The 31-year-old took 81 wickets in 21 Tests but had his appearances limited by a series of injuries.
Leading into pre-season I really wanted to give the Ashes a crack, Pattinson said.
‘If I was to be part of the Ashes I would need to do myself and my team-mates justice.
Pattinson, who also played in 15 one-day internationals and four Twenty20 internationals, made his Test debut in 2011.
He took 14 wickets in his first series against New Zealand and was named player of the series, while his 81 Test wickets overall came at an average of 26.33.
Pattinson had major back surgery in 2017 but recovered sufficiently to play in the Ashes in England two years later.
However, another injury in the build-up to Australia’s domestic season has led him to retire, with the Ashes set to begin on 8 December in Brisbane.
‘In the end I haven’t had the preparation I would have liked heading into the coming season,’ he said.
‘I didn’t want to be in a position of battling with my body when you need to be ready to go at any time.’
He added that he would continue to focus on developing young players at Victoria, where he plays state cricket, and would consider playing county cricket in England.
‘James epitomises what it means to play for Australia, the sacrifices you make and the hurdles you have to overcome on the journey,’ chief national selector George Bailey said.
‘He has been a wonderful player. We would have loved to have seen a lot more of him over the years.’