Brown was selected in the England squad to face Hungary on Wednesday, having missed out on the World Cup this summer.
But the Football Association confirmed Brown visited England's team hotel in Watford on Sunday night and told manager Fabio Capello he no longer wished to be involved.
Brown then confirmed the news by a short statement which read: "After a lot of thought and with a very heavy heart, I have decided the time is right for me to retire from international football.
"At the age of 30 I feel it is right for me to stand aside and let younger players come through, which allows me to concentrate on my club career.
"I regard it as an honour and a privilege to have represented my country at every level from Under-15s upwards.
"I have always been very proud to play for England and wish them well in future tournaments."