The competition had not come during the most prosperous time for the team. Two years earlier, they had been humiliated at Euro 2004 and the head coach arrived at the tournament with a young roster, looking to start a new era.
Excitement reached fever pitch and, unsurprisingly, a number of World Cup anthems soon started surfacing. There was the rather optimistic, '54, '74, '90, 2006 by Sportfreunde Stiller (which had its title and chorus changed within hours of their semi-final exit at the hands of Italy) but the enduring song was Oliver Pocher's Schwarz und Weiss (Black and White).
A tribute to the colours of the national team, the song harked back to former glories with old clips of commentary from West Germany's previous World Cup wins - the comedian appearing in the guises of various heroes of past and present.
The rousing chorus of "Black and white, we stand by your side and we'll win together with you. Now, let's go!" served as the soundtrack for much of the tournament eight years ago and remains just as popular today.
The significance of the song was not lost on Klinsmann's players and, after Germany's quarter-final victory over Argentina in a tense penalty shoot-out, Lukas Podolski went into the crowd to hand Pocher his jersey from the match.
Pocher's anthem is still played before all games which Germany play at home and fans all sing along just as heartily as they did all those years ago.
Pocher himself was something of a colourful figure. His love of football had long been well-known and he once did a famous impression of former Bayern Munich and Germany goalkeeper Oliver Kahn, which is still doing the rounds on YouTube today.
And, his song for 2006 is being played all around the country this year as Joachim Low's side look to make up for their semi-final exits in the last two tournaments. If they win the trophy, it will be the soundtrack for the summer.
This is how Germany supporters share their musical football experience. Now it's your turn to show your passion and share your music for 2014 FIFA World Cup Brazil™ with Hyundai's Pin Your Fan Park. Create your own Fan Park by sharing live footage of you enjoying the 2014 FIFA World Cup Brazil™. Post your supporting message, upload photos and videos and make football playlists at worldcup.hyundai.com