Bayern looked a shadow of its normal self in the opening 75 minutes of the Bundesliga clash and found itself staring at a first domestic defeat of the season until the closing stages.
Vedad Ibisevic opened the scoring for Stuttgart in the 29th minute with a poacher's goal from 10 yards, but the striker and Martin Harnik both wasted further chances to double the home side's lead.
That cost it as substitute Claudio Pizarro levelled for Bayern with 14 minutes remaining before Thiago's late acrobatic volley stole the points and extended the defending champions' unbeaten Bundesliga run to 43 games.
Guardiola acknowledged his side was far from its best and had some fortune with the late comeback.
"At the end we were lucky, it was a fantastic goal and an incredible victory," he reflected. "After the substitutions of Pizarro and [Mario] Mandzukic we created more chances.
"Stuttgart played very well for 30 minutes, were more aggressive and won many battles. We had trouble finding our rhthym."
Bayern's win moves it 13 points clear of Bayer Leverkusen at the top of the Bundesliga and also means it has scored in each of its last 28 away games in the league.