Bayern looked a shadow of their normal selves in the opening 75 minutes of the Bundesliga clash and found themselves staring at their first domestic defeat of the season until the closing stages.
Vedad Ibisevic opened the scoring for Stuttgart in the 29th minute with a poacher's goal from 10 yards, but the striker and Martin Harnik both wasted further chances to double the home side's lead.
That cost them as substitute Claudio Pizarro levelled for Bayern with 14 minutes remaining before Thiago's late acrobatic volley stole the points and extended the defending champions' unbeaten Bundesliga run to 43 games.
Guardiola acknowledged his side were far from their best and had some fortune with their late comeback.
"At the end we were lucky, it was a fantastic goal and an incredible victory," he reflected.
"After the substitutions of Pizarro and [Mario] Mandzukic we created more chances.
"Stuttgart played very well for 30 minutes, were more aggressive and won many battles. We had trouble finding our rhthym."
Bayern's win moves them 13 points clear of Bayer Leverkusen at the top of the Bundesliga and also means they have scored in each of their last 28 away games in the league.