The Bavarians were left shocked after Niklas Sule was able to slip the ball past Manuel Neuer to give the home side a 1-0 lead.
However, Bayern equalized five minutes later when Mario Mandzukic deflected home a Franck Ribery free kick, before Thomas Muller scored the winner 15 minutes from time.
"It certainly was not our best performance, but we won," Guardiola told Bayern Munich's official website. "Not everything went perfectly, so we will need to correct the way we play. The encounters with Hoffenheim were always tough in recent years, but we had a bit of luck."
The Spaniard was not the only figure in the Bayern dressing room to question the team's performance on the pitch, with Mario Gotze unhappy with his side's inability to finish off several of their chances.
"We did improve in the first half, but very little. We also had a few great chances to score again, though I don't know why we didn't score. This is of course not great."
Victory saw Guardiola's side equal Hamburg's 30-year-old league record of 36 consecutive matches unbeaten.