The Spaniard was not a happy man after watching his side blow the opportunity to rack up a 20th consecutive victory in the Bundesliga ahead of its midweek trip to Old Trafford.
The Bavarians had appeared to be cruising towards a 20th consecutive win in the Bundesliga after responding to Anthony Modeste's opener with a Claudio Pizarro double and a Xherdan Shaqiri strike.
However, Hoffenheim pulled one back just before the break through Sejad Salihovic and then claimed their first ever point away to Bayern courtesy of Roberto Firmino's 75th-minute equaliser.
"Compliments to Hoffenheim," Guardiola told his club's official website after the dramatic 3-3 draw. "They showed a good ability to press. In the first half we found good solutions. In the second half, though, we were not good.
"We need to control the ball. If this is not the case, then we have a problem. We need to analyse this game and then quickly focus on Tuesday. We need to improve."
The first leg of Bayern's Champions League quarterfinal clash with United will take place at Old Trafford.