Robert Lewandowski converted an 82nd minute penalty at the second attempt before Arjen Robben struck in stoppage time to seal all three points.
It appeared as though new boss Kovac would be forced to settle for a point when Adam Szalai produced a classy 57th-minute finish to cancel out Thomas Muller's earlier header.
But the reigning champions, who lost Kingsley Coman to a concerning ankle injury, were given a lifeline when substitute Havard Nordtveit brought down Franck Ribery with under 10 minutes remaining.
Lewandowski, recently vocal in the press about feeling unsupported by Bayern, had his first attempt at the winner saved by Oliver Baumann, and Robben's successful follow-up was ruled out for encroachment.
The prolific Poland striker was subsequently sent back to the spot and made no mistake as Robben eventually joined him on the scoresheet with a clinical finish at the death.
6 - @esmuellert_ (@FCBayernEN) has scored his 6th goal on the opening weekend of a #Bundesliga season, no other current Bundesliga player has scored more (Lewandowski also 6). Opener. #FCBTSG pic.twitter.com/JjobLYEUM4— OptaFranz (@OptaFranz) August 24, 2018
Bayern had started like a team that finished 21 points clear last season and their ominous opening was rewarded thanks to Muller's header from a Joshua Kimmich corner in the 23rd minute.
They did suffer a setback when Coman hobbled off following a crude late challenge from Nico Schulz, and Kovac's headaches mounted 12 minutes after the interval as Szalai punished some sleepy defending.
A response was needed and it eventually came through two goals in a frantic final 10 minutes, Lewandowski and Robben combining to end Kovac's curious record of having never previously beaten Hoffenheim.
What does it mean: Questions over Lewandowski
Though his spot-kick blushes were spared courtesy of a retake, Lewandowski was some way short of the standards he has set in recent seasons. The 30-year-old, openly critical of Bayern chiefs this week, was quiet prior to the penalty and did little to suggest he has moved on from the frustration of being denied a move to Real Madrid.
Pat on the back: Joshua Kimmich
Now an ever-present at right-back, Joshua Kimmich showed he has overcome Germany's nightmare World Cup with a typically confident display. He had more touches than anyone else on the pitch and was consistently a driving force for Bayern.
Boot up the backside: Grifo's return to forget
Julian Nagelsmann's proactive decision to bring off the ineffective Vincenzo Grifo at half-time was a wise move. Having returned in the close-season for his second spell at the club, the midfielder will want to quickly forget a 45 minutes that featured just 10 passes, one tackle and scant forward momentum.