The 23-year-old curled home the opener away to Stuttgart before teeing up Robert Lewandowski to make it 2-0 at Mercedes-Benz Arena.
Speaking to reporters about Goretzka post-match, Kovac said: "I'm very happy. He did a good job in the pre-season. He played a little bit less minutes than others all the time.
"But for today we had the feeling that he is the right player to start with his ability to create space, his aggressiveness and his pace. He can help us a lot with that.
"The opening was a shot from 16 meters so from then on, the game was a bit easier for us.
"I'm happy for him that he scored his first goal in his first Bundesliga game with Bayern. Or was it his second game already? Yes, right his second. So it was his first goal and I'm happy for him."
Bayern continued their winning start under Kovac thanks to a three-goal second half away from home.
The first half proved frustrating for Bayern, who finally made the breakthrough in the second 45 via Goretzka, Lewandowski and Thomas Muller.
"I'm very happy about my team's performance," Kovac added. "We played very concentrated over the whole 90 minutes. Defensively and offensively. How many chances we had in the second half shows that we are able to do a lot of good things with the ball.
"I hope that we can keep our game on that level. I think we can do that in the future, too. But I know every new game is different. And we have to prove what we are capable to do in every single game."