The 22-year-old moved to the Allianz Arena last summer in a €37 million deal but did not hit the heights expected of him, despite helping the Bavarians to the Bundesliga title.
After the heroics for the national team in Brazil this summer - with Gotze volleying home in extra-time against Argentina at the Maracana to seal the trophy - Rummenigge feels he must cement his reputation for his club this campaign.
"It's important that Mario makes his mark here," he told Sport1.
"The fans and media always accused him of not arriving yet, but he needs to start in that direction in his second year.
"I am convinced that he knows what is expected of him - and he will do that. He will now justify fully the confidence we have in him.
"He is a great player at Bayern, a very great one."
Bayern have been linked with an audacious move for Marco Reus and Rummenigge insists that he will not disrespect Borussia Dortmund by speaking publicly about the 25-year-old.
"We have a professional relationship with Dortmund and I don't want to create any problems with a premature, unnecessary mark. That wouldn't be fair," he said.
"Dortmund could buy out his release clause or sign a new contract."
Rummenigge also claimed that Dortmund and Bayern will face stiffer competition for the Bundesliga title in the coming years as their top-flight rivals continue to strengthen.
"There are a few clubs trying to get closer to Bayern and Dortmund. We'll have to wait and see if this is possible," he added.
"[Bayer] Leverkusen have had sponsorship money from Bayer for the next three months paid in advance to buy new players, Wolfsburg have great possibilities. Let's see what happens by the end of the transfer window."