Sane – who is contracted to City until 2021 – has been linked with a move to Bayern following back-to-back Premier League titles in Manchester.
With Bayern greats Robben and Ribery leaving Allianz Arena, Germany international forward Sane has emerged as a target for the Bavarian giants.
And Rummenigge talked up a move for Sane, who left boyhood club and Bundesliga side Schalke for City in 2016.
"You shouldn't forget that he still has two years contract and that Manchester City don't just sells their players because they need to," Rummenigge said.
"This will be difficult. But I think that the player has to be convinced to have the joy for playing here at Bayern.
"He would be the successor of Arjen Robben and Franck Ribery which is very interesting. As a player, this would be a good incentive I think."
France internationals and World Cup winners Lucas Hernandez and Benjamin Pavard have already agreed to join Bayern in the off-season, however, James Rodriguez could be set to leave Munich.
But James struggled with injuries and game time under head coach Niko Kovac in 2018-19 – the 27-year-old starting just 13 of his 20 Bundesliga appearances.
Asked about James, Rummenigge said: "We'll definitely meet next week and decide how to proceed. We have to put all the options on the table. And then decide together how to deal with these things, which now still have a question mark, so that they become exclamation marks.
"You can imagine if James does not play continuously from the beginning, that is not his claim, he is a superstar and of course he always wants to play. Now he has been injured in the last few weeks and we must not forget that. We will find a solution in line with him, I can promise you that."
When pressed on whether Bayern could buy James permanently before reselling him, Rummenigge replied: "No, that's not the style of Bayern. We don't want to do any human trafficking to earn anything from this story. Bayern has never done this before."