Kane has been tipped to succeed in La Liga by Madrid captain Sergio Ramos, but the 25-year-old has stayed quiet when quizzed on a possible move.
"For one thing it would upset the Tottenham fans, and that’s something he couldn’t do because the relationship he has with them is special.
"I know he loves Spurs. You can tell by the way he talks about the club.
"He might want to stay there for his whole career, breaking all the scoring records, being there for the first 10 years of the new stadium, being a Tottenham legend.
"It would have to be something special for him to leave the club. But if it’s going to happen, it will have to be soon.
"He’s 25 now. If he does not go in the next two years, by the time he is 27, he will never leave.
"If he did, Real are the only club that would be right for him. Real shine brighter than any other club in the world.
"If it happened, everyone would understand, nobody could be angry."
Although Kane is the star player at Spurs, Matthaus believes he and chairman Daniel Levy may have already worked out an agreement over a potential exit.
"Levy knows what Harry’s done for the club," the former midfielder said.
"It wouldn’t surprise me if there is an understanding, a gentleman’s agreement, between them.
"He would not have anything written into his contract. He is not that sort of player.
"But if he knocked on Daniel’s door, at the right time, and there was a truly serious offer on the table, it may be different."