The Germany international came through the ranks of the Schalke youth academy, but opted to leave his boyhood club when Bayern came calling.
He has since been subjected to offensive chants on more than one occasion when the two clubs have met, and Neuer feels the supporters are crossing a line with their behaviour.
READ MORE | Matthaus tips Ancelotti for Bayern success
"I don't mind it if they whistle me when I'm on the ball. But I cannot understand it when they insult me with the words they used," Neuer told Kicker.
"Emotion is part of the game, but I find it incomprehensible when I hear people shout at me and call me a son of a w****.
"Everything that happened when I left Schalke for Bayern made me stronger as a person. It was quite an experience and it was not easy for me.
"But if you look at the career I have already had at Munich, it's clear that joining Bayern was the right choice for me."
Neuer kept a clean sheet when Bayern beat Schalke 3-0 at the Allianz Arena on April 16.