The 49-year-old has opened up about the hard time he has when he socializes, and claims that his spell at the club will not be in vain if he fails to triumph in Europe.
The Portuguese boss has developed a reputation as a controversial figure due to his outspoken nature, and his stay in the Spanish capital has witnessed several fierce battles with main rival Barcelona.
He has now revealed that he is routinely targeted by the public, who have also taunted his children when he is with them.
"I hate my social life," he told CNN. "I hate not to be a normal father who goes with his son to his football match and be there with the 20 other fathers watching the game. The people have to come for photos. The people have to come for autographs. The people have come to insult me. The people have to go behind the goal of my kid to insult a 12-year-old kid.
"People think they know me, but they don't. People know the manager, especially the manager during 90 minutes. And during 90 minutes, I'm not there to have fun. I'm there to do my job, I'm there to win. I'm there with my team to try to win. I'm there and I live the game. I live the match as if it was the last of my career.
"People look at me and see what they see. It's the other place where people know me."
Having ended Barcelona's stranglehold on the Liga title, Mourinho is currently bidding to bring home los Blancos' 10th Champions League title, but does not feel his spell at the club will be deemed a failure should he not manage to do so.
"Will I see my stay here as a failure? No. In relation to the Champions League there is an edge where the difference between succeeding and not succeeding is very narrow," he added. "The world will consider it a failure because I won it so many times and brought so much success to my teams that everyone is probably expecting that."
Follow GOAL.COM USA on