The Portuguese lifted La Liga in 2012 but much publicised disputes with goalkeeper Iker Casillas and defender Pepe brought about the end of Mourinho's reign at the Santiago Bernabeu last season.
Los Blancos finished runners-up to Barcelona in the league while bowing out of the Champions League in the semi-finals to Borussia Dortmund, before capping off a miserable season with defeat in the Copa del Rey final at the hands of city rivals Atletico Madrid.
Mourinho was subsequently sacked at the end of the campaign, with Albiol following him out of the exit door as he joined Napoli and the 28-year-old says the writing was on the wall for his former boss as the season drew to a close.
"With Mourinho there was never a bad relationship, I always got along with him. That is normal with my boss," he told El Pais.
"The last three months at Madrid were unpleasant, because we lost the league, we were eliminated from the Champions League and lost the Copa del Rey Final.
"Maybe last year was the worst in respect of the dressing room atmosphere, but I will stick with the good and my relationship with Mourinho was normal."
Albiol has played both of Napoli's opening Serie A games this season, with Rafael Benitez's side emerging as the early leaders in the division and the Spain international admits he is enjoying playing under the man who gave him his first chance with former club Valencia.
"I am back to enjoying the confidence of the coach, the tension of the games, it is what satisfies you as a professional," he continued.
"(Completing the move) took no time, in a few days we arranged everything. It was easy to decide - it was Benitez, they had already signed [Jose] Callejon, they had a project and Naples has always been a city for football.
"Benitez’s influence? I didn’t talk to the coach until I was taking the medical. But I have known him for years, in fact I made my debut at Valencia with him, in his final year, against AEK Solna in the UEFA Cup."