The three-time World Cup finalist was asked to leave after he attempted to console his compatriots following their shock defeat against Germany
The two-time World Cup winner - who made 142 appearances for Brazil during his active career - visited his compatriots after their humbling defeat in an attempt to console them, yet was unceremoniously told to leave by CBF president Jose Maria Marin.
"President Marin said he didn't want any strangers in the dressing room," Cafu told ESPN.
"I told him I'm no stranger. I was only there to embrace the boys, to give them some comfort. I didn't want to talk about anything else. I was only there because the boys needed support, that's why I came to the dressing room.
"I was surprised that I was practically expelled from the dressing room. Humbly, I left the changing room.
"I endured a similar thing back in 1998 and I know what a World Cup defeat feels like.
"I knew the boys needed some attention and needed to be comforted at that moment. I wasn't there for self-promotion, I don't need this kind of thing. I went to give a hug to Felipao and [Carlos Alberto] Parreira."
Brazil will meet either Netherlands or Argentina in the third-place play-off in Brasilia on Saturday.