The midfielder was pictured at a bus stop near Craven Cottage after being substituted in his side's 6-0 defeat against Martin Jol's men.
But Warnock has admitted he can understand why Taarabt left the game on Sunday and has claimed that would have probably done the same, given the circumstances.
Warnock told the Daily Mail: “Put it this way, if I was subbed at half-time after that first half, I would have left.
“There was nowhere for him to sit and watch the game, so I don't blame him for going. It's been blown out of all proportion.”
Taarabt was pictured with QPR fan Kerry Smith who had left the match as the Loftus Road outfit conceded the fourth goal.
She told the Daily Mail: “We asked Taarabt what the hell he was doing there. He just shrugged. He asked what the score was.
"It was 4-0 by that point, but we heard another goal go in and he just rolled his eyes and kind of smiled in disbelief.”