The Black Cats were reduced to nine men in the defeat to Hull City after the midfielder and Andrea Dossena were dismissed, leaving Tigers boss Steve Bruce furious with the Italian
The midfielder was given a straight red card by referee Andre Marriner after a late tackle on Ahmed Elmohamady in first half stoppage time, and he was followed down the tunnel just two minutes later after Andrea Dossena stamped on the shin of David Meyler, leaving Sunderland with nine men.
Poyet was left to openly criticise Marriner's decision to dismiss Cattermole, who has a poor disciplinary record.
"I'm convinced it wasn't a red card," the Uruguayan told Sky Sports. "I think sometimes people just see Lee Cattermole making a tackle and that's it.
"I think we need to be fair to him. It's Lee Cattermole. People put things in the mind of [referees]."
Hull boss Steve Bruce offered sympathy for Cattermole, but was furious with Dossena after the Italian's reckless challenge on Meyler.
He said: "I've known Lee for a long time as everybody knows. I've signed him twice, but the big thing with that challenge [is] in my day [it] probably would only be a yellow card.
"The directive now is that if you leave the floor with your studs showing, even if you don't touch the player, then it's a straight red card.
"The other one [Dossena's tackle] should be three red cards. They're ones that hurt professional footballers; over the top and a stamp on his ankle. Meyler is lucky he has got away [without an injury].
"For someone who has had horrific injuries, I am pleased he has got away with it. It was an awful tackle that no one wants to see."