The Swans drew 0-0 with the Terriers at John Smith's Stadium but finished the match with ten men after the Ghana international was dismissed.
After being sent off by referee Michael Oliver for a reckless challenge on Jonathan Hogg, Ayew is facing a three-match ban but his manager claims the Swans may have grounds for a plea.
“It was right in front of me,” Carvalhal told club website.
“I don’t usually talk about refereeing decisions, but I think this one was from the fourth official.
“To me, it was not a red card. I thought that at the time and I think the same after watching the video.
“Two players fought very hard for the ball and their feet were in exactly the same position. They were both fighting for the ball.
“If it’s a red card to one then it should be a red card to both of them. I think Jordan slipped a little on the pitch and their player did the same. They were like two bulls colliding.
“We will analyse it and see if we have reason to appeal. Let’s see.”
However, Carvalhal is confident his team would find a way to survive.
“Of course I would rather have Jordan available than not for the next three games,” he continued.
“He is a player I like a lot but, if Jordan does not play, we will play with another player.
“When we lost (Leroy) Fer, he was our best player at that time. We lost Renato Sanches who would be in the 11 and we then lost Wilfried Bony.
“We are still playing and still fighting even without them. We have shown character to get results and that is what we will keep doing.”