Stoke took the lead through Oussama Assaidi at St James’ Park on Boxing Day, but then saw Glenn Whelan and Marc Wilson sent off within three minutes of each other prior to half-time.
Despite seeing Loic Remy have a penalty saved following Wilson's dismissal, Newcastle duly ran riot to secure a seventh win in nine games.
"I think the referee needs to look at his performance," Hughes told reporters. "From one of our more senior referees, I was absolutely dismayed by his performance if I’m honest.
"Up to the point when the first sending off was given I thought we were totally in control of the game.
"We were 1-0 to the good; Newcastle were finding it very difficult to really have an impact on the game and we had a number of other chances. Then the referee makes a couple of decisions which change the course of the game."
In addition to criticising Atkinson for both red cards, Hughes pointed to a handball prior to Newcastle's equaliser through Remy and also believed the ball had gone out of play in the build-up to the hosts' second goal, scored by Gouffran.
"We save the penalty (that followed Wilson's dismissal). We go down the other end with nine men and we have an opportunity to play somebody in for a clear opportunity on goal," added Hughes.
"It gets handballed by (Mike) Williamson, the referee is two yards away, plays on and they go up the other end and score. So you can understand why we're a little bit aggrieved by what's happened."
Hughes was sent to the stands by Atkinson after Whelan had been sent off, but had no complaints about that decision.
"That's because there was a ball near me and I needed to get rid of some frustration so I booted it on the pitch. It was totally the wrong thing to do so I apologise for that," explained the Welshman.
Magpies boss Alan Pardew meanwhile felt Stoke could have few complaints about the officiating.
"The game changed on a couple of decisions that, on reflection, I haven't really got a problem with," he told reporters.
"I thought the first booking was slightly harsh on Whelan and maybe Yohan (Cabaye) was at fault as well and was lucky to get away without having a yellow himself.
"But the second one I have absolutely no issue with because Yohan gets a lot of treatment at this place and I don't think that tackle had any indication of playing the ball," he added.