Zaha made his 11th Premier League appearance on Saturday but struggled to make a key impact as second-half goals from Tammy Abraham and Christian Pulisic condemned them to defeat.
The 26-year-old was in action for 90 minutes, making just one key pass but did not register a single shot on target.
Zaha’s influence in Roy Hodgson's team was limited by 19-year-old Recce James on Saturday afternoon and his performance got Cole frustrated.
“I’m disappointed with Zaha today. Reece James has got in his head,” Cole told BT Sport.
“He’s let a young full-back dominate him. This Zaha five minutes ago was saying he wanted a move to the big clubs. He said he wanted to go and play for this club, that club.
“Reece James has come on his debut, got in his head and a player of his quality letting the ball under his feet. I think credit to Reece James but Wilf Zaha needs to show more if he wants to make that next step.”
Owen Hargreaves, on his part, praised James for putting on a good show in stopping Zaha who is considered as one of the best dribblers in the Premier League.
“Nobody will play Wilf Zaha better this season than Reece James did today,” Hargreaves added.
“He was quick, he was physical and he’s a good athlete that’s what these young guys give you.
“He stood up to him every time. Nobody does that with Wilf Zaha everybody runs away from him because they’re scared they will dribble by him. I loved his game he was close to perfect.”