The Sunderland manager says he is "one of the few" not overly concerned by his American striker's slump.
After earning a summer move to Sunderland with a 31-goal campaign for Dutch side AZ, the 24-year-old scored just twice this past season — including a solitary goal in 31 league appearances.
"He was always trying his best," Poyet told SiriusXM. "Unfortunately the goals didn't come out for him, but I'm relaxed because if you see the power he's got, if you see the shot, he can shoot. I can tell you that because I see it every day.
"I'm quite relaxed with Jozy, to be honest. I'm one of the few. I know it's difficult to offset that striker when he doesn't score, but I like him. I like him as a boy and I like him as a player. It's just that he couldn't find the net. You know, sometimes it happens. You change clubs, and you can't do it at one club for whatever reason."
Altidore's struggles were particularly startling considering he scored eight goals for the U.S. national team in 2013. Despite his recent club form, Altidore was named to Jurgen Klinsmann's 30-man preliminary roster Monday and remains the favorite to start up top for the Americans in Brazil.
"I think you've seen him on goal for the national team for whatever reasons," Poyet said. "The base of the games with the national team are different in England. I don't know, but something of course is not working for us with Jozy, but I have to say that he (handled it) very well."