The Frenchman popped up deep into stoppage time to lash home a late winner for Alan Pardew's men, ending the side's barren run of over seven hours without a goal.
Newcastle are 2/1 with bet365 to beat Norwich on 1 March, 2014
"I am relieved because we lost some important players like [Yohan] Cabaye (who joined Paris Saint-Germain in January) and we had so many injuries, so for us it was very important [to win] for the confidence in the team," Remy told Sky Sports.
"It is good to win even at home. The fans are expecting a lot from us so now we are very pleased and I think the team are fine for the rest of this season."
Remy's late goal made the difference on his return from a three-match ban after being sent off in the 0-0 stalemate at Norwich City last month, and the 27-year-old is determined to make up for lost time.
"I was very hungry," he continued. "I missed three games. I left my team-mates. I missed some good opportunities so maybe for the next game I will try to score more goals.
"I never feel my chance has gone because I really want to score or give assists, so for me when the game is not finished I believe I can score."
Remy dedicated his match-winning strike to assistant boss John Carver, whose father recently passed away.
"I would like to dedicate this one to JC [John Carver] because of his father as unfortunately not all the players could be there for his funeral," he added.