The Blues slipped to their second defeat of the season as the Portuguese continues to search for a first Premier League victory at St James' Park after four failed attempts
The Chelsea manager entered the match with just two points from his previous three visits to Newcastle, and watched on as his side wasted a chance to move to the top of the Premier League table on Saturday.
Chelsea controlled a first half devoid of clear-cut chances – their best effort saw John Terry hit the crossbar with a header – but were left reeling when Gouffran headed in an inviting Yohan Cabaye free-kick in the 68th minute.
Substitutes Willian and Samuel Eto'o went close for Chelsea in the latter stages, but Remy hammered home Vurnon Anita's cut-back with two minutes left to seal Newcastle's win.
The loss was Chelsea's second of the league campaign, meaning Arsenal could go five points clear if they beat Liverpool on Saturday.
|VIEW FROM ST JAMES' PARK
|By Chris Myson at St James' Park
"Well, who would have seen that coming?! At half-time it looked like we were heading for a narrow Chelsea win or, at worst for the visitors, a goalless draw. The Blues were dominating possession and Newcastle offered little, other than some surprisingly solid defending.
"After the break, though, Alan Pardew's team came alive. Loic Remy, Yann Gouffran, Moussa Sissoko and Mathieu Debuchy were all superb and the goals were not against the run of play by any means. The home fans were bouncing in the second half as their team produced a passionate display and morale at the club has been given a much-needed boost.
"Jose Mourinho is still waiting for a league win at St James' Park – and the Blues’ title rivals will have loved watching this uninspiring attacking display."
Newcastle also made two alterations, with defender Mapou Yanga-Mbiwa returning from suspension and Shola Ameobi selected, as Hatem Ben Arfa and Paul Dummett (hamstring) made way.
In a scrappy first half, Chelsea came close to an opener when Terry's thumping header hit the bar in the 13th minute.
Chelsea begun strongly but failed to create a clear openings until the captain's chance, which came when he rose above Mike Williamson to nod Mata's corner against the woodwork, while Branislav Ivanovic's follow-up deflected onto the top of the bar.
Another Terry header almost led to a goal, with his effort cleared off the line by Davide Santon, and at the other end, Moussa Sissoko – scorer of two goals in Newcastle's 3-2 win over Chelsea in February – forced a smart save from Chelsea goalkeeper Petr Cech at the near post seven minutes before the break.
Newcastle skipper Chiekh Tiote limped off with an apparent thigh injury eight minutes after the break but that did not stop them from enjoying a terrific spell around the hour-mark.
Sissoko was first to go close, forcing Cech to tip wide when released by Mathieu Debuchy, before Remy shot straight at the goalkeeper from close range.
Gouffran had a good effort saved by Cech at the near post in the 66th minute and three minutes later, Newcastle's pressure told, with the Frenchman breaking the deadlock by nodding in Cabaye's excellent free-kick.
They quickly went in search of a second, with Terry making an excellent sliding tackle to deny Remy after he linked well with Sissoko, before Eto'o had an effort deflected wide by Yanga-Mbiwa.
A clever Oscar cut-back let Willian in with 10 minutes remaining, but his shot was well saved by Newcastle goalkeeper Tim Krul, and a follow-up from Eto'o hit Debuchy and went wide.
And Newcastle wrapped up their victory in the 88th minute, as Remy produced a powerful finish to convert Anita's clever assist.