The London side's record signing struck a wonderful goal to win the game at the death to give new boss Tony Pulis a huge boost while heaping pressure on Villains coach Paul LambertDwight Gayle's last-gasp gem snatched Crystal Palace a 1-0 win at Aston Villa to lift it out of the Premier League's bottom three.
The Boxing Day clash at Villa Park appeared set to finish goalless until Gayle collected a pass from fellow substitute Jonathan Williams and curled the ball home from 20 yards.
The result moved Palace to 17th in the Premier League, above Fulham on goal difference, and left Villa just three points clear of the relegation zone following a fourth successive defeat.
The Villains have now won just one of their last 13 matches on Boxing Day and have failed to score in eight of their last 11 outings.
Paul Lambert was without Christian Benteke and Ashley Westwood due to injury and suspension respectively, but Gabby Agbonlahor returned after completing a one-match ban. Palace's top scorer Marouane Chamakh missed out after picking up his fifth yellow card of the season last time out.
The home side dominated possession in the opening 25 minutes, but its final ball let them down, with Marc Albrighton particularly wasteful.
Palace improved as the half progressed, largely thanks to Yannick Bolasie. The 24-year-old winger collected possession on the left, skipped past Matthew Lowton and fed Cameron Jerome, who over-hit a through ball to Barry Bannan when he really should have shot.
After a limp first 45 minutes, the second half exploded into life, with an off-balance Baker doing well to deny Bolasie a clear sight of goal.
Fabian Delph's vision and immaculate pass then presented Andreas Weimann with a chance, but the Austria international's half-volley went straight into the arms of Julian Speroni.
Bannan, against his former club, almost put Palace in front on 54 minutes, but Brad Guzan did superbly to tip the ball onto the post.
Agbonlahor raced into the area but hit his shot straight at Speroni and, following a delightful move, Albrighton's clever cut-back set up Weimann, who fired over from inside the box.
Bolasie tricked his way past two opponents on 65 minutes and crossed for Jason Puncheon, whose fierce shot from 10 yards was saved by Guzan.
A scrappy finish looked destined for a draw until Gayle grabbed his third Premier League goal, and all three points for Tony Pulis' side.
Palace has now won three of seven games under the former Stoke City boss, but faces a daunting trip to Manchester City on Saturday.