The Belgian scored from the spot having been felled by Diego Lugano to send his side into the top half of the table after a first half which saw each side score three times.
Aston Villa notched its 300th Premier League win by battling from two goals down to beat Midlands rival West Brom 4-3 at Villa Park.
A thunderous strike from Chris Brunt and a Fabian Delph own goal inside the first 10 minutes seemed to have put West Brom well on its way to three points on Wednesday.
Villa was reading from a different script, however, and Andreas Weimann's lobbed finish in the 12th minute inspired the home side, with Leandro Bacuna equalizing before the half-hour mark.
Against all odds, Paul Lambert's side looked set to take a lead into the break as Delph scored in the correct goal this time, but Youssouf Mulumbu leveled the scores again with a composed finish in the 43rd minute.
West Brom coach Pepe Mel admitted in the build-up to the game that he tried to sign Christian Benteke in his days at Real Betis and it was the Belgium star who netted the winner when he converted a second-half penalty to move Villa into 10th.
Matthew Lowton replaced the injured Gabriel Agbonlahor in Villa's only change from the 2-2 draw with Liverpool in the team's last Premier League outing.
Mel restored Mulumbu and Victor Anichebe to his starting XI after missing out against Everton, replacing Zoltan Gera and Matej Vydra.
It took the visiting side just four minutes to take the lead and it came in style as Brunt lashed home a fearsome half-volley from 30 yards out, giving Villa goalkeeper Brad Guzan no chance.
Villa had a major struggle on its hands five minutes later as a well-worked free kick released James Morrison in the left side of the home team's area and Delph directed the eventual cross into his own goal.
The home team responded almost immediately, however. Guzan's long ball from the back was inadvertently flicked on by Diego Lugano, allowing Weimann to drop the ball over Ben Foster and into the net.
Bacuna continued Villa's stunning comeback in the 25th minute as he fortuitously sliced Lowton's low cross over the diving Guzan before Delph made amends to score at the right end as he burst into the area and unleashed an unstoppable drive eight minutes before the break.
There was still enough time for one more goal before halftime as West Brom leveled the scores again, with Mulumbu confidently slotting past Guzan following a delicate first touch.
Ten minutes after the break, Lugano missed a glorious chance to give West Brom the lead as he headed Brunt's free kick wide at the back post and the Uruguayan played a big role in Villa's fourth goal in the 64th minute.
Lugano hauled down Benteke in the penalty area as he jumped to reach a cross and the Belgian buried the spot kick, scoring for the third consecutive match.