The 26-year-old striker came off the bench to net a late winner after fellow substitute Craig Noone had cancelled out Papiss Cisse's opener, handing the Norwegian boss a maiden win
The Magpies twice hit the woodwork through Hatem Ben Arfa before finally breaking the deadlock on the hour-mark, with Papiss Cisse latching onto a loose ball inside the box and tucking the ball beyond David Marshall.
But the away team responded with two quick-fire goals. First, Craig Noone sent a fizzing long-range effort into the back of the net, before fellow substitute Frazier Campbell completed the turnaround by converting Peter Whittingham's corner moments later.
|VIEW FROM ST. JAMES' PARK
|By Harry Slavin
Ole Gunnar Solskjaer’s introduction to English football is an impressive one as he showed tactical nous to lead his side to a thrilling comeback at St. James’ Park.
The Norweigan saw his side fall behind to a Papiss Cisse effort before two substitutions from the Norweigan changed the tide of the match as firstly Craig Noone – from fully thirty-yards – and Fraizer Campbell turned the game on its head.
The long wait for silverware inevitably carries on for another year for the Magpies, but the small batch of Cardiff fans that made the trip will be travelling home dreaming of Wembley and a brighter future under Solskjaer.
Solskjaer, meanwhile, made five changes to Cardiff's starting XI from the 2-0 Premier League defeat to Arsenal, with Andreas Cornelius and Peter Odemwingie paired up front for the visitors.
Cardiff made the better start to proceedings and looked to have taken the lead when Mark Hudson headed home Whittingham's corner after nine minutes, only for the goal to be ruled out for a push in the area.
Kevin McNaughton and Kim Bo-Kyung then went close from distance as Newcastle struggled to gain a foothold in the game.
However, the hosts did finally threaten when Hatem Ben Arfa crashed an effort against the left-hand post after 24 minutes.
Yoan Gouffran then shot wide on the half-hour mark, but Newcastle generally struggled to test the Cardiff goalkeeper.
Marshall was called into action twice in the early stages of the second half as he denied both Ben Arfa and Moussa Sissoko.
But Newcastle were rewarded for their positive second-half showing in the 61st minute, Cisse converting from point-blank range for only his third goal of the season after Sissoko's initial shot had been only partially cleared.
Cardiff's response was a strong one and Solskjaer's men came close to levelling when Campbell struck the post with an effort from 18 yards.
It was Noone's introduction that would prove pivotal, though, with the midfielder's driving run and long-range piledriver flying past Rob Elliot and into the top corner.
The tie appeared poised for a replay at that point, but, with both sides pushing for a winner late on, it was the Welsh side who ultimately got it - as with 10 minutes left Campbell rose highest to head Cardiff into the fourth round.