A goal at the death brought Sunderland level after Cardiff had stormed out to a 2-0 lead.
The Welsh side looked to be in complete control in its first game following the sacking of boss Malky Mackay by controversial club owner Vincent Tan, after Jordon Mutch had opened the scoring with a deflected strike after just six minutes.
Three points appeared to be a formality when former Sunderland man Fraizer Campbell struck in the early stages of the second half.
But Sunderland refused to lie down, and found some hope when Steven Fletcher pulled a goal back with seven minutes left to set up a frantic finish.
There was still time for more late drama and the comeback was completed with the last meaningful kick of the game when Colback found the net with a deflected effort to spark jubilant scenes in the away end and leave the Premier League's bottom side just two points adrift of safety.
|By Tim Poole
Cardiff committed the cardinal sin of not killing a game off that was there for the taking against Sunderland on Saturday evening. After dominating for the majority of the encounter, the home side dropped deeper and deeper during the second half and paid the price after Steven Fletcher and Jack Colback scored late on for the visiting side.
Jordan Mutch had earlier given Cardiff the lead, while he then set up Fraizer Campbell for the second. Vincent Tan left the stands at full time as soon as he could. And so too did thousands of Cardiff fans.
There was worrying news for Sunderland, though, after reports that Fabio Borini had been taken to hospital after collapsing at halftime.
Gary Medel was restored to the Cardiff midfield as one of three changes made by interim coaches David Kerslake and Joe McBride, while Sunderland boss Poyet - without Adam Johnson out through illness - selected Jozy Altidore to lead the line in place of Fletcher.
Cardiff made a bright start and was denied a penalty in the first minute as referee Chris Foy ruled that Mutch had dived after a challenge with Valentin Roberge.
The 22-year-old was instead booked for simulation, and Mutch was again involved in the game's first real chance shortly afterwards, the midfielder meeting Craig Noone's cross with a powerful header that flew over the crossbar.
He was finally rewarded for his initiative after six minutes when he capitalized on ponderous defending from the visiting side to burst forward and fire a shot that deflected off Modibo Diakite and into the bottom corner.
Sunderland struggled to gain a foothold in the contest and owed much to goalkeeper Vito Mannone, who produced an excellent fingertip save to keep out Kim Bo-Kyung's fierce long-range drive.
The chances continued to flow for the home side as Steven Caulker, Peter Whittingham and Campbell all went close to adding to the lead.
However, it was Sunderland which perhaps should have scored the game's second goal just after the half-hour mark when Altidore somehow shot wide with the goal at his mercy after David Marshall had failed to deal with an effort from Ki Sung-Yueng.
Sunderland brought Fletcher on in place of Borini at the break - the Liverpool loanee apparently suffering from some kind of illness - and the task was made even harder 12 minutes into the second period when Campbell converted Mutch's pull back.
Lee Cattermole went close to finding a reply soon after and Fletcher frayed the home nerves as he turned home an Emanuele Giaccherini cross seven minutes from time.
That set up a late Sunderland onslaught, with Fletcher and Roberge both going close, but it was Colback who proved to be the hero in the fifth minute of injury time as his effort from the edge of the box took a wicked deflection past a despairing Marshall.