The Premier League newcomers overturned Edin Dzeko's 52nd-minute strike to beat last season's runners-up, with Frazier Campbell's two goals leading the way.
The victory was Cardiff's first in the Premier League. as Manuel Pellegrini's Manchester City suffered an early blow to its title ambitions in Wales.
Cardiff came from behind to secure the famous win at the Cardiff City Stadium, after Edin Dzeko fired Manchester City into the lead in the 52nd minute with an outstanding strike from distance. The lead lasted for just eight minutes, though, as Iceland international Aron Gunnarsson was on hand to net a close-range finish after Campbell saw his effort saved by Manchester City goalkeeper Joe Hart.
The ever-threatening Campbell nodded in a corner at the far post with 11 minutes to play to fire Cardiff in front, and he struck again three minutes from time to cap a terrific performance. Manchester City substitute Alvaro Negredo set up a nervy finish with a header, but Cardiff hung on for a sensational victory.
As expected, the home fans created a superb atmosphere as they celebrated their club's first home top-flight match in 51 years and first in the Premier League.
And they saw their side burst out of the blocks as Campbell had ambitious appeals for a penalty turned down in the first minute, while an early free-kick from the impressive Peter Whittingham sailed just wide. Hart then made a smart save to deny Steven Caulker from another Whittingham free kick, although the offside flag was up in any case.
Cardiff, led by the tenacious Gary Medel, continued to get in the faces of its more-fancied opponents and rarely allowed them to settle on the ball.
There were fleeting signs of Manchester City's class, as David Silva and Sergio Aguero combined excellently on the edge of the penalty area before the former saw his cross hacked clear for a corner, while Dzeko nodded wide following a neat Silva ball.
But Cardiff kept the pressure on, restricting its guest to further half chances from Dzeko and Silva, before Campbell went close twice on the stroke of halftime.
Both came from balls over the top designed to exploit Pellegrini's side for pace, with Campbell poking wide from the initial opportunity, before Hart charged off his line to save following a poor header from Javi Garcia that allowed the former Manchester United and Sunderland striker in.
Cardiff's failure to convert their chances was compounded seven minutes into the second half when Dzeko scored a terrific goal. The striker won the ball after Aguero flicked a Yaya Toure ball into his path, and he beat Cardiff goalkeeper David Marshall at his near post with a superb 25-yard effort that appeared to receive a slight deflection.
It was an advantage that lasted just mere moments, though, as Gunnarsson sent the home fans wild with his equalizer on the hour mark. South Korean Kim Bo-Kyung was the creator, skipping past an array of challenges and into the penalty area before cutting back for Campbell. And although Campbell's shot was saved by Hart, Gunnarsson followed in and was on hand to net the rebound inside the six-yard box.
Manchester City responded with a consistent spell of possession, but Toure, substitute Samir Nasri and Silva failed to put their efforts on target.
Pellegrini's men were stunned with 11 minutes to play as Campbell escaped the attention of Pablo Zabaleta following an excellent Whittingham corner and headed in from two yards. Campbell then netted another late header, and although Negredo set up a tense finale, Cardiff did enough to win and spark a raucous celebration around Cardiff City Stadium.