Goals from Islam Feruz and Olivier Ntcham ensured the tie was all square after 120 minutes as the visitors prevailed in the shoot-out at the Etihad Stadium
Ruben Loftus-Cheek score was the hero for the Blues scoring the winning penalty as Chelsea go through to meet Liverpool or Manchester United in the final.
The hosts began the game strongly and had a couple of good chances in the early stages. Seko Fofana clipping a superb through ball for Marcos Lopes but Jamal Blackman in the Chelsea goal was alert and off his line quickly to clear the danger.
Minutes later and Shay Facey put Jordi Hiwula through on goal but again Blackman was on hand to deny him.
Chelsea then took the lead on 12 minutes when Islam Feruz headed the hosts into the lead.
Lewis Baker was the creator, with a great burst forward, before rounding the keeper and clipping the ball to Feruz at the far post who headed back across goal and into the net.
Ten minutes later and Chelsea could have doubled their lead. A mistake in the City defence allowing Isaiah Brown to nick in but his cross shot flew inches wide with the goal gaping.
Manchester City came back into it, however, and were a little unlucky not to equalise before the break after creating some good chances.
First Adam Drury did superbly to get to the byline and cross but the ball just evaded the incoming Sinan Bytyqi. Fofana was then in the thick of the action once again, sliding the ball into Lopes whose shot at goal beat Blackman, but just crept the wrong side of the post.
City came out strongly after the break with Lopes missing a glorious chance to level within five minutes of the restart.
The captain was put through on goal with just Blackman to beat but put his shot far too close to the goalkeeper, who managed to block.
Chelsea continued to pose at threat and substitute Kasey Palmer almost added a second on the hour mark, firing just wide after good work by Feruz before George Glendon was forced to hook Ruben Loftus-Cheek's header off the line from a Blues corner.
However, City got back on level terms with just 15 minutes of normal time remaining, a surging run from Thierry Ambrose ended with Olivier Ntcham turning in the box and firing home strongly from close range to make it 1-1.
Having pulled level City almost went ahead with five minutes to go, Ambrose with a fiercely-struck shot which was tipped over by a flying Blackman.
The action then switched immediately to the other end with Baker denied by a brilliant block by Plummer before Feruz smashed a shot wide from close range in a frantic finale as the game ended all square.
Extra-time brought few chances for either side, although the hosts looked the brighter and thought they had won a penalty midway through the extra period, only for the referee to correctly book Ambrose for simulation after going down with just Blackman to beat.