In a controversial match, Sergio Aguero had a penalty appeal turned down and a goal disallowed by referee Craig Pawson, who also rejected Sanchez's demands for a first-half spot-kick.
City striker Aguero did score for the 12th time in as many games to put City ahead just after the hour mark, but Nacho Monreal hit back with his first goal in over two years.
Yaya Toure and Fernandinho both hit the woodwork for City as the match moved into extra time, where Sanchez's strike proved decisive.
Arsene Wenger is under intense pressure with his Arsenal contract due to expire in June, but he now has the chance to win his seventh FA Cup, which would give him the outright record, following up on his side's 2014 and 2015 triumphs.
City had won their last eight FA Cup semi-final appearances, but this defeat means that, despite huge expectations upon his arrival, Pep Guardiola is almost certain to finish a managerial campaign without a trophy for the first time in his career.
4 - Alexis Sanchez is now Arsenal's all-time top scorer in games at Wembley (4 goals). Hero. pic.twitter.com/HoxR5i93fv— OptaJoe (@OptaJoe) April 23, 2017
Guardiola's focus must immediately switch to ensuring his side can at least salvage a top-four Premier League finish, with a crucial derby against Manchester United coming up on Thursday.
Arsenal threatened first when Olivier Giroud sent a header straight at Claudio Bravo, before Petr Cech had to work harder at the other end to save David Silva's aerial effort from Aguero's cross.
Laurent Koscielny had a goal correctly ruled out for offside after he raced onto Giroud's flick-on to beat Bravo, while City suffered a major blow when Silva went off injured after a challenge from Gabriel, Raheem Sterling replacing him.
City's mood was not helped when Aguero had a penalty claim turned down by Pawson, having gone down under slight contact from Alex Oxlade-Chamberlain in the box.
Guardiola's men then suffered more frustration, Aguero forcing Leroy Sane's curling left-wing delivery over the line at the far post, only for the officials to rule the Germany winger's delivery had crossed the line before reaching the striker – a decision that looked marginal at best.
Arsenal themselves had an appeal for a spot-kick rejected just before the break, Sanchez claiming he was tugged to the ground by Jesus Navas as he tried to latch on to Oxlade-Chamberlain's free-kick.
The Gunners, again playing with a three-man defence, had made a solid start to the second half but fell behind after 62 minutes.
Aaron Ramsey lost possession to Toure just outside the City area and it only took one long pass from the Ivorian to send Aguero racing clear of Monreal, before dinking a clever finish over Cech as the blue half of Wembley erupted.
Arsenal produced a strong response and Mesut Ozil fired wide after a neat passing move before Wenger's men found a dramatic equaliser with 19 minutes remaining.
The influential Oxlade-Chamberlain delivered a cross from the right which found the opposite wing-back Monreal racing in at the back post and he stabbed in a right-footed volley from six yards.
City almost won it when Toure unleashed a stunning 20-yard volley that was tipped onto the post by Cech.
And the woodwork denied City again in the closing stages as Fernandinho's header from eight yards crashed against the crossbar from Kevin De Bruyne's corner.
Wenger brought on Danny Welbeck for Giroud and the England striker beat Nicolas Otamendi before flashing an effort narrowly wide as the game moved into extra time.
Rob Holding was the first to squander a big chance in the additional period, sending a free header over the bar from six yards, but Arsenal did strike after the clock had ticked over to 100 minutes.
Welbeck missed Koscielny's knock-down from a corner but Sanchez pounced to take a touch and fire in from close range, the Chile international starring in an FA Cup semi-final again having netted twice against Reading in 2015.
Aguero had hobbled off for City just before that goal, with Fabian Delph coming on, meaning De Bruyne moved into a centre-forward role, but Arsenal were on top with Welbeck missing the target from a good headed chance.
Guardiola threw on Kelechi Iheanacho for the second period and City embarked on a late charge, but the Nigeria international missed the target with a header, with De Bruyne also shooting off target and Arsenal substitute Hector Bellerin brilliantly blocking a Delph effort as Wenger's men held on for a famous victory.