Despite leading through Sergio Aguero's 18th-minute effort, City were behind at the break after Nabil Bentaleb scored a pair of spot-kicks.
The first of those, awarded after a lengthy period of deliberation and confusion, was given for a handball against Nicolas Otamendi, who was booked for that transgression and then picked up a second yellow card with 22 minutes remaining.
Pep Guardiola appeared content to settle for just a one-goal deficit at that point, yet substitute Sane had other ideas, brilliantly firing in an 85th-minute free-kick to equalise against his former club, and Sterling then latched on to Ederson's long clearance to score a 90th-minute winner on a wild night in Gelsenkirchen.
It looked like a straightforward outing was in store for City when they went ahead, David Silva robbing Salif Sane on the edge of the box and squaring for Aguero to roll home a simple finish.
Yet Guardiola's side were rocked by two penalties awarded in the latter stages of the half.
Referee Carlos del Cerro Grande initially gave a corner when Daniel Caligiuri's shot crashed into Otamendi's right arm - which the Argentinian had tried to withdraw - only for the decision to be overturned following a VAR review.
Due to a reported fault with the pitch-side monitor, it took almost three minutes for that call to be made, and another two before Bentaleb sent Ederson the wrong way from 12 yards.
And five minutes later, Schalke had another penalty as Fernandinho bundled into Salif Sane at a free-kick, Bentaleb dispatching this spot-kick high into the top-left corner even though Ederson got a hand to his effort.
City's night took another turn for the worse in the second period when Otamendi was dismissed after upending Guido Burgstaller, and Guardiola immediately responded by withdrawing Silva for Vincent Kompany.
However, it was another substitute, Leroy Sane, who would spark a comeback against his former employers by blasting a free-kick into the top corner from 25 yards out.
And Sterling had the final say, shrugging off Bastian Oczipka and collecting Ederson's long ball to finish past Ralf Fahrmann as City turned the game, and the tie, on its head.
What does it mean? City in charge after dramatic finish
Trailing 2-1 with five minutes to go and down to 10 men, Guardiola would surely have settled for a one-goal deficit ahead of the second leg in three weeks' time. But City now have one foot in the last eight thanks to Sane and Sterling's late goals. Schalke will be kicking themselves for not hanging on.
Ex-Schalke man Sane stuns German side
It was at Schalke where Sane made his name, though he had to initially settle for a place among the substitutes on his return. When City were awarded a free-kick around 25 yards out late on it was Sane who wanted to step up. Most of those who saw Sane in a darker shade of blue must have feared what was coming and he provided an emphatic finish.
Otamendi nearly costs City
There may have been little Otamendi could have done about his first caution – though most defenders put their hands behind their backs to avoid such possibilities – yet his second booking was for a stupid foul on Burgstaller, who was going nowhere. He should be the first to thank Sane and Sterling.
Key Opta facts
- Raheem Sterling's goal was Manchester City's first 90th-minute winner in the Champions League since Kevin De Bruyne against Sevilla in October 2015.
- City are unbeaten in their last eight Champions League games against German opponents (W7 D1), since a 1-0 defeat to Bayern Munich in September 2014.
- Schalke have seen their nine-game unbeaten home run in Europe ended by Manchester City (W6 D3 previously), with their last loss coming in February 2016 against Shakhtar Donetsk.
- City striker Sergio Aguero has scored 10 goals in his last seven appearances in all competitions, and is the top goalscorer in the top five European leagues in 2019 (11).
- Leroy Sane has been directly involved in eight goals in his last seven appearances in all competitions for City (three goals, five assists).