The Bhoys won away from home in the Champions League for the first time since 1986 thanks to the Greece forward's dramatic late winner after Juan Insaurralde's red card
Georgios Samaras scored in injury-time to secure a dramatic 3-2 victory for Celtic against Spartak Moscow at the Luzhniki Stadium.
Gary Hooper gave Neil Lennon's team the lead but two tap-ins from Emmanuel Emenike put the home side in front.
The game changed, however, when Juan Insaurralde was sent off for a challenge on Hooper and Dmitri Kombarov sent the ball into his own net to level the score after James Forrest had been denied at the end of a sweeping attacking move.
And Samaras rose to head home Emilio Izaguirre's in-swinging cross as Celtic sealed a long overdue away victory in the Champions League.
The Bhoys have struggled away from home in Europe, winning last at the continent's top table away from Glasgow in 1986 and, ahead of the contest, Lennon would be drawn only as far to say that he hoped his team could make the game in Moscow "competitive", while acknowledging that the Russian top flight was in a stronger state than its Scottish counterpart.
That assessment looked accurate in the early stages but Spartak's attacking approach left gaps for the visitors to exploit – and they did that to great effect when Victor Wanyama initiated a move which ended in Hooper sweeping home Mikael Lustig's low cross to send Celtic a surprise lead.
Spartak continued to control possession but when Celtic were able to win the ball cleanly, they counter-attacked into the acres of space left vacant in the opposition half. Wanyama was influential in that respect and Hooper found the target again after Izaguirre had released Samaras down the left, only to be denied by a flag for offside.
The breaks kept coming for Celtic but, in the meantime, Spartak were cranking up the pressure themselves. Dani Pareja's free kick was well saved by Fraser Forster but the goalkeeper could do nothing to stop Emenike, who tapped home after Ari had sent Kallstrom's superb long pass behind Kelvin Wilson across goal.
Shortly after the halfway point the turnaround was complete. Again, Ambrose and Wilson were too slow to react when the ball got behind them, this time as a result of an Ari shot which was parried weakly by Forster. Demy de Zeeuw pounced and slid the ball across goal for Emenike to touch over the line.
Hooper, though, continued to threaten and was brought down when through on goal by Insaurralde after racing on to Charlie Mulgrew's pass. After initially seeming to ignore the challenge, referee Tony Chapron finally produced a red card for the Argentine defender following the disbelieving protests of the Celtic players. Mulgrew, however, could only direct the subsequent free kick straight at goalkeeper Sergei Pesyakov.
It was not long, however, before Lennon's team made the man advantage count. Mulgrew's low cross from the left was dummied by Samaras and passed on by Hooper for substitute Forrest to shoot. The winger's effort was saved by Pesyakov but rebounded into the net off of Kombarov for an own goal.
Substitute Joe Ledley went close with a long-range effort as both teams searched for a decisive goal. Celtic, sensing a rare opportunity to pick up three points away from home, did most of the attacking but their hopes seemed to end with a Hooper effort from a difficult angle that was comfortably saved.