Andrei Finonchenko and Sergey Khizhnichenko struck either side of half time to give the Kazakh champions a famous win and leave Neil Lennon's men in deep trouble
Andrei Finonchenko gave the Kazakh champions the lead on 12 minutes, latching onto a monstrous long throw from Gediminas Vicius to poke home.
And Sergey Khizhnichenko headed in a second for the home side 13 minutes from time to leave Neil Lennon's men needing a heroic display when the two sides meet again at Celtic Park in eight days' time.
Lennon made three changes to the side that defeated Aberdeen on Saturday, with Virgil van Dijk making his first start since his close-season move from Eredivisie side Groningen. Mikael Lustig and James Forrest also came into the starting XI.
Viktor Kumykov made 10 changes to the Karagandy team that beat Astana 2-1 on Saturday, and was clearly keen to focus on the team's Champions League aspirations.
It was Celtic who made the brighter start, pressurising goalkeeper Alexandr Mokin at corners, although he did make an excellent point-blank save from a Georgios Samaras header.
Fraser Forster had been a bystander in the Celtic goal until Karagandy opened the scoring with 12 minutes gone.
The visitors failed to clear their lines from a Vicius long throw and Finonchenko ghosted in between Joe Ledley and Emilio Izaguirre to poke home from close range.
Kris Commons came close to producing a swift response when he unleashed a fierce 25-yard effort that struck the crossbar and bounced on the line before spinning safely into the hands of Mokin.
Karagandy posed a considerable threat in the air and missed a good opportunity to double their advantage when an Aleksandar Simcevic centre was headed off target by Sergei Maliy.
And they were nearly punished on the half-hour mark when Samaras glanced a Charlie Mulgrew free-kick from the left-hand side just wide of the far post.
Celtic knew they needed to try and work harder to get a foothold in the game, and should have equalised in the 52nd minute.
Izaguirre did well to cut in from the left-hand side, but blazed well over the crossbar on his weaker right foot.
And Celtic should have been level with just under 20 minutes remaining when Forrest timed his run into the box to perfection to meet Izaguirre's curling cross, but he could only head straight at the goalkeeper.
That missed proved costly as Karagandy grabbed a second when more poor defending from Vicius' effort led to Khizhnichenko having the simplest of headers from six yards to send the home fans into raptures.
Celtic's hopes now hang by a thread and they will look to overturn the deficit at Celtic Park on August 28.