Andrei Finonchenko and Sergey Khizhnichenko struck on either side of halftime to give the Kazakh champions a famous win and leave Neil Lennon's men in deep trouble.
Andrei Finonchenko gave the Kazakh champion the lead on 12 minutes, latching onto a monstrous long throw from Gediminas Vicius to poke home. Sergey Khizhnichenko headed in a second for the home side 13 minutes from time to leave Neil Lennon's men needing a heroic display when the two sides meet again at Celtic Park in eight days' time.
Lennon made three changes to the side that defeated Aberdeen on Saturday, with Virgil van Dijk making his first start since his offseason move from Eredivisie side Groningen. Mikael Lustig and James Forrest also came into the starting XI.
Viktor Kumykov made 10 changes to the Karagandy team that beat Astana 2-1 on Saturday, and was clearly keen to focus on the team's Champions League aspirations.
It was Celtic that made the brighter start, pressuring goalkeeper Alexandr Mokin at corners, although he did make an excellent point-blank save from a Georgios Samaras header.
Fraser Forster had been a bystander in the Celtic goal until Karagandy opened the scoring with 12 minutes gone.
The visitors failed to clear their lines from a Vicius long throw and Finonchenko ghosted in between Joe Ledley and Emilio Izaguirre to poke home from close range.
Kris Commons came close to producing a swift response when he unleashed a fierce 25-yard effort that struck the crossbar and bounced on the line before spinning safely into the hands of Mokin.
Karagandy posed a considerable threat in the air and missed a good opportunity to double the advantage when an Aleksandar Simcevic center was headed off target by Sergei Maliy. The hosts were nearly punished on the half-hour mark when Samaras glanced a Charlie Mulgrew freekick from the left-hand side just wide of the far post.
Celtic knew it needed to try and work harder to get a foothold in the game, and should have equalized in the 52nd minute. Izaguirre did well to cut in from the left-hand side, but he blazed well over the crossbar on his weaker right foot.
Celtic should have been level with just less than 20 minutes remaining when Forrest timed his run into the box to perfection to meet Izaguirre's curling cross, but he could only head straight at the goalkeeper.
That miss proved costly, as Karagandy grabbed a second when more poor defending from Vicius' effort led to Khizhnichenko having the simplest of headers from six yards to send the home fans into raptures.