After Twumasi's side lost 5-0 in the first leg, Celtic's Kristoffer Vassbakk Ajer scored an own goal to hand the home side the lead on 23rd minute, reigniting hope of overturning the deficit.
However, Scott Sinclair equalised for the visitors before Serikzhan Muzhikov made it 2-1 three minutes from the break to restore Astana's lead.
Twumasi then scored twice by the 49th and 69th minutes respectively to push the scoreline to 4-1.
However, two quick goals in the last ten minutes of the game from Celtic’s Olivier Ntcham and Leigh Griffiths reduced the deficit to 4-3, as the Kazakhstan champions crash out of the competition.
Although, Astana are out of the race, they were able to achieve an enviable feat by ending Celtic's unbeaten away record of 26 games in all competitions since September 2016.
They have also become fourth club to score four or more goals against Celtic in the Champions League since 2000 aside FC Petrzalka, Barcelona, Legia
Twumasi will now play in the Europa League playoffs for a place in the group stage.