The game looked to be heading for a draw before the Fortuna Dusseldorf striker slotted home an 86th minute spot kick.
The Fortuna Dusseldorf striker made up for a first-half miss from the spot when he converted a controversial 86th minute spot kick after Essaid Belkalem was controversially adjudged to have handled inside the area.
Jonathan Pitroipa opened the scoring for Burkina Faso on the verge of halftime but the lead was quickly canceled out by Sofiane Feghouli.
Djakaridja Kone put the home side back in front just after the hour, but Algeria again responded quickly through Carl Medjani's towering header.
Bance scored with just four minutes to play though, as his late penalty delighted the home crowd in Ougadougou.
Burkina Faso now has the advantage heading into the second leg, to be played in Blida on Nov. 19.
Both sides struggled to create chances of note in the opening 30 minutes of the match, however as the game drifted towards the break it burst into life.
First, Burkina Faso survived a penalty shout after Feghouli went down under a challenge from Jean-Noel Lingani, however the Valencia midfielder saw his penalty claims waved away by the referee.
A minute later the referee was pointing to the spot, but at the other end, after Pitroipa was brought down by a rash challenge from Belkalem.
Bance stepped up to take the penalty but hit a tame effort that Algeria goalkeeper Adi Mbolhi saved relatively comfortably.
Luckily for Bance, his miss was quickly forgotten as Pitroipa struck in stoppage time at the end of the half.
Lingani delivered a cross into the penalty area from deep and Pitroipa – who had got away from Madjid Bougherra – headed past Mbolhi to give the home side the lead.
The visiting side was level early in the second half as Feghouli netted his third goal of qualifying with a low shot after neat link up with Islam Slimani inside the penalty area.
Despite Algeria's second-half dominance the team found itself behind again in the 65th minute as Kone produced a moment of magic.
The midfielder controlled a loose ball on his chest, took the ball past Mehdi Mostefa-Sbaa and lashed the ball into the bottom corner for his second international goal.
Burkina Faso's lead only lasted three minutes though as Medjani rose to meet a left-wing corner to head past goalkeeper Daouda Diakite.
The home team would not be denied though, claiming the winner with four minutes left after Belkalem was penalized for handling the ball.
It was not referee Janny Sikazwe who gave the penalty though, instead his assistant, and although Mbolhi got a hand to Bance's effort, it was second time lucky for the Burkina Faso man.