The Villans secured their third League Cup quarterfinal spot in four seasons despite losing a 2-0 lead with 20 minutes remaining, as the Belgian's second goal secured progression.
Paolo Di Canio's men looked far from out of their depth and fought back from Christian Benteke and Gabriel Agbonlahor strikes with Nicky Storey's double before the Belgian broke their hearts in the final moments off a pass from American fullback Eric Lichaj.
The tie started brightly, with Benteke wasting a decent chance for the visitors very early on when he failed to make any real contact on a Agbonlahor cross. However, it was the League One side that edged the following 15 minutes or so. Though they couldn't conjure up any clear opportunities, Di Canio's were able to boast the better share of possession.
Back came Lambert's men, though, as the likes of Andreas Weimann began to settle; a mix-up between the Austrian forward and Stephen Ireland denying a clear shot on goal once more. Swindon then finally mustered the first real chance of the night, Paul Benson passing Shay Given before dispatching a low shot across goal, however, Matthew Lowton was on hand to clear it form the line.
Di Canio was left rueing that miss just five minutes later when Benteke nodded past a helpless Wes Foderingham. Ireland made the goal with some neat work on the left side, as well providing a pin-point cross for the striker.
Though some resistance came courtesy of Matt Ritchie, who fired a number of dangerous balls into the box, Agbonlahor doubled the away side's lead with a little more than five minutes left in the half - smashing his effort past the keeper after more good play from Weimann and Ireland. And it could have been even worse when Weimann sent an effort goal-wards with the last action of the first period, but Foderingham pulled off a superb save to keep the hosts in the tie.
In similar fashion to the opening stages of first half, the Robins came out from the break fighting. Benson and Gary Roberts were both guilty of failing to connect with good crosses while Villa on the other hand looked off the pace.
As the half continued, though, Di Canio's side's tempo again died down as the visitors successfully disrupted the flow of the game; however, almost out of nowhere Roberts had a golden chance to hand the League One side a glimmer of hope, with the winger somehow heading wide from another fine Ritchie delivery.
Then, with the game looking all but settled, the hosts managed to give themselves a glimmer of hope with around 20 minutes remaining. Miles Storey, on as a substitute just moments earlier, beat Given with a smart finish.
Storey netted an improbable second less than five minutes later to draw the sides level. Yet another cross flew into the Villa box, with the replacement finishing the move off with a brilliantly improvised back-heel, sending the game into extra time.
The drama wasn't over, though, with Lambert having the last laugh. After a poor second half from the Premier League outfit, Benteke salvaged the win very late on after bringing down Lichaj's ball and keeping his cool to beat the keeper.