Two late goals gave Arsenal a win on the road.
Arsenal looked down and out after Andy Najar's 71st-minute header gave Anderlecht a deserved lead in the Group D UEFA Champions League encounter.
But Anthony Vanden Borre hit the post and Anderlecht wasted other late chances, for which the club was made to pay.
Gibbs first drew Arsenal level in the 88th minute, volleying home at the back post before Podolski rifled in from close range in the 91st minute.
Chancel Mbemba Mangulu could not clear Alexis Sanchez's cut back and the rebound fell invitingly to the Germany international, who sent the away fans wild when he fired home from four yards.
The win means Arsenal is in a strong position to qualify from Group D, with the last-gasp triumph putting the club five points clear of both Anderlecht and Galatasaray - which was beaten 4-0 by Borussia Dortmund - in second.
With Wojciech Szczesny suspended and backup David Ospina sidelined with a thigh injury, Arsene Wenger handed a start in goal to former Argentina youth international Emiliano Martinez.
Martinez conceded five times in his last Arsenal appearance, an incredible 7-5 win over Reading after extra time in the League Cup's last 16 in Oct. 2012.
But a repeat never looked likely in an uninspiring first half, which saw Arsenal shade the contest but fail to create many meaningful chances.
The closest Arsenal came was in the 16th minute, when Danny Welbeck headed Alexis' cross over after a quick break, while the Chilean, Santi Cazorla and Per Mertesacker all missed the target with opportunities.
Anderlecht was far from threatening, either, and a series of blocks and defensive headers from Arsenal meant it took until the 39th minute before Martinez was tested, as he rushed off his line to deny to Dennis Praet from an acute angle.
Alexis had a good chance to give Arsenal the lead five minutes into the second half, but with his back to goal, his flick - which came off his shoulder - could not beat goalkeeper Silvio Proto from Cazorla's free kick.
Aaron Ramsey then fired wide after Alexis' cut back, before Gohi Cyriac shot at Martinez at the other end.
Anderlecht was growing into the game, as Praet fired over, and shortly after Cazorla's close call - which saw Proto deny him with an excellent save - the home side took the lead.
Najar inspired the sweeping counterattack, bursting down the right before feeding Praet, who composed himself and swung in an excellent cross.
Najar continued his run, nodding a free header past Martinez and into the bottom-right corner.
Vanden Borre could have wrapped up the win for Anderlecht with 11 minutes left, but his mis-hit effort struck the post after good work from Najar and Praet, and Arsenal made the home side pay - Gibbs volleying a side-footed finish past Proto from Calum Chambers' cross.
The goal lifted Arsenal and the Gunners surged forward in search of a winner - one that came from the boot of Podolski.