Striker scores deflected winner five minutes from time after the Championship side had twice equalised as Olivier Giroud scores a double for the visitors
The Arsenal forward was rested by manager Arsene Wenger but saved the Gunners' blushes five minutes from time with a deflected strike.
Olivier Giroud had twice given Arsenal the lead but the Premier League side were pegged back on both occasions by their Championship opponents through goals from Ashley Barnes and debutant Leonardo Ulloa.
The result was harsh on Brighton, who knocked out Newcastle in the third round, but Arsenal showed fighting spirit of their own as the north London club seek their first trophy in eight years.
The hosts started brightly as they looked to exploit Arsenal’s weakness on the left side, where Andre Santos endured yet another torrid afternoon in a Gunners shirt.
Brighton had the first real opportunity of the match in the 15th minute when David Lopez lifted a ball over the top to Ashley Barnes, who was played onside by the Arsenal defence but saw his fierce shot brilliantly saved by Wojciech Szczesny.
Less than a minute later, Arsenal were ahead. The visitors broke rapidly from the resulting corner and when Giroud took the ball from Lukas Podolski on the edge of the penalty area, he knew exactly what he was going to do.
The Frenchman worked half a yard of space and launched an unstoppable left-footed strike into the top corner.
Yet the Gunners were struggling to cope with the intensity of Brighton’s play as Poyet’s side set the tempo and pressed for an equaliser.
It arrived in the simplest of circumstances in the 33rd minute. David Lopez swung in a cross from the right and Barnes was allowed to rise unmarked to head home from inside the six-yard box as Gunners goalkeeper Wojciech Szczesny flapped at thin air.
Arsenal began the second half with more purpose, Podolski rattling the crossbar with a thumping effort before Giroud restored the visitors' lead in the 56th minute with his second goal of the game.
Abou Diaby played a lovely long ball over the Brighton back four, which Giroud plucked out of the air with a gorgeous first touch and then steered his finish past Casper Ankergren on the stretch.
Giroud forced Ankergren into a sharp save as he sought a hat-trick after running on to Tomas Rosicky’s long pass, but Brighton continued to threaten in wide areas.
In the 62nd minute, they were level again. Barnes turned provider this time, whipping in a superb cross from the right, where Santos was again AWOL, for Ulloa to score a debut goal with a diving header.
Wenger responded by introducing Walcott and Wilshere from the bench - and in the 85th minute the striker made the difference with the winning goal.
Brighton failed to properly clear an Arsenal corner and the ball fell to Walcott on the edge of the area, whose shot took a huge deflection off Adam El Abd before nestling in the corner.