Nketiah, who made his debut for the Gunners against BATE Borisov earlier this season, was introduced in the 85th minute at the Emirates Stadium with his side trailing to Josh Murphy's cool first-half finish.
The youngster made an immediate impact, equalising with almost his first touch to send the game into extra-time.
He then ensured the headlines would be his in the 96th minute with a superb header from Mohamed Elneny's corner to secure Arsene Wenger's side's place in the next round.
Norwich – unbeaten in nine in all competitions going into the game – were the better side for long spells and should have been out of sight in the second half, but Murphy, Nelson Oliveira and Mario Vrancic all squandered golden opportunities.
Wenger named Olivier Giroud, Theo Walcott, Jack Wilshere and Alex Iwobi in his starting line-up, but none of them could make an impression against a stubborn Norwich defence, which only wilted following the introduction of Nketiah.
Norwich, buoyed by their 1-0 derby win over Ipswich Town on Sunday, started in confident mood and comfortably kept Arsenal at arm's length in the opening exchanges.
Their positive display was rewarded in the 34th minute when Murphy netted his fourth EFL Cup goal of the campaign.
The 22-year-old latched onto James Maddison's through ball and clipped a cool finish over debutant goalkeeper Matt Macey to give the visitors a deserved half-time lead.
Any thoughts that they might sit back and hold onto their advantage in the second half were soon dispelled by more impressive attacking play.
An incisive break in the 55th minute should have resulted in a red card for Elneny, but referee Andrew Madley only gave him a caution for a foul on Oliveira despite the Egyptian being the last man.
Norwich then missed a hat-trick of opportunities to put the game out of sight. First, Oliveira blasted over after 69 minutes before Murphy sent a half-volley wildly over with only Macey to beat.
Vrancic then inexplicably fired wide from close range after taking two Arsenal defenders out of the game with a fine dummy.
They were made to pay for that profligacy in the 85th minute when substitute Nketiah stabbed home from close range to take the game into extra-time.
Theo Walcott was denied by a fine Angus Gunn save, but from the resulting corner Nketiah rose highest to send a thumping header into the top corner.
Norwich were denied a late penalty when James Husband was hauled down in the area by Mathieu Debuchy, as Arsenal held on thanks to Nketiah's memorable intervention.