After 120 minutes of action ended in a 1-1 draw, Jonas Hector netted the winning spot-kick after seven players failed to convert in an unbelievable sequence of penalties on Saturday.
Simone Zaza, Graziano Pelle, Leonardo Bonucci and Matteo Darmian all missed for Germany, who saw Thomas Muller, Mesut Ozil and captain Bastian Schweinsteiger also fluff their lines.
Ozil looked to have put Germany on course for victory in normal time, but Bonucci's late penalty – after a handball from Jerome Boateng – ensured the game progressed to extra time and then spot-kicks.
The thrilling win helped Germany gain a measure of revenge for a run of eight consecutive defeats to the Azzurri in major tournaments, including a semi-final loss four years ago at Euro 2012.
World champions Germany are now the first team to make the last four at six consecutive major tournaments.
Joachim Low's side will face either hosts France or Iceland in Marseille next Thursday when they will bid to make a record seventh European Championship final, although Mats Hummels will miss the match through suspension after picking up a booking against Italy.
Germany switched to three at the back with Benedikt Howedes replacing Julian Draxler, but they suffered a blow when Sami Khedira limped off after only 16 minutes as skipper Schweinsteiger came on to replace him.
Schweinsteiger, whose outing saw him equal Miroslav Klose's European appearance record at major tournaments, had a goal disallowed for shoving Mattia De Sciglio to the ground before getting on the end of Hummels' cross and planting a header beyond Gianluigi Buffon.
Germany had the best chance of a quiet first half when Muller scuffed a presentable opportunity straight at Buffon, and the Bayern Munich man was denied by a flying Alessandro Florenzi on the goal-line shortly after the interval.
But the breakthrough did arrive with 25 minutes remaining when Mario Gomez slipped a pass through to Jonas Hector down the left and his cross deflected off Bonucci, allowing Ozil to stab home from eight yards for his first goal of the tournament.
Italy stayed in the game when a magnificent reaction save from Buffon prevented Giorgio Chiellini's tackle on Gomez from flying into the net, but Boateng gave them a lifeline after 77 minutes.
Boateng inexplicably raised his hands to block a Chiellini flick and referee Viktor Kassai immediately pointed to the spot.
Bonucci had not scored for his country since September 2014, but he confidently found the bottom-right corner from the spot despite Neuer guessing correctly as he conceded for the first time in the competition.
Germany substitute Draxler came closest to finding an extra-time goal when his flick narrowly missed the target before he over-hit a pass when he could have released Muller on the break a few moments later.
After both teams scored their first spot-kicks thanks to Lorenzo Insigne and Toni Kroos respectively, unbelievable drama ensued.
Zaza had been brought on in the final seconds of extra time to take a penalty, but he blazed over after a bizarre run-up, before being relieved to see Buffon immediately save from Muller.
Andrea Barzagli's successful conversion was then followed by Ozil striking the post, but, astonishingly, Pelle tamely struck wide to allow Draxler to level the shoot-out.
Bonucci was then thwarted impressively by Neuer, giving Schweinsteiger the chance to seal victory, only for the midfielder to incredibly shoot way over as the first five penalties ended with the score level at 2-2.
Emanuele Giaccherini, Hummels, Marco Parolo, Joshua Kimmich, De Sciglio and Boateng all held their nerve as tension reached breaking point.
But Darmian, Italy's ninth taker, was soon thwarted by Neuer and Hector stepped up to score low under Buffon - who left the pitch in tears - and send Germany's players wild.