The former Blucerchiati striker converted from the spot with just seconds of normal time remaining, after Zdravko Kuzmanovic had been felled in the box by Alessio Romagnoli
The Nerazzurri looked set to be held at home for the third successive game in all competitions - which would have heaped further pressure on coach Walter Mazzarri - but Zdravko Kuzmanovic was felled by Alessio Romagnoli with just seconds of normal time remaining.
Icardi, who had been repeatedly frustrated by Sergio Romero in the Sampdoria goal, then stepped up to beat his fellow Argentine from 12 yards out to send the Giuseppe Meazza wild and end the visitors' unbeaten start to the season in the process.
Mazzarri had found himself up against one of the reported favourites to replace him at the helm, in the shape of Sampdoria coach Sinisa Mihajlovic, but he came out on top against the ex-Nerazzurri defender.
The hosts were wasteful in the first half, with Rodrigo Palacio in particular spurning several good chances, but they managed to snatch all three points at the death through Icardi to move to within a point of third-placed AC Milan.
Daniele Gastaldello was lucky to escape punishment for elbowing former team-mate Icardi in the face early on, before a wayward pass in midfield allowed Mateo Kovacic to play the ball in to Palacio, who inexplicably missed the target from close range.
Palacio was again guilty of wasting a great opportunity in the 13th minute, as the Argentine latched on to Gary Medel's incisive through-ball and poked a shot straight at Romero from 12 yards.
Inter continued to dictate the tempo as the first half progressed, but they found themselves up against an in-form Romero, who produced an excellent reaction save to keep Icardi's deflected effort out five minutes before the break.
And they were almost made to pay for their wastefulness in the final minute of the half, as Alfred Duncan struck the crossbar after pouncing on a loose ball inside the area.
The start of the second half followed a similar pattern to the first as Palacio spurned another opportunity in the 50th minute, shooting wide from six yards.
Inter's chances were seemingly dealt a blow a few moments later when Hernanes was forced off with an injury.
Kuzmanovic, the Brazilian's replacement, wasted a glorious chance with 28 minutes remaining, hitting a tame effort straight at Romero when through on goal.
But Inter continued to press and salvaged a dramatic late winner, with Kuzmanovic again central to the action.
He was tripped by Romagnoli when racing through and Icardi fired the resulting penalty into the right-hand side of the net to move Inter up to seventh and end Samp's unbeaten start at eight matches - one game shy of their club record.