Three goals in the space of eight second-half minutes - plus a 79th-minute effort from Khouma Babacar - looked to have wrapped up the points for Fiorentina, but two late goals from Icardi set-up a grandstand finish in an incredible clash.
Having initially come from a goal behind thanks to strikes from Ivan Perisic and Icardi, Stefano Pioli's Inter looked in good shape heading into the second half.
However, despite Federico Bernardeschi squandering an opportunity from the penalty spot, Fiorentina - who had taken a 23rd-minute lead courtesy of Matias Vecino's first goal of the season - blitzed their way through Inter's ramshackle defence midway through a thrilling second period.
Samir Handanovic had made a string of fine saves prior to the 62nd minute, but his untimely slip allowed Davide Astori to level matters with a looping header before Vecino netted his second and Babacar got in on the act with two brilliantly-taken goals.
Icardi's goals either side of the 90-minute mark applied the pressure late on, but Fiorentina ultimately held firm to secure a famous victory, moving to within a point of seventh-placed Inter in the process.
Inter started well - both Perisic and Icardi testing Ciprian Tatarusanu from acute angles before Borja Valero and Babacar forced Handanovic into action at the other end.
Handanovic could do nothing to prevent Fiorentina's opener, though, as Vecino thumped in an emphatic first-time finish after connecting with Hrvoje Milic's low cross.
Fiorentina's lead was short-lived though, as Perisic restored parity at the culmination of a free-flowing counter.
Inter were ahead six minutes later - Icardi brushing aside Carlos Sanchez before dispatching a low finish into the left corner.
Handanovic had to be at his best once more on the stroke of half-time, reacting superbly to spare Miranda's blushes, and Inter's goalkeeper was the hero again after the interval.
Danilo D'Ambrosio was harshly adjudged to have brought down Babacar in the area, but justice was done when Bernardeschi drilled the resulting penalty straight at Handanovic.
Fiorentina were level soon after, however, as Handanovic's slip prevented him from keeping out Astori's header.
Fiorentina were looking rampant, and had their third a minute later when Vecino slotted home from the edge of the area at the end of a superb run.
Inter's shocking second-half display was made worse moments later, Babacar adding a fourth with a wonderfully disguised finish into the bottom-right corner and then nudging home his second of the match nine minutes later.
And despite Fiorentina's three-goal cushion, Babacar's second goal ultimately proved decisive, after Icardi tucked in Perisic's cross before bringing up his hat-trick with a powerful header in stoppage time.
There was to be more drama, though, as - in the final minute of stoppage time - Astori hooked Marcelo Brozovic's strike off the line to preserve the three points.