Kroos found the net with a wickedly curling set-piece in the 95th minute of a game that brought out the best and worst of Joachim Low's world champions, who were reduced to 10 men when Jerome Boateng was sent off with eight minutes remaining.
It capped a dramatic turnaround in a match that Germany could not lose after Die Mannschaft were beaten by Mexico in their opening Group F game, but such a scenario was on the cards when Ola Toivonen's superb lob gave Sweden a lead they held until half-time.
But Low's team-talk at the break obviously struck a chord with the world champions, who levelled early in the second half through Marco Reus' scrappy effort.
Boateng's dismissal threatened to unravel their efforts to get back into the match and it looked as though they would have to settle for a point that would have left their hopes of progressing from the group hanging in the balance. However, with the clock winding down, Kroos' stunner sparked jubilant German celebrations and simultaneously broke Swedish hearts.
The result leaves Group F wide open with no team yet mathematically through. Mexico lead the way on six points, while Germany and Sweden are on three, and South Korea still on zero.
Germany dazzled Sweden with lightning-fast passing in the opening exchanges and Julian Draxler had a shot blocked from close range before he toe-poked a left-footed effort narrowly wide from a tight angle soon after.
Emil Forsberg was pulled back by Thomas Muller on Sweden's first foray into German territory but no free-kick was given and, after Victor Lindelof cleared a Reus cross away from danger, Janne Andersson's men were denied what looked like a clear penalty.
Viktor Claesson's pass put Marcus Berg through one-on-one with Manuel Neuer when Boateng appeared to bundle the Sweden striker to the ground without playing the ball, but referee Szymon Marciniak ignored Sweden's ardent protests.
Germany lost Sebastian Rudy when he left the field with a bloody nose after a collision with Toivonen moments before the Sweden forward landed another blow as he burst into the box to control a Claesson cross on his chest and lift his shot over Neuer into the far corner of the net in some style.
Substitute Ilkay Gundogan and Berg exchanged dangerous long-range shots at the end of the first half and Low sent on Mario Gomez for Draxler at the start of the second half.
Less than three minutes later Germany drew level when Timo Werner's cross evaded Gomez but found Reus, who met it with his knee and turned the ball beyond Robin Olsen and into the bottom corner of the net.
Low's men did not stop there, and Muller headed narrowly wide before Jonas Hector forced Olsen into a good save to keep the scores level.
Reus should have made it 2-1 when his attempted back-heel failed to connect with Joshua Kimmich's low cross.
With just under 10 minutes left, Boateng was dismissed after picking up a second yellow card for a dangerous tackle on Berg, and Neuer then made a dramatic one-handed save to deny substitute John Guidetti a winning goal with a downward header.
Olsen pulled off an even better save at the other end, parrying Gomez's powerful header over the crossbar, and Julian Brandt crashed a thunderous shot onto the post from an offside position as time looked to have run out on Germany.
But Kroos hooked a sweetly struck effort from a well-worked free-kick with Reus on the left into the far corner of the net from a tight angle to allow his country to breathe a huge sigh of relief.
Key Opta stats:
- Germany came from behind at half-time to win a World Cup match for the first time since 1974 – which was also against Sweden.
- Toni Kroos' winning goal after 94 minutes and 42 seconds was the latest goal scored by Germany in World Cup history (excluding extra-time).
- Sweden's Ola Toivonen failed to score in 23 Ligue 1 games for Toulouse in 2017-18, attempting 19 shots without success – he scored with his second shot at the 2018 World Cup.
- This was the first match at a major international tournament (Euros and World Cups) that Mesut Ozil had not started for Germany since he made his debut in 2010, ending a run of 26 consecutive starts.
- Sweden have lost a World Cup group stage match for the first time since June 1990, when they lost to Costa Rica – this defeat ends a run of 10 group games unbeaten.