Desperate for a win in St Petersburg after being held to a 1-1 draw by Switzerland in their opening fixture, Brazil looked like missing out on the three points when the decision to award Neymar a spot-kick was overturned in the 79th minute.
The former Barcelona star went down softly under the challenge of Giancarlo Gonzalez, with referee Bjorn Kuipers initially pointing to the spot.
However, VAR intervention led to a reversal of the decision, as replays showed minimal contact between Gonzalez and Neymar, who had fallen backwards as if pulled down.
The incident came late in a match Brazil dominated but in which for long spells they struggled to break down a determined Costa Rica defence.
However, with the clock having ticked past 90 minutes, Coutinho and Neymar stepped forward to break Costa Rican hearts and spare Brazilian blushes, sending their side top of Group E ahead of the clash between Serbia and Switzerland.
Neymar dropped to his knees in tears on the final whistle after the tournament's most successful side came through a real test of character against a Costa Rica side whose hopes of progression are now over.
Costa Rica – rightly anticipating that Brazil would enjoy the lion's share of possession – sat deep but looked dangerous when they were given the rare opportunity to counter.
Celso Borges should have done better when he dragged an effort wide of the target from a Cristian Gamboa cutback after 13 minutes and, for all of their time on the ball, Brazil found it difficult to find a way through the Costa Rica defence.
Gabriel Jesus hammered a shot into the back of the net from an offside position in the 25th minute, rightly prompting a flag from the linesman, but the incident sparked a period of Brazil pressure, with Marcelo and Coutinho trying their luck from distance and a Neymar cross necessitating a desperate defensive header over the crossbar.
Marcelo had looked most likely to make something happen for Brazil but found Keylor Navas equal to his low drive five minutes from the break, the Real Madrid goalkeeper stooping to halt the game's first shot on target from his clubmate.
Neymar – named in the starting XI despite struggling with an ankle injury this week - confronted the referee on the pitch and in the tunnel at half-time, presumably taking objection to Costa Rica's rough-and-ready approach to dealing with him.
And Brazil returned to the field with added purpose after the break, with Jesus heading against the crossbar before Navas diverted Coutinho's follow-up wide.
Navas – a growing thorn in Brazil's side – pulled off even greater heroics to deny Neymar with a fingertip save soon after, while another Coutinho strike was directed straight at the goalkeeper.
Neymar curled around the right-hand post from 18 yards after the defence opened up for him in the 72nd minute, before the officials arrived at the correct decision via VAR to deny him a penalty.
That incident threatened to dominate the headlines until Coutinho prodded Brazil in front in the 91st minute, getting on the end of a Roberto Firmino nod down and a helpful touch from Jesus.
And Neymar was on hand to tap home Douglas Costa's delicate chipped cross even deeper into stoppage time to wrap up three points that had appeared like slipping through Brazilian fingers.
The relief was clear as the tears rolled down the face of the world's most expensive player, who surpassed Romario in third on the country's all-time list of goalscorers.
Key Opta stats:
- This was Brazil's first World Cup victory thanks to a goal scored in the 90th minute.
- Brazil have won 10 of their 11 meetings with Costa Rica in all competitions, losing the other in a friendly in 1960.
- Costa Rica are winless in their last five World Cup matches (D3 L2).
- Costa Rica had just three shots in this match, with all of those coming in the opening 15 minutes.
- Brazil's second goal was timed at 96:49 – the latest recorded 90th minute goal in a World Cup match since 1966.
- In all international matches, Brazil are unbeaten in 13 games (W9 D4 L0), conceding just three goals in that run.
- Neymar has been fouled 14 times so far in the 2018 World Cup, more than any other player.
- Neymar has scored 56 goals in 87 matches for Brazil – only Pele (77) and Ronaldo (62) have scored more.