Trailing 2-0 with 11 minutes remaining in Asuncion, under-fire coach Dunga and Brazil appeared destined for defeat after Dario Lezcano and Edgar Benitez scored either side of the break.
But despite the absence of suspended captain Neymar and David Luiz, Brazil - who coughed up a two-goal lead in their 2-2 draw at home to Uruguay last week - refused to surrender as veteran Ricardo Oliveira pulled a goal back in the 79th minute before Alves completed the last-gasp comeback with a curled shot into the far corner of the net.
The result left Brazil and Paraguay level on nine points, four adrift of leaders Uruguay.
Despite being forced into an early substitution, with Jorge Benitez making way for Roque Santa Cruz in the latter's first appearance since last year's Copa America, Paraguay - who drew 2-2 with Ecuador last time out - pegged back Brazil and were only denied by the woodwork and some Alisson brilliance.
A flick on from Richard Ortiz hit the post before goalkeeper Alisson was on hand to block the rebound in the 18th minute.
Alisson was at it again just a minute later as he produced a stunning reaction save to thwart Gustavo Gomez at point-blank range.
Brazil managed to weather the storm and almost took an unlikely lead in the 27th minute after Willian combined with Fernandinho before squaring the ball to Oliveira, who rattled the crossbar.
The Brazilians finally came unstuck five minutes before the break as Paraguay - a team with just five goals in as many matches prior to kick-off - took a deserved lead through Lezcano - his third goal in two games.
Edgar Benitez made an overlapping run down the wing and picked out Lezcano, who fired his shot into the ground and up past Alisson.
Hulk was introduced in place of Fernandinho at the break but Dunga was left fuming as Benitez went from provider to goalscorer three minutes into the second half.
Santa Cruz beat three opponents and found Ortiz, who then crossed into the box as Benitez nipped in front of his opponent, poking the ball forward and slotting underneath Alisson.
Brazil did manage to find the back of the net in the 63rd minute but Gil's header from a corner was ruled out after towering over his opponent.
They were not to be denied with 11 minutes remaining as Oliveira was on hand to tuck home the rebound after Justo Villar failed to deal with Hulk's long-range effort.
And just as Paraguay looked set to claim all three points, Alves cut into the box and beat Villar as he spared Dunga's blushes.