Didier Deschamps saw his side dominate the match, and though the Sociedad man struck with fewer than 10 minutes left, Caceres levelled the game late on
The Real Sociedad forward, winning only his third cap, looked to have secured a fourth successive victory for France with an 82nd-minute strike in Nice.
However, there was still time for Paraguay to level through a header from captain Victor Caceres, denying Deschamps' men another success ahead of the FIFA World Cup.
Deschamps, who was still without the injured Franck Ribery, saw his men start with plenty of energy, the front three of Loic Remy, Olivier Giroud and Mathieu Valbuena all keen to run at the Paraguay defence with the ball at their feet.
Remy however, was guilty of wasting several good opportunities in the first half.
His first sight of goal came as early as the fourth minute, but it was a neat link-up with Giroud in the 18th minute that nearly brought an opener.
The Arsenal striker met a high ball into the penalty area and cushioned a header into the path of Remy, but his strike partner was unable to control his volley from 12 yards out.
Paul Pogba shone in midfield for France, showing energy and agility as they dominated proceedings, and the 21-year-old Juventus midfielder sent two low shots fizzing wide of Antony Silva's goal.
At the other end, Hugo Lloris was rarely tested and when Paraguay did play balls into the penalty area, the France goalkeeper dealt with them comfortably.
Lloris also began a move just after the half-hour mark that led to another chance for Remy.
The ‘keeper's ball found Giroud on the right and, after beating two defenders, the striker played a cross into Remy, whose acrobatic effort flew agonisingly wide.
Having kept out France, or more accurately Remy during the first 45 minutes, Paraguay started the second half with a renewed impetus and brought the first save of note out of Lloris. After Bacary Sagna had brought down Caceres, the Tottenham goalie got down well to save Miguel Samudio's free-kick.
That was enough of a wake-up call to bring France back to life and Silva was again on hand to deny Giroud on two occasions from accurate Valbuena crosses.
With time running out France, who sorely missed Ribery's creativity, continued to apply pressure and were finally rewarded as Griezmann curled home a shot that may have taken a slight deflection on its way to goal, after Silva had saved well from Giroud.
However, the hosts were denied victory in the final minutes when Caceres met a Jorge Rojas cross, glancing a header past Lloris, although he may feel he should could have done better.
France will now hope for positive news regarding Ribery, who is a reported doubt for the World Cup as he continues to battle a back problem.