The Uruguayan forward became his club's all-time leading scorer in the French top-flight when he converted a 75th-minute penalty, going clear of Zlatan Ibrahimovic in the record books.
Having halved the deficit, Cavani made sure the title winners avoided suffering a first home defeat in the league this season by heading home Thomas Meunier's cross.
The result was tough on Guingamp, who are mathematically safe from the drop and still in with an outside chance of finishing in the top five and played like a side with nothing to lose and everything to gain.
They took the lead in spectacular fashion, Ludovic Blas letting fly with a left-footed rocket to beat Alphonse Areola in the 45th minute, the ball curling away from the goalkeeper as it flashed into the net.
Jimmy Briand completed a swift counter started by Clement Grenier to double their advantage, only to carelessly give away a penalty at the other end that triggered PSG's comeback bid.
115 - Edinson Cavani is now Paris top goalscorer in the top-flight with 115 goals, overtaking Zlatan Ibrahimovic best tally (113). Matador. pic.twitter.com/R9TmHMCm1b— OptaJean (@OptaJean) April 29, 2018
Still, the result saw the runaway champions drop points at home for the first time in the league this season at the end of a week in which coach Unai Emery announced he will leave the club at the end of the campaign.
The outgoing boss witnessed his star-studded side turn in a below-par performance that drew whistles from a disgruntled home crowd.
Briand was guilty of twice missing the target in the opening 15 minutes, curling wide with his left foot before fluffing a header from Felix Eboa Eboa's delightful cross, to let a sluggish PSG side off the hook.
The hosts also needed assistance from the woodwork, Marcus Thuram's clever deflection of Etienne Didot's low drive sending the ball onto the post, to avoid conceding. However, Guingamp finally broke the deadlock – and in some style, too – in the 45th minute with a wonder strike.
PSG may have thought they were out of trouble when they headed a corner away, only for Blas to unleash a first-time drive that flew beyond a stationary Areola.
The home side, in contrast, barely threatened. Giovani Lo Celso dragged their one notable first-half chance wide with his left foot, as the home team's powerful forward line – including the fit-again Angel Di Maria – failed to catch fire in the pouring rain.
Kylian Mbappe did have the ball in the net in the 54th minute, only for his toe-poke beyond goalkeeper Karl-Johan Johnsson to be ruled out for offside.
There was no flag to chalk off Briand's finish to a counter-attack though, the veteran forward reaching double figures in Ligue 1 for the season when he stretched out to push the ball between the advancing Areola's legs.
However, Guingamp's captain gifted PSG a lifeline with an ill-judged sliding tackle on Lo Celso, the official failing to spot that the initial contact occurred outside the area.
Cavani converted to create history before going on to find the equaliser six minutes later, his diving header too strong for goalkeeper Johnsson to deny Guingamp a famous victory in the French capital.