The French giants took the lead in the quarter-final tie at the Parc des Princes through Neymar but the visitors bounced back with late penalties scored by Yeni N'Gbakoto and Marcus Thuram.
PSG have been the dominant force in France since they were bought over by Qatar Sports Investments. As well as five of the last six Ligue 1 titles, they have won the last five editions of the Coupe de la Ligue, four straight Coupe de France titles and six Trophee des Champions.
Their incredible run in domestic cups saw them unbeaten in 49 games until Guingamp overturned the deficit in midweek.
They are the first team to get the better of them in a cup game since Montpellier did it in January 2014 in the Coupe de France.
70 - Neymar has been involved in 70 goals in all competitions with Paris in 51 games played since August 2017 (46 goals, 24 assists), with his first headed goal with PSG all comps. Monstrous. @PSG_English #PSGEAG pic.twitter.com/ZBtI5B6MtU— OptaJean (@OptaJean) 9 januari 2019
The result may have been disappointing, but PSG star Neymar may take some joy from his first ever header for the French side.
The Brazilian has now been involved in 70 goals since joining the capital club in August 2017, scoring 46 of his own and adding 24 assists.
He has needed just 51 games to reach the mark.
PSG are still comfortably ahead in Ligue 1, 13 point clear of nearest challenger Lille with two games in hand.
They also won’t have long to wait for a shot at a bit of revenge on Guingamp, with the two clubs set to face off in league play on January 19.