The French champions appeared to be heading for a stalemate against their spirited opponents after 90 minutes of frustrating football for the home faithful, until the 18-year-old bundled home a rebound in injury time before the Swede settled matters just before the final whistle.
PSG boss Laurent Blanc named an unchanged side from last week's 2-1 win away to Nantes, with Mamadou Sakho not involved in the squad as rumours of a move to Liverpool build.
Visitors Guingamp were without veteran striker Cedric Faure due to a calf injury, while Gregory Cerdan was also not included.
PSG came into the game looking for their first home win of the season and almost snatched the perfect start as Ibrahimovic curled a 25-yard free-kick onto the post just two minutes in.
It was to very much set the tone for the early exchanges, with the hosts dominating much of the possession in midfield, but they struggled to unlock a determined Guingamp defence. The visitors themselves were a threat on the break as Claudio Beauvue and Sadio Diallo both went close.
Ten minutes before the break, the hosts who created a flurry of chances, with Thiago Silva’s header being denied by Mamadou Samassa in the six-yard box.
Moments later it was the post to come to the visitors' rescue, with Thiago Silva once again foiled.
Samassa was then called into action as Thiago Motta released Cavani. The Uruguayan took the ball wide of the goalkeeper before finding Javier Pastore, but the 23-year-old recovered well to deny the Argentinian.
PSG began brightly after the break, with Samassa coming to the rescue of his team again to parry away a low Cavani effort after a well-worked corner.
But the three-pronged attack of Ezequiel Lavezzi, Ibrahimovic and Cavani continued to be frustrated by a dogged Guingamp defence, with the Swede enduring a particularly difficult afternoon, skying an effort from 25 yards just after the hour mark.
Lavezzi did not enjoy much luck either, getting on the end of a brilliant ball from Motta to race away from his man, before hitting a tame drive straight at the keeper.
PSG continued to knock on the door and nearly broke the deadlock in spectacular circumstances when Cavani sent a powerful effort narrowly wide on 67 minutes, before setting up Ibrahimovic just a minute later, with the Swede also off target.
And the champions were nearly made to pay for their wastefulness in front of goal several minutes later when Mustapha Yatabare raced into the box, before shrugging off his man and firing wide.
It looked as though the visitors would hold on for a draw, until a whipped free-kick was met by Alex, whose header was parried straight to substitute Rabiot, who applied a calm finish from close range.
And just two minutes later the win was sealed, when Ibrahimovic latched onto a long ball to round Samassa and roll the ball into an empty net.