Christian Atsu scored in the 20th minute to cancel out Virgil van Dijk’s early opener for Liverpool.
Mohamed Salah struck back before the half hour, only for Salomon Rondon to claw the Magpies back level shortly after half-time, leaving the Reds pushing for a crucial winner to keep their title hopes alive.
But Origi headed in an 86th-minute goal to give Liverpool a 3-2 win on Saturday and see them return to the Premier League summit.
It was the second time this season that the 24-year-old Belgium international has scored a crucial goal towards the end of the game to earn a win for his team.
Origi also scored against Everton in the 96th minute back in December to earn his club a 1-0 victory.
The headed set-piece was supplied by another player who has had limited chances – Xherdan Shaqiri – and Klopp hailed the duo's impact despite their limited chances.
"So crucial. What can I say? It is just brilliant. It is like a fairy tale," he told a news conference.
"You can say whatever you want about me – they don't play often enough, Shaqiri not often enough. I make this decision.
"It is great that they do it, I love it. That was really a big moment."
Liverpool will host Wolves on the final day while Manchester City make the trip to Brighton and Hove Albion.
The Reds need a stumble from City to have any hopes of winning the Premier League, while the club face a major uphill battle in the Champions League, trailing Barcelona 3-0 after the first leg of their semi-final tie.
They will host the Spanish giants on Tuesday at Anfield as they look to make a second straight Champions League final.