The two local rivals had been held to a gripping, tense stalemate heading into the 96th minute, but substitute Origi took advantage of a Jordan Pickford mistake to give the Reds a 1-0 victory.
Origi reacted quickest when Everton shot-stopper Pickford spilled a speculative ball from Virgil van Dijk on to his goalline.
The visitors conceding in the hotly contested derby game sent the home fans - and their manager - into ecstatic celebrations.
Klopp, who had been animated on the Anfield sidelines all match, couldn't contain his emotion at the late winner and ran on to celebrate with goalkeeper Alisson.
Wow!!! Jurgen Klopp charges onto the pitch to celebrate with Alisson! 😱— Soccer AM (@SoccerAM) December 2, 2018
Liverpool score a bizzare LATE goal to win the Merseyside derby 1-0. Simply incredible!🤯 pic.twitter.com/7NM4P0pltV
Klopp pumped his fists as he leapt for joy following a hug with Alisson, before returning to celebrate with his coaching staff.
After the match, the former Borussia Dortmund coach said sorry for his pitch invasion, saying he wasn't able to control his emotions following Origi's strike.
"I have to apologise because I didn't want to be disrespectful but I couldn't stop myself," he told Sky Sports. "It just happened!
"The game was with unbelievable intense. Both teams had chances. Both goalkeepers had outstanding performances. In the end it was a very weird goal.
"I can't imagine the disappointment of Everton. But we got it and we will take it. It is a brilliant night. If I could describe it [how he feels] then I would have control over it.
"We don't want to look for excuses but that's how it is. I'm not sure what the FA has to do in a situation like this but I will have to wait.
"I didn't want to be disrespectful, [but] I couldn't stop myself. I don't know how it happened.”
When asked if it was deliberate that he ran to Alisson, who saved crucial Everton efforts to keep Liverpool level, Klopp replied: “No, the plan was not to run!
“But he was surprised – you should’ve seen his eyes!”
Origi told Sky Sports after the match: "I’m enjoying the moment. It’s a special moment for the club and city, and I’m enjoying that.
"It was just instinct. I tried to be sharp and in the end Virgil gave a nice assist! I just want to try to help the team, and I’m happy that I could do that today."
Van Dijk, who bagged the unexpected assist, was thrilled with the emotional finale in what he felt was an even clash that the Reds deserved to edge.
“[The end] was fantastic, it is such a great feeling to win," he told Sky Sports. "We had big chances.
"They were a good side today but we were better. We are happy to keep a clean sheet and score this goal. I am very happy.
"Last-minute goals are always a bit special. It was a tough game. They gave us a good game but we deserved the win and I’m glad we did."
The victory sent Liverpool back to within two points of reigning champions and table-toppers Manchester City, who defeated Bournemouth 3-1 on Sunday.