The Red Devils made it four wins out of four under Solskjaer , meaning the caretaker manager has equalled Matt Busby's best-ever start to a United tenure.
The visitors had to bide their time at St James' Park, with the breakthrough coming as substitute Romelu Lukaku made an instant 64th-minute impact after Martin Dubravka spilt Marcus Rashford's free-kick.
Rashford then sealed the points 10 minutes from time, producing a composed finish after good link-up play from Lukaku and Alexis Sanchez.
Solskjaer has already managed to restore the feel-good factor at the club and he admitted post-match that he doesn't want to depart at the end of his projected six-month spell at the helm.
When asked if he will leave United in May, Solskjaer replied: "I don't want to."
He went on to address matching Busby's record, before bemoaning a 'sluggish' first-half display from his team.
"I'm just thinking about the next game because if you win four you can win another four with this club," he told Sky Sports. "That's the challenge, that's the standard we're used to."
"It was a very professional performance. We never hit the heights that we can do but I felt we were in control, kept plugging away,"
"It was a great goal again by Marcus and a great reaction again by Rom – good first touch as a sub.
"In the first half it was sluggish, it was almost like a testimonial at times – too slow, too many touches and too many square passes and back
"But the big, big plus was the clean sheet."
4 - Ole Gunnar Solskjaer is only the second manager in Manchester United history to win his first four league games in charge of the club, after Matt Busby in 1946. Revival. #NEWMUN pic.twitter.com/Dt9XpCfGms— OptaJoe (@OptaJoe) January 2, 2019
Lukaku has two in two as a substitute, while Rashford was again United's stand-out forward operating centrally .
Solskjaer believes this gives him a nice problem.
"You can play with two of them, no problem," he added. "One can play on the right, they can play two through the middle. It gives me options."
Newcastle boss Rafael Benitez can only dream of such riches in attack after his team drew a blank for the third time in four matches.
"We know that the final third is the key," he told Sky Sports. "When you pay big money for players, normally you are talking about offensive players who make the difference.
"You have to give credit to our players because for 80 minutes they were in the game."