The Red Devils made hard work of their League One opponents on Wednesday night with Mason Greenwood's opener cancelled out by a 76th-minute equaliser from 16-year-old Luke Matheson.
With scores locked up at full-time, Solskjaer's side held their nerve in a shootout with Sergio Romero making the decisive save against Jimmy Keohane.
The nervous win continued United's shaky start to the season with the club losing 2-0 to West Ham on Saturday to sit eighth in the Premier League.
While Solskjaer was relieved with the victory, he hinted a few big changes are required at the club to ensure it gets back on track.
"As you normally are when it gets tight, you’re happy when you win, you’re through and you learn again about different things," Solskjaer told Sky Sports post-match.
"That’s our tradition, to do it the hard way. The crowd was fantastic tonight so of course we just have to take one game at a time but also we’ve got more than one eye on the future.
"We know we have to change things throughout the club and we are doing that."
Though the win over Rochdale was scrappy, the match gave a number of young Red Devils a chance to impress with 17-year-old Greenwood once again getting his name on the scoresheet after striking in the Europa League last week.
Fellow teenagers Brandon Williams and Tahith Chong also got minutes and they were able to impress their manager.
"It’s a great stage for some of the youngsters to prove themselves and there were one or two that really did," Solskjaer said.
"I think Mason shows again he’s a goal-scorer and you’ve seen Brandon coming on, really assured performance on his debut, fantastic debut."
Going within inches of a massive upset, Rochdale were praised by the United boss for their unique style of play.
"The big plus for everyone I think should be Rochdale’s way of playing football," Solskjaer said.
"I’ve got to say, when we’ve analysed them the last few days, the way they want to play, it’s a bit unorthodox, but [manager Brian Barry-Murphy] trusts his players, he wants them to play and I thought they were excellent."
The Red Devils have been drawn against Chelsea in the next round of Carabao Cup to be played in late October.