The tie seemed destined for a replay at Old Trafford after Park Ji-sung cancelled out Daniel Agger's opener, but the Dutchman secured his side's progression in the final minutes
Daniel Agger opened the scoring for the Reds, heading home a Steven Gerrard corner, but Park Ji-sung brought the visitors level shortly before halftime.
United enjoyed plenty of possession in midfield but struggled to find its way past the Liverpool defense and was punished when Kuyt provided the decisive strike.
Tensions between the two clubs had been high prior to this match following the fallout of their last meeting at Anfield, but while Patrice Evra was booed by the home fans throughout, the game itself proved a low-key affair until Kuyt's dramatic winner.
The battle of the opening 15 minutes took place largely in midfield. In terrific form but without his partner in crime, Nani, on the opposite flank, Antonio Valencia would be relied upon as a primary creative force for United on the right-hand side and he burst past Rodriguez before thumping a 20-yard shot against Pepe Reina's far post.
United goalkeeper David De Gea has sometimes looked uncertain when dealing with high balls into the penalty area this season and Liverpool was determined to exploit that potential weakness from the first minute. Each set-piece saw the Spaniard crowded by three or four red shirts and he was left stranded when Steven Gerrard curled a corner into the six-yard box that Agger rose to head home to give the home side the lead.
The away side dominated the ball for long periods from that point, patiently probing for a gap to take advantage of in the Liverpool defense. United found it when Valencia's flick drew Jose Enrique and Rafael into a race for the ball that the Brazilian won. The right-back cut a cross back into the penalty area into the path of Park, who hammered a first-time shot beyond Reina.
Agger and Chris Smalling both survived handball appeals in their own area early in the second half; the home side's claim on the latter incident was strongest, but both were from close range and neither could be judged as deliberate.
Minutes later Welbeck timed a run between Kelly and Martin Skrtel to perfection to bear down on Reina, but after flicking the ball past the onrushing Spanish goalkeeper, Skrtel recovered to clear into touch.
Dalglish and Ferguson threw Bellamy and Javier Hernandez into the fray as the match entered its closing stages, neither manager apparently keen on the idea of a replay.
Liverpool finished strongly, with Carroll a willing runner, and while the big center forward could not find the target himself, he provided the crucial flick-on in the 88th minute to allow substitute Kuyt to race through behind Evra and fire beyond De Gea.
The Dutchman might have put a gloss on the victory when he followed up a Carroll header that had bounced back off the post, but he sent his shot on the run wide from close range. It mattered not, as Dalglish's team held out to seal its place in the fifth round.