Nigel Adkins' side was knocked out of the Capital One Cup at Elland Roads, with the home team impressing throughout
The first half saw Luke Varney waste a number of chances, but after good work from the forward, Tonge scored from the edge of the area, with the hosts dominating.
Little changed after the break, with Varney wasteful and Southampton uninterested until the final 15 minutes, and Leeds held on to its second Premier League scalp of the competition.
The visitors made 11 changes from the side that lost 2-1 to Tottenham at the weekend with key players such as Jay Rodriguez, Rickie Lambert and Adam Lallana not even appearing on the bench.
Neil Warnock also made changes to his Leeds side, but not as extreme as his opposite number. Just three players missed out from the team that lost against Birmingham at the weekend, with David Norris, Danny Pugh and Luciano Becchio replaced by Aidan White, Michael Brown and Luke Varney.
The match started off at a good pace with both sides wanting possession, but it was Leeds which looked the more dangerous, with right back Sam Byram looking a particularly potent weapon as he surged forward at every opportunity.
Around the 15 minute mark the first chance of the match came for Leeds, and how it did not take the lead will remain a mystery. El-Hadji Diouf knocked a deep delivery back across goal and just when Varney looked set to score in an open goal, the forward put the ball wide of the post from just two yards out.
Varney went close on two more occasions, but he eventually made up for those misses around the half hour mark as he beat Daniel Seaborne before firing into the area, and although Kelvin Davis made the save, Tonge fired in the rebound on the edge of the area, with Leeds continuing to dominate through to halftime.
The players came out after halftime and one would have been forgiven for thinking the Saints would show enthusiasm, but the pattern of the match stayed the same, with Leeds very much on the front foot.
Chances were at a premium and the hosts generally controlled possession, but Saints did eventually have a decent chance as Jason Pearce’s poor clearance fell kindly for Emmanuel Mayuka in the area, but after bringing it down poorly, his strike went well wide.
Nigel Adkins’ charges showed a bit more urgency in the final ten minutes, with a Richard Chaplow cross nearly dropping nicely for Mayuka, while a Lloyd Isgrove delivery was forced behind by Tom Lees, but Leeds held strong.
Leeds secured the win in the 87th minute, with Rodolph marching forward brilliantly before shooting on the edge of the box, and although Kelvin Davis made the save, in came Diouf to knock in the rebound before Becchio smashed in a penalty in stoppage time.
Follow GOAL.COM USA on