The managerless Championship side booked an FA Cup meeting with Premier League Wigan despite being pegged back quickly on their opening goal by Michael Keane's strike
The managerless visitors took the lead through Adam Clayton before Michael Keane equalised with a matter of minutes. However, caretaker boss Mark Lillis's men sealed their passage through to the next stage when Sean Scannell grabbed the winner late in the second half.
Proceedings got off to a frantic start, the visitors taking the lead through a Clayton free kick after Scannell had been fouled by Wes Morgan, before Nigel Pearson's side levelled as Keane latched onto a Anthony Knockaert corner just two minutes later.
The game died down as the opening period wore on, with only Paul Gallagher and Murray Wallace going close for either side in a half that was largely dominated by the defences.
The hosts made the better start to the second 45 minutes, with Richie Wellens and David Nugent both having chances to put the Foxes ahead for the first time.
Still it was Scannell who was at the hub of everything good for the visitors, the former Crystal Palace man crossing for Scott Arfield, who drew a save from Kasper Schmeichel as the Terriers looked to restore their advantage.
The home defence failed to heed the warning signs, however, and it was Scannell who made the telling contribution with just over 15 minutes to play, firing into the bottom corner from close range to put the West Yorkshire club on the brink of progression to the fifth round.
Pearson's side pushed to find a late leveller and take the game into extra time but, although Gallagher was able to craft a late chance, firing wide late on, Huddersfield held on to earn a win that means that they will welcome the Premier League strugglers to the John Smiths Stadium on Saturday.