The Brazilian scored a screamer early on but Sebastian Kehl's goal was enough to seal an aggregate win for the Germans despite Rondon's winner on the night
The Russians grabbed the lead through a stunning strike from Hulk early in the first half before Sebastian Kehl equalized before the break.
Rondon restored the visitors' advantage in the second period but Jurgen Klopp's side was able to hold on to a 5-4 aggregate advantage to reach the last eight.
Sergey Semak, taking charge of his final Zenit match before Andre Villas-Boas is officially installed as coach on Thursday, at least saw his reign close with a victory at Signal Iduna Park, although his side failed to turn around its first-leg deficit.
Axel Witsel had the game's first effort on target as the Belgian tested Dortmund goalkeeper Roman Weidenfeller with a long-range drive.
Vyacheslav Malafeev was then forced into a relatively routine save from Robert Lewandowski before Hulk produced a stunning effort to open the scoring.
Having cut in from the right, the Brazilian hit a wonderful, swerving left-footed strike that gave Roman Weidenfeller no chance.
As the Germans responded, Kevin Grosskreutz saw his effort well saved by Malafeev before Sokratis Papastathopoulos headed over from a corner. For all its possession and forward momentum though, Dortmund struggled to break down Zenit's resolute back line.
That was the case until Kehl rose highest to head past Malafeev - breaking his Champions League duck in the process at the 30th attempt.
Hulk threatened a repeat of his first-half effort shortly after the break when he sent a free kick over the crossbar before Viktor Fayzulin's shot drifted wide of Weidenfeller's post.
Rondon's winner may have spoiled the evening for Dortmund somewhat - the Venezuelan arriving to meet a cross from the left - but it changed little as Jurgen Klopp's men followed Bundesliga rival and European champion Bayern Munich into the last eight.