Klopp's side were frustrated by 1860 Munich at the Allianz Arena and struggled to break down their lower-division opponents, but extra-time strikes from Pierre-Emerick Aubameyang and Henrikh Mkhitaryan eventually sealed a 2-0 win for BVB.
And Klopp was pleased with number of opportunities his side created against a dogged 1860 side.
"We must and can win tough games," said Klopp. "We had four great opportunities and hit the post and the crossbar.
"They (Munich) defended very deep, which is not easy for any team. If we had scored early it might have been different."
The introduction of Aubameyang sparked Dortmund - who took 15,000 fans with them for the clash - into life and he had a hand in both goals.
Elsewhere in the competition, Bayer Leverkusen had little trouble against second division Arminia Bielefeld as they recorded a 2-0 win of their own.
There were also wins for Hamburg, who secured a 1-0 victory at home to Greuther Furth, and Wolfsburg, who beat Aalen 2-0.
Hoffenheim overcame Energie Cottbus in extra time, while Augsburg saw off third division Preussen Muster, 3-0. Second division Sandhausen came from a goal behind to beat fourth tier Widenbruck 3-1.
Second division Cologne caused the upset of the night as they beat Bundesliga side Mainz 1-0.