BVB picked up their first three points of the new season with a 3-2 win at the SGL Arena but only just, after conceding twice in the final eight minutes
BVB looked set to bounce back from their shocking opening-day defeat at home to Bayer Leverkusen with a resounding with at the SGL Arena, with Marco Reus and Sokratis Papastathopoulos both netting inside the opening quarter before Adrian Ramos seemingly put the outcome beyond all doubt with just over 10 minutes to go.
However, Raul Bobadilla pulled a goal back and then Tim Matavz set up a frantic final by scoring with just over 60 seconds of normal time remaining.
Dortmund ultimately held on to collect their first points of the season, but Klopp was less than impressed with the way in which his troops went to pieces in the closing stages.
"Many things got out of control," the BVB boss told Sky. "But the result today helps us after the brutality of the opening loss.
"However, this is just the beginning of the season."
Meanwhile, Dortmund centre-half Neven Subotic was just as upset with the concession of two late goals.
"We did not do well as a team in the closing stages," he stated. "We stopped defending and that should never happen."
Dortmund now have a Bundesliga break due to the upcoming international action before taking on Freiburg at Signal Iduna Park on September 13.