Die Schwarzgelben dominated for large spells but imploded in the final 10 minutes as Max Kruse and Raffael scored for the hosts, after Mats Hummels had earned his marching orders.
The loss prompted Klopp to demand his side - who subsequently surrended top spot in the Bundesliga following Bayern Munich's 1-1 draw at Bayer Leverkusen - become more clinical.
"We are self-critical and certainly believe that we have our share of this defeat," the 46-year-old coach admitted to reporters after the match. "We left the game open and left Monchengladbach alive.
"It was an unusual game. My team did exceptionally well for a long time, we put our opponents under pressure and counter-pressed well - but did not score.
Despite defeat, Klopp maintained his side could still take plenty from their performance and was wary of criticising the officials over Hummels' red card and the resulting penalty.
"The penalty, I have not seen on TV," he said. "With 10 men, it was hard, then came the second goal, and we lost.
"We will also take the positive things in the assessment, so the defeat hurts a little less."
Dortmund had surged to the top of the table after five consecutive opening league wins, but have now been usurped at the summit by Pep Guardiola's Bayern.