Bayern fell 1-0 behind early on when Marco Reus made the most of a mistake from Javi Martinez, but the Spanish defender then levelled the scoring, while Mats Hummels handed the Bavarians the lead shortly before the interval.
Robert Lewandowski and Arjen Robben spurned gilt-edged chances to add a third and Bayern were made to pay for their profligacy when Pierre-Emerick Aubameyang restored parity, with Ousmane Dembele eventually ending their dreams of retaining the DFB-Pokal.
"We are very disappointed," said Ancelotti, whose side failed to win for a fifth time this month.
"We started April with a pretty good feeling. We did not deserve the loss against Real Madrid, but we only have ourselves to blame for this one.
"We simply did not take our chances.
"We now have to wrap up the Bundesliga title as soon as possible."
David Alaba was equally disappointed and thinks Bayern would likely have progressed had they scored a third goal.
"We got enough chances to put the match to bed," added Alaba.
"They then took every chance they got.
"Mistakes like the one for the opener can happen. We got back into it and got a number of chances to make it 3-1. Things would have ended differently had one of those gone in."