Diego Simeone’s team beat the Reds 3-2 after extra time in the second leg of their last-16 tie at Anfield, sending the defending European champions crashing out of the competition.
Georginio Wijnaldum’s first half strike took the game into extra time and Roberto Firmino put the European champions in the driving seat just four minutes in.
But Atletico recovered thanks to a Marcos Llorente double and a late goal from Alvaro Morata to make it 4-2 on aggregate.
The Premier League outfit dominated possession and had twice as many shots on target as their Spanish opponents, but saw their title defence brought to an end.
And coach Klopp feels Atletico should be capable of much better given the quality of their squad.
"I am completely happy with the performance. It’s so difficult to play a side like this,” he told BT Sport .
"I don’t understand with the quality they have the football they play. They could play proper football but they stand deep and have counterattacks.
"We accept it of course but it doesn’t feel right tonight. I realise I am a really bad loser, especially when the boys put such an effort in against world class players on the other side who defend with two rows of four.
"We know in the last two years we had some lucky moments in the Champions League, you have to, to reach two finals, but today it was everything was against us in the decisive moments."
He added: "For the score to go 2-1 was not cool, it was a blow but it's not a big problem. The second goal was a bigger problem.
"After the second goal the legs were a bit tired. Everything that looked really natural in the first 90 minutes became a bit stiff. The crosses did not have the same quality We wanted the first time crosses like for the first goal, which was brilliant and the boys forget to do that.”