Liverpool's defensive frailties were again exposed as they suffered a 2-0 loss to exit the EFL Cup in the third round on Tuesday.
It extended their winless run to four matches, a period during which Klopp's men have conceded 10 goals.
The German retains belief in his team despite their dip in form, taking plenty out of a strong first half.
"It doesn't feel too cool but it's the truth that we are still a good football team," Klopp told a news conference.
"I see it but unfortunately it doesn't feel it at the moment because it's a cup competition.
"We want to go to the next round. Nobody was interested in who plays the better first half. Everybody was interested in who would go through and we are not the team who's in the next round – and for some good reasons."
Liverpool have kept just two clean sheets this season in nine games, and Shinji Okazaki and Islam Slimani scored second-half goals for Leicester – the first after Liverpool had cleared a corner and the second from a throw-in.
Klopp said: "Defending is different things. It's common together defending and it's individual defending, it's one-on-one situations, it's two-on-one situations.
"I don't think teams shoot a lot against us, I know they don't, they have not a lot of finishing situations.
"We defend really well in a lot of moments – how everyone can imagine, we know how to defend a set-piece, a throw-in, and then you change two or three times and then in the end, obviously it doesn't work.
"I'm in the boat, it's not that I say 'I told you and you don't know'. I feel really bad in the moment about this, of course it's not about not knowing it's about doing it in that moment – that's hard in the moment, it feels really hard in this moment to go out because of this [and] after that performance in the first half.
"But we have to play like this, we made seven or eight chances and we still can play football, that's cool, good, good, good, but as long as you concede goals like this it makes life really difficult."