Adam Lallana's last-minute strike sealed an incredible 5-4 win at Carrow Road on Saturday, although set-pieces had threatened to undo Liverpool again.
Twice the visitors were guilty of not clearing their lines, which allowed Dieumerci Mbokani and Sebastien Bassong to fire past Simon Mignolet, the latter doing so to level the score at 4-4 in stoppage time before Lallana's crucial strike.
"I think we were deserved winners in a spectacular game," Klopp told BT Sport.
"But it is crazy what we are doing there [at set-pieces] we have to solve this problem because it is not always possible to score five goals.
READ MORE | Firmino starting to make £29m impact
"Today we knew it would be difficult because they are not a normal team in this position in the table because they play football, it's not only long balls and deep defending.
"In the game we felt we could cause them problems but then we should not always concede goals from set plays, because it is easy to play against us.
"You don't have to create chances you just have to throw the ball over the line and have a set play. Then bring your whole team into our box and something will happen.
"After the game it feels funny but it's really rubbish what we are doing, we have to solve this."