The Premier League champions were beaten by Shaun Cummings' last-gasp goal as the League One side shrugged off the dismissal of Jake Cooper.
A much-changed Foxes line-up had started brightly, but failed to make their promising first-half display count and looked void of ideas after Cooper's red card in the 52nd minute.
"I am very disappointed," said Leicester boss Ranieri, whose side sit just one point above the Premier League relegation zone.
"We played well in the first half, so-so at the beginning of the second until it went to 10 v 11. After that, they played better than us, with more desire and heart and deserved to win.
"When we have the opportunity to go to the next round, we have to win. We have to show desire. I gave a great opportunity to my players.
"In the first half, I watched good football. We were very prepared. I was happy at the end of the first half. I said 'keep going, try to score but keep a clean sheet' because in the second leg [a replay] the pitch might be better at home and we can play our football but we lost at the last minute.
"We can learn. When a team from League One beats the champions we say 'why?' and have to react as soon as possible. We are better than Millwall, but Millwall deserved to win."