The hosts surrendered a two-goal lead to Terry Butcher's side and failed to overtake Rangers at the top of the Scottish Premier League.
Lennon's men were cruising with 25 minutes to go, after a wonder strike from Paddy McCourt doubled Celtic's advantage following Ki Sung-Yeung's opener before half-time.
The commanding lead was subsequently thrown away, as Inverness scored two goals to dent the Celtic's title ambitions.
"I'm pretty livid, to say the leas," Lennon told reporters.
"It's the manner of the goals we're conceding that's most alarming. I should be talking about Paddy McCourt, the goal he scored.
"It was a wonder goal. But, unfortunately, I'm not talking about him, and we really have to do better.
"It's individual glaring mistakes that you expect from novices, really.
"The manner of the goals deflate the team and we didn't have a lot to help the team cover.
"I didn't think Inverness were a threat at all and we gifted them a goal out of nothing. That's the most galling thing. It's not good enough."
Lennon also spoke of his concern over his side's home form, which has stuttered so far this season.
He said: "You want to win your home games, so it's a worrying trend to be at home, concede leads and drop points.
"We'll have to lift ourselves and get over it as quickly as possible, but we can't keep making individual errors and defend set-pieces a bit better. We have to dispel this quickly."