The 35-year-old defender was ruled out by a shin injury on Saturday and watched his team-mates struggle at the back and slump to a 3-1 loss against Mauricio Pochettino's team
Carragher missed the clash at St Mary's with a shin injury so Daniel Agger and Martin Skrtel were paired at centre-back.
Rodgers acknowledged that his team must improve as a defensive unit but insisted that the problem ran all the way through the team and refused to pin the blame on any specific players.
"There's no question," he told reporters when asked if Liverpool had missed Carragher. "For us, it's an area we feel we need to be better in.
"We defend from the front. It's the team. We never defended well as a team so it's unfair to criticise those two [Agger and Skrtel].
"They'll be disappointed of course, as defenders, when we concede goals. We've been in a run of late where we either don't concede or we do, two or three.
"So it was one where our defensive line always starts from the front of the pitch. This is a group, when we lose, we lose together."
Several Liverpool players will now leave the club for the international break and Rodgers hoped the brief departure from club football would prove beneficial.
"Maybe you've just got to go away and reflect on the last few weeks," he added. "Because it's important for us to maintain the efforts that we have been putting in that have got us results.
"Today we let those standards slip a wee bit so we just need to get back now and refocus on that and I'm sure we'll see a different side against Aston Villa."