The Dane was dismissed after tangling with Shane Long with the Irishman clear on goal and the Reds already 1-0 down to Steve Clarke's men.
Long went on to miss from the spot, but Agger believes that referee Phil Dowd made an error in giving the foul in the first place - which also appears to have been outside the box.
| LIVERPOOL LATEST
|11/2||Liverpool are 11/2 to draw 1-1 with Manchester City with bet365
"Of course. I've seen it again on television and there is no contact," the 27-year-old insisted when asked if he would appeal.
"Referees are only human. They make mistakes as well.
"I was unlucky to be involved in one of the mistakes. It's frustrating but that's part of the game."
The Reds got off to the worst possible start in Brendan Rodgers' first league game in charge and the Denmark international is desperate to put things right over the course of the season.
"I know we will bounce back from this," added Agger. "I think we controlled the game before the sending-off.
"We had the momentum, we kept the ball well and we created chances. The red card changed the game.
"Everyone is frustrated. I don't think the result was right. It doesn't really make sense. But that's football and it happens.
"We have to continue. We showed a bit of the football we want to play. The manager said the same - that we controlled the game, created some chances and were a bit unlucky with some decisions.
"At Liverpool, the spotlight is always there. There is no reason to panic. I think the Premier League is getting better and better. There will always be tough games ahead."