Ledley sustained a hip problem in the Eagles' 1-1 draw at Swansea City on Sunday and has opted to withdraw from the squad to focus on recovering at the south London club.
|CRYSTAL PALACE LATEST
|17/2||Crystal Palace are 17/2 with bet365 to beat Southampton 1-0|
"Joe Ledley will be returning to Palace," Coleman said.
"He has got a hip problem that never really settled down after Sunday's game. We won't take the chance with Joe."
Defender Ashley Williams is fit despite experiencing sickness during Swansea's clash with Palace, while Real Madrid's Gareth Bale is expected to play a full part in the game in Cardiff following his participation in the derby with Atletico Madrid.
"I was never worried about 'Baley'. He has been here before when he's known he is not going to play and has made the trip anyway to be with us. It is great to have him," Coleman added.
"He was involved in a tough game at the weekend - the Madrid derby was brutal at times. My heart was in my mouth on one or two occasions but he has come through that. He is here and raring to go."