Koulibaly was shown two yellow cards in quick succession during the 1-0 defeat at San Siro on Boxing Day, the second coming after he sarcastically applauding the referee.
The Senegal defender was targeted by racist abuse from the stands throughout the game in Milan and head coach Carlo Ancelotti said Koulibaly was "nervous and the mood at the time was not his best".
Napoli said the game should have been suspended due to the taunts from the crowd and the club's lawyer vowed to fight any suspension.
However, the Italian Football Federation (FIGC) appeals court rejected his claim on Friday, leading to condemnation from Napoli, who believe the governing body have "degraded" football and hampered their fight to stop racism.
The club statement read: "The decision of the FIGC appeals court to turn down our appeal against Koulibaly's ban is a sad defeat both for football and the wider issue raised by this whole affair, namely, the fight against discrimination of all sorts, which is still part of football and society.
"A battle that UEFA has been fighting for years – and which Napoli has always supported – has thus been degraded.
"UEFA and FIFA have condemned what happened, saying that protocol was not respected and that the match should have been suspended. All manner of public figures, from mayors to artists, footballers and coaches have offered their solidarity while expressing their indignation.
"Rarely in Italy have we witnessed such unanimous condemnation from society as a whole.
"Then there was the appeal itself, where the judges said they are completely aware of what happened and that they are on Koulibaly's side as a man. They encouraged him not to give up or feel alone, which, he explained, in a sincere and moving speech, is how he felt that evening. Just as he explained to the judges how embarrassed he felt when he had to tell his parents, and his mother in particular, what had happened.
"After all these extraordinary events, the appeal was turned down. A procedural rule was incapable of doing the only thing there was to do: give back to Kalidou Koulibaly – this Senegalese lad who, in light of these events, represents everything that is good and bad in Italy at the moment – the dignity he deserves.
"Koulibaly, Italian football and the institutions all come out of this degraded. Koulibaly's ban should have been overturned regardless of the rules and the bureaucracy. All this does is kill football. Because football is above all a passion that unites billions of people around the world, and it should not be derided like this.
"A great opportunity has been missed today. Sadly it proves there is still much to do and many things to change."