The experienced Netherlands midfielder fears racists might feel empowered after the San Siro side's decision to abandon Thursday's match in the midst of abuse from the stands.
M'Baye Niang, Sulley Muntari and Kevin-Prince Boateng were the victims of racist chants during the match, and the latter stopped playing 26 minutes into the game, picked up the ball, and angrily kicked it towards the Pro Patria support before walking off, followed by his teammates.
The former Ghana international has since been praised for taking a stance against racism, but Seedorf is unsure Milan have given off the right signal.
"I don't feel it's such a fabulous thing," Seedorf told BBC Radio 5 live. "These people will feel empowered now. They should just be identified and kicked out of the stadium."
Seedorf left AC Milan in the summer transfer window after a 10-year spell at the club and has since joined Brazilian side Botafago.