The Bayern Munich midfielder took a boot to the face after just over half an hour of the game, in which Germany fell behind to an Ola Toivonen goal before Marco Reus equalised following the interval.
Toni Kroos then scored a 95th-minute winner to complete a gripping comeback and move Germany up to second place in Group F with two games played.
Victory over South Korea on Wednesday would make sure of Germany's place in the knockout phase but head coach Joachim Low told reporters he could be without Rudy in central midfield.
"Sebastian Rudy may well have broken his nose. It wouldn't stop bleeding, so we had to take him off," said Low, who replaced Rudy with Ilkay Gundogan.
"He may still be able to play on Wednesday though. We'll have to wait and see."
Jerome Boateng will miss the clash with South Korea through suspension, having been sent off eight minutes from time against Sweden, and Low assessed the damage to his squad, saying: "We have to take stock and see what the situation is.
"The players were tired at the end – Marco Reus started having cramps in his legs 10 minutes before the whistle, Boateng won't be playing, Rudy broke his nose.
"We'll start by resting and then prepare for South Korea on Monday. Our scouts have watched them twice already, but we'll have to see what players are available.
"Mats Hummels will probably be able to play and that offers another variant in the back line."