The Ivorian headed in a cross in the dying minutes after Javier Hernandez cancelled out Wayne Routledge's opener and substitute Fabio saw red.
Wayne Routledge put the visitors on course for the upset when he lobbed David De Gea in the 12th minute, only for Javier Hernandez to convert Alexander Buttner's cross and pull United level within four minutes.
But Moyes' frustration was compounded when substitute Fabio was shown a red card 10 minutes from time. As both sides pushed forward, Bony headed in Routledge's cross to win it.
Rio Ferdinand also hobbled off injured during another disappointing afternoon for the Premier League champions, while Swansea can now look forward to a fourth-round match against Birmingham City, Crawley Town or Bristol Rovers.
Moyes handed starting spots to goalkeeper Anders Lindegaard and defenders Ferdinand and Buttner, while Danny Welbeck and Hernandez led the line in the absence of injured pair Wayne Rooney and Robin van Persie.
There were five changes from the Swansea side that lost to Manchester City in the Premier League on Wednesday. Captain Ashley Williams was on the bench with manager Michael Laudrup forced to contend with an injury list featuring the likes of Michu and Michel Vorm.
The hosts began the brighter and should have gone ahead after 11 minutes, but Hernandez failed to connect with an effort from inside the area after good link-up play between Welbeck and Chris Smalling.
United was made to pay soon after, when Alejandro Pozuelo played in Routledge, who produced a delightful lob over Lindegaard to put Swansea ahead.
Four minutes later, United was level, Hernandez making amends for his earlier miss to prod home Buttner's cross inside the 6-yard box.
After the early goals, chances dried up for both sides before Jonathan de Guzman and Buttner threatened from set pieces for either side.
Hernandez wasted another opportunity a minute before the break, volleying Jonny Evans' lofted ball wide of Gerhard Tremmel's right-hand post.
The sides cancelled each another out in the early stages of the second half, with Swansea perhaps enjoying greater possession without causing any problems.
Bony's swerving effort had Lindegaard worried on the hour before it whistled wide, while Swansea's resolute defending at the other end stifled United's forward line.
United's cause was not helped by the loss of Ferdinand to a suspected knee injury and the dismissal of his replacement Fabio, who was shown a red card for a late challenge on Jose Canas 10 minutes from time.
And Bony compounded Moyes' misery late on to secure Laudrup a first win in eight matches.