The Ivorian headed in Wayne Routledge's cross in the dying minutes after Chicharito had cancelled out the Englishman's opener and substitute Fabio had seen red
Wayne Routledge put the visitors on course for a famous upset when he lobbed Anders Lindegaard on 12 minutes, only for Chicharito to convert Alexander Buttner's cross and pull United level within four minutes.
But Moyes' frustration was compounded when substitute Fabio was shown a red card 10 minutes from time and as both sides pushed forward, Bony headed in Routledge's cross to win it.
Rio Ferdinand also hobbled off injured during another disappointing afternoon for the Premier League champions, while Swansea can now look forward to a fourth-round tie against either Birmingham City, Crawley Town or Bristol Rovers.
Moyes handed starting spots to goalkeeper Lindegaard and defenders Ferdinand and Buttner, while Danny Welbeck and Chicharito led the line in the absence of injured pair Wayne Rooney and Robin van Persie.
There were five changes from the Swansea side that lost to Manchester City in the Premier League on Wednesday. Captain Ashley Williams was on the bench with manager Michael Laudrup forced to contend with an injury list featuring the likes of Michu and Michel Vorm.
The hosts began the brighter and should have gone ahead after 11 minutes, but Hernandez failed to connect with an effort from inside the area after good link-up play between Welbeck and Chris Smalling.
United were made to pay soon after, when Alejandro Pozuelo played in Routledge, who produced a delightful lob over Lindegaard to put Swansea ahead.
Four minutes later, United were level, Chicharito making amends for his earlier miss to prod home Buttner's cross inside the six-yard box.
After the early goals, chances dried up for both sides before Jonathan de Guzman and Buttner threatened from set-pieces for either side.
Chicharito wasted another opportunity a minute before the break, volleying Jonny Evans' lofted ball wide of Gerhard Tremmel's right-hand post.
The sides cancelled one another out in the early stages of the second half, with Swansea perhaps enjoying greater possession, without causing any problems.
Bony's swerving effort had Lindegaard worried on the hour before it whistled wide, while Swansea's resolute defending at the other end stifled United's forward line.
United's cause was not helped by the loss of Ferdinand to a suspected knee injury and the dismissal of his replacement Fabio, who was shown a red card for a late challenge on Jose Canas 10 minutes from time.
And Bony compounded Moyes' misery late on to secure Laudrup a first win in eight matches.