The FA have chosen to act following ugly scenes in the closing stages of the Premier League clash between the two teams on December 9.
A scuffle between Hull's Yannick Sagbo and Swansea's Ashley Williams prompted a melee.
Sagbo and Williams were both booked, but their clubs could now face further sanctions.
"Both Swansea City and Hull City have been charged by The FA in that they failed to ensure their players conducted themselves in an orderly fashion in or around the 80th minute of their fixture on 9 December 2013," read a statement on the FA website.
"The clubs have until 6pm on 17 December 2013 to respond to the charge."