The 33-year-old said the mood in the Red Devils' changing room following Cazorla's late goal mirrored that of a loss, despite the fact the win moved them to the top of the league.
An early opener from former Gunner Robin van Persie gave Sir Alex Ferguson's men the lead; however, with Wayne Rooney missing a penalty and the hosts squandering a number of other chances, the visitors were allowed to remain in the contest.
"We should have finished that game off ages ago," Ferdinand told reporters. "That is the most disappointing thing.
Patrice Evra doubled the lead with more than 20 minutes left, before Jack Wilshere was sent off for Arsene Wenger's side, but a fine strike from Santi Cazorla denied United a clean sheet at Old Trafford. The late consolation goal meant nothing in terms of the result but Ferguson's annoyance on the touchline was clear as a chance to boost their goal difference further, with Chelsea drawing, went begging.
Ferdinand was also disappointed with the margin of victory, and the 33-year-old insisted the Red Devils should have had the result dead and buried much in advance, suggesting the late goal was hard to take.
"If you had been in our changing room you'd have thought we lost the game, the way the mood was. At the end of the day it is three points," Ferdinand said. "I thought it was the best shape we have had for a while, this season, and we defended as a team.
"That showed in the amount of chances we created but we should have maybe had the game won by five, six or seven but it wasn't to be."