The former Everton midfielder scored from 25 yards out for the Gunners with three minutes remaining on the clock to leave the City's title hopes in tatters
The ex-Everton midfielder, 30, was given acres of space on 87 minutes and lashed a 25-yard strike into the bottom right corner of the Joe Hart's net.
The victory puts the Gunners back in third place, two points clear of Tottenham in fourth and five points ahead of Chelsea and Newcastle.
“It was so important,” Arteta told Sky Sports after the match.
“Chelsea won yesterday, Newcastle haven’t really conceded in the last few games and Tottenham are still there, so it’s going to be a big fight from here until the end of the season.
“But we knew that today was one of the more difficult ones we had until the end.”
The game was littered with half-chances and hard tackles, with Mario Balotelli seeing a second yellow card after Arteta’s goal, and Arteta said he believes Arsenal were the deserved winners.
He added: “It was a tough game to be honest. I think we were the better team overall and we had the most chances.
“But it's always tough against a team like Man City, and when you’re not scoring it’s always difficult, but we scored at the end and this is massive for us.
“You’re not going to get 10 chances, so when you have three or four you have to take them because they’ve got the quality up front to trouble you.
“Today we did at the end and I’m so glad.”
Robin van Persie had seen his header cleared off the line by team-mate Thomas Vermaelen and hit the post in the second half, so it fell to Arteta to score the winning goal.
He said: “I saw I had some space and the centre-half was coming up to me, and I thought, ‘I have to strike it’, and it went right in the net.”
The victory leaves Manchester City eight points off rivals Manchester United, and looks to have ended any realistic opportunity of a maiden Premier League title for Mancini's men.
Arteta added: “It was a big win because now it’s going to be really tough for them to catch Man United.”