The Gunners had not been victorious on the Premier League's opening weekend since thrashing Everton 6-1 in 2009 and lost last year's against Aston Villa but, despite going behind to the Eagles, rescued it late on Saturday.
Laurent Koscielny equalised Brede Hangeland's opener and, after Jason Puncheon was shown a second yellow card in the 89th minute, Ramsey pounced to bundle home from close range in the final moments - a welcome relief for the Welshman.
|CRYSTAL PALACE LATEST|
|5/4||Crystal Palace are 5/4 with William Hill to beat West Ham|
"I didn't have the best of games but I kept on trying, kept getting into the right areas and finally, in the 93rd minute, it went in."
Having scored the winner against Hull City in last season's FA Cup final and Arsenal's second against City in the Community Shield, Ramsey is enjoying a hot streak in front of goal.
"He has something of a rare gift - that is to be where you need to be in the box," said his manager, Arsene Wenger, to Sky Sports as he offered praise to Palace for their display.
"Credit to Palace; they worked hard, were physically very strong and very organised. We gave ourselves a difficult task. They gave us big problems and defended very well.
"We keep going - that's one of our strengths and it paid off. The change of pace in the final third [could have been better]. That is usually one of our strengths.
"Is it down to the quality of defending or the fact that we are not there physically yet? Maybe a bit of both."
Ramsey agreed with his manager's assessment and noted that Pulis's influence lived on in Palace's performance.
"They were very hard to break down. They lost their manager earlier in the week but you would never have known that," he added.
"Tony Pulis got them playing really well and they picked up a lot of points towards the end of last season. You could see they were well organised and it was difficult but we came away with the three points."