Mikel Arteta scored the first of two penalties that secured three points for Arsene Wenger's team after Cazorla fell in the box despite having evaded Steven Reid's tackle.
Olsson suggested that players, rather than referees, deserve criticism for the prevalence of simulation but admitted that Arsenal merited their victory on the balance of play.
"I don't like people diving," Olsson told reporters. "I'm not very objective either being a defender. Most of the time I like this league because of the fair play it contains.
"The ref doesn't have the privilege of watching replays. It's a difficult job. If it's a dive the focus should be on the person diving as well.
|BACKING THE BAGGIES?
|13/2||West Brom are 13/2 to finish in the top six this season with bet365|
West Brom began the campaign in fine form under Steve Clarke but have now slipped to sixth place following their recent disappointing run.
Swansea and Stoke saw off Clarke's high-flyers prior to their defeat against Arsenal but Olsson is confident they are experiencing only a minor blip.
"All teams have down periods," the Sweden international insisted. "Hopefully ours is now and well get back on track as soon as possible. This is a difficult place to play, Swansea too.
"I don't think we should over-analyse them. Stoke at home is harder to come to terms with. For us it's just important to get back on track."