The Gunners fell to a 2-1 defeat against Chelsea at Stamford Bridge on Sunday, despite an improved second-half display.
And Wenger fears that loss to Rafa Benitez's side could have very bad mathematical implications for his side's chances of finishing in a Champions League spot.
|PREMIER LEAGUE LATEST
|11/1||Arsenal are 11/1 with William Hill to beat West Ham 3-1
"The game we lost on Sunday had difficult mathematical consequences."
Arsenal sit sixth as they prepare to host West Ham after back-to-back defeats against Manchester City and the Blues.
And Wenger has become increasingly concerned that his side have only taken five points from a possible 21 against sides above them in the league.
"We do not win the big games," admitted the Frenchman. "That is more worrying than the difference in points.
"The difference in points mathematically is not dramatic, but the consistency of our performances within games is worrying.
"When you see us in the first-half and the second-half it is difficult to believe that we are the same team.
"The position we are in is a bit backs-to-the-wall. We have to respond. I’m confident that we will, but the team was very down after the Chelsea game.
"The players come away thinking: “We could have done better”. But when you look at the first-half we lost goals we shouldn’t lose, 50-50s you cannot lose in big games."