Marko Arnautovic gave the visitors a 25th-minute lead, with Nacho Monreal restoring parity five minutes later.
An entertaining and error-strewn contest continued to flow from end to end but an injury to Arnautovic robbed West Ham of their prime attacking threat before Diop erred.
Welbeck came off the bench to gleefully blast in from close range, ensuring Manuel Pellegrini's fledgling West Ham tenure stands at three defeats out of three.
Arnautovic asked constant questions of Arsenal's high and shaky backline and had already thrashed into the side netting before he exchanged passes with Felipe Anderson to arrow a 20-yard drive into the bottom corner.
Monreal made light of an earlier knock to slam home the equaliser when West Ham failed to clear Hector Bellerin's cross.
Bellerin's mistakes at the other end remained a problem, with Sokratis Papastathopoulos forced into a desperate block from Felipe Anderson before the Spain full-back played Robert Snodgrass onside to shoot too close to Petr Cech.
Cech stood firm to thwart Arnautovic in the 55th minute and apparent knee injury he sustained during that attack proved pivotal.
Diop allowed Alexandre Lacazette too much room to turn and shoot, with ex-Arsenal goalkeeper Lukasz Fabianski beating the shot away, and the 21-year-old centre-back was the architect of his own misery 20 minutes from time.
A miscued clearance spiralled up in the air and Aaron Ramsey nodded down for Lacazette to drive into the goalmouth, with Diop only able to divert into his own net.
Felipe Anderson continued to probe, flashing agonisingly across the face of goal in injury time, meaning Arsenal nerves only settled when Welbeck swivelled to dispatch Bellerin's pass.
What it means: Off the mark but work ahead for Emery
Arsenal arguably attacked more fluently in their 3-2 defeat to Chelsea last time out, but Emery will be delighted with a three-point haul to boost an ongoing teething process. This was West Ham's best showing under Pellegrini – clearing an admittedly low bar – but any prolonged absence for Arnautovic would represent a major problem.
Pat on the back: Arnautovic cut off in his prime
Some sharper finishing from the Austria forward might have seen West Ham take all the points back across London but Arnautovic was very impressive as their focal point, revelling in Arsenal's defensive uncertainty. His goal was well taken and the 29-year-old – handed the captain's armband by Pellegrini – performed with a level of maturity that has not always been evident during a career that has thrilled and frustrated in equal measure.
Boot up the backside: Sokratis with plenty to ponder
Whipping the Arsenal defenders he inherited into shape will understandably take Emery time. However, it is hard to escape the impression his board landed him with an extra problem by bringing in the sluggish and untidy Sokratis, who does little to inspire confidence.
Key Opta facts
14 – Only Mohamed Salah (22) has been directly involved in more Premier League goals in 2018 than Marko Arnautovic (8 goals, 6 assists). Threat. pic.twitter.com/HvAQlssoMh— OptaJoe (@OptaJoe) August 25, 2018
- West Ham have lost their first three games of the Premier League season for the third time this decade (also 2010-11 and 2017-18) after doing so just five times between 1920 and 2010.
- Arnautovic has had a hand in 14 Premier League goals in 2018 (eight goals, six assists), second only to Liverpool's Mohamed Salah (22).
- Arsenal's Monreal has scored four Premier League goals in 2018; Crystal Palace's Patrick van Aanholt is the only defender to score more this year (five).
- Issa Diop is only the eighth player to score an own goal on his Premier League debut and the first to do so for West Ham in the competition.