Spurs opened the scoring with just 10 minutes on the clock at the Emirates Stadium, when Christian Eriksen tapped in a close-range rebound after Bernd Leno failed to hold a tame Eric Lamela shot from outside the box.
Harry Kane doubled the visitors' account from the penalty spot 30 minutes later, after Granit Xhaka was punished for a reckless tackle on Son Heung-min in the box.
Arsenal found a way back into the game when Alexandre Lacazette struck right on half-time, which swung the momentum of the match in the home side's favour.
The Gunners were the dominant force for large portions of the second period and were rewarded for their efforts when Pierre-Emerick Aubameyang notched his third goal of the season to make it 2-2 in the 71st minute.
Neither side could find a decisive third goal thereafter, with Alli left bemoaning Tottenham's defensive lapses against a "talented" Arsenal team.
"Being 2-0 up we have to be more relentless defensively and a little more mature in the way we played in the second half," the 23-year-old told Sky Sports.
"We knew with Arsenal getting the goal before half time they were going to come out and push for the second goal, which does create space for us to counter-attack.
"But at the same time we have to be cleverer with our defending and we're obviously disappointed.
"I think the boys played very well in the first half. Obviously this isn't an easy place to come, you just have to be a bit cleverer with the way you play.
"But we showed great character to not lose in the end because of the amount of possession they had. They are a talented team.
"Obviously we are disappointed but there are a few positives to take."
Mauricio Pochettino's men are ninth in the Premier League table after the first four matches of the 2019-20 season and without a win since the opening weekend.
The England international came off the bench at the Emirates and was happy to be back in the fold after a lot of "hard work" behind the scenes.
"It felt good to be back today and I'd been doing a lot of hard work in training to ensure I'd come back ready, we were disappointed with the result but it is nice to be back," Alli added.
"I think we've had some tough games to start. A point away at City is a good one, if you look at today we should've left with three, but it's only a good point if we win the next game."