Raheem Sterling put the visitors in front in the eighth minute at White Hart Lane after Mario Balotelli had almost headed in a debut goal even before that, and Steven Gerrard struck from the penalty four minutes into the second period of the Premier League encounter.
|5/2||Tottenham are 5/2 with William Hill to finish in the top four|
"It's a very disappointing result and two key moments at the start of each half cost us," bemoaned Pochettino.
"After we went 1-0 down, we stayed in the game and created chances, but in football you need luck at key moments and [it] was difficult."
Tottenham should have had a penalty themselves in the closing stages when Emmanuel Adebayor had his shirt pulled by Dejan Lovren, but it would have only given them the chance to net a consolation score.
Having struggled against the top teams last term, including 4-0 and 5-0 losses to Liverpool, this result burst their early-season bubble following league wins over West Ham and QPR.
However, Pochettino insists it is too early to tell if it will be the same story throughout this campaign, adding: "We need to improve and look forward. It's the beginning of the season and only three games in."
Striker Adebayor echoed his manager's thoughts and expressed a desire to respond.
"We have to find a way to correct the mistakes and move forward. It's very important to bounce back from this," he added.
"We are all disappointed but the season has a long way to go and we have to learn from our mistakes."