The French defender is concerned that the three-match ban earned by the Togo international in Saturday's north London derby could have a detrimental effect on the team.
Adebayor, who scored the opener, was given a straight red card early on after a reckless lunge on Santi Cazorla and then watched from the sidelines as his side capitulated to a 5-2 defeat.
"I think it will be a big loss for us because Adebayor has a lot of quality," Gallas told the Hackney Gazette. "He can keep the ball, he can score and we can play long or short."
The card came after Adebayor went in two-footed on Cazorla, but Gallas says it wasn't as clear-cut as it seemed.
"From the pitch it [the red card] was difficult for the players to accept," he said. "We spoke with the referee. We said to him ‘it’s not fair’, but some people on the TV said he deserved the red card.
"I think he [Adebayor] was sorry, even if he didn’t say anything. He was really sorry, we could see that on his face. Now we have to forget that game because it was painful."
The defender went on to explain that Adebayor's ejection was the clear turning point in the match.
"It is very difficult to accept [the result]. At the beginning of the game we played very well and we scored the first goal. I thought we controlled the game but after the sending-off of Adebayor it was really difficult for us," he went on.
"We were unlucky because we were down to 10 men and that’s hard, especially when you play away and when you play against a team like Arsenal, who play really good football. It’s really difficult to keep that score so I think we were unlucky. In the Man City game, and against Chelsea, I think we missed something but we have to work again and try to come back.
"The season is still long. We have to work hard and we know how hard it is to qualify for the Champions League."
Tottenham has the chance to put the derby defeat to one side as the club faces West Ham next Sunday and Gallas believes victory is crucial to end a recent run of successive defeats.
"I know it’s not easy sometimes but we have to come back to training, work again and try to have confidence because it’s true that confidence can be low," he added. "We lost 5-2 so the confidence can be low, but we will be okay.
"Every game is very good because it means you can forget about the game you have played before. West Ham will be one more difficult game but we have to carry on. We have a target, we have to keep working in training and it will come."
Follow GOAL.COM USA on