The Ghanaians conceded twice by the 50th minute, and although Richard Arthur halved the deficit eight minutes later, the visitors scored again in the 63rd minute to consolidate their lead in the preliminary round first leg encounter at the Tamale Sports Stadium.
All Stars have their job cut out ahead of the second leg, which will be played in Tunisia due to security concerns in Libya.
“It is unfortunate we lost by such margin,” Adipah said.
“We were the better side but things didn’t go our way,
“I know they [Ahly] never believed they could come and get such score line here.
“We can also go and beat them by a big margin.
“They will be playing home away from home in Tunisia and that could be an advantage for us.
“We are not down and we are going to fight in the return encounter.”
Sunday’s fixture was All Stars’ first ever Caf continental club game.