Tottenham knocked City out at the quarter-final stage on Wednesday, losing 4-3 on the night at the Etihad Stadium but progressing on away goals after a 4-4 aggregate scoreline.
The result marked the second consecutive season in which City exited the Champions League in the last eight, with Liverpool knocking Pep Guardiola's side out last time around.
Mendy expressed his frustration with his side's inability to break through in Europe, with Guardiola still chasing his first Champions League crown outside of Barcelona.
"It was terrible elimination," Mendy told RMC. "It's historic for them, not for us. We prefer to be qualified.
"It's a very difficult time. It is several times that we go out in the quarters," the defender added. "Next season we will give the maximum to aim even higher.
"What are we missing? The recipe, we have not found it yet. We could have done better, it's going to be hard this evening, but we'll be back in a few days."
City's chances of a historic quadruple have been dashed but Guardiola's side can still aim for a treble, having already won the Carabao Cup, reached the FA Cup final, and currently sitting two points behind first-place Liverpool in the league but holding a game in hand.
"We will now think about the Premier League and the cup," Mendy said.
Mendy's side will not have to wait long to try and gain some measure of revenge on Spurs, with the two sides set to face off again on Saturday in Premier League action.