With over €120 million spent on the likes of Roberto Soldado, Erik Lamela, Paulinho and Christian Eriksen, the Argentine feels the blame must go to the players
Andre Villas-Boas was sacked in December last year and Tim Sherwood is expected to be fired at the end of the season with Tottenham likely to finish sixth in the Premier League.
But Ardiles said the summer buys were to blame with upwards of €120 million spent on Roberto Soldado, Erik Lamela, Paulinho, Christian Eriksen, Etienne Capoue, Vlad Chiriches and Nacer Chadli.
"When we started this season we were full of optimism, but somehow it has been a very disappointing season," the Argentine told talkSPORT.
"I believe that the players we have bought in this season have been the problem. They are not performing the way we expected. I don't want to name names, but basically all the people we have bought.
"This is a serious, serious problem.
"At the end of the day, whoever the manager is, AVB [Villas-Boas] earlier in the season and later on Tim, there is not a lot you can do with the kind of players we have right now.
"Whoever comes in, or if Tim stays, he has to make a big rebuilding of the team."