The Croatian tried to leave the North London club for Chelsea last summer and is now looking to move to Spanish champions Real Madrid.
He reportedly failed to travel with Spurs for their pre-season tour of USA and head coach Villas-Boas was scathing in his assessment of the wantaway star.
The Spurs boss told AS: "I think Modric is wrong. This will go against him.
|20/1||Andre Villas-Boas is 20/1 with William Hill to be the first Premier League manager to leave his post in 2012-13
"This is also not a good time for Daniel Levy, who is in the USA at the moment, because his wife is ill."
However, the Portuguese manager admitted that the loss of Modric would be a major one for Spurs, who play three games in the space of a week, including Liverpool at the M & T Bank Stadium in Baltimore.
Villas-Boas added: "He's good, very good. I tried to sign him last summer for Chelsea.
“He will be a big loss. He's a player who can make the difference, who gives a lot of clarity to the play when the team needs it and who can play in several positions.”
Modric has also been fined £80,000 by Tottenham for refusing to train, while Spurs will face MLS side LA Galaxy in their first match on July 24 at the Home Depot Center.