In trying to force a move to Spain, the Croatian midfielder has angered his club chairman by refusing to travel with Spurs on their pre-season tour of the United States
The Croatian tried to leave the North London club for Chelsea last summer, and is now looking to move to Spanish champions Real Madrid.
He has now reportedly failed to travel with Spurs for their pre-season tour of the United States, and Villas-Boas has been scathing on his want-away star.
He told Spanish newspaper AS: "I think Modric is wrong. This will go against him.
“He has worsened the situation with what he's doing and now the chairman is very angry.
"This is also not a good time for Daniel Levy, who is in the USA at the moment, because his wife is ill."
However, the Portuguese manager admitted that the loss of Modric would be a major loss for Spurs, who play three games in the space of a week, including Liverpool at the M & T Bank Stadium in Baltimore.
Villas-Boas added: "He's good, very good. I tried to sign him last summer for Chelsea.
“He will be a big loss. He's a player who can make the difference, who gives a lot of clarity to the play when the team needs it and who can play in several positions.”
Modric has also been fined £80,000 by Tottenham for refusing to train, while Spurs will face Major League Soccer side LA Galaxy in their first match on July 24th at the Home Depot Center.