Bale was cautioned for diving – the third time he has been penalised for such an offence this season – despite contact from Craig Gardner meaning he is suspended from his side’s New Year’s Day match against Reading.
"It's a big mistake from the referee," the 35-year-old insisted to Sky Sports.
| TOTTENHAM LATEST
|11/2||Tottenham are 11/2 to beat Reading 2-1 with Paddy Power
"If I'm not wrong, we had the same situation with Defoe in the first half and... it looks like persecution [against Bale].
"Players have a responsibility and we have to take that into consideration. We want to take it out of the game. I think our game is respectful and there is fair play. But with the speed they go and then there is contact it is a difficult one for the referee. To see Gareth punished again is a little bit unfair.
"We have to let it go. We lose him for the Reading game, we just recover the boys and continue a magnificent run of form and results."
The winger also criticised the referee's decision adding that he feels as though he has been unfairly targeted in recent weeks.
"It's the same thing every time," he added. "It's the third time now this season I've been clipped and I've got booked for no reason again.
"It's just one of those things. There's nothing I can do about it, the referees just need to look a bit closer I think. It's not something I'm doing, it's people thinking I'm diving where there's actually contact, so at the end of the day if I'm getting contact it's a penalty or a free kick so I can't do anything about that.
"If there's contact I fall over, what do you expect me to do? Hit my head on the floor? I've got to put my hands down and protect myself. If people are kicking me I'm not going to stop going over because it's a foul.
"It doesn't bother me. People can keep saying what they want, it doesn't bother me whatsoever. At the end of the day, it's a foul and I know that myself."
Spurs' 2-1 win at the Stadium of Light briefly put them third, but Villas-Boas was keen to play down rumours that his side could go on to launch a tilt at the top of the table, insisting he would continue to take things game by game.
He told reporters: "We can't believe in unrealistic situations we just need to keep gathering the most amount of points we can.
"Our comfort comes from we've gone to Old Trafford, Goodison Park and the Emirates and we have difficult trips to Stamford Bridge and Anfield, stadiums where our opponents can lose points.
"Our satisfaction comes from that and over the festive period with two away fixtures and we've won both of them."