Following a stunning start to the 2015-16 season, the England international has been linked with a shock transfer to Rafael Benitez's side, while Liverpool and Tottenham have also been reportedly planning a move.
Vardy has scored in nine consecutive Premier League matches - putting him one short of Ruud van Nistelrooy's record of 10 - and insisted he has no plans to leave Leicester after such a fine goalscoring run.
"It is all speculation, it is as simple as that," the 28-year-old said to Sky Sports when asked about the talk of a move to Madrid.
"I am happy with what I am doing at Leicester and long may that continue. It has been a brilliant start for myself as well as the team."
Vardy has scored 12 goals in as many league games this season but was reluctant to set a total figure he would like to have reached by the end of the campaign after winning the Premier League Player of the Month award for October.
He continued: "I have already doubled my tally from last year but I don't set myself any targets.
"I just want to keep on improving as much as possible and the way the season has started I could not ask for much more really."
Vardy has the chance to match Van Nistelrooy's record when Leicester play Newcastle United at St James' Park after the international break.