The player's representative feels a return to Serie A is possible, but not for the moment following the closure of the transfer window last week
The 24-year-old Italy international, who could feature in Tuesday night's Euro 2012 qualifier against Slovenia, was linked with a summer exit from the Spanish club.
A move to Barcelona never took flight despite talks with the club, while interest from Napoli, AC Milan, Juventus and Inter fizzled out as he stayed at the Madrigal.
Napoli have already expressed their desire to sign Rossi in future with president Aurelio De Laurentiis recently admitting a move could be possible.
And Pastorello is not totally against the idea.
"It is nice to hear that Napoli want Rossi, but for the moment there are no chances that any deal can take place," the agent told Radio Kiss Kiss.
"However, like we always say in football, never say never."
Napoli take on Cesena on Saturday night, while Villarreal hope to recover from their 5-0 thrashing against Barcelona with a home game against Sevilla.