The Real Madrid and Argentina winger insists he was not reacting to supporters booing him and claims he is infuriated at the way his actions are constantly scrutinised
The Argentine appeared to grasp his groin as he left the field in the victory at Santiago Bernabeu after some sections of the home support booed him for his performance.
Di Maria insists he was not attempting to make any gesture and claims his actions - unlike those of his team-mates' - are scrutinised to a frustrating extent whenever he takes to the field.
"It wasn't for anyone. I adjusted myself and kept running," the 25-year-old told Fox Sports.
"I've never had problems with the fans. I could feel that half were applauding and half booing.
"I went out totally naturally. They always expect me to do something. I always have to clarify what I do. With everyone else nothing happens and with me it's always a complete mess.
"I fight every day for a place at Real Madrid and it makes me angry that things are always being invented about me. But I am calm because I know it was nothing."
Carlo Ancelotti had earlier suggested any attempted gesture from Di Maria would have been "a mistake" but stressed he was not concerned over the incident.
Two goals from Cristiano Ronaldo - his 399th and 400th of his career - and a strike from Karim Benzema sealed the points for the hosts against Celta.