The 22-year-old was shown just a yellow card for the incident, but could quite easily have been punished with a red had referee Yuichi Nishimura adopted a stricter approach.
The Japanese official came in for criticism from Croatia boss Niko Kovac for awarding Brazil a controversial penalty and harshly disallowing Ivan Perisic's second half goal.
But Neymar believes he was right to let him off with a booking after insisting his altercation with Modric was nothing more than an accident.
When asked about the incident, the Barcelona star told Times of India: "Well, it was unintentional. The referee was close to us. If he could see any malice in it, he would have taken a more stringent call."
The yellow card leaves Neymar one bookable offence short of a one-match suspension, sparking fears he could miss a potentially important clash.
"I am not worried," he explained. "I shall be playing my natural game. During the match anything can happen and we have to take it as it comes. We cannot apprehend anything and sit idle."
The Barcelona star scored twice against Croatia, the first a left foot drive from 20 yards before slotting home a tame penalty that goalkeeper Stipe Pletikosa ought to have saved.
"For a second my heart was in my mouth!" he said "Ultimately it went in and I felt relieved. We needed to take the lead and it was a perfect opportunity for us. I was not tense but changed my mind while taking it."
Mexico are Brazil's next Group A opponents on Tuesday.