The Selecao fell behind after an own-goal from Marcelo after 11 minutes before two goals from Neymar, the second a hotly-debated penalty, and a late Oscar strike gave them all three points in a 3-1 win.
Croatia boss Kovac was furious at what he saw as an extremely soft penalty when Fred went down under a challenge by Dejan Lovren, but Scolari claimed he saw nothing wrong with referee Yuichi Nishimura's decision to point to the spot.
"The ref saw a penalty. He gave the penalty, and he's the one who decides," the 65-year-old told reporters after the game.
"And we think it was a penalty as well.
"I'm not going to talk about what the other coach said. All I say is Brazil has five world titles. So Brazil had five circuses to win five cups?
"I understand what my colleague has said, and I respect it, because I think the reason he said was due to the result, and I would probably be complaining if I had lost the match as well. But I didn't see us getting favoured by the referee. That's just something that has been said and it will stay said and there's no point in arguing about it."
The Brazil boss was positive about his side's chances in the tournament after the victory and also reserved special praise for Chelsea midfielder Oscar, who delivered an impressive performance in Sao Paulo.
"Neymar took the man-of-the-match award, but if there was a second award it would have to be Oscar's," he continued.
"Oscar is a good boy. Before the game I went to him and told him he still owed me a goal and that today was the day, and afterwards he came to me and said he paid his debt.
"But one of the things that we all wanted to see and had doubts about was over the resilience of this young group. Today we know we have evolved at least a bit. There's still a lot we need to improve to see if this group can take a punch and get it back together to make it to the end of a match with a win."