The South American sides are two of the favourites to win the tournament this year, with the hosts largely tipped to go all the way and pick up a sixth world crown.
And Scolari says he is ready to take his side to the final and admits he hopes to meet Alejandro Sabella's men in the Maracana on July 13 to decide who takes home the trophy.
"I don't have to worry about Argentina," he told reporters. "I hope we can make it a Brazil-Argentina final, a South American final, with players that have great technical ability.
"I don't want Argentina to drop out beforehand. I want them to play their best football. If they lose, it's not my problem. My problem is Brazil. I have to take Brazil to the final, taking on whoever stands in our way."
The 65-year-old went on to praise Neymar, who he insists is a vital player for Brazil even when he is on an off day.
"Even when he doesn't play well, if he makes an effort, for us it's marvellous, because not only is he excellent as a player, but also as a person and member of the group.
"Neymar is part of a group, he has extraordinary talent but he has his off days like everyone does. The group has to help him out."