Costa netted in Chelsea's 2-0 FA Cup third round win over Scunthorpe United on Sunday before limping off in the final minutes.
Eden Hazard picked up a groin injury in his side's 3-0 win at Crystal Palace last week and was expected to be back for Everton's visit to Stamford Bridge next Saturday.
But Hiddink says he will be patient with the Belgium international in a bid to avoid further injuries.
Hiddink said: "We want to be cautious with Eden. We don't want him to fall again with muscle problems.
READ MORE | Oscar denies Costa bust-up
"I want to be careful with him so that he doesn't start too early. It will maybe be a week later [than necessary] rather than too early."Oscar also picked up a toe injury in the victory against League One Scunthorpe as Chelsea produced a performance that Hiddink was not fully satisfied with.
"There were some vulnerabilities in the second half, it's true. So in that way we have to improve," Hiddink added.
"We scored a beautiful goal and then we got a bit sloppy finishing it off.
"They were still in the game and there was a big threat from free-kicks and corners. As long as it was 1-0 the threat was actual for us.
"This was an important game for us. We are very determined to go on and get silverware at the end and that's why you have to play these difficult games."