World Cup winners N'Golo Kante and Olivier Giroud have spent little time working with the new head coach, who succeeded fellow Italian Antonio Conte at Stamford Bridge.
Sarri also has a question mark hanging over Hazard's Belgium team-mate Thibaut Courtois, who skipped training this week as the goalkeeper seemingly tries to force a move to Real Madrid.
Kante, Giroud and Hazard came off the bench in the second half of a 0-0 draw against Lyon in the International Champions Cup on Tuesday, but Sarri feels it is too soon for them to play a competitive game.
"It was better for them to play 25 minutes than to have a training session alone with the physical coach. They have to work, obviously.
"I'm very happy. I can work with all my players for the first time tomorrow, so we have only three days to prepare for the next match. And the next match will be very important. Very, very, very important."
Asked directly if it would be difficult for the trio to face Huddersfield, Sarri said: "Yes, I think so. Especially all together."
While Kante is a doubt to play in Saturday's Premier League opener after helping Les Bleus to win the World Cup, Ruben Loftus-Cheek shone in midfield.
Hazard has called for the England international to be given more opportunities at Chelsea and Sarri is excited by the 22-year-old's potential.
"He's potentially a great player," Sarri said. "He has to improve from the tactical point of view.
"But I think that's not a problem. He's very young. I think he can be a very beautiful player for us."
Chelsea are not known for showing patience with managers but Sarri feels it will take time for his plan to come together.
"I think that we have to work," Sarri added. "But we started to work, to work two days ago.
"But I think, potentially, we can become in two or three months a very good team."