Caballero was reunited with Pellegrini in July - the 32-year-old goalkeeper previously played under the Chilean at Malaga - as City prepare to launch their Premier League title defence.
Hart was dropped last season following a string of high-profile mistakes, allowing Romanian Costel Pantilimon a run in the first-team, and will now face competition from the Argentine.
"I have two number one goalkeepers," Pellegrini said ahead of Sunday's Community Shield clash with Arsenal.
"I am sure Willy will be very important for us but I also think we have the best goalkeeper in England in Joe Hart.
"Top teams need two players for each position because we have to play so many games during the year."