The Tottenham midfielder played 90 minutes as the Three Lions lost their opening UEFA Nations League game to Spain at Wembley on Saturday.
However, a slight injury sustained during that game has now seen him pull out of the squad, and Gareth Southgate has opted against calling up a replacement for the friendly on Tuesday.
Alli has played in all four of Spurs' games thus far this season, scoring in the opening day win over Newcastle United, and played a key role in the Three Lions' run to the World Cup semi-finals over the summer.
Chilwell, meanwhile, is set to replace Luke Shaw at the King Power Stadium, who suffered a concussion during the loss to La Roja.
Gray has won 21 caps for the England Under-21 team, while Chilwell has played for that age group nine times.