The match appeared to be heading for stalemate after Jordan Pickford denied Milivojevic from 12 yards on the hour, but Everton scored twice in two minutes with the clock winding down to secure a third successive Premier League win.
Everton created several chances in a first half they generally controlled, but Palace went closest when Cheikhou Kouyate hit the frame of the goal with a header.
Palace then saw Milivojevic denied from the spot and Everton took full advantage, as Calvert-Lewin headed them into an 87th-minute lead before Tosun added a second on the break.
Despite their control of the game, Everton did not test Wayne Hennessey in the Palace goal until the 28th minute when Gylfi Sigurdsson met Seamus Coleman's cut-back and forced a vital save.
The visitors soon enjoyed a promising spell of their own and went close to the breakthrough – Kouyate heading against the crossbar from close range after James Tomkins nodded Andros Townsend's corner across goal.
Wilfried Zaha won Palace a penalty in the 60th minute when tripped by Coleman, but Milivojevic saw his effort – struck straight down the middle – saved by Pickford's foot.
Theo Walcott wasted a chance soon after, but his replacement – Ademola Lookman – ultimately set up the opener three minutes from time, producing a fine cross for Calvert-Lewin to head home.
Tosun then put the game beyond Palace two minutes later, racing on to Michael Keane's long ball and coolly slotting past the helpless Hennessey, moving Everton up to eighth in the Premier League.
What it means: Silva's subs a masterstroke
With Andre Gomes, Bernard and Walcott not making much of an impact in the second half, Silva withdrew them all. Tosun, Lookman and Calvert-Lewin entered and all three made a vital contribution. The coach could not have hoped for a better answer from his substitutes.
Jordan pick of the bunch
Aside from his generally authoritative display, Pickford's importance to Everton on Sunday was huge. Although Calvert-Lewin and Tosun ultimately took the plaudits with their late goals, had he not saved Milivojevic's penalty, there is every chance Everton would have been too deflated to go on and score.
Walcott wasteful and ineffective
An otherwise quiet day for former Arsenal man Walcott could have been really livened up when he was present with a great opportunity in the second half, but he shot straight at Hennessey. It summed up his day.
Everton face a trip to Old Trafford to face Manchester United next Sunday, while Palace host Arsenal a few hours earlier.