The visitors had won on their last three visits to Selhurst Park but struggled on this occasion to open up the determined Eagles, managed by former Liverpool and England boss Roy Hodgson, in Monday's clash.
However, Milner crucially converted from the penalty spot after a foul on Mohamed Salah at the end of the first half, with Mane's breakaway goal wrapping up the points deep into stoppage time.
Palace opened their campaign by winning at Fulham but never looked likely to equalise after falling behind, especially once the otherwise outstanding Aaron Wan-Bissaka saw red for a clumsy foul on Salah, with Jurgen Klopp's men looking on track for a title challenge.
Liverpool thought they had scored in the 21st minute as Wayne Hennessey inadvertently threw the ball in his own net, but referee Michael Oliver gave a foul for Mane's challenge on the Wales goalkeeper, who also collided with team-mate Wan-Bissaka in mid-air.
Naby Keita sent Salah racing clear but while last season's Golden Boot winner collected the ball neatly, his lobbed finish easily cleared the crossbar. Andros Townsend was closer at the other end, though, seeing a 30-yard curler crash back off the woodwork.
But Liverpool took the lead on the stroke of half-time, Milner confidently tucking home a penalty when former Liverpool defender Mamadou Sakho was adjudged to have tripped Salah in the box.
Luka Milivojevic's free-kick forced a flying save from Alisson early in the second half, but hopes of a home fightback were extinguished with 15 minutes to go when Wan-Bissaka walked for bumping over Salah as the Egypt international raced through on goal.
And Palace were duly caught out in the 93rd minute, Salah sending Mane through as Liverpool countered from defending a corner, the Senegal forward holding off Patrick van Aanholt before rounding Hennessey to tuck in his sixth goal in eight games against the Eagles.
What it means: Reds look serious contenders
Liverpool dropped too many points on the road last season, a mediocre away record costing them dearly as Manchester City romped to the title.
But, bolstered by the arrival of midfield dynamo Keita, Klopp's men held Palace at bay as they demonstrated the steely edge that is required to contend. Palace lacked a goal threat but a top-half finish does not look beyond Hodgson's well-organised side, despite slipping to defeat in their home opener.
Pat on the back: Reliable Milner hits the spot
Salah's stardust overshadows his team-mates but it was Milner's penalty that put Liverpool on course for what could be a vital three points in the final reckoning. The midfielder, who kept club captain Jordan Henderson on the bench and wore the armband, has scored eight consecutive Premier League goals from penalties - setting a record - and recorded a passing accuracy of 91.8 per cent.
Boot up the backside: Benteke blunt for Eagles
Supported by flying wingers Townsend and Zaha, Christian Benteke needed to hold the ball up and set a platform for Palace to attack. Instead, the ex-Liverpool striker, who only scored three Premier League goals last term, was dominated in the air by Virgil van Dijk before being replaced by Alexander Sorloth.
Key Opta facts
- Liverpool won their 34th Premier League game played on a Monday – only Man Utd (43) and Arsenal (36) have more victories.
- Crystal Palace have won just two of their previous 20 Premier League games played against 'big six' sides at Selhurst Park (D2 L16).
- Roy Hodgson suffered his 100th defeat in the Premier League as a manager (W87 D65 L100).
- Aaron Wan-Bissaka is the youngest Crystal Palace player to be shown a red card in the Premier League (20 years, 267 days).
- James Milner extended his current record for the most Premier League games scored in without losing any of them (48 Games: W38 D10 L0).
- Sadio Mane has scored six goals in his eight Premier League appearances against Crystal Palace, more than he has against any other side.
Liverpool return home for a Saturday fixture against Brighton and Hove Albion, who beat Manchester United last time out, while Palace make the short trip to Watford - Javi Gracia's men have two wins from two - the following day.