The Premier League champions failed to find the net throughout the first half at Rodney Parade as the fourth-tier hosts initially frustrated their more fancied opposition.
Leroy Sane broke the resistance in the 51st minute, while England youth international Foden scored a brace - either side of Padraig Amond's reply for Newport - and Riyad Mahrez was also on target.
Veteran midfielder Silva, one of many City regulars who started in south Wales, completed 82 minutes in difficult conditions and earned praise for his professionalism.
"Incredible," Guardiola told reporters when asked to describe the World Cup winner's performance.
"[He is] 33 years old, fights, wins duels. Really, really good.
"David is a fantastic player. He played here at a League Two club like an 18-year-old, like Phil Foden."
Newport, who are 15th in England's fourth tier, put in a superb display against their illustrious visitors and Guardiola added: "I was really impressed. The home crowd was amazing. I am surprised they are in the position they are.
"Here I realised English football better than ever and why the cup is special for a team in those conditions - they use their strength points and that's why the cup always has big surprises."
"We are still in four competitions and we accept all the challenges," said Guardiola. "Today it was League Two and in four days we play on the biggest stage. We have to change mentality."