Nuno Espirito Santo's side went ahead during a dominant first-half performance when Cavaleiro finished a neat passing move with his first goal for the club since his decisive strike in the replay against Shrewsbury Town in the previous round.
City, who are the Championship's form team under Lee Johnson, improved in the second half but, crucially, lacked a cutting edge.
Defeat leaves the home team free to focus on their promotion push that has seen them win their last seven consecutive league matches, while Wolves may start to dream of FA Cup glory for the first time since 1960.
Wolves grabbed the game's only goal after 28 minutes. The impressive Matt Doherty cut in from the right and pulled the ball back to the middle of the penalty area, allowing Cavaleiro to calmly apply a side-footed finish.
Doherty hit the post from a tight angle moments later, while Leander Dendoncker stung the palms of Frank Fielding when he went one-on-one with the City goalkeeper, opting for power over accuracy with his finish.
Jay Dasilva had the whole goal to aim at when he blazed a shot over the crossbar from just inside the box as City made a bright start to the second half.
Substitute Matty Taylor provided John Ruddy with his first test of the game with a shot from close range, while the Wolves goalkeeper made an even better stop to deny Marlon Pack, diving low to keep out a long-range drive.
But it was Fielding who made the save of the game when he leaped to push Joao Moutinho's dipping free-kick over the crossbar 15 minutes from time, paving the way for a last-ditch comeback that failed to materialise.