The Czech Republic international recorded his first strike for his new club moments after replacing the talismanic striker, with Brad Guzan having earlier saved a penaltyLibor Kozak's first Aston Villa goal gave his side their second Premier League away win of the season at Norwich City on Saturday.
The Czech - a first-half substitute for the injured Christian Benteke - scored the only goal of the game within moments of coming on to seal all three points for Paul Lambert's side.
Villa's goal came against the run of play and after an early missed penalty from Robert Snodgrass, given when Ciaran Clark was adjudged to have handled inside the area.
Even the introduction of Gary Hooper for his Premier League debut could not lift Norwich, who slipped to their third top-flight defeat of the season.
Canaries boss Chris Hughton made two changes to the side beaten by Tottenham, with Russell Martin and Jonny Howson replacing Steven Whittaker and Bradley Johnson.
|VIEW FROM CARROW ROAD
|By Theo Squires
Paul Lambert has the last laugh as he leaves Carrow Road with all three points, all of Norwich City's big-name strikers once again failing to deliver in front of goal.
The final whistle was met with boos from Canaries fans, while Chris Hughton's withdrawal of Nathan Redmond earned cries of "You don't know what you're doing!".
It could have been so different had Robert Snodgrass not seen an early penalty saved by the superb Brad Guzan. Libor Kozak scored the deciding goal and, while it may not have been pretty, it is Villa who claim the three points.
Villa started the game well and could have opened the scoring in the first five minutes as Andreas Weimann saw one effort well saved by John Ruddy and a second strike the foot of the post.
Having survived that scare, Norwich were gifted an opportunity to break the deadlock when referee Chris Foy awarded them a penalty after Clark had handled in the sixth minute.
The decision looked harsh on the defender but Brad Guzan came to his rescue with a fine save diving to his left to parry Snodgrass's poor spot kick.
Villa's injury problems continued midway through the half as top scorer Benteke was substituted with a groin problem. The Belgium international slipped when trying to control a long ball out of defence and, despite trying to play on, had to be replaced by Kozak.
Ricky van Wolfswinkel should have tested Guzan while Villa were readying the replacement but the Dutchman fired wide from a low cross from Martin and they were punished for their wastefulness moments later as the Czech Republic international grabbed his first goal for the club on the half-hour mark.
Kozak missed his initial effort on goal but fired into the net from six yards when Gabriel Agbonlahor rolled the loose ball back to him.
Norwich began the second half at a high tempo and spent large parts camped in the Villa half but still struggled to test Guzan. Good crosses from Snodgrass and Leroy Fer failed to find any team-mates inside the penalty area and Villa were able to clear with ease.
The visitors threatened on the counterattack but could not take their opportunities when they had sight of goal, with Tonev particularly guilty of missing chances.
Hooper then saw a late chance brilliantly tipped over by Guzan as the American secured Villa's first clean sheet in 27 Premier League matches.