The young goalkeeper was punished for failing to claim a corner in an otherwise fine display as Georgy Kondratyev's side claimed an important victory in their opening match
The defensive midfielder capitalised on an error by goalkeeper Michael O'Keeffe to nod home a corner aimed at the back post on the stroke of half-time.
It was not the first time Belarus had threatened from a set-piece as the discipline and organisation of Georgy Kondratyev's side triumphed over the promising and inventive but, in the end, fruitless passing play of New Zealand.
Neil Emblen paired Shane Smeltz and Chris Wood in attack for the Oly Whites, with Kostas Barbarouses and Marco Rojas providing support from the flanks, while Ryan Nelsen anchored the defence.
Sergei Kornilenko, as expected, led the line for Belarus, with manager Kondratyev selecting five midfielders in a 4-2-3-1 formation.
Belarus broke forward immediately from the kick-off, with centre-back Tommy Smith guilty of tugging at the shirt of Kornilenko, an offence for which he received a yellow card, in the opening minute. The Krylia Sovetov striker failed to take advantage from 20 yards, pulling a low free-kick well wide.
New Zealand quickly settled, however, and resisted the temptation to launch passes at the 6' 3" target of Wood, building their attacks from the back and through midfield lynchpin Michael McGlinchey.
Left-back Ian Hogg had goalkeeper Aleksandr Gutor scrambling when his cross from deep looked like it might drop underneath the crossbar before defender Sergei Politevich was required to divert a dangerous low ball from Rojas behind for a corner.
Emblen's team did take advantage of Wood's aerial presence when McGlinchey curled a free-kick from the left on to the head of the 20-year-old striker, but his effort lacked power and was retrieved by Gutor, who made a good save low to his right.
Set-pieces looked to be Belarus' only route to goal as New Zealand remained in control, and both Kornilenko and Mikhail Gordeichuk came close to pouncing on one such opportunity from a corner kick. After a delivery from the right had ricocheted between two New Zealand players, the former's shot was superbly blocked by McGlinchey before O'Keeffe made an excellent save to deny the latter.
Gutor had reacted angrily when Wood challenged him from a high ball with some zeal and Smeltz took the opportunity to attempt to get under the goalkeeper's skin from a similar situation, drawing further choice words from the BATE Borisov player.
He was focused on the job at hand, at least when Rojas tore through the middle of the pitch before unleashing a left-footed shot that Gutor did well to tip over the bar.
New Zealand's play up to the edge of the opposition penalty area remained positive but Emblen's concern at their failure to carve out clear-cut chances will have grown as Belarus' counter-attacking threat became more potent. Gordeichuk sent a poor shot wide when running into all kinds of space on the right flank.
Eventually, with the last attack of the half, the Oly Whites were punished for their profligacy. The inexperienced O'Keeffe, who currently plays for Fairfield University in the United States, attempted to punch Renan Bressan's corner clear but missed the ball entirely, leaving Baga to stoop and head home at the far post.
In the second half, Belarus were content to protect their lead by any means necessary. Kornilenko was fortunate not to go into the referee's book when he missed Rojas with a wild hack and Politevich and Baga also got away with bundling the winger down higher up the pitch.
Smeltz was played through on goal by Wood shortly after but the Perth Glory striker's poor touch took him back into trouble as New Zealand struggled to conjure the quality needed to break down their opponents' defence.
Kornilenko, somewhat muted as an attacking force until this point, almost put the game to bed after 70 minutes when he spun on the edge of the box before firing a left-footed shot at O'Keeffe, but the goalkeeper made an excellent save.
Emblen introduced 17-year-old Cameron Howieson, of Burnley, for the impressive Rojas in an attempt to find the breakthrough but Belarus took control in the final 10 minutes.
O'Keeffe will be left disappointed to have made a crucial error in an otherwise fine display, as he kept his country in the game with two outstanding saves from substitute Andrei Voronkov in the final minutes.
After beating away the Neman Grodno player's shot from the edge of the box, the 21-year-old brilliantly tipped his header from the resulting corner over the bar.
Kornilenko then had time to slam a shot against the crossbar from a tight angle after turning Nelsen before Ilya Aleksievich was denied by O'Keeffe and Tim Payne hacked a rebound off the line.