The defender reacted quickly after Nicolas Otamendi's header came back off the upright and tapped home to make the difference shortly before the break.
Sampaoli fielded an attacking XI in his first game as Argentina boss, including Lionel Messi, Gonzalo Higuain, Paulo Dybala and Angel Di Maria.
Brazil, meanwhile, were without star attacker Neymar, but Tite did name Premier League trio Gabriel Jesus, Willian and Philippe Coutinho in his line-up.
The Selecao were left to rue an astonishing miss in the second half, though, as Jesus struck the post with the goal gaping as Tite tasted his first defeat since replacing Dunga at the helm, bringing an end to a run of nine consecutive victories.
Sampaoli's side made an encouraging start and came close to opening the scoring after just six minutes, Di Maria blasting a powerful shot against the left upright from a narrow angle after being set up by Dybala.
Brazil first threatened midway through the half when Willian escaped the attentions of his marker down the left and unselfishly squared for the unmarked Coutinho, only for the Liverpool man to take too much time and allow Otamendi to make a last-ditch block.
Di Maria then nearly surprised Weverton with an ambitious volley from the corner of the box after Gil failed to adequately clear a cross from the left, the goalkeeper only just managing to keep the winger's attempt out with his chest.
But there was no denying Argentina in the 45th minute as Mercado broke the deadlock with a calm finish from close range, reacting first to the rebound after Otamendi hit the upright following a cross from Di Maria.
Brazil went in search of the equaliser after the break and Jesus was first to try his luck when he forced goalkeeper Sergio Romero into a one-handed save with a curled shot toward the top corner from 15 yards out.
Jesus refused to give up, though, and the Manchester City man should have levelled the scoring at the hour mark. He rounded Romero after a brilliant throughball from Fernandinho, only to then hit the post, with Willian rattling the other upright from the rebound.
And those misses would prove to be costly for Tite's men as the Argentina defence held firm in the remainder of the game to get Sampaoli off to a triumphant start.