Edivaldo Rojas stunned the home side by putting the unfancied Bolivians infront early in the second half, and the Albiceleste had Sergio Aguero to thank for rescuing a point for Sergio Batista's men.
Star player Lionel Messi was none too impressed by Bolivia's goal, but was grateful that his side secured a draw.
"They surprised us with a sh*tty goal and we drew levelled thanks to Kun [Aguero]," he told Goal.com after the game. "But we are upset because we didn't achieve our goal.
"We did some things well, but we have to keep growing and improving."
Argentina's hero Aguero also urged the team to put their early setback behind and bounce back stronger in the upcoming games.
"It is the first game, they are all going to be difficult. Bolivia were very tight at the back, but the key is that we did not lose," he said after the match.
"I think that in the second half Checho [coach Sergio Batista] went looking for the game, he put in another forward and we looked for the goal. It couldn't be done. Bolivia scored a goal, I don't really know how, but the key here is we took a point away."You always get better as the games go on. We will try to correct the errors made tonight to make sure they don't happen again."