The Barcelona defender has leapt to the defence of his team-mate after he was handed a four-match international ban
The 23-year-old kicked a ball at Pablo Armero at full-time and was then involved in an altercation with Carlos Bacca, which saw both players given their marching orders.
Neymar was subsequently handed a suspension that has ruled him out of the rest of the tournament, but Alba understands the frustration of his Barcelona team-mate.
"I think the suspension is really unfair," he told Sport.es. "The opposition look for him a lot. Neymar is very patient and has been silent for a long time.
"I would have exploded long before Neymar did."
Alba also responded to Barcelona presidential candidate Jordi Majo's claim he was offered Real Madrid defender Sergio Ramos by an intermediary as an "electoral asset".
Further reports claimed the centre-back is unhappy at the Bernabeu and would be willing to join his side's arch-rivals.
Alba added: "I would never join Real Madrid."