Second-half goals from Michael Antonio and Cheikhou Kouyate gave the Hammers all three points at Upton Park with Villa now 10 points off safety at the foot of the Premier League table.
Asked to clarify reports suggesting Garde may consider his position at the club at the end of the season, the Frenchman - who only took charge in November - was unwilling to comment.
"I'm not speaking about that," he told BT Sport. "This not the time at all."
Garde was also reluctant to comment on Villa’s lack of activity in the January transfer window, saying: "After a game, it is really not the time to speak about that because there is a lot of emotion."
Aston Villa had to play 75 minutes with 10 men following the early dismissal of striker Jordan Ayew for a reckless elbow on Aaron Cresswell, but the Villa boss was unwilling to discuss the striker's indiscretions.
Instead, he preferred to focus on referee Jon Moss' failure to award the away side a penalty following Antonio's handball in the box and a questionable offside in the build-up to the Hammers' opening goal that was not picked up on by the officials.
READ MORE | Bilic delighted with patient West Ham
"I feel very sorry for the guys – to be beaten 2-0 after the bravery that they showed," he added
"It's a big mountain for 10 men. We started well, we were well in the game, we were denied a penalty – it's the third in a row and it is a big thing for us.
"I'm sorry for the ref but it was a clear penalty. It was in the time that we were well in the game; we didn't control the game but 11 versus 11 we started well.
"Without this penalty and [with] this stupid red card, the game was over.
"You have to defend well and that's what we did until the first goal. I've seen it again and I'm not sure if it's not an offside as well.
"I didn't speak yet with Jordan. Anyway, it will stay private between him and me."