The tournament debutants were within two minutes of all-but securing a place in the last 16 on Saturday, only for Birkir Saevarsson's unfortunate own goal to hand Hungary a 1-1 draw and prolong their wait.
A win against the Austrians at the Stade de France on Wednesday will see them through while a draw may be enough depending on events elsewhere. However Gudmundsson would rather not take that chance.
He said: "A draw could be enough but we want to win the game. If we win we are definitely through, that's our aim.
"We try to win every game and that will be the same against Austria.
"It will be tough but we've shown that we can beat these kind of teams. We are really positive and hopefully it will happen as it would be massive for the country."
Gudmundsson, who plays his club football for Charlton Athletic in England, was understandably downbeat after Iceland's late heartbreak at the Stade Velodrome but admitted the equaliser was brought on, in part, themselves.
He added: "We were almost through so it's massively disappointing. We defended so well, got the goal, and in the 88th minute they score. It was a mistake by us and that's not good enough at this level.
"We just need to forget about this game now and focus on the next one."
The 25-year-old feels an improved defensive display should prove enough to overcome Austria.
"We are really tough to beat, we don’t concede too many goals and we always score," he said.
"It's tough to break us down but we have conceded two goals in two games and we think that’s not good enough. So hopefully we can keep a clean sheet in the next game because we always score. If you keep a clean sheet you win games."