The in-form West Ham forward has netted four in seven Premier League matches this season and his perseverance against a determined Northern Ireland side paid off with a goal in the 71st minute.
Michael O'Neill's men withstood waves of pressure in Vienna and looked set to earn their first point in the competition after goalkeeper Bailey Peacock-Farrell made two crucial saves.
Visiting captain Steven Davis should have claimed the points when he was presented with a one-on-one against goalkeeper Heinz Lindner but he squandered the chance and Arnautovic made sure he would rue the miss by calmly slotting home the winner.
Northern Ireland goalkeeper Peacock-Farrell picked up a backpass to present Austria with a first-half set-piece inside the penalty area but Arnautovic's shot hit the defensive wall.
Paddy McNair drew a good save from Lindner after half-time before Peacock-Farrell did well to fingertip Arnautovic's low shot around the post following neat footwork from the West Ham forward.
Davis missed a gilt-edged chance midway through the second half when Stuart Dallas put him through on goal, the midfielder outpacing the Austrian defence but then dragging his shot wide.
The game's decisive moment arrived at the other end when Peter Zulj's pass found Arnautovic onside and he beat McNair for pace before side-footing a shot into the far corner, beating Peacock-Farrell low to the goalkeeper's left.
With six minutes left, substitute Will Grigg glanced a header onto the post from a McNair corner but the ball bounced up into Lindner's hands to sum up a frustrating night for Northern Ireland.
What it means: Austria off the mark as Northern Ireland struggle
After a disappointing defeat to Bosnia-Herzegovina in their Nations League opener, Austria claimed a morale-boosting victory that keeps them in contention, while Northern Ireland must console themselves after following up their own loss to the Group B3 leaders by taking nothing from another gritty display.
Peacock-Farrell recovers from early error
Having looked naïve after picking up a backpass in the first half, Leeds United's Peacock-Farrell emerged as one of Northern Ireland's best players with impressive stops from Arnautovic and Marcel Sabitzer.
Lewis slow to react
While McNair was marking Arnautovic when he scored, it was left-back Jamal Lewis who played the forward onside in a moment that settled the match, the Norwich City defender failing to step up with his team-mates to play the offside trap.
As they look to avoid Nations League relegation, Northern Ireland have the chance to gain revenge for a 2-1 home defeat to Bosnia-Herzegovina when they travel for a return clash on Monday, while Austria play away to Denmark in a friendly a day later.