Substitute Fahad Al Muwallad's second-half strike was enough to seal the win over the Group B winners, who had already secured their place at Russia 2018.
The result means Saudi Arabia join Japan in the group stage of next year's tournament, while Australia, who earlier beat Thailand 2-1, must settle for a place in the AFC section play-off.
The home side, backed by a vociferous crowd at King Fahd Stadium, had Japan on the back foot early on and came close to opening the scoring through a fierce Yahia Al Shehri drive.
They failed to create much in the way of meaningful chances in the first half despite their dominance of possession, though, and Abdullah Al-Mayouf had to be alert to parry a superbly controlled volley from Hotaru Yamaguchi five minutes before the interval.
Mansour Al-Harbi clipped the woodwork with a strike from a tight angle after a swift break down the left, but the frustrations of the home fans were beginning to build as Japan started to assume more control of the play.
Al Muwallad was introduced at half-time to a welcome reception as Saudi Arabia sought a greater attacking threat, and the forward's first contribution was to send in a teasing low cross that Yuto Nagatomo raced back to clear.
Hiroki Sakai saw an effort hooked off the line from a corner before Genki Haraguchi shot narrowly wide, but Al Muwallad missed the best chance of the first hour moments later, wrestling the ball away from Nagatomo only to be thwarted by Eiji Kawashima's foot when through one-on-one.
But Al Muwallad was not to be denied on 63 minutes, collecting Nawaf Al Abid's pass before blasting the ball beyond Kawashima and into the roof of the net from close range, sending the packed crowd into raptures and Saudi Arabia to Russia.
Japan progress to next year's group stage along with them, while Australia must contest a play-off against Syria, with the winner going on to meet the fourth-placed team from CONCACAF in an intercontinental tie in November.