Scotland recorded a 2-0 win away to San Marino to recover some pride but the Scottish Football Association (SFA) still opted to end McLeish's second spell in charge after just 14 months.
McLeish led Scotland to promotion in their inaugural Nations League campaign last year but defeat in Kazakhstan piled pressure on the former Rangers boss.
And McLeish's departure was confirmed by the SFA on Thursday, with Scotland not returning to action until a double-header against Cyprus and Belgium in June.
Assistants Peter Grant, James McFadden and Stevie Woods will also leave their posts.
"I am grateful for the opportunity to have managed my country for a second time and leave knowing that I gave my all in the pursuit of success," McLeish said in a statement.
"I am proud that together we finished top of our Nations League group and qualified for the Euro 2020 play-offs, which gives us a real opportunity to reach a major tournament for the first time in over 20 years.
"I am also pleased to have given many younger players a first taste of international football that will stand them – and the country – in good stead for the future."
Scottish Football Association chief executive Ian Maxwell said the country's status as one of the co-hosts for Euro 2020 was partly behind the move to change manager.
"The decision to part company was not an easy one, especially given Alex's status as a Scotland Hall of Fame member, earning 77 international caps, having played in three World Cups and taken charge of the national team for two spells," Maxwell said.
"It was only taken after full consideration by the board and after an honest and respectful conversation between myself and Alex earlier in the week. Ultimately, the performances and results as a whole in the past year – and, in particular, the manner of the defeat in Kazakhstan – did not indicate the progress expected with a squad we believe to be capable of achieving more.
"With that in mind, it was agreed that we would seek to recruit a new management team to take us forward for the remainder of European qualifying campaign.
"At this point I would like to pay tribute to Alex for the good grace with which he has accepted the decision and for remaining unstinting in his support of the team for the remainder of the campaign. It befits his status as a Scotland legend and Hall of Fame inductee.
"We firmly believe we have the potential to recover from the opening group defeat and unlock the potential of this exciting squad of players for the remainder of the campaign, safe in the knowledge we also have the play-offs as an additional route to Euro 2020."