His side are bottom of World Cup qualifying Group A and have yet to win a match, losing on their away trips to Wales on Friday and Belgium on Tuesday.
Though Levein has received backing from members of his squad, Smith has joined those calling for the Scotland boss to step down.
"I think the best thing would be for him to resign and take that pressure away from the SFA," Smith told the BBC.
|100/1||Scotland are 100/1 with William Hill to qualify for World Cup 2014
"If you say that and you don't qualify, you become the common denominator of it all."
Though adamant that Levein needs to leave his position, Smith suspects the SFA will stand by him due to the good work he has done with the Scottish youth set-up.
"I would personally be very surprised if they did move him on because he's been very much involved with the SFA, more so than just managing the national team," he continued.
"Now, in my opinion, his remit was to manage the national team and to take the national team to a major championships - and he hasn't done it.
"Regardless of what other work he's doing, he's not the performance director, that's Mark Wotte."