The Team of Choice edged out Phunya Sele Sele 2-1 in a quarter-final encounter which was played at the Harry Gwala Stadium in Pietermaritzburg.
Lebohang Maboe put the hosts into the lead just a minute into the game to make it 1-0 to Maritzburg much to the delight of the home fans.
Celtic then pushed for the equalizing goal, but they found it difficult to unlock the Maritzburg defence which was led by the experienced Siyanda Xulu.
The score was 1-0 to the Team of Choice during the half-time break with Phunya Sele Sele having failed to level matters in the first half.
The visitors did restore parity in the closing stages of the match through Jacky Motshegwa, who beat Maritzburg goalkeeper Richard Ofori to make it 1-1 in the 76th minute.
However, Andrea Fileccia had other ideas as the Belgian striker snatched a late winning goal to had Maritzburg a 2-1 win over Celtic on the day.
Meanwhile, Stars progressed into the semi-finals of South Africa's FA Cup after defeating Ubuntu Cape Town.
Ea Lla Koto beat Ubuntu 4-2 on penalties following a 1-1 stalemate in the quarter-final match which was played at the Athlone Stadium in Cape Town.
The hosts, who are campaigning in the National First Division (NFD), broke the deadlock two minutes before the half-hour mark through Taahir Ganga to make it 1-0.
However, Stars fought back and the Premier Soccer League (PSL) outfit managed to level matters on the stroke of half-time to make it 1-1 through Goodman Dlamini.
The two teams were locked at 1-1 at the interval between Ubuntu and Stars.
A tightly contested second half followed as the two teams searched for more goals, but they could not score and match ended in a 1-1 draw in regulation time, and extra-time followed.
There were no goals in extra-time and a penalty shootout followed with Ubuntu looking to eliminate another PSL club having knocked out Polokwane City, and AmaZulu.
Stars emerged 4-2 on penalties after Ubuntu midfielders Duncan Adonis and Dillon Goss missed their spot-kicks.