Phunya Sele Sele played host to National First Division (NFD) side Richards Bay at the Dr. Molemela in the City of Roses.
The deadlock was broken by Mpho Maruping, who beat Richards Bay goalkeeper Xolani Ngcobo to make it 1-0 to Celtic five minutes before the half-time break.
Celtic were leading 1-0 at the interval thanks to Maruping's goal.
The visitors pushed forward in numbers in search of the equalizing goal in the second half, but they were struggling to penetrate the Phunya Sele Sele defence which was marshalled by Alfred Ndengane.
The hosts snatched two late goals in the closing stages of the game through promising youngsters Neo Maema and Kabelo Dlamini.
Richards Bay managed to grab a consolation goal through Thabiso Nkoana in stoppage time and ultimately, Celtic won 3-1 on the day.
At the Princess Magogo Stadium, NFD club Royal Eagles welcomed their Kwa-Zulu-Natal rivals Maritzburg.
The Team of Choice opened the scoring through Deolin Mekoa, who hit the back of the net on the stroke of half-time.
Maritzburg were leading 1-0 at the interval in the biggest city in KwaZulu-Natal, Durban.
Royals pushed for the equalizing goal in the second half and they did restore parity through Mame Niang.
The experienced striker scored from the penalty spot-kick to make it 2-1 with 18 minutes left on the clock.
However, Siyanda Xulu had other ideas as the defender scored a late winning goal to seal Maritzburg's 2-1 victory in the dying minutes of the game.
Celtic and Maritzburg joined PSL sides Kaizer Chiefs and Free State Stars, and NFD outfit Ubuntu Cape Town in the quarter-finals.
The last two round of 16 games will be played in midweek with Mamelodi Sundowns scheduled to host ABC Motsepe side EC Bees at the Loftus Versfeld Stadium in Tshwane on Tuesday.
Cape Town City will take on fellow PSL side Orlando Pirates at the Cape Town Stadium in the Mother City on Wednesday.