The deadlock was broken just 10 minutes into the game and it was Tshegofatso Mabaso, who hit the back of the net to make it 1-0 to Celtic.
Former Orlando Pirates striker Ndumiso Mabena set up the young forward and he netted his third goal of the season with a left-footed effort.
The score was 1-0 to Celtic during the half-time break with Leopards having failed to convert their chances.
A tightly contested second half followed as Leopards looked to equalize, but Celtic defence which was marshalled by Kaizer Chiefs loanee Lorenzo Gordinho, looked solid.
Celtic were able to hold on to their slender lead in the closing stages of the game and ultimately they emerged 1-0 winners over Leopards on the day.
Despite the win, Celtic remained second on the league standings, while Leopards slipped down to 10th on the table.
Elsehwere, Highlands Park recorded their first win of the season in the league against Golden Arrows at the Makhulong Stadium.
The Lions of the North took the lead 20 minutes into the game after the hosts were awarded a penalty.
Mothobi Mvala stepped up to beat Arrows goalkeeper Nkosingiphile Gumede from the spot-kick to make 1-0 to Highlands Park.
However, Abafana Bes'thende levelled matters through Danny Venter, who has established himself as one of Arrows' key players since joining them from Free State Stars.
The left-footed attacker beat goalkeeper Tapiwa Kapini with a decent effort to make it 1-1 two minutes after the half-hour mark.
Parity was only restored for four minutes as Highlands Park captain Mvala scored to make it 2-1 to his side with the half-time break fast approaching.
Highlands Park were leading 2-1 at the interval following a lively first half.
Arrows tried to equalize again in the second half with Clinton Larsen having introduced exciting attacker Siphelele Magubane.
However, the Lions of the North defence was able to contain Abafana Bes'thende in the latter stages of the encounter.
Ultimately, Highlands Park ran out 2-1 winners over a struggling Arrows side.
The victory took Highlands Park to sixth on the league standings, while Arrows moved down to 11th spot on the table.