Stuart Armstrong crowned a tireless performance by arrowing in a superb 35th-minute opener – his 11th goal of the campaign but Brendan Rodgers' side were short of their best and flagged during the closing stages.
Rangers displayed impressive fighting spirit, with incoming boss Pedro Caixinha looking on from the stands, and Hill pounced from close range in the 87th minute to end Graeme Murty's spell in caretaker charge on a pleasing note.
The visitors acquitted themselves well in the opening stages and should have led midway through the first half.
A bad misjudgement by Celtic centre-back Dedryck Boyata let Martyn Waghorn through on goal but Craig Gordon stood firm to save impressively with his left boot.
Armstrong was denied when Wes Foderingham faintly touched his whipped free-kick on to the post but the Rangers goalkeeper was powerless as he lashed into the bottom right corner from the edge of the box.
Scott Sinclair jinked in from the left in the 56th minute and played a pass through for Armstrong - who was rarely out of the action – to force a sharp stop from Foderingham.
The duo's duel continued as Foderingham tipped a swerving 25-yard effort away from his top corner, while opposite number Gordon again excelled himself to thwart Waghorn at close quarters.
That chance encouraged Rangers to reassert themselves as an attacking force, Jason Holt curling agonisingly past the far post, and Hill was on hand to spark wild celebrations among the travelling supporters when an outstretched Gordon parried Emerson Hyndman's shot.
Hill survived a strong penalty shout from Leigh Griffiths but Rangers edged to within eight points of second-placed Aberdeen, who lie 25 points shy of Celtic.
88' GOAL! Clint Hill equalises for the Gers! pic.twitter.com/SjJchh0kQK— Rangers FC (@RangersFC) March 12, 2017