The Australian star continued his recent good form with a crucial header after Luis Robles kept the Red Bulls in the game with a penalty stop.
Robles was called into action after Brandon Barklage took out Eddie Gaven's legs inside the penalty area in the 71st minute. Robles guessed correctly to deny Federico Higuain spot kick glory, parrying the Argentine's effort wide.
Cahill was in the right place in the 80th minute to score the winner, getting in the way of a rebound and nodding his looping header just out of reach of Andy Gruenebaum's grasping fingers.
The victory is New York's third straight in MLS, and has the Red Bulls equal with first-placed Montreal on points, trailing only on a single goal in goal difference. Columbus now has a 3-3-3 record.