The Championship club are only three points away from the playoff positions but have decided that a new boss would be able to get better performances out of their squad
Youth team boss Neil Redfearn has taken temporary charge of the Elland Road outfit who are currently in 10th place in the Championship, three points behind the playoff spots.
Leeds chief executive, Shaun Harvey, told the club's official website: "We have 18 games to go this season and are still within touching distance of the playoffs, but felt with the transfer window now closed we needed to make the change at this time in the belief that a new managerial team will be able to get more out of the existing squad of players and make the difference."
Grayson had been at the club for three years and steered Leeds to promotion from League One in 2009-10.
The Whites had also sold their captain, Jonny Howson, to Norwich City during the January transfer window.
Yorkshire-born Grayson's last game in charge was a 4-1 loss to Birmingham City at Elland Road on Tuesday.