The former Raith Rovers boss leaves the Edinburgh team sitting bottom of the SPL following Wednesday's 2-0 defeat away to fellow relegation strugglers St Mirren, his side having reached the Scottish League Cup final but floundered in the table.
A club spokesperson told the team's offical website: "When John came to the club, Hearts was embarking on a transitional phase in the club's history.
"He began that important period of change and it is now imperative we find the right man to take the transition forward. The overriding factor in this decision was the long-term well-being of Heart of Midlothian Football Club.
|28/1||Hearts are 28/1 with William Hill to lose 3-0 to Motherwell in their next SPL match
McGlynn spent 10 years at Tynecastle as a coach prior to his successful stint at Raith Rovers and admits his sadness at leaving a club who hold such a special place in his heart.
"I was very grateful to be given the chance to become the Hearts manager as it was my dream job," he declared. "It was an honour and privilege to serve the club in this capacity and I thoroughly enjoyed working with the players, who have been top professionals.
"It is unfortunate that things didn't turn out the way I wanted them to and I am gutted about that. I am a worker and want to get back into football as quickly as possible.
"I don't think I could have done any more but the league table doesn't lie and I know it is a results-driven business. We had a lot of difficult circumstances to deal with and they have taken their toll in the end but no-one will be more delighted than me if the team can go on and win the League Cup.
"I would like to thank all the backroom staff - the physios, medical staff and video analysis department - also the club's media department and everyone at Tynecastle for being good work-mates."