The former Liverpool defender has paid the price after a dreadful run of just three wins in 14 Bundesliga games for the Champions League hopeful.
The Bundesliga club has endured a woeful run of form, which has seen the team pick up just three wins from the last 14 games, culminating in another defeat on Friday.
The result leaves the club in increasing danger of missing out on Champions League qualification. Both Wolfsburg and Borussia Monchengladbach can leapfrog Leverkusen in the table if they win their games in hand this weekend.
Reports have suggested youth coach Sascha Lewandowski will now take charge of the team until the end of the season, having previously worked with the first team last term.
Hyypia took charge of Leverkusen in May 2012 but has been heavily criticized since the turn of the year, with the club suffering a 6-1 aggregate defeat to Paris Saint-Germain in the last 16 of the Champions League.
The 40-year-old's contract was due to expire in the summer of 2015.