Croatia topped Group H at the start of this month, but a goalless draw with Azerbaijan followed by a 2-0 defeat to Norway meant they dropped to third, with Kovac accusing his side of lacking heart in the matches.
Captain Luka Modric was one of those to refute suggestions of a lack of effort and Kovac ultimately paid the price for a turbulent spell following an executive board meeting of the Croatian Football Federation (HNS) on Wednesday.
HNS president Davor Suker said at a news conference: "Based on the unanimous decision of the board, we have decided to terminate the contract with Niko Kovac and with all members of the coaching staff<
"We spoke as two friends, and when he heard the news he accepted it like a gentleman."
Having replaced Igor Stimac in charge of the national side on a contract until the end of Euro 2016 qualification, Kovac guided Croatia to last year's World Cup in Brazil, via a play-off victory over Iceland, where they exited at the group stage with one win from three matches.
A positive start to qualification for Euro 2016 gradually went downhill, with Lokomotiv Zagreb coach Ante Cacic reported as a potential replacement for the former Croatia international.
However, Cacic told Croatian media this week that, while he was "proud" to be mooted, nobody had contacted him regarding the role.The HNS posted via Twitter: "Search for a new head coach will be led by president Davor Suker and technical director Romeo Jozak. New coach is expected by 21 September."