The Cape Coast-based side have picked just one win after six games played and sit one place above the foot of the log with five points.
"I'm packing my belongings and handing over notes to leave Cape Coast because I know the club will soon ask me to step aside looking at the results we've picked after six games," Hayford told Oyerepa FM.
"I have never been sacked by a club since I always part ways mutually whenever things are not going right and the tradition will continue if necessary," he said.
Hayford, however, believes referees have had a major role in his recent troubles following a series of bad decisions against his team.
"I believe the referees are mostly behind the troubles of coaches in Ghana. They have been given too much power on the field, including the power to kill you since some of their decisions can bring about hypertension or lead one to commit suicide," he said.
"I always complain about some decisions because they affect my progress immensely. Several wrong calls have aided opponents to win games against us which shouldn't have happened like that," the outspoken coach added.
Dwarfs lost 2-1 to Asante Kotoko on Wednesday as they finished the game with nine men, after Albert Gaisie was shown red card while Paul Abeka suffered a broken leg.