The manager admitted to feeling exhausted after handing in his resignation on Wednesday and his former assistant believes the Eagles boss worked himself too hard
Holloway resigned from his post on Wednesday after earning just three points from eight games following promotion to the Premier League, citing exhaustion as one reason for his departure.
When asked whether he noticed his boss was suffering from tiredness, assistant manager Millen admitted: "I did yes but it is not just recently.
"When you look back at the last three or four years he has been in the play-offs and getting promoted and hasn't really had a summer off. When we won promotion this year he was straight in to the recruitment of new players and was pretty tired when we came back from pre-season.
"When you don't get results it does take its toll and I think the frustration of not quite rejuvenating the group was quite hard for him to accept."
Having worked alongside Holloway at both Blackpool and Palace, Millen admits it will be strange carrying on without him at Selhurst Park but is determined to inspire a positive reaction from the players.
“It was difficult because Ian is a good friend of mine and he always will be and I love working with him," he continued.
"Unfortunately I've been in this situation before and I know the day after it has happened it is almost a transitional period and you have to explain to the players what has happened.
"We had a meeting with the players and I explained to them what the chairman had told me and the reason for Ian leaving. Everyone at the club is hurting because everyone enjoys working with him.
"We have got to get the group united better because if you haven't got the best players you need a good team spirit and get them working for each other.
"There is enough quality in that squad to get enough points to stay up. It is a fresh start, I have got no personal issues with the lads."