The England international, who originally joined the club on a season-long loan last year, became the Hammers' record signing over the summer, but has yet to feature this term.
After impressing on loan at Upton Park last season, the 24-year-old completed a permanent switch from Liverpool in the summer for an undisclosed fee reported to be 15 million pounds.
However, a heel injury has prevented the England forward from featuring since his big-money move, with Sam Allardyce's men struggling in his absence, and Sullivan says a club of the Hammers' stature cannot afford to be without such an expensive asset.
"Had we known he would be out for this long, we would not have signed him," Sullivan told the BBC. "We are not a rich enough club to deal with that. You know any player can get injured, but we can't buy a player knowing he is going to be out for half the season. When we signed him we were assured by the medical staff that the very, very latest he would be back was Sept. 1. That would have meant he would only miss two league games.
"He is a fantastic player, don't get me wrong, but I only wouldn't have signed him because I can't have that amount of wages and that amount of transfer fee on a player who isn't going to play a minimum 20 or 30 games a season, hopefully 38 games a season."
Sullivan has no doubts over Carroll's ability and remains hopeful the forward can inspire a West Ham resurgence in the New Year.
"If you ask me the same question at the end of the season, I hope I will be able to give you a different answer and say, 'Yes, it was the right decision to sign Andy Carroll,'" he said. "On his day he is unplayable. We love him and that is why we signed him. He is desperate to come back. He is sweating blood in training.
"He has been all over Europe for treatment, [but] even now we haven't got a date that he could be back. He could be back in three weeks, he could be back in six or eight weeks."