The Scot admits he feels the "best referee" has been selected for the Premier League title encounter with the Blues, but insists he hopes it is his team's turn for a bit of luck.
World Cup final ref Howard Webb has been placed in charge of the crunch clash and while Ferguson admits the "best referee" has been selected for the encounter, he is still wary of a mistake working against his side.
"We are getting the best referee, there is no doubt about that," Ferguson told reporters on Friday.
"But [getting a bad decision] is definitely our big fear.
"We have the players to do it all right. We just hope it's our turn for a little bit of luck."
The 69-year-old believes the challenge that awaits Chelsea at Old Trafford will be extremely difficult for Carlo Ancelotti's men, despite the Blues' recent form.
"Anybody coming to Old Trafford must know they're going to have a hard game," said the Scot. "And every team that comes to Old Trafford has a game-plan. That doesn't really change.
"What's important is how we approach the game and how we perform. Our home form has been very good. We've only dropped two points at home all season, so our record must be one of the best in Europe.
"I don't know how many points we were in front of Chelsea at any given time, but it doesn't matter. Where we are today is exactly where we are.
"It doesn't matter what happened last Sunday, four Sundays ago, four months ago or four years ago. We are where we are.
"We're three points ahead, with the same goal difference as Chelsea. We're playing them at home, with three games left - two at home and one away. That's it, that's where we are. Nothing can change it."
A victory for United would take them one step closer to a 19th title which would overtake Liverpool although Ferguson is not phased by the prospect.
"Personally, it doesn't mean anything at all," he said. "It's more for the history of the club that it is important.
"It's like with the European Cup. There is the history of Real Madrid and AC Milan and then there's another bracket - Liverpool, Ajax and Bayern Munich. We should definitely be in there as well."
The United manager took charge of the Red Devils in 1986 and admits he didn't believe he could potentially surpass Liverpool's 18 titles.
"No, I certainly didn't envisage it at that time," he said. "The aim was to match Liverpool and be successful ourselves. That was the target.
"That's why I came here - to win the title. Once we won it, the club took off.
"I couldn't envisage being here so long. It's an exceptional period of time that I've been here and you don't tend to think about the number of titles we've won.
"Getting the first was the target."
Ferguson has also confirmed that Darren Fletcher will not start against Chelsea on Sunday but has plans for the midfielder to feature in the Champions League final against Barcelona.
"He won't start on Sunday but his training will be stepped up in terms of the sharpness needed for such an important game ahead," he added.
"As far as the final's concerned, he's got almost three weeks. That's his target and hopefully if he reaches it then it will be a great boost to us because, as we all know, he's a big-game player."Follow Goal.com on to get the latest soccer news directly. Check out Goal.com's page; be part of the best soccer fan community in the world!