Roberto Mancini's men reduced the points gap at the top of the Premier League to 12 points at Old Trafford but, with just seven games remaining, are unlikely to overturn the deficit entirely.
However, Kompany described the win as "massive" for the visitors as they look toward a probable fight to regain their title in the next campaign.
"A derby is always a special game and for us to be able to win this game is massive. Our fans are still celebrating – it’ll be a great night for us," he told Sky Sports.
|12/1||Roberto Mancini is 12/1 to be the next Premier League manager to leave his post with BetVictor|
The Belgian also hailed City's workrate off the ball as the reason behind their victory over Sir Alex Ferguson's side.
"We all worked very hard. They played very offensively," he added. "They have an unbelievable strike force so we had to close the gaps.
"We had to be tight to them and we did that very well – I don’t think we gave a lot chances away and if you look at our overall play I think we were the better team. So I’m really happy with it."
Kompany's team-mate James Milner, meanwhile, insisted his side were not more determined due to the distance between the teams in the table.
"Not really. Regardless of where we are in the table, a derby is a derby and you massively want to beat your rivals and that’s the case for every game," he said.
"It was massively important, we knew that. We came here and worked hard and got a good win.
"They’ve got such a great record at Old Trafford. They’re very tough to break down and they score a lot of goals here. We’re a bit disappointed to concede one from a set piece as I thought we defended really well."