Roberto Mancini's men reduced the points gap at the top of the Premier League to 12 points at Old Trafford but, with just seven games remaining, are unlikely to overturn the deficit entirely.
However, Kompany described the win as "massive" for the visitors as they look toward a probable fight to regain their title in the next campaign.
"A derby is always a special game and for us to be able to win this game is massive. Our fans are still celebrating – it’ll be a great night for us," he told Sky Sports.
"Every game you have to play to win and I do believe that the way we finish this season is going to have an impact on next season and it’s always going to start from the beginning again. So this is a good game to have under our belt and we’ll take it into next season."
The Belgian also hailed City's workrate off the ball as the reason behind their victory over Sir Alex Ferguson's side.
"We all worked very hard. They played very offensively," he added. "They have an unbelievable strike force so we had to close the gaps.
"We had to be tight to them and we did that very well – I don’t think we gave a lot chances away and if you look at our overall play I think we were the better team. So I’m really happy with it."
Kompany's teammate James Milner, meanwhile, insisted his side was not more determined due to the distance between the teams in the table.
"Not really. Regardless of where we are in the table, a derby is a derby and you massively want to beat your rivals and that’s the case for every game," he said.
"It was massively important, we knew that. We came here and worked hard and got a good win.
"They’ve got such a great record at Old Trafford. They’re very tough to break down and they score a lot of goals here. We’re a bit disappointed to concede one from a set piece as I thought we defended really well."