Milan looked on course for a well-deserved draw after having the better chances in the match, however Costa struck his fifth goal in four European games this season to seal victory.
The Serie A side had the better of the first half with excellent opportunities to open the scoring but lacked quality in front of goal as Andrea Poli and Kaka both missed good chances, while visiting goalkeeper Thibaut Courtois was in inspired form.
Atleti improved after the interval, but chances for either side were few and far between, with Christian Abbiati only called into action once in the whole game by the visitors until Costa powered a header beyond him in the closing stages.
The result gives Atleti a crucial away goal going into the second leg at the Vicente Calderon in three weeks' time.
Diego Simeone's side attacked from the off, though Abbiati was untested and Milan slowly worked its way back into the game with good spells of possession.
The team created a number of good opportunities to break the deadlock but suffered from a lack of accuracy in its finishing and found Courtois in fine form when it did get the ball on target.
The Belgian goalkeeper diverted Kaka’s effort onto the crossbar in the 15th minute and three minutes later he produced an outstanding stop to tip Poli’s close-range header onto the post.
Kaka went close again for the hosts after latching onto a back heel from Mario Balotelli, but the Brazilian curled his effort just over the bar.
Clarence Seedorf will have been impressed with his side's performance, but the hosts failed to capitalize, Balotelli missing the last chance of the first half as his long-range effort bent wide of goal.
With the words of Simeone no doubt ringing in their ears, Atleti came out for the second half with renewed energy, but clear-cut chances were still not forthcoming.
After a quiet opening period, Costa saw an attempted acrobatic volley fly way over the bar, and an effort from Raul Garcia went the same way with the midfielder's low effort smothered by Abbiati as chances of a goal looked to be running out.
However, with seven minutes to play, the veteran goalkeeper was unable to get anywhere near a header from Costa, with the Brazilian pulling away from his marker to give the visitors victory.
Adil Rami went close to leveling for Milan with a powerful long-range effort, but it flew wide of Courtois' right-hand post, leaving Milan with a big hill to climb Spain on March 11.