Teenager Patrick Cutrone was the late hero at San Siro on Thursday, netting the injury-time winner after visiting Rijeka had overturned a 2-0 deficit in the final 10 minutes.
Goals from Boadu Acosty and Josip Elez levelled matters for Rijeka, who had appeared set to steal an unlikely point.
And Montella, while understandably disappointed to see his side squander the two-goal lead earned by efforts from Andre Silva and Mateo Musacchio, was delighted with Milan's spirit.
"I want to underline the spirit of the team that brought the victory home," he told Sky Sport Italia. "There was indescribable joy to see the team play with this energy and determination in the final minute.
"It's a great result, maybe not so much in terms of the scoreline, but it is a great step forward."
Montella's positive outlook, though, was offset by a slight frustration at Milan's game management.
"We've got to grow in the way we handle games," he added. "This was a valuable learning experience
"You have to control a match at the 70th minute. We must improve our defensive movements, as we have perhaps struggled a little in terms of temperament under the weight of expectations.
"I see the glass half-full. We learned some valuable lessons and we won."