The beaten team's captain acknowledged that his opposite number proved adept at drawing fouls during the Sky Blues' triumph at Allianz Stadium
The Italian veteran struck what proved to be a match-winning first-half penalty for Sydney, who claimed a 3-2 win over Victory despite playing over an hour with 10 men.
Del Piero slowed up play throughout, drawing 12 fouls with clever play and the help of referee Peter Green and overeager Victory defenders.
Midfielder Milligan, 28 was annoyed during the encounter, but said his side needed to back off Del Piero.
"As a player it is frustrating," Milligan said on Fox Sports.
"You want to speed it up and players are playing a bit for it. We didn't help the situation, we were aware of it, and we still jumped in on him."
Milligan lamented his team's inability to break down the 10 men of the Sky Blues, who defended resolutely.
He rued failing to take his chance in a one-on-one with Sydney goalkeeper Vedran Janjetovic when the score was locked at 2-2.
"We didn't create the chances we normally create," Milligan said.
"Last night we just weren't good enough and players didn't stick to our system. When it went to 2-2 we were very confident that we would go on with it from there.
"I had a good chance shortly after that, if I'm able to finish that it's a different story from there."
Click here to catch all the action from Friday night's A-League game LIVE