Conte, out of work since leaving Premier League side Chelsea at the end of the 2017-18 campaign, has been tipped to take over from Luciano Spalletti at San Siro following the conclusion of the season.
Chiellini was a key part of those championship-winning teams and he is uncomfortable with the possibility of Conte coaching one of Juve's major rivals.
"I would hate to see him there, but it's part of life," the defender told Tuttosport.
"I understand that they are professionals, but to see him at Inter would make me [feel] weird.
"It is not a criticism, everyone knows that with Antonio I have a special relationship.
"What can I say, I hope everything goes well for him, but nothing can win at Inter.
"It is the truth, so much so that when he coached Chelsea, I rooted for him. At Inter, I can't do it, really."
Conte won Serie A in every one of his three years in charge of Juve, before winning the Premier League and the FA Cup at Chelsea after a stint with the Italy national team.
He made a whopping 420 appearances in all competitions for the Turin giants, scoring 44 goals from central midfield.
Inter would benefit greatly from his expertise, having allowed their grip on third place to slip and are now fourth, with their Champions League hopes in the balance ahead of the final round of fixtures.
Should results go against them, they could find themselves in the Europa League next season. They host an Empoli side fighting desperately for survival, while Atalanta and Milan play mid-table sides Sassuolo and SPAL, respectively, who have nothing to play for.