A statement on the club's official website reads: "Stefano Pioli is the new Palermo coach. He has signed a two-year contract with president Maurizio Zamparini's club and there is an option for the third."
Pioli was favourite for the role prior to Rossi's departure.
The former coach severed ties with the Rosanero after holding talks with Zamparini on Wednesday. Their agreement was severed by mutual consent despite the club qualifying for the Europa League.
Rossi had returned to Palermo in April after being sacked in February, but speculation was rife of a less than perfect relationship with his boss.
The incoming coach was previously linked with Roma after holding recent talks in the capital when he left Chievo at the end of the season.
However, last week Pioli warned the capital-based club that he would not wait around for them as he had other offers.
He is expected to be unveiled to the media on Thursday afternoon in Sicily.