Morocco international Feghouli opened the scoring for Galatasaray with his strike in the ninth minute after he was set up by Nigeria winger Henry Onyekuru.
The lead was short-lived by Vedat Muriqi's equaliser on the stroke of half-time.
After the restart, Umar gave the Rizespor the lead in the 76th minute before late goals from Diagne turned the game at the Yeni Rize Sehir Stadium around.
The Senegal international scored two goals within five minutes of stoppage time to stretch his lead at the summit of the top-scoring chart to 30 goals after 28 games.
The victory fired Galatasaray to the top of the Turkish Super Lig table with 66 points from 32 games, three points above their closest rival Istanbul Basaksehir who have a game in hand.
Rizespor dropped to the ninth spot after gathering 40 points from the same numbers of games.
Next Sunday, Galatasaray will take on Istanbul Basaksehir for their next league game while Rizespor visit Alanyaspor.