A 3-0 win at Kenyatta Stadium ensured the Rangers picked up the points to move to sixth on the KPL log.
A goal from William Odera in the 51st minute gave the Mailmen the lead before Joseph Mbugi stretched their advantage with a 75th-minute penalty before Elvis Osok added the third four minutes later.
With two wins, two draws and a loss, Sammy Omollo's side looks an improved outfit, especially after battling relegation all through last season.
They kept their league status after a successful play-off against Nairobi Stima and their next match will be away to Zoo FC at Kericho Green Stadium on October 20.
Meanwhile, Kisumu All-Stars' struggles will persist into week six of the KPL season as they have managed to pick up just one point in a draw against KCB.
Their four losses mean they sit deep into the relegation zone at position 17. They will next face unbeaten Mathare United on October 19 before concluding the month with a tie against another struggling side - Sony Sugar.