The Ghana international found the back of the net with an absolute beauty, set up an equaliser, before delivering a free-kick which ultimately resulted in the match winner at Wanda Metropolitano.
Saturday's game was Partey 10th league appearance of the season, the goal his second and the assist also his second.
Interestingly, it was the away side who took the lead on the day. Spain striker of Ghanaian descent Inaki Williams, who is currently being courted by the Black Stars for a switch of international allegiance, opened the scoring with a simple goal-line tap-in in the 36th minute.
Partey then hit back for Atletico in the 61st minute, sending a thunderbolt from 26 yards into the net to tie the game at 1-1.
But Williams was not done just yet. Three minutes after the equaliser, the 24-year-old put his side back in front with a cool finish on the run.
Ten minutes to full-time, Partey showed up once again for Atletico, this time, delivering a corner which found the head of Rodrigo Hernandez and then into the net for another equaliser.
It looked like the spoils were going to be shared at Wanda Metropolitano before the story sensationally changed in the first minute of injury time.
Partey whipped a long free-kick into the box, Saul Niguez flicked it towards the face of goal and captain Diego Godin was on hand to head into the net from close range.
There was a momentary scare for Atletico as the goal was ruled out for offside before being reinstated after a video review. 3-2 it ended in dramatic fashion.
The win has taken the Rojiblancos to second on the league table, while Bilbao sit just one place above the relegation zone on the 17th position.
Partey's next assignment is international duty as he has been called up for Ghana's 2019 Africa Cup of Nations qualifier against Ethiopia on November 18