England star Kane was vital in the victory, as he scored an equaliser in the 78th minute and then saw his late header deflect in off Trent Sainsbury for the winning goal.
Luuk de Jong had earlier given PSV the lead just 61 seconds into the match, but Spurs otherwise dominated and were only prevented from securing a healthier scoreline by the heroics of Jeroen Zoet in goal.
Although Spurs were frustrated for long periods, Kane felt the display was an impressive one, as Spurs left themselves three points behind second-placed Inter with two games left.
"We said we were disappointed coming away from there [Eindhoven, with a draw]," Kane told BT Sport.
"Today it wasn't ideal conceding early, but then we were dominant and created chances. We could have been more clinical, but we'll just take a win in the Champions League.
"We had to dig deep and find another level when it mattered. We have been grinding results out here and there all season.
"We are getting wins. That was one of our best performances of the season."
PSV coach Mark van Bommel was proud of his team's efforts despite the defeat, and singled out Zoet for praise.
"They fought like lions," he said of his players. "After one minute we got a goal and we tried to defend it until the end.
"We did everything, had an outstanding goalkeeper [Zoet]. Everyone was fighting for the result. Of course, Tottenham got some big chances, that's why I'm giving our keeper a big compliment."
Defeat for PSV leaves them on the brink, sitting six points behind Inter with two matches remaining.