Padraig Amond's first-half header deservedly put Newport ahead and they went close to hanging on, but they were ultimately denied by Kane eight minutes from time to earn Spurs another bite of the cherry in a replay.
Spurs showed precious little evidence of occupying a division three rungs higher in the English football pyramid, particularly in the first period when Newport started with great intent and looked far more threatening.
The visitors found themselves trailing seven minutes before the break, as Amond headed in a teasing right-wing delivery, sending the home fans into raptures.
Spurs improved after the break, though a bad Kane miss just before the hour mark saw Mauricio Pochettino's men waste a rare opportunity.
4 - Harry Kane has scored in each of his last four games for Tottenham in all competitions, his best scoring streak since November 2016. Equaliser.— OptaJoe (@OptaJoe) January 27, 2018
Spurs' pressure towards the end became increasingly intense and, although Newport often found themselves with 11 men behind the ball, their desire failed to trump the visitors' desperation, as Kane tapped in to give the away side the opportunity to do it all over again at Wembley.
For all the quality in their starting XI, Spurs looked rattled early on and could count themselves lucky not to be behind in the fourth minute.
Joss Labadie did well on the right flank and squeezed a low delivery into the penalty area which fell to the unmarked Frank Nouble, but the former West Ham forward inexplicably fired over from 12 yards.
Further chances fell to the home side, as Scot Bennett sliced wide a few minutes later and Kieran Trippier sent a glancing header into the side-netting of his own goal.
Spurs appeared to get to grips with Newport as the half progressed and Kane forced Joe Day into a smart stop in the 32nd minute.
38’ NEWPORT COUNTY LEAD TOTTENHAM HOTSPUR! What a goal! Padraig Amond has headed in at the back post and the Exiles are 1-0 up! Unbelievable! pic.twitter.com/3sSX8eSCFk— NCAFCLive (@NCAFCLive) January 27, 2018
But Newport took advantage of slack defending soon after to take a deserved lead – Amond heading in at the back post after Trippier failed to deal with Robbie Willmott's cross.
Spurs looked more focused after the restart and they should have levelled in the 58th minute, but Kane headed over from inside the six-yard box.
That ushered in a period of real dominance for Spurs, who withdrew the ineffective Fernando Llorente for Dele Alli with just over 20 minutes to go.
And he nearly made an immediate impact, feeding Son Hueng-min into the left side of the area and he tested Day.
But Day was beaten in the 82nd minute as Newport's defences were finally breached.
Son's corner delivery went all the way through to the back post where Kane was left unmarked to tap in one of the simplest finishes he is every likely to produce, saving Spurs from an embarrassing defeat.
Key Opta stats:
- Spurs failed to beat a fourth-tier side in an FA Cup match for the first time since January 1996, when they drew away at Hereford United.
- Newport avoided defeat against Spurs for the first time in their history, having previously lost all four meetings.
- Harry Kane has now scored 30 goals in all competitions this season, more than any other Premier League player.
- Kane has scored in four consecutive games for Spurs in all competitions, his best run since November 2016.
- Padraig Amond scored his first FA Cup goal since November 2015, when he scored a brace against St Albans City for Grimsby Town.
- Robbie Willmott has assisted seven goals for Newport in all competitions this season, more than any other Exiles player.