The Three Lions had seemingly snatched defeat from the jaws of victory in Group F of World Cup qualifying, as two brilliant late Leigh Griffiths free-kicks cancelled out Alex Oxlade-Chamberlain's opener.
But, with the clock winding down, Kane – captain in the absence of the omitted Wayne Rooney – met Raheem Sterling's cross to clinch a point at Hampden Park.
And Southgate was disappointed England only emerged with a point in a game he felt they controlled.
"It's a game we should win, a game we were in total control of," the England boss told ITV.
"It was two moments of brilliant quality to turn the match on its head in seconds. But we should never be beaten. We should have that mentality and we showed that at the end.
"We had good moments, we got into good positions in the final third. We've got to take the finish. We played a team with four or five at the back. I understand why they did that, but we had decent positions and didn't take the chance."
Southgate refused to blame his side for conceding the second late free-kick from which Scotland scored, and is happy group leaders England still have qualification for Russia in their own hands – although Slovakia will close with two points by beating Lithuania, while Slovenia are three points back.
"The first free-kick is careless, the second is a slip so I'm not sure there's a lot we could have done about that," he added.
"We came to a hostile atmosphere and controlled game. We should have been further ahead, but if you don't get the second goal you can be punished.
"Our objective is to qualify and we're still top of the group."