After a 1-1 draw in Moscow, Jordan Henderson saw his spot-kick saved by David Ospina, but England came from behind thanks to misses from Mateus Uribe and Carlos Bacca.
Sweden now await in the last eight of the World Cup and Southgate believes a first World Cup shoot-out win for England will give his side an important boost.
"It's something they can look back on and draw on as an experience," Southgate said to ITV.
"We've talked about creating our own stories, our own history. They've done a little bit of that tonight, we go to a quarter-final, but I don't want to go home yet!
"I thought we played so well in the 90 minutes. We've shown incredible resilience to come back from huge disappointment at the final whistle.
"Huge credit to all of the players and every member of staff because so much effort has gone into winning the way we did tonight."
Southgate has plenty of penalty history of his own, having missed from the spot as England lost in the semi-finals of Euro 96 to Germany.
But Jordan Pickford saved from Bacca and Eric Dier tucked home the winning kick to book the Three Lions a clash with Sweden in Samara on Saturday.
"We'd talked long and hard about owning the process. We kept calm, great credit to everyone for taking everything on board," Southgate added of the shoot-out.
"We looked at individual technique, how we needed to be as a team, and the goalkeeper has been very important in that as well, of course it's a special moment, but I'm now thinking about Sweden. This was special, but I want us to go on now.
"Sweden is another team we have a poor record against. We've underestimated them for years, they're brilliant at what they do, know exactly how they play. It's going to be a real tough test."