Yussuf Poulsen slotted home on 59 minutes after being put through on goal by Tottenham midfielder Christian Eriksen, but Denmark rode their luck at times against Ricardo Gareca's attack-minded team.
Christian Cueva missed a penalty after he was tripped by Poulsen at the end of a first half in which Peru created the better chances, and Hareide was quick to credit the South Americans after their first World Cup game in 36 years.
"Peru are an outstanding team," the Norwegian told reporters. "They are a good team, and we knew that. They have a quick style of play, and we know and have a lot of respect for them.
"They got very little and probably deserved more – sometimes that happens.
"We were on the lucky side."
Hareide was left with injury concerns over two players at the end of a physical game in which his players had to defend hard for long periods.
Denmark midfielder Kvist was carried off on a stretcher after 36 minutes following a blow to the ribs and defender Andreas Christensen was taken off nine minutes from full-time.
When asked about Kvist's condition, Hareide said: "We think it's a fractured rib – we'll have confirmation later on.
"If it's a fractured rib he's out for the rest of the World Cup for sure. We cannot replace a player at this point in the tournament so we'll be down to 22 if he is out."
"Christensen had a bit of a cramp so we couldn't take any chances with him," he added. "Fortunately we had other good defenders."