Thomas Muller twice gave Joachim Low's men the lead, but sloppy defending allowed the hosts back into the game on both occasions before Ilkay Gundogan eventually scored the winner after the break.
With two games reamining in Group D, Germany remain top of the table by two points but the Bayern Munich stopper was not satisfied with the team's defensive performance.
"We made mistakes from set pieces, which meant that Scotland were able to equalise twice," Neuer told the official DFB website.
"It’s annoying to concede goals, but we performed well and were the better team overall.
"We want to keep going, as we have played well in the previous two matches. We have to keep the momentum going."
Goalscorer Gundogan, meanwhile, was quick to concede Scotland did not make life easy for the world champions.
"The Scots sat very, very deep. They made it difficult for us. But we played good football and were dangerous throughout. I’m pleased to have scored again," the Dortmund midfielder added.
Muller was once more the hero for Germany and the prolific attacker was happy with his fine form in recent games, having scored eight goals in his past seven matches for his country.
"I have been clinical in the last couple of games, which is always good. We knew what kind of game to expect. We were calm on the ball, it was a case of being patient," concluded the Bayern star.