Sokratis Papastathopoulos and Shinji Kagawa scored second-half goals as the Austrian made a fine start after replacing the sacked Peter Bosz.
The visitors survived a couple of early scares, including Suat Serdar's shot against the crossbar, to register their first victory and clean sheet in the Bundesliga since September.
It was an encouraging show of resolve from a side which crumbled with alarming regularity in recent months and Stoger, himself dismissed by Cologne this month, hopes to provide the platform for improvement.
"My aim is simply to make the best use of the qualities of the players," he told Sky Sport Germany.
"The timeframe before the game was tight. Overall, we are very pleased.
"At half-time we discussed a few things we wanted to correct. The second half was much better, we took control and maybe could have closed it out earlier.
"We showed in phases that we can play really good football."