The Americans improved to 2-4-1 under the new coach following a first half that saw the team score three goals.
Edson Buddle, Clint Dempsey and Jozy Altidore scored in the first half for the Americans, who had more goals in the opening 43 minutes than they did in the first six games since Klinsmann took over for Bob Bradley last summer.
Tim Matavz scored on both sides of halftime for Slovenia.
Klinsmann is known for his attacking style of play, but it had looked like the U.S. had taken a step back with the new coach until the team's strong offensive performance against Slovenia following a 1-0 loss to France four days earlier.
"It looked much better," Klinsmann said. "It's a process, and that process, besides results, is going really well."
Playing with a 4-4-2 formation for the first time in seven games, the U.S. pushed forward from kickoff and knocked in its first goal in the ninth minute.
The U.S. took advantage of a turnover from Slovenia midfielder Darijan Matic, who lost the ball after a challenge from Dempsey. Buddle, making a surprising start, was right there to gain possession and fire a strike from outside the box and off the left post past Slovenia goalkeeper Samir Handanovic.
Moments later, the U.S. once again applied pressure to Slovenia's back four and nearly came up with its second goal of the day. Fabian Johnson delivered a perfect pass across the six-yard box to the feet of Jozy Altidore but somehow the striker put the ball over the crossbar.
But it would be Slovenia to strike next.
In the 25th minute, Matavz was able to beat the offside trap to receive a pass from Zlatan Ljubijankic with a clear path to the U.S. net. With only Howard to beat, the forward drilled a right-footed shot to the left to tie the game at one apiece.
After going back and forth on the foggy pitch, the U.S. took the lead again off a corner kick when Michael Bradley found Dempsey for a header. And it was only two minutes later when Altidore would record his first goal under Klinsmann on a penalty kick.
Johnson, who was making his debut in the starting lineup, was taken down in the box by defender Miso Brecko. Altidore was the man to step up for the spot kick and went left while Handanovic went right.
Slovenia became the offensive team in the opening minutes of the second half and had several scoring opportunities. It had missed an easy tap-in at the goalline and hit the post within a two-minute span before Matavz put in his second goal from inside the penalty area in the 61st minute.
The U.S. responded by putting in an extra midfielder, replacing Buddle with Maurice Edu and leaving Altidore as the lone striker for the final minutes of the games.
Not only did the Americans pick up their second win with Klinsmann, they also had some revenge to a disappointing 2-2 draw against Slovenia in a memorable encounter in last year's World Cup when Edu had a potential winner ruled out in controversial circumstances.
The U.S. brought back eight players from that World Cup match for Tuesday's friendly, including seven starters. Among the returning players was captain Carlos Bocanegra, who made his 100th appearance on the international level in Tuesday's win.
"It was a special night," Klinsmann said. "The team was pumped up all day."