Bielsa had won each of his four games in charge of Leeds in all competitions but his team had to battle for a point at the Liberty Stadium, with Kemar Roofe and Pablo Hernandez on target.
In an entertaining game between two unbeaten sides, McBurnie gave Swansea the lead in the 24th minute by striding on to a Bersant Celina pass to poke a finish through goalkeeper Bailey Peacock-Farrell's grasp and complete a brilliant counter-attacking move.
That led Bielsa to send on Lewis Baker to replace Kalvin Phillips and Leeds levelled before the break, Roofe tucking home his fourth league goal of the season after brilliant work from teenage full-back Jamie Shackleton on his second Championship appearance.
Swansea's lead was restored by McBurnie in the 51st minute, the striker powering home a fabulous header from a Martin Olsson cross after an excellent run down the left wing by the Sweden international.
But Leeds refused to be beaten, Hernandez coming back to haunt his old club with a fine first-time finish from Patrick Bamford's low centre, the substitute striker cleverly tricking his way into space on the left flank to create the opportunity for his team-mate.
Lampard's first home win in charge of Derby County, meanwhile, was set up by a 20-yard drive from Joe Ledley, who came on as a first-half substitute for the injured George Evans.
A corner was cleared to the Wales international and his strike flew into Bartosz Bialkowski's top-left corner after taking a deflection off a defender.
And the points were secured in the 68th minute when Tom Lawrence fired home a free-kick that slipped beyond Bialkowski to find the net.
Matty Taylor and Andreas Weimann, who scored twice, earned Bristol City a 3-0 win at Loftus Road, giving Lee Johnson's men their first win of the season and thrusting QPR deeper into trouble.
Meanwhile, Richard Wood headed Rotherham United into a 16th-minute lead at home to Hull City, but the Tigers roared back to claim a 3-2 win.
Jackson Irvine struck twice and Fraizer Campbell was also on target for Nigel Adkins' side, who held on to claim the points despite Jamie Proctor's 75th-minute effort setting up a grandstand finish.