Scott Carson saved from Phil Jones to earn the Championship side a place in the fourth round, progressing 8-7 at the end of a high-class shoot-out that followed a dramatic 2-2 draw at Old Trafford.
Lampard, in his first season as a manager, triumphed against former boss Jose Mourinho despite Marouane Fellaini's late headed equaliser.
And Lampard could hardly believe Derby, who trailed thanks to Juan Mata's third-minute goal, were able to dump United out.
"I'm shell-shocked. What a performance," Lampard said to Sky Sports.
"To go down early at Old Trafford and play as well as they did was superb. We had the personality to take penalties and play the way we did. I am a proud manager.
"It was the easiest selection I have had, the players were superb against Brentford [in Saturday's 3-1 Championship win] and they deserved the chance to play at Old Trafford.
"I wanted to bring a team who could really compete. I am so proud of them. I am trying to build a group to be successful and nights like this are incredible. To go against world-class players and play that way, it's right up there."
Derby came from behind to lead thanks to a brilliant long-range free-kick from Liverpool loanee Harry Wilson, with Jack Marriott nodding the Rams ahead.
And it was another loanee, Chelsea's Mason Mount, who was singled out for praise by the former Blues star.
"Harry Wilson does do that in training, he proved me wrong as I thought it was too far out," Lampard added. "Both he and Mason Mount are going to go a long way."