Last season's Champions League finalists named a changed side at the JobServe Community Stadium on Tuesday, but still included regulars like Eric Dier, Davinson Sanchez, Ben Davies, Lucas Moura and Dele Alli, among others.
But Spurs were unable to score on their fourth-tier opposition over 90 minutes, with the match entering a penalty shootout tied at 0-0.
In the shootout, substitute Christian Eriksen and Moura both failed to convert from the spot, with Colchester defeating their Premier League opposition 4-3 on penalty kicks.
Remarkably, Tottenham are not the first Premier League side that Colchester have knocked out of this year's Carabao Cup.
Colchester also defeated Crystal Palace in the second round last month, prevailing 5-4 on penalties after again holding their opponents scoreless over 90 minutes.
Colchester are now the first lower league team to progress from two penalty shootouts against Premier League sides within the same League Cup campaign since Bradford in 2012-13.
That season the Bantams went all the way to the final, where they were defeated 5-0 by Swansea City.
Tuesday also marked the first time Tottenham have been eliminated by a side from a lower division in the League Cup since 2005-06 against Grimsby.
The elimination continued a rough patch of form for Tottenham, who have now gone three games without a win in all competitions.
That run includes a 2-2 draw at Olimpiacos in the Champions League last week, with Spurs having thrown away a two-goal advantage in the first half.
Tottenham also dropped points from a winning position at the weekend, losing 2-1 to Leicester City after taking a first-half lead through Harry Kane.
That result dropped Spurs down to seventh in the Premier League table, with Mauricio Pochettino's side having won two, lost two and drawn two of their opening six league matches.
Spurs will look to turn things around on Saturday when they host Southampton in Premier League action.
Pochettino's men will then take on Bayern Munich in Champions League play three days later as they look to once again reach the competition's final this season.