The former goalkeeper would like to see young Americans playing regularly and isn't convinced that would happen if the rumored deal goes through.
MORE: Biggest summer transfers | Soccer's beautiful fans | WAGs
Keller's ESPN colleague, Taylor Twellman, tweeted on Wednesday that the English club is now leading the race for the signature of the 21-year-old, who was one of the breakout stars of the World Cup for the United States. But the 44-year-old, who played four seasons with the North London side, was reluctant to fully endorse the move away from MLS.
"Spurs is a tricky place to play," Keller told Goal USA at Major League Soccer's Castrol Media Cup. "They're a team that has a lot of turnover in players.
"Spurs are a team that has the money, and they are always looking for a bargain, but it's a tough place to go. I just want to see young American players get to situations where they're going to play regularly, week in and week out. If that's at Tottenham, great. But if it's not, then it's not the right move.
"It's hard to develop if you don't play."
The former Spurs and Sounders shot-stopper also said that, given his lack of national team involvement, Yedlin might not even be eligible to play for Spurs.
"I'm not sure DeAndre would even qualify for a work permit," Keller said. "He's never started a game for the national team, (England) is a tough place to get a work permit. My guess would be that if Tottenham is in the race, it would be to sign DeAndre and loan him out somewhere."