The South American side fell behind in their knockout tie against the Three Lions through a second-half penalty and had six players booked over the 120 minutes before losing 4-3 in the shootout.
Colombia conceded 23 fouls as well, 10 fewer than Gareth Southgate's side in a tight encounter, but Falcao believes his team were at a disadvantage because of the United States referee.
"I found it peculiar that they put an American referee in this instance... to tell you the truth, the process leaves a lot of doubts," he told reporters.
“He only spoke English, some bias was certain. Through small calls he was pushing us [toward] our goal, that was clear for me.
"The referee disturbed us a lot, in the 50-50 plays, he always made the calls in favour of England. This situation was undermining us. He didn't act with the same criteria for both teams.
"When in doubt he always went to the England side. It's shameful that this happens in the round of 16 of a World Cup."
It took a late header from Yerry Mina to pull Colombia level and send the tie into extra-time, and Monaco star Falcao was confident of his side getting through to the quarter-finals of the competition.
And, after experiencing his first World Cup, Falcao believes his team will only get better in the next few years.
"During the 120 minutes we fought with our weapons, with our strength, we carried a plan to the end and we managed to draw," he added.
"On penalties the victory could go either way. It leaves a sour taste, because we thought we could have qualified.
"This team is very strong. The players are growing and this Colombia side will become stronger; we have to keep working."