Frank Lampard gave the Premier League side the lead but a goal from Vicente was followed by another from Ashley Barnes to give the hosts the advantage. New signing Andrew Crofts completed the turnaround for Gus Poyet's men, leaving Di Matteo visibly frustrated on the sidelines.
The loss was Chelsea's third in row, following on from defeats to both AC Milan and an MLS All-stars side in their final two games of their tour of the United States.
"I was expecting more from my team in terms of urgency and movement," the Italian told the club's official website. "I was looking for a better performance than the one we gave today.
"We were too stretched, never aggressive enough and didn't get the ball back quickly enough when we lost it. With the ball, there wasn't enough movement and there were a lot of things we didn't do. We have to be a bit more focused and sharper."Di Matteo pointed out that the club's new signings still need time to find their feet whilst noting some of the players involved in Euro 2012 had only been back with the squad for a couple of weeks.
"We haven't had the whole team together yet," he added. "Some players have only been back nine days, some 16, some 21 and some are not back at all, but when they are back we can all work together.
"Hazard played on the left, we want him to be incisive, he had good spells but we need to be patient, he's a young man coming from a different league and it takes time to adapt.
"I was looking to get some players 90 minutes so I didn't want to make many changes, seven played the whole game and we left some players back at Cobham to train."
Chelsea take on Manchester City at Villa Park in the Community Shield next Sunday.