The Wales captain has urged his team-mates to remain focused despite having already won the Capital One Cup and reached the 40-point mark for Premier League safety
Michael Laudrup's team were aggrieved by the offside flag that denied Roland Lamah an equaliser at the Hawthorns after Romelu Lukaku and a Jonathan de Guzman own goal gave the Baggies the lead.
Williams, though, refused to blame the officials and accused the Swans of taking their foot off the pedal after Luke Moore had put them in front.
"Looking at our performance as a team we could have done better," he told reporters. "The incident at the end has gone, there's no point in talking about it.
"Some of the lads spoke to the linesman, I think he admitted it was a mistake but I'm not too sure. I've seen it on video and it's a clear goal.
"It's gone, we need to talk about our performances. When we scored, we stopped playing for some reason. We stopped attacking them, they're a good side and a team like that you have to finish off.
"That's the frustrating thing about this game. We did brilliantly last week against Newcastle. This time I felt we took our foot off the pedal and let them back into it."
Williams highlighted the need for Swansea to remain motivated for their last nine fixtures having already reached the 40-point mark for Premier League safety and won the Capital One Cup.
"As a dressing room we're frustrated," he added. "As a team we have to be better than that. We've got to continue to push ourselves, it's not about being safe.
"This was a game we could have won, we let it slip. We'll be disappointed if we don't finish in the top 10."