Despite taking the lead at White Hart Lane on Monday, Sunderland slumped to a 5-1 defeat against Tottenham to further dent their hopes of avoiding relegation.
|1/5||Sunderland are 1/5 with Bet365 to be relegated this season|
"Their player has got behind me for the first and second goals. We can talk all we want to about having to cut out the mistakes, but we have to," Brown told the Sunderland Echo.
"We have to do better when we're going forwards too. We're not helping ourselves.
"Unless we can produce 90 minutes where we're 100 per cent, we can only blame ourselves."
Brown insists Sunderland cannot give up hope of pulling off a great escape ahead of their final seven games of the season.
"The table doesn't lie, it's bad," he added. "But we've got to keep going. That's all you can do.
"Of course it’s difficult. You can try not to look at how bad it is, but you do.
"We've got to show a bit of pride in ourselves and keep on grinding it out."