Hojbjerg's superb strike gave Southampton the lead at St Mary's on Monday and Danny Ings made it 2-0 to the hosts from the penalty spot after the interval.
But a Shane Duffy header gave Brighton hope and when the defender was adjudged to have been fouled by James Ward-Prowse at a corner, Glenn Murray scored a 91st-minute equaliser from the penalty spot.
Hojbjerg's goal was his second in two matches for Southampton but the Denmark international could not contain his frustration after Southampton surrendered their advantage.
"It's not good enough," Hojbjerg told Sky Sports. "2-0 up against an opponent we have to beat.
"If you want to be serious this year, you don't do it [lose a lead]. We had all the possibilities and somehow we didn’t manage to do what we had to do.
"We did good for most of the game. If you do good and go 2-0 up and let them come back it's not good enough.
"What did they have in the first 60 minutes? One long ball and a header?
"If you want to be serious and a contender you cannot let games like this go. We needed to take three points today.
"Maybe there's a hangover from last year, nervousness. Maybe we need to get that out of the system."
Mark Hughes was annoyed with the late penalty decision that went against his side, Anthony Taylor pointing to the spot after Ward-Prowse held Duffy from a corner.
"We went into the game wanting to win - we wanted back-to-back results," Hughes said to BBC Sport. "It wasn't to be, we got into a winning position and maybe didn't have the nous to make sure we kept the margin at 2-0.
"They had nothing to lose and threw people forward and got balls into our area and it's about the percentages. You just hope you can defend those percentages.
"We're disappointed with the award of the penalty. Shane Duffy is a big lad - about six foot five [inches] - and he's gone down easily under an innocuous challenge. They will look for anything they can get and the referee's bought that.
"We just have to see games out to a conclusion. We're in a position where we know we're progressing and key moments we will manage better in the future I'm sure."