Ogbonna headed home in the last minute of extra time, after Philippe Coutinho's driven free-kick had cancelled out Michail Antonio's first-half opener at Upton Park.
West Ham will now face Blackburn Rovers in the fifth round of the competition, and Ogbonna has spoken of his pride at netting for the first time in English football.
"It was my first goal for West Ham and in front of our supporters, so I am really happy for my teammates, for everybody. I really enjoyed it," the Italian told the club's official website.
READ MORE | Bilic revels in FA Cup triumph
"I'm still excited, and I want to enjoy this moment. It's unbelievable. It was a great ball from Dimitri.
"It's a good result for us and we have to look forward to the next step now. We'll enjoy this moment, but we will keep our feet on the ground.
"This is the fourth game we have played against Liverpool this season, and we have given them just one goal.
"We had a really good mentality, we defended together, attacked together, switched the ball and were really good on Tuesday."