It is the first time two non-league sides have reached that stage since the foundation of the Football League.
Skipper Jamie Collins' 53rd minute penalty was enough to give his side a deserved 1-0 victory over the Championship club and secure their place in the last 16.
It comes a day after almost 10,000 fans roared National League leaders Lincoln City to a stunning 3-1 victory over Brighton and Hove Albion.
2 - For the first time since the foundation of the Football League, there will be two non-league clubs in the FA Cup 5th round. Historic.— OptaJoe (@OptaJoe) January 29, 2017
The draw for the fifth round takes place on Monday night when supporters of both clubs will find out who they’ll play on the weekend of February 18 and 19.