Lolley helped the Conference Premier outfit earn a Fourth Round tie at Premier League Sunderland when his goal secured victory at London Road.
Tommy Rowe gave Peterborough the lead at half-time before Michael Gash’s headed equaliser.
The FA Cup with Budweiser
Third Round proper
Tuesday 14 January 2014
Winners receive £67,500 from FA Prize Fund
The comeback seemed complete when Jack Byrne put the Harriers ahead, only for Britt Assombalonga to level from the penalty spot.
But Lolley caused the shock of the night with his 76th minute winner to set up a trip to the Stadium of Light.
Charlton Athletic were almost the victims of an FA Cup giant-killing before they came back from two goals down to draw 2-2 at home and earn a replay against League Two Oxford United.
John Mullins scored first for Oxford and Scott Davies doubled their lead before half-time.
Sheffield Wednesday 4-1 Macclesfield Town.
Charlton found their form in the second half and Michael Morrison headed them back into the game before Yann Kermorgant struck a cracking equaliser eight minutes from time.
Port Vale will play Brighton & Hove Albion after twice coming from behind to win their replay 3-2 against League Two Plymouth Argyle.
Sheffield Wednesday now face Rochdale in the next round after surviving a scare to defeat Conference side Macclesfield Town 4-1.
Meanwhile, Championship Watford await the winners of Manchester City and Blackburn Rovers’ replay after goals from Marco Davide Faraoni and Lewis McGugan were enough to record a 2-0 win over Bristol City at Vicarage Road.
