Nicky Clark scored a last-minute winner as Rangers booked their place in the Scottish Cup quarter-finals with a 2-1 victory over Kilmarnock at Rugby Park.
Martyn Waghorn netted his 28th goal of the season on Tuesday night with an early spot-kick to put Rangers ahead in the fifth-round replay, but had to limp off because of injury soon after.
Killie quickly got back on level terms through Rory McKenzie but the Gers continued to dominate, eventually getting the goal they so badly craved late on to set up a quarter-final clash with the winner of next week's tie involving Dundee and Dumbarton.
Gers had lost to top-flight opposition St Johnstone in a League Cup tie back in October and were held by the Rugby Park side at Ibrox two weeks ago.
The Ayrshiremen have since appointed Lee Clark as Gary Locke's successor and he was presented to the home support before kick-off.
But the former Huddersfield, Birmingham and Blackpool boss decided to let former Gers skipper Lee McCulloch take charge of the side for the final time as he took the opportunity to assess his new squad from the stand.
The fifth-round replay was part two of Rangers' three-match run on plastic, but it was the hosts who slipped up inside the first two minutes as Stevie Smith got on the wrong side of Waghorn before tripping the Gers top scorer's heel.
Waghorn showed no sign of nerves after five games without a goal, though, as he sent Jamie MacDonald the wrong way from the spot.
But it was his final involvement after suffering an injury in the initial incident and he was substituted by Clark.
As if that was not bad enough a blow for Warburton's team, McKenzie then cancelled out their opener after eight minutes as he cut in from the right before slotting a perfect left-footed strike past Wes Foderingham.
It was an impressive response by Killie as they harried after Rangers high up the pitch.
Their hunger almost paid off again when Josh Magennis robbed Rob Kiernan, who came close to conceding a penalty himself when he scrambled back to barge the Killie striker over before he could shoot.
Rangers were awarded a free-kick inside the hosts' box after MacDonald bizarrely picked up Conrad Balatoni's back-pass, but it went unpunished as the Championship leaders failed to read James Tavernier's ball across goal.
Barrie McKay's swerving volley was another near miss 10 minutes after the interval, while MacDonald looked solid as he pushed away Tavernier's shot from range.
Another save from MacDonald denied Jason Holt's long-range effort, but Rangers were gradually beginning to dominate. A goal-saving block from Balatoni on substitute Harry Forrester frustrated them again, however.
But with extra-time looming large, they got the breakthrough as Clark darted to the front post to fire the 90th-minute winner from McKay's corner.