Scott Sinclair scored his first goal for Aston Villa in a 2-1 win over Leicester to send them through to the quarter-finals of the FA Cup.
Leandro Bacuna's fine effort in the 68th minute opened the scoring for the home side after a goalless first half, with new boss Tim Sherwood having been in the Villa changing room at the break.
Sinclair added the second a minute before full-time after Christian Benteke chipped the ball over the City defence and Mark Schwarzer fumbled.
But as Villa thought they were through with the score at 2-0, up popped Andrej Kramaric in the 90th minute to head past Shay Given.
Leicester’s late rally unnerved Villa but they held firm and secured their spot in Monday's draw for the FA Cup quarter-finals.
First-team coach Scott Marshall took charge of Villa with new full-time manager Sherwood watching on from the stands following Paul Lambert's dismissal on Wednesday.
With Benteke restored to the starting line-up, Villa had the first chance after six minutes when Alan Hutton sent the ball into the box from the right wing, but Andreas Weimann’s shot flew over the bar.
Leicester’s first chance came from Villa’s poor concentration, after Ciaran Clark headed Paul Konchesky’s cross out. Matty James sent the resulting corner to the near post and Marcin Wasilewski headed it onto the outside of the post.
Another Villa mistake in the 24th minute nearly ended in embarrassment for Given, who received a backpass from Fabian Delph and played it straight to Riyad Mahrez inside the six-yard box. Fortunately for Given, the Leicester man could not get the ball under control and conceded a goal kick.
Given redeemed himself in the 30th minute with a superb one-handed save to deny a goalbound effort from James following good play on the edge of the box from Esteban Cambiasso and Jeff Schlupp.
Benteke had the ball in the net minutes before half-time as Bacuna and Weimann played a tidy one-two, but the striker was flagged offside.
The goalless first half prompted Sherwood to visit to the dressing room at half-time and Villa came out looking like they wanted to impress their new boss.
Fabian Delph had a chance within a minute of the restart, but he sent it over the bar. Then seven minutes later, Hutton's cross set up Benteke but Mark Schwarzer cleared with his feet.
Villa took control of the game and opened the scoring in the 68th minute. Ron Vlaar did well to hold the ball up and played in Bacuna on the edge of the box, who made his way towards the keeper and unleashed a curling shot beyond the despairing dive of Schwarzer.
Sinclair replaced Bacuna and he doubled Villa's lead in the 89th minute after Benteke chipped a perfect ball over the Leicester defence and the on-loan winger shot from inside the box. Schwarzer should have held on to the effort, but he fumbled the ball and it went over the line.
Leicester got a consolation in the first minute of injury time as Schlupp's cross found Kramaric for the striker to head past Given and set up a nervous conclusion.
But Villa hung on to the delight of their new manager, who will officially take charge of the team for the first time in next weekend's Premier League clash with Stoke.