Liverpool 0 Crystal Palace 2
The Eagles were the heavy underdogs prior to kick-off and could have been behind early on had Michael Owen and Sami Hyypia done better with their respective chances.
The Reds continued to dominate their first division opponents but simply could not finish any of their well-worked opportunities.
Palace, to their credit, held their shape and were rewarded in the second half when Julian Gray lost his marker and buried a superb left-footed drive past Jerzy Dudek.
The signs did not look good for the visitors when substitute Dougie Freedman was shown a straight red card for an elbow on Hyypia.
Despite being down a man Palace wrapped up the win when the impressive Gray broke away and his cross was turned into his own net by Stephane Henchoz.
Palace's reward for a sensational Cup win at Anfield is a clash against debt-ridden Leeds United at Selhurst Park in the fifth round.