In a pulsating contest at Anfield, Dennis Bergkamp's deflected goal just past the hour mark looked as if it would prove decisive until Emile Heskey pounced in injury-time.
After withstanding early pressure from The Reds, Arsenal were ahead on nine minutes after Robert Pires slotted into an empty net following Jerzy Dudek's save from Thierry Henry.
The French livewire caused the home rearguard countless problems with his pace but was consistently thwarted in his efforts to bag a second for the visitors.
John Arne Riise restored parity on 52 minutes with a rocket into the bottom corner before Bergkamp appeared to have delivered the points to Highbury.
But in the first minute of injury-time, Salif Diao's cross picked out Heskey unmarked at the far post and the England international headed back across goal and into the corner of the net.
At White Hart Lane, Tottenham had the best of the first half and in the second, Gustavo Poyet had a goal disallowed for offside but The Magpies would not be denied.
With the match heading for a stalemate, the ever-dangerous Craig Bellamy jinked his way into the penalty area and had his shot saved by Kasey Keller.
But teenage midfielder Jermaine Jenas was on hand to nudge the ball over the line for three precious points for The Magpies.
Manchester City cemented their spot in the top half of the table with a crushing victory over Fulham.
The visitors had the temerity to take the lead through Steed Malbranque on two minutes as they sought to take their excellent cup form into the league.
But from the moment Nicolas Anelka equalised on 21 minutes after latching onto Ali Benarbia's pass, City never looked back.
Benarbia put the hosts in front just two minutes after the break and the lead was extended on 61 minutes through Marc Vivien Foe.
Substitute Shaun Wright-Phillips rubbed further salt into Fulham's wounds by lobbing the ball home for a fourth goal.
West Ham welcomed back talismanic Italian Paolo Di Canio as they went in search of a first Premiership win at Upton Park this season.
It was Blackburn, though, who grabbed the lead as Dwight Yorke rammed the ball home at the second attempt.
Di Canio, though, was the focal point for The Hammers and after he was fouled by Andy Todd, he rifled the resulting spot kick into the roof of the net.
With time running out, The Hammers finally ended their home hoodoo when substitute Jermain Defoe's drive whistled into the back of the net.
In a dour tussle at The Hawthorns, West Brom sunk to the bottom of the table as they failed to gain any respite in their battle against relegation.
Charlton continue to entrench themselves in the sanctuary of mid-table with Shaun Bartlett's 60th minute goal separating the sides.
Click below for an individual match summary:
Liverpool 2-2 Arsenal
Manchester City 4-1 Fulham
Tottenham 0-1 Newcastle
West Brom 0-1 Charlton
West Ham 2-1 Blackburn
