Crystal Palace stunned Manchester United with a 2-1 win at Old Trafford thanks to Patrick van Aanholt's injury-time winner.
Daniel James appeared to have earned United a point with an 89th-minute equaliser.
But David de Gea couldn't keep out Van Aanholt's drive as Palace claimed their first win of the new Premier League season.
United had fallen behind in the first half to Jordan Ayew's breakaway goal before Marcus Rashford missed a penalty on a deeply frustrating afternoon for Ole Gunnar Solskjaer's team.
- How to watch Premier League highlights on Sky
- The Premier League table
- Premier League reports and highlights - matchday three
Despite dominating possession, United rarely threatened Palace's goal and the full-time whistle was greeted by a chorus of boos from those remaining inside Old Trafford.
Palace's win was their first ever against United in the Premier League era.
United fail to make their possession count
United ruled the ball in the opening stages and twice came close to scoring when Rashford's free-kick flew just over the bar and James' shot was blocked almost on the line by Van Aanholt.
But the visitors gradually settled into the game and undid the United defence in straightforward fashion when Jeffrey Schlupp out-jumped Victor Lindelof to head Vicente Guaita's clearance into the path of Ayew.
Clean through on goal, and with Harry Maguire conspicuous by his absence, the forward calmly finished past a startled De Gea.
Where had United's defence gone? There was a subplot to tell. Moments before, Luke Shaw had pulled up with an apparent hamstring injury only to then return to the fray. Had Maguire felt compelled to shuffle over to the left of United's defence to offer extra protection against an apparent weakness? Only he will know, but the distance between Maguire and Lindelof was a chasm and Shaw made his belated exit before the restart.
"It was the first time they were in our half," rued Solskjaer. But if the United boss was unimpressed at the interval, his frustration was only just beginning.
United pay the penalty again
After the furore in midweek when Paul Pogba's missed penalty cost United victory at Wolves, there was a sense of inevitability around the award of their third spot-kick in as many matches this term when Scott McTominay was bundled over.
But there was an immediate twist in the expected tale when Rashford, having converted on the opening weekend against Chelsea, struck the inside of the post with his kick.
United could and perhaps should have had a second penalty later on when Rashford tumbled over in the box. While the referee and VAR didn't detect an offence, the United boss saw matters differently. "It was nailed on," Solskjaer later reflected.
But United weren't to be denied for much longer. James, a lively presence throughout despite an alleged dive drawing another caution, curled a fine finish into the far post after Pogba had robbed Wilfried Zaha.
That looked to have set up a grandstand finish, only for roles to be reversed when Pogba lost the ball on the halfway line, Zaha drove deep into United territory and then slipped the ball back to Van Aanholt for the most unlikely winner of the season so far.
What the managers said
Crystal Palace boss Roy Hodgson: "It was a heroic victory. The players worked very hard. The penalty was unfortunate for us. It was strong for us to come back and win the game. The players can take a lot of credit. At one-nil with five minutes to go, I thought we would hold on.
"If you judge games on possession, then people will say we didn't deserve it. But when you win the game, it's very foolish of any manager to say we didn't deserve it. Discipline and desire is also part of a football match."
Manchester United boss Ole Gunnar Solskjaer: "We just weren't clinical enough. They scored two very easy goals. Two games, two missed penalties. If you score them, it's a different scenario. Sometimes it happens. The more we get into those positions, the more we will score.
"We dominated the game but we didn't control it. We should have dealt better with the last five minutes.
"We created enough to win the game. But we didn't defend well enough."
Man of the match - Patrick van Aanholt
Deserved the award just for the spirit of adventure behind his match-winning goal. A left-back supporting an injury-time attack at Old Trafford when the scores are level? That deserves special praise.
But alongside the returning Gary Cahill, Van Aanholt was impressively resolute against the effervescent James and his last-ditch block to deny the winger a first-half opener was to prove critical.
- Manchester United have won one of their last nine matches across all competitions (D3 L5), scoring just eight goals while conceding 16 in this stretch.
- Manchester United are only the second team to win a penalty in each of their first three matches to a Premier League season, after Chelsea in 2012-13. However, the Blues scored all three of theirs while the Red Devils have missed two.
- Dan James is the seventh Manchester United player to score in both of his first two Premier League appearances at Old Trafford for the club, and the first since Zlatan Ibrahimovic in September 2016.
Both Palace and Manchester United return to Premier League action on Saturday. United will travel to Southampton while Palace will host Aston Villa in a 3pm kick off.