Shaun Barker and captain Martin McIntosh headed the goals for Ronnie Moore's Millers before Dean Holdsworth scored a consolation goal for the home side.
Wimbledon started quickly with Mike Pollitt forced to save a Wayne Gray volley, but Rotherham responded with John Mullin's 35-yard shot.
Rotherham's aerial ability troubled Wimbledon and after Chris Swailes headed against the post, McIntosh could only find Steve Banks with his effort.
But Rotherham's reward came when Barker gave them the lead on the stroke of half time, the defender scoring his second goal in a week from a Paul Warne cross.
Wimbledon pressed for an equaliser after the break with Adam Nowland thundering his shot against the woodwork and Nigel Reo-Coker blasting the rebound wide.
Rotherham quickly added their second. Richie Barker won a corner, Carl Robinson took it and McIntosh rose unchallenged to score.
Warne could have added a third but instead conceded the penalty to give Wimbledon a lifeline. He brought down Patrick Agyemang and Holdsworth slotted home to set up a tense finale.
