Fulham owner Shahid Khan has withdrawn his offer to buy Wembley Stadium because of the division it has caused within the game.