England bowling coach David Saker has revealed how tough it was to pass up the chance to take over at Warwickshire.