St Helens coach Nathan Brown admits he stayed on too long at former side Huddersfield before taking up his position with the Saints.