Despite 26 points from Pascal Siakam, Toronto fell to a narrow 122-117 defeat at the hands of Portland.