Former U.S. attorney Heffelfinger was early target for removal