Lisle Carter was born on 18 November 1925 in New York City, New York, USA. He was married to Jane Livingston and Emily Ellis. He died on 10 September 2009 in Warrenton, Virginia, USA.
His mother, Eunice Carter, was an assistant district attorney for then-New York District Attorney Thomas E. Dewey and put together the case that led to the conviction of mobster Lucky Luciano.