His father, George, was a Post Office worker and his mother, Anne, was a bookkeeper in her father's hat company. His maternal grandparents lived downstairs and his grandmother proved to be remarkably influential in Goldberg's life: 'Anything that happened to me that's been good really goes back to Brooklyn and...my grandmother's apartment,' he once said.