The boxer in the bar is Floyd Patterson. At the time Patterson was the heavyweight champion. He would lose the title to Sonny Liston a month later in September of 1962.
The address Freddy Rumsen gives the cab driver when they leave the underground casino is 152 Riverside Drive. The same address as Joe Fox (played by Tom Hanks) in You've Got Mail.
When Don tells Roger that punching Jimmy Barrett "...was a real Archibald Whitman maneuver" and that he was "a hothead drunk I used to know", Don is referring to his biological father.
As this episode revolves around the death of Marilyn Monroe, it's only appropriate that it ends with her singing "I'm Through With Love".