The scene where McHale and Parker ask "What do you want to do?", "I don't know what do you want to do" is an homage to the movie Marty (1955), the film for which Ernest Borgnine won an Academy Award as Best Actor.
The title refers to one leaving a party or other activity without bidding farewell to the host.