The visual look of the video was inspired by the work of Brazilian photographer Sebastião Salgado on diamond mining.
The band used the video to debut their new glam rock wardrobes, notably Billy Corgan's black shirt with the word "Zero" written in silver, and silver pants.
The video marks the last filmed appearance of Billy Corgan prior to his decision to shave his head.
When asked why the band chose Bullet for the first video, Corgan responded "the record company did a survey of K-Mart shoppers between 30 and 40 and this is the song they came up with". "This is the blue light special", said Chamberlin. Corgan originally wanted the song "Jellybelly" to be the first single.
The song had its origins during the recording of 1993's Siamese Dream. According to Billy Corgan, "Somewhere, I have a tape of us from 1993 endlessly playing the 'world is a vampire' part over and over." It was not until 1995 that Corgan finished the song with the noted chorus "rat in a cage," on an acoustic guitar during the same session that the cover of Fleetwood Mac's "Landslide" was recorded.