Cap & Gown Club was founded in 1891 by a group of undergraduates from the Class of 1893 who had formed an eating society called the "Oliver Twist Club." With some help from the godfather of so many of the clubs, Moses Taylor Pyne, Class of 1877, the group was reborn as Cap & Gown. Under the direction of Thomas O. Speir, Class of 1887, the new club constructed a building for itself on the south side of Prospect Avenue, across from the University Field, on the lot that Cap still occupies. (Cap & Gown thus holds the distinction of being the only one of Princeton's eating clubs to stay in the same geographic location for its entire existence.)

Speir's design called for a modest cottage with a gambrel roof and a wide front porch. It was completed in March 1892, but within three years, Cap & Gown had outgrown it and sought to build a larger structure. The Speir building was moved across the street to a location near the eastern corner of Olden Street and Prospect Avenue, approximately where the Mudd Library stands today.

The smallest purpose-built clubhouse at Princeton, this building played a key role in the development of the club system. In its Olden Street location, this building was the famous "Incubator," so called because it served as the birthplace to a series of emerging clubs, including Cannon, Campus, Charter, Terrace, and Tower. Other clubs temporarily occupied the Incubator while their own clubhouses were under construction or renovation.