Ships of the Sea Maritime Museum features nine galleries of ship models, maritime paintings, and artifacts. The vast majority of ship models were commissioned by the Museum to interpret Savannah's rich maritime history. The museum is housed within a National Historic Landmark built in 1819 for William Scarbrough, the principal owner of the Steamship Savannah (the first steamship to cross the Atlantic Ocean).

Scarbrough's architect, William Jay from Bath, England, created one of the earliest examples of domestic Greek Revival architecture in the South. Used as a public school from the 1870s–1960s, the mansion was then abandoned for a brief period but later restored by Historic Savannah Foundation in the 1970s. After another period of vacancy, Ships of the Sea Maritime Museum restored the house again in 1996–1997, building a new roof based on a documented William Jay design, adding a new rear portico and enlarging the gardens.