On February 4, 1864 at approximately 8:15 in the morning, workers in the East Armory discovered smoke coming through an attic window. The frantic pistol manufacturers ran  hoses up to the attic and tried to extinguish the blaze. The fire exploded from a drying room used and ignited leather pulley belts that drove machinery in the floors. This, combined with the complex’s empty water reserve caused the workers to flee the building. By 9:00 the fire had spread to the roof and quickly engulfed the iconic onion dome in flames. Soon after, the dome burst through the roof of the burning Armory building. The speed of the fire was accelerated by oils and chemicals used in the manufacture of pistols. 

The fire spread across a bridge connecting the East Armory to the company’s office. The powerhouse and several other small buildings were lost. Factory workers and the volunteer Hartford Fire Department in preventing the fire from spreading to the West Armory. At the onset of the fire, workers threw paperwork out of the office windows, saving a majority of the records. Workers also saved much of the machinery and tooling inside the building, carrying out as much as possible before the second floor collapsed. A factory worker, E. K. Fox, died in the collapse while trying to move equipment from the burning building. 

Although the fire was contained by noon, it was not fully extinguished for two days. What was once the basement of the East Armory was full of melted metal from the machines not saved from the blaze. More than a thousand  machines were lost and the complex’s primary power sources, two 300- and 400-horsepower engines, and the main boiler were destroyed. At the time damages were estimated at more than $2 million dollars (adjusted for inflation, this is an estimated $35 million in 2021), but insurance only covered $600,000. Faced with this knowledge, Elizabeth Colt chose to rebuild. The new building was modeled after the original and created a blue dome similar to the first. The new building was five stories, larger than the original three story building.