The Illinois State Penitentiary opened in 1833 with 33 cells. It expanded to 256 cells over the years, including a hospital and other supporting buildings. The penitentiary closed in July, 1860 but was reopened 2 years later after the Union victory at Fort Donelson, Tennessee which captured many confederate prisoners of war. The first confederates came to the prison in February, 1863, and numbered 1,640 men. The numerous prisoners, lack of adequate space, and poor living conditions created an environment rife with misery and illness. Latrines were placed too close to drinking water; some cells held so many men they could not lay down or unfold their knees. Over 11,000 men passed through the prison over the course of only three years.
The same year the prisoners were brought to Alton, cases of smallpox began to circulate throughout the prison. The virulent disease, sometimes taking up to 17 days to incubate in the body, quickly ravaged the prison population with high fever, rashes, and lesions and two hastily constructed hospitals were created on islands in the middle of the Mississippi River to quarantine the ill- Sunflower Island (later called Smallpox Island) and Tow Island. Both islands are mostly underwater today and have been heavily changed by the river landscape and flooding.

An estimated 240 confederate soldiers and an unknown number of Union guards died and were buried on Tow Island. Hundreds more died in the prison from battlefield injuries, infection, elemental exposure, and other illnesses and were interred at a burial ground north of Alton- the same plot state penitentiary prisoners were buried in before its closing. Wooden stakes were erected as markers but fell into disrepair over time and individual grave identifiers were lost. The Commission for Marking Graves of Confederate Dead tried to identify the remains of the soldiers but ultimately failed.

In 1908-1909 an obelisk, towering at 58 feet, was erected made of rusticated granite with a stepped base and concrete plinth. 1,354 confederate soldier's names are engraved on four sides of the obelisk including those that died in both the prison and on the islands.

Both the cemetery and the prison are purportedly haunted locations. A rumor spread that bricks from the native limestone walls of the prison were used in the construction of Alton homes and businesses but it has since been proved a myth- according to records, almost all of the bricks were broken upon demolition of the prison in the decades after its closure on July, 7, 1865, and thrown into the river. All that remains of the prison today is the outer corner of one wall, a testament to both history and the Civil War.