Slavery existed both in the north and in the south of the United States. The industrialisation of the north and the expansion of demand for cotton in the south shifted the balance so that it became a regional issue, as the southern economy grew increasingly reliant on cheap labour.

The expression in the question title was a euphemistic term that white southerners used for slavery. John C. Calhoun defended the "peculiar labor" of the South in 1828 and the "peculiar domestick institution" in 1830. The term came into general use in the 1830s when the abolitionist followers of William Lloyd Garrison began to attack slavery.

Slaveholders began to refer to slavery as the "peculiar institution" to differentiate it from other examples of forced labour. They justified it as less cruel than the free labour of the North. The implicit message from the term "peculiar institution" was that slavery in the American South was different from the very harsh slave systems existing in other countries and that southern slavery had no impact on those living in northern states.

More Info: en.wikipedia.org