Slavery is certainly at the core of racism in the U.S. but I think that the overt manifestation of racism became firmly entrenched as a part of American culture in the post-civil war era with the implementation of Jim Crow laws (racial segregation laws enacted between 1876 and 1965 in the United States at the state and local level). At least a credible argument can be made that slavery had roots in the economic infrastructure of the South. Jim Crow was just plain meanness, a legalized system of hate and disenfranchisement. Racial hatred based purely on skin color. (Examples of Jim Crow Laws by state)
For a brief period following the Civil War, there was an effort to educate former slaves and their children. For a brief period, there was an effort to offer some level of reparations via land and housing. Former slaves were even elected to political office but after nursing its hurt feelings and bruised ego, the South released its wrath against anyone of African descent and enacted laws to take away what little advances had been achieved and Jim Crow became more powerful and widespread than slavery had ever been. Owning slaves had been the luxury of the landowners, the landed gentry. Anyone with white skin could be superior to the new underclass of blacks.
The implementation of Jim Crow is the something rotten in the U.S. It's what no one wants to acknowledge, that racism isn't some remnant left over from slavery; racism was created and nurtured to ensure that black people remained only a step above chattel, no longer bought and sold but still deemed inferior to even the poorest of whites. Jim Crow cast us as the underclass, and denied us all access to "life, liberty, and the pursuit of happiness."
The dialogue that needs to begin is about the post slavery era of Jim Crow. There are still plenty of us around who lived under Jim Crow laws and also plenty of white Americans who benefitted from Jim Crow laws. Invariably when there are discussions about racism in mixed company, someone white will question why we (black people) keep talking about slavery. The commentary goes something like this, "It's long been over and done with and no one is still alive who owned slaves or was a slave."
True, and I have no desire to talk about slavery. What I want to talk about is my childhood, my adolescence, my young adulthood and the laws that restricted where I played, went to school, went out to eat, went to the hospital, received medical care, where and how I traveled, where I sat in the movie theater (assuming it admitted me at all) and every other aspect of my life and the lives of all the black people that I knew. It's a lengthy and long overdue conversation and this country still hasn't engaged in it.