Recently I have read about segregated special education school buildings being incorrectly described as safe places for students with developmental disabilities that benefit families and communities, when in fact they were places of limited opportunity, prejudice, and danger.
I am familiar with practices in segregated schools with students on IEPS only. Students were bussed from all neighborhoods to one building far from neighborhood peers and siblings. No one in the neighborhood knew the family from school events or classroom volunteering, scouts, sports, or parks department programs. Neighborhood children and families did not experience the true natural diversity of their community. SpEd students did not experience the sense of community and natural accommodations that were taken for granted by their mainstream peers. Incidental opportunities were rare in segregated schools and the highest expectations were lower than low expectations in inclusive schools.
There were no creative options or accidental / unplanned opportunities. The community was unaware of the diversity and individuality of students in the segregated school. The community's understanding of childhood disability was creepy.
Prejudice against students of color, families in poverty, and cultural differences was as prevalent as neighborhood schools with even less protection from discrimination and harm. The community did not visit segregated schools, and for many families, distance and resistance to participation provided few opportunities to observe or encourage learning.

We lived in a neighborhood that had a new elementary school when children with developmental disabilities were first "allowed" to go to public schools. Instead of disabled students going to their neighborhood schools, the new neighborhood school was assigned for all students with IEPs in the district.
Many years later, a school on the outskirts of the district was designated and students 3-21 attended classrooms in that building.
The neighborhood school was designated an alternative high school. Eventually, the school on the outskirts was remodeled extensively and became a kindergarten center for three overcrowded elementary schools with SpEd preschool classrooms in one wing.
For the first time, it was painted in fresh, bright colors instead of institutional drab, plumbing was repaired, and a beautiful playground was built. Neighborhood families were brought into what was then a neighborhood school, and school board members visited. The SpEd preschoolers were described as developmentally delayed instead of "severely disabled" even though they were the same children.
One beautiful summer afternoon, I was walking home from the community garden near the river that ran through that part of town with my son. He was in an inclusive classroom with neighborhood mainstream peers and his sister. We met an elderly neighbor who had lived there since she was a newly married young woman.
They were tearing down the old brick building down the street from her home, and planned to build a new elementary just a few blocks away. I told her that I wished it had been built in time for my children to attend.
The old woman grabbed my wrist and squeezed it until her sharp fingernails left a mark. She said that the old brick building was brand new when her children were young, and the whole neighborhood was happy to have a place they could call their own, a center for that neighborhood where everyone knew one another and their children could walk to school.
But the district gave it to disabled kids. She did not use the word disabled, and had horrible adjectives describing the mean and inappropriate terms she did use, describing how foreign and worthless the children were who took their school away from the neighborhood.
It was like happening upon satan at the hellmouth. My young son with DS was right there with me. She looked at him with such hate that I had to jump between them. But she was finished and walking away.
Of course, the children and teens with disabilities were not to blame, and the practice was more damaging for them than our community. But she held on to that hate into her late old age.
So there are many issues and hopefully unintended consequences from segregated schools.