Among the angels and others for whom morality is the deciding factor in their actions it would be reasonable to expect that all people would disapprove of him, and apologize and reconnect with you. But we live among very very fallible and broken human beings. And there will be a mixed reaction - and it is hard (at least on the other side of the computer screen) to know which way the tide will tilt at the end. I don't want you to have to move, and I don't think you should move out of fear. AND I want you to be prepared, because while "the good guys" carry a lot of the battles, they don't carry all of them. And you have a mom and baby to care for and keep safe and sane, and that might not be the best place to care for them if the tide tilts the wrong direction. (bad metaphor, sorry.)
I hope you live in a town of sensible and compassionate people, and that they "get it right" and it becomes a nurturing place for you and your family.