One reason I tend to avoid family-friendly places is because parents often don't know how to control their children, or they simply refuse to do so. You have a child, which means you signed up to handle life and all it has in addition to your child and all they have. Death in the family? Well, now you have a death in the family and diaper changes. Car accident? Car accident and kid that wants to put gum in his sister's hair. You don't get a special pass because you're having a stressful day.
That having been said, if your kid happens to be screaming his head off and it bothers me, I do have the option to move most of the time. As I said in my previous post, I can move to the other side of the store, ask to be re-seated at a restaurant, go out for a smoke, put in my headphones, or what have you. 99.9% of the time, that is my option. Where your freedom stops is when it infringes on my right to eat/shop/whatever in peace. More often than not, this happens at a restaurant. I'll be seated in my booth, and a family will sit down behind me. Then, the restless child gets up and starts pulling at my bags or my coat.
"Oh, let's not do that, okay? That doesn't belong to you." I'll say this in a way that's appropriate for the child's age. A smile and sing-song voice with a toddler. A more stern tone with a 5 or 6-year old.
By then, the parent should take over. If they don't and the child continues, then I will move my items away from the child and turn to the parent. "Excuse me, but (s)he's grabbing my things and disrupting me." Nine times out of ten, the parent will call them back. Usually they tell me the kid is small and can't be controlled. Oddly enough, I have yet to meet a parent of a child with special needs who won't keep tabs on their kids. It's the parents of normal but disruptive brats who are most often irresponsible.
"I can't control him."
"Well then, I'll have to." At that point I will turn back to the child and--in a more severe tone--point back to their family. "You need to sit down."
If that doesn't do anything or if the parent wants to jump on me for trying to do their job, then I'll get restaurant management involved. The waiter/waitress and manager don't want to deal with the kid and his family any more than I do, but they have the authority to tell the family to leave. If I'm told to leave, I will do so, and I will not be back. The family has no right to infringe upon my right to peace, and I will do everything within my ethics and the law to defend that.
Some parents complain that they can't control their children in such places. The solution, then, is to keep children out of those places. Hire a babysitter. Get a friend to babysit as a favor. If you can't afford a babysitter or can't find one, you get to stay home to deal with your kid. It's called responsibility. If your kid is disruptive, it is your responsibility to calm the kid down. Parents of children with special needs seem to understand this. I have several friends whose children are autistic and prone to outbursts. Before the kid has even gotten loud enough to be a disruption, the parent is handling it. Why parents of "normal" kids don't seem to always understand that, I don't know. There are certainly many, many parents who do; I'm not saying it's a problem with all of them.