I have a different perspective on this: I worked as a pastor of a small church while my kids were growing up. Fortunately, the church was a loving church, and my kids were not subjected to some of the things that you have faced, but there were still expectations placed on them that would not be placed on other kids, and I know that was a challenge to the kids.
There is one time though that my daughter brings up from time to time. I was serving as an associate pastor at a church, and was speaking from the pulpit when one of the ladies in the church started to verbally assault me from the congregation. She criticized me, and my parenting abilities right in front of the church, and especially in front of my kids. It really affected my daughter. So being a pastor's kid is hard, and living in fishbowl is really hard, and the church needs to be reminded that the pastor's kids are still just kids, and will do what kids do.
For the record: My kids are all adults. They have never had a drug problem, an alcohol problem. They are all active in their church, and involved in ministry. Two of the three are married and have children of their own, and are doing well. The third is not yet married, but is working as a 911 operator and using the extra money she makes to travel and enjoy herself.