It was more upset about a lack of supportive, caring family, rather than missing them. Then it started to grate around holiday season and meeting new people -eventually they ask about your family.
I get this. It felt awful for so long when people would ask about my family, because an honest answer wasn't exactly what they were looking for. I've started just saying, "we don't talk" and leaving it at that. If they press the issue I say "he's not a nice man" and leave them to draw their own conclusions. Although closer friends know more of what goes on all around the family.
Mostly, I find that I have very little desire to become close to anyone. I don't think this is a social fear (which I definitely have social anxiety) so much as it is a general disinterest in relationships. I wonder if I should be worried about that?