From your post, it sounds like he's trying to rescue you, which could be his issues coming in to play, my x wife use to do me the same way as your treating your man, you might come to realize you are not good for your husband and your husband is not good for you, keep your recovery family close, he's probably exhausted from this situation, and constantly dealing with the absence of intimacy or support. How have you been supportive of him? Concerning trust, like me I don't believe you ever had a environment to develop trust, it's something foreign to you, you have no reference or frame of mind for trust, and dysfunctional abusive relationships are the only one's your mind are familiar with, so you naturally gravitate towards recreating what's familiar, rather than unlearning and re-learning new interactive skills and behaviors with other's I'm surprised he's not left already, he has alot more patients than I have. It's not that he's giving up on you, it's just his life, and there's other things he desires to experience about life besides constantly dealing with your issues and personal problems, that he can't fix, is not responsible to fix nor desires to fix, and actually him leaving might be the best thing for your life.
From your post, I feel like your giving to much persona identity to this one relationship, so if the relationship vanishes, so does your identity, big mistake to give yourself identity with any one relationship, you've become invisible through the time you've been in this relationship, you've probably subconsciously surrendered alot of your personal powers over to this relationship, that in the end, will end. Perhaps you'll look back someday in time, and ask yourself, why did I put myself in such a warped and twisted situation? Why wasn't I strong enough to stand up for myself and demand the best out of life. Write down a fear inventory for yourself, your scared of abandonment, but just imagine your life as a book, each day, is a blank page, it's up to you, to write the book, and some chapters of your life have to end, but now there's a new chapter, make it a interesting one. It sounds like you and him are not really good for each other, maybe some space and time in the past you were, but your going one direction and he's going another, which is ok, it doesn't mean he's abandoning you, it means, it's ending, and something new is beginning, the fear of the unknown, fear of abandonment, fear of loss, is what your left with dealing with. It's like your life has become consumed with dealing with this one relationship on a constant bases, besides that, what else is going on in your life? There's 7 billion people living on this planet, surely your not surrending all your power over to one of those humans.
David