In regards to understanding why your body reacted the way it did, <added by staff: Dead Link Removed>, an article you may find helpful. The physiological fear reaction you experienced is a well recognised phenomenon, and it's something you had no control over.
There is absolutely no doubt that this is a case of criminal sexual assault. You did everything right. You made it clear before you went to his flat that you did not want to have sex with him, and he assured you that he believed in an affirmative yes, and would never do anything you didn't want to do. Despite that, he ignored your physical unresponsiveness, he ignored you when you said you didn't want to do anything, and he responded to your shaking and crying by telling you that you wanted it and asked for it.
You say you never said no, but actually you did: you had a very clear mutual understanding before you went back to his flat, and when he went further than you were comfortable with, you told him that you didn't want to do anything, and he ignored you.
You could not have physically said or done more, because of your body's physiological fear reaction. And he ignored your physical reactions too - even to the extent of telling you you wanted it and asked for it while you were shaking and crying.
Looking at the facts logically and objectively, how can this be anything other than a clear, unambiguous case of sexual assault? To be clear, I believe there are grey areas when it comes to sexual assault. But I see no grey areas in this case. Like I said, you did everything right.
If this man is still harassing you, or contacting you in anyway, then the only way for you to be completely safe is to make a report to the police (even if you don't intend to press charges), and apply for a restraining order.
And even if you don't want to press charges, it's very important the police know about this, as you're probably not the only woman he's done this to, and you probably won't be the last.
If you can find the strength and courage to do this, then you will be able to feel safe, and you will have the satisfaction of knowing you've done what is within your power to protect other women from this man.
My heart goes out to you. I hope you are able to recognise that you are in absolutely no way to blame for what happened. As I keep on saying, you did everything right: you were extremely clear about your own boundaries, he pretended that he respected your boundaries, and then proceeded to treat you like an object once he had you inside his flat, and wilfully ignored every single clear signal that you did not want this. Like I said, this is an absolutely unambiguous case of criminal sexual assault.