Sexual assault is a terrible, awful thing, and I don't blame you for wanting to put it behind you. But there are residual emotions that are disrupting your life, therefore I strongly suggest you see a counselor, like Zed suggested.
I discussed this in another question recently, but I was once friends with a man who was (borderline) sexually abusive. Like you, I'm still terrified by the idea of telling my family. They know that he wasn't a good friend to me, but they have no idea just how messed up the relationship was. My mom is my best friend, and I tell her everything, but not that. I'm afraid.
I'm afraid that people will judge me for it, even though I know that's a warped perspective. My family has always considered me a strong and strong-willed person, and I don't want to shatter that belief. I want to seem strong, even if I once wasn't. I'm slowly trying to build the courage to tell my mother the extent of everything.
Of course I can't pretend to understand how painful your situation must be. I'm so, so sorry that it happened to you and that you still have to deal with the repercussions. But I'm hoping that maybe my process of trying to get over it could be helpful to you. Though again, speaking with a certified counselor will probably be much more effective than my emotional rambling.
First off, please, please know that any feelings of guilt you have are entirely unfounded. It doesn't matter what you did or where you went. His actions were unjustified and inexcusable in every way. You are not responsible, and you should not have to feel responsible.
I'm not at a point in my recovery where I can tell you how to overcome guilt. I'm not there. I sometimes still feel like everything is my fault. The decisions I made facilitated the fact that he took advantage of me, and it's hard not to blame myself for that. But logically I know that I shouldn't feel that way. I have to actively tell myself that it's not my fault.
Second, I've actually found comfort in sharing my experience. I would never just bring it up in an everyday conversation, and in fact I've never brought up the details with anyone. But I have one friend in particular whom I trust completely, and talking to him about my feelings made me feel a lot better. It's helpful to know I'm not alone, and to know that people will still love me in spite of everything.
Before I found the courage to talk about it, however discreetly, it felt like the experience was defining me. Because of it, I was terrified of making friends. I was afraid of being emotionally open, therefore I closed myself off completely. It felt like my whole life was being consumed by memories of that friendship, and I was miserable because despite my best efforts, it took a long time to get over the man who abused me.
But talking about it allowed me to make deep friendships, and those friends helped remind me that not everyone is a selfish, arrogant a**. Even the recent discussions of it here on the Board have felt oddly therapeutic, because it's a low-pressure way to admit my mistakes and relive myself of painful secrets.
In short, I think talking about it with someone you trust could help you confront your feelings and hopefully move forward.
If you want to keep it a secret from friends and family, I'm here. You can email me if you want to, and it can remain completely anonymous and confidential. But please, don't feel like this is something you have to deal with alone. You have friends and family out there who love you and want you to be healed and happy. There are counselors out there who are trained to help you recover from the experience. And personally, if you want to form a posse and beat the crap out of this guy, I will happily join you.