Dear aren't we all,
Despite being hyper-aware of every mention of my SO now, I can't help but feel that it's applicable here, especially since you relate to me a lot and I am a very different person from my SO.
I am very extroverted, love junk food, work extremely hard in school while still being relatively undisciplined in other areas and am not always the most politically conservative person. I am not as careful as I should be with my finances. I have strong opinions and I was inactive in the church for several years. My SO, on the other hand, is the most disciplined person I know in every regard but school, is extremely frugal (I pay for most things when we go out), has never questioned the church and is very conservative politically. He is more reserved, is in way better shape than I am and has had a great family life and hasn't had a tenth of the trials I have had. We're just very, very different people, but our core values are the same, and really I think that's what matters the most.
Like you, I have not had a lot of gentleness or kindness in my life, and one reason I stuck with my SO during the first few weeks when I was freaking out about our differences and the newness of the relationship was that he was so gentle and kind with me. When it came down to it, we had some major differences, but he was a really good guy and I knew that if we wanted to make it work, we could because the core things that I looked for in a guy were all there.
With that being said, I don't think it's selfish (or even shallow) of you in the slightest to consider your differences and take those into consideration. You need to take care of yourself, first. However, I don't think that letting him into your life would necessarily mean letting go of the things that you love. If he really cares for you, and you for him, you'll both want the other to be happy and to do the things that the other enjoys. If you end up dating him and care for him in that way, you'll probably find you want to do things with him that you dislike just because it means spending more time with him and vice versa. And even if he doesn't do all of the things you love with you, you can still absolutely do them by yourself. Being in a relationship with someone doesn't mean giving up the parts that make up you. Alone time or time with friends is perfectly acceptable from time to time, even when in a serious relationship.
My SO hates the newer Les Miserables movie and I absolutely love it. He set up a special date where he watched it with me just because it meant spending more time with me and he knew that it would make me happy. I remember I was really surprised that he would do that for me and I'm still getting used to the idea that I rank so highly in his life. Like you, probably, I have never ranked highly on anyone else's list of things to care about and I'm used to fending for myself. Being in a relationship has surprised me in what he is willing to do for and with me. Similarly, I think he might surprise you, even if you think he doesn't have a love of the outdoors or of nature, as to what he's willing to do for you and with you.
I guess the bottom line is that if you want a reason to not date someone, you'll always find it. Always. You can always find a way to talk yourself out of something, but the question is why you're trying to do that. Are you afraid of the possibilities? Genuinely concerned that he won't treat you well or care for you? I don't think it's the latter because he sounds like a decent guy and as a decent guy, he's never going to ask you to give up the things that mean the most to you and make you who you are. If he likes you and the things that make you who you are, he's not going to have a problem with your love of exercise or vegetables.
And as for the stress of having super different interests, I don't think that they have to be all that stressful. In fact, I think it's kind of fun to be so different because it exposes you to a different viewpoint and set of tastes and opinions. If you're both similar, sure, it's fun that you can do lots of things together, but being different is just as rewarding. While being very extroverted, I hate dancing. Absolutely loathe it. I can do slow dances, but anything beyond that I freak out, get super self-conscious and immediately look for the nearest corner to hide in. It actually induces stress in me because I'm so terrified to look stupid. My SO is a ridiculous dancer and loves to dance. He's awful at it, but he's 100% okay with looking goofy and I thought that was going to be a huge issue, knowing how much he loves it, and how much I refuse to do it. We went to a ball recently and I was surprised (and so relieved) to discover how cool he was about it. He never once pressured me or made me feel uncomfortable. Instead, he went and hid with me in the corner, which was really sweet of him. And in return I danced as many slow songs with him as possible and made sure he knew that I was okay with him going to dance to some of the faster songs.
So like I said, it's a give and a take, but it's not even necessarily taking as much as it is giving. I'd give this guy a chance and give him an opportunity to show how willing he is to compromise with you and be introduced to the things you love. He doesn't have to love doing what you love, but you shouldn't have to give anything up, either.