Dear Sad Astronomer,
I'm going to level with you; pursuing your dream of Astronomy will probably not be easy. I considered myself to be "good at math and science" in high school, and right now my plain old Physics major is chewing me to bits, preparing to spit what's left of me out mid-December, at which point I will crawl away and try to find solace in Star Wars Episode VII (okay, it may not be quite that bad, but it's bad). The math and physics classes required are rigorous and demanding and will likely take a concentrated effort to overcome.
Now, after having read that harrowing paragraph, completely forget that I said it.
See, here's the thing: you have something I don't have, a secret weapon that could cause you to succeed where I seem to be failing. That secret weapon is desire. You're interested in Astronomy and want to pursue it. If that's what you want, if that's what's really going to make you happy, then there is nothing that can stop you. When times get hard, hold onto that desire, that dream, and push forward. Will it be easy? Probably not. Will you graduate in four years? Maybe, but it could very easily take longer. In pursuit of your goal, you may have to rearrange other goals or priorities and make some sacrifices. But, if all of that ends up in you doing what you love, it is absolutely worth it.
Following your dream is not hopeless. If you have one, chase it.
(I guess I feel so strongly about this because I've never really known what I wanted to do, and that fact has frustrated me for a while. Physics seemed like a good fit at the start, but as time goes on it seems decidedly less so. When struggling through my hard classes, I find myself thinking it would be so much easier, psychologically speaking, if I had some goal or career I was working toward instead of just plunging blindly into harder and harder classes, hoping to find some aspect that sticks and I enjoy. But that's another story...)
Hopefully you're sufficiently inspired, so now let's talk plans/strategy. Here is a very useful flowchart, showing all of the courses you'll need to take in order to graduate with a B.S. in Astronomy. If you're looking to get your feet wet, I'd recommend taking at least Math 112 (Calculus 1) and Physics 127 (Descriptive Astronomy). Getting back into math might be hard at first, but give it some time, and spend as much time in the Math lab as you need. Physics 127 shouldn't be too math-heavy, so don't worry about not knowing enough math to be able to understand that course.
As you can probably tell from the flowchart, after taking those it would probably be wise to take Math 113 (Calculus 2), Physics 121 (Mechanics), and Physics 191 (a class that surveys the different research options here at BYU). You'll need some concepts from Math 112 in Physics 121, which is why I recommend it here, and it'll be good to keep going with the math side of things so you don't risk forgetting what you've learned. 191 will get you a little more familiar with what astronomers do and will also help you get in contact with professors who can further advise you (if you're feeling really good about things, you could also take CS 142, which is the Intro to Programming class. It's not super related to what you'll be studying at this point, but programming is a skill that will help you later on).
Now, this part is important: I wouldn't fully commit to Astronomy until you've at least taken all of the courses heretofore described. I know there's a lot of pressure to graduate as fast as you can, especially if you're on scholarship, but use these first few semesters at BYU to explore what you like. Don't just grind through your generals; use them to explore different areas of study that you might not have considered before. It'll help you be more sure of yourself when the time comes to finalize your major.
Good luck to you, and never stop dreaming.