When I show my ray tracer project to people, by far and away the most common response is "I wish I knew enough mathematics to do this." This is the exact same thought that I had when I started. I had to learn as I went. For a lot of the project, I was relearning mathematics that I had forgotten, then I was learning new mathematics. It has been fun and rewarding, and I want to encourage everyone to do it, because the big secret to mathematics is that it is fun.
There's never been a better time to learn. The internet is stuffed full of mathematics learning sites. OK, it's actually stuffed full of cat pictures. But there is a lot of maths online. You can visit the Kahn Academy, look things up on Eric Weisstrams world of maths, check the wolfram site, ask questions on the xkcd or many other maths forums, read books, textbooks and historical papers... Even ten years ago, this information wasn't available.
Maths is a subject that you can dip into when you have time. You can do a little bit every day, or dive in and immerse yourself for a whole day. It's the cheapest hobby in existence, and the one that can have the most surprising outcomes.