Forget 3D books when it comes to maths. Some of them are good but most of them don't give you nearly enough knowledge on the mathematics.
What you need is a few basic books such as mathematical engineering ones along with a few dealing with manifolds and tensors. Recently I've been studying tensors and have looked at tensor surface patches. These things are vital for basic computer graphics applications.
Linear algebra is an absolute must, trig will follow through easily.
Calculus is also extremely essential, as are things like Fourier Series and Fourier Transforms for signal processing and you need to understand the analysis behind the fields that use these tools.
In fact when it comes to basic 3D stuff there is no such thing as enough mathematics.
Even a PhD in mathematics isn't enough as far as I'm concerned.
The most common sense thing to do is to learn the basics of 3D via some computer graphics book and if you enjoy it you can get stuck into the mathematics little by little.
In the end however if you want to write a paper about 3D graphics you need to be able to do things as simple as convert between spaces, for reasons beyond what I want to discuss at 4:44am (as this is taking enough time as it is), a good and very example is to take a look at Euclidean and Polar spaces along with how complex numbers are used.
A very good book:
http://www.amazon.com/Advanced-Engi...ef=sr_1_1/104-6772202-3968737?ie=UTF8&s=books
Another very good book:
http://www.amazon.com/Interactive-Computer-Graphics-Top-Down-Approach/dp/0321321375
I've read many computer graphics books and many more game programming ones, all the game programming books are rubbish at best.
Mathematics is where it is all at, if anyone tells you otherwise, they don't know enough mathematics to understand where and/or how it can be applied to applications in computer graphics.