There are 3rd party libraries in any game project. But you might be surprised at how many in-house solutions and algorithms you must use in any given project. Game developer is too vague to really understand what it is one might be doing in such a project. Most of the 'engine' is most likely a 3rd party system like Unreal 3, Gamebryo, Unity, Source, etc. But more and more studios are creating in-house proprietary systems and for reasons that are beyond the scope of this post.
Basically if you are a gameplay programmer you might not touch the nasty stuff or you might touch nastier stuff when implementing AI and the like. If you are a graphics programmer you will certainly encounter some nasty math but most is made quite simple b/c of vector and matrix classes. The math involved is complex but not all that hard. One matrix multiply is 16 separate multiplies but all that is hidden inside of matrix classes.
That being said if you wish to get into game programming you will most likely start in scripting and UI and then move from there. I believe math is extremely important but you could theoretically get in without tons of math although if you wish to move up or laterally to the engine and graphics area you will definitely need it.