1. Vertex normal generation

I am currently working on a fractal based game engine. This engine generates a fractal world (yes it's actually roundish) from a set of functions and refines it and unrefines it in real time as you run across the surface. Currently I have the world generation, collision detection, and camera working so I can move around the surface. However I haven't started shading it yet. I have face normals and I more or less know how to generate vertex normal's from them. I intend to use procedural shading.

However there is a problem. Sometimes the fractal algorithm generates sharp peaks and stuff like that. I want to be able to set a threshold so that sharp peaks (i.e. drastic changes in normals from one face to the next) are not smooth shaded so they look realistic. I know how to handle this in DirectX but I need a good algorithm to figure out how many normals are needed per vertex and to which triangles they apply. Does anyone know of a good algorithm that will smooth shade up to a point but allows for non smooth shading for sharp corners? I have a couple possible ideas I but I wanted to find out if there was some standard way to handle this.

2. Originally Posted by SyntaxError
I am currently working on a fractal based game engine. This engine generates a fractal world (yes it's actually roundish) from a set of functions and refines it and unrefines it in real time as you run across the surface.
Yer a nut!

Anyway, if this provides any perspective, in openGL "smooth" vs. "sharp" or curved vs. faceted surfaces are dealt with by using additional normals for each vertex of a polygon/quadratic surface, vs. using only one normal for each flat segment.

3. Originally Posted by MK27
Yer a nut!
I won't argue with that However, now that I have all the bugs out of the collision detector (which took me an eon) it actually works quite well and it gives me some big advantages over a normal game engine. a) I can make my game world as large as I wan't. b) I don't have to store terrain data anywhere. c) I can generate a whole new world to play on in a matter of seconds.

Several other people are working on the same thing so there must be a few other nuts out there. When someone finally builds a viable engine my feeling is it's going to take over MMORPGs. Think of being able to put all players on a single shard (ala EVE online) however with full terrain and everything.

Originally Posted by MK27
Anyway, if this provides any perspective, in openGL "smooth" vs. "sharp" or curved vs. faceted surfaces are dealt with by using additional normals for each vertex of a polygon/quadratic surface, vs. using only one normal for each flat segment.
Actually that part I know. It's just I'm not sure who to determine what faces get the version of a vertex with a particular normal. For instance if you have a vertex surrounded by 6 triangles you could have 1 normal, or 6 different normals or 2 different normals or various other combinations depending on the triangles.

4. Originally Posted by SyntaxError
I won't argue with that However, now that I have all the bugs out of the collision detector (which took me an eon) it actually works quite well and it gives me some big advantages over a normal game engine. a) I can make my game world as large as I wan't. b) I don't have to store terrain data anywhere. c) I can generate a whole new world to play on in a matter of seconds.
I meant nut in a good way. Hooray! Vis, the thing with OGL and "normals for quadratics", I just got myself introduced to it yesterday (which is why it was stuck in my brain), so I can't comment any further. But I think there are a couple people who could who might be around here sooner or later...ITMT I would try 2 different normals before I would try 6 :x

5. Well I think I figured something out on my own. You can just walk the faces surrounding the vertex in a loop, sum up the normals and divide by the number of faces. The caveat is wherever the angle of normals between two consecutive faces are greater than your threshold, you assign your current normal average to the vertex in question of the faces you have already traversed, then clear your sum and count and keep going. Hopefully this will let you shade ridgelines correctly while keeping smooth shading where you want it.