Printable View
This is very weird and I did not encounter this last time I coded this quad-tree frustum culled terrain.
Since this is a recursive algorithm if the function works for 1 patch it should theoretically work for all patches. How in the world am I getting some patches that just do not show up? I've written the terrain data to 5 files each separating the different types of data and I cannot find the problem.
Look at this mess:
Here is an updated screenie with the new shaders.
Still holes in the terrain. Dunno why. But now I have multiple terrain types with smooth transitions between them.
This screenshot is clamped at 100 FPS so I'm not losing any frames so far with just the rendering.
So this problem is due to the frustum culling, e.g. when you have the frustum test turned off everything is rendered properly?
Man that's odd. I don't know if its a frustum culling problem because patches before and after the hole are rendered.
The texture transitions look good.
Is it just a random tile here and there or is there any consistency to it? Have you used this algorithm before? Granted I don't know much at all about 3d/Driect X, but I'm just wondering if perhaps there's a pattern to the missing patches.
How are you doing the texture transitions? Are you loading all of your textures into a 3d texture and interpolating the texture z value when you want to transition, or is it a multi-pass kind of thing?
I turned frustum culling off and still have the same problem. There are a few areas I can think of that may be causing this issue. I'll let you know what I discover.Quote:
So this problem is due to the frustum culling, e.g. when you have the frustum test turned off everything is rendered properly?
My sentiments exactly when I ran it and saw what it was doing. And thanks I think the texture transitions look very good for a single pass shader.Quote:
Man that's odd. I don't know if its a frustum culling problem because patches before and after the hole are rendered.
The texture transitions look good.
It's not quite random and it appears as if the southeast portion of the quad tree is falling apart. However there is a serious problem with this theory. If the quadtree was broken, almost every terrain patch would reveal this since this is a recursive algo. So if it's broke for one patch it should be just as broken for the next. Also what's extremely baffling is that 4 patches in the center of the terrain are completely missing. This is a quad tree boundary so this would mean that all 4 quad tree nodes would have failed in order to produce such a render. I'm almost certain it cannot be the quad tree. It may be in the actual render code that traverses the tree and drills down to the leaf nodes.Quote:
Is it just a random tile here and there or is there any consistency to it? Have you used this algorithm before? Granted I don't know much at all about 3d/Driect X, but I'm just wondering if perhaps there's a pattern to the missing patches.
Terrain texturing and lighting is done in a one pass blending shader that does diffuse and specular lighting. It's a well known technique. I'm using a blend map to blend several different textures. Each texture uses a channel of the blend map to determine how much of it's color contributes to the final pixel. A tiling coefficient is used on the texture samplers and a non tiled coefficient is used on the blend map. This technique is in several books but is explained very well in Frank Luna's recent book. I'm only using the basic technique here and it works well for now.Quote:
How are you doing the texture transitions? Are you loading all of your textures into a 3d texture and interpolating the texture z value when you want to transition, or is it a multi-pass kind of thing?
Once I discover the source of my 'holes' I'm going to add bump-mapping to the shader. Future plans also include billboarded grass, 3D trees/rocks/plants and whatever else I can throw at this. The eventual goal is to implement this renderer as the planet renderer for my space game project so I will also need billboards on the horizon so I can show far off planets, moon, suns, etc. Hopefully I can also implement some CLOD so I can perform seamless space to planet transitions with no loading and no pausing.
The skybox is not actually a box. Instead it is a cubic environment mapped icosphere which gives pretty good results.
To narrow down problems specific to the polygons that make up the world I've implemented a debug 'point and shoot' function which tells me which polygon I'm pointing at. I have then examined the parent algorithm (in your case the recursive rendering functionality?) at the part of the function that should be handling the polygon in question. Perhaps you can do the same?
It seems the culling is indeed the problem. I messed with the min and max vectors of the AABBs for the patches and now I have some very odd results.
Here is a screenshot from way above the mesh patches. There is a clear pattern of holes and suspiciously one entire quadrant is missing. This points to a quad tree generation problem.
EDIT:
I'm an idiot. Found the problem and it was the SW corner was not generating correctly. Probably because I was passing midwz as the start z and the end z.
oh oh! I think I see a pattern!
Hehe. Yep. It's a pattern and an ugly one. Perfect example of how looking at the big picture can fix a simple issue.
Now it's working fine and the culling also is working quite well. I dumped the P and N optimization that every web site out there tells you to use. Why? Because it quite honestly does not work for all cases or at least I could not get it to work. So I'm testing all 8 corners of the AABB against the 6 planes of the frustum. The test I'm not performing is the 8 corners of the frustum against the 8 corners of the AABB.
I also have in mind some different culling algos in case this one does not yield good results. I've looked also at restrictive quad-tree's and triangle bin-trees for CLOD but have not attempted any of them yet.
Next question is now how do I generate these patches on the fly and generate the normals? I have code that will read in data from a file given a central offset and a range around the offset. The idea is to load in 'patches' in real time as the player moves.
I think the key word here is "paging", you may want to try googling for terrain paging algos. I never ventured there myself, always assumed everything could be buffered in memory.
I've found several articles on various techniques but not sure how to go about implementing them. Moving from a white paper to actual code is a bit of a leap especially when I hardly understand all of the white paper.
Hughes Hoppe page describes geomip-mapping which I've been wanting to try. I think paging data in would be easier since everything is still maintained in a regular grid. My idea is to pre-process the entire world and place it on disk in vertex format. The index buffers for the patches will never change so I could store it once and just use the data for every patch. The only difference between the LODs is the step size across the data. I'll deal with T junctions when I get there but I have a few ideas on how to deal with them. The biggest issue is fitting all of this into a quad-tree at run time. It's easy to pre-gen the quad-tree but it's not so easy to embed data down into the quad tree at run time.
Another issue is view distance. I'm assuming that total view distance based on height above the terrain is the projection of the altitude vector onto the horizontal view vector or look vector. This would be true out to a certain maximum distance.
Other issues that come into play are shaders. I need to do sky scattering both on the skydome or skybox and on the terrain. But the terrain also needs diffuse lighting with the blend texture map system to arrive at the final terrain. This is a lot of inputs into the shader. I simply do not know how to 'chain' shaders. Ideally one shader would take care of the diffuse and blend map and the other shader would then do the sky scattering on the result of the previous shader. Anyone know how to chain these together?
Here is a screenshot of terrain with detail levels. I'm trying to get excellent texture detail closeup and lessen it the farther away from the viewer the terrain is. This sample uses 5 textures, grass, grass_far, rock, sand, and the blend map. Closeup UV's are multiplied by 512.0f, mid UV's are 32.0f, and far UV's are 1.0f.
Here is a sample of my cloud shader and associated source.
Code:uniform extern texture cloud_tex1;
uniform extern texture cloud_tex2;
float2 cloud_scroll1;
float2 cloud_scroll2;
float4x4 matWorldViewProj;
sampler Cloud0 = sampler_state
{
texture = <cloud_tex1>;
};
sampler Cloud1 = sampler_state
{
texture = <cloud_tex2>;
};
struct VS_OUT
{
float4 Pos : POSITION;
float2 TexUV1 : TEXCOORD0;
float2 TexUV2 : TEXCOORD1;
};
VS_OUT VS(float4 Pos : POSITION,float2 TexUV1 : TEXCOORD0,float2 TexUV2 : TEXCOORD1)
{
VS_OUT outData = (VS_OUT)0;
outData.Pos = mul(Pos,matWorldViewProj);
TexUV1 += cloud_scroll1;
TexUV2 += cloud_scroll2;
outData.TexUV1 = TexUV1;
outData.TexUV2 = TexUV2;
return outData;
}
float4 PS(VS_OUT inData) : COLOR
{
float c0 = tex2D(Cloud0, inData.TexUV1);
float c1 = tex2D(Cloud1, inData.TexUV2);
float total_inverse = 1.0f / (c0 + c1);
//c0 *= c0 * total_inverse;
//c1 *= c1 * total_inverse;
return (1.0f - total_inverse) * 3.0f;
}
technique Clouds
{
pass P0
{
VertexShader = compile vs_3_0 VS();
PixelShader = compile ps_3_0 PS();
}
}
Man I love your work Bubba, it's really impressive...
I know you're working on an engine, or at least were. Do you plan to release it? Commercially or otherwise?
On issue with your cloud shader: the clouds should be dark in the innermost parts and white and fading on the outside. That would make them look a lot more realistic.
Agreed. Here is a render of the changes.
Oh, yes. Much nicer.
Sorry Bubba I don't want to hijack the thread so I will make this brief. I noticed in your code you did this:
Are those typedefs or classes? Right now I am thinking about reworking my own vector and matrix headers, and right now I am just using typedefs such as:Code:float2 cloud_scroll1;
float2 cloud_scroll2;
float4x4 matWorldViewProj;
I am thinking about making classes instead but I am just unsure if I want to take the time to do so, and if there would really be any benefit from doing so.Code:typedef float[2] .....etc....
Those are HLSL intrinsic data types. The shader code is not C++.
I've optimized the system to create 1 index buffer and then do a quick memcpy from that buffer to the actual Direct3D index buffer. I've also optimized the normal creation so that a normal map is generated once at load time and then re-used when needed.
I have a huge problem now. I need to page in terrain data and somehow maintain my quad-tree structure. Right now if I set the quad tree range to twice as large as one grid then 2 terrains are displayed as expected. Likewise when I set the range to 4.0f then 4 terrains are displayed and so on. So the code is in place to begin loading terrain and creating patches in world space. I just don't know how to place them into the quad tree at run-time without re-building the entire tree.
UPDATE:
I finally have endlessly repeating terrain. After trying several algorithms I decided on this one b/c it fits nicely into my quad tree.
- Each terrain 'patch' has a world position.
- All vertices in the patch are in model space.
- The height for the vertices is grabbed from the heightmap as if the vertices were already in world space.
- The size of the quad-tree is constant so the max view distance must be specified at program start.
- Each patch is rendered separately using an ID3DXEffect that can be specified to alter the appearance of the terrain.
- Terrain::Update() calls Terrain::UpdatePatches() which subtracts the camera position from the patch position. If the x difference or z difference is greater/less than the positive/negative world map width the patch pos is translated positive/negative world map width. This causes the terrain pieces to wrap around if they get too far from the camera. At this point, new terrain data could be loaded in which really resolves to just changing the y vertex pos of the current mesh. In this way terrain can be paged in at run-time.
Major issues so far:
- View distance kills framerate. The best view distance is: -worldwidth * 4.0f, worlddepth * 4.0f, worldwidth * 4.0f, -worlddepth * 4.0f. One world depth is 256 * cellsize, 256 * cellsize. Cellsize looks best at around 32 or 64 units. As you can tell this is a ton of data. The good news is that the patch size is only 32x32 so I'm only sending 32x32 patches to the card at any one time.
- The frustum culling is still wacky. In certain directions it works fine but in others patches pop in/out which should not happen. I'm still unsure as to why this is happening.
- Terrain pieces 'pop' in on the horizon which doesn't look the best
- There is currently no CLOD but this scheme is nearly ready for geo-mip mapping so it should not be too much to implement.
I can either try to fix the AABB frustum culling or just use spheres. Spheres do not suffer from the same issues as the AABBs. Very odd. Next on the list would be to implement the geo-mip mapping. I believe what this amounts to is having various levels of detail in static vertex buffers. The index buffers do not change. At run time the algo would decide which vertex buffer to use based on the total screen space error or whatever decision metric I use to decide the LOD. After this the next step is to page in new data. Since I already know which 'patches' are leaving the viewing area I can cache in new data based on the patches new world position and pull that data from a file or heightmap.
Had to stop development and clean my computer. Started feeling hot air while coding and realized my CPU was at 76C. My terrain graphics takes my card from 64c to around 80c. The code is running 3 vertex and pixel shaders and pushing just under 1 million tri's which shouldn't really tax it too much. My card is supposed to support 7 pixel shader units. This was using a view area of 65,536 units on x and z. I tried to push the viewing area to 262,144 but the vertex buffer failed. I'm assuming I probably ran out of video memory.
Obviously some LOD techniques are needed before proceeding any further. Arctic Silver for my poor CPU is also in order. After cleaning the system a bit I removed the CPU, power supply, etc. Put it all back together and the computer wouldn't stay on for more than 10 seconds. The thermocouple under my CPU was shutting the system down. I finally got it working but I think a new system is a definite must now.
I recently bought an entirely new system. My GPU at the time (7800GS) was on it's way out - overheating like you wouldn't believe. T'was packed with dust, the horrible sticks-to-everything-electronic dust so I actually dismantled it, which probably wasn't a good idea, even less so since I didn't have an Arctic Silver.
So yeah I murdered my last GPU. Good times.
Hehe, my last GPU burned out because I didn't clean it in time. All the dust killed the fan, and then it was a matter of minutes until I smelled burnt dust and the screen went completely insane.
If your system is at 64c in idle, you've already got a problem.
My last dust killer was just the opposite. It had the same sticky brown dust on everything electronic, so I went to vacuum it out with a crevice tool on the end of the vacuum hose.Quote:
Originally Posted by CornedBee
I'm at a friend's home, and she's got a Dyson big vacuum, all yellow and such - which wouldn't suck a postage stamp off the ground, on a July day.
I was SO disappointed with Dyson - he looks so clever in his commercials -
So I blew the dust off the boards, and then vacuumed up what the pitiful Dyson would pick up, with it's $400 suction. :(
Next week the thing overheated! The dust had been blown around enough that it had managed to smartly settle on the thermal grease of the cpu, making it stop working.
Of course, my friend had tried to re-start it and actually ran it for short periods of time, throughout the week. Mobo was cooked.
I went there to back up her data and install a new HD, but I should have brought the thermal compound along with me.
After cleaning everything including the power supply internals my temps are now:
Idle
CPU: 48-50 C
Mobo: 30 C
GPU: 47 C
Under load
CPU: 52 - 55 C
Mobo: 32 C
GPU: 52 C
Before cleaning:
Idle
CPU: 64 - 67 C
Mobo: 47 C
GPU: 64 C
Under load
CPU: 72 - 76 C
Mobo: 49 C
GPU: 90 C (wow!)
Holly!Quote:
Originally Posted by Bubba
The fact you can still use it, makes that the sexiest graphics card I've ever seen. Brand and model please :)
BFG Tech GeForce 7800 GS 256 MB AGP OC
I'm surprised it still works. Max temps for NVidia GPU's are actually around 120C before the chip starts to actually fry itself.
Max temps for AMD CPUs are rumored to be around the 115C to 125C mark. I have a feeling you would know it was getting that hot long before it failed. I saw an old Athlon 1 GHz run up to 205 to 215F way back in the day. The customer requested a new chip so my friend who owned the biz took it and put it into one of his office systems. It ran just fine after that and prob would still run today.
You betcha. If memory is still fully usable it certainly qualifies as a "good job guys!" email to BFG.
It would probably pay for you to install some monitoring software. I'm not confident enough to suggest any. What I seem to know is that there is no way to gauge gpu temperatures from an agp slot, which if true is a bummer. Unless the card comes with a probe and the needed software, you are out of luck.
Started to implement geo-mip mapping but I'm still running out of video memory. I'm thinking I may have to store some vertices in system RAM and use DrawPrimitiveUP() on those. It will be the most distant patches so it should not be too much of a hit. Right now I do not have any 'transitory' patches that connect the different LODs and with a view distance of 262,144 units I'm completely out of video memory.
With a view distance of 262,144 and 6 levels of LOD spaced every 1600 units I'm getting just over 50 FPS which is terrible.
Running out of algos and options here. There is something someone is not sharing over the internet on how to get this thing to really fly. I have some ideas for updating the patches in which I will only check patches that are on the edges or on the horizon. Patches that are elsewhere will never be moving out of view so it is a waste of time to check them for wraparound.
More details later.
Early screenshot in wireframe: