VS3.0 in NV40

[Ante P]

Any really nice usage scenarios?
Or do we perhaps need some sort of HOS for it to be really usefull?

Pre-calculated displacement mapping doesn't seem all that fancy right now IMHO.

So any other in-game usages that one can image?

 

[rendezvous]

It'd be nice to have in a dynamic environment where you'd like to displace the vertices, just like in SM3.0 boat demo that nvidia made.

 

[Ilfirin]

Dynamic displacement maps (as, I'm sure, has been mentioned a hundred times) - evaluate some fluid dynamics in the pixel shader, save the result to a texture. Do a per-vertex texture lookup in the next pass, and tada, totally GPU based 3D water.

Not particularly interesting, but really the uses are pretty minimal with 3.0 since you still can't realistically do any silhouette extractions on the GPU (and hence any modifications you make to the vertex positions will produce incorrect shadowing on anything that requires silhouettes). So, unless you're using a strictly ad-hoc shadow map solution (not a hybrid/smoothie version) or are just plain not doing dynamic shadowing, the only geometry you can apply these types of effects to are the ones which wouldn't get shadowing anyway (ala, water).

And displacing a pre-tesselated model with a static displacement mapping is just plain dumb (why not just displace it once at load-time and forget about it?). Soo..

3.0 is a big tease - you can *almost* do a lot of really cool stuff, but not quite. 4.0 addresses these issues

 

[Ante P]

3.0 is a big tease - you can *almost* do a lot of really cool stuff, but not quite. 4.0 addresses these issues

hehe sort of having that impression too yeah

 

[DemoCoder]

Not particularly interesting, but really the uses are pretty minimal with 3.0 since you still can't realistically do any silhouette extractions on the GPU (and hence any modifications you make to the vertex positions will produce incorrect shadowing on anything that requires silhouettes).

This particular use case would also rule out all vertex shaders that modify vertex position, which pretty much is everything. How would you propose that models which used skinned vertices get around this problem without doing the skinning on the CPU first (before calculating shadow hulls) and duplicating it on the GPU?

That it causes trouble for stencil shadow volumes (vertex shaders) is not really a legitimate criticism of VS3.0 in particular. Anyway, for any surface that needed vertex shader extrusion, the degenerate quad method could be used on those displaced surfaces, since displacement textures are likely to be used for particular effects, not on everything.

 

[krychek]

And displacing a pre-tesselated model with a static displacement mapping is just plain dumb (why not just displace it once at load-time and forget about it?).

How about LOD in large terrains? You could have a single flat patch of vertices which are displaced based on the heightmap. Would be helpful if you are bandwidth restricted (during large movements across terrain). Storing as heightmaps also has the advantage that they can be compressed. Noise textures may also be used to combine when the resolution of the heightmap texture becomes too low for thje geometry.

 

[Ilfirin]

DemoCoder:
I was very careful in putting 'realistically' in there for a reason I don't consider the degenerate quad method to be a realistic solution in the general case considering the number of added vertices required. Now, if your software isn't too heavy on the vertex work, go ahead, but it's not something you can recommend for every case and expect it to always work out.

Also, it's not just shadow volumes that need silhouette information.. lots of the hard-edged hybrid shadow mapping solutions also require it, and in many (all?) of those cases the degenerate quad method can't be used because you actually need the silhouette itself, not the shadow volume (that results from extruding and then projecting the silhouette). Smoothies (soft shadow maps) also require silhouettes and can't use the degenerate quad method.



It should also be noted that games requiring dynamic shadowing (that aren't using just plain ad-hoc shadow maps) *do* still do all their animation on the CPU. (Though there are a few notable exceptions where GPU based shadow volumes were used)

 

[Ilfirin]

And displacing a pre-tesselated model with a static displacement mapping is just plain dumb (why not just displace it once at load-time and forget about it?).

How about LOD in large terrains? You could have a single flat patch of vertices which are displaced based on the heightmap. Would be helpful if you are bandwidth restricted (during large movements across terrain). Storing as heightmaps also has the advantage that they can be compressed. Noise textures may also be used to combine when the resolution of the heightmap texture becomes too low for thje geometry.

That'd be fine, if we had hardware tesellators, but we don't.. (and hence you're going to end up building the LODs on the host anyway.. so why not do the displacements while tesellating. Then you're only displacing once, rather than every single pass, on every single frame)

 

[DemoCoder]

My point Ilfrin is that your criticism applies to the very notion of vertex shaders. If dynamic stencil volumes are the absolute priority, than NVidia and ATI should produce cards without vertex shaders since they cannot be used at all for vertex transformations.

Basically, the 'T' part of T&L can't be used.

I think it's too extreme. There is a balance that can be struct being using stencil shadow volumes, and using vertex shaders (including displacements in vertex shaders) For example, large static sections of the world can have static shadow volumes calculated, whereas dynamic models or displaced surfaces can have volumes extruded on the GPU. Or, some game engines will just use shadow buffers.

Secondly, pre-tesellated meshes for displacements are primarily useful for *animated* displacements like in Half Life 2. Hardware tesselation adds a form of geometry compression.

 

[Ilfirin]

Again I want to emphasize that silhouettes are required for lots of graphics algorithms, not just shadow volumes (and not just shadows). Not all cases have versions that can be run totally on the GPU (yet.. again, 4.0 solves this).

My point Ilfrin is that your criticism applies to the very notion of vertex shaders. If dynamic stencil volumes are the absolute priority, than NVidia and ATI should produce cards without vertex shaders since they cannot be used at all for vertex transformations.

Basically, the 'T' part of T&L can't be used.

Currently you still get all the 'T' part from non-deforming models (which should be about 90% of the on-screen models) even when using CPU-based shadow volumes. And you generally don't do the final perspective projection even when handling animated models on the host, so there's still some 'T' help there.

And there's lots of stuff to do in vertex shaders that don't modify the topology of the model (such as rotating a spherical harmonic light probe into tangent space).

I think it's too extreme. There is a balance that can be struct being using stencil shadow volumes, and using vertex shaders (including displacements in vertex shaders) For example, large static sections of the world can have static shadow volumes calculated, whereas dynamic models or displaced surfaces can have volumes extruded on the GPU. Or, some game engines will just use shadow buffers.

But, remember, shadow volumes have to be recalculated everytime the light changes too.. not just the geometry. And things get trickier with soft shadowing (depending on which of the huge myriad of choices you choose for an implementation).

OT: Why does no one ever spell my name right?

 

[krychek]

Hey Ilfirin, I got your name right back at the ion storm boards

That'd be fine, if we had hardware tesellators, but we don't.. (and hence you're going to end up building the LODs on the host anyway.. so why not do the displacements while tesellating. Then you're only displacing once, rather than every single pass, on every single frame)

What do you mean by hardware tesselators? Why would we need that? My point was that storing as heightmap save a lot of space. 1/2 bytes compared to 12/20 bytes per vertex. The x-z info is same for the terrains anyway. Granted that It'll be unnecessarily displacing the same patch in subseqent passes but with this method much more goemetry (heightmaps) can be passed than before so moving through detailed terrain or rather panning a zoomed view on large terrain should be pretty smooth. Maybe SM4.0 will allow to displace once and save the result.
If any compilcated lighting is used for the tarrain.. the vertex shader will be idle anyways so we lose nothing by displacing every frame. Ofcourse this is a very specific use, I agee with your points in general but I felt it does have some advantages in this case.

 

[Guden Oden]

Dynamic displacement maps (as, I'm sure, has been mentioned a hundred times) - evaluate some fluid dynamics in the pixel shader, save the result to a texture. Do a per-vertex texture lookup in the next pass, and tada, totally GPU based 3D water.

Sorry for replying to such an old post, heh, but it seems to me such an approach would just lead to waves that bob straight through anything placed on them - such as boats, for example, since the GPU isn't running the overall physics model as well...

Also, wouldn't this risk affecting waves as the framerate varies, or is there some sort of hardware timer provided by the GPU to provide a steady input signal to synchronize to? Or does the CPU do this?

 

[nAo]

Sorry for replying to such an old post, heh, but it seems to me such an approach would just lead to waves that bob straight through anything placed on them - such as boats, for example, since the GPU isn't running the overall physics model as well...

That's not true in the general case.
GPU can be aware of boundary conditions or ties, at least in this case (a simple 2D water simulation with small waves)

ciao,
Marco

 

[Guden Oden]

But imagine a game with several speedboats leaving wakes that interact with each other and other boats, or on choppy seas. The boats would have inertia, center of gravity, they would have to react to player input, surface drag, possibe impacts with other boats or buoys or such, or even land/underwater objects.

How would you do all this on the GPU? Would the game ever know the precise location of a wave if the wave is being rendered entirely by the GPU?

Doing waves with a combination of pixel/vertex shaders sounds as if they might make for wonderful screen savers that don't consume much CPU time, but how useful is this in an actual GAME sceneario? Humus' waving flag demo clips through itself sometimes, wouldn't similar things happen on a routine basis if we put too much work on the GPU?

 

[nAo]

But imagine a game with several speedboats leaving wakes that interact with each other and other boats, or on choppy seas. The boats would have inertia, center of gravity, they would have to react to player input, surface drag, possibe impacts with other boats or buoys or such, or even land/underwater objects.

It's entirely possible. Even if at this time one have to use some trick (like rendering into a texture or vertex buffer .or stuff like that)

How would you do all this on the GPU?

A wave operator can be evaluated very easily on a GPU, even with weird dynamic boundary conditions.
GPU are powerful with convolutions, with exotic kernels too, you know..

Would the game ever know the precise location of a wave if the wave is being rendered entirely by the GPU?

The real question is: does the game want to know where the wave or the boat is? Maybe..it depenpds on the game.
The GPU could render all the data the game-logic has to know in a (very small) buffer and the CPU can read it back a frame later.

Humus' waving flag demo clips through itself sometimes, wouldn't similar things happen on a routine basis if we put too much work on the GPU?

Humus's flag demo shows compenetration like tons of other cloth simulation demos that run on common CPUs.
Can a GPU handle a more correct simulation? the answer is: yes!

ciao,
Marco

 

[Ilfirin]

What do you mean by hardware tesselators? Why would we need that? My point was that storing as heightmap save a lot of space. 1/2 bytes compared to 12/20 bytes per vertex. The x-z info is same for the terrains anyway. Granted that It'll be unnecessarily displacing the same patch in subseqent passes but with this method much more goemetry (heightmaps) can be passed than before so moving through detailed terrain or rather panning a zoomed view on large terrain should be pretty smooth. Maybe SM4.0 will allow to displace once and save the result.
If any compilcated lighting is used for the tarrain.. the vertex shader will be idle anyways so we lose nothing by displacing every frame. Ofcourse this is a very specific use, I agee with your points in general but I felt it does have some advantages in this case.

I just misread your original post. Yeah, I suppose if you're extremely pressed for vertex sizes this could be one method to further reduce that number, but I don't expect such cases to be particularly common.

Actually, that's kinda clever.. you'd only need to store a uv-coord per terrain vertex for positional data, the rest (even tangent, binormal and normal vectors) could be either generated or read from a texture in the vertex shader. Effectively reducing the vertex size by a factor of 7. Neat (ignoring the fact that GPUs work best when sizeof(vertex) is a multiple of 32 bytes, that is).

"Hey Ilfirin, I got your name right back at the ion storm boards"
I thought your name looked familiar, Hi.

Sorry for replying to such an old post, heh, but it seems to me such an approach would just lead to waves that bob straight through anything placed on them - such as boats, for example, since the GPU isn't running the overall physics model as well...

The initial example I gave *was* just for a gimmicky, morrowind-esque water effect. Not for anything useful

But, you could project the center of mass of some object that's supposed to be floating into the plane of the water to generate a uv-coord which you could then use to index the water's displacement map completely on the GPU. Using this general idea with a bit more information about the physics of the model that's passing over the water, you should be able to get a aesthetically pleasing physics simulation of something like a boat bobbing in water (just generate the vertex's transformation matrix in the vertex shader and then transform that vertex by said matrix).

This still wouldn't handle multiple objects on water colliding with each other, but a similar method should be acheivable using render to texturing w/ vertex shader texture lookups. Though I question how efficient (or accurate) such a method would be..

And there's still a problem that the CPU will inevitably need all this information back, to feed the AI and sound systems and such (plus, there's still the dynamic shadowing problem), and with AGP the cost of getting that information back is usually more expensive than just doing everything on the CPU to begin with, so it'll all have to wait until PCI-Express is in the mainstream.

 

[Guden Oden]

But imagine a game with several speedboats leaving wakes that interact with each other and other boats, or on choppy seas. The boats would have inertia, center of gravity, they would have to react to player input, surface drag, possibe impacts with other boats or buoys or such, or even land/underwater objects.

It's entirely possible.

It's entirely possible to do a realistic water physics model for several boats including accurate collision detection and the application of forces from waves coming in different directions) entirely on the GPU at playable frame rates?

The real question is: does the game want to know where the wave or the boat is? Maybe..it depenpds on the game.

What game would not want to know? You'd want to hear splashes and such of course and thunks when you crash into stuff. How can the game know if the player is on course if it has no idea where the boat is, and how can a player or AI character react to side-drift or wave crests etc or even knocks and bumps if there's no feedback between the game engine and the actual rendered graphics on-screen?

Furthermore, it might be a multiplayer title, and then it would be beneficial if all players had the same waves on their screen, and indeed if the boats are in the same positions.

The GPU could render all the data the game-logic has to know in a (very small) buffer and the CPU can read it back a frame later.

Hm, is it really realistic to expect the game and the player to run a frame *behind* the graphics? That just seems pretty backwards to me.

Humus's flag demo shows compenetration like tons of other cloth simulation demos that run on common CPUs.
Can a GPU handle a more correct simulation? the answer is: yes!

But how much of a performance penalty will that mean? The flag must do collision detection on itself on a per-polygon basis. Now, on a flag it probably doesn't matter, I know of no game where there are like millions of flags waving around, but on clothes for example it could have a somewhat larger performance impact.

On the other hand it would be performance well spent, I'm sure we've all seen games where parts of bodies - like a head for example - clip through the clothes the character wears when looking around or moving. Shirt collars and hair/chins in particular seem a problematic combination.

 

[nAo]

It's entirely possible to do a realistic water physics model for several boats including accurate collision detection and the application of forces from waves coming in different directions) entirely on the GPU at playable frame rates?

Very weird question..actually GPUs are MUCH better than CPUs in this kind of tasks.

What game would not want to know?

Many games wouldn't want to know. A game can do fast physics simulations via GPU on objects that don't directly interact with gameplay.
One could setup an entire particles system (with collisions detection) or a dynamic fur simulation on the GPU without needing to readback any data with the CPU.

You'd want to hear splashes and such of course and thunks when you crash into stuff.

That's why I wrote that depends upon the application.
If in a game enviroment it's raining the GPU can simulate raindrops but the game logic don't need to know where and/or when the raindrop fall to play a 'rain sound'

Furthermore, it might be a multiplayer title, and then it would be beneficial if all players had the same waves on their screen, and indeed if the boats are in the same positions.

Actually all the multiplayer games have those kind of problems, physics simulations on the GPUs don't add additional burden regarding this particular problem (except different precision support on different hw running on different platforms...)

Hm, is it really realistic to expect the game and the player to run a frame *behind* the graphics? That just seems pretty backwards to me.

what?! actually a lot of games run a frame 'behind' the graphics!
And no one ever complained
What about a lot of games that do occlusion detection (to save draw calls or to calculate an occlusion term to better simulate special effects such as lens flares, halos, etc..) via GPU?

But how much of a performance penalty will that mean? The flag must do collision detection on itself on a per-polygon basis.

I don't understand your point. What's basically wrong with that?
In the future more and more games will offload part of their physical simulation on the GPU.

Now, on a flag it probably doesn't matter, I know of no game where there are like millions of flags waving around, but on clothes for example it could have a somewhat larger performance impact.

Like in every game a developer should know how to make tradeoffs..how to balance calculations between CPU and GPU.
GPU+programmability is an huge and powerful new field where we are just learning how to play with.

ciao,
Marco

 

[Nick]

3.0 is a big tease - you can *almost* do a lot of really cool stuff, but not quite. 4.0 addresses these issues

Do you have 4.0 specifications?

 

[Ilfirin]

The same stuff everyone else has. (Where they explicitly state unblocking those exact blocked scenarios as goals in its development)