Graphic pipeline stages - I'm lost

[boris16]

Hiya

I’m interested in learning how a 3d picture ( in a 3d game) is formed and finally displayed on monitor, but I’m really having trouble figuring out the process.

a) In what order and through which different stages must an image go for it to be displayed on the screen?


b) At what stages do MTUs, pixel and vertex shaders fit in?


c) What I’ve found thus far:

One article claims there are two main stages : geometry and rendering stages,

while wikipedia claims the main stages are:

1)Modeling transformation
2)Per-Vertex Lighting
3)Viewing transformation
4)Projection transformation
5)Clipping
6)rasterization
7)Texturing
8)Display


How do the “wikipediaâ€￾ stages relate geometry and rendering stages which the first article claimed to be?



thank you

 

[Davros]

not that i know much on the subject but to me they seem on the wrong order

per vertex lightng for example sureley gets done after texturing as the light affects the brightness/colour of the textures

and rasterisation is the process of turning the scene into dots (or pixels) that must come after texturing

/me waits for someone to point out how wrong i am

 

[trinibwoy]

IIRC vertex lighting is driven by attributes of the vertex itself - the color does not come from a texture. The order seems fine. Texturing is a per-pixel operation so it comes after rasterization.

 

[Rys]

Given that these days you can also sample at the vertex processing stage and that shading is programmable, you could probably just simplify it to the following (for an IMR), while remembering that there's various feedback throughout the pipeline and that there's another render stage to consider with D3D10:

Vertex Shading
Triangle Setup
Fragment Rasterisation
Pixel Shading
ROP ops
Display

 

[eSa]

Hiya

c) What I’ve found thus far:

One article claims there are two main stages : geometry and rendering stages,

while wikipedia claims the main stages are:

1)Modeling transformation
2)Per-Vertex Lighting
3)Viewing transformation
4)Projection transformation
5)Clipping
6)rasterization
7)Texturing
8)Display


How do the “wikipedia” stages relate geometry and rendering stages which the first article claimed to be?

thank you

The very basic (and most common) way of composing a 3d gfx image is loosely as follows; 3d objects are build using simple geometric primitives, usually triangles. Each triangle has three corner points, situated in three dimensional space. Corner points have three coordinates X,Y and Z.

Triangles form a mesh. This mesh represents the outer shell of object, for example the outer surface of a ball. When an objects is moved in 3d space, each corner point of each triangle is moved using same operation(s). These operations are called transforms.

Finally projection transform is done and three dimensional triangles are projected to the two dimensional plane. This phase mimics the way we humans see world around us. Objects far away from us are drawn smaller that ones near us. Even if they are originally all the same size.


These all belong to the "geometry stage";

1)Modeling transformation - We move object(s) around in 3d world. For example turn character in game.
2)Per-Vertex Lighting - We calculate how lightsources affect each polygon of each object
3)Viewing transformation - We situate a camera (also called viewer) in to the 3d world
4)Projection transformation - We project areas the camera can see to the 2d plane
5)Clipping - We remove the objects that are outside of our view area



These belong to the so called rendering stage;

6)rasterization
7)Texturing
8)Display

Allthought it is bit misleading to use term "rendering" because it usually means the whole process of creating a 3d image. That is, stages 1-8.

Do note that above stages 6,7,8 are bit wrong. In normal terminology they are all more or less fused together. And called as one stage, rasterization.

Basic rasterization includes;

polygons (usually triangles) are drawn and during the drawing the pixels inside polygon are colored. The final color for each pixels comes usually as combination of different sources; variation of light across surface, the texture(s) used to "cover" the surface and so on.

 

[Pete]

Sure, why not.

Edit: The people above are smarter, listen to them. Rys and eSa got more explicit while I was hacking together my well-intentioned but ill-advised post. I write these things in the selfish hope of learning more myself by being corrected.

At the risk of furthering ignorance (Mint especially loves these posts), I'll try saving the smarter people here some time and offer some comments.

a) I think you mean "what are the stages of transforming a scene into an image?"

b) I think you mean TMU and vertex & fragment shaders. A GPU starts with triangles in the form of vertices, turns these into fragments, and pixels are the final display picture elements (picel -> pixel).

c) Application (models) > Geometry (vertices) > Rasterization (pixels). Very basically, shapes > layout > colors. That list looks mostly correct, but I'd add fragment/pixel shading to 7) while noting that step 6 should be Triangle Setup and you can refer to steps 6-8) as "rasterization." Better yet, read this intro (helpful diagrams): An Introductory Tour of Interactive Rendering (from the very relevant Real-Time Rendering's site).

Just using the GeForce (Geometry Force, which pushed TnL from the CPU onto the GPU) as a guide, I very roughly (and ignoring SM3+ with its TMU-aware vertex shaders and new geometry shaders, which further complicates things) estimate:

1) CPU
2-4) "TnL" (transform and lighting), so vertex shaders
5) still occurs on the GPU, still dealing with vertices and so geometry, though we're not shading anything
6) convert (vertex-based) triangles into (screen-space) fragments/pixels, so it's pixels from here on out
7) TMUs and fragment shaders (technically we're not at pixels yet, just fragments, according to OGL--I believe the more "relaxed" use of pixel was popularized by [earlier versions of?] D3D) join the fray
8) ROPs graduate fragments into pixels in VRAM and they're ready for display (via RAMDAC or I forget what DVI uses [TDMA?]).

 

[Davros]

IIRC vertex lighting is driven by attributes of the vertex itself - the color does not come from a texture..

were am i going wrong

surely if i had a triangle texured white and 1 of the vetex's was lit with a red light then part of the white texture would be graduated pink so ergo lighting has to be done after texturing

edit : just thought about it it could be done the other way round

 

[trinibwoy]

were am i going wrong

surely if i had a triangle texured white and 1 of the vetex's was lit with a red light then part of the white texture would be graduated pink so ergo lighting has to be done after texturing

My understanding of traditional per-vertex lighting is that the vertex color is part of the vertex information sent by the CPU and is not read from a texture. During rasterization the intial unshaded color of the pixel is a result of interpolating the colors of the three vertices making up the triangle. So in your example if all three vertices were red then this unshaded pixel will be red when entering the shading pipeline and will be blended with the white texel there. This is what I consider per-vertex lighting and like Pete I'll await correction from someone who actually knows what they're talking about

 

[Pete]

I'm not an authority here, but I'm pretty sure trini's right. When the texture color is applied to fragment, it's blended with the color derived from interpolating the per-vertex lighting values. It doesn't need to be in order in the sense that the surface has a pattern first, then is lit, as it's a simple blend.

See slides 29 and 33 here for diagrams detailing per-vertex lighting and when it occurs in the rendering pipeline.

 

[Zengar]

Basically what triniboy said applies. In the fixed-function pipeline, the lighting is computer per-vertex in the first stage. Let us say, the vertex transformation produces a set of attributees per vertex (transformed position, lit color, texture coordinates etc.) The rasterizer then interpolates this attributes across the polygon, so that each fragment (pixel) gets his own set of attributes. When texturing is applied (per-fragment!), the texture color and the vertex color are combined together in some way (that is usaully programmable). Most usually, the texture value replaces the vertex color, or they got multiplied (resulting in the effect Davros is describing). With introduction of multitexturing, more complex combinations were possible. That all led to a utterly complicated but still inflexible model (I remember Nvidia's texture_shader extensions, brr). Thank God we have new hardware, that get rids of all that stuff now.

 

[Sc4freak]

were am i going wrong

surely if i had a triangle texured white and 1 of the vetex's was lit with a red light then part of the white texture would be graduated pink so ergo lighting has to be done after texturing

edit : just thought about it it could be done the other way round

Yep, it's done the other way round. This is because the pixel shader (where texturing is done) is applied after the vertex shader (where vertex transformation is done). In per-vertex lighting, usually all the vertex shader does is pass on the texture coordinates to be interpolated across the triangle, ready for the pixel shader to actually fetch the texture colour.

 

[boris16]

Hiya


It's starting to make some sense

I'd just like to ask two additional question, if I may:

1)
Can you also tell me which of the two shaders calculates:

- Anisotropic filtering
- trilinear filtering
- bump mapping
- FSAA



2)
And more importantly, does pixel shader also include the functionality of TMU, or is TMU still a separate unit?



cheers

 

[trinibwoy]

Texture sampling and associated filtering (bi, tri, aniso) happens in the TMU's.

Bump mapping is done in a pixel shader.

FSAA is done in the ROPs.

TMU is a separate unit. Just think of it as a worker for the shader units - it fetches texels used by the shaders.

 

[Rys]

Texture sampling and associated filtering (bi, tri, aniso) happens in the TMU's.

Bump mapping is done in a pixel shader.

FSAA is done in the ROPs.

TMU is a separate unit. Just think of it as a worker for the shader units - it fetches texels used by the shaders.

Bump mapping requires sampling too, remember :smile: It's probably also worth noting that TMU is (IMO anyway) a poor term for the unit these days.

 

[Simon F]

4)Projection transformation - We project areas the camera can see to the 2d plane
5)Clipping - We remove the objects that are outside of our view area

Clipping needs to be done before the projection to the 2D plane.

 

[Xmas]

That would be
4) Projection transformation
5) Clipping
6) Division by homogeneous W

 

[Simon F]

That would be
4) Projection transformation
5) Clipping
6) Division by homogeneous W

Yes... but I was too lazy to type all that