How to do Shader AA?

[Humus]

Dunno. I'm still trying to understand why HDR isn't rendered directly into the framebuffer - what is all this compositing stuff?

You'd need an FP16 framebuffer. The compositing stuff is mainly adding bloom and/or working around lack of blending by rendering to different buffers and the combining them in the end. Also, you need the tonemapping pass in the end. Unless you're just going to use a plain scale operation to reduce the range you can't express the math with blending (1 - exp(-exposure * x)), so you'd have to have the rendering in a separate buffer as an input for a final pass to that goes to the framebuffer.

 

[Jawed]

OK, so an example of compositing is the bloom effect shown on page 20:

http://www.ati.com/developer/gdc/D3DTutorial08_FarCryAndDX9.pdf

Is it correct to summarise the bloom process as?:

1. render the full scene to FP16 framebuffer
2. copy the framebuffer to an FP16 texture
3. apply the bloom
4. blend the bloomed-texture directly onto the framebuffer

What bugs me about this is that MSAA in step 1 would in no way impact the applicability of compositing.

Is this how tone-mapping goes?:

1. copy the FP16 framebuffer to a FP16 texture
2. clear framebuffer
3. set the framebuffer to integer 8
4. tone-map FP16 texture -> FX8 and write to framebuffer

Again, I don't see how MSAA would "get in the way" of this process. The framebuffer enters tonemapping already AA'd.

Jawed

 

[nAo]

Is this how tone-mapping goes?:

1. copy the FP16 framebuffer to a FP16 texture
2. clear framebuffer
3. set the framebuffer to integer 8
4. tone-map FP16 texture -> FX8 and write to framebuffer

Hopefully step 1 is not needed on a closed platform, and step 2 it's not needed at all since
you're going to fill the whole FX8 buffer anyway

 

[Jawed]

Just having some fun with tone-mapping in Photoshop:

The bloom-map from the PDF doesn't cover the entire screenshot and doesn't seem to line up entirely. Also, the tonemapping reveals some nasty JPEG artefacts in the No HDR screenshot. Oh well.

Jawed

 

[Hyp-X]

Is it correct to summarise the bloom process as?:

1. render the full scene to FP16 framebuffer
2. copy the framebuffer to an FP16 texture
3. apply the bloom
4. blend the bloomed-texture directly onto the framebuffer

What bugs me about this is that MSAA in step 1 would in no way impact the applicability of compositing.

As Humus has pointed out there's no such thing as an FP16 framebuffer (at least on current cards).
So this method doesn't work with FP16.
Make that:

1. Render the full scene to an FP16 offscreen render target
2. Copy the offscreen render target to an FP16 texture
3. Draw the texture with conversion to the framebuffer (FX8) (possibly tone mapping)
4. Blend the texture blurred & scaled to the framebuffer (bloom)

step 1-2 is only separate to support MSAA, otherwise the first step is:
1. Render the full scene to an FP16 render target texture

step 3-4 can be a single step or multiple steps

 

[bloodbob]

Again, I don't see how MSAA would "get in the way" of this process. The framebuffer enters tonemapping already AA'd.
Jawed

Cause they don't aa FP16 back buffers and second to correctly aa you need to know the tone mapping processes would be better then nothing still. ( similar to gamma correct AA)

 

[psurge]

bloodbob - what if you skipped the downsampling step on the FP16 buffer entirely?
If you arranged the MSAA samples to be constrained to a regular subgrid
(e.g. for 4xAA:

 -------- -------- 
| .  1   |        |
|        |  . 2   |
 -------- --------
|        |    .   |
| . 4    |  3     |
 -------- --------

)

then could one reasonably treat the buffer as oversampled and tonemap normally?

Also, do current real-time tone-mapping implementations actually operate on a full resolution framebuffer, or do they do additional/manual downsampling?

 

[Jawed]

As Humus has pointed out there's no such thing as an FP16 framebuffer (at least on current cards).

I don't think that's what Humus said at all

Jawed

 

[KimB]

1. Render the full scene to an FP16 offscreen render target
2. Copy the offscreen render target to an FP16 texture
3. Draw the texture with conversion to the framebuffer (FX8) (possibly tone mapping)
4. Blend the texture blurred & scaled to the framebuffer (bloom)

Hrm, I was pretty sure that step 2 was transparent to the software. That is, if a multisample render target is used, it will automatically be downsampled when read in as a texture.

 

[Jawed]

Thanks bloodbob for some ideas.

This page shows why gamma-correct anti-aliasing is important:

http://home.no.net/dmaurer/~dersch/gamma/gamma.html

This would be the sequence of events when performing HDR rendering with gamma-correct MSAA:

1. render frame (in linear space) including AA samples (calculated by the ROPs)
2. resolve AA samples (assuming gamma 2.2 rendering on screen) to generate AA'd frame
3. tonemap the frame (still in linear space)
4. render on screen (includes conversion to gamma 2.2 screen-space)

The tonemapping, step 3, destroys the integrity of the resolved AA samples in step 2.

So to do this properly, tonemapping would have to be performed on the un-resolved frame (i.e on all AA samples). And the problem with that is the GPU can't access the MSAA samples once the ROPs have calculated them. The only kind of AA samples that the GPU could get at would be super-samples.

So it seems that to do tone-mapping of MSAA samples would require dedicated ROP tonemapping hardware - which would prolly mean that the GPU couldn't support fully programmable tone-mapping (calculated in a shader).

I suppose programmable tonemapping in the ROPs would utilise a bias plus scale, with the option to apply a variable S-curve and a variable gamma-curve.

Jawed

 

[KimB]

The tonemapping, step 3, destroys the integrity of the resolved AA samples in step 2.

Er, why? If the program starts off working in linear space, and only does the gamma correction at the end, everything should be correct, from texture filtering to AA sample recombination to tone mapping.

 

[Jawed]

Jawed

 

[KimB]

Is your tonemapping algorithm not assuming a linear color space?

 

[Jawed]

No, the tonemapping is in linear space.

Tonemapping in this scenario is a post-process. This means that the AA data has to be resolved before tonemapping. Once you tone-map AA data that assumes gamma 2.2 (which is the left hand side) then the result is broken AA.

I can't think of a way around this, if tonemapping is performed as a post-process. Because the input to the tonemapping pass is the framebuffer copied into a texture, the AA samples are already resolved.

Catch-22.

So tonemapping and AA can only be compatible if performed in the ROPs, if you want gamma-correct MSAA.

Jawed

 

[KimB]

I still don't buy it. If you intend to have a smooth gradient from white to black in a specific area, it should look like a smooth gradient whether you enable tone mapping or not.

What you might consider is working in a color space where the tone mapping function is linear across the 0-1 range.

 

[Jawed]

Look at the histogram for the scene, before tonemapping:

this shows that there's no image data between 140 and 255. If the engine was using FP16, that might represent the range 0-8 (roughly) for inside, and full-white is, say, 16 (for the sake of argument).

In the tonemapped version of the image:

a wealth of data now appears between 140 and 255 - or in the example 0-16 FP mapping, the full range from 0 to 16 is now occupied by the tonemapped scene.

So, what we're seeing here is the tonemapping pass converting a dark scene into one where the "eyes have adjusted", where the interior with the shafts of light occupies most of the tonal range on screen.

You can't get to the latter from the former without performing a non-linear tonemapping.

If you want to render a scene with a linear tonemapping from 0-1, then you'd have to know, in advance, how to "invert" the tonal values of every element of the scene before rendering them, based on the final "exposure" you'd want for the scene.

Which is basically what an FX8 rendering algorithm that simulates HDR does

Jawed

 

[Hyp-X]

Hrm, I was pretty sure that step 2 was transparent to the software. That is, if a multisample render target is used, it will automatically be downsampled when read in as a texture.

Well it isn't transparent in D3D.
Not sure about GL.

The only 'transparent' thing is when you render to RT textures directly, but in that case there's no copying. (And no AA)

 

[Hyp-X]

[*] resolve AA samples (assuming gamma 2.2 rendering on screen) to generate AA'd frame

No, you shouldn't resolve AA using gamma correction since you are still in linear color space.
Gamma correction in AA resolving is optional on every card that supports it.
Of course since there's no API to turn it on/off the driver tries to 'figure out' what the developer wanted (at least on ATI cards).
This can lead to unexpected results...

 

[Jawed]

ATI cards resolve AA in linear space. It seems to me that NVidia cards resolve in screen space - hence the poor quality.

Jawed

 

[Hyp-X]

ATI cards resolve AA in linear space. It seems to me that NVidia cards resolve in screen space - hence the poor quality.

Jawed

Ok.
Wording.

I could say: nVidia cards resolve AA assuming the render target is in linear space, while ATI cards resolve AA assuming the render target is in sRGB.

Or I could say: nVidia cards resolve AA linearly (eg: (sample1 + sample2)/2), while ATI cards resolve AA with gamma correction (eg: pow((pow(sample1, 2.2) + pow(sample2, 2.2))/2, 1/2.2) ).

You say 'resolve in linear space' referring to the color space in which the addition is performed in - but that wording comes with the assumption that the buffers are in sRGB (which they might not be).

Anyway my point was that for ATI cards gamma corrected AA is optional.