chrominance subsampling in gbuffer albedo

[joysui]

Hi everyone ,
I want to know how to implement the gbuffer albedo "chrominance subsampling" technique which is used in the CRYSIS3, the article "The Rendering Technologies of Crysis 3" has metioned that. I know the theory about this, but I can not figure out how to do it when I generate the Gbuffer,Can someone help me? thanks!:smile:

 

[traptd]

On their slides it says that CbCr is interleaved, therefore I assume that they store Cb for odd pixels and Cr for even pixels, and they decide on what to store by looking at gl_FragCoord/HLSL equivalent.

 

[joysui]

hi traptd, thanks for your reply, you mean when I fill the Gbuffer via the PS, I use gl_FragCoord(opengl) or VPOS(D3D) to select the right pixels which store the Cb and Cr for this pixel,right? If the pixel is odd I save the Cb info of this pixel, and if the pixel is even, I save the Cr info of this pixel,right? Ok,if it is possible( I dont test it ), then when I decode this info from Gbuffer, eg, if I encountered a odd pixel, I know it stores the Cb info of itself,and the right neibour stores the Cr info of its right neibour, so I directly fetch the Cb info from it, and fetch the Cr info from its right neibour, right? finally do the convertion for itself.

 

[sebbbi]

G-buffer rendering:
Yes, you should use VPOS (DX9) / SV_Position (DX10/11) input semantic pixel shader, to get the screen pixel coordinates. Depending if the screen_pos.x is even or odd, choose the Cb or Cr data for that pixel. Perform exactly the same instructions for even/odd pixels (do not write a large branch, because that branch will be guaranteed to be divergent, and it hurts the performance). Add branch-less select or [flatten] directive (or equivalent on your platform) to make sure that the Cb/Cr selection line doesn't get compiled to a branch (afterwards look at the shader microcode to be sure).

Lighting or combine pass:
If you are using tile based deferred rendering, you need to gather Cr/Cb back only once a pixel. For geometry based (stencil masked) deferred rendering, you need to do it several times (once per overlapping light). Alternatively you can combine the result later. Use similar VPOS / SV_Position trick to determine odd/even pixels. To fetch the other of the Cr/Cb channels, for even pixels do UV.x+=one_pixel, for odd pixels do UV.x-=one_pixel. Use branch-less select (or [flatten]) and check the resulting microcode so that you do not get a dynamic branch (it would be always divergent).

 

[joysui]

thank you very much! I tried it in a demo, and yes it works, but the visual effect is not good, I think there is something I need to solve, I dont use branch and the code is here, maybe some error? :
Encode :

float Y = 0.299 * oCol.r + 0.587 * oCol.g + 0.114 * oCol.b;
float Cb = 0.5 + ( -0.168 * oCol.r - 0.331 * oCol.g + 0.5 * oCol.b );
float Cr = 0.5 + ( 0.5 * oCol.r - 0.418 * oCol.g - 0.081 * oCol.b );
oCol.r = Y;
oCol.g = fmod( screenSpace.x,2.0 ) * Cb + ( 1 - fmod( screenSpace.x,2.0 ) ) * Cr;


Decode :

float4 color = tex2D( g_samScene, Tex );
float Y = color.r;
float Cb = fmod( screenSpace.x,2.0 ) * color.g + ( 1.0 - fmod( screenSpace.x,2.0 ) ) * color.g; // even : Cr, odd : Cb
float invWidth = (-1.0) / 640.0f;
float2 offset = Tex + float2( ( fmod( screenSpace.x,2.0 ) * 2 - 1 ) * invWidth, 0 );
float Cr = tex2D( g_samScene, offset ).g;
float2 CbCr = float2( Cb,Cr );
float2 dotP = float2( fmod( screenSpace.x,2.0 ), 1.0 - fmod(screenSpace.x,2.0) );
float CR = dot( CbCr.xy, dotP );
float CB = dot( CbCr.yx, dotP );
float R = Y + 1.402 * ( CB - 0.5 );
float G = Y - 0.344 * ( CR - 0.5 ) - 0.714 * ( CB - 0.5 );
float B = Y + 1.772 * ( CR - 0.5 );

the code is not nice . but the function is here.

 

[Ethatron]

Use gather and process in 2x2 quads, safes you lots of hassle and bandwidth.

 

[joysui]

Ethatron, thanks for your reply, Yes but I implement it via SM3 not SM4 plus, I wonder if I can use via SM2.0 hardware.Thanks.

 

[Laa-Yosh]

Clever as it is, I hope they'll drop this compromise on nextgen consoles...

 

[sebbbi]

Clever as it is, I hope they'll drop this compromise on nextgen consoles...

Why?

Blu-ray is 4:2:0, and that is half the chroma resolution compared to Crytek's method (4:2:2). I have heard no complaints about Blu-ray image quality. Also many HDTV sets do not support 4:4:4 (full resolution) chroma (esp. those HDTVs with frame interpolation, according to AV forums). So the extra chroma information gets lost either way.

Full res chroma is required for rendering small (colored) text (and thus supported by all PC monitors), but console games are not using small (8-15 pixel) fonts. And the game developer can of course render the UI without chroma subsampling if that's the case. But that's not a good idea since many HDTV sets still downscale the chroma before displaying the image. Console TCRs usually mandate that fonts must be large enough to be clear on all kinds of TVs.

I would prefer higher resolution or better frame rate instead of full resolution chroma.

This is a good image to visualize how hard it is for the human eye to see small chroma details. Luminance on the other hand must be pixel perfect to look good.

 

[Laa-Yosh]

I'm an offline CG asset guy. Every bit of quality loss hurts me

Don't get me started on how Gametrailers or Youtube butchers our stuff...

 

[sebbbi]

Don't get me started on how Gametrailers or Youtube butchers our stuff...

I know... I am the 60 fps guy. Most online video services gap the frame rate down to 30

 

[Pavlos]

You can find many details about this technique, including a simple WebGL demo with source code, on this page: http://www.pmavridis.com/research/fbcompression/

The encoding part of your code looks correct, but the decoding needs more attention. If you just copy the missing chrominance from the nearest pixel (as your code does), you will get visual artifacts at polygon edges and other areas with strong chrominance transitions.

One solution to this problem is to use an "edge-directed" reconstruction filter, as the one described in the link above (the paper includes simple GLSL code). However, I'm not sure if Crytek uses the same solution, since they don't provide many details in the presentation.

Btw, we have published this technique last year in JCGT, after getting some feedback from game developers, but I was not aware that it got used in actual games. Thank you for bringing this to my attention!

 

[Ethatron]

Ethatron, thanks for your reply, Yes but I implement it via SM3 not SM4 plus, I wonder if I can use via SM2.0 hardware.Thanks.

Not all, just ATI, and you have to separate channels into different textures, then you can use Fetch4. Probably not very practical in general.

Anyway, if you want to improve quality you can upsample the chroma-planes at least linearly:

* - * -
- a - *
b X c -
- d - *

You need to form X from abcd, I use a quincunx layout here as it improves quality, odd = (x ^ y) & 1. Otherwise you can do this:

* - * -          a - b -
a X b -    or    c X d -
* - * -          e - f -
* - * -          * - * -

The left is context-free, the right could for example measure the slope of ab and ef or af and eb, and lerp cd according to a constructed weight.

 

[joysui]

Pavlos,Thanks for your reply and valuable information,the work you done is very helpful to me,thanks again.
Ethatron, Thank you for your advices and what you metioned is similar to the work Pavlos has done, yes, you both right, now I know I should improve it via some filtering method.

 

[Bryant]

Subsampled screenspace effects, subsampled render targets, and now subsampled chroma on a video game? It's bad enough digital video uses chroma subsampling when digital compression is efficient enough to not require it, but now video games? When is this stuff going to stop?

 

[milk]

Graphic programers don't go through all this trouble to get their game looking worse. Technology is made of trade-offs, and if devs end up choosing chroma subsampling is because the savings outweight the compromises. Considering most modern renderers do everything on pretty fat linear space HDR buffers, you are not really loosing that much. Its subsampling of a supersampled buffer in some ways.

 

[Bryant]

Regardless of the resolution, the subsampling will still let errors creep into the final buffer.

 

[Novum]

I couldn't even tell if an image is chroma subsampled if I would know without searching a while. Human vision is far more sensitive to contrast than to slight color changes. This has been known and used in video codecs since ages and nobody ever complains.

 

[Ethatron]

Depends. 4:2:2 JPEG subsampling is quite visible. Chroma bands need upto twice the precision of the luminance band for the same bandwidth in various YCC codings, so you get aggregate effects from loss of precision + loss of resolution.
Said that, the approach per se isn't too problematic, it's the quality of the implementation which counts most. And the JPEG one is kind of bad, as is the MPEG one, the h.264 is much better, as is the Crysis 3 one.

 

[HAL-10K]

Regardless of the resolution, the subsampling will still let errors creep into the final buffer.

Why do you have an opinion about a subject where you don't understand basic principles and don't care to think about what people say?

Resources are limited, thus exploiting physiological properties like the eye's lower chromatic resolution makes it possible to INCREASE graphics quality.