2-bit texture compression

[aths]

This is not finished work, but I think I finally found an approach.

4-bit TC delivers very good quality. Due to the 2^n addressing scheme, the next step must be 2-bit.

Previous 2-bit techniques (I read the work of Simon Fenney) either reduce the number of used color gradients per tile from four to two, or reduce the spatial resolution with a checkboard pattern.

The PACKMAN approach uses just one color per tile (however per 4x2 tile instead of 4x4 tile.) My approach also uses just one color, but 4x4 tiles like S3TC. To get smooth color gradients, color (CbCr) should be bilinear interpolated among the tiles. For photographs, this actually works better than one thinks.

First Idea: 12 bit color, 3 bit multiplier (this is also a rip-off from PACKMAN) and 1 bit direction. The direction indicates either 4x1 or 1x4 texel subtiles. For each subtile, a 4-bit code represents one of 16 possible wave-forms. That wave-form is multiplied by the multiplier and applied to the color.

Second Idea: Use 2x2 subtiles instead. The direction bit can be used to choose from two sets of 16 2d-waveforms (each 2d-waveform of course is a 2x2 brightness modulation tile.)

Third Idea: Combine both approaches with 12 bit color, 2 bit multiplier and 2 bit direction/waveform set.

The idea behind this approach is to show little respect to single bright (or dark) texels which would be lost. That 2-bit approach would still be OK to encode the general shape. Because it needs hard-wired wave-forms which can be addressed with the 4-bit codes per subtile, an implemention is not too cheap but allows truecolor texture compression with just 2 bits per texel.

 

[MfA]

Oops,nevermind ... read that wrong.

I think oscillating basis functions are overrated and you should be more concerned with edge quality. The advantage of S3TC and PVR-TC is that they can do sharp edges without lots of mosquito noise because they both have two colours to work with ... in theory you also have more than 1 colour to work with (the one from the present block and the ones from neighbouring ones). Only using that for bilinear interpolation seems like a waste.

 

[aths]

I think oscillating basis functions are overrated and you should be more concerned with edge quality. The advantage of S3TC and PVR-TC is that they can do sharp edges without lots of mosquito noise because they both have two colours to work with ... in theory you also have more than 1 colour to work with (the one from the present block and the ones from neighbouring ones). Only using that for bilinear interpolation seems like a waste.

One of the 2-bit PVRTC modes is just decreased spatial resolution. You could use 4-bit TC and stretch a smaller texture over the same area to get "2-bit" with downscaling each texture dimension by factor 1.414. The other one uses only two brightness levels (or different colors) for a 4x4 tile.

I consider the option of many brightness levels more important than good spatial color resolution. With 2-bit TC you can store a texture with higher resolution in the same amount of memory, making each 4x4 tile smaller. This somewhat reduces the issue of having only 1 color per tile.

Mosquito noise should only be a visible issue if the texture is rendered with magnification. Texture compression should allow to store high-res textures which are not beeing magnified. No matter how good the set of the predefined waveforms is, my approach would kill some detail but this should often be compensated by allowing higher base texture resolution due to a smaller memory footprint.

Storing two colors per tile like S3TC seems like a waste to me because it is likely that a similar color is used for a neighbour tile. The waveform approach can still be expanded to use two colors:

7 bit for first color, 7 bit for second color, 2 bits for direction (horizontal, vertical, 2x2 set 1, 2x2 set 2.) Each 4-bit waveform now is a predefines set of how much each color is weighted in.

I even considered to store only Cb *or* Cr per tile in a checkboard pattern and calculate the other color component as the average value from the neighbours. This would allow 5 bits for each 4x1/2x2 subtile. However this Cb/Cr split could lead to strange color bleeding.

 

[MfA]

Storing two colors per tile like S3TC seems like a waste to me because it is likely that a similar color is used for a neighbour tile.

Yes I agree, that's why I said you already have more than 1 colour to work with ... but you are limiting that work to static bilinear blending.

 

[Simon F]

This is not finished work, but I think I finally found an approach.

4-bit TC delivers very good quality. Due to the 2^n addressing scheme, the next step must be 2-bit.

Not necessarily. The original experiments in PVRTC (not sure it was mentioned very precisely in the paper) was to have 2.5bpp. IIRC, the colour channels were scaled by a factor of 8x but using bicubic interpolation, with 2bits/pixel of modulation. That was not used as it didn't fit the tight gate budget of MBX.

Previous 2-bit techniques (I read the work of Simon Fenney) either reduce the number of used color gradients per tile from four to two, or reduce the spatial resolution with a checkboard pattern.

Well, it can switch between them on a region-by-region basis, but supporting 2bpp that fits nicely in with the 4bpp mode wasn't trivial.

First Idea: 12 bit color, 3 bit multiplier (this is also a rip-off from PACKMAN) and 1 bit direction. The direction indicates either 4x1 or 1x4 texel subtiles. For each subtile, a 4-bit code represents one of 16 possible wave-forms. That wave-form is multiplied by the multiplier and applied to the color.

I also experimented with transform coding (DCT/Hadamard etc) and only storing the significant coefficients, and I found that, while it works fine for photographs etc, as soon as you try something that is more "graphical" the results weren't pretty.

Anyway, good luck with the experiments.

 

[aths]

Yes I agree, that's why I said you already have more than 1 colour to work with ... but you are limiting that work to static bilinear blending.

Yes because I have no idea how to compress with references to neighbour tiles colors. This would include some re-encoding of existing tiles to get an optimal result.

Not necessarily. The original experiments in PVRTC (not sure it was mentioned very precisely in the paper) was to have 2.5bpp. IIRC, the colour channels were scaled by a factor of 8x but using bicubic interpolation, with 2bits/pixel of modulation. That was not used as it didn't fit the tight gate budget of MBX.

In my opinion, cubic interpolations raises more issues than it solves; but I have to admit that some of my "philosophical" arguments why cubic interpolation does not work are disproved by actual results.

I had good success with color downsampling using bilinear CbCr interpolation. Compared to the original image it looks more grayish but if you don't know the original copy you don't miss the color details.

At least to my eyes, brightness information per pixel is much more important than spatial color resolution. With 7 bit for Y and only 1x CbCr per 4x4 tile (with 8 bit per C component) we already have 3:1 "compression" from the truecolor original. The bilinear interpolation of CbCr eliminates the color tile structure. That even works for sudden color changes like with photographs of the face of Christina Aguilera with her signature all-red lips. I got the idea to interpolate colors from your work and have to agree that this work works.

Well, it can switch between them on a region-by-region basis, but supporting 2bpp that fits nicely in with the 4bpp mode wasn't trivial.

I also experimented with transform coding (DCT/Hadamard etc) and only storing the significant coefficients,

Storing coefficents probably works only with larger tiles, so I had the idea of predefined modulation waveforms.

The secret behind fixed-bitrate image compression is to encode very similar looking tiles with the same bit sequence, leaving room to encode reasonable different tiles. In other words, to find a mathematical transformation which delivers an even distribution of all "meaningful" possible tiles over the bits, sorting out as much as "nonsense" or "less helpful" looking tiles as possible.

This is a very far-fetched idea: Analyze a lot of image and its brightess modulations per 4x4 tile to find out the ones which are used a lot. Then generate a logic circuit and optimize it for transistor count even if this means that some of the transforming efficiency is lost.

and I found that, while it works fine for photographs etc, as soon as you try something that is more "graphical" the results weren't pretty..

I concluded my diploma thesis that S3TC delivers that good all-round texture compression for 4 bits per texel that the only future work for 4 bbt TC should be S3TC encoding and that 2 bbt is not possible without great image quality sacrifices. If I only could prove me wrong.

Anyway, good luck with the experiments.

Thanks; but I currently don't have time to experiment with modulation waveforms.

By the way, I also have some ideas to compress framebuffers with using 2x supersampling but storing only one color channel for each subsample. So a full downsampled pixel has full color but a subpixel only has Cb or Cr. This will generate weird colors on edges but the correct brightness is far more important. With the VSA-100 LOD Bias trick (-0.25 for one subpixel, +0.25 for the other) for Y (brightness) you also can get "half trilinear" filtering with 2x bilinear filtering TMUs. 2x supersampling also gives nice improvement options for interlaced monitors.

 

[aths]

I hope it's ok I use a second post instead of adding to the previous to improve the clear view.

For 4x1 predefined waveform it is obvious that we need these three pieces. (L = lower brightness, H = higher brightness.)

L L L B
L L B B
L B B B

And all the way back

B L L L
B B L L
B B B L

We also need a subtile for no change (N)

N N N N

and all lower

L L L L

and all brighter

B B B B

Now we already used 9 out of 16 options. The remaining possible subtiles can be used to encode S curves and U / reversed U curves. Again, the idea is to show little respect to single texels with high or low brightness.




For 2x2 wave tiles we have 16 possible forms, too. 8 forms are used up to encode "go brighter" in each of 8 directions (left right up down and all 4 diagonales), we also need 8 to encode "go darker" in all those directions.

Since there is no bit left to encode "no change from base color" we must have the option to pick from two subtile sets per 4x4 tile and reduce the possible shade-directions of each one.

 

[aths]

[]Reply to a strange general S3TC explanation[]

There is actually only one real S3TC algorithm with some additional variations or extra alpha channel compression. ATI's 3Dc uses two of those alpha channel compression approaches to encode a 2-channel texture/map.

I am currently only examine the compression of coloured images. To get a compression twice as high as S3TC ("DXT1") I try to discuss an approach using just one color (like PACKMAN) and with interpolating that base color over the tiles (as proposed for PVRTC) combined with the idea to use prefefined brightness modulation wave-forms for sequences of texels.

 

[AlexV]

Pretty sure that was a bot you just replied to. I guess we'll soon find out if I'm right

 

[aths]

I also have a 2-bit TC for alpha channel.

6 bit average alpha value, 2 bits multiplicator. The 4x4 tile is devided into 8 subtiles of 2x1 texels each. For each subtile we use 3 bits. These 3 bits allow some modulation: higher, lower, more alpha, less alpha. Those modulation is multiplied with the multiplier and then applied to the average alpha value.