About Image compression, streaming, the meaning of life...

[andypski]

From Simon's testing, it does seem possible that the Compressonator's MSE calculation is cream crackered (again).

You might be using an old version - the version I have (which is current, naturally) gives an MSE of 7.23 when compressing DXT1 with the standard options.

Fafalada - I was reading the other thread, and I was quite surprised to use an example like this (which is not really a classical 'texture' image) in this thread. I had assumed that in the other thread you were really talking about quite a narrow definition of textures rather than images in general - ie. something with a relatively limited colour range that is used to add texture to a surface, rather than something that holds large amounts of colour detail.

In the case of textures such as woodgrain or stone palettisation can be very effective, and might do better in terms of quality in some cases than DXTC. It should be noted these sorts of image do tend to be very good cases for DXTC as well, so although you might do a bit better in some cases you almost certainly wouldn't be getting anything like your money's worth out of the doubled storage cost for the paletted versions.

As a generalised solution over a wider variety of images DXTC will usually beat palettisation handily in terms of quality, and at half the storage cost.

 

[Dio]

From Simon's testing, it does seem possible that the Compressonator's MSE calculation is cream crackered (again).

You might be using an old version - the version I have (which is current, naturally) gives an MSE of 7.23 when compressing DXT1 with the standard options.

I worked it out. If you do the compression it tells me the MSE is 7.52. However, if you save it and reload it, it claims 3.63 instead. (FYI, on this scale 8bit is 4.59, that dreadful S3TC version was 6.56, and VQ was 7.85, so all about in line with Simon's results). The diff images look identical in both cases.

I'll file a bug report. Nobody let this little inconsistency put you off The Compressonator, which is extremely good at it's main job (i.e. compressing textures!).

 

[nobie]

Image quality with palletized textures is heavily dependant on the type of image being compressed. One advantage with DXTC is that it works well with basically any type of image. This is a worst case scenario for palletized textures:

source
DXTC
8-bit
4-bit

Now, a texture which is a very good case for palletized textures (their is a narrow range of colors):

source
DXTC
8-bit
4-bit

Here, even the 4-bit texture looks decent. Of course, even with a best case scenario, palletized textures are not good candidates for multitexturing or use in a shader program, which is why they have no future.

 

[Simon F]

This is a worst case scenario for palletized textures:

source

FWIW PVR-TC should return an almost 0% error with that one. ... Sigh, if only all images were like it

 

[cybamerc]

nobie:

> This is a worst case scenario for palletized textures:

Hmm... it can look better than your examples. Not accurate mind you but more visually pleasing.

 

[marconelly!]

Image quality with palletized textures is heavily dependant on the type of image being compressed. One advantage with DXTC is that it works well with basically any type of image. This is a worst case scenario for palletized textures:

I have just opened your worst case scenario in Photoshop 7, converted it to 8 bit, and got the resulting picture that is so much better than your 8bit example, that it's ridiculous. That's pretty sad when according to some here, Photoshop is quite poor tool for this kind of job

I'm really curious as to what you've used to make your 8bit conversion.

*edit* Same goes for 4bit conversion. Your version is much worse than what photoshop churns out.

 

[Dio]

Saying DXTC works well with any type of texture is a bit generous as well. Even I won't go that far. The usual problems are:

- cartoon-type images it is usually poor on (they are rarely used as textures so this isn't much of a problem)
- diagonal meshes (chickenwire-type textures) can be problematic
- ones with very gentle gradient fades (sky textures are the usual ones that get hit by this)
- smooth 'glow' effects can go a bit lumpy
- a white specular highlight near an edge of something coloured which fades to black can cause blocking artifacts (again this is rarely seen in textures; the lighting gets applied afterwards!)

It is at its best with highly noisy textures that don't change chrominance very quickly. These actually have quite poor MSE's with DXTC, but the difference just isn't visible (noise is noise - it doesn't matter if it's quite the right noise as long as the character of the noise is roughly maintained, which it is).

One suggestion is that developers should make all textures DXTC up until alpha test. At that point QA should feedback what doesn't quite work and that texture should be bounced to an alternate format (potentially reducing the number of mip levels to keep within memory budget).

 

[andypski]

FWIW PVR-TC should return an almost 0% error with that one.

Interesting - if you'd stuck with the bicubic upscaling I guess the error on this image would actually increase a bit compared to the bilinear upscale, although I guess it would probably be so small a difference that you wouldn't notice much.

So I guess cheaper is better... (in isolation)

... Sigh, if only all images were like it

What a wonderfully psychedelic world you must wish to inhabit...

 

[andypski]

Image quality with palletized textures is heavily dependant on the type of image being compressed. One advantage with DXTC is that it works well with basically any type of image. This is a worst case scenario for palletized textures:

...

Interestingly this also isn't a great case for DXTC (although not really bad) because the colour gradients cut across blocks diagonally, which can increase blocking artifacts significantly.

 

[Ty]

Who's that cute girl ?

(what ? what ?! I'm perfectly on topic )

That was my exact thoughts. Who cares about all this technobabble?!?! Check out the girl!! Wow!

 

[akira888]

Image quality with palletized textures is heavily dependant on the type of image being compressed. One advantage with DXTC is that it works well with basically any type of image. This is a worst case scenario for palletized textures:

I have just opened your worst case scenario in Photoshop 7, converted it to 8 bit, and got the resulting picture that is so much better than your 8bit example, that it's ridiculous. That's pretty sad when according to some here, Photoshop is quite poor tool for this kind of job

I'm really curious as to what you've used to make your 8bit conversion.

*edit* Same goes for 4bit conversion. Your version is much worse than what photoshop churns out.

Marco, is the image dithered? If so, that really tends to look bad when the texture is magnified and filtered. Found that one out the hard way a few years back I'm afraid.

Nota bene: If anyone knows the model's name please, please, please[/p] post it.

 

[Genghis Presley]

From Simon's testing, it does seem possible that the Compressonator's MSE calculation is cream crackered (again).

You might be using an old version - the version I have (which is current, naturally) gives an MSE of 7.23 when compressing DXT1 with the standard options.

I worked it out. If you do the compression it tells me the MSE is 7.52. However, if you save it and reload it, it claims 3.63 instead. (FYI, on this scale 8bit is 4.59, that dreadful S3TC version was 6.56, and VQ was 7.85, so all about in line with Simon's results). The diff images look identical in both cases.

I'll file a bug report. Nobody let this little inconsistency put you off The Compressonator, which is extremely good at it's main job (i.e. compressing textures!).

Doh. I'm reporting the all channels MSE in after compressing and the all channels weighted MSE when doing the compare. I'll fix it to use to same in all cases but there may be a three hour conversation with a Russian mathematician as to which is the correct one to use.

GP.

 

[Fafalada]

Whee, so many things to reply to

First thing's first, I think some have misunderstood the point I was trying to make. It was mainly a reply to "Clut & photos don't mix" comment made prior - this wasn't really about making a "competion" after all it would take much more then one test image to make that
I included other compressed versions as a point of reference that some might find interesting. I accidently made S3 look bad because I used an older version of M$s compressor(grabbed the first thing on my HDD), which wasn't really my intention - and it also made guessing a lot easier because of the too obvious block artifacts.

Now to the core of the stuff.

Who's that cute girl ?

IIRC her name is Aki Kawamura, a Japanese Idol.

The guesses by the gurus were of course correct, but then it wasn't that hard (If I wanted to make it difficult I'd choose a different image and add a few more exotic formats to the mix).

It does make that image a rather aggressive test for block-based compression, while not affecting palettised compression significantly

I used this as one of the test images while working on testing various compression schemes - the source may contain artifacts but it tends to work well to expose weaknesses in many compressors.
On that note, 8bit struggles pretty bad(normally I stay away from using dithering) with this image, I have better examples of how good it can perform with photo matterial (but I'll get to that later).

Using the Dreamcast VQ compressor, this image gets a slightly better RMS score of 12.8 at an approximate cost of 2bpp. It does look a bit noisy though.

Funny you should mention this, as I've used your DCVQ tool to do this one I have another VQ compressor but it's both slow and not all that good most of the time, so I've stuck with yours.
I didn't want to go into numerical tests because I dithered both VQ and 8bit images which changes stuff like RMS score differently then it affects perceptual quality. I am also a little skeptical about the MSE number for the 8bit version of this image in particular.
As for copyrights, the photo comes from one of those fansite galleries, so I doubt there's any. Btw, you wouldn't mind sending the fixed version of your new VQ tool over my way would you? 8)


Andy,

In addition to this the palettised image is clearly relying on dithering to improve the apparent image quality, which is fine when viewed at 1:1 pixel:texel ratio, but when the same image is bilinearly enlarged (as typically happens with textures) the dithering naturally becomes highly noticable

You're correct of course, but this image was not a particularly favourable scenario for Clut to begin with.
I still disagree with the assertion that you need to stay away from lots of colour detail though - for example:
http://homepage.mac.com/archie4oz/.Pictures/Fafalada/temple01.png
http://homepage.mac.com/archie4oz/.Pictures/Fafalada/temple02.png
http://homepage.mac.com/archie4oz/.Pictures/Fafalada/temple03.png
http://homepage.mac.com/archie4oz/.Pictures/Fafalada/temple04.png

Here we have source that has nearly 0.5 milion colors, and the quantised version of image doesn't use dithering at all, looks perceptually flawless, and has a MSE as low as 4.18.
Btw, the mix of formats is the same as before, but they aren't in same order, anyone wants to make guesses again?

Incidentially the S3TC compressed with ATI tool for this image has a worse MSE(11.10) then the S3 version I actually posted(8.30), but I think it looks perceptually better - didn't have time to upload it though as Archie isn't around right now. The source image in this case is much better quality though (and it's a photo I took personally).
Maybe if I am up to it later I can post a Luminance compressed texture (~2bpp) for this sample too (previous pic was absolute worse case scenario for that compression so I skipped it).


Phew, and I still wanted to post about streaming -_-

 

[Fafalada]

archie4oz, it was already covered that Halo doesn't stream its level in continuously like, for example, Jak2. It puts all the separate sections of a level on to the hard drive and then swaps the section currently in memory when you hit a checkpoint.

It could be argued that doesn't really streaming as such though - it's just loading whole levels on a paralel thread to the main game.
JetSetRadio did the same thing way back, and with DC's "slowish" optical drive, no less. They had loading areas between city parts (disguised as tunnels and such) but the game was never interrupted - there wasn't even a pause.

Either way, I just wanted to mention that there's a whole slew of games on PS2 that do continous streaming of level data and seamless worlds - games like SSX3, Primal, NFS:Underground... just to name a few.
NFS:U in particular is interesting because the levels aren't stretched very far out and you don't get to hide streaming parts with LOD or clever level design(like Jak1 did for instance). On the flipside, the game starts looking pretty funny if you run it off a bad/scratched disc

Of course HDD makes things easier, but didn't the original statement say that seamless worlds of that sort shouldn't be feasible at all without it?

 

[Dio]

dxtc, source, vq, palettised
That image falls into the category I described above. The trees in the background are highly noisy, and DXTC munges the noise making it different but even with a side by side there isn't one that's right. As I said - high MSE, but no image quality difference. To me, the paletted one has some nasty posterisation in places.

However, there seems to be a significant 'sharpening' of the image (which shouldn't happen with the S3 reference compressor). It may be that the decompressor used isn't quite to spec (which the one in the Compressonator is) and that explains the 'perceptibly better' bit you observed.

With that fixed, I would hope it looks great.

 

[Thowllly]

Image comparison 2:

temple01=#st3c#
temple02=source
temple03=#VQ#
temple04=#8bit#

 

[Lazy8s]

Fafalada:

It could be argued that doesn't really streaming as such though

Right, the term was being used loosely to bring Halo's method for extending level size into comparison with games that are actually streaming.

Of course HDD makes things easier, but didn't the original statement say that seamless worlds of that sort shouldn't be feasible at all without it?

I think the original statement could reasonably be interpretted as being within the context of 'Halo's sort' and the features it used the HDD for, such as continually keeping track of every grenade/item/etc, and recalling the current level after having quit without a loading screen (even after turning the machine off).

OK, for the compression example of the temple photo, I guess:
*
***spoiler guess (still don't know how to shrink my text)***
*
*
*
*
8-bit palletized, source, DCVQ, DXTC
*
*
*

 

[marconelly!]

Marco, is the image dithered? If so, that really tends to look bad when the texture is magnified and filtered. Found that one out the hard way a few years back I'm afraid.

Yes, it was dithered, but his 8bit example was dithered as well. That was the the only way I could compare them on an equal footing.

 

[andypski]

I still disagree with the assertion that you need to stay away from lots of colour detail though - for example:
http://homepage.mac.com/archie4oz/.Pictures/Fafalada/temple01.png
http://homepage.mac.com/archie4oz/.Pictures/Fafalada/temple02.png
http://homepage.mac.com/archie4oz/.Pictures/Fafalada/temple03.png
http://homepage.mac.com/archie4oz/.Pictures/Fafalada/temple04.png

Here we have source that has nearly 0.5 milion colors, and the quantised version of image doesn't use dithering at all, looks perceptually flawless, and has a MSE as low as 4.18.

It does look pretty good.

However...

Although there are a lot of colours in the image it appears, from visual inspection, that they are really clustered into just a few groups in the colourspace, which is actually not too bad a case for palettisation at all. As such the actual range of truly different colours in the image is not really that great, and you get what you expect from the palette - some loss of subtle variations.

Looking at the pattern on the end of the large cylinder, and at the more subtle changes in red hue across the pillars, the limits of the palettisation are still fairly clear, even without blowing up the image.

Secondly, although the image is not dithered as such, the background area of trees is already simply a wash of high frequency noise, so even a large loss of colour fidelity would be masked by this noise. This effect benefits all three compression schemes here. Generally speaking the palettised version is probably doing a better job of the background than the DXTC compression, but it's just not very noticable because of the noise.

DXTC also has problems with the roof - this is a bad case for the compressor because the colours are cutting through blocks at a diagonal, and things can therefore get quite blocky, but again, because the variations are high-frequency, even relatively large errors in the compression turn out to be ok visually.

Conversely, on areas of relatively constant colour such as the pillars the lack of high-frequency noise will tend to make problems with the compression schemes much more visible. Here DXTC does particularly well because it's good at compressing low frequency colour variations.

It's an interesting image, as it allows us to point out various strengths and weaknesses in each technique - one key reason why DXTC is generally preferrable for texture compression over palettisation is that usually the cases where it has problems are relatively less noticable than the areas where palettisation can have problems - witness the differences in the background compression vs. the compression of the red areas.

Btw, the mix of formats is the same as before, but they aren't in same order, anyone wants to make guesses again?

temple01 - DXTC, temple02 - Original, temple03 - VQ, temple04 - Palette

Incidentially the S3TC compressed with ATI tool for this image has a worse MSE(11.10) then the S3 version I actually posted(8.30), but I think it looks perceptually better - didn't have time to upload it though as Archie isn't around right now. The source image in this case is much better quality though (and it's a photo I took personally).

Are you sure that you posted the S3 image, because I recompressed it here with that compressor and the results are quite different from your image - the S3 compressor seems to do a lot better on the red areas that I mentioned than the compressor that you used. I'll try and check up and see what's going on.

 

[Squeak]

Re: About Image compression, streaming, the meaning of life.

Regardless, working with a texture pool sized 10-15MB per scene, I can tell you from personal experience that we were always juggling between using either lower resolution or more of 4bit (and this includes the use of JPeg and 2bit Luminance compression for certain texture types) - and the latter usually prevails.

2bit Luminance? Are we still talking PS2 here, or is it some exotic other hardware that has 2bit CLUTs ?
No matter how you blend two 4bit textures, you will always end up with +4bits per texel, no?

I vaguely remember you writing something about this compression method before, about it not being compatible with MIP-mapping.
Wouldn’t it just be a matter of simply not doing it on lower MIP levels, because colourspace requirements tend to shrink with texturesize?

Another thing, shouldn’t it be called Croma compression instead, because isn’t it the colour information that’s compromised?