About Image compression, streaming, the meaning of life...

Dio,
you're probably right about 'out of spec' thing, I get much more consistent results using the Ati tool.
To me, the paletted one has some nasty posterisation in places.
Can't say I see it without blowing up the image. But either way, if you think that's 'nasty', you better not look at how the more known tools mangle this image :)

andy said:
Generally speaking the palettised version is probably doing a better job of the background than the DXTC compression, but it's just not very noticable because of the noise.
It's perhaps where the disparity in MSE comes from also - like Dio said, it may be more wrong but visually noise is noise after all.
Are you sure that you posted the S3 image
The image I posted was using another S3 tool, not ATI one (I sent images to Archie before making the latter).

Squeak,
2bit textures are made by interleaving two of them into one 4bit image and use two different 4bit palettes to draw them (each having 2bits of the palette masked out). You can go down to 1bit textures this way too, but so far I haven't found any real use for them in games :p

Wouldn?t it just be a matter of simply not doing it on lower MIP levels, because colourspace requirements tend to shrink with texturesize?
You can do that, but at the cost of no longer using hardware mip-mapping - hw can't mix different texture formats across mip-levels.

Another thing, shouldn?t it be called Croma compression instead, because isn?t it the colour information that?s compromised?
Actually I don't know if it has official name yet :p
And well, quantizing luminance to 2bits is certainly not lossless so technically both components are stored lossy to some degree (how much exactly depends on the type of source image).
 
Faf, Archie, can you post a link to the best CLUT quantizer that you know of. Or is it a proprietary piece of software that can't just be downloaded for free?
 
Genghis Presley said:
Doh. I'm reporting the all channels MSE in after compressing and the all channels weighted MSE when doing the compare. I'll fix it to use to same in all cases but there may be a three hour conversation with a Russian mathematician as to which is the correct one to use.
GP.
AFAIK, use unweighted channel values. My understanding is that the assumption that blue is less important than the other channels is not really true. I believe it stems from the fact that the spatial resolution of the eye in blue is not as high as the other channels but that the eye is actually quite sensitive to blue levels. (Unfortunately I can't remember the source of this information).

The 'formula' I use for the RMS Error is:
Code:
#define SQ(X) ((X)*(X))

Error = 0.0;

for( y =  etc)
  for(x = 0 etc)
  {
     Error+= SQ( Src[y][x].Red - Apprx[y][x].Red) + SQ (..green) + etc;
  }

Error = sqrt ( Error / NumPixels);


Fafalada said:
Whee, so many things to reply to :p

Simon said:
Using the Dreamcast VQ compressor, this image gets a slightly better RMS score of 12.8 at an approximate cost of 2bpp. It does look a bit noisy though.
Funny you should mention this, as I've used your DCVQ tool to do this one :) I have another VQ compressor but it's both slow and not all that good most of the time, so I've stuck with yours.
/me does double take!!

I didn't want to go into numerical tests because I dithered both VQ and 8bit images which changes stuff like RMS score differently then it affects perceptual quality. I am also a little skeptical about the MSE number for the 8bit version of this image in particular.
If you have got VQGen.exe + VQdll.dll (the latter may have shipped with Sega's 'wrapper' program) ignore the RMS error it outputs - I clearly had a bug somewhere in the code and I'm not bothering to fix it now :)

The version for the DLL (which is the important bit) is 2.02.00.0002a, dated 21 May 2002. (MD5 Sum: 87df5f3af46bbe9aa44129a813908fba)

As for copyrights, the photo comes from one of those fansite galleries, so I doubt there's any.
Never assume anything. There is information on the web saying that the owners of the famous "Lena" image (i.e. Playboy) were allowing it to be used for CG research. I checked with their legal department and they categorically said this was not so. Not that they've actually benefited in anyway from research done using that image...

Btw, you wouldn't mind sending the fixed version of your new VQ tool over my way would you? 8)
We'll if it looks like you do have an old version drop me a PM and I'll see about sending the "latest" (!!).

BTW I'll have a go using the PVR-TC compressor with your temple image shortly - nice picture BTW.
 
Simon F said:
AFAIK, use unweighted channel values. My understanding is that the assumption that blue is less important than the other channels is not really true. I believe it stems from the fact that the spatial resolution of the eye in blue is not as high as the other channels but that the eye is actually quite sensitive to blue levels.
I found something similar compressing DXTC. Originally I assumed that finding compression axes in YCrCb would be best and it actually was very slightly worse in percieved quality and much worse in MSE. Arguably, there was a marginally better result perceptually if I used a munged colourspace of (R, G, (G+B)/2) but it was pretty close to call.
 
Fafalada said:
I still disagree with the assertion that you need to stay away from lots of colour detail though - for example:
http://homepage.mac.com/archie4oz/.Pictures/Fafalada/temple01.png
http://homepage.mac.com/archie4oz/.Pictures/Fafalada/temple02.png
http://homepage.mac.com/archie4oz/.Pictures/Fafalada/temple03.png
http://homepage.mac.com/archie4oz/.Pictures/Fafalada/temple04.png

Here we have source that has nearly 0.5 milion colors, and the quantised version of image doesn't use dithering at all, looks perceptually flawless, and has a MSE as low as 4.18.
Are you sure? According to my program the RMSE for the 8bpp palette texture is 8.46.

Also it seems like your S3TC is scoring ~18 whereas the S3TC compressor gives 10.9.

The VQ score seems about the same as the compression results on my PC.
 
Fafalada said:
2bit textures are made by interleaving two of them into one 4bit image and use two different 4bit palettes to draw them (each having 2bits of the palette masked out). You can go down to 1bit textures this way too, but so far I haven't found any real use for them in games :p

Couldn’t 1bit CLUTs be used for stuff like letters, the outline in cartoonish textures, or even glowing line patterns like in Ico?

Not that I don’t believe your explanation of course, but it seems odd to me, how the CLUT hardware can understand the 2bit texture data.
It doesn’t sound like a terribly “official†format, so I would be surprised if it was planned for by the designers? What I’m trying to say here is, won’t the CLUT hardware miss the remaining 2 bits somehow? Maybe it just skips them, but then you would still have to send the same amount of data, and nothing would be saved?

Otherwise I must say it’s seems like a really clever approach (not that I’m the best judge of that, though). It could help out a lot in bridging the gab to newer kinds of TC.
Do you by any chance know who first came up with the idea, and if there is any released games that use it to any extent?

I did a search on your “name†and luminance and got this: http://www.beyond3d.com/forum/viewtopic.php?p=156748&highlight=luminance#156748
Fafalada said:
Simon said:
I was refering to native compression but ignoring that, yes using multiple passes can emulate other texture compression schemes. What you describe sounds sort of YUV based; I could see that working to some extent.
One relatively simple method is to encode texture luminance as a high resolution 2-4bit map and combine it with low resolution high color map.
It tends to work best with maps with lots of colors (we use it for skymaps), and pretty handily beats straight quantization methods on those.
If I had somewhere to host I could give you samples of how it looks :p

Well, I know it’s lot to ask, but I think it could be fun to see how the multitexture approach compares to the four other compression methods, so if you have time and want to… :)

Fafalada said:
Simon said:
That would be an impressive hack (I'm not sure how you would do it), but surely rather time consuming?
I've got it to work at 2ops/texel for decompression (but it must use an intermediate decompression buffer), which is basically same cost as the above luminance trick or what Blue Shift team claims for their compression.
I know of one team that got some kind of VQ derivative to work at a smaller cost then that, but I haven't been told enough to figure out exactly how it works yet. It goes to 4-2/bits/texel too, but unlike my method it's not the same as VQ encoding that DC used.

Have you found out more about it now?

Simon F--> About the new compression scheme you are working on (have worked on?), are we going to see it in some actual product anytime soon? I know that releases of new PVRcards are far and few between, but in your paper http://web.onetel.net.uk/~simonnihal/assorted3d/fenney03texcomp.pdf you emphasized that one criterion for the technology was, that it should be suitable for low end stuff (portables, PDAs, mobile phones etc.(although 2bpp. with full alpha, is very impressive on any platform), so have you had any calls from SCE or Nintendo? :)

I know I asked this before, but I never did quite understand the underlying idea fully, so here goes:
From what I could see (and as explained in the first pages of the paper), the compressor works by extracting the luminance, and then the remaining colour information is downsampled.
But then, and here comes the part I don’t understand, the colour map is split in to two different kinds of map, but for what good?
 
Squeak, are you sure you are talking about that paper? The modulation signal will be quite well correlated with luminance, but that is more like a side effect. The main aim is to give each pixel a localized choice between 2 full colour approximations of the signal.
 
Anyways, here's chapstick's 4-bit CLUT (no diffusion) just for fun... Obviously the trees got hit pretty bad with the limited chromince values to work with... I tried to push more green but it ate too much into the red and killed the shadow detail...

http://homepage.mac.com/archie4oz/.Pictures/Fafalada/temple05.png

My understanding is that the assumption that blue is less important than the other channels is not really true. I believe it stems from the fact that the spatial resolution of the eye in blue is not as high as the other channels but that the eye is actually quite sensitive to blue levels.

Well the assumption comes from the fovea of the eye being largely dominated with red and green cones (as opposed to an even balance of all three), hence blue data is very coarse (and carries little luminance information).

As far as levels go, I think that's more of function of the brain than that of the eye. I'd say it's the dynamic range that blue gives you (or the lack there of if no blue) since it's wavelength peak is quite a bit more distant that green is to red (or blue), is probably the factor... Red obviously carries the most luminance data, and blue gives you more chromatic range... Green tends to become rather redundant (carries a lot of unecessary luminance you can extract from red)), and you can reconstruct more green from red and blue than you can of blue from red and green....
 
Squeak said:
Simon F--> About the new compression scheme you are working on (have worked on?), are we going to see it in some actual product anytime soon?
It's not my place to say when products will be available. It would entirely depend on those companies that are making MBX-based products.
...in your paper http://web.onetel.net.uk/~simonnihal/assorted3d/fenney03texcomp.pdf you emphasized that one criterion for the technology was, that it should be suitable for low end stuff (portables, PDAs, mobile phones etc.(although 2bpp. with full alpha, is very impressive on any platform), so have you had any calls from SCE or Nintendo? :)
I'm just a researcher/engineer - I leave that sort of thing to the marketing people!

I know I asked this before, but I never did quite understand the underlying idea fully, so here goes:
From what I could see (and as explained in the first pages of the paper), the compressor works by extracting the luminance, and then the remaining colour information is downsampled.
But then, and here comes the part I don’t understand, the colour map is split in to two different kinds of map, but for what good?
No, it doesn't try to extract "luminance", per se.

This is a bit of a gross simplification but what it does is find two "bounding" images, A and B. These are continuous (eg bilinearly or bicubicly expanded) low frequency images and so each only needs ~1/16th (4bpp) or 1/32nd(2bpp) of the colour data of the original image. A pixel of the original image usually lies very close to the line in colour space joining the corresponding two pixels from A and B.

The compressed data also contains a full frequency, but very low precision, modulation signal that basically blends these two bounding images together to obtain the decompressed result.

You might be thinking that the luminence is extracted because for 'natural textures', many corresponding pixels in A and B sometimes differ by a luminence component, but frequently this is not the case. In "Lena", for example, some pixels in A and B differed mainly in the blue channel.

Does that make sense to you?
 
Fafalada said:
FWIW here these are the PVR-TC results for the temple image:

4bpp 10.1 , 2bpp 16.7

I'm still far from happy with the compressor but it will have to wait for a bit.
 
Simon F said:
Squeak said:
Simon F--> About the new compression scheme you are working on (have worked on?), are we going to see it in some actual product anytime soon?
It's not my place to say when products will be available. It would entirely depend on those companies that are making MBX-based products.
I purposefully didn't mention MBX, because isn’t that suited for portable equipment only? Maybe the compression technology can be licensed separately like S3 did?
http://web.onetel.net.uk/~simonnihal/assorted3d/fenney03texcomp.pdf you emphasized that one criterion for the technology was, that it should be suitable for low end stuff (portables, PDAs, mobile phones etc.(although 2bpp. with full alpha, is very impressive on any platform), so have you had any calls from SCE or Nintendo? :)
I'm just a researcher/engineer - I leave that sort of thing to the marketing people!
I know, hence the smiley. Even if you did know, you would probably have signed an NDA agreement.
I know I asked this before, but I never did quite understand the underlying idea fully, so here goes:
From what I could see (and as explained in the first pages of the paper), the compressor works by extracting the luminance, and then the remaining colour information is downsampled.
But then, and here comes the part I don’t understand, the colour map is split in to two different kinds of map, but for what good?
No, it doesn't try to extract "luminance", per se.

This is a bit of a gross simplification but what it does is find two "bounding" images, A and B. These are continuous (eg bilinearly or bicubicly expanded) low frequency images and so each only needs ~1/16th (4bpp) or 1/32nd(2bpp) of the colour data of the original image. A pixel of the original image usually lies very close to the line in colour space joining the corresponding two pixels from A and B.

The compressed data also contains a full frequency, but very low precision, modulation signal that basically blends these two bounding images together to obtain the decompressed result.

You might be thinking that the luminence is extracted because for 'natural textures', many corresponding pixels in A and B sometimes differ by a luminence component, but frequently this is not the case. In "Lena", for example, some pixels in A and B differed mainly in the blue channel.

Does that make sense to you?
Yes it does, and thank you for the explanation!
I guess one of the keywords here is delta compression, and although I thought I knew what it was, I didn't. :oops: But I do now :D
london-boy said:
I'm still waiting for someone to tell me the meaning of life...

Edit: And don't say i'm off topic, cause it's in the title. :D
Actually the answer to the ultimate question, was already posted at the bottom of the third post on the first page.
 
Oh heck, I'll do another post later...

Marc, since Archie was too lazy to post it :p , here
http://www.anime.net/~sasami/ura/xpadiepro.html

Simon said:
BTW I'll have a go using the PVR-TC compressor with your temple image shortly - nice picture BTW.
FWIW here these are the PVR-TC results for the temple image:
4bpp 10.1 , 2bpp 16.7
Thanks, and I'll be happy to compare RMS results if I can get hands on the utility ;) As it's obvious by now ATIs tool uses a bit different formula to calculate the error :?

Never assume anything.
Yeah you're probably right. But then I had no intention of publishing papers using that image, I merely like to use something nice to look at for my tests :p

Squeak said:
Couldn?t 1bit CLUTs be used for stuff like letters, the outline in cartoonish textures, or even glowing line patterns like in Ico?
1bit stuff is very prone to aliasing, so it has limited use in 3d. Although in cartoony stuff it might be usefull yeah, but still not something that I think would be used enough to noticeably affect overall memory usage.

Not that I don?t believe your explanation of course, but it seems odd to me, how the CLUT hardware can understand the 2bit texture data.It doesn?t sound like a terribly ?official? format, so I would be surprised if it was planned for by the designers? What I?m trying to say here is, won?t the CLUT hardware miss the remaining 2 bits somehow? Maybe it just skips them, but then you would still have to send the same amount of data, and nothing would be saved?
I don't think you understood me quite right. It's just applying binary arithmetics, any CLUT hardware can use this.
Take 2 2-bit textures (2x1 pixels), in binary representation:
A)00, 11
B)10, 01

Now you interleave them into 4bit (A uses left, B right 2bits).
00 10, 11 01

It's just a regular 4bit texture 1x1pixel. Now you use it with two 4bit palettes, each filled with only 4different RGB values based on relevant 2bits.
For A 00XX-11XX, XX meaning all combinations of the other 2 bits.
For B XX00-XX11
Palette still has 16entries, but for example, 0000 = 0011 = 0001 = 0010 = 00 from A.
Etc.
Of course, A and B can just be two halves of one 2bit map that is double the width.

Do you by any chance know who first came up with the idea, and if there is any released games that use it to any extent?
Well afaik idea of separating image into luminance and chroma is pretty old. Don't know who first came up with this application in realtime though.

Anyway I'll post samples when Archie uploads them later today :p

Have you found out more about it now?
I still have some scriblles lying around of the stuff I was thinking to try, but it's been awhile since I had time to do research work :p
 
Fafalada said:
Simon said:
BTW I'll have a go using the PVR-TC compressor with your temple image shortly - nice picture BTW.
FWIW here these are the PVR-TC results for the temple image:
4bpp 10.1 , 2bpp 16.7
Thanks, and I'll be happy to compare RMS results if I can get hands on the utility ;)
Which utility do you mean?
As it's obvious by now ATIs tool uses a bit different formula to calculate the error :?
By the sound of the discussion between Dio et al, I'd say it won't be long till a fixed version of the Compressenator is released :).

Do you by any chance know who first came up with the idea, and if there is any released games that use it to any extent?
Well afaik idea of separating image into luminance and chroma is pretty old. Don't know who first came up with this application in realtime though.
Well separate Luma and Chroma (with lower chroma bandwidth) has been used in colour TV transmission since day one and that is definitely realtime :)
 
Simon F said:
Do you by any chance know who first came up with the idea, and if there is any released games that use it to any extent?
Well afaik idea of separating image into luminance and chroma is pretty old. Don't know who first came up with this application in realtime though.
Well separate Luma and Chroma (with lower chroma bandwidth) has been used in colour TV transmission since day one and that is definitely realtime :)

This is a very primitive example, but in old movie houses during the black-and-white era sometimes colored screens were put over the projector lense to give a colored effect. In a scene with a burning building a red screen would be used, a scene on an alien world would use a green screen, etc. In this case the chrominance of the original filmed scene is completely lost, and then artificially restored with the colored screen.
 
nobie said:
This is a very primitive example, but in old movie houses during the black-and-white era sometimes colored screens were put over the projector lense to give a colored effect. In a scene with a burning building a red screen would be used, a scene on an alien world would use a green screen, etc. In this case the chrominance of the original filmed scene is completely lost, and then artificially restored with the colored screen.

Fair enough example.

More relevant is that at least one old flight simulator only had textures which encoded only a "luma" component. This was then multiplied by a constant-per-polygon colour. It works OKish for a flight simulator's grass, trees, rock, and concrete but is pretty dull for games :)
 
Fafalada said:
Do you by any chance know who first came up with the idea, and if there is any released games that use it to any extent?
Well afaik idea of separating image into luminance and chroma is pretty old. Don't know who first came up with this application in realtime though.
Sorry, I should have been clearer.
I meant, who first came up with the idea of interleaving two CLUT images?
To me, it seems like one of those ideas that are very simple once know them, but are hard to get.

But anyway.
Simon F said:
Well separate Luma and Chroma (with lower chroma bandwidth) has been used in colour TV transmission since day one and that is definitely realtime :)
If we are talking public transmissions then yes, but if we are talking colour TV in general, I think Bairds mechanical television, from the nineteen thirties had a spinning translucent disc with the primary colours, thus no separate luma-channel.
 
Back
Top