Next-Generation NVMe SSD and I/O Technology [PC, PS5, XBSX|S]

[davis.anthony]

Oodle plus LZ can be done in hardware. Most folks consider Zip to be LZ.

I hope you won't go off into another in the weeds discussion because you're looking to pick fault with common vernacular.

I didn't realize Oodle texture could be handled by XSX's hardware decompressor, when was it confirmed?

 

[BRiT]

I didn't realize Oodle texture could be handled by XSX's hardware decompressor, when was it confirmed?

You seemed to be focused on LZ vs Zip, so I was commenting about that aspect. Not sure of Oodle aspect, but its been used on last gen Jaguar cores without detriment. So I should probably edit that post.

 

[davis.anthony]

You seemed to be focused on LZ vs Zip, so I was commenting about that aspect.

No I wasn't. That's not the center point of the discussion I was having or the center point the post of mine you replied too.

 

[BRiT]

No I wasn't. That's not the center point of the discussion I was having or the center point of reply of mine you replied too.

Well that's how your post came off to me. Sorry.

 

[function]

I didn't realize Oodle texture could be handled by XSX's hardware decompressor, when was it confirmed?

Oodle happens before zip compression. It's a way of preparing textures for block compression.

This confirms you don't actually understand the things you're console/platform warring about.

Dude sort yourself out. You even linked to the stuff that told you this and showed compression rates.

It's a difficult job for mods to differentiate between honest mistakes and concerted platform warring. Fortunately, you're doing a good job of making your intentions clear.

 

[dobwal]

No, they say the average compression ratio achieved in games using both Kraken and Oodle texture is 2:1. The 3.x figure you quote was a single texture set example. The blog is very clear what the average ratio is and that it results in an 11GB/s average throughput.

Kraken compression is typically 2:1. Oodle Texture is a BCN encoder that performs better than the typical 4:1 - 8:1 compression of basic encoders. Oodle has a RDO step that allows you to trade off more compression for less quality by looking for areas in the texture where the loss of quality is less perceivable. You oodle first then you kraken. LOL

 

[iroboto]

Oodle is separate from Kraken. you can Oodle and zlib and get similar compression sizes to oodle+kraken. But kraken will decompress faster; at least that is my general take on it.

 

[dobwal]

Oodle is separate from Kraken. you can Oodle and zlib and get similar compression sizes to oodle+kraken. But kraken will decompress faster; at least that is my general take on it.

Yes. And I think Microsoft process is similar where BCPack is just MS's version of Oodle. And performance comparison between the two would be a question of quality versus compression and not compression rates alone.

 

[pjbliverpool]

No, they state:

The blog is very clear, in an actual real life game (Not some demo made to bring the best results possible) they achieved a 3.16:1 compression ratio using Kraken+Oodle texture

Go and ask them, they're very open on the blog about answering questions.

http://cbloomrants.blogspot.com/2020/09/how-oodle-kraken-and-oodle-texture.html

It's literally right there in your own link.

Sony has previously published that the SSD is capable of 5.5 GB/s and expected decompressed bandwidth around 8-9 GB/s, based on measurements of average compression ratios of games around 1.5 to 1. While Kraken is an excellent generic compressor, it struggled to find usable patterns on a crucial type of content : GPU textures, which make up a large fraction of game content. Since then we've made huge progress on improving the compression ratio of GPU textures, with Oodle Texture which encodes them such that subsequent Kraken compression can find patterns it can exploit. The result is that we expect the average compression ratio of games to be much better in the future, closer to 2 to 1.

That is absolutely as clear as day. The 3.16 to 1 example given is for a single texture set from a single game. Not the entire game content which would be lower on average. That particular set was likely chosen as a good example of a highly compressible workload to highlight the differences between the different compression methods.

 

[davis.anthony]

It's literally right there in your own link.

That is absolutely as clear as day. The 3.16 to 1 example given is for a single texture set from a single game. Not the entire game content which would be lower on average. That particular set was likely chosen as a good example of a highly compressible workload to highlight the differences between the different compression methods.

You do know my original comment back a few pages wasn't comparing average for a whole game right?

There's a lot selective reading and selective picking up of comments from quite a lot of members in this forum.

No wonder this place struggles to attract and retain new members.

 

[davis.anthony]

Oodle happens before zip compression. It's a way of preparing textures for block compression.

This confirms you don't actually understand the things you're console/platform warring about.

Dude sort yourself out. You even linked to the stuff that told you this and showed compression rates.

It's a difficult job for mods to differentiate between honest mistakes and concerted platform warring. Fortunately, you're doing a good job of making your intentions clear.

You sort yourself out, I fully understand what happens but being constantly back in to a corner by people selectively reading what they want is enough to cause anyone to stumble.

So no, you and few others need to sort yourselves out.

 

[Allandor]

XSX's hardware decompressor doesn't have support for Kraken, so Oodle+Zip would need to be done on the CPU (or GPU) taking away resources, so will it ever be used? Where as on PS5 it is supported by it's decompression hardware.

There's no agenda at all, just people looking for anything to pick fault with.

Again and again. Oodle texture is pre-data-compression stuff. You can store the textures in this format on disk or even in the GPUs memory. Oodle texture is a texture compression (or combines many) with data-loss (just like any other texture compression format). Oodle texture also prepares the texture in a way that is is better compressable for the data-compression (with a hopefully minimal sacrifice on texture quality). The hardware inside of the xbox and PS5 just decompress stuff compressed with data-compression. Those are lossless formats (like Zip, LZ, kraken, ...). So you can use oodle texture in combination with any other data-compression, this is just a licensing thing.
The data-compression in the end won't make textures magically much smaller 3:1 as this is just content-dependent. In general kraken should be a bit better that zlib, but not by much. The texture compression format is way more important.
If games go the way of small artifacts, the difference in data-compression will be even less important as many smaller files will loose some effectiveness vs a few big library files. Therefore the SSD-only concept allows only to read what is really needed which saves duplicates and therefore in the end should be smaller. Which on the other hand will be sooner or later eaten up by the larger texture sizes the new hardware and memory allows.

 

[Shifty Geezer]

You sort yourself out, I fully understand what happens but being constantly back in to a corner by people selectively reading what they want is enough to cause anyone to stumble.

So no, you and few others need to sort yourselves out.

This can often happen in public discussions and I've been on both sides, though more often on this side at the moment! Top tip - when a conversation starts doing this, here or anywhere else, you've only two sane options:

1. Take a large step back and spell out the full discussion to try to restart the conversation with everyone on the same wavelength. If you have a smart enough group who aren't too emotional yet, they'll see the different interpretations and/or the different interweaving sentiments between what you were arguing with half a dozen different other people and be able to refocus on your specific points.
   
   
2. Walk away. If you aren't on the same wavelength, it's a waste of time broadcasting. The discussion just stalemates with people talking at cross purposes. This can feel hard to do at first, but eventually it's very liberating and you save a truckload of time to spend on more meaningful pursuits!

What you should never do, anyone involved in the discussion, is persist in one/two line repetitions of a point saying, "See? See? This is what I'm trying to say, see?" because they didn't see the first or second times and won't the fifth or sixth if it's just the same point presented in the same way.

Shifty Geezer 1.0 would be clearing this up now, but Shifty Geezer version 2 is going to trust to the system. As the community is largely balanced against davis.anthony showing his idea is lost, I recommend davis.anthony spend a bit of time formulating a clear argument assembled all in one post, "I think this because of evidences A, B and C, where A shows...and B shows..." Then hopefully, honest, piecewise analysis of the evidences will show they either support or refute the theory in a way everyone can agree with, with those who got the wrong end of the stick either in the communication or the understanding of the data can say, "oh, okay. I thought it was that. My bad, you're right."

 

[davis.anthony]

Again and again. Oodle texture is pre-data-compression stuff. You can store the textures in this format on disk or even in the GPUs memory. Oodle texture is a texture compression (or combines many) with data-loss (just like any other texture compression format). Oodle texture also prepares the texture in a way that is is better compressable for the data-compression (with a hopefully minimal sacrifice on texture quality). The hardware inside of the xbox and PS5 just decompress stuff compressed with data-compression. Those are lossless formats (like Zip, LZ, kraken, ...). So you can use oodle texture in combination with any other data-compression, this is just a licensing thing.
The data-compression in the end won't make textures magically much smaller 3:1 as this is just content-dependent. In general kraken should be a bit better that zlib, but not by much. The texture compression format is way more important.
If games go the way of small artifacts, the difference in data-compression will be even less important as many smaller files will loose some effectiveness vs a few big library files. Therefore the SSD-only concept allows only to read what is really needed which saves duplicates and therefore in the end should be smaller. Which on the other hand will be sooner or later eaten up by the larger texture sizes the new hardware and memory allows.

Thank you for explaining something that:

1. I'm already aware of
2. Has been spoken about already above.
3. Doesn't really adding anything to the discussion

 

[davis.anthony]

This can often happen in public discussions and I've been on both sides, though more often on this side at the moment! Top tip - when a conversation starts doing this, here or anywhere else, you've only two sane options:

1. Take a large step back and spell out the full discussion to try to restart the conversation with everyone on the same wavelength. If you have a smart enough group who aren't too emotional yet, they'll see the different interpretations and/or the different interweaving sentiments between what you were arguing with half a dozen different other people and be able to refocus on your specific points.
   
   
2. Walk away. If you aren't on the same wavelength, it's a waste of time broadcasting. The discussion just stalemates with people talking at cross purposes. This can feel hard to do at first, but eventually it's very liberating and you save a truckload of time to spend on more meaningful pursuits!

What you should never do, anyone involved in the discussion, is persist in one/two line repetitions of a point saying, "See? See? This is what I'm trying to say, see?" because they didn't see the first or second times and won't the fifth or sixth if it's just the same point presented in the same way.

Shifty Geezer 1.0 would be clearing this up now, but Shifty Geezer version 2 is going to trust to the system. As the community is largely balanced against davis.anthony showing his idea is lost, I recommend davis.anthony spend a bit of time formulating a clear argument assembled all in one post, "I think this because of evidences A, B and C, where A shows...and B shows..." Then hopefully, honest, piecewise analysis of the evidences will show they either support or refute the theory in a way everyone can agree with, with those who got the wrong end of the stick either in the communication or the understanding of the data can say, "oh, okay. I thought it was that. My bad, you're right."

I'm going with option 3.

Ditching this forum, I should have known better really, this place does after all have a reputation for having members with shitty and somewhat arrogant attitudes.

I keep seeing some people saying they would love this place to grow and be as active as it once was, never going to happen unless people here change their attitudes.

In the last 3 days I've had people not read what I've said properly or only read what they want too - Some apologised and others ignored their fault.

Brit at the top of this page being one such example.

If the owners of this forum genuinely want it to grow then I suggest looking at the attitudes of the long established members and potentially getting shot of some of them as all their doing is destroying any hopes this place has of growing.

I'll be sticking to overclock.net and Twitter and leaving this place to carry on dying out it's slow death.

Peace out.

 

[Allandor]

Thank you for explaining something that:

1. I'm already aware of
2. Has been spoken about already above.
3. Doesn't really adding anything to the discussion

Your answer

XSX's hardware decompressor doesn't have support for Kraken, so Oodle+Zip would need to be done on the CPU (or GPU) taking away resources, so will it ever be used? Where as on PS5 it is supported by it's decompression hardware.

There's no agenda at all, just people looking for anything to pick fault with.

does show, that you did not really understand what oddle texture is. Oddle texture does not need "hardware"-support to get used. That was the whole point of my post, because your post indicated that some sort of hardware is needed to use oodle texture compression.
Oddle textures + zlib can be used on xbox without the need of any other CPU resources (not more or less than the PS5 needs).
Same applies to oodle textures + kraken on PS5.

Has the PS5 a higher theoretical bandwidth with the decompression of data? Yes it has. The point of all the discussion the pages before was, that you indicated that the PS5 could do things in hardware the xbox couldn't do because of texture compression. And that is just not true at all. On a pure texture compression level, PS5 and xbox behave the same. Only question that remains is what BCpack might change here. There might be a few additional compression formats the GPU understands, but it might also just be a "precompiler" for textures, just like oodle texture. Will it change much? Probably not. In the best case scenario I can think of it is just a bit more efficient. In the worst case scenario it is just a bit less efficient than oodle textures. But so far, I don't know what BCpack really is. I just assume that it is something like oodle textures. MS wasn't really open about this so far (or at least I haven't seen any deeper explanation).

 

[iroboto]

sorry I wasn't following this thread, I just sort tossed my hat in with what I knew. I'm reading upwards now on the discussion and what's being asked is if Oodle Texture is done on CPU or GPU.
So the answer is, it's available on _any gpu_ that supports BCn 1-7 type encoding. Which typically most directX GPUs will support. What Oodle Texture does, is not compress the texture, but re-encode the texture to a RDO format such that the decoding you would use for a typical BCn1-7 would output the same result. So the technology is in the encoding process (which is long and slow) to find the encoding that would output better results but arrange the data to be patterned in such a way that your compressors (like Kraken, or Lz) can find repeatable patterns to compress the texture down.

Hope that helps. Here's the official tweet. Let's try to be more civil everyone. Thanks.

https://twitter.com/x/status/1273015452273266689

wrt BCPack - I believe this is a similar RDO encoding of textures. I have doubts MS created a compression algorithm that competes with Kraken and Lz and is just as fast but has decided not to give it out freely.

We will see likely more use of Oodle Texture or BCPack. I do see Oodle texture get used quite a bit on those game logo loading screens. AC Valhalla is re-packaging itself, so all new binaries are required. PS5 still certainly doing better on delta patches though. Xbox Series file sizes has decreased dramatically from the original release, in fact coming out smaller than PS5 which I'll be honest in saying I didn't expect, a total of 44GB saved.

Data Restructuring - Title Update 1.4.1 | Ubisoft Discussion Forums
Download size from 1.4.0 to 1.4.1:

• PC ~78 GB
• PS4 ~67 GB
• PS5 ~40 GB
• X1 ~62 GB
• XS ~71 GB

Disc space used after installing 1.4.1

• PC ~77 GB
• PS4 ~75 GB
• PS5 ~77 GB
• X1 ~63 GB
• XS ~72 GB

Disc space saved after rebase:

• PC ~34 GB
• PS4 ~30 GB
• PS5 ~13 GB
• X1 ~30 GB
• XS ~44 GB

 

[dobwal]

sorry I wasn't following this thread, I just sort tossed my hat in with what I knew. I'm reading upwards now on the discussion and what's being asked is if Oodle Texture is done on CPU or GPU.
So the answer is, it's available on _any gpu_ that supports BCn 1-7 type encoding. Which typically most directX GPUs will support. What Oodle Texture does, is not compress the texture, but re-encode the texture to a RDO format such that the decoding you would use for a typical BCn1-7 would output the same result. So the technology is in the encoding process (which is long and slow) to find the encoding that would output better results but arrange the data to be patterned in such a way that your compressors (like Kraken, or Lz) can find repeatable patterns to compress the texture down.

Hope that helps. Here's the official tweet. Let's try to be more civil everyone. Thanks.

https://twitter.com/x/status/1273015452273266689

wrt BCPack - I believe this is a similar RDO encoding of textures. I have doubts MS created a compression algorithm that competes with Kraken and Lz and is just as fast but has decided not to give it out freely.

We will see likely more use of Oodle Texture or BCPack. I do see Oodle texture get used quite a bit on those game logo loading screens. AC Valhalla is re-packaging itself, so all new binaries are required. PS5 still certainly doing better on delta patches though. Xbox Series file sizes has decreased dramatically from the original release, in fact coming out smaller than PS5 which I'll be honest in saying I didn't expect, a total of 44GB saved.

Data Restructuring - Title Update 1.4.1 | Ubisoft Discussion Forums
Download size from 1.4.0 to 1.4.1:

• PC ~78 GB
• PS4 ~67 GB
• PS5 ~40 GB
• X1 ~62 GB
• XS ~71 GB

Disc space used after installing 1.4.1

• PC ~77 GB
• PS4 ~75 GB
• PS5 ~77 GB
• X1 ~63 GB
• XS ~72 GB

Disc space saved after rebase:

• PC ~34 GB
• PS4 ~30 GB
• PS5 ~13 GB
• X1 ~30 GB
• XS ~44 GB

Yep. And RDO isn't something thats new. Its been around a while and Oodle Texture is just RAD games version of it.

http://cbloomrants.blogspot.com/2018/03/improving-compression-of-block.html

RDO texture creation is a modification of the step that creates the block compressed texture (BCn or whatever) from the original (RGBA32 or whatever). Instead of simply choosing the compressed texture blocks that minimize error, blocks are chosen to minimize rate + distortion. That is, sometimes larger error is intentionally chosen when it improves rate. In this case, we want to minimize the rate *after* a following LZ compressor. The block compressed textures always have the same size, but some choices are more compressible than others. The basic idea is to choose blocks that have some relation to preceding blocks, thereby making them more compressible. Common examples are trying to reuse selector bits, or to choose endpoints that match neighbors.

RDO encoding of block compressed textures should always be done from the original non-compressed version of the texture, *not* from a previous block compressed encoding. eg. don't take something already in BC1 and try to run RDO to shrink it further. Doing that would cause the errors to add up, a bit like taking a JPEG and lowering it's "quality" setting to make it smaller - that should always be done from original data.

Now, block compressed textures are already lossy. BC1 is quite bad; BC7 and ASTC less so. So adding more error may not be acceptable at all. If large amounts of error are acceptable in your texture, you may not ever be seeing the largest mip levels. Sending mip levels that are too large and never visible is a *far* larger waste of size than anything we do here, so it's important to have a process in your game to find those textures and shrink them.

I imagine most devs will choose Oodle Textures as it supports all platforms. I have a hard time believing that you can use BCPack on the PS5 though it should be technically feasible unless PS5 doesn't support the BCN format.

 

[iroboto]

Yep. And RDO isn't something thats new. Its been around a while and Oodle Texture is just RAD games version of it.

http://cbloomrants.blogspot.com/2018/03/improving-compression-of-block.html



I imagine most devs will choose Oodle Textures as it supports all platforms. I have a hard time believing that you can use BCPack on the PS5 though it should be technically feasible unless PS5 doesn't support the BCN format.

would be curious going forward to see if texture quality improves on Xbox. I do recall quite a few times things being attributed to being poor AF on Series consoles even if the AF values were the same, so the only explanation at the time people thought was texture streaming was poor, but I wonder if it comes down to RDO encoding quality or the lack of, resulting in Series having to use lower quality BCn due to space issues and PS5 using the highest quality BCn but oodle texture encoding allows it to drop back in size to not make a difference.

 

[dobwal]

would be curious going forward to see if texture quality improves on Xbox. I do recall quite a few times things being attributed to being poor AF on Series consoles even if the AF values were the same, so the only explanation at the time people thought was texture streaming was poor, but I wonder if it comes down to RDO encoding quality or the lack of, resulting in Series having to use lower quality BCn due to space issues and PS5 using the highest quality BCn but oodle texture encoding allows it to drop back in size to not make a difference.

IDK. Maybe its possible for a dev that didn't want to pay the licensing fees and rolled a free solution for the Series consoles. Rad games states on their website that most pubs have already purchased bulk licenses for their devs so I would like to think there is little stopping most from utilizing Oodle Texture on the Series consoles.

I lean toward MS still working out the kinks of their tech.