Scorpio 4k texture format, size discussion

[Jay]

I didn't want to drag any other threads down, so I thought I'd start a new one as it covers a couple things I've come across in other threads.
I'm curious about discussing the possibilities and cons.

I think there's problems or maybe more so concerns/issues with 4k assets. With some 4k asset packs currently around the 40GB mark, and even then compromises probably need to be made.

These are the issues surrounding 4k assets as I see it:
1. Distribution size
Physical
Digital
2. Storage size
HDD
3. Console
RAM capacity (Scorpio has additional 3GB)
Bandwidth (Scorpio has 326Gbs)
HDD Transfer speed (50% improvement over X1)

Answer:
Even taking the console improvements into account, I wonder if it's enough.
I appreciate mip map, lod levels so wouldn't be storing and streaming all 4k assets in. But are those improvements really enough, before I said yes, but that's probably more so because it has to be, devs will make do, than ne truly knowing and having confidence in it.

So simple answer that would solve everything is a new texture compression format.
Is there any new formats that are suitable to decompress in hardware on a console?
Even a 50% improvement would be a big deal, that would affectively double the additional texture memory, and halve the pack size.
Even better 3-4x the compression ratio, that would be a really big deal.

So the problems as I see it:
1. Encoding time will be a magnitude longer to the point of making development with them impractical.
2. If using standard compressed textures as place holders, dev machine would require lot more memory & faster storage to be able to stream them in.
3. Higher memory bandwidth on dev machine when using standard compressed placeholders.
4. Only currently available on Scorpio, therefore no benefit of multiple platform support.

Answer:
1. MS provides azure servers to do the compression, low cost/free maybe for final authoring.
2. Scorpio dev machines have 24GB of memory and 1TB of SSD
3. Bandwidth is the same, I'm not sure about answer to this, have an idea but sounds stupid even to me.
4. Maybe it will be in Vega, windows store could then provide correct pack for supported cards. Even if not in Vega, as long as it doesn't give dev's much over head and is cost effective, it may not matter.

Would also explain to me why the forza tech demo whilst using 4k assets was still at only 5GB. Yep conspiracy theory

Could have situation where Scorpio has the best assets including compared to pc 4k texture packs, with all the benefits the smaller size offers.

So my questions are (although everything can be discussed)
Are there any currently known suitable compression formats
Are they currently supported in gpu's, but not used due to compression time or some other reasons
What are the reasons for MS not adding it to the soc, would it take up that much space? Could older texture compression formats be removed and deprecated to make space?
If the very old formats are used by couple games, ms could recompress them into new format, and supply assets as a patch.

If it did have this I'd consider it to actually really be drop the mic good, that's why I don't think it does.

 

[Allandor]

Developers don't do any extra work they didn't already do for the PC. Look at Gears or War 4 for example. It already has more than 100GB on the PC.
As for Forza Apex, it is already a PC build made with 4k in mind. There is nothing special about it. If you use a lower resolution or lower textures, you don't need the highest texture resolution. Works on PC for ages.
Same now applies to xbox one and Scorpio.

 

[Jay]

you say don't do extra work, that may not be true. Dependent on performance profiles, may have to decide to use lower quality assets in places due to size.

particular texture formats are used due to decompression hardware available, how long it takes to compress, etc.

I'm not talking about using different source assets, but better texture compression to get it down in size, as you said look how big it is on pc.

forza apex one wasn't what was used it was forza tech which is a proving ground. Which they may have updated to work with new texture format.
also as i said could've used the 4k pc assets, but with new texture format.

the point is the extra work for devs could be very little, with the biggest part being the compression to new format

 

[Cyan]

Developers don't do any extra work they didn't already do for the PC. Look at Gears or War 4 for example. It already has more than 100GB on the PC.
As for Forza Apex, it is already a PC build made with 4k in mind. There is nothing special about it. If you use a lower resolution or lower textures, you don't need the highest texture resolution. Works on PC for ages.
Same now applies to xbox one and Scorpio.

I don't have the numbers for the Xbox One version of Forza 6, but how much space does Forza Apex needs?

Forza 5 is pretty hefty iirc. I have the physical version and it is like a 40GB install. I was impressed by that at the time, little I knew about the future, but it was along with Ryse a game that surprised me and welcomed me into the new generation, games on the X360 needed a lot less space. I can quite understand it, to some extent, but patches are abusive on consoles these days.

Dunno if it was witchcraft, but I sorely miss the days of 2MB or 3MB patches on the X360.

 

[brunogm]

Maybe if the new hardware supports ASTC?

Sent from my Moto G using Tapatalk

 

[Jay]

Maybe if the new hardware supports ASTC?

Sent from my Moto G using Tapatalk

that's the sort of thing i was thinking about, but the thread is about the low level aspects.
so from the quick read, didn't really get to know how much better it is compared to what we have, if it's already implemented in hardware, if it's the sort of algorithm that azure would deal with well to use to compress to that format.

pc 4k textures, may even be able to be improved depending on format used, i don't know.
even if quality is the same, just being compressed smaller would be a big win in my books.

 

[brunogm]

that's the sort of thing i was thinking about, but the thread is about the low level aspects.
so from the quick read, didn't really get to know how much better it is compared to what we have, if it's already implemented in hardware, if it's the sort of algorithm that azure would deal with well to use to compress to that format.

pc 4k textures, may even be able to be improved depending on format used, i don't know.
even if quality is the same, just being compressed smaller would be a big win in my books.

Search the forum there was good discussions and comparison of ASTC versus BC6H/BC7 (current formats).

Here is a nice presentation: https://www.slideshare.net/mobile/IntelSoftware/in-the-trenches-optimizing-ue4-for-intel

Sent from my Moto G using Tapatalk

 

[Jay]

as i said I've seen the thread.
that is part of the reason i didn't want to pollute it.
it's also talking about one specific format, and didn't seem to answer my questions.

if it's the only suitable format, why isn't it currently being used more?
the slides didn't really tell me much, I'm assuming it means intel has it integrated into their apus

 

[brunogm]

GPUs texture units support a limited set of formats based on power/speed constraints. The random way textures are accessed, limits the compression techniques used in those formats. For sure people have written innovative ways to pack textures, in the form of shaders/software. The problem lies in the standardization and hardware implementation/support.

When PS4/XBO were announced ASTC where already standardized but no GPU hardware was available, but it was the perfect feature to customize the TextureUnits, in reality no such support was added.
I remember a KZ:SF presentation that focused on benefits of BC7/ BC6H, so big productions were starting to use these incremental formats.

Now ASTC allows for a revolution in texture uses including HDR. But with hardware support solves issues in complex shader decoding, memory usage, loading times....

Sent from my Moto G using Tapatalk

 

[Jay]

GPUs texture units support a limited set of formats based on power/speed constraints. The random way textures are accessed, limits the compression techniques used in those formats. For sure people have written innovative ways to pack textures, in the form of shaders/software. The problem lies in the standardization and hardware implementation/support.

When PS4/XBO were announced ASTC where already standardized but no GPU hardware was available, but it was the perfect feature to customize the TextureUnits, in reality no such support was added.
I remember a KZ:SF presentation that focused on benefits of BC7/ BC6H, so big productions were starting to use these incremental formats.

Now ASTC allows for a revolution in texture uses including HDR. But with hardware support solves issues in complex shader decoding, memory usage, loading times....

Sent from my Moto G using Tapatalk

thanks.

ASTC sounds like a good candidate. Do you know any particular reason it's not been adopted by gpu's yet? would it take up obscene amount of die space at the moment?

i understood with my limited understanding that there's a range of requirements that makes a texture format suitable to be implemented in hardware. but it sounds like ASTC could've been a known enough quantity to be a consideration.

i suspect we've had the same formats in hardware for a while now.

edit: Seems like it does have hardware support, but not by amd

 

[brunogm]

Like I said the were really not good arguments to not include in the first PS4/XBO. Even Nintendo should have pushed this . Hell some legacy texture; formats could been scraped for a custom die.

Sent from my Moto G using Tapatalk

 

[Jay]

Like I said the were really not good arguments to not include in the first PS4/XBO. Even Nintendo should have pushed this . Hell some legacy texture; formats could been scraped for a custom die.

Sent from my Moto G using Tapatalk

see you've been discussing this from 2013.
hopefully it will make it into Scorpio.
could be one of the vega things they pull across, i wouldn't expect amd to be singing ASTC from the rooftops if they've added it to vega

maybe bc7 is as good as it gets for now, and maybe that's more than good enough?