PC system impacts from tech like UE5? [Storage, RAM] *spawn*

[PSman1700]

the PC crowd is in deep shit and this tech will take years to appear there

A wet dream never comes to fruition. PC tech as compared to ps5 is already on another level, even ssd’s this summer will be faster.

Besides, Tim Sweeny isn’t lying, and he probably knows his things.

unless you consider Star Citizen a game...

It’s a playable tech demo, more so then the 9minutes of UE5. That warping to another planet was even more impressive.

 

[pjbliverpool]

I did interpret it correctly and you're confirming that the decompression alone is taking the equivalent of 3 Zen2 cores. Which neither the SeriesX or the majority of PC gamers can afford to lose.
And this is even assuming the decompression performance would scale linearly with just throwing more CPU cores working in parallel - which it may not.

Yes for full disk real time decryption of BCPACK. Which the PC doesn't do.

AFAIK the majority of PC games use zlib. Why would devs choose not to compress the files using the same methods as the console counterparts?

But is this for everything streaming off the disk in real time like the consoles? I admit I don't know much about this but that message is from GTA's load screen, and GTA is notorious for it's long load times. So no doubt it's decompressing some assets into memory at load time in readiness for gameplay, but is it doing that for everything that's streaming from the disk in live gameplay too? I'm doubtful. If it comes down to being unable to run a next gen game on the PC without severely crippling it, or increasing install sizes by 50-100% so that the PC doesn't have to decompress more data than it can cope with in real time, I think it's a fairly safe bet that the developer will choose the latter.

Also it does seem that the UE5 demo is running on PC's which further suggests there's nothing specific to PS5's storage system or hardware decompression required here:

https://www.pcgamer.com/unreal-engine-5-tech-demo/

They don't kill CPU performance because up until this point they were designed for storage units capable of ~40MB/s which is what you find in current consoles with HDDs. Therefore, all current and older games - console or PC - just use one CPU thread to decompress the data.
On PCs using high-speed SSDs, the loading times are limited by single-core performance.

It's not like devs could design a higher-performance decompression method for PCs with NVMe drives. There are no games demanding SSDs so far, unless you consider Star Citizen a game...

SSD's have been around for years so I'm sure developers are allowing for potentially faster streaming speeds than 40MB/s but we do know that the ancient IO protocols currently in use are a limiting factor here so whether they're using one core or 8 cores, we'd still expect to see CPU limitations on faster drivers due to IO alone.

 

[Deleted member 13524]

PC tech as compared to ps5 is already on another level, even ssd’s this summer will be faster.

Besides, Tim Sweeny isn’t lying, and he probably knows his things.

Oh the irony...

Sony is pioneering here with the PlayStation 5 architecture. It's got a God-tier storage system which is pretty far ahead of PCs

PC SSDs this summer won't be faster than PS5's 5.5GB/s minimum / >8GB/s effective throughput. 5.5GB/s already close to saturating whatever you can extract from a 4x PCIe 4.0 bus, and then the PC has no decompression hardware to reach a higher effective throughput.

Yes for full disk real time decryption of BCPACK. Which the PC doesn't do.

BCPACK and Kraken is for textures.. Zlib is for everything: textures, audio, geometry, probably some scripts too, etc. It's a zip file, and every game has its assets inside zip files (regardless of file extension).

The PS5's decompression block supports zlib (everything) and Kraken (textures only), the SeriesX supports zlib (everything) and BCpack (textures only).

But is this for everything streaming off the disk in real time like the consoles?

Yes. Every game uses compressed assets in the PC or consoles.
Even the Switch. The main advantage gamers found with the CPU boost mode was faster loading times -> meaning the games are compressed in the cartridge / microSD and need to be decompressed by the CPU when sending data to the RAM.

SSD's have been around for years so I'm sure developers are allowing for potentially faster streaming speeds than 40MB/s

I wish they had, but in truth I think Star Citizen's eternal alpha is the only SSD-dependent implementation I've seen so far.

 

[pjbliverpool]

PC SSDs this summer won't be faster than PS5's 5.5GB/s minimum / >8GB/s effective throughput. 5.5GB/s already close to saturating whatever you can extract from a 4x PCIe 4.0 bus, and then the PC has no decompression hardware to reach a higher effective throughput.

7GB/s SSD's are already due this year.

https://www.anandtech.com/show/1472...18-pcie-40-ssd-controller-up-to-7-gbs-nvme-14
https://mackintosh-hd.net/new-lexar-nvme-ssd-speeds-over-7-gb-s/

BCPACK and Kraken is for textures.. Zlib is for everything: textures, audio, geometry, probably some scripts too, etc. It's a zip file, and every game has its assets inside zip files (regardless of file extension).

The PS5's decompression block supports zlib (everything) and Kraken (textures only), the SeriesX supports zlib (everything) and BCpack (textures only).

Indeed, even the PS4 and XBO used hardware zlib decompression for steaming files off the BRD I believe. However I've yet to find any evidence of zlib or any other whole disk compression algorithm being used for all data streaming off disk on the PC to be decompressed in real time. Compressed textures which are read natively by the GPU of course, perhaps some other select data types too, but textures make up the vast majority of the data that's being streamed from disk during gameplay. And a requirement for the CPU to decompress everything streaming from disk in real time, textures included (like the consoles)? I can't find any evidence of that. Although I'd welcome a source if you can provide one.

The concept just doesn't make any sense to me. You're basically saying that next gen games which are going to require console speed storage solutions to function won't be done on the PC, not because they won't have sufficiently fast storage solutions (because they will, without full disk compression) but because devs will insist on compressing everything on disk to save disk space on the users PC that they're not paying for, thus creating a decompression bottleneck that doesn.t need to exist but nevertheless will mean they can't sell their game on PC, or at least not in it's full form.

 

[PSman1700]

Oh the irony...

Actually by the end of this year, SSD's have already surpassed PS5's transfer speeds. Besides that, Optane is already superior to the fastest NVME drives available, despite not having the fastest theoretical transfer speeds.

With the PC, there's also gobs of bandwith available, not a paltry 448gb/s that's being shared with just about everything else. Also, everything on the SSD needs to be finally rendered by something (the GPU), which is already a huge advantage to the PC right now. Soon we have 20TF GPU's around with advanced DLSS and ray tracing features. DX12 and velocity architecture and the whole MS/Xbox unified platform, in comibination with todays scaling, and the power of UE these days, a game running on 3080Ti/RDNA2 GPU with 16GB HBM2/GDDR6 ram at high speeds, fast clocked zen 3 CPU's (8 cores or more), and the mentioned 7GB/s drives, UE5 games/demos will put the PS5's output to shame.

Since UE5 is multiplatform, with probably amazing scaling, and with the XSX being what it is (most powerfull console), those new GPU's won't be left without some true next generation titles and tech demo's.

 

[Deleted member 13524]

I can't find any evidence of that.

Besides the screenshot I already posted?
How mamy will you need to consider valid?

Besides that, Optane is already superior to the fastest NVME drives available, despite not having the fastest theoretical transfer speeds.

Tim Sweeney who knows his things:

“The hardware that Sony is launching is absolutely phenomenal. Not only an unprecedented amount of graphics power, but also a completely new storage architecture that blows past architectures out of the water, and it’s so far ahead of even the state-of-the-art in the highest-end PCs you can buy.”

https://twinfinite.net/2020/05/epic...ge-blows-past-architectures-out-of-the-water/

 

[PSman1700]

Tim Sweeney who knows his things:

Ye, lol, today. The PS5 isn't out yet, so ain't the 7GB/s SSD's, but both will be at the end of this year.

 

[iroboto]

Besides the screenshot I already posted?
How mamy will you need to consider valid?



Tim Sweeney who knows his things:



https://twinfinite.net/2020/05/epic...ge-blows-past-architectures-out-of-the-water/

hmmm... but did he know about this!??

[queue the epic music]
https://www.gigabyte.com/ca/Solid-State-Drive/AORUS-Gen4-AIC-SSD-8TB#kf

 

[PSman1700]

hmmm... does he know about this!??

It's marketing. But anyway, yes the storage solution is faster then anything you can buy.... well ye, it can be, but the PS5 you can't buy either. When you can, you can also buy something even faster then the PS5's though.

 

[MrFox]

hmmm... does he know about this!??
https://www.gigabyte.com/ca/Solid-State-Drive/AORUS-Gen4-AIC-SSD-8TB#kf

You can put as many nvme in parallel as you want, but it won't magically solve the rest of the pipeline. It adds more compexity in the I/O, gets super expensive, reliability goes down, and doesn't solve the underlying problem.

On PC right now you end up with this:

That 3,500MB/s drive loads the game at the same speed as the 550MB/s ssd. The architecture is already bottlenecked. It's both a hardware and software architecture problem. This was not addressed on PC yet, not even close.

 

[BRiT]

Will need to wait until MS DirectStorage hits the PC space.

 

[Davros]

and every game has its assets inside zip files (regardless of file extension).

Oh, you really dont want to make claims like that....

 

[orangpelupa]

yep PC use compressed game files (except on loose files). Not always zlib.

IIRC call of duty series usually used standard zip but renamed (you even can open them using winzip). Unreal with its PAK file alllows it to be compressed or not https://docs.unrealengine.com/en-US.../Packaging/index.html#compressingyour.pakfile

Destiny 2 PKG files used oodle https://zenhax.com/viewtopic.php?t=4823

 

[pjbliverpool]

Besides the screenshot I already posted?
How mamy will you need to consider valid?

I can't find any evidence of full disk, realtime decompression of all data streaming off the disk on PC. I don't consider one game decompressing an unknown subset of files during a loading screen evidence of that I'm afraid. I've already agreed that some files will be compressed on disk on PC and that they may be decompressed either at load time or in real time streaming if practical, but that's quite different to literally everything on the disk having to be decompressed in real time as it streams into memory. I already mentioned the point of textures which make up the majority of a games package size which as far as I'm aware, don't get any additional compression on top of their native formats along the lines that Kraken and BCPACK provide.

Just look at the link Orangelupa posted to UE above stating that compression is optional on PAK files and actually not necessarily recommended on Steam and Oculus. However it also states that PS4 uses it natively already, and that it's recommended on XBO (presumably because of the hardware zlib decompression that both consoles use which I mentioned earlier). So yes, both current gen consoles and both next gen consoles compress everything on the disk for real time decompression when transferring off the disk either at load screens or during streaming. But the same does not seem to apply in the PC space.

 

[pjbliverpool]

That 3,500MB/s drive loads the game at the same speed as the 550MB/s ssd. The architecture is already bottlenecked. It's both a hardware and software architecture problem. This was not addressed on PC yet, not even close.

Why is it a hardware architecture problem? I understand that it's a software architecture problem. Ancient IO protocols and software not designed to utilise that kind of read speed will certainly be a bottleneck, although DirectStorage and next gen software designed with these kinds of speeds in mind may solve both issues.

However on a Zen2/x570 platform which is all that particular storage drive works on anyway, there doesn't seem to be any specific hardware bottleneck that I can see provided the Zen2 IO die can handle such insane speeds! The drive would be connected directly to the IO die on the Zen 2 chip over PCIe 4.0 16x which provides ample bandwidth for this level of data transfer. That same die manages the onwards distribution of the data into either system memory or GPU memory over the same PCIe 4.0 interface. Seems pretty straight forward to me. I guess one limitation would be that the SSD is stealing most of the PCIe bandwidth so while you could pull the data into system memory at these speeds you've be limited to only 7-8GB/s when sending it to the GPU over the 4 remaining PCIe lanes. So in that respect this isn't really targeted as a storage device to be used for gaming.

 

[manux]

Why is it a hardware architecture problem? I understand that it's a software architecture problem. Ancient IO protocols and software not designed to utilise that kind of read speed will certainly be a bottleneck, although DirectStorage and next gen software designed with these kinds of speeds in mind may solve both issues.

Follow the data. In ps5 case it's likely the data goes straight from controller-ram. On PC It might move like controller-ram-cpu-[main ram, video ram].

Depending how the pc hw is implemented the flow of data could be pretty bad and the data might move much more around than needed. In this case console having dedicated decompression in controller, dma engines+automated cache scrubbing and unified memory pool can have pretty significant advantage.

If DirectStorage allows for hw decompression engine to be used then the IO controller chip in new amd cpu's would be great place to implement it. Flow would then be controller-io-chip(decompression]-[main ram, video ram].

 

[pjbliverpool]

Follow the data. In ps5 case it's likely the data goes straight from controller-ram. On PC It might move like controller-ram-cpu-[main ram, video ram].

Depending how the pc hw is implemented the flow of data could be pretty bad and the data might move much more around than needed. In this case console having dedicated decompression in controller, dma engines+automated cache scrubbing and unified memory pool can have pretty significant advantage.

If DirectStorage allows for hw decompression engine to be used then the IO controller chip in new amd cpu's would be great place to implement it. Flow would then be controller-io-chip(decompression]-[main ram, video ram].

Agreed if the data needs to go via the CPU. Doesn't AMD's HBCC allow data to be read directly from the SSD into GPU memory though (via the Zen IO controller and a PCIe bus either side of it course)? I'm not saying that's necessarily enabled in drivers or software yet, but from a pure hardware perspective this is already in place isn't it?

That's not to say all hardware implementations can/will operate like this, but in terms of whether the problem has been addressed in hardware, AMD at least seems to have done so with it's latest platform and GPU's (going back to Vega in fact).

 

[eastmen]

Follow the data. In ps5 case it's likely the data goes straight from controller-ram. On PC It might move like controller-ram-cpu-[main ram, video ram].

Depending how the pc hw is implemented the flow of data could be pretty bad and the data might move much more around than needed. In this case console having dedicated decompression in controller, dma engines+automated cache scrubbing and unified memory pool can have pretty significant advantage.

If DirectStorage allows for hw decompression engine to be used then the IO controller chip in new amd cpu's would be great place to implement it. Flow would then be controller-io-chip(decompression]-[main ram, video ram].

If this tech takes off we could simply see storage connectors on the graphics chips to by pass any such limitations. The technology already exists. Before that I am sure the speed diffrences can be mitigated by additional storage on the graphics card. I expect the new navi 2 and GeForce 3x00 series to have 12-16 gigs of ram on the models. The ps5 has 16 gigs total ram

 

[manux]

If this tech takes off we could simply see storage connectors on the graphics chips to by pass any such limitations. The technology already exists. Before that I am sure the speed diffrences can be mitigated by additional storage on the graphics card. I expect the new navi 2 and GeForce 3x00 series to have 12-16 gigs of ram on the models. The ps5 has 16 gigs total ram

The data has to be loaded into ram. There is no way to use ram to cache things without the loading. Some games might just be too big to be cached efficiently(think of flying a plane in gta in low altitude, jump out of the plane, run in street, enter building, exit building, enter car, everything with higest possible level of detail with a lot of unique assets).

One big promise in this tech is less load time/less pop in effect/allow higher level assets as the streaming should be able to pull in whatever is needed. We will know a lot more once we see what DirectStorage in windows is and is ue5 streaming taking advantage of it.

Just dropping texture resolution to half would drop bandwidth needed to 1/4th. It's not rocket science to scale this technology to work with lower quality for those systems that have slower ssd.

 

[eastmen]

The data has to be loaded into ram. There is no way to use ram to cache things without the loading. Some games might just be too big to be cached efficiently(think of flying a plane in gta in low altitude, jump out of the plane, run in street, enter building, exit building, enter car, everything with higest possible level of detail with a lot of unique assets).

One big promise in this tech is less load time/less pop in effect/allow higher level assets as the streaming should be able to pull in whatever is needed. We will know a lot more once we see what DirectStorage in windows is and is ue5 streaming taking advantage of it.

Just dropping texture resolution to half would drop bandwidth needed to 1/4th. It's not rocket science to scale this technology to work with lower quality for those systems that have slower ssd.

Right but if you add a ssd , even using pci-e 4 tech lets say you put a 8x pci-e nvme slot directly on the board so that it by passes the need to go through the cpu to access it. You'd have a super large fast cache of 8GB/s or more storage feeding directly into the graphjics ram