PC system impacts from tech like UE5? [Storage, RAM] *spawn*

[Deleted member 13524]

If any company were extremely bold, they would release a PC add-on PCI-Express card with hardware-level Kraken decompression engine and NVME slots. That is not something I expect either Sony or Microsoft to do.

But Asus, MSI, Gigabyte, etc. could, if another company (like AMD, Realtek, VIA, etc.) comes up with a dedicated ASIC for that.
The problem is that for a 8-9GB/s effective throughput, this add-in board would need either a PCIe 3.0 16x connection or a PCIe 4.0 8x one, and that's a level of bandwidth that most modern consumer CPUs can't really spare.

Higher-end PCIe 4.0 motherboards could split the x16 PCIe bus into two, effectively using 8 lanes for the dGPU and 8 lanes for the storage card, assuming the CPU-dGPU connection doesn't become the bottleneck.
I'm guessing they could embed such a chip into the motherboard itself, and by using it then people wouldn't get access to CPU-driven additional storage, only the storage driven by the chipset.



I think there's also the possibility of AMD releasing their future video cards with IFLInk, not for multi-GPU but for a high-bandwidth connection with a storage card. Though if nvidia doesn't come onboard with this (with a similar NVLink card), then they just won't have a relevant enough marketshare to make devs dedicate their time with this.

So in the meantime, both consoles extreme storage systems may go underutilized for cross-platform games.

Or the PC crowd will just get random loading times and/or loading screens until a solution arrives.
It wouldn't be the first time the PC gets a very late adoption of a feature/technology in games. Just look at HDR, it's been almost flawlessly working on consoles since 2016 whereas in the PC it's a mess even today.

 

[eastmen]

did they just look for a card similar in Tflops as the ps5 ?

 

[Shifty Geezer]

Probably. That and Epic saying the demo would run on a 2070 with an SSD. It's not a researched bit of info at all.

 

[eastmen]

We would have to ask the journalists the source. Could be they are all getting them from one corrupt source or misquote. But by the wording it appears someone was at least told sata was insufficient.

It would make sense as a sata drive is limited to 600MB/s but with over head I haven't seen many drives pass 570MB/s . Compared to sony's 6,000MB/s that's slow. But you step up to nvme and your looking at 2GBs minimum compared to 6GBs and if you had pci-4 nvme's the 6GBs is just the start.

 

[Rodéric]

NVMe not begin much faster is odd, it seems Windows NTFS might be the bottleneck. I wonder if any tests were made with an OS using a different file system.

 

[Scott_Arm]

@Shifty Geezer The start of a console gen is definitely a good time to do a refresh of the min spec requirements on PC. In fact, it's bound to happen this time around because the consoles actually have reasonably good hardware. I know people with slow SSDs and old-ish quad core CPUs that expect their hardware to stay relevant because they assume it's better than console. Probably a rude awakening coming for them.

 

[Scott_Arm]

@ToTTenTranz That's why I'm wondering about DirectStorage on PC. They've said it's coming, but haven't said it won't require new hardware. Seems possible that to be direct storage compatible they could require decompression hardware built into the south bridge or something. Maybe it'll be two parts. One where you can address the nvme controller directly instead of through the filesystem, and another where you have decompression hardware in the chipset.

 

[chris1515]

https://twitter.com/x/status/1262581494968508416

Unbuffered I/O on PC after it must need a very good CPU

https://twitter.com/x/status/1248426555094609924

https://social.technet.microsoft.co...obocopy-unbuffered-io?forum=win10itprogeneral

Some explanation about unbuffered I/O

 

[Deleted member 13524]

@ToTTenTranz That's why I'm wondering about DirectStorage on PC. They've said it's coming, but haven't said it won't require new hardware. Seems possible that to be direct storage compatible they could require decompression hardware built into the south bridge or something. Maybe it'll be two parts. One where you can address the nvme controller directly instead of through the filesystem, and another where you have decompression hardware in the chipset.

I think putting DirectStorage hardware on the chipset might not be very effective, since it's using only a 4x PCIe 4.0 link to the CPU and therefore the RAM, meaning it has to share those 4x lanes with peripherals, USB ports, SATA, etc. To get the most out of a NVMe nowadays, we need to put it in one of the CPU-driven NVMe links.

I think it'd need to be on a PCIe board that links directly to the CPU's free lanes, or eventually within the CPU/APU itself.

 

[Scott_Arm]

@ToTTenTranz Oh, you're right. I didn't look at it carefully enough. I didn't notice the link between the CPU and the southbridge was only 4 lanes. For some reason I thought it was 8 lanes.

Definitely an interesting problem that would require a new solution. If they had to build decompression into the CPU, I think you'd be looking at a very long timeline for that to happen. Same with adding extra lanes to the southbridge from the cpu.

Edit: Only other thing I can think of is allowing the southbridge direct memory access. Basically cpu sends southbridge a command to load something, and then the southbridge can load from disk, decompress and write directly to memory. But the northbridge has control of RAM as it is right now, so that complicates the cpu again, since they have the northbridge on chip (correct?).

Edit: AMD will have to switch sockets to expose more pcie lanes. AM4 is limited to 20 lanes. Ryzen 4000 series is still AM4, so maybe in 5000 they'll have an opportunity to switch sockets and expose 8 lanes to the CPU from the southbridge.

 

[Deleted member 7537]

Another issue related to this is whether the complexity that exist on PC is part of a desirable level of abstraction. Cerny said their implementation is fully transparent to the developer, would be the same on PC? I say this because I remember mantel, DX12 and the "coding to the metal" mantra of last gen that didn't really pan out. I'm on the side that on PC, they will eventually brute force it. Specially if there isn't really a need. PS5 SSD will shine with PS5 exclusives most surely.

 

[Scott_Arm]

Well, DirectStorage API is coming to Windows, so they'd need a simple api for loading that basically handled decompression for you. As it is PC games are doing decompression on the cpu, so they're most likely using established decompression libraries at runtime.

Edit: I'm holding off on upgrading my cpu until I see how DirectStorage works on PC and whether it needs hardware compatibility. If it doesn't cover decompression I'll probably have to wait for 12 core or 16 core CPUs.

 

[Pinstripe]

I suppose the most logical place for decompressing game textures would be on the GPU? Perhaps Gaming Ampere and RDNA2 will have that block integrated on the die. Otherwise I can't see Microsoft simply omitting such a large chunk of their ecosystem by leaving even most recent PC builds behind.

 

[Jay]

Well, DirectStorage API is coming to Windows, so they'd need a simple api for loading that basically handled decompression for you. As it is PC games are doing decompression on the cpu, so they're most likely using established decompression libraries at runtime.

Edit: I'm holding off on upgrading my cpu until I see how DirectStorage works on PC and whether it needs hardware compatibility. If it doesn't cover decompression I'll probably have to wait for 12 core or 16 core CPUs.

Is there a conflating of two different things going on here?
Velocity Architecture - inc hardware decompression, direct storage etc
Direct Storage - more efficient access to ssd, possibly file format etc

Direct storage is coming to pc, but I don't remember ever hearing VA is also.

 

[Scott_Arm]

@DSoup Microsoft is claiming that DirectStorage will reduce 5 CPU cores worth of processing (3 hardware decompression, 2 improvements to I/O) down to one tenth of one core. They haven't explained how the new I/O system works, but it sounds like it basically bypasses your typical filesystem access and all of that Windows OS bureaucracy.

The final component in the triumvirate is an extension to DirectX - DirectStorage - a necessary upgrade bearing in mind that existing file I/O protocols are knocking on for 30 years old, and in their current form would require two Zen CPU cores simply to cover the overhead, which DirectStorage reduces to just one tenth of single core.

"Plus it has other benefits," enthuses Andrew Goossen. "It's less latent and it saves a ton of CPU. With the best competitive solution, we found doing decompression software to match the SSD rate would have consumed three Zen 2 CPU cores. When you add in the IO CPU overhead, that's another two cores. So the resulting workload would have completely consumed five Zen 2 CPU cores when now it only takes a tenth of a CPU core.

https://www.eurogamer.net/articles/digitalfoundry-2020-inside-xbox-series-x-full-specs

 

[Scott_Arm]

Is there a conflating of two different things going on here?
Velocity Architecture - inc hardware decompression, direct storage etc
Direct Storage - more efficient access to ssd, possibly file format etc

Direct storage is coming to pc, but I don't remember ever hearing VA is also.

Under the definitons here, hardware decompressions is included as part of directstorage. Not sure. If you look at the definition for the velocity architecture it looks like directstorage and decompression are different things. Who knows. I guess we'll find out whenever they give details for Windows implementation.

https://news.xbox.com/en-us/2020/03/16/xbox-series-x-glossary/

 

[Jay]

Under the definitons here, hardware decompressions is included as part of directstorage. Not sure. If you look at the definition for the velocity architecture it looks like directstorage and decompression are different things. Who knows. I guess we'll find out whenever they give details for Windows implementation.

https://news.xbox.com/en-us/2020/03/16/xbox-series-x-glossary/

Reading it there it does sound a bit confusing to me, so what is velocity architecture then? then it's mentioned in the hardware decompression section.

Could just be bad marketing blurb. I'll freely admit I'll believe that as I don't expect hardware decompression to be added to pc. So I could easily be wrong.

 

[Deleted member 7537]

Reading it there it does sound a bit confusing to me, so what is velocity architecture then? then it's mentioned in the hardware decompression section.

Could just be bad marketing blurb. I'll freely admit I'll believe that as I don't expect hardware decompression to be added to pc. So I could easily be wrong.

I don't either, I expect CPUs to do most of the job, especially now 8/16 is becoming the norm I'm sure there is some headroom available. Also PC have (usually) more system and video RAM available so you don't need to be constantly moving data in and out. PS5 is a closed system, that can't be upgraded, and needs to last 6-7 years before becoming obsolete.

 

[PSman1700]

I'll probably have to wait for 12 core or 16 core CPUs.

You won't have to wait, AMD has excellent 12 core 16 thread CPU's, clocking amazingly high considering the core count aswell. Zen 3 is around the corner though, which is an even more improvement. AMD really is killing it with their CPU's.

I don't either, I expect CPUs to do most of the job, especially now 8/16 is becoming the norm I'm sure there is some headroom available. Also PC have (usually) more system and video RAM available so you don't need to be constantly moving data in and out. PS5 is a closed system, that can't be upgraded, and needs to last 6-7 years before becoming obsolete.

That and perhaps storing bigger assets on the local drive, so there's less need for compression. On PC, users are more prone to have more then just a paltry 800gb of storage.

 

[Inuhanyou]

You won't have to wait, AMD has excellent 12 core 16 thread CPU's, clocking amazingly high considering the core count aswell. Zen 3 is around the corner though, which is an even more improvement. AMD really is killing it with their CPU's.



That and perhaps storing bigger assets on the local drive, so there's less need for compression. On PC, users are more prone to have more then just a paltry 800gb of storage.

-kicks rocks with my 200gb hdd laptop space-