PC system impacts from tech like UE5? [Storage, RAM] *spawn*

[Orion]

We don't know that until we know how virtualisation works. Remember, 100 objects with 4k textures requiring loads of data (~5 GBs raw 24 bit colour) can be represented with 64 MBs of textures when healthily virtualised. The actual amount of geometry needed to render a frame is as many triangles as there are pixels in a perfect case, so 8 million triangles for 4K. The amount to transfer will be somewhere between that 8 million vertices and all the geometry for all the object. Until we know what that mid-point is, we don't know what minimum speed is needed on storage.

we also know that some sections of the demo required careful data layout in the ssd. One might think that such optimization was done for sata speeds, but many articles comment epic noted that sata ssd was insufficient. This suggests that the careful layout was necessary for the development pcs with nvme ssds.

We don't know how much an open world without careful layout will take, might be that not even direct storage or the ps5 can handle it or might be they can. What we do know is that the current pcs without directx12 ultimate likely cannot handle open world at fast speed at this level of detail. At least without further optimization

 

[Deleted member 11852]

If any company were extremely bold, they would release a PC add-on PCI-Express card with hardware-level Kraken decompression engine and NVME slots. That is not something I expect either Sony or Microsoft to do. So in the meantime, both consoles extreme storage systems may go underutilized for cross-platform games.

This would be a good start but this would only work on future games packaged with a supported decompressor and there are many more stages in the traditional I/O pipeline that most games are using. The desktop filesystem is not optimised for handling hundreds thousands of small asset files (objects, textures, sounds files etc) that compromise many games hence why they are packaged up. On a desktop OS, if my engine needs two assets, one for CPU and one for GPU, this is the typical I/O chain:

1. Games asks filesystem for assets, calls a bunch of APIs to the various files.
2. API calls OS filesystem.
3. Filesystem calls device drivers (bridge and devices)
4. CPU drives the device driver/filesystem to find and pull data over southbridge into main RAM. (seeks/reads, seeks/reads).
5. CPU unpacks/decompresses/processes (reads/writes, eating bandwidth/time) data now in main in RAM.
6. Data for CPU-side now ready. Job done.
7. Data for GPU sent over Northbridge to GPU. Job done.

There is a staggeringly amount of faffing around at the kernel level as data is passed from one component (the SSD) to another (bridges) to get to where it needs to be. You don't think about it because it's what the filesystems APIs abstract from user code (games/applications) but it's all happening and it's all relatively slow which is part of the reason why it's generally quicker to work with packs of data then all the little individual files. This will change with nextgen console but it won't change on Windows without a new filesystem and API and even then all the handing off of data from one piece of hardware to another will continue to exist.

 

[Scott_Arm]

@DSoup Really curious for more detail on DirectStorage and how it affects that chain. Wonder if it only affects step 2, 3, by replacing the normal filesystem access, or if it has other repercussions to the rest of the chain.

 

[Davros]

we also know that some sections of the demo required careful data layout in the ssd.

I was under the impression ssd's dont care where the data is one of the reasons (along with wear) that we dont defrag them.

 

[BRiT]

I was under the impression ssd's dont care where the data is one of the reasons (along with wear) that we dont defrag them.

It doesn't. However there is extreme overhead with PC IO that some data layout optimizations needed to happen in order to be able to read at sustained rates required for the Flying Tombraider UE5 Tech Demo.

Perhaps it's in organizing smaller assets to be part of the same 512K or 1 Meg block sizes. It might even be just using NTFS partition with larger cluster sizes.

I don't have the links right now, but @chris1515, @DmitryKo and a few others have done a fine job of posting various findings on PC IO and optimal cluster sizes and different filesystems to get the most out of block devices. I think they're in the Console Technology Next-Gen Threads. It's a trove of useful information if you skip pass all the other posts filled with posturing.

 

[Davros]

 

[BRiT]

Quite a few posts from this Thread [ https://forum.beyond3d.com/threads/next-gen-console-versus-pc-comparison-spawn.61408/ ]

https://forum.beyond3d.com/posts/2084731/
https://forum.beyond3d.com/posts/2085030/
https://forum.beyond3d.com/posts/2085032/
https://forum.beyond3d.com/posts/2085593/​

 

[BRiT]

...

I think any statements as to what's required or not is premature by an entire year. The TechDemo was from 2020-03 with nearly an entire year of additional work being required before Early Access release.

That's a metric ton of time for performance tuning to happen on all platforms.

 

[Malo]

Quoting an image capture from somewhere called "World Today News" as an authority on this insanely high tech? Am I on the right forum?

 

[Inuhanyou]

Playstation 5's SSD advancements are pretty advanced, but im sure PC can pretty much brute force past needing such things. If we take a look at the next generation of graphics and CPU advancements, its going to be a significant jump from the next gen machines and we'll be back to the normal PC and console dynamic in a year or two's time, even without PC components having those tricks.

To begin with, since all games need to be multiplatform and run on PC well..lets assume most third party devs will just target things like sata SSD..so they dont need specialized NVME speeds or whatever to gain parity or surpass the consoles.

The fact that the consoles will make HDD's obsolete for gaming in a little while is pretty big though considering what Star citizen can do while designing without HDD limitation(although for HDD users i hear its a pretty miserable experience)

 

[Remij]

To begin with, since all games need to be multiplatform and run on PC well..lets assume most third party devs will just target things like sata SSD..so they dont need specialized NVME speeds or whatever to gain parity or surpass the consoles.

I'm a PC gamer first and foremost, but I honestly hope that doesn't happen. I want them to push Gen3 NVMe drive as a recommended spec. Maybe a SATA SSD will be good enough for most games, but I'd like to see the big publishers push NVMe adoption with their big games. We need a new baseline as far as I'm concerned, both consoles have NVMe so that should be the standard..SATA drives just don't cut it IMO.

 

[Scott_Arm]

I'm sure nvme drives will come out with 12 channel controllers eventually. Mine has 8 channels, but cheaper ones can have as little as 2. To be honest, Sony needs them to come out otherwise it'll be pretty much impossible to expand on storage, and 850 GB or whatever it is does not sound like a lot after looking at the UE5 demo.

 

[Inuhanyou]

I'm a PC gamer first and foremost, but I honestly hope that doesn't happen. I want them to push Gen3 NVMe drive as a recommended spec. Maybe a SATA SSD will be good enough for most games, but I'd like to see the big publishers push NVMe adoption with their big games. We need a new baseline as far as I'm concerned, both consoles have NVMe so that should be the standard..SATA drives just don't cut it IMO.

but they cant garuntee everyone(or even most people) to have top tier HW in their units like that and it would become a miserable experience targeting software for drives 10x faster than the baseline of SSD with sata...

like dont you think its logical for general third party devs to go for lowest common denominator while still pushing things forward somewhat with sata 3 SSD baseline?

I understand from a gamer point of view we want technology to be pushed...but in the modern industry where devs have to account for all kinds of handicaps and lower denominators, it just seems like an inevitable reality. The consoles are pushing tech standards in this new way but even with their adoption its way too fast for the industry to adapt as a whole for a while.

 

[Deleted member 11852]

Playstation 5's SSD advancements are pretty advanced, but im sure PC can pretty much brute force past needing such things.

The barrier to brute-forcing it on PC is you have to brute-force more than half of the PC architecture to accommodate all the bits of hardware in the I/O chain - including the SSD, the controller, the Southbridge, the CPU and the Northbridge - and that doesn't do anything to mitigate the inefficiencies of the software stack, nor overcoming two RAM pools.

What hardware developments do you think are on the horizon that will allow a PC to brute force it? Because I don't see anything and new standards are public a good 4-5+ years before that are implemented widely.

 

[Deleted member 7537]

I'm sure nvme drives will come out with 12 channel controllers eventually. Mine has 8 channels, but cheaper ones can have as little as 2. To be honest, Sony needs them to come out otherwise it'll be pretty much impossible to expand on storage, and 850 GB or whatever it is does not sound like a lot after looking at the UE5 demo.

I expect Sony to be the only manufacturer of PS5 SSDs for a while, unless their SSD solution has been codeveloped with an external manufacturer like Samsung which could accelerate 12 channel SSDs to the market but I doubt it. I hope they name them memory cards. xD

Also, Cerny mentioned they would be testing 3rd party SSDs and will be recommending those that meet PS5 standards, maybe some SSDs, even with 8 channels, do not hinder PS5 performance.

The barrier to brute-forcing it on PC is you have to brute-force more than half of the PC architecture to accommodate all the bits of hardware in the I/O chain - including the SSD, the controller, the Southbridge, the CPU and the Northbridge - and that doesn't do anything to mitigate the inefficiencies of the software stack, nor overcoming two RAM pools.

What hardware developments do you think are on the horizon that will allow a PC to brute force it? Because I don't see anything and new standards are public a good 4-5+ years before that are implemented widely.

And brute forcing cost money. You can always build a 3000 dollar pc, but then you are comparing apples to oranges.

 

[pjbliverpool]

One might think that such optimization was done for sata speeds, but many articles comment epic noted that sata ssd was insufficient.

Do you have some links to these please? There's obviously this quote from Tim Sweeney suggesting non NVMe SSD's will be fine, but while it's in the wider context of discussing the demo he could easily have just been referring to the UE5 in general (which it really goes without saying should be supported on PC's with regular SSD's).

https://www.eurogamer.net/articles/...xt-gen-unreal-engine-running-on-playstation-5

One of the big efforts that's been done and is ongoing in Unreal Engine 5 now is optimising for next generation storage to make loading faster by multiples of current performance. Not just a little bit faster but a lot faster, so that you can bring in this geometry and display it, despite it not all fitting and memory, you know, taking advantage of next generation SSD architectures and everything else... Sony is pioneering here with the PlayStation 5 architecture. It's got a God-tier storage system which is pretty far ahead of PCs, but on a high-end PC with an SSD and especially with NVMe, you get awesome performance too.

 

[Shifty Geezer]

but they cant garuntee everyone(or even most people) to have top tier HW in their units like that and it would become a miserable experience targeting software for drives 10x faster than the baseline of SSD with sata...

like dont you think its logical for general third party devs to go for lowest common denominator while still pushing things forward somewhat with sata 3 SSD baseline?

I understand from a gamer point of view we want technology to be pushed...but in the modern industry where devs have to account for all kinds of handicaps and lower denominators, it just seems like an inevitable reality.

I disagree. For top-drawer AAA titles that need streaming, abandoning the too slow sector of the PC space the same way they'll abandon 100+ million last-gen consoles will be a sensible option. The cost reduction in creating the game could be significant if you don't have to force it down onto slower machines. Something like the next GTA could force a hardware refresh in the PC space which is necessary, same as in the console space, if software is to stretch its legs and not be hampered by legacy hardware. Economically, this isn't a bad call either IMO. I think a few top tier games will shift the evolution.

Unlike GPU work that can scale down, IO work needs a new baseline. Once that's established, your game can stream different stuff for slower and faster drives, but until then, targeting HDD is like targeting 'no GPU required'. I also doubt super-fast PC storage will sell much to consumers just being faster, so it'll only progress thanks to expensive workstations and gaming. If Gaming doesn't push the industry forwards, PC will be really far behind and a real drag on game development.

 

[Nesh]

I disagree. For top-drawer AAA titles that need streaming, abandoning the too slow sector of the PC space the same way they'll abandon 100+ million last-gen consoles will be a sensible option. The cost reduction in creating the game could be significant if you don't have to force it down onto slower machines. Something like the next GTA could force a hardware refresh in the PC space which is necessary, same as in the console space, if software is to stretch its legs and not be hampered by legacy hardware. Economically, this isn't a bad call either IMO. I think a few top tier games will shift the evolution.

Unlike GPU work that can scale down, IO work needs a new baseline. Once that's established, your game can stream different stuff for slower and faster drives, but until then, targeting HDD is like targeting 'no GPU required'. I also doubt super-fast PC storage will sell much to consumers just being faster, so it'll only progress thanks to expensive workstations and gaming. If Gaming doesn't push the industry forwards, PC will be really far behind and a real drag on game development.

It was about time the PC Master Race would finally put down some of their egos

 

[PSman1700]

It was about time the PC Master Race would finally put down some of their egos

Not only the master race

 

[Orion]

Do you have some links to these please? There's obviously this quote from Tim Sweeney suggesting non NVMe SSD's will be fine, but while it's in the wider context of discussing the demo he could easily have just been referring to the UE5 in general (which it really goes without saying should be supported on PC's with regular SSD's).

https://www.eurogamer.net/articles/...xt-gen-unreal-engine-running-on-playstation-5

Epic also appreciated the benefits of the PS5's SSD, noting that such graphics cannot be achieved using a hard drive or even a SATA SSD as
https://www.guru3d.com/news-story/f...to-run-the-unreal-engine-5-demo-smoothly.html

Finally, Epic noted that these graphics were made even more possible by the use of the PS5’s SSD, even going as far to say that a normal hard drive or even a SATA SSD wouldn’t be fast enough to load the high res textures used in the demo, and an NVME SSD would be needed in order to load the textures at the required speed on a PC.
https://www.game-debate.com/news/28...east-run-unreal-engine-5-demo-on-pc-says-epic

We would have to ask the journalists the source. Could be they are all getting them from one corrupt source or misquote. But by the wording it appears someone was at least told sata was insufficient.