Next-gen console versus PC comparison *spawn

[DmitryKo]

I could see MS creating a gaming file system that is used on Scarlett, but can also be used on PC, maybe some type of containerized game install.

That's full hardware virtualisation - i.e. each container has to include a hosted OS and drivers/runtime, similarly to the statically compiled game executable on the Xbox One, where the hypervisor hosts two separate virtual machine partitions for the Windows 10 OS and the game code.

Given the variety of hardware that has to be supported, that hypervisor becomes the full Windows OS, probably with Hyper-V isolation that runs a lightweight Windows OS for each VM, and the container-hosted OS becomes the full Windows OS (like Nano Server in Windows Docker containers), which kind of defeats the whole point.

 

[Shifty Geezer]

Sure, but caching would not improve start up times when you need to load large data sets for the first time. You have to read it from the media first - if your random disk read throughput is slow, RAM cache will not improve it.

Sure, the games will take longer to load but at least they'll be able to stream the same and cross-platform titles can target the same ballpark performance in such a scenario. It's a matter of the PC potentially being able to match consoles in terms of streaming performance, possibly at a cost.

 

[DmitryKo]

It's a matter of the PC potentially being able to match consoles in terms of streaming performance

I can't really see how Windows PCs can offer a similar level of performance without some radical design decisions.

Suppose I'm a developer updating my game engine to provide incredibly fast load times on Windows 10. Here's my plan:

1. target a mid-range PC with 32 GB of RAM and a fast PCI 3.0 x4 NVMe disk, at least Core i5-3000 series (Ivy Bridge) or higher;
2. set up a separate disk partition and format it with large 2 MB clusters (and hope there will be little fragmentation);
3. use a few relatively large files for game data containers, not duplicating any resource;
4. reorganize my data formats to require as little CPU processing as possible, with no data compression and geometry/texture formats based on graphics API conventions, so that faster decoding is possible;
5. on loading, allocate 8 to 16 GB of physical memory in a few contiguous region(s) with 2 MB or 1GB pages and map my entire data file(s) to this memory (and hope the OS uses large buffers and fast DMA modes to quickly transfer this data over that fast PCIe bus);
6. create several file views to access my resources, to show some pre-rendered animations (and distract the user), and to parse data and copy/reallocate the resources into 'regular' physical memory with a separate background process(es).

This should be as fast as it can possibly get, without forcing the user to buy a custom HPC server and stack it with Optane DC persistent memory modules.


Unfortunately, all of the above probably won't allow me to load my game in less than 1s, because:

1. my giant/large page physical memory allocation may fail because of memory fragmentation, so I will have to micro-manage data access with much smaller windows;
2. the actual disk IO subsystem and filesystem driver will be unable to achieve random access throughput of 2-3 GByte/s, so I will be unable to read and process game data in just a few seconds - and there is nothing I can do about it, because in general the developers:
3. cannot request the OS memory manager to defragment the physical memory and free a large contiguous block;
4. have no control over disk access modes or direct (virtualized) access to the NVMe controller;
5. have no control over the disk layout of game data - can't request the OS to use a contigous block of certain size;
6. cannot assume the PC has certain high-performance hardware - can only depend on the end user to make excellent buying choices and to excel in performing disk management tasks which oftens require a professional level of IT competence.

But all of this would be easy to allow on an embedded game console system with a proprietary OS, which does not have to concern itself with compatibility.


On Windows, something similar to the XBox One hypervisor, with games packaged in Docker-like containers along with Direct3D runtime and most other user-space libraries, but without a full hosted OS kernel, could be possible - but that's not really likely. This can only work on Xbox because it's standard hardware with no need for installable drivers, and this can work on Linux because officially it's the Linux Kernel and all its APIs are public, while everything else lives in the user space. But Microsoft is very secretive about their internal kernel-mode APIs, and they won't separate the real OS kernel from Kernel32.dll and similar user-mode thunks.

 

[PSman1700]

MS could lay some resources on the problem as xbox=windows and vice versa, and gamepass etc. I'm sure they will work something sometime, it's their market. Cross platform games regarding MS is actuallya thing. Besides that we don'y know at all how fast the consoles are gonna be. Intersting HW times atleast, AMD RT, Zen 3, ddr5, pci4/5, Intel gpu's, HBM2/3, rtx3000, and next gen consoles too.

 

[vjPiedPiper]

Pretty true there,

MS could easily create some sort of DX SSD, type of virtual device, out of an a users existing SSD, and then let games acces that resource similar to how it might work in the next Xbox.
Of course not everyone will have 128GB-1TB of spare high speed SSD just sitting there free for use by DX.

Although i still think it's likely we'll see a smaller SSD + HD solution form at least one of the console makers.
But maybe we see a dual SKU approach similar to the first 360 release, eg. 128GB SSD, at the low end, backed by a USBC connection for users to attach their own HD, and a 1Tb version, again with the USBC for external HD space, but also allowing the other SSD space to be used as storage.

It would be interesting to hear some developer comments regarding how much space they think would be the right amount for 4-5GB/s SSD cache?
Would 2 x the RAM amount be useful? what about 10X? It could totally change the way open world games are designed.
If any given frame can draw on 16GB of 400GB/s memory AND 256Gb of 4GB/s memory

 

[Tkumpathenurple]

Your quote attributes me, but I didn't write that.

Are you sure?

I like big butts and I can not lie
You other brothers can't deny
That when a girl walks in with an itty bitty waist
And a round thing in your face
You get sprung, want to pull up tough
'Cause you notice that butt was stuffed
Deep in the jeans she's wearing
I'm hooked and I can't stop staring.

Well said.

 

[chris1515]

I'm not that sure about page size though. It's common to think that larger page size is more efficient, but it's rarely that obvious when running real world workloads. Most OS are still using 4KB pages, though iOS changed to 16KB on 64 bits systems. Of course, maybe gaming workloads are different, but if that's the case then consoles should be using those sizes right now (I don't know if that's the case though).

On a console a simpler file system can be used to reduce overhead, as most large files are static, but the gain probably won't be too large (maybe in single digit percentage). High performance games on PC also uses memory mapped files to reduce API overhead. So I think the advantage is there, but probably not huge.

The patent explain the can work with 4kib files but if you need to stream or load many 4kib files the parrallel I/O is not efficient anymore and the devs need to merge all the 4kb files into one files.

After the virtual filesystem is not different to a PC with a free BSD for the x86 CPU inside the patent. But there is a filesystem internal to the SSD call file Archive. They don't give details on it but the main CPU Zen 2 never access this filesystem, there is a secondary CPU(ARM?) inside the SSD taking care of all SSD management and an hardware decompressor and a custom SSD controller optimized for read access(the speed will be asymetrical between read and write). They explain SSD on PC are often slow down by the CPU load and they want to delegate all of to the other CPU managing the SSD. The main CPU ask for some files with a thread and receive the file when all is ready, no decompression on the x 86CPU or GPU or managemen task of the SSD.

Same thing for cost and latency, the adress table translation is not store into RAM but in SRAM. There is different patents covering a two years period 2015 and 2016. Cerny explained in the Wired interview the PS5 development begins four years ago. The first work seems to be the SSD. They have an internal system of queue where they can give some data prioritires in another patent and a third patent showing less consumption in sleep mode because of SRAM usage.

But they explain the faster is the SSD, the most powerful the secondary CPU and hardware decompressor needs to be increasing cost. I don't expect something better than 5 GB/s or pushing it to 10 GB/but I doubt it will go so fast.

And if it is 10 GB/s or faster, very easy more RAM on PC solve the problem, the first loading time will just be longer.

 

[PSman1700]

From rumors and leaks to patents

 

[chris1515]

From rumors and leaks to patents

This one I am ready to take a definitive ban bet it will be something looking like the patent because I heard it by someone I trust. He reveals it because he was surprise someone find the patent. But he did not want to talk about the speed. But the speed of the SSD in PS5 impress Todd Howard(Bethesda) and two japaneses devs from Capcom if I remember well.

But like I said this is more probably something like 4/5 GB/s real speed in game. Because consoles are made of compromise and Sony need to keep the SSD cost as lowest of possible.

If you are so sure of you, take the bet.

I have no problem with it because if I trust KleeGamefan* I will not take a bet from someone else leak/rumor.

Like I said the day this person talk to me she said finally they find it and ask if someone has find the dualshock 5 patent. I told him no and I did not want to search because too many dualshock patent but searching every dualshock patent in resetera post by gofreak and anexhanume after DS5 functionnality reveal in Wired interview I find it.

EDIT: And he was clever, he hides the most important part the speed.

EDIT2: And I search there is no other Sony patent with a SSD post 2015.

*Since 18 years I read his post from GAF and the proof of his identity is enough on my side.

EDIT3: Something easy to verify after the reveal of the PS5 and some test, he told me that non optimized games will load as fast as a SSD with RAMdisk. And optimized games will load and stream data much faster but he did not want to give me details but he kills my hope of game launching in less than 1 seconds at boot time because this is not only a question of data but of setting up the game engine, more powerful CPU will help but it is long and he expects to see some games with 3d menu because it will help to reduce the boot time. It will probably take a few seconds at least.

 

[Deleted member 13524]

After the virtual filesystem is not different to a PC with a free BSD for the x86 CPU inside the patent. But there is a filesystem internal to the SSD call file Archive. They don't give details on it but the main CPU Zen 2 never access this filesystem, there is a secondary CPU(ARM?) inside the SSD taking care of all SSD management and an hardware decompressor and a custom SSD controller optimized for read access(the speed will be asymetrical between read and write). They explain SSD on PC are often slow down by the CPU load and they want to delegate all of to the other CPU managing the SSD. The main CPU ask for some files with a thread and receive the file when all is ready, no decompression on the x 86CPU or GPU or managemen task of the SSD.

If true, it'll probably be a long time until the PC can have any hardware that emulates this behavior, so more and more games will be starting to recommend 32 or even 64GB of RAM to provide a similar experience to the consoles.

 

[chris1515]

If true, it'll probably be a long time until the PC can have any hardware that emulates this behavior, so more and more games will be starting to recommend 32 or even 64GB of RAM to provide a similar experience to the consoles.

No, it is not so difficult because everything is internal to the SSD. The only problem is they need to reach a standard and they will not optimize only for read-only... But they could probably do better, I don't expect crazy speed. I expect 4/5GB /s but not theoretical or in a benchmark.

You can do exactly the same thing with Windows or linux or Mac OS.

 

[Deleted member 13524]

No, it is not so difficult because everything is internal to the SSD. The only problem is they need to reach a standard and they will not optimize only for read-only...

It may not be difficult to achieve in the hardware, but if it's properly patented then it might be hard to do without some brute force.

 

[PSman1700]

No, it is not so difficult because everything is internal to the SSD. The only problem is they need to reach a standard and they will not optimize only for read-only... But they could probably do better, I don't expect crazy speed. I expect 4/5GB /s but not theoretical or in a benchmark.

You can do exactly the same thing with Windows or linux or Mac OS.

The PC will offer the same experience as the scarlett xbox, one or another way. If not at the start it will catch up. Btw it's about the only thing that they have going for it
If price wasn't a matter, give me that 16+ TF ray tracing monster, a zen 3 12 core with some brutal speeds, 64gb ddr4 or 5, intel optane SSD's etc.Silent and powerfull.

 

[chris1515]

The PC will offer the same experience as the scarlett xbox, one or another way. If not at the start it will catch up. Btw it's about the only thing that they have going for it
If price wasn't a matter, give me that 16+ TF ray tracing monster, a zen 3 12 core with some brutal speeds, 64gb ddr4 or 5, intel optane SSD's etc.Silent and powerfull.

I just say that there is nothing crazy, this looks clever but one company making SSD can come with the same solution probably easily. Add a beefier ARM CPU and some SSD have some hardware decompressor. The most difficult is the software side because you probably need to work with the MS on the OS side and other SSD makers.

If the SSD is faster than the one in Xbox Scarlett they will not catch up, there is a hardware part... PC will catch up.

 

[PSman1700]

I just say that there is nothing crazy, this looks clever but one company making SSD can come with the same solution probably easily. Add a beefier ARM CPU and some SSD have some hardware decompressor. The most difficult is the software side because you probably need to work with the MS on the OS side and other SSD makers.

Patents are just that, patents, they could mean the real thing. They could also totally not. It sure is possible though, one thing though, adding arm processors and logic means extra hardware too, and it will add up to the total cost. Their going to sell 100million units again most likely so that's a factor. Somewhere they will have to compromise then, if not that arm cpu could be used for other things like standby downloading, like the PS4 does now.

 

[chris1515]

Patents are just that, patents, they could mean the real thing. They could also totally not. It sure is possible though, one thing though, adding arm processors and logic means extra hardware too, and it will add up to the total cost. Their going to sell 100million units again most likely so that's a factor. Somewhere they will have to compromise then, if not that arm cpu could be used for other things like standby downloading, like the PS4 does now.

Like I say take a definitive ban bet from B3D I have no problem with that and I have my account since 2005, this one I am 100% sure it will be this. This is not a second-hand thing but someone I know for 14 years. Like I said I don't expect a very fast SSD because of the cost, they said inside the patent they have the possibility to use multiple ARM core and increase the number of hardware decompressor working parallelly to improve speed but it increases the cost too. Consoles are made of compromise, I would not even give a speed number if Mark Cerny did not say faster than any PC SSD on April 4 to 5 GB/seems realistic.

 

[PSman1700]

No one has to take ban-bets, that's where forums are for, discussions. It doesn't matter if your wrong, or me or anyone
How does this person know PS5 hardware details though, have i missed something? He must be close to the HW design team at sony or AMD i might think?
Using multiple arm cores adds to the size of the total chip, so it must be worth it.

 

[chris1515]

No one has to take ban-bets, that's where forums are for, discussions. It doesn't matter if your wrong, or me or anyone
How does this person know PS5 hardware details though, have i missed something? He must be close to the HW design team at sony or AMD i might think?
Using multiple arm cores adds to the size of the total chip, so it must be worth it.

Because you think people working for the first-party studios have no idea about the PS5 and third party too. They need to know how the PS5 work. After he is under NDA and he was just surprised when I show him the patent. He knows if it is 8 Tflops or 10 Tflops, he probably has an idea of Scarlett power now than it is on the hand of tons of the third party because he knows people in the industry but NDA.

 

[PSman1700]

After he is under NDA and he was just surprised when I show him the patent.

That's like tipping off, someone under NDA shouldn't hint either. Maybe your right, no idea, only a good thing then, isn't it?
So it's confirmed that the PS5 will have arm co-processors for the SSD, and 10TF is it, for the GPU?

Edit: I just find it surprising you seem to posess info on the hardware of the PS5 no one else has, outside of Sony and AMD etc.

 

[chris1515]

That's like tipping off, someone under NDA shouldn't hint either. Maybe your right, no idea, only a good thing then, isn't it?
So it's confirmed that the PS5 will have arm co-processors for the SSD, and 10TF is it, for the GPU?

I have no idea for the GPU, he just said the SSD is coming from the work of these patents, that's all and he asks if someone finds the DualShock 5 patent. Nothing more and not the juicy thing the SSD speed. I will not take a bet on it because I have no idea. If gofreak on B3D did not find the patent I would think it is an SSD NVMe PCIE 4.0.

I don't know if the second CPU will be an ARM maybe it is a CELL this is a joke. He just said it comes from the work on this patent. And the patent just says secondary CPU and the most logical choice is ARM.

He did not talk about the CPU or GPU or the RAM.