Next-gen console versus PC comparison *spawn

[PSman1700]

Maybe. You were talking about "improved software solutions" making things better, I assume you meant current PCs. PCIe 4.x will make future PCs and future components designed to support the standard perform better overall. Anybody who already has a PC, or is buying one now, isn't going to experience some revolutionary benefit from improved software because the constraints are in hardware.

Ah ok, well then yes, the PS5 isn't even out yet so it is hard to compare against 2020 hardware. Software stack is also important and probably will see improvements, too.

 

[cheapchips]

Upgrading my PC is way down my spending priorities these days. It took about 3 yrs to complete an upgrade to a respectable mid-range PC with SSD. It worked out ok. Don't think I really missed much prior to upgrading my CPU/mb/ram. This gens poor Jaguars helped on that front.

It doesn't feel it's going to happen that way this time. It's likely I'd have to do a full upgrade all at once + add high end storage to play next gen games.

It used to be the case that a mid tier PC was by far the more economical choice Vs consoles and had the most diverse games.

Indie game are pretty much all on consoles now. That's a big change from the start of the generation. From the point of view of software value, Gamepass shifts things significantly.

I might end up with a Scarlett this gen.

 

[PSman1700]

It used to be the case that a mid tier PC was by far the more economical choice Vs consoles and had the most diverse games.

The other way around, it was much more expensive to keep up before the 8th generation. We have seen a market shift where there are less console exclusives and less pc-exclusives, nowadays it doesn't matter so much what platform (PC, XB, PS), you won't miss out much, and that trend isn't slowing down. Well, aside from PS which still offers the most exclusives, but it ain't like PS2 days either there

 

[DmitryKo]

PCIe 4.x will make future PCs and future components designed to support the standard perform better overall. Anybody who already has a PC, or is buying one now, isn't going to experience some revolutionary benefit from improved software because the constraints are in hardware.

These are bottlenecks consoles do not have to live with, they can be engineered out. You can have insane bandwidth storage feeding directly into RAM used by both the CPU and GPU and this is pretty much my take from what Mark Cerny said in his first Wired interview and so ably demonstrated by the Spider-Man tech demo running on a PS5 devkit.

Exactly! Sony can avoid all these issues, because the PS5 is a closed embedded system, with no requirement for backward compatibility with 3rd party hardware and multiple 3rd party drivers, or hundred thousand titles of 3rd party software released during the last 25 years. They may do as they wish and tune the performance for one single hardware configuration, and fix any incompatibilities with emulation and hardware virtualization. There are only several thousand game titles from previous Playstations, so that should be manageable.

there are innate bottlenecks that fast solid state storage and RAM can't overcome and that's the fact that a PC is a bunch of external buses connected over a bunch of internal buses - and this design is fundamental to the PC architecture being so flexible and extensible

Soon there will be no other buses than PCIe - even internal processor buses are now based on the PCIe protocol, though with extension for multiprocessor cache coherency, and soon there will be USB4/ThunderBolt3 and DisplayPort2 with a PCIe-derived ThunderBolt data protocol.

late 2020 might see improved software solutions. Read speeds are close to 8GB/s already now. A better software stack could fix alot.

I wouldn't expect wonders from the current Windows team. Been a member of Microsoft Insider Program for the last five years, and all they can come with are subtle bugs that they don't even care to admit, save to fix, and stupid UX features that nobody asked for.

32GB main ram or more might be the solution then, with ram prices sinking this could be a faster solution even

RAM alone will not help much if the OS and applications cannot make efficient use of it.

It's not until recent times with AMD becoming relevant that 8 cores are getting more popular, with R7 chips that wont slow down atleast. With the arrival of next gen consoles people finally would need to upgrade.

Given that most motherboards can't really upgrade to a 8-core CPU and/or PCIe 4.0 on either of AMD or Intel platforms, that would be a large spending towards a high-end configuration, especially since new AMD sockets and DDR5 memory are on the horizon.

Cross platform games that need ssd's to play because it's not feasible on hdd will come and if its games the low end want they will upgrade. They have been sitting on what their using for many many years now.
If a game really needs an ssd to work, you won't be able to feasibly scale it down to work on a hdd.

Strictly saying, they promised to reduce load times - which may or may not work on a HDD or a slower SATA SSD depending on the implementation of these claimed improvements. I'm afraid cheap SATA SSD upgrades will not achieve game console performance, and fast PCIe 4.0 NVMe SSD would require a new expensive platform.

What percentage of those system stats are from China (or other extremely price sensitive regions)?

I don't think China is the low-cost PC market it used to be 15 years ago at the time of PlayStation 3, their GDP grew by several times and so did average wages. Anyone who needed a low-end PC back then now buys a mid-range Android phone instead.

 

[DmitryKo]

I think you are wrong if you really think nothing else will match the PS5's storage solution, anyways we don't even know what will be in there, maybe let's wait and see how things will turn out.

OK, let's consider some practical solutions.

SSD benchmarks show us how large block IO and deep command queue results in maximum possible throughput. Using block size of 256 to 1MB and queue depth of 8 to 16 commands would often lead to an order of magnitude better read-write performance than the default values of 4KB and QD 1.

So just reformat the disk to use 2 MB clusters and that should boost real-world throughput in random access loads, right?

(There will be a LOT of wasted spase on small files, but let's just ingnore it for this excercise of thought, since we only care about better random access speeds).


Unfortunately Windows has to maintain compatibility with legacy storage devices and drivers, and that might prevent them from effectively implementing these changes.

Enabling 2MB clusters today will only result in write amplification, when small 4KB buffers in the the IO subsystem will prevent efficient block IO resulting in unnecessary read-modify-write operations, when the entire large cluster will have to be read then written back to update only a small block of data.

First we need to change typical application usage patterns to accomodate zero latency and enormous read bandwidth of NVMe SSDs, and so far this did not happen, as the file handling APIs and block disk IO interfaces did not really change since 1993, even though current storage can work faster than DRAM of these days.

Consider some of the most obvious changes I mentioned above, like modifying the file system by introducing 8 KB and 16 KB physical sector sizes that match the write page size of NV flash memory, and 1 MB and 2 MB cluster size that matches the erase block size.

Only exFAT and NTFS currently support large clusters, but not FAT32 or ReFS. exFAT supports up to 32 MByte clusters, and by default Windows 10 assigns 128 KByte clusters for volumes below 512 GB, then comes in a progression of 256/512/1024/2048 KB etc. cluster sizes for 512GB and 1/2/4 TB etc. volume sizes. (exFAT formatted media also contains the Flash Parameters block, which can be used to optimize access patterns by the OS).
NTFS supports cluster sizes from 512 bytes to 64 KB (since Windows 2000), while 128/256/512 KB, and 1/2 MB clusters were added in Windows 10 version 1709 - though Microsoft does not recommend anything but 4 KB.

Unfortunately Microsoft only supports 512B and 4K physical sectors in the IO subsystem; other sizes are not supported, even though both NVMe and exFAT support arbitrary number of bytes per sector, and NTFS has some reserved fields that may be accomodated for this.
Most SSDs advertise 512e (emulation of 512B sectors) and some advertise 4Kn (4K Native, an Advanced disk format first introduced in Windows8/Server2012), but neither of these match the native write page size.


The problem is virtual memory paging (pagefile.sys) - Windows has always been using page size of 4 KB on both 32-bit (x86) and 64-bit (x64, x86-64) platforms, and so it does on the recent ARM64 platform.
Windows 10 kernel is closely related to Windows 2000 (NT 5.0) kernel, which ran on 16 MB of RAM (three orders of magnitude lower than my current 16 GB) and 5 GB hard disk drives (two orders of magnitude lower than current 500 GByte SSDs) with transfer rates of 5 MB/s (1.5 order of magnitude slower than current 200 MB/s hard disks, and three orders of magnitude slower than the fastest PCIe 4.0 NVMe SSDs that can achieve 5 GB/s).
Yet the page size did not really change through all these years (and neither has the NTFS file system, which remains at version 3.1 as was introduced in Windows 2000).

Nowadays all x64 (Intel64/x86-64) processors support page size of 2 MB (Page Size Extension flag), and most recent ones ones support page size of 1 GB (pdpe1gb flag), but Windows kernel has not been changed to fully support large pages on the same level as 4 KB pages.
Why? Mostly because of driver and application compatibility issues that such a giant leap, from 4KB to 2MB, would have created. Even on 64-bit Itanium, where the virtual memory subsystem supports eight different page sizes from 4 KB to 256 MB, they've chosen 8 KB page size as the least disruptive - and that was a dedicated server/HPC platform with practically no requirements for backward compatibility.

Future Intel processors might support arbitrary page allocation units, such as 64 KB page size, but it would take years before such support is widespread.​

And without support for large physical sectors - and related support for large memory page sizes - you cannot enable large clusters that match the properties of flash memory devices and eliminate overhead from small sector emulation and internal garbage collection in the SSD.

So there is a lot of work ahead to make the improvements happen, but it's only taking baby steps, with no announced commitment to solve the problem systematically. Not that I can really see how they could practically solve it without some clear break from existing hardware compatibility requirements - something that Apple does with OS X on an yearly basis.

 

[Deleted member 11852]

Soon there will be no other buses than PCIe - even internal processor buses are now based on the PCIe protocol, though with extension for multiprocessor cache coherency, and soon there will be USB4/ThunderBolt3 and DisplayPort2 with a PCIe-derived ThunderBolt data protocol.

PCIe will solve a lot of problems until, like its predecessor and its predecessors before them, it's not fast enough. This is the story of every I/O bus standard popularised: SCSI, PCI, Zorro, ADB, AGP, IDE, RS-232, Centronics, VESA, SATA, USB, Firewire, Thunderbolt. The need for a new faster bus probably isn't going anyway any time soon.

At some, inevitably, you're ripping out your motherboard to get a faster base architecture.

 

[DmitryKo]

PCIe will solve a lot of problems until, like its predecessor and its predecessors before them, it's not fast enough.

Sure, but right now Intel is skipping PCIe 4.0 and goes directly to PCIe 5.0 then 6.0, at least in the server/HPC market. That's 4 or 8 GB/s per lane, 63 or 126 GB/s for x16 link, and it would need quad-channel DDR5-8000 (PC5-64000) memory to fully saturate the bandwidth.

And it's basically another order of magnitude faster than today's fastest SSDs, if flash memory technology can keep up - or Intel and Micron will make fortunes from end sales and IP licenses for their 3D Xpoint (Optane/QuantX) memory...

 

[Deleted member 11852]

And it's basically another order of magnitude faster than today's fastest SSDs, if flash memory technology can keep up - or Intel and Micron will make fortunes from end sales and IP licenses for their 3D Xpoint (Optane/QuantX) memory...

Mark Cerny has already stated that cheap consoles next year will have far greater solid state performance than "today's fastest SSDs". Sony (and likely Microsoft) are looking to the future. I'm not convinced the solid state tech in the PS5 is based on PCIe 4.0.

It's also worth noting that Intel's numbers for their predicted benefits of PCIe 4.0 for gamers do not, and can not, factor in how game engines may be expected to perform in 1-5 years once cheap consoles have virtually eliminated loading times. What demands will be put upon this will not be known until devs are actually demanding the hardware do this.

 

[Nisaaru]

You really expect them to add some custom "bus" for flash?)

PCIe 4.0 with 4 lanes provides 8GB/s. If they really need more and the flash can actually provide that they would just add more lanes.

 

[Shifty Geezer]

It's not about the bus, but the entire software stack. See here. PS5 could well be using PCIe to connect to the SSD. A PC using the same PCIe connection wouldn't be getting the same fast load speeds because the file system isn't geared towards it.

 

[Globalisateur]

It's not about the bus, but the entire software stack. See here. PS5 could well be using PCIe to connect to the SSD. A PC using the same PCIe connection would be getting the same fast load speeds because the file system isn't geared towards it.

wouldn't ?

 

[PSman1700]

OK, let's consider some practical solutions.

If the PS5 is going to be much faster then any pc out there, both now and i the future, and you seem so sure, why didn't the genius cerney not say that instead of just faster then 'current today pc SSD solutions'? I mean he might not be the best hardware expert in the world but he sure would know what to say in a interview about their next generation console. Saying a future product is faster then today's tech... well kinda obvious for the most, atleast now on most forums people seem to think it's cause he compared it to PCI-E gen 3 standards.

Also, we really don't know what it's going to be, all we know is it's going to be faster then a spring 2019 pc SSD solution. Aside from that, hyping has happened before, latest with PS4 which was going to be that SuperCharged PC Architecutre etc, and we all know what we got.

https://www.polygon.com/2013/2/20/4009940/playstation-4-tech-specs-hardware-details

I just want to see it before i believe it, not PR like 8k 120fps faster then anything out there etc

wouldn't ?

What you think yourself?

 

[pcchen]

It's not about the bus, but the entire software stack. See here. PS5 could well be using PCIe to connect to the SSD. A PC using the same PCIe connection would be getting the same fast load speeds because the file system isn't geared towards it.

I'm not that sure about page size though. It's common to think that larger page size is more efficient, but it's rarely that obvious when running real world workloads. Most OS are still using 4KB pages, though iOS changed to 16KB on 64 bits systems. Of course, maybe gaming workloads are different, but if that's the case then consoles should be using those sizes right now (I don't know if that's the case though).

On a console a simpler file system can be used to reduce overhead, as most large files are static, but the gain probably won't be too large (maybe in single digit percentage). High performance games on PC also uses memory mapped files to reduce API overhead. So I think the advantage is there, but probably not huge.

 

[Shifty Geezer]

If the PS5 is going to be much faster then any pc out there, both now and i the future, and you seem so sure, why didn't the genius cerney not say that instead of just faster then 'current today pc SSD solutions'?

He can't predict the future. He has no idea what MS will do with their PC OS. Or Apple for that matter.They also may employ alternative solutions like massive RAM amounts and RAM caching.

I just want to see it before i believe it, not PR like 8k 120fps faster then anything out there etc

Well, it's already been demoed at 19x faster loading for Spider-Man.

 

[PSman1700]

He can't predict the future. He has no idea what MS will do with their PC OS. Or Apple for that matter.They also may employ alternative solutions like massive RAM amounts and RAM caching.

Going to be more expensive as RAM will always cost more then SSD's. Anyway, isn't it possible that a portion, or a small soldered SSD is going to be in the PS5? A whole 1TB seems much and will be used to install games to aswell.

 

[Jay]

Strictly saying, they promised to reduce load times - which may or may not work on a HDD or a slower SATA SSD depending on the implementation of these claimed improvements. I'm afraid cheap SATA SSD upgrades will not achieve game console performance, and fast PCIe 4.0 NVMe SSD would require a new expensive platform.

I agree with you.
But if we're talking about the low end, then they will be willing to put up with a very sub par experience running off ssd. The kind of experience no one believes people actually put up with but do.

The thought that studios will wait (few years+) for the low end to upgrade to ssd's just isn't going to happen. The low end will just sit on what they have while they can, that's the nature of the low end.

Middle section will probably upgrade/have SSD and enough memory to give it a usable experience compared to console.

High end will throw money at it, pcie 4+,GB's memory, etc.

I could see MS creating a gaming file system that is used on Scarlett, but can also be used on PC, maybe some type of containerized game install.

 

[Shifty Geezer]

Going to be more expensive as RAM will always cost more then SSD's.

You asked why Cerny wouldn't claim faster than future tech. I explained why. What has the cost of any solutions Cerny can't predict got to do with anything?

 

[DmitryKo]

If the PS5 is going to be much faster then any pc out there, both now and i the future, and you seem so sure, why didn't the genius cerney not say that instead of just faster then 'current today pc SSD solutions '?

The PC can be faster, PCIe 5.0 and DDR5 are just around the corner. But it would require a lot of effort to move the PC ecosystem towards NVMe oriented programming design, due to backward compatibility issues which PS5 can simply avoid.

They also may employ alternative solutions like massive RAM amounts and RAM caching.

Sure, but caching would not improve start up times when you need to load large data sets for the first time. You have to read it from the media first - if your random disk read throughput is slow, RAM cache will not improve it.

 

[Jay]

Your quote attributes me, but I didn't write that.

Sure, but caching would not improve start up times when you need to load large data sets for. You have to read it from the media first - if your random disk read throughput is slow, RAM cache will not improve it.

Wouldn't improve start times but would make the games more viable.
May have to use combination of memory and LOD levels etc to make it work.
Consoles will also probably be using the increase in random access speed and throughput to reload assets it used to have to keep in memory, pc can keep it resident in ram.

 

[DmitryKo]

It's common to think that larger page size is more efficient, but it's rarely that obvious when running real world workloads. Most OS are still using 4KB pages, though iOS changed to 16KB on 64 bits systems.
High performance games on PC also uses memory mapped files to reduce API overhead.

In context of improving file system read throughput by using large disk sectors (such as 8, 16 or 32 KB to match write page size of flash memory) and large disk clusters (258KB - 2 MB to match erase block size), large virtual pages would allow big continuous buffers for direct memory access, without additional translation layers, software logic and TLB/PTE management issues. This would also apply to memory mapped files, which are implemented using virtual memory.

Of course there are downsides - big clusters are not effective for disk space usage with a lot of small files (250.000 files on my current Windows installation), and typical memory allocation strategies in most programming language runtimes are still based on small pages, as you mentioned.

Current x86-64 processors support only 4 KB, 2 MB and 1 GB pages, but apprently 2 MB is too large to use as a default allocation unit for compatibility reasons. Even though Windows has a limited support for such 'large pages' (and Linux includes 'transparent huge pages'), these may become impossible to allocate due to memory fragmentation from standard 4KB pages.


The good news, Intel is slowly taking some steps to address the problem. First, the Sunny Cove architecture introduces 5-level paging, with an additional level of virtual memory translation to support 57 bits of virtual address space and 52 bit physical memory (up to 4 PB).

Then there is a recent 2018 patent US9858198 "64KB page system that supports 4KB page operations", which enhances the 5-level page table scheme with 64 KB page allocations while transparently supporting 4 KB sub-pages for backward compatibility (found here). The key points are:

1) the page table entry (PTE) is expanded to 128 bits to support a 64-bit physical address and additional attributes;
2) the memory allocation unit is changed to 64 KB, while still allowing the 4K pages;
3) 64 KB pages allow standard 4-level lookup to map full 64-bit physical memory to 64-bit virtual memory, and 5-level lookup scheme remaps 64 KB pages into 4 KB subpages (or generally any allocation unit size to a smaller page size).

This way, the OS kernel can gradually start using the new hierarchy and transition to a larger allocation unit while still supporting operations with a smaller page size.


However it remains to be seen if and when the improvements from this patent will be implemented in actual CPUs and supported by the operating systems. And that's certainly not happening in the near future.

Excerpts:

A general problem of paging systems that convert virtual addresses to physical addresses involves a size of pages on a free memory list. Typically a paging system that supports allocation of memory in 64 KB pages does not support a 4 KB page size. In order to be backwards compatible with some currently operating systems and applications, a 4 KB page size needs to be supported. Typically, support of the 4 KB page size is accomplished by allocation of memory in 4 KB units. Pages of size 64 KB may be assembled from 4 KB units by one of several methods. Support of a 4 KB page size by a paging system that supports the 64 KB size in this manner may result in serious inefficiencies in lookup when in the 64 KB page size lookup mode. Typically a whole process or segment must use the same page size, so if a legacy application requests protections or addresses that are aligned on a 4 KB boundary, all of the pages are to be stored pages as 4 KB pages.

In addition to backwards compatibility, a related issue involves memory caches, which typically match high order address bits against a tag stored with each memory value and use low order address bits to directly index into the cache memory. With a 4 KB page size, only the lower 12 address bits are unchanged by address translation, which can add complexity to implementation of a direct mapped cache to index the memory location. Use of 64 KB pages is conducive to design of larger memory caches.

Another general problem of paging systems is that the operating system needs to manage the mapping of individual pages of physical memory into various virtual memory spaces, which can require tracking information about the individual pages. A larger page size can result in a reduction of page management that the operating system performs and can also result in reduction of tracking information to be stored. However, backward compatibility is of major importance for many operating systems.

Embodiements provide an efficient scheme for an operating system to support allocation of memory in units of 64 KB, while fully supporting applications that operate with a 4 KB page size. Note that these sizes are illustrative, and that more generally the embodiments presented herein permit an operating system to support operating on pages smaller than the memory allocation unit used in a free memory list. Embodiments presented herein also permit mapping of larger address spaces than current paging systems without an increase in the number of page walks required. Additionally embodiments provide means to self-map page tables into virtual address space.

… a new paging system (“ VA64 ”) includes the following features:

1 ) VA64 increases a page table entry (PTE) size from 64-bits to 128-bits, which provides space to support a larger physical address (PA) range.
2 ) VA64 changes a memory allocation unit size from 4 KB to 64 KB, which allows 4K page table entries (PTEs), each PTE with size 128-bits.
3 ) VA64 provides a 4-level lookup that supports a 64-bit virtual address (VA) range and that can map virtual to a 64-bit physical address (PA) range.
4 ) VA64 supports a 5-level lookup that allows mapping a 64 KB page into 4 KB sub-pages, each of which has its own separate page table entry.
5 ) VA64 supports legacy 2 GB and 1 TB large page sizes as well as supporting native 256 GB and 1 TB large pages, and even native 4 PB pages if desired.
6 ) VA64 allows optimization of client and low end systems by reduction in page table walks that result in 52-bit and 40-bit virtual address ranges.​

As a result of support of the mapping of the 64 KB page into 4 KB subpages, VA64 directly supports all currently defined operations on 4 KB pages, including independent protection bits per 4 KB page and arbitrary 4 KB-aligned address mappings. VA64 also supports OS kernel page management on 4 KB boundaries, even when the OS kernel allocates memory in 64 KB units. As a result of support of large pages, VA64 supports all divisions of the virtual address range into pages that an existing paging system such as Intel Corporation's IA-32e paging system supports. Therefore, VA64 supports applications and hardware devices that work with a 4 KB-page Windows® OS kernel, while also taking full advantage of 64 KB pages when 64 KB pages can be used.

The capabilities of VA64 can be adopted gradually by the OS kernel, rather than requiring them all to be supported in the first generation VA64-capable OS kernel. For example, a VA64-capable OS kernel could start by mapping all pages to current sizes (e.g., 4 KB/2 GB/1 TB in Intel Corporation's IA-32e paging system), but changing to a new page table format. After the change in page table format, the OS kernel could be modified to map virtual memory in 64 KB units and change to store 64 KB pages in its free list. Then the OS kernel could start using 64 KB pages whenever alignment and protections permit, and add support for other VA64 capabilities.

Finally, OS kernel variants could be defined to support 64-bit, 52-bit and 40-bit VA ranges for server, client, and low end systems. These address range choices could be supported using compile-time constants with a common code base. Alternately, hardware could be provided that allows the reduced number of page walks permitted by the reduced address ranges, while using a single executable version of the OS kernel.