Next-gen console versus PC comparison *spawn

The question was which exact software and hardware implementations the above Sony patent (if granted) could prevent on Windows.

Since a) istallable file systems were already implemented decades ago, b) SSD controller designers have no interest in hardware data compression, and c) game developers are unlikely to deploy their own custom solutions - the patent is not relevant for the PC in any meaningful way, and it cannot prevent Microsoft from improving the throughput of their block I/O stack to match the speeds of NVMe disks and recent PCIe revisions.

 

Imo, this is "easy" to implement on PC and I doubt Sony will do anything against it or want to use the solution on PC. But this need the collaboration of Microsoft with SSD industry and this is mostly software. SSD PC Industry don't need asymetrical read/write speed and they don't need to use SRAM.

 

To you're whole post, I refer you back to my post of over-a-week ago.

Patents are their to protect specific methods and I've not seen anybody use anything like Sony's specific method. The big reason for this is because filesystems and I/O systems mature slowly and solid state storage is still relatively new compared to decades-old Winchester-derived mechanisms.

The advantages of an implementation like Sony outline are clear; increasingly I/O isn't reading a few bytes/kilobytes here and there, it's reading megabytes/gigabytes and to make that faster you need to rethink I/O top-to-bottom - tossing out the granularity now it's not required while offeriong provision for bundling smaller files together to minimise wasted space.

You've mentioned data compression several times but, now, having read Sony's patent application dated 6 April 2017, I can't see any description of data compression. Are we looking at the same application?

 

We had this discussion before.

 

The patent does not describe file systems or APIs - only data decompression at the NVMe controller that performs the translation of a LBA number to a memory address.

The 'archive file system' looks conceptually similar to write-once optical media which uses contiguous allocation and read-only access, with larger 64KB blocks to improve read speeds. Then again Windows already supports large clusters in standard NTFS and exFAT file systems, and the new file compression algorithms use contiguous allocation and no longer suffer from heavy fragmentation of compressed files. It only takes a few simple steps forward, such as support for large 64 KB sectors in NVMe drives and CPU support for 64KB page sizes.

BTW Phison has shown the PS5018-E18 controller that offers 7.0 GByte/s sequential read and write speeds and 1M IOPS, with SSDs expected by Fall 2020. It probably won't improve random read/write speeds though without these large LBA sectors.

So are the limitations.

I refer you to my reply to your post above.

It's US20170097897A1 - if you search the full text on Google Patents, decompression is explicitly mentioned in claims [46] through [85].

 

This is good I hope soon Microsoft will think about doing something on this side to improve SSD NVME speed. And maybe use the Xbox Scarlett to do something for improve SSD speed on gaming side, treat it as a special case on PC.

 

You can't patent an API.

Thanks, I was looking at a different patent.

 

FYI, Xbox Series X includes a new disk I/O subsystem, the DirectStorage API - which will be ported to Windows 10 - as well as hardware decompression, and the Sony patent did not prevent it from happening.

 

This is good, at least there is a standard and a great idea to use a special format for further texture compression on disk. Curious to see the Sony solution now I bet on at least 8 GB compressed.

 

The XSX falls nicely in line with my theory that the GPU will be end up being a mid range GPU by the time it launches. Corona impact aside, it'd be competing against RDN2 and 3070 level range.

 

I think it really depends on what Ampere will bring under the hood. Right now the rumor is a 50% performance improvement over Turing, that would put the 3070 above both consoles and the 3060 next to the 10TF mark. However, it's not clear whether those are raw performance numbers or overall performance taking into account the optimizations/changes NVIDIA may have implemented.

I do believe new consoles will be closer to the 3060 than to the 3070.

 

I'm taking into account the benefit the consoles (XSX mainly for now since it's known) will have the ability to have their toolset fully catered to the hardware and can squeeze more out of the platform. On a raw TF basis, I agree with you. In demonstrable terms, the console HW should be able to surpass it's counterpart.

 

BTW, NVMe v1.3 introduces controller hints for optimal I/O boundary, optimal write size, preferred write granularity/alignment, and preferred deallocation granularity/alignment.

This extends the LBA sector size (LBADS), which can be any degree of 2 in NVMe 1.x specs, but is traditionally set to 512 Bytes by most SSD devices.


StorNVMe.sys driver in Windows 10 currently supports optimal I/O boundary hint (NOIOB), but not write size (NOWS) or other alignment/granularity hints (NPWG, NPWA, NPDG, NPDA).

https://docs.microsoft.com/en-us/windows-hardware/drivers/storage/stornvme-feature-support


https://nvmexpress.org/specifications/

Identify – LBA Format Data Structure, NVM Command Set Specific

LBA Data Size (LBADS):
This field indicates the LBA data size supported. The value is reported in terms of a power of two (2^n). A value smaller than 9 (i.e., 512 bytes) is not supported.

Identify – Identify Controller Data Structure

Maximum Data Transfer Size (MDTS): This field indicates the maximum data transfer size for a command that transfers data between host-accessible memory (refer to section 1.6.17) and the controller ... The value is in units of the minimum memory page size (CAP.MPSMIN) and is reported as a power of two (2^n).

Identify – Identify Namespace Data Structure, NVM Command Set Specific

Namespace Optimal I/O Boundary (NOIOB):
This field indicates the optimal I/O boundary for this namespace. This field is specified in logical blocks. The host should construct Read and Write commands that do not cross the I/O boundary to achieve optimal performance.

Namespace Optimal Write Size (NOWS):
This field indicates the size in logical blocks for optimal write performance for this namespace.

Namespace Preferred Write Granularity (NPWG):
This field indicates the smallest recommended write granularity in logical blocks for this namespace.

Namespace Preferred Write Alignment (NPWA):
This field indicates the recommended write alignment in logical blocks for this namespace.

Namespace Preferred Deallocate Granularity (NPDG):
This field indicates the recommended granularity in logical blocks for the Dataset Management command...

Namespace Preferred Deallocate Alignment (NPDA):
This field indicates the recommended alignment in logical blocks for the Dataset Management command...

 

There are two recently ratified specifications from the upcoming NVMe 2.0 standard:

NVMe Zoned Namespaces (ZNS) and ZNS Command set (TP 4053), primarily sponsored by Western Digital who authored to their Zoned Block Commands (ZBC) and Zoned ATA Command set (ZAC) for SMR hard disks.

https://nvmexpress.org/new-nvmetm-s...-namespaces-zns-as-go-to-industry-technology/
https://blog.westerndigital.com/nvme-spec-ratification-zns-ssd/

Zones are based on physical flash memory dies and use sequential writes and append-only write policy, with native flash memory write/erase block I/O sizea. This arrangement moves garbage collection and wear levelling to the OS driver level from the SSD controller, reducing write amplification and overprovisioning, and improves access latencies by eliminating the Flash Translation Layer (FTL). This is now part of NVMe 1.4a.

ZNS protocols are primarily aimed at enterprise storage at this point, with initial support coming in Linux Kernel 5.9.

https://www.anandtech.com/show/15959/nvme-zoned-namespaces-explained/3
https://blog.westerndigital.com/what-is-zoned-storage-initiative/
https://zonedstorage.io/introduction/zns/

From Open-Channel SSDs to Zoned Namespaces
youtu.be/CaCURnVcql4
youtu.be/mwnbsH1bYuo
usenix.org/conference/vault19/presentation/bjorling


NVMe Simple Copy Command (TP 4065)
New NVM I/O command that copies logical blocks from one or more logical block ranges to a single contiguous destination logical block range

https://www.phoronix.com/scan.php?page=news_item&px=NVMe-Simple-Copy-Linux
https://snia.org/sites/default/file...nds-NVMe-2.0-Specification-Preview.pdf#page=6
https://www.snia.org/educational-library/nvme-20-specification-preview-2020
https://www.snia.org/sites/default/files/SDCEMEA/2020/3 - Javier Gonzalez Zoned namespacese.PDF




PS. NVMe 2.0 specification has been released on June 4, 2021:
https://nvmexpress.org/nvme-2-0-specifications-infographic/
https://nvmexpress.org/everything-y...0-specifications-and-new-technical-proposals/

https://www.anandtech.com/show/16702/nvme-20-specification-released
https://www.tomshardware.com/news/nvme-2-0-supports-hard-disk-drives

 

FYI, Windows Insider WDK headers (NVMe.h storport.h winioctl.h) are being updated to support NVMe 1.3/2.0 features above, specifically Namespace-specific hints for optimal I/O boundary and write size and preferred write/deallocate granularity/alignment (WDK build 19624), and Zoned Namespaces with ZNS command set (WDK build 20246). LBADS has been already supported for a while. These headers define data structures for StorPort port class driver and StorNVMe miniport driver to use.

Unfortunately Windows Iron (21H1) turns out to be another Insider-only development branch, similarily to Windows Manganese (196xx), so these features will be generally available no earlier than Windows Cobalt (21H2).

 

Consoles are fairing better this time for sure. GPUs are acceptable, CPUs are good and I/O is great. Less memory than some expected, but probably fine given that better I/O and game size limits are going to limit the need for more memory IMO.

 

If the videos at the beginning of last generation are anything to go by, we should hold off on comparing a GPU for now. It'll be a different setup now compared to a year from now.

Zero chance of creating a PC with similar specifications for the price. Especially considering the IO system.