Blazing Fast NVMEs and Direct Storage API for PCs *spawn*

Discussion in 'PC Hardware, Software and Displays' started by DavidGraham, May 18, 2020.

  1. DavidGraham

    Veteran

    Joined:
    Dec 22, 2009
    Messages:
    3,279
    Likes Received:
    3,527
    Not exactly, even 6 months ago there were solutions that are way faster than PS5. Gigabyte launched an SSD that is capable of reaching 15GB/s, it's called AORUS Gen4 AIC SSD 8 TB.
    https://www.amazon.com/Gigabyte-Performance-Advanced-Solution-GP-ASACNE6800TTTDA/dp/B081BSF14V

    Other less exotic solutions include the Viper VP4100, Corsair MP600, and AORUS NVMe Gen 4, all of which already provide 5GB/s speeds.
     
  2. Arwin

    Arwin Now Officially a Top 10 Poster
    Moderator Legend

    Joined:
    May 17, 2006
    Messages:
    18,038
    Likes Received:
    1,627
    Location:
    Maastricht, The Netherlands
    But that is not the whole story. He mentioned that the way the SSD is integrated into the system is beyond what Windows offers currently. And I do believe that the optimized file system, the additional interrupts and the integration into the graphics pipeline all combine to make it very effective.

    That is not to say PC won’t catch up of course. Eventually they always do and they certainly likely will have by the time the new consoles are out, perhaps not quite as efficient but then through brute force. Still it will be interesting to see when the PC can match the OS5s extremely low latency from SSD to graphics pipeline.
     
    egoless, Jov, DSoup and 1 other person like this.
  3. DSoup

    DSoup meh
    Legend Veteran Subscriber

    Joined:
    Nov 23, 2007
    Messages:
    12,494
    Likes Received:
    7,746
    Location:
    London, UK
    These parts do have theoretical speeds rivalling/exceeding PS5, what these parts do not solve is the problem of pulling data off the SSD into RAM, for CPU unpacking/decompression and for anything for the GPU, shoving that all over another bus before it's ready to use.
     
    Mitchings and milk like this.
  4. DavidGraham

    Veteran

    Joined:
    Dec 22, 2009
    Messages:
    3,279
    Likes Received:
    3,527
    Shouldn't DirectStorage be the PC answer to all of this?
    Several games don't require compression at all on PCs, a famous case is Titanfall, it came with 30GB of uncompressed sound data because the developer wanted the game to run on low end CPUs.

    Yes they were, it's pointless for the engineer to run the demo with reduced detail on a laptop and compare that directly to PS5. He did it to prove a point, that current gen runs the new features. Epic themselves stated the demo runs with good performance on current NVMe drives, they didn't mention once that the demo got downgraded. In fact we now have double confirmation that current PCs can run the demo.


    First: RTX 2070 Super and fast NVMe drive runs the demo with pretty good performance.
    Second: RTX 2080 laptop and Smasung Evo 970 ran the demo with 40fps.

    So that's two similar information bits in a row right now, with more likely incoming in the upcoming weeks
     
    PSman1700 likes this.
  5. DSoup

    DSoup meh
    Legend Veteran Subscriber

    Joined:
    Nov 23, 2007
    Messages:
    12,494
    Likes Received:
    7,746
    Location:
    London, UK
    For the last time, no. A software API does not negate all of the driver and device I/O that Windows does. The fact you keep suggesting this strongly suggests you need to spend a good hour reading this Microsoft documentation.

    Once you realise how much work is actually involved in pulling one asset from a pack (or even an isolated file) in Windows, you'll understand why DirectStorage cannot undo all of that because fundamentally to Windows your PC is just a bunch of components that needs Windows, its filesystem and Windows kernel drivers to move small bits of data around in an incredibly administratively burdensome way. I'm talking thousands of lines off code and API calls just to open and find data in a file because to get to the data you need the device drivers for the storage system, all of the interconnect buses between the CPU and RAM and the storage and Windows ranges each movement and handover.

    The only way Microsoft can fix this in software is to toss pretty much most of the Windows code in the bin, along with the filesystem, and start over. This will break everything. It's a non starter.

    This is why no amount of money can buy an SSD RAID array that eliminates all load times in Windows games. There are too many individual pieces of hardware involved, all need to talk to each other through a different bunch of device drivers that themselves cannot communicate directly with each other. And this chain doesn't operate at the same speed so bottlenecks are common, and throughout the process Windows manages the data in ways to keep it secure from any rogue code that might be running.
     
  6. chris1515

    Veteran Regular

    Joined:
    Jul 24, 2005
    Messages:
    4,679
    Likes Received:
    3,578
    Location:
    Barcelona Spain
    #6 chris1515, May 19, 2020
    Last edited: May 19, 2020
  7. zupallinere

    Regular Subscriber

    Joined:
    Sep 8, 2006
    Messages:
    736
    Likes Received:
    86
    With Windows it's Bueracracies All The Way Down.
     
    egoless and DSoup like this.
  8. Shifty Geezer

    Shifty Geezer uber-Troll!
    Moderator Legend

    Joined:
    Dec 7, 2004
    Messages:
    43,525
    Likes Received:
    15,981
    Location:
    Under my bridge
    Is this open to games?
     
    PSman1700 likes this.
  9. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    14,162
    Likes Received:
    5,463
    @DSoup Microsoft is claiming that DirectStorage will reduce 5 CPU cores worth of processing (3 hardware decompression, 2 improvements to I/O) down to one tenth of one core. They haven't explained how the new I/O system works, but it sounds like it basically bypasses your typical filesystem access and all of that Windows OS bureaucracy.

    https://www.eurogamer.net/articles/digitalfoundry-2020-inside-xbox-series-x-full-specs
     
    Lightman, tinokun, John Norum and 4 others like this.
  10. Jay

    Jay
    Veteran Regular

    Joined:
    Aug 3, 2013
    Messages:
    2,615
    Likes Received:
    1,683
  11. DavidGraham

    Veteran

    Joined:
    Dec 22, 2009
    Messages:
    3,279
    Likes Received:
    3,527
    But that's precisely what Microsoft is promising .. I don't see how can you be so decisive with this no, like that. DirectStorage will be basically a lower level API just like DX12/Vulkan, bypassing a lot of OS/driver overhead.
     
    boxerlvr, Rootax, tinokun and 3 others like this.
  12. DSoup

    DSoup meh
    Legend Veteran Subscriber

    Joined:
    Nov 23, 2007
    Messages:
    12,494
    Likes Received:
    7,746
    Location:
    London, UK
    On Xbox Series X, hence the references to Zen cores as the metric to demonstrate performance improvements. What Microsoft's Andrew Goossen is demonstrating here is how much the Velocity Architecture I/O would have saturated the software stack if Microsoft had not changed it and introduced DirectStorage.

    Microsoft are "precisely" "promising" nothing with regard to the Windows version of DirectStorage, other than "we plan to bring to Windows". A Windows PC is nothing like an Xbox. Microsoft know exactly what hardware and specification of hardware is in every Xbox meaning they are not reliant on a bunch of device drivers to manage the I/O handover. PCs will still have two RAM pools and much longer chain of interconnect buses to get data from the SSD to the controller to the bus to main RAM for the CPU and across the PCI to GDDR.

    You need to stop believing that this software unicorn can re-architect your hardware, because it cannot. Every single desktop OS works like this for a reason, and can only work like this. The only way to change it would be for an OS to mandate very specific hardware and remove the need for a bunch of different device drivers. Have a look at Device Manager at all the different device drivers that make your PC work. Learn how your PC works, then you'll understand.

    No, for the same reason.
     
    #12 DSoup, May 20, 2020
    Last edited: May 20, 2020
    Ivan likes this.
  13. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    14,162
    Likes Received:
    5,463
    @DSoup why would Microsoft bring directstorage to pc if it didn’t address any issues? I’m expecting it’ll work around some file system performance issues, even though it can’t address the issues with the hardware architecture.
     
    tinokun, AzBat, BRiT and 3 others like this.
  14. Unknown Soldier

    Veteran

    Joined:
    Jul 28, 2002
    Messages:
    2,461
    Likes Received:
    178
    DSoup said as much ;)

     
  15. Unknown Soldier

    Veteran

    Joined:
    Jul 28, 2002
    Messages:
    2,461
    Likes Received:
    178
    But DSoup won't know what MS know. If MS feel there's a benefit, even if it's 10%, MS will add DirectStorage to DirectX Ultimate for PC. Any benefit is good. Will it reach the XBSX level benefit? I highly doubt but again any benefit would be good.
     
  16. DavidGraham

    Veteran

    Joined:
    Dec 22, 2009
    Messages:
    3,279
    Likes Received:
    3,527
    We know this chain will always be here, in fact it is one of PC stregnths (the double RAM pool), to allow for more powerful hardware without compromise. This isn't the crux of the problem at all by the way.

    You previously stated that the main problem is the file system and the OS/APIs, now when Microsoft is stating they can alleviate alot of that you are just going to shrug it off?

    Whose to say that DirectStorage will not require compatible SSDs just like DX requires compatible video and audio cards?

    And you need to stop this authoritative confirmatory tone about something you know basically nothing about.
     
    boxerlvr, Rootax, AzBat and 3 others like this.
  17. DSoup

    DSoup meh
    Legend Veteran Subscriber

    Joined:
    Nov 23, 2007
    Messages:
    12,494
    Likes Received:
    7,746
    Location:
    London, UK
    No, two pools, or rather the existence of a further bus and device drive (PCI, GPU) exasperates the problem.

    Microsoft's comments are about Xbox, not PC. They have said and promised nothing regarding PC other than planning to bring DirectStorage to Windows. If you think this, please cite it or stop claiming this.

    How does compatible SSD drive help? How does that undue the chain of SSD -> controller -> southbridge -> RAM -> Northbridge -> GDDR -> GPU. How does that change anything? Your comments only demonstrate you don't know how your PC works, or how for data to transfer from SSD to RAM you need to involve the device driver for the SSD, the overarching interface driver, and the interconnect bus driver. And that's just for starters. What different does one SSD or another make?

    Please. Read the links I posted before. It explains how Windows is architecturally designed.

    I've written Windows device drivers. You don't even appear to understand what they are. :???:
     
    #17 DSoup, May 20, 2020
    Last edited: May 20, 2020
    Ivan, chris1515 and Unknown Soldier like this.
  18. DavidGraham

    Veteran

    Joined:
    Dec 22, 2009
    Messages:
    3,279
    Likes Received:
    3,527
    Their message to DigitalFoundry certainly came across as planning to bring the same kind of improvements to PCs, but even if they are not the same, there stands to be some definite improvements otherwise why would they even bother?

    There are two problems to the current paradigm that you gracefully explained:
    1-Too many buses
    2-Cumbersome File Systems/APIs/OS
    Are you saying there is no way in hell any of these problems would be alleviated by a new API?

    Pulling the old "I've written drivers and you didn't" trick seems both childish and immature at this point, especially in the face of the massive shift in storage paradigms, I could throw this right back at ya by asking how many APIs have you written? For which the answer would be?
     
    boxerlvr, AzBat and PSman1700 like this.
  19. DSoup

    DSoup meh
    Legend Veteran Subscriber

    Joined:
    Nov 23, 2007
    Messages:
    12,494
    Likes Received:
    7,746
    Location:
    London, UK
    I didn't read it as Microsoft promising improvements. They just they're planning to bring it Windows. Why would they bring an API that des nothing to Windows? Because Microsoft need API parity for games that work on both Xbox and Windows. Even if DirectStorage effectively does nothing - it needs to be there for game code using it on Xbox not to break when running on PC.

    A new API does not make all the buses in your PC disappear. This is how your hardware is all connected to each other. This won't change. DirectStorage is not a new filesystem. Could it bring some improvements? Sure but given the current Windows driver architecture, I'm not seeing how unless Microsoft want to fundamentally change the Windows model. They never have, we've never had a revolution in architecture, only evolution.

    Bullshit. You accused me of using "this authoritative confirmatory tone about something you know basically nothing about.". If you attack someone's perceived knowledge don't be surprised them they cite their experience or credentials. You invited that response.

    As for your query on APIs, I've also written a lot of APIs for bespoke hardware and shared software libraries, far too many to remember them all. I bet most experienced programmers have. Anybody who creates something for use by others has written an API, that is a vast, vast amount of coding effort in the world. But I don't see the relevance?
     
  20. DavidGraham

    Veteran

    Joined:
    Dec 22, 2009
    Messages:
    3,279
    Likes Received:
    3,527
    Highly unconvincing to be frank, If DirectStorage is directed mainly at Series X, then there would be no need to bring it to PCs, console APIs don't get transferred to PCs to maintain feature parity, there is no need to (especially with something completely busted as you describe it), games on PC work with the already established systems. You don't
    See Sony bringing their APIs on PC.

    But it can enable various workarounds. Give rise to new devices with new capabilities.

    You don't know that, that was my original remark in the first place, you don't know what Microsoft's plan for DirectStorage is, they might be planning for a revolutionary change (just like Series X), they might not be, we still don't know, yet if we follow your line of thinking we would have already declared their effort worthless.

    It's very good that you know about this rule, sadly you don't seem to follow it at all, you invited this response and others when you used a confirmatory tone about something you know nothing about "DirectStorage", and when you constantly implied my (and others) lack of knowledge about basic filesystem/OS principles. No one doubted your knowledge and experience in the first place, yet you proceeded on your own to hand everyone around you (You are an ignorant) card, and that behavior is simply unacceptable and quite frankly rude.
     
    boxerlvr, Rootax, AzBat and 1 other person like this.
Loading...

Share This Page

  • About Us

    Beyond3D has been around for over a decade and prides itself on being the best place on the web for in-depth, technically-driven discussion and analysis of 3D graphics hardware. If you love pixels and transistors, you've come to the right place!

    Beyond3D is proudly published by GPU Tools Ltd.
Loading...