Blazing Fast NVMEs and Direct Storage API for PCs *spawn*

Discussion in 'PC Hardware, Software and Displays' started by DavidGraham, May 18, 2020.

  1. milk

    milk Like Verified
    Veteran Regular

    Joined:
    Jun 6, 2012
    Messages:
    3,686
    Likes Received:
    3,735
    This is a really good point and I see it as a perfectly valid consideration. I just want to add some nuance to it, despite agreeing on a broader range.

    Even if you have 16Gb RAM on next gen machines, you won't be filling that up to the brim with static assets brought from storage. Of course we never did that before, we need a place to store game state, physics/animation data, dinamuc lighting, framebuffers, volumetric data etc... But the thing is, once you know you can grab random assets from storage at a finger's snap and at fine granularity, engines might start using much more of those 16Gbs for the aformentioned dynamic stuff and the actual pool for static asset data might actually shrink to as low as 4Gb or less. So that will trade in more agressive constant streaming for more available RAM to be used for the actual cool shit.

    Also, I can think of many scenarios where you are constantly cycling through repeated data without even noticing it.

    Imagine SpiderMan, you are cruising through NY. In the game, there are four types of food stand (hypothetical here) Hot Dog, Burgers, Ice-cream and Boiled goose. As you cruise across town at speed, you may cross through nultiple dozens of those within a minute or so of gameplay, yet there is usually only one of those on view at full detail LOD. That means the game is streaming in, out, and in again the data for those carts dozens of times every few seconds. Something a game would never do past gen, but now becomes sensible.
     
    #441 milk, Nov 20, 2020
    Last edited: Nov 20, 2020
    PSman1700, Remij and pjbliverpool like this.
  2. milk

    milk Like Verified
    Veteran Regular

    Joined:
    Jun 6, 2012
    Messages:
    3,686
    Likes Received:
    3,735
    Or an even simpler example here: Imagine a shooter where you constantly switch weapons. On last gen(s), all the weapon models (and related animations, sound, particle effects, UI graphics, etc) would stay resident in RAM constantly. Now it can all be pulled out from storage at weapon switch. That means, during a single level, a player can potentially switch weapons enough times for the game to pull thousands of GB through the streaming system, all by cycling through a relatively small set of repeating data.
     
    #442 milk, Nov 20, 2020
    Last edited: Nov 21, 2020
  3. milk

    milk Like Verified
    Veteran Regular

    Joined:
    Jun 6, 2012
    Messages:
    3,686
    Likes Received:
    3,735
    Now imagine all sounds are streamed exactly on demand from SSD. Every time you shoot a gun, RAM only has the first 1/4 second of that sound clip resident in ram, and pulls the rest as it starts playing. That's what? A couple Kbs of data? But those couple Kbs of data will be streamed over and over and over throught a game session literally thousands of times.

    Imagine every sound effect, gun shot, explosion, foot-step, line of dialogue is pulled on demand, in small chuncks, loaded just in time. All those are assets (sound assets in this case) that end up being re-used repeatedly millions of times during a game.

    The same can be done with charcter animations, by the way...

    Once you open up to modern, ambitious uses of fast streaming, the idea of the crazy speeds being used more often than only at level load becomes way more plausible.
     
    #443 milk, Nov 20, 2020
    Last edited: Nov 20, 2020
  4. Davros

    Legend

    Joined:
    Jun 7, 2004
    Messages:
    16,809
    Likes Received:
    4,105
    Remember the X-FI with it's 64mb of x-ram
     
    PSman1700 likes this.
  5. milk

    milk Like Verified
    Veteran Regular

    Joined:
    Jun 6, 2012
    Messages:
    3,686
    Likes Received:
    3,735
    no, actually.
     
  6. PSman1700

    Veteran Newcomer

    Joined:
    Mar 22, 2019
    Messages:
    4,657
    Likes Received:
    2,123
    How did the X-ram work actually? I remember having two XFi champion series with the 64mb xram. I think only the 64mb versions could enable Ultra in BF2.
     
  7. Davros

    Legend

    Joined:
    Jun 7, 2004
    Messages:
    16,809
    Likes Received:
    4,105
    Theoretically the game would upload the game audio into the x-ram so it didnt have to use ram or stream it off the hdd
    the cheaper x-fi's had 2mb of x-ram and i think it was just used as a cache and the game didnt need to be coded to take advantage of it. (could be wrong on that)

    Here's some info on X-Ram :
    http://ixbtlabs.com/articles2/multimedia/creative-x-fi-part3.html
     
    #447 Davros, Nov 21, 2020
    Last edited: Nov 21, 2020
    PSman1700 likes this.
  8. PSman1700

    Veteran Newcomer

    Joined:
    Mar 22, 2019
    Messages:
    4,657
    Likes Received:
    2,123
    Oh ok thanks yes i remember that. I have to say, those higher end Xfi models did deliver. Not all games did support it, but quite many did, considering the niche market the Xfi 64mb versions where.
    No idea about performance, but in BF2 for example, there was a huge difference in sound quality and placement (channels too).
     
  9. Davros

    Legend

    Joined:
    Jun 7, 2004
    Messages:
    16,809
    Likes Received:
    4,105
    The link claimed 25% increase in fps in doom3
     
    PSman1700 likes this.
  10. pharma

    Veteran Regular

    Joined:
    Mar 29, 2004
    Messages:
    4,219
    Likes Received:
    3,422

    November 30, 2020
    What's New in DirectX 12? Understanding DirectML, DirectX Raytracing and DirectStorage (techspot.com)
     
    pjbliverpool and PSman1700 like this.
  11. pjbliverpool

    pjbliverpool B3D Scallywag
    Legend

    Joined:
    May 8, 2005
    Messages:
    8,577
    Likes Received:
    2,940
    Location:
    Guess...
    Disappointing if accurate but they do come across as if they know this rather than are speculating.

    That said, they do say this at the start of the article which suggests they don't have any more information than we do and thus could simply be drawing their own conclusions:

    "The third enhancement we're going to cover hasn't been released yet, and it's still yet to hit the developer preview stage. That means details are still scarce besides what Microsoft has told us via their developer blog."
     
    pharma, BRiT and PSman1700 like this.
  12. pharma

    Veteran Regular

    Joined:
    Mar 29, 2004
    Messages:
    4,219
    Likes Received:
    3,422
    True, at this point it's a lot of speculating whether the two features perform the same functionality. From past reading I know Nvidia was keen on developing a means of direct transfers to GPU memory in lieu of reliance on CPU memory. While the end result might end up being the same, it is possible Nvidia has their own take on the API's purpose which is less dependence on the CPU for memory transfers. Time will tell ...
     
    pjbliverpool and PSman1700 like this.
  13. Remij

    Regular Newcomer

    Joined:
    May 3, 2008
    Messages:
    250
    Likes Received:
    406
    So Direct Storage improves batching and I/O request handling on the CPU side, and allows Nvidia and AMD to "plug in" their respective GPU decompression technologies which gives the GPU access to local storage directly, further reducing CPU related I/O work.
     
  14. MfA

    MfA
    Legend

    Joined:
    Feb 6, 2002
    Messages:
    7,330
    Likes Received:
    674
    So it seems that indeed DirectStorage on PC is crippled and NVIDIA is uncrippling it, shame on Microsoft.
     
  15. chris1515

    Legend Regular

    Joined:
    Jul 24, 2005
    Messages:
    6,194
    Likes Received:
    6,548
    Location:
    Barcelona Spain
    I read the devblog again, they never said the CPU need to read the data before sending it to the GPU. It looks like techspot article is false.

     
    DmitryKo and BRiT like this.
  16. pjbliverpool

    pjbliverpool B3D Scallywag
    Legend

    Joined:
    May 8, 2005
    Messages:
    8,577
    Likes Received:
    2,940
    Location:
    Guess...
    If that is the case, which still seems to be a bit up in the air, then the problem is that AMD haven't announced any GPU based decompression capability, and the ideal time to have done so has already passed.
     
    PSman1700 likes this.
  17. Jay

    Jay
    Veteran Regular

    Joined:
    Aug 3, 2013
    Messages:
    3,522
    Likes Received:
    2,859
    Ideal time possibly.

    But as long as they announce and ready prior to direct storage coming out and used, it shouldn't be a problem at all.

    May have some bar graphs on competing products but its not being used yet.
     
  18. Kaotik

    Kaotik Drunk Member
    Legend

    Joined:
    Apr 16, 2003
    Messages:
    9,814
    Likes Received:
    3,971
    Location:
    Finland
    Except that what NVIDIA has described is literally nothing new, Vega did it all already.

    "GPU based decompression capability" is nothing but a program running on shaders of the GPU, what's to advertise there?
     
    DmitryKo, Remij, BRiT and 1 other person like this.
  19. pharma

    Veteran Regular

    Joined:
    Mar 29, 2004
    Messages:
    4,219
    Likes Received:
    3,422
    Doesn't Vega (HBCC) use system memory as cache, where as RTX IO bypasses system memory?
    [​IMG]
     
  20. Kaotik

    Kaotik Drunk Member
    Legend

    Joined:
    Apr 16, 2003
    Messages:
    9,814
    Likes Received:
    3,971
    Location:
    Finland
    Pretty sure it can but doesn't need to, like shown in Radeon SSGs with direct connected SSDs.
    In that slide System DRAM is just another memory it can connect to, among NVRAM, Network Storage etc
     
Loading...

Share This Page

  • About Us

    Beyond3D has been around for over a decade and prides itself on being the best place on the web for in-depth, technically-driven discussion and analysis of 3D graphics hardware. If you love pixels and transistors, you've come to the right place!

    Beyond3D is proudly published by GPU Tools Ltd.
Loading...