Support of Machine Learning (ML) on PS5 and Series X?

Discussion in 'Console Technology' started by Shifty Geezer, Mar 18, 2020.

  1. Shifty Geezer

    Shifty Geezer uber-Troll!
    Moderator Legend

    Joined:
    Dec 7, 2004
    Messages:
    42,906
    Likes Received:
    14,974
    Location:
    Under my bridge
    MS talked ML stuff where Sony didn't. Is that additional silicon for XBSX, or a standard RDNA2 feature Cerny didn't mention this time around? I imagine the latter (would suck for game progress if both platforms didn't have similar ML features).
     
    Alucardx23 likes this.
  2. AzBat

    AzBat Agent of the Bat
    Legend Veteran

    Joined:
    Apr 1, 2002
    Messages:
    6,368
    Likes Received:
    2,356
    Location:
    Alma, AR
    Wouldn't the lower CUs mean they won't have any headroom to take advantage of the ML stuff?

    Tommy McClain
     
    PSman1700 likes this.
  3. pTmdfx

    Regular Newcomer

    Joined:
    May 27, 2014
    Messages:
    275
    Likes Received:
    175
    "ML stuff" is essentially a bunch of compute kernels doing maths for you, and your typical latest-gen GPU can already run them great with stuff like packed FP16 and Int8 support. Assuming there is no CU capability divergence given both being RDNA 2, the only possible difference is actually backing some of these kernels with accelerators (e.g. "tensor cores" for running convolution and friends).
     
  4. patsu

    Legend

    Joined:
    Jun 25, 2005
    Messages:
    27,686
    Likes Received:
    132
    Use Tempest. On-device ML inference should be doable there. You can also use CPU and GPU.

    On iOS device, CoreML runs on all 3 hardware parts: CPU, GPU, Neural Engine ASIC. If the developer doesn’t specify, the software middleware chooses which part to run a given ML model.

    On PS5, if Sony doesn’t do anything, developers will have to roll their own on these parts.
     
  5. chris1515

    Veteran Regular

    Joined:
    Jul 24, 2005
    Messages:
    4,510
    Likes Received:
    3,337
    Location:
    Barcelona Spain
    Sony did not talk ML but Laura Miele from EA talked about ML for next generation in second Wired article. I think the Road to PS5 was moslty about the differentiator of PS5. I suppose they don't want to talk about standard RDNA 2 features before AMD will talk about it. Microsoft needed to present DX12 Ultimate and it is officialliy support by Turing GPU and XSX no official word for RDNA 2 GPU and PS5.

    https://www.wired.com/story/exclusive-playstation-5/

     
    disco_ likes this.
  6. Jay

    Jay
    Veteran Regular

    Joined:
    Aug 3, 2013
    Messages:
    2,453
    Likes Received:
    1,555
    The ML that they added as part of the collaboration with the azure team was the INT4, INT8, FP8.
    They've not mentioned anything else beyond that.
    So i doubt it's much more die space at all. Just not something that would've been added otherwise, xsx gets it as a nice bonus.
     
    PSman1700 likes this.
  7. iroboto

    iroboto Daft Funk
    Legend Regular Subscriber

    Joined:
    Mar 6, 2014
    Messages:
    9,861
    Likes Received:
    9,192
    Location:
    Self Imposed Work Exile: The North
    it's extremely challenging to have low overhead fast running models. A lot of the libraries made for ML are entirely made for data science and we don't give a rats ass about processing a neural network with as little overhead or latency as possible. Processing speed matters, but for real time applications things get much more complex.
    It will be a challenge for them I suspect. Developers will not be interested in rolling their own low-level APIs to run ML with minimal overhead. This will be on Sony to provide.
     
    BRiT and PSman1700 like this.
  8. patsu

    Legend

    Joined:
    Jun 25, 2005
    Messages:
    27,686
    Likes Received:
    132
    The toolkits for *on-device* ML inference should be light weight and low latency because they run on cell phones.

    AMD and Sony should both already have optimized math libraries, but these may lack ML-specific hardware support (e.g. INT4 ?).

    It's the end-to-end ML toolsets that's harder to come together (on AMD platforms). Nvidia has it all covered. I haven't personally tried AMD ROCm.
     
    #8 patsu, Apr 16, 2020
    Last edited: Apr 16, 2020
    Cyan likes this.
  9. iroboto

    iroboto Daft Funk
    Legend Regular Subscriber

    Joined:
    Mar 6, 2014
    Messages:
    9,861
    Likes Received:
    9,192
    Location:
    Self Imposed Work Exile: The North
    on-device celluar ML inference does not provide the level of complexity that SSAA + 4K upresolution does in 4ms or less. The number of outputs from that type of model would be unlikely to run at all on a cell phone just looking at resources required.

    Cell phones are capable of ML as are any devices but nothing close to that.

    The API used are provided by Apple (coreML by your example), is not that low level at all.
    They're actually quite simplistic in terms of calling for use. Apple develops their own model specifically for use in ios devices, so they don't let the developers actually touch that aspect of running the model.

    This is literally their call to run a model pulled from their most basic tutorial:

    Code:
    guard let marsHabitatPricerOutput = try? model.prediction(solarPanels: solarPanels, greenhouses: greenhouses, size: size) else {
       fatalError("Unexpected runtime error.")
    }
    
    Developers don't need to worry about specifying each layer, what to do with it, etc. because likely all that work is done by Apple packaging.

    Everyone else needs to work with their own custom performance model types, or universal ones like ONNX.

    DirectML - is basically like coding DX12. You're doing all the resource allocation, sync, memory management and everything. You get full control on how ML is run on the GPU.

    This is what DirectML looks like to do the same thing:
    https://github.com/microsoft/Direct...MLSuperResolution/DirectMLSuperResolution.cpp
     
    #9 iroboto, Apr 16, 2020
    Last edited: Apr 16, 2020
    Arwin and Dictator like this.
  10. patsu

    Legend

    Joined:
    Jun 25, 2005
    Messages:
    27,686
    Likes Received:
    132
    Probably talking about different types of ML models. For that level of discussions, you'll probably need to look at a specific use cases and then find a suitable solution. One, two liner can only give a rough bearing.

    CoreML stack is built on top of several optimized, low level math libraries on iOS. They are all accessible via public APIs (e.g., Accelerate, BNNS, Metal Performance Shaders).

    AMD ROCm and others basically also pull together multiple libraries to deliver the total package. You should be able to find the references on their sites; but they may not be well setup for your particular config. Formal support from Sony for PS5 developers is definitely preferred.
     
  11. iroboto

    iroboto Daft Funk
    Legend Regular Subscriber

    Joined:
    Mar 6, 2014
    Messages:
    9,861
    Likes Received:
    9,192
    Location:
    Self Imposed Work Exile: The North
    Lets assume i'm wrong about it, even if the API is low level (I haven't seen any code bases that use Metal to break down a specific coreML model to run it custom for the GPU, but that doesn't mean someone hasn't done it), it's still supported and provided by Apple for developers to use. And that's something that MS does for DirectX, but something Sony will need to provide for GNM, which we haven't heard yet.
     
    PSman1700 likes this.
  12. patsu

    Legend

    Joined:
    Jun 25, 2005
    Messages:
    27,686
    Likes Received:
    132
    https://developer.apple.com/videos/play/wwdc2019/614/

    Check with the expert IOS developers for real use.

    Who is "we" ?

    Does PS5 still run BSD/Linux ?
     
    #12 patsu, Apr 16, 2020
    Last edited: Apr 16, 2020
  13. iroboto

    iroboto Daft Funk
    Legend Regular Subscriber

    Joined:
    Mar 6, 2014
    Messages:
    9,861
    Likes Received:
    9,192
    Location:
    Self Imposed Work Exile: The North
    As in the public as an official announcement.
     
  14. patsu

    Legend

    Joined:
    Jun 25, 2005
    Messages:
    27,686
    Likes Received:
    132
    But they are only serving the developers at the moment. Even their latest "Road to PS5" presentation is geared togethers developers.
    I'll check with the developers if you have access.

    They also didn't talk about VRS and whatever. But others have confirmed it's there. ‍
    We'll need a lot more info.
     
  15. iroboto

    iroboto Daft Funk
    Legend Regular Subscriber

    Joined:
    Mar 6, 2014
    Messages:
    9,861
    Likes Received:
    9,192
    Location:
    Self Imposed Work Exile: The North
    API and hardware features are sort of separate. DirectML is an entire API, not just a single function.

    Just to be clear on my position: I'm not saying Sony doesn't have a low level API for ML.
    I'm just saying they will be the ones responsible for providing it to developers to use for ML. Not AMD, not MS, not anyone else.

    If they don't provide an API to do it, then developers have to roll their own solution using GNM/ HLSL shaders and that's going to be extremely _PAINFUL_. They won't integrate ML into their games without it. I'm sure of this. This is like back in the before CUDA came along, ML engineers tried to make GPUs do datascience work, and they were using colours values to calculate what they wanted. Painful process. CUDA came along and the rest became straight forward.

    The main goals of something like DirectML is to ensure you get that maximum hardware acceleration, while being vendor agnostic. So whether you have CUDA cores, Tensor Cores, RDNA 2 CUs , some Intel thing, or some other form of acceleration, like the specific AI Accelerator ASICs you find in a couple of devices like IOS, it's going to try to take advantage of that, and that requires vendors to do their work on the driver side of things as well.
     
    #15 iroboto, Apr 16, 2020
    Last edited: Apr 16, 2020
  16. Scott_Arm

    Legend

    Joined:
    Jun 16, 2004
    Messages:
    14,074
    Likes Received:
    5,302
    Tempest looks like the worst place to do it, at least for any real-time image processing for a game. PS5 gpu should support like 20.0 TFLOPS with FP16. Tempest is maybe 0.2 - 0.4 TFLOPS with FP16. An RTX2060 is 52 TFLOPS with FP16 on tensor cores, for comparison.
     
    tinokun, PSman1700 and BRiT like this.
  17. TheAlSpark

    TheAlSpark Moderator
    Moderator Legend

    Joined:
    Feb 29, 2004
    Messages:
    21,429
    Likes Received:
    6,874
    Location:
    ಠ_ಠ
    I don't suppose they're working with Khronos for a VulkanML off-shoot of sorts.
     
  18. iroboto

    iroboto Daft Funk
    Legend Regular Subscriber

    Joined:
    Mar 6, 2014
    Messages:
    9,861
    Likes Received:
    9,192
    Location:
    Self Imposed Work Exile: The North
    shrug.

    it's possible, I dunno. GNM has to be the API of choice, ti's close to Vulkan, but that doesn't mean you can just port it to GNM.

    I think wrt to this thread, any piece of hardware can be given a low level API to do ML. Software APi development is a non issue with regards to BOM.
    I don't know if the int4/8 on XSX is going to increase the BOM of the device, and I don't know what PS5 supports I guess with respect to this.

    I don't know if either machine is powerful enough to do something like DLSS2.0 at 60fps
     
    PSman1700 and TheAlSpark like this.
  19. JoeJ

    Regular Newcomer

    Joined:
    Apr 1, 2018
    Messages:
    942
    Likes Received:
    1,058
    AFAIK, RDNA2 extended support for ML means low precision types and dot product instructions executed on SFUs. I think Navi 12 already has them, which is in (future) products exclusive for Apple?
    Assuming dot product SFU takes much less die area than NVs tensors with matrix ops, it's quite likely Sony would have adopted them and there is no hardware difference between consoles.
     
  20. dobwal

    Legend Veteran

    Joined:
    Oct 26, 2005
    Messages:
    5,315
    Likes Received:
    1,345
    RDNA allows for variation of CUs. The RDNA white paper describes such variants, "Some variants of the dual compute unit expose additional mixed-precision dot-product modes in the ALUs, primarily for accelerating machine learning inference. A mixed-precision FMA dot2 will compute two half-precision multiplications and then add the results to a single-precision accumulator. For even greater throughput, some ALUs will support 8-bit integer dot4 operations and 4-bit dot8 operations, all of which use 32-bit accumulators to avoid any overflows."
     
Loading...

Share This Page

  • About Us

    Beyond3D has been around for over a decade and prides itself on being the best place on the web for in-depth, technically-driven discussion and analysis of 3D graphics hardware. If you love pixels and transistors, you've come to the right place!

    Beyond3D is proudly published by GPU Tools Ltd.
Loading...