MS talked ML stuff where Sony didn't. Is that additional silicon for XBSX, or a standard RDNA2 feature Cerny didn't mention this time around? I imagine the latter (would suck for game progress if both platforms didn't have similar ML features).
Support for Machine Learning (ML) on PS5 and Series X?
- Thread starter Shifty Geezer
- Start date
Wouldn't the lower CUs mean they won't have any headroom to take advantage of the ML stuff?
Tommy McClain
"ML stuff" is essentially a bunch of compute kernels doing maths for you, and your typical latest-gen GPU can already run them great with stuff like packed FP16 and Int8 support. Assuming there is no CU capability divergence given both being RDNA 2, the only possible difference is actually backing some of these kernels with accelerators (e.g. "tensor cores" for running convolution and friends).
Use Tempest. On-device ML inference should be doable there. You can also use CPU and GPU.
On iOS device, CoreML runs on all 3 hardware parts: CPU, GPU, Neural Engine ASIC. If the developer doesn’t specify, the software middleware chooses which part to run a given ML model.
On PS5, if Sony doesn’t do anything, developers will have to roll their own on these parts.
Sony did not talk ML but Laura Miele from EA talked about ML for next generation in second Wired article. I think the Road to PS5 was moslty about the differentiator of PS5. I suppose they don't want to talk about standard RDNA 2 features before AMD will talk about it. Microsoft needed to present DX12 Ultimate and it is officialliy support by Turing GPU and XSX no official word for RDNA 2 GPU and PS5.
https://www.wired.com/story/exclusive-playstation-5/
https://www.wired.com/story/exclusive-playstation-5/
The ML that they added as part of the collaboration with the azure team was the INT4, INT8, FP8.
They've not mentioned anything else beyond that.
So i doubt it's much more die space at all. Just not something that would've been added otherwise, xsx gets it as a nice bonus.
it's extremely challenging to have low overhead fast running models. A lot of the libraries made for ML are entirely made for data science and we don't give a rats ass about processing a neural network with as little overhead or latency as possible. Processing speed matters, but for real time applications things get much more complex.
It will be a challenge for them I suspect. Developers will not be interested in rolling their own low-level APIs to run ML with minimal overhead. This will be on Sony to provide.
The toolkits for *on-device* ML inference should be light weight and low latency because they run on cell phones.
AMD and Sony should both already have optimized math libraries, but these may lack ML-specific hardware support (e.g. INT4 ?).
It's the end-to-end ML toolsets that's harder to come together (on AMD platforms). Nvidia has it all covered. I haven't personally tried AMD ROCm.
Last edited:
on-device celluar ML inference does not provide the level of complexity that SSAA + 4K upresolution does in 4ms or less. The number of outputs from that type of model would be unlikely to run at all on a cell phone just looking at resources required.
Cell phones are capable of ML as are any devices but nothing close to that.
The API used are provided by Apple (coreML by your example), is not that low level at all.
They're actually quite simplistic in terms of calling for use. Apple develops their own model specifically for use in ios devices, so they don't let the developers actually touch that aspect of running the model.
This is literally their call to run a model pulled from their most basic tutorial:
Code:
guard let marsHabitatPricerOutput = try? model.prediction(solarPanels: solarPanels, greenhouses: greenhouses, size: size) else {
fatalError("Unexpected runtime error.")
}Everyone else needs to work with their own custom performance model types, or universal ones like ONNX.
DirectML - is basically like coding DX12. You're doing all the resource allocation, sync, memory management and everything. You get full control on how ML is run on the GPU.
This is what DirectML looks like to do the same thing:
https://github.com/microsoft/Direct...MLSuperResolution/DirectMLSuperResolution.cpp
Last edited:
Probably talking about different types of ML models. For that level of discussions, you'll probably need to look at a specific use cases and then find a suitable solution. One, two liner can only give a rough bearing.
CoreML stack is built on top of several optimized, low level math libraries on iOS. They are all accessible via public APIs (e.g., Accelerate, BNNS, Metal Performance Shaders).
AMD ROCm and others basically also pull together multiple libraries to deliver the total package. You should be able to find the references on their sites; but they may not be well setup for your particular config. Formal support from Sony for PS5 developers is definitely preferred.
Lets assume i'm wrong about it, even if the API is low level (I haven't seen any code bases that use Metal to break down a specific coreML model to run it custom for the GPU, but that doesn't mean someone hasn't done it), it's still supported and provided by Apple for developers to use. And that's something that MS does for DirectX, but something Sony will need to provide for GNM, which we haven't heard yet.
https://developer.apple.com/videos/play/wwdc2019/614/
Check with the expert IOS developers for real use.
Who is "we" ?
Does PS5 still run BSD/Linux ?
Last edited:
As in the public as an official announcement.
But they are only serving the developers at the moment. Even their latest "Road to PS5" presentation is geared togethers developers.
I'll check with the developers if you have access.
They also didn't talk about VRS and whatever. But others have confirmed it's there.
We'll need a lot more info.
API and hardware features are sort of separate. DirectML is an entire API, not just a single function.
Just to be clear on my position: I'm not saying Sony doesn't have a low level API for ML.
I'm just saying they will be the ones responsible for providing it to developers to use for ML. Not AMD, not MS, not anyone else.
If they don't provide an API to do it, then developers have to roll their own solution using GNM/ HLSL shaders and that's going to be extremely _PAINFUL_. They won't integrate ML into their games without it. I'm sure of this. This is like back in the before CUDA came along, ML engineers tried to make GPUs do datascience work, and they were using colours values to calculate what they wanted. Painful process. CUDA came along and the rest became straight forward.
The main goals of something like DirectML is to ensure you get that maximum hardware acceleration, while being vendor agnostic. So whether you have CUDA cores, Tensor Cores, RDNA 2 CUs , some Intel thing, or some other form of acceleration, like the specific AI Accelerator ASICs you find in a couple of devices like IOS, it's going to try to take advantage of that, and that requires vendors to do their work on the driver side of things as well.
Last edited:
Tempest looks like the worst place to do it, at least for any real-time image processing for a game. PS5 gpu should support like 20.0 TFLOPS with FP16. Tempest is maybe 0.2 - 0.4 TFLOPS with FP16. An RTX2060 is 52 TFLOPS with FP16 on tensor cores, for comparison.
I don't suppose they're working with Khronos for a VulkanML off-shoot of sorts.
shrug.
it's possible, I dunno. GNM has to be the API of choice, ti's close to Vulkan, but that doesn't mean you can just port it to GNM.
I think wrt to this thread, any piece of hardware can be given a low level API to do ML. Software APi development is a non issue with regards to BOM.
I don't know if the int4/8 on XSX is going to increase the BOM of the device, and I don't know what PS5 supports I guess with respect to this.
I don't know if either machine is powerful enough to do something like DLSS2.0 at 60fps
AFAIK, RDNA2 extended support for ML means low precision types and dot product instructions executed on SFUs. I think Navi 12 already has them, which is in (future) products exclusive for Apple?
Assuming dot product SFU takes much less die area than NVs tensors with matrix ops, it's quite likely Sony would have adopted them and there is no hardware difference between consoles.
Assuming dot product SFU takes much less die area than NVs tensors with matrix ops, it's quite likely Sony would have adopted them and there is no hardware difference between consoles.
RDNA allows for variation of CUs. The RDNA white paper describes such variants, "Some variants of the dual compute unit expose additional mixed-precision dot-product modes in the ALUs, primarily for accelerating machine learning inference. A mixed-precision FMA dot2 will compute two half-precision multiplications and then add the results to a single-precision accumulator. For even greater throughput, some ALUs will support 8-bit integer dot4 operations and 4-bit dot8 operations, all of which use 32-bit accumulators to avoid any overflows."
Similar threads
- Sticky
