AMD FidelityFX on Consoles

what this have to do with anything discussed here?

 

Buying up studios and have everything multiplat instead of greedy exclusives might be a way to kill off Playstation.

Perhaps, just like with dolby atmos, the PS5 had to have this exotic audio solution no one outside first party studios will utilize (and have a less impressive effect to boot).

PS4 is doing ray tracing in some games, it has hardware featured ray tracing, case closed.

Indeed, in the same vein, using ML for a subtle thing like muscle deformation on a character as opposed to reconstructing an image from 1080p all the way to 4k is a total different thing.

 

running ML alongside game code as in parallel? I think only tensor cores can do that.

If you mean being able to run ML inference at fast enough speeds, the main attributors to making models run faster are the size of the model and any sort of feature engineering that needs to be done prior to running it through the model. If none is required, that is ideal. Then you're just looking at the algorithm it's running and how many layers there are for it to provide an acceptable return. Once you figure out the absolute minimums, you can choose to encode it or mix encode it to lower precision and see if you lose any more accuracy.

And this is where int8 would come in. And if for some crazy reason you can encode it further down to int4 and not lose any accuracy, then go for it.

Afterwards, it's all about the number of available ALU you have to run the network. More available threads means you can process larger networks faster (not true necessarily) - so if you have for some reason, and I'm just throwing random numbers, if you wanted to transform a 1080p image into a 4K image, your input threads will be 2.0M pixels long. Your output will be 8.3M pixels out. To run the later layers of the NN in a single shot (all threads being processed at the same time), requires you to have 8.3M threads available. I'm not even talking about the calculations that are going to happen right, just threads. And even maxing out the most number of threads may not be ideal for performance, but that's a different topic.

So basically you're going to have to run multiple cycles per layer to get through all those pixels before you're allowed to move onto the next layer. NN are serialized in this fashion, there are many nodes in a layer that needs processing, but you can not process the next layer without the previous layer being completed. So having wide GPUs and bandwidth will matter in this type of thing. Then you can start factoring in int8/int4 encoding and mixed precision processing to improve your performance further.

This is the way deep learning works, but this isn't how all machine learning algorithms work. Without knowing what the model is (it may not be a NN) and without knowing what encoding schemes are being used, if any at all, there may be no advantages that XSX holds over PS5 except in available ALU and bandwidth to feed them because not all ML algorithms will be leverage those features (int4/8 and mixed precision dot products)

 

Just because you do or don't care for a result doesn't mean that it's not relevant to the conversation regarding machine learning on consoles.

I was assuming that AMD would do the training, since we are discussing "AMD Fidelity FX On Consoles" and not anything specific the Xbox or Microsoft. So we can add that assumption as another link in the assumption chain for guaranteed success. But it is possible that Microsoft have a specialized solution that leverages Series hardware. And again, if it's easy enough to implement, and offers performance or IQ benefits, developers will use it.

 

Sorry, I hope this isn't too OT...

But my understanding of the Auto-HDR feature,
It is simple a custom SDR - HDR transform applied on a game by game basis, and that while it is TRAINED by ML,
The training and ML is simply to get the right values for the colourspace conversion, that forms the SDR ->HDR transform.

Whats more, i think, this happens in the Output block of the Series consoles, and not strictly within the GPU itself.
Which is very different to what we are talking about regarding using "traditional" GPU resources to implement an AI inference based up-scaling method.

I actually dont know how much separation there is in the current gen consoles between the GPU silicon and the "Output Block" ( which i would consider to include the hdmi encoder).

I know in the past i have speculated that by combining the use of GPU based upscaling and intelligent work in the output block, might leed to the best possible result for any upscaling system,
This would be simply due to the powerful and flexible nature of the output block on series consoles.

 

You're a broken record, @invictis. I suggest dropping this persistent rumour of yours, it's really going nowhere other than round and round in circles.

It's happened several times before now in this very thread and frankly it's boring to repeatedly go through it every few pages.

You realise the majority of people on B3D don't care for the platform warring and bias? You own an Xbox, good for you - they're good consoles. Just drop the boring "Microsoft has X feature, Sony does not". You've done it before, and again, and again. We heard you, it's acknowledged that there may be differences in the ML implementation.

Let's now check out how the different implementations pan out.

 

By inference, the dogs were discussing whether one, or more, of the blocks were blue. A human stops and says the one on the right is Blue. So by inference the left one is not Blue otherwise the human would have said they are both Blue and not just picked out one.

I'm not disagreeing with your point, just saying being OCD about the example

 

Nvidia doesn't and won't share it.

AMD would, simple example is fidelity fx, its open source and can run on Nvidia cards also.

Some of the reaons behind the difference in approach is where they are in the market.
AMD due to market share needs as much uptake and usage of their tools as possible, as using Nvidia toolkit locks them out of it and leaves them at a disadvantage.

In regards to MS they can approach it a few different ways, or all.

• Make it free for use with DX12U, this would lockout Vulcan and PS5 and give additional reasons for a studio to use DX.
• Put it in something like playfab. So that can be used on Vulcan and PS5 and the more people use features from playfab the more likely they are to use other features that it has to offer like azure, basically end up upselling.

Playfab is an MS suite that can be used by anyone including Sony.

Models being able to run everywhere doesn't mean its free to.

 

While I don't think nVidia isn't going to continue working on DLSS, the current implementation does not require per game training. I think they use their market position and marketing deals to leverage developers into supporting it. We don't know if the AMD solution will require per game training, but recent comments from them hint that FidelityFX Super Resolution might not even use ML.

I'm pretty interested to see how Microsoft's solution is different than AMD's solution. We know Microsoft is working on ML projects for DirectX, and we know AMD is working on an upscaling solution to compete with DLSS, and obviously they partner in plenty of projects. But people talk as if Fidelity FX Super Resolution is not going to be a separate thing from a DirectML upscaler evangelized by Microsoft. I think everything Microsoft has shown about their solution was some Forza footage, and IIRC it was running on nVidia hardware.

 

I'm sure there are many developers that simply don't want to spend the time incorporating DLSS into their game, or their game engine has issues that prevent DLSS integration. There's probably several reasons why most games don't have DLSS beyond technical as well as we're not really privy to Nvidia's requirements. It makes sense though for as many developers integrate it as possible, provided it's worth their time, and their game actually benefits from it (ie. the game's graphics are demanding at higher resolutions)

A solution by AMD would likely suffer worse than DLSS on PC for widespread use, even if it's open.

 

I personally would hope AMD and MS are working independently, and I kind of expect that to be the case.
Different solutions and options are good.
AMD non ML based, MS ML based.

Why do you think this?
Are you talking about if AMD's solution is ML based also, even so I'm unsure why you would have that view.

They would need to go the same route as Nvidia and get it incorporated into unity and unreal.

 

Nvidia have 80% market share and a large install base of Turning/Ampere GPUs, yet DLSS, even without the need to train per game anymore, has a relatively small amount of integrations. Nvidia also historically have far larger dev relations than AMD. Assuming the need for RDNA1/2 for an AMD solution, I don't see why suddenly we'd see a lot more adoption of an open solution when adding in a relatively very small addition of RDNA-based GPUs into the potential user base.

Yeah, definitely this should be the big priority at the start.

 

Frustratingly not mentioning FidelityFX VRS that could have ended much of the debate here. lol

 

Would be very confusing if PS5/XSX etc didnt support those FidelityFX features.

 

How so?
Could implement a software version.