Apple (PowerVR) TBDR GPU-architecture speculation thread

[Ailuros]

They're unlikely to license any microarchitecture which the recent press release talked about due to its lack of "multi-use" language, it's probably again a wide-range architecture and IP license as an extension of what they're already had for half a decade. Also note people saying Apple broke up with them; those press releases are scrubbed from the web and as far as I know the royalties never stopped flowing (till the new deal). Apple played dumb in regards to the "clean-room GPU" claim, indicating again that it never existed.

Meaning the new deal might skip royalties after all?

There's still a larger market outside of those, for example they're still leading the TV SoC market.

I severely doubt they went back to profitability ever since the first announcement that Apple plans to supposedly no longer use their IP. If automotive, TV SoC and the rest are not enough to cover their expenses then Imagination truly needs to completely change course as they can't survive in the longrun, whether there's a Chinese government umbrella related or not.

The entire Alborix philosophy strikes me for far more ambitious plans then just something to render a TV UI or automotive dashboards/infotainment (high end goes to NV IMHO).

 

[Nebuchadnezzar]

Meaning the new deal might skip royalties after all?

Based on the new language omitting "multi-use", it could mean royalties are gone in favour of some flat rate license. But that was the natural course of the original agreement anyhow with royalties diminishing to nothing over the years.

I severely doubt they went back to profitability ever since the first announcement that Apple plans to supposedly no longer use their IP.

Again, assuming they actually ever lost profitability and that the Apple money actually ever stopped, disregarding the public drama.

 

[Ailuros]

Based on the new language omitting "multi-use", it could mean royalties are gone in favour of some flat rate license. But that was the natural course of the original agreement anyhow with royalties diminishing to nothing over the years.

I recall from the past the license fees per year to be less than 2 millions at best and the majority of income coming from royalties. Considering Apple is selling several hundreds of million devices per year the loss is there in the longrun.

Again, assuming they actually ever lost profitability and that the Apple money actually ever stopped, disregarding the public drama.

At some point if they're not paying royalties it'll inevitably come as far if it hasn't been the case already. TV and automotive markets aren't by far as large or fast moving as the mobile markets, meaning IMG will or plans to reduce the Apple royalty loss with what exactly?

I thought that at least Mediatek would balance the persentages between ARM and IMG GPU IP usage ever since, but the scale is increasingly going in ARM's ballpark.

 

[Leovinus]

According to The China Times Apples in-house GPU silicon will feature in the new iMac and is progressing well under the codename "Lifuka". Information is a bit sparse really. But at least it's something to show it's puttering along.

Macrumours.com quote the article as saying:

According to relevant sources, Apple's self-developed GPU is progressing smoothly. The research and development code is Lifuka. Like the upcoming A14X processor, it is produced using TSMC's 5nm process. Apple has designed a series of processors for Mac personal computers. The new GPU will provide better performance per watt and higher computing performance. It has tile-based deferred rendering technology that allows application developers to write more powerful professional application software and game software.

For me personally it's going to be interesting to see wether this is a beefing up of igpu performance for high resolution desktop use in lower end Macs, or if Apple intends to create what is essentially their own dGPU family throughout the performance spectrum.

Given that Apple has standardised on the RDNA architecture for their Intel based products, and given that support for these platforms will persist for some time, I do wonder if they won't leave mid- to high end products in the hands of AMD. At least to start with. But one can only speculate.

Considering the power and performance envelope their licensed GPU IP is designed for it feels like an investment too far for Apple to rework it to a high performance part without wholly owning the result. Why go through that effort and still have to pay royalties? Then again I don't know how the Apple-PowerVR deal is set up. Or perhaps Apple really does have it's own IP in the works?

 

[Lurkmass]

For me personally it's going to be interesting to see wether this is a beefing up of igpu performance for high resolution desktop use in lower end Macs, or if Apple intends to create what is essentially their own dGPU family throughout the performance spectrum.

Given that Apple has standardised on the RDNA architecture for their Intel based products, and given that support for these platforms will persist for some time, I do wonder if they won't leave mid- to high end products in the hands of AMD. At least to start with. But one can only speculate.

Considering the power and performance envelope their licensed GPU IP is designed for it feels like an investment too far for Apple to rework it to a high performance part without wholly owning the result. Why go through that effort and still have to pay royalties? Then again I don't know how the Apple-PowerVR deal is set up. Or perhaps Apple really does have it's own IP in the works?

I can safely say that it's not going to end well because tile-based GPUs have a totally different driver design paradigm compared to desktop GPUs. The differences between their HW designs becomes very apparent at the gfx API level. Renderpasses existed since the beginning of the introduction of Metal/Vulkan because mobile GPU drivers used this information to pick the optimal tiling strategy for their hardware. D3D12 by comparison didn't initially expose the concept of renderpasses until recently because that abstraction wasn't necessary for desktop GPUs to achieve optimal performance. Even after ignoring the other gfx APIs, developers can still approach Metal's renderpass API more liberally on AMD HW and developers would be potentially unaware of the bad performance on Apple silicon since they never intended on deploying their application to that platform.

If AMD and Intel drivers are deemed less than favourable because they're behaviour is not consistent compared to Nvidia's drivers then Apple silicon would effectively blacklist these common industry practices that's harmful against their HW design. Performance in this case isn't just solely dependent on HW design but it's also a subject of attaining parity in driver behaviour as well. There's a limit to Apple alienating a programming culture they don't like especially the one where desktop GPUs derive their value on running software unique to them which is high-end graphics or professional visualizations ...

 

[Leovinus]

I can safely say that it's not going to end well because tile-based GPUs have a totally different driver design paradigm compared to desktop GPUs.

Which is why I would find it curious if Apple intended to to use a PowerVR like design throughout their entire lineup. Producers with iOS experience have a basis to start migrating from. But it makes a poor argument for basically anyone else.

Apple did show Tomb Raider being run via translation layer on their A14 silicon at the Big Sur preview. If the translation layer is efficient enough and the hardware performant enough the overall experience might not be terrible even if the actual performance is. Then again Tomb Raider was already ported to MacOS and Metal, adhering to code that is presumably tile-benign to begin with, and a known quantity for the translation layer.

Is it possible for Apple to offer developers tools that are effective and work-flow efficient enough to limit some of the pains that would come with porting to their hardware, or does it go as deep as the core design of rendering engines and philosophies?

 

[Pressure]

Apple did show Tomb Raider being run via translation layer on their A14 silicon at the Big Sur preview.

Everything from that keynote was done on the A12Z.

 

[Leovinus]

Everything from that keynote was done on the A12Z.

My apologies, of course it was. Temporary memory spasm.

 

[MfA]

There's a limit to Apple alienating a programming culture they don't like especially the one where desktop GPUs derive their value on running software unique to them which is high-end graphics or professional visualizations ...

There's only a handful of important professional software packages with realtime previews/visualization, they have no immediate need to replace the offline render farms. For those they could in theory just throw money at embedded developers.

 

[Lurkmass]

Which is why I would find it curious if Apple intended to to use a PowerVR like design throughout their entire lineup. Producers with iOS experience have a basis to start migrating from. But it makes a poor argument for basically anyone else.

Apple did show Tomb Raider being run via translation layer on their A14 silicon at the Big Sur preview. If the translation layer is efficient enough and the hardware performant enough the overall experience might not be terrible even if the actual performance is. Then again Tomb Raider was already ported to MacOS and Metal, adhering to code that is presumably tile-benign to begin with, and a known quantity for the translation layer.

Is it possible for Apple to offer developers tools that are effective and work-flow efficient enough to limit some of the pains that would come with porting to their hardware, or does it go as deep as the core design of rendering engines and philosophies?

Aside from middlewares like engines, no amount of tooling will be able to reconcile the fundamental differences between tile-based GPUs and immediate mode GPUs, The ideal place to hide these architectural differences would've been at the driver side but not even Apple was able to do this with Metal so the harsh reality is that invasive API design is needed to cater to tile-based architectures and it will become apparent to any graphics programmer that have to deal with those APIs.

Not dealing with renderpasses is one of the big reasons why many AAA developers prefer D3D12 over either Metal or Vulkan. With DirectX 12 Ultimate on the horizon, Microsoft are going to capitalize even further on the strengths of desktop GPUs by exposing HW features such as ray tracing, mesh shaders and true bindless resources which will only mean even less reasons to port applications to tile-based GPUs. Does anyone realistically think Apple silicon will have competing implementations of these technologies in the near future ?

 

[Leovinus]

Well, cross-platform middleware like Unreal engine would continue to support it for iOS releases. As for professional users, @MfA is correct. Apple can just throw money at that problem.

As for the hardware I'm honestly curious. Tiled rendering has its drawbacks, but I've always thought it was a nifty approach since the Kyro days. IMG's A-series IP especially seems eminently suitable to be scaled into higher performance parts. IMG also has ray-tracing patents and implementations, though I haven't seen any performance analysis of it or what Metal/Vulkan extensions are used, and don't know if Apple will make use of it.

As for the rest, mesh shaders and mindless resources, I don't know enough to say anything on the subject.

I just hope we'll get more info on Lifuka soon. For all my questions as to the validity of Apples approach here I'm excited to see what they have in mind. Both in hardware and software.

 

[Entropy]

With DirectX 12 Ultimate on the horizon, Microsoft are going to capitalize even further on the strengths of desktop GPUs by exposing HW features such as ray tracing, mesh shaders and true bindless resources which will only mean even less reasons to port applications to tile-based GPUs.

There is only one reason to produce software for a platform.

Revenue.

It’s a pretty good one though, and if you are leaving money on the table, you can rest assured someone else will be there to pick it up.

 

[Lurkmass]

Well, cross-platform middleware like Unreal engine would continue to support it for iOS releases. As for professional users, @MfA is correct. Apple can just throw money at that problem.

As for the hardware I'm honestly curious. Tiled rendering has its drawbacks, but I've always thought it was a nifty approach since the Kyro days. IMG's A-series IP especially seems eminently suitable to be scaled into higher performance parts. IMG also has ray-tracing patents and implementations, though I haven't seen any performance analysis of it or what Metal/Vulkan extensions are used, and don't know if Apple will make use of it.

It's because tile-based GPUs couldn't compete on a technological basis. Tiled rendering died off on desktop around the time the architects behind PowerVR fumbled hard by being 3 years late (2002) trying to integrate hardware accelerated T&L functionality (which was introduced in late 1999 with Direct3D 7) into their hardware designs. Their work was going to be obsoleted with the introduction of Direct3D 8 and it's vertex shaders which entirely replaced fixed function T&L.

Immediate mode GPUs were sometimes several years ahead of the curve compared to tile-based GPUs in terms of HW features like geometry shaders, compute shaders, and tessellation. Immediate mode GPUs were also standardizing features such as predication and streamout which negatively affected performance on tile-based GPUs.

Tile-based GPUs still lacks support for HW features like pointers (bindless) or texture handles. When can we also expect hardware accelerated ray tracing too ? It's questions like these why we should doubt if Apple is truly sincere about targeting PC developers ...

 

[Leovinus]

Is it possible that you're analysing Apple's actions and intents from PC centric position that leads to easy dismissal? Apple certainly doesn't target "PC developers" broadly and never have. There should be no surprise in Apple walking a path where others must adapt to follow. Which is fair to scrutinise.

As you mention the PowerVR solution lacks certain aspects that are expected in the PC domain. I and other have pointed out that Apple has invested in a Mobile (Arm, tiled rendering, etc) and strictly Apple domain. There are already vestiges of compatibility for developers to draw on from the mobile space and most likely money for those that Apple feel need to be coaxed. Success depends on how gracefully Apple manages to bridge the capability and design divide they're forming between the Apple and PC domain. Also, it's not certain that Apple will drop AMD GPU's for their high end, meaning some assumptions of cross-compatibility between tiled and immediate mode would have to be catered for in software and support. At least for some time yet. (Who knows, Apple might just be crazy enough to invest in some fixed function translation help for certain "inescapable" compatibility-centric workloads. The A-series comes with a RISC-V core for programmable firmware that could theoretically be expanded on. But I'm way out of my depth with that theorising)

IMG's A-series IP is extremely efficient and performant at least for classical tasks from the stuff I have seen. Anandtech has a very good overview of the new architecture. And it looks impressive enough in its own right. I'm more interested in what it is potentially capable of than what it isn't.

 

[Rootax]

"Tile-based GPUs still lacks support for HW features"

But it is "just" because IMG is too slow to add them, or it is because of the TBDR nature of the gpu ?

 

[MfA]

Does anyone realistically think Apple silicon will have competing implementations of these technologies in the near future ?

Raytracing hardware is really simple comparitively at the moment ... once it gets more flexible, with on the fly LoD and animation, it will get more complex ... but for the moment it's very straightfoward. They have the patents from Imagination, they have enough expertise. If they want to competetively accelerate the preview windows in handful of the top of the line professional applications, they can build the hardware and help implement the software for that quickly. Those renderers are less complex and less hacky than game rendering pipelines.

Their existing customer base are not big gamers. Just relying on dedicated game ports for the developers who will volunteer their time to wrangle Metal is good enough for now, stealing everyone's lunch can wait. Just being viable for the most important professional applications is enough.

 

[Lurkmass]

"Tile-based GPUs still lacks support for HW features"

But it is "just" because IMG is too slow to add them, or it is because of the TBDR nature of the gpu ?

Sometimes it is down to the nature of their architecture ...

That's another gripe about the Metal API not exposing geometry shaders even though AMD or Intel HW could perfectly support it. Even if they are somewhat slow, some PC games actually see usage of geometry shaders and it is very useful for debugging purposes as well

Raytracing hardware is really simple comparitively at the moment ... once it gets more flexible, with on the fly LoD and animation, it will get more complex ... but for the moment it's very straightfoward. They have the patents from Imagination, they have enough expertise. If they want to competetively accelerate the preview windows in handful of the top of the line professional applications, they can build the hardware and help implement the software for that quickly. Those renderers are less complex and less hacky than game rendering pipelines.

Their existing customer base are not big gamers. Just relying on dedicated game ports for the developers who will volunteer their time to wrangle Metal is good enough for now, stealing everyone's lunch can wait. Just being viable for the most important professional applications is enough.

Most professional graphics applications don't even have a Metal backend. It's either CUDA or OpenGL that dominates so if Apple blocks Nvidia which has the best API implementations for them or the only API implementation in some cases then there's no reason to ever consider an Apple platform for professional work ...

Just to give you an idea quite a few CAD developers are actually considering transitioning from OpenGL to either D3D12 or Vulkan because they are interested in using mesh shaders meanwhile Metal is not on their radar since they don't have a competing feature ...

Metal is not even suitable for machine learning or data science either. If you look at RadeonML and the models supported by MPS (Metal performance shaders), it only supports models that do either denoising or upscaling. Meanwhile on DirectML, you can inference all models in RadeonML and with some form of Tensorflow you can even train models using DirectML ...

When I look at tile-based GPUs, all I see are the dark ages and Apple only cares about winning benchmarks in low value software like smartphone games so it's pretty obvious that the industry wants the future direction of graphics technology dictated by the more sane desktop GPU vendors ...

 

[MfA]

They have no pressing need to get people on board who aren't using Macs to begin with ... if during the transition they just don't lose the ones they have that is enough. Eating everyone's lunch can wait.

CAD isn't an existing market for Apple either. Neither are ML bubble companies. The very few operations where real time neural networks make sense for Apple users can be programmed by Apple in Core ML ... and almost everyone else can just run their stuff on servers and in the cloud regardless of the OS they are using, just like they do now. Hell even Maya isn't really their domain. It's the (semi-)2D and video people who use them, so mostly they need to get MARI and Z-Brush well supported (Final Cut Pro is obviously not a problem).

And if they want to throw some embedded developers at Autodesk, I doubt they'll refuse. NVIDIA might be offended, but the Apple market potential is too juicy to pass up ...

 

[rikrak]

I can safely say that it's not going to end well because tile-based GPUs have a totally different driver design paradigm compared to desktop GPUs. The differences between their HW designs becomes very apparent at the gfx API level. Renderpasses existed since the beginning of the introduction of Metal/Vulkan because mobile GPU drivers used this information to pick the optimal tiling strategy for their hardware. D3D12 by comparison didn't initially expose the concept of renderpasses until recently because that abstraction wasn't necessary for desktop GPUs to achieve optimal performance. Even after ignoring the other gfx APIs, developers can still approach Metal's renderpass API more liberally on AMD HW and developers would be potentially unaware of the bad performance on Apple silicon since they never intended on deploying their application to that platform.

And yet render passes are a thing in modern APIs now and are here to stay, especially with advent of WebGPU. I really don't see why you are making such a big issue out of it.

Not dealing with renderpasses is one of the big reasons why many AAA developers prefer D3D12 over either Metal or Vulkan.

Many AAA developers prefer DX12 because they only target Windows and because DX12 has superior tooling.

Sometimes it is down to the nature of their architecture ...
That's another gripe about the Metal API not exposing geometry shaders even though AMD or Intel HW could perfectly support it. Even if they are somewhat slow, some PC games actually see usage of geometry shaders and it is very useful for debugging purposes as well

Geometry shaders are a bad abstraction for GPU execution model. They don't scale. Metal only exposes stuff that makes sense for the hardware. If you need functionality of geometry shaders, use compute shaders with GPU-driven render loop.

With DirectX 12 Ultimate on the horizon, Microsoft are going to capitalize even further on the strengths of desktop GPUs by exposing HW features such as ray tracing, mesh shaders and true bindless resources which will only mean even less reasons to port applications to tile-based GPUs. Does anyone realistically think Apple silicon will have competing implementations of these technologies in the near future ?

I think your information might be a bit outdated? Metal has supported bindless ressources (they call them Argument Buffers) for a while now, including arbitrary pointer chasing, writeable resource handles and whatever you want. You can build entire hierarchies of resource bindings on the GPU. This years update to Metal includes fully-featured implementation of ray tracing, with programmable intersection shaders, function pointers for dynamic function loading, recursive shader execution etc. The performance is not the best right now, since ray tracing is done in "software" (using compute shaders), but it is obvious that hardware ray tracing support is coming. Regarding mesh shaders: I agree that they are not the best fit for the TBDR hardware, since the generated vertices cannot be just passed through to the rasterizer immediately. So I am not sure whether we will ever see them in Metal. But if all you need to do is generating some geometry on the GPU, well, compute shaders + GPU-driven rendering loops got you covered. And of course, Metal has sparse textures, variable rate shading, fully programmable multisampling and so on.

Besides, you are leaving out some advantages TDBR brings to the table. Like programmable blending or direct control over tile memory. Explicit render passes, memory-less render targets and tile shaders simplify implementation of many advanced rendering techniques and end up being much more efficient.

 

[MfA]

With raytracing mesh shaders would have to dump the tris to memory any way ... they are going to be a poor fit going forward, tiling or not.