GPU performance analysis on mobile operating systems

[ams]

Good feedback gentlemen

We need some better mobile benchmarks :S

Amen to that!!!

Why are consumers still unable to get basic framerate data on a game-by-game basis as we are accustomed to on the PC?

Image quality analysis would be great too, but that would clearly be influenced by the quality and color accuracy of the screen itself too.

 

[french toast]

Thanks for answering, guys. This thread has been great




What hostility are you talking about? The questions I've seen in this thread are just what you would expect to find in a formal interview with a journalist from the hardware media.
Rys, Dominik D and JohnH just answered the questions in a honest and straight-forward way (thank you for that, honestly) and that was it.

I asked about the FP16 subject because I've seen speculation about that in lots of places. And if we're given the opportunity from the source to explain this, then we should ask the question for the benefit of all.

Moreover, this was a peaceful and adult conversation. There's no need to come forth and protect someone who doesn't need protection.




I've been wondering about that.
Is it possible to make at least a FRAPS clone for Android?

Good feedback gentlemen



Amen to that!!!

Why are consumers still unable to get basic framerate data on a game-by-game basis as we are accustomed to on the PC?

Image quality analysis would be great too, but that would clearly be influenced by the quality and color accuracy of the screen itself too.

There are currently two FRAPS like apps for android, one is called fps meter and requires root, the other is called gamebench which does not. In the case of game bench it can be used with most apps not just games and has helped me optimize my smartphone gaming experience, i have thought about suggesting this method of soc performance analysis to anandtech as to my knowledge no review site uses it,(although some youtube reviewers have used t) and would be a more representative of real world performance for consumers rather than benchmarks .

 

[Ailuros]

Shouldn't someone split the FRAPS stuff into another thread since it might be an interesting topic, but has absolutely nothing to do with the topic at hand?

 

[Rys]

ams brings up a great topic: performance analysis techniques that are possible on PC tend to be difficult or impossible to use on mobile operating systems like iOS and Android.

The primary reason is one of access: to hook into client drivers and display pipe internals in iOS and Android requires either specially packaged applications or privileged access. As a normal user of the OS, you can't insert yourself between the application and the operating system.

On Android, if you have root access then your options become a lot more interesting. Android's architecture is such that it already has a proxy graphics layer that you can redirect to an intercepting one (which is how tools like FRAPS work).

On iOS, you're stuck. Xcode will let you profile an application you have source for and can launch with profiling, but there's no easy way to turn the sampled data into something you might want to publish online in a nice way, or visualise differently, or profile arbitrary applications (unless you're Apple).

Then there are secondary methods of performance analysis that aren't sampling profilers, such as looking at compiler disassembly, which are similarly difficult on mobile operating systems. The main barrier to adoption of secondary methods is there's no way to easily update the GPU driver outside of the OS vendor giving you new binaries.

Even if I was to publish the source code of our Android driver, for example, there's no honestly easy way for you to use that to replace the driver on your device. A chunk of the driver isn't supplied by the GPU vendor (the memory allocator and display controller integration mainly) on Android so you'd need to get your SoC vendor to give you those parts too, then you'd need privileged access to push it to the device.

If it breaks, there's no easy recovery either, like on Windows. The GPU driver interface to the rest of the OS is therefore brittle, difficult to work with and really unsuitable for end user replacement.

Then you have issues of frequency. It's not like Windows or Linux or OS X where the GPU vendor is in control of pushing updates, even if the mechanism existed to make it easy and the previous issues were solved. Out-of-band updates for GPU drivers just aren't a thing; they come packaged with full OS firmware updates and often don't change at all between releases.

Because Android is a fast moving target and QA is expensive, SoC vendors tend to converge on the first driver that passes their QA and then never update it. So getting access to things like performance and image quality improvements that we make in our drivers might never be possible on your device, because it's expensive to QA new drivers and no good mechanisms exists to frequently ship you new drivers anyway.

On iOS you're actually more likely to see rolling improvements in the GPU driver, but the cadence is entirely Apple's and there's nothing you can do to work on the driver yourself or put yourself between it and an app, or anything like that.

There are other reasons why tools like FRAPS just aren't more pervasive and performance analysis of the GPU on mobile devices is difficult, but those above are the main ones.

 

[Ailuros]

Are there any technical obstacles for frame counters in mobile games?

 

[Rys]

Nope, there's nothing (other than good code) stopping apps from implementing it themselves.

 

[Arwin]

So we basically need something like a high-speed videocamera to look at the smart-device for framerates, right?

 

[Helmore]

So we basically need something like a high-speed videocamera to look at the smart-device for framerates, right?

That would only tell you half the story. It would tell you whether a game really is running smoothly, which admittedly is probably the most important metric for end users. It wouldn't be very accurate for GPU to GPU or device to device comparisons. Other things you'd run into are V-sync and things like that.

 

[BRiT]

So we basically need something like a high-speed videocamera to look at the smart-device for framerates, right?

Unless the device has ability to send display out via hdmi , into HDMI capture. Not many do, and those that do, I don't know if it mirrors or used as secondary display or only usable for video, not games.

 

[Deleted member 13524]

Unless the device has ability to send display out via hdmi , into HDMI capture. Not many do, and those that do, I don't know if it mirrors or used as secondary display or only usable for video, not games.

The most usual display output out there is the MHL port, which basically "reroutes" the micro-USB pins from a smartphone into HDMI pins for screen mirroring to regular displays.

 

[Nebuchadnezzar]

Rys is pretty spot on. Tools have GameBench have been released on Android which do exactly this - but they require root and therefore eliminate much of the possibility to be able to run it on many review devices.

 

[french toast]

Gamebench does NOT require root, just that you sync with pc app once only,so i do not see why reviews cant use this method. other than that very interesting post rys.

 

[Ailuros]

Nope, there's nothing (other than good code) stopping apps from implementing it themselves.

Then trying to convince ISVs to implement framecounters in their applications sounds like the best solution to me under current conditions.

 

[tuna]

ams brings up a great topic: performance analysis techniques that are possible on PC tend to be difficult or impossible to use on mobile operating systems like iOS and Android.

The primary reason is one of access: to hook into client drivers and display pipe internals in iOS and Android requires either specially packaged applications or privileged access. As a normal user of the OS, you can't insert yourself between the application and the operating system.

....

Even if I was to publish the source code of our Android driver, for example, there's no honestly easy way for you to use that to replace the driver on your device. A chunk of the driver isn't supplied by the GPU vendor (the memory allocator and display controller integration mainly) on Android so you'd need to get your SoC vendor to give you those parts too, then you'd need privileged access to push it to the device.

If you do not publish the source code of the Linux part of your Android drivers you are on very thin legal ice and open up yourself to the possibility of lawsuits from a lot of people.

Also, like Cyogen Mod and similar have shown, you can rebuild Android, and if you want to, put in a lot of extra features in the graphics stack.

 

[Rys]

Gamebench does NOT require root, just that you sync with pc app once only,so i do not see why reviews cant use this method. other than that very interesting post rys.

It's been a while since I spoke to the Gamebench people, but if I remember correctly their method wraps the entire application you want to profile, which introduces its own issues (and doesn't work for everything).

 

[Rys]

If you do not publish the source code of the Linux part of your Android drivers you are on very thin legal ice and open up yourself to the possibility of lawsuits from a lot of people.

Also, like Cyogen Mod and similar have shown, you can rebuild Android, and if you want to, put in a lot of extra features in the graphics stack.

Sure, you can rebuild it, but that's not something an end-user should be expected to do to get updated drivers.

 

[tuna]

Sure, you can rebuild it, but that's not something an end-user should be expected to do to get updated drivers.

No, but a dedicated tester could do it to benchmark stuff.

 

[Dominik D]

Then trying to convince ISVs to implement framecounters in their applications sounds like the best solution to me under current conditions.

Framecounters are not terribly useful. I'd rather have history of frame render times in ms or whatever available. You can always draw FPS from that but you also see spikes and variability of fluidity (lulz).

 

[Ailuros]

Framecounters are not terribly useful. I'd rather have history of frame render times in ms or whatever available. You can always draw FPS from that but you also see spikes and variability of fluidity (lulz).

Better than nothing. I'd rather have any potential reviewer being forced to actually play a game in real time on X devices than running just a benchmark and call it a day.

On a sidenote I've run a couple of TRex benchmarks on low end/integrated PC GPUs lately and despite that a GeForce 210 gets only a fraction of an integrated Intel GPU in terms of fps, there isn't a single Intel iGPU I've tried in that test up to now that doesn't stutter like hell especially at the beginning of the test.