3DMark for Windows 8 to unify test suite between ARM and x86 solutions

Discussion in 'Mobile Software' started by ToTTenTranz, Nov 14, 2011.

  1. ToTTenTranz

    Legend Veteran Subscriber

    Joined:
    Jul 7, 2008
    Messages:
    11,486
    Likes Received:
    6,228
    3DMark for Windows 8 to unify test suite between ARM and x86. UPDATE: Android too.


    So there'll be both DX9 and DX11 modes.
    DX9 hasn't been supported in 3DMark since 3DMark06, but we'll now get apples-to-apples comparisons between ARM SoCs and x86 with integrated and discrete graphics.

    This is going to be really interesting.
     
    #1 ToTTenTranz, Nov 14, 2011
    Last edited by a moderator: Nov 30, 2011
  2. darkblu

    Veteran

    Joined:
    Feb 7, 2002
    Messages:
    2,642
    Likes Received:
    22
    While this surely is a good thing, this release makes it sound as if nobody ever ran benchmarks comparing mobile and non-mobile gpus. GLES2 has been around for quite a while, you know.
     
  3. ToTTenTranz

    Legend Veteran Subscriber

    Joined:
    Jul 7, 2008
    Messages:
    11,486
    Likes Received:
    6,228
    What's "GLES2"?
    OpenGL ES 2? That's an API, not a benchmark.

    What benchmark is available for both x86 Windows and any ARM-supported OS, with the same test suite?
     
  4. darkblu

    Veteran

    Joined:
    Feb 7, 2002
    Messages:
    2,642
    Likes Received:
    22
    Likewise with DX9 and DX11, support for which has been a big bulletpoint in the announcement.

    Why would a benchmark need to be available for windows for the benchmark to be valid? What's wrong with benchmarks that don't include windows? Are you trying to compare GPUs or something else?

    As to the crux of your question: any game/rendering engine that has GL + GLES2 and x86Linux + ARMLinux/iOS/Android support and some sort of rudimentary free benchmarking mode/samples/demos. That includes (off the top of my hat):

    http://unity3d.com/unity/
    http://ogre3d.org
    http://irrlicht.sourceforge.net/
    http://code.google.com/p/gamekit/ (can run with ether ogre or irrlicht)

    The missing component in all those is a scores aggregator site. Something that can be put together in less than a man-week. Actually, GLBenchmark could easily extend their benchmarks and score site to include desktop GL scores as well. Apparently they have not had the motivation so far. But outside of the public space, everybody who'd been interested in cross-platform game development has been doing this (i.e. comparing performances across various classes of GPUs) internally for years now.

    Don't get me wrong. As I said, it's good that Futuremark are doing it with their well-accepted benchmark. But there's nothing special technology-wise about it. It's just that they have the motivation (and business model) to do it. And since they are windows-centric, they can finally do it on GPUs that were previously beyond their reach.

    edit: to give you a hint of what can be done using GL+GLES, here's GLview which is not much of a benchmark (program's main focus is elsewhere), but has been supporting GL + GLES and a database site for years now.
     
    #4 darkblu, Nov 15, 2011
    Last edited by a moderator: Nov 15, 2011
  5. Exophase

    Veteran

    Joined:
    Mar 25, 2010
    Messages:
    2,406
    Likes Received:
    430
    Location:
    Cleveland, OH
    I guess the biggest problem is that to date the SGX drivers for Atom platforms have been even worse on Linux than they are on Windows. And that most people would be using Windows.

    No idea if things are going to finally change here.
     
  6. ToTTenTranz

    Legend Veteran Subscriber

    Joined:
    Jul 7, 2008
    Messages:
    11,486
    Likes Received:
    6,228
    Easy answer there, GPU driver development and optimization is near exclusively oriented to Windows..
    OpenGL gaming benchmarks show a difference to almost 5 times more performance in Windows than Linux.. Because let's face it: who cares for anything other than Windows in PC gaming?
    Even the recently ported MacOS games lack a substantial chunk of performance compared to the exact same system in Windows environment.

    So I don't think you can count all those benchmarks as valid for comparing GPUs with ARM SoCs to GPUs with x86 systems.




    Systems. CPU+Memory+GPU combinations. We'll finally see how far ARM SoCs will be from low-power x86 solutions in 2012.

    Sure, one will easily reach the conclusion that a Radeon HD6970 is some 2 orders of magnitude faster than a SGX535 in DX9..
    But will a C-60 be faster than a dual-core Krait @ 2.5GHz + Adreno 320 for gaming?
    They're both going into tablets.
     
    #6 ToTTenTranz, Nov 15, 2011
    Last edited by a moderator: Nov 15, 2011
  7. darkblu

    Veteran

    Joined:
    Feb 7, 2002
    Messages:
    2,642
    Likes Received:
    22
    IMO, Intel is the worst licensee IMG has ever had. Trying to judge about SGX from the Paulsbo chipset et al is pointless.

    Luckily, we have SGX in a bunch of other platforms. iOS, for instance, is a good base for comparison now. Until recently one could not use FSAA there due to the fact apps always rendered to FBOs, but I hear that was fixed in iOS4 (my dev license expired so i cannot check myself).

    This is simply not true in the current mobile landscape, where GLES across linux, android, iOS and, heck, Symbian^3 make DX@W8 night irrelevant.

    The one aspect GL (and GLES) has lagged behind DX for the past couple of iterations is drawcalls overhead (from the POV of the same functional subset, apparently). Disregarding any current or future API advancements there, (1) I'm not advocating comparisons among parts under DX and GL stacks, but among GLES2 and GL2, and (2) drawcalls overhead can be countered by properly designed benchmarks.

    So yes, I think that for the same functional subset, I can compare ARM SoCs GPUs with higher-class GPUs under x86, and the results can be as valid as any. And there's nothing extraordinary about it.
     
    #7 darkblu, Nov 15, 2011
    Last edited by a moderator: Nov 15, 2011
  8. Exophase

    Veteran

    Joined:
    Mar 25, 2010
    Messages:
    2,406
    Likes Received:
    430
    Location:
    Cleveland, OH
    But what if you want to judge not SGX but Atom + SGX in conjunction? Seems like a pretty reasonable desire.
     
  9. darkblu

    Veteran

    Joined:
    Feb 7, 2002
    Messages:
    2,642
    Likes Received:
    22
    Ah, I admit did miss your original point. Yes, by conjecture (or perhaps, historically), one could expect that SGX@Atom driver support would be better under DX@W8. But still, there's no guarantee it won't be botched in one way or another, bar from the HW vendor's best of interest to have their HW reaching its advertised potential. And we know how that went last time with Intel and SGX. Heck, Intel used to underhand their own GMA designs for generations (not just the SGX-based ones), across both windows and linux (latter notably worse, of course). I don't see what new W8 can bring to the table here.
     
  10. ToTTenTranz

    Legend Veteran Subscriber

    Joined:
    Jul 7, 2008
    Messages:
    11,486
    Likes Received:
    6,228
    And how is "the current mobile landscape" (which lacks any kind of AAA 3D game development and games are all made for the lowest common denominator) even remotely relevant for this thread, which is talking about a benchmark appearing somewhere in Q3 2012, using at least DX9 compatible GPUs?


    Let's just agree to disagree, then.




    You really don't?
    W8 will bring a tsunami of competition from ARM SoCs for mobile solutions (Win8 tablets, Win8 netbooks, Win8 UMPCs in the long run, etc), which is going to set Intel's ass on fire and force them to get a lot more aggressive with Atom, both through the integrated GPU and through the process nodes they use.
     
  11. darkblu

    Veteran

    Joined:
    Feb 7, 2002
    Messages:
    2,642
    Likes Received:
    22
    Ah, the 'AAA 3D games coming to a tablet near you any day now' argument. W8-powered mobiles can hope to have as much 'AAA' games as your iOS device gets today. Not more, not less. Surely, the picture gradually improves with each hw generation, but don't expect W8 to make a splash there. It won't. Because the exact same guys who are making the iOS and Android games today will be making the W8 ones. And if Futuremark would hypothetically spawn a GLES2-based version of their benchmark, you'd have been able to see the 'future of benchmarking' already today, across the iOS and Android ecosystems. But apparently that does not bode with their business model, so enter W8 the gamechanger.

    BTW, you don't believe any of the current GLES2-compliant gpus are not DX9-level (or better), do you? Or that Epic's UE3 port to iOS and Android would take anything less than GLES2? DX9-class tech is alive and kicking in the mobile space today, without a trace of the actual DX9 sw stack to be seen ; )

    I'm perfectly ok with that.


    Surely W8 will put pressure on Intel. But do you think that as a result Intel will all of a sudden become a proper GPU vendor, and one in the SoC segment at that? How come W7 did not make them one in the desktop space before, which is Intel's home turf?
     
  12. sebbbi

    Veteran

    Joined:
    Nov 14, 2007
    Messages:
    2,924
    Likes Received:
    5,293
    Location:
    Helsinki, Finland
    Win8 can run all the Win7/Vista games. You can install Steam to it, and download all the AAA games on it right now. The Samsung Series 7 slate (the Win8 demo tablet that Microsoft gave developers) has exactly the same hardware inside it than Macbook Air and the newest Intel Ultrabooks (4 GB of RAM, newest generation Sandy Bridge i5 CPU that runs four HW threads and supports AVX vector instruction set, full speed HD 3000 GPU, a fast 64/128 GB SSD and a 1366x768 screen). HD 3000 GPU is enough to run many console ports and biggest PC AA titles at 30 fps (see the list below). If Win8 tablets and Win8 Ultrabooks become as popular as Intel is hoping (over 50% of shipped PCs) developers will surely start to optimize their engines specially for those platforms as well.

    Some Intel HD 3000 GPU scores (at 1024x768, based on Anandtech's numbers):
    Modern Warfare 2: 42.2 fps
    Bioshock 2: 35.1 fps
    Starcraft 2: 32.6 fps
    Dirt II: 30.1 fps
    World of Warcraft: 42.8 fps

    Intel is launching Ivy Bridge before Win8 launch. Intel claims it has 60% faster GPU and 20% faster CPU, and lower power consumption. I am sure there will be many Ivy Bridge based tablets at Win8 launch. And I am pretty sure there will be many players playing World of Warcraft (the king of all AAA games) on those tablets.
     
  13. darkblu

    Veteran

    Joined:
    Feb 7, 2002
    Messages:
    2,642
    Likes Received:
    22
    You lost me here. You can run all Win7 games on a x86 tablet? Disregarding for a sec all likelihood that the tablet might not meet the basic hw requirements of a high-profile PC game, how is one expected to interact with those games? By virtual kbd and d-pad?

    For those AAA W8@mobile games to appear, somebody has to write them to the specs (that includes basic things as HID too) of those devices, and there's no way around this. You do realize that part of the reason Apple have been so successful with their AppStore is because they have not been trying to push desktop sw to their iOS users, right?

    Ok, so you actually did not mean what you wrote in the 1st paragraph. Phew ; )

    Of course developers will start to develop to specs. And of course there will be a decent selection of games. What I don't understand from your post, is how you believe that situation will differ from the current iOS/Android ecosystems, where, low and behold, we have middleware developers like Epic with their UE3, Unity, etc, with a myriad of down-ported higher-end tech, which allows for write-once-deploy-everywhere development.

    I can't see how that fits the entry-level mobile segment. Is that an argument against what I said about Intel not being a proper GPU vendor? Because you could have made a much stronger point by showing me how Intel have started properly supporting their not-so-old GMA designs, across the OSs they ship those under. One GMA iteration doing well under Windows is not impressing me much - I've seen those before.

    I betcha, those WoW players are of the play-on-the-go type. I mean, look at all those (formerly common) x86 Winbooks which could run WoW more-or-less adequately - they were mostly used by WoW players. Or heck, any players. I'm sure the Ultrabooks will repeat the netbooks' success. If not, the Gigabooks surely will.
     
  14. Ailuros

    Ailuros Epsilon plus three
    Legend Subscriber

    Joined:
    Feb 7, 2002
    Messages:
    9,455
    Likes Received:
    186
    Location:
    Chania
    Embedded platforms have still (and will continue) to have a large distance to anything like netbooks or even worse low end PC/notebooks, first above all in per device TDP. Throwing them blindly all into the same melting pot just because one ISV decides to create a unified cross platform synthetic benchmark is utter nonsense.

    Unless I've missed anything darkblu is IMO correct on all of his points. There's no such thing as any sort of concentration on windows driver development in the embedded space up to recently. Major focus is and will continue to be OGL_ES driver development and yes win8 will signify a new era, but if it'll really be able to come even close in terms of deployment to linux kernel based OSs of the embedded space is in the stars (I personally also doubt it, but prefer to see how it rolls out before jumping to any conclusions).

    Even worse if someone would want to hold a crystal ball about future evolutions in terms of embedded APIs I'm afraid we'd need to know more about upcoming OGL_ES3.0/Halti, how it'll co-operate with OpenCL and how developers will react to it. Win8 on embedded platforms doesn't necessarily mean obligatory DX11 for embedded GPUs. I severely doubt that even Halti will go as high simply because embedded APIs get defined by a multitude of ISVs and not all of them have the luxury to have a DX11 equivalent design on their road-maps even much less by the time win8 actually ships.

    Besides DX11 means quite a high overhead in terms of transistors for a GPU and I'd much rather prefer that things take their time, instead of sacrificing a high persentage of transistors into higher level features and sacrifice performance.

    With probably only a few exceptions current embedded GPUs are not even fully DX9 compliant; neither FP16/20 nor 16bit only precision amongst others fullfill DX9. Embedded IHVs have to make a number of cutbacks in order to save die area and in extension power consumption wherever they can and it's not going to change in the future either. So I'm really wondering what some folks are actually wishing for here.
     
  15. sebbbi

    Veteran

    Joined:
    Nov 14, 2007
    Messages:
    2,924
    Likes Received:
    5,293
    Location:
    Helsinki, Finland
    Did you read my post? I listed several AAA games that run perfectly well on current generation x86 tablet hardware that already exist. Yes, those games run with full network multiplayer features, etc. You can buy a tablet immediately if you want to start playing. The hardware is fine, however current high end tablets feature Win7, and thus are not yet ready for mainstream (Win8 should fix that problem).

    At Win8 launch we will surely see even more powerful Ivy Bridge based tablets. We are not talking about tablets like iPad that are running default phone hardware on a bigger screen, but fully capable x86 PC notebook hardware. You do not need a high end desktop PC to play console ports anymore (high end PCs have over 10x performance compared to consoles). Notebooks are just fine if low quality settings (equal to consoles) are enough for you.
    The touchscreen emulates mouse by default pretty well. Existing games developed for mouse control will play pretty well. Virtual dpad is great for some games, but not for all. Series 7 slate has also USB ports, and you can for example plug in a Xbox 360 controller and it works right away with all PC games that support it (most console ports do). You can also use wireless (bluetooth) mouse and keyboard if you want (very good if you want to use your tablet for real work as well).

    Of course there will be games designed mainly for touchscreen experience. But Win8 requires all Metro apps to work properly on keyboard + mouse as well, since Win8 is not only a tablet OS. All Metro apps should work on tablets, notebooks and desktop computers.
    What this has to do with bringing AAA games to Windows 8 tablets? I don't see the connection.
    How many real AAA console ports have you seen on iOS and Android? How many AAA console ports have you seen on Windows XP/7/Vista? Win8 is fully backwards compatible, and thus all cross platform game developers have technology ready to support Win8. Win8 isn't just about tablets, it is the most recent Windows version for desktops and notebooks as well, and thus will be the operating system with most AAA games released in the next years. I am sure that most AAA game developers are willing to spend the miniscule amount of time required to implement virtual dpads to their PC games, so that the games run properly on Win8 tablets as well. Some developers are of course willing to go further, and will implement highly tuned touchscreen focused interfaces.

    Win8 release will the the most interesting thing in PC gaming for a long long time.
     
  16. sebbbi

    Veteran

    Joined:
    Nov 14, 2007
    Messages:
    2,924
    Likes Received:
    5,293
    Location:
    Helsinki, Finland
    ARM Mali T-604 supports full DirectX 11 feature set. First products using it are expected to launch early next year. Samsung has licensed it already (for Android phones I believe).

    I compared mobile GPU specs (and extensions) to the Win8 DX11.1 API DX9.3 feature requirement list in another thread. Tegra seems to be 9.3 capable already, and others are mainly lacking MRT (multiple render target) support. Some Imagination tech insiders said their hardware supports MRT, but the feature is not yet exposed by the OpenGL extensions. If DX 9.3 will be the minimum requirement for all Win8 devices, I expect minimal HW changes are needed to bring all mobile GPUs to the required specs.
     
  17. Ailuros

    Ailuros Epsilon plus three
    Legend Subscriber

    Joined:
    Feb 7, 2002
    Messages:
    9,455
    Likes Received:
    186
    Location:
    Chania
    So does IMG's Rogue where they haven't failed to note that the architecture will scale depending on needs in between DX10 and DX11. What are you going to need DX11 exactly for in an android environment? If you'd tell it's going to be a windows phone with win8 there would be some reasoning behind it. As for projections it's the job of different marketing departments to be overly optimistic. Just like AMD promised that there will be 28nm based GCN GPUs this year, don't place your hopes too high that you'll see any next generation embedded GPU before 2013.

    Samsung has also licensed Series5XT just a couple of days ago, so I'd say that its reasonable to expect that one first before T604 and no it's not going to launch tomorrow either.

    It's DX9 Level3 for the record (DX11 certified DX9) and no FP20 PS ALUs, 16bit Z precision and 2048*2048 TMUs (Tegra) are not good enough even for plain DX9. When you lack even one requirement then it's simply not DX9 in the strict sense. However if Microsoft should want to close a couple of eyes here and there for win8 or whatever else then it's a completely different chapter. Judging by desktop DX9 and in extension DX9 L3, the story is as above.

    SGX has FP32 ALUs, 24 bit Z precision and their DX9 L3 cores (SGX544, 554) have 4096*4096 TMUs amongst other details. MRTs are supported, but amongst all the far more important details you're stuck on MRTs? Unless you have some weird win8 GUI in mind that uses deferred shading I have a hard time imagining why it should be of any particular importance.

    Still all the above doesn't change one bit of what I said in my former post. What some of you might be seeing in anything windows is some sort of Messiah that would transform the embedded space over night. Au contraire what windows will add is a transistor overhead that isn't absolutely necessary in the embedded space and could be invested far better in higher performance. Or maybe some want to run endlessly around the infamous dragon in Unigine for embedded tessellation pissing contests.
     
  18. Exophase

    Veteran

    Joined:
    Mar 25, 2010
    Messages:
    2,406
    Likes Received:
    430
    Location:
    Cleveland, OH
    I can't find any substantiation behind that 20% improved CPU claim that's mentioned on Wikipedia. The Anandtech article it links to makes no say of it, and I know I've seen much smaller IPC improvement claims from Anand in the past, with no indication that clock speed will increase at all (top bin is only 77W)

    And that 60% improvement is going to be comparing top bin to top bin. The ULV 17W option won't come anywhere close to that performance and you'd better believe it'll use all that power while running anything resembling a modern game.

    Same thing goes for current generation. Look at this direct comparison:

    http://www.tomshardware.com/reviews/eee-slate-ep121,2986-7.html

    Of course, the battery life on this thing is a joke:

    http://www.tomshardware.com/reviews/eee-slate-ep121,2986-9.html

    It doesn't say what it's like while gaming but HD 3000 is not known to be especially efficient and if there's any point where you're exercising the full 18W TDP of the chip it's going to be then. If it can't even get 2 hours during h.264 playback, which shouldn't be exercising the CPU that much nor the non-video parts of the GPU how well can it really do while exercising GPU full tilt?

    This is of course for a huge slate that's twice the weight of an iPad 2 and over twice as thick (never mind comparisons to tablets with smaller screens).

    So I'm sure there'll be at least some IB tablets early on, probably not "many", and they'll still suck for gaming. There'll be more in the way of "convertibles", but contrary to market I don't consider these tablets but specialized laptops (and they've been around for years)
     
  19. darkblu

    Veteran

    Joined:
    Feb 7, 2002
    Messages:
    2,642
    Likes Received:
    22
    I did. You did not specify they were ran on the Samsung series7, just that they were benchmarked on a HD3k-equipped system and tablet-like screen res. So, of those games, how many were ran on the slate just for the benchmark, and how many were playable on the slate, without bringing your own set of peripherals? As you can't seriously say virtual kbd and d-pad are viable control schemes. The situation is akin to playing your GBA emulator roms on your touch-screen phone. Some games might be playable, but most are ruined by the foreign controls. Which makes them hardly 'AAA' games.

    See, the whole premise of 'here's a tablet that has a notebook chipset, if your slap you desktop peripherals to it it becomes a PC' is not really driving the point with me. That's what notebooks/netbooks have been for generations - they are already PCs one can carry around.

    Why would I wan't to buy a device where i could play gimped versions of games I can play on a desktop to a much better experience? /rhetorical

    Exactly. Notebooks are fine. Tablets - not so much (without the extra HID hw). Now, if you're saying that people will drop their notebooks for tablets (vis-a-vis getting a tablet along with the notebook/desktop), I have nothing but skepticism for you.

    Yep, so far I'm with you, but that has nigh to do with the segment we're discussing here. Desktop GL (say, 2) and GLES2 are also very compatible technologies. Heck, OSX and iOS are ultra compatible technologies in terms of APIs et al. Of course, claiming that my OSX apps somehow can automagically make the transition to iOS' entirely-touch-based ecosystem would be considered at best naive by anybody who'd every tacked the problem.

    And this is where we part. Apparently you and I have radically different notions of 'games running properly' on a tablet. What you consider as 'some developers might decide to go further' has been proven by Apple to be the very fundamental of successful tablet/touch-screen ecosystems. It's not optional in any way. You can technically neglect it, but that'd result in a badly designed software. So much for the AAA games.

    For those who don't have a PC already - it surely will.

    Co-signed.
     
  20. sebbbi

    Veteran

    Joined:
    Nov 14, 2007
    Messages:
    2,924
    Likes Received:
    5,293
    Location:
    Helsinki, Finland
    Intel has recently upgraded their integrated GPU performance pretty much by a factor of 2 every time they have introduced new products. I don't find it in any way hard to believe they achieved 60% boost by going to a much smaller process (and 3d transistors to boot). Anything less than that would be an utter failure.

    I have programmed for those old Intel intergrated DX9 chips (GMA 900/950). Things have changed a lot since. The new Intel chips are fully DX10 capable, and can run exactly the same code as AMD/NVidia chips (you don't have to code separate Intel paths anymore). And the performance has increased dramatically as well. These chips can run modern games (at 720p).
    That's a completely different product, based on older (and cheaper) technology. The new energy efficient Sandy Bridge CPU models are very energy efficient compared to those old i3/i5 CPUs, and the GPU performance is around 3x higher as well. In comparison, the Samsung Series 7 Slate has 7 hour battery life. It's the same thing as saying Galaxy S2 must have bad battery life, because some last gen Android phone had bad battery life. If you want to see some battery life tests on those new energy efficient Sandy Bridge models, I suggest looking at some Macbook Air battery tests. Both have identical hardware (Apple likes to use lots of Samsung components in their designs).
    iOS cannot run OSX applications. Win8 tablets can run Win XP/7/Vista applications. The binary files work right away, no problems.
     
Loading...

Share This Page

  • About Us

    Beyond3D has been around for over a decade and prides itself on being the best place on the web for in-depth, technically-driven discussion and analysis of 3D graphics hardware. If you love pixels and transistors, you've come to the right place!

    Beyond3D is proudly published by GPU Tools Ltd.
Loading...