AMD Mantle API [updating]

[itsmydamnation]

There only way to get a semblance of convergence on BF4 in a live multi-player environment is by playing hours for each configuration. And even then with similar game play. That may be enjoyable for the reviewer if he's into that kind of game, but it will somewhat kill his writing productivity.

i offered..... all i need is someone to send me the hardware

 

[silent_guy]

i offered..... all i need is someone to send me the hardware

Constructive comments offering help: we need more of those!

 

[Andrew Lauritzen]

http://bf4stats.com/pc/itsmydamnation

I take it you play a lot of infantry maps/modes

Sure its hardly perfect, but it doesn't need to be

It does need to be - we're talking about ~10% differences here. I remain completely unconvinced that you could do any useful benchmarking in multiplayer on public servers.

You don't really need Mantle to prove or disprove that part of the equation. Just take a pile of frame time logs from different games/servers and see how much variance there is.

 

[itsmydamnation]

I take it you play a lot of infantry maps/modes

nope just only play inf, most played maps would be zavod and paracel in conquest mode 64 player. If i played inf only my KDR would be way higher ( i have a C4'ing tank obsession) .....lol

It does need to be - we're talking about ~10% differences here. I remain completely unconvinced that you could do any useful benchmarking in multiplayer on public servers.

You don't really need Mantle to prove or disprove that part of the equation. Just take a pile of frame time logs from different games/servers and see how much variance there is.

But again so long as you capture enough data from both DX and mantle on the same group of servers you will account for the viability. also its quite easy to tell when a BF4 server isn't running well, especially when you have network smoothing turned off (like i do, there are 50 odd 64p server within 10ms of me) so you can just omit that run.

variance is not a bad thing because that's the real world, instead of trying to remove it, just account for it.

 

[Andrew Lauritzen]

variance is not a bad thing because that's the real world, instead of trying to remove it, just account for it.

I'm a math major, I don't need a lecture on stats I'll remain sceptical until you/someone proves you can "account" for variations to the level required in any reasonable amount of time. There are just too many variables (and too many correlated ones too) to "trust" that someone has factored everything else out. I could trivially show you any result you want from the noise and make it look legit.

 

[itsmydamnation]

thats why peer review exists :smile:. , im a trustworthy guy right

i will do some numbers when i get around to it, but im a bit atypical 6950 @ 3840x1024 on 3x 5:4 17inch LCD monitors

 

[sir doris]

I would be interested if you could use the BF4 frame time capture to bench 3 or 4 server ticks on the same map, running the exact same settings and API, to see how close they are. It should give an idea of how useful that method could be.

 

[itsmydamnation]

had a quick bash tonight on zavod 64P conquest, full server.

I have just done frame time
setup is
I7-3770@ 4.3ghz
16gb 1866 ramm
6970 ( 6950 bois flashed to 6970) @ 3840x1024 most settings @ medium no AA no AO, 75 FOV

 

[Jaaanosik]

"Proletarians of all countries, unite!" ... the Marxist approach might be a way to go.

The testers should unite and create a multiplayer 'scripts' that they would run on their testing server.
Everybody should be able to report more repeatable results.

 

[lanek]

Have just run some basic test on Starswarm demo ( 1.0 steam version )

System used:

- Benchmark v. 1.0 ( steam version )
- Single 7970 @ stock speed ( 1050mhz )
- I7 4930K @ stock speed
- Ram: 16Gb 2133mhz C9


Follow + extreme setting ( default benchmark resolution: 1080p ( on 2560x1440p screen ).


Mantle: 51.92fps

Oxide Games
Star Swarm Stress Test - ©2013
C:\Users\lanek\Documents\Star Swarm\Output_14_02_06_1540.txt
Version 1.00
02/06/2014 15:40
================================================== =========

== Hardware Configuration =================================
GPU: AMD Radeon HD 7900 Series
CPU: GenuineIntel
Intel(R) Core(TM) i7-4930K CPU @ 3.40GHz
Physical Cores: 6
Logical Cores: 12
Physical Memory: 17116585984
Allocatable Memory: 8796092891136
================================================== =========


== Configuration ==========================================
API: Mantle
Scenario: ScenarioFollow.csv
User Input: Disabled
Resolution: 1920x1080
Fullscreen: True
GameCore Update: 16.6 ms
Bloom Quality: High
PointLight Quality: High
ToneCurve Quality: High
Glare Overdraw: 16
Shading Samples: 64
Shade Quality: Mid
Deferred Contexts: Disabled
Temporal AA Duration: 16
Temporal AA Time Slice: 2
Detailed Frame Info: Off
================================================== =========


== Results ================================================
Test Duration: 360 Seconds
Total Frames: 18692

Average FPS: 51.92
Average Unit Count: 4585
Maximum Unit Count: 6068
Average Batches/MS: 835.36
Maximum Batches/MS: 5654.34
Average Batch Count: 18702
Maximum Batch Count: 106006

DirectX: 31.32fps

Oxide Games
Star Swarm Stress Test - ©2013
C:\Users\lanek\Documents\Star Swarm\Output_14_02_06_1548.txt
Version 1.00
02/06/2014 15:48
================================================== =========

== Hardware Configuration =================================
GPU: AMD Radeon HD 7900 Series
CPU: GenuineIntel
Intel(R) Core(TM) i7-4930K CPU @ 3.40GHz
Physical Cores: 6
Logical Cores: 12
Physical Memory: 17116585984
Allocatable Memory: 8796092891136
================================================== =========


== Configuration ==========================================
API: DirectX
Scenario: ScenarioFollow.csv
User Input: Disabled
Resolution: 1920x1080
Fullscreen: True
GameCore Update: 16.6 ms
Bloom Quality: High
PointLight Quality: High
ToneCurve Quality: High
Glare Overdraw: 16
Shading Samples: 64
Shade Quality: Mid
Deferred Contexts: Disabled
Temporal AA Duration: 16
Temporal AA Time Slice: 2
Detailed Frame Info: Off
================================================== =========


== Results ================================================
Test Duration: 360 Seconds
Total Frames: 11276

Average FPS: 31.32
Average Unit Count: 4346
Maximum Unit Count: 5443
Average Batches/MS: 539.25
Maximum Batches/MS: 1067.38
Average Batch Count: 19492
Maximum Batch Count: 140942
================================================== =========

Attract scenario, extreme setting, default benchmark resolution (1080p )

Mantle: 48.26fps

Quote:
================================================== =========
Oxide Games
Star Swarm Stress Test - ©2013
C:\Users\lanek\Documents\Star Swarm\Output_14_02_06_1603.txt
Version 1.00
02/06/2014 16:03
================================================== =========

== Hardware Configuration =================================
GPU: AMD Radeon HD 7900 Series
CPU: GenuineIntel
Intel(R) Core(TM) i7-4930K CPU @ 3.40GHz
Physical Cores: 6
Logical Cores: 12
Physical Memory: 17116585984
Allocatable Memory: 8796092891136
================================================== =========


== Configuration ==========================================
API: Mantle
Scenario: ScenarioAttract.csv
User Input: Disabled
Resolution: 1920x1080
Fullscreen: True
GameCore Update: 16.6 ms
Bloom Quality: High
PointLight Quality: High
ToneCurve Quality: High
Glare Overdraw: 16
Shading Samples: 64
Shade Quality: Mid
Deferred Contexts: Disabled
Temporal AA Duration: 16
Temporal AA Time Slice: 2
Detailed Frame Info: Off
================================================== =========


== Results ================================================
Test Duration: 360 Seconds
Total Frames: 17377

Average FPS: 48.26
Average Unit Count: 4422
Maximum Unit Count: 5733
Average Batches/MS: 1256.60
Maximum Batches/MS: 8297.07
Average Batch Count: 29482
Maximum Batch Count: 201757
================================================== =========

DirectX: 25.60fps

================================================== =========
Oxide Games
Star Swarm Stress Test - ©2013
C:\Users\lanek\Documents\Star Swarm\Output_14_02_06_1609.txt
Version 1.00
02/06/2014 16:09
================================================== =========

== Hardware Configuration =================================
GPU: AMD Radeon HD 7900 Series
CPU: GenuineIntel
Intel(R) Core(TM) i7-4930K CPU @ 3.40GHz
Physical Cores: 6
Logical Cores: 12
Physical Memory: 17116585984
Allocatable Memory: 8796092891136
================================================== =========


== Configuration ==========================================
API: DirectX
Scenario: ScenarioAttract.csv
User Input: Disabled
Resolution: 1920x1080
Fullscreen: True
GameCore Update: 16.6 ms
Bloom Quality: High
PointLight Quality: High
ToneCurve Quality: High
Glare Overdraw: 16
Shading Samples: 64
Shade Quality: Mid
Deferred Contexts: Disabled
Temporal AA Duration: 16
Temporal AA Time Slice: 2
Detailed Frame Info: Off
================================================== =========


== Results ================================================
Test Duration: 360 Seconds
Total Frames: 9215

Average FPS: 25.60
Average Unit Count: 3939
Maximum Unit Count: 5507
Average Batches/MS: 643.51
Maximum Batches/MS: 1172.64
Average Batch Count: 26256
Maximum Batch Count: 187408
================================================== =========

 

[Jawed]

Thanks, Jawed also posted golem.de way back, I missed it.

Nemo posted them!

http://forum.beyond3d.com/showpost.php?p=1824756&postcount=1040

I just linked them in a follow-up post.

 

[Barbarian]

So I had an idea. How about this:
Two machines, identical setup, one runs DX11, the other Mantle.
Join same server, go to observer, go to same spot, look the same way.
Record FPS.
Done

 

[DavidGraham]

So I had an idea. How about this:
Two machines, identical setup, one runs DX11, the other Mantle.
Join same server, go to observer, go to same spot, look the same way.
Record FPS.
Done

Brilliant! that might just do it!

 

[Alexko]

Wouldn't there be a somewhat lower CPU load in observer mode?

 

[silent_guy]

So I had an idea. How about this: Two machines, identical setup, one runs DX11, the other Mantle. Join same server, go to observer, go to same spot, look the same way. Record FPS. Done

Excellent! But even better: 3 machines. 1 player. 2 observers: one with Mantle, the other without.

Edit: I'm dumb. They're both already in observer.

 

[3dilettante]

To get at least what TR got for its early analysis, however, this would only cover one CPU/GPU combination for a comparison.

One more Mantle/non-Mantle would be needed for a different CPU.
The setups could be different when getting numbers for Nvidia and a weak/strong CPU.

However, with just two players you now have no simultaneous runs, so the different configurations stop being comparable without multiple runs and averaging things out for comparison.
Unless you get six players and six setups in observation mode.
Hopefully on the same network?
Does observer mode change things?

And then, unless someone claims the testing wasn't representative because that game might have had an unreasonably low/high number of actions/events.
Better capture footage.
If someone claims player X had "teh bias", better capture all six...

 

[rpg.314]

In my opinion, it goes to show that for all its inefficiency, the full real-world impact of the APIs an industry has spent countless hours and millions to billions of dollars to use effectively is overblown.
Yes, drivers and APIs can fall down, but the people working on this haven't been so universally incompetent that they don't get the majority of peak most of the time.

I beg to differ.

DX is certainly works well, if you are GPU limited. And that right there is the crux of the problem. An API's job is to get out of the way, and not dictate that one half of the silicon must be ignored.

 

[3dilettante]

I beg to differ.

DX is certainly works well, if you are GPU limited. And that right there is the crux of the problem. An API's job is to get out of the way, and not dictate that one half of the silicon must be ignored.

Those not with an AMD processor are GPU limited because DX and the developers using it work so well. It doesn't take much effort to become GPU limited, and with AMD's lack of progress on multiple fronts and contrived examples like Oxide, the effort to get to AMD's sweet spot becomes progressively unreasonable as time goes on.

The 290X's middling scaling of Mantle when using an Intel CPU--in comparison with itself or the 780 still on D3D, shows that even if the API gets out of the way, AMD's GPUs barely take you further.
AMD can present a stronger case for the API holding its wonderful GPUs back when they don't stall 12 FPS higher than with said API.

 

[DavidGraham]

Unless you get six players and six setups in observation mode.
Hopefully on the same network?
Does observer mode change things?

Actually I think we can get by with 3 identical test beds for now .. one with 290X Mantle , one with 290X DX , and then one with NVIDIA DX.

A different comparison can be made with a low CPU using the same 3 test beds.

The observer mode just works like a "no clipped" player , you can fly around to any spot, follow players, change camera from a multitude of camera views (including 1st person, 3rd person and others). If you choose your spot carefully you can get a wide picture of the whole map with many intense fights taking place.

 

[3dilettante]

Which CPU is going to be chosen?
The TR testing compared along that axis as well.