[Doomtrooper]

Comanche 4 benchmark utility, something OTHER than Quake3

http://www2.novalogic.com/downloads.html#c4demo

Radeon 8500 users make sure you edit the C4.cfg, Although C4 detects shaders it disables them since this was designed on a Geforce 3.
You have to go into C4.cfg and set Disable_Shaders=1 to 0

[Dave Baumann]

Fortuitous timing - I was just wondering what other benchmarks I could use for this Radeon review. Even more useful to see what the Pixel Shader hit is like.

[Dave Baumann]

Thank christ for that as well - it has a batch mode as well!

[kl899]

is there an alternative download site? gigex isn't working

[Doomtrooper]

Demo looks sweet but I think the engine needs some platform optimization work. I get 30 fps @ 1024 and 37 fps @ 800 x 600. I also don't like having to edit the .cfg file to enable shaders, whats the game look for when checking the card, DX compatability or just looks for a Geforce 3.

Anyhow nice to see a new benchmark anyways.

[rhink]

grr.... is there anywhere to download it without using that gigex crap?

[Doomtrooper]

None so far, Gigex seems to work ok for me. 15 second download and away you go ??

[rhink]

it ain't working for me... won't connect once the pos gigex xfer utility downloads.

[msan_ewu]

I hate Gigex too but downloaded it anyway.... I had a 24.27fps score but I'm sure most will post higher as I didn't drop any IQ settings. It sure felt faster than 25fps though... smooth

The demo is nice but nothing special compared to what's out already. I liked the water and the rotor wash though

Tbird@1.2
GF3
512 pc-133
XP

edit: my settings were all default i.e. 8x6, etc....

[Randell]

its strange, I get ~28 without AA or aniso, ~25 with 2xQualityAA and 16x aniso.

nvNews forums seem to be full of people with Gf3/8500 class hardware getting 25-30 fps, but a slew of Ti200 owners getting 37+fps!

[Ante P]

There's some guy over at nvnews with a 2.2 Ghz P4 and a overclocked Ti 4600 and he just managed to pass the 40 fps mark

something is not right
but if you look at the scene shifting you'll see that it actually blends two fully rendered scenes
crazy stuff that'll never happe during actual gameplay

it also seems like it's very CPU limited

[Mephisto]

What a cool benchmark! That's what we need, a modern dx8 benchmark made directly from a scene within a game! Finally, good bye Quake 3!

[Tahir2]

This benchmark is a load of crap. The game is meant to be fun to play but it doesn't look too hot IMO. I don't think the results are really all that useful. The only thing it tells me is that the graphics engine needs some serious attention. Dave I don't reccommend using it in your review.
Doesn't tell you anything useful really.


Results below:

33 fps at 800*600
30 fps at 1024*768

CPU - AthlonXP @ 1564.54 MHz (10.5*148.91)
Motherboard - EPoX 8KHA+ KT266A
RAM - Nanya 512MB PC2100 at CAS 2 148.91 MHz DDR
RadeonLE GPU at 295MHz RAM at 270MHz

Using the 6025 XP driver for the Radeon (I like to play DVD with hardware enabled and later drivers seem to have screwed that up).

Regards,

[Entropy]

Originally Posted by misae

This benchmark is a load of crap. The game is meant to be fun to play but it doesn't look too hot IMO. I don't think the results are really all that useful. The only thing it tells me is that the graphics engine needs some serious attention. Dave I don't reccommend using it in your review.
Doesn't tell you anything useful really.

I'd like to echo the above sentiment.
Besides managing to look pretty ugly in spite of an average of 200 000 polygons per frame, it seems to be mostly limited by the CPU/memory/buses(?) subsystem rather than the graphics.

In that respect it is similar to Unreal/UT, only that UT manages to be mostly CPU/memory limited in spite of having a tenth to a hundredth as many polygons per frame.... And still has a hard time achieving consistently good framerates on any hardware in existance.

I guess Carmack is a genious after all, or perhaps simply a competent craftsman among hackers and wanna-bes.

It scares me a bit when Tim Sweeney goes on and on about how many polygons Unreal 2 will use. Will it be another game(engine) that will never run particularly well on any hardware in relation to how it looks? I remember when Tim first started talking about the new engine and said that it wouldn't use any method of reducing the number of polygons for distant terrain because "it never looked good" but instead they would rely on the GPUs to handle it fast enough. Amazing to hear a man _brag_ about how they will cripple either performance or visual quality out of laziness.

Oh. I'm ranting. Sorry. Comanche4? Not impressed.

Entropy

[Mephisto]

How can you judge this benchmark based on postings based on different systems? Who knows, one might have AGP1x without SBA and SDR system memory, the other AGP4x with SBA with nForce and DDR. And obviosly the benchmark does different things with different gfx class ( DX7/8 ) hardware, so you would have to set the things manualy in the config file to get really comparable results.

BTW: I just get the impression that some people just don't like the benchmark because they see that their system with silly expensive shader graphics card does not get more thant 30 fps.

[Humus]

I wasn't especially impressed either, neither by graphics or by gameplay. After 5 minutes or so I decided I had enough and uninstalled it.


Entropy, don't forget that the Unreal engine was initially developed mainly for software rendering, then later upgraded to Glide, and then to DirectX/OpenGL later on. The techniques it was based on is much more suitable for software rendering than what Carmacks Q2/Q3 engines were. Unreal had way better graphics that Q2 for instance, with high performance on Voodoo2 and lower hardware. What made Unreal so fast on old cards like V2 was that it has almost no overdraw. But as GPU's continued to get faster very quickly the optimizations in the Unreal engine became a brake instead. Today the GPU sits idle waiting for the CPU to sort out visibility in Unreal engine based games that the GPU itself would sort out faster. The quake 2/3 engines put more work on the CPU instead, thus they scale very well with newer GPU's. On my old G400 UT would run much faster than Q3 for instance, on my old Radeon the performance was about the same, on my current Radeon8500 Q3 run much faster.

I don't think we should complain about Tim Sweeney's work, his engine lasted quite long, but it's gotten outdated now. But he has realized that himself too, thus the Unreal2 engine is taking something similar to Carmacks approach. That's probably why he choose not to do any fancy LOD stuff on the terrain, it would probably end up being slower anyway if he spent to much cycles on sorting out how much detail he should draw the terrain with, he's probably only doing some really rough visibility stuff and throws the rest on the GPU, it'll probably be a better solution in the long run.

[Randell]

Well I got bored very quickly with the gameplay in the demo.

People playing the game get much higher fps than they get in the benchmark, utilising the same settings, what.

As for SBA making a differnce between systems? How much?

The reality is across DDR and SDR systems, 1gig Athlons to 2 gig pentiums, almost everybody is throwing out similar results. The one anomaly seems to be Ti200 owners who are claiming higher frame rates than Gf3 and Ti500 owners. Again without side by side comparisons of system specs you 'cant' draw conclusions. Other than some Ti200 owners on nvNews like to brag

[Doomtrooper]

I'm sure we'll see a patch to address some of the issues, this is one of the 1st bechmarks besides Aquamark that uses some advanced effects and I'm surprised how quickly people are writing it off as crapola.

Besides I'm a sucker for seeing things blow up and if I see another Quake 3 timedemo I'm gonna lose it :P

[Entropy]

Originally Posted by Humus

I don't think we should complain about Tim Sweeney's work, ...

I disagree.
Though I can certainly forgive the man that he zigged instead of zagged on a particular technical point, I don't much care for his cathegorical statements, judgemental attitude and recently his infantile harping on how Unreal will have 100 000+ polys per frame.
First off, it's no guarantee that it will look particularly good. The Comanche4 Demo makes that embarrassingly obvious. Secondly, it implies that they don't do much (or any) LOD work, making for a gameworld bogged down with unnecessary detail at high distances in order to look reasonably good close up. Your comments about LOD are not without validity of course, but it is not as if it takes a huge amount of work to do, nor does it lhave to look bad, as long as you are not constrained by trying to reduce the scene to an absolute bare minimum number of polygons. Safe, conservative LOD is way better than none at all. Not using it is asking for trouble when you are creating a general engine, or an engine intended for multiplayer use.

Have _anyone_ here seen good framerates in any games when the polycounts climb beyond 6 digits? No, benchmarks such as 3DMarks don't count. Not too many such games around of course, and those that are may not be optimally coded, but.... I think a certain sceptisism is warranted. As long as you can reduce the polycount to improve framerate this won't be a problem, but that reduces all this "over a hundred thousand polygons" to marketing and little else.

We could probably have a good discussion here about how a large polygon budget should be used though. And what bottlenecks/pitfalls there are. New thread, someone?

And I'm sorry, but I can't help but comparing Sweeneys posturing with Carmacks nerdy description of how they make relatively few polys look really good, and how they allocate the available computing budget. it's probably a bit silly of me, but I know which attitude I'm comfortable with.

Entropy

[Dave Baumann]

BTW: I just get the impression that some people just don't like the benchmark because they see that their system with silly expensive shader graphics card does not get more thant 30 fps.

No, the benchmark is clearly CPU limited – mapping out the fillrate graph shows some real oddities with the Radeon 8500 because the drop off at 1600x1200 is far too much. Also the Pixel Shader don’t make much sense as so far all the testing indicates that its actually faster with PS enabled than disabled!

[Oompa Loompa]

Originally Posted by Entropy

It scares me a bit when Tim Sweeney goes on and on about how many polygons Unreal 2 will use. Will it be another game(engine) that will never run particularly well on any hardware in relation to how it looks?

Hmm. Well, Unreal/UT may have never been a good benchmark, but these were good games, and licensees made at least as many good looking and performing UT-engine games as Q3-engine games. So I'm not sure all of your criticism is justified.

But in any case, things seem to be different this time. According to the "Unreal Peformance Test 2002", results seem to scale very much according to GPU power. http://www.anandtech.com/video/showd...=1583&p=12

(Hmm. When did Anand start embedding his charts in such a sneaky fashion? As far as I can see the charts don't exist as discrete GIFs or JPEGs, but each bar is a separate object. Makes it hard to steal )

[darkblu]

We could probably have a good discussion here about how a large polygon budget should be used though. And what bottlenecks/pitfalls there are. New thread, someone?

that'd be interesting.

[Mephisto]

No, the benchmark is clearly CPU limited – mapping out the fillrate graph shows some real oddities with the Radeon 8500 because the drop off at 1600x1200 is far too much. Also the Pixel Shader don’t make much sense as so far all the testing indicates that its actually faster with PS enabled than disabled!

Sure? here with my GF3 on a Thunderbird 1 GHz (<- !) it definitly shows a fillrate limited fall down.

May be you shoud enable the vertex shading on R8500 (off by default ...).

[Reverend]

The benchmark itself isn't indicative of gameplay. I have the full retail version and the gameplay really is rather good, being a good mix of straight-out action that many FPS'ers will be familiar with and that of a "sim", a not-so-serious one at that however. That's my opinion of gameplay and as usual gameplay is more subjective than something like IQ.

That said, the IQ is, well, simple. It is nothing to shout about. The only stuff worth mentioning are the rotor water vertex effects (which aren't that realistic anyway) and highish polies. Everything else seem dated to me (look at the explosion effect of the ship when it eventually goes down during the benchmark... it's cartoonish).

The full retail version is like this demo/benchmark when it comes to performance - it is incredibly CPU and bus limited and the explosion effects gobbles up fillrate (which is the primary reason for all the lowest recorded framerates during the benchmark run). I ran a batched benchmark (FYI. I never was able to use the LAUNCH.EXE for batched benchmark so I tried without the LAUNCH.EXE and simply used c4demo.exe for batched benchmarking, which ran fine) :

===========================================

Comanche 4 Benchmark Results for 640x480x32 FSAA=0 VSync=OFF DXTC=ENABLED AUDIO=OFF SHADERS=ENABLED

Frames per second: 40.41 avg
Tris per second: 8,047,552 avg


Comanche 4 Benchmark Results for 1024x768x32 FSAA=0 VSync=OFF DXTC=ENABLED AUDIO=OFF SHADERS=ENABLED

Frames per second: 40.30 avg
Tris per second: 8,025,587 avg


Comanche 4 Benchmark Results for 1600x1200x32 FSAA=0 VSync=OFF DXTC=ENABLED AUDIO=OFF SHADERS=ENABLED

Frames per second: 37.89 avg
Tris per second: 7,546,306 avg


Comanche 4 Benchmark Results for 640x480x32 FSAA=2 VSync=OFF DXTC=ENABLED AUDIO=OFF SHADERS=ENABLED

Frames per second: 40.17 avg
Tris per second: 8,000,690 avg


Comanche 4 Benchmark Results for 1024x768x32 FSAA=2 VSync=OFF DXTC=ENABLED AUDIO=OFF SHADERS=ENABLED

Frames per second: 39.49 avg
Tris per second: 7,863,706 avg


Comanche 4 Benchmark Results for 1600x1200x32 FSAA=2 VSync=OFF DXTC=ENABLED AUDIO=OFF SHADERS=ENABLED

Frames per second: 29.03 avg
Tris per second: 5,782,285 avg


Comanche 4 Benchmark Results for 640x480x32 FSAA=4 VSync=OFF DXTC=ENABLED AUDIO=OFF SHADERS=ENABLED

Frames per second: 39.91 avg
Tris per second: 7,947,273 avg


Comanche 4 Benchmark Results for 1024x768x32 FSAA=4 VSync=OFF DXTC=ENABLED AUDIO=OFF SHADERS=ENABLED

Frames per second: 35.65 avg
Tris per second: 7,098,793 avg


Comanche 4 Benchmark Results for 1600x1200x32 FSAA=4 VSync=OFF DXTC=ENABLED AUDIO=OFF SHADERS=ENABLED

Frames per second: 16.76 avg
Tris per second: 3,338,692 avg


Comanche 4 Benchmark Results for 640x480x32 FSAA=0 VSync=OFF DXTC=ENABLED AUDIO=OFF SHADERS=DISABLED

Frames per second: 38.76 avg
Tris per second: 7,676,165 avg


Comanche 4 Benchmark Results for 1024x768x32 FSAA=0 VSync=OFF DXTC=ENABLED AUDIO=OFF SHADERS=DISABLED

Frames per second: 38.65 avg
Tris per second: 7,655,600 avg


Comanche 4 Benchmark Results for 1600x1200x32 FSAA=0 VSync=OFF DXTC=ENABLED AUDIO=OFF SHADERS=DISABLED

Frames per second: 36.89 avg
Tris per second: 7,307,617 avg


Comanche 4 Benchmark Results for 640x480x32 FSAA=2 VSync=OFF DXTC=ENABLED AUDIO=OFF SHADERS=DISABLED

Frames per second: 38.53 avg
Tris per second: 7,630,637 avg


Comanche 4 Benchmark Results for 1024x768x32 FSAA=2 VSync=OFF DXTC=ENABLED AUDIO=OFF SHADERS=DISABLED

Frames per second: 38.04 avg
Tris per second: 7,534,190 avg


Comanche 4 Benchmark Results for 1600x1200x32 FSAA=2 VSync=OFF DXTC=ENABLED AUDIO=OFF SHADERS=DISABLED

Frames per second: 29.10 avg
Tris per second: 5,764,551 avg


Comanche 4 Benchmark Results for 640x480x32 FSAA=4 VSync=OFF DXTC=ENABLED AUDIO=OFF SHADERS=DISABLED

Frames per second: 38.43 avg
Tris per second: 7,612,581 avg


Comanche 4 Benchmark Results for 1024x768x32 FSAA=4 VSync=OFF DXTC=ENABLED AUDIO=OFF SHADERS=DISABLED

Frames per second: 34.77 avg
Tris per second: 6,885,814 avg


Comanche 4 Benchmark Results for 1600x1200x32 FSAA=4 VSync=OFF DXTC=ENABLED AUDIO=OFF SHADERS=DISABLED

Frames per second: 16.93 avg
Tris per second: 3,353,467 avg

===========================================

That's on a XP2000+, 256MB PC2100, GF4 Ti4600 with official 28.32 drivers, DX8.1, WinXP. Sound was disabled via the game's specified command line for doing so and all AA was implemented via command line option rather than thru NV's display properties. That SHADERS= option is a result of using either "dx7" or "dx8" in the command line. It corresponds to dx7=SHADERS=DISABLED and dx8=SHADERS=ENABLED.

A few notes :
- Note the DX7 vs DX8 scores for all tests, AA or NoAA
- WRT the above, I don't believe the game uses any PS effects, only VS. Accordingly, if I'm correct, DX7 would mean vertex shaders using host CPU and DX8 would mean using vid card supporting VS. I also believe that the VS effects largely has to do with the rotor-on-water effects - and using the command line option to use "DX7" or "DX8" resulted in the same such effect. Note Wavey's comments on this - I think it comes down to which is faster for the VS effects used in this particular game/benchmark - the host CPU or the GPU.
- Note triangle throughput differences at diff rez

This is another game that can be indicative of the way the latest crop of games are being developed - they will become more dependant on the host CPU up until a rez like 1024x768. Stuff like physics, sound and AI will ensure CPU limitation. Look at this particular game, MOHAA, Max Payne...

[Humus]

Originally Posted by Randell

The reality is across DDR and SDR systems, 1gig Athlons to 2 gig pentiums, almost everybody is throwing out similar results.

I suppose that means it's limited by AGP bandwidth. They are probably storing all the geometry in AGP memory, and with that massive polygoncount I suppose it will easily become the bottleneck.