fraps Benchmarking

[Mark]

While reading Nite-hawk's post, I was suddenly reminded of Isaac Asimov's "Foundation" series...

 

[WaltC]

While reading Nite-hawk's post, I was suddenly reminded of Isaac Asimov's "Foundation" series...

Don't remind me...I just read the entire Foundation series (again) from start to finish (in the order recommended by Asimov), including the three Foundation sequels by the "three Bees" (Bear, Benford and...what's his name...)...

I swear, I think if I ever see the name "Harry Seldon" again, I'll puke...

You know, in the end Seldon reminded me of nothing so much as a far-future counterpart to yesterday's stereotype of a book-burning Nazi... "The Plan," indeed....I actually felt that in a really good story Daneel and Seldon would have ended up in opposition...which would have made Seldon a lot more likable to me in the end, as well as making for a far more interesting story. Ah well...didn't mean to steer off topic...

 

[Nathan]

I raced each of the four races twice (Rally, Hillclimb, Ice Racing, RallyCross) using a random course/track and car. I logged FRAPs framerate from the start of the race to the finish, and logged again while watching the reply of the race from the default replay camera. This gave me around 2-3 minutes per sample and 16 samples when everything was said and done (8 races, 2 samples per race). I then averaged the results once again.

Out of interest, how close were the replay frame rates to the actual race frame rates?

That was truly a joy to read. I've been lamenting the lack of statistical analysis in this industry for a long time. Thank you for the wonderful piece of work.

Thanks

I'm currently working on a project that would be helped significantly by tests like the one you just performed. The idea is that with enough samples of data from around the web a database can be compiled and analysis done to make predictions about how variations in systems will affect certain benchmark scores (Say, that across the board we have 1000 different samples of the radeon 9700 running ut2k3 botmatch on various kinds of hardware. With what degree of accuracy can we predict variations in scores versus changes in say, CPU speed? Alternatively, how do different cards perform given the same test system?). In addition, with enough data we should be able to start making predictions about how a specific configuration should perform based on how others do. I think there is a *lot* of really neat things that could be done if someone was willing to do them.

An interesting idea. I think that you could potentially have a system capable of providing a massive amount of high quality information. It always annoys me when one review shows video card A beating video card B, while another one shows B beating A. Which one do you believe? Your database could provide a definite answer.

A concern I do have, is that the amount of information available may not be sufficient to provide a decent statistical basis for the results. There is a mind boggling amount of different hardware configurations - and only so many reviews on the internet.

 

[Mark]

I raced each of the four races twice (Rally, Hillclimb, Ice Racing, RallyCross) using a random course/track and car. I logged FRAPs framerate from the start of the race to the finish, and logged again while watching the reply of the race from the default replay camera. This gave me around 2-3 minutes per sample and 16 samples when everything was said and done (8 races, 2 samples per race). I then averaged the results once again.

Out of interest, how close were the replay frame rates to the actual race frame rates?

That was truly a joy to read. I've been lamenting the lack of statistical analysis in this industry for a long time. Thank you for the wonderful piece of work.

Thanks

I'm currently working on a project that would be helped significantly by tests like the one you just performed. The idea is that with enough samples of data from around the web a database can be compiled and analysis done to make predictions about how variations in systems will affect certain benchmark scores (Say, that across the board we have 1000 different samples of the radeon 9700 running ut2k3 botmatch on various kinds of hardware. With what degree of accuracy can we predict variations in scores versus changes in say, CPU speed? Alternatively, how do different cards perform given the same test system?). In addition, with enough data we should be able to start making predictions about how a specific configuration should perform based on how others do. I think there is a *lot* of really neat things that could be done if someone was willing to do them.

An interesting idea. I think that you could potentially have a system capable of providing a massive amount of high quality information. It always annoys me when one review shows video card A beating video card B, while another one shows B beating A. Which one do you believe? Your database could provide a definite answer.

A concern I do have, is that the amount of information available may not be sufficient to provide a decent statistical basis for the results. There is a mind boggling amount of different hardware configurations - and only so many reviews on the internet.

about 20 fps less on average for the replay mode. The reason is because when I race, I use the "bumper cam" which doesn't show your own car. The replay mode does a sort of cinematic thing, which shows lots more stuff.

 

[Quitch]

Don't remind me...I just read the entire Foundation series (again) from start to finish (in the order recommended by Asimov), including the three Foundation sequels by the "three Bees" (Bear, Benford and...what's his name...

Sorry to wander, but I love this series... what is the reccomended order? And hell, I didn't know of any prequel, unless Forward the Foundation counts.

Gimme!

 

[WaltC]

Don't remind me...I just read the entire Foundation series (again) from start to finish (in the order recommended by Asimov), including the three Foundation sequels by the "three Bees" (Bear, Benford and...what's his name...

Sorry to wander, but I love this series... what is the reccomended order? And hell, I didn't know of any prequel, unless Forward the Foundation counts.

Gimme!

According to Asimov, you should read them in this order:

(1) Prelude to Foundation (1988)
(2) Foundation (1951)
(3) Foundation and Empire (1952)
(4) Second Foundation (1953)
(5) Foundation's Edge (1982)
(6) Forward the Foundation (1983)

Then of course read the three Foundation novels written after Asimov's death by the three Bees...Bear, Benford, and Brin...which complete the story and are very entertaining, I thought. Still, Harry Seldon...he tasks me...he tasks me...