Measuring framerate on shipped titles

[assen]

So, after the method for measuring framebuffer resolution has already been more or less proven to work, we as a anal-retentive, magifier-glass-wielding technofreak community need a way to measure the _real_ framerate on release titles. Do you have any ideas? I can't think of anything except for high-speed cameras... I figure to measure reliably framerates in the 30-60 range you'd need at least 150-200 fps. Is there anything in the consumer space that can do this sort of thing?

What got me wondering was a little birdie who told me that an upcoming high-profile multiplatform title will be running at a significantly better framerate on one of the platforms, although none of the versions will be locked at either 30 or 60 fps, and it won't be advertised because of, ahem, a gentle warning given by the vendor of the framerate-challenged platform.

Since the gaming press has already written tons of bullshit about previous versions of the same game, I don't think we can trust them to find and expose the difference - if they receive marketing materials saying "the visuals are exactly the same on both platforms", they will parrot it, and add some "fluid" and "smooth"-s on their own.

 

[Shifty Geezer]

B3D should set up its own labs. We should be dissecting our own 40GB PS3's rather than waiting on some other site to do it, and doing our own grizzly game analysis that sends developers running for the hills as we unearth the secrets of performance they always got away with before (who counted pixels on PS2 and identified upscaling games there?!).

 

[Spy Hunter]

latency

It might be hard to measure framerates with a high speed camera because I imagine they don't record for long periods of time and framerate can vary so much over time.

I was recently thinking that it would be cool to have one of those high speed cameras, not to measure frame rates but to measure *latency*. Right now manufacturers often throw around response time numbers (mostly for LCDs) and there isn't a whole lot of independent verification of those numbers. I think one of the hardware review sites should invest in a high-speed camera and start benchmarking the latency of all kinds of devices. I would love to see USB vs PS/2 mice, optical vs ball mice, wireless vs wired peripherals, LCD vs CRT monitors, various LCDs vs their published specs, DVI/HDMI vs VGA/Component, Wiimote pointer/accelerometer latency, etc etc.

 

[kyleb]

You just need a capture card to grab a video and count unique frames.

 

[Bohdy]

You just need a capture card to grab a video and count unique frames.

Yep. There is an avisynth filter that will remove all duplicate frames, leaving the real average framerate over a video sequence.

It's more tricky with 60i video, however, as it must be deinterlaced first then the frames must be checked by some similarity threshhold which won't be exact.

 

[ERP]

I've used a VHS recorder before (used to find them exremely useful for debugging), but anything that captures the signal and will advance one field will work.

 

[Spy Hunter]

Video capture cards could easily measure the framerate of SD video, but are there any consumer-level HD capture cards out there? How would you measure the framerate of 1080p over component or HDMI?

 

[kyleb]

SD capture is good enough for many 360 games, as the render the same regardless of if you are running 1080p over HDMI or 480i widescreen though composite. Though, yeah, at higher resolutions for other 360 games and likely all PS3 games, the capture card needed doesn't come cheap.