The Framerate Analysis Thread part 2

[grandmaster]

Rules of Engagement : Read before posting or run the risk of losing posting rights in the Tech Forum!

This is a technical analysis thread. It is allied to the IQ and Framebuffer Analysis threads and shares the same rules as those which you should familiarise yourself with. It is not the place to discuss developers, port quality, or otherwise engage in pointless platform favouritism. Neither is it a place to discuss the findings of any analyses. Do not follow up 'game x is running at 29.94 fps average on Console A and 27.43 fps average on Console B' with 'this goes to show Console A is better' or 'the developers clearly didn't bother with Console B'. The purpose of this thread is to collate factual information regarding framerates and tearing, and where appropriate discuss techniques/methodologies and highlight any technical considerations. Any discussion on the results should be held in relevant threads referencing this thread as needed.

Copied over are a selection of posts from the original Framerate Analysis Thread. It includes interesting, relevant debate on a technique employed by PS3 and a couple of framerate analyses. If you are not posting to this standard, your posts will be removed, and persistent unwanted contributions will see you locked out of the Technology Forum.

The previous thread can be revisited here. Major kudos are bestowed on grandmaster for performing his excellent service!

_____

(This post was a reply to talk of Mirror's Edge framerate fluctuations)

I can see where GarretASUS is coming from in that the section where Faith leaps off the end of the big red thing (around 45 seconds in the video) is one of the sections where 360 is tearing and therefore dropping frames. Most of the time - 98.4% of the time - it is v-locked and 30fps.

PS3 on the other hand doesn't appear to be v-locked at all, which boosts frame rate, but introduces more screen tear. Case in point - on the flyby section at 58 seconds in, PS3 is rendering at 26fps vs 360 at 30fps. For the most part, however, the tearing is confined to the very top pixels of the screen.

It seems that DICE's optimisations favour each port in different areas. PS3 not being v-locked gives it a .3fps advantage over the course of the entire 10,875 frames - 29.6fps vs 29.9fps.

 

[T.B.]

PS3 on the other hand doesn't appear to be v-locked at all, which boosts frame rate, but introduces more screen tear. Case in point - on the flyby section at 58 seconds in, PS3 is rendering at 26fps vs 360 at 30fps. For the most part, however, the tearing is confined to the very top pixels of the screen.

One way to implement the 30fps-or-less kind of vsync on the PS3 will cause tearing in the first few lines, independent on the actual engine-side frame rate. So I would guess the engine is actually as "locked" as the 360 is, they just have this artifact. You can basically expect the same effect on the PS3 version of Sacred2, as I do the same thing. So unless you can find instances where is runs with more than 30fps, I would assume locking. Yes, this little trick makes measuring frame-tear pretty hard to do. Maybe you should just discard the upper few lines...

 

[Mintmaster]

It seems that DICE's optimisations favour each port in different areas. PS3 not being v-locked gives it a .3fps advantage over the course of the entire 10,875 frames - 29.6fps vs 29.9fps.

So I assume that the 360 being conditionally V-locked means that every frame in the 60p capture stream has at least one duplicate (as defined by your scanline comparison). A few times you saw the same scanline 3 times, hence bringing the average down to 29.6.

For the PS3, however, you should have some unique frames, right? If you can count those, then we can get an idea of what the average framerate would be if it was also V-locked.

 

[grandmaster]

So I assume that the 360 being conditionally V-locked means that every frame in the 60p capture stream has at least one duplicate (as defined by your scanline comparison). A few times you saw the same scanline 3 times, hence bringing the average down to 29.6.

Yup

For the PS3, however, you should have some unique frames, right? If you can count those, then we can get an idea of what the average framerate would be if it was also V-locked.

The amount of unique frames in the PS3 game is very little owing to a tear at the top of the screen. Sometimes it is a single scanline, but it puts average frame rate at around 59fps, so the amount of unique frames is virtually non-existent.

 

[catisfit]

One way to implement the 30fps-or-less kind of vsync on the PS3 will cause tearing in the first few lines, independent on the actual engine-side frame rate. So I would guess the engine is actually as "locked" as the 360 is, they just have this artifact. You can basically expect the same effect on the PS3 version of Sacred2, as I do the same thing. So unless you can find instances where is runs with more than 30fps, I would assume locking. Yes, this little trick makes measuring frame-tear pretty hard to do. Maybe you should just discard the upper few lines...

That would tally with grandmasters suggestion of a lack of unique frames in the PS3 recording.

 

[grandmaster]

I discarded the top 16 lines in my final analysis to be on the safe side, so the actual results are sound - 3.8% frames torn. The same result occurs if I drop the first 64 too. Mirror's Edge does indeed tear pretty badly in certain sections, and in that case a unique frame count does indeed go beyond 30fps.

 

[Mintmaster]

The amount of unique frames in the PS3 game is very little owing to a tear at the top of the screen. Sometimes it is a single scanline, but it puts average frame rate at around 59fps, so the amount of unique frames is virtually non-existent.

In that case I would have to concur with T.B. and say that the PS3 version is also conditionally locked at 30fps, just like the 360 version. If there are very few unique frames then the PS3 isn't gaining any advantage with its v-lock settings.

A tear consistently at the top of the screen actually suggests a 30fps framerate. The hardware just went slightly over 33ms for a few frames to start the tear but in later frames couldn't render faster than 33ms to quickly remove it.

 

[FirewalkR]

One way to implement the 30fps-or-less kind of vsync on the PS3 will cause tearing in the first few lines, independent on the actual engine-side frame rate. So I would guess the engine is actually as "locked" as the 360 is, they just have this artifact. You can basically expect the same effect on the PS3 version of Sacred2, as I do the same thing. So unless you can find instances where is runs with more than 30fps, I would assume locking. Yes, this little trick makes measuring frame-tear pretty hard to do. Maybe you should just discard the upper few lines...

That's interesting, I recall Grandmaster talk about some other game where the same thing happened... Wipeout HD perhaps?

Can you expand on how this technique works or is it under NDA?

 

[grandmaster]

Yup, WipEout HD does it in 720p mode, but the screen-tear goes walkabout all over the place in 1080p mode, even with the adaptive frame buffer.

 

[T.B.]

Can you expand on how this technique works or is it under NDA?

Basically you defer the flip until at least two scanouts have been performed, then you perform an unsync'd flip. Timing issues make you sometimes miss the vsync-interval, hence the tearing.

Now *how* you do that is something you have to get out of someone less NDA-shy.

 

[grandmaster]

Cables don't affect the image being rendered in a console's framebuffer. There are many reasons why this might be happening:

* The engine on one console may be more optimised than it is on the other - the games may simply be different
* The video may be coming from earlier/later builds on each platform
* The video output of the console is 59.94fps. Internet video is 29.97fps or 30fps. Every other frame is being thrown away - it may well be that some torn frames are being thrown away
* Conversely, it may well be that the torn frames aren't being thrown away, meaning that they stay onscreen twice as long as they would do otherwise.

The second two points are why I always upload 720p60 videos on my blog if there's any hint of v-lock issues. If there's screen tear in any given game, you won't be seeing it properly unless the full output of the console is being displayed. I'm sure there are other reasons too.

 

[grandmaster]

I know it's an 'old' game but about Devil may cry 4 I suspect that the frame rate on 360 is 30 fps and not 60. Anyone can verify that? The doubt is born when I have read Lost planet play sometimes 60 fps on ps3 vs 30 fps of 360...or is it false? I have notice too on 360 the v-sync sometimes is disabled and some texture are worse (the part on the snow mountains, the reflex is missed and the bump it seems worsen to my eyes).

Lost Planet PS3 has far more frame drops than the 360 version - it's a very poor conversion. Devil May Cry is also definitely 60fps on both platforms.

 

[grandmaster]

Far Cry 2 is in the next Eurogamer face-off, so I'll have an opportunity to measure that one. Gears 2 I'm not so sure.

I'm working 14 hour days at the moment updating my blog and doing Eurogamer work so apologies if I can't meet all the demands here.

Forgive me but in a technical forum I think every graphic feature is interesting...or not? Supposing the frame rate dips to 60 on 30 fps on ps3: it isn't the case to revaluate the consideration of terrible framerate?

Not really because it dips lower than 30fps... Lost Planet is a famously sub-optimal conversion from the 360 original.

 

[grandmaster]

For a refresher on frame rate analysis, I've written a new blog entry:

http://www.digitalfoundry.org/blog/?p=167

It includes a Gears of War 2 analysis - one clip only - showing gameplay mostly at 30fps (with fluctuations, not v-locked), cut-scene at 20fps and an overall 21.7% torn frames from the 60Hz output. Most of the single-player campaign is unlocked and I can find some time to do more analysis if people ask for specific points within the game.

 

[Betanumerical]

For a refresher on frame rate analysis, I've written a new blog entry:

http://www.digitalfoundry.org/blog/?p=167

It includes a Gears of War 2 analysis - one clip only - showing gameplay mostly at 30fps (with fluctuations, not v-locked), cut-scene at 20fps and an overall 21.7% torn frames from the 60Hz output. Most of the single-player campaign is unlocked and I can find some time to do more analysis if people ask for specific points within the game.

I've heard the higher levels of Horde mode are rather stressing on the game and would be interested to see the framerate.

 

[RobertR1]

I've heard the higher levels of Horde mode are rather stressing on the game and would be interested to see the framerate.

Framerate on Horde seem to be map sensitive for whatever reason. Play Jacinto and camp by the helipad and you'll see the frame rate drop like crazy. Play security/blooddrive/avalanche and its' not an issue.

 

[grandmaster]

I've compiled a frame rate analysis of Call of Duty 4 in preparation for the work I'm doing on CoD: WaW. It's analysis the way I wish I had time to do it on this thread: complementary vids, indications where the frames are actually being dropped etc.

http://www.digitalfoundry.org/blog/?p=179

 

[assen]

Framerate on Horde seem to be map sensitive for whatever reason.

How 'bout "because the maps are different"?

Different in number of draw calls, draw distance, ballance of near/far geometry, occlusion opportunities, particle systems, collision environment, how busy the background is, computational needs of AI of the enemies (e.g. are there lots of obstacles or cover points), ...

The only thing that is probably virtually the same between maps is the texture budgets, which are somehow enforced by the engine; everything else hasn't been equalized across maps so aggressively.

 

[chambers]

Thanks for all your hard work grandmaster. Megaupload has pulled the first set of videos for you post on COD4. I can't say that I didn't expect the COD4 analysis to turn out the way it did, but the difference is almost impossible to notice without the equipment you use. I hope Infinity Ward keeps up the good work in the future.

 

[AlNom]

Very nice work GM. Looking forward to the WaW analysis. As I didn't get time to grab the videos, I was wondering if the framerate drops on PS3 were during scenes of intense alpha blending/smoke.