Digital Foundry Article Technical Discussion Archive [2010]

[Shifty Geezer]

I remember some people with 120hz TV sets claiming a few months back about how their 30FPS tittles "feel" as if its running at 60FPS on their TV. I donno if its true but if its indeed the case then I assume its nothing more than duplicating the frames ?

Various motion smoothing techs interpolate frames, so it's not just a duplication of the same frame which would look the same as 30hz. In a TV it'll be introducing lag as part of the processing step, as it needs two frames to tween between them.

 

[Arwin]

I remember some people with 120hz TV sets claiming a few months back about how their 30FPS tittles "feel" as if its running at 60FPS on their TV. I donno if its true but if its indeed the case then I assume its nothing more than duplicating the frames ? And doesn't this obviously means an increased perception to input lag experienced by the observer/user, since the game will still respond like a 30FPS title. ?

From the linked article:

Andreev first got the idea for the technique by studying 120Hz TVs that interpolate two frames in order to produce an intermediate image, producing a smoother picture. Software filters on some media players (for example Philips' Trimension as seen on the WinDVD player) were also considered. If this approach could be replicated within the game engine, an effect far more pleasing than most motion blur algorithms could be produced. Discussions after SIGGRAPH 2008 soon led to prototyping.

 

[Laa-Yosh]

It is very interesting... But I'm not sure if what we need is another step down in quality. It really needs more analysis to see if it really looks better than 30fps images, I'm not sure that the interpolated frames turn out to be good enough. Yeah, if you stand still there's no loss and if you move it's easier to get away with artifacts - but will it be too much?

Sure, if the PS5+ generation does indeed move on to raytracing, it'd probably be the only way to reach 60fps, but I've never been a believer of such interpolation techniques. I don't yet have a TV that does this but I'm already convinced that I'd turn it off for everything and watch movies at 24fps instead.
I'd still prefer to have it as an optional feature.

 

[patsu]

Finished reading the slides. It's pretty interesting. So does the game look good compared to other 30fps and 60fps titles ? The game will have similar lag as a 30fps game since the tech does not address/affect input right ?

 

[Xenus]

Yes it doesn't actually change the lag it just gives you more frames to show what you are doing but not really changing the time between the motion and what you are doing. However unlike other methods it's not adding any lag to equation either.

 

[Alucardx23]

Yes it doesn't actually change the lag it just gives you more frames to show what you are doing but not really changing the time between the motion and what you are doing. However unlike other methods it's not adding any lag to equation either.

I think that qualifies as reducing lag since you are seeing a reaction 16.6ms sooner on screen when you press a button.

 

[Xenus]

Not really though. Say you are running in one direction if at the rending of one frame. The interpolated frame form that point will still show you running in the same direction even if afterward you switch before the interpolated frame is shown. So I think in fact it could have the opposite effect at times as wanted. Even though the actual input lag is the same the perceived lag could be greater to the end user. That is if I understand it right and all the movement vectors are based on the previos frame and not some 60fps sample rate.

 

[Alucardx23]

Not really though. Say you are running in one direction if at the rending of one frame. The interpolated frame form that point will still show you running in the same direction even if afterward you switch before the interpolated frame is shown. So I think in fact it could have the opposite effect at times as wanted. Even though the actual input lag is the same the perceived lag could be greater to the end user. That is if I understand it right and all the movement vectors are based on the previos frame and not some 60fps sample rate.

What I understand from the article is that information is taken from the next frame that it still being rendered, it’s not about predicting where the player or anything else is going to move since that information is already available on the next real frame, what is happening is that a new frame is inserted in between, so I don’t think the example you gave can happen.

You'd think that this technique would cause lag, but as the interpolated image is being generated using elements from the next "real" frame, it actually reduces latency. Andreev's technique is single-frame based rather than dual-frame. The latter approach would require buffering two images so has a big memory and latency overhead, while the technique Andreev used effectively interpolates on the fly using past and future rendering elements.

"The most simple and efficient solution is to do the interpolation in-place, during the current frame, while the previous one is on screen. This way the previous front buffer can be mapped as a texture (on Xbox 360) and used for interpolation directly," he explains.

"In terms of latency there is something interesting going on. I said that there is no extra latency, which is true. But if you think about it, latency is actually reduced because we get the new visual result 16.6 ms earlier. You see the result of your actions earlier."

 

[Shifty Geezer]

I think that qualifies as reducing lag since you are seeing a reaction 16.6ms sooner on screen when you press a button.

Animated characters are update at 30fps, so there's no change there.

 

[Billy Idol]

Would it make sense to interpolate even higher?
Let's say up to 120Hz?

 

[Xenus]

Except that explanation makes no sense unless those elements are sampled at a higher rate then the rest of the frames. I think we'll have to wait for a more detailed description to be certain as that is dumbed down o the point it could be taken to mean many different things.

Only issue with that is by the looks of it as with anything the grater the interpolation the more artifacts as shown by the immediately noticeable ones from 15 to 60 fps unless there is a lower limit on the interpolation and that 15 was just too low.

 

[Alucardx23]

Animated characters are update at 30fps, so there's no change there.

Yeah but how about just pushing a button to open a door or turning in car game, wouldn’t you start seeing the reaction sooner?

 

[Alucardx23]

Except that explanation makes no sense unless those elements are sampled at a higher rate then the rest of the frames. I think we'll have to wait for a more detailed description to be certain as that is dumbed down o the point it could be taken to mean many different things.

Here you can see it explained in a lot more detail.
http://and.intercon.ru/rtfrucvg_html_slides/

"The most simple and efficient solution is to do the interpolation in-place, during the current frame, while the previous one is on screen. This way the previous front buffer can be mapped as a texture (X360) and used for interpolation directly. In terms of latency there is something interesting going on. I said that there is no extra latency, which is true. But if you think about it, latency is actually reduced. Because we get the new visual result 16.6 ms earlier. You see the result of your actions earlier."

 

[Shifty Geezer]

Yeah but how about just pushing a button to open a door or turning in car game, wouldn’t you start seeing the reaction sooner?

No, because the character animation that shows you he's opening a car door will start at the same 1/30th second interval as whether interpolated to 60 hz or not. The actual user input responsiveness is down to game engine which can run at 60 fps independent of refresh rate, but that won't be changing what's drawn on screen any sooner.

 

[Alucardx23]

No, because the character animation that shows you he's opening a car door will start at the same 1/30th second interval as whether interpolated to 60 hz or not. The actual user input responsiveness is down to game engine which can run at 60 fps independent of refresh rate, but that won't be changing what's drawn on screen any sooner.

I was referring to this type of door opening :smile:
http://www.youtube.com/watch?v=6R0w3T7ovcs

I understand what you say about 30 frames animations but I was thinking it doesn’t really matter, if you where to use Ben Heck controller board on Halo 3 with the same frame interpolation upscaling it should register that the jump animation starts 16.6ms earlier than normal 30fps gameplay.
http://www.eurogamer.net/articles/digitalfoundry-lag-factor-article?page=2

 

[Shifty Geezer]

Well, I suppose so, as the scenery would move during the interrim frame.

 

[patsu]

What I understand from the article is that information is taken from the next frame that it still being rendered, it’s not about predicting where the player or anything else is going to move since that information is already available on the next real frame, what is happening is that a new frame is inserted in between, so I don’t think the example you gave can happen.

It looks like...

That quote applies to the single frame implementation (slide 23). In this case, dynamic geometries (e.g. character animation) will need to be rendered separately or re-rendered at 60fps. In other words, the relevant part of the rendering system is running like a 60fps game. As a result, the frame can be presented 16.7ms earlier (say... reduced lag from 100-133ms, but still not 66-99ms like 60fps games). OTOH, the input responsiveness depends on whether the developer samples the input at higher frequency (separate system not covered in discussion).

For the two frame implementation (slide 32), they need both frames done already (like 120Hz TV interpolation). In this case, there will be additional lag. Whether you can perceive the additional lag depends on individual's threshold.

 

[Alucardx23]

It looks like...

That quote applies to the single frame implementation (slide 23). In this case, dynamic geometries (e.g. character animation) will need to be rendered separately or re-rendered at 60fps. In other words, part of the rendering system is running like a 60fps game. As a result, the frame can be presented 16.7ms earlier (say... reduced lag from 100-133ms, but still not 66-99ms like 60fps games). OTOH, the input responsiveness depends on whether the developer samples the input at higher frequency (separate system not covered in discussion).

For the two frame implementation (slide 32), they need both frames done already (like 120Hz TV interpolation). In this case, there will be additional lag. Whether you can perceive the additional lag depends on individual's threshold.

The Force Unleashed II tech demo was using the single frame solution and it was regarded as free since it required less time that the motion blur it was replacing 2.2ms VS 1.4ms on 360, I don’t think that would be the case if they had to render parts of the game at 60 frames. I could be wrong but the way I see it is that they use some kind of forward motion blur, but instead of rendering a trace just create one extra frame and put it in between the real frames, this gives the impression of a more fluid image since is actually giving more information about the trajectory of an object.

At least that’s what I understand from slide 19. :smile:

The idea is very very simple. We do frame rate up-conversion
Render the velocity buffer as we would do for motion blur. Build the middle frame. And present it at the moment in time it was intended for.

Note that in case of 30 to 60 fps conversion, the inner frame has to be presented at the middle of the frame. This is all it is, no more, no less. The rest is implementation itself, which is rather tricky.

Andreev's technique is single-frame based rather than dual-frame. The latter approach would require buffering two images so has a big memory and latency overhead, while the technique Andreev used effectively interpolates on the fly using past and future rendering elements."

 

[patsu]

I was asking in the general case. If it's covered up by another more time consuming task, then it's possible that there is no saving in lag time -- which was my original question.

The technology helps to generate smoother pictures (akin to a 60fps game), but may or may not help to make it play like a 60fps title.

May be like Fafalada said in the other thread, parts of it can be tailored to render a slightly different perspective for stereoscopic 3D. That's useful as well.

 

[Alucardx23]

I was asking in the general case. If it's covered up by another more time consuming task, then it's possible that there is no saving in lag time -- which was my original question.

The technology helps to generate smoother pictures (akin to a 60fps game), but may or may not help to make it play like a 60fps title.

May be like Fafalada said in the other thread, parts of it can be tailored to render a slightly different perspective for stereoscopic 3D. That's useful as well.

As long as they don’t use the two frame implementation I don’t see why there would be additional lag, there are no additional buffers and the extra frame is created before the next real one.