How does motion blur in modern games work?

Grall

Invisible Member
Legend
I'm thinking about recent Source engine titles such as HL Ep.2, TF2 etc, Crysis/Warhead and so on. This stuff looks very much like real photographic motion blur, not at all like the fake T-buffer crap from the 3dfx voodoo5 days.

Btw, those games mentioned all seem to use the same basic technology for motion blur, with some technical implementation differences. Crysis' blur tech seems to run slower for example, especially with MSAA activated, while Valve's blur is very well optimised and doesn't impact framerates nearly as much.

One weird aspect I've noticed is the blur is very dependent on framerate. Fast framerate = virtually no blur visible even when shaking the camera wildly. Bit of a shame really. :) Like it does its magic by somehow interpolating current and previous frames.
 
The actual motion blur filter portion is just a weighted blend of n neighbor pixels leading in a straight line going in some direction. The direction in this case comes from a piece of per-pixel velocity information that means you do your blur along vector v. Of note is the fact that some people will do the motion blur when rendering the objects themselves while others may do it as a full-screen pass, producing different results and with their own respective difficulties when it comes to properly sampling for your filter.

And pretty much along the lines of what you were seeing with frame-rate dependence, the velocity vector is based on the motion for that frame. This alone isn't really unrealistic since motion blur in real film capture is a product of the exposure time and latency between frames, so when you're at a higher framerate, the time spans are shorter and therefore, there's less motion info captured. The error is not so much that but rather, the stuff that is blurred and when. A lot of games, for instance, will use the per-frame motion vectors of the object in world space for velocity -- this is fine for a static camera, but since world-space velocity vector is not necessarily equal to view-space velocity vector, you might get locally too much or too little blur, and blur won't be dependent on camera movement.
 
Interesting. Thanks for the reply!

How is this velocity information stored for frames rendered, for the case when blurring is done per-object?

I'm aware the similarity so to speak of blurring being diminished in photography with increasing framerate, but since it requires a separate operation (or sets of operations) in realtime rendered 3D, it's sort of a waste of computing resources if higher framerates makes the blur virtually undetectable... Something ought to be done about that, methinks. :)
 
The actual motion blur filter portion is just a weighted blend of n neighbor pixels leading in a straight line going in some direction. The direction in this case comes from a piece of per-pixel velocity information that means you do your blur along vector v. Of note is the fact that some people will do the motion blur when rendering the objects themselves while others may do it as a full-screen pass, producing different results and with their own respective difficulties when it comes to properly sampling for your filter.

And pretty much along the lines of what you were seeing with frame-rate dependence, the velocity vector is based on the motion for that frame. This alone isn't really unrealistic since motion blur in real film capture is a product of the exposure time and latency between frames, so when you're at a higher framerate, the time spans are shorter and therefore, there's less motion info captured. The error is not so much that but rather, the stuff that is blurred and when. A lot of games, for instance, will use the per-frame motion vectors of the object in world space for velocity -- this is fine for a static camera, but since world-space velocity vector is not necessarily equal to view-space velocity vector, you might get locally too much or too little blur, and blur won't be dependent on camera movement.

Very interesting. Are you from the industry yourself?

I always thought it's got to be vector-based and I'm glad to have this confirmed by somebody who knows what he's talking about.

I think there is one critical difference you've seen in games these days.

1) In some games, there is a rudimentary kind of motion blur, which I guess is easily applied as a post process based on per pixel vector between two frames, which simply gives an effect of blur when the camera is turned left right, up down. (for example Gears of War 1)

2) There is a very hardware intense kind of motion blur, which applies just the objects that move over a certain threshold velocity. (for example the first Crysis game. I remember I lost over 30% performance when turning it on (it was under shader quality= ultra high))
For example in a cutscene in which one character would throw a map at some other character who then catches it mid air, there would be only the map becoming blurry in mid air and maybe the other character's arm when he reaches out swiftly to catch it mid air. This is a much more cinematic way of rendering motion blur and first and foremost it realistically simulates "shutter speed" of a camera lens.

And in one of the latest games, the doom reboot of 2016, there is a kind of motion blur i haven't seen before.
It seems that if i record a scene and watch clips of it where there is a lot of movement frame by frame, it seems that the very character models become distorted based on vector calculations.
Could that be?
That for example camera turn-speed would alter the geometry of a character model? I guess this would make motion blur less heavy on the post processing side and more integrated into the first render pass. I mean if each point of each surface could be mapped, it would be possible to generate "real" motion blur, namely geometry that "morphs" based on vectors.
But I'm not entirely sure.
 
There's a short blurb about Doom's motion blur, here.

http://www.eurogamer.net/articles/digitalfoundry-2016-doom-tech-interview

Tiago Sousa: From a perspective of correctness/believability, motion blur is essentially simulating the accumulated light between an image exposure for a certain amount of time into film/digital sensor. For approximating such we need to reconstruct the pixel movement history. For real-time purposes that's usually achieved via outputting the relative velocity of a projected surface into the viewing plane, between current and previous frame, while the next frame is usually extrapolated. So, from a physically plausible view, separating object (ie dynamics) and camera (ie statics or just camera rotation) doesn't make much sense. It's programmatically possible, but would introduce noticeable artefacts and not look that nice in end.

I'm not sure they do it all that much differently than many other modern games.

Regards,
SB
 
I'm particularly curious about the sampling strategies used for bluring games in velocity-buffer based aproaches. It seems like a problem inherintly ill fitted for scatter as you gather algorithms, but you have to make it that way so it performs. The perfect implementation would tap the entire screen for every single pixel to see which ones happen to have such a velocity amount and direction that would affect it. Of course that's completely unfeasible, hence why I wonder what approximation/optimization devs typically use. I saw the one by one of the CoD devs posted in a past siggraph, i believe, but don't know much about what other devs do.
 
a properly implemented motion blur algorithm should be able to render motion blur for objects that have actually left the screen. How is that handled - if at all?
 
a properly implemented motion blur algorithm should be able to render motion blur for objects that have actually left the screen. How is that handled - if at all?
Same as all screen space methods, it's not.

Did test with UE4 in which I increased render buffer size so it would go over the edges of screen and it did look nice for all screenspace post fx. (Reflections were amazing with 200% buffer.)
Would love to try it with the low resolution tricks nvidia introduced for VR or similar tech to reduce the cost.
Even really low resolution might be enough for convincing FX.
 
Last edited:
The perfect implementation would tap the entire screen for every single pixel to see which ones happen to have such a velocity amount and direction that would affect it.

Even that isn't sufficient for a "perfect" implementation. You still wouldn't be able to handle cases where features that are occluded in your current frame, but weren't occluded for the entirety of the shutter period. As an example, take this image of real-world motion blur from a camera:

0904061649261abstract_motion_blur_qx.jpg


If you look at the horse's legs, you'll notice that you can see the green grass through them even though they're opaque. This is because the grass was visible for some portion of the time that the shutter was open, and was only occluded for a part of that time.

Now with your typical 2D motion blur that's used in games, you would have a big problem in this scenario. Every frame is basically an instantaneously sample of the scene from an infinitesimally small period of time, as if you used a camera with an insanely high shutter speed. So in the case of the horse you would get an image with no motion blur, where the legs are occluding the area of grass behind them based on where they were posed at the current simulation state. When it comes time for the 2D motion blur to do its thing, the shader will typically look at the velocity (and probably neighboring velocities) and try to gather a whole bunch of samples to simulate the partial coverage that naturally happens from having a non-instantaneous shutter period. This works okay when the entire neighborhood is moving in the same direction, or for areas where the moving elements have "spread" onto nearby static elements. In both of those cases, gathering your neighbors along the direction velocity works well enough for approximating the real motion-blurred result. However it fails for the partial occlusion case that you have on the legs, since your 2D image simply doesn't enough information to tell you what as in the occluded area. You also have a similar situation when a non-moving foreground element (such as the right horse's head) occluding a moving background element: you need to know the contents of the occluded background so that you can integrate it with the visible neighboring areas.

In general, games will try to deal with this by re-using neighboring pixels to "make up" the contents of the partially occluded elements. So in the case of the horse legs, it might actually work okay since the grass is pretty uniform in color, and so you probably wouldn't notice that the post-process sampled the neighboring grass color instead of sampling the occluded grass. But if the scene is more complex the error may be more jarring. It's also worth pointing out that defocus blur (depth of field) has almost the exact same problem, except the partial occlusion happens due to the lens aperture instead of a shutter.

The other problem you can run into with 2D motion blur is errors from complex motion. Most games store a per-pixel velocity that's computed by taking the different between the pixel's current screen position and it's screen position from the previous frame. This essentially assumes that the surface being sampled for that pixel moved in a straight line between the previous frame in the current frame. In some cases this might be true, in other cases it's definitely not. The classic example is a spinning wheel: a typical motion blur algorithm will end blurring along a straight line based on a pixel's instantaneous velocity, but in real-life the motion blur will follow the curve of the wheel.
 
Well there's the old Accumulation Buffer technique, which shouldn't suffer from that problem.
 
Even that isn't sufficient for a "perfect" implementation. You still wouldn't be able to handle cases where features that are occluded in your current frame, but weren't occluded for the entirety of the shutter period. As an example, take this image of real-world motion blur from a camera:

If you look at the horse's legs, you'll notice that you can see the green grass through them even though they're opaque. This is because the grass was visible for some portion of the time that the shutter was open, and was only occluded for a part of that time.

Now with your typical 2D motion blur that's used in games, you would have a big problem in this scenario. Every frame is basically an instantaneously sample of the scene from an infinitesimally small period of time, as if you used a camera with an insanely high shutter speed. So in the case of the horse you would get an image with no motion blur, where the legs are occluding the area of grass behind them based on where they were posed at the current simulation state. When it comes time for the 2D motion blur to do its thing, the shader will typically look at the velocity (and probably neighboring velocities) and try to gather a whole bunch of samples to simulate the partial coverage that naturally happens from having a non-instantaneous shutter period. This works okay when the entire neighborhood is moving in the same direction, or for areas where the moving elements have "spread" onto nearby static elements. In both of those cases, gathering your neighbors along the direction velocity works well enough for approximating the real motion-blurred result. However it fails for the partial occlusion case that you have on the legs, since your 2D image simply doesn't enough information to tell you what as in the occluded area. You also have a similar situation when a non-moving foreground element (such as the right horse's head) occluding a moving background element: you need to know the contents of the occluded background so that you can integrate it with the visible neighboring areas.

In general, games will try to deal with this by re-using neighboring pixels to "make up" the contents of the partially occluded elements. So in the case of the horse legs, it might actually work okay since the grass is pretty uniform in color, and so you probably wouldn't notice that the post-process sampled the neighboring grass color instead of sampling the occluded grass. But if the scene is more complex the error may be more jarring. It's also worth pointing out that defocus blur (depth of field) has almost the exact same problem, except the partial occlusion happens due to the lens aperture instead of a shutter.

The other problem you can run into with 2D motion blur is errors from complex motion. Most games store a per-pixel velocity that's computed by taking the different between the pixel's current screen position and it's screen position from the previous frame. This essentially assumes that the surface being sampled for that pixel moved in a straight line between the previous frame in the current frame. In some cases this might be true, in other cases it's definitely not. The classic example is a spinning wheel: a typical motion blur algorithm will end blurring along a straight line based on a pixel's instantaneous velocity, but in real-life the motion blur will follow the curve of the wheel.
You can clearly see these kinds of errors in games even if you're not necessarily looking for them. I think the problem with wheels is easily solved (just have different textures which are increasingly (properly) blurred for the wheels depending on the speed of the car lol). There are probably lots of similar tricks devs use in racing games for this.
 
(just have different textures which are increasingly (properly) blurred for the wheels
Well, we're kinda past the stage where a car wheel is simply a textured cylinder... :p So I don't know how effective that really would be.
 
Well, we're kinda past the stage where a car wheel is simply a textured cylinder... :p So I don't know how effective that really would be.
Yea I'm just pointing out that there are ways around that specific limitation. My suggestion probably isn't the best :)
 
Even that isn't sufficient for a "perfect" implementation. You still wouldn't be able to handle cases where features that are occluded in your current frame, but weren't occluded for the entirety of the shutter period. As an example, take this image of real-world motion blur from a camera:

0904061649261abstract_motion_blur_qx.jpg


If you look at the horse's legs, you'll notice that you can see the green grass through them even though they're opaque. This is because the grass was visible for some portion of the time that the shutter was open, and was only occluded for a part of that time.

Now with your typical 2D motion blur that's used in games, you would have a big problem in this scenario. Every frame is basically an instantaneously sample of the scene from an infinitesimally small period of time, as if you used a camera with an insanely high shutter speed. So in the case of the horse you would get an image with no motion blur, where the legs are occluding the area of grass behind them based on where they were posed at the current simulation state. When it comes time for the 2D motion blur to do its thing, the shader will typically look at the velocity (and probably neighboring velocities) and try to gather a whole bunch of samples to simulate the partial coverage that naturally happens from having a non-instantaneous shutter period. This works okay when the entire neighborhood is moving in the same direction, or for areas where the moving elements have "spread" onto nearby static elements. In both of those cases, gathering your neighbors along the direction velocity works well enough for approximating the real motion-blurred result. However it fails for the partial occlusion case that you have on the legs, since your 2D image simply doesn't enough information to tell you what as in the occluded area. You also have a similar situation when a non-moving foreground element (such as the right horse's head) occluding a moving background element: you need to know the contents of the occluded background so that you can integrate it with the visible neighboring areas.

In general, games will try to deal with this by re-using neighboring pixels to "make up" the contents of the partially occluded elements. So in the case of the horse legs, it might actually work okay since the grass is pretty uniform in color, and so you probably wouldn't notice that the post-process sampled the neighboring grass color instead of sampling the occluded grass. But if the scene is more complex the error may be more jarring. It's also worth pointing out that defocus blur (depth of field) has almost the exact same problem, except the partial occlusion happens due to the lens aperture instead of a shutter.

The other problem you can run into with 2D motion blur is errors from complex motion. Most games store a per-pixel velocity that's computed by taking the different between the pixel's current screen position and it's screen position from the previous frame. This essentially assumes that the surface being sampled for that pixel moved in a straight line between the previous frame in the current frame. In some cases this might be true, in other cases it's definitely not. The classic example is a spinning wheel: a typical motion blur algorithm will end blurring along a straight line based on a pixel's instantaneous velocity, but in real-life the motion blur will follow the curve of the wheel.
Yeah, I know about all the disoclusion issues, and they are similar to the problem related to objects outside the frustrum. Its the nature of ss effects.
Maybe perfect was used too loosely. I meant, within the scope of SSMB, with all artifacts and limitations that come with it, what are further optimisations/acxeleration techniches and simplifications comonly used by devs to decide where to tap samples from.
 
Well there's the old Accumulation Buffer technique, which shouldn't suffer from that problem.

Sure, as long as you're willing to render your scene N times per frame. :D

You can clearly see these kinds of errors in games even if you're not necessarily looking for them. I think the problem with wheels is easily solved (just have different textures which are increasingly (properly) blurred for the wheels depending on the speed of the car lol). There are probably lots of similar tricks devs use in racing games for this.

Well, I don't know if I would call that "solved". Sure it may look better to use a pre-blurred wheel instead of relying on screen-space motion blur, but you get some of the standard issues associated with pre-integration/pre-filtering. For instance, to get the same specular response you would need to modify the specular NDF to represent the distribution that you get from moving the wheel geometry + normal maps across the pixel. Or if you had a shadow fall across the wheel, if you pre-integrate the blur then the shadow will maintain its hard edge instead of being filtered in the motion direction.
 
Or if you had a shadow fall across the wheel, if you pre-integrate the blur then the shadow will maintain its hard edge instead of being filtered in the motion direction.

Actually, that would be the correct.
 
Deep buffers could solve some of the problems with occlusion etc.
To get 'perfect' motion blur there are some harder problems though.

All lights, shadows and surface sampling should be motion blurred.

Moving light might be possible to fake using area/volume lights.
Line light for point lights, not sure how one could emulate intensity changes within time shutter is open.

I have no idea how to do proper shadowing without raytracing.

Reflections/speculars might be somewhat fake able with blurring of cubemaps, no idea what to do with SSR.

Surface changes during the open shutter like in case of water, perhaps pre-filter normals in temporal as well as mipmap etc.

The simplest way is to use continuous ray-tracing for everything, but it certainly would have it's own problems in terms of performance. ;)
 
Last edited:
Back
Top