Digital Foundry Article Technical Discussion [2024]

[pjbliverpool]

The problem with 'stuttering', at least when presented by outlets like DF who I trust to correctly identify it, is that the frametime disruptions are far in excess of individual frames - this is why VRR can't smooth out the console framedrops, the ms disruption may only present as brief blip on the realtime framerate counter, but the actual 'feel' of the drop is far more drastic.

I play all my games on a 60hz TV and I've never had a game that has known traversal/shader stuttering be rectified by just having a 60fps cap. For the sake of covering any angle yeah it probably should have been mentioned, but aside from the camera stutter I doubt a framecap would really materially affect the actual stuttering issues of the game.

In line with what @Remij said, I think we all agree that shader and traversal stutter generally won't be solved by frame rate caps but at least some of what is being shown here seems to be more general frame time variation which definitely can be solved by a cap.

A great recent example of this for me is Days Gone which we recently picked up. Unlocked it runs at something like 80-90 fps but it's pretty uneven with some pretty big spikes that are definitely noticeable during gameplay. I locked it to 60fps with RTSS and it was absolutely perfect, every frame locked to 16.6ms and smooth as butter. Tested under the same circumstances as Suicide Squad though and the conclusion would have been similar that the PC version stutters more than the console.

I do take note of John's comments on Twitter that even with a cap the PC stutters more, but is that at matched settings or with the PC maxed out, because as noted in the video, if RT enabled (a setting the consoles lack) makes a 12900k drop into the 30's, then it will certainly make a 7800X3D drop below 60.

 

[Cyan]

It optimizes when things load more so than how the engine handles *how* those things load... still resulting in stutters when the engine has to load them. It's not fixing the root issue by itself.

Yes, hopefully the threading stuff they're working on will help solve this issue.. but how many games will release between now and whenever that may possibly happen? Sad to think about.

Epic should have been working on this stuff far earlier than they did. And the reason why I believe they weren't.. is because they were more focused on being the bridge between film production and real-time graphics, than they were about being the best optimized video game engine it could be. I hate saying that as I'm sure Epic employees would disagree and I'm obviously not privy to the realities of their plans... but that's certainly what it feels like as a gamer, watching the industry adopt this engine en mass, while games with issues inherent to the engine have carried on for multiple generations now.

These issues are what you'd expect an engine more focused on production would have, over an engine focused specifically on games and optimization.

I have a much clearer understanding now of why Hideo Kojima and his team opted to use Decima over Unreal.

glad Hideo Kojima was smart enough to avoid UE. Even traditional PC developers like CD Projekt might have trouble with that in the future if the engine doesn't fix its issues. The only solution to stuttering as I see it when it comes to an Unreal Engine game is to cap the framerate to 30fps -so stutter is part of the framerate xD - and use Lossless Scaling plus Black Frame Insertion from your display to actually make you believe the game is running at 60fps without stuttering.

 

[ILikeFeet]

glad Hideo Kojima was smart enough to avoid UE

his microsoft game is using Unreal Engine

 

[Flappy Pannus]

A great recent example of this for me is Days Gone which we recently picked up. Unlocked it runs at something like 80-90 fps but it's pretty uneven with some pretty big spikes that are definitely noticeable during gameplay. I locked it to 60fps with RTSS and it was absolutely perfect, every frame locked to 16.6ms and smooth as butter. Tested under the same circumstances as Suicide Squad though and the conclusion would have been similar that the PC version stutters more than the console.

And also an example of how it is actually possible, even for an open-world game, for UE4 to perform consistent on PC. Now, it is obvious that the nature of UE4 probably makes this more difficult than other engines no doubt, but there are likely just as many UE4 games on PC that have minimal, if any stuttering issues as ones that do. Days Gone, Dead Island 2, Sackboy (eventually), Hi Fi Rush - all excellent performers. But that consistency has to be a priority, and likely early in development.

It wouldn't entirely surprise me to see patches which reduce the shader stuttering for Suicide Squad come into play, as at least there's evidence with the precompiling step they're at least aware of the requirements. Traversal stutter though...eh.

 

[Dictator]

The problem with 'stuttering', at least when presented by outlets like DF who I trust to correctly identify it, is that the frametime disruptions are far in excess of individual frames - this is why VRR can't smooth out the console framedrops, the ms disruption may only present as brief blip on the realtime framerate counter, but the actual 'feel' of the drop is far more drastic.

I play all my games on a 60hz TV and I've never had a game that has known traversal/shader stuttering be rectified by just having a 60fps cap. For the sake of covering any angle yeah it probably should have been mentioned, but aside from the camera stutter I doubt a framecap would really materially affect the actual stuttering issues of the game.

It will indeed do nothing of note - the issue persists.

@Remij @pjbliverpool Take a look at those frame-times, specifically the ones in excess of 20 ms. All of those are still there with a 60 FPS cap. The game feels like doodoo even with one. I would say it feels like doodoo on consoles as well based on John's reporting.

The only reason we do Not have recordings of the Game at 60 FPS is because I got sick, Rich tried it Out on His 13900K and Had an awful experience at 60 FPS cap. He Talks about it in DF direct this week.

Caps do nothing for shader comp or traversal stutter - and on top of this stutters the Game also has umeven camera movement when Not at 30 or 60 or 120. So it looks even worse when Drops occur.

 

[Shifty Geezer]

The only reason we do Not have recordings of the Game at 60 FPS is because I got sick, Rich tried it Out on His 13900K and Had an awful experience at 60 FPS cap.

Stutter is bad for your health!

 

[pjbliverpool]

It will indeed do nothing of note - the issue persists.

@Remij @pjbliverpool Take a look at those frame-times, specifically the ones in excess of 20 ms. All of those are still there with a 60 FPS cap. The game feels like doodoo even with one. I would say it feels like doodoo on consoles as well based on John's reporting.

The only reason we do Not have recordings of the Game at 60 FPS is because I got sick, Rich tried it Out on His 13900K and Had an awful experience at 60 FPS cap. He Talks about it in DF direct this week.

Caps do nothing for shader comp or traversal stutter - and on top of this stutters the Game also has umeven camera movement when Not at 30 or 60 or 120. So it looks even worse when Drops occur.

Thanks for the clarification and yes, re-looking at the frame times graph I can see there are lots a drops above 16ms with RT enabled so a 60fps cap wouldn't help a whole lot there.

I also failed to notice on the first watch that John included some performance metrics with RT turned off which I suspected would be one of the main causes for stuttering based on his commentary early in the video. This was pretty interesting as turning off RT did seem to remove most of the frame times in excess of 16.6ms, at least in the footage that was shown, I guess it could be worse in other areas. The below shot was one of the worst I could find and while I agree that this would feel stuttery as per the commentary, a 60fps cap here would remove most of that stuttering while also resolving the camera stutter issue.



And then there is this stuttery scene a few seconds later that should be a perfect lock with a 60fps cap:



Obviously this won't resolve the shader or traversal stutters though which aren't shown in these screens but would presumably far exceed anything a cap can resolve.

 

[Remij]

It will indeed do nothing of note - the issue persists.

@Remij @pjbliverpool Take a look at those frame-times, specifically the ones in excess of 20 ms. All of those are still there with a 60 FPS cap. The game feels like doodoo even with one. I would say it feels like doodoo on consoles as well based on John's reporting.

The only reason we do Not have recordings of the Game at 60 FPS is because I got sick, Rich tried it Out on His 13900K and Had an awful experience at 60 FPS cap. He Talks about it in DF direct this week.

Caps do nothing for shader comp or traversal stutter - and on top of this stutters the Game also has umeven camera movement when Not at 30 or 60 or 120. So it looks even worse when Drops occur.

I figured as much wrt you getting sick. Hope you're beginning to feel a bit better and on the mend!

Yea, there are definitely still a lot of spikes in excess of 16ms so it's not going to fix anything completely, especially not any traversal/shader stutters.. but I was thinking mostly in terms of the jittery camera movement and that sort of thing. Still terrible regardless though.. and plenty of drops all over the place. Very unfortunate. I hope Rockstar put some effort into cleaning it up.. but as always I have my doubts.

@pjbliverpool Yea, Unreal Engine 4 and RT are just not a good pair.. add "open world" to that and it's a recipe for disaster unless done by a team that is really passionate about perf and optimization for PC.

 

[Andrew Lauritzen]

[snip]

I get it, games will have to hitch from time to time, and the best developers are masters at hiding it and working that limitation into the game design.. and they take very careful considering about the player experience.. but far more release without that consideration IMO, at least when it comes to PC where it's imperative that they do so.

I think you're putting far more of this on the engine than is realistic in *both* directions. There are plenty of UE games that don't have these sorts of hitches and there are plenty of non-UE games that do. It's more a function of polish, attention to detail and good design than technology at a certain point. Obviously it would be great if it were somehow impossible to make games that have hitches or fall below framerate cap or otherwise, but that sort of restriction is closer to a custom map or small mod, not a game engine.

The downside of this of course is that you should not be waiting around for some magical "fix" because people will always be able to create hitchy, bad experiences no matter what engine or technology they use. Calling attention to it so that developers take it seriously early on and throughout development is the correct thing to do, not lament or wait for some magical technological solution. Ultimately game developers are responsible for what they ship. Engines are a set of tools.

 

[Remij]

I think you're putting far more of this on the engine than is realistic in *both* directions. There are plenty of UE games that don't have these sorts of hitches and there are plenty of non-UE games that do. It's more a function of polish, attention to detail and good design than technology at a certain point. Obviously it would be great if it were somehow impossible to make games that have hitches or fall below framerate cap or otherwise, but that sort of restriction is closer to a custom map or small mod, not a game engine.

The downside of this of course is that you should not be waiting around for some magical "fix" because people will always be able to create hitchy, bad experiences no matter what engine or technology they use. Calling attention to it so that developers take it seriously early on and throughout development is the correct thing to do, not lament or wait for some magical technological solution. Ultimately game developers are responsible for what they ship. Engines are a set of tools.

I get it. I just don't know what to do or who to blame anymore. People make me feel bad if I put too much pressure on the devs/pubs, and same goes for the engine providers. I know it's ultimately the company who makes the game's fault for releasing it as it is. I do still believe that there's things the engine side could do better to ensure that less of these games fall through the cracks. I wouldn't be saying any of this stuff if there wasn't a clear pattern emerging here.

It just seems impossible to get through to whoever needs to hear it. I want preventative measures, not fixes after the fact. The majority simply aren't passionate enough about performance to speak up, and they often speak out against people who do actually speak up, which is an entirely different level of frustrating when you're trying to bring attention to issues.

I pick on Unreal Engine only because of how prolific and popular it is. It's the most important thing to focus on atm.. because there's just so many future games coming using that engine which I want to look forward to, but constantly have doubts about them. I hate seeing the same mistake repeated again and again.

 

[davis.anthony]

I think you're putting far more of this on the engine than is realistic in *both* directions. There are plenty of UE games that don't have these sorts of hitches and there are plenty of non-UE games that do.

I get what you're saying but, you're pretty much guaranteed to get shader stutter in a new UE powered game on PC.

The same however, can not be said for games that don't use UE.

That, to me, is an engine/tool problem with UE.

 

[Andrew Lauritzen]

It just seems impossible to get through to whoever needs to hear it. I want preventative measures, not fixes after the fact.

We've been through this before so I won't rehash too much, but I don't think there's any communication problem. There are resource and priority problems. The best thing you can continue to do is what you just did: refund and/or don't buy the game if it doesn't meet your standards. And DF should continue to call out issues they find in all games, regardless of the engine or toolset. There's really no need for consumers to try and hone in on the complex relationship between game and engine tech in terms of "blaming" someone... the reality is it will usually be different for each of these cases anyways even if the symptoms are the same. Call out the games, and the game devs can talk to their engine/middleware providers if they need support on fixing or avoiding issues.

I get what you're saying but, you're pretty much guaranteed to get shader stutter in a new UE powered game on PC.

The same however, can not be said for games that don't use UE.

I think you're falling into a bit of confirmation bias and searching for patterns. Several recent UE games covered by DF (to keep this thread vaguely on topic) have had no stutter. Conversely DF has covered several non-UE games that have had stutter recently too. Just go browse through the wealth of examples with DF's handy hashtag. It's clearly not just a UE problem.

There's another line of fallacious reasoning here: if many/most games of a certain scope and level of complexity are made with UE then you should expect many/most of the games that have (insert issue related to that) to be using UE as well, even if the toolset was equivalent to any other toolset in that regard. I'm not saying this 100% accounts for the perceived relationship, but it's a big part of why just picking out examples of games that have [that issue] that use UE is not particularly meaningful in and of itself.

There are places where it's easier or harder to fall into various pitfalls with certain tools. UE does allow folks to shoot themselves in the foot in a number of areas, by the very nature of giving folks access to powerful tools like the material editor. By this point the tradeoffs and best practices around that have been known for a decade though so I don't think that's a reasonable excuse any more for AAA developers. For newer stuff like Nanite and Lumen there's more leeway as both the engine and best practices are being developed, so there are going to be some rough edges. But if you're shipping a game on UE4 in 2024 with major issues, that's on you IMO.

And of course the opposite is true as well: when a game comes out and it's great, praise the game itself. In an ideal world tools help game devs realize their visions, but it's still a lot of work and expertise to make a good game even with advanced tools. Anyone can go try and make something great with the Unreal Editor if they think otherwise

 

[Scott_Arm]

I would probably say it’s not worth getting emotionally invested in games, never preorder and wait for reviews (user or media), especially if you’re the type of person that gets bothered by things shader compilation stutter.

Ultimately if you stop giving money, devs will have to respond and they’ll likely start working with their partners to correct the issues. Buying and hoping for fixes just enables whatever is going on.

In terms of UE I think it’s difficult to just look at the number 4 or 5 and make assumptions about implementation. Some developers make big changes to the engine to suit their needs and some don’t. On top of that UE will sometimes have multiple options to solve the same problem. Those solutions are all customizable with different pros and cons or limitations. I think you’d need fairly open developer interviews to really understand technical problems but usually they don’t want to publicly discuss problems unless fixes are on the way.

I think developer interviews are DFs best content and it would be nice if they could get someone to discuss problems like traversal stutter directly either for a particular game or more generally. Pretty sure they’ve had developers discuss shader compilation. Their spotlight on tech problems I think makes some difference, but I imagine it’s tough to balance criticism and still have developers wanting to talk to you. I think it would be cool if the did some Godot, UE and Unity interviews to maybe publicize best practices in some of these areas without having to mention specific games.

A round table discussion with a bunch of engine people from different companies could be cool, just discussing different approaches to common problems. Maybe some engine folks and some people that worked on games that produced good results.

May be kind of outside DFs focus on consumer content.

 

[Shifty Geezer]

Ultimately if you stop giving money, devs will have to respond and they’ll likely start working with their partners to correct the issues. Buying and hoping for fixes just enables whatever is going on.

Ultimately, that should be true. However, you'll be a tiny drop in the ocean as most people won't care (enough to go without their game) and will give their money freely. The average pundit is willing to put up with a lot of jank in the everyday. That's why you need to start a 'movement'. You need to get people to be unwilling to accept the issue and protest/not buy. Which is where leadership and the likes of DF taking point is necessary.

 

[Andrew Lauritzen]

On top of that UE will sometimes have multiple options to solve the same problem.

/me cries into >20 different shadow techniques in UE

I too really enjoy the developer interviews, even if they are more "soft ball" questions as may be required. Usually get lots of interesting insight regardless, so thanks DF and hope there are more of those!

The average pundit is willing to put up with a lot of jank in the everyday. That's why you need to start a 'movement'. You need to get people to be unwilling to accept the issue and protest/not buy. Which is where leadership and the likes of DF taking point is necessary.

Raising awareness about something that you feel important is fine (ex. pushing for the switch to looking at frame times in the first place), but it feels weird to try and make people mad about something if it legitimately doesn't bother them to start with.

At a certain point I just have to accept that most people legitimately like gigantic heavy SUVs and trucks and not small light cars anymore no matter how much I try to convince them otherwise

 

[Shifty Geezer]

Raising awareness about something that you feel important is fine (ex. pushing for the switch to looking at frame times in the first place), but it feels weird to try and make people mad about something if it legitimately doesn't bother them to start with.

I agree. But if you want change and they aren't changing, you gotta try and change them; protesting on your own won't affect anything. Or just let it go.

 

[Bludd]

DF Direct Weekly #148: State of Play - PS5's 2024 Revealed - Suicide Squad Reaction - Switch 2 LCD!

0:00:00 Introduction
0:01:00 News 01: State of Play reaction: Death Stranding 2
0:17:45 News 01: Stellar Blade, Dragon’s Dogma 2
0:28:50 News 01: Sonic X Shadow Generations, Judas
0:38:21 News 01: Metro Awakening, Until Dawn
0:50:23 News 01: Silent Hill: The Short Message, Silent Hill 2
1:01:48 News 02: Suicide Squad impressions!
1:19:14 News 03: New Switch rumoured to have 8 inch LCD
1:31:02 News 04: GT Sport patched with offline save support
1:37:49 Supporter Q1: How can Epic’s TSR beat DLSS 2?
1:42:00 Supporter Q2: Why does per-pixel motion blur look good, while display sample-and-hold blur looks bad?
1:45:50 Supporter Q3: Will frame generation make VRR obsolete?
1:49:07 Supporter Q4: What speed of hardware adoption should we expect from gamers?
1:54:27 Supporter Q5: Rich, when did you become interested in PC hardware and technology?

 

[Andrew Lauritzen]

1:45:50 Supporter Q3: Will frame generation make VRR obsolete?

This is always an interesting question but I think people forget that current frame gen is *interpolation*, and thus by its very nature it will always add latency. While the notion of generating a frame on the fly if it gets missed (vsync or otherwise) is tempting, there are a number of issues with using it in that kind of "just in time"/extrapolation context like VR does.

It's not entirely clear if you can get sufficient quality frame generation with extrapolation; the main reason it works decently for VR is because the primary motion is often your head, which is easy to predict. When there's any significant animation-based motion onscreen, space-warp tend to add significant smearing and judder, and I doubt that's something you would want to have on all the time. Interpolation is a much easier problem because you will usually have the majority of information you need in one of the two frames.

 

[GuardHei]

Will DF be covering Granblue Fantasy Relink? The in-game camera has such a weird stutter, making 60fps feels like sub-30s
Also, I'm having trouble reaching 4k60 even with lowest settings (besides textures) on a 4070ti with 13900. There's no reason this poly count gonna push the 4070ti to its limit...

 

[troyan]

FrameGeneration makes VRR more usable. On LCD it helps to stay in the right window for overdrive, it lifts performance over the VRR underlimit.