Digital Foundry Article Technical Discussion [2024]

[Remij]

0:38:07 News 03: Lords of the Fallen receives performance boost​

I've got to echo Alex's point again that he brings up during the Lords of the Fallen section.

You get these huge repeatable traversal stutters.. and you just have to wonder how these things don't get profiled and fixed. This isn't just finding that this stuff is a problem right at the end of development... surely they know this is an issue long before that due to how they load/stream in assets.. so why not design the game around it? Having gameplay critical things happen which can cross from zone to zone where these stutters happen, should either be optimized so that it doesn't happen, or the design of the area/level changed so that they don't happen in critical moments.

I've also noticed that in many games that Xbox and PC tend to get hit a bit harder in comparison to PS5 when it comes to this stuff. I just don't get why the focus is so much on adding more demanding things to engines/APIs, before clearly fixing these fundamental issues.

It's 2024... we should be able to fly through a game world and have it seamlessly stream in without stuttering. For crying out loud devs were doing better in the past with CDROM based streaming off the disc. The priorities are all wrong. Get the fundamentals right and build up off that without compromising it.

 

[DavidGraham]

It's 2024... we should be able to fly through a game world and have it seamlessly stream in without stuttering

I was playing the Callisto Protocol the other day on my 2080Ti, and the game was so full of traversal stutters it was down right ridiculous.

Dead Space Remake is full of that too, and so is Jedi Survivor, Resident Evil 4 Remake and many recent games. It is so widespread now that it is down right depressing.

We are are barely seeing moderate progress in solving the compilation stutters, which is why we can't really afford the emergence of another type of unsolvable stutters, enough is enough.

 

[function]

Fallout 4 quality mode broken on Xbox Series X, detail pared back from Xbox One X. Quality mode works on PS5.

Another MS game where the PS5 game is more polished, but this time it's a juggernaut of a series that's just had a smash TV show.

MS are like the anti-Nintendo.

 

[Seanspeed]

I've got to echo Alex's point again that he brings up during the Lords of the Fallen section.

You get these huge repeatable traversal stutters.. and you just have to wonder how these things don't get profiled and fixed. This isn't just finding that this stuff is a problem right at the end of development... surely they know this is an issue long before that due to how they load/stream in assets.. so why not design the game around it? Having gameplay critical things happen which can cross from zone to zone where these stutters happen, should either be optimized so that it doesn't happen, or the design of the area/level changed so that they don't happen in critical moments.

I've also noticed that in many games that Xbox and PC tend to get hit a bit harder in comparison to PS5 when it comes to this stuff. I just don't get why the focus is so much on adding more demanding things to engines/APIs, before clearly fixing these fundamental issues.

It's 2024... we should be able to fly through a game world and have it seamlessly stream in without stuttering. For crying out loud devs were doing better in the past with CDROM based streaming off the disc. The priorities are all wrong. Get the fundamentals right and build up off that without compromising it.

Being that this is a common issue among professional developers who literally do this stuff for a living, I'm guessing it's harder than you're making it sound. It's also likely no coincidence this issue is more common with Unreal Engine games.

 

[Remij]

Being that this is a common issue among professional developers who literally do this stuff for a living, I'm guessing it's harder than you're making it sound.

Making games is hard.. boo hoo.. Problem is... professional developers of years past.. could do it.

If you follow developers on twitter at all... there's a LOT of them which call out the inability for these professional developers to be able to do the same these days.

 

[Seanspeed]

Making games is hard.. boo hoo.. Problem is... professional developers of years past.. could do it.

If you follow developers on twitter at all... there's a LOT of them which call out the inability for these professional developers to be able to do the same these days.

Years past are the key words there. Games used to be magnitudes less complex to make, both in terms of game design but also technical ambitions. A single texture on its own nowadays with everything involved is like 50x more complex and demanding than anything in any game from 1998.

And yes, devs of old can call out things all they want, if they're not actually doing it themselves, it means nothing. It's just old people shouting at clouds. The days of DIY engine coding and a team of four people being able to hash and out and bug fix an entire game are long, LONG gone.

Times simply cant be compared.

It cannot be some coincidence that Unreal Engine games have these same exact streaming/stuttering issues. And it's not all from lackluster or rookie devs or anything. Even highly competent studios like Respawn and Square Enix fall foul of these issues, and it's also not some coincidence that the UE games that dont tend to suffer from these problems are simply less ambition ones.

Ignoring the difficulties of modern game development is insane. And it's beyond annoying seeing people who have no idea what's involved trying to call people incompetent and being extremely unappreciative for the work they do to make games for us to enjoy. I still maintain that gaming is the only hobby where the people who consume the media regularly hold the creators of said media in disdain.

 

[Remij]

Years past are the key words there. Games used to be magnitudes less complex to make, both in terms of game design but also technical ambitions. A single texture on its own nowadays with everything involved is like 50x more complex and demanding than anything in any game from 1998.

And yes, devs of old can call out things all they want, if they're not actually doing it themselves, it means nothing. It's just old people shouting at clouds. The days of DIY engine coding and a team of four people being able to hash and out and bug fix an entire game are long, LONG gone.

Times simply cant be compared.

It cannot be some coincidence that Unreal Engine games have these same exact streaming/stuttering issues. And it's not all from lackluster or rookie devs or anything. Even highly competent studios like Respawn and Square Enix fall foul of these issues, and it's also not some coincidence that the UE games that dont tend to suffer from these problems are simply less ambition ones.

Ignoring the difficulties of modern game development is insane. And it's beyond annoying seeing people who have no idea what's involved trying to call people incompetent and being extremely unappreciative for the work they do to make games for us to enjoy. I still maintain that gaming is the only hobby where the people who consume the media regularly hold the creators of said media in disdain.

Yes, times CAN be compared. Back in the day developers worked within the limitations of that hardware just like they do now.

This whole entire "oh things are more complex so you have to settle for these issues" is complete bullshit. Cut it out.

Do you follow ANY developers on twitter, like at all? They call each other out FAR HARDER than any gamer does... lets be real. And I'll tell you... the far more annoying thing than seeing people who have no idea what's involved calling others incompetent... is seeing people defend clearly flawed products. I don't give a shit how hard the job is dude.... if I buy a car.. it has to work properly... that's not an unrealistic expectation.

 

[Seanspeed]

This whole entire "oh things are more complex so you have to settle for these issues" is complete bullshit. Cut it out.

This is just delusional, I dont know how else to say it. If you honestly think game development nowadays isn't massively more complex and challenging than decades ago, you simply dont know what you're talking about.

Listening to some grumpy old developers of old who no longer make games means nothing. I absolutely follow modern interviews and GDC talks and whatnot from developers and whatnot where possible.

"if I buy a car.. it has to work properly... that's not an unrealistic expectation."

All these games do work, they just dont work as perfectly as you want them to. Every single car in existence has problems, too. But making a car move down a road is not at all the same as making a bunch of magic happen in a virtual world flawlessly. Most cars are all doing mostly variations of the same thing as has been done for more than five decades. Every game is basically an entirely new massive ground up challenge.

 

[Remij]

This is just delusional, I dont know how else to say it. If you honestly think game development nowadays isn't massively more complex and challenging, you simply dont know what you're talking about.

I'm not saying it isn't massively more complex and challenging.. I'm saying it doesn't change the fact that the end product has to work properly... and that's an expectation which should NEVER change...

but guess what? It is... because of people like you making excuses for them.

 

[Flappy Pannus]

It cannot be some coincidence that Unreal Engine games have these same exact streaming/stuttering issues. And it's not all from lackluster or rookie devs or anything. Even highly competent studios like Respawn and Square Enix fall foul of these issues, and it's also not some coincidence that the UE games that dont tend to suffer from these problems are simply less ambition ones.

Except that many of them don't though? There have been plenty of cases as well (at least with UE4 titles) that games shipping with horrible shader stuttering issues that were greatly alleviated by patches that added a minimal delay to startup by just compiling the shaders that they could, if imperfectly - even from small, inexperienced dev teams. The outcry from a community now educated on the source of these issues no doubt fueled the movement on those patches, and has lit a fire under Epic to continually improve these deficits of the UE with every point release of UE5. Like the devs of The Ascent said they were taken aback by the complaints of stuttering for their game when it dropped, as they weren't experienced devs and really didn't have a clue what that was - but they researched it and added in a patch that massively improved things. On the other end of the spectrum, you have the developers of the the Alone in the Dark remake, who's solution to the awful shader stuttering was to add in an .ini setting that they found by scanning reddit gamers.

So, do I think when these games ship with these errors, it's because of 'lazy/stupid devs'? Of course not, I'm sure they work harder than I do. As others have brought up on this topic though, I do think there is a certain element of brain-drain that has occurred in the industry, and it's not hard to fathom why considering the far more lucrative employment offerings you can get in other fields where your skills are actually rewarded, vs the grind of a modern day gaming publisher.

(BTW, one of the most highly-praised games from the perspective of consistent performance lately is Stellar Blade - an UE4 title.)

Ignoring the difficulties of modern game development is insane. And it's beyond annoying seeing people who have no idea what's involved trying to call people incompetent and being extremely unappreciative for the work they do to make games for us to enjoy. I still maintain that gaming is the only hobby where the people who consume the media regularly hold the creators of said media in disdain.

There is a toxic element within a decent portion of the gaming community no doubt, but I don't think that falls on the shoulders of those drawing attention to games that literally freeze for several seconds in the middle of gameplay. That simply should not have shipped, and the only way this kind of thing is going to be actually addressed if if there's a consistent PR price to pay when it occurs. In the end, it's a product, and if it has significant flaws, you don't get to hide behind "well, it's complex" - making games is not an unavoidable genetic condition. The fault points for these issues may be deep and entangled and are certainly a product of the financials of the industry, but they're perfectly valid to be called out by gamers, and it's the only way you'll see some improvement so publishers understand that allocating more resources to value the skillsets needed for performance is valuable.

Do you follow ANY developers on twitter, like at all? They call each other out FAR HARDER than any gamer does... lets be real. And I'll tell you... the far more annoying thing than seeing people who have no idea what's involved calling others incompetent... is seeing people defend clearly flawed products. I don't give a shit how hard the job is dude.... if I buy a car.. it has to work properly... that's not an unrealistic expectation.

He can be needlessly acerbic at times, but it was amusing when a dev was talking about the optimization issues and lack of competence with the development of modern games when he got hit with the "Well what have you done, game development is hard!" reply - only to link them to his site and the patch he made where he largely solved one of the biggest reasons for the stuttering in Arkham Knight in a week.

I was blown away, and quite a bit disappointed, by the fact that such a straightforward optimization (implemented in under a week) was able to get such a dramatic performance improvement in a game this notorious for bad performance. I wonder how the people responsible for this implementation found it acceptable, and whether they realize how much money was thrown away by something that could’ve been fixed in a single man-week of work.

I argue a lot for craftsmanship-for-craftsmanship’s-sake, and I realize how that usually doesn’t make financial sense, but in this case a little attention and care could have saved the publisher millions of dollars in actual costs, and only god-knows-how-much in reputation and goodwill.

Sloppy engineering has a cost, good craftsmanship has actual value, and the next time someone is allowed to ship something this terrible we might not be lucky enough to be able to develop such a fix.

 

[trinibwoy]

Yes modern games are more complex. Modern aircraft are more complex too. There’s an expectation that we build on knowledge, technology and lessons learned from the past. This new generation of games is definitely not starting from scratch.

 

[Seanspeed]

I'm not saying it isn't massively more complex and challenging.. I'm saying it doesn't change the fact that the end product has to work properly... and that's an expectation which should NEVER change...

but guess what? It is... because of people like you making excuses for them.

If you say the times *can* be compared from back in the day to now as you did, then you're not actually acknowledging the insane levels of additional complexity involved.

You can demand things all you want, but it's ridiculous to do so without actually recognizing what goes on into doing so, while also having ever more ambitious games. I honest to god cant imagine the mentality where you can want something to be much better than before while also not recognizing what might have to go into that.

Maybe I'm just a more inherently sympathetic person than you are or something? Or maybe you still, despite what you say, dont realize that video games are genuinely some of the most insanely complex pieces of software in existence? You say you follow developers, so you should know perfectly well how most of them will tell you that getting a game finished and in decent working order is basically the closest thing that humanity will come to an actual divine 'miracle', right? I'm betting you have conveniently ignored all those sorts of comments or probably just haven't actually been following as many developers as you claim to have.

 

[Seanspeed]

Yes modern games are more complex. Modern aircraft are more complex too. There’s an expectation that we build on knowledge, technology and lessons learned from the past. This new generation of games is definitely not starting from scratch.

Modern passenger aircraft are really not that much more complex, though. Main areas of priority are maintenance and efficiency. It's evolutions and working all very well known principles.

But if we're talking 5th gen military jets vs upcoming 6th gen jets, suddenly it's not so 'simple evolution' anymore, is it? Now we are absolutely talking incredible complexity and a huge balancing act that will likely require compromises even with huge costs to try and minimize them. And things will absolutely go wrong in the early releases, as they always do for basically any new complex military platform.

Games are genuinely at the cutting edge of software. They aren't some boring evolution from before meant to improve financial efficiency of the business. Games require improving the technical performance by a large degree and satisfying a customer base that demands that. Games are constantly trying to be the Concorde of the aviation world.

 

[Remij]

He can be needlessly acerbic at times, but it was amusing when a dev was talking about the optimization issues and lack of competence with the development of modern games when he got hit with the "Well what have you done, game development is hard!" reply - only to link them to his site and the patch he made where he largely solved one of the biggest reasons for the stuttering in Arkham Knight in a week.

I remember that lmao. Now, obviously.. coming from someone in the industry has far more merit than coming from a lowly gamer.. but it just goes to show you a simple reality of today... a lot of programmers simply don't know how, or for whatever reason can't (time, budget, ect.) properly write/optimized code.

It's basically like cursive writing these days. Kids these days don't learn how to write in cursive. You could literally write something in cursive and kids these days probably wouldn't know how to read it. It looks like a foreign language to them. Now, do they NEED to know how to write in cursive? Nope.. not with today's technology. However, they've lost out on understanding the fundamentals of writing. Think of all the information of the past written in cursive.. which is essentially lost to these new generations. For example, my mom recently passed away, and all of her cooking recipes were handwritten in cursive.. and while my wife and I both can read and write handwriting, our young daughter certainly can't. If we want to pass that information on to the next generation, we'll have to rewrite and print everything out.

That's basically how programming is these days as far as I see it. A lot of them never learn the fundamentals, because they're no longer needed...so it's basically building unoptimized code on top of unoptimized code in a lot of cases. A lot of reasons contribute to it.. but let's not act like expecting a game to run properly and not literally freeze while you're playing it... is some new unrealistic expectation. It's the bare minimum. If your game ain't doing it... learn why it isn't and fix it.

 

[Seanspeed]

I remember that lmao. Now, obviously.. coming from someone in the industry has far more merit than coming from a lowly gamer.. but it just goes to show you a simple reality of today... a lot of programmers simply don't know how, or for whatever reason can't (time, budget, ect.) properly write/optimized code.

It's basically like cursive writing these days. Kids these days don't learn how to write in cursive. You could literally write something in cursive and kids these days probably wouldn't know how to read it. It looks like a foreign language to them. Now, do they NEED to know how to write in cursive? Nope.. not with today's technology. However, they've lost out on understanding the fundamentals of writing. Think of all the information of the past written in cursive.. which is essentially lost to these new generations. For example, my mom recently passed away, and all of her cooking recipes were handwritten in cursive.. and while my wife and I both can read and write handwriting, our young daughter certainly can't. If we want to pass that information on to the next generation, we'll have to rewrite and print everything out.

That's basically how programming is these days as far as I see it. A lot of them never learn the fundamentals, because they're no longer needed...so it's basically building unoptimized code on top of unoptimized code in a lot of cases. A lot of reasons contribute to it.. but let's not act like expecting a game to run properly and not literally freeze while you're playing it... is some new unrealistic expectation. It's the bare minimum. If your game ain't doing it... learn why it isn't and fix it.

It's interesting that you claim to follow a lot of supposed game developers, yet have only heard how terrible game developers are, and not how incredibly complex modern game development is, especially finishing and polishing a highly ambitious title.

It sounds an awful like you dont really follow many game developers at all, and certainly not many from the 21st century. But I'm guessing if you did, you would unfollow them cuz they wouldn't be saying the things you want to hear.

 

[Remij]

If you say the times *can* be compared from back in the day to now as you did, then you're not actually acknowledging the insane levels of additional complexity involved.

You can demand things all you want, but it's ridiculous to do so without actually recognizing what goes on into doing so, while also having ever more ambitious games. I honest to god cant imagine the mentality where you can want something to be much better than before while also not recognizing what might have to go into that.

Maybe I'm just a more inherently sympathetic person than you are or something? Or maybe you still, despite what you say, dont realize that video games are genuinely some of the most insanely complex pieces of software in existence? You say you follow developers, so you should know perfectly well how most of them will tell you that getting a game finished and in decent working order is basically the closest thing that humanity will come to an actual divine 'miracle', right? I'm betting you have conveniently ignored all those sorts of comments or probably just haven't actually been following as many developers as you claim to have.

Give me a break. I completely acknowledge that game complexity is insane. Again, that will never ever change the fact that there's a certain expectation of basic function regarding games. If I can't expect a game to function properly... then is it REALLY a complexity issue, or is it an overestimation by the developer of their ability to create a product which meets basic expectations?

You keep saying I never recognize what goes into it... that's just a complete cop out. You keep regurgitating that response like as if it's going to change anything.

How about you get the basic premise of my argument through your head.. If game complexity is such that MOST developers are unable to release products which don't literally freeze while you play them.. then PERHAPS they shouldn't be making such complex games....

 

[homerdog]

Maybe all the past devs wrote their code in cursive and now nobody can understand how they avoided traversal stutter.

 

[Flappy Pannus]

I feel it's worth to be as clear as possible when describing the frustration some gamers may have at the current state of optimization, and Alex does that pretty well here (timestamped):

We've always had titles that gamers felt were poorly optimized for the target hardware in the past, yes. But this issue of stuttering is indeed relatively new, at least at this frequency that we're seeing in modern titles. This is a distinct performance phenomena separate from frame rate drops. Hell, with the proliferation of VRR displays, frame rate drops are probably less impactful than ever before, but stutters - pauses in frame delivery of 50->300ms or more - are actually relatively rare in older titles despite the 'average' benchmark of those games likely being far below modern titles even on middling rigs. It's a new level of immersion disruption.

Richard makes an amusing comparison earlier in that segment, where imagine if you were watching Star Wars and the entire film paused for 4 frames when a light saber was turned on for the first time. Audiences would wonder wtf, and if anything it's more disruptive to an interactive medium.

Yes, game development is eminently complex and is often a tortuous process - which is why it's so frustrating to see the player yanked out of all that effort poured into artwork and story crafting when the delivery of those assets is so fumbled. I can recognize the immense talent it takes to create these experiences, but also recognize that at some point, priorities were at the very least, misallocated.

I mean for fucks sake, it may be slightly better now, but for most of 2023 it was 'hold your breath see if this PC port compiles shaders', because we had title after title where the concept of "let's playtest this on a PC that hasn't run it yet" was a herculean ask apparently. That issue was compiling shaders, something PC developers had been doing for decades.

That's an oversight that you can't hand wave away as the trappings of modern game development - it's the most basic requirement you have when developing a game on the PC using a technology that debuted over 20 years ago. To not take that into account in any form, no matter how imperfect, is the result of publisher/management indifference or indeed the necessarily skills being lacking on the development side. It is perfectly reasonable to expect that to be addressed.

 

[trinibwoy]

I’ve worked with great software developers and terrible software developers. Not on games specifically but games aren’t special in that regard. There are talented and less talented people in every profession.

I could never support the idea that game developers deserve special consideration because their jobs are hard. Lots of jobs are hard and making games isn’t a particularly critical job. The idea that someone can think designing planes carrying hundreds of people is a simpler job than making Fortnite is kinda hilarious. There is so much prior art in the games industry that a lot of the hard stuff has already been solved and current devs are standing on the shoulders of giants.

So both things are true. Making games is hard. But we also should expect quality for our money. There is no room here for excusing shoddy work (including poor QA).

 

[Scott_Arm]

Maybe all the past devs wrote their code in cursive and now nobody can understand how they avoided traversal stutter.

I think there's a multitude of problems. From the outside it seems like the game developer world has a mentoring problem. If new devs don't know how to do x, it's because old devs didn't teach them x. That's how this stuff works. It's also an education problem. People who go through universities to get their Computer Science degree are not getting the same education as people in the past. The software landscape has changed and most of the industry's needs are not the same as real-time applications. I also think it's disingenuous to compare the games of today to the games of the past. Games get more complex every year. If modern games used the limited material systems of the past, maybe we wouldn't see the same issues. There's also the whole landscape of shader languages, shader compilers, APIs that are all outside developer control. There are just fundamental problems in the PC space. Could some devs do better? Sure. I just don't think the "get good" mentality is really helpful. I can probably think of more reasons why there are problems, but this is just riffing off the top of my head. Ultimately if you're a gamer, just wait for reviews and if a game is stuttering in a way you can't tolerate, don't buy it. It's all we can really do.