Digital Foundry Article Technical Discussion [2023]

[pjbliverpool]

It's a big part of it, but no - not isolated at least. If it was the vast majority of the problem then these issues these AAA games exhibit could be solved by faster GPU's, but they often can't.

I didn't mean to imply that the GPU pricing issue was connected in any way to the performance issues in some recent high profile titles.

I think its pretty clear for the most part that GPU performance is scaling just as we should expect it to, so faster GPU's won't help much if at all in that regard. The issue (as Andy has pointed out in this very thread) is one of coding and how that interacts on the CPU side.

And in fact PC CPU performance increases are superb right now with pricing being pretty reasonable IMO, so there's no issues there.

It's not DF's job though to worry about how the 'narrative' is formed by less informed sources. There's a reason, while perhaps exaggerated at points, that this narrative has taken hold - it's because so often, PC AAA titles are launching in a very poor state, and that's assuming these problem titles actually get fixed. Not all do.

DF - and frankly anyone - shouldn't be in the business of worrying about platform retention when it comes to delivering evidence-based critique on technical shortcomings of games at launch. The best way to combat the threat of PC gaming losing its appeal is to keep up pressure on publishers to release better ports from the outset. I'm not sure how you combat this 'narrative that is taking hold across the Internet' - like what are you suggesting? If it's not for DF to alter their coverage in any way then I'm not sure what you mean by this being a 'fine line'. How are you suggesting this outlook be combated, if you think it's somewhat disingenuous?

Frankly if more outlets started covering this kind of thing with greater frequency and did it from the outset, we may not be in quite this situation now. Like really, if your rebuttal to concern about the platform is "It's doing fine, you just have to wait 3-4 months after new AAA releases as they're probably be patched", that is...not reassuring!

This isn't a critique of DF at all. I'm a huge DF fan and subscriber. As noted above, I agree that poor ports should be called out and I enjoy Alexs savaging of them when called for.

That said, I don't agree that there is a particular crisis with PC AAA gaming for the reasons mentioned above. I agree these issues suck if A) you like to purchase games as soon as they release, or shortly after and B) are particularly bothered by these types of issues which people like us absolutely will be, but I know for sure not all people are. For example TLOU, widely considered one of the absolute worst PC ports for quite some time was picked up near day 1 by my wife and she thoroughly enjoyed it without any complaints whatsoever. And the game has improved significantly since then with another major patch landing just yesterday.

In terms of the narrative that's building, I agree that's not caused by, or the responsibility of DF, but I have for example seen a couple instances now from high profile sources (Oliver being one in the latest DF Direct) making statements like "even though I have a 4090 and high end CPU I'm choosing to play these games in my PS5 instead" which really helps to fuel that narrative. I can't help but think 'WTF' when I hear that. I mean, even in the abomination that is Jedi Survivor, on such a system you should be able to run the game at max details, native 4k (so a much more visually pleasing experience) with a frame rate cap of say 40-50 fps depending on your CPU performance for a pretty consistent experience. How that's worse than the widely recommended 30fps quality mode on the PS5 which can also drop below and has traversal stutters while upscaling from around 1080p, I have no idea.

 

[davis.anthony]

games are just starting to push systems too far that were never really designed to handle the complexity thrown at them.

PC gamers are also to blame, they've been so used to running ports of games built around 2013 console hardware at 100, 200 or even 300fps that they've built up an unrealistic expectation of what performance they expect in multiplatform games and Sony ports.

So when they play a game built with 2020 console hardware and their frame rates are no longer in the hundreds they blame the game and the developers and shout shit like "Dumb devs releasing an unoptimized game" instead of keeping their expectations in check.

TLOU had its share of problems at launch but to see people with RTX 2060's (a GPU slower in raster than PS5's) complaining they can't at least match PS5's performance and quality settings in what is a port of a highly tuned PS5 exclusive was absurd.

 

[Dictator]

I didn't mean to imply that the GPU pricing issue was connected in any way to the performance issues in some recent high profile titles.

I think its pretty clear for the most part that GPU performance is scaling just as we should expect it to, so faster GPU's won't help much if at all in that regard. The issue (as Andy has pointed out in this very thread) is one of coding and how that interacts on the CPU side.

And in fact PC CPU performance increases are superb right now with pricing being pretty reasonable IMO, so there's no issues there.



This isn't a critique of DF at all. I'm a huge DF fan and subscriber. As noted above, I agree that poor ports should be called out and I enjoy Alexs savaging of them when called for.

That said, I don't agree that there is a particular crisis with PC AAA gaming for the reasons mentioned above. I agree these issues suck if A) you like to purchase games as soon as they release, or shortly after and B) are particularly bothered by these types of issues which people like us absolutely will be, but I know for sure not all people are. For example TLOU, widely considered one of the absolute worst PC ports for quite some time was picked up near day 1 by my wife and she thoroughly enjoyed it without any complaints whatsoever. And the game has improved significantly since then with another major patch landing just yesterday.

In terms of the narrative that's building, I agree that's not caused by, or the responsibility of DF, but I have for example seen a couple instances now from high profile sources (Oliver being one in the latest DF Direct) making statements like "even though I have a 4090 and high end CPU I'm choosing to play these games in my PS5 instead" which really helps to fuel that narrative. I can't help but think 'WTF' when I hear that. I mean, even in the abomination that is Jedi Survivor, on such a system you should be able to run the game at max details, native 4k (so a much more visually pleasing experience) with a frame rate cap of say 40-50 fps depending on your CPU performance for a pretty consistent experience. How that's worse than the widely recommended 30fps quality mode on the PS5 which can also drop below and has traversal stutters while upscaling from around 1080p, I have no idea.

I agreed with you that usually even a Bad PC Port can be way better than the PS5/XSX experience on high enough Hardware. Unlese the Game has extra stuttering on PC not on console in a big way!

The issue I think is that people on PC expecting more - i.e. No stuttering, good menus, and hopefully some scalability. Stuttering for me personally is Just the absolute worst as No Hardware can get rid of it no matter how much you throw at it - and portentially can exist for a decade after the fact in a Game.

 

[DavidGraham]

UE4 was never designed to handle open worlds.

These problems crop up even when UE4 is not involved. Even on engines known to be well multi-threaded, the games often become CPU limited in some way or form.

And as games get bigger and more complex, handling these more complex API doesn't help. Previously with the old APIs, the driver could handle a lot more, now the driver handles far fewer things, and the job of making the whole thing work optimally rests now on the shoulders of the developers who are often time and resources constrained.

I am saying these APIs make the already hard job even more harder and harder. The end result: half the DX12/Vulkan AAA games don't work as advertized, indie games often avoid DX12/Vulkan altogether, and thus have far fewer problems.

These lower level APIs may be better on some overall scale, but only if you take the time to actually do them well, and that time is now a scarce resource that is no longer available anymore. I think by now we all have a 10 years experience witnessing contemporary developers making contemporary games not devoting the necessary time to perfect these APIs, whether on purpose, whether due to lack of knowledge, resources, time or whatever. We've wintessed this time and time again.

That's the real problem with these APIs, we no longer have the luxury of time and resources dedicated to their optimal operation, thus we have the situation we are in now, half the ports are sub optimal or bad. Course corrections (API changes) are needed to make the job easier for those developers who don't have the necessary time/resources/knowledge, and there are lots of them nowadays.

 

[troyan]

While it is going to vary game to game I think the DF discussion on this in terms of UE4 stuff is likely a good chunk of the picture - games are just starting to push systems too far that were never really designed to handle the complexity thrown at them. UE4 was never designed to handle open worlds. Unreal in general has always had a tension between being easy to use with stuff like blueprints and very flexible actors and components vs. maintaining good performance. When scaled to large counts these systems simply bog down on serial CPU logic because they were never designed to handle such cases. UE5 has made improvements of course, but as with most engines it is an ongoing process and adherence to supporting legacy content makes it even more complicated.

I dont understand why you blame "open world" settings. Rocksteady used UE3 for Arhkam: City and Knight and outside of the port problem of Knight these games run much better than the lastest "open world" UE4 games.
There are a lot more games with UE4 showing the same PC specific problems - top down shooter, side scroller, smaller third person games. Every game has at least two things in common: UE4 and DX12 (and sometimes raytracing).

And i understand that it is unfair to use UE4 as an example but when UE4 is used by nearly everyone it shows that there is a generell problem.

 

[Lurkmass]

Spitballing but I'm starting to think that programmable shaders were a mistake ...

Hardware vendors should implement fixed function material evaluator so that way hardware is capable of "toggling" different material states as opposed "compiling" different material shader and just be done with it if end users are going to make a big stink long compilation times ...

Who's in favour of "dynamic states for materials" over "programmable materials/static states" ?

 

[Scott_Arm]

@troyan I think UE is getting some unfair blame in a lot of cases. If a developer decides to make an open world game with UE it’s on them to enhance the engine. They should be investigating the engine limitations before they settle on a game design. They get access to the full source so they can heavily modify the parts of the engine that they need to. I think most big AAA UE games have modified something. If someone selects UE for open world, and they know the limitations, then it’s on them to remove those limitations.

 

[Shortbread]

Hogwarts Legacy is a demanding title across all current-gen systems - and even high-powered PCs. So did the last-gen consoles ever stand a chance of getting a decent rendition of the game? In the DF tech review, Oliver discovers that a wide range of cutbacks, nips and tucks have been made - but the experience holds together well overall considering how old the base technology is.

 

[Silent_Buddha]

Almost always going to have better results scaling down to worse hardware then scaling up to better hardware. Make the better version first, then figure out what needs to be done to scale it to worse hardware and make it run well.

Regards,
SB

 

[Seanspeed]

Yet more evidence that developers simply give Xbox less priority than Playstation versions.

Still, the overall last gen optimizations seem pretty well done and smart overall, to where the game is still really nice looking for an XB1/PS4 game. Performance is actually better than I expected, too.

 

[Seanspeed]

@troyan I think UE is getting some unfair blame in a lot of cases. If a developer decides to make an open world game with UE it’s on them to enhance the engine. They should be investigating the engine limitations before they settle on a game design. They get access to the full source so they can heavily modify the parts of the engine that they need to. I think most big AAA UE games have modified something. If someone selects UE for open world, and they know the limitations, then it’s on them to remove those limitations.

Of course, but I bet there's a lot of "I can fix him" sort of mentality that goes on here that just gets them in trouble later on when there's things they, in fact, cannot fix, at least to the standard they'd like to do.

Still their own fault at the end of the day, but it's not like there's just a sea of great engine alternatives out there up for grabs.

 

[Shortbread]

Almost always going to have better results scaling down to worse hardware then scaling up to better hardware. Make the better version first, then figure out what needs to be done to scale it to worse hardware and make it run well.

Regards,
SB

Well, One X simply received the XBO code treatment (other than running at 1440p), while the PS4/Pro textures and other game assets looks quite better. They should have scaled down from One X to One, and not the other way around.

 

[Shortbread]

Yet more evidence that developers simply give Xbox less priority than Playstation versions.

Sony's market leading position, more moderate/balanced hardware, and relatively easy to use Dev-tools, would do that!

 

[Riddlewire]

Sony's market leading position, more moderate/balanced hardware, and relatively easy to use Dev-tools, would do that!

The PS4's hardware isn't balanced at all.
The CPU is easily the worst of the 8th gen consoles and not suited to its powerful GPU.

The results in cross-gen games for the past few years (ever since Mass Effect Legendary Edition, imo) would suggest that developers have simply given up on making games for 8th gen Xboxes. "Whatever boots" seems to be the rule for Xbox One S/X.

 

[Shortbread]

The PS4's hardware isn't balanced at all.
The CPU is easily the worst of the 8th gen consoles and not suited to its powerful GPU.

The results in cross-gen games for the past few years (ever since Mass Effect Legendary Edition, imo) would suggest that developers have simply given up on making games for 8th gen Xboxes. "Whatever boots" seems to be the rule for Xbox One S/X.

I was talking about the PS5 in this instance, and porting / scaling down to the prior generation (hence, the development/optimization priority for the larger PS4 userbase when dealing with Hogwarts). Plus, Sony having marketing rights also helps.

 

[Flappy Pannus]

In terms of the narrative that's building, I agree that's not caused by, or the responsibility of DF, but I have for example seen a couple instances now from high profile sources (Oliver being one in the latest DF Direct) making statements like "even though I have a 4090 and high end CPU I'm choosing to play these games in my PS5 instead" which really helps to fuel that narrative.

Didn't see that comment, but yeah that seems a bit much. I'd have to know what games he's talking about specifically to categorize it as 'feuling a narrative' though vs someone who just really values framerate consistency though (which is the majority of the DF staff, thankfully for us), depends on the game. I mean if it's his personal preference he's fine to state it, and I'm fairly confident his system wouldn't be misconfigured or he doesn't know what to look for. Still, I agree that the list of games where a 4090 system provides overall a lesser experience than the PS5 is rather small to justify such a blanket statement, but I don't know what he's playing.

I can't help but think 'WTF' when I hear that. I mean, even in the abomination that is Jedi Survivor, on such a system you should be able to run the game at max details, native 4k (so a much more visually pleasing experience) with a frame rate cap of say 40-50 fps depending on your CPU performance for a pretty consistent experience.

Capping the framerate, even at 40, would still present with shader and compilation stutters though (note the latest patch also increased performance mode framerate on PS5 significant as well).

I agree at least Jedi Survivor is probably not the best indicator of health as related to the platform itself, it was more the salt on the wound coming right after TLOU. It's more accurately an indicator of poor quality of recent AAA releases in general as it has significant issues on all platforms.

 

[Flappy Pannus]

I dont understand why you blame "open world" settings. Rocksteady used UE3 for Arhkam: City and Knight and outside of the port problem of Knight these games run much better than the lastest "open world" UE4 games\

The only way I found to get rid of Arkham City stuttering was to use DXVK. I mean it's well-known for this, and UE3 titles in general! Lots of UE3 titles had stuttering. It's also a tiny open-world compared to games even from the PS4 era.

It also has shader compilation stutters, but people didn't know what those were back then, or rather the tech press didn't. It's partly of matter of these kinds of performance bottlenecks just weren't covered at the time.

(tbc I'm not saying this to deflect attention away from the critique of UE4, if anything it's more condemning that shader/traversal stutter was a known problem in UE3 titles too!)

 

[Andrew Lauritzen]

I dont understand why you blame "open world" settings. Rocksteady used UE3 for Arhkam: City and Knight and outside of the port problem of Knight these games run much better than the lastest "open world" UE4 games.

Yes true, I'm using "open world" as an imperfect proxy for a lot of other things: more complex worlds with more assets and more dynamic behavior. And the huge one when comparing to those older games in particular: dynamic lighting! If you're fine with baked/static lighting then a lot of things get much simpler, and fit much better with UE4's lighting systems.

There are a lot more games with UE4 showing the same PC specific problems - top down shooter, side scroller, smaller third person games. Every game has at least two things in common: UE4 and DX12 (and sometimes raytracing).

The end result: half the DX12/Vulkan AAA games don't work as advertized, indie games often avoid DX12/Vulkan altogether, and thus have far fewer problems.

On specifically the issue of shader compilation stutter, then yes the new APIs make that more difficult in some ways as has been discussed to death at this point. Although worth mentioning it is not a problem that is exclusive to the new APIs.

Beyond that issue though I don't think there's as much commonality in the causes and there's a lot of spurious correlations that people jump to (trend is towards more complex games, trend is to newer APIs, etc). Certainly from the games I've looked at at Intel and otherwise the submission thread issues were indeed mostly resolved by the new APIs. In Unreal it's much more common to be bound by the "render thread" - which critically does not interact with the graphics API - than by the "RHI thread", which is the one that does all the API stuff. The render thread does lots of stuff like collecting lights and primitives from the scene, managing things like visibility and occlusion and related data structures and constructing a high level graph of passes and dependencies that will later turn into the command stream on the RHI thread. Different games/engines/platforms organize this slightly differently of course but I haven't really heard of many folks complaining about the CPU overhead of the new APIs outside of PSOs in particular.

 

[Andrew Lauritzen]

Stuttering for me personally is Just the absolute worst as No Hardware can get rid of it no matter how much you throw at it - and portentially can exist for a decade after the fact in a Game.

Agreed.

Judging from some of my less technically inclined friends, I think a main point of confusion here is that you guys are often pre-addressing the "well just get better hardware idiot" stupid replies by saying "and look at how bad it runs even on a very high end system". But often the message that gets through to the non-tech-savvy folks is "well DF says Jedi Survivor only runs at 40fps on a 4090 so on my 2080 it's going to run at 5fps!" which obviously isn't the case, especially with CPU bottlenecks. The complaint is really that it's not scaling up well to the high end, but sometimes that gets misunderstood by people as the absolute workload is just so high that even beastly cards can't handle it.

To be clear I don't think it is anything wrong with the way you are communicating nor do I have suggestions on how to avoid this second tier of tech reporting that makes the message a lot blurrier to the wider range of folks, but it's an effect I've seen happen a lot more recently.

 

[Shifty Geezer]

I dont understand why you blame "open world" settings. Rocksteady used UE3 for Arhkam: City and Knight and outside of the port problem of Knight these games run much better than the lastest "open world" UE4 games.

How much customisation of the engine was done though? Sometimes devs start with an engine and modify it themselves - the source-code is available to modify. As games get more complex and UE gets more complex, the opportunity/cost to adapt the engine at a lower level is diminished, meaning a need to work either within the real limits of the engine (which you may not know until you've made a game or three and learnt how you assumptions for assets don't actually work that well) or make the game you want and cross your fingers.

I think it rare for devs to limit their game vision to the tech available, which is why 60 fps remained so rare. They'd rqather more eye-candy and effects and physics and a ropey, stuttery framerate than target smooth 60fps. And Nintendo shows how well you can target hardware and design for it, art and all. I also guess devs fear that a game that runs well because it fits the tech (hardware and engine) will look weak in stills and videos which is where their marketing happens.