So watching that video, it's good that Epic appears to be taking this issue seriously, because it's a serious issue. They DID say that the new automated method will continue to be improved over time, and that's all good and great. I just wanted to chime in though.. They sort of mention how bad it looks that Epic's own engine in their big crown jewel product Fortnite.. has these shader comp stutters. But the reality is the despite the game's status for the company, and it being their engine, that actually couldn't be further from the case, because Fortnite is actually a worst-case type scenario for this particular issue. DF more than likely knows this too of course.
It really doesn't have anything to do with realism vs cartoony visuals.. you can have super complex shaders with cartoonish visuals.. rather it's the sheer amount of different assets and materials and effects the game has, and the customization it offers the player to mix and match them all. When you think about a typical single player game, the developers create all their assets and everything is very curated and mostly known ahead of time. You play a level, and they know exactly what's coming. This generally makes it easier for developers to collect PSO data.
However, with a game like Fortnite, you have tons of different characters with different visual styles, hundreds upon hundreds of different customizable options for those characters.. and you can also mix and match equipment, and there's all sorts of different effects and materials in play for the various equipment and guns and so on. You also have all the game's environment assets, tons of effects like when breaking objects because everything is destructible, all the different guns and so on. So the fact that any of the 100 players you encounter could have any combination of those things.. means that capturing all that PSO data during development, would likely be very challenging compared to the traditional game. Not to mention, that even running the game at different graphics settings could necessitate shader recompiles.. which means you have a combinatorial nightmare to trying to capture it all, and I could see how that is impacted by the realities of game development.
Which had me thinking again about what I read in the Unreal Engine documentation a while back about it being possible to crowdsource PSO data from users. I think for this type of game (live-service MP game with tons of customization and assets) crowdsourcing the PSO data from players is exactly the way they should go about building as complete of a PSO data cache as possible. In the documentation it says that there's no infrastructure built into Unreal Engine to do it.. but that it's possible. So what Epic should do is build that directly into the game. Have it log and generate the PSO data by default (player can opt out if they want), then after the game client is closed, that file gets uploaded to Epic's servers. Very quickly you'd get massive coverage of all different combinations, and all at various graphics settings. Then they could update the game with the better PSO coverage.
Due to the sheer amount of shaders, having that automatically begin compiling as the game starts wouldn't be a good idea, since it would likely take quite a few minutes. So in that case, they could add it as an option in the menu, for people who want to do it and wait until the shaders are pre-compiled. They could even have it go in the background while you're in the menu, and that way you could look around the store, or do other things while it's compiling.
I think crowdsourcing PSO caches for these types of games just makes a lot of sense, and then giving the pre-compilation process as an option in the menu would be perfect. I also think that Valve should implement this type of feature directly into Steam's framework, for developers who could utilize this type of thing. Just like uploading cloud saves.. If a developer sets it up, it automatically uploads PSO caches created by their games, and allows them to take all that data from all the players and get the most coverage possible to be able to integrate it into future updates.