I've read many papers in my life.
I've also ran many games through profilers (pix, etc.). I've also heard many horror stories about developers from IHVs.
I'm very confident that developers don't (and haven't) made the best use of DirectX. You're right that some get closer than others (and then write papers about it!). But honestly I think you're talking about a market that you don't fully understand. If you don't believe me just try profiling a few games (start with GTA 4).
But whether to make the best use of DirectX is the same as being better off designing a rendering pipeline that matches what your game is trying to do more closely, or allows for completely different types of effects, is a completely different discussion. Why are there so very few games that morph vertices in realtime? Why is animation blending so rare? Why couldn't raycasting be used for more things on PC easily? I'm a noob, admittedly, but how efficient DirectX is being used is not entirely the point. Point is, you're not showing anything about papers you've read, you're just telling me that I'm wrong, with little in the way of backing it up? I'm very reasonable and very open to argumentation and evidence, and contrary to popular belief, don't care about winning an argument . So give me some reasons to change my mind other than on faith.
And going back to topic I'll ask you one more question: Why do you think a fixed platform couldn't possibly hold back a fluid platform 7 years later?
I've said the same - by the end of the console generation a lot of PCs are so far ahead of consoles that there was a sizeable number of PCs capable of running higher details, better physics, etc. If a game can target those with far better graphics and make a profit though, then it would happen, and towards the end of the console cycle, you saw some of that happening (high-res textures packs, etc.).
However, I was arguing that your 'fixed' platform is DirectX9 for much of last-gen, before DirectX11 become widely enough available. You think it's hardware that is fixed, but if it is 'open' enough to programmers, it is more software limited than hardware limited. Of course that is not entirely true - at the end of the day, we all know that Art creation has been the primary limitation and eventually a PC can push more by brute forcing things, but they power-to-performance ratio isn't always great, and has typically been worse.
You admitted we've made technological advancements since then (and that those technological advancements might require new approaches to various problems). But those advancements require time and resources to fully exploit. Time and resources have a finite quantity. Having to support a fixed platform diverts time and resources away from researching those technological advancements (since you need to spend resources researching how to extract the best possible performance from the fixed platform). I don't think I'm making any crazy claims here!
Do you think if given more time for pc ports that developers could do a better job? Because if we go by your logic the answer should be no (they've already "maxed out" DirectX). I suspect though the answer might be yes...
There are at least as many PC to console ports as the other way around, and right now in the current generation that holds far more than ever. Not even all
exclusives were developed on consoles last gen. On PC, for a long time they had only DirectX9, and getting at the hardware improvements was a minefield because of the huge variation in GPUs (still an issue). Then there's this whole mentality of we'll 'per game' optimise everything away in the graphics driver ... Again, I'm pretty sure you know more about PC development than I do, but I still don't understand your position.
So the question is whether current "average" gaming PC is faster/more capable than the best consoles available right now ?
Likely yes. (Could check steam hardware survey to have an idea.)
I've seen numbers that suggest the number of Steam PCs more powerful than the PS4 is not even half the number of PS4s out there (remember, already more than 22 million). So perhaps it would be worthwhile looking at that number to be sure. Then I would like to know if you think PC hardware can be used as efficiently as console hardware in principle (we can discuss in practice more later - right now I'm suspecting console optimisation is still in 'early days').