I can't discuss internals publicly so it's better if Rys tackles this one, sorry.
One thing I can say is that driver is doing its best to provide compiler with as many context around the stuff being compiled as possible. You get hints from the API, you understand what's being executed and on what resources (unless you're doing bindless), you know what shader it is (pixel, vertex,...), what inputs and outputs are, and so on. You take this knowledge and pass it to the compiler which does whatever it can to maximize HW utilization and minimize all the bad things (latency, resource usage, etc.). If there's something that compiler could do better - this will be discussed, assessed against wider spectrum of APIs and needs and potentially implemented. This has been discussed in the driver optimization thread before and is an absolute must for every driver on every HW.
On top of that driver has to (well, should) be conservative so if e.g. we know that we could optimize something and gain 10% boost 90% of the time but it'd break a bunch of apps - we won't do this. If there's something that can be done in the API to mitigate this - this should be discussed with API people (Khronos, MS). If there's something that HW could do, it's being discussed with HW folks and if sane - prototyped, measured and budgeted (people, time, die area). This is how every business operates (or should).
So, yeah, I can't state how things work internally w/o getting shot in the head but people should follow Occam's razor in discussions about HW. Changes in HW - from every vendor I'm sure - happen for a reason. It's absolutely counterproductive and probably counter-factual to assume, that companies do stuff in the HW for political reasons. There are engineers measuring and tweaking stuff. They are the people who make calls - sometimes correct, sometimes not. Happens. So how work in the GPU is scheduled, what precision is being used - those things come from real world data, not from a religious leader. And in general things improve from core to core.