The reason I brought up divergence handling is because that's one of the only things I could immediately come up with.
I thought GCN had already a pretty efficient shader core. But I'm all ears about other potential improvements.
(*late correction: Meant to say 4 threads per CU below)
GCN does have a pretty coarse threshold for getting utilization, testing of compute loads shows the architecture takes longer to spin up, needing groups that approach the significantly wider wavefronts and having at least 4 threads per SIMD (then probably at least 4 more to help hide a death by thousand low-latency cuts and now the number of register per wave comes into play).
I suppose divergence handling can partially apply to that.
Another way is finding something to match Volta's ability to not roll over and die with divergent or irreducible control flow with synchronization in the mix.
Outside of that, there was discussion about how nice it would be if there were a scalar equivalent to the floating-point and other capabilities in the SIMDs. Also, there were some discussions from a console context that probably apply generally about kernels or shaders benefiting if it were possible to pull more than one scalar or LDS value per clock into a vector operation.
There's some signs that AMD has tried to improve the shared fetch front end and instruction caches, which might be a scaling barrier.
Register file capacity has not changed, so occupancy is still a source of pressure. I suppose FP16 is the hoped-for mitigation, but one reason why it's not as flexible is that packed math doesn't expand items like the conditional masks or EXE mask.
There was some discussion about wanting some way to reduce occupancy pressure or allow execution on pending registers when utilizing GCN's ability to fire off multiple loads. Vega's waitcnt for memory is significantly higher, with an odd way of being split up so as to not conflict with existing encodings that points to NCU perhaps not being as large a departure as some statements indicate. Whether that means new things or perhaps something really increasing the memory subsystem's latency is unknown.
Then there's the static allocation of registers based on the worst-case consumption of a kernel. I think AMD might have some speculative work on this, but no mention in any roadmap or marketing.
There's the desire to have better memory consistency/coherence (quick release, timestamped coherence, etc.), with various papers from AMD but no mention so far here.
It's more of an aesthetic preference on my part, but I feel some of the low-level details leaking into the architecture with regards to wait states or specific architectural quirks could use tightening up.
Maybe more bandwidth from the L1 (Knights Landing is up to 128B)?
*edit: Maybe start looking the whole cadence and CU implementation at some point?
