Another hypothesis is power. You want the main power and ground lines to go straight through to the chip, and the socket is designed for Zeppelins to be where those chiplets were placed.
That could be a limit on placement, as some elements like IO and to a lesser extent where the DDR lines go to leave the package seem to have some similarity within some margin of error. That margin gets larger where the IO die is, which concentrates PHY for DRAM and IO nearer the middle.
If the concern is how direct this is from the pins for power, there's a fair amount of distribution and metal used for Zen's voltage regulation that muddles things. Slide 19 from
https://www.slideshare.net/AMD/isscc-2018-zeppelin-an-soc-for-multichip-architectures seems to show some power pins align with the CPUs, but a very large chunk of it would now be under the IO die, and another large chunk is concentrated on one side of the MCM.
Slides 9 and 10 show a possibly undesirable amount of latency could be involved, if AMD hasn't adjusted more of its fabric's features. Possibly, the data fabric on the CPU chiplets will be simplified by having a number of its clients moved elsewhere. I'm curious if there's potentially some special case handling possible since the coherent CPUs are known to be on one side of an IF package link, and the home agents and routing hardware are on the other.
Could someone give me more insight?
I've seen many negative comments in Anand saying the new product is 50% more dense but only 20% better in performance?
I'm not sure if it's necessarily negative, but that seems to be accurate. It's been a very long time since density scaling was linked strongly to performance or power scaling, with 90nm or 65nm being the threshold where a number of vendors got hit by significant problems.
Density governs how many transistors there can be in a chip, whereas performance in this context is more about the straight-line speed of individual circuits or pipelines. Only a subset of the overall set of transistors can take part in any local block or circuit, and in general more is not better due to each contributing some amount of delay or each increasing the length of wires required to connect them.
There are other physical conditions and architectural choices that can influence how much power, transistors, or wires come into play within a given time window. The faster a desired clock period, the more costly tradeoffs will become.
Trying to focus on parallelism can utilize the 50% increase in density without travelling as far up the steep upward curve of pushing clock circuit performance, if that option is available.
1 - Where's the L3 cache? One would think it would belong in the chiplets to reduce latency and the new IF links don't seem to be wide enough to keep up with the bandwidth. OTOH the chiplets are tiny (way less than half of a Zeppelin) and that I/O die is huge. There's no way the I/O die has only DDR4 PHYs and IF glue, it has to have some huge cache for coherency (L4?).
It seems probable there's cache or SRAM arrays on the IO die, at least because unless AMD has changed its system topology the home agents that maintain memory ordering and handle coherence would be there (unless AMD changed this for Infinity Fabric 2.0). How fast some of these processes can be, if they are always a link or more in distance and are on a die not optimized for speed may be a question mark (perhaps more for client-oriented products?).
A local L3 absorbing traffic before it traverses the IF links and contends for the centralized resources also seems worthwhile.
There's also PCIe, the PSP(s?), encryption hardware, USB and disk controllers, potentially. AMD promised expanded enterprise features, which may belong closer to the memory controllers and IO complexes. One question I have is whether this changes what happens for some of the error handling, where it used to be that errors pertaining to off-chip links would have a nearby CPU and its local memory to handle them. If there's a link problem with a CPU chiplet, is there a resource on the IO die that can step in?
Have any other CPU benchmarks been revealed?
