AMD Architecture Discussion

The fabric is described as a superset of Hypertransport, and also that there is only a coherent version of it.
It states that Vega should introduce a mesh version, with bandwidths 512GB/s and up.
Classical Hypertransport has a baked-in crossbar, which would be a significant contributor to the complexity of an SOC change, which it seems like AMD has changed somehow.

It's curious how that plays with GCN as we know it.
If Vega starts at 512 GB/s for memory bandwidth, it's a step down if the fabric is introduced between the GPU L2 and L1, which already greater that 1 TB/s in high-end GPUs, and given the protocol's overhead and coherent nature seems excessive as long as the dozens of L1s are write-through every 4 cycles.
If it's outside L2s or in the memory controllers, that should allow the promised 1:1 memory and fabric bandwidth ratio and wouldn't disrupt the classic GCN L1-L2 model. In that case, though, it would seem to be bit excessive as a coherent fabric in a discrete GPU context if coherence is handled the way it usually is.

The SenseMI system mentioned for Zen would be using some form of the fabric, or at least a control variant of it. Could the control protocol be kept as a parallel fabric, rather than running over the data paths?

 

Assuming the fabric is designed to make a distinction between non-coherent vs coherent accesses (for accelerator & peripheral IPs), they could move L2 into shader engines (or finer grained blocks) by exploiting the fact (IIRC) that CU R/W accesses generally are non-coherent, except when GLC/SLC bits are set, or it is an atomic op. The coherent protocol can then be used to back atomic operations and coherent accesses to the system.

How the export bus and ROPs gonna play their part is a serious question though...

 

Naively moving the L2 slices to a per-engine arrangement would break GCN's coherence method, which relies on per-channel assignment that physically prevents a location from being cached in more than one coherent location.

If it's a superset of Hypertransport, it does support non-coherent accesses. There's a non-coherent form of it, although the article seems to be saying there's no non-coherent option for the Infinity Fabric.
For HT, it's a bit set in the command portion of the packet. That may be more costly if moved into the parts of the GPU that were more "dumber" in the past if it were to have the 8 or 12 bytes of overhead per 4-64 bytes of payload that the main protocol has.

 

As said in the previous post, L2 acts as a point of coherence only for memory accesses that meet any of the three conditions (GLC/SLC/atomics). For the bulk of the accesses, the L2 is not coherent due to non-coherent L1 caching. So for accesses under those conditions, the private L2 instances can be coordinated by the coherent data fabric.

I am rather inclined to read it as "having no variant that drops all the cache coherent mechanics". Non-coherent accesses are rather essential for a SoC-friendly (rather than a CPU centric) interconnect to achieve optimal performance AFAIK.

I believe nothing prevents on-chip network from using its custom internal encoding though — they just have to appear to be HT at a certain boundaries. Moreover, AMD's generations of on-chip HT interconnect all separate the control datapath from the data datapath IIRC.

 

It would be a mesh that is capable of supporting full memory bandwidth, which historically AMD has not been able to offer coherently in its chips due to the mentioned mixture of multiple on-die interconnects. A crossbar would be the most consistent in terms of latency, but as implemented it is not modular enough to isolate its parts of the system from knock-on effects and validation for changes in client count or the types of clients in it.
AMD's APUs and CPUs have generally had limited changes to the number of crossbar clients and a rather confused mixture of on-die connections for various sub-components.

By contrast, a modular system that has interconnect stops of a fixed complexity and more abstracted message protocol can be scaled and individual stops modified without potentially revalidating or cascading changes across the whole chip, which might be what AMD is mentioning.
For a potentially similar scenario:
http://www.realworldtech.com/sandy-bridge/8/

However, per that same article the ring bus is still simpler than a mesh, although due to scaling issues at high client counts Knights Landing also goes with a mesh. That might be more like a system of perpendicular rings, maybe.

Since Vega is on an interposer, there has been research done by AMD on implementing a mesh network in the interposer. However, a practical implementation became some kind of butterfly network or a concentrated the mesh into a smaller number of stops shared by multiple clients on the chip above it.

Graphics and compute can feed into one another, and the L2 also supports the atomics the CUs rely on. It's comparatively cheap the way it's implemented now.

 

Coherence is maintained by a read/write miss to the next level of the hierarchy as needed, since there is no snooping built into the system. In the case of localized L2s and their atomic units, that would mean SLC and L2 atomics resolve to some kind global miss, unless the L2s start snooping or there's another cache.

For non-coherent accesses, unless there's a change to the sliced L2 design they're going to somehow behave at an SE level as if they are striped across all memory channels. Capacity-wise, they would act as if they were an L2 1/(SE count) the size of the overall L2.

On a side note, there's also a system request queue that is centralized in the classical HT northbridge whose fate is somewhat unclear in this superset fabric. A mesh and the more distributed architecture of a GPU seem to indicate that this may have changed, although as a superset of the original protocol some provision for a queue would remain.

 

Yes, cache coherence between distributed L2s is what I meant. It feels like an inevitable step to me, as they can't possibly indefinitely grow the crossbar. (The crossbar in Fiji is probably a fat tree instead of a full 64x16 one. But still.)

Though I just recalled from my memory that L1D is write-through with dirty byte mask. Decentralizing the L2 would definitely be a problem even for non-coherent accesses in this case, hmm.

Edit: Hmm, a second thought tells it is not if the dirty byte mask is carried through by the fabric to the memory controller.

SRI seems to aggregate requests from cores to reduce the complexity of the NB crossbar. So I guess it would be gone if one has a ring or a mesh, especially when Zen quad-core blocks presumably presents only one outfacing I/F.

 

It is a problem for non-coherent accesses if the interconnect handles dirty lines at a granularity of lines, but not a granularity of bytes (i.e. dirty byte masks). Specifically, if two CUs edit the same cache line with unequal dirty byte mask, while the order is indeterministic for writes to the same byte, the non-overlapping changes of both sides should be visible regardless. If the interconnect handles writes without the dirty byte mask, the memory controller would not be able to "diff" the change but simply overwrite the conflicting line, i.e. either version of it wins.

That's said I believe modern interconnects already do this to minimise the bandwidth use (so as GCN L1 -> L2 despite having full 64B interconnect), so it might not be a problem.

 

would that be the skybridge arm soc?

 

IIRC Skybridge was scrapped in 2015.

 

too bad it would be very good for mobile

 

The ARM "semi-custom design win" is still to show up, unless it's also been scrapped.

Regardless, there are AotS results in there, so even if it's an APU, it's using a x86 CPU.

 

Perhaps in a compute, APU, or multi-chip scenario it matters more. What's not entirely clear right now is how Vega makes use of it since the number is both very high for a lot of interconnect work, but also low for GCN's historical numbers for internal cache bandwidth.

I'm not sure all interconnects could do that. The interposers are 65nm and the actual bumps like with HBM are ~40-50 um pitch. There's supposed to be improvement there someday, but it's orders of magnitude above the size that some of the highest-bandwidth on-die paths. This was one area where Intel was in the past skeptical of how quickly the pitches could be scaled.

 

That's not quite relevant to my point though. Sorry if it wasn't cleared enough. I was commenting on a certain aspect of memory consistency in GCN, which AFAIK relies on write combining in L2 with dirty byte mask.

To illustrate with a simple example, let's say you have two CUs referenced the same cache line. Initially it is AAAAAAAA. The first CU modifies it locally as ABABDDDD, and the second CU modifies it locally as CACAEEEE. Write combining with dirty byte mask would eventually yield CBCB????, where `?` is indeterministic.

If it is winner-takes-all, all locations would be indeterministic without synchronisation. In other words, non-competing writes by different CUs to the same cache line would intervene with each other. This is definitely bad.

 

Time to bring this up again: http://research.cs.wisc.edu/multifacet/papers/hpca14_quick_release.pdf