AMD CDNA Discussion Thread

https://www.nextplatform.com/2022/0...computer-tests-show-decent-performance-leaps/

 

https://github.com/llvm/llvm-project/commit/47bac63d3f6b9e64fdf997aff1f145bc948f02d9
It seems like cache coherence in GFX940 (CDNA 3?) is achieved by:

* memory local to L2: reads & writes are cacheable
* remote memory (other L2s or CPU): reads are uncached; writes are write through and send an invalidation to the home L2.
* Each L2 keeps a probe filter for CPU cached lines from its local memory, and forces CPU invalidation or writeback as appropriate.

It appears that GFX940 no longer has a unified L2 cache shared by all CUs. It is configurable:

1. from many smaller agents/virtual devices, each having their own private L2 cache;

2. to one single agent having multiple L2 caches.

Each L2 owns a disjoint(?) region of the device memory now, while they appear to still have internal interleaved “channel” partitions.

Makes perfect sense in multi-agent mode where each small agent gets a fixed contiguous region, which can be owned outright by a single L2. But I am not sure how more “monolithic” configurations where one agent sees multiple standalone L2s would work effectively. Page level interleaving, eh?

 

https://videocardz.com/newz/amd-ins...t-compute-dies-support-for-hbm3-and-pcie-gen5

 

A100 compared to MI250 in HPC workloads:

 

Does that "MI250X/1" indicate, that it's running on half a MI250X, i.e. only on one of the two devices?

 

Footnote in the first pic says "single GPU of Crusher/MI250X".

 

Yes, same as for SVD. They clearly stated "single GPU" in figure description and "MI250X/1" in graphs legends and "Single GCD" for SVD figure desc.

Also, some comparisons are for V100 vs MI100 with old ROCm etc.

 

Linked document (page 48) for explanation of HPC benchmark results and performance for MI250X vs MI100, A100, V100.
March 31, 2022
CEED ECP Milestone Report: High-order algorithmic developments and optimizations for more robust exascale applications | Zenodo