NVIDIA COPA - Composable On-Package Architecture

This seems to be purely for data centre compute:

2104.02188.pdf (arxiv.org)

"In this work, we demonstrate that diverging architectural requirements between the HPC and DL application domains put converged GPU designs on a trajectory to become significantly under-provisioned for DL and over-provisioned for HPC. We propose a new composable GPU architecture that leverages emerging circuit and packaging technologies to provide specialization, while maintaining substantial compatibility across product lines. We demonstrate that COPA-GPU architectures can enable selective deployment of on-package cache and off-chip DRAM resources, allowing manufacturers to easily tailor designs to individual domains."

So the focus here is that cache and memory controllers are interchangeable, collectively called Memory System Module. They attach using custom links to the GPU Module (GPM).

Along the way I learnt that 826mm² appears to be the current reticle limit.

 

NVidia is pretty safe with its software infrastructure for, let's say, five years.

This snippet is pretty worrying:

"Figure 12 shows that doubling and quadrupling the number of baseline GPU-N instances (2× GPU-Ns and 4× GPU-Ns) results in mean 29% and 43% performance gains respectively for our training workloads. We find that a DL-optimized HBML+L3 COPA-GPU configuration (with 27% performance gain) provides similar levels of performance to 2× GPU-Ns, yet should cost significantly less than buying and hosting 2× larger installations of traditional GPU-Ns.

[...]

HBML+L3 integrates 1.6× more HBM memory, resulting in total aggregate cost lower than 2× of GPU-N. Thus, DL-optimized COPA-GPUs will provide substantially better cost-performance at scale, saving on not just overall GPU cost but additional system-level collateral such as datacenter floorspace, CPUs, network switches, and other peripheral devices."

It demonstrates that in DL, scaling by using more GPUs is almost a dead-end - and you can bet NVidia's customers have noticed. Sure, putting 1GB of L3 cache and much more HBM bandwidth is a radical, difficult change, but NVidia's competitors are doing radical difficult things too.

Much like patent documents are usually a sliver of the future, because the products are way more complex than any single document can hint, I think it's safe to assume that NVidia is also planning to do far more radical things inside the GPM.

Perhaps processing in memory is where DL is headed.

 

So, erm, Hopper being "chiplet" based might be where we start to see COPA?:



Thread indicates that there's a growing consensus that Hopper is data centre (Lovelace is gaming).

Videocardz's spin-off article refers back to tweets from May about the configuration of GH100:

NVIDIA Hopper GPU rumored to tape out soon - VideoCardz.com

So I suppose we now need to be aware of NVidia codenames for products that use the "composable" concepts. Hopper might be the architecture of the chiplets, but something else might be the name of the AI accelerator and something else again might be the name of the non-AI accelerator.

So a collection of codenames related to Hopper could be the first real clue that it is at the centre of COPA-based family of products.

Of course, it might be too early for anything COPA-based.

 

Yes, identical URLs.