GTC 2024
- Thread starter pharma
- Start date
So, basically nVidia has done Ampere/Hopper just with two dies. Impressive. Oh and delivering 10 PFlops FP8 performance within 208 billion transistors is so out of this world. 10 TB/s interconnect means full bandwidth between both dies from the HBM3e.
Better than expected. NVLink with 1.8 TB/s is another surpise.
Better than expected. NVLink with 1.8 TB/s is another surpise.
Any whitepapers yet?
Bondrewd
Veteran
NVIDIA DGX B200 Datasheet
The latest addition to the NVIDIA DGX systems, DGX B200 delivers a unified platform for training, fine-tuning and inferencing in a single solution optimized for enterprise AI workloads, powered by the NVIDIA Blackwell GPU.
resources.nvidia.com
the funny bit is 800G being still MIA (the catastrophe continues).
Architecture white paper usually comes out within 24 hours of the announcement.
NVIDIA Announces New Switches Optimized for Trillion-Parameter GPU Computing and AI Infrastructure
NVIDIA today announced a new wave of networking switches, the X800 series, designed for massive-scale AI.
nvidianews.nvidia.com
DavidGraham
Veteran
GB200 is memory coherent between the Grace CPU and Blackwell GPUs, I think this is an important milestone.
Technical brief: https://nvdam.widen.net/s/xqt56dflgh/nvidia-blackwell-architecture-technical-brief
Going by this Blackwell at 700W delivers 75% more performance over H100/H200. Or one GPU would be around 87.5% of the performance at 350W.
Blackwell supports PCIe 6.0. And it looks like that nVidia has cut back on traditional compute performance (at least for FP64).
Going by this Blackwell at 700W delivers 75% more performance over H100/H200. Or one GPU would be around 87.5% of the performance at 350W.
Blackwell supports PCIe 6.0. And it looks like that nVidia has cut back on traditional compute performance (at least for FP64).
DavidGraham
Veteran
There are 4 versions of Blackwell releasing simultaneously.
B100 (air): 14PF of FP4 at 700w
B200 (air): 18PF of FP4 at 1000w
B200 (water): 20PF of FP4 at 1200w
GB200 (water): 40PF of FP4 at 2700w (1 Grace CPU + 2 B200 GPUs)
Makes sense for an AI chip. I wonder if nV will make a separate server GPU product line for FP64, or if they will just cede the market to AMD. It's a much smaller market than AI, but it's quite prestigious and sees a lot of software investment.
Bondrewd
Veteran
This.
Can't win against the APU.
APU wins so unbelievably hard for people who roll their own codebases.
ML farm is the real battlefield anyway.
B100/200 are priced very aggresively given how much more expensive they are relative to H100.
The market seemed to respond positively to the price announcement after some jitters earlier this morning.
$30-$40K isn’t cheap but Blackwell perf/$ should be significantly more attractive than Hopper. Nvidia is playing their usual game. Get your foot in the door everywhere and make it hard for people to switch ecosystems. Their biggest competition would be CSPs rolling their own hardware and they will try their very best to make that an unattractive option.
Bondrewd
Veteran
Finally, $AMD is down, people can load on it moar.
That's Hopper pricing (hyperscale will pay less as usual) for 2x the Si and 2 moar HBMs.
Margins kaput but hey, competition!
The 'ecosystem' is at large, PyTorch.
If your stuff is in mainline PyTorch and runs good perf out of the box, you're good.
(that doesn't account for truly boutique solutions like Cerebras WSE though).
The real moat is and has always been the hardware.
Ever since V100 NV led the market on matrix math piles at speed.
A100 in particular was kinda a watershed moment for the market at large.
No they're doing a competitive response. lol.
Their biggest competition is MI400. They know it.
Eh, AMD has a long history of producing attractive hardware that goes nowhere. Let’s see them deliver the good first before claiming Nvidia is running scared.
Bondrewd
Veteran
Ughhhh it's been 5 years since Rome.
If there's anything current AMD is good at, it's running a consistent roadmap at all costs.
When it becomes not-consistent, they do a reorg and knife a bunch of parts (which, they did! byebye the og mi350x. and byebye the og horrific Si spam MI400).
Oh they do.
B100 would've been 50k otherwise.
Gotta defend the CSP share at all costs.
Frankly it's just competitive response.
Panic button stuff is the Intel staple (more on that later).
DavidGraham
Veteran
They would rather focus on Sovereign AI, they seem to have a division for custom chips and can make cheaper Grace APUs easily if there is a need for it, but right now there isn't.
Yep, remember MI100? MI250X? Even MI300 is having a rather tame reception compared to H100/H200.
Bondrewd
Veteran
what even
AMD's been running semicustom biz for like 13 years.
that's is NOT an APU. not at all.
Good devkit.
Hard to forget Frontier.
? This is outright cope.
It's a new product from (effectively) a new vendor and a first real engagement from AMD with hyperscale wrt GPGPU stuff and it's doing hella numbers starting end of this Q.
Similar threads
