Click bait nonsense. “Server” by itself means nothing and clearly Blackwell servers come in much larger configs than prior gens. An equivalent Blackwell config will not be 10x the price.
NVIDIA discussion [2024]
- Thread starter pharma
- Start date
Click bait nonsense. “Server” by itself means nothing and clearly Blackwell servers come in much larger configs than prior gens. An equivalent Blackwell config will not be 10x the price.
DavidGraham
Veteran
NVIDIA's Data Center GPUs Shipments Crossed Reached Almost 4 Million in 2023, Gained 98% AI Market Dominance
NVIDIA's Data Center & AI GPU shipments hit almost 4 million units in 2024 which is incredible growth for the company.
wccftech.com
Not sure how much to trust that, it feels like someone just dividing public revenue numbers by ASP estimates maybe? NVIDIA had full-year datacenter revenue of $47.5B, but that's "end of january 2023 to end of january 2024" rather than "2023". And it includes Mellanox/networking which is a large amount of money as well. And H100 80GiB probably had >2x ASP of A100 40GiB. So it kinda-sorta tracks but doesn't say anything very useful if it is even real data imo.
Also, AMD/Intel are meant to be 50K and 40K respectively, not 500K and 400K, for it to add up. That's a pretty big typo...
Also, AMD/Intel are meant to be 50K and 40K respectively, not 500K and 400K, for it to add up. That's a pretty big typo...
DavidGraham
Veteran
Latest ML4.0 Benchmarks, NVIDIA is far ahead of the competition (Gaudi 2 and TPUv5p).
In GPT3 for example:
512 H100: 50 minutes
512 Gaudi 2 : 132 minutes (normalized)
512 TPUv5p: 144 minutes
IIama 70b:
8 H100: 29 minutes
8 Gaudi 2: 78 minutes
AMD is missing for the third time. No idea when they will post their results for MI300x.
NVIDIA is also talking about a 100k Hopper factory in 2024 and a 300k Blackwell GPU factory in 2025.
In GPT3 for example:
512 H100: 50 minutes
512 Gaudi 2 : 132 minutes (normalized)
512 TPUv5p: 144 minutes
IIama 70b:
8 H100: 29 minutes
8 Gaudi 2: 78 minutes
AMD is missing for the third time. No idea when they will post their results for MI300x.
NVIDIA is also talking about a 100k Hopper factory in 2024 and a 300k Blackwell GPU factory in 2025.
Last edited:
https://www.nvidia.com/en-us/events/siggraph/
https://videocardz.com/newz/nvidia-ceo-to-deliver-a-keynote-at-siggraph-2024-on-july-29
Keynote Outline:
1. Introduction
2. Discuss what Nvidia has done with AI
3. Discuss what Nvidia is currently doing with AI
4. Say "the more you buy, the more you save"
5. Discuss the future of AI
6. Ensure AI is said a few more times to raise stock price
7. Conclusion
https://videocardz.com/newz/nvidia-ceo-to-deliver-a-keynote-at-siggraph-2024-on-july-29
Keynote Outline:
1. Introduction
2. Discuss what Nvidia has done with AI
3. Discuss what Nvidia is currently doing with AI
4. Say "the more you buy, the more you save"
5. Discuss the future of AI
6. Ensure AI is said a few more times to raise stock price
7. Conclusion
they already post it , it´s 19% faster than RTX 4090 in OpenCL benchmark , that´s why China wanted all 4090s since it´s release
https://wccftech.com/amd-instinct-m...kbench-opencl-benchmark-faster-than-rtx-4090/
Last edited:
H100 slower than 4090? Why?they already post it , it´s 15% faster than RTX 4090 in OpenCL benchmark , that´s why China wanted all 4090s since it´s release
https://wccftech.com/amd-instinct-m...kbench-opencl-benchmark-faster-than-rtx-4090/
RTX 4090 has more SMs and higher clock rates, if you're not DRAM limited or Tensor Core limited (and don't need FP64 or FP16), it is actually much faster than H100! Also, H100 PCIe only has a 350W TDP (H100 SXM5 has a massive 700W TDP) so it's extremely power limited for many workloads, while RTX 4090 has a 450W TDP so it also wins on that front.
Same reason why I wouldn't put too much weight into MI300X not being that much better in Geekbench OpenCL, it's really not aimed at that kind of workload at all.
DavidGraham
Veteran
Speaking of which, I wanna discuss the difference between H100 and MI300X die size and cost wise
We know MI300X uses about 920mm² of combined TSMC 5nm chiplets (8 compute chiplets) stacked on top of 1460mm² of TSMC 6nm chiplets (4 IO chiplets). Combined this makes for a total silicon area of 2380mm², this is compared to a 814mm² single die of the H100.
https://www.semianalysis.com/p/amd-mi300-taming-the-hype-ai-performance
NVIDIA's transistor footprint is much smaller, but yield is worse, however the H100 is not a full die product, it has about 20% of it disabled to improve yields. So the difference in yields between the two may not be that large.
MI300X costs significantly more due to it's larger overall size, more complex packaging and the need for additional 6nm chiplets.
Any thoughts?
We know MI300X uses about 920mm² of combined TSMC 5nm chiplets (8 compute chiplets) stacked on top of 1460mm² of TSMC 6nm chiplets (4 IO chiplets). Combined this makes for a total silicon area of 2380mm², this is compared to a 814mm² single die of the H100.
https://www.semianalysis.com/p/amd-mi300-taming-the-hype-ai-performance
NVIDIA's transistor footprint is much smaller, but yield is worse, however the H100 is not a full die product, it has about 20% of it disabled to improve yields. So the difference in yields between the two may not be that large.
MI300X costs significantly more due to it's larger overall size, more complex packaging and the need for additional 6nm chiplets.
Any thoughts?
DavidGraham
Veteran
To understand where NVIDIA is going with Blackwell. The next step for AI breakthrough is to massively increase the amount of compute dedicated to a single very large model, train a multi-trillion parameter multimodal transformer with massive amounts of video, image, audio, and text. Then run it on a cluster of a 100K H100/B100/B200, which will be the basis of these large clusters.
www.semianalysis.com
100k H100 Clusters: Power, Network Topology, Ethernet vs InfiniBand, Reliability, Failures, Checkpointing
Frontier Model Scaling Challenges and Requirements, Fault Recovery through Memory Reconstruction, Rack Layouts
www.semianalysis.com
Potato Head
Newcomer
Nvidia just surpassed MS to become the worlds most valuable company.
finance.yahoo.com
Nvidia overtakes Microsoft as most valuable stock in the world
Nvidia is now the largest company by market cap, surpassing Microsoft and completing a stunning rise for the chipmaker over the last two years.
DavidGraham
Veteran
