AMD CDNA: MI300 & MI400 (Analysis, Speculation and Rumors in 2024)

trinibwoy said:

Testing AMD’s Giant MI300X​


https://chipsandcheese.com/2024/06/25/testing-amds-giant-mi300x/
Click to expand...

Chips and Cheese is also comparing the MI300X primarily to the PCIe version of the H100, which is the weakest version of the H100 with the lowest specs
Click to expand...

Chips and Cheese also mentions getting specific help from AMD with its testing, but doesn't appear to have received equivalent input from Nvidia, so there could be some bias in the benchmark results
Click to expand...

The introduction says, "We would also like to thank Elio from NScale who assisted us with optimizing our LLM runs as well as a few folks from AMD who helped with making sure our results were reproducible on other MI300X systems." No mention is made of any consultation with any Nvidia folks, and that suggests this is more of an AMD-sponsored look at the MI300X
Click to expand...

www.tomshardware.com

AMD MI300X performance compared with Nvidia H100 — low-level benchmarks testing cache, latency, inference, and more show strong results for single GPUs

MI300X has a very strong architecture, based on testing by Chips and Cheese.
www.tomshardware.com www.tomshardware.com
 
DavidGraham said:
www.tomshardware.com

AMD MI300X performance compared with Nvidia H100 — low-level benchmarks testing cache, latency, inference, and more show strong results for single GPUs

MI300X has a very strong architecture, based on testing by Chips and Cheese.
www.tomshardware.com www.tomshardware.com
Click to expand...
I got the same feeling that TomsHardware did when I read the C&C article. Similar to the performance hpc tit-tat between AMD and Nvidia a few months ago with optimizations only for one side. I think AMD sponsored a couple more tests earlier this year with AI partners using similar optimizations.

If there was any credibility to these claims why avoid MLPerf or the recent AMD refusal of Tiny Corp using MI300x in MLPerf testing?
Can't blame marketing for these decisions ...
 
pharma said:
I got the same feeling that TomsHardware did when I read the C&C article. Similar to the performance hpc tit-tat between AMD and Nvidia a few months ago with optimizations only for one side. I think AMD sponsored a couple more tests earlier this year with AI partners using similar optimizations.

If there was any credibility to these claims why avoid MLPerf or the recent AMD refusal of Tiny Corp using MI300x in MLPerf testing?
Can't blame marketing for these decisions ...
Click to expand...
So you got wrong feeling. They explained that AMD did not provide them any special optimization. Just result validation.
 
pharma said:
I got the same feeling that TomsHardware did when I read the C&C article. Similar to the performance hpc tit-tat between AMD and Nvidia a few months ago with optimizations only for one side. I think AMD sponsored a couple more tests earlier this year with AI partners using similar optimizations.

If there was any credibility to these claims why avoid MLPerf or the recent AMD refusal of Tiny Corp using MI300x in MLPerf testing?
Can't blame marketing for these decisions ...
Click to expand...
The biggest issue I have with this article is that they use the generic vLLM and not TensorRT-LLM or LMDeploy that are much faster on Nvidia accelerators. AMD is clearly behind this article and they are not in a good spot right now with all their recent benchmark shenanigans...

PS: a recent test that compares different LLM inference backends:
www.bentoml.com

Benchmarking LLM Inference Backends

Compare the Llama 3 serving performance with vLLM, LMDeploy, MLC-LLM, TensorRT-LLM, and Hugging Face TGI on BentoCloud.
www.bentoml.com www.bentoml.com
 
Granath said:
So you got wrong feeling. They explained that AMD did not provide them any special optimization. Just result validation.
Click to expand...
It is a bit weird to receive only AMD optimizations from Nscale when they could have provided the same for Nvidia. Granted Nscale is a primary AMD partner and though the do offer access to Nvidia hardware did not offer any optimations for the testing. Could have something to do with increased marketing effort for their huge MI300x purchase earlier this year.

Let's see if C & C provides balanced testing/optimizations with the Nvidia contacts provided to them by TomsHardware.
 
pharma said:
It is a bit weird to receive only AMD optimizations from Nscale when they could have provided the same for Nvidia. Granted Nscale is a primary AMD partner and though the do offer access to Nvidia hardware did not offer any optimations for the testing. Could have something to do with increased marketing effort for their huge MI300x purchase earlier this year.

Let's see if C & C provides balanced testing/optimizations with the Nvidia contacts provided to them by TomsHardware.
Click to expand...
But they said that no sw optimizations were provided. Vanilla vLLM for both vendors.
 
Granath said:
But they said that no sw optimizations were provided. Vanilla vLLM for both vendors.
Click to expand...
"NScale who assisted us with optimizing our LLM runs as well as a few folks from AMD who helped with making sure our results were reproducible on other MI300X systems."

Also, why would you need AMD engineers to help make sure your results were similar to those already published by AMD?
 
Last edited:
pharma said:
"NScale who assisted us with optimizing our LLM runs as well as a few folks from AMD who helped with making sure our results were reproducible on other MI300X systems."

Also, why would you need AMD engineers to help make sure your results were similar to those already published by AMD?
Click to expand...
Guys from AMD only checked results if they were consistent.
 
Granath said:
Guys from AMD only checked results if they were consistent.
Click to expand...
Your speculation or from the article? Why the need for the results to be consistent with AMD published numbers?
What happens in the case the results are not consistent? Independent or managed test results?
 
pharma said:
Your speculation or from the article? Why the need for the results to be consistent with AMD published numbers?
What happens in the case the results are not consistent? Independent or managed test results?
Click to expand...
That's what C&C explained in comments. Seems that word selection in article was unfortunate. But you know it's a small site. Private run, right?
 
You must log in or register to reply here.
Back
Top