Bondrewd
Veteran
Yea.
No it's a more straightforward dual-issue.
940 has no VOPD (the ISA manual is out).
Tonga I think.
I'm not sure if that's correct given even the monodies have two HWS per die.
AMD CDNA: MI300 & MI400 (Analysis, Speculation and Rumors in 2024)
- Thread starter trinibwoy
- Start date
940 has no VOPD (the ISA manual is out).
Chipsandcheese delivering the goods as usual with their very detailed CDNA 3 write up. Proper dual issue and an upgraded scheduler with 24!! Wavefronts per SIMD to improve chances of two ready instructions per cycle. As the article states though register bandwidth may be a limitation.
Bondrewd
Veteran
I know right.
We're all good friends.
Same rules as Ampere/Ada/Hopper so yea.
AMD and Supermicro: Pioneering AI Solutions
In the constantly evolving landscape of AI and machine learning, the synergy between hardware and software is paramount. Enter AMD and Supermicro, two industry titans who have joined forces to empower organizations in the new world of AI with cutting-edge solutions.
Unleashing the Open-Source Power of AI through AMD Instinct Accelerators and ROCm
At AMD, we believe that AI is the most important thing to impact computing and technology in the past 50 years, and we remain dedicated to driving innovation and collaboration within the AI industry. Through our open software ecosystem, ROCm™, and our cutting-edge AI accelerators, we are...
Multi-node LLM Training on AMD GPUs
Lamini is the enterprise LLM platform for existing software teams to quickly develop and control their own LLMs. Lamini has built-in best practices for specializing LLMs on billions of proprietary documents to improve performance, reduce hallucinations, offer citations, and ensure safety. Lamini...
www.lamini.ai
DavidGraham
Veteran
Looks like MI350 is back on track and is not cancelled.
Bondrewd
Veteran
Not the OG one, no. That one was different XCD n stuff.
Read the poast clearly, not that hard at all!
Hot Chips 2024, Monday 26th August 2024
AMD Instinct MI300X Generative AI Accelerator and Platform Architecture - Alan Smith and Vamsi Krishna Alla, AMD
AMD Instinct MI300X Generative AI Accelerator and Platform Architecture - Alan Smith and Vamsi Krishna Alla, AMD
MI325X is the HBM3E CDNA3 refresh with 288GB and 6TB/s bandwidth with increased compute coming this year.
MI350 is CDNA4 on 3nm coming in 2025. 35x inference improvement(FP4/FP6) compared to CDNA3.
1.5x more memory and 1.2x "AI compute" TFlops compared to B200.
MI400 is CDNA Next coming in 2026.
AMD Computex 2024 stream link
MI350 is CDNA4 on 3nm coming in 2025. 35x inference improvement(FP4/FP6) compared to CDNA3.
1.5x more memory and 1.2x "AI compute" TFlops compared to B200.
MI400 is CDNA Next coming in 2026.
AMD Computex 2024 stream link
Last edited:
AMD’s MI300X Outperforms NVIDIA’s H100 for LLM Inference
www.blog.tensorwave.com
Edit: Tensorwave seems to have chosen the slowest and unoptimized inference engine for H100.
AMD’s MI300X Outperforms NVIDIA’s H100 for LLM Inference
Discover if AMD's MI300X accelerator can outperform NVIDIA's H100 in real-world AI workloads. Early results are in!
www.blog.tensorwave.com
Edit: Tensorwave seems to have chosen the slowest and unoptimized inference engine for H100.
Last edited:
The thing is that NVDA has tripled H100 performance by huge software optimizations since it was announced. AMD is not competing a still target like they did with Intel CPUs...AMD’s MI300X Outperforms NVIDIA’s H100 for LLM Inference
AMD’s MI300X Outperforms NVIDIA’s H100 for LLM Inference
Discover if AMD's MI300X accelerator can outperform NVIDIA's H100 in real-world AI workloads. Early results are in!
Edit: Tensorwave seems to have chosen the slowest and unoptimized inference engine for H100.
Last edited:
AMD’s MI300X Outperforms NVIDIA’s H100 for LLM Inference
AMD’s MI300X Outperforms NVIDIA’s H100 for LLM Inference
Discover if AMD's MI300X accelerator can outperform NVIDIA's H100 in real-world AI workloads. Early results are in!
Edit: Tensorwave seems to have chosen the slowest and unoptimized inference engine for H100.
They’re an AMD shop I think.
Probably
Based on authenticated and published test results only Intel is the second best option according the MLPerf tests.
It's a marketing attempt by Tensorwave to generate sales as they did something similar a few months ago.
Pretty unusual for AMD to miss both MLPerf tests with hardware available. They seem to want to move on pretty quick from MI300x.
It's a marketing attempt by Tensorwave to generate sales as they did something similar a few months ago.
Pretty unusual for AMD to miss both MLPerf tests with hardware available. They seem to want to move on pretty quick from MI300x.
AFAICT, they have a significant ~27% performance gain for 512 H100s (iso-GPU-count) from June 2023 to June 2024, but they are also comparing 2023 to 2024 numbers at 3.2x performance with 3.2x GPUs to claim "perfect scaling" - even though the June 2024 numbers for 3584 H100s wouldn be noticeably higher than that, so they do not actually achieve perfect scaling, but more like ~85% which is pretty standard...
EDIT: ... to bring it back to AMD/MI300X ...
For training, AMD is competing to something "close enough" to a fixed target in my opinion, +27% by SW in a year is nothing amazing in this industry, there have been a lot of optimisations in PyTorch and generic things like Flash Attention 2 which also applies to AMD. I wouldn't be surprised if NVIDIA-specific perf improvements for training were more like 10-15% which is kinda what you'd expect in that timeframe for a new-ish architecture.
For inference, both NVIDIA *and* AMD are extremely fast moving targets, partly due to inference framework improvements, and NVIDIA's unique advantage here is TensorRT which AMD doesn't have an in-house alternative to. But there are 3rd party optimised inference frameworks that can be compared, e.g. MK1 Flywheel which is used in TensorWave's blog post - however comparing that to vllm on NVIDIA is a bit silly. Either you compare the same frameworks (e.g. vllm or MK1 on both NVIDIA and AMD) or you compare the best you can find on AMD (e.g. MK1 Flywheel?) to the best you can find on NVIDIA (e.g. TensorRT). So TensorWave's blog post is definitely not a fair comparison, and there's a bit of a pattern from them of not very rigorous claims imo...
I don't find to much confusing about the statement. "Nearly perfect scaling" and "perfect scaling" are not the same thing, and they also mentioned the 3.2x increase was due to larger scale and software improvements.
Anyway not the thread for this discussion.
AMD MI300X and Nvidia H100 benchmarking in FFT: VkFFT, cuFFT and rocFFT comparison
https://videocardz.com/newz/amd-to-present-neural-texture-block-compression-technology