AMD Execution Thread [2023]

[DavidGraham]

As it is, revenue for the current high pricing of video cards isn't sustainable at covid lockdown/crypto mining levels (pricing + volume)

NVIDIA's shipment is around Q2 2019 levels, so not that far off from one of their lowest points, AMD -on the other hand- is far below their lowest point.

Desktop GPU Sales Lowest in Decades: Report

Nvidia continued to dominate in Q2 2023 amid inventory correction, says Jon Peddie Research.

www.tomshardware.com

 

[Kaotik]

NVIDIA's shipment is around Q2 2019 levels, so not that far off from one of their lowest points, AMD -on the other hand- is far below their lowest point.

Desktop GPU Sales Lowest in Decades: Report

Nvidia continued to dominate in Q2 2023 amid inventory correction, says Jon Peddie Research.

www.tomshardware.com

Regardless of where NVIDIA or AMD or anyone is currently last time we were this low was in the early 90s and back then we were on the way up, as in, it had been way lower until then

 

[trinibwoy]

Revenue should go up as you're using the same wafers for more expensive and higher margin parts. At least until demand for AI silicon (NV) or CPU silicon (AMD) dies off.

As it is, revenue for the current high pricing of video cards isn't sustainable at covid lockdown/crypto mining levels (pricing + volume), so it's either greatly reduce prices or ship less cards.

Unlike previous generations, outside of the 4090, no GPU has really been sold out for weeks after launch, everything has been easily obtainable reflecting demand that is significantly less than supply. Usually launch represents the highest demand:supply ratio due to lower supply combined with high demand (anticipation).

Regards,
SB

Yeah meant consumer segment revenue which to your point may not be a priority given demand from other sectors.

Price/performance stagnation is definitely an issue. The other side of it that people don't really talk about is that there really is no need to upgrade especially if you were forced into a higher than usual price bracket during covid. Today if you had $200 you could pick up a 6600, $300 a 3060 12 GB or 6600 XT or $400 gets you a 3060 Ti or 6700 XT. A lot of people already overpaid for cards in those performance tiers in 2021 and 2022 and there isn't anything faster at their usual budget.

Even if there was something faster those cards still play everything just fine at 1080p / 1440p so why upgrade? Diablo IV is the biggest game in a long time and runs well on a potato.

 

[Lurkmass]

Some suggest the guy isn't worth his fame these days, he has done some dirty tricks apparently

Hector Martin (@marcan@treehouse.systems)

Lol, it took 2 weeks for geohot to go from grandiosely announcing he's going to make ML good on AMD hardware with his own code to giving up after running into some bugs. https://geohot.github.io/blog/jekyll/update/2023/05/24/the-tiny-corp-raised-5M.html...

social.treehouse.systems

the tiny corp raised $5.1M

Here we go again. I started another company. The money is in the bank.

geohot.github.io

Apparently he planned on making his own compute driver ...

https://social.treehouse.systems/@python@fosstodon.org/110505386098461905

Why indeed ...

People are giving him too much credit if he can't even figure out that ROCm includes the drivers itself ...

 

[Silent_Buddha]

Being sold out is hardly an indication of anything besides the fact that momentary demand is higher than supply. Most GPUs hasn't been "sold out" after their launch if we're looking at historical data.

It's certainly indicative because it also comes with greatly lowered shipments of consumer GPUs. Supply has been reduced and demand is still significantly below supply.

Regards,
SB

 

[^M^]

With those volumes discrete GPU can't make that much cash for AMD.
I imagine the business wouldn't even be viable if not for the consoles.

 

[Silent_Buddha]

The other factor for the low shipments is clearing out of old inventory. Both RDNA 2 and Ampere are still readily available brand new. That's another factor for neither AMD nor NV being that interested in releasing larger quantities of lower priced current gen. cards.

That's the other reason for why demand for the RDNA 3 and Ada are so low. While the chart above that DavidGraham posted shows the 2080 briefly dipping down for one quarter, demand and shipments for that recovered relatively quickly. Ada OTH dipped down and then continues to dip downwards in the 3rd quarter after release. RDNA 3 dipped WAY down and has basically been flat.

There's just not a compelling reason to increase shipments of current gen GPUs when there's a stockpile of previous gen GPUs still with which the new generation doesn't necessarily compare to favorably enough to to justify Covid/Cryptomining pricing levels.

If either NV or AMD want to ship more consumer GPUs and actually have them sell, they'd need to lower the pricing of their current gen GPUs. And why would they do that if instead they can ship more CPUs (AMD) and AI focused products (NV).

With those volumes discrete GPU can't make that much cash for AMD.
I imagine the business wouldn't even be viable if not for the consoles.

Which is a great incentive for AMD to continue investing into R&D for graphics products even if the current GPU market is tougher for consumer discrete graphics cards.

Regards,
SB

 

[DegustatoR]

It's certainly indicative because it also comes with greatly lowered shipments of consumer GPUs. Supply has been reduced and demand is still significantly below supply.

Regards,
SB

Shipments are (were?) low because of the overshipping of the previous year, they don't necessarily say anything about current demand. This should be looked at through other means I think.

There's just not a compelling reason to increase shipments of current gen GPUs when there's a stockpile of previous gen GPUs still with which the new generation doesn't necessarily compare to favorably enough to to justify Covid/Cryptomining pricing levels.

You got it. All IHVs are waiting on inventory to sell off before resuming "normal" shipments.

If either NV or AMD want to ship more consumer GPUs and actually have them sell, they'd need to lower the pricing of their current gen GPUs.

Current gen GPUs are available at basically the same prices as previous gen did, and the gen prior to that, and even the gen prior to that one. There's nothing new in pricing of current gen of GPUs. From this POV if they want to resume shipments on the scale of your typical pre-covid/mining period (2019?) the only thing which they need is to wait for the inventory to clear out. And I think it's completely unrealistic to expect shipments to return to mining peaks so any comparison with what we saw in 2021/22 would be really misleading here.

 

[TopSpoiler]

AMD seems to be having a little bit of self-contradiction in promoting their AI hardware strategy. They announced the MI200 two years ago as "the world's fastest HPC and AI accelerator", but it failed to gain significant customers outside of the Frontier supercomputer amid the AI boom, which is evident from last quarter's results. Now they've announced the MI300 as if it were their first serious AI accelerator and will have to try to hide the MI200's poor sales until Q4 of this year.

 

[DavidGraham]

According to @CarstenS .. MI300A will deliver ~2500 TFLOPS at FP8 with Sparsity. Knowing this, MI300X should deliver ~3000 TFLOPS FP8 with Sparcity vs ~4000 TFLOPS FP8 with Sparsity for the H100.

https://twitter.com/x/status/1669376538146492416

 

[Father_Murphy]

According to @CarstenS .. MI300A will deliver ~2500 TFLOPS at FP8 with Sparsity. Knowing this, MI300X should deliver ~3000 TFLOPS FP8 with Sparcity vs ~4000 TFLOPS FP8 with Sparsity for the H100.

According to the endnotes of the presentation, the TBP of the MI300X is 750W. So, swapping out the 24 CPU cores for an additional 64 GB HBM3 and 2 more GCDs nets 100W from the MI300A. I wonder what the relative clocks and performance look like.

 

[DavidGraham]

They announced the MI200 two years ago as "the world's fastest HPC and AI accelerator", but it failed to gain significant customers outside of the Frontier supercomputer amid the AI boom

Problem with MI200 is that it was two GPU dies slapped onto one stick, and it appears as two GPUs to the system with two separate memory pools, in simple terms AMD was crossfiring two dies, and calling them MI200. So in reality it's real performance is halved, and in a single GPU vs single GPU, it lost by a mile vs A100 in pure performance. However, MI200 was good for customers who wanted increased compute density with limited room, as you can put more GPUs per rack with the MI200.

If MI300 is repeat of the same story as MI200, then I don't think things would change much, it's important for MI300 to appear as a single GPU to the system, and not as a multitude of GPUs. I don't know if this is indeed the case or not, AMD hasn't shared much of official information yet.

So, swapping out the 24 CPU cores for an additional 64 GB HBM3 and 2 more GCDs nets 100W from the MI300A. I wonder what the relative clocks and performance look like

MI300X is pure GPU, it's rated for 750w indeed. It's an 8 GPU tile part.

MI300A is an APU, with 3 CPU tiles and 6 GPU tiles, it's rated for ~850w?

There are two scenarios here, either MI300X and MI300A share the same clocks, in which case MI300X is going to be the faster version GPU wise, or that both are the same performance, in which case MI300A operates at higher clocks to compensate for the fewer GPU tiles.

 

[Granath]

Problem with MI200 is that it was two GPU dies slapped onto one stick, and it appears as two GPUs to the system with two separate memory pools, in simple terms AMD was crossfiring two dies, and calling them MI200. So in reality it's real performance is halved, and in a single GPU vs single GPU, it lost by a mile vs A100 in pure performance. However, MI200 was good for customers who wanted increased compute density with limited room, as you can put more GPUs per rack with the MI200.

If MI300 is repeat of the same story as MI200, then I don't think things would change much, it's important for MI300 to appear as a single GPU to the system, and not as a multitude of GPUs. I don't know if this is indeed the case or not, AMD hasn't shared much of official information yet.



MI300X is pure GPU, it's rated for 750w indeed. It's an 8 GPU tile part.

MI300A is an APU, with 3 CPU tiles and 6 GPU tiles, it's rated for ~850w?

There are two scenarios here, either MI300X and MI300A share the same clocks, in which case MI300X is going to be the faster version GPU wise, or that both are the same performance, in which case MI300A operates at higher clocks to compensate for the fewer GPU tiles.

Do you know how that in HPC you are using MPI most of the time ? And it was designed for FP64, where it was several Times faster than A100

 

[Deleted member 2197]

With the MI300 launch were any secured customers expected to use the product announced (other than El Capitan)?
There was a news release not to long ago about Microsoft sharing AI development resources but did not see any MS commitment regarding MI300 during presentation or post launch.

Hopefully we will get more speed metrics information soon regarding AI training/inference capability and comparative MI300 performance data in AI training/inference benchmark suites.

 

[TopSpoiler]

Do you know how that in HPC you are using MPI most of the time ? And it was designed for FP64, where it was several Times faster than A100

https://twitter.com/x/status/1552226821219008513

 

[TopSpoiler]

Hopefully we will get more speed metrics information soon regarding AI training/inference capability and comparative MI300 performance data in AI training/inference benchmark suites.

AMD has been intentionally ignoring MLPerf so far, hopefully they're taking a different stance this time.

 

[TopSpoiler]

Lisa Su Reaffirms Commitment To Improving AMD ROCm Support, Engaging The Community - Phoronix

www.phoronix.com

 

[CarstenS]

If MI300 is repeat of the same story as MI200, then I don't think things would change much, it's important for MI300 to appear as a single GPU to the system, and not as a multitude of GPUs. I don't know if this is indeed the case or not, AMD hasn't shared much of official information yet.

MI300 will appear as one device to the system and every XCD will have full speed access to all memory stacks. That's what AMD told me (article in german), when I asked.

With the MI300 launch were any secured customers expected to use the product announced (other than El Capitan)?

Yes, Atos will be building a computer for german Max-Planck-Institut in Garching near Munich: https://atos.net/en/2023/press-rele...ys-new-bullsequana-xh3000-based-supercomputer

 

[del42sa]

looks like magic refresh isn´t happening .....

it really amazes me, how AMD, whille being so successful in CPU business, is able produce such a failure in gaming GPU´s ? for example Fiji, a gigantic GPU with HBM on interposer and yet slow, then VEGA 64 with HBM2, running hot, power hungry and slower than competition and after quite sucessful RDNA2 generation followed with "chiplet" hype AMD pulled out another "Vega II" aka RDNA3 ... In light of these facts I must conclude that I have a very little faith in RDNA4, because if this is all they can do, I suggest they have to try much much harder

 

[del42sa]

AMD Officially Bringing ROCm Support To RDNA 3 GPUs This Fall, Starting With Radeon RX 7900 XTX & Pro W7900