NVIDIA discussion [2024]

  • Thread starter Deleted member 2197
  • Start date
AWS CEO on NVIDIA: "I actually think most will probably be NVIDIA for a long time, because they’re 99% of the workloads today, and so that’s probably not going to change. But, hopefully, Trainium 2 (AWS's custom AI chip) can carve out a good niche where I actually think it’s going to be a great option for many workloads—not all workloads"

https://twitter.com/x/status/1864459680644517983

I think we like to view it as a supplement to NVIDIA's GPUs. NVIDIA has a fantastic product, and their team has done an outstanding job executing. We believe the vast majority of workloads will continue to run on NVIDIA processors for a long time.
Click to expand...

However, customers want choices—options that can offer lower costs. For certain workloads, we think alternative solutions can provide customers with 30–40% cost-performance benefits compared to today's GPU-powered instances. This is a significant win for customers, particularly as they aim to reduce the cost of general AI workloads.
Click to expand...

> As of now, NVIDIA is estimated to hold about 95% market share. Do I agree with that number? I think it's probably even higher. The vast majority of general AI workloads today run on NVIDIA technology. They've undeniably been the leaders in this space.
Click to expand...

 
DavidGraham said:
Elon Musk is going bonkers now. Expanding his Colossus super cluster facility to 1 million NVIDIA GPUs. Colossus has 100K H100s right now, they are currently adding an additional 100K of H200, but now Musk is expanding that to 1 million, unknown whether they will be H100, H200 or GB200.





www.ft.com

Elon Musk plans to expand Colossus AI supercomputer tenfold

Facility in Memphis expected to incorporate more than 1mn GPUs as billionaire’s xAI aims to catch up with rivals
www.ft.com www.ft.com

Meanwhile, Meta just announced it will build a $10 billion artificial intelligence data center in northeast Louisiana, it will most probably be powered by NVIDIA as well.

abcnews.go.com

Meta to build $10 billion AI data center in Louisiana as Elon Musk expands his Tennessee AI facility

Facebook’s parent company Meta says it will build a $10 billion artificial intelligence data center in northeast Louisiana
abcnews.go.com abcnews.go.com
Click to expand...
I am still skeptical any of this will work out. In my line of work I’ve been seeing outlandish requests for electric service (say, 1GW of load ramped up over 5-10 years, essentially impossible on the distribution system). The infrastructure to build megadatacenters doesn’t really exist and I don’t see these companies actually building co-located nuclear plants in the middle of metro areas.
 
Cappuccino said:
I am still skeptical any of this will work out. In my line of work I’ve been seeing outlandish requests for electric service (say, 1GW of load ramped up over 5-10 years, essentially impossible on the distribution system). The infrastructure to build megadatacenters doesn’t really exist and I don’t see these companies actually building co-located nuclear plants in the middle of metro areas.
Click to expand...

They probably don't need to be in a metro area. For example, Google's main datacenter in Taiwan is inside an industrial park near a coastal area (it's not marked on Google Map, by the way).
There is a solar PV plant there (100MW IIRC) and a planned offshore windfarm (more than 1GW in capacity) which will be online in 2025.
 
pcchen said:
They probably don't need to be in a metro area. For example, Google's main datacenter in Taiwan is inside an industrial park near a coastal area (it's not marked on Google Map, by the way).
There is a solar PV plant there (100MW IIRC) and a planned offshore windfarm (more than 1GW in capacity) which will be online in 2025.
Click to expand...
Usually offshore wind is planned to power a lot more than a single data center though.

I specified metro area because that’s where this particular customer wanted to put their site, within 30 minutes of two major cities lol.

It’s not even really about generation, it’s about the ratings of the distribution infrastructure that will ultimately serve the site. They’d have to pursue some sort of transmission-level solution or serve their own load with co-located generation to the tune of roughly a nuclear reactor or two.
 
Discrete attach rate is cratering hard while overall PC shipments are soaring. Integrated GPUs still aren’t that great so OEMs must be selling a lot more non-gaming machines.
 
trinibwoy said:
Discrete attach rate is cratering hard while overall PC shipments are soaring. Integrated GPUs still aren’t that great so OEMs must be selling a lot more non-gaming machines.
Click to expand...
Having spent way too much time in doctor's offices and hospitals lately it was sort of shocking to see how many mini PCs were strapped to the back of monitors. Rolling ones with batteries, at least one in every room, just all the freaking f-word over the place. Just little SFF PCs with a display out hooked up, network, and a bunch of USB stuff for various crap. I get the practibility but the sheer quantity I've been seeing all over makes me think you are very correct.

Don't need a ton of GPU power for what their doing and if they do they can have a couple of souped up rigs for that. Add regular businesses who also use the same type and that makes for a whole lot of cheap PCs with integrated graphics.

BTW-I heard JHH's cousin won Time's 2024 CEO of the year! ;)
 
digitalwanderer said:
Having spent way too much time in doctor's offices and hospitals lately it was sort of shocking to see how many mini PCs were strapped to the back of monitors. Rolling ones with batteries, at least one in every room, just all the freaking f-word over the place. Just little SFF PCs with a display out hooked up, network, and a bunch of USB stuff for various crap. I get the practibility but the sheer quantity I've been seeing all over makes me think you are very correct.

Don't need a ton of GPU power for what their doing and if they do they can have a couple of souped up rigs for that. Add regular businesses who also use the same type and that makes for a whole lot of cheap PCs with integrated graphics.

BTW-I heard JHH's cousin won Time's 2024 CEO of the year! ;)
Click to expand...
The vast majority of PCs we sell are SFF. Used to be all NUCs until Intel got out of the market.

They work fine unless the fan dies. Then they don't say anything and just keep turning it back on after overheat shutdown until the whole computer fries itself.
 
homerdog said:
The vast majority of PCs we sell are SFF. Used to be all NUCs until Intel got out of the market.

They work fine unless the fan dies. Then they don't say anything and just keep turning it back on after overheat shutdown until the whole computer fries itself.
Click to expand...

A friend's company is making these "digital signage" boxes. Unfortunately many customers still like to use Windows boxes because it's easier to customize (although many of these so-called "customization" are just a full screen web page).
It's really much better to just use a box (no matter it's a SFF or something) with no moving parts. It's probably not a huge market but I bet it's still worth doing.
 
pcchen said:
A friend's company is making these "digital signage" boxes. Unfortunately many customers still like to use Windows boxes because it's easier to customize (although many of these so-called "customization" are just a full screen web page).
It's really much better to just use a box (no matter it's a SFF or something) with no moving parts. It's probably not a huge market but I bet it's still worth doing.
Click to expand...
I'm not familiar with fanless computers but that sounds nice.

There is some kind of defect with Intel 8th gen NUCs. The fans are dying like crazy. It may affect some of the 7th and 10th gen as well (they are basically the same), but the 8th gen is killing us. And we sold a lot of them :( Never had a problem with the 6th gen and prior.
 
DavidGraham said:
Jensen is congratulating Sega on their announcement of the next gen of Virtua Fighter on their 30th anniversary, while also reminiscing about the first NV1 chip in collaboration with Sega.

Click to expand...
NVIDIA has been fascinating to watch for the last 20 years. They saw the potential of GPGPU when nobody took it seriously at all. Got CUDA into universities etc. I'd say they have at least another decade of absolute domination.
 
homerdog said:
NVIDIA has been fascinating to watch for the last 20 years. They saw the potential of GPGPU when nobody took it seriously at all. Got CUDA into universities etc. I'd say they have at least another decade of absolute domination.
Click to expand...

Yeah looking back now it’s kinda wild how hard they were pushing CUDA in academia when there was no money in it. Some real crystal ball stuff.

They were pushing gaming stuff just as hard at the same time although for some reason people seem to think they’ve abandoned gaming (funny accusation given their 90% market share). CES will be interesting. Can’t imagine what new thing they can try to push without broader ecosystem support.
 
DavidGraham said:
Jensen is congratulating Sega on their announcement of the next gen of Virtua Fighter on their 30th anniversary, while also reminiscing about the first NV1 chip in collaboration with Sega.

Click to expand...
nVidia and Sega, a partnership made in heaven.

On a different note...

We tested the Nvidia App performance problems — games can run up to 15 percent slower with the app (Updated)

www.tomshardware.com

We tested the Nvidia App performance problems — games can run up to 15 percent slower with the app (Updated)

Maybe it should have stayed in beta a while longer?
www.tomshardware.com www.tomshardware.com
 
Scott_Arm said:
Hardware unboxed had a video. Disabling game filters fixed the performance regression for them.
Click to expand...
Thank you for this. I don't know why, but I've been running into a lot more issues since Nvidia App has come onboard. Like disappearing VRAM and such, I was watching my VRAM tracker in HB2 jump up and down from 6.5GB up to 8GB, and then back to 6.5GB. Which makes no sense, because the upper limit of memory should have always stayed at 8GB of VRAM. But as soon as the upper limit dropped to 6.5GB of VRAM, I would run out of memory and frame rate would croak until I rebooted the game because it would never recover after that.
 
DavidGraham said:
Jensen is congratulating Sega on their announcement of the next gen of Virtua Fighter on their 30th anniversary, while also reminiscing about the first NV1 chip in collaboration with Sega.

Click to expand...
With what he must earn, he could already change his jacket, which he has been with for at least 5 years
 
You must log in or register to reply here.

Similar threads

Cyan
NPU (Neural Processing Unit). AI Hardware Acceleration discussion.
2
Replies
34
Views
4K
pcchen
P
D
GTC 2024
2 3
Replies
56
Views
6K
trinibwoy
T
xpea
Datacenter GPU market supply and demand (2024)
2
Replies
28
Views
3K
Erinyes
E
F
Dedicated gaming GPUs bottom of the priority list?
2
Replies
34
Views
3K
Seanspeed
S
Shortbread
Nvidia is Now The Second-Largest Company in The US Market
Replies
1
Views
3K
Shortbread
Shortbread
Back
Top