In the call with Wall Street analysts, Huang did not break the Nvidia datacenter revenue stream down by application type – HPC simulation and modeling, AI machine learning training and inference, database acceleration (analytics and Spark databases), virtual desktops, and such – but he did say that Nvidia has over 25,000 customers who are using its AI platform for machine learning inference, and probably a similar number have been using it for machine learning training.
That would imply that the AI part of the Nvidia business has a customer set that is perhaps two orders of magnitude larger in terms of customer count (but significantly less in terms of aggregate compute, we think) than its HPC business. There are some truly large GPU-accelerated HPC systems out there in the world, and they have a lot of oomph.
AI is on a hockey stick exponential growth path, and HPC seems to be growing more linearly.
...
Huang and Colette Kress, Nvidia’s chief financial officer, rattled off a bunch of interesting statistics during the call. Let’s just rattle them off for a second, and forgive us for once for using bullets:
- There are over 3 million CUDA application developers and the CUDA development tools have been downloaded over 30 million times in the past decade and a half. Which means people are keeping current.
- The company’s gaming business is going like gangbusters, and only about a quarter of the installed base has moved to the RTX architecture, so there is just a huge wave of business that can be done here. And it isn’t cryptocurrency, either.
- The cryptocurrency-specific GPUs, which do not have their hashing functions crimped, only generated $105 million of revenue in fiscal Q3, so this is not a mining bubble.
- The AI Enterprise software stack is now generally available and is certified to run atop VMware’s vSphere/ESXi server virtualization stack, making it easily consumable by enterprises.
- The Launchpad co-location stacks with partner Equinix are now available in nine different locations, and of course, all the major clouds also have GPU-accelerated gear long-since.
- In networking, demand for the former Mellanox products outstripped supply, with older 200 Gb/sec Spectrum and Quantum InfiniBand switches and ASICs and ConnectX-5 and ConnectX-6 adapter cards, and the company is poised to ramp up sales of 400 Gb/sec Quantum-2 InfiniBand and their companion ConnectX-7 adapter cards and BlueField-3 DPUs.
- No matter how skeptical you might be about GPU-based inference, the TensorRT software stack and the Triton inference server stack are being adopted by the market, and we think this is not necessarily because the GPU offers the best or cheapest inference processing, but because of its compatibility with machine learning training platforms. Nvidia says that inference revenues are growing faster than the overall growth in the datacenter business.
- There are over 700 companies that are evaluating the Omniverse 3D design and digital twin software stack, and since the open beta started in December 2020, it has been downloaded over 70,000 times. Nvidia estimates that there are over 40 million 3D designers in the world, which is, by the way, a much larger installed base than Java or PHP or C/C++ programmers individually and probably on the same order of magnitude collectively across those three languages, just to give you some scale. The Omniverse opportunity is going to accelerate the Nvidia business further, and it will be half hardware and half software.
We have been wondering why Nvidia is not more freaked out by AMD having such big wins with the exascale-class supercomputers in the United States and smaller machines around the globe in HPC centers.
We have spent some time with Huang puzzling out the Omniverse opportunity, and now we understand the scope of the datacenter business. AI, and its extension and merging with HPC and other design technologies and VR/AR adjuncts is just so much more of an opportunity.
There are 300 or so hyperscalers, cloud builders, and service providers who represent more than half of the aggregate compute consumed in the world and probably a little less than half of the revenue. There are somewhere on the order of 50,000 large enterprises that do interesting things that consume most of the other half.
The 3,000 or so HPC centers in the world do interesting things and are solving hard problems, but they represent only a sliver of the compute and money.
......
We have been in the trenches for so long, watching HPC accelerate with GPUs and then AI emerge as a new computing paradigm,
that we didn’t see the hugeness of the Omniverse, which will obliterate such distinctions and which is such a huge opportunity that even Nvidia has trouble getting its arms around it
Huang put it this way: In the next five years, tens of millions of retail outlets will have some form of conversational AI avatar speaking to customers instead of actual human beings, who don’t want a low-paying job. The AI avatar doesn’t get tired or sick, like people do, and it will cost less than people do we suspect. And this population of virtual people will require enormous amounts of hardware and software, and companies will spend money on this just like they did 50 years ago to get rid of file clerks shuffling through file cabinets with manila folders and accountants working with pencil on greenbar in the back office.
No one will talk about it that way, but it is damned well going to happen. Every avatar will have an Omniverse license, and you can bet it will cost less than health insurance for a real person, and the hardware to drive those avatars will cost less than a salary for a real person.