Oracle says that the forthcoming Blackwell supercluster with 131,072 Blackwell GPUs is rated at 2.4 zettaflops. That math checks out at FP4 precision, with the B200 rated at 18 petaflops of aggregate oomph on the tensor cores in Blackwell. If you multiply that out, you get 2,359.3 exaflops of FP4 peak, and that rounds up to 2.4 zettaflops. However, divide that by four to use the FP16 precision that most LLM makers want to use if they can, and that is only 589.8 exaflops.
For FP64 performance, which is important for segments of certain AI workloads and for HPC simulation and modeling, the Big Larry cluster coming next year, then the vector cores or tensor cores only deliver 5.24 exaflops of FP64 oomph across that fleet of Blackwell GPUs. That is five times the peak performance of the “Frontier” supercomputer at Oak Ridge National Laboratories and probably two times the peak FP64 performance of the impending “El Capitan” supercomputer at Lawrence Livermore National Laboratory.
Mind you, at 5.24 exaflops, that Oracle “machine” would still count as the largest HPC system in the world, if Oracle would let you rent it all at once. The odds certainly favor Oracle selling this machine in chunks to many people, but with that number of Blackwell allocations all in one place, maybe not. Perhaps there will only be a few customers who get access to the OCI supercluster so they can train their models
...
And a few minutes later, when Ellison was talking again, he added a thought about the power these AI datacenters require.
“Let me say something that’s going to sound really bizarre. Well, you would probably say, well, he says bizarre things all the time, so why is he announcing this one? It must be really bizarre. So we are in the middle of designing a datacenter that’s north of a gigawatt – we found the location and the power for the place. We look at it, and they have already got building permits for three nuclear reactors. These are the small modular nuclear reactors to power the datacenter. This is how crazy it is getting.”