RIP CPU-based AI supercomputers? NVIDIA just announced DGX GH200, an AI supercomputer that combines whooping 256 Grace CPUs & Hopper GPUs in the same package with 144 Terabytes of shared memory! Here's the breakdown

1. Eliminates Traditional CPU-to-GPU Connections GH200 integrates NVIDIA's Arm-based Grace CPU and H100 Tensor Core GPU in the same package This increases bandwidth between GPU and CPU by 7x compared to latest PCIe while reducing power consumption by over 5x

2. Increased Scale and Speed DGX GH200 is the first supercomputer to use Grace Hopper Superchips and the NVIDIA NVLink Switch System, allowing them to work together as one. Allowing simplicity of programming a single GPU This increases 48x more than previous generation

3. Support for Larger AI Models With current pace of AI model growth in both size and complexity, they require powerful infrastructure that can scale to meet these demands. This is where DGX GH200 w/ 1 Exaflop & 144 Terabytes of Shared Memory comes in.

4. More Efficient AI Training Training large AI models is super resource and time consuming tasks. With DGX GH200's ability to work with Terabytes of datasets, developers can conduct research at larger scale and much faster speeds.

Press Release: Blog: NVIDIA Keynote:

If you enjoyed this tweet, please consider to 1. Follow me @minchoi for more AI contents 2. Like, Retweet & Reply on first tweet below

I understand Nvidia wants to cash in on this bonanza, but it may well be the real gating factor to better models is not bigger hardware, but better training data.

💯Quality data should be a focus

Whenever was a CPU based AI computer a thing? It has always been about the GPUs, even in large supercomputers.

what will the price range be ?

@JedStack They didn't announce it yet but won't be cheap considering 8x H100s on a carrier board goes for $200K