nVIDIA H100 CPU server, train GPT3 in 3 minutes

Nvidia proves it has the best AI GPU in the world.
NVIDIA trained all the data of the GPT-3 model in just 46 hours by using 1,548 GPUs provided by cloud services.
NVIDIA's strengths lie in the platform, network and software developed by NVIDIA.
Despite Nvidia's higher price and powerful performance, it's an option worth considering for those who want the best performance.
V&K demonstrates how to leverage state-of-the-art technology for large-scale computing.
NVIDIA demonstrated a new interface, InfiniBand, which can connect multiple GPUs together to form a huge GPU and achieve near-linear scalability.
In this case, if fifty-eight hundred red cards are combined into one hundred and one red cards, the latter will be twice as fast as the former; adding another hundred will double the speed again.
This is the first time Nvidia has shown that they can achieve near-linear scalability with their highest-end graphics cards and their most important new interface, InfiniBand.