Nvidia introduced the H200 on November 13, 2023, which is a cutting-edge graphics processing unit (GPU) tailored for training and deploying the advanced artificial intelligence models driving the current generative AI revolution. This new GPU represents an upgrade from its predecessor, the H100, utilized by OpenAI to train its most sophisticated language model, GPT-4. A limited supply of these chips has triggered intense competition among major corporations, startups, and government agencies.
According to an estimate from Raymond James, the cost of H100 chips ranges between $25,000 and $40,000, and a substantial number of these chips must be combined to facilitate the training of the largest models, a process known as "training." The impact of Nvidia's AI GPUs on the market has resulted in a remarkable surge in the company's stock value, increasing by over 230% thus far in 2023. Nvidia envisions generating approximately $16 billion in revenue for its fiscal third quarter, reflecting a staggering 170% growth compared to the previous year. The H200 ushers in a key advancement by incorporating 141GB of cutting-edge "HBM3" memory, bolstering the chip's performance during "inference." Inference involves deploying a trained model to generate text, images, or predictions.
Nvidia claims that the H200 can generate output nearly twice as quickly as its predecessor, the H100, as evidenced by a test utilizing Meta's Llama 2 LLM. Set to be released in the second quarter of 2024, the H200 will rival AMD's MI300X GPU, which shares similar attributes and also boasts enhanced memory capabilities to accommodate large models for inference.
A notable advantage of the H200 is its compatibility with the H100, allowing AI companies currently employing the prior model to seamlessly integrate the new version without the need for server system or software changes. The H200 can be incorporated into four-GPU or eight-GPU server configurations through Nvidia's HGX complete systems, and there is also a chip, known as GH200, that combines the H200 GPU with an Arm-based processor. However, the H200's reign as Nvidia's fastest AI chip may be temporary.
While Nvidia offers various chip configurations, significant advancements typically occur every two years when manufacturers adopt a new architecture, unlocking substantial performance gains beyond mere memory increases or other smaller optimizations. Both the H100 and H200 are based on Nvidia's Hopper architecture. In October, Nvidia indicated to investors that it would transition to a one-year release pattern instead of a two-year architecture cadence due to the high demand for its GPUs. The company shared a slide suggesting the announcement and release of its B100 chip, based on the forthcoming Blackwell architecture, in 2024.
Article Source: https://www.cnbc.com/2023/11/13/nvidia-unveils-h200-its-newest-high-end-chip-for-training-ai-models.html
