Intel released its newest artificial intelligence chip called Gaudi 3 introduced. Intel Gaudi 3 promises significant performance and efficiency improvements over the NVIDIA H100 GPU.
Intel Gaudi 3 Features
According to Intel, the Gaudi 3 is twice as energy efficient as the H100 and can perform AI tasks one and a half times faster. This improved efficiency could help reduce the prohibitive costs associated with training large AI models. The chip also comes in different modules, including an 8-chip configuration on the motherboard and a board compatible with current Intel server designs.
Intel says it tested the chip in popular AI models from Meta’s Llama 2 and Falcon projects and showed that Gaudi 3 is capable of training transformers and running complex applications.
According to Intel benchmarks, Gaudi 3 is on average 50% faster at training large language models such as Llama with 7 billion and 13 billion parameters, and GPT-3 with 175 billion parameters. Additionally, Gaudi 3 is expected to achieve 40% better power efficiency than the H100 in terms of inference.
Compared to NVIDIA H100, Intel Gaudi 3 is predicted to offer an average of 50% faster training time on Llama2 models with 7B and 13B parameters and the GPT-3 175B parameter model. Additionally, Gaudi 3 accelerator inference efficiency is predicted to outperform H100 by 501% on average for Llama 7B and 70B parameters and Falcon 180B parameter models, and 40% for inference power efficiency average2.
Gaudi 3 connects to its main CPU via a high-speed PCIe Gen 5 x16 link. Intel is also developing liquid-cooled models for higher performance.
Intel states that Gaudi 3 will be available to OEMs including Dell Technologies, HPE, Lenovo and Supermicro in the second quarter of 2024.
