Nvidia isn’t the king of every AI hill.
There’s no question that Nvidia‘s (NVDA -1.71%) H100 data center GPU is the fastest AI accelerator on the market. In a series of industry-standard benchmarks that cover a wide variety of workloads including image classification, natural language processing, and large language models, the H100 easily bested the competition. There’s a reason why Nvidia’s GPUs are in such high demand.
Intel‘s (INTC 4.18%) Gaudi 2 AI chip is a worthy competitor. While Gaudi 2 isn’t nearly as fast as Nvidia’s H100 across the same set of benchmarks, it’s a compelling alternative once performance-per-dollar is factored in. Intel saw its AI accelerator pipeline expand by a factor of 6 in the second quarter, largely driven by interest in its Gaudi 2 chip.
Intel beats Nvidia in some cases
It turns out that Intel’s Gaudi 2 chip leaves Nvidia’s H100 in the dust in a particular type of AI workload that makes use of some specialized hardware built into Intel’s AI chip. AI start-up Hugging Face updated a blog post this week that detailed performance benchmarks for Intel’s Gaudi 2 and Nvidia’s H100. The benchmark in question was fine-tuning BridgeTower, a vision-language model that processes both images and natural language.
The results of Hugging Face’s tests throw cold water on the idea that Nvidia is untouchable in the AI accelerator market. The process of fine-tuning a vision-language model typically involves a bunch of work done on the CPU prior to sending data to the AI accelerator. The CPU reads image data, decodes the images, transforms them, and finally sends them to the AI accelerator. The problem is that the AI accelerator may be sitting idle as it waits for data.
Intel’s Gaudi 2 chip features hardware-based decoders, which allows the chip to offload some of this work from the CPU. Hugging Face found that by implementing all of the data-loading optimizations possible for both the Gaudi 2 and the H100, the Gaudi 2 outperformed the H100 by a whopping 41%. Gaudi 2 was also 2.5 times faster than Nvidia’s last-gen A100 GPU.
While Nvidia’s H100 continues to reign supreme when a broad range of AI workloads is considered, it’s hard to argue after this result that Intel’s Gaudi 2 isn’t a serious competitor in the AI accelerator market.
Gaudi 3 is coming next year
While Intel has been killing off products and projects as part of its turnaround effort, its family of Gaudi AI chips has survived. Sometime in 2024, Intel plans to launch its next-generation Gaudi 3 chip. While not much information has been revealed, the chip will reportedly be built on a 5nm process and come with new features, more memory, and greater processing power than Gaudi 2.
After Gaudi 3, Intel’s AI roadmap gets more complicated. Intel also sells its own data center GPUs, which are more general-purpose than its Gaudi chips and can be used for a variety of workloads beyond AI. In 2025, the company plans to launch its Falcon Shores GPUs, which will seemingly merge the two product families and bring Gaudi features to its GPU products. Details are scarce and plans could change, but as of now, it doesn’t look like there will be a direct successor to Gaudi 3.
Despite the uncertainty around Intel’s AI accelerator roadmap, the company’s AI chips are capable alternatives to Nvidia’s dominant data center GPUs. Nvidia may be the undisputed market leader for now, but Intel should not be counted out.
