Nvidia and AMD Lead Shift to AI Inference Market
The AI infrastructure market is shifting towards inference, with Bloomberg Intelligence projecting a 32% annual compound growth rate through 2032, reaching $1.3 trillion. Nvidia and AMD are positioning themselves for growth in this market with strategic acquisitions and technologies.
Key points
- Nvidia acquired Groq, a company specializing in language processing units (LPUs), which can reduce latency in AI inference.
- Bloomberg Intelligence projects the AI inference market will grow at a 32% annual compound rate through 2032, reaching $1.3 trillion.
- The AI training market is expected to reach $658 billion over the same period, about half the size of the inference market.
- Nvidia's GPUs, packaged with high-bandwidth memory (HBM), are well-suited for the pre-fill phase of inference, while LPUs excel in the decode phase.
- Advanced Micro Devices (AMD) is also positioning itself for growth in the AI inference market with its own technologies.
The AI infrastructure market is undergoing a significant shift, with inference becoming the primary growth driver. This change is driven by the increasing demand for AI applications in various industries. Bloomberg Intelligence projects that the inference market will grow at a 32% annual compound rate through 2032, reaching $1.3 trillion.
Nvidia and AMD are positioning themselves for growth in this market with strategic acquisitions and technologies. Nvidia acquired Groq, a company specializing in language processing units (LPUs), which can reduce latency in AI inference. LPUs have on-chip SRAM built into them, making them great chips for handling the decode phase of inference.
Meanwhile, Nvidia's GPUs, packaged with high-bandwidth memory (HBM), are well-suited for the pre-fill phase of inference, when the model reads a query. This combination of technologies makes Nvidia an important player in the inference space moving forward.
Advanced Micro Devices (AMD) is also positioning itself for growth in the AI inference market with its own technologies. While AI training is all about raw computing power, inference requires a different set of skills. By focusing on inference, Nvidia and AMD are well-positioned to capture a significant share of the growing market.
Sources
The WireByte editorial team synthesises technology news from multiple primary sources, verifies the facts, and links every source. Articles are produced with AI assistance and reviewed under our editorial policy.