CPUs Regain Prominence in AI Inference
Intel's shift in CPU-GPU ratio for large language model inference is driven by growing reliance on tool calls and multistep reasoning, with CPUs now handling a significant portion of compute, according to industry analysts.
Key points
- Intel notes a shift in CPU-to-GPU ratio from 1:8 to 1:1 in training workloads and 4:1 in agentic deployments.
- CPUs are well-suited for tasks like tool calls, multistep reasoning, and orchestration across small models, changing the compute landscape.
- Industry analysts say this shift is driven by the growing complexity of AI applications and the need for more specialized models.
- The CPU's resurgence in AI inference is expected to have significant implications for the industry, with potential changes in hardware and software development.
The traditional dominance of graphics processing units (GPUs) in large language model (LLM) inference is being challenged by a growing reliance on tool calls, multistep reasoning, and orchestration across small, specialized models. Industry analysts point to Intel's shift in CPU-to-GPU ratio as evidence of this trend, with the ratio moving from 1:8 in training workloads to 1:1 and even 4:1 in agentic deployments.
This shift is driven by the growing complexity of AI applications, which require more specialized models and a greater emphasis on tool calls and multistep reasoning. As a result, central processing units (CPUs) are now handling a significant portion of the compute, rather than just a fraction of the total compute per request.
The implications of this shift are significant, with potential changes in hardware and software development on the horizon. As the industry adapts to this new reality, we can expect to see a greater emphasis on CPU-based serving and a reevaluation of the traditional GPU-centric approach to LLM inference.
Sources
The WireByte editorial team synthesises technology news from multiple primary sources, verifies the facts, and links every source. Articles are produced with AI assistance and reviewed under our editorial policy.