Fable Cuts Costs with Image-Based Input Tokens
Fable, a GitHub-based AI model, has reduced costs by converting code and bulky context into images, which are then read by the model using OCR. This approach cuts input tokens by up to 70%, resulting in lower bills for users. The innovation is workload-dependent and measured by token reduction, with dollars following as a secondary effect.
Key points
- Fable, a GitHub-based AI model, has developed pxpipe, a local proxy that converts bulky parts of requests into compact PNG images.
- The image-based approach reduces input tokens by up to 70% compared to text-based input, resulting in lower end-to-end bills for users.
- The cost savings are workload-dependent and measured by token reduction, with dollars following as a secondary effect.
- Fable's list prices can change, but the token count remains the same, making tokens the number to watch for cost savings.
- The innovation is expected to benefit users with dense system prompts, tool docs, and history, which can be compressed into compact images.
Fable, a GitHub-based AI model, has introduced a novel approach to reduce costs by converting code and bulky context into images. The model uses OCR to read the images, resulting in a significant reduction in input tokens. According to Fable, the image-based approach cuts input tokens by up to 70%, leading to lower end-to-end bills for users.
The cost savings are workload-dependent and measured by token reduction, with dollars following as a secondary effect. Fable's list prices can change, but the token count remains the same, making tokens the number to watch for cost savings. The innovation is expected to benefit users with dense system prompts, tool docs, and history, which can be compressed into compact images.
The pxpipe local proxy is responsible for rewriting bulky parts of requests into compact PNGs before the request leaves the user's machine. This approach has been measured to reduce input tokens by approximately 59-70% compared to text-based input. The primary result is input-token reduction, with the token count remaining the same even if list prices change.
Sources
The WireByte editorial team synthesises technology news from multiple primary sources, verifies the facts, and links every source. Articles are produced with AI assistance and reviewed under our editorial policy.