August 25, 2026, (Inside AI) — OpenAI has disclosed the first measured results for its custom inference chip, Jalapeño, showing a 1.5 to 1.9 times increase in AI work per watt at peak throughput compared to leading commercial systems. The chip also cut end-to-end latency by 1.7 to 3.6 times across three public models: GPT‑OSS 120B, DeepSeek R1, and Kimi K2.5 1T.
The announcement marks a shift in the AI hardware race. For years, inference accelerators forced a tradeoff between throughput and latency. Jalapeño appears to break that pattern by delivering both in a single architecture, a feat that could reshape how interactive AI agents are deployed at scale.
OpenAI said the chip is rated at 700 watts, but measured sustained power stayed at or below 550 watts on tested workloads. That efficiency gain matters because AI inference costs are now a dominant factor in the economics of large language models.
Why Agents Expose the Limits of Current Inference Hardware
Agentic workloads are uniquely punishing for inference systems. An agent may need dozens of sequential model calls to complete one task. If each call adds 100 milliseconds of latency, the total delay compounds into seconds of perceived unresponsiveness.
Read: OpenAI and Samsung Deepen Partnership on Next-Generation AI Chips
Jalapeño was designed specifically for this problem. The chip minimizes data movement and communication delays by keeping model state, including the KV cache, local to the compute units that need it. The network is integral to the architecture, allowing an entire workload to remain within one connected system.
Language-model inference has two distinct phases with different bottlenecks. Prefill, when the system processes a prompt, is compute-intensive. Decode, when the system generates tokens one by one, is constrained by memory bandwidth. A system that excels at one phase can lose that advantage while waiting for data to move between cores and chips.
Jalapeño addresses this by explicitly placing model state and activating the right combination of compute, memory, and networking for each phase. The result is a balanced accelerator that can adapt as the balance between prefill and decode changes, a defining feature of agentic workloads.
AI Helped Design the Chip That Runs AI
OpenAI said AI played a direct role in Jalapeño’s development, enabling the team to move from initial design to tapeout in nine months. AI explored implementations, shortened design and verification loops, and optimized arithmetic circuits to fit more compute performance into the chip on schedule.
The company also used its Codex model with GPT‑Astra to bring three open-weight models that were not part of the original production plan to high performance within two months. For selected GPT‑OSS attention and mixture-of-experts blocks, AI-generated implementations ran 1.5 to 1.8 times faster than existing human-expert-written implementations.
Those figures apply only to selected blocks, not the full model, but they point to a powerful new development loop. Engineers can describe work through local tensors, explicit communication, and predictable synchronization. AI can then optimize how that work is mapped, placed, scheduled, and coordinated across the system.
Read: Microsoft Plans Maia 300 AI Chip Unveiling in September, Report Says
OpenAI tested Jalapeño on InferenceX, a public benchmark from SemiAnalysis that measures the full process of serving an AI request. The company compared Jalapeño with leading commercially available AI systems across the tested operating range, from high-throughput serving to highly interactive, low-latency use.
On Kimi K2.5 1T, the largest public model tested, Jalapeño delivered approximately 1.5 times higher peak performance per watt and 3.4 times lower end-to-end latency than the comparison system. In internal testing, the advantage widened further on frontier OpenAI models, suggesting the architecture becomes more valuable as workloads grow larger.
OpenAI plans to begin deploying Jalapeño within its compute infrastructure by the end of the year. The company said Gen 2 is deep in development and Gen 3 is taking shape. Each generation will build on what the team learns and further advance both efficiency and speed.
Meeting growing demand for AI will require more compute from every available source. OpenAI said it will continue to widely deploy accelerators from NVIDIA and other partners for both training and inference workloads. The mission, the company said, is to ensure that artificial general intelligence benefits all of humanity.
