OpenAI Jalapeño Chip 2026: Why It Beats NVIDIA Blackwell
Category: Tech Deep Dives
This analysis was written by the aifreetool Editorial Team — a group of full-time AI-industry researchers and writers who verify every claim against primary sources. Last updated August 27, 2026. We keep no affiliate relationship with the companies covered here.
Quick answer: Yes, OpenAI's Jalapeño inference chip beats NVIDIA's Blackwell generation on the workloads that matter most for running a chatbot. On SemiAnalysis's public InferenceX benchmark, across three open models, Jalapeño delivers 1.5 to 1.9 times more AI work per watt, cuts end-to-end response latency by 1.7 to 3.6 times, and on the highly interactive traffic that drives ChatGPT today, runs 2.1 to 4.1 times faster. The chip is real, third-party-monitored, and built for OpenAI's own data centers — not for sale.
From Design to Tape-Out in Nine Months
On August 25, 2026, OpenAI's hardware lead Richard Ho stood on the stage at Hot Chips at Stanford and put a number next to NVIDIA's name. The chip is called Jalapeño. It was co-designed with Broadcom, announced publicly in October 2025, and fabricated starting in November 2025. By the time it hit the benchmark, it had gone from first design to manufacturing tape-out in roughly nine months.
The reason that timeline matters is that chips traditionally take two to four years from concept to fabrication. OpenAI says its own models helped compress parts of the design work — older models drove chip layout decisions, newer models handled programming and optimization. The team is now deep into a second-generation design, with a third on the drawing board.
What the InferenceX Benchmarks Actually Show
The headline number is the perf-per-watt story, because that is what determines whether a chatbot is cheap to run or bankrupts the company that runs it. The test used SemiAnalysis's public InferenceX suite — a benchmark OpenAI does not control and that SemiAnalysis verified on-site in person during the run.
Three open models were tested: OpenAI's own GPT-OSS 120B, DeepSeek's R1 at 670B parameters, and Moonshot AI's Kimi K2.5 at roughly 1T parameters. On GPT-OSS 120B, Jalapeño hit about 85,448 mixed tokens per second per kilowatt against Blackwell's 44,960 — a 1.9× advantage. On DeepSeek R1, end-to-end latency dropped from 5.99 seconds on NVIDIA's GB300 to 1.65 seconds on Jalapeño, a 3.6× improvement. On Kimi K2.5, the largest of the three, Jalapeño produced roughly 1.5× more tokens per watt and 3.4× lower latency.
The interactive gap is the one OpenAI cares about most. On the back-and-forth traffic that actually loads ChatGPT — short prompts, concurrent users, fast replies — Jalapeño ran 2.1× to 4.1× faster than a comparable Blackwell system. Single-user generation on DeepSeek R1 reached 700 tokens per second versus 169 on a GB300, which is the difference between an instant reply and a noticeable wait.
Key Takeaways: Jalapeño vs Blackwell at a Glance
| Metric (GPT-OSS 120B) | NVIDIA GB300 | OpenAI Jalapeño | Delta |
|---|---|---|---|
| Throughput per kilowatt | 44,960 TPS/kW | 85,448 TPS/kW | 1.9× better |
| End-to-end latency (R1) | 5.99 s | 1.65 s | 3.6× lower |
| Single-user tok/s (R1) | 169 | 700 | 4.1× faster |
| Rated power | 1,200–1,400 W | 700 W (≤550 W sustained) | ~50% less |
| Memory | HBM3e | HBM4 | Gen ahead |
The chip is rated at 700 watts and held sustained power at or below 550 watts during testing. A single Jalapeño rack packs 128 accelerators, 1.7 exaFLOPS of 4-bit compute, and 27.5 TB of HBM4 memory at roughly 2 petabytes per second of bandwidth. The cooling is air, not liquid.
Why OpenAI Built It, and Why It Matters Now
Richard Ho, OpenAI's VP of hardware, framed it bluntly: "The bottom line is that the results show a very, very significant performance advance over state of the art." Sam Altman summed it up on X the same day: "we made a chip and it is fast".
The bigger story is the timing. Six days before the Hot Chips talk, Anthropic confirmed it had hired Amir Salek, a co-founder of Google's TPU program, to lead its own in-house silicon effort. That makes two frontier labs with custom inference hardware in production in the same week. Add Google TPUs, Amazon Trainium, Microsoft Maia, Apple silicon, and Meta's MTIA, and every major AI customer now builds at least some of its own accelerator. NVIDIA's moat was never that its chips were unbeatable. It was that no one had the engineering depth to compete at the same time. That excuse is gone.
OpenAI CFO Sarah Friar was clear that Jalapeño complements rather than replaces its existing compute stack. NVIDIA, AMD, AWS, Cerebras, and CoreWeave stay as partners. The chip plugs into the same Stargate data-center buildout announced last year, with small-volume deployment by the end of 2026 and a broader rollout through 2027. Browse the AI model landscape on aifreetool to see where this fits in the broader stack.
The Catch: Vera Rubin, CUDA, and a 2027 Rollout
SemiAnalysis pushed back on OpenAI's framing in one specific way. The fairer comparison is not Blackwell but NVIDIA's newer Vera Rubin platform, which also uses HBM4 memory. On output tokens per megawatt, Jalapeño still edges ahead. On total cost per token, including networking and software overhead, the two are roughly even. NVIDIA has shipped larger-model benchmarks on Vera Rubin for DeepSeek V4 Pro and Kimi K3 that have not yet been tested on Jalapeño.
Two more caveats matter for buyers and builders. First, Jalapeño is an inference-only chip. It serves models, it does not train them. OpenAI still needs GPUs for pre-training and fine-tuning. Second, the software moat around CUDA is not gone — it has just become a moving target. OpenAI says its results were achieved without speculative decoding or multi-token prediction, two techniques NVIDIA's submitted runs used. Future benchmarks will need to control for that.
FAQ
Q: Did OpenAI really beat NVIDIA on inference?
A: On the SemiAnalysis InferenceX suite across three open models, yes — 1.5 to 1.9× better tokens per watt, with the gap widening on interactive workloads. The fairer NVIDIA comparison is the not-yet-tested Vera Rubin.
Q: Can anyone buy or rent a Jalapeño?
A: No. Richard Ho said OpenAI is "struggling to have enough" compute for its own needs. The chip will not be sold or offered through the API.
Q: What does this mean for the price of running a chatbot?
A: If other labs replicate the same design philosophy, end-to-end inference cost could drop 30 to 50 percent within two years. NVIDIA's own Vera Rubin pricing is the closest public benchmark.
Q: Is this the end of NVIDIA?
A: No. NVIDIA still owns the training market, the existing software ecosystem, and the next two product cycles. The headline is that the inference side is now genuinely competitive for the first time.
My Take / The Bottom Line
This is the moment the AI compute stack stopped looking like a one-vendor shop. Two frontier labs shipping custom inference silicon in the same week, both benchmarked at 1.5× to 4× advantage on workloads that real users actually run — that is not a leak or a marketing slide. It is the start of a multi-vendor inference market.
What it means for readers of aifreetool: if you build on any of the major closed APIs, expect pricing pressure through 2027. If you train your own models, your NVIDIA bill is still safe — for now. The interesting question is what Anthropic's chip looks like when it shows up at next year's Hot Chips.
