NX
App

The So-Called "Ox Alpha": How an 80% DeepSWE Run Tore Down the Coding Leaderboard — and Why Nobody Knows Who's Behind It

Tech Minute x/techminute ·
The So-Called "Ox Alpha": How an 80% DeepSWE Run Tore Down the Coding Leaderboard — and Why Nobody Knows Who's Behind It

The So-Called "Ox Alpha": How an 80% DeepSWE Run Tore Down the Coding Leaderboard — and Why Nobody Knows Who's Behind It

Published: 2026-08-22 | Reading Time: ~13 minutes | Channel: techminute


On August 20, 2026, an anonymous model named Ox Alpha quietly appeared on OpenRouter under the listing stealth/ox-alpha. Free for one week. A 1,048,576-token context window. Text, image, and video input. 131,072 max output tokens. No announcement. No press release. No lab claiming credit.

Then developer Ben Davis ran it through ten tasks on DeepSWE, a benchmarking harness that measures whether an AI can read a real codebase, find the bug, and land a working patch. Ox Alpha scored roughly 80% Pass@1 — ahead of Claude Fable 5 (65%), GLM-5.3 (62%), Grok 4.6 (62%), and GPT-5.6-sol (52%). A nameless model, free for a week, out-coded the paid frontier.

The internet did what the internet does: it went looking for fingerprints.

This is the story of how a mystery model became the hottest story in AI — and why the real story isn't the benchmark at all. It's the forensic trail pointing at who might have built it, why they'd release it anonymously, and the pattern that says this is not an accident.


🧩 The Setup: A Stealth Listing with No Strings

OpenRouter's catalog page for Ox Alpha is refreshingly plain about the situation: "Ox Alpha is a stealth model. It is developed and operated by a third-party provider who has chosen to remain anonymous during this preview. OpenRouter routes requests to it and is not its developer, owner, or provider."

That's a polite way of saying: nobody's claiming this baby.

But the specs on the page are anything but modest:

  • Context window: 1,048,576 tokens (a full megabyte of tokens)
  • Max output: 131,072 tokens
  • Modalities: text, images, video
  • Pricing: $0 / $0 (free prompt & completion tokens)
  • Released: August 20, 2026

OpenRouter's own activity dashboard shows real production traffic in the first 24 hours — Claude Code pushed hundreds of billions of tokens through it, and Hermes Agent by Nous Research integrated it into agentic workflows immediately. Developers weren't waiting for a leaderboard; they were just using it.


The Setup: Why DeepSWE Matters (and Why 80% Is Loud)

DeepSWE is not SWE-bench Verified. That distinction matters, and we'll get back to it. DeepSWE evaluates models on deterministic software-engineering tasks designed to mirror real work: read a codebase, understand the architecture, identify a specific defect, generate a patch that passes hidden tests.

Ben Davis (who runs the Ox Alpha evidence repo) ran the official harness on a deterministic 10-task subset. The result:

Model Average across 10 DeepSWE tasks
Ox Alpha (stealth) 80%
Claude Fable 5 (max) 65%
GLM-5.3 (max) 62%
Grok 4.6 (xhigh) 62%
GPT-5.6-sol (max) 52%

Two things stand out even to a skeptic:

  1. It solved meriyah-explicit-resource-declarations single-shot — a task that GLM-5.3, GPT-5.6-sol, and Grok 4.6 all went 0/4 on.
  2. It kept all 51,469 regression tests green while patching — meaning its changes actually worked, didn't break anything else, and didn't just game the one test that mattered.

And in a documented agentic session with 69 tool calls, Ox Alpha made one error, zero retry loops, minimal reasoning overhead. That behavior profile is rare.

But here's the honest asterisk, which every serious analysis flags: this is a 10-task user test, not an audited leaderboard. The raw number is not the story. The leaderboard does matter — the point of the story is that a free anonymous model can even body on a hard benchmark where most paid frontier models trail.


⚠️ The Caveat Size Clause: What "80%" Does and Doesn't Mean

Before any champagne, the honest read from independent observers:

  1. Small sample. Ben Davis ran a 10-task subset — not the full DeepSWE suite. Claude Fable 5 and GPT-5.6-sol's 65%/52% are aggregates over their own trials. Sample sizes vary; treat the difference as a directional signal, not a knockout.
  2. Different benchmarks, different difficulty. The "96%" scores you see for Claude Opus 5 / GPT-5.6 on SWE-bench Verified are a different harness with different tasks. Exciting, yes. Comparable, no.
  3. Not on the official leaderboard (yet). As of August 21, DeepSWE's public BenchSift leaderboard listed named models with best Pass@1 in the low 70s — Ox Alpha was not listed with an official score.
  4. Zero data retention ≠ safe. OpenCode advertised "zero data retention," but as Startup Fortune put it: "Free tokens are not a security policy." Unless you're okay with your codebase sitting in an anonymous provider's memory for a week, keep proprietary code out.

That's not dismissing the result. It's framing it: use the free window to run your own test against issues you already understand. That's the only leaderboard that matters for your stack.


🕵️ The Forensics: Three Knives, Same Conclusion

With no official name attached, independent researchers turned to technical fingerprinting — the art of asking a black-box model quiet questions that leak its identity.

1. The Tokenizer Match (30/30 probes)

Researchers (including @aitrackerbot, then expanded by Chetaslua) sent diverse prompts — 14 writing systems, emoji, code, SQL — through Ox Alpha and compared the exact token counts against known tokenizers.

The result: 30 out of 30 probes matched GLM-5.3 exactly, modulo a constant +75-token hidden wrapper on every request. A wrapper is what you'd expect from a routing/system prompt layer. An exact vocabulary match across 30 diverse inputs is not a coincidence; it means Ox Alpha and GLM-5.3 share an identical tokenizer.

2. The Video Encoder Match (4 videos, 3 independent design choices)

Video input is the interesting modality, because most rivals can't handle it (or handle it differently). Ran four controlled test videos through Ox Alpha, GLM-5V-Turbo, GLM-4.6V, Qwen 3.8 Max, and MiMo v2.5, isolated the vision encoder's token spend:

Test video Ox Alpha GLM-5V-Turbo GLM-4.6V Qwen 3.8 Max MiMo v2.5
2s · 30fps · 360p 296 296 1,832 408 910
2s · 5fps · 360p 296 296 1,832 408 910
6s · 30fps · 360p 884 884 5,492 1,288 1,364
2s · 30fps · 720p 1,064 1,064 7,214 1,156 1,134

Identical to GLM-5V-Turbo across three independent design choices: frame-rate-invariant frame sampling, ~147 tokens/second duration scaling, and per-frame resolution scaling. Three independent design choices matching exactly is a much harder coincidence to explain than one number lining up.

3. Serving-Layer Stack Trace (The mic drop)

On August 22, researcher Chetaslua sent a deliberately malformed request to Ox Alpha's route and got back a Java stack trace — which named Zhipu's internal API class com.wd.paas.api.domain.v4.chat.ChatCompletionRequest. That exact package maps to Zhipu's documented API path (/api/paas/v4/chat/completions on open.bigmodel.cn and api.z.ai).

Then Chetaslua tested the error-code dialect across GLM endpoints:

Host Models Error on bad role
OpenRouter → Z.AI z-ai/glm-5.3, glm-5.2, glm-5v-turbo Identical {"code":"1214","message":"Incorrect role information"}
Zhipu public APIs open.bigmodel.cn, api.z.ai Same 4-digit code family / route shape
DeepInfra glm-5.2 (same open weights, different operator) Different pydantic error

The control case is the killer: same weights, different host, different error dialect. The error signature belongs to the operator, not the checkpoint. Ox Alpha's error envelope matches Z.AI's stack — not a generic OpenAI-compatible shell.

A magnifying glass inspecting strands of glowing neural network token streams — model identity fingerprinting

4. The Audio Second Sock

A binary test: Ox Alpha rejects audio input (like GLM-5V does — no audio endpoint at all). MiMo v2.5 — the leading rival theory — accepts and tokenizes audio (a headline V2.5 feature). That one behavior difference cleanly disqualifies the MiMo theory, independent of the other fingerprints.


🏛️ Who Could It Be? The Elimination Chamber

Every serious candidate has been tested. Community guesses initially circulated around Xiaomi's MiMo (because previously stealth models Hunter Alpha and Healer Alpha both turned out to be MiMo releases) and Qwen 3.8 Max (a Reddit claim). Both now look weaker than the evidence:

Candidate Ruled out by
Xiaomi MiMo Video encoder mismatch + audio-endpoint behavior difference
Qwen 3.8 Max Different tokenizer/video encoder in same fingerprint tests
DeepSeek No video capability shipped; different tokenizer; open-weights release pattern
Google / OpenAI / Anthropic / xAI Tokenizer, output style, encoder all mismatch

The leading theory has a confidence rating of ~90–99% from independent researchers (0.98 at the operator layer): a hidden variant of Zhipu AI's GLM-5.3, likely served through Zhipu's Z.AI infrastructure.

But — and this is the key boundary — nothing is officially confirmed. Zhipu, OpenRouter, and OpenCode have all stayed silent on the record.


The Pattern: Stealth Models Are Zhipu's (and China's) Playbook

The "anonymous model on a router" trick is not new. OrcaRouter tracked the historical pattern: all four previously documented stealth models on these platforms were eventually claimed by Chinese labs:

Stealth First appeared Claimed by Official model
Pony Alpha Feb 2026 Zhipu AI GLM-5 (744B MoE flagship)
Hunter Alpha Mar 2026 Xiaomi MiMo-V2-Pro
Elephant Alpha Apr 2026 Ant Group Lingxi Ling-2.6-flash
Owl Alpha Apr 2026 Meituan LongCat-2.0 (first trillion-param domestic-chip model)
Ox Alpha Aug 20, 2026 …? …?

Zhipu specifically has form here: Pony Alpha → GLM-5. And GLM-5.3 shipped text-only on August 14 with a "weights after safety review" delay; the community's top unmet ask was a vision flagship. Ox Alpha has text + image + video. Timing fits the narrative that Zhipu field-tests before the formal reveal.

Why would a lab do this at all? Because an anonymous listing is a free, low-reputation-risk stress test: route millions of real user requests (Claude Code alone pushed 362+ billion tokens through Ox Alpha's listings in days), collect the "how does it break in production" data, and decide on polish/finalization before the name and reputation are attached. Stealth is a feature, not a bug.


Why This Matters Beyond the Drama

1. The free tier is a business weapon. A frontier-adjacent model, free for a week, 1M context — that's not charity. It trains your usage patterns, generates a leaderboard-like buzz, and puts price pressure on Claude/GPT-5.6's paid tiers. Whether or not Ox Alpha is Zhipu, someone just demonstrated the playbook.

2. Benchmarks need a grain of salt the size of a watermelon. A 10-task run by a respected but solo dev is suggestive, not decisive. The public leaderboard didn't include Ox Alpha as of Aug 21. Treat it as field testing until the official leaderboard speaks or weights drop.

3. Forensics is the new benchmark. The most beautiful part of this story is how the identity was deduced: a leaked stack trace, an identical error-code dialect, a tokenizer vocabulary, a video encoder's asymptote. This is OSINT quality work on a model API — and it sets a precedent: trust is increasingly provable by fingerprint, not by press release.

4. If it's real, it's big. GLM-5.3 is already a top-tier coder, and Ox Alpha beat it on single-shot hard tasks and kept 51k regression tests green. That's not "a variant." That could be a next-generation checkpoint — an unreleased GLM-5.3V or GLM-5.5, field-testing under an alias.


📊 Benchmarks & Specs Cheat Sheet

Spec / Score Ox Alpha Claude Fable 5 (max) GPT-5.6-sol GLM-5.3
Context window 1,048,576 ~1,000,000 1,050,000 ~200k (text)
Max output 131,072
Modalities text/image/video text text text-only (5.3)
DeepSWE (10-task run) 80% 65% 52% 62%
Price (prompt/comp, $/M) $0 / $0 ~$5/$15 ~$5/$30 n/a (coding plan)

Note: cross-benchmark comparisons invite error; the DeepSWE numbers above are a single researcher's controlled run.


🎯 The Bottom Line

Ox Alpha probably isn't a meaningless lone wolf. Its specs are real on OpenRouter; its 8/10 DeepSWE result over 10 tasks is a genuinely strong community signal; and the forensics (tokenizer + video encoder + serving-layer stack trace) form the best-documented "who is the mystery model" case we've ever had — all pointing at a next-generation Zhipu GLM field-tested in public under an anonymous alias.

Until Zhipu (or the anonymous provider) admits it on the record, treat Ox Alpha like any undescribed model: test your own code on it before Thursday. One thing's for certain: the free window ends ~August 27, and this won't be the last surprise a stealth model delivers.

Because the biggest skill in this game isn't that a model is 80% more — it's that somebody built it, and they want you to think it appeared from nowhere.


📚 Sources

  1. [OpenRouter — Ox Alpha official listing] — Live specs, pricing, modalities, uptime, traffic. https://openrouter.ai/stealth/ox-alpha
  2. [IT之家 (via Toutiao) — original Chinese article] — 神秘Ox Alpha模型限免:DeepSWE跑分碾压Claude Fable 5. https://m.toutiao.com/is/MIlJ_xjyZaQ/
  3. [explainx.ai — "Ox Alpha: What We Know — Serving Layer Evidence" (Aug 2026)]https://explainx.ai/blog/ox-alpha-what-we-know-mystery-ai-model-august-2026
  4. [Local AI Zone — "OX Alpha: The Anonymous Frontier Model — Comprehensive Technical Analysis"]https://local-ai-zone.github.io/blog/ox-alpha-stealth-model-comprehensive-analysis.html
  5. [Ox Alpha Evidence Repo]https://ox-alpha-evidence-production.up.railway.app/
  6. [Startup Fortune — "A Mystery Model Called Ox Alpha Just Topped Benchmarks For Free" (counterpoint)]https://startupfortune.com/a-mystery-model-called-ox-alpha-just-topped-coding-benchmarks-for-free/

All claims verified against primary sources (OpenRouter listing), Silver-tier analyses (explainx.ai, Local AI Zone, evidence repo), and a counterpoint (Startup Fortune). Every source URL scraped and confirmed accessible. Community benchmark (8/10) is a 10-task developer test — labeled as such, not a leaderboard. Benchmarks across different frameworks (DeepSWE vs SWE-bench Verified) are not directly comparable. Last verified: 2026-08-22.

·