Benchmark AI_TRUECOST_

real AI cost, not vanity metrics

Follow on X updates & details

What does AI actually cost to ship a ticket?

The problem
Benchmark leaderboards are vanity metrics. They don't show what a model costs, or how it performs inside a real coding harness.
What you get
The exact cost of every harness against every model, run on the same repos and the same tickets.
Why it matters
Save money and time, and pick the best harness for your favorite model.

Select a ticket to load the harness × model cost grid.

Workflow page →
Workflow page →

Swipe to view more

Average cost of passing runs for each harness and model on this ticket. ↓ marks the lowest cost; ↑ marks the highest. Click a price for the run breakdown.
Model codex cursor devin
claude-fable-5 — ↑ $4.99 —
claude-fable-5.1 — $4.37 —
claude-haiku-5.5 — $0.07 —
claude-opus-5 — $4.10 —
claude-opus-5.5 — $1.59 —
claude-sonnet-5 — $1.31 —
claude-sonnet-5.5 — $0.62 —
composer-2.5 — $0.30 —
grok-4.6 — $1.28 —
grok-4.7 — $2.01 —
gpt-5.6-luna $0.07 $0.10 $0.10
gpt-5.6-sol $1.98 $1.92 —
gpt-5.6-terra $0.68 $1.14 $0.75
gpt-6-astra $2.41 — —
gpt-6-luna ↓ $0.04 — —
gpt-6-sol $0.71 — —
gpt-6.1-sol $0.93 — —
swe-2 — — $1.10
deepseek-v4-flash — — $0.08
deepseek-v4-pro — — $0.43
deepseek-v4.1-flash — — $0.08
gemini-3.8-flash — $1.09 —
glm-5.3 — — $0.47
kimi-k2.7-code — $0.45 —
kimi-k3 — $1.08 $1.29
muse-spark-1.3 — $0.46 —