Benchmark AI_TRUECOST_

real AI cost, not vanity metrics

Follow on X updates & details

claude-opus-5.5

Running claude-opus-5.5 inside cursor.

FastAPI: Add POST /api/v1/projects/{project_id}/tasks endpoint Plan → Review plan → Execute → Review & fix → Open PR → Verify PR → Verify e2e

All steps run as one workflow, inside a single Docker container.

Swipe to view more

5 of the 10 steps are agent sessions, each started fresh: only the repository working tree and the previous step's output carry over. Host steps orchestrate and verify, and cost nothing.
# Step Kind What happens
1 pullIssue host Fetches the GitHub issue and creates a branch
2 plan agent Drafts a plan from the ticket, including tests
3 reviewPlan agent Always runs; returns the full final plan
4 execute agent Implements the landing page; may run repeated preview captures; no harness step timeout
5 reviewAndCorrect agent Visual review against the brief and reference render; fixes layout and copy; no harness step timeout
6 openPullRequest agent Commits, pushes, opens a pull request into main
7 verifyPullRequest host Checks that the exact pull request is open
8 verifyPullRequestCheck host Waits for the required end-to-end check to pass
9 summarizeUsage host Totals tokens and cost
10 score host Records the final result
Speed
Reasoning
Run 5 2026-10-08 · standard · high $1.59 PASS

Steps

Swipe to view more

Agent steps start a fresh session. Host steps are included in the run cost.
Step Status Cost Time In tok Out tok Tools
pullIssue succeeded $0.00 — 0 0 0
plan succeeded $0.45 2m 21s 14 12,754 27
reviewPlan succeeded $0.21 1m 04s 14 5,855 24
execute succeeded $0.39 1m 34s 24 7,367 14
reviewAndCorrect succeeded $0.32 1m 13s 14 5,225 8
openPullRequest succeeded $0.21 0m 54s 22 3,380 10
verifyPullRequest succeeded $0.00 0m 01s 0 0 0
verifyPullRequestCheck succeeded $0.00 0m 01s 0 0 0
summarizeUsage succeeded $0.00 — 0 0 0
score succeeded $0.00 — 0 0 0

Run totals

Cost
$1.59
Agent time
7m 07s
Input tokens
88
Output tokens
34,581
Cache-read tokens
1,231,985
Tool calls
83

Tool call mix

read
43
shell
25
edit
9
glob
3
createPlan
2
grep
1
Run 4 2026-10-07 · standard · high $1.58 PASS

Steps

Swipe to view more

Agent steps start a fresh session. Host steps are included in the run cost.
Step Status Cost Time In tok Out tok Tools
pullIssue succeeded $0.00 — 0 0 0
plan succeeded $0.45 2m 19s 14 12,492 27
reviewPlan succeeded $0.21 1m 03s 14 5,817 24
execute succeeded $0.39 1m 34s 24 7,325 14
reviewAndCorrect succeeded $0.32 1m 14s 14 5,161 8
openPullRequest succeeded $0.21 0m 53s 22 3,309 10
verifyPullRequest succeeded $0.00 0m 01s 0 0 0
verifyPullRequestCheck succeeded $0.00 0m 01s 0 0 0
summarizeUsage succeeded $0.00 — 0 0 0
score succeeded $0.00 — 0 0 0

Run totals

Cost
$1.58
Agent time
7m 04s
Input tokens
88
Output tokens
34,104
Cache-read tokens
1,226,107
Tool calls
83

Tool call mix

read
42
shell
28
edit
7
createPlan
3
glob
2
grep
1
Run 3 2026-10-07 · standard · high $1.61 PASS

Steps

Swipe to view more

Agent steps start a fresh session. Host steps are included in the run cost.
Step Status Cost Time In tok Out tok Tools
pullIssue succeeded $0.00 — 0 0 0
plan succeeded $0.46 2m 18s 14 13,016 27
reviewPlan succeeded $0.22 1m 05s 14 5,860 24
execute succeeded $0.39 1m 33s 24 7,528 14
reviewAndCorrect succeeded $0.33 1m 12s 14 5,320 8
openPullRequest succeeded $0.22 0m 54s 22 3,360 10
verifyPullRequest succeeded $0.00 0m 01s 0 0 0
verifyPullRequestCheck succeeded $0.00 0m 00s 0 0 0
summarizeUsage succeeded $0.00 — 0 0 0
score succeeded $0.00 — 0 0 0

Run totals

Cost
$1.61
Agent time
7m 02s
Input tokens
88
Output tokens
35,084
Cache-read tokens
1,239,222
Tool calls
83

Tool call mix

read
42
shell
29
edit
5
createPlan
3
glob
3
grep
1
Run 2 2026-10-07 · standard · high $1.60 PASS

Steps

Swipe to view more

Agent steps start a fresh session. Host steps are included in the run cost.
Step Status Cost Time In tok Out tok Tools
pullIssue succeeded $0.00 — 0 0 0
plan succeeded $0.46 2m 22s 14 12,856 27
reviewPlan succeeded $0.22 1m 05s 14 6,000 24
execute succeeded $0.39 1m 34s 24 7,328 14
reviewAndCorrect succeeded $0.33 1m 13s 14 5,311 8
openPullRequest succeeded $0.21 0m 55s 22 3,402 10
verifyPullRequest succeeded $0.00 0m 01s 0 0 0
verifyPullRequestCheck succeeded $0.00 0m 00s 0 0 0
summarizeUsage succeeded $0.00 — 0 0 0
score succeeded $0.00 — 0 0 0

Run totals

Cost
$1.60
Agent time
7m 08s
Input tokens
88
Output tokens
34,897
Cache-read tokens
1,243,467
Tool calls
83

Tool call mix

read
43
shell
25
edit
10
glob
3
createPlan
1
grep
1
Run 1 2026-10-07 · standard · high $1.58 PASS

Steps

Swipe to view more

Agent steps start a fresh session. Host steps are included in the run cost.
Step Status Cost Time In tok Out tok Tools
pullIssue succeeded $0.00 — 0 0 0
plan succeeded $0.45 2m 20s 14 13,025 27
reviewPlan succeeded $0.22 1m 04s 14 5,886 24
execute succeeded $0.38 1m 34s 24 7,284 14
reviewAndCorrect succeeded $0.32 1m 13s 14 5,133 8
openPullRequest succeeded $0.21 0m 54s 22 3,310 10
verifyPullRequest succeeded $0.00 0m 01s 0 0 0
verifyPullRequestCheck succeeded $0.00 0m 01s 0 0 0
summarizeUsage succeeded $0.00 — 0 0 0
score succeeded $0.00 — 0 0 0

Run totals

Cost
$1.58
Agent time
7m 06s
Input tokens
88
Output tokens
34,638
Cache-read tokens
1,227,558
Tool calls
83

Tool call mix

read
42
shell
24
edit
10
glob
3
createPlan
2
grep
2