Benchmark AI_TRUECOST_

real AI cost, not vanity metrics

Follow on X updates & details

gpt-5.6-terra

Running gpt-5.6-terra inside cursor.

FastAPI: Add POST /api/v1/projects/{project_id}/tasks endpoint Plan → Review plan → Execute → Review & fix → Open PR → Verify PR → Verify e2e

All steps run as one workflow, inside a single Docker container.

Swipe to view more

5 of the 10 steps are agent sessions, each started fresh: only the repository working tree and the previous step's output carry over. Host steps orchestrate and verify, and cost nothing.
# Step Kind What happens
1 pullIssue host Fetches the GitHub issue and creates a branch
2 plan agent Drafts a plan from the ticket, including tests
3 reviewPlan agent Always runs; returns the full final plan
4 execute agent Implements the landing page; may run repeated preview captures; no harness step timeout
5 reviewAndCorrect agent Visual review against the brief and reference render; fixes layout and copy; no harness step timeout
6 openPullRequest agent Commits, pushes, opens a pull request into main
7 verifyPullRequest host Checks that the exact pull request is open
8 verifyPullRequestCheck host Waits for the required end-to-end check to pass
9 summarizeUsage host Totals tokens and cost
10 score host Records the final result
Speed
Reasoning
Run 5 2026-09-01 · standard · high $1.42 PASS

Steps

Swipe to view more

Agent steps start a fresh session. Host steps are included in the run cost.
Step Status Cost Time In tok Out tok Tools
pullIssue succeeded $0.00 — 0 0 0
plan succeeded $0.24 1m 16s 21 5,356 35
reviewPlan succeeded $0.14 1m 18s 11 2,910 22
execute succeeded $0.38 1m 57s 74 8,343 50
reviewAndCorrect succeeded $0.45 3m 08s 69 10,061 66
openPullRequest succeeded $0.21 1m 00s 34 3,939 16
verifyPullRequest succeeded $0.00 0m 01s 0 0 0
verifyPullRequestCheck succeeded $0.00 0m 26s 0 0 0
summarizeUsage succeeded $0.00 — 0 0 0
score succeeded $0.00 — 0 0 0

Run totals

Cost
$1.42
Agent time
8m 39s
Input tokens
209
Output tokens
30,609
Cache-read tokens
2,332,680
Tool calls
189

Tool call mix

read
76
shell
58
edit
17
glob
15
grep
11
task
8
createPlan
3
getMcpTools
1
Run 5 2026-09-01 · standard · medium $1.05 PASS

Steps

Swipe to view more

Agent steps start a fresh session. Host steps are included in the run cost.
Step Status Cost Time In tok Out tok Tools
pullIssue succeeded $0.00 — 0 0 0
plan succeeded $0.18 0m 53s 19 3,304 39
reviewPlan succeeded $0.19 0m 48s 30 3,492 34
execute succeeded $0.26 1m 18s 49 5,892 37
reviewAndCorrect succeeded $0.22 1m 04s 36 3,798 41
openPullRequest succeeded $0.19 0m 52s 36 2,850 18
verifyPullRequest succeeded $0.00 0m 01s 0 0 0
verifyPullRequestCheck succeeded $0.00 0m 24s 0 0 0
summarizeUsage succeeded $0.00 — 0 0 0
score succeeded $0.00 — 0 0 0

Run totals

Cost
$1.05
Agent time
4m 55s
Input tokens
170
Output tokens
19,336
Cache-read tokens
1,568,338
Tool calls
169

Tool call mix

read
76
shell
35
grep
18
glob
17
edit
10
task
6
createPlan
3
await
2
webFetch
2
Run 4 2026-08-31 · standard · high $1.36 PASS

Steps

Swipe to view more

Agent steps start a fresh session. Host steps are included in the run cost.
Step Status Cost Time In tok Out tok Tools
pullIssue succeeded $0.00 — 0 0 0
plan succeeded $0.23 1m 05s 23 5,173 39
reviewPlan succeeded $0.13 1m 09s 11 2,806 22
execute succeeded $0.37 1m 43s 70 8,170 45
reviewAndCorrect succeeded $0.43 2m 39s 65 9,690 68
openPullRequest succeeded $0.20 0m 50s 31 3,801 19
verifyPullRequest succeeded $0.00 0m 00s 0 0 0
verifyPullRequestCheck succeeded $0.00 0m 24s 0 0 0
summarizeUsage succeeded $0.00 — 0 0 0
score succeeded $0.00 — 0 0 0

Run totals

Cost
$1.36
Agent time
7m 26s
Input tokens
200
Output tokens
29,640
Cache-read tokens
2,236,542
Tool calls
193

Tool call mix

read
79
shell
60
glob
16
edit
15
grep
11
task
7
createPlan
3
getMcpTools
2
Run 4 2026-08-31 · standard · medium $0.99 PASS

Steps

Swipe to view more

Agent steps start a fresh session. Host steps are included in the run cost.
Step Status Cost Time In tok Out tok Tools
pullIssue succeeded $0.00 — 0 0 0
plan succeeded $0.18 0m 44s 22 3,160 37
reviewPlan succeeded $0.18 0m 42s 26 3,295 35
execute succeeded $0.25 1m 03s 51 5,387 41
reviewAndCorrect succeeded $0.21 1m 00s 36 3,495 38
openPullRequest succeeded $0.18 0m 46s 36 2,636 16
verifyPullRequest succeeded $0.00 0m 01s 0 0 0
verifyPullRequestCheck succeeded $0.00 0m 23s 0 0 0
summarizeUsage succeeded $0.00 — 0 0 0
score succeeded $0.00 — 0 0 0

Run totals

Cost
$0.99
Agent time
4m 15s
Input tokens
171
Output tokens
17,973
Cache-read tokens
1,476,801
Tool calls
167

Tool call mix

read
73
shell
35
grep
19
glob
15
edit
14
task
5
createPlan
3
await
2
webFetch
1
Run 3 2026-08-31 · standard · high $1.27 PASS

Steps

Swipe to view more

Agent steps start a fresh session. Host steps are included in the run cost.
Step Status Cost Time In tok Out tok Tools
pullIssue succeeded $0.00 — 0 0 0
plan succeeded $0.22 1m 12s 22 4,747 35
reviewPlan succeeded $0.12 1m 11s 13 2,442 19
execute succeeded $0.34 1m 55s 74 7,556 41
reviewAndCorrect succeeded $0.40 2m 55s 64 8,520 62
openPullRequest succeeded $0.19 0m 56s 32 3,436 18
verifyPullRequest succeeded $0.00 0m 01s 0 0 0
verifyPullRequestCheck succeeded $0.00 0m 22s 0 0 0
summarizeUsage succeeded $0.00 — 0 0 0
score succeeded $0.00 — 0 0 0

Run totals

Cost
$1.27
Agent time
8m 08s
Input tokens
205
Output tokens
26,701
Cache-read tokens
2,139,202
Tool calls
175

Tool call mix

read
71
shell
55
glob
14
edit
13
grep
10
task
7
createPlan
3
getMcpTools
2
Run 3 2026-08-31 · standard · medium $0.96 PASS

Steps

Swipe to view more

Agent steps start a fresh session. Host steps are included in the run cost.
Step Status Cost Time In tok Out tok Tools
pullIssue succeeded $0.00 — 0 0 0
plan succeeded $0.17 0m 51s 23 3,028 37
reviewPlan succeeded $0.17 0m 45s 28 3,189 30
execute succeeded $0.24 1m 17s 46 5,393 35
reviewAndCorrect succeeded $0.20 1m 06s 31 3,539 35
openPullRequest succeeded $0.17 0m 53s 34 2,569 14
verifyPullRequest succeeded $0.00 0m 00s 0 0 0
verifyPullRequestCheck succeeded $0.00 0m 23s 0 0 0
summarizeUsage succeeded $0.00 — 0 0 0
score succeeded $0.00 — 0 0 0

Run totals

Cost
$0.96
Agent time
4m 52s
Input tokens
162
Output tokens
17,718
Cache-read tokens
1,400,066
Tool calls
151

Tool call mix

read
69
shell
29
grep
16
glob
15
edit
10
task
5
await
3
createPlan
3
webFetch
1
Run 2 2026-08-31 · standard · high $1.19 PASS

Steps

Swipe to view more

Agent steps start a fresh session. Host steps are included in the run cost.
Step Status Cost Time In tok Out tok Tools
pullIssue succeeded $0.00 — 0 0 0
plan succeeded $0.20 1m 03s 23 4,610 32
reviewPlan succeeded $0.12 1m 02s 11 2,438 15
execute succeeded $0.31 1m 39s 74 7,089 40
reviewAndCorrect succeeded $0.38 2m 31s 64 8,564 55
openPullRequest succeeded $0.18 0m 50s 33 3,375 14
verifyPullRequest succeeded $0.00 0m 00s 0 0 0
verifyPullRequestCheck succeeded $0.00 0m 18s 0 0 0
summarizeUsage succeeded $0.00 — 0 0 0
score succeeded $0.00 — 0 0 0

Run totals

Cost
$1.19
Agent time
7m 05s
Input tokens
205
Output tokens
26,076
Cache-read tokens
1,959,961
Tool calls
156

Tool call mix

read
68
shell
51
edit
13
glob
10
grep
7
task
4
createPlan
2
getMcpTools
1
Run 2 2026-08-31 · standard · medium $0.89 PASS

Steps

Swipe to view more

Agent steps start a fresh session. Host steps are included in the run cost.
Step Status Cost Time In tok Out tok Tools
pullIssue succeeded $0.00 — 0 0 0
plan succeeded $0.16 0m 44s 19 2,958 31
reviewPlan succeeded $0.16 0m 36s 30 2,889 25
execute succeeded $0.22 1m 02s 46 4,951 29
reviewAndCorrect succeeded $0.18 0m 55s 34 3,221 31
openPullRequest succeeded $0.17 0m 41s 34 2,290 14
verifyPullRequest succeeded $0.00 0m 01s 0 0 0
verifyPullRequestCheck succeeded $0.00 0m 18s 0 0 0
summarizeUsage succeeded $0.00 — 0 0 0
score succeeded $0.00 — 0 0 0

Run totals

Cost
$0.89
Agent time
3m 58s
Input tokens
163
Output tokens
16,309
Cache-read tokens
1,305,281
Tool calls
130

Tool call mix

read
68
shell
28
glob
10
grep
9
edit
8
await
2
createPlan
2
task
2
webFetch
1
Run 1 2026-08-31 · standard · high $1.28 PASS

Steps

Swipe to view more

Agent steps start a fresh session. Host steps are included in the run cost.
Step Status Cost Time In tok Out tok Tools
pullIssue succeeded $0.00 — 0 0 0
plan succeeded $0.22 1m 08s 21 5,021 34
reviewPlan succeeded $0.12 1m 08s 12 2,594 18
execute succeeded $0.34 1m 44s 72 7,577 43
reviewAndCorrect succeeded $0.40 2m 43s 66 8,933 60
openPullRequest succeeded $0.19 0m 54s 33 3,564 16
verifyPullRequest succeeded $0.00 0m 01s 0 0 0
verifyPullRequestCheck succeeded $0.00 0m 22s 0 0 0
summarizeUsage succeeded $0.00 — 0 0 0
score succeeded $0.00 — 0 0 0

Run totals

Cost
$1.28
Agent time
7m 37s
Input tokens
204
Output tokens
27,689
Cache-read tokens
2,121,051
Tool calls
171

Tool call mix

read
72
shell
56
edit
14
glob
12
grep
8
task
6
createPlan
2
getMcpTools
1
Run 1 2026-08-31 · standard · medium $0.95 PASS

Steps

Swipe to view more

Agent steps start a fresh session. Host steps are included in the run cost.
Step Status Cost Time In tok Out tok Tools
pullIssue succeeded $0.00 — 0 0 0
plan succeeded $0.17 0m 48s 21 3,056 34
reviewPlan succeeded $0.17 0m 42s 27 3,095 30
execute succeeded $0.24 1m 09s 48 5,264 34
reviewAndCorrect succeeded $0.20 1m 00s 33 3,414 34
openPullRequest succeeded $0.17 0m 47s 33 2,511 15
verifyPullRequest succeeded $0.00 0m 01s 0 0 0
verifyPullRequestCheck succeeded $0.00 0m 22s 0 0 0
summarizeUsage succeeded $0.00 — 0 0 0
score succeeded $0.00 — 0 0 0

Run totals

Cost
$0.95
Agent time
4m 27s
Input tokens
162
Output tokens
17,340
Cache-read tokens
1,415,201
Tool calls
147

Tool call mix

read
71
shell
30
grep
14
glob
13
edit
10
task
4
await
2
createPlan
2
webFetch
1