Benchmark AI_TRUECOST_

real AI cost, not vanity metrics

Follow on X updates & details

Add POST /api/v1/projects/{project_id}/tasks endpoint

The target repo is a FastAPI service that stores projects and the tasks on them. Clients can already create a project and list its tasks.

This ticket adds POST /api/v1/projects/{project_id}/tasks so a client can create a task on an existing project.

Stack it runs on

Language
Python 3.13
API
FastAPI + Uvicorn
Validation
Pydantic v2
ORM / DB
SQLAlchemy 2.0 + SQLite
Package / runtime
uv
Lint / types / tests
Ruff, ty, pytest
CI
GitHub Actions

Build a marketing landing page from a design spec

The target repo is an Astro static site whose home page is an empty shell.

The run builds the whole page from three files in `brief/`: `copy.md` for the content and section order, `DESIGN.md` for the design, and `reference.png` for the intended result.

Nothing is missing: every value the spec names is already a Tailwind utility, the three font families are self-hosted, and `npm run preview` renders the page so the agent can look at its own work.

Stack it runs on

Language
TypeScript
Framework
Astro 7
Styling
Tailwind CSS v4
Fonts
Inter + Plus Jakarta Sans (self-hosted)
Package / runtime
npm + Node 22
Capture
Playwright
CI
GitHub Actions

All steps run as one workflow, inside a single Docker container.

Swipe to view more

5 of the 10 steps are agent sessions, each started fresh: only the repository working tree and the previous step's output carry over. Host steps orchestrate and verify, and cost nothing.
# Step Kind What happens
1 pullIssue host Fetches the GitHub issue and creates a branch
2 plan agent Drafts a plan from the ticket, including tests
3 reviewPlan agent Always runs; returns the full final plan
4 execute agent Implements the landing page; may run repeated preview captures; no harness step timeout
5 reviewAndCorrect agent Visual review against the brief and reference render; fixes layout and copy; no harness step timeout
6 openPullRequest agent Commits, pushes, opens a pull request into main
7 verifyPullRequest host Checks that the exact pull request is open
8 verifyPullRequestCheck host Waits for the required end-to-end check to pass
9 summarizeUsage host Totals tokens and cost
10 score host Records the final result

How a run is scored

A run passes when the workflow succeeded, the pull request is open, and the required end-to-end check passed on CI.

What this ticket requires

  • 404 for an unknown project, 409 for a duplicate external_id, 422 for an invalid body
  • The created task survives a restart
  • Unit and integration tests for the new endpoint
  • No edits to the e2e acceptance suite or the CI workflows

What this ticket requires

  • Every section of the copy deck is present, in that order, with that copy
  • Layout, color roles, typography, spacing, and elevation follow the design spec and match the 1440px reference render
  • Responsive behavior matches the breakpoint rules in the design spec
  • Token-driven styling only: no raw hex colors, one-off pixel type sizes, or a second radius scale
  • No font CDN; the self-hosted families stay
  • npm ci && npm run build succeeds
  • No edits to the brief, the capture scripts, or the CI workflows