Saffronv0.1.0 · free to use
GitHub
Gherkin in · zero tokens out

Write tests in plain Gherkin.Pay for AI once.Replay forever.

An AI agent executes each scenario in a real browser on the first run and records a deterministic cache. Every later run replays with plain Playwright at zero tokens, around 100ms per scenario. When the UI changes, the agent heals mid-run and files a reviewable proposal — like snapshot testing for user journeys.

Get startedGitHub
cached adapted failed
$ saffron run
 
GREENSign the guestbook and leave a note    cache · 147ms
YELLOWNote after the standard guests        4/5 seeded ✓
GREENGuest count via step set               cache · 160ms
GREENSuccessful login                       cache · 90ms
GREENLogin with a credentials table         cache · 99ms
GREENFailed login shows an error — ex. 1    cache · 84ms
GREENFailed login shows an error — ex. 2    cache · 91ms
 
6 passed, 1 adapted, 0 failed · 5 AI calls · $0.61
report: .saffron/reports/latest.html

actual output from the bundled example suite · 6/7 replayed from cache · one novel scenario recorded and verified

The rules

Six rules that keep it honest

Saffron's culture is honesty over convenience. The runner refuses to hide costs, refuses to heal assertions, and refuses to change your suite without a commit.

01 · Assertions

Assertions are sacred

No Then is healed by default — enforced mechanically in the runner, not by prompt discipline. Projects may opt in to adaptable mid-scenario checkpoints; the final assertion block of every scenario stays strict under every policy.

02 · States

Three result states

Green — replayed from cache. Gold — recorded or adapted by the agent this run; the proposal awaits review, stamped verified or honestly unverified. Red — failed. No fourth category.

03 · Diagnosis

Diagnose before adapting

The agent decides whether the failure is UI drift or a real application defect. Defects surface as red, not as a silent heal.

04 · Caches

Caches are git artifacts

.saffron/cache lives in your repo and gets reviewed like snapshots. Diffs are readable. Nothing changes on CI without a commit.

05 · Proposals

Verified proposals

Every recording is proof-replayed with zero AI before it is filed. A proposal that cannot replay deterministically never leaves the machine.

06 · Cost

Honest cost reporting

Prompt-cache reads and writes are shown alongside token counts. The number at the bottom of the report is the real bill.

How it works

Record once, replay forever, heal when the UI moves

Every scenario travels one of three paths. The runner picks the path automatically based on cache state and the outcome of the replay.

First run. An AI agent drives Playwright via MCP and records semantic actions — ARIA role and accessible name first, CSS only as a fallback. The cache is verified by a zero-AI replay before it is written.

Every later run. Plain Playwright replays the recorded actions against your app. No agent is loaded, no tokens are spent, and scenarios finish in about a hundred milliseconds each.

Mid-run failure. When a cached step fails, the agent takes over in the same browser via CDP, adapts around the drift, finishes the scenario, and files a proposal. The proposal includes a cache diff and a suggested feature-file edit — nothing is applied without your review.

.feature scenarios parse cache hit no/stale cache zero-AI replay green seeded islands replay zero-AI step fails retries agent heals in-place (CDP) agent records only the gaps candidate cache zero-AI proof replay fails one refine pass re-verify proposal · verified / unverified accept [--propagate | --with-feature-edit] committed cache HTML + JSON report status · AI calls · tokens · cache traffic · cost · adaptations · .feature diffs
Execution model
The .saffron dialect

A superset of Gherkin, one keyword deeper

.saffron is to Gherkin what TypeScript is to JavaScript. Files that end in .saffron add one keyword — StepSet — and expand back to plain Gherkin at parse time so caching, seeding, and healing behave identically.

StepSet: Complete guest information
    Given I am on guest information page
    When I enter guest name "Chathuranga Jayasinghe"
    And I click continue button
    Then I am not on guest information page
 
Scenario: Verify checkout happy path
    Given I am on calendar page
    And I click choose button
    StepSet Complete guest information
    Then I am on checkout page

Define a flow once, invoke it anywhere. A StepSet is a named sequence of steps. Call it from a scenario and Saffron inlines the steps before the runner ever sees them.

Because the expansion happens at parse time, caching, seeding, and healing are unchanged. The step index still sees leaf steps only.

Editing a set makes every caller honestly stale. Callers re-record with the shared inner steps already seeded, so the incremental cost is the diff — not the whole scenario.

Sets are project-wide. Keep application-wide flows — login, cookie banners — in a sets-only library file (convention: features/shared.steps.saffron) and invoke them from any feature. Set names are unique across the project, which also catches the colon typo at an invocation site.

checkout.saffron — what you write StepSet: Complete guest information Given I am on guest information page When I enter guest name “Chathuranga” And I click continue button Then I am not on guest information page Scenario: Verify checkout happy path Given I am on calendar page … (search, choose package) And I click choose button StepSet Complete guest information Then I am on checkout page what Saffron runs Given I am on calendar page … (search, choose package) And I click choose button Given I am on guest information page When I enter guest name “Chathuranga” And I click continue button Then I am not on guest information page ↓ from StepSet Then I am on checkout page the set’s first step is a guard, its last step an exit assertion — every invocation checks itself per-step caches · step index recorded once — seeded everywhere sub-steps keep their exact text, so caching, seeding, healing and propagation work unchanged when you edit a step set… edit the StepSet once, in one place invoking scenarios go stale honest — never silently wrong re-record, mostly seeded only changed steps cost AI green at zero tokens parse-time expansion
Step sets: source vs. what the runner sees
The data contract

Edit values, replay free

Recordings reference your scenario's own data instead of baking it in. Value edits replay at zero tokens; structural changes are honestly stale and re-record — mostly seeded.

Tables

Tables record as references

A credentials table records as <table:username>, a guest list as <table:1:firstName>. Edit the values — replay stays free. Change keys, headers, or row counts — honestly stale, re-recorded.

Doc strings

Doc strings stay editable

A """ block records as <docstring>. Reword the note and the run replays at zero tokens — even a later assertion on that text follows the edit.

Dynamic values

Dates and prices survive

Literal dates near today become {date+1} templates so date pickers work tomorrow. Displayed prices are captured by name and asserted with differs/matches, not baked in as literals.

Step vocabulary

A derived index of every step your project knows

The vocabulary is compiled from your .feature and .saffron files plus the cache. It is never hand-maintained, never stale, and it powers everything that reads or writes Gherkin — humans, AI assistants, and IDEs.

recorded — replays free divergent — same text, different recordings unrecorded — will trigger the agent

Duplicate wording is proven by behavior. When two phrasings record identical actions, Saffron files an automatic rename proposal so the vocabulary stays lean.

.feature / .saffron files the wording — including steps not recorded yet .saffron/cache recorded actions + status per step the step vocabulary a derived index — never hand-maintained, never stale IDE completion & diagnostics humans typing (LSP) saffron mcp AI assistants writing files in your IDE saffron author plain paragraphs in, feature file out duplicate detection identical actions, different wording → rename proposal one index — humans, AI assistants and the runner all speak the same step vocabulary ● recorded — replays free ● divergent — needs a look ○ written, not recorded yet every suggestion shows whether the step is already paid for accepted renames flow back into the files — the vocabulary converges instead of drifting
Step vocabulary sources and consumers
VS Code

VS Code extension

Completion with status badges, hover cards, go-to-definition for step sets, and near-duplicate warnings. Reads the same vocabulary as the runner.

JetBrains

JetBrains via LSP

saffron lsp powers IntelliJ, WebStorm, PyCharm and Rider — including Community editions, via the free LSP4IJ plugin.

MCP

AI assistants via MCP

saffron mcp serves the vocabulary to Claude, Cursor, and any MCP client — assistants reuse your wordings instead of inventing near-duplicates.

Economics

Cost scales with novel steps, not with runs

Real numbers from an internal checkout suite. A new scenario gets cheaper as the suite grows because the agent seeds from every step it has already recorded.

ScenarioCost
Full unseeded recording$2.07
Tuned inner loop$1.16
Seeded recording, 6-step scenario, one novel step$0.15
Every replay after that$0.00

New scenarios get cheaper as the suite grows, because cost scales with novel steps only.

CLI reference

Eight commands, one honest line each

saffron runReplay cached scenarios — cross-browser with --browser firefox|webkit, parallel with --workers N. Record what is missing (--heal-model picks a cheaper model for heals). Fail on assertions.
saffron accept --with-feature-edit --propagateAccept a proposal, apply the suggested feature edit, propagate to callers.
saffron rejectDiscard a proposal without touching the cache or the feature file.
saffron steps --json --snippetsList the vocabulary with status badges; --json exports it, --snippets writes VS Code completion snippets.
saffron authorTurn plain paragraphs into a .saffron file that reuses the current vocabulary.
saffron mcpServe the step vocabulary to AI assistants over MCP (search_steps, list_step_sets).
saffron lspRun the language server for JetBrains and any LSP-capable editor.
saffron reportOpen the latest run report in the browser.
Get started

Five commands to your first zero-token run

npm i -D saffron-ai
npx playwright install chromium
 
npx saffron run            # agent records, files proposals
npx saffron accept --all   # promote proposals to caches
npx saffron run            # 0 tokens, ~100ms per scenario

No step definitions, no glue code. Write plain Gherkin (or .saffron), point saffron.config.json at your app, and run. The agent needs Claude Code auth or an ANTHROPIC_API_KEY on the first run only — replay-only CI needs no AI access at all, runs cross-browser (--browser firefox|webkit), and parallelizes with --workers N.

Saffron is live on npm — free to use, including commercially. The source opens up later: until then the GitHub button points at this site's repository, which also hosts the issue tracker.

Roadmap

What is shipped, what is ahead

Shipped
  • MVP runner
  • Verified proposals
  • Level-1 step reuse
  • History and trends
  • Assertion policy
  • Data vocabulary
  • Step sets
  • IDE integration
  • Cross-browser replay, parallel workers, healModel
  • Public npm release — saffron-ai, free for any use
Ahead
  • Secrets: {env:VAR} tokens — never in files, caches, or reports
  • Open-sourcing the runner
  • LLM-less MCP replay
  • Multi-provider agents
  • Open-core SaaS layer — CLI never requires it