Saved cases

Save a real turn from a thread as a regression case on disk: its history, the model's replies, tool results and what must happen.

When an agent does something right (or you've just fixed something it did wrong), saveCase / save_case turns that turn into a self-contained case directory you can commit next to your code.

Save a case

A case restores a fork point and replays the next user input after it. You say which events the replay must produce.

const dir = mkdtempSync(join(tmpdir(), "cases-"));
const saved = await thread.saveCase("reads-notes", {
  expect: {
    must: [{ type: "tool_result", data: { is_error: false } }],
    expect: [{ type: "turn_completed" }],
  },
  externalEffects: "stub",
  at: point.event_id,
  dir,
});
if (!saved.ok) throw new Error(saved.error.message);
console.log(saved.value.path, saved.value.portable);

thread here is a handle opened with a sandbox, as in Fork. CaseExpectation is imported from threads in Python; in TypeScript it is a plain object.

Options

OptionRequiredWhat it does
nameyesLowercase-kebab name. The case is written to <dir>/<name>/
expect.mustyes, at least oneEvent matchers the replay must produce, or the case fails
expect.expectnoEvent matchers that are only reported
externalEffects / external_effectsyesMust be "stub": a case never makes real calls
atnoThe fork point to restore. Default: the latest one. There must be a user input after it
dirnoParent directory. Default: cases

A matcher is { type, data? }, plus optional seq, actor_kind, epoch, branch_id or critical. type and the listed envelope keys match exactly; data matches as a subset, so { is_error: false } matches any successful tool result. In TypeScript, each must matcher is also checked against the recorded turn when you save, so a case that couldn't pass is refused with invalid_request.

What it writes

cases/reads-notes/
  case.json                  # name, input text, expectations, which scripts to use
  log.threads-ts.jsonl       # the thread's log up to the snapshot
  log.threads-py.jsonl
  expected.threads-ts.json   # the state that log reduces to
  expected.threads-py.json
  model.json                 # the model replies recorded after the snapshot
  stubs.json                 # the recorded results of external calls
  artifacts/                 # files the log refers to

TypeScript also writes sandbox.json with the sandbox reads from that turn. The log is saved once per language, so the same case works for a TypeScript and a Python implementation of the agent.

portable is true when the snapshot came from the in-memory fake sandbox, so the case needs no sandbox provider to replay. It is false when it depends on a real provider such as E2B.

The case fails to save with egress_policy_unsupported if the thread's sandbox can't block network access, since a replay must never reach the outside world.

Using saved cases today

threads doesn't ship a runner for saved cases yet.

A case uses the same file layout as the conformance cases in the threads repo (spec/conformance/), and it is plain JSON and JSONL. Today you can:

  • commit it as a record of a known-good turn and review changes to it in code review;
  • replay the same turn yourself with a scripted model built from model.json;
  • re-run the scenario on a stub fork (Python) and compare against must.
Edit on GitHub

On this page