Reference

Subcommands #

explain, doc, manifest, and list all load and validate the spec first — exit code 2 on a schema error — so any of them doubles as a lint step in CI.

Selecting scenarios #

Selection flags compose with any spec: --filter NAME (repeatable, and comma-separated for OR — --filter a,b or --filter a --filter b runs scenarios whose name contains a or b), --tag T and --skip-tag T (tags match exactly, not by substring — atago list shows the available tags), --parallel N, --fail-fast, and --rerun-failed. atago run --rerun-failed re-runs only the scenarios the previous run recorded as failed in .atago/last-failed.json, so the fix-and-recheck loop replays just the failures instead of the whole suite. While authoring, --verbose traces every command, capture, and assertion verdict — for passing scenarios too. Under --ci, a selection that matches no scenario fails the run (exit 3) rather than passing an empty suite. Flags may be written before, after, or between the paths — atago run ./specs --report json and atago run --report json ./specs are the same command — and -- ends flag parsing, so a path spelled like a flag can still be named. atago record is the exception: everything after its -- is the recorded program's own command line.

Snapshot testing #

snapshot matchers compare output against committed golden files; ANSI colors, temp paths, UUIDs, timestamps, ports, and CRLF are normalized so snapshots stay stable across machines. Record or refresh them with:

atago snapshot update spec.atago.yaml

For volatile patterns the built-ins do not cover — auto-increment IDs, request identifiers, epoch times — declare spec-wide scrub: rules that rewrite each regex match to a placeholder before the compare (applied after secrets: masking):

scrub:
  - {pattern: 'id=\d+', placeholder: 'id=<ID>'}

See the scrub example.

Spec file keys #

Every key a spec file accepts, generated from the committed JSON Schema — the same document that powers editor completion, so this reference cannot drift from what the loader accepts. Indentation shows nesting; a type that links (like step) is documented in its own section rather than repeated inline.

All keys belong to spec format version 1 — the only format version so far; version: "1" is the first line of every spec. The Since column is the atago release that introduced the key (unreleased = merged to main, not yet in a tagged release).

Directory manifest #

A directory of specs may carry an atago.project.yaml holding what belongs to the tree rather than to one file:

env:
  MYTOOL_REGISTRY: "http://127.0.0.1:8080"
defaults:
  run:
    sandbox_home: true
fixtures_dir: testdata

It can also declare the binary under test:

subject:
  name: mytool
  artifact: bin/mytool
  build:
    command: "go build -o ${artifact} ."
    cwd: ".."
profiles:
  cover:
    build:
      command: "go build -cover -covermode=atomic -coverpkg=./... -o ${artifact} ."
      cwd: ".."
    env:
      GOCOVERDIR: "${env:GOCOVERDIR}"

atago run builds it once per invocation, before any scenario, and prepends the artifact's directory to PATH. --profile NAME swaps in that profile's whole build block, cwd included, and layers its env. A failing build — or one that exits 0 without writing ${artifact} — is a run-level error and no scenario executes.

It is discovered by walking up from a spec to the nearest one, so atago run ./e2e and atago run ./e2e/one.atago.yaml resolve the same configuration. Precedence is host < project < suite < scenario < step for env, and a spec file's own defaults: beat the manifest's. fixtures_dir resolves against the manifest's directory and must exist at load time. atago explain prints the manifest that applied and the resolved fixtures directory. Its own schema is atago.project.schema.json.

All four are absolute, because a scenario runs somewhere other than where its spec lives.

Editor support (JSON Schema) #

A JSON Schema lives at schema/atago.schema.json. With the YAML language server you get completion and validation as you type — step types, every matcher, and the ${workdir} / ${env:NAME} / ${name} / $${...} expansion rules. atago init and atago record already emit this header as the first line of every generated spec, so scaffolded specs get completion out of the box. To add it to an existing spec, use the absolute URL (it resolves in any project, unlike a repo-relative path):

# yaml-language-server: $schema=https://raw.githubusercontent.com/nao1215/atago/main/schema/atago.schema.json
version: "1"

The report and manifest outputs have schemas too: report.schema.json and manifest.schema.json.

Platform support #

atago runs on Linux, macOS, and Windows, and CI tests all three: the unit suite on every OS, the self-hosted E2E suite on Linux and macOS, and on Windows both under the native cmd.exe and under a POSIX shell. Almost everything behaves identically. This section is the short list of what does not, and why.

skip: and only: gate on the host as Go names it, so besides linux, darwin and windows they accept freebsd, openbsd and netbsd. Each is a host of its own rather than one BSD family: what FreeBSD ps prints is not what OpenBSD ps prints, so a gate names the one it means. bsd is not a value, and neither is dragonfly, where atago does not build. atago ships no binary for the three it accepts; go install builds one, and a gate for a host you never run on simply never fires.

How a command is split into argv #

Without shell: true, atago splits run.command itself. Both quotes group an argument and are stripped, and \" inside a double-quoted group is a literal quote that does not end the group — on every OS. Every other backslash differs: on Windows it stays literal, because it is a path separator there, so "C:\dir\x", "\\server\share" and "a\\b" arrive unchanged; on Linux and macOS the string goes through go-shellwords, whose escapes are C-style, so \t becomes a tab and C:\dir\x loses its backslashes. An argument that contains quotes or backslashes is therefore best written in single quotes, where nothing is an escape.

One consequence on Windows: a double-quoted path ending in a backslash ("C:\dir\") reads its closing quote as a literal one, so the argument is never closed and atago reports ATG4001 rather than guessing. Write that path in single quotes.

Choosing the shell #

shell: true runs the platform's own interpreter, so a command written for /bin/sh does not run under cmd.exe. Two ways out. Keep the command portable — echo and exit are builtins of both, and run.env:, run.stdin:, run.stdout_to: cover the variable prefixes and redirects a spec usually reaches for a shell to get. Or point atago at the shell you want:

ATAGO_SHELL='C:\Program Files\Git\bin\bash.exe' atago run ./e2e

ATAGO_SHELL takes an absolute path, or a name found on PATH, on either platform. atago picks the calling convention from the name: /S /C for cmd.exe, -c for anything else, which covers the bash that ships with Git for Windows and MSYS2 as well as PowerShell. One caveat when pairing a POSIX shell with Windows paths: ${workdir}, ${specdir}, and ${atago} expand to backslash paths, and a POSIX shell reads a backslash as an escape — so interpolate them into argv-form commands (shell: false) rather than into shell commands.

Shell completion #

atago completion <bash|zsh|fish|powershell> prints a completion script for your shell.

Exit codes #

Ctrl-C/SIGTERM stops the run cleanly: in-flight processes, services, and sessions are torn down, partial results are reported, and the run exits 4.

Errors also carry a diagnostic code such as ATG2201, whose first digit is the exit code above — so ATG2xxx always exits 2. The codes are searchable and stable across rewordings of the message; Error codes lists what each one means, what to change, and which families carry codes today. Assertion failures carry none: exit 1 is a result, not an error. atago explain ATG2201 prints the same entry without a browser, and the JSON report carries the code as a code field on each failure so a dashboard can group by cause.