Use it in CI

On this page

Real E2E suites flake (timing, ports, external tools). --retry-failed N re-runs failed scenarios in a fresh workdir and reports recovered ones as flaky — green for the exit code, but loud in every report format; silent retries are explicitly a non-goal. --repeat N does the opposite job: run each scenario N times to detect flakiness before it reaches CI.

atago run --ci --retry-failed 2 ./specs          # keep CI green, report instability loudly
atago run --repeat 20 --filter "race prone" ./specs   # flake detection

setup-atago installs a released binary:

name: behavior-specs
on: [push, pull_request]
jobs:
  atago:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: nao1215/setup-atago@v0
      - run: atago run --ci --report gha ./specs

On GitLab CI (or any CI that starts from a container image), use the published GHCR image:

image: ghcr.io/nao1215/atago:latest

stages: [test]

behavior-specs:
  stage: test
  script:
    - atago run --ci --report junit ./specs > junit.xml
  artifacts:
    when: always
    reports:
      junit: junit.xml

The image contains atago and ca-certificates; if your scenarios drive git, jq, a browser, or your own CLI binary, build FROM ghcr.io/nao1215/atago:latest and layer those tools on top.

  • --report json|junit|gha|tap picks the report format; the JSON shape is stable and versioned (sample JSON, JUnit, TAP).
  • --ci enables deterministic, color-free output. It also turns an empty selection into a hard error: a --filter/--tag/--skip-tag that matches no scenario fails the run (exit 3) instead of passing an empty suite, so a typo cannot silently disable your specs. Without --ci the same case is a warning that still exits 0.
  • --artifacts-dir DIR persists the exact payloads a failed assertion compared — plus, for a failed scenario, its background services’ logs and each mock server’s recorded requests — so a failure stays reviewable after the job ends.
  • Environment variable names listed under secrets: are masked as *** in all reports and snapshots.

Review specs without running them #

explain describes what a spec does, doc generates Markdown (with fixtures, expected payloads, and golden files inlined), manifest emits a stable JSON summary for tooling, and list shows scenarios, tags, and artifacts. All of them load and validate the spec first — exit code 2 on a schema error — so any of them doubles as a lint step in CI:

atago explain and doc rendering a spec

atago explain spec.atago.yaml
atago doc --out docs/specs.md ./specs
atago manifest ./specs
atago list ./specs

The real-world pages on this site are atago doc output, committed and drift-tested.