Real CLIs tested with atago
These real programs run their end-to-end suites on atago; most were migrated from ShellSpec. The generated behavior docs live in doc/e2e/.
Every suite here was written by the atago project on its own initiative and runs in atago’s own CI — the point is to exercise atago against real programs of every shape. The third-party projects listed below are not affiliated with atago, did not commission these tests, and these specs are not their official test suites.
Programs maintained by the author, starting with atago itself (test/e2e/tools/):
| Tool | Feature | Specs | Docs |
|---|---|---|---|
| atago | atago tested by atago: the self-hosted specs run the real built binary in CI (make e2e). | specs | docs |
| gup | Updates and manages the Go command-line tools in $GOBIN. | specs | docs |
| sqly | Runs SQL against CSV/TSV/LTSV/JSON/Parquet/Excel/ACH/Fedwire files. | specs | docs |
| truss | Image transformation (convert/resize/re-encode). | specs | docs |
| iso8583tool | Debugs and inspects ISO 8583 payment messages. | specs | docs |
| jose | Signs and encrypts with JOSE. | specs | docs |
| career | Renders résumé PDFs from a single YAML file. | specs | docs |
| mimixbox | Packs many Unix commands into one BusyBox-style binary. | specs | docs |
| mobilepkg | Inspects Android packages for metadata and security findings. | specs | docs |
Third-party programs (test/e2e/thirdparty/). atago runs each as an unmodified binary and ships only its own spec YAML — no third-party code is copied or redistributed, which every listed license permits:
| Program | Feature | License | Specs | Docs |
|---|---|---|---|---|
| git | Version control — runs in CI on all three OSes. | GPL-2.0 | specs | docs |
| jq | JSON processor: stdin-driven filters, –arg injection, and its documented exit-code contract (1 for -e false, 3 for a bad program, parse failures). | MIT | specs | docs |
| fzf | Interactive fuzzy finder driven inside a real pseudo-terminal: expect/send sessions type queries, multi-select, and abort (exit 130); –filter pins the non-TTY contract. | MIT | specs | docs |
| htop | Full-screen alternate-screen TUI as a pty screen-rendering testbed: the rendered frame shows the meters and column header, q quits with a clean lifecycle, and an unknown flag is rejected before the TUI opens. | GPL-2.0 | specs | docs |
| redis | Server+client pair: redis-server as a scenario service (readiness by log and by port), redis-cli round-trips (PING/SET/GET/INCR/TTL), error contracts, and a polled graceful shutdown. | RSALv2/SSPLv1 | specs | docs |
| hugo | Scaffold + build + serve in one binary: hugo new site tree assertions, a theme-less --minify build, and hugo server bootstrapped in suite.setup then queried over HTTP. | Apache-2.0 | specs | docs |
| openssl | Cryptography toolkit: exact digests, keygen → sign → verify (and tamper detection), encrypt/decrypt round-trips with the wrong-password failure mode, self-signed certificates. | Apache-2.0 | specs | docs |
| sqlite3 | Embedded-database shell driven as a real binary: one-shot SQL, -json/-csv output modes, .dump → .read rebuilds, .import from CSV, bad-SQL diagnostics. | Public Domain | specs | docs |
| caddy | Self-hosted web server, booted from an authored Caddyfile and queried over HTTP. | Apache-2.0 | specs | docs |
| coredns | Self-hosted DNS server: an authored zone queried with real dig — authoritative answers, CNAME chasing, NXDOMAIN/REFUSED, and the health plugin over HTTP. | Apache-2.0 | specs | docs |
| gitea | Self-hosted git service: booted with SQLite, administered via its CLI, driven over the REST API (repos, commits, issues), then cloned with real git. | MIT | specs | docs |
| gotify | Self-hosted notification server: app provisioning, token-authenticated pushes, and the app icon uploaded as real multipart/form-data, downloaded back, and verified as a PNG. | MIT | specs | docs |
| grafana | Self-hosted observability platform: health/build info, the login redirect asserted with follow_redirects: false, and a dashboard + datasource lifecycle over the REST API. | AGPL-3.0 | specs | docs |
| mailpit | Self-hosted email testing: messages delivered over real SMTP (stock curl), then asserted via the REST API — capture, search, MIME attachments, teardown. | MIT | specs | docs |
| minio | Self-hosted S3-compatible object storage: full object lifecycle via mc, versioning, anonymous bucket policies, S3 XML error contract. | AGPL-3.0 | specs | docs |
| aws-cli | The AWS CLI driven against a local MinIO S3 endpoint — a cloud CLI tested offline: bucket/object lifecycle, byte-identical round-trip, head-object JSONPath, presigned-URL fetch, missing-key error contract. | Apache-2.0 | specs | docs |
| python3 | The REPL as an interactive pty testbed: prompt detection, multi-exchange expect/send, EOF exit, traceback recovery, and TTY branching — a copy-paste template for testing your own REPL. | PSF-2.0 | specs | docs |
| ssh-keygen | OpenSSH key generation: non-interactive and interactive-passphrase (pty) key pairs, generated-file asserts, the exact fingerprint contract, and verification-failure exit codes. | BSD | specs | docs |
| ffmpeg | Media pipeline: lavfi video synthesis, ffprobe stream JSON (JSONPath), frame extraction verified with image assertions (format/dimensions/pixel similarity), a webm transcode, and error contracts. | LGPL/GPL | specs | docs |
| pandoc | Document conversion: markdown→HTML, a docx artifact verified by unzipping its OOXML, a stdin→stdout filter (via stdin: {file:}), the JSON AST queried with JSONPath, metadata-driven standalone output, a markdown↔HTML round-trip, and the unknown-format and missing-input error contracts. | GPL-2.0+ | specs | docs |
| terraform | The multi-exit-code IaC CLI, fully offline via the builtin terraform_data resource: init/validate/plan/apply/destroy, the plan -detailed-exitcode 0/1/2 contract, state JSON, and fmt -check exit 3. | BUSL-1.1 | specs | docs |
| age | Modern file encryption: keygen, binary-safe encrypt/decrypt round-trips, armored (PEM) output, interactive-passphrase mode (pty), and clean failure semantics. | BSD-3-Clause | specs | docs |
| sops | Secrets encryption with age as the key backend: value-only encryption that keeps keys readable and adds a MAC, an encrypt→decrypt value round-trip, single-field --extract, wrong-key failure (128), a tampered ciphertext failing the MAC (25), and --encrypted-regex scoping which keys are encrypted. Every age key is generated in the workdir. | MPL-2.0 | specs | docs |
| kustomize | Declarative Kubernetes manifest rendering, fully offline and deterministic: build transforms (prefix/namespace/labels/images), generator content-hash suffixes (and disabling them), strategic-merge and JSON6902 patches, the load-restrictor security contract, byte-identical repeated builds, and the create/edit/fix authoring side effects. | Apache-2.0 | specs | docs |
| actionlint | Static linter for GitHub Actions workflows, offline and deterministic: a clean workflow exits 0 silently, distinct rule categories (undefined needs, unknown runner label, invalid expression) surface as tagged findings on stdout, the JSON format is a structured oracle over them, -ignore suppresses a match, and stdin mode lints piped content. | MIT | specs | docs |
| aqua | Declarative CLI version manager: install pulls a tool from a local registry + local HTTP file server, lands it executable under the managed root, and aqua exec runs it — its essential install→resolve→run flow, hermetic. Plus the offline surface: init (idempotent), policy init, root-dir/completion generators, and exit codes separating a usage error (3) from a runtime error (1, which on an unmanaged command). | MIT | specs | docs |
| ecspresso | Amazon ECS deploy tool: a real deploy against a moto mock AWS endpoint registers a task definition and creates the service, then a changed image rolls it to a new revision — the ECS state change observed with aws ecs describe-services. Plus the offline render surface: env/must_env template functions and defaults, a jsonnet task definition with --ext-str, config resolution, and failure modes at distinct exit codes, checked as structured JSON. | MIT | specs | docs |
| nats | Self-hosted messaging: request/reply through the real broker, JetStream persistence (create → publish → count → purge), the KV store, and the monitoring endpoint. | Apache-2.0 | specs | docs |
| ntfy | Self-hosted push notifications: publish with headers, poll the JSON feed, topic isolation, and deny-all access control unlocked via the admin CLI. | Apache-2.0 | specs | docs |
| prometheus | Self-hosted monitoring: promtool config/rule checks and rule unit tests, the query API, and a self-scrape polled with http retry. | Apache-2.0 | specs | docs |
| pushgateway | Self-hosted metrics gateway: raw text body: pushes scraped back with their group labels, a malformed payload rejected (400) and never ingested, POST-merges-vs-PUT-replaces group semantics, an added instance label, and job-group deletion. | Apache-2.0 | specs | docs |
| rclone | Self-hosted file sync: copy/sync/check semantics (including the corruption failure mode), JSON listings, and rclone serve http. | MIT | specs | docs |
| restic | Self-hosted backup: init → backup → restore round-trip, snapshot JSON, diff, integrity check, retention, wrong-password failure mode. | BSD-2-Clause | specs | docs |
| transfer.sh | Self-hosted file sharing: a PNG uploaded as the raw request body (body_file), downloaded back to disk (body_to), and verified byte-for-byte; multipart uploads too. | MIT | specs | docs |
| webhook | Self-hosted webhook receiver driven by config fixtures: a request runs a command whose file side effect and output are both checked, a value trigger-rule gates execution (and blocks it, proven by the absent side effect), an HMAC signature verified against an openssl-computed oracle, the http-methods allowlist (405), and a non-zero command surfaced as 500. | MIT | specs | docs |