Testing
On this page
gup reads the build information of binaries it did not build, rewrites the
files in your $GOBIN, and drives go install for every tool you have. Each
technique below targets one way that can go wrong. The README lists the same
techniques in one table; this page has the commands, files and CI jobs.
Unit tests #
Every package has table-driven tests next to its source. CI runs them with the
race detector on Linux, macOS and Windows, on every supported Go minor and on
the latest Go release (.github/workflows/unit_test.yml).
go test -race ./...
End-to-end tests and snapshots #
The specs in e2e/atago
run the real gup binary and the real go command against an offline module
proxy in a throwaway directory, using atago.
Golden-output snapshots pin what gup prints, a pseudo terminal drives the
interactive gup remove prompt, and e2e/os_matrix.tsv records which specs
run on which operating system and why. CI runs them on Linux, macOS and
Windows for every pull request and every day (.github/workflows/e2e.yml).
go run ./e2e/runner
Property-based tests #
internal/goutil/property_test.go
checks that version comparison is a total order (reflexive, antisymmetric,
consistent with the parser) over generated module and Go versions.
internal/config/property_test.go
writes generated tool sets to gup.json and reads them back unchanged.
Differential tests #
internal/goutil/differential_test.go
reads the same files with gup and with go version -m, the go command’s own
reader of embedded build information, and fails when they disagree on whether
a binary is manageable, its import path, its module, its version or the Go that
built it. The files are the committed ELF and PE fixtures, files that are not
Go binaries, the toolchain’s go and gofmt, and binaries built during the
test for Linux, macOS (Mach-O) and Windows. go test -short skips the builds.
The same file compares gup’s Go toolchain ordering, which decides whether a
binary counts as built with an old Go, with go/version.Compare from the
standard library over release names generated with
rapid. Every go test draws 100
cases; the nightly rapid properties job in .github/workflows/fuzz.yml
draws 100,000.
go test ./internal/goutil -run agreesWith -rapid.checks=100000
Fuzzing #
FuzzParseConf (gup.json), FuzzParsePinArgs (gup pin arguments) and
FuzzValidatePinnedVersion (pinned versions) run their seed corpus in every
go test. .github/workflows/fuzz.yml searches for new inputs every night.
go test -run '^$' -fuzz '^FuzzParseConf$' -fuzztime 1m ./internal/config
Mutation testing #
gremlins rewrites one expression at
a time and checks that some test fails. .github/workflows/mutation.yml mutates
the lines a pull request changes and the whole module every week; the report is
uploaded as the mutation-report artifact.
gremlins unleash --workers 2 --timeout-coefficient 10 .
Run it from a clean checkout: gremlins mutates every Go file under the
directory, including the copies go-licenses leaves in the ignored
third_party_licenses/.
Baseline, measured on 2026-11-19 with gremlins v0.6.0 over the whole module: 800 killed, 77 lived, 222 not covered, 4 timed out. Test efficacy, killed / (killed + lived), is 91.22%; mutator coverage is 79.80%. Lived mutants include equivalent ones, so the score is tracked against this baseline rather than held to 100%.
Benchmarks #
himorime measures the gup binary in
bench/ and compares each
pull request with its base in the same job (.github/workflows/bench.yml).
make bench