imagemagick

The atago project wrote these specs on its own initiative and runs them in its own CI, to exercise atago against a real program. They are not imagemagick’s official test suite, and the imagemagick project is not affiliated with atago.

Summary #

1 suite · 13 scenarios

Contents #

ImageMagick (image conversion pipeline) #

ImageMagick converts, resizes, and inspects images. What this suite pins is the part a converter must not get wrong: that the file it wrote really is the image it claims.

Every output is decoded and checked as an image — format, pixel dimensions, alpha channel — never merely “a file appeared”. A lossless round trip is verified twice over: once by atago comparing the pixels against the original, and once by ImageMagick’s own compare -metric AE, an independent oracle that has to agree. The failure side is pinned per kind (missing input, corrupt input, unwritable format), and the input images are always generated by the tool itself, so the suite carries no third-party assets.

Source: test/e2e/thirdparty/imagemagick/imagemagick.atago.yaml

Scenario: a conversion writes the requested format and nothing else #

only when convert -version succeeds

Given #

  • The command runs with an isolated home under ${workdir}/.atago-home (HOME/XDG or APPDATA redirected).

When #

convert -size 8x6 xc:red source.png
convert source.png out.jpg

Then #

  • after convert -size 8x6 xc:red source.png:
    • exit code is 0
  • after convert source.png out.jpg:
    • exit code is 0
    • the step changed exactly created out.jpg, modified nothing, deleted nothing
    • image out.jpg is jpeg, width 8, height 6

Generated artifacts #

  • out.jpg

Scenario: resize honors the aspect ratio unless it is forced #

only when convert -version succeeds

When #

convert -size 40x20 xc:white source.png
convert source.png -resize 20x20 fitted.png
convert source.png -resize 20x20! forced.png

Then #

  • after convert -size 40x20 xc:white source.png:
    • exit code is 0
  • after convert source.png -resize 20x20 fitted.png:
    • exit code is 0
    • image fitted.png width 20, height 10
  • after convert source.png -resize 20x20! forced.png:
    • exit code is 0
    • image forced.png width 20, height 20

Generated artifacts #

  • fitted.png
  • forced.png

Scenario: a 1x1 canvas survives the smallest possible conversion #

only when convert -version succeeds

When #

convert -size 1x1 xc:black tiny.png

Then #

  • exit code is 0
  • image tiny.png is png, width 1, height 1

Generated artifacts #

  • tiny.png

Scenario: an alpha channel survives PNG but is dropped by JPEG #

only when convert -version succeeds

When #

convert -size 8x6 xc:none transparent.png
convert transparent.png -background white -flatten flat.jpg

Then #

  • after convert -size 8x6 xc:none transparent.png:
    • exit code is 0
    • image transparent.png is png, has alpha
  • after convert transparent.png -background white -flatten flat.jpg:
    • exit code is 0
    • image flat.jpg is jpeg, has no alpha

Generated artifacts #

  • transparent.png
  • flat.jpg

Scenario: a PNG to PPM round trip restores the original pixels #

only when convert -version succeeds

When #

convert -size 16x12 gradient:red-blue source.png
convert source.png intermediate.ppm
convert intermediate.ppm restored.png
compare -metric AE source.png restored.png null:

Then #

  • after convert -size 16x12 gradient:red-blue source.png:
    • exit code is 0
  • after convert source.png intermediate.ppm:
    • exit code is 0
  • after convert intermediate.ppm restored.png:
    • exit code is 0
    • image restored.png similar to source.png
  • after compare -metric AE source.png restored.png null::
    • exit code is 0

Generated artifacts #

  • restored.png

Scenario: comparing different images reports a nonzero difference #

only when convert -version succeeds

When #

convert -size 8x6 xc:red red.png
convert -size 8x6 xc:blue blue.png
compare -metric AE red.png blue.png null:

Then #

  • after convert -size 8x6 xc:red red.png:
    • exit code is 0
  • after convert -size 8x6 xc:blue blue.png:
    • exit code is 0
  • after compare -metric AE red.png blue.png null::
    • exit code is 1
    • stderr contains 48

Scenario: the JSON report describes the image it just wrote #

only when convert -version succeeds

When #

convert -size 24x18 xc:red report.png
convert report.png json:

Then #

  • after convert -size 24x18 xc:red report.png:
    • exit code is 0
  • after convert report.png json::
    • exit code is 0
    • stdout at $[0].image.format equals PNG; at $[0].image.geometry.width equals 24; at $[0].image.geometry.height equals 18; at $[0].image.mimeType equals image/png

Scenario: strip removes an embedded comment from the output bytes #

only when convert -version succeeds

When #

convert -size 8x6 xc:red -set comment CONFIDENTIAL-MARK marked.png
convert marked.png -strip stripped.png

Then #

  • after convert -size 8x6 xc:red -set comment CONFIDENTIAL-MARK marked.png:
    • exit code is 0
    • file marked.png contains CONFIDENTIAL-MARK
  • after convert marked.png -strip stripped.png:
    • exit code is 0
    • file stripped.png does not contain CONFIDENTIAL-MARK
    • image stripped.png similar to marked.png

Generated artifacts #

  • stripped.png

Scenario: a missing input fails without writing an output #

only when convert -version succeeds

When #

convert no-such-input.png out.png

Then #

  • exit code is 1
  • stdout is empty
  • stderr contains no-such-input.png
  • file out.png does not exist

Scenario: a corrupt input is rejected rather than converted #

only when convert -version succeeds

Given #

  • Fixture file broken.png is created.

Inputs #

Fixture broken.png:

this is not a PNG at all

When #

convert broken.png out.png

Then #

  • exit code is 1
  • stdout is empty
  • stderr contains broken.png
  • file out.png does not exist

Scenario: an unknown output extension keeps the input encoding #

only when convert -version succeeds

When #

convert -size 8x6 xc:red source.png
convert source.png out.notaformat

Then #

  • after convert -size 8x6 xc:red source.png:
    • exit code is 0
  • after convert source.png out.notaformat:
    • exit code is 0
    • image out.notaformat is png, width 8, height 6

Generated artifacts #

  • out.notaformat

Scenario: identify reads the dimensions back out of a written file #

only when convert -version succeeds

When #

convert -size 37x11 xc:gray sized.png
identify -format "%wx%h %m" sized.png

Then #

  • after convert -size 37x11 xc:gray sized.png:
    • exit code is 0
  • after identify -format "%wx%h %m" sized.png:
    • exit code is 0
    • stdout equals an exact value

Scenario: identify on a missing file fails and prints nothing to stdout #

only when convert -version succeeds

When #

identify absent.png

Then #

  • exit code is 1
  • stdout is empty
  • stderr contains absent.png