imagemagick

The atago project wrote these specs on its own initiative and runs them in its own CI, to exercise atago against a real program. They are not imagemagick’s official test suite, and the imagemagick project is not affiliated with atago.

Summary #

1 suite · 13 scenarios

Contents #

ImageMagick (image conversion pipeline) #

ImageMagick converts, resizes, and inspects images. What this suite pins is the part a converter must not get wrong: that the file it wrote really is the image it claims.

Every output is decoded and checked as an image — format, pixel dimensions, alpha channel — never merely “a file appeared”. A lossless round trip is verified twice over: once by atago comparing the pixels against the original, and once by ImageMagick’s own compare -metric AE, an independent oracle that has to agree. The failure side is pinned per kind (missing input, corrupt input, unwritable format), and the input images are always generated by the tool itself, so the suite carries no third-party assets.

Source: test/e2e/thirdparty/imagemagick/imagemagick.atago.yaml

Scenario: a conversion writes the requested format and nothing else #

only when magick -version succeeds

Given #

  • The command runs with an isolated home under ${workdir}/.atago-home (HOME/XDG or APPDATA redirected).

When #

magick -size 8x6 xc:red source.png
magick source.png out.jpg

Then #

  • after magick -size 8x6 xc:red source.png:
    • exit code is 0
  • after magick source.png out.jpg:
    • exit code is 0
    • the step changed exactly created out.jpg, modified nothing, deleted nothing
    • image out.jpg is jpeg, width 8, height 6

Generated artifacts #

  • out.jpg

Scenario: resize honors the aspect ratio unless it is forced #

only when magick -version succeeds

When #

magick -size 40x20 xc:white source.png
magick source.png -resize 20x20 fitted.png
magick source.png -resize 20x20! forced.png

Then #

  • after magick -size 40x20 xc:white source.png:
    • exit code is 0
  • after magick source.png -resize 20x20 fitted.png:
    • exit code is 0
    • image fitted.png width 20, height 10
  • after magick source.png -resize 20x20! forced.png:
    • exit code is 0
    • image forced.png width 20, height 20

Generated artifacts #

  • fitted.png
  • forced.png

Scenario: a 1x1 canvas survives the smallest possible conversion #

only when magick -version succeeds

When #

magick -size 1x1 xc:black tiny.png

Then #

  • exit code is 0
  • image tiny.png is png, width 1, height 1

Generated artifacts #

  • tiny.png

Scenario: an alpha channel survives PNG but is dropped by JPEG #

only when magick -version succeeds

When #

magick -size 8x6 xc:none transparent.png
magick transparent.png -background white -flatten flat.jpg

Then #

  • after magick -size 8x6 xc:none transparent.png:
    • exit code is 0
    • image transparent.png is png, has alpha
  • after magick transparent.png -background white -flatten flat.jpg:
    • exit code is 0
    • image flat.jpg is jpeg, has no alpha

Generated artifacts #

  • transparent.png
  • flat.jpg

Scenario: a PNG to PPM round trip restores the original pixels #

only when magick -version succeeds

When #

magick -size 16x12 gradient:red-blue source.png
magick source.png intermediate.ppm
magick intermediate.ppm restored.png
magick compare -metric AE source.png restored.png null:

Then #

  • after magick -size 16x12 gradient:red-blue source.png:
    • exit code is 0
  • after magick source.png intermediate.ppm:
    • exit code is 0
  • after magick intermediate.ppm restored.png:
    • exit code is 0
    • image restored.png similar to source.png
  • after magick compare -metric AE source.png restored.png null::
    • exit code is 0

Generated artifacts #

  • restored.png

Scenario: comparing different images reports a nonzero difference #

only when magick -version succeeds

When #

magick -size 8x6 xc:red red.png
magick -size 8x6 xc:blue blue.png
magick compare -metric AE red.png blue.png null:

Then #

  • after magick -size 8x6 xc:red red.png:
    • exit code is 0
  • after magick -size 8x6 xc:blue blue.png:
    • exit code is 0
  • after magick compare -metric AE red.png blue.png null::
    • exit code is 1
    • stderr contains 48

Scenario: the JSON report describes the image it just wrote #

only when magick -version succeeds

When #

magick -size 24x18 xc:red report.png
magick report.png json:

Then #

  • after magick -size 24x18 xc:red report.png:
    • exit code is 0
  • after magick report.png json::
    • exit code is 0
    • stdout at $[0].image.format equals PNG; at $[0].image.geometry.width equals 24; at $[0].image.geometry.height equals 18; at $[0].image.mimeType equals image/png

Scenario: strip removes an embedded comment from the output bytes #

only when magick -version succeeds

When #

magick -size 8x6 xc:red -set comment CONFIDENTIAL-MARK marked.png
magick marked.png -strip stripped.png

Then #

  • after magick -size 8x6 xc:red -set comment CONFIDENTIAL-MARK marked.png:
    • exit code is 0
    • file marked.png contains CONFIDENTIAL-MARK
  • after magick marked.png -strip stripped.png:
    • exit code is 0
    • file stripped.png does not contain CONFIDENTIAL-MARK
    • image stripped.png similar to marked.png

Generated artifacts #

  • stripped.png

Scenario: a missing input fails without writing an output #

only when magick -version succeeds

When #

magick no-such-input.png out.png

Then #

  • exit code is 1
  • stdout is empty
  • stderr contains no-such-input.png
  • file out.png does not exist

Scenario: a corrupt input is rejected rather than converted #

only when magick -version succeeds

Given #

  • Fixture file broken.png is created.

Inputs #

Fixture broken.png:

this is not a PNG at all

When #

magick broken.png out.png

Then #

  • exit code is 1
  • stdout is empty
  • stderr contains broken.png
  • file out.png does not exist

Scenario: an unknown output extension keeps the input encoding #

only when magick -version succeeds

When #

magick -size 8x6 xc:red source.png
magick source.png out.notaformat

Then #

  • after magick -size 8x6 xc:red source.png:
    • exit code is 0
  • after magick source.png out.notaformat:
    • exit code is 0
    • image out.notaformat is png, width 8, height 6

Generated artifacts #

  • out.notaformat

Scenario: identify reads the dimensions back out of a written file #

only when magick -version succeeds

When #

magick -size 37x11 xc:gray sized.png
magick identify -format "%wx%h %m" sized.png

Then #

  • after magick -size 37x11 xc:gray sized.png:
    • exit code is 0
  • after magick identify -format "%wx%h %m" sized.png:
    • exit code is 0
    • stdout equals an exact value

Scenario: identify on a missing file fails and prints nothing to stdout #

only when magick -version succeeds

When #

magick identify absent.png

Then #

  • exit code is 1
  • stdout is empty
  • stderr contains absent.png