openssl

The atago project wrote these specs on its own initiative and runs them in its own CI, to exercise atago against a real program. They are not openssl’s official test suite, and the openssl project is not affiliated with atago.

Summary #

2 suites · 9 scenarios

Contents #

openssl + changes (key and cert generation footprints) #

A tool that generates private key material should write exactly the file you named and nothing else — no copy, no temporary file left in the directory, no cache entry elsewhere. With secrets, a stray extra file is a leak.

This suite states that as an exhaustive contract: after generating a key, and after generating a certificate, the directory contains precisely the one new artifact that was asked for.

Source: test/e2e/thirdparty/openssl/changes.atago.yaml

Scenario: genrsa writes exactly the key file #

only when openssl version succeeds

When #

openssl genrsa -out key.pem 2048

Then #

  • exit code is 0
  • the step changed exactly created key.pem, modified nothing, deleted nothing

Scenario: a self-signed cert is the only new file the req step creates #

only when openssl version succeeds

When #

openssl genrsa -out key.pem 2048
openssl req -new -x509 -key key.pem -out cert.pem -subj /CN=atago-test -days 1

Then #

  • after openssl genrsa -out key.pem 2048:
    • exit code is 0
  • after openssl req -new -x509 -key key.pem -out cert.pem -subj /CN=atago-test -days 1:
    • exit code is 0
    • the step changed exactly created cert.pem, modified nothing, deleted nothing
    • file cert.pem contains BEGIN CERTIFICATE

openssl (third-party CLI, no build required) #

OpenSSL is the cryptography tool most scripts reach for, and cryptography is where “it produced output” is the least convincing evidence of anything.

So each guarantee here is checked against something independent. Digests are pinned to their known values. A generated key is proven to work by deriving its public half and then signing and verifying with the pair — not by checking that a file appeared. Encryption is proven by decrypting back to the original bytes.

And the failure case is treated as a feature: decrypting with the wrong password must fail loudly, because a crypto tool that silently emits garbage on a bad key is worse than one that crashes. A self-signed certificate is checked to carry the subject that was requested.

Source: test/e2e/thirdparty/openssl/openssl.atago.yaml

Scenario: sha256 digest of a fixed input is exact and stable #

Given #

  • Fixture file msg.txt is created.

Inputs #

Fixture msg.txt:

the quick brown fox

When #

openssl dgst -sha256 -r msg.txt

Then #

  • exit code is 0
  • stdout contains 6e459fed18ddb06d57c8e9f0d000c302c7e01389926db6e89884bfbe91a2a5df

Scenario: base64 encode and decode round-trip via stdin #

Inputs #

stdin for openssl:

round-trip me

stdin for openssl:

${encoded}

When #

openssl base64
# capture ${encoded} from stdout
openssl base64 -d

Then #

  • after openssl base64 -d:
    • exit code is 0
    • stdout contains round-trip me

Scenario: rand -hex emits exactly the requested entropy #

When #

openssl rand -hex 16

Then #

  • exit code is 0
  • stdout matches /^[0-9a-f]{32} ?$/

Scenario: a generated private key is valid and yields its public half #

When #

openssl genpkey -algorithm ed25519 -out key.pem
openssl pkey -in key.pem -check -noout
openssl pkey -in key.pem -pubout -out pub.pem

Then #

  • after openssl genpkey -algorithm ed25519 -out key.pem:
    • exit code is 0
    • file key.pem contains BEGIN PRIVATE KEY
  • after openssl pkey -in key.pem -check -noout:
    • exit code is 0
    • stdout contains Key is valid
  • after openssl pkey -in key.pem -pubout -out pub.pem:
    • exit code is 0
    • file pub.pem contains BEGIN PUBLIC KEY

Scenario: symmetric encryption round-trips and a wrong password fails loudly #

Given #

  • Fixture file secret.txt is created.

Inputs #

Fixture secret.txt:

attack at dawn

When #

openssl enc -aes-256-cbc -pbkdf2 -pass pass:correct-horse -in secret.txt -out secret.enc
openssl enc -d -aes-256-cbc -pbkdf2 -pass pass:correct-horse -in secret.enc -out roundtrip.txt
openssl enc -d -aes-256-cbc -pbkdf2 -pass pass:wrong-password -in secret.enc -out garbage.txt

Then #

  • after openssl enc -aes-256-cbc -pbkdf2 -pass pass:correct-horse -in secret.txt -out secret.enc:
    • exit code is 0
    • file secret.enc exists
    • file secret.enc does not contain attack at dawn
  • after openssl enc -d -aes-256-cbc -pbkdf2 -pass pass:correct-horse -in secret.enc -out roundtrip.txt:
    • exit code is 0
    • file roundtrip.txt contains attack at dawn
  • after openssl enc -d -aes-256-cbc -pbkdf2 -pass pass:wrong-password -in secret.enc -out garbage.txt:
    • exit code is not 0
    • stderr contains bad decrypt

Generated artifacts #

  • secret.enc

Scenario: a self-signed certificate carries the requested subject #

When #

openssl req -x509 -newkey ed25519 -keyout ca.key -out ca.crt -nodes -days 1 -subj /CN=atago-test
openssl x509 -in ca.crt -noout -subject
openssl verify -CAfile ca.crt ca.crt

Then #

  • after openssl req -x509 -newkey ed25519 -keyout ca.key -out ca.crt -nodes -days 1 -subj /CN=atago-test:
    • exit code is 0
  • after openssl x509 -in ca.crt -noout -subject:
    • exit code is 0
    • stdout contains atago-test
  • after openssl verify -CAfile ca.crt ca.crt:
    • exit code is 0
    • stdout contains OK

Scenario: signing with the private key verifies with the public key #

Given #

  • Fixture file payload.txt is created.
  • Fixture file payload.txt is created.

Inputs #

Fixture payload.txt:

sign me

Fixture payload.txt:

sign me (tampered)

When #

openssl genpkey -algorithm ed25519 -out sk.pem
openssl pkey -in sk.pem -pubout -out vk.pem
openssl pkeyutl -sign -inkey sk.pem -rawin -in payload.txt -out payload.sig
openssl pkeyutl -verify -pubin -inkey vk.pem -rawin -in payload.txt -sigfile payload.sig
openssl pkeyutl -verify -pubin -inkey vk.pem -rawin -in payload.txt -sigfile payload.sig

Then #

  • after openssl pkeyutl -sign -inkey sk.pem -rawin -in payload.txt -out payload.sig:
    • exit code is 0
  • after openssl pkeyutl -verify -pubin -inkey vk.pem -rawin -in payload.txt -sigfile payload.sig:
    • exit code is 0
    • stdout contains Signature Verified Successfully
  • after openssl pkeyutl -verify -pubin -inkey vk.pem -rawin -in payload.txt -sigfile payload.sig:
    • exit code is not 0