Handle a metric this platform cannot measure
The same suite runs on Linux and Windows, and one metric cannot be measured on one of them.
# yaml-language-server: $schema=../../schema/himorime.schema.json
#
# Recipe: handle a metric this platform cannot measure.
# https://nao1215.github.io/himorime/cookbook/#handle-a-metric-this-platform-cannot-measure
version: "1"
name: unsupported metrics
description: Peak RSS of a command that starts a child process, which Windows cannot report.
build:
command: [go, build, -o, "${artifact}", ../tools/sleepy]
defaults:
warmup: 0
runs: 5
metrics:
cpu: true
memory: true
# The default, fail, stops before measuring when a requested metric
# cannot be measured here. skip measures everything else and reports the
# metric as unsupported, with its budgets and comparisons skipped.
unsupported: skip
benchmarks:
- name: parent and child
commands:
sleepy:
command: ["${artifact}", -ms, "20", -children, "1"]
budget:
peak_rss:
max: "<= 256MiB"
$ himorime run examples/unsupported-metrics
On Linux and macOS every metric is measured and the budget passes. On Windows the command starts a child process, so peak RSS is reported as unsupported, its budget as SKIPPED with the reason, CPU time is still measured, and himorime exits 0.
unsupported: skipand the defaultfailbehavior are defined in Metrics.- An unsupported metric is never reported as zero: its
statsarenull, its status saysunsupported, and the reason says why; a supported collection failure exits 6.
Example: examples/unsupported-metrics