Performance Evidence Policy
Relevant source files
- docs/performance/README.md
- docs/performance/methodology.md
- docs/performance/current-results.md
- docs/performance/evidence
- scripts/verify/benchmark-provenance.py
- scripts/benchmark
axiomos separates performance claims into method, current evidence, and history. A number counts as current release evidence only if its campaign records all of:
| Requirement | Purpose |
|---|---|
| Exact source commit | Ties the number to code |
| Pinned toolchain | Rules out compiler drift |
| Exact command | Reproducibility |
| Host or hardware boundary | Scope of the claim |
| Tracked raw output | Numbers must be present in the raw log |
| SHA-256 of raw log and measured executable or image | Identifies the artifact that produced the result |
Each campaign lives under docs/performance/evidence/COMMIT/ with a manifest.toml and the raw captures.
No QEMU, Raspberry Pi, Linux-comparison, boot-time, interrupt-latency, WCET, or comparative-speed headline is current release evidence until it is rerun under this contract. A developer-host campaign does not establish a hardware latency objective.
Provenance gate
python3 -B scripts/verify/benchmark-provenance.pyThe checker (also run as benchmark-provenance-static in the audit gate) verifies the raw-log hash, confirms the Cargo-reported executable path and every declared result marker appear in the log, validates source-input hashes against the recorded Git commit, and requires a well-formed executable hash. It does not claim byte-identical binaries across different hosts or linkers.
Evidence tiers
Adding a campaign
- Start from a clean, committed tree with the pinned toolchain.
- Store raw output and
manifest.tomlunderdocs/performance/evidence/COMMIT/. - Record the Cargo-reported executable, its SHA-256, host/hardware controls, exact command, and hashes of benchmark source,
Cargo.lockandrust-toolchain.tomlat that commit. - Add only results present in the raw output. Hardware campaigns must also retain UART and logic-analyzer captures and deployed-image hashes.
- Run the benchmark-provenance and documentation-link checks.
Reproducible Pi 5 images
Pi 5 kernel8.img builds are bit-reproducible on the same host and toolchain because both build.rs and kernel/build.rs pin the ext2 UUID, a non-zero hash_seed, and SOURCE_DATE_EPOCH (otherwise mke2fs stamps random values into every image). Reproduction needs the same feature set and trust-root inputs:
export AXIOM_BPF_TRUSTED_KEY_PATH=PATH_TO_LAB_PUBLIC_KEY
export AXIOM_SIGNED_BPF_STARTUP_PATH=PATH_TO_SIGNED_STARTUP_RBPF
cargo clean
cargo xtask build rpi5 -- release FEATURES_FROM_BUILD_TOML
sha256sum target/aarch64-unknown-none/release/kernel8.imgAn image whose build.toml records worktree_dirty = true cannot be reproduced and stays provisional regardless of how well its hashes are recorded.