1.2 KiB
Compatibility harness
The compatibility harness verifies that two retained validator runs implement the same declared run-artifact contract. It is useful when comparing native and container execution, or when validating a refactor against a fixed offline input.
verify_run_abi.sh validates the required files and semantic fields for one successful run. It is independent of absolute paths and wall-clock timestamps:
tests/compat/verify_run_abi.sh /path/to/runs/run_0001
compare_runs.py compares two run directories. It checks run status and counts, the validation contract, report content, and canonical CIR, CCR, VRP, and VAP payloads. Only the documented canonicalization exceptions in baseline-manifest.toml are normalized:
tests/compat/compare_runs.py \
/path/to/left/runs/run_0001 \
/path/to/right/runs/run_0001
Use a fixed validation time and a reproducible input for equivalence testing. Live RRDP and rsync sources can change during a comparison; investigate an input difference before interpreting a file-level mismatch as validator behaviour.
The test policy and the output contract are documented in the Sphinx site: docs/source/testing.rst and docs/source/output-abi.rst.