29 lines
1.2 KiB
Markdown
29 lines
1.2 KiB
Markdown
# Compatibility harness
|
|
|
|
The compatibility harness verifies that two retained validator runs implement
|
|
the same declared run-artifact contract. It is useful when comparing native and
|
|
container execution, or when validating a refactor against a fixed offline
|
|
input.
|
|
|
|
verify_run_abi.sh validates the required files and semantic fields for one
|
|
successful run. It is independent of absolute paths and wall-clock timestamps:
|
|
|
|
tests/compat/verify_run_abi.sh /path/to/runs/run_0001
|
|
|
|
compare_runs.py compares two run directories. It checks run status and counts,
|
|
the validation contract, report content, and canonical CIR, CCR, VRP, and VAP
|
|
payloads. Only the documented canonicalization exceptions in
|
|
baseline-manifest.toml are normalized:
|
|
|
|
tests/compat/compare_runs.py \
|
|
/path/to/left/runs/run_0001 \
|
|
/path/to/right/runs/run_0001
|
|
|
|
Use a fixed validation time and a reproducible input for equivalence testing.
|
|
Live RRDP and rsync sources can change during a comparison; investigate an
|
|
input difference before interpreting a file-level mismatch as validator
|
|
behaviour.
|
|
|
|
The test policy and the output contract are documented in the Sphinx site:
|
|
docs/source/testing.rst and docs/source/output-abi.rst.
|