20260902 完成公开树清理与热点拆分
This commit is contained in:
@@ -1,49 +1,28 @@
|
||||
# Compatibility harness (M5)
|
||||
# Compatibility harness
|
||||
|
||||
`verify_run_abi.sh` validates the required files and semantic fields for a
|
||||
successful normal run. It is intentionally independent of absolute paths and
|
||||
wall-clock timestamps, so it can be used for native and Docker run directories:
|
||||
The compatibility harness verifies that two retained validator runs implement
|
||||
the same declared run-artifact contract. It is useful when comparing native and
|
||||
container execution, or when validating a refactor against a fixed offline
|
||||
input.
|
||||
|
||||
```bash
|
||||
tests/compat/verify_run_abi.sh /path/to/runs/run_0001
|
||||
```
|
||||
verify_run_abi.sh validates the required files and semantic fields for one
|
||||
successful run. It is independent of absolute paths and wall-clock timestamps:
|
||||
|
||||
`compare_runs.py` compares an original-runtime run with a component run. It
|
||||
checks the normal-run status/count contract, report and validation contract,
|
||||
and canonical CIR/CCR/VRP/VAP payloads. It normalises only the exceptions in
|
||||
`baseline-manifest.toml` (including runtime telemetry that is measured
|
||||
separately). Validation time is treated as a fixed input, while only lifecycle
|
||||
timestamps and CCR `producedAt` are normalised; all other semantic payload
|
||||
differences are failures:
|
||||
tests/compat/verify_run_abi.sh /path/to/runs/run_0001
|
||||
|
||||
```bash
|
||||
tests/compat/compare_runs.py \
|
||||
/path/to/original/runs/run_0001 \
|
||||
/path/to/panda/runs/run_0001
|
||||
```
|
||||
compare_runs.py compares two run directories. It checks run status and counts,
|
||||
the validation contract, report content, and canonical CIR, CCR, VRP, and VAP
|
||||
payloads. Only the documented canonicalization exceptions in
|
||||
baseline-manifest.toml are normalized:
|
||||
|
||||
The M5 harness will run the same frozen input against the original `rpki`
|
||||
runtime, native `panda-rpki-validator`, and the Docker runtime. It will compare
|
||||
the normal-run output ABI in `docs/output-abi.md`, then record stage timing,
|
||||
CPU, RSS, state size and artifact size for five serial repetitions per profile.
|
||||
tests/compat/compare_runs.py \
|
||||
/path/to/left/runs/run_0001 \
|
||||
/path/to/right/runs/run_0001
|
||||
|
||||
M4 provides the ABI verifier and a real snapshot/warm-delta staging run. The
|
||||
canonical comparator is now available for M5; five-run performance comparison
|
||||
and the complete profile matrix remain M5 deliverables.
|
||||
`verification-only` is excluded.
|
||||
Use a fixed validation time and a reproducible input for equivalence testing.
|
||||
Live RRDP and rsync sources can change during a comparison; investigate an
|
||||
input difference before interpreting a file-level mismatch as validator
|
||||
behaviour.
|
||||
|
||||
For live-RIR evidence, run both images with the same `RIRS` and fixed
|
||||
`--validation-time`, then compare `input.cir` and the decoded `result.ccr` in
|
||||
separate steps. `compare_runs.py` deliberately remains strict about report,
|
||||
validation-event and CSV order, so a live RRDP/rsync source race can produce a
|
||||
file-level mismatch even when the CCR state digest matches. Use
|
||||
`ccr_state_compare` plus `triage_ccr_cir_pair` to distinguish source/input
|
||||
drift from validator behavior; the 2026-09-01 APNIC/all5 evidence is recorded in
|
||||
`specs/develop/20260901/m5_live_rir_image_artifact_comparison_milestone_report.md`.
|
||||
The follow-up remote-231 serial all5 run (old image first, then new image;
|
||||
one snapshot plus three deltas per side with cache/prefetch/parallel flags) is
|
||||
recorded in
|
||||
`specs/develop/20260901_2/m5_live_all5_cache_prefetch_serial_4run_milestone_report.md`.
|
||||
Its strict file mismatches remain failures until the same frozen input is
|
||||
replayed on both sides; live CCR/CIR differences must first be triaged with
|
||||
the state comparator rather than treated as code parity.
|
||||
The test policy and the output contract are documented in the Sphinx site:
|
||||
docs/source/testing.rst and docs/source/output-abi.rst.
|
||||
|
||||
@@ -1,7 +1,6 @@
|
||||
schema_version = 1
|
||||
source_commit = "74cbebbd3334ac0063761c1a97a88ee000cc2a57"
|
||||
component = "panda-rpki-validator"
|
||||
comparison_status = "m5-canonical-snapshot-delta-baseline-in-progress"
|
||||
comparison_scope = "fixed-input-runtime-equivalence"
|
||||
verification_only = "excluded"
|
||||
normal_profiles = [
|
||||
"snapshot",
|
||||
|
||||
Reference in New Issue
Block a user