Manual RRDP sync (APNIC-focused)

This directory contains manual, command-line scripts to reproduce the workflow described in:

specs/develop/20260226/apnic_rrdp_delta_analysis_after_manifest_revalidation_fix_20260227T022606Z.md

They are meant for hands-on validation / acceptance runs, not for CI.

Prerequisites

Rust toolchain (cargo)
rsync available on PATH (for rsync fallback/objects)
Network access (RRDP over HTTPS)

What the scripts do

`full_sync.sh`

Creates a fresh RocksDB directory
Runs a full serial validation from a TAL URL (default: APNIC RFC7730 TAL)
Writes:
- run log
- audit report JSON
- run meta JSON (includes durations + download_stats)
- short summary Markdown (includes durations + download_stats)
- RocksDB key statistics (db_stats --exact)
- RRDP repo state dump (rrdp_state_dump)

`delta_sync.sh`

Copies an existing “baseline snapshot DB” to a new DB directory (so the baseline is not modified)
Runs another validation against the copied DB (RRDP will prefer delta when available)
Produces the same artifacts as full_sync.sh
Additionally generates a Markdown delta analysis report by comparing:
- base vs delta report JSON
- base vs delta rrdp_state_dump TSV
- and includes a duration comparison (base vs delta) if the base meta JSON is available
- delta meta JSON includes download_stats copied from delta report JSON

Audit report fields (report.json)

The rpki binary writes an audit report JSON with:

format_version: 2
downloads: per-download RRDP/rsync events (URI, timestamps, duration, ok/fail, error, bytes, objects stats)
download_stats: aggregate counters (by kind)

These are useful for diagnosing why a run is slow (e.g. RRDP snapshot vs delta vs rsync fallback).

Meta fields (meta.json)

The scripts generate *_meta.json next to *_report.json and include:

durations_secs: wall-clock duration breakdown for the script steps
download_stats: copied from report_json.download_stats

Usage

Run from rpki/:

./scripts/manual_sync/full_sync.sh

After you have a baseline run, run delta against it:

./scripts/manual_sync/delta_sync.sh target/live/manual_sync/apnic_full_db_YYYYMMDDTHHMMSSZ \
  target/live/manual_sync/apnic_full_report_YYYYMMDDTHHMMSSZ.json

If the baseline was produced by full_sync.sh, the delta script will auto-discover the base meta JSON next to the base report (by replacing _report.json with _meta.json) and include base durations in the delta analysis report.

Configuration (env vars)

Both scripts accept overrides via env vars:

TAL_URL (default: APNIC TAL URL)
HTTP_TIMEOUT_SECS (default: 1800)
RSYNC_TIMEOUT_SECS (default: 1800)
RSYNC_MIRROR_ROOT (default: disabled; when set, passes --rsync-mirror-root to rpki)
VALIDATION_TIME (RFC3339; default: now UTC)
OUT_DIR (default: rpki/target/live/manual_sync)
RUN_NAME (default: auto timestamped)

Example:

TAL_URL="https://tal.apnic.net/tal-archive/apnic-rfc7730-https.tal" \
HTTP_TIMEOUT_SECS=1800 RSYNC_TIMEOUT_SECS=1800 \
./scripts/manual_sync/full_sync.sh

3.1 KiB Raw Blame History