rpki/deploy/docker-installer/docs/troubleshooting.en.md

Troubleshooting (__PACKAGE_ARCH__ / __PACKAGE_PLATFORM__)

Docker or Compose Is Unavailable

Run:

docker version
docker compose version

If missing, run:

./scripts/install.sh

Package Architecture Mismatch

If status.sh, install.sh, or self-check.sh reports:

package architecture mismatch

the host uname -m does not match the packaged PACKAGE_ARCH=__PACKAGE_ARCH__. The default behavior is to fail clearly rather than silently switching to emulation.

Only if you intentionally accept QEMU/binfmt cross-architecture execution should you set:

ALLOW_CROSS_ARCH=1

Then rerun install or self-check.

__PACKAGE_ARCH__ Image Cannot Run

Running __PACKAGE_PLATFORM__ images on a different host architecture requires binfmt/qemu. The installer only attempts this automatically when ALLOW_CROSS_ARCH=1; you can also enable it manually:

docker run --rm --privileged tonistiigi/binfmt --install __PACKAGE_ARCH__
docker run --rm --platform __PACKAGE_PLATFORM__ debian:bookworm-slim uname -m

For arm64 packages the expected output is aarch64; for amd64 packages the expected output is x86_64.

To confirm the metrics image is also loaded correctly:

docker image inspect "$(grep '^METRICS_IMAGE=' .env | cut -d= -f2-)"
docker run --rm --platform __PACKAGE_PLATFORM__ \
  "$(grep '^METRICS_IMAGE=' .env | cut -d= -f2-)" \
  /opt/ours-rp/bin/rpki_artifact_metrics --help

RTR report metrics are empty

First check registration and the report directory:

./scripts/status.sh
./scripts/register_rtr_monitor.sh --report-dir /root/rpki/report --dry-run

Check the metrics container mount:

docker inspect "$(grep '^COMPOSE_PROJECT_NAME=' .env | cut -d= -f2-)-artifact-metrics" \
  | jq '.[0].Mounts[] | select(.Destination=="/var/lib/ours-rp/rtr-report")'

First Snapshot Times Out

All-five snapshot can be slow, especially under QEMU. Increase timeout:

./scripts/start.sh --timeout-secs 14400

Output Counts Are Too Low

Check:

grep LIVE_TA_REFRESH_BEFORE_SNAPSHOT .env
ls -l __HOST_DATA_DIR__/state/live-ta
tail -100 __HOST_DATA_DIR__/logs/live-ta-refresh-*.log

In file-live-ta mode, snapshot should wait until live TA refresh succeeds.

Grafana Login Fails

Check .env:

GRAFANA_ADMIN_USER=admin
GRAFANA_ADMIN_PASSWORD=admin

If Grafana has already started, changing .env may not reset the existing Grafana database. Stop services and back up/clean ${HOST_DATA_DIR}/grafana if needed.

A Finite Acceptance Test Starts an Extra Run

If .env sets a finite MAX_RUNS=3 while SOAK_RESTART_POLICY=unless-stopped, Docker Compose restarts the soak container after it exits successfully.

For finite tests, set:

SOAK_RESTART_POLICY=no

How to Confirm a Periodic Forced Snapshot

Check the latest run metadata:

latest="$(find ${HOST_DATA_DIR}/runs -maxdepth 1 -type d -name 'run_*' | sort | tail -1)"
jq '{run_id,sync_mode,snapshot_reason,periodic_snapshot_delta_count,periodic_snapshot_forced,reset_db_cleanup_status}' "$latest/run-meta.json"

For a threshold-triggered reset you should see:

• sync_mode: "snapshot"
• snapshot_reason: "periodic_snapshot_delta_limit"
• periodic_snapshot_forced: true

To confirm delta counting still advances after retained runs are pruned, also inspect:

jq '{last_success_snapshot,successful_deltas_since_snapshot,recent_runs_count:(.recent_runs|length)}' \
  "${HOST_DATA_DIR}/state/run-lifecycle-state.json"

Lifecycle State File Is Corrupt

The script backs up the corrupt file before best-effort bootstrap from retained runs:

ls -1 "${HOST_DATA_DIR}/state"/run-lifecycle-state.json.corrupt.*
jq . "${HOST_DATA_DIR}/state/run-lifecycle-state.json"