# Operations Guide (`__PACKAGE_ARCH__`)

## Install

```bash
./scripts/install.sh
```

The installer is idempotent:

- existing `.env` is kept;
- existing Docker/Compose installation is reused;
- repeated loading of packaged runtime, metrics, Prometheus and Grafana images for the package architecture is safe;
- existing data directory is reused;
- matching host/package architecture is required by default.

## Start

```bash
./scripts/start.sh
```

Start without waiting for the first snapshot:

```bash
./scripts/start.sh --no-wait-first-run
```

If you intentionally want to run this package on a different host architecture through QEMU/binfmt, first set:

```bash
ALLOW_CROSS_ARCH=1
```

Then rerun `./scripts/install.sh` or `./scripts/self-check.sh`, and only then will the scripts attempt to enable the matching binfmt handler.

## Stop and Restart

```bash
./scripts/stop.sh
./scripts/restart.sh
```

## Status Checks

```bash
./scripts/status.sh
./scripts/self-check.sh
```

Important checks:

- Docker/Compose availability;
- runtime, metrics, Prometheus and Grafana images exist;
- `HOST_DATA_DIR` is writable;
- Compose config is valid;
- latest run status;
- metrics, Prometheus and Grafana endpoints;
- whether `host_arch`, `package_arch`, and `arch_mode` match expectations.

`status.sh` also prints:

- `package_arch`
- `package_platform`
- `runtime_image`
- `metrics_image`
- `allow_cross_arch`
- `periodic_snapshot_reset`
- `periodic_snapshot_max_deltas`
- `rtr_report_dir`
- `rtr_report_container_dir`

To verify RTR report ingestion:

```bash
./scripts/register_rtr_monitor.sh --report-dir /root/rpki/report --dry-run
./scripts/register_rtr_monitor.sh --report-dir /root/rpki/report
./scripts/status.sh
curl -s "http://127.0.0.1:${METRICS_PORT:-9556}/metrics" | grep '^ours_rp_rtr_' | head
```

Registration accepts only an absolute directory on the same host. The command retains a timestamped `.env` backup and updates only the metrics, Prometheus, and Grafana sidecars; the `ours-rp-soak` container ID must remain unchanged.

## Upgrade

Extract the new package into a new directory and explicitly reuse the existing `.env` through the upgrade script:

```bash
./scripts/upgrade.sh --reuse-env-from /path/to/old-installer/.env
```

Configuration is merged in this order, with a key on the right overriding the same key on the left:

```text
new .env.example < old .env < environment variables on this upgrade command
```

For example, override the next-run interval and Grafana password for this upgrade:

```bash
INTERVAL_SECS=300 GRAFANA_ADMIN_PASSWORD='new-password' \
  ./scripts/upgrade.sh --reuse-env-from /path/to/old-installer/.env
```

Before any image is loaded or container is started, upgrade verifies that every key declared by the new template is present and non-empty in the final `.env`. An explicit empty assignment fails, for example `INTERVAL_SECS= ./scripts/upgrade.sh ...`. Removed keys from an old `.env` are not copied, and the script reports key names only, never password-like values.

The installer reads `.env` as Compose-compatible dotenv assignments. Do not put shell command substitutions or environment references such as `$HOME` in the file; use concrete values or command-level overrides instead.

If the new package directory already has a `.env`, the upgrade script keeps it. In that case `--reuse-env-from` is not imported again, but environment variables from the current command still override the existing `.env`.

When an older `.env` lacks a key introduced by the new package, the new `.env.example` default remains in the final `.env`. Upgrade fails before Docker operations if any final declared value is empty.

Upgrade does not delete:

- `runs/`
- `logs/`
- `state/rsync-mirror`
- runtime configuration referenced by `.env`
- Prometheus / Grafana data

To validate periodic forced snapshot behavior, temporarily set:

```bash
PERIODIC_SNAPSHOT_RESET=1
PERIODIC_SNAPSHOT_MAX_DELTAS=2
```

Then confirm the latest `run-meta.json` contains:

```bash
snapshot_reason=periodic_snapshot_delta_limit
```

And inspect the independent lifecycle state:

```bash
jq '{last_run,last_success_snapshot,successful_deltas_since_snapshot,state_health}' \
  "${HOST_DATA_DIR}/state/run-lifecycle-state.json"
```

After validation, restore:

```bash
PERIODIC_SNAPSHOT_MAX_DELTAS=100
```

## Cleanup

```bash
./scripts/cleanup.sh --keep-runs 100
./scripts/cleanup.sh --keep-runs 100 --execute
```

Cleanup is dry-run by default. Add `--execute` after reviewing the output.