1 Commits

Author SHA1 Message Date
850ff218ce add custom TAL installer patch 2026-08-06 10:48:38 +08:00
128 changed files with 6569 additions and 13099 deletions

View File

@@ -37,12 +37,27 @@ RETAIN_RUNS=100
# TAL/TA input mode: # TAL/TA input mode:
# file-with-ta: use packaged fixture TAL + TA only. # file-with-ta: use packaged fixture TAL + TA only.
# file-live-ta: use packaged fixture TAL; snapshot waits for live TA refresh, delta refreshes TA in background. # file-live-ta: use packaged fixture TAL; snapshot waits for live TA refresh, delta refreshes TA in background.
# custom-file-with-ta: use one custom TAL + TA from the read-only custom fixture mount.
# url: pass TAL URL to child process. # url: pass TAL URL to child process.
TAL_INPUT_MODE=file-live-ta TAL_INPUT_MODE=file-live-ta
LIVE_TA_REFRESH_BEFORE_SNAPSHOT=1 LIVE_TA_REFRESH_BEFORE_SNAPSHOT=1
LIVE_TA_REFRESH_CONNECT_TIMEOUT_SECS=15 LIVE_TA_REFRESH_CONNECT_TIMEOUT_SECS=15
LIVE_TA_REFRESH_MAX_TIME_SECS=120 LIVE_TA_REFRESH_MAX_TIME_SECS=120
# Custom TAL patch mode. Keep TAL_INPUT_MODE=file-live-ta and the five-RIR RIRS
# list above for the original package behavior. For a private fixture, set:
# RIRS=custom
# TAL_INPUT_MODE=custom-file-with-ta
# The paths below are container paths; CUSTOM_FIXTURE_HOST_DIR is a host path
# resolved relative to the compose directory when left at its default.
CUSTOM_FIXTURE_HOST_DIR=../custom-fixtures
CUSTOM_TAL_PATH=/opt/ours-rp/custom-fixtures/tal/custom.tal
CUSTOM_TA_PATH=/opt/ours-rp/custom-fixtures/ta/custom-ta.cer
CUSTOM_TAL_URI=https://host.docker.internal:18443/tal/custom.tal
# Space-separated container paths; paths must not contain spaces. Set this in
# custom mode when the RRDP server uses a private/self-signed CA.
HTTP_ROOT_CERT_PATHS=
# Sync and runtime behavior. # Sync and runtime behavior.
RSYNC_SCOPE=module-root RSYNC_SCOPE=module-root
DISABLE_COMPETING_RPS=0 DISABLE_COMPETING_RPS=0

View File

@@ -11,8 +11,15 @@ services:
RUN_ROOT: /var/lib/ours-rp RUN_ROOT: /var/lib/ours-rp
BIN_DIR: /opt/ours-rp/bin BIN_DIR: /opt/ours-rp/bin
FIXTURE_DIR: /opt/ours-rp/fixtures FIXTURE_DIR: /opt/ours-rp/fixtures
CUSTOM_FIXTURE_DIR: /opt/ours-rp/custom-fixtures
extra_hosts:
- "host.docker.internal:host-gateway"
volumes: volumes:
- ../.env:/opt/ours-rp/.env:ro - ../.env:/opt/ours-rp/.env:ro
# Overlay the patched runner while keeping the original runtime image
# and its exact rpki binaries unchanged.
- ../scripts/soak/run_soak.sh:/opt/ours-rp/run_soak.sh:ro
- ${CUSTOM_FIXTURE_HOST_DIR:-../custom-fixtures}:/opt/ours-rp/custom-fixtures:ro
- ${HOST_DATA_DIR:-/var/lib/ours-rp-package-installer}/state:/var/lib/ours-rp/state - ${HOST_DATA_DIR:-/var/lib/ours-rp-package-installer}/state:/var/lib/ours-rp/state
- ${HOST_DATA_DIR:-/var/lib/ours-rp-package-installer}/runs:/var/lib/ours-rp/runs - ${HOST_DATA_DIR:-/var/lib/ours-rp-package-installer}/runs:/var/lib/ours-rp/runs
- ${HOST_DATA_DIR:-/var/lib/ours-rp-package-installer}/logs:/var/lib/ours-rp/logs - ${HOST_DATA_DIR:-/var/lib/ours-rp-package-installer}/logs:/var/lib/ours-rp/logs

View File

@@ -0,0 +1,14 @@
# Custom TAL fixture mount
This directory is intentionally shipped as a mount point, not as a repository
of private keys. Run the patch tool `tools/generate_custom_fixture.py` to create
the test-only TAL, TA, RPKI objects, HTTPS CA/server certificates, RRDP files,
and rsync module data below it.
The generated fixture contains `baseline-v1`, `sync-hash-mismatch` and
`baseline-v2` for snapshot / hash / delta tests, plus
`validation-expired`, `validation-max-length`,
`validation-max-length-invalid`, `validation-roa-prefix-outside`,
`validation-out-of-resource`, `validation-ca-over-resource`, and
`validation-nonstandard` for validation cases. Generated keys are test-only
and must not be used in a production RPKI hierarchy.

View File

@@ -0,0 +1,30 @@
# 自建 TAL/TA 补丁
本目录描述并生成一个叠加到客户 Arm64 Ours RP 组件包上的测试补丁。补丁基线是源码 commit `9f4f4cf069e9fa8f6f97ed0f1065adac38abe82f`,不会替换客户包中的 runtime image 或 `bin/rpki`、`bin/rpki_daemon` 二进制。
补丁做三件事:
1. 在 runner 中增加 `RIRS=custom`、`TAL_INPUT_MODE=custom-file-with-ta`、自定义 TAL/TA 路径和 `HTTP_ROOT_CERT_PATHS`;
2. 通过 Compose 只读挂载 patched runner、`custom-fixtures/` 和 `host.docker.internal`,连接固定 HTTPS RRDP 与 rsync 服务;
3. 提供固定证书链、RPKI 对象生成器和服务启动脚本,不依赖 Barry、Rapport 或公网 RIR。
先停止 Ours RP,再执行:
```bash
./apply_custom_tal_patch.sh \
--component-root /opt/ours-rp-rtr-stack/components/ours-rp \
--enable-custom
```
`--enable-custom` 会修改组件 `.env` 的 custom 模式入口;不加该选项只安装 overlay,不改变当前 `.env`。回滚:
```bash
./rollback_custom_tal_patch.sh \
--component-root /opt/ours-rp-rtr-stack/components/ours-rp
```
补丁应用脚本会校验 `PACKAGE-MANIFEST.env` 的源码 commit 和 `arm64` 架构,并留下 `CUSTOM-TAL-PATCH-MANIFEST.env` 与 `.custom-tal-patch-state/` 备份。升级组件后需要重新检查 patch marker;不要把该 overlay 误当成客户原始安装包。
生成器默认把容器访问地址写成 `host.docker.internal`,固定服务端口为 HTTPS RRDP `18443`、rsync `1873`,rsync module 为 `custom`。可用 `--host localhost` 生成给宿主机直接运行 `bin/rpki` 的本地变体。
用例包括:`baseline-v1`、`sync-hash-mismatch`、`baseline-v2`、`validation-expired`、`validation-max-length`、`validation-max-length-invalid`、`validation-roa-prefix-outside`、`validation-out-of-resource`、`validation-ca-over-resource`、`validation-nonstandard`。其中 `validation-ca-over-resource` 在 `RESOURCE_VALIDATION_MODE=rfc6487` 下验证严格拒绝;`validation-update-03` 是原包默认的兼容处理模式。

View File

@@ -0,0 +1,187 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PATCH_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
PAYLOAD_ROOT="$SCRIPT_DIR/payload"
STATE_DIR_NAME=".custom-tal-patch-state"
EXPECTED_SOURCE_COMMIT="9f4f4cf069e9fa8f6f97ed0f1065adac38abe82f"
usage() {
cat <<'USAGE'
Usage:
./apply_custom_tal_patch.sh --component-root <installed-ours-rp-root>
./apply_custom_tal_patch.sh --component-root <root> --enable-custom
The default operation installs the overlay but does not change the existing
.env. --enable-custom writes the custom TAL/TA/root-cert example values into
the component .env so the next run uses the fixed local fixture.
Stop the ours RP service before applying or rolling back the overlay.
USAGE
}
die() {
echo "error: $*" >&2
exit 2
}
COMPONENT_ROOT=""
ENABLE_CUSTOM=0
while [[ $# -gt 0 ]]; do
case "$1" in
--component-root)
COMPONENT_ROOT="${2:-}"
shift 2
;;
--enable-custom)
ENABLE_CUSTOM=1
shift
;;
-h|--help)
usage
exit 0
;;
*)
die "unknown option: $1"
;;
esac
done
[[ -n "$COMPONENT_ROOT" ]] || die "--component-root is required"
COMPONENT_ROOT="$(cd "$COMPONENT_ROOT" 2>/dev/null && pwd)" \
|| die "component root does not exist: $COMPONENT_ROOT"
[[ -f "$COMPONENT_ROOT/PACKAGE-MANIFEST.env" ]] \
|| die "missing component PACKAGE-MANIFEST.env: $COMPONENT_ROOT"
[[ -d "$PAYLOAD_ROOT" ]] || die "missing patch payload: $PAYLOAD_ROOT"
# shellcheck disable=SC1091
source "$COMPONENT_ROOT/PACKAGE-MANIFEST.env"
[[ "${source_commit:-}" == "$EXPECTED_SOURCE_COMMIT" ]] \
|| die "base source commit mismatch: ${source_commit:-missing} != $EXPECTED_SOURCE_COMMIT"
[[ "${package_arch:-${PACKAGE_ARCH:-}}" == "arm64" ]] \
|| die "this patch is for the customer Arm64 component, got ${package_arch:-${PACKAGE_ARCH:-missing}}"
STATE_ROOT="$COMPONENT_ROOT/$STATE_DIR_NAME"
BACKUP_ROOT="$STATE_ROOT/original"
marker="$COMPONENT_ROOT/CUSTOM-TAL-PATCH-MANIFEST.env"
set_env_value() {
local env_path="$1"
local key="$2"
local value="$3"
local tmp_path="${env_path}.custom-tal-patch.tmp"
[[ -f "$env_path" ]] || die "missing component .env: $env_path"
awk -v key="$key" -v value="$value" '
BEGIN { found = 0 }
$0 ~ "^" key "=" { print key "=" value; found = 1; next }
{ print }
END { if (!found) print key "=" value }
' "$env_path" > "$tmp_path"
mv "$tmp_path" "$env_path"
}
copy_payload_file() {
local source_path="$1"
local target_path="$2"
mkdir -p "$(dirname "$target_path")"
cp "$source_path" "$target_path"
chmod --reference="$source_path" "$target_path" 2>/dev/null || true
}
append_custom_env_example() {
local target_path="$1"
if ! grep -Eq '^CUSTOM_TAL_URI=' "$target_path"; then
{
printf '\n# #142 custom TAL patch settings; original package values above are preserved.\n'
cat "$SCRIPT_DIR/custom-tal.env.example"
} >> "$target_path"
fi
}
if [[ -e "$STATE_ROOT" ]]; then
die "patch is already applied or an incomplete state exists: $STATE_ROOT"
fi
mkdir -p "$BACKUP_ROOT"
for relative_path in compose/docker-compose.yml .env.example; do
[[ -f "$COMPONENT_ROOT/$relative_path" ]] \
|| die "missing base file: $COMPONENT_ROOT/$relative_path"
mkdir -p "$BACKUP_ROOT/$(dirname "$relative_path")"
cp -a "$COMPONENT_ROOT/$relative_path" "$BACKUP_ROOT/$relative_path"
done
runner_path="$COMPONENT_ROOT/scripts/soak/run_soak.sh"
if [[ -e "$runner_path" ]]; then
mkdir -p "$BACKUP_ROOT/scripts/soak"
cp -a "$runner_path" "$BACKUP_ROOT/scripts/soak/run_soak.sh"
printf 'runner_was_present=1\n' > "$STATE_ROOT/state.env"
else
printf 'runner_was_present=0\n' > "$STATE_ROOT/state.env"
fi
if [[ -d "$COMPONENT_ROOT/custom-fixtures" ]]; then
cp -a "$COMPONENT_ROOT/custom-fixtures" "$BACKUP_ROOT/custom-fixtures"
printf 'custom_fixtures_were_present=1\n' >> "$STATE_ROOT/state.env"
else
printf 'custom_fixtures_were_present=0\n' >> "$STATE_ROOT/state.env"
fi
if [[ -d "$COMPONENT_ROOT/custom-tal-patch" ]]; then
cp -a "$COMPONENT_ROOT/custom-tal-patch" "$BACKUP_ROOT/custom-tal-patch"
printf 'patch_tools_were_present=1\n' >> "$STATE_ROOT/state.env"
else
printf 'patch_tools_were_present=0\n' >> "$STATE_ROOT/state.env"
fi
if [[ -f "$COMPONENT_ROOT/custom-tal.env.example" ]]; then
cp -a "$COMPONENT_ROOT/custom-tal.env.example" "$BACKUP_ROOT/custom-tal.env.example"
printf 'custom_env_example_was_present=1\n' >> "$STATE_ROOT/state.env"
else
printf 'custom_env_example_was_present=0\n' >> "$STATE_ROOT/state.env"
fi
if [[ "$ENABLE_CUSTOM" == "1" ]]; then
[[ -f "$COMPONENT_ROOT/.env" ]] || die "missing component .env: $COMPONENT_ROOT/.env"
cp -a "$COMPONENT_ROOT/.env" "$BACKUP_ROOT/.env"
fi
copy_payload_file "$PAYLOAD_ROOT/compose/docker-compose.yml" "$COMPONENT_ROOT/compose/docker-compose.yml"
append_custom_env_example "$COMPONENT_ROOT/.env.example"
copy_payload_file "$PAYLOAD_ROOT/scripts/soak/run_soak.sh" "$runner_path"
mkdir -p "$COMPONENT_ROOT/custom-fixtures/tal" \
"$COMPONENT_ROOT/custom-fixtures/ta" \
"$COMPONENT_ROOT/custom-fixtures/certs" \
"$COMPONENT_ROOT/custom-fixtures/services"
if [[ -d "$PAYLOAD_ROOT/custom-fixtures" ]]; then
cp -a "$PAYLOAD_ROOT/custom-fixtures/." "$COMPONENT_ROOT/custom-fixtures/"
fi
mkdir -p "$COMPONENT_ROOT/custom-tal-patch/tools"
cp -a "$PAYLOAD_ROOT/tools/." "$COMPONENT_ROOT/custom-tal-patch/tools/"
copy_payload_file "$SCRIPT_DIR/custom-tal.env.example" \
"$COMPONENT_ROOT/custom-tal.env.example"
if [[ "$ENABLE_CUSTOM" == "1" ]]; then
set_env_value "$COMPONENT_ROOT/.env" RIRS custom
set_env_value "$COMPONENT_ROOT/.env" TAL_INPUT_MODE custom-file-with-ta
set_env_value "$COMPONENT_ROOT/.env" CUSTOM_FIXTURE_HOST_DIR ../custom-fixtures
set_env_value "$COMPONENT_ROOT/.env" CUSTOM_TAL_PATH /opt/ours-rp/custom-fixtures/tal/custom.tal
set_env_value "$COMPONENT_ROOT/.env" CUSTOM_TA_PATH /opt/ours-rp/custom-fixtures/ta/custom-ta.cer
set_env_value "$COMPONENT_ROOT/.env" CUSTOM_TAL_URI https://host.docker.internal:18443/tal/custom.tal
set_env_value "$COMPONENT_ROOT/.env" HTTP_ROOT_CERT_PATHS /opt/ours-rp/custom-fixtures/certs/rrdp-ca.pem
printf 'env_was_enabled=1\n' >> "$STATE_ROOT/state.env"
else
printf 'env_was_enabled=0\n' >> "$STATE_ROOT/state.env"
fi
[[ -f "$SCRIPT_DIR/PATCH-MANIFEST.env" ]] \
|| die "missing PATCH-MANIFEST.env"
cp "$SCRIPT_DIR/PATCH-MANIFEST.env" "$marker"
printf 'patch_root=%s\n' "$PATCH_ROOT" >> "$STATE_ROOT/state.env"
printf 'applied_at_utc=%s\n' "$(date -u +%Y-%m-%dT%H:%M:%SZ)" >> "$STATE_ROOT/state.env"
echo "custom TAL patch applied to $COMPONENT_ROOT"
if [[ "$ENABLE_CUSTOM" == "1" ]]; then
echo "custom mode enabled in $COMPONENT_ROOT/.env"
else
echo "custom mode remains disabled; review custom-tal.env.example before enabling it"
fi

View File

@@ -0,0 +1,15 @@
# Copy these values into the component .env after applying the patch.
# The patch's --enable-custom option writes the same values automatically.
RIRS=custom
TAL_INPUT_MODE=custom-file-with-ta
CUSTOM_FIXTURE_HOST_DIR=../custom-fixtures
CUSTOM_TAL_PATH=/opt/ours-rp/custom-fixtures/tal/custom.tal
CUSTOM_TA_PATH=/opt/ours-rp/custom-fixtures/ta/custom-ta.cer
CUSTOM_TAL_URI=https://host.docker.internal:18443/tal/custom.tal
HTTP_ROOT_CERT_PATHS=/opt/ours-rp/custom-fixtures/certs/rrdp-ca.pem
# Recommended for a finite acceptance run; keep the customer default for a
# long-running test only after the fixture service is supervised.
MAX_RUNS=1
INTERVAL_SECS=0
SOAK_RESTART_POLICY=no

View File

@@ -0,0 +1,80 @@
#!/usr/bin/env bash
set -euo pipefail
usage() {
cat <<'USAGE'
Usage:
./rollback_custom_tal_patch.sh --component-root <installed-ours-rp-root>
Stop the ours RP service before rolling back the overlay.
USAGE
}
die() {
echo "error: $*" >&2
exit 2
}
COMPONENT_ROOT=""
while [[ $# -gt 0 ]]; do
case "$1" in
--component-root)
COMPONENT_ROOT="${2:-}"
shift 2
;;
-h|--help)
usage
exit 0
;;
*)
die "unknown option: $1"
;;
esac
done
[[ -n "$COMPONENT_ROOT" ]] || die "--component-root is required"
COMPONENT_ROOT="$(cd "$COMPONENT_ROOT" 2>/dev/null && pwd)" \
|| die "component root does not exist: $COMPONENT_ROOT"
STATE_ROOT="$COMPONENT_ROOT/.custom-tal-patch-state"
BACKUP_ROOT="$STATE_ROOT/original"
[[ -d "$STATE_ROOT" ]] || die "patch state not found: $STATE_ROOT"
[[ -f "$BACKUP_ROOT/compose/docker-compose.yml" ]] \
|| die "patch backup is incomplete: $BACKUP_ROOT"
cp -a "$BACKUP_ROOT/compose/docker-compose.yml" "$COMPONENT_ROOT/compose/docker-compose.yml"
cp -a "$BACKUP_ROOT/.env.example" "$COMPONENT_ROOT/.env.example"
if [[ -f "$BACKUP_ROOT/scripts/soak/run_soak.sh" ]]; then
mkdir -p "$COMPONENT_ROOT/scripts/soak"
cp -a "$BACKUP_ROOT/scripts/soak/run_soak.sh" "$COMPONENT_ROOT/scripts/soak/run_soak.sh"
else
rm -f "$COMPONENT_ROOT/scripts/soak/run_soak.sh"
rmdir "$COMPONENT_ROOT/scripts/soak" 2>/dev/null || true
fi
if [[ -d "$BACKUP_ROOT/custom-fixtures" ]]; then
rm -rf "$COMPONENT_ROOT/custom-fixtures"
cp -a "$BACKUP_ROOT/custom-fixtures" "$COMPONENT_ROOT/custom-fixtures"
else
rm -rf "$COMPONENT_ROOT/custom-fixtures"
fi
if [[ -d "$BACKUP_ROOT/custom-tal-patch" ]]; then
rm -rf "$COMPONENT_ROOT/custom-tal-patch"
cp -a "$BACKUP_ROOT/custom-tal-patch" "$COMPONENT_ROOT/custom-tal-patch"
else
rm -rf "$COMPONENT_ROOT/custom-tal-patch"
fi
if [[ -f "$BACKUP_ROOT/custom-tal.env.example" ]]; then
cp -a "$BACKUP_ROOT/custom-tal.env.example" "$COMPONENT_ROOT/custom-tal.env.example"
else
rm -f "$COMPONENT_ROOT/custom-tal.env.example"
fi
if [[ -f "$BACKUP_ROOT/.env" ]]; then
cp -a "$BACKUP_ROOT/.env" "$COMPONENT_ROOT/.env"
fi
rm -f "$COMPONENT_ROOT/CUSTOM-TAL-PATCH-MANIFEST.env"
rm -rf "$STATE_ROOT"
echo "custom TAL patch rolled back from $COMPONENT_ROOT"

View File

@@ -0,0 +1,636 @@
#!/usr/bin/env python3
"""Generate a small self-contained RPKI repository for installer tests.
The generator uses Python cryptography and the OpenSSL CLI for CMS signing.
It does not invoke Barry, Rapport, or a public RIR service.
"""
from __future__ import annotations
import argparse
import base64
import hashlib
import ipaddress
import shutil
import subprocess
import tempfile
from datetime import datetime, timedelta, timezone
from pathlib import Path
from cryptography import x509
from cryptography.hazmat.primitives import hashes, serialization
from cryptography.hazmat.primitives.asymmetric import rsa
from cryptography.hazmat.primitives.serialization import Encoding, NoEncryption, PrivateFormat, PublicFormat
from cryptography.x509 import AccessDescription, AuthorityInformationAccess, DistributionPoint, UniformResourceIdentifier
from cryptography.x509.oid import AuthorityInformationAccessOID, NameOID, ObjectIdentifier
RRDP_HOST = "host.docker.internal"
RRDP_PORT = 18443
RSYNC_PORT = 1873
TAL_URI = f"rsync://{RRDP_HOST}:{RSYNC_PORT}/custom/ta.cer"
TAL_HTTPS_URI = f"https://{RRDP_HOST}:{RRDP_PORT}/ta/custom-ta.cer"
RRDP_URI = f"https://{RRDP_HOST}:{RRDP_PORT}/rrdp/notification.xml"
ROOT_REPO_URI = f"rsync://{RRDP_HOST}:{RSYNC_PORT}/custom/root/"
ROOT_MFT_URI = f"{ROOT_REPO_URI}root.mft"
CHILD_REPO_URI = f"rsync://{RRDP_HOST}:{RSYNC_PORT}/custom/child/"
CHILD_MFT_URI = f"{CHILD_REPO_URI}child.mft"
OID_SIA = ObjectIdentifier("1.3.6.1.5.5.7.1.11")
OID_RESOURCES_IP = ObjectIdentifier("1.3.6.1.5.5.7.1.7")
OID_RESOURCES_AS = ObjectIdentifier("1.3.6.1.5.5.7.1.8")
OID_POLICY_IP_AS = ObjectIdentifier("1.3.6.1.5.5.7.14.2")
OID_ROA_ECONTENT = "1.2.840.113549.1.9.16.1.24"
OID_MANIFEST_ECONTENT = "1.2.840.113549.1.9.16.1.26"
OID_SHA256 = "2.16.840.1.101.3.4.2.1"
SIA_CA_REPOSITORY = "1.3.6.1.5.5.7.48.5"
SIA_RPKI_MANIFEST = "1.3.6.1.5.5.7.48.10"
SIA_SIGNED_OBJECT = "1.3.6.1.5.5.7.48.11"
SIA_RPKI_NOTIFY = "1.3.6.1.5.5.7.48.13"
UTC = timezone.utc
VALID_FROM = datetime(2026, 1, 1, tzinfo=UTC)
VALID_TO = datetime(2027, 12, 31, 23, 59, 59, tzinfo=UTC)
CHILD_PREFIX = ipaddress.ip_network("203.0.113.0/24")
SECOND_PREFIX = ipaddress.ip_network("203.0.113.128/25")
OUTSIDE_PREFIX = ipaddress.ip_network("203.0.114.0/24")
TEST_ASN = 64496
SESSION_ID = "11111111-2222-4333-8444-555555555555"
def tlv(tag: int, value: bytes) -> bytes:
length = len(value)
if length < 128:
encoded_length = bytes([length])
else:
raw = length.to_bytes((length.bit_length() + 7) // 8, "big")
encoded_length = bytes([0x80 | len(raw)]) + raw
return bytes([tag]) + encoded_length + value
def seq(*values: bytes) -> bytes:
return tlv(0x30, b"".join(values))
def integer(value: int) -> bytes:
raw = value.to_bytes(max(1, (value.bit_length() + 7) // 8), "big")
if raw[0] & 0x80:
raw = b"\x00" + raw
return tlv(0x02, raw)
def oid(oid_value: str) -> bytes:
parts = [int(part) for part in oid_value.split(".")]
encoded = bytearray([40 * parts[0] + parts[1]])
for part in parts[2:]:
chunks = [part & 0x7F]
part >>= 7
while part:
chunks.append(0x80 | (part & 0x7F))
part >>= 7
encoded.extend(reversed(chunks))
return tlv(0x06, bytes(encoded))
def octet(value: bytes) -> bytes:
return tlv(0x04, value)
def uri(value: str) -> bytes:
return tlv(0x86, value.encode("ascii"))
def bit_string(value: bytes, unused: int = 0) -> bytes:
return tlv(0x03, bytes([unused]) + value)
def generalized_time(value: datetime) -> bytes:
return tlv(0x18, value.astimezone(UTC).strftime("%Y%m%d%H%M%SZ").encode("ascii"))
def context_zero(value: bytes) -> bytes:
return tlv(0xA0, value)
def ip_choice(networks: list[ipaddress._BaseNetwork]) -> bytes:
entries = []
for network in networks:
length = (network.prefixlen + 7) // 8
raw = network.network_address.packed[:length]
entries.append(bit_string(raw, length * 8 - network.prefixlen))
return seq(*entries)
def ip_resources(networks: list[ipaddress._BaseNetwork]) -> bytes:
families: dict[int, list[ipaddress._BaseNetwork]] = {1: [], 2: []}
for network in networks:
families[1 if network.version == 4 else 2].append(network)
return seq(
*(
seq(octet(afi.to_bytes(2, "big")), ip_choice(items))
for afi, items in families.items()
if items
)
)
def ip_inherit() -> bytes:
return seq(
seq(octet(b"\x00\x01"), tlv(0x05, b"")),
seq(octet(b"\x00\x02"), tlv(0x05, b"")),
)
def as_resources(asns: list[int] | None) -> bytes:
if asns is None:
return seq(context_zero(tlv(0x05, b"")))
return seq(context_zero(seq(*(integer(asn) for asn in asns))))
def sia(entries: list[tuple[str, str]]) -> bytes:
return seq(*(seq(oid(method), uri(location)) for method, location in entries))
def write(path: Path, data: bytes, mode: int | None = None) -> None:
path.parent.mkdir(parents=True, exist_ok=True)
path.write_bytes(data)
if mode is not None:
path.chmod(mode)
def pem_cert(cert: x509.Certificate) -> bytes:
return cert.public_bytes(Encoding.PEM)
def pem_key(key: rsa.RSAPrivateKey) -> bytes:
return key.private_bytes(Encoding.PEM, PrivateFormat.TraditionalOpenSSL, NoEncryption())
def write_key(path: Path, key: rsa.RSAPrivateKey) -> None:
write(path, pem_key(key), 0o600)
def sign_cert(
subject: x509.Name,
issuer: x509.Name,
public_key,
signing_key: rsa.RSAPrivateKey,
serial: int,
is_ca: bool,
ski_key: rsa.RSAPrivateKey,
aki_key: rsa.RSAPrivateKey | None = None,
ip_ext: bytes | None = None,
as_ext: bytes | None = None,
ca_repository: str | None = None,
manifest_uri: str | None = None,
notify_uri: str | None = None,
signed_object_uri: str | None = None,
crl_uri: str | None = None,
ca_issuer_uri: str | None = None,
not_before: datetime = VALID_FROM,
not_after: datetime = VALID_TO,
) -> x509.Certificate:
builder = (
x509.CertificateBuilder()
.subject_name(subject)
.issuer_name(issuer)
.public_key(public_key)
.serial_number(serial)
.not_valid_before(not_before)
.not_valid_after(not_after)
)
if is_ca:
builder = builder.add_extension(x509.BasicConstraints(ca=True, path_length=None), critical=True)
if is_ca:
usage = x509.KeyUsage(False, False, False, False, False, True, True, False, False)
else:
usage = x509.KeyUsage(True, False, False, False, False, False, False, False, False)
builder = builder.add_extension(usage, critical=True)
builder = builder.add_extension(x509.SubjectKeyIdentifier.from_public_key(ski_key.public_key()), critical=False)
if aki_key is not None:
builder = builder.add_extension(x509.AuthorityKeyIdentifier.from_issuer_public_key(aki_key.public_key()), critical=False)
builder = builder.add_extension(
x509.CertificatePolicies([x509.PolicyInformation(OID_POLICY_IP_AS, None)]),
critical=True,
)
if crl_uri is not None:
builder = builder.add_extension(
x509.CRLDistributionPoints(
[DistributionPoint([UniformResourceIdentifier(crl_uri)], None, None, None)]
),
critical=False,
)
if ca_issuer_uri is not None:
builder = builder.add_extension(
AuthorityInformationAccess(
[AccessDescription(AuthorityInformationAccessOID.CA_ISSUERS, UniformResourceIdentifier(ca_issuer_uri))]
),
critical=False,
)
sia_entries: list[tuple[str, str]] = []
if ca_repository is not None:
sia_entries.append((SIA_CA_REPOSITORY, ca_repository))
if manifest_uri is not None:
sia_entries.append((SIA_RPKI_MANIFEST, manifest_uri))
if notify_uri is not None:
sia_entries.append((SIA_RPKI_NOTIFY, notify_uri))
if signed_object_uri is not None:
sia_entries.append((SIA_SIGNED_OBJECT, signed_object_uri))
if sia_entries:
builder = builder.add_extension(x509.UnrecognizedExtension(OID_SIA, sia(sia_entries)), critical=False)
if ip_ext is not None:
builder = builder.add_extension(x509.UnrecognizedExtension(OID_RESOURCES_IP, ip_ext), critical=True)
if as_ext is not None:
builder = builder.add_extension(x509.UnrecognizedExtension(OID_RESOURCES_AS, as_ext), critical=True)
return builder.sign(signing_key, hashes.SHA256())
def make_crl(subject: x509.Name, key: rsa.RSAPrivateKey, number: int) -> x509.CertificateRevocationList:
return (
x509.CertificateRevocationListBuilder()
.issuer_name(subject)
.last_update(VALID_FROM + timedelta(days=30))
.next_update(VALID_TO - timedelta(days=30))
.add_extension(x509.AuthorityKeyIdentifier.from_issuer_public_key(key.public_key()), critical=False)
.add_extension(x509.CRLNumber(number), critical=False)
.sign(key, hashes.SHA256())
)
def roa_content(prefix: ipaddress._BaseNetwork, max_length: int) -> bytes:
length = (prefix.prefixlen + 7) // 8
raw = prefix.network_address.packed[:length]
address = seq(bit_string(raw, length * 8 - prefix.prefixlen), integer(max_length))
family = seq(octet((1 if prefix.version == 4 else 2).to_bytes(2, "big")), seq(address))
return seq(integer(TEST_ASN), seq(family))
def manifest_content(number: int, files: list[tuple[str, bytes]]) -> bytes:
entries = [
seq(tlv(0x16, name.encode("ascii")), bit_string(hashlib.sha256(content).digest()))
for name, content in files
]
return seq(
integer(number),
generalized_time(VALID_FROM + timedelta(days=30)),
generalized_time(VALID_TO - timedelta(days=30)),
oid(OID_SHA256),
seq(*entries),
)
def write_cms(content: bytes, content_type: str, signer_cert: x509.Certificate, signer_key: rsa.RSAPrivateKey, output: Path, work: Path) -> None:
content_path = work / f"{output.name}.content.der"
signer_path = work / f"{output.name}.signer.pem"
key_path = work / f"{output.name}.key.pem"
write(content_path, content)
write(signer_path, pem_cert(signer_cert))
write_key(key_path, signer_key)
subprocess.run(
[
"openssl", "cms", "-sign", "-binary", "-in", str(content_path),
"-signer", str(signer_path), "-inkey", str(key_path),
"-outform", "DER", "-nodetach", "-nosmimecap", "-keyid", "-md", "sha256",
"-econtent_type", content_type, "-out", str(output),
],
check=True,
stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE,
)
def make_roa(
name: str,
prefix,
max_length: int,
not_before: datetime,
not_after: datetime,
child_name,
child_key,
common: Path,
work: Path,
certificate_prefix=None,
) -> None:
key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
cert = sign_cert(
x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, f"#142 EE {name}")]),
child_name,
key.public_key(),
child_key,
x509.random_serial_number(),
False,
key,
child_key,
ip_ext=ip_resources([certificate_prefix or prefix]),
signed_object_uri=f"{CHILD_REPO_URI}{name}",
crl_uri=f"{CHILD_REPO_URI}child.crl",
ca_issuer_uri=f"{CHILD_REPO_URI}child.cer",
not_before=not_before,
not_after=not_after,
)
write_cms(roa_content(prefix, max_length), OID_ROA_ECONTENT, cert, key, common / name, work)
def make_manifest(name: str, number: int, files: list[tuple[str, bytes]], child_name, child_key, common: Path, work: Path) -> bytes:
key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
cert = sign_cert(
x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, f"#142 Manifest EE {name}")]),
child_name,
key.public_key(),
child_key,
x509.random_serial_number(),
False,
key,
child_key,
ip_ext=ip_inherit(),
as_ext=as_resources(None),
signed_object_uri=CHILD_MFT_URI if name != "root" else ROOT_MFT_URI,
crl_uri=f"{CHILD_REPO_URI}child.crl" if name != "root" else f"{ROOT_REPO_URI}root.crl",
ca_issuer_uri=f"{CHILD_REPO_URI}child.cer" if name != "root" else f"{ROOT_REPO_URI}ta.cer",
)
output = common / f"{name}.mft"
write_cms(manifest_content(number, files), OID_MANIFEST_ECONTENT, cert, key, output, work)
return output.read_bytes()
def publish(uri: str, path: Path) -> str:
encoded = base64.b64encode(path.read_bytes()).decode("ascii")
return f' <publish uri="{uri}">{encoded}</publish>\n'
def snapshot(case_root: Path, names: list[str], serial: int, extra_root_names: list[str] | None = None) -> bytes:
repository = case_root / "repository"
objects = [
(f"{ROOT_REPO_URI}root.crl", repository / "root/root.crl"),
(f"{ROOT_REPO_URI}child.cer", repository / "root/child.cer"),
*[
(f"{ROOT_REPO_URI}{name}", repository / f"root/{name}")
for name in extra_root_names or []
],
(f"{CHILD_REPO_URI}child.crl", repository / "child/child.crl"),
*[(f"{CHILD_REPO_URI}{name}", repository / f"child/{name}") for name in names],
(ROOT_MFT_URI, repository / "root/root.mft"),
(CHILD_MFT_URI, repository / "child/child.mft"),
]
body = "".join(publish(uri, path) for uri, path in objects)
return (
f'<snapshot xmlns="http://www.ripe.net/rpki/rrdp" session_id="{SESSION_ID}" serial="{serial}">\n'
f"{body}</snapshot>\n"
).encode()
def notification(serial: int, snap: bytes, delta: bytes | None) -> bytes:
lines = [
f'<notification xmlns="http://www.ripe.net/rpki/rrdp" version="1" session_id="{SESSION_ID}" serial="{serial}">',
f' <snapshot uri="https://{RRDP_HOST}:{RRDP_PORT}/rrdp/snapshot.xml" hash="{hashlib.sha256(snap).hexdigest()}"/>',
]
if delta is not None:
lines.append(
f' <delta serial="{serial}" uri="https://{RRDP_HOST}:{RRDP_PORT}/rrdp/delta-{serial}.xml" hash="{hashlib.sha256(delta).hexdigest()}"/>'
)
return ("\n".join(lines) + "\n</notification>\n").encode()
def corrupt_first_hash(xml: bytes) -> bytes:
marker = b'hash="'
start = xml.index(marker) + len(marker)
end = xml.index(b'"', start)
value = bytearray(xml[start:end])
value[0] = ord("0") if value[0] != ord("0") else ord("1")
return xml[:start] + bytes(value) + xml[end:]
def delta(add_name: str, add_path: Path, new_mft: Path) -> bytes:
body = publish(CHILD_MFT_URI, new_mft)
body += publish(f"{CHILD_REPO_URI}{add_name}", add_path)
return f'<delta xmlns="http://www.ripe.net/rpki/rrdp" session_id="{SESSION_ID}" serial="2">\n{body}</delta>\n'.encode()
def build_case(
output: Path,
common: Path,
tal: bytes,
ta: bytes,
name: str,
number: int,
names: list[str],
child_mft: bytes,
change: bytes | None = None,
root_mft: bytes | None = None,
extra_root_objects: list[tuple[str, bytes]] | None = None,
notification_hash_mismatch: bool = False,
) -> None:
case_root = output / "cases" / name
repo = case_root / "repository"
(repo / "root").mkdir(parents=True, exist_ok=True)
(repo / "child").mkdir(parents=True, exist_ok=True)
shutil.copy2(common / "ta.cer", repo / "ta.cer")
for item in ("root.crl", "child.cer"):
shutil.copy2(common / item, repo / "root" / item)
(repo / "root/root.mft").write_bytes(root_mft if root_mft is not None else (common / "root.mft").read_bytes())
for item, content in extra_root_objects or []:
(repo / "root" / item).write_bytes(content)
for item in (
"child.crl", "valid.roa", "valid2.roa", "expired.roa", "max-length.roa",
"max-length-invalid.roa", "roa-prefix-outside.roa", "out-of-resource.roa", "bad-format.roa",
):
if (common / item).exists():
shutil.copy2(common / item, repo / "child" / item)
(repo / "child/child.mft").write_bytes(child_mft)
snap = snapshot(case_root, names, number, [item for item, _ in extra_root_objects or []])
(case_root / "http/rrdp").mkdir(parents=True, exist_ok=True)
(case_root / "http/tal").mkdir(parents=True, exist_ok=True)
(case_root / "http/ta").mkdir(parents=True, exist_ok=True)
(case_root / "http/rrdp/snapshot.xml").write_bytes(snap)
notification_xml = notification(number, snap, change)
if notification_hash_mismatch:
notification_xml = corrupt_first_hash(notification_xml)
(case_root / "http/rrdp/notification.xml").write_bytes(notification_xml)
if change is not None:
(case_root / "http/rrdp/delta-2.xml").write_bytes(change)
(case_root / "http/tal/custom.tal").write_bytes(tal)
(case_root / "http/ta/custom-ta.cer").write_bytes(ta)
(case_root / "CASE-MANIFEST.txt").write_text(
f"case={name}\nserial={number}\nobjects={','.join(names)}\nsnapshot_sha256={hashlib.sha256(snap).hexdigest()}\n",
encoding="utf-8",
)
def make_https_certificates(output: Path) -> None:
ca_key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
ca_name = x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, "#142 fixed RRDP test CA")])
ca_cert = (
x509.CertificateBuilder()
.subject_name(ca_name).issuer_name(ca_name).public_key(ca_key.public_key())
.serial_number(x509.random_serial_number()).not_valid_before(VALID_FROM).not_valid_after(VALID_TO)
.add_extension(x509.BasicConstraints(ca=True, path_length=1), critical=True)
.add_extension(x509.KeyUsage(False, False, False, False, False, True, True, False, False), critical=True)
.sign(ca_key, hashes.SHA256())
)
server_key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
server_name = x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, RRDP_HOST)])
try:
server_host = x509.IPAddress(ipaddress.ip_address(RRDP_HOST))
except ValueError:
server_host = x509.DNSName(RRDP_HOST)
san_entries = [server_host]
if RRDP_HOST != "localhost":
san_entries.append(x509.DNSName("localhost"))
server_cert = (
x509.CertificateBuilder()
.subject_name(server_name).issuer_name(ca_cert.subject).public_key(server_key.public_key())
.serial_number(x509.random_serial_number()).not_valid_before(VALID_FROM).not_valid_after(VALID_TO)
.add_extension(x509.BasicConstraints(ca=False, path_length=None), critical=True)
.add_extension(x509.KeyUsage(True, False, False, False, False, False, False, False, False), critical=True)
.add_extension(x509.SubjectAlternativeName(san_entries), critical=False)
.sign(ca_key, hashes.SHA256())
)
write(output / "certs/rrdp-ca.pem", pem_cert(ca_cert))
write_key(output / "certs/rrdp-ca.key", ca_key)
write(output / "certs/rrdp-server.pem", pem_cert(server_cert))
write_key(output / "certs/rrdp-server.key", server_key)
def main() -> None:
global RRDP_HOST, TAL_URI, TAL_HTTPS_URI, RRDP_URI
global ROOT_REPO_URI, ROOT_MFT_URI, CHILD_REPO_URI, CHILD_MFT_URI
parser = argparse.ArgumentParser()
parser.add_argument("--output", type=Path, required=True)
parser.add_argument(
"--host",
default=RRDP_HOST,
help="host embedded in TAL, certificate SIA and RRDP data (default: host.docker.internal)",
)
args = parser.parse_args()
RRDP_HOST = args.host
TAL_URI = f"rsync://{RRDP_HOST}:{RSYNC_PORT}/custom/ta.cer"
TAL_HTTPS_URI = f"https://{RRDP_HOST}:{RRDP_PORT}/ta/custom-ta.cer"
RRDP_URI = f"https://{RRDP_HOST}:{RRDP_PORT}/rrdp/notification.xml"
ROOT_REPO_URI = f"rsync://{RRDP_HOST}:{RSYNC_PORT}/custom/root/"
ROOT_MFT_URI = f"{ROOT_REPO_URI}root.mft"
CHILD_REPO_URI = f"rsync://{RRDP_HOST}:{RSYNC_PORT}/custom/child/"
CHILD_MFT_URI = f"{CHILD_REPO_URI}child.mft"
output = args.output.resolve()
if output.exists():
shutil.rmtree(output)
for directory in ("tal", "ta", "certs", "services", "keys"):
(output / directory).mkdir(parents=True)
root_key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
child_key = rsa.generate_private_key(public_exponent=65537, key_size=2048)
root_name = x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, "#142 Custom RPKI Root")])
child_name = x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, "#142 Custom RPKI Child")])
root_cert = sign_cert(
root_name, root_name, root_key.public_key(), root_key, 0x14200001, True, root_key,
ip_ext=ip_resources([CHILD_PREFIX]),
as_ext=as_resources([TEST_ASN]), ca_repository=ROOT_REPO_URI, manifest_uri=ROOT_MFT_URI,
notify_uri=RRDP_URI,
)
child_cert = sign_cert(
child_name, root_name, child_key.public_key(), root_key, 0x14200002, True, child_key, root_key,
ip_ext=ip_resources([CHILD_PREFIX]), as_ext=as_resources([TEST_ASN]),
ca_repository=CHILD_REPO_URI, manifest_uri=CHILD_MFT_URI, notify_uri=RRDP_URI,
crl_uri=f"{ROOT_REPO_URI}root.crl", ca_issuer_uri=f"{ROOT_REPO_URI}child.cer",
)
bad_child_name = x509.Name([x509.NameAttribute(NameOID.COMMON_NAME, "#142 Custom RPKI Overclaiming Child")])
bad_child_repo_uri = f"rsync://{RRDP_HOST}:{RSYNC_PORT}/custom/bad-child/"
bad_child_cert = sign_cert(
bad_child_name, root_name, child_key.public_key(), root_key, 0x14200003, True, child_key, root_key,
ip_ext=ip_resources([OUTSIDE_PREFIX]), as_ext=as_resources([TEST_ASN]),
ca_repository=bad_child_repo_uri, manifest_uri=f"{bad_child_repo_uri}bad-child.mft", notify_uri=RRDP_URI,
crl_uri=f"{ROOT_REPO_URI}root.crl", ca_issuer_uri=f"{ROOT_REPO_URI}child.cer",
)
root_crl = make_crl(root_name, root_key, 1)
child_crl = make_crl(child_name, child_key, 1)
tal = (
f"{TAL_URI}\n{TAL_HTTPS_URI}\n\n"
+ base64.b64encode(root_key.public_key().public_bytes(Encoding.DER, PublicFormat.SubjectPublicKeyInfo)).decode()
+ "\n"
).encode()
write(output / "tal/custom.tal", tal)
write(output / "ta/custom-ta.cer", root_cert.public_bytes(Encoding.DER))
with tempfile.TemporaryDirectory(prefix="custom-rpki-cms-") as temp:
work = Path(temp)
common = output / ".common"
common.mkdir()
write(common / "ta.cer", root_cert.public_bytes(Encoding.DER))
write(common / "child.cer", child_cert.public_bytes(Encoding.DER))
write(common / "bad-child.cer", bad_child_cert.public_bytes(Encoding.DER))
write(common / "root.crl", root_crl.public_bytes(Encoding.DER))
write(common / "child.crl", child_crl.public_bytes(Encoding.DER))
for name, prefix, max_length, before, after, certificate_prefix in (
("valid.roa", CHILD_PREFIX, 24, VALID_FROM + timedelta(days=31), VALID_TO - timedelta(days=31), None),
("valid2.roa", SECOND_PREFIX, 25, VALID_FROM + timedelta(days=31), VALID_TO - timedelta(days=31), None),
("expired.roa", CHILD_PREFIX, 24, VALID_FROM - timedelta(days=30), VALID_FROM - timedelta(days=1), None),
("max-length.roa", CHILD_PREFIX, 25, VALID_FROM + timedelta(days=31), VALID_TO - timedelta(days=31), None),
("max-length-invalid.roa", CHILD_PREFIX, 33, VALID_FROM + timedelta(days=31), VALID_TO - timedelta(days=31), None),
("roa-prefix-outside.roa", OUTSIDE_PREFIX, 24, VALID_FROM + timedelta(days=31), VALID_TO - timedelta(days=31), CHILD_PREFIX),
("out-of-resource.roa", OUTSIDE_PREFIX, 24, VALID_FROM + timedelta(days=31), VALID_TO - timedelta(days=31), None),
):
make_roa(
name, prefix, max_length, before, after, child_name, child_key, common, work,
certificate_prefix=certificate_prefix,
)
bad = bytearray((common / "valid.roa").read_bytes())
bad[-1] ^= 1
write(common / "bad-format.roa", bytes(bad))
root_mft = make_manifest(
"root", 1, [
("root.crl", (common / "root.crl").read_bytes()),
("child.cer", (common / "child.cer").read_bytes()),
], root_name, root_key, common, work,
)
write(common / "root.mft", root_mft)
bad_root_mft = make_manifest(
"root", 1, [
("root.crl", (common / "root.crl").read_bytes()),
("child.cer", (common / "child.cer").read_bytes()),
("bad-child.cer", (common / "bad-child.cer").read_bytes()),
], root_name, root_key, common, work,
)
write(common / "root.mft", root_mft)
case_specs = (
("baseline-v1", 1, ["child.crl", "valid.roa"]),
("sync-hash-mismatch", 1, ["child.crl", "valid.roa"]),
("baseline-v2", 2, ["child.crl", "valid.roa", "valid2.roa"]),
("validation-expired", 1, ["child.crl", "expired.roa"]),
("validation-max-length", 1, ["child.crl", "max-length.roa"]),
("validation-max-length-invalid", 1, ["child.crl", "max-length-invalid.roa"]),
("validation-roa-prefix-outside", 1, ["child.crl", "roa-prefix-outside.roa"]),
("validation-out-of-resource", 1, ["child.crl", "out-of-resource.roa"]),
("validation-ca-over-resource", 1, ["child.crl", "valid.roa"]),
("validation-nonstandard", 1, ["child.crl", "bad-format.roa"]),
)
child_mfts = {}
for name, number, names in case_specs:
child_mfts[name] = make_manifest(
name, number, [(item, (common / item).read_bytes()) for item in names],
child_name, child_key, common, work,
)
for name, number, names in case_specs:
change = None
if name == "baseline-v2":
change = delta("valid2.roa", common / "valid2.roa", common / "baseline-v2.mft")
build_case(
output, common, tal, root_cert.public_bytes(Encoding.DER), name, number, names,
child_mfts[name], change,
root_mft=bad_root_mft if name == "validation-ca-over-resource" else None,
extra_root_objects=[("bad-child.cer", (common / "bad-child.cer").read_bytes())]
if name == "validation-ca-over-resource" else None,
notification_hash_mismatch=name == "sync-hash-mismatch",
)
write_key(output / "keys/root.key", root_key)
write_key(output / "keys/child.key", child_key)
make_https_certificates(output)
shutil.rmtree(output / ".common")
(output / "FIXTURE-MANIFEST.txt").write_text(
"fixture_schema_version=1\nfixture_kind=custom-tal-fixed-local-rpki\n"
f"tal_uri={TAL_URI}\nrrdp_uri={RRDP_URI}\nrsync_module=custom\n"
f"rrdp_port={RRDP_PORT}\nrsync_port={RSYNC_PORT}\n"
"cases=baseline-v1,sync-hash-mismatch,baseline-v2,validation-expired,validation-max-length,validation-max-length-invalid,"
"validation-roa-prefix-outside,validation-out-of-resource,validation-ca-over-resource,validation-nonstandard\n",
encoding="utf-8",
)
print(f"generated custom fixture: {output}")
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,36 @@
#!/usr/bin/env python3
"""Serve a fixture directory over HTTPS with request logging."""
from __future__ import annotations
import argparse
import http.server
import ssl
from pathlib import Path
class RequestHandler(http.server.SimpleHTTPRequestHandler):
def log_message(self, format: str, *args) -> None:
print("%s %s" % (self.log_date_time_string(), format % args), flush=True)
def main() -> None:
parser = argparse.ArgumentParser()
parser.add_argument("--directory", type=Path, required=True)
parser.add_argument("--certfile", type=Path, required=True)
parser.add_argument("--keyfile", type=Path, required=True)
parser.add_argument("--host", default="0.0.0.0")
parser.add_argument("--port", type=int, default=18443)
args = parser.parse_args()
handler = lambda *handler_args, directory=str(args.directory): RequestHandler(
*handler_args, directory=directory
)
server = http.server.ThreadingHTTPServer((args.host, args.port), handler)
context = ssl.SSLContext(ssl.PROTOCOL_TLS_SERVER)
context.load_cert_chain(certfile=args.certfile, keyfile=args.keyfile)
server.socket = context.wrap_socket(server.socket, server_side=True)
print(f"https fixture server listening on {args.host}:{args.port} root={args.directory}", flush=True)
server.serve_forever()
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,87 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$BASH_SOURCE")" && pwd)"
FIXTURE_ROOT=""
CASE_NAME="baseline-v1"
RRDP_PORT="$(printenv RRDP_PORT 2>/dev/null || true)"
RSYNC_PORT="$(printenv RSYNC_PORT 2>/dev/null || true)"
PID_ROOT=""
[[ -n "$RRDP_PORT" ]] || RRDP_PORT=18443
[[ -n "$RSYNC_PORT" ]] || RSYNC_PORT=1873
usage() {
cat <<'USAGE'
Usage:
./start_fixed_services.sh --fixture-root <generated-fixtures> [--case NAME]
Cases: baseline-v1, sync-hash-mismatch, baseline-v2, validation-expired,
validation-max-length, validation-max-length-invalid,
validation-roa-prefix-outside, validation-out-of-resource,
validation-ca-over-resource, validation-nonstandard
Environment: RRDP_PORT (default 18443), RSYNC_PORT (default 1873)
USAGE
}
die() {
echo "error: $*" >&2
exit 2
}
while [[ $# -gt 0 ]]; do
case "$1" in
--fixture-root) FIXTURE_ROOT="$2"; shift 2 ;;
--case) CASE_NAME="$2"; shift 2 ;;
--pid-root) PID_ROOT="$2"; shift 2 ;;
-h|--help) usage; exit 0 ;;
*) die "unknown option: $1" ;;
esac
done
[[ -n "$FIXTURE_ROOT" ]] || die "--fixture-root is required"
FIXTURE_ROOT="$(cd "$FIXTURE_ROOT" 2>/dev/null && pwd)" || die "fixture root not found"
CASE_ROOT="$FIXTURE_ROOT/cases/$CASE_NAME"
[[ -d "$CASE_ROOT/repository" && -d "$CASE_ROOT/http" ]] || die "fixture case not found: $CASE_NAME"
if [[ -z "$PID_ROOT" ]]; then
PID_ROOT="$FIXTURE_ROOT/services/$CASE_NAME"
fi
mkdir -p "$PID_ROOT"
[[ ! -f "$PID_ROOT/pids.env" ]] || die "services already appear to be running: $PID_ROOT/pids.env"
rm -f "$PID_ROOT/rsyncd.pid" "$PID_ROOT/rsyncd.lock"
RSYNCD_CONF="$PID_ROOT/rsyncd.conf"
RSYNC_UID="$(id -un)"
RSYNC_GID="$(id -gn)"
cat > "$RSYNCD_CONF" <<EOF
uid = $RSYNC_UID
gid = $RSYNC_GID
use chroot = no
read only = yes
port = $RSYNC_PORT
pid file = $PID_ROOT/rsyncd.pid
lock file = $PID_ROOT/rsyncd.lock
log file = $PID_ROOT/rsyncd.log
[custom]
path = $CASE_ROOT/repository
comment = #142 fixed custom RPKI repository
read only = yes
EOF
rsync --daemon --no-detach --config="$RSYNCD_CONF" >"$PID_ROOT/rsyncd.stdout.log" 2>&1 &
RSYNC_PID=$!
python3 "$SCRIPT_DIR/serve_https.py" \
--directory "$CASE_ROOT/http" \
--certfile "$FIXTURE_ROOT/certs/rrdp-server.pem" \
--keyfile "$FIXTURE_ROOT/certs/rrdp-server.key" \
--port "$RRDP_PORT" \
>"$PID_ROOT/https.log" 2>&1 &
HTTPS_PID=$!
printf 'case=%s\nrrdp_pid=%s\nrsync_pid=%s\nrrdp_port=%s\nrsync_port=%s\n' \
"$CASE_NAME" "$HTTPS_PID" "$RSYNC_PID" "$RRDP_PORT" "$RSYNC_PORT" > "$PID_ROOT/pids.env"
sleep 1
kill -0 "$HTTPS_PID" 2>/dev/null || die "HTTPS fixture server exited; see $PID_ROOT/https.log"
kill -0 "$RSYNC_PID" 2>/dev/null || die "rsync fixture server exited; see $PID_ROOT/rsyncd.stdout.log"
echo "fixed RRDP/rsync services started case=$CASE_NAME rrdp=$RRDP_PORT rsync=$RSYNC_PORT pid_root=$PID_ROOT"

View File

@@ -0,0 +1,23 @@
#!/usr/bin/env bash
set -euo pipefail
PID_ROOT="$1"
[[ -n "$PID_ROOT" ]] || { echo "Usage: $0 <pid-root>" >&2; exit 2; }
[[ -f "$PID_ROOT/pids.env" ]] || { echo "services are not running: $PID_ROOT" >&2; exit 2; }
# shellcheck disable=SC1090
source "$PID_ROOT/pids.env"
service_pids=("${rrdp_pid:-}" "${rsync_pid:-}")
for pid in "${service_pids[@]}"; do
if [[ "$pid" =~ ^[0-9]+$ ]] && kill -0 "$pid" 2>/dev/null; then
kill "$pid" 2>/dev/null || true
fi
done
for pid in "${service_pids[@]}"; do
[[ "$pid" =~ ^[0-9]+$ ]] || continue
for _ in $(seq 1 50); do
kill -0 "$pid" 2>/dev/null || break
sleep 0.1
done
done
rm -f "$PID_ROOT/pids.env"
echo "fixed RRDP/rsync services stopped: $PID_ROOT"

View File

@@ -1,107 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
usage() {
cat <<'EOF'
Usage:
check_rpki_client_ccr.sh \
--rpki-client <path> \
--ccr-root <directory-or-ccr-file> \
--out <result.jsonl> \
[--limit <count>]
Runs the supplied rpki-client importer against CCR files. A non-zero result
means that rpki-client reported a CCR-file parsing error or could not start.
Per-file command output is retained next to the JSONL result file.
EOF
}
RPKI_CLIENT=""
CCR_ROOT=""
OUT=""
LIMIT=0
while [[ $# -gt 0 ]]; do
case "$1" in
--rpki-client) RPKI_CLIENT=${2:?missing value for --rpki-client}; shift 2 ;;
--ccr-root) CCR_ROOT=${2:?missing value for --ccr-root}; shift 2 ;;
--out) OUT=${2:?missing value for --out}; shift 2 ;;
--limit) LIMIT=${2:?missing value for --limit}; shift 2 ;;
-h|--help) usage; exit 0 ;;
*) echo "error: unknown argument: $1" >&2; usage >&2; exit 2 ;;
esac
done
[[ -n "$RPKI_CLIENT" && -n "$CCR_ROOT" && -n "$OUT" ]] || {
echo "error: --rpki-client, --ccr-root, and --out are required" >&2
usage >&2
exit 2
}
[[ -x "$RPKI_CLIENT" ]] || { echo "error: rpki-client is not executable: $RPKI_CLIENT" >&2; exit 2; }
[[ "$LIMIT" =~ ^[0-9]+$ ]] || { echo "error: --limit must be a non-negative integer" >&2; exit 2; }
mkdir -p "$(dirname "$OUT")"
LOG_DIR="${OUT%.jsonl}.logs"
mkdir -p "$LOG_DIR"
: > "$OUT"
declare -a CCR_FILES=()
if [[ -f "$CCR_ROOT" ]]; then
CCR_FILES=("$(readlink -f "$CCR_ROOT")")
elif [[ -d "$CCR_ROOT" ]]; then
while IFS= read -r -d '' file; do
CCR_FILES+=("$file")
done < <(find "$CCR_ROOT" -type f -name '*.ccr' -print0 | sort -z)
else
echo "error: CCR root does not exist: $CCR_ROOT" >&2
exit 2
fi
if (( LIMIT > 0 && ${#CCR_FILES[@]} > LIMIT )); then
CCR_FILES=("${CCR_FILES[@]:0:LIMIT}")
fi
if (( ${#CCR_FILES[@]} == 0 )); then
echo "error: no .ccr files found under: $CCR_ROOT" >&2
exit 2
fi
passed=0
failed=0
for ccr in "${CCR_FILES[@]}"; do
digest=$(sha256sum "$ccr" | awk '{print $1}')
log="$LOG_DIR/${digest}.log"
set +e
"$RPKI_CLIENT" -f "$ccr" >"$log" 2>&1
exit_code=$?
set -e
classification=pass
if (( exit_code != 0 )); then
classification=tool_error
elif grep -Fq "rpki-client: ${ccr}:" "$log"; then
classification=ccr_parse_error
fi
if [[ "$classification" == pass ]]; then
((passed += 1))
else
((failed += 1))
fi
python3 - "$OUT" "$ccr" "$digest" "$exit_code" "$classification" "$log" <<'PY'
import json
import sys
out, ccr, sha256, exit_code, classification, log = sys.argv[1:]
with open(out, "a", encoding="utf-8") as stream:
stream.write(json.dumps({
"ccr": ccr,
"sha256": sha256,
"rpkiClientExitCode": int(exit_code),
"classification": classification,
"log": log,
}, ensure_ascii=False, sort_keys=True) + "\n")
PY
done
echo "checked=${#CCR_FILES[@]} passed=${passed} failed=${failed} jsonl=${OUT} logs=${LOG_DIR}"
(( failed == 0 ))

View File

@@ -0,0 +1,146 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "$BASH_SOURCE")" && pwd)"
REPO_ROOT="$(cd "$SCRIPT_DIR/../.." && pwd)"
PATCH_SOURCE="$REPO_ROOT/deploy/docker-installer/custom-tal-patch"
BASE_COMPONENT_PACKAGE=""
OUT_DIR="$(printenv OUT_DIR 2>/dev/null || true)"
PREFIX="$(printenv PREFIX 2>/dev/null || true)"
ALLOW_DIRTY=0
[[ -n "$OUT_DIR" ]] || OUT_DIR="$REPO_ROOT/target/custom-tal-patch"
[[ -n "$PREFIX" ]] || PREFIX="ours-rp-custom-tal-patch"
usage() {
cat <<'USAGE'
Usage:
scripts/docker/build_custom_tal_patch.sh \
--base-component-package <ours-rp-installer-*.tar.gz> \
[--out-dir <dir>] [--prefix <name>] [--allow-dirty]
The patch targets the Arm64 component package from the customer stack. It
overlays the runner, Compose, fixture mount and test-service tools; it does not
rebuild or replace the runtime image.
USAGE
}
die() {
echo "error: $*" >&2
exit 2
}
while [[ $# -gt 0 ]]; do
case "$1" in
--base-component-package) BASE_COMPONENT_PACKAGE="$2"; shift 2 ;;
--out-dir) OUT_DIR="$2"; shift 2 ;;
--prefix) PREFIX="$2"; shift 2 ;;
--allow-dirty) ALLOW_DIRTY=1; shift ;;
-h|--help) usage; exit 0 ;;
*) die "unknown option: $1" ;;
esac
done
[[ -f "$BASE_COMPONENT_PACKAGE" ]] || die "missing base component package: $BASE_COMPONENT_PACKAGE"
[[ -d "$PATCH_SOURCE" ]] || die "missing patch source: $PATCH_SOURCE"
SOURCE_COMMIT="$(git -C "$REPO_ROOT" rev-parse HEAD)"
SOURCE_COMMIT_SHORT="$(git -C "$REPO_ROOT" rev-parse --short=8 HEAD)"
SOURCE_DIRTY=false
if [[ -n "$(git -C "$REPO_ROOT" status --short)" ]]; then
SOURCE_DIRTY=true
fi
if [[ "$SOURCE_DIRTY" == true && "$ALLOW_DIRTY" != 1 ]]; then
die "source worktree is dirty; use --allow-dirty for a development patch"
fi
package_root="$(tar -tzf "$BASE_COMPONENT_PACKAGE" | awk -F/ '!seen { print $1; seen = 1 }')"
[[ -n "$package_root" ]] || die "cannot determine component package root"
manifest_text="$(tar -xOf "$BASE_COMPONENT_PACKAGE" "$package_root/PACKAGE-MANIFEST.env")" \
|| die "base component package has no PACKAGE-MANIFEST.env"
read_manifest_value() {
printf '%s\n' "$manifest_text" | awk -F= -v key="$1" '$1 == key { print substr($0, index($0, "=") + 1); exit }'
}
base_source_commit="$(read_manifest_value source_commit)"
base_package_arch="$(read_manifest_value package_arch)"
base_runtime_image="$(read_manifest_value runtime_image)"
[[ "$base_source_commit" == 9f4f4cf069e9fa8f6f97ed0f1065adac38abe82f ]] \
|| die "base source commit mismatch: $base_source_commit"
[[ "$base_package_arch" == arm64 ]] || die "base package must be arm64: $base_package_arch"
timestamp="$(date -u +%Y%m%dT%H%M%SZ)"
patch_name="$PREFIX-$SOURCE_COMMIT_SHORT"
if [[ "$SOURCE_DIRTY" == true ]]; then
patch_name="$patch_name-dirty"
fi
patch_name="$patch_name-$timestamp"
stage="$OUT_DIR/$patch_name"
tar_path="$OUT_DIR/$patch_name.tar.gz"
base_sha256="$(sha256sum "$BASE_COMPONENT_PACKAGE" | awk '{ print $1 }')"
rm -rf "$stage"
mkdir -p "$stage/payload/compose" "$stage/payload/scripts/soak" \
"$stage/payload/custom-fixtures" "$stage/payload/tools"
cp "$PATCH_SOURCE/apply_custom_tal_patch.sh" "$stage/"
cp "$PATCH_SOURCE/rollback_custom_tal_patch.sh" "$stage/"
cp "$PATCH_SOURCE/custom-tal.env.example" "$stage/"
cp "$REPO_ROOT/deploy/docker-installer/compose/docker-compose.yml" "$stage/payload/compose/"
cp "$REPO_ROOT/scripts/soak/run_soak.sh" "$stage/payload/scripts/soak/"
cp "$REPO_ROOT/deploy/docker-installer/custom-fixtures/README.md" "$stage/payload/custom-fixtures/"
cp "$PATCH_SOURCE/tools/generate_custom_fixture.py" "$stage/payload/tools/"
cp "$PATCH_SOURCE/tools/serve_https.py" "$stage/payload/tools/"
cp "$PATCH_SOURCE/tools/start_fixed_services.sh" "$stage/payload/tools/"
cp "$PATCH_SOURCE/tools/stop_fixed_services.sh" "$stage/payload/tools/"
chmod +x "$stage/apply_custom_tal_patch.sh" "$stage/rollback_custom_tal_patch.sh" \
"$stage/payload/scripts/soak/run_soak.sh" "$stage/payload/tools/"*.sh \
"$stage/payload/tools/"*.py
{
echo "patch_schema_version=1"
echo "patch_name=$patch_name"
echo "created_at_utc=$timestamp"
echo "source_commit=$SOURCE_COMMIT"
echo "source_commit_short=$SOURCE_COMMIT_SHORT"
echo "source_dirty=$SOURCE_DIRTY"
echo "base_component_archive=$(basename "$BASE_COMPONENT_PACKAGE")"
echo "base_component_archive_sha256=$base_sha256"
echo "base_component_root=$package_root"
echo "base_source_commit=$base_source_commit"
echo "base_package_arch=$base_package_arch"
echo "base_runtime_image=$base_runtime_image"
echo "overlay_runtime_image_unchanged=true"
echo "custom_mode=RIRS=custom,TAL_INPUT_MODE=custom-file-with-ta"
echo "rrdp_port=18443"
echo "rsync_port=1873"
echo "payload_root=payload"
while IFS= read -r relative_path; do
safe_name="$(printf '%s' "$relative_path" | tr '/.' '__')"
file_sha256="$(sha256sum "$stage/payload/$relative_path" | awk '{ print $1 }')"
echo "payload_"$safe_name"_sha256=$file_sha256"
done < <(cd "$stage/payload" && find . -type f -printf '%P\n' | sort)
} > "$stage/PATCH-MANIFEST.env"
cat > "$stage/PATCH-SUMMARY.txt" <<EOF
patch_name: $patch_name
base_component_archive: $(basename "$BASE_COMPONENT_PACKAGE")
base_component_archive_sha256: $base_sha256
base_source_commit: $base_source_commit
source_commit: $SOURCE_COMMIT
source_dirty: $SOURCE_DIRTY
runtime_image: $base_runtime_image
runtime_image_changed: false
custom_fixture_generator: payload/tools/generate_custom_fixture.py
fixed_services: payload/tools/start_fixed_services.sh
EOF
tar -C "$OUT_DIR" -czf "$tar_path" "$patch_name"
patch_sha256="$(sha256sum "$tar_path" | awk '{ print $1 }')"
{
echo "patch=$tar_path"
echo "patch_dir=$stage"
echo "patch_sha256=$patch_sha256"
echo "manifest=$stage/PATCH-MANIFEST.env"
echo "base_component_archive_sha256=$base_sha256"
echo "source_commit=$SOURCE_COMMIT"
echo "source_dirty=$SOURCE_DIRTY"
} > "$OUT_DIR/$patch_name.summary.env"
echo "patch built: $tar_path"

View File

@@ -1,22 +1,18 @@
# rpctl # rpctl
`rpctl` 是五款 RP 本地控制 CLI,用于在实验服务器本地独立启动、停止和观察单个 RP/profile 的 all5 实验。一个实验可包含一次 snapshot 和多次复用 state 的子 run。 `rpctl` 是四款 RP 本地控制 CLI,用于在实验服务器本地独立启动、停止和观察单个 RP/profile 的 all5 实验。一个实验可包含一次 snapshot 和多次 delta 子 run。
## 部署 ## 部署
```bash ```bash
mkdir -p /data/rpki_4rp_ctl mkdir -p /root/rpki_4rp_ctl
rsync -a rpki_2/rpki/scripts/rpctl/ root@47.251.127.231:/data/rpki_4rp_ctl/ rsync -a rpki_2/rpki/scripts/rpctl/ root@47.77.237.41:/root/rpki_4rp_ctl/
rsync -a artifacts/rpki-prover/rpki-prover-6857d4bf-linux-amd64.docker.tar.gz root@47.251.127.231:/data/ rsync -a specs/develop/20260707_3/feature100_four_rp_interval_archive/runtime_bundle/ root@47.77.237.41:/root/rpki_4rp_ctl_runtime_bundle/
ssh root@47.251.127.231 'cd /data/rpki_4rp_ctl && ./rpctl init \ ssh root@47.77.237.41 'cd /root/rpki_4rp_ctl && ./rpctl init --bundle /root/rpki_4rp_ctl_runtime_bundle'
--bundle /data/rpki_4rp_ctl_runtime_bundle \
--rpki-prover-image /data/rpki-prover-6857d4bf-linux-amd64.docker.tar.gz'
``` ```
`--bundle` 目录需要包含 #100 runtime bundle 的 `bin/`、`lib/`、`fixtures/`、`scripts/` 和 `versions.json`。不要使用早期 precheck 目录作为标准 bundle,因为它可能缺少 FORT 运行所需动态库。 `--bundle` 目录需要包含 #100 runtime bundle 的 `bin/`、`lib/`、`fixtures/`、`scripts/` 和 `versions.json`。不要使用早期 precheck 目录作为标准 bundle,因为它可能缺少 FORT 运行所需动态库。
`--rpki-prover-image` 是可选项;不配置时原有四款 RP 仍可运行,但启动 rpki-prover 会明确报错。配置后 `init` 会复制并加载离线镜像,校验宿主机和镜像均为 `amd64`,确认 tag 为 `rpki-prover:6857d4bf`、版本为 `rpki-prover-0.10.1`,并将 image ID 与 archive SHA-256 写入 `config.json`。运行时不会在线拉取镜像。
## 常用命令 ## 常用命令
```bash ```bash
@@ -24,8 +20,6 @@ ssh root@47.251.127.231 'cd /data/rpki_4rp_ctl && ./rpctl init \
./rpctl status ./rpctl status
./rpctl start --rp ours-pp-object-cache --interval 10m --runs 6 --rirs all5 ./rpctl start --rp ours-pp-object-cache --interval 10m --runs 6 --rirs all5
./rpctl start --rp routinator --interval 0 --runs 1 --rirs apnic ./rpctl start --rp routinator --interval 0 --runs 1 --rirs apnic
./rpctl start --rp rpki-prover --interval 0 --runs 2 --rirs all5
./rpctl start --rp rpki-prover-full --interval 0 --runs 1 --rirs all5
./rpctl continue --exp-id <successful-exp-id> --runs 3 --interval 10m ./rpctl continue --exp-id <successful-exp-id> --runs 3 --interval 10m
./rpctl tail ./rpctl tail
./rpctl show ./rpctl show
@@ -46,10 +40,6 @@ ssh root@47.251.127.231 'cd /data/rpki_4rp_ctl && ./rpctl init \
| `routinator` | `routinator-latest-release` | | `routinator` | `routinator-latest-release` |
| `rpki-client` | `rpki-client-latest-release` | | `rpki-client` | `rpki-client-latest-release` |
| `fort` | `fort-latest-release` | | `fort` | `fort-latest-release` |
| `rpki-prover`, `rpki-prover-incremental` | `rpki-prover-incremental` |
| `rpki-prover-full`, `rpki-prover-no-incremental` | `rpki-prover-full` |
`rpki-prover-incremental` 使用上游默认增量算法;`rpki-prover-full` 显式传入 `--no-incremental-validation`。两者的 state 位于各自实验/profile 目录,互不复用。首轮显示为 `snapshot`;后续和 `continue` 追加轮显示为 `warm`,表示保留了上轮 state,实际 RRDP snapshot/delta 决策仍由 rpki-prover 自身完成。
## 数据目录 ## 数据目录
@@ -63,9 +53,7 @@ ssh root@47.251.127.231 'cd /data/rpki_4rp_ctl && ./rpctl init \
runs/<profile>/<exp-id>/profiles/<profile>/runs/run_0001/ runs/<profile>/<exp-id>/profiles/<profile>/runs/run_0001/
``` ```
每个子 run 目录包含 `stdout.log`、`stderr.log`、`exit-code.txt`、`run-meta.json` 以及各 RP 产物。传统四 RP 使用 `process-time.txt`;rpki-prover 额外保存 `container-inspect.json`、`rss-samples.tsv` 和 `vrps.csv`。 每个子 run 目录包含 `stdout.log`、`stderr.log`、`process-time.txt`、`exit-code.txt`、`run-meta.json` 以及各 RP 产物。
rpki-prover 的 `run-meta.json` 和 `view` 同时记录 VRP 原始行数与按 `(ASN, prefix, maxLength)` 去重数。其当前 CLI 不输出 VAP,所以统一显示 `n/a`,不是 `0`。max RSS 是容器全部相关 PID 的 RSS 峰值汇总,不是短暂 Docker client 进程的 RSS。
## 历史查看与删除 ## 历史查看与删除

View File

@@ -4,10 +4,8 @@ from __future__ import annotations
import argparse import argparse
import csv import csv
import gzip import gzip
import hashlib
import json import json
import os import os
import platform
import re import re
import shutil import shutil
import signal import signal
@@ -39,8 +37,6 @@ PROFILES = {
"routinator-latest-release": "routinator", "routinator-latest-release": "routinator",
"rpki-client-latest-release": "rpki-client", "rpki-client-latest-release": "rpki-client",
"fort-latest-release": "fort", "fort-latest-release": "fort",
"rpki-prover-incremental": "rpki-prover",
"rpki-prover-full": "rpki-prover",
} }
ALIASES = { ALIASES = {
@@ -56,15 +52,8 @@ ALIASES = {
"rpki-client-latest-release": "rpki-client-latest-release", "rpki-client-latest-release": "rpki-client-latest-release",
"fort": "fort-latest-release", "fort": "fort-latest-release",
"fort-latest-release": "fort-latest-release", "fort-latest-release": "fort-latest-release",
"rpki-prover": "rpki-prover-incremental",
"rpki-prover-incremental": "rpki-prover-incremental",
"rpki-prover-full": "rpki-prover-full",
"rpki-prover-no-incremental": "rpki-prover-full",
} }
RPKI_PROVER_IMAGE_TAG = "rpki-prover:6857d4bf"
RPKI_PROVER_VERSION = "rpki-prover-0.10.1"
DEFAULT_CONFIG = { DEFAULT_CONFIG = {
"root": "/root/rpki_4rp_ctl", "root": "/root/rpki_4rp_ctl",
"bundle": "", "bundle": "",
@@ -218,14 +207,6 @@ def command_string(argv: list[str]) -> str:
return " ".join(shlex.quote(x) for x in argv) return " ".join(shlex.quote(x) for x in argv)
def sha256_file(path: Path) -> str:
digest = hashlib.sha256()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
digest.update(chunk)
return digest.hexdigest()
def check_bundle(root: Path) -> None: def check_bundle(root: Path) -> None:
required = ["bin/rpki", "bin/routinator", "bin/rpki-client", "bin/fort", "fixtures/tal", "scripts/normalize_and_collect.py", "scripts/write_group_report.py", "versions.json"] required = ["bin/rpki", "bin/routinator", "bin/rpki-client", "bin/fort", "fixtures/tal", "scripts/normalize_and_collect.py", "scripts/write_group_report.py", "versions.json"]
missing = [item for item in required if not (root / item).exists()] missing = [item for item in required if not (root / item).exists()]
@@ -251,102 +232,6 @@ def install_from_bundle(bundle: Path) -> None:
item.chmod(item.stat().st_mode | 0o111) item.chmod(item.stat().st_mode | 0o111)
def validate_rpki_prover_host() -> None:
arch = platform.machine().lower()
if arch not in {"x86_64", "amd64"}:
raise SystemExit(f"rpki-prover image requires amd64 host, found: {arch}")
if shutil.which("docker") is None:
raise SystemExit("docker is required for rpki-prover profiles")
result = subprocess.run(
["docker", "info"],
stdout=subprocess.DEVNULL,
stderr=subprocess.PIPE,
text=True,
check=False,
)
if result.returncode != 0:
raise SystemExit(f"docker is not available: {result.stderr.strip()}")
def inspect_rpki_prover_image() -> dict[str, str]:
inspect = subprocess.run(
["docker", "image", "inspect", RPKI_PROVER_IMAGE_TAG],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
check=False,
)
if inspect.returncode != 0:
raise SystemExit(f"loaded image is missing tag {RPKI_PROVER_IMAGE_TAG}: {inspect.stderr.strip()}")
data = json.loads(inspect.stdout)[0]
architecture = str(data.get("Architecture") or "")
if architecture != "amd64":
raise SystemExit(f"rpki-prover image architecture mismatch: expected amd64, found {architecture or '<unknown>'}")
version = subprocess.run(
["docker", "run", "--rm", "--entrypoint", "/opt/app/rpki-prover", RPKI_PROVER_IMAGE_TAG, "--version"],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
check=False,
)
version_text = version.stdout.strip()
if version.returncode != 0 or version_text != RPKI_PROVER_VERSION:
raise SystemExit(
f"rpki-prover version check failed: expected {RPKI_PROVER_VERSION}, "
f"exit={version.returncode} output={version_text or version.stderr.strip()}"
)
return {
"tag": RPKI_PROVER_IMAGE_TAG,
"imageId": str(data.get("Id") or ""),
"architecture": architecture,
"version": version_text,
}
def install_rpki_prover_image(archive: Path) -> dict[str, str]:
if not archive.is_file():
raise SystemExit(f"rpki-prover image archive does not exist: {archive}")
validate_rpki_prover_host()
image_dir = ctl_root() / "images"
image_dir.mkdir(parents=True, exist_ok=True)
destination = image_dir / archive.name
if archive != destination:
shutil.copy2(archive, destination)
load = subprocess.run(
["docker", "load", "--input", str(destination)],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
check=False,
)
if load.returncode != 0:
raise SystemExit(f"failed to load rpki-prover image archive: {load.stderr.strip()}")
metadata = inspect_rpki_prover_image()
metadata.update({
"archive": str(destination),
"archiveSha256": sha256_file(destination),
"loadedAtUtc": utc_now(),
})
return metadata
def require_rpki_prover_image() -> dict[str, Any]:
metadata = load_config().get("rpkiProverImage")
if not isinstance(metadata, dict) or metadata.get("tag") != RPKI_PROVER_IMAGE_TAG:
raise SystemExit(
"rpki-prover image is not initialized; run ./rpctl init --bundle <bundle> "
"--rpki-prover-image <docker.tar.gz>"
)
validate_rpki_prover_host()
inspected = inspect_rpki_prover_image()
if metadata.get("imageId") and metadata.get("imageId") != inspected.get("imageId"):
raise SystemExit(
f"rpki-prover image id changed: configured={metadata.get('imageId')} "
f"loaded={inspected.get('imageId')}"
)
return metadata
def cmd_init(args: argparse.Namespace) -> None: def cmd_init(args: argparse.Namespace) -> None:
ensure_dirs() ensure_dirs()
bundle = Path(args.bundle).resolve() bundle = Path(args.bundle).resolve()
@@ -354,16 +239,9 @@ def cmd_init(args: argparse.Namespace) -> None:
cfg = load_config() cfg = load_config()
cfg["bundle"] = str(bundle) cfg["bundle"] = str(bundle)
cfg["initializedAtUtc"] = utc_now() cfg["initializedAtUtc"] = utc_now()
if args.rpki_prover_image:
cfg["rpkiProverImage"] = install_rpki_prover_image(Path(args.rpki_prover_image).resolve())
save_config(cfg) save_config(cfg)
print(f"initialized root={ctl_root()}") print(f"initialized root={ctl_root()}")
print(f"bundle={bundle}") print(f"bundle={bundle}")
if cfg.get("rpkiProverImage"):
image = cfg["rpkiProverImage"]
print(f"rpki_prover_image={image.get('tag')}")
print(f"rpki_prover_image_id={image.get('imageId')}")
print(f"rpki_prover_archive_sha256={image.get('archiveSha256')}")
def timed_run(run_dir: Path, argv: list[str], env: dict[str, str] | None = None) -> int: def timed_run(run_dir: Path, argv: list[str], env: dict[str, str] | None = None) -> int:
@@ -518,240 +396,6 @@ def build_fort_args(run_dir: Path, state: Path, rirs: list[str]) -> tuple[list[s
return argv, env return argv, env
def prepare_rpki_prover_state(state: Path, rirs: list[str]) -> None:
tal_dir = state / "tals"
tal_dir.mkdir(parents=True, exist_ok=True)
for old_tal in tal_dir.glob("*.tal"):
old_tal.unlink()
for rir in rirs:
shutil.copy2(ctl_root() / "fixtures" / "tal" / RIR_TAL[rir], tal_dir / RIR_TAL[rir])
def rpki_prover_arguments(profile: str) -> list[str]:
argv = [
"--once",
"--vrp-output", "/outputs/vrps.csv",
"--no-rir-tals",
"--rpki-root-directory", "/rpki-data",
"--cpu-count", "4",
"--fetcher-count", "8",
"--log-level", "info",
]
if profile == "rpki-prover-full":
argv.append("--no-incremental-validation")
return argv
def build_rpki_prover_command(profile: str, run_dir: Path, state: Path, container_name: str) -> list[str]:
return [
"docker", "run", "--name", container_name,
"-v", f"{state.resolve()}:/rpki-data",
"-v", f"{run_dir.resolve()}:/outputs",
RPKI_PROVER_IMAGE_TAG,
*rpki_prover_arguments(profile),
]
def container_rss_kb(container_name: str) -> tuple[int, int]:
top = subprocess.run(
["docker", "top", container_name, "-eo", "pid"],
stdout=subprocess.PIPE,
stderr=subprocess.DEVNULL,
text=True,
check=False,
)
total = 0
maximum = 0
for line in top.stdout.splitlines():
pid = line.strip()
if not pid.isdigit():
continue
try:
status = Path(f"/proc/{pid}/status").read_text(encoding="utf-8", errors="replace")
except OSError:
continue
match = re.search(r"^VmRSS:\s+(\d+)\s+kB$", status, re.MULTILINE)
if not match:
continue
rss = int(match.group(1))
total += rss
maximum = max(maximum, rss)
return total, maximum
def timed_rpki_prover_run(
profile: str,
run_dir: Path,
state: Path,
container_name: str,
) -> tuple[int, int, int, int, dict[str, Any]]:
run_dir.mkdir(parents=True, exist_ok=True)
start_utc = utc_now()
start_ns = time.monotonic_ns()
(run_dir / "start-utc.txt").write_text(start_utc + "\n", encoding="utf-8")
create_argv = [
"docker", "create", "--name", container_name,
"-v", f"{state.resolve()}:/rpki-data",
"-v", f"{run_dir.resolve()}:/outputs",
RPKI_PROVER_IMAGE_TAG,
*rpki_prover_arguments(profile),
]
create = subprocess.run(create_argv, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True, check=False)
if create.returncode != 0:
(run_dir / "stderr.log").write_text(create.stderr, encoding="utf-8")
(run_dir / "stdout.log").write_text(create.stdout, encoding="utf-8")
(run_dir / "end-utc.txt").write_text(utc_now() + "\n", encoding="utf-8")
(run_dir / "exit-code.txt").write_text(str(create.returncode) + "\n", encoding="utf-8")
return create.returncode, 0, 0, (time.monotonic_ns() - start_ns) // 1_000_000, {}
previous_sigterm = signal.getsignal(signal.SIGTERM)
def terminate_container(signum: int, _frame: Any) -> None:
subprocess.run(
["docker", "rm", "--force", container_name],
stdout=subprocess.DEVNULL,
stderr=subprocess.DEVNULL,
check=False,
)
raise SystemExit(128 + signum)
signal.signal(signal.SIGTERM, terminate_container)
max_total_rss_kb = 0
max_process_rss_kb = 0
inspect_data: dict[str, Any] = {}
samples_path = run_dir / "rss-samples.tsv"
with (run_dir / "stdout.log").open("wb") as stdout, (run_dir / "stderr.log").open("wb") as stderr, samples_path.open("w", encoding="utf-8") as samples:
samples.write("epoch_ms\ttotal_rss_kb\tmax_process_rss_kb\n")
start = subprocess.Popen(["docker", "start", "--attach", container_name], stdout=stdout, stderr=stderr)
while start.poll() is None:
total_rss_kb, process_rss_kb = container_rss_kb(container_name)
max_total_rss_kb = max(max_total_rss_kb, total_rss_kb)
max_process_rss_kb = max(max_process_rss_kb, process_rss_kb)
samples.write(f"{int(time.time() * 1000)}\t{total_rss_kb}\t{process_rss_kb}\n")
samples.flush()
time.sleep(0.25)
start.wait()
try:
inspect = subprocess.run(
["docker", "inspect", container_name],
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
check=False,
)
if inspect.returncode == 0:
parsed = json.loads(inspect.stdout)
if parsed:
inspect_data = parsed[0]
write_json(run_dir / "container-inspect.json", inspect_data)
state_data = inspect_data.get("State", {}) if inspect_data else {}
exit_code = int(state_data.get("ExitCode", start.returncode if start.returncode is not None else 1))
finally:
subprocess.run(["docker", "rm", "--force", container_name], stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL, check=False)
signal.signal(signal.SIGTERM, previous_sigterm)
wall_ms = (time.monotonic_ns() - start_ns) // 1_000_000
(run_dir / "end-utc.txt").write_text(utc_now() + "\n", encoding="utf-8")
(run_dir / "exit-code.txt").write_text(str(exit_code) + "\n", encoding="utf-8")
return exit_code, max_total_rss_kb, max_process_rss_kb, wall_ms, inspect_data
def collect_rpki_prover_vrps(path: Path) -> tuple[int, set[str]]:
unique: set[str] = set()
raw = 0
if not path.is_file():
return raw, unique
with path.open("r", encoding="utf-8", errors="replace", newline="") as handle:
for row in csv.reader(handle):
if not row or row[0].strip().lower() == "asn":
continue
if len(row) < 3:
continue
raw += 1
unique.add(",".join(value.strip() for value in row[:3]))
return raw, unique
def write_rpki_prover_run_metadata(
run_root: Path,
profile: str,
run_seq: int,
mode: str,
wall_ms: int,
max_rss_kb: int,
max_process_rss_kb: int,
exit_code: int,
) -> dict[str, Any]:
run_dir = run_root / "profiles" / profile / "runs" / f"run_{run_seq:04d}"
raw_vrps, unique_vrps = collect_rpki_prover_vrps(run_dir / "vrps.csv")
with gzip.open(run_dir / "normalized-vrps.txt.gz", "wt", encoding="utf-8") as handle:
for row in sorted(unique_vrps):
handle.write(row + "\n")
image = load_config().get("rpkiProverImage", {})
meta = {
"profile": profile,
"rp": "rpki-prover",
"runSeq": run_seq,
"syncMode": mode,
"startUtc": read_text_strip(run_dir / "start-utc.txt"),
"endUtc": read_text_strip(run_dir / "end-utc.txt"),
"exitCode": exit_code,
"timing": {
"wallMs": wall_ms,
"maxRssKb": max_rss_kb,
"maxProcessRssKb": max_process_rss_kb,
},
"counts": {
"vrps": raw_vrps,
"vrpsRaw": raw_vrps,
"vrpsUnique": len(unique_vrps),
"vaps": "n/a",
},
"vapsAvailable": False,
"validationAlgorithm": "full-every-iteration" if profile == "rpki-prover-full" else "incremental",
"image": image,
"remoteRunDir": str(run_dir),
}
write_json(run_dir / "run-meta.json", meta)
return meta
def write_rpki_prover_group_report(run_root: Path, profile: str) -> None:
rows: list[dict[str, Any]] = []
run_dirs = sorted((run_root / "profiles" / profile / "runs").glob("run_*"))
for run_dir in run_dirs:
meta = read_json(run_dir / "run-meta.json", {})
if not meta:
continue
rows.append({
"profile": profile,
"rp": "rpki-prover",
"run_seq": meta.get("runSeq", ""),
"sync_mode": meta.get("syncMode", ""),
"start_utc": meta.get("startUtc", ""),
"end_utc": meta.get("endUtc", ""),
"exit_code": meta.get("exitCode", ""),
"wall_ms": meta.get("timing", {}).get("wallMs", ""),
"max_rss_kb": meta.get("timing", {}).get("maxRssKb", ""),
"max_process_rss_kb": meta.get("timing", {}).get("maxProcessRssKb", ""),
"vrps": meta.get("counts", {}).get("vrpsRaw", ""),
"vrps_unique": meta.get("counts", {}).get("vrpsUnique", ""),
"vaps": "n/a",
"vaps_available": False,
"validation_algorithm": meta.get("validationAlgorithm", ""),
"remote_run_dir": str(run_dir),
})
reports = run_root / "reports"
reports.mkdir(parents=True, exist_ok=True)
if rows:
with (reports / "per_run_metrics.csv").open("w", encoding="utf-8", newline="") as handle:
writer = csv.DictWriter(handle, fieldnames=list(rows[0]))
writer.writeheader()
writer.writerows(rows)
write_json(reports / "per_run_metrics.json", {"generatedAtUtc": utc_now(), "rows": rows})
def normalize_one(run_root: Path, profile: str, run_seq: int) -> None: def normalize_one(run_root: Path, profile: str, run_seq: int) -> None:
script = ctl_root() / "scripts" / "normalize_and_collect.py" script = ctl_root() / "scripts" / "normalize_and_collect.py"
if not script.exists(): if not script.exists():
@@ -816,18 +460,6 @@ def parse_entry_rirs(entry: dict[str, Any]) -> list[str]:
return parse_rirs(str(value)) return parse_rirs(str(value))
def write_experiment_versions(run_root: Path) -> None:
versions = read_json(ctl_root() / "versions.json", {})
image = load_config().get("rpkiProverImage")
if image:
versions["rpki-prover"] = image
profiles = versions.setdefault("profiles", [])
for profile in ["rpki-prover-incremental", "rpki-prover-full"]:
if profile not in profiles:
profiles.append(profile)
write_json(run_root / "versions.json", versions)
def run_profile(profile: str, exp_id: str, runs: int, interval_secs: int, rirs: list[str], append: bool = False) -> int: def run_profile(profile: str, exp_id: str, runs: int, interval_secs: int, rirs: list[str], append: bool = False) -> int:
ensure_dirs() ensure_dirs()
rp = PROFILES[profile] rp = PROFILES[profile]
@@ -835,7 +467,7 @@ def run_profile(profile: str, exp_id: str, runs: int, interval_secs: int, rirs:
profile_root = run_root / "profiles" / profile profile_root = run_root / "profiles" / profile
(run_root / "logs").mkdir(parents=True, exist_ok=True) (run_root / "logs").mkdir(parents=True, exist_ok=True)
(run_root / "reports").mkdir(parents=True, exist_ok=True) (run_root / "reports").mkdir(parents=True, exist_ok=True)
write_experiment_versions(run_root) (run_root / "versions.json").write_text((ctl_root() / "versions.json").read_text(encoding="utf-8"), encoding="utf-8")
now = utc_now() now = utc_now()
if append: if append:
run_info = normalize_experiment_metadata(read_json(run_root / "run-info.json", {})) run_info = normalize_experiment_metadata(read_json(run_root / "run-info.json", {}))
@@ -872,9 +504,6 @@ def run_profile(profile: str, exp_id: str, runs: int, interval_secs: int, rirs:
"experimentRoot": str(run_root), "experimentRoot": str(run_root),
"stateRoot": str(profile_root / "state"), "stateRoot": str(profile_root / "state"),
} }
if rp == "rpki-prover":
run_info["validationAlgorithm"] = cache_profile_name(profile)
run_info["rpkiProverImage"] = load_config().get("rpkiProverImage", {})
write_json(run_root / "run-info.json", run_info) write_json(run_root / "run-info.json", run_info)
status = { status = {
"state": "running", "state": "running",
@@ -906,7 +535,7 @@ def run_profile(profile: str, exp_id: str, runs: int, interval_secs: int, rirs:
time.sleep(sleep_seconds) time.sleep(sleep_seconds)
last_start = time.time() last_start = time.time()
snapshot = run_seq == 1 snapshot = run_seq == 1
mode = "snapshot" if snapshot else ("warm" if rp == "rpki-prover" else "delta") mode = "snapshot" if snapshot else "delta"
run_dir = profile_root / "runs" / f"run_{run_seq:04d}" run_dir = profile_root / "runs" / f"run_{run_seq:04d}"
if run_dir.exists() and any(run_dir.iterdir()): if run_dir.exists() and any(run_dir.iterdir()):
raise RuntimeError(f"refusing to overwrite existing run directory: {run_dir}") raise RuntimeError(f"refusing to overwrite existing run directory: {run_dir}")
@@ -921,31 +550,10 @@ def run_profile(profile: str, exp_id: str, runs: int, interval_secs: int, rirs:
argv, env = build_rpki_client_args(run_dir, state, rirs) argv, env = build_rpki_client_args(run_dir, state, rirs)
elif rp == "fort": elif rp == "fort":
argv, env = build_fort_args(run_dir, state, rirs) argv, env = build_fort_args(run_dir, state, rirs)
elif rp == "rpki-prover":
prepare_rpki_prover_state(state, rirs)
container_suffix = hashlib.sha1(f"{run_root}:{run_seq}".encode("utf-8")).hexdigest()[:12]
container_name = f"rpctl-prover-{container_suffix}"
argv = build_rpki_prover_command(profile, run_dir, state, container_name)
else: else:
raise RuntimeError(rp) raise RuntimeError(rp)
run_dir.mkdir(parents=True, exist_ok=True) run_dir.mkdir(parents=True, exist_ok=True)
(run_dir / "command.txt").write_text(command_string(argv) + "\n", encoding="utf-8") (run_dir / "command.txt").write_text(command_string(argv) + "\n", encoding="utf-8")
if rp == "rpki-prover":
code, max_rss_kb, max_process_rss_kb, wall_ms, _ = timed_rpki_prover_run(
profile, run_dir, state, container_name
)
meta = write_rpki_prover_run_metadata(
run_root,
profile,
run_seq,
mode,
wall_ms,
max_rss_kb,
max_process_rss_kb,
code,
)
write_rpki_prover_group_report(run_root, profile)
else:
code = timed_run(run_dir, argv, env=env) code = timed_run(run_dir, argv, env=env)
normalize_one(run_root, profile, run_seq) normalize_one(run_root, profile, run_seq)
meta = parse_latest_meta(run_root, profile, run_seq) meta = parse_latest_meta(run_root, profile, run_seq)
@@ -963,9 +571,6 @@ def run_profile(profile: str, exp_id: str, runs: int, interval_secs: int, rirs:
if code != 0: if code != 0:
exit_code = code exit_code = code
break break
if rp == "rpki-prover":
write_rpki_prover_group_report(run_root, profile)
else:
write_group_report(run_root, interval_secs, completed_seq) write_group_report(run_root, interval_secs, completed_seq)
final_state = "success" if exit_code == 0 else "failed" final_state = "success" if exit_code == 0 else "failed"
update_current(state=final_state, finishedAtUtc=utc_now(), exitCode=exit_code) update_current(state=final_state, finishedAtUtc=utc_now(), exitCode=exit_code)
@@ -1040,10 +645,6 @@ def cmd_paths(args: argparse.Namespace) -> None:
print(f"runs={runs_dir()}") print(f"runs={runs_dir()}")
print(f"logs={logs_dir()}") print(f"logs={logs_dir()}")
print(f"bundle={cfg.get('bundle','')}") print(f"bundle={cfg.get('bundle','')}")
image = cfg.get("rpkiProverImage", {})
print(f"rpki_prover_image={image.get('tag', '')}")
print(f"rpki_prover_image_id={image.get('imageId', '')}")
print(f"rpki_prover_image_archive={image.get('archive', '')}")
cur = current_live_status() cur = current_live_status()
if cur.get("experimentRoot"): if cur.get("experimentRoot"):
print(f"current_experiment_root={cur['experimentRoot']}") print(f"current_experiment_root={cur['experimentRoot']}")
@@ -1067,8 +668,6 @@ def cmd_start(args: argparse.Namespace) -> None:
raise SystemExit("--runs must be >= 1") raise SystemExit("--runs must be >= 1")
if not (ctl_root() / "versions.json").exists(): if not (ctl_root() / "versions.json").exists():
raise SystemExit("rpctl is not initialized; run ./rpctl init --bundle <bundle-root> first") raise SystemExit("rpctl is not initialized; run ./rpctl init --bundle <bundle-root> first")
if PROFILES[profile] == "rpki-prover":
require_rpki_prover_image()
exp_id = args.exp_id or f"{utc_id()}_{profile}_{args.runs}runs_{interval_secs}s" exp_id = args.exp_id or f"{utc_id()}_{profile}_{args.runs}runs_{interval_secs}s"
runner_args = [ runner_args = [
sys.executable, sys.executable,
@@ -1118,8 +717,6 @@ def cmd_continue(args: argparse.Namespace) -> None:
if entry["state"] != "success": if entry["state"] != "success":
raise SystemExit(f"can only continue a successful experiment: exp_id={args.exp_id} state={entry['state']}") raise SystemExit(f"can only continue a successful experiment: exp_id={args.exp_id} state={entry['state']}")
profile = entry["profile"] profile = entry["profile"]
if PROFILES[profile] == "rpki-prover":
require_rpki_prover_image()
rirs = parse_entry_rirs(entry) rirs = parse_entry_rirs(entry)
interval_secs = parse_interval(args.interval) interval_secs = parse_interval(args.interval)
runner_args = [ runner_args = [
@@ -1270,9 +867,6 @@ def parse_command_summary(command: str) -> str:
keys.append(f"--rsync-scope={tokens[idx + 1]}") keys.append(f"--rsync-scope={tokens[idx + 1]}")
elif token.startswith("--rsync-scope="): elif token.startswith("--rsync-scope="):
keys.append(token) keys.append(token)
if RPKI_PROVER_IMAGE_TAG in token_set:
algorithm = "full-every-iteration" if "--no-incremental-validation" in token_set else "incremental"
keys.append(f"algorithm={algorithm}")
return " ".join(keys) return " ".join(keys)
@@ -1351,10 +945,6 @@ def cache_profile_name(profile: str) -> str:
return "pp+roa+crt" return "pp+roa+crt"
if profile == "ours-no-cache": if profile == "ours-no-cache":
return "off" return "off"
if profile == "rpki-prover-incremental":
return "incremental"
if profile == "rpki-prover-full":
return "full-every-iteration"
return "n/a" return "n/a"
@@ -1381,8 +971,6 @@ def profile_label(profile: str) -> str:
"routinator-latest-release": "routinator", "routinator-latest-release": "routinator",
"rpki-client-latest-release": "rpki-client", "rpki-client-latest-release": "rpki-client",
"fort-latest-release": "fort", "fort-latest-release": "fort",
"rpki-prover-incremental": "prover-inc",
"rpki-prover-full": "prover-full",
} }
return labels.get(profile, profile) return labels.get(profile, profile)
@@ -1455,9 +1043,6 @@ def infer_output_counts(profile: str, run_dir: Path) -> tuple[str, str]:
return vrps, vaps return vrps, vaps
if rp == "fort": if rp == "fort":
return count_csv_data_rows(run_dir / "vrps.csv"), "0" if (run_dir / "vrps.csv").exists() else "" return count_csv_data_rows(run_dir / "vrps.csv"), "0" if (run_dir / "vrps.csv").exists() else ""
if rp == "rpki-prover":
raw, _ = collect_rpki_prover_vrps(run_dir / "vrps.csv")
return str(raw) if (run_dir / "vrps.csv").exists() else "", "n/a" if (run_dir / "vrps.csv").exists() else ""
return "", "" return "", ""
@@ -1770,7 +1355,6 @@ def fallback_run_details(entry: dict[str, Any]) -> list[dict[str, Any]]:
"wall_ms": timing.get("wallMs", ""), "wall_ms": timing.get("wallMs", ""),
"max_rss_kb": timing.get("maxRssKb", ""), "max_rss_kb": timing.get("maxRssKb", ""),
"vrps": first_non_empty(meta.get("counts", {}).get("vrps") if meta else "", inferred_vrps), "vrps": first_non_empty(meta.get("counts", {}).get("vrps") if meta else "", inferred_vrps),
"vrps_unique": first_non_empty(meta.get("counts", {}).get("vrpsUnique") if meta else ""),
"vaps": first_non_empty(meta.get("counts", {}).get("vaps") if meta else "", inferred_vaps), "vaps": first_non_empty(meta.get("counts", {}).get("vaps") if meta else "", inferred_vaps),
"remote_run_dir": str(run_dir), "remote_run_dir": str(run_dir),
}) })
@@ -1807,7 +1391,7 @@ def cmd_view(args: argparse.Namespace) -> None:
print("command=" + entry["command"]) print("command=" + entry["command"])
if rows: if rows:
print("\nper-run:") print("\nper-run:")
headers = ["run", "mode", "exit", "start", "end", "wall_ms", "rss_kb", "vrps", "vrps_unique", "vaps", "path"] headers = ["run", "mode", "exit", "start", "end", "wall_ms", "rss_kb", "vrps", "vaps", "path"]
table_rows = [] table_rows = []
for row in rows: for row in rows:
table_rows.append([ table_rows.append([
@@ -1819,7 +1403,6 @@ def cmd_view(args: argparse.Namespace) -> None:
first_non_empty(row.get("wall_ms"), row.get("wallMs")), first_non_empty(row.get("wall_ms"), row.get("wallMs")),
first_non_empty(row.get("max_rss_kb"), row.get("maxRssKb")), first_non_empty(row.get("max_rss_kb"), row.get("maxRssKb")),
first_non_empty(row.get("vrps"), row.get("VRPs")), first_non_empty(row.get("vrps"), row.get("VRPs")),
first_non_empty(row.get("vrps_unique"), row.get("vrpsUnique")),
first_non_empty(row.get("vaps"), row.get("VAPs")), first_non_empty(row.get("vaps"), row.get("VAPs")),
first_non_empty(row.get("remote_run_dir"), row.get("remoteRunDir")), first_non_empty(row.get("remote_run_dir"), row.get("remoteRunDir")),
]) ])
@@ -1829,7 +1412,7 @@ def cmd_view(args: argparse.Namespace) -> None:
def build_parser() -> argparse.ArgumentParser: def build_parser() -> argparse.ArgumentParser:
formatter = argparse.RawDescriptionHelpFormatter formatter = argparse.RawDescriptionHelpFormatter
parser = argparse.ArgumentParser( parser = argparse.ArgumentParser(
description="Control one local five-RP performance run on this server.", description="Control one local four-RP performance run on this server.",
formatter_class=formatter, formatter_class=formatter,
epilog=""" epilog="""
RP/profile naming: RP/profile naming:
@@ -1852,14 +1435,6 @@ RP/profile naming:
fort fort
Alias of fort-latest-release. Uses archived FORT validator release binary. Alias of fort-latest-release. Uses archived FORT validator release binary.
rpki-prover / rpki-prover-incremental
Uses the archived rpki-prover Docker image with its default incremental
validation algorithm and persistent state.
rpki-prover-full / rpki-prover-no-incremental
Uses the same image with --no-incremental-validation. Its state is kept
separate from the incremental profile.
Cache difference for ours RP: Cache difference for ours RP:
--rp ours adds: --rp ours adds:
--enable-publication-point-validation-cache --enable-publication-point-validation-cache
@@ -1869,12 +1444,10 @@ Cache difference for ours RP:
--rp ours-no-cache does not add those flags. --rp ours-no-cache does not add those flags.
Typical usage: Typical usage:
./rpctl init --bundle /data/rpki_4rp_ctl_runtime_bundle \\ ./rpctl init --bundle /root/rpki_4rp_ctl_runtime_bundle
--rpki-prover-image /data/rpki-prover-6857d4bf-linux-amd64.docker.tar.gz
./rpctl list ./rpctl list
./rpctl start --rp ours --interval 10m --runs 6 --rirs all5 ./rpctl start --rp ours --interval 10m --runs 6 --rirs all5
./rpctl start --rp ours-no-cache --interval 10m --runs 6 --rirs all5 ./rpctl start --rp ours-no-cache --interval 10m --runs 6 --rirs all5
./rpctl start --rp rpki-prover --interval 0 --runs 2 --rirs all5
./rpctl status ./rpctl status
./rpctl show ./rpctl show
./rpctl show --failed ./rpctl show --failed
@@ -1885,8 +1458,7 @@ Typical usage:
Data model: Data model:
- Experiment data stays under ./runs/<profile>/<exp-id>/. - Experiment data stays under ./runs/<profile>/<exp-id>/.
- Each experiment starts with a snapshot; later child runs reuse the profile state. - Each experiment starts with a snapshot; later child runs are deltas and reuse the profile state.
- rpki-prover labels reused-state runs as warm because its native storage decides snapshot/delta behavior.
- By default only one RP/profile can run at a time. - By default only one RP/profile can run at a time.
""", """,
) )
@@ -1909,24 +1481,14 @@ The bundle must contain:
scripts/write_group_report.py scripts/write_group_report.py
versions.json versions.json
Optional rpki-prover image archive:
Pass --rpki-prover-image to copy and load the saved linux/amd64 Docker image.
Init verifies image architecture, tag and rpki-prover version without pulling online.
Recommended source on 47.77.237.41: Recommended source on 47.77.237.41:
/root/rpki_4rp_ctl_runtime_bundle /root/rpki_4rp_ctl_runtime_bundle
Example: Example:
./rpctl init --bundle /data/rpki_4rp_ctl_runtime_bundle \\ ./rpctl init --bundle /root/rpki_4rp_ctl_runtime_bundle
--rpki-prover-image /data/rpki-prover-6857d4bf-linux-amd64.docker.tar.gz
""", """,
) )
p.add_argument("--bundle", required=True, help="Path to the archived #100 runtime bundle root.") p.add_argument("--bundle", required=True, help="Path to the archived #100 runtime bundle root.")
p.add_argument(
"--rpki-prover-image",
default="",
help="Optional saved rpki-prover linux/amd64 Docker image archive. Existing four-RP profiles do not require it.",
)
p.set_defaults(func=cmd_init) p.set_defaults(func=cmd_init)
p = sub.add_parser( p = sub.add_parser(
@@ -1940,10 +1502,6 @@ Use this command when unsure which value to pass to --rp.
Important ours RP profiles: Important ours RP profiles:
ours-pp-object-cache cache-enabled profile ours-pp-object-cache cache-enabled profile
ours-no-cache no-cache baseline profile ours-no-cache no-cache baseline profile
rpki-prover profiles:
rpki-prover-incremental default incremental validation
rpki-prover-full --no-incremental-validation baseline
""", """,
) )
p.set_defaults(func=cmd_list) p.set_defaults(func=cmd_list)
@@ -1967,8 +1525,6 @@ RP/profile examples:
./rpctl start --rp routinator --interval 15m --runs 2 --rirs apnic ./rpctl start --rp routinator --interval 15m --runs 2 --rirs apnic
./rpctl start --rp rpki-client --interval 0 --runs 1 --rirs apnic,arin ./rpctl start --rp rpki-client --interval 0 --runs 1 --rirs apnic,arin
./rpctl start --rp fort --interval 0 --runs 1 --rirs all5 ./rpctl start --rp fort --interval 0 --runs 1 --rirs all5
./rpctl start --rp rpki-prover --interval 0 --runs 2 --rirs all5
./rpctl start --rp rpki-prover-full --interval 0 --runs 1 --rirs all5
Cache mapping: Cache mapping:
--rp ours enables PP cache + ROA cache + child certificate cache. --rp ours enables PP cache + ROA cache + child certificate cache.
@@ -1981,7 +1537,7 @@ Safety:
p.add_argument( p.add_argument(
"--rp", "--rp",
required=True, required=True,
help="RP/profile or alias. Common values: ours, ours-no-cache, routinator, rpki-client, fort, rpki-prover, rpki-prover-full.", help="RP/profile or alias. Common values: ours, ours-no-cache, routinator, rpki-client, fort.",
) )
p.add_argument( p.add_argument(
"--interval", "--interval",
@@ -2022,7 +1578,6 @@ Run semantics:
Examples: Examples:
./rpctl continue --exp-id 20260709T081122Z_ours-pp-object-cache_2runs_600s --runs 3 --interval 10m ./rpctl continue --exp-id 20260709T081122Z_ours-pp-object-cache_2runs_600s --runs 3 --interval 10m
./rpctl continue --rp routinator --exp-id m18_237_routinator_all5_snapshot_delta_20260710T002726Z --runs 1 --interval 0 ./rpctl continue --rp routinator --exp-id m18_237_routinator_all5_snapshot_delta_20260710T002726Z --runs 1 --interval 0
./rpctl continue --rp rpki-prover --exp-id prover_all5 --runs 1 --interval 0
""", """,
) )
p.add_argument("--exp-id", required=True, help="Existing successful experiment id to append to.") p.add_argument("--exp-id", required=True, help="Existing successful experiment id to append to.")
@@ -2137,7 +1692,7 @@ Examples:
./rpctl show --json ./rpctl show --json
""", """,
) )
p.add_argument("--rp", default="", help="Optional RP/profile filter, e.g. ours, ours-no-cache, routinator, rpki-client, fort, rpki-prover.") p.add_argument("--rp", default="", help="Optional RP/profile filter, e.g. ours, ours-no-cache, routinator, rpki-client, fort.")
p.add_argument("--failed", action="store_true", help="Only show failed, partial, incomplete or exited experiments.") p.add_argument("--failed", action="store_true", help="Only show failed, partial, incomplete or exited experiments.")
p.add_argument("--limit", type=int, default=50, help="Maximum rows to show. 0 means no limit. Default: 50.") p.add_argument("--limit", type=int, default=50, help="Maximum rows to show. 0 means no limit. Default: 50.")
p.add_argument("--paths", action="store_true", help="Also show full experiment root paths. Hidden by default to keep the table readable.") p.add_argument("--paths", action="store_true", help="Also show full experiment root paths. Hidden by default to keep the table readable.")

View File

@@ -1,113 +0,0 @@
from __future__ import annotations
import importlib.machinery
import importlib.util
import json
import tempfile
import unittest
from pathlib import Path
def load_rpctl():
path = Path(__file__).with_name("rpctl")
loader = importlib.machinery.SourceFileLoader("rpctl_module", str(path))
spec = importlib.util.spec_from_loader(loader.name, loader)
module = importlib.util.module_from_spec(spec)
loader.exec_module(module)
return module
class RpkiProverProfileTests(unittest.TestCase):
@classmethod
def setUpClass(cls) -> None:
cls.rpctl = load_rpctl()
def test_profile_aliases_and_algorithms(self) -> None:
self.assertEqual(self.rpctl.normalize_profile("rpki-prover"), "rpki-prover-incremental")
self.assertEqual(self.rpctl.normalize_profile("rpki-prover-no-incremental"), "rpki-prover-full")
self.assertNotIn("--no-incremental-validation", self.rpctl.rpki_prover_arguments("rpki-prover-incremental"))
self.assertIn("--no-incremental-validation", self.rpctl.rpki_prover_arguments("rpki-prover-full"))
incremental = self.rpctl.parse_command_summary(
f"docker run {self.rpctl.RPKI_PROVER_IMAGE_TAG} --once"
)
full = self.rpctl.parse_command_summary(
f"docker run {self.rpctl.RPKI_PROVER_IMAGE_TAG} --once --no-incremental-validation"
)
self.assertEqual(incremental, "algorithm=incremental")
self.assertEqual(full, "algorithm=full-every-iteration")
def test_vrp_count_preserves_raw_and_unique(self) -> None:
with tempfile.TemporaryDirectory() as temp:
csv_path = Path(temp) / "vrps.csv"
csv_path.write_text(
"ASN,IP Prefix,Max Length,Trust Anchor\n"
"AS64496,192.0.2.0/24,24,TA-A\n"
"AS64496,192.0.2.0/24,24,TA-B\n"
"AS64497,2001:db8::/32,48,TA-A\n",
encoding="utf-8",
)
raw, unique = self.rpctl.collect_rpki_prover_vrps(csv_path)
self.assertEqual(raw, 3)
self.assertEqual(len(unique), 2)
def test_run_metadata_marks_vaps_not_applicable(self) -> None:
with tempfile.TemporaryDirectory() as temp:
root = Path(temp)
old_ctl_root = self.rpctl.ctl_root
self.rpctl.ctl_root = lambda: root
try:
self.rpctl.save_config({"rpkiProverImage": {"tag": self.rpctl.RPKI_PROVER_IMAGE_TAG}})
run_root = root / "runs" / "rpki-prover-incremental" / "test"
run_dir = run_root / "profiles" / "rpki-prover-incremental" / "runs" / "run_0001"
run_dir.mkdir(parents=True)
(run_dir / "start-utc.txt").write_text("2026-07-17T00:00:00Z\n", encoding="utf-8")
(run_dir / "end-utc.txt").write_text("2026-07-17T00:01:00Z\n", encoding="utf-8")
(run_dir / "vrps.csv").write_text(
"ASN,IP Prefix,Max Length,Trust Anchor\nAS64496,192.0.2.0/24,24,TA-A\n",
encoding="utf-8",
)
meta = self.rpctl.write_rpki_prover_run_metadata(
run_root, "rpki-prover-incremental", 1, "snapshot", 60_000, 1234, 1200, 0
)
self.assertEqual(meta["counts"]["vrpsRaw"], 1)
self.assertEqual(meta["counts"]["vrpsUnique"], 1)
self.assertEqual(meta["counts"]["vaps"], "n/a")
self.assertFalse(meta["vapsAvailable"])
self.rpctl.write_rpki_prover_group_report(run_root, "rpki-prover-incremental")
report = (run_root / "reports" / "per_run_metrics.csv").read_text(encoding="utf-8")
self.assertIn("vrps_unique", report)
self.assertIn("n/a", report)
finally:
self.rpctl.ctl_root = old_ctl_root
def test_missing_image_configuration_fails_before_docker(self) -> None:
with tempfile.TemporaryDirectory() as temp:
root = Path(temp)
old_ctl_root = self.rpctl.ctl_root
self.rpctl.ctl_root = lambda: root
try:
self.rpctl.save_config({})
with self.assertRaisesRegex(SystemExit, "not initialized"):
self.rpctl.require_rpki_prover_image()
finally:
self.rpctl.ctl_root = old_ctl_root
def test_versions_include_prover_profiles(self) -> None:
with tempfile.TemporaryDirectory() as temp:
root = Path(temp)
old_ctl_root = self.rpctl.ctl_root
self.rpctl.ctl_root = lambda: root
try:
(root / "versions.json").write_text(json.dumps({"profiles": ["fort-latest-release"]}), encoding="utf-8")
self.rpctl.save_config({"rpkiProverImage": {"tag": self.rpctl.RPKI_PROVER_IMAGE_TAG}})
run_root = root / "experiment"
self.rpctl.write_experiment_versions(run_root)
versions = json.loads((run_root / "versions.json").read_text(encoding="utf-8"))
self.assertIn("rpki-prover-incremental", versions["profiles"])
self.assertIn("rpki-prover-full", versions["profiles"])
finally:
self.rpctl.ctl_root = old_ctl_root
if __name__ == "__main__":
unittest.main()

View File

@@ -14,7 +14,7 @@ Usage:
scripts/soak/build_portable_soak_package.sh [--out-dir <path>] [--profile <profile>] scripts/soak/build_portable_soak_package.sh [--out-dir <path>] [--profile <profile>]
Requires release binaries to already exist. Build them first, for example: Requires release binaries to already exist. Build them first, for example:
cargo build --release --bin rpki --bin rpki_daemon --bin db_stats --bin rpki_artifact_metrics --bin rpki_query_service --bin rpki_query_indexer --bin verification_ccr_compare cargo build --release --bin rpki --bin rpki_daemon --bin db_stats --bin rpki_artifact_metrics --bin rpki_query_service --bin rpki_query_indexer
USAGE USAGE
} }
@@ -53,7 +53,7 @@ else
TARGET_BIN_DIR="$REPO_ROOT/target/$PROFILE" TARGET_BIN_DIR="$REPO_ROOT/target/$PROFILE"
fi fi
REQUIRED_BINS=(rpki rpki_daemon db_stats rpki_artifact_metrics rpki_query_service rpki_query_indexer verification_ccr_compare) REQUIRED_BINS=(rpki rpki_daemon db_stats rpki_artifact_metrics rpki_query_service rpki_query_indexer)
OPTIONAL_BINS=( OPTIONAL_BINS=(
ccr_dump ccr_dump
ccr_state_compare ccr_state_compare
@@ -82,7 +82,7 @@ STAGE_DIR="$BUILD_ROOT/$PACKAGE_DIR_NAME"
ARCHIVE_PATH="$OUT_DIR/$PACKAGE_NAME.tar.gz" ARCHIVE_PATH="$OUT_DIR/$PACKAGE_NAME.tar.gz"
rm -rf "$STAGE_DIR" "$ARCHIVE_PATH" rm -rf "$STAGE_DIR" "$ARCHIVE_PATH"
mkdir -p "$STAGE_DIR/bin" "$STAGE_DIR/fixtures" "$STAGE_DIR/scripts" "$STAGE_DIR/scripts/soak" "$STAGE_DIR/scripts/verification" \ mkdir -p "$STAGE_DIR/bin" "$STAGE_DIR/fixtures" "$STAGE_DIR/scripts" "$STAGE_DIR/scripts/soak" \
"$STAGE_DIR/runs" "$STAGE_DIR/state" "$STAGE_DIR/logs" "$STAGE_DIR/tmp" "$STAGE_DIR/runs" "$STAGE_DIR/state" "$STAGE_DIR/logs" "$STAGE_DIR/tmp"
install -m 0755 "$SCRIPT_DIR/run_soak.sh" "$STAGE_DIR/run_soak.sh" install -m 0755 "$SCRIPT_DIR/run_soak.sh" "$STAGE_DIR/run_soak.sh"
@@ -92,8 +92,6 @@ install -m 0755 "$SCRIPT_DIR/fixed_phase_loop.sh" "$STAGE_DIR/scripts/soak/fixed
install -m 0755 "$SCRIPT_DIR/hourly_soak_report.py" "$STAGE_DIR/scripts/soak/hourly_soak_report.py" install -m 0755 "$SCRIPT_DIR/hourly_soak_report.py" "$STAGE_DIR/scripts/soak/hourly_soak_report.py"
install -m 0755 "$SCRIPT_DIR/publish_remote231.sh" "$STAGE_DIR/scripts/soak/publish_remote231.sh" install -m 0755 "$SCRIPT_DIR/publish_remote231.sh" "$STAGE_DIR/scripts/soak/publish_remote231.sh"
install -m 0755 "$SCRIPT_DIR/publish_remote231_full.sh" "$STAGE_DIR/scripts/soak/publish_remote231_full.sh" install -m 0755 "$SCRIPT_DIR/publish_remote231_full.sh" "$STAGE_DIR/scripts/soak/publish_remote231_full.sh"
install -m 0755 "$REPO_ROOT/scripts/verification/run_verification_only.sh" "$STAGE_DIR/scripts/verification/run_verification_only.sh"
install -m 0644 "$REPO_ROOT/scripts/verification/README.md" "$STAGE_DIR/scripts/verification/README.md"
install -m 0644 "$SCRIPT_DIR/portable-soak.env.example" "$STAGE_DIR/.env" install -m 0644 "$SCRIPT_DIR/portable-soak.env.example" "$STAGE_DIR/.env"
install -m 0644 "$SCRIPT_DIR/portable-soak.env.example" "$STAGE_DIR/portable-soak.env.example" install -m 0644 "$SCRIPT_DIR/portable-soak.env.example" "$STAGE_DIR/portable-soak.env.example"

View File

@@ -15,6 +15,10 @@ INTERVAL_SECS="${INTERVAL_SECS:-0}"
STOP_AFTER_SECS="${STOP_AFTER_SECS:-0}" STOP_AFTER_SECS="${STOP_AFTER_SECS:-0}"
RIRS="${RIRS:-afrinic,apnic,arin,lacnic,ripe}" RIRS="${RIRS:-afrinic,apnic,arin,lacnic,ripe}"
TAL_INPUT_MODE="${TAL_INPUT_MODE:-file-with-ta}" TAL_INPUT_MODE="${TAL_INPUT_MODE:-file-with-ta}"
CUSTOM_TAL_PATH="${CUSTOM_TAL_PATH:-$PACKAGE_ROOT/custom-fixtures/tal/custom.tal}"
CUSTOM_TA_PATH="${CUSTOM_TA_PATH:-$PACKAGE_ROOT/custom-fixtures/ta/custom-ta.cer}"
CUSTOM_TAL_URI="${CUSTOM_TAL_URI:-https://host.docker.internal:18443/tal/custom.tal}"
HTTP_ROOT_CERT_PATHS="${HTTP_ROOT_CERT_PATHS:-}"
RESOURCE_VALIDATION_MODE="${RESOURCE_VALIDATION_MODE:-validation-update-03}" RESOURCE_VALIDATION_MODE="${RESOURCE_VALIDATION_MODE:-validation-update-03}"
RUN_ROOT="${RUN_ROOT:-$PACKAGE_ROOT}" RUN_ROOT="${RUN_ROOT:-$PACKAGE_ROOT}"
RETAIN_RUNS="${RETAIN_RUNS:-10}" RETAIN_RUNS="${RETAIN_RUNS:-10}"
@@ -48,7 +52,6 @@ TMP_DIR="${TMP_DIR:-$RUN_ROOT/tmp}"
RSYNC_MIRROR_ROOT="${RSYNC_MIRROR_ROOT:-$STATE_ROOT/rsync-mirror}" RSYNC_MIRROR_ROOT="${RSYNC_MIRROR_ROOT:-$STATE_ROOT/rsync-mirror}"
INVALID_ROOT="$STATE_ROOT/invalid" INVALID_ROOT="$STATE_ROOT/invalid"
RESET_STAGING_ROOT="$STATE_ROOT/reset-staging" RESET_STAGING_ROOT="$STATE_ROOT/reset-staging"
STATE_EXECUTION_LOCK_PATH="$STATE_ROOT/locks/state-execution.lock"
LIVE_TA_REFRESH_DIR="${LIVE_TA_REFRESH_DIR:-$META_DIR/live-ta-refresh}" LIVE_TA_REFRESH_DIR="${LIVE_TA_REFRESH_DIR:-$META_DIR/live-ta-refresh}"
LIVE_TA_REFRESH_CONNECT_TIMEOUT_SECS="${LIVE_TA_REFRESH_CONNECT_TIMEOUT_SECS:-15}" LIVE_TA_REFRESH_CONNECT_TIMEOUT_SECS="${LIVE_TA_REFRESH_CONNECT_TIMEOUT_SECS:-15}"
LIVE_TA_REFRESH_MAX_TIME_SECS="${LIVE_TA_REFRESH_MAX_TIME_SECS:-120}" LIVE_TA_REFRESH_MAX_TIME_SECS="${LIVE_TA_REFRESH_MAX_TIME_SECS:-120}"
@@ -119,10 +122,10 @@ validate_rsync_scope() {
validate_tal_input_mode() { validate_tal_input_mode() {
case "$TAL_INPUT_MODE" in case "$TAL_INPUT_MODE" in
file-with-ta|file-live-ta|url) file-with-ta|file-live-ta|custom-file-with-ta|url)
;; ;;
*) *)
die "TAL_INPUT_MODE must be file-with-ta, file-live-ta or url: $TAL_INPUT_MODE" die "TAL_INPUT_MODE must be file-with-ta, file-live-ta, custom-file-with-ta or url: $TAL_INPUT_MODE"
;; ;;
esac esac
} }
@@ -156,14 +159,46 @@ parse_rirs() {
afrinic|apnic|arin|lacnic|ripe) afrinic|apnic|arin|lacnic|ripe)
RIR_LIST+=("$normalized") RIR_LIST+=("$normalized")
;; ;;
custom)
[[ "$TAL_INPUT_MODE" == "custom-file-with-ta" ]] \
|| die "RIRS=custom requires TAL_INPUT_MODE=custom-file-with-ta"
RIR_LIST+=("$normalized")
;;
*) *)
die "invalid RIRS entry: $raw_token; allowed: afrinic,apnic,arin,lacnic,ripe" die "invalid RIRS entry: $raw_token; allowed: afrinic,apnic,arin,lacnic,ripe,custom (custom mode only)"
;; ;;
esac esac
done done
[[ "${#RIR_LIST[@]}" -gt 0 ]] || die "RIRS must contain at least one RIR" [[ "${#RIR_LIST[@]}" -gt 0 ]] || die "RIRS must contain at least one RIR"
} }
validate_custom_inputs() {
local root_cert_path
local -a root_cert_paths=()
if [[ "$TAL_INPUT_MODE" == "custom-file-with-ta" ]]; then
[[ "${#RIR_LIST[@]}" -eq 1 && "${RIR_LIST[0]}" == "custom" ]] \
|| die "custom-file-with-ta requires exactly RIRS=custom"
[[ -s "$CUSTOM_TAL_PATH" ]] \
|| die "missing custom TAL: $CUSTOM_TAL_PATH"
[[ -s "$CUSTOM_TA_PATH" ]] \
|| die "missing custom TA DER: $CUSTOM_TA_PATH"
[[ -n "$CUSTOM_TAL_URI" ]] \
|| die "CUSTOM_TAL_URI is required in custom-file-with-ta mode"
elif [[ "$RIRS" == *custom* ]]; then
die "RIRS=custom requires TAL_INPUT_MODE=custom-file-with-ta"
fi
if [[ -n "$HTTP_ROOT_CERT_PATHS" ]]; then
# shellcheck disable=SC2206
root_cert_paths=( $HTTP_ROOT_CERT_PATHS )
for root_cert_path in "${root_cert_paths[@]}"; do
[[ -s "$root_cert_path" ]] \
|| die "missing HTTP root certificate: $root_cert_path"
done
fi
}
tal_file_for_rir() { tal_file_for_rir() {
case "$1" in case "$1" in
afrinic) printf '%s' "$FIXTURE_DIR/tal/afrinic.tal" ;; afrinic) printf '%s' "$FIXTURE_DIR/tal/afrinic.tal" ;;
@@ -171,6 +206,7 @@ tal_file_for_rir() {
arin) printf '%s' "$FIXTURE_DIR/tal/arin.tal" ;; arin) printf '%s' "$FIXTURE_DIR/tal/arin.tal" ;;
lacnic) printf '%s' "$FIXTURE_DIR/tal/lacnic.tal" ;; lacnic) printf '%s' "$FIXTURE_DIR/tal/lacnic.tal" ;;
ripe) printf '%s' "$FIXTURE_DIR/tal/ripe-ncc.tal" ;; ripe) printf '%s' "$FIXTURE_DIR/tal/ripe-ncc.tal" ;;
custom) printf '%s' "$CUSTOM_TAL_PATH" ;;
*) die "unknown RIR: $1" ;; *) die "unknown RIR: $1" ;;
esac esac
} }
@@ -182,6 +218,7 @@ ta_file_for_rir() {
arin) printf '%s' "$FIXTURE_DIR/ta/arin-ta.cer" ;; arin) printf '%s' "$FIXTURE_DIR/ta/arin-ta.cer" ;;
lacnic) printf '%s' "$FIXTURE_DIR/ta/lacnic-ta.cer" ;; lacnic) printf '%s' "$FIXTURE_DIR/ta/lacnic-ta.cer" ;;
ripe) printf '%s' "$FIXTURE_DIR/ta/ripe-ncc-ta.cer" ;; ripe) printf '%s' "$FIXTURE_DIR/ta/ripe-ncc-ta.cer" ;;
custom) printf '%s' "$CUSTOM_TA_PATH" ;;
*) die "unknown RIR: $1" ;; *) die "unknown RIR: $1" ;;
esac esac
} }
@@ -193,6 +230,7 @@ tal_url_for_rir() {
arin) printf '%s' "https://www.arin.net/resources/manage/rpki/arin.tal" ;; arin) printf '%s' "https://www.arin.net/resources/manage/rpki/arin.tal" ;;
lacnic) printf '%s' "https://www.lacnic.net/innovaportal/file/4983/1/lacnic.tal" ;; lacnic) printf '%s' "https://www.lacnic.net/innovaportal/file/4983/1/lacnic.tal" ;;
ripe) printf '%s' "https://tal.rpki.ripe.net/ripe-ncc.tal" ;; ripe) printf '%s' "https://tal.rpki.ripe.net/ripe-ncc.tal" ;;
custom) printf '%s' "$CUSTOM_TAL_URI" ;;
*) die "unknown RIR: $1" ;; *) die "unknown RIR: $1" ;;
esac esac
} }
@@ -1058,6 +1096,16 @@ build_child_args() {
CHILD_ARGS+=(--rsync-mirror-root "$TMP_DIR/rsync-mirror-{run_id}") CHILD_ARGS+=(--rsync-mirror-root "$TMP_DIR/rsync-mirror-{run_id}")
fi fi
local root_cert_path
local -a root_cert_paths=()
if [[ -n "$HTTP_ROOT_CERT_PATHS" ]]; then
# shellcheck disable=SC2206
root_cert_paths=( $HTTP_ROOT_CERT_PATHS )
for root_cert_path in "${root_cert_paths[@]}"; do
CHILD_ARGS+=(--http-root-cert "$root_cert_path")
done
fi
CHILD_ARGS+=( CHILD_ARGS+=(
--parallel-phase2-ready-batch-size 256 --parallel-phase2-ready-batch-size 256
--parallel-phase2-ready-batch-wall-time-budget-ms 100 --parallel-phase2-ready-batch-wall-time-budget-ms 100
@@ -1081,7 +1129,6 @@ build_child_args() {
CHILD_ARGS+=( CHILD_ARGS+=(
--report-json "{run_out}/report.json" --report-json "{run_out}/report.json"
--validation-contract-out "{run_out}/validation-contract.json"
) )
if is_true "$OUTPUT_COMPACT_REPORT"; then if is_true "$OUTPUT_COMPACT_REPORT"; then
CHILD_ARGS+=(--report-json-compact) CHILD_ARGS+=(--report-json-compact)
@@ -1111,59 +1158,6 @@ build_child_args() {
fi fi
} }
write_current_state_binding() {
local run_dir="$1"
local run_id="$2"
local output_path="$META_DIR/current-state-binding.json"
python3 - "$run_dir" "$run_id" "$output_path" <<'PY'
import hashlib
import json
import os
import sys
import tempfile
from datetime import datetime, timezone
from pathlib import Path
run_dir = Path(sys.argv[1])
run_id = sys.argv[2]
output_path = Path(sys.argv[3])
summary = json.loads((run_dir / "run-summary.json").read_text(encoding="utf-8"))
if summary.get("status") != "success":
raise SystemExit(f"cannot bind unsuccessful source run: {run_id}")
def digest(name):
path = run_dir / name
if not path.is_file():
raise SystemExit(f"current state binding input is missing: {path}")
hasher = hashlib.sha256()
with path.open("rb") as handle:
for chunk in iter(lambda: handle.read(1024 * 1024), b""):
hasher.update(chunk)
return hasher.hexdigest()
binding = {
"schemaVersion": 1,
"currentRunId": run_id,
"sourceCirSha256": digest("input.cir"),
"sourceCcrSha256": digest("result.ccr"),
"validationContractSha256": digest("validation-contract.json"),
"updatedAtRfc3339Utc": datetime.now(timezone.utc).isoformat().replace("+00:00", "Z"),
}
output_path.parent.mkdir(parents=True, exist_ok=True)
fd, tmp_name = tempfile.mkstemp(prefix=output_path.name + ".", dir=output_path.parent)
try:
with os.fdopen(fd, "w", encoding="utf-8") as handle:
json.dump(binding, handle, indent=2, sort_keys=True)
handle.write("\n")
handle.flush()
os.fsync(handle.fileno())
os.replace(tmp_name, output_path)
finally:
if os.path.exists(tmp_name):
os.unlink(tmp_name)
PY
}
copy_inner_run_outputs() { copy_inner_run_outputs() {
local daemon_state_root="$1" local daemon_state_root="$1"
local run_dir="$2" local run_dir="$2"
@@ -1359,12 +1353,6 @@ run_one_round() {
final_status="failed" final_status="failed"
fi fi
fi fi
if [[ "$final_status" == "success" ]]; then
if ! write_current_state_binding "$run_dir" "$run_id"; then
warn "failed to bind successful run to current state: $run_id"
final_status="failed"
fi
fi
write_run_meta "$run_dir/run-meta.json" "$final_status" "$run_index" "$run_id" "$sync_mode" \ write_run_meta "$run_dir/run-meta.json" "$final_status" "$run_index" "$run_id" "$sync_mode" \
"$snapshot_reason" "$previous_run_id" "$previous_success_value" "$started_at" "$completed_at" \ "$snapshot_reason" "$previous_run_id" "$previous_success_value" "$started_at" "$completed_at" \
"$INVALID_DB_PATH" "$INVALID_STATE_PATH" "$INVALID_TMP_PATH" "$daemon_exit_code" "$PACKAGE_ROOT" "$ENV_FILE" \ "$INVALID_DB_PATH" "$INVALID_STATE_PATH" "$INVALID_TMP_PATH" "$daemon_exit_code" "$PACKAGE_ROOT" "$ENV_FILE" \
@@ -1388,7 +1376,6 @@ main() {
require_command python3 require_command python3
require_command date require_command date
require_command find require_command find
require_command flock
if [[ "$TAL_INPUT_MODE" == "file-live-ta" ]]; then if [[ "$TAL_INPUT_MODE" == "file-live-ta" ]]; then
require_command curl require_command curl
validate_positive_int "LIVE_TA_REFRESH_CONNECT_TIMEOUT_SECS" "$LIVE_TA_REFRESH_CONNECT_TIMEOUT_SECS" validate_positive_int "LIVE_TA_REFRESH_CONNECT_TIMEOUT_SECS" "$LIVE_TA_REFRESH_CONNECT_TIMEOUT_SECS"
@@ -1406,6 +1393,7 @@ main() {
validate_positive_int "DB_STATS_EXACT_EVERY" "$DB_STATS_EXACT_EVERY" validate_positive_int "DB_STATS_EXACT_EVERY" "$DB_STATS_EXACT_EVERY"
fi fi
parse_rirs parse_rirs
validate_custom_inputs
[[ -x "$RPKI_BIN" ]] || die "missing executable: $RPKI_BIN" [[ -x "$RPKI_BIN" ]] || die "missing executable: $RPKI_BIN"
[[ -x "$RPKI_DAEMON_BIN" ]] || die "missing executable: $RPKI_DAEMON_BIN" [[ -x "$RPKI_DAEMON_BIN" ]] || die "missing executable: $RPKI_DAEMON_BIN"
@@ -1422,7 +1410,7 @@ main() {
fi fi
done done
mkdir -p "$RUNS_ROOT" "$LOG_ROOT" "$DB_DIR" "$META_DIR" "$TMP_DIR" "$INVALID_ROOT" "$RESET_STAGING_ROOT" "$LIVE_TA_REFRESH_DIR" "$(dirname "$STATE_EXECUTION_LOCK_PATH")" mkdir -p "$RUNS_ROOT" "$LOG_ROOT" "$DB_DIR" "$META_DIR" "$TMP_DIR" "$INVALID_ROOT" "$RESET_STAGING_ROOT" "$LIVE_TA_REFRESH_DIR"
if is_true "$ALLOW_RSYNC_MIRROR_REUSE"; then if is_true "$ALLOW_RSYNC_MIRROR_REUSE"; then
mkdir -p "$RSYNC_MIRROR_ROOT" mkdir -p "$RSYNC_MIRROR_ROOT"
fi fi
@@ -1448,9 +1436,6 @@ main() {
local any_failed=0 local any_failed=0
while (( run_forever == 1 || next_index <= stop_index )); do while (( run_forever == 1 || next_index <= stop_index )); do
local state_lock_fd
exec {state_lock_fd}>"$STATE_EXECUTION_LOCK_PATH"
flock "$state_lock_fd"
INVALID_DB_PATH="" INVALID_DB_PATH=""
INVALID_STATE_PATH="" INVALID_STATE_PATH=""
INVALID_TMP_PATH="" INVALID_TMP_PATH=""
@@ -1537,8 +1522,6 @@ main() {
echo "completed run $(printf 'run_%04d' "$next_index") status=failed" >&2 echo "completed run $(printf 'run_%04d' "$next_index") status=failed" >&2
any_failed=1 any_failed=1
fi fi
flock -u "$state_lock_fd"
exec {state_lock_fd}>&-
if (( STOP_AFTER_SECS > 0 )); then if (( STOP_AFTER_SECS > 0 )); then
elapsed_secs=$(( $(date +%s) - started_epoch )) elapsed_secs=$(( $(date +%s) - started_epoch ))
if (( elapsed_secs >= STOP_AFTER_SECS )); then if (( elapsed_secs >= STOP_AFTER_SECS )); then

View File

@@ -1,50 +0,0 @@
# verification-only 使用说明
`verification-only` 用于复核启用验证缓存的最新成功 normal run。它冻结当前仓库视图,关闭 PP、ROA、child certificate cache、prefetch、VCIR 持久化和网络访问,在源 CIR 的同一验证时间重新执行完整验证,并比较 CCR 的 MFT、VRP、VAP、TA、router key 五类 state digest。
## 前提
- 源 run 必须是当前 state DB 对应的最新成功 normal run。
- 源 run 必须包含 `input.cir`、`result.ccr`、`run-summary.json` 和 `validation-contract.json`。
- state root 必须包含 `db/work-db`、`db/repo-bytes.db` 和 `meta/current-state-binding.json`。
- 源 normal run 必须启用至少一种验证缓存;仅启用 transport prefetch 不算验证缓存。
- verification-only 与 normal soak 共用 `state/locks/state-execution.lock`,不能绕过 wrapper 并发执行。
## 执行
在 portable package 根目录执行:
```bash
./scripts/verification/run_verification_only.sh \
--source-run-dir /data/ours-rp/runs/run_0123 \
--source-state-root /data/ours-rp/state \
--out-dir /data/ours-rp/verification-runs/run_0123
```
验证时间自动从源 `input.cir` 读取,不接受命令行覆盖。成功后 scratch work-db 默认删除;排障时可加 `--keep-scratch`。
## 输出
- `input.cir`、`result.ccr`、`report.json`
- `vrps.csv`、`vaps.csv`
- `stage-timing.json`、`validation-events.jsonl`
- `validation-contract.json`、`verification-meta.json`
- `compare/ccr-state-compare.json`、`compare/ccr-state-compare.md`
- `stdout.log`、`stderr.log`、`process-time.txt`、`wrapper-timing.json`
通过条件是 compare JSON 中 `stateDigestMatch=true`。`producedAt` 不参与比较。
## 隔离保证
- 不分配 normal run id,不更新 lifecycle、periodic snapshot 计数和 retention。
- 源 work-db 只用于创建 checkpoint,后续写入只发生在独立 scratch work-db。
- 源 repo-bytes.db 以只读模式打开。
- 输出目录必须不存在,防止覆盖旧验证证据。
## 常见失败
- `current state binding`:所选 run 已不是当前 state 对应的最新成功 run,需选择 binding 指向的 run。
- `differs from current state binding`:源 CIR、CCR 或 contract 已变化,不能继续复核。
- `blob is missing`:repository view 引用的原始 bytes 不完整,验证不会启动。
- `did not enable a validation cache`:源 run 不是缓存正确性对比的有效基准。
- `CCR state digest mismatch`:查看 `compare/` 下 JSON/Markdown,按不匹配状态集合继续排查。

View File

@@ -1,102 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
usage() {
cat <<'USAGE'
Usage:
run_verification_only.sh \
--source-run-dir <run_dir> \
--source-state-root <state_dir> \
--out-dir <verification_dir> \
[--rpki-bin <path>] [--keep-scratch]
validation time is always read from the source CIR. The wrapper serializes access
with normal soak runs and never writes into the normal run sequence.
USAGE
}
SOURCE_RUN_DIR=""
SOURCE_STATE_ROOT=""
OUT_DIR=""
RPKI_BIN=""
KEEP_SCRATCH=0
while [[ $# -gt 0 ]]; do
case "$1" in
--source-run-dir) SOURCE_RUN_DIR="${2:?missing value}"; shift 2 ;;
--source-state-root) SOURCE_STATE_ROOT="${2:?missing value}"; shift 2 ;;
--out-dir) OUT_DIR="${2:?missing value}"; shift 2 ;;
--rpki-bin) RPKI_BIN="${2:?missing value}"; shift 2 ;;
--keep-scratch) KEEP_SCRATCH=1; shift ;;
-h|--help) usage; exit 0 ;;
*) echo "unknown argument: $1" >&2; usage >&2; exit 2 ;;
esac
done
[[ -n "$SOURCE_RUN_DIR" ]] || { echo "--source-run-dir is required" >&2; exit 2; }
[[ -n "$SOURCE_STATE_ROOT" ]] || { echo "--source-state-root is required" >&2; exit 2; }
[[ -n "$OUT_DIR" ]] || { echo "--out-dir is required" >&2; exit 2; }
if [[ -z "$RPKI_BIN" ]]; then
RPKI_BIN="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)/target/release/rpki"
fi
[[ -x "$RPKI_BIN" ]] || { echo "rpki binary is not executable: $RPKI_BIN" >&2; exit 2; }
command -v flock >/dev/null || { echo "flock is required" >&2; exit 2; }
command -v python3 >/dev/null || { echo "python3 is required" >&2; exit 2; }
now_ms() {
python3 -c 'import time; print(time.time_ns() // 1_000_000)'
}
mkdir -p "$SOURCE_STATE_ROOT/locks" "$(dirname "$OUT_DIR")"
[[ ! -e "$OUT_DIR" ]] || { echo "verification output already exists: $OUT_DIR" >&2; exit 2; }
LOCK_PATH="$SOURCE_STATE_ROOT/locks/state-execution.lock"
STDOUT_TMP="${OUT_DIR}.stdout.tmp"
STDERR_TMP="${OUT_DIR}.stderr.tmp"
PROCESS_TIME_TMP="${OUT_DIR}.process-time.tmp"
WAIT_STARTED_MS="$(now_ms)"
exec {LOCK_FD}>"$LOCK_PATH"
flock "$LOCK_FD"
LOCK_ACQUIRED_MS="$(now_ms)"
ARGS=(
--verification-only
--verification-source-run "$SOURCE_RUN_DIR"
--verification-source-state "$SOURCE_STATE_ROOT"
--verification-out "$OUT_DIR"
)
if [[ "$KEEP_SCRATCH" -eq 1 ]]; then
ARGS+=(--keep-verification-scratch)
fi
set +e
if [[ -x /usr/bin/time ]]; then
RPKI_STATE_EXECUTION_LOCK_HELD=1 /usr/bin/time -v -o "$PROCESS_TIME_TMP" -- \
"$RPKI_BIN" "${ARGS[@]}" >"$STDOUT_TMP" 2>"$STDERR_TMP"
else
RPKI_STATE_EXECUTION_LOCK_HELD=1 "$RPKI_BIN" "${ARGS[@]}" >"$STDOUT_TMP" 2>"$STDERR_TMP"
fi
EXIT_CODE=$?
set -e
LOCK_RELEASED_MS="$(now_ms)"
flock -u "$LOCK_FD"
exec {LOCK_FD}>&-
mkdir -p "$OUT_DIR"
mv "$STDOUT_TMP" "$OUT_DIR/stdout.log"
mv "$STDERR_TMP" "$OUT_DIR/stderr.log"
if [[ -f "$PROCESS_TIME_TMP" ]]; then
mv "$PROCESS_TIME_TMP" "$OUT_DIR/process-time.txt"
fi
python3 - "$OUT_DIR/wrapper-timing.json" "$WAIT_STARTED_MS" "$LOCK_ACQUIRED_MS" "$LOCK_RELEASED_MS" "$EXIT_CODE" <<'PY'
import json
import sys
from pathlib import Path
path = Path(sys.argv[1])
wait_started_ms, lock_acquired_ms, lock_released_ms, exit_code = map(int, sys.argv[2:])
path.write_text(json.dumps({
"lockWaitMs": lock_acquired_ms - wait_started_ms,
"lockHeldMs": lock_released_ms - lock_acquired_ms,
"exitCode": exit_code,
}, indent=2, sort_keys=True) + "\n", encoding="utf-8")
PY
exit "$EXIT_CODE"

View File

@@ -7,11 +7,9 @@ use std::process::Command;
use std::sync::{Arc, Mutex}; use std::sync::{Arc, Mutex};
use rpki::blob_store::ExternalRepoBytesDb; use rpki::blob_store::ExternalRepoBytesDb;
use rpki::query::report_stream::{ObjectFilter, ObjectScope};
use rpki::query::vrp::VrpLookup;
use rpki::query_db::{ use rpki::query_db::{
ChainEdgeRecord, ExportJobRecord, ObjectInstanceRecord, ObjectUriIndexRecord, QueryDb, ChainEdgeRecord, ExportJobRecord, ObjectInstanceRecord, QueryDb, QueryDbError,
QueryDbError, ValidationExplainRecord, ValidationExplainRecord,
}; };
use serde::Serialize; use serde::Serialize;
use serde_json::{Value, json}; use serde_json::{Value, json};
@@ -546,18 +544,6 @@ fn route_request(
return Ok(json!({"data":{"service":"rpki_query_service","version":1}})); return Ok(json!({"data":{"service":"rpki_query_service","version":1}}));
} }
match segments.as_slice() { match segments.as_slice() {
["healthz"] => {
let latest_ready_run = db.latest_ready_run()?;
data_response(
&json!({
"status": "ok",
"service": "rpki_query_service",
"version": 1,
"latestReadyRun": latest_ready_run,
}),
None,
)
}
["runs"] => page_response(db.list_runs(limit(&query), cursor(&query))?, None), ["runs"] => page_response(db.list_runs(limit(&query), cursor(&query))?, None),
["latest_run"] => { ["latest_run"] => {
let run_id = db let run_id = db
@@ -599,22 +585,6 @@ fn route_request(
Some(run_id), Some(run_id),
) )
} }
["runs", raw_run_id, "vrps", "lookup"] => {
let run_id = resolve_run(db, raw_run_id)?;
let lookup = vrp_lookup_from_query(&query)?;
let asn = query
.get("asn")
.map(|value| {
value.parse::<u32>().map_err(|_| {
ApiError::new(400, format!("invalid ASN query parameter: {value}"))
})
})
.transpose()?;
page_response(
db.lookup_vrps(&run_id, &lookup, asn, limit(&query), cursor(&query))?,
Some(run_id),
)
}
["runs", raw_run_id, "repos"] => { ["runs", raw_run_id, "repos"] => {
let run_id = resolve_run(db, raw_run_id)?; let run_id = resolve_run(db, raw_run_id)?;
page_response( page_response(
@@ -660,15 +630,8 @@ fn route_request(
} }
["runs", raw_run_id, "repos", repo_id, "objects"] => { ["runs", raw_run_id, "repos", repo_id, "objects"] => {
let run_id = resolve_run(db, raw_run_id)?; let run_id = resolve_run(db, raw_run_id)?;
let filter = object_filter_from_query(&query)?;
page_response( page_response(
db.list_objects_filtered( db.list_objects_for_repo(&run_id, repo_id, limit(&query), cursor(&query))?,
&run_id,
ObjectScope::Repo(repo_id.to_string()),
&filter,
limit(&query),
cursor(&query),
)?,
Some(run_id), Some(run_id),
) )
} }
@@ -706,63 +669,18 @@ fn route_request(
} }
["runs", raw_run_id, "publication-points", pp_id, "objects"] => { ["runs", raw_run_id, "publication-points", pp_id, "objects"] => {
let run_id = resolve_run(db, raw_run_id)?; let run_id = resolve_run(db, raw_run_id)?;
let filter = object_filter_from_query(&query)?;
page_response( page_response(
db.list_objects_filtered( db.list_objects_for_pp(&run_id, pp_id, limit(&query), cursor(&query))?,
&run_id,
ObjectScope::PublicationPoint(pp_id.to_string()),
&filter,
limit(&query),
cursor(&query),
)?,
Some(run_id), Some(run_id),
) )
} }
["runs", raw_run_id, "objects"] => { ["runs", raw_run_id, "objects"] => {
let run_id = resolve_run(db, raw_run_id)?; let run_id = resolve_run(db, raw_run_id)?;
let filter = object_filter_from_query(&query)?;
page_response( page_response(
db.list_objects_filtered( db.list_objects(&run_id, limit(&query), cursor(&query))?,
&run_id,
ObjectScope::All,
&filter,
limit(&query),
cursor(&query),
)?,
Some(run_id), Some(run_id),
) )
} }
["runs", raw_run_id, "issues"] => {
let run_id = resolve_run(db, raw_run_id)?;
let mut filter = object_filter_from_query(&query)?;
filter.issues_only = true;
filter.repo_id = query.get("repoId").cloned();
filter.pp_id = query.get("ppId").cloned();
page_response(
db.list_objects_filtered(
&run_id,
ObjectScope::All,
&filter,
limit(&query),
cursor(&query),
)?,
Some(run_id),
)
}
["runs", raw_run_id, "search"] => {
let run_id = resolve_run(db, raw_run_id)?;
let q = query
.get("q")
.map(String::as_str)
.unwrap_or("")
.trim()
.to_string();
if q.is_empty() {
return Err(ApiError::new(400, "q query parameter is required"));
}
let limit = search_limit(&query);
data_response(&search_run(db, &run_id, &q, limit)?, Some(run_id))
}
["runs", raw_run_id, "objects", "by-uri"] => { ["runs", raw_run_id, "objects", "by-uri"] => {
let run_id = resolve_run(db, raw_run_id)?; let run_id = resolve_run(db, raw_run_id)?;
let uri = query let uri = query
@@ -874,18 +792,6 @@ fn route_request(
offset_cursor(&query), offset_cursor(&query),
) )
} }
["runs", raw_run_id, "exports"] => {
let run_id = resolve_run(db, raw_run_id)?;
let limit = exports_limit(&query);
page_response(
rpki::query_db::QueryPage {
data: list_export_jobs(export_jobs, &run_id, limit)?,
next_cursor: None,
limit,
},
Some(run_id),
)
}
["runs", raw_run_id, "exports", job_id] => { ["runs", raw_run_id, "exports", job_id] => {
let run_id = resolve_run(db, raw_run_id)?; let run_id = resolve_run(db, raw_run_id)?;
let job = get_export_job(db, export_jobs, &run_id, job_id)? let job = get_export_job(db, export_jobs, &run_id, job_id)?
@@ -1244,24 +1150,6 @@ fn get_export_job(
Ok(db.get_export_job(run_id, job_id)?) Ok(db.get_export_job(run_id, job_id)?)
} }
fn list_export_jobs(
export_jobs: &ExportJobStore,
run_id: &str,
limit: usize,
) -> Result<Vec<ExportJobRecord>, ApiError> {
let jobs = export_jobs
.lock()
.map_err(|_| ApiError::new(500, "export job store lock poisoned"))?;
let mut out = jobs
.values()
.filter(|job| job.run_id == run_id)
.cloned()
.collect::<Vec<_>>();
out.sort_by(|left, right| right.created_at.cmp(&left.created_at));
out.truncate(limit);
Ok(out)
}
fn run_export_job( fn run_export_job(
db: &QueryDb, db: &QueryDb,
repo_bytes: &ExternalRepoBytesDb, repo_bytes: &ExternalRepoBytesDb,
@@ -1687,103 +1575,6 @@ fn projection_for_explain(
Ok(Some(projection)) Ok(Some(projection))
} }
fn search_run(db: &QueryDb, run_id: &str, q: &str, limit: usize) -> Result<Value, ApiError> {
let objects = search_run_objects(db, run_id, q, limit)?;
let lower = q.to_lowercase();
let repos = db.search_repos(run_id, &lower, limit)?;
let publication_points = db.search_publication_points(run_id, &lower, limit)?;
Ok(json!({
"objects": objects,
"repos": repos,
"publicationPoints": publication_points,
}))
}
fn search_run_objects(
db: &QueryDb,
run_id: &str,
q: &str,
limit: usize,
) -> Result<Vec<ObjectInstanceRecord>, ApiError> {
if q.starts_with("rsync://") || q.starts_with("http://") || q.starts_with("https://") {
let Some(index) = db.get_object_by_uri(run_id, q)? else {
return Ok(Vec::new());
};
let object = match db.get_object_by_instance_id(run_id, &index.object_instance_id)? {
Some(full) => full,
None => object_instance_from_uri_index(&index),
};
return Ok(vec![object]);
}
let lower = q.to_lowercase();
if is_sha256_hex(q) {
if let Some(projection) = db.get_object_projection(&lower)? {
if let Some(full) = db.get_cached_object_by_sha256(run_id, &projection.sha256)? {
return Ok(vec![full]);
}
return Ok(vec![object_instance_from_projection(run_id, &projection)]);
}
if let Some(object) = db.get_object_by_sha256(run_id, &lower)? {
return Ok(vec![object]);
}
return Ok(Vec::new());
}
let is_hex = lower.len() >= 8 && lower.bytes().all(|byte| byte.is_ascii_hexdigit());
if is_hex {
let mut objects = Vec::new();
for projection in db.search_object_projections_by_hash_prefix(&lower, limit)? {
let object = match db.get_cached_object_by_sha256(run_id, &projection.sha256)? {
Some(full) => full,
None => object_instance_from_projection(run_id, &projection),
};
objects.push(object);
}
return Ok(objects);
}
Ok(Vec::new())
}
fn object_instance_from_uri_index(index: &ObjectUriIndexRecord) -> ObjectInstanceRecord {
ObjectInstanceRecord {
schema_version: 1,
run_id: index.run_id.clone(),
object_instance_id: index.object_instance_id.clone(),
uri: index.uri.clone(),
uri_hash: String::new(),
sha256: index.sha256.clone(),
object_type: String::new(),
result: String::new(),
detail_summary: None,
repo_id: index.repo_id.clone(),
pp_id: index.pp_id.clone(),
source_section: "uri_index".to_string(),
rejected: false,
reject_reason: None,
}
}
fn object_instance_from_projection(
run_id: &str,
projection: &rpki::object_projection::ObjectProjectionRecord,
) -> ObjectInstanceRecord {
ObjectInstanceRecord {
schema_version: 1,
run_id: run_id.to_string(),
object_instance_id: String::new(),
uri: String::new(),
uri_hash: String::new(),
sha256: projection.sha256.clone(),
object_type: projection.object_type.clone(),
result: projection.parse_status.clone(),
detail_summary: projection.error_summary.clone(),
repo_id: String::new(),
pp_id: String::new(),
source_section: "hash_index".to_string(),
rejected: false,
reject_reason: None,
}
}
fn global_projection_for_hash( fn global_projection_for_hash(
db: &QueryDb, db: &QueryDb,
repo_bytes: Option<&ExternalRepoBytesDb>, repo_bytes: Option<&ExternalRepoBytesDb>,
@@ -2087,93 +1878,6 @@ fn limit(query: &BTreeMap<String, String>) -> usize {
.clamp(1, 1000) .clamp(1, 1000)
} }
fn search_limit(query: &BTreeMap<String, String>) -> usize {
query
.get("limit")
.and_then(|value| value.parse::<usize>().ok())
.unwrap_or(10)
.clamp(1, 50)
}
fn exports_limit(query: &BTreeMap<String, String>) -> usize {
query
.get("limit")
.and_then(|value| value.parse::<usize>().ok())
.unwrap_or(50)
.clamp(1, 200)
}
fn object_filter_from_query(query: &BTreeMap<String, String>) -> Result<ObjectFilter, ApiError> {
let mut filter = ObjectFilter::default();
if let Some(value) = query.get("type") {
if value.is_empty() {
return Err(ApiError::new(400, "type filter must not be empty"));
}
// Accept projection-vocabulary aliases so both `mft` and `manifest`
// (etc.) match the URI-derived report stream types.
let lower = value.to_ascii_lowercase();
let normalized = match lower.as_str() {
"mft" => "manifest",
"cer" => "certificate",
"asa" => "aspa",
_ => lower.as_str(),
};
filter.object_type = Some(normalized.to_string());
}
if let Some(value) = query.get("result") {
if !matches!(
value.to_ascii_lowercase().as_str(),
"ok" | "error" | "skipped"
) {
return Err(ApiError::new(
400,
format!("invalid result filter: {value}"),
));
}
filter.result = Some(value.clone());
}
if let Some(value) = query.get("rejected") {
filter.rejected = Some(match value.as_str() {
"true" => true,
"false" => false,
_ => {
return Err(ApiError::new(
400,
format!("invalid rejected filter: {value}"),
));
}
});
}
if let Some(value) = query.get("reason") {
filter.reason = Some(value.to_lowercase());
}
if let Some(value) = query.get("q") {
filter.query = Some(value.to_lowercase());
}
Ok(filter)
}
fn vrp_lookup_from_query(query: &BTreeMap<String, String>) -> Result<VrpLookup, ApiError> {
let ip = query.get("ip");
let prefix = query.get("prefix");
match (ip, prefix) {
(Some(_), Some(_)) => Err(ApiError::new(
400,
"ip and prefix query parameters are mutually exclusive",
)),
(None, None) => Err(ApiError::new(
400,
"one of ip or prefix query parameters is required",
)),
(Some(value), None) => {
VrpLookup::parse_ip(value).map_err(|error| ApiError::new(400, error))
}
(None, Some(value)) => {
VrpLookup::parse_prefix(value).map_err(|error| ApiError::new(400, error))
}
}
}
fn cursor(query: &BTreeMap<String, String>) -> Option<&str> { fn cursor(query: &BTreeMap<String, String>) -> Option<&str> {
query.get("cursor").map(String::as_str) query.get("cursor").map(String::as_str)
} }
@@ -2519,28 +2223,6 @@ mod tests {
.as_u64(), .as_u64(),
Some(1) Some(1)
); );
let vrp_ip = test_route_request(
&db,
Some(&repo_bytes),
"/api/v1/runs/latest/vrps/lookup?ip=2001:da8::1",
)
.expect("VRP IP lookup");
assert_eq!(vrp_ip["data"][0]["asn"].as_u64(), Some(4538));
assert_eq!(vrp_ip["meta"]["runId"].as_str(), Some("run_0001"));
let vrp_prefix = test_route_request(
&db,
Some(&repo_bytes),
"/api/v1/runs/latest/vrps/lookup?prefix=2001%3Ada8%3A%3A%2F32&asn=4538",
)
.expect("VRP prefix lookup");
assert_eq!(vrp_prefix["data"].as_array().unwrap().len(), 1);
let invalid = test_route_request(
&db,
Some(&repo_bytes),
"/api/v1/runs/latest/vrps/lookup?ip=192.0.2.1&prefix=192.0.2.0%2F24",
)
.expect_err("conflicting VRP lookup parameters");
assert_eq!(invalid.status, 400);
for target in [ for target in [
"/api/v1/runs/latest/repos", "/api/v1/runs/latest/repos",

View File

@@ -1,171 +0,0 @@
use std::path::PathBuf;
fn usage() -> &'static str {
"Usage: verification_ccr_compare --source-ccr <path> --verification-ccr <path> --out-json <path> [--out-md <path>]"
}
#[derive(Debug, PartialEq, Eq)]
struct Args {
source_ccr: PathBuf,
verification_ccr: PathBuf,
out_json: PathBuf,
out_md: Option<PathBuf>,
}
fn parse_args(argv: &[String]) -> Result<Args, String> {
let mut source_ccr = None;
let mut verification_ccr = None;
let mut out_json = None;
let mut out_md = None;
let mut index = 1usize;
while index < argv.len() {
let target = match argv[index].as_str() {
"--source-ccr" => &mut source_ccr,
"--verification-ccr" => &mut verification_ccr,
"--out-json" => &mut out_json,
"--out-md" => &mut out_md,
"-h" | "--help" => return Err(usage().to_string()),
other => return Err(format!("unknown argument: {other}\n{}", usage())),
};
index += 1;
*target =
Some(PathBuf::from(argv.get(index).ok_or_else(|| {
format!("{} requires a value", argv[index - 1])
})?));
index += 1;
}
let source_ccr = source_ccr.ok_or_else(|| format!("--source-ccr is required\n{}", usage()))?;
let verification_ccr =
verification_ccr.ok_or_else(|| format!("--verification-ccr is required\n{}", usage()))?;
let out_json = out_json.ok_or_else(|| format!("--out-json is required\n{}", usage()))?;
Ok(Args {
source_ccr,
verification_ccr,
out_json,
out_md,
})
}
fn run(args: &Args) -> Result<(), String> {
let comparison =
rpki::verification_only::compare_ccr_files(&args.source_ccr, &args.verification_ccr)?;
rpki::verification_only::write_ccr_comparison(
&args.out_json,
args.out_md.as_deref(),
&comparison,
)?;
if !comparison.state_digest_match {
return Err(format!(
"CCR state digest mismatch: {}",
comparison.mismatched_states.join(",")
));
}
println!("CCR state digests match: {}", args.out_json.display());
Ok(())
}
fn main() -> Result<(), String> {
let argv = std::env::args().collect::<Vec<_>>();
run(&parse_args(&argv)?)
}
#[cfg(test)]
mod tests {
use super::*;
use rpki::ccr::{
CcrContentInfo, CcrDigestAlgorithm, RpkiCanonicalCacheRepresentation,
build_roa_payload_state, encode_content_info,
};
use rpki::data_model::roa::{IpPrefix, RoaAfi};
use rpki::validation::objects::Vrp;
use std::fs;
fn argv(values: &[&str]) -> Vec<String> {
std::iter::once("verification_ccr_compare")
.chain(values.iter().copied())
.map(str::to_string)
.collect()
}
fn sample_ccr(asn: u32) -> Vec<u8> {
let vrps = build_roa_payload_state(&[Vrp {
asn,
prefix: IpPrefix {
afi: RoaAfi::Ipv4,
prefix_len: 24,
addr: [192, 0, 2, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
},
max_length: 24,
}])
.expect("build VRP state");
encode_content_info(&CcrContentInfo::new(RpkiCanonicalCacheRepresentation {
version: 0,
hash_alg: CcrDigestAlgorithm::Sha256,
produced_at: time::OffsetDateTime::UNIX_EPOCH,
mfts: None,
vrps: Some(vrps),
vaps: None,
tas: None,
rks: None,
}))
.expect("encode CCR")
}
#[test]
fn parse_args_accepts_complete_input_and_rejects_bad_forms() {
let parsed = parse_args(&argv(&[
"--source-ccr",
"source.ccr",
"--verification-ccr",
"verification.ccr",
"--out-json",
"result.json",
"--out-md",
"result.md",
]))
.expect("parse");
assert_eq!(parsed.source_ccr, PathBuf::from("source.ccr"));
assert_eq!(parsed.out_md, Some(PathBuf::from("result.md")));
assert!(parse_args(&argv(&[])).unwrap_err().contains("source-ccr"));
assert!(
parse_args(&argv(&["--source-ccr"]))
.unwrap_err()
.contains("requires a value")
);
assert!(
parse_args(&argv(&["--unknown"]))
.unwrap_err()
.contains("unknown argument")
);
assert!(parse_args(&argv(&["--help"])).is_err());
}
#[test]
fn run_writes_matching_outputs_and_rejects_mismatch() {
let temp = tempfile::tempdir().expect("tempdir");
let source = temp.path().join("source.ccr");
let verification = temp.path().join("verification.ccr");
let out_json = temp.path().join("out/result.json");
let out_md = temp.path().join("out/result.md");
fs::write(&source, sample_ccr(64496)).expect("source");
fs::write(&verification, sample_ccr(64496)).expect("verification");
let args = Args {
source_ccr: source.clone(),
verification_ccr: verification.clone(),
out_json: out_json.clone(),
out_md: Some(out_md.clone()),
};
run(&args).expect("matching run");
assert!(out_json.is_file());
assert!(out_md.is_file());
fs::write(&verification, sample_ccr(64497)).expect("mismatch");
let error = run(&args).unwrap_err();
assert!(error.contains("vrps"));
assert!(
fs::read_to_string(out_json)
.expect("comparison JSON")
.contains("mismatchedStates")
);
}
}

View File

@@ -77,7 +77,6 @@ pub struct ExternalRawStoreDb {
pub struct ExternalRepoBytesDb { pub struct ExternalRepoBytesDb {
path: PathBuf, path: PathBuf,
db: Arc<DB>, db: Arc<DB>,
read_only: bool,
} }
impl ExternalRawStoreDb { impl ExternalRawStoreDb {
@@ -190,7 +189,6 @@ impl ExternalRepoBytesDb {
Ok(Self { Ok(Self {
path, path,
db: Arc::new(db), db: Arc::new(db),
read_only: false,
}) })
} }
@@ -203,7 +201,6 @@ impl ExternalRepoBytesDb {
Ok(Self { Ok(Self {
path, path,
db: Arc::new(db), db: Arc::new(db),
read_only: true,
}) })
} }
@@ -211,12 +208,6 @@ impl ExternalRepoBytesDb {
if blobs.is_empty() { if blobs.is_empty() {
return Ok(()); return Ok(());
} }
if self.read_only {
return Err(StorageError::RocksDb(format!(
"repo-bytes DB is read-only: {}",
self.path.display()
)));
}
let mut batch = WriteBatch::default(); let mut batch = WriteBatch::default();
for (sha256_hex, bytes) in blobs { for (sha256_hex, bytes) in blobs {
validate_blob_sha256_hex(sha256_hex)?; validate_blob_sha256_hex(sha256_hex)?;
@@ -230,42 +221,6 @@ impl ExternalRepoBytesDb {
Ok(()) Ok(())
} }
pub(crate) fn is_read_only(&self) -> bool {
self.read_only
}
pub(crate) fn require_existing_blob_bytes_batch(
&self,
blobs: &[(String, Vec<u8>)],
) -> StorageResult<()> {
if blobs.is_empty() {
return Ok(());
}
let hashes = blobs
.iter()
.map(|(sha256_hex, _)| sha256_hex.clone())
.collect::<Vec<_>>();
let existing = self.get_blob_bytes_batch(&hashes)?;
for ((sha256_hex, expected_bytes), actual_bytes) in blobs.iter().zip(existing) {
match actual_bytes {
Some(actual_bytes) if actual_bytes == *expected_bytes => {}
Some(_) => {
return Err(StorageError::InvalidData {
entity: "read_only_repo_bytes",
detail: format!("existing bytes differ for SHA-256 {sha256_hex}"),
});
}
None => {
return Err(StorageError::InvalidData {
entity: "read_only_repo_bytes",
detail: format!("blob is missing for SHA-256 {sha256_hex}"),
});
}
}
}
Ok(())
}
pub fn get_blob_bytes(&self, sha256_hex: &str) -> StorageResult<Option<Vec<u8>>> { pub fn get_blob_bytes(&self, sha256_hex: &str) -> StorageResult<Option<Vec<u8>>> {
validate_blob_sha256_hex(sha256_hex)?; validate_blob_sha256_hex(sha256_hex)?;
let key = repo_bytes_key(sha256_hex); let key = repo_bytes_key(sha256_hex);

View File

@@ -6,7 +6,6 @@ use crate::ccr::build::{
}; };
use crate::ccr::encode::encode_manifest_state_payload_der; use crate::ccr::encode::encode_manifest_state_payload_der;
use crate::ccr::hash::compute_state_hash; use crate::ccr::hash::compute_state_hash;
use crate::ccr::manifest_location::select_manifest_signed_object_location_from_der;
use crate::ccr::model::{ use crate::ccr::model::{
CcrDigestAlgorithm, ManifestInstance, ManifestState, RpkiCanonicalCacheRepresentation, CcrDigestAlgorithm, ManifestInstance, ManifestState, RpkiCanonicalCacheRepresentation,
}; };
@@ -55,10 +54,7 @@ impl CcrManifestContribution {
aki: projection.manifest_ee_aki.clone(), aki: projection.manifest_ee_aki.clone(),
manifest_number_be: projection.manifest_number_be.clone(), manifest_number_be: projection.manifest_number_be.clone(),
this_update, this_update,
locations_der: vec![select_manifest_signed_object_location_from_der( locations_der: projection.manifest_sia_locations_der.clone(),
&projection.manifest_rsync_uri,
&projection.manifest_sia_locations_der,
)?],
subordinate_skis: projection.subordinate_skis.clone(), subordinate_skis: projection.subordinate_skis.clone(),
}) })
} }
@@ -250,8 +246,7 @@ mod tests {
use crate::ccr::export::build_ccr_from_run; use crate::ccr::export::build_ccr_from_run;
use crate::ccr::verify::verify_content_info; use crate::ccr::verify::verify_content_info;
use crate::data_model::manifest::ManifestObject; use crate::data_model::manifest::ManifestObject;
use crate::data_model::oid::{OID_AD_RPKI_NOTIFY, OID_AD_SIGNED_OBJECT}; use crate::data_model::rc::SubjectInfoAccess;
use crate::data_model::rc::{AccessDescription, SubjectInfoAccess};
use crate::data_model::roa::{IpPrefix, RoaAfi}; use crate::data_model::roa::{IpPrefix, RoaAfi};
use crate::data_model::ta::TrustAnchor; use crate::data_model::ta::TrustAnchor;
use crate::data_model::tal::Tal; use crate::data_model::tal::Tal;
@@ -288,35 +283,16 @@ mod tests {
) )
.expect("read manifest"); .expect("read manifest");
let manifest = ManifestObject::decode_der(&manifest_der).expect("decode manifest"); let manifest = ManifestObject::decode_der(&manifest_der).expect("decode manifest");
let manifest_uri = match manifest.signed_object.signed_data.certificates[0]
.resource_cert
.tbs
.extensions
.subject_info_access
.as_ref()
.expect("manifest sia")
{
SubjectInfoAccess::Ee(ee_sia) => ee_sia
.access_descriptions
.iter()
.find(|ad| {
ad.access_method_oid == OID_AD_SIGNED_OBJECT
&& ad.access_location.starts_with("rsync://")
})
.expect("manifest rsync signedObject")
.access_location
.clone(),
SubjectInfoAccess::Ca(_) => panic!("manifest ee sia should not be CA variant"),
};
let manifest_hash = hex::encode(sha2::Sha256::digest(&manifest_der)); let manifest_hash = hex::encode(sha2::Sha256::digest(&manifest_der));
let mut raw = RawByHashEntry::from_bytes(manifest_hash.clone(), manifest_der.clone()); let mut raw = RawByHashEntry::from_bytes(manifest_hash.clone(), manifest_der.clone());
raw.origin_uris.push(manifest_uri.clone()); raw.origin_uris
.push("rsync://example.test/repo/current.mft".to_string());
raw.object_type = Some("mft".to_string()); raw.object_type = Some("mft".to_string());
raw.encoding = Some("der".to_string()); raw.encoding = Some("der".to_string());
store.put_raw_by_hash_entry(&raw).expect("put raw"); store.put_raw_by_hash_entry(&raw).expect("put raw");
let projection = VcirCcrManifestProjection { let projection = VcirCcrManifestProjection {
manifest_rsync_uri: manifest_uri.clone(), manifest_rsync_uri: "rsync://example.test/repo/current.mft".to_string(),
manifest_sha256: sha2::Sha256::digest(&manifest_der).to_vec(), manifest_sha256: sha2::Sha256::digest(&manifest_der).to_vec(),
manifest_size: manifest_der.len() as u64, manifest_size: manifest_der.len() as u64,
manifest_ee_aki: manifest.signed_object.signed_data.certificates[0] manifest_ee_aki: manifest.signed_object.signed_data.certificates[0]
@@ -336,20 +312,23 @@ mod tests {
.as_ref() .as_ref()
.expect("manifest sia") .expect("manifest sia")
{ {
SubjectInfoAccess::Ee(ee_sia) => vec![ SubjectInfoAccess::Ee(ee_sia) => ee_sia
crate::ccr::manifest_location::select_manifest_signed_object_location( .access_descriptions
&manifest_uri, .iter()
&ee_sia.access_descriptions, .map(|ad| {
) let oid = encode_oid_for_test(&ad.access_method_oid)
.expect("select manifest signedObject"), .expect("encode access method oid");
], let uri = encode_tlv_for_test(0x86, ad.access_location.as_bytes().to_vec());
encode_sequence_for_test(&[oid, uri])
})
.collect(),
SubjectInfoAccess::Ca(_) => panic!("manifest ee sia should not be CA variant"), SubjectInfoAccess::Ca(_) => panic!("manifest ee sia should not be CA variant"),
}, },
subordinate_skis: vec![vec![0x33; 20]], subordinate_skis: vec![vec![0x33; 20]],
}; };
let vcir = ValidatedCaInstanceResult { let vcir = ValidatedCaInstanceResult {
manifest_rsync_uri: manifest_uri.clone(), manifest_rsync_uri: "rsync://example.test/repo/current.mft".to_string(),
parent_manifest_rsync_uri: None, parent_manifest_rsync_uri: None,
tal_id: "apnic".to_string(), tal_id: "apnic".to_string(),
ca_subject_name: "CN=test".to_string(), ca_subject_name: "CN=test".to_string(),
@@ -358,8 +337,8 @@ mod tests {
last_successful_validation_time: PackTime::from_utc_offset_datetime( last_successful_validation_time: PackTime::from_utc_offset_datetime(
manifest.manifest.this_update, manifest.manifest.this_update,
), ),
current_manifest_rsync_uri: manifest_uri.clone(), current_manifest_rsync_uri: "rsync://example.test/repo/current.mft".to_string(),
current_crl_rsync_uri: format!("{manifest_uri}.crl"), current_crl_rsync_uri: "rsync://example.test/repo/current.crl".to_string(),
validated_manifest_meta: ValidatedManifestMeta { validated_manifest_meta: ValidatedManifestMeta {
validated_manifest_number: manifest.manifest.manifest_number.bytes_be.clone(), validated_manifest_number: manifest.manifest.manifest_number.bytes_be.clone(),
validated_manifest_this_update: PackTime::from_utc_offset_datetime( validated_manifest_this_update: PackTime::from_utc_offset_datetime(
@@ -402,7 +381,7 @@ mod tests {
related_artifacts: vec![VcirRelatedArtifact { related_artifacts: vec![VcirRelatedArtifact {
artifact_role: VcirArtifactRole::Manifest, artifact_role: VcirArtifactRole::Manifest,
artifact_kind: VcirArtifactKind::Mft, artifact_kind: VcirArtifactKind::Mft,
uri: Some(manifest_uri.clone()), uri: Some("rsync://example.test/repo/current.mft".to_string()),
sha256: manifest_hash, sha256: manifest_hash,
object_type: Some("mft".to_string()), object_type: Some("mft".to_string()),
validation_status: VcirArtifactValidationStatus::Accepted, validation_status: VcirArtifactValidationStatus::Accepted,
@@ -450,6 +429,55 @@ mod tests {
(vcir, vrps, aspas, router_keys) (vcir, vrps, aspas, router_keys)
} }
fn encode_oid_for_test(oid: &str) -> Result<Vec<u8>, String> {
let arcs = oid
.split('.')
.map(|part| part.parse::<u64>().map_err(|_| format!("bad oid: {oid}")))
.collect::<Result<Vec<_>, _>>()?;
if arcs.len() < 2 {
return Err(format!("bad oid: {oid}"));
}
let mut body = Vec::new();
body.push((arcs[0] * 40 + arcs[1]) as u8);
for arc in &arcs[2..] {
encode_base128_for_test(*arc, &mut body);
}
Ok(encode_tlv_for_test(0x06, body))
}
fn encode_base128_for_test(mut value: u64, out: &mut Vec<u8>) {
let mut tmp = vec![(value & 0x7F) as u8];
value >>= 7;
while value > 0 {
tmp.push(((value & 0x7F) as u8) | 0x80);
value >>= 7;
}
tmp.reverse();
out.extend_from_slice(&tmp);
}
fn encode_sequence_for_test(elements: &[Vec<u8>]) -> Vec<u8> {
let total_len: usize = elements.iter().map(Vec::len).sum();
let mut buf = Vec::with_capacity(total_len);
for element in elements {
buf.extend_from_slice(element);
}
encode_tlv_for_test(0x30, buf)
}
fn encode_tlv_for_test(tag: u8, value: Vec<u8>) -> Vec<u8> {
let mut out = Vec::with_capacity(1 + 9 + value.len());
out.push(tag);
if value.len() < 0x80 {
out.push(value.len() as u8);
} else {
out.push(0x81);
out.push(value.len() as u8);
}
out.extend_from_slice(&value);
out
}
#[test] #[test]
fn accumulator_finish_matches_builder_on_fresh_vcir_inputs() { fn accumulator_finish_matches_builder_on_fresh_vcir_inputs() {
let td = tempfile::tempdir().expect("tempdir"); let td = tempfile::tempdir().expect("tempdir");
@@ -484,32 +512,4 @@ mod tests {
let ci = crate::ccr::model::CcrContentInfo::new(accumulated_ccr); let ci = crate::ccr::model::CcrContentInfo::new(accumulated_ccr);
verify_content_info(&ci).expect("verify accumulated ccr"); verify_content_info(&ci).expect("verify accumulated ccr");
} }
#[test]
fn accumulator_sanitizes_historical_projection_with_rpki_notify() {
let td = tempfile::tempdir().expect("tempdir");
let store = RocksStore::open(td.path()).expect("open rocksdb");
let (mut vcir, _, _, _) = sample_vcir_and_manifest(&store);
let notify_der =
crate::ccr::manifest_location::encode_access_description_der(&AccessDescription {
access_method_oid: OID_AD_RPKI_NOTIFY.to_string(),
access_location: "https://rrdp.example.test/notification.xml".to_string(),
})
.expect("encode rpkiNotify");
let expected_location = vcir.ccr_manifest_projection.manifest_sia_locations_der[0].clone();
vcir.ccr_manifest_projection
.manifest_sia_locations_der
.push(notify_der);
let mut accumulator = CcrAccumulator::new(Vec::new());
accumulator
.append_manifest_projection(&vcir.ccr_manifest_projection)
.expect("sanitize historical projection");
let contribution = accumulator
.manifests_by_hash
.values()
.next()
.expect("manifest contribution");
assert_eq!(contribution.locations_der, vec![expected_location]);
}
} }

View File

@@ -9,13 +9,12 @@ use crate::ccr::encode::{
encode_trust_anchor_state_payload_der, encode_trust_anchor_state_payload_der,
}; };
use crate::ccr::hash::compute_state_hash; use crate::ccr::hash::compute_state_hash;
use crate::ccr::manifest_location::select_manifest_signed_object_location;
use crate::ccr::model::{ use crate::ccr::model::{
AspaPayloadSet, AspaPayloadState, ManifestInstance, ManifestState, RoaPayloadSet, AspaPayloadSet, AspaPayloadState, ManifestInstance, ManifestState, RoaPayloadSet,
RoaPayloadState, RouterKey, RouterKeySet, RouterKeyState, TrustAnchorState, RoaPayloadState, RouterKey, RouterKeySet, RouterKeyState, TrustAnchorState,
}; };
use crate::data_model::manifest::ManifestObject; use crate::data_model::manifest::ManifestObject;
use crate::data_model::rc::SubjectInfoAccess; use crate::data_model::rc::{AccessDescription, SubjectInfoAccess};
use crate::data_model::roa::RoaAfi; use crate::data_model::roa::RoaAfi;
use crate::data_model::router_cert::BgpsecRouterCertificate; use crate::data_model::router_cert::BgpsecRouterCertificate;
use crate::data_model::ta::TrustAnchor; use crate::data_model::ta::TrustAnchor;
@@ -69,17 +68,12 @@ pub enum CcrBuildError {
#[error("manifest EE certificate SIA is not the EE form: {0}")] #[error("manifest EE certificate SIA is not the EE form: {0}")]
ManifestEeSiaWrongVariant(String), ManifestEeSiaWrongVariant(String),
#[error(
"manifest EE certificate SIA location selection failed for {manifest_rsync_uri}: {detail}"
)]
ManifestSiaLocationSelection {
manifest_rsync_uri: String,
detail: String,
},
#[error("manifest child SKI is not valid lowercase hex or not 20 bytes: {0}")] #[error("manifest child SKI is not valid lowercase hex or not 20 bytes: {0}")]
InvalidChildSki(String), InvalidChildSki(String),
#[error("manifest AccessDescription contains unsupported accessMethod OID: {0}")]
UnsupportedAccessMethodOid(String),
#[error("manifest state encoding failed: {0}")] #[error("manifest state encoding failed: {0}")]
ManifestEncode(String), ManifestEncode(String),
@@ -268,16 +262,11 @@ pub fn build_manifest_state_from_vcirs_with_breakdown(
CcrBuildError::ManifestEeMissingSia(vcir.current_manifest_rsync_uri.clone()) CcrBuildError::ManifestEeMissingSia(vcir.current_manifest_rsync_uri.clone())
})?; })?;
let locations = match sia { let locations = match sia {
SubjectInfoAccess::Ee(ee_sia) => vec![ SubjectInfoAccess::Ee(ee_sia) => ee_sia
select_manifest_signed_object_location( .access_descriptions
&vcir.current_manifest_rsync_uri, .iter()
&ee_sia.access_descriptions, .map(encode_access_description_der)
) .collect::<Result<Vec<_>, _>>()?,
.map_err(|detail| CcrBuildError::ManifestSiaLocationSelection {
manifest_rsync_uri: vcir.current_manifest_rsync_uri.clone(),
detail,
})?,
],
SubjectInfoAccess::Ca(_) => { SubjectInfoAccess::Ca(_) => {
return Err(CcrBuildError::ManifestEeSiaWrongVariant( return Err(CcrBuildError::ManifestEeSiaWrongVariant(
vcir.current_manifest_rsync_uri.clone(), vcir.current_manifest_rsync_uri.clone(),
@@ -364,6 +353,45 @@ fn collect_subordinate_ski_bytes(
Ok(skis.into_iter().collect()) Ok(skis.into_iter().collect())
} }
fn encode_access_description_der(ad: &AccessDescription) -> Result<Vec<u8>, CcrBuildError> {
let oid = encode_oid_from_string(&ad.access_method_oid)?;
let uri = encode_tlv(0x86, ad.access_location.as_bytes().to_vec());
Ok(encode_sequence(&[oid, uri]))
}
fn encode_oid_from_string(oid: &str) -> Result<Vec<u8>, CcrBuildError> {
let arcs = oid
.split('.')
.map(|part| {
part.parse::<u64>()
.map_err(|_| CcrBuildError::UnsupportedAccessMethodOid(oid.to_string()))
})
.collect::<Result<Vec<_>, _>>()?;
if arcs.len() < 2 {
return Err(CcrBuildError::UnsupportedAccessMethodOid(oid.to_string()));
}
if arcs[0] > 2 || (arcs[0] < 2 && arcs[1] >= 40) {
return Err(CcrBuildError::UnsupportedAccessMethodOid(oid.to_string()));
}
let mut body = Vec::new();
body.push((arcs[0] * 40 + arcs[1]) as u8);
for arc in &arcs[2..] {
encode_base128(*arc, &mut body);
}
Ok(encode_tlv(0x06, body))
}
fn encode_base128(mut value: u64, out: &mut Vec<u8>) {
let mut tmp = vec![(value & 0x7F) as u8];
value >>= 7;
while value > 0 {
tmp.push(((value & 0x7F) as u8) | 0x80);
value >>= 7;
}
tmp.reverse();
out.extend_from_slice(&tmp);
}
pub fn build_router_key_state( pub fn build_router_key_state(
router_certs: &[BgpsecRouterCertificate], router_certs: &[BgpsecRouterCertificate],
) -> Result<RouterKeyState, CcrBuildError> { ) -> Result<RouterKeyState, CcrBuildError> {
@@ -622,33 +650,17 @@ mod tests {
} }
} }
fn sample_manifest_vcir(manifest_der: &[u8], child_ski_hex: &str) -> ValidatedCaInstanceResult { fn sample_manifest_vcir(
manifest_uri: &str,
manifest_der: &[u8],
child_ski_hex: &str,
) -> ValidatedCaInstanceResult {
let manifest = ManifestObject::decode_der(manifest_der).expect("decode manifest fixture"); let manifest = ManifestObject::decode_der(manifest_der).expect("decode manifest fixture");
let manifest_uri = match manifest.signed_object.signed_data.certificates[0]
.resource_cert
.tbs
.extensions
.subject_info_access
.as_ref()
.expect("manifest sia")
{
SubjectInfoAccess::Ee(ee_sia) => ee_sia
.access_descriptions
.iter()
.find(|ad| {
ad.access_method_oid == crate::data_model::oid::OID_AD_SIGNED_OBJECT
&& ad.access_location.starts_with("rsync://")
})
.expect("manifest rsync signedObject")
.access_location
.clone(),
SubjectInfoAccess::Ca(_) => panic!("manifest ee sia should not be CA variant"),
};
let hash = hex::encode(sha2::Sha256::digest(manifest_der)); let hash = hex::encode(sha2::Sha256::digest(manifest_der));
let now = manifest.manifest.this_update; let now = manifest.manifest.this_update;
let next = manifest.manifest.next_update; let next = manifest.manifest.next_update;
let projection = crate::storage::VcirCcrManifestProjection { let projection = crate::storage::VcirCcrManifestProjection {
manifest_rsync_uri: manifest_uri.clone(), manifest_rsync_uri: manifest_uri.to_string(),
manifest_sha256: sha2::Sha256::digest(manifest_der).to_vec(), manifest_sha256: sha2::Sha256::digest(manifest_der).to_vec(),
manifest_size: manifest_der.len() as u64, manifest_size: manifest_der.len() as u64,
manifest_ee_aki: manifest.signed_object.signed_data.certificates[0] manifest_ee_aki: manifest.signed_object.signed_data.certificates[0]
@@ -668,19 +680,18 @@ mod tests {
.as_ref() .as_ref()
.expect("manifest sia") .expect("manifest sia")
{ {
SubjectInfoAccess::Ee(ee_sia) => vec![ SubjectInfoAccess::Ee(ee_sia) => ee_sia
select_manifest_signed_object_location( .access_descriptions
&manifest_uri, .iter()
&ee_sia.access_descriptions, .map(encode_access_description_der)
) .collect::<Result<Vec<_>, _>>()
.expect("select manifest signedObject"), .expect("encode access descriptions"),
],
SubjectInfoAccess::Ca(_) => panic!("manifest ee sia should not be CA variant"), SubjectInfoAccess::Ca(_) => panic!("manifest ee sia should not be CA variant"),
}, },
subordinate_skis: vec![hex::decode(child_ski_hex).expect("decode child ski")], subordinate_skis: vec![hex::decode(child_ski_hex).expect("decode child ski")],
}; };
crate::storage::ValidatedCaInstanceResult { crate::storage::ValidatedCaInstanceResult {
manifest_rsync_uri: manifest_uri.clone(), manifest_rsync_uri: manifest_uri.to_string(),
parent_manifest_rsync_uri: None, parent_manifest_rsync_uri: None,
tal_id: "test-tal".to_string(), tal_id: "test-tal".to_string(),
ca_subject_name: "CN=test".to_string(), ca_subject_name: "CN=test".to_string(),
@@ -689,7 +700,7 @@ mod tests {
last_successful_validation_time: crate::storage::PackTime::from_utc_offset_datetime( last_successful_validation_time: crate::storage::PackTime::from_utc_offset_datetime(
now, now,
), ),
current_manifest_rsync_uri: manifest_uri.clone(), current_manifest_rsync_uri: manifest_uri.to_string(),
current_crl_rsync_uri: format!("{manifest_uri}.crl"), current_crl_rsync_uri: format!("{manifest_uri}.crl"),
validated_manifest_meta: crate::storage::ValidatedManifestMeta { validated_manifest_meta: crate::storage::ValidatedManifestMeta {
validated_manifest_number: manifest.manifest.manifest_number.bytes_be.clone(), validated_manifest_number: manifest.manifest.manifest_number.bytes_be.clone(),
@@ -725,7 +736,7 @@ mod tests {
related_artifacts: vec![crate::storage::VcirRelatedArtifact { related_artifacts: vec![crate::storage::VcirRelatedArtifact {
artifact_role: crate::storage::VcirArtifactRole::Manifest, artifact_role: crate::storage::VcirArtifactRole::Manifest,
artifact_kind: crate::storage::VcirArtifactKind::Mft, artifact_kind: crate::storage::VcirArtifactKind::Mft,
uri: Some(manifest_uri.clone()), uri: Some(manifest_uri.to_string()),
sha256: hash, sha256: hash,
object_type: Some("mft".to_string()), object_type: Some("mft".to_string()),
validation_status: crate::storage::VcirArtifactValidationStatus::Accepted, validation_status: crate::storage::VcirArtifactValidationStatus::Accepted,
@@ -755,8 +766,10 @@ mod tests {
"tests/fixtures/repository/ca.rg.net/rpki/RGnet-OU/bW-_qXU9uNhGQz21NR2ansB8lr0.mft", "tests/fixtures/repository/ca.rg.net/rpki/RGnet-OU/bW-_qXU9uNhGQz21NR2ansB8lr0.mft",
)) ))
.expect("read manifest b"); .expect("read manifest b");
let vcir_a = sample_manifest_vcir(&manifest_a, &"33".repeat(20)); let vcir_a =
let vcir_b = sample_manifest_vcir(&manifest_b, &"44".repeat(20)); sample_manifest_vcir("rsync://example.test/a.mft", &manifest_a, &"33".repeat(20));
let vcir_b =
sample_manifest_vcir("rsync://example.test/b.mft", &manifest_b, &"44".repeat(20));
let store_dir = tempfile::tempdir().expect("tempdir"); let store_dir = tempfile::tempdir().expect("tempdir");
let store = RocksStore::open(store_dir.path()).expect("open rocksdb"); let store = RocksStore::open(store_dir.path()).expect("open rocksdb");
for (vcir, bytes) in [(&vcir_a, &manifest_a), (&vcir_b, &manifest_b)] { for (vcir, bytes) in [(&vcir_a, &manifest_a), (&vcir_b, &manifest_b)] {

View File

@@ -471,71 +471,3 @@ fn decode_big_unsigned(bytes: &[u8]) -> Result<BigUnsigned, CcrDecodeError> {
}; };
Ok(BigUnsigned { bytes_be }) Ok(BigUnsigned { bytes_be })
} }
#[cfg(all(test, feature = "full"))]
mod tests {
use super::*;
use crate::ccr::encode::{encode_content_info, encode_manifest_state_payload_der};
use crate::ccr::hash::compute_state_hash;
use crate::ccr::manifest_location::encode_access_description_der;
use crate::ccr::model::{
CcrContentInfo, CcrDigestAlgorithm, ManifestState, RpkiCanonicalCacheRepresentation,
};
use crate::data_model::oid::{OID_AD_RPKI_NOTIFY, OID_AD_SIGNED_OBJECT};
use crate::data_model::rc::AccessDescription;
#[test]
fn generic_decoder_preserves_multiple_manifest_locations() {
let signed_object = encode_access_description_der(&AccessDescription {
access_method_oid: OID_AD_SIGNED_OBJECT.to_string(),
access_location: "rsync://example.test/repository/current.mft".to_string(),
})
.expect("encode signedObject");
let rpki_notify = encode_access_description_der(&AccessDescription {
access_method_oid: OID_AD_RPKI_NOTIFY.to_string(),
access_location: "https://rrdp.example.test/notification.xml".to_string(),
})
.expect("encode rpkiNotify");
let manifest = ManifestInstance {
hash: vec![0x11; 32],
size: 1024,
aki: vec![0x22; 20],
manifest_number: BigUnsigned { bytes_be: vec![1] },
this_update: time::OffsetDateTime::parse(
"2026-07-20T00:00:00Z",
&time::format_description::well_known::Rfc3339,
)
.expect("parse time"),
locations: vec![signed_object, rpki_notify],
subordinates: Vec::new(),
};
let manifest_payload = encode_manifest_state_payload_der(&[manifest.clone()])
.expect("encode manifest payload");
let content = CcrContentInfo::new(RpkiCanonicalCacheRepresentation {
version: 0,
hash_alg: CcrDigestAlgorithm::Sha256,
produced_at: time::OffsetDateTime::parse(
"2026-07-20T00:00:00Z",
&time::format_description::well_known::Rfc3339,
)
.expect("parse time"),
mfts: Some(ManifestState {
mis: vec![manifest],
most_recent_update: time::OffsetDateTime::parse(
"2026-07-20T00:00:00Z",
&time::format_description::well_known::Rfc3339,
)
.expect("parse time"),
hash: compute_state_hash(&manifest_payload),
}),
vrps: None,
vaps: None,
tas: None,
rks: None,
});
let encoded = encode_content_info(&content).expect("encode CCR");
let decoded = decode_content_info(&encoded).expect("decode CCR");
assert_eq!(decoded.content.mfts.unwrap().mis[0].locations.len(), 2);
}
}

View File

@@ -124,8 +124,6 @@ mod tests {
use super::*; use super::*;
use crate::ccr::decode::decode_content_info; use crate::ccr::decode::decode_content_info;
use crate::data_model::manifest::ManifestObject; use crate::data_model::manifest::ManifestObject;
use crate::data_model::oid::OID_AD_SIGNED_OBJECT;
use crate::data_model::rc::SubjectInfoAccess;
use crate::data_model::roa::{IpPrefix, RoaAfi}; use crate::data_model::roa::{IpPrefix, RoaAfi};
use crate::data_model::ta::TrustAnchor; use crate::data_model::ta::TrustAnchor;
use crate::data_model::tal::Tal; use crate::data_model::tal::Tal;
@@ -151,34 +149,15 @@ mod tests {
let base = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR")); let base = std::path::PathBuf::from(env!("CARGO_MANIFEST_DIR"));
let manifest_der = std::fs::read(base.join("tests/fixtures/repository/rpki.cernet.net/repo/cernet/0/05FC9C5B88506F7C0D3F862C8895BED67E9F8EBA.mft")).expect("read manifest"); let manifest_der = std::fs::read(base.join("tests/fixtures/repository/rpki.cernet.net/repo/cernet/0/05FC9C5B88506F7C0D3F862C8895BED67E9F8EBA.mft")).expect("read manifest");
let manifest = ManifestObject::decode_der(&manifest_der).expect("decode manifest"); let manifest = ManifestObject::decode_der(&manifest_der).expect("decode manifest");
let manifest_uri = match manifest.signed_object.signed_data.certificates[0]
.resource_cert
.tbs
.extensions
.subject_info_access
.as_ref()
.expect("manifest sia")
{
SubjectInfoAccess::Ee(ee_sia) => ee_sia
.access_descriptions
.iter()
.find(|ad| {
ad.access_method_oid == OID_AD_SIGNED_OBJECT
&& ad.access_location.starts_with("rsync://")
})
.expect("manifest rsync signedObject")
.access_location
.clone(),
SubjectInfoAccess::Ca(_) => panic!("manifest EE SIA should not be CA variant"),
};
let hash = hex::encode(sha2::Sha256::digest(&manifest_der)); let hash = hex::encode(sha2::Sha256::digest(&manifest_der));
let mut raw = RawByHashEntry::from_bytes(hash.clone(), manifest_der.clone()); let mut raw = RawByHashEntry::from_bytes(hash.clone(), manifest_der.clone());
raw.origin_uris.push(manifest_uri.clone()); raw.origin_uris
.push("rsync://example.test/repo/current.mft".to_string());
raw.object_type = Some("mft".to_string()); raw.object_type = Some("mft".to_string());
raw.encoding = Some("der".to_string()); raw.encoding = Some("der".to_string());
store.put_raw_by_hash_entry(&raw).expect("put raw"); store.put_raw_by_hash_entry(&raw).expect("put raw");
let projection = VcirCcrManifestProjection { let projection = VcirCcrManifestProjection {
manifest_rsync_uri: manifest_uri.clone(), manifest_rsync_uri: "rsync://example.test/repo/current.mft".to_string(),
manifest_sha256: sha2::Sha256::digest(&manifest_der).to_vec(), manifest_sha256: sha2::Sha256::digest(&manifest_der).to_vec(),
manifest_size: manifest_der.len() as u64, manifest_size: manifest_der.len() as u64,
manifest_ee_aki: manifest.signed_object.signed_data.certificates[0] manifest_ee_aki: manifest.signed_object.signed_data.certificates[0]
@@ -190,27 +169,14 @@ mod tests {
.expect("manifest aki"), .expect("manifest aki"),
manifest_number_be: manifest.manifest.manifest_number.bytes_be.clone(), manifest_number_be: manifest.manifest.manifest_number.bytes_be.clone(),
manifest_this_update: PackTime::from_utc_offset_datetime(manifest.manifest.this_update), manifest_this_update: PackTime::from_utc_offset_datetime(manifest.manifest.this_update),
manifest_sia_locations_der: match manifest.signed_object.signed_data.certificates[0] manifest_sia_locations_der: vec![vec![
.resource_cert 0x30, 0x11, 0x06, 0x08, 0x2b, 0x06, 0x01, 0x05, 0x05, 0x07, 0x30, 0x05, 0x86, 0x05,
.tbs b'r', b's', b'y', b'n', b'c',
.extensions ]],
.subject_info_access
.as_ref()
.expect("manifest sia")
{
SubjectInfoAccess::Ee(ee_sia) => vec![
crate::ccr::manifest_location::select_manifest_signed_object_location(
&manifest_uri,
&ee_sia.access_descriptions,
)
.expect("select manifest signedObject"),
],
SubjectInfoAccess::Ca(_) => panic!("manifest EE SIA should not be CA variant"),
},
subordinate_skis: vec![vec![0x33; 20]], subordinate_skis: vec![vec![0x33; 20]],
}; };
ValidatedCaInstanceResult { ValidatedCaInstanceResult {
manifest_rsync_uri: manifest_uri.clone(), manifest_rsync_uri: "rsync://example.test/repo/current.mft".to_string(),
parent_manifest_rsync_uri: None, parent_manifest_rsync_uri: None,
tal_id: "apnic".to_string(), tal_id: "apnic".to_string(),
ca_subject_name: "CN=test".to_string(), ca_subject_name: "CN=test".to_string(),
@@ -219,7 +185,7 @@ mod tests {
last_successful_validation_time: PackTime::from_utc_offset_datetime( last_successful_validation_time: PackTime::from_utc_offset_datetime(
manifest.manifest.this_update, manifest.manifest.this_update,
), ),
current_manifest_rsync_uri: manifest_uri.clone(), current_manifest_rsync_uri: "rsync://example.test/repo/current.mft".to_string(),
current_crl_rsync_uri: "rsync://example.test/repo/current.crl".to_string(), current_crl_rsync_uri: "rsync://example.test/repo/current.crl".to_string(),
validated_manifest_meta: ValidatedManifestMeta { validated_manifest_meta: ValidatedManifestMeta {
validated_manifest_number: manifest.manifest.manifest_number.bytes_be.clone(), validated_manifest_number: manifest.manifest.manifest_number.bytes_be.clone(),
@@ -263,7 +229,7 @@ mod tests {
related_artifacts: vec![VcirRelatedArtifact { related_artifacts: vec![VcirRelatedArtifact {
artifact_role: VcirArtifactRole::Manifest, artifact_role: VcirArtifactRole::Manifest,
artifact_kind: VcirArtifactKind::Mft, artifact_kind: VcirArtifactKind::Mft,
uri: Some(manifest_uri), uri: Some("rsync://example.test/repo/current.mft".to_string()),
sha256: hash, sha256: hash,
object_type: Some("mft".to_string()), object_type: Some("mft".to_string()),
validation_status: VcirArtifactValidationStatus::Accepted, validation_status: VcirArtifactValidationStatus::Accepted,

View File

@@ -1,424 +0,0 @@
use crate::data_model::common::DerReader;
use crate::data_model::oid::OID_AD_SIGNED_OBJECT;
use crate::data_model::rc::AccessDescription;
pub(crate) fn select_manifest_signed_object_location(
manifest_rsync_uri: &str,
access_descriptions: &[AccessDescription],
) -> Result<Vec<u8>, String> {
let matching = access_descriptions
.iter()
.filter(|access_description| {
access_description.access_method_oid == OID_AD_SIGNED_OBJECT
&& access_description.access_location == manifest_rsync_uri
})
.collect::<Vec<_>>();
let access_description = expect_single_matching_location(
manifest_rsync_uri,
matching.len(),
"parsed Manifest EE SIA",
)?;
encode_access_description_der(matching[access_description])
}
pub(crate) fn select_manifest_signed_object_location_from_der(
manifest_rsync_uri: &str,
locations_der: &[Vec<u8>],
) -> Result<Vec<u8>, String> {
let matching = locations_der
.iter()
.filter_map(
|location_der| match decode_access_description_der(location_der) {
Ok(access_description)
if access_description.access_method_oid == OID_AD_SIGNED_OBJECT
&& access_description.access_location == manifest_rsync_uri =>
{
Some(Ok(location_der.clone()))
}
Ok(_) => None,
Err(detail) => Some(Err(detail)),
},
)
.collect::<Result<Vec<_>, _>>()?;
let index = expect_single_matching_location(
manifest_rsync_uri,
matching.len(),
"persisted VCIR CCR projection",
)?;
Ok(matching[index].clone())
}
pub(crate) fn encode_access_description_der(
access_description: &AccessDescription,
) -> Result<Vec<u8>, String> {
let oid = encode_oid_der(&access_description.access_method_oid)?;
let uri = encode_tlv(0x86, access_description.access_location.as_bytes().to_vec());
Ok(encode_sequence(&[oid, uri]))
}
fn expect_single_matching_location(
manifest_rsync_uri: &str,
matching_count: usize,
source: &str,
) -> Result<usize, String> {
if matching_count == 1 {
return Ok(0);
}
Err(format!(
"{source} contains {matching_count} id-ad-signedObject locations matching manifest URI {manifest_rsync_uri}; expected exactly one"
))
}
fn decode_access_description_der(der: &[u8]) -> Result<AccessDescription, String> {
let mut top = DerReader::new(der);
let mut sequence = top.take_sequence()?;
if !top.is_empty() {
return Err("trailing bytes after AccessDescription".to_string());
}
let access_method_oid = decode_oid_der(sequence.take_tag(0x06)?)?;
let access_location = std::str::from_utf8(sequence.take_tag(0x86)?)
.map_err(|error| format!("AccessDescription URI is not UTF-8: {error}"))?
.to_string();
if !sequence.is_empty() {
return Err("trailing fields in AccessDescription".to_string());
}
Ok(AccessDescription {
access_method_oid,
access_location,
})
}
fn decode_oid_der(value: &[u8]) -> Result<String, String> {
let mut offset = 0usize;
let first = decode_base128(value, &mut offset)?;
let (first_arc, second_arc) = match first {
0..=39 => (0, first),
40..=79 => (1, first - 40),
value => (2, value - 80),
};
let mut arcs = vec![first_arc, second_arc];
while offset < value.len() {
arcs.push(decode_base128(value, &mut offset)?);
}
Ok(arcs
.into_iter()
.map(|arc| arc.to_string())
.collect::<Vec<_>>()
.join("."))
}
fn decode_base128(value: &[u8], offset: &mut usize) -> Result<u64, String> {
let first = *value
.get(*offset)
.ok_or_else(|| "truncated OBJECT IDENTIFIER".to_string())?;
if first == 0x80 {
return Err("non-minimal OBJECT IDENTIFIER base-128 encoding".to_string());
}
let mut out = 0u64;
loop {
let byte = *value
.get(*offset)
.ok_or_else(|| "truncated OBJECT IDENTIFIER".to_string())?;
*offset += 1;
out = out
.checked_shl(7)
.ok_or_else(|| "OBJECT IDENTIFIER arc overflows u64".to_string())?
.checked_add((byte & 0x7f) as u64)
.ok_or_else(|| "OBJECT IDENTIFIER arc overflows u64".to_string())?;
if byte & 0x80 == 0 {
return Ok(out);
}
}
}
fn encode_oid_der(oid: &str) -> Result<Vec<u8>, String> {
let arcs = oid
.split('.')
.map(|part| {
part.parse::<u64>()
.map_err(|_| format!("unsupported accessMethod OID: {oid}"))
})
.collect::<Result<Vec<_>, _>>()?;
if arcs.len() < 2 || arcs[0] > 2 || (arcs[0] < 2 && arcs[1] >= 40) {
return Err(format!("unsupported accessMethod OID: {oid}"));
}
let mut body = Vec::new();
encode_base128(
arcs[0]
.checked_mul(40)
.and_then(|value| value.checked_add(arcs[1]))
.ok_or_else(|| format!("unsupported accessMethod OID: {oid}"))?,
&mut body,
);
for arc in &arcs[2..] {
encode_base128(*arc, &mut body);
}
Ok(encode_tlv(0x06, body))
}
fn encode_base128(mut value: u64, out: &mut Vec<u8>) {
let mut encoded = vec![(value & 0x7f) as u8];
value >>= 7;
while value > 0 {
encoded.push(((value & 0x7f) as u8) | 0x80);
value >>= 7;
}
encoded.reverse();
out.extend_from_slice(&encoded);
}
fn encode_sequence(elements: &[Vec<u8>]) -> Vec<u8> {
let total_len = elements.iter().map(Vec::len).sum();
let mut value = Vec::with_capacity(total_len);
for element in elements {
value.extend_from_slice(element);
}
encode_tlv(0x30, value)
}
fn encode_tlv(tag: u8, value: Vec<u8>) -> Vec<u8> {
let mut out = Vec::with_capacity(1 + 9 + value.len());
out.push(tag);
encode_length(value.len(), &mut out);
out.extend_from_slice(&value);
out
}
fn encode_length(len: usize, out: &mut Vec<u8>) {
if len < 0x80 {
out.push(len as u8);
return;
}
let mut bytes = Vec::new();
let mut value = len;
while value > 0 {
bytes.push((value & 0xff) as u8);
value >>= 8;
}
bytes.reverse();
out.push(0x80 | bytes.len() as u8);
out.extend_from_slice(&bytes);
}
#[cfg(test)]
mod tests {
use super::*;
use crate::data_model::oid::OID_AD_RPKI_NOTIFY;
const MANIFEST_URI: &str = "rsync://example.test/repo/manifest.mft";
fn access_description(access_method_oid: &str, access_location: &str) -> AccessDescription {
AccessDescription {
access_method_oid: access_method_oid.to_string(),
access_location: access_location.to_string(),
}
}
#[test]
fn selects_matching_signed_object_and_excludes_rpki_notify() {
let signed_object = access_description(OID_AD_SIGNED_OBJECT, MANIFEST_URI);
let selected = select_manifest_signed_object_location(
MANIFEST_URI,
&[
signed_object.clone(),
access_description(
OID_AD_RPKI_NOTIFY,
"https://rrdp.example.test/notification.xml",
),
],
)
.expect("select signed object");
assert_eq!(
selected,
encode_access_description_der(&signed_object).unwrap()
);
}
#[test]
fn selects_only_the_signed_object_matching_manifest_uri() {
let expected = access_description(OID_AD_SIGNED_OBJECT, MANIFEST_URI);
let selected = select_manifest_signed_object_location(
MANIFEST_URI,
&[
access_description(
OID_AD_SIGNED_OBJECT,
"https://backup.example.test/manifest.mft",
),
expected.clone(),
],
)
.expect("select matching signed object");
assert_eq!(selected, encode_access_description_der(&expected).unwrap());
}
#[test]
fn rejects_missing_or_duplicate_matching_signed_object() {
let missing = select_manifest_signed_object_location(
MANIFEST_URI,
&[access_description(
OID_AD_RPKI_NOTIFY,
"https://rrdp.example.test/notification.xml",
)],
)
.expect_err("missing signed object must fail");
assert!(missing.contains("contains 0"), "{missing}");
let duplicate = select_manifest_signed_object_location(
MANIFEST_URI,
&[
access_description(OID_AD_SIGNED_OBJECT, MANIFEST_URI),
access_description(OID_AD_SIGNED_OBJECT, MANIFEST_URI),
],
)
.expect_err("duplicate signed object must fail");
assert!(duplicate.contains("contains 2"), "{duplicate}");
}
#[test]
fn selects_matching_signed_object_from_historical_projection() {
let signed_object = access_description(OID_AD_SIGNED_OBJECT, MANIFEST_URI);
let signed_object_der = encode_access_description_der(&signed_object).unwrap();
let notify_der = encode_access_description_der(&access_description(
OID_AD_RPKI_NOTIFY,
"https://rrdp.example.test/notification.xml",
))
.unwrap();
let selected = select_manifest_signed_object_location_from_der(
MANIFEST_URI,
&[notify_der, signed_object_der.clone()],
)
.expect("select historical signed object");
assert_eq!(selected, signed_object_der);
}
#[test]
fn rejects_malformed_historical_projection() {
let error = select_manifest_signed_object_location_from_der(
MANIFEST_URI,
&[vec![0x30, 0x01, 0x06]],
)
.expect_err("malformed historical projection must fail");
assert!(error.contains("truncated DER"), "{error}");
}
#[test]
fn access_description_der_codec_covers_long_and_invalid_forms() {
let long = access_description(
"2.999.200.1",
&format!("rsync://example.test/repo/{}", "x".repeat(160)),
);
let encoded = encode_access_description_der(&long).expect("encode long access description");
assert_eq!(
decode_access_description_der(&encoded).expect("decode long access description"),
long
);
let mut trailing = encoded.clone();
trailing.push(0);
assert!(
decode_access_description_der(&trailing)
.expect_err("trailing bytes must fail")
.contains("trailing bytes")
);
let oid = encode_oid_der(OID_AD_SIGNED_OBJECT).expect("encode signedObject OID");
let uri = encode_tlv(0x86, MANIFEST_URI.as_bytes().to_vec());
assert!(
decode_access_description_der(&encode_sequence(&[
oid.clone(),
uri.clone(),
encode_tlv(0x05, Vec::new()),
]))
.expect_err("trailing field must fail")
.contains("trailing fields")
);
assert!(
decode_access_description_der(&encode_sequence(&[oid, encode_tlv(0x86, vec![0xff]),]))
.expect_err("non-UTF8 URI must fail")
.contains("not UTF-8")
);
assert!(
decode_access_description_der(&encode_sequence(&[
encode_tlv(0x06, vec![0x80, 0x00]),
uri.clone(),
]))
.expect_err("non-minimal OID must fail")
.contains("non-minimal")
);
assert!(
decode_access_description_der(&encode_sequence(&[encode_tlv(0x06, vec![0x81]), uri,]))
.expect_err("truncated OID must fail")
.contains("truncated OBJECT IDENTIFIER")
);
assert!(
encode_access_description_der(&access_description("3.1", MANIFEST_URI))
.expect_err("invalid OID must fail")
.contains("unsupported accessMethod OID")
);
}
#[test]
fn access_description_der_codec_reports_malformed_selector_inputs() {
let malformed_sequence = decode_access_description_der(&[0x31, 0x00])
.expect_err("non-sequence AccessDescription must fail");
assert!(!malformed_sequence.is_empty());
let missing_method = decode_access_description_der(&encode_sequence(&[encode_tlv(
0x86,
MANIFEST_URI.as_bytes().to_vec(),
)]))
.expect_err("missing accessMethod must fail");
assert!(!missing_method.is_empty());
let oid = encode_oid_der(OID_AD_SIGNED_OBJECT).expect("encode signedObject OID");
let missing_location = decode_access_description_der(&encode_sequence(&[oid.clone()]))
.expect_err("missing accessLocation must fail");
assert!(!missing_location.is_empty());
let wrong_location_tag = decode_access_description_der(&encode_sequence(&[
oid.clone(),
encode_tlv(0x04, MANIFEST_URI.as_bytes().to_vec()),
]))
.expect_err("wrong accessLocation tag must fail");
assert!(!wrong_location_tag.is_empty());
let empty_oid = decode_access_description_der(&encode_sequence(&[
encode_tlv(0x06, Vec::new()),
encode_tlv(0x86, MANIFEST_URI.as_bytes().to_vec()),
]))
.expect_err("empty OID must fail");
assert!(!empty_oid.is_empty());
let first_arc_zero = decode_access_description_der(&encode_sequence(&[
encode_tlv(0x06, vec![0x01, 0x02]),
encode_tlv(0x86, MANIFEST_URI.as_bytes().to_vec()),
]))
.expect("decode first OID arc zero");
assert_eq!(first_arc_zero.access_method_oid, "0.1.2");
assert_eq!(first_arc_zero.access_location, MANIFEST_URI);
let non_numeric =
encode_access_description_der(&access_description("no.such.oid", MANIFEST_URI))
.expect_err("non-numeric OID must fail");
assert!(!non_numeric.is_empty());
let too_short = encode_access_description_der(&access_description("1", MANIFEST_URI))
.expect_err("OID without second arc must fail");
assert!(!too_short.is_empty());
let invalid_second_arc =
encode_access_description_der(&access_description("1.40", MANIFEST_URI))
.expect_err("invalid second OID arc must fail");
assert!(!invalid_second_arc.is_empty());
let overflowing_first_subidentifier = encode_access_description_der(&access_description(
"2.18446744073709551615",
MANIFEST_URI,
))
.expect_err("overflowing first OID subidentifier must fail");
assert!(!overflowing_first_subidentifier.is_empty());
}
}

View File

@@ -10,8 +10,6 @@ pub mod encode;
#[cfg(feature = "full")] #[cfg(feature = "full")]
pub mod export; pub mod export;
pub mod hash; pub mod hash;
#[cfg(feature = "full")]
pub(crate) mod manifest_location;
pub mod model; pub mod model;
pub mod state_digest; pub mod state_digest;
#[cfg(feature = "full")] #[cfg(feature = "full")]

View File

@@ -38,7 +38,7 @@ use crate::validation::run_tree_from_tal::{
run_tree_from_tal_url_parallel_phase2_audit, run_tree_from_tal_url_parallel_phase2_audit,
run_tree_from_tal_url_parallel_phase2_audit_with_timing, run_tree_from_tal_url_parallel_phase2_audit_with_timing,
}; };
use crate::validation::tree::{DEFAULT_MAX_CA_DEPTH, TreeRunConfig}; use crate::validation::tree::TreeRunConfig;
#[cfg(test)] #[cfg(test)]
use output::write_json; use output::write_json;
use output::{ use output::{
@@ -115,12 +115,6 @@ fn vcir_storage_summary_enabled() -> bool {
#[derive(Clone, Debug, PartialEq, Eq)] #[derive(Clone, Debug, PartialEq, Eq)]
pub struct CliArgs { pub struct CliArgs {
pub verification_only: bool,
pub verification_source_run_dir: Option<PathBuf>,
pub verification_source_state_root: Option<PathBuf>,
pub verification_out_dir: Option<PathBuf>,
pub keep_verification_scratch: bool,
pub validation_contract_out_path: Option<PathBuf>,
pub tal_urls: Vec<String>, pub tal_urls: Vec<String>,
pub tal_paths: Vec<PathBuf>, pub tal_paths: Vec<PathBuf>,
pub ta_paths: Vec<PathBuf>, pub ta_paths: Vec<PathBuf>,
@@ -174,7 +168,7 @@ pub struct CliArgs {
pub rsync_mirror_root: Option<PathBuf>, pub rsync_mirror_root: Option<PathBuf>,
pub rsync_scope_policy: RsyncScopePolicy, pub rsync_scope_policy: RsyncScopePolicy,
pub max_ca_depth: usize, pub max_depth: Option<usize>,
pub max_instances: Option<usize>, pub max_instances: Option<usize>,
pub validation_time: Option<time::OffsetDateTime>, pub validation_time: Option<time::OffsetDateTime>,
@@ -190,19 +184,8 @@ fn usage() -> String {
Usage: Usage:
{bin} --db <path> --tal-url <url> [--tal-url <url> ...] [options] {bin} --db <path> --tal-url <url> [--tal-url <url> ...] [options]
{bin} --db <path> --tal-path <path> --ta-path <path> [--tal-path <path> --ta-path <path> ...] [options] {bin} --db <path> --tal-path <path> --ta-path <path> [--tal-path <path> --ta-path <path> ...] [options]
{bin} --verification-only --verification-source-run <path> --verification-source-state <path> --verification-out <path> [options]
Options: Options:
--verification-only Revalidate the latest successful run without validation caches or network access
--verification-source-run <path>
Source normal run directory containing input.cir/result.ccr/validation-contract.json
--verification-source-state <path>
Source state root containing db/work-db and db/repo-bytes.db
--verification-out <path> Independent output directory for the one-shot verification run
--keep-verification-scratch
Keep the private work-db checkpoint after a successful verification
--validation-contract-out <path>
Write the effective normal-run validation contract (not valid in verification-only mode)
--db <path> RocksDB directory path (required) --db <path> RocksDB directory path (required)
--raw-store-db <path> External raw-by-hash store DB path (optional) --raw-store-db <path> External raw-by-hash store DB path (optional)
--repo-bytes-db <path> External repo object bytes DB path (optional) --repo-bytes-db <path> External repo object bytes DB path (optional)
@@ -276,8 +259,7 @@ Options:
--rsync-timeout-secs <n> rsync I/O timeout seconds (default: 60) --rsync-timeout-secs <n> rsync I/O timeout seconds (default: 60)
--rsync-mirror-root <path> Persist rsync mirrors under this directory (default: disabled) --rsync-mirror-root <path> Persist rsync mirrors under this directory (default: disabled)
--rsync-scope <policy> rsync scope policy: host, publication-point, or module-root (default: module-root) --rsync-scope <policy> rsync scope policy: host, publication-point, or module-root (default: module-root)
--max-ca-depth <n> Maximum CA depth from a trust anchor (root = 0, default: {DEFAULT_MAX_CA_DEPTH}) --max-depth <n> Max CA instance depth (0 = root only)
--max-depth <n> Deprecated alias for --max-ca-depth
--max-instances <n> Max number of CA instances to process --max-instances <n> Max number of CA instances to process
--validation-time <rfc3339> Validation time in RFC3339 (default: now UTC) --validation-time <rfc3339> Validation time in RFC3339 (default: now UTC)
--analyze Write timing analysis JSON under target/live/analyze/<timestamp>/ --analyze Write timing analysis JSON under target/live/analyze/<timestamp>/
@@ -290,12 +272,6 @@ Options:
} }
pub fn parse_args(argv: &[String]) -> Result<CliArgs, String> { pub fn parse_args(argv: &[String]) -> Result<CliArgs, String> {
let mut verification_only = false;
let mut verification_source_run_dir: Option<PathBuf> = None;
let mut verification_source_state_root: Option<PathBuf> = None;
let mut verification_out_dir: Option<PathBuf> = None;
let mut keep_verification_scratch = false;
let mut validation_contract_out_path: Option<PathBuf> = None;
let mut tal_urls: Vec<String> = Vec::new(); let mut tal_urls: Vec<String> = Vec::new();
let mut tal_paths: Vec<PathBuf> = Vec::new(); let mut tal_paths: Vec<PathBuf> = Vec::new();
let mut ta_paths: Vec<PathBuf> = Vec::new(); let mut ta_paths: Vec<PathBuf> = Vec::new();
@@ -343,8 +319,7 @@ pub fn parse_args(argv: &[String]) -> Result<CliArgs, String> {
let mut rsync_timeout_secs: u64 = 30; let mut rsync_timeout_secs: u64 = 30;
let mut rsync_mirror_root: Option<PathBuf> = None; let mut rsync_mirror_root: Option<PathBuf> = None;
let mut rsync_scope_policy = RsyncScopePolicy::default(); let mut rsync_scope_policy = RsyncScopePolicy::default();
let mut max_ca_depth: Option<usize> = None; let mut max_depth: Option<usize> = None;
let mut max_ca_depth_option: Option<&'static str> = None;
let mut max_instances: Option<usize> = None; let mut max_instances: Option<usize> = None;
let mut validation_time: Option<time::OffsetDateTime> = None; let mut validation_time: Option<time::OffsetDateTime> = None;
let mut analyze: bool = false; let mut analyze: bool = false;
@@ -356,39 +331,6 @@ pub fn parse_args(argv: &[String]) -> Result<CliArgs, String> {
let arg = argv[i].as_str(); let arg = argv[i].as_str();
match arg { match arg {
"--help" | "-h" => return Err(usage()), "--help" | "-h" => return Err(usage()),
"--verification-only" => {
verification_only = true;
}
"--verification-source-run" => {
i += 1;
verification_source_run_dir = Some(PathBuf::from(
argv.get(i)
.ok_or("--verification-source-run requires a value")?,
));
}
"--verification-source-state" => {
i += 1;
verification_source_state_root = Some(PathBuf::from(
argv.get(i)
.ok_or("--verification-source-state requires a value")?,
));
}
"--verification-out" => {
i += 1;
verification_out_dir = Some(PathBuf::from(
argv.get(i).ok_or("--verification-out requires a value")?,
));
}
"--keep-verification-scratch" => {
keep_verification_scratch = true;
}
"--validation-contract-out" => {
i += 1;
validation_contract_out_path = Some(PathBuf::from(
argv.get(i)
.ok_or("--validation-contract-out requires a value")?,
));
}
"--tal-url" => { "--tal-url" => {
i += 1; i += 1;
let v = argv.get(i).ok_or("--tal-url requires a value")?; let v = argv.get(i).ok_or("--tal-url requires a value")?;
@@ -719,26 +661,13 @@ pub fn parse_args(argv: &[String]) -> Result<CliArgs, String> {
let v = argv.get(i).ok_or("--rsync-scope requires a value")?; let v = argv.get(i).ok_or("--rsync-scope requires a value")?;
rsync_scope_policy = RsyncScopePolicy::parse_cli_value(v)?; rsync_scope_policy = RsyncScopePolicy::parse_cli_value(v)?;
} }
"--max-ca-depth" | "--max-depth" => { "--max-depth" => {
i += 1; i += 1;
let option = arg; let v = argv.get(i).ok_or("--max-depth requires a value")?;
let v = argv max_depth = Some(
.get(i)
.ok_or_else(|| format!("{option} requires a value"))?;
if let Some(previous_option) = max_ca_depth_option {
return Err(format!(
"{option} cannot be combined with {previous_option}; use only --max-ca-depth"
));
}
max_ca_depth = Some(
v.parse::<usize>() v.parse::<usize>()
.map_err(|_| format!("invalid {option}: {v}"))?, .map_err(|_| format!("invalid --max-depth: {v}"))?,
); );
max_ca_depth_option = Some(match option {
"--max-ca-depth" => "--max-ca-depth",
"--max-depth" => "--max-depth",
_ => unreachable!("matched CA depth option"),
});
} }
"--max-instances" => { "--max-instances" => {
i += 1; i += 1;
@@ -773,91 +702,10 @@ pub fn parse_args(argv: &[String]) -> Result<CliArgs, String> {
i += 1; i += 1;
} }
if verification_only {
if verification_source_run_dir.is_none() {
return Err(format!(
"--verification-only requires --verification-source-run\n\n{}",
usage()
));
}
if verification_source_state_root.is_none() {
return Err(format!(
"--verification-only requires --verification-source-state\n\n{}",
usage()
));
}
if verification_out_dir.is_none() {
return Err(format!(
"--verification-only requires --verification-out\n\n{}",
usage()
));
}
if db_path.is_some()
|| raw_store_db.is_some()
|| repo_bytes_db.is_some()
|| !tal_urls.is_empty()
|| !tal_paths.is_empty()
|| !ta_paths.is_empty()
|| policy_path.is_some()
|| strict_policy.is_some()
|| resource_validation_mode.is_some()
|| validation_time.is_some()
|| max_ca_depth_option.is_some()
|| max_instances.is_some()
|| enable_roa_validation_cache
|| enable_child_certificate_validation_cache
|| publication_point_cache_observe_only
|| enable_publication_point_validation_cache
|| enable_transport_request_prefetch
|| rsync_local_dir.is_some()
|| rsync_command.is_some()
|| !http_root_cert_paths.is_empty()
|| payload_replay_archive.is_some()
|| payload_replay_locks.is_some()
|| payload_base_archive.is_some()
|| payload_base_locks.is_some()
|| payload_delta_archive.is_some()
|| payload_delta_locks.is_some()
|| validation_contract_out_path.is_some()
{
return Err(format!(
"verification-only derives DB, TAL, validation time, policy and cache settings from the source run; conflicting normal-run options are not allowed\n\n{}",
usage()
));
}
if report_json_path.is_some()
|| ccr_out_path.is_some()
|| vrps_csv_out_path.is_some()
|| vaps_csv_out_path.is_some()
|| cir_enabled
|| cir_out_path.is_some()
{
return Err(format!(
"verification-only output paths are derived from --verification-out\n\n{}",
usage()
));
}
db_path = Some(
verification_out_dir
.as_ref()
.expect("checked verification out")
.join("scratch/work-db"),
);
} else if verification_source_run_dir.is_some()
|| verification_source_state_root.is_some()
|| verification_out_dir.is_some()
|| keep_verification_scratch
{
return Err(format!(
"verification source/output options require --verification-only\n\n{}",
usage()
));
}
let db_path = db_path.ok_or_else(|| format!("--db is required\n\n{}", usage()))?; let db_path = db_path.ok_or_else(|| format!("--db is required\n\n{}", usage()))?;
let tal_mode_count = (!tal_urls.is_empty()) as u8 + (!tal_paths.is_empty()) as u8; let tal_mode_count = (!tal_urls.is_empty()) as u8 + (!tal_paths.is_empty()) as u8;
if !verification_only && tal_mode_count != 1 { if tal_mode_count != 1 {
return Err(format!( return Err(format!(
"must specify either one-or-more --tal-url or one-or-more --tal-path/--ta-path pairs\n\n{}", "must specify either one-or-more --tal-url or one-or-more --tal-path/--ta-path pairs\n\n{}",
usage() usage()
@@ -911,13 +759,13 @@ pub fn parse_args(argv: &[String]) -> Result<CliArgs, String> {
usage() usage()
)); ));
} }
if !verification_only && !tal_urls.is_empty() && !ta_paths.is_empty() { if !tal_urls.is_empty() && !ta_paths.is_empty() {
return Err(format!( return Err(format!(
"--ta-path cannot be used with --tal-url mode\n\n{}", "--ta-path cannot be used with --tal-url mode\n\n{}",
usage() usage()
)); ));
} }
if !verification_only && !tal_paths.is_empty() { if !tal_paths.is_empty() {
if !ta_paths.is_empty() { if !ta_paths.is_empty() {
if ta_paths.len() != tal_paths.len() { if ta_paths.len() != tal_paths.len() {
return Err(format!( return Err(format!(
@@ -1092,12 +940,6 @@ pub fn parse_args(argv: &[String]) -> Result<CliArgs, String> {
} }
Ok(CliArgs { Ok(CliArgs {
verification_only,
verification_source_run_dir,
verification_source_state_root,
verification_out_dir,
keep_verification_scratch,
validation_contract_out_path,
tal_urls, tal_urls,
tal_paths, tal_paths,
ta_paths, ta_paths,
@@ -1147,7 +989,7 @@ pub fn parse_args(argv: &[String]) -> Result<CliArgs, String> {
rsync_timeout_secs, rsync_timeout_secs,
rsync_mirror_root, rsync_mirror_root,
rsync_scope_policy, rsync_scope_policy,
max_ca_depth: max_ca_depth.unwrap_or(DEFAULT_MAX_CA_DEPTH), max_depth,
max_instances, max_instances,
validation_time, validation_time,
analyze, analyze,
@@ -2012,7 +1854,7 @@ where
H: crate::sync::rrdp::Fetcher + Clone + 'static, H: crate::sync::rrdp::Fetcher + Clone + 'static,
R: crate::fetch::rsync::RsyncFetcher + Clone + 'static, R: crate::fetch::rsync::RsyncFetcher + Clone + 'static,
{ {
if args.verification_only || args.tal_inputs.len() > 1 { if args.tal_inputs.len() > 1 {
return if let Some(t) = timing { return if let Some(t) = timing {
run_tree_from_multiple_tals_parallel_phase2_audit_with_timing( run_tree_from_multiple_tals_parallel_phase2_audit_with_timing(
store, store,
@@ -2153,54 +1995,9 @@ where
} }
pub fn run(argv: &[String]) -> Result<(), String> { pub fn run(argv: &[String]) -> Result<(), String> {
let mut args = parse_args(argv)?; let args = parse_args(argv)?;
let mut prepared_verification = if args.verification_only {
let prepared = crate::verification_only::prepare_verification(
args.verification_source_run_dir
.as_deref()
.expect("validated by parse_args"),
args.verification_source_state_root
.as_deref()
.expect("validated by parse_args"),
args.verification_out_dir
.as_deref()
.expect("validated by parse_args"),
)?;
args.db_path = prepared.artifacts.scratch_work_db.clone();
args.repo_bytes_db = Some(prepared.artifacts.source_repo_bytes_db.clone());
args.report_json_path = Some(prepared.artifacts.report_json.clone());
args.report_json_compact = true;
args.skip_report_build = false;
args.skip_vcir_persist = true;
args.enable_roa_validation_cache = false;
args.enable_child_certificate_validation_cache = false;
args.publication_point_cache_observe_only = false;
args.enable_publication_point_validation_cache = false;
args.enable_transport_request_prefetch = false;
args.ccr_out_path = Some(prepared.artifacts.result_ccr.clone());
args.vrps_csv_out_path = Some(prepared.artifacts.vrps_csv.clone());
args.vaps_csv_out_path = Some(prepared.artifacts.vaps_csv.clone());
args.compare_view_trust_anchor = Some("verification-only".to_string());
args.cir_enabled = true;
args.cir_out_path = Some(prepared.artifacts.result_cir.clone());
args.cir_tal_uris = crate::verification_only::cir_tal_uris(&prepared.cir);
args.tal_inputs = crate::verification_only::tal_inputs_from_cir(&prepared.cir);
args.validation_time = Some(prepared.cir.validation_time);
args.max_ca_depth = prepared.contract.max_ca_depth;
args.max_instances = prepared.contract.max_instances;
args.disable_rrdp = true;
args.rsync_local_dir = None;
Some(prepared)
} else {
None
};
let mut policy = if let Some(prepared) = prepared_verification.as_ref() { let mut policy = read_policy(args.policy_path.as_deref())?;
prepared.contract.policy.clone()
} else {
read_policy(args.policy_path.as_deref())?
};
if !args.verification_only {
if let Some(strict_policy) = args.strict_policy { if let Some(strict_policy) = args.strict_policy {
policy.strict = strict_policy; policy.strict = strict_policy;
} }
@@ -2210,31 +2007,9 @@ pub fn run(argv: &[String]) -> Result<(), String> {
if args.disable_rrdp { if args.disable_rrdp {
policy.sync_preference = crate::policy::SyncPreference::RsyncOnly; policy.sync_preference = crate::policy::SyncPreference::RsyncOnly;
} }
} else {
policy.sync_preference = crate::policy::SyncPreference::RsyncOnly;
policy.ca_failed_fetch_policy = crate::policy::CaFailedFetchPolicy::StopAllOutput;
}
let validation_time = args let validation_time = args
.validation_time .validation_time
.unwrap_or_else(time::OffsetDateTime::now_utc); .unwrap_or_else(time::OffsetDateTime::now_utc);
let validation_time =
time::OffsetDateTime::from_unix_timestamp(validation_time.unix_timestamp())
.map_err(|error| format!("normalize validation time failed: {error}"))?;
if let Some(path) = args.validation_contract_out_path.as_deref() {
let contract = crate::verification_only::ValidationContract::for_current_binary(
validation_time,
policy.clone(),
args.max_ca_depth,
args.max_instances,
crate::verification_only::ValidationCacheContract {
publication_point: args.enable_publication_point_validation_cache,
roa: args.enable_roa_validation_cache,
child_certificate: args.enable_child_certificate_validation_cache,
transport_prefetch: args.enable_transport_request_prefetch,
},
)?;
crate::verification_only::write_validation_contract(path, &contract)?;
}
let http_root_certificates_pem = args let http_root_certificates_pem = args
.http_root_cert_paths .http_root_cert_paths
.iter() .iter()
@@ -2244,12 +2019,7 @@ pub fn run(argv: &[String]) -> Result<(), String> {
}) })
.collect::<Result<Vec<_>, _>>()?; .collect::<Result<Vec<_>, _>>()?;
let store = if let Some(prepared) = prepared_verification.as_mut() { let store = if args.raw_store_db.is_some() || args.repo_bytes_db.is_some() {
prepared
.store
.take()
.expect("prepared verification store must be available")
} else if args.raw_store_db.is_some() || args.repo_bytes_db.is_some() {
Arc::new( Arc::new(
RocksStore::open_with_external_stores( RocksStore::open_with_external_stores(
&args.db_path, &args.db_path,
@@ -2262,7 +2032,7 @@ pub fn run(argv: &[String]) -> Result<(), String> {
Arc::new(RocksStore::open(&args.db_path).map_err(|e| e.to_string())?) Arc::new(RocksStore::open(&args.db_path).map_err(|e| e.to_string())?)
}; };
let config = TreeRunConfig { let config = TreeRunConfig {
max_depth: Some(args.max_ca_depth), max_depth: args.max_depth,
max_instances: args.max_instances, max_instances: args.max_instances,
compact_audit: args.skip_report_build compact_audit: args.skip_report_build
&& args.report_json_path.is_none() && args.report_json_path.is_none()
@@ -2352,23 +2122,7 @@ pub fn run(argv: &[String]) -> Result<(), String> {
); );
let validation_started = std::time::Instant::now(); let validation_started = std::time::Instant::now();
let collect_current_repo_objects = false; let collect_current_repo_objects = false;
let out = if args.verification_only { let out = if delta_replay_mode {
let http = crate::verification_only::NetworkDisabledHttpFetcher;
let rsync = crate::fetch::current_repository::CurrentRepositoryViewRsyncFetcher::new(
Arc::clone(&store),
);
run_online_validation_with_fetchers(
Arc::clone(&store),
&policy,
&args,
&http,
&rsync,
validation_time,
&config,
collect_current_repo_objects,
timing.as_ref().map(|(_, t)| t),
)?
} else if delta_replay_mode {
let tal_path = args let tal_path = args
.tal_path .tal_path
.as_ref() .as_ref()
@@ -2897,62 +2651,6 @@ pub fn run(argv: &[String]) -> Result<(), String> {
} }
print_summary_from_shared(validation_time, &shared); print_summary_from_shared(validation_time, &shared);
if let Some(prepared) = prepared_verification.as_ref() {
std::fs::create_dir_all(&prepared.artifacts.compare_dir).map_err(|error| {
format!(
"create verification compare directory failed: {}: {error}",
prepared.artifacts.compare_dir.display()
)
})?;
let comparison = crate::verification_only::compare_ccr_files(
&prepared.artifacts.source_ccr,
&prepared.artifacts.result_ccr,
)?;
crate::verification_only::write_ccr_comparison(
&prepared.artifacts.compare_dir.join("ccr-state-digest.json"),
Some(&prepared.artifacts.compare_dir.join("ccr-state-digest.md")),
&comparison,
)?;
let scratch_retained = !comparison.state_digest_match || args.keep_verification_scratch;
let source_run_id = prepared
.artifacts
.source_run_dir
.file_name()
.and_then(|value| value.to_str())
.unwrap_or("unknown")
.to_string();
crate::verification_only::write_verification_meta(
&prepared.artifacts.verification_meta,
&crate::verification_only::VerificationRunMeta {
schema_version: 1,
status: if comparison.state_digest_match {
"success".to_string()
} else {
"mismatch".to_string()
},
source_run_id,
validation_time: crate::verification_only::format_validation_time(validation_time)?,
wall_ms: total_started.elapsed().as_millis() as u64,
repository_objects_checked: prepared.blob_summary.current_objects,
repository_bytes_checked: prepared.blob_summary.bytes_verified,
state_digest_match: comparison.state_digest_match,
scratch_retained,
},
)?;
if !comparison.state_digest_match {
return Err(format!(
"verification-only CCR state digest mismatch: {}",
comparison.mismatched_states.join(",")
));
}
if !args.keep_verification_scratch {
drop(shared);
drop(store);
std::fs::remove_dir_all(prepared.artifacts.output_dir.join("scratch")).map_err(
|error| format!("remove verification scratch directory failed: {error}"),
)?;
}
}
Ok(()) Ok(())
} }

View File

@@ -1,6 +1,4 @@
use super::*; use super::*;
use crate::data_model::oid::OID_AD_SIGNED_OBJECT;
use crate::data_model::rc::AccessDescription;
use crate::memory_telemetry::{ use crate::memory_telemetry::{
MemoryTelemetryCheckpoint, MemoryTelemetrySummary, ProcessMemorySnapshot, MemoryTelemetryCheckpoint, MemoryTelemetrySummary, ProcessMemorySnapshot,
}; };
@@ -23,8 +21,6 @@ fn parse_help_returns_usage() {
assert!(err.contains("--memory-trim-after-validation"), "{err}"); assert!(err.contains("--memory-trim-after-validation"), "{err}");
assert!(err.contains("--enable-roa-validation-cache"), "{err}"); assert!(err.contains("--enable-roa-validation-cache"), "{err}");
assert!(err.contains("--resource-validation-mode"), "{err}"); assert!(err.contains("--resource-validation-mode"), "{err}");
assert!(err.contains("--max-ca-depth"), "{err}");
assert!(err.contains("default: 32"), "{err}");
assert!( assert!(
err.contains("--publication-point-cache-observe-only"), err.contains("--publication-point-cache-observe-only"),
"{err}" "{err}"
@@ -51,63 +47,6 @@ fn parse_rejects_unknown_argument() {
assert!(err.contains("unknown argument"), "{err}"); assert!(err.contains("unknown argument"), "{err}");
} }
#[test]
fn parse_accepts_verification_only_without_manual_validation_time() {
let argv = vec![
"rpki".to_string(),
"--verification-only".to_string(),
"--verification-source-run".to_string(),
"/run/run_0002".to_string(),
"--verification-source-state".to_string(),
"/run/state".to_string(),
"--verification-out".to_string(),
"/verification/check_0002".to_string(),
];
let args = parse_args(&argv).expect("parse verification-only args");
assert!(args.verification_only);
assert!(args.validation_time.is_none());
assert_eq!(
args.db_path,
PathBuf::from("/verification/check_0002/scratch/work-db")
);
}
#[test]
fn parse_rejects_manual_validation_time_in_verification_only() {
let argv = vec![
"rpki".to_string(),
"--verification-only".to_string(),
"--verification-source-run".to_string(),
"/run/run_0002".to_string(),
"--verification-source-state".to_string(),
"/run/state".to_string(),
"--verification-out".to_string(),
"/verification/check_0002".to_string(),
"--validation-time".to_string(),
"2026-07-16T00:00:00Z".to_string(),
];
let error = parse_args(&argv).expect_err("manual time must be rejected");
assert!(error.contains("validation time"), "{error}");
}
#[test]
fn parse_accepts_normal_validation_contract_output() {
let argv = vec![
"rpki".to_string(),
"--db".to_string(),
"db".to_string(),
"--tal-url".to_string(),
"https://example.test/root.tal".to_string(),
"--validation-contract-out".to_string(),
"out/validation-contract.json".to_string(),
];
let args = parse_args(&argv).expect("parse normal contract output");
assert_eq!(
args.validation_contract_out_path,
Some(PathBuf::from("out/validation-contract.json"))
);
}
#[test] #[test]
fn parse_rejects_both_tal_url_and_tal_path() { fn parse_rejects_both_tal_url_and_tal_path() {
let argv = vec![ let argv = vec![
@@ -141,51 +80,6 @@ fn parse_rejects_invalid_max_depth() {
assert!(err.contains("invalid --max-depth"), "{err}"); assert!(err.contains("invalid --max-depth"), "{err}");
} }
#[test]
fn parse_rejects_invalid_max_ca_depth() {
let argv = vec![
"rpki".to_string(),
"--db".to_string(),
"db".to_string(),
"--tal-url".to_string(),
"https://example.test/x.tal".to_string(),
"--max-ca-depth".to_string(),
"nope".to_string(),
];
let err = parse_args(&argv).unwrap_err();
assert!(err.contains("invalid --max-ca-depth"), "{err}");
}
#[test]
fn parse_rejects_both_ca_depth_options() {
let argv = vec![
"rpki".to_string(),
"--db".to_string(),
"db".to_string(),
"--tal-url".to_string(),
"https://example.test/x.tal".to_string(),
"--max-ca-depth".to_string(),
"32".to_string(),
"--max-depth".to_string(),
"12".to_string(),
];
let err = parse_args(&argv).unwrap_err();
assert!(err.contains("cannot be combined"), "{err}");
}
#[test]
fn parse_defaults_max_ca_depth_to_32() {
let argv = vec![
"rpki".to_string(),
"--db".to_string(),
"db".to_string(),
"--tal-url".to_string(),
"https://example.test/x.tal".to_string(),
];
let args = parse_args(&argv).expect("parse");
assert_eq!(args.max_ca_depth, 32);
}
#[test] #[test]
fn parse_accepts_ccr_out_path() { fn parse_accepts_ccr_out_path() {
let argv = vec![ let argv = vec![
@@ -1209,7 +1103,7 @@ fn parse_accepts_offline_mode_requires_ta() {
assert_eq!(args.ta_paths, vec![PathBuf::from("ta.cer")]); assert_eq!(args.ta_paths, vec![PathBuf::from("ta.cer")]);
assert_eq!(args.tal_path.as_deref(), Some(Path::new("a.tal"))); assert_eq!(args.tal_path.as_deref(), Some(Path::new("a.tal")));
assert_eq!(args.ta_path.as_deref(), Some(Path::new("ta.cer"))); assert_eq!(args.ta_path.as_deref(), Some(Path::new("ta.cer")));
assert_eq!(args.max_ca_depth, 0); assert_eq!(args.max_depth, Some(0));
} }
#[test] #[test]
@@ -1724,9 +1618,8 @@ fn sample_cli_ccr_accumulator() -> CcrAccumulator {
) )
.expect("discover root"); .expect("discover root");
let mut accumulator = CcrAccumulator::new(vec![discovery.trust_anchor.clone()]); let mut accumulator = CcrAccumulator::new(vec![discovery.trust_anchor.clone()]);
let manifest_uri = "rsync://example.test/repo/current.mft".to_string();
let projection = crate::storage::VcirCcrManifestProjection { let projection = crate::storage::VcirCcrManifestProjection {
manifest_rsync_uri: manifest_uri.clone(), manifest_rsync_uri: "rsync://example.test/repo/current.mft".to_string(),
manifest_sha256: vec![0x44; 32], manifest_sha256: vec![0x44; 32],
manifest_size: 2048, manifest_size: 2048,
manifest_ee_aki: vec![0x55; 20], manifest_ee_aki: vec![0x55; 20],
@@ -1734,13 +1627,10 @@ fn sample_cli_ccr_accumulator() -> CcrAccumulator {
manifest_this_update: crate::storage::PackTime::from_utc_offset_datetime( manifest_this_update: crate::storage::PackTime::from_utc_offset_datetime(
time::OffsetDateTime::now_utc(), time::OffsetDateTime::now_utc(),
), ),
manifest_sia_locations_der: vec![ manifest_sia_locations_der: vec![vec![
crate::ccr::manifest_location::encode_access_description_der(&AccessDescription { 0x30, 0x11, 0x06, 0x08, 0x2b, 0x06, 0x01, 0x05, 0x05, 0x07, 0x30, 0x05, 0x86, 0x05,
access_method_oid: OID_AD_SIGNED_OBJECT.to_string(), b'r', b's', b'y', b'n', b'c',
access_location: manifest_uri, ]],
})
.expect("encode signedObject"),
],
subordinate_skis: vec![vec![0x33; 20]], subordinate_skis: vec![vec![0x33; 20]],
}; };
accumulator accumulator

View File

@@ -1,178 +0,0 @@
use std::sync::Arc;
use crate::fetch::rsync::{
RsyncFetchError, RsyncFetchResult, RsyncFetcher, normalize_rsync_base_uri,
};
use crate::storage::{RepositoryViewState, RocksStore};
#[derive(Clone)]
pub struct CurrentRepositoryViewRsyncFetcher {
store: Arc<RocksStore>,
}
impl CurrentRepositoryViewRsyncFetcher {
pub fn new(store: Arc<RocksStore>) -> Self {
Self { store }
}
}
impl RsyncFetcher for CurrentRepositoryViewRsyncFetcher {
fn fetch_objects(&self, rsync_base_uri: &str) -> RsyncFetchResult<Vec<(String, Vec<u8>)>> {
let base = normalize_rsync_base_uri(rsync_base_uri);
let entries = self
.store
.list_repository_view_entries_with_prefix(&base)
.map_err(|error| {
RsyncFetchError::Fetch(format!(
"list frozen repository view failed for {base}: {error}"
))
})?;
let mut objects = Vec::with_capacity(entries.len());
for entry in entries {
if !matches!(
entry.state,
RepositoryViewState::Present | RepositoryViewState::Replaced
) {
continue;
}
let bytes = self
.store
.load_current_object_bytes_by_uri(&entry.rsync_uri)
.map_err(|error| {
RsyncFetchError::Fetch(format!(
"load frozen repository object failed for {}: {error}",
entry.rsync_uri
))
})?
.ok_or_else(|| {
RsyncFetchError::Fetch(format!(
"frozen repository object missing for {}",
entry.rsync_uri
))
})?;
objects.push((entry.rsync_uri, bytes));
}
objects.sort_by(|left, right| left.0.cmp(&right.0));
if objects.is_empty() {
return Err(RsyncFetchError::Fetch(format!(
"frozen repository view contains no current objects under {base}"
)));
}
Ok(objects)
}
fn fetch_object(&self, rsync_uri: &str) -> RsyncFetchResult<Vec<u8>> {
self.store
.load_current_object_bytes_by_uri(rsync_uri)
.map_err(|error| {
RsyncFetchError::Fetch(format!(
"load frozen repository object failed for {rsync_uri}: {error}"
))
})?
.ok_or_else(|| {
RsyncFetchError::Fetch(format!("frozen repository object not found: {rsync_uri}"))
})
}
fn dedup_key(&self, rsync_base_uri: &str) -> String {
normalize_rsync_base_uri(rsync_base_uri)
}
}
#[cfg(test)]
mod tests {
use super::*;
use crate::storage::{RepositoryViewEntry, RocksStore};
#[test]
fn current_repository_fetcher_returns_sorted_present_objects() {
let dir = tempfile::tempdir().expect("tempdir");
let store = Arc::new(RocksStore::open(dir.path()).expect("open store"));
let entries = [
("rsync://example.test/repo/b.roa", b"b".as_slice()),
("rsync://example.test/repo/a.mft", b"a".as_slice()),
];
for (uri, bytes) in entries {
let hash = hex::encode(crate::cir::sha256(bytes));
store
.put_blob_bytes_batch(&[(hash.clone(), bytes.to_vec())])
.expect("put blob");
store
.put_repository_view_entry(&RepositoryViewEntry {
rsync_uri: uri.to_string(),
repository_source: Some("fixture".to_string()),
object_type: Some("roa".to_string()),
state: RepositoryViewState::Present,
current_hash: Some(hash),
})
.expect("put view");
}
let fetcher = CurrentRepositoryViewRsyncFetcher::new(store);
let objects = fetcher
.fetch_objects("rsync://example.test/repo/")
.expect("fetch objects");
assert_eq!(
objects
.iter()
.map(|(uri, _)| uri.as_str())
.collect::<Vec<_>>(),
vec![
"rsync://example.test/repo/a.mft",
"rsync://example.test/repo/b.roa"
]
);
assert_eq!(
fetcher
.fetch_object("rsync://example.test/repo/a.mft")
.expect("fetch one"),
b"a"
);
assert!(
fetcher
.fetch_object("rsync://example.test/repo/missing.roa")
.is_err()
);
assert_eq!(
fetcher.dedup_key("rsync://example.test/repo"),
"rsync://example.test/repo/"
);
assert!(
fetcher
.fetch_objects("rsync://empty.example/repo/")
.unwrap_err()
.to_string()
.contains("no current objects")
);
}
#[test]
fn current_repository_fetcher_ignores_withdrawn_and_reports_missing_blob() {
let dir = tempfile::tempdir().expect("tempdir");
let store = Arc::new(RocksStore::open(dir.path()).expect("open store"));
store
.put_repository_view_entry(&RepositoryViewEntry {
rsync_uri: "rsync://example.test/repo/withdrawn.roa".to_string(),
repository_source: Some("fixture".to_string()),
object_type: Some("roa".to_string()),
state: RepositoryViewState::Withdrawn,
current_hash: None,
})
.expect("put withdrawn view");
let missing_hash = "ab".repeat(32);
store
.put_repository_view_entry(&RepositoryViewEntry {
rsync_uri: "rsync://example.test/repo/missing.roa".to_string(),
repository_source: Some("fixture".to_string()),
object_type: Some("roa".to_string()),
state: RepositoryViewState::Present,
current_hash: Some(missing_hash),
})
.expect("put missing view");
let fetcher = CurrentRepositoryViewRsyncFetcher::new(store);
let error = fetcher
.fetch_objects("rsync://example.test/repo/")
.unwrap_err()
.to_string();
assert!(error.contains("blob bytes missing"));
}
}

View File

@@ -1,4 +1,3 @@
pub mod current_repository;
pub mod http; pub mod http;
pub mod rsync; pub mod rsync;
pub mod rsync_system; pub mod rsync_system;

View File

@@ -42,5 +42,3 @@ pub mod sync;
pub mod tools; pub mod tools;
#[cfg(feature = "full")] #[cfg(feature = "full")]
pub mod validation; pub mod validation;
#[cfg(feature = "full")]
pub mod verification_only;

View File

@@ -1,4 +1,3 @@
pub mod artifact_manifest; pub mod artifact_manifest;
pub mod object_resolver; pub mod object_resolver;
pub mod report_stream; pub mod report_stream;
pub mod vrp;

View File

@@ -9,7 +9,6 @@ use serde_json::value::RawValue;
use serde_json::{Value, json}; use serde_json::{Value, json};
use sha2::{Digest, Sha256}; use sha2::{Digest, Sha256};
use crate::query::vrp::VrpRecord;
use crate::query_db::{ use crate::query_db::{
ObjectInstanceRecord, PublicationPointRecord, QUERY_DB_SCHEMA_VERSION, QueryDbError, ObjectInstanceRecord, PublicationPointRecord, QUERY_DB_SCHEMA_VERSION, QueryDbError,
QueryDbResult, QueryPage, RepositoryRecord, StatsRecord, QueryDbResult, QueryPage, RepositoryRecord, StatsRecord,
@@ -24,7 +23,6 @@ pub struct ReportSummary {
pub query_audit: Option<Value>, pub query_audit: Option<Value>,
pub download_stats: Option<Value>, pub download_stats: Option<Value>,
pub vrps_count: u64, pub vrps_count: u64,
pub vrps: Vec<VrpRecord>,
pub aspas_count: u64, pub aspas_count: u64,
pub warnings_count: u64, pub warnings_count: u64,
pub objects_count: u64, pub objects_count: u64,
@@ -52,68 +50,6 @@ pub enum ObjectLookup {
Sha256(String), Sha256(String),
} }
/// Optional predicates applied while streaming report objects. `reason` and
/// `query` must be pre-lowercased by the caller; they are matched as
/// case-insensitive substrings. The other fields are exact matches.
#[derive(Clone, Debug, Default, PartialEq, Eq)]
pub struct ObjectFilter {
pub object_type: Option<String>,
pub result: Option<String>,
pub rejected: Option<bool>,
pub reason: Option<String>,
pub query: Option<String>,
pub repo_id: Option<String>,
pub pp_id: Option<String>,
pub issues_only: bool,
}
impl ObjectFilter {
pub fn matches(&self, object: &ObjectInstanceRecord) -> bool {
if self.issues_only && !(object.rejected || object.result == "error") {
return false;
}
if let Some(rejected) = self.rejected
&& object.rejected != rejected
{
return false;
}
if let Some(result) = self.result.as_deref()
&& !object.result.eq_ignore_ascii_case(result)
{
return false;
}
if let Some(object_type) = self.object_type.as_deref()
&& !object.object_type.eq_ignore_ascii_case(object_type)
{
return false;
}
if let Some(repo_id) = self.repo_id.as_deref()
&& object.repo_id != repo_id
{
return false;
}
if let Some(pp_id) = self.pp_id.as_deref()
&& object.pp_id != pp_id
{
return false;
}
if let Some(reason) = self.reason.as_deref() {
let Some(reject_reason) = object.reject_reason.as_deref() else {
return false;
};
if !reject_reason.to_lowercase().contains(reason) {
return false;
}
}
if let Some(query) = self.query.as_deref()
&& !object.uri.to_lowercase().contains(query)
{
return false;
}
true
}
}
#[derive(Clone, Debug, PartialEq, Eq, Serialize)] #[derive(Clone, Debug, PartialEq, Eq, Serialize)]
#[serde(rename_all = "camelCase")] #[serde(rename_all = "camelCase")]
pub struct ObjectResolutionMeta { pub struct ObjectResolutionMeta {
@@ -145,37 +81,13 @@ pub fn list_report_objects(
scope: ObjectScope, scope: ObjectScope,
limit: usize, limit: usize,
cursor: Option<&str>, cursor: Option<&str>,
) -> QueryDbResult<QueryPage<ObjectInstanceRecord>> {
list_report_objects_filtered(
report_path,
run_id,
scope,
&ObjectFilter::default(),
limit,
cursor,
)
}
pub fn list_report_objects_filtered(
report_path: &Path,
run_id: &str,
scope: ObjectScope,
filter: &ObjectFilter,
limit: usize,
cursor: Option<&str>,
) -> QueryDbResult<QueryPage<ObjectInstanceRecord>> { ) -> QueryDbResult<QueryPage<ObjectInstanceRecord>> {
let (start_pp_index, start_object_index) = parse_object_cursor(cursor)?; let (start_pp_index, start_object_index) = parse_object_cursor(cursor)?;
let file = File::open(report_path)?; let file = File::open(report_path)?;
let reader = BufReader::new(file); let reader = BufReader::new(file);
let mut deserializer = serde_json::Deserializer::from_reader(reader); let mut deserializer = serde_json::Deserializer::from_reader(reader);
let mut state = ObjectScanState::new_list( let mut state =
run_id, ObjectScanState::new_list(run_id, scope, limit, start_pp_index, start_object_index);
scope,
filter.clone(),
limit,
start_pp_index,
start_object_index,
);
ObjectScanSeed { state: &mut state } ObjectScanSeed { state: &mut state }
.deserialize(&mut deserializer) .deserialize(&mut deserializer)
.map_err(QueryDbError::from)?; .map_err(QueryDbError::from)?;
@@ -289,13 +201,7 @@ impl<'de> Visitor<'de> for ReportSummaryVisitor<'_> {
out.repos = pps.repos; out.repos = pps.repos;
out.stats = pps.stats; out.stats = pps.stats;
} }
"vrps" => { "vrps" => out.vrps_count = map.next_value::<CountSeq>()?.0,
let vrps = map.next_value_seed(VrpSequenceSeed {
run_id: self.run_id,
})?;
out.vrps_count = vrps.count;
out.vrps = vrps.records;
}
"aspas" => out.aspas_count = map.next_value::<CountSeq>()?.0, "aspas" => out.aspas_count = map.next_value::<CountSeq>()?.0,
"download_stats" => out.download_stats = Some(map.next_value()?), "download_stats" => out.download_stats = Some(map.next_value()?),
"queryAudit" => out.query_audit = Some(map.next_value()?), "queryAudit" => out.query_audit = Some(map.next_value()?),
@@ -333,62 +239,6 @@ struct PublicationPointsSummary {
objects_count: u64, objects_count: u64,
} }
#[derive(Deserialize)]
struct RawVrpRecord {
asn: u32,
prefix: String,
max_length: u8,
}
struct VrpSequenceSeed<'a> {
run_id: &'a str,
}
struct VrpSequence {
count: u64,
records: Vec<VrpRecord>,
}
impl<'de> DeserializeSeed<'de> for VrpSequenceSeed<'_> {
type Value = VrpSequence;
fn deserialize<D>(self, deserializer: D) -> Result<Self::Value, D::Error>
where
D: Deserializer<'de>,
{
deserializer.deserialize_seq(VrpSequenceVisitor {
run_id: self.run_id,
})
}
}
struct VrpSequenceVisitor<'a> {
run_id: &'a str,
}
impl<'de> Visitor<'de> for VrpSequenceVisitor<'_> {
type Value = VrpSequence;
fn expecting(&self, formatter: &mut std::fmt::Formatter) -> std::fmt::Result {
formatter.write_str("VRP array")
}
fn visit_seq<A>(self, mut seq: A) -> Result<Self::Value, A::Error>
where
A: SeqAccess<'de>,
{
let mut count = 0u64;
let mut records = Vec::new();
while let Some(raw) = seq.next_element::<RawVrpRecord>()? {
let record = VrpRecord::from_report(self.run_id, raw.asn, &raw.prefix, raw.max_length)
.map_err(de::Error::custom)?;
count += 1;
records.push(record);
}
Ok(VrpSequence { count, records })
}
}
struct PublicationPointsSummaryVisitor<'a> { struct PublicationPointsSummaryVisitor<'a> {
run_id: &'a str, run_id: &'a str,
} }
@@ -680,7 +530,6 @@ struct ObjectScanState<'a> {
run_id: &'a str, run_id: &'a str,
scope: ObjectScope, scope: ObjectScope,
mode: ObjectScanMode, mode: ObjectScanMode,
filter: ObjectFilter,
limit: usize, limit: usize,
start_pp_index: u64, start_pp_index: u64,
start_object_index: u64, start_object_index: u64,
@@ -695,7 +544,6 @@ impl<'a> ObjectScanState<'a> {
fn new_list( fn new_list(
run_id: &'a str, run_id: &'a str,
scope: ObjectScope, scope: ObjectScope,
filter: ObjectFilter,
limit: usize, limit: usize,
start_pp_index: u64, start_pp_index: u64,
start_object_index: u64, start_object_index: u64,
@@ -704,7 +552,6 @@ impl<'a> ObjectScanState<'a> {
run_id, run_id,
scope, scope,
mode: ObjectScanMode::List, mode: ObjectScanMode::List,
filter,
limit, limit,
start_pp_index, start_pp_index,
start_object_index, start_object_index,
@@ -721,7 +568,6 @@ impl<'a> ObjectScanState<'a> {
run_id, run_id,
scope, scope,
mode: ObjectScanMode::Lookup(lookup), mode: ObjectScanMode::Lookup(lookup),
filter: ObjectFilter::default(),
limit: 1, limit: 1,
start_pp_index: 0, start_pp_index: 0,
start_object_index: 0, start_object_index: 0,
@@ -836,7 +682,7 @@ fn process_raw_publication_point(
let after_cursor = pp_index > state.start_pp_index let after_cursor = pp_index > state.start_pp_index
|| (pp_index == state.start_pp_index || (pp_index == state.start_pp_index
&& object_index >= state.start_object_index); && object_index >= state.start_object_index);
if after_cursor && state.data.len() < state.limit && state.filter.matches(&record) { if after_cursor && state.data.len() < state.limit {
state.data.push(record); state.data.push(record);
if state.data.len() == state.limit { if state.data.len() == state.limit {
state.next_cursor = Some(object_cursor(pp_index, object_index + 1)); state.next_cursor = Some(object_cursor(pp_index, object_index + 1));
@@ -1311,10 +1157,7 @@ mod tests {
{"rsync_uri":"rsync://repo.example/rpki/a.roa","sha256_hex":"22","kind":"roa","result":"error","detail":"bad roa"} {"rsync_uri":"rsync://repo.example/rpki/a.roa","sha256_hex":"22","kind":"roa","result":"error","detail":"bad roa"}
] ]
}], }],
"vrps":[ "vrps":[{},{}],
{"asn":64496,"prefix":"192.0.2.0/24","max_length":24},
{"asn":64497,"prefix":"2001:db8::/32","max_length":48}
],
"aspas":[{}], "aspas":[{}],
"download_stats":{"eventsTotal":0}, "download_stats":{"eventsTotal":0},
"queryAudit":{"status":"complete","eventsPath":"validation-events.jsonl"} "queryAudit":{"status":"complete","eventsPath":"validation-events.jsonl"}

View File

@@ -1,275 +0,0 @@
use std::net::{IpAddr, Ipv4Addr, Ipv6Addr};
use serde::{Deserialize, Serialize};
#[derive(Clone, Debug, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "camelCase")]
pub struct VrpRecord {
pub run_id: String,
pub asn: u32,
pub prefix: String,
pub max_length: u8,
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub struct IpPrefix {
addr: IpAddr,
prefix_length: u8,
}
#[derive(Clone, Debug, PartialEq, Eq)]
pub enum VrpLookup {
Ip(IpAddr),
Prefix(IpPrefix),
}
impl VrpRecord {
pub fn from_report(
run_id: impl Into<String>,
asn: u32,
prefix: &str,
max_length: u8,
) -> Result<Self, String> {
let network = IpPrefix::parse(prefix)?;
if max_length < network.prefix_length {
return Err(format!(
"VRP max_length {max_length} is shorter than prefix length {}",
network.prefix_length
));
}
if max_length > network.max_prefix_length() {
return Err(format!(
"VRP max_length {max_length} exceeds address family limit {}",
network.max_prefix_length()
));
}
Ok(Self {
run_id: run_id.into(),
asn,
prefix: network.to_cidr(),
max_length,
})
}
pub fn prefix_data(&self) -> Result<IpPrefix, String> {
IpPrefix::parse(&self.prefix)
}
pub fn index_key(&self) -> Result<String, String> {
let prefix = self.prefix_data()?;
Ok(format!(
"vrp/{}/{}/{:03}/{}/{:010}/{:03}",
self.run_id,
prefix.family(),
prefix.prefix_length,
prefix.network_hex(),
self.asn,
self.max_length
))
}
}
impl IpPrefix {
pub fn parse(value: &str) -> Result<Self, String> {
let (addr, raw_length) = value
.rsplit_once('/')
.ok_or_else(|| format!("invalid CIDR prefix: {value}"))?;
let ip = addr
.parse::<IpAddr>()
.map_err(|_| format!("invalid IP address in prefix: {value}"))?;
let prefix_length = raw_length
.parse::<u8>()
.map_err(|_| format!("invalid prefix length in: {value}"))?;
Self::new(ip, prefix_length)
}
pub fn new(addr: IpAddr, prefix_length: u8) -> Result<Self, String> {
if prefix_length > max_prefix_length(addr) {
return Err(format!(
"prefix length {prefix_length} exceeds address family limit {}",
max_prefix_length(addr)
));
}
Ok(Self {
addr: network_address(addr, prefix_length),
prefix_length,
})
}
pub fn prefix_length(&self) -> u8 {
self.prefix_length
}
pub fn max_prefix_length(&self) -> u8 {
max_prefix_length(self.addr)
}
pub fn family(&self) -> &'static str {
match self.addr {
IpAddr::V4(_) => "v4",
IpAddr::V6(_) => "v6",
}
}
pub fn network_hex(&self) -> String {
match self.addr {
IpAddr::V4(addr) => hex::encode(addr.octets()),
IpAddr::V6(addr) => hex::encode(addr.octets()),
}
}
pub fn to_cidr(&self) -> String {
format!("{}/{}", self.addr, self.prefix_length)
}
pub fn contains_ip(&self, ip: IpAddr) -> bool {
if std::mem::discriminant(&self.addr) != std::mem::discriminant(&ip) {
return false;
}
network_address(ip, self.prefix_length) == self.addr
}
pub fn contains_prefix(&self, other: &Self) -> bool {
self.family() == other.family()
&& self.prefix_length <= other.prefix_length
&& self.contains_ip(other.addr)
}
pub fn ancestor_prefixes(&self) -> Vec<Self> {
(0..=self.prefix_length)
.filter_map(|length| Self::new(self.addr, length).ok())
.collect()
}
}
impl VrpLookup {
pub fn parse_ip(value: &str) -> Result<Self, String> {
value
.parse::<IpAddr>()
.map(VrpLookup::Ip)
.map_err(|_| format!("invalid IP address: {value}"))
}
pub fn parse_prefix(value: &str) -> Result<Self, String> {
IpPrefix::parse(value).map(VrpLookup::Prefix)
}
pub fn candidate_prefixes(&self) -> Vec<IpPrefix> {
match self {
VrpLookup::Ip(ip) => (0..=max_prefix_length(*ip))
.filter_map(|length| IpPrefix::new(*ip, length).ok())
.collect(),
VrpLookup::Prefix(prefix) => prefix.ancestor_prefixes(),
}
}
pub fn matches(&self, record: &VrpRecord) -> Result<bool, String> {
let record_prefix = record.prefix_data()?;
Ok(match self {
VrpLookup::Ip(ip) => record_prefix.contains_ip(*ip),
VrpLookup::Prefix(prefix) => {
record_prefix.contains_prefix(prefix) && record.max_length >= prefix.prefix_length
}
})
}
}
pub fn vrp_key_prefix(run_id: &str, prefix: &IpPrefix) -> String {
format!(
"vrp/{}/{}/{:03}/{}/",
run_id,
prefix.family(),
prefix.prefix_length,
prefix.network_hex()
)
}
pub fn parse_cursor(cursor: Option<&str>) -> Result<usize, String> {
let Some(cursor) = cursor else {
return Ok(0);
};
let offset = cursor
.strip_prefix("vrp:")
.ok_or_else(|| format!("invalid VRP cursor: {cursor}"))?
.parse::<usize>()
.map_err(|_| format!("invalid VRP cursor: {cursor}"))?;
Ok(offset)
}
pub fn next_cursor(offset: usize, limit: usize, total: usize) -> Option<String> {
let next = offset.saturating_add(limit);
(next < total).then(|| format!("vrp:{next}"))
}
fn max_prefix_length(addr: IpAddr) -> u8 {
match addr {
IpAddr::V4(_) => 32,
IpAddr::V6(_) => 128,
}
}
fn network_address(addr: IpAddr, prefix_length: u8) -> IpAddr {
match addr {
IpAddr::V4(addr) => {
let value = u32::from(addr);
let mask = if prefix_length == 0 {
0
} else {
u32::MAX << (32 - prefix_length)
};
IpAddr::V4(Ipv4Addr::from(value & mask))
}
IpAddr::V6(addr) => {
let mut octets = addr.octets();
let full_bytes = usize::from(prefix_length / 8);
let remaining_bits = prefix_length % 8;
if full_bytes < octets.len() {
if remaining_bits > 0 {
octets[full_bytes] &= 0xff << (8 - remaining_bits);
}
let first_zero = full_bytes + usize::from(remaining_bits > 0);
for byte in &mut octets[first_zero..] {
*byte = 0;
}
}
IpAddr::V6(Ipv6Addr::from(octets))
}
}
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn normalizes_ipv4_and_ipv6_networks() {
assert_eq!(
IpPrefix::parse("192.0.2.129/24").unwrap().to_cidr(),
"192.0.2.0/24"
);
assert_eq!(
IpPrefix::parse("2001:db8:1::1234/48").unwrap().to_cidr(),
"2001:db8:1::/48"
);
}
#[test]
fn prefix_lookup_uses_covering_and_max_length_semantics() {
let lookup = VrpLookup::parse_prefix("192.0.2.0/24").unwrap();
let covering = VrpRecord::from_report("run_1", 64496, "192.0.0.0/16", 24).unwrap();
let too_short = VrpRecord::from_report("run_1", 64497, "192.0.0.0/16", 23).unwrap();
let intersecting = VrpRecord::from_report("run_1", 64498, "192.0.2.128/25", 25).unwrap();
assert!(lookup.matches(&covering).unwrap());
assert!(!lookup.matches(&too_short).unwrap());
assert!(!lookup.matches(&intersecting).unwrap());
}
#[test]
fn cursor_is_scoped_and_bounded() {
assert_eq!(parse_cursor(None).unwrap(), 0);
assert_eq!(parse_cursor(Some("vrp:12")).unwrap(), 12);
assert!(parse_cursor(Some("12")).is_err());
assert_eq!(next_cursor(0, 10, 12).as_deref(), Some("vrp:10"));
assert_eq!(next_cursor(10, 10, 12), None);
}
}

View File

@@ -9,11 +9,10 @@ use sha2::{Digest, Sha256};
use crate::query::artifact_manifest::build_artifact_manifest; use crate::query::artifact_manifest::build_artifact_manifest;
use crate::query::report_stream::{self, ObjectLookup, ObjectScope, ReportSummary}; use crate::query::report_stream::{self, ObjectLookup, ObjectScope, ReportSummary};
use crate::query::vrp::{self, VrpLookup, VrpRecord};
use crate::blob_store::ExternalRepoBytesDb; use crate::blob_store::ExternalRepoBytesDb;
pub const QUERY_DB_SCHEMA_VERSION: u32 = 2; pub const QUERY_DB_SCHEMA_VERSION: u32 = 1;
pub const CF_META: &str = "meta"; pub const CF_META: &str = "meta";
pub const CF_RUNS: &str = "runs"; pub const CF_RUNS: &str = "runs";
@@ -27,7 +26,6 @@ pub const CF_VALIDATION_EXPLAIN_CACHE: &str = "validation_explain_cache";
pub const CF_EXPORT_JOBS: &str = "export_jobs"; pub const CF_EXPORT_JOBS: &str = "export_jobs";
pub const CF_STATS: &str = "stats"; pub const CF_STATS: &str = "stats";
pub const CF_REASON_INDEX: &str = "reason_index"; pub const CF_REASON_INDEX: &str = "reason_index";
pub const CF_VRPS: &str = "vrps";
pub const QUERY_DB_COLUMN_FAMILIES: &[&str] = &[ pub const QUERY_DB_COLUMN_FAMILIES: &[&str] = &[
CF_META, CF_META,
@@ -42,7 +40,6 @@ pub const QUERY_DB_COLUMN_FAMILIES: &[&str] = &[
CF_EXPORT_JOBS, CF_EXPORT_JOBS,
CF_STATS, CF_STATS,
CF_REASON_INDEX, CF_REASON_INDEX,
CF_VRPS,
]; ];
const KEY_SCHEMA_VERSION: &[u8] = b"schema_version"; const KEY_SCHEMA_VERSION: &[u8] = b"schema_version";
@@ -95,7 +92,6 @@ pub struct QueryIndexSummary {
pub object_instances_indexed: u64, pub object_instances_indexed: u64,
pub object_projections_indexed: u64, pub object_projections_indexed: u64,
pub stats_indexed: u64, pub stats_indexed: u64,
pub vrps_indexed: u64,
pub latest_ready_run: Option<String>, pub latest_ready_run: Option<String>,
pub errors: Vec<String>, pub errors: Vec<String>,
} }
@@ -373,52 +369,6 @@ impl QueryDb {
self.list_json_by_prefix(CF_REPOS, &format!("repo/{run_id}/"), limit, cursor) self.list_json_by_prefix(CF_REPOS, &format!("repo/{run_id}/"), limit, cursor)
} }
pub fn lookup_vrps(
&self,
run_id: &str,
lookup: &VrpLookup,
asn: Option<u32>,
limit: usize,
cursor: Option<&str>,
) -> QueryDbResult<QueryPage<VrpRecord>> {
let cf = self.cf(CF_VRPS)?;
let mut records = BTreeMap::<String, VrpRecord>::new();
for candidate in lookup.candidate_prefixes() {
let key_prefix = vrp::vrp_key_prefix(run_id, &candidate);
let mode = IteratorMode::From(key_prefix.as_bytes(), rocksdb::Direction::Forward);
for item in self.db.iterator_cf(cf, mode) {
let (key, value) = item?;
if !key.starts_with(key_prefix.as_bytes()) {
break;
}
let key_string = String::from_utf8_lossy(&key).to_string();
let record: VrpRecord = serde_json::from_slice(&value)?;
if asn.is_some_and(|expected| expected != record.asn)
|| !lookup
.matches(&record)
.map_err(QueryDbError::InvalidArtifact)?
{
continue;
}
records.insert(key_string, record);
}
}
let offset = vrp::parse_cursor(cursor).map_err(QueryDbError::InvalidArtifact)?;
let bounded_limit = limit.clamp(1, 1000);
let total = records.len();
let data = records
.into_values()
.skip(offset)
.take(bounded_limit)
.collect::<Vec<_>>();
Ok(QueryPage {
data,
next_cursor: vrp::next_cursor(offset, bounded_limit, total),
limit: bounded_limit,
})
}
pub fn get_repo(&self, run_id: &str, repo_id: &str) -> QueryDbResult<Option<RepositoryRecord>> { pub fn get_repo(&self, run_id: &str, repo_id: &str) -> QueryDbResult<Option<RepositoryRecord>> {
self.get_json_cf(CF_REPOS, repo_key(run_id, repo_id).as_bytes()) self.get_json_cf(CF_REPOS, repo_key(run_id, repo_id).as_bytes())
} }
@@ -523,31 +473,6 @@ impl QueryDb {
) )
} }
pub fn list_objects_filtered(
&self,
run_id: &str,
scope: ObjectScope,
filter: &report_stream::ObjectFilter,
limit: usize,
cursor: Option<&str>,
) -> QueryDbResult<QueryPage<ObjectInstanceRecord>> {
let Some(report_path) = self.report_path_for_run(run_id)? else {
return Ok(QueryPage {
data: Vec::new(),
next_cursor: None,
limit: limit.clamp(1, 1000),
});
};
report_stream::list_report_objects_filtered(
&report_path,
run_id,
scope,
filter,
limit,
cursor,
)
}
pub fn get_object_by_instance_id( pub fn get_object_by_instance_id(
&self, &self,
run_id: &str, run_id: &str,
@@ -703,118 +628,6 @@ impl QueryDb {
self.get_json_cf(CF_OBJECTS_BY_HASH, object_hash_key(sha256).as_bytes()) self.get_json_cf(CF_OBJECTS_BY_HASH, object_hash_key(sha256).as_bytes())
} }
pub fn search_object_projections_by_hash_prefix(
&self,
sha256_prefix: &str,
limit: usize,
) -> QueryDbResult<Vec<crate::object_projection::ObjectProjectionRecord>> {
let cf = self.cf(CF_OBJECTS_BY_HASH)?;
let prefix = format!("objhash/{sha256_prefix}");
let mut out = Vec::new();
let mode = IteratorMode::From(prefix.as_bytes(), rocksdb::Direction::Forward);
for item in self.db.iterator_cf(cf, mode) {
let (key, value) = item?;
if !String::from_utf8_lossy(&key).starts_with(&prefix) {
break;
}
if out.len() >= limit {
break;
}
out.push(serde_json::from_slice(&value)?);
}
Ok(out)
}
pub fn get_cached_object_by_sha256(
&self,
run_id: &str,
sha256: &str,
) -> QueryDbResult<Option<ObjectInstanceRecord>> {
let mut cursor = None;
loop {
let page: QueryPage<ObjectInstanceRecord> = self.list_json_by_prefix(
CF_OBJECT_INSTANCES,
&format!("objinst/{run_id}/"),
1000,
cursor.as_deref(),
)?;
if let Some(found) = page
.data
.into_iter()
.find(|item| item.sha256.eq_ignore_ascii_case(sha256))
{
return Ok(Some(found));
}
let Some(next_cursor) = page.next_cursor else {
return Ok(None);
};
cursor = Some(next_cursor);
}
}
pub fn search_repos(
&self,
run_id: &str,
query_lower: &str,
limit: usize,
) -> QueryDbResult<Vec<RepositoryRecord>> {
let cf = self.cf(CF_REPOS)?;
let prefix = format!("repo/{run_id}/");
let mut out = Vec::new();
let mode = IteratorMode::From(prefix.as_bytes(), rocksdb::Direction::Forward);
for item in self.db.iterator_cf(cf, mode) {
let (key, value) = item?;
if !String::from_utf8_lossy(&key).starts_with(&prefix) {
break;
}
if out.len() >= limit {
break;
}
let repo: RepositoryRecord = serde_json::from_slice(&value)?;
if repo.host.to_lowercase().contains(query_lower)
|| repo.uri.to_lowercase().contains(query_lower)
{
out.push(repo);
}
}
Ok(out)
}
pub fn search_publication_points(
&self,
run_id: &str,
query_lower: &str,
limit: usize,
) -> QueryDbResult<Vec<PublicationPointRecord>> {
let cf = self.cf(CF_PUBLICATION_POINTS)?;
let prefix = format!("pp/{run_id}/");
let mut out = Vec::new();
let mode = IteratorMode::From(prefix.as_bytes(), rocksdb::Direction::Forward);
for item in self.db.iterator_cf(cf, mode) {
let (key, value) = item?;
if !String::from_utf8_lossy(&key).starts_with(&prefix) {
break;
}
if out.len() >= limit {
break;
}
let pp: PublicationPointRecord = serde_json::from_slice(&value)?;
let uri_matches = [
pp.manifest_rsync_uri.as_deref(),
pp.publication_point_rsync_uri.as_deref(),
pp.rrdp_notification_uri.as_deref(),
pp.rsync_base_uri.as_deref(),
]
.into_iter()
.flatten()
.any(|uri| uri.to_lowercase().contains(query_lower));
if uri_matches {
out.push(pp);
}
}
Ok(out)
}
pub fn get_stat( pub fn get_stat(
&self, &self,
run_id: &str, run_id: &str,
@@ -949,20 +762,12 @@ impl QueryDb {
(CF_EXPORT_JOBS, format!("export/{}/", run.run_id)), (CF_EXPORT_JOBS, format!("export/{}/", run.run_id)),
(CF_STATS, format!("stats/{}/", run.run_id)), (CF_STATS, format!("stats/{}/", run.run_id)),
(CF_REASON_INDEX, format!("reason/{}/", run.run_id)), (CF_REASON_INDEX, format!("reason/{}/", run.run_id)),
(CF_VRPS, format!("vrp/{}/", run.run_id)),
] { ] {
self.delete_prefix_range(&mut batch, cf_name, &prefix)?; self.delete_prefix_range(&mut batch, cf_name, &prefix)?;
} }
self.write_batch(batch) self.write_batch(batch)
} }
fn clear_vrp_index(&self, run_id: &str) -> QueryDbResult<()> {
let mut batch = WriteBatch::default();
let prefix = format!("vrp/{run_id}/");
self.delete_prefix_range(&mut batch, CF_VRPS, &prefix)?;
self.write_batch(batch)
}
fn cf(&self, name: &'static str) -> QueryDbResult<&rocksdb::ColumnFamily> { fn cf(&self, name: &'static str) -> QueryDbResult<&rocksdb::ColumnFamily> {
self.db self.db
.cf_handle(name) .cf_handle(name)
@@ -1138,7 +943,6 @@ pub fn index_artifacts_with_open_db(
summary.object_instances_indexed += run_summary.object_instances_indexed; summary.object_instances_indexed += run_summary.object_instances_indexed;
summary.object_projections_indexed += run_summary.object_projections_indexed; summary.object_projections_indexed += run_summary.object_projections_indexed;
summary.stats_indexed += run_summary.stats_indexed; summary.stats_indexed += run_summary.stats_indexed;
summary.vrps_indexed += run_summary.vrps_indexed;
summary.latest_ready_run = run_summary.latest_ready_run; summary.latest_ready_run = run_summary.latest_ready_run;
} }
Err(err) => summary.errors.push(format!("{}: {err}", run_dir.display())), Err(err) => summary.errors.push(format!("{}: {err}", run_dir.display())),
@@ -1211,7 +1015,6 @@ struct SingleRunIndexSummary {
object_instances_indexed: u64, object_instances_indexed: u64,
object_projections_indexed: u64, object_projections_indexed: u64,
stats_indexed: u64, stats_indexed: u64,
vrps_indexed: u64,
latest_ready_run: Option<String>, latest_ready_run: Option<String>,
} }
@@ -1253,7 +1056,6 @@ fn index_run_dir(
run_record.index_status = "building".to_string(); run_record.index_status = "building".to_string();
run_record.index_error = None; run_record.index_error = None;
db.put_json_cf(CF_RUNS, run_key(&run_id).as_bytes(), &run_record)?; db.put_json_cf(CF_RUNS, run_key(&run_id).as_bytes(), &run_record)?;
db.clear_vrp_index(&run_id)?;
let indexed = let indexed =
match write_summary_index_records(db, &run_id, &report_summary, &artifact_manifest) { match write_summary_index_records(db, &run_id, &report_summary, &artifact_manifest) {
@@ -1297,7 +1099,6 @@ fn index_run_dir(
object_instances_indexed: indexed.object_instances_indexed, object_instances_indexed: indexed.object_instances_indexed,
object_projections_indexed: indexed.object_projections_indexed, object_projections_indexed: indexed.object_projections_indexed,
stats_indexed: indexed.stats_indexed, stats_indexed: indexed.stats_indexed,
vrps_indexed: indexed.vrps_indexed,
latest_ready_run: if should_update_latest { latest_ready_run: if should_update_latest {
Some(run_id) Some(run_id)
} else { } else {
@@ -1337,7 +1138,6 @@ struct IndexWriteSummary {
object_instances_indexed: u64, object_instances_indexed: u64,
object_projections_indexed: u64, object_projections_indexed: u64,
stats_indexed: u64, stats_indexed: u64,
vrps_indexed: u64,
} }
fn write_summary_index_records( fn write_summary_index_records(
@@ -1350,7 +1150,6 @@ fn write_summary_index_records(
let mut batch = WriteBatch::default(); let mut batch = WriteBatch::default();
let mut pending = 0usize; let mut pending = 0usize;
let mut summary = IndexWriteSummary::default(); let mut summary = IndexWriteSummary::default();
let mut seen_vrp_keys = BTreeSet::new();
macro_rules! flush_if_needed { macro_rules! flush_if_needed {
() => { () => {
@@ -1388,18 +1187,6 @@ fn write_summary_index_records(
flush_if_needed!(); flush_if_needed!();
} }
for vrp_record in &report_summary.vrps {
let key = vrp_record
.index_key()
.map_err(QueryDbError::InvalidArtifact)?;
if seen_vrp_keys.insert(key.clone()) {
put_json_batch(&mut batch, db, CF_VRPS, key.as_bytes(), vrp_record)?;
pending += 1;
summary.vrps_indexed += 1;
flush_if_needed!();
}
}
let artifacts_value = serde_json::to_value(artifact_manifest)?; let artifacts_value = serde_json::to_value(artifact_manifest)?;
let stats = report_stream::stats_records_from_summary(run_id, report_summary, artifacts_value); let stats = report_stream::stats_records_from_summary(run_id, report_summary, artifacts_value);
for record in &stats { for record in &stats {
@@ -1660,56 +1447,6 @@ mod tests {
assert_eq!(db.count_cf(CF_OBJECT_INSTANCES).unwrap(), 0); assert_eq!(db.count_cf(CF_OBJECT_INSTANCES).unwrap(), 0);
} }
#[test]
fn vrp_index_supports_ip_prefix_asn_and_pagination_queries() {
let temp = tempfile::tempdir().expect("tempdir");
let run_dir = temp.path().join("runs/run_0001");
fs::create_dir_all(&run_dir).expect("run dir");
write_sample_run(&run_dir, "run_0001", 1);
let mut report = read_json_file(&run_dir.join("report.json")).expect("read report");
let duplicate = report["vrps"][0].clone();
report["vrps"].as_array_mut().unwrap().push(duplicate);
fs::write(
run_dir.join("report.json"),
serde_json::to_vec(&report).unwrap(),
)
.expect("write duplicate VRP report");
let query_db_path = temp.path().join("query-db");
let summary = index_artifacts(&ArtifactIndexerConfig {
query_db_path: query_db_path.clone(),
run_root: Some(temp.path().to_path_buf()),
run_dir: None,
repo_bytes_db_path: None,
projection_entry_limit: 50,
min_run_seq: None,
retain_indexed_runs: None,
})
.expect("index");
assert_eq!(summary.vrps_indexed, 1);
let db = QueryDb::open(&query_db_path).expect("query db");
assert_eq!(db.count_cf(CF_VRPS).unwrap(), 1);
let ip = VrpLookup::parse_ip("192.0.2.1").expect("ip lookup");
let page = db
.lookup_vrps("run_0001", &ip, None, 1, None)
.expect("ip query");
assert_eq!(page.data.len(), 1);
assert_eq!(page.data[0].asn, 64496);
assert_eq!(page.data[0].prefix, "192.0.2.0/24");
assert!(page.next_cursor.is_none());
let prefix = VrpLookup::parse_prefix("192.0.2.0/24").expect("prefix lookup");
let filtered = db
.lookup_vrps("run_0001", &prefix, Some(64497), 100, None)
.expect("asn query");
assert!(filtered.data.is_empty());
let latest = db
.lookup_vrps("run_0001", &prefix, Some(64496), 100, None)
.expect("filtered prefix query");
assert_eq!(latest.data.len(), 1);
}
#[test] #[test]
fn failed_run_does_not_replace_previous_latest() { fn failed_run_does_not_replace_previous_latest() {
let temp = tempfile::tempdir().expect("tempdir"); let temp = tempfile::tempdir().expect("tempdir");

View File

@@ -8,7 +8,6 @@ use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex}; use std::sync::{Arc, Mutex};
use base64::Engine; use base64::Engine;
use rocksdb::checkpoint::Checkpoint;
use rocksdb::{ColumnFamily, DB, Direction, IteratorMode, Options, WriteBatch}; use rocksdb::{ColumnFamily, DB, Direction, IteratorMode, Options, WriteBatch};
use serde::{Deserialize, Serialize}; use serde::{Deserialize, Serialize};
@@ -54,13 +53,6 @@ pub enum StorageError {
pub type StorageResult<T> = Result<T, StorageError>; pub type StorageResult<T> = Result<T, StorageError>;
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize)]
pub struct RepositoryBlobVerificationSummary {
pub current_objects: u64,
pub bytes_verified: u64,
pub batches: u64,
}
#[derive(Clone, Debug, Default, PartialEq, Eq, Serialize)] #[derive(Clone, Debug, Default, PartialEq, Eq, Serialize)]
pub struct ChildCertificateCacheMmapLookup { pub struct ChildCertificateCacheMmapLookup {
pub projections: Vec<Option<ChildCertificateCacheProjection>>, pub projections: Vec<Option<ChildCertificateCacheProjection>>,
@@ -603,7 +595,7 @@ pub struct VcirFailedFetchReuseIdentity {
pub ta_context_digest: [u8; 32], pub ta_context_digest: [u8; 32],
#[serde(rename = "p")] #[serde(rename = "p")]
#[serde(with = "serde_bytes_32")] #[serde(with = "serde_bytes_32")]
pub ca_validation_context_digest: [u8; 32], pub parent_context_digest: [u8; 32],
#[serde(rename = "f")] #[serde(rename = "f")]
#[serde(with = "serde_bytes_32")] #[serde(with = "serde_bytes_32")]
pub policy_fingerprint: [u8; 32], pub policy_fingerprint: [u8; 32],
@@ -1125,7 +1117,7 @@ pub struct RoaCacheObjectMeta {
#[derive(Clone, Debug, PartialEq, Eq)] #[derive(Clone, Debug, PartialEq, Eq)]
pub struct RoaCacheProjectionContext { pub struct RoaCacheProjectionContext {
pub ca_validation_context_digest: [u8; 32], pub parent_context_digest: [u8; 32],
pub policy_fingerprint: [u8; 32], pub policy_fingerprint: [u8; 32],
pub object_meta: Vec<RoaCacheObjectMeta>, pub object_meta: Vec<RoaCacheObjectMeta>,
} }
@@ -1266,7 +1258,7 @@ pub struct RoaCacheProjection {
pub issuer_ca_sha256_hex: Option<String>, pub issuer_ca_sha256_hex: Option<String>,
#[serde(rename = "p", default, skip_serializing_if = "Option::is_none")] #[serde(rename = "p", default, skip_serializing_if = "Option::is_none")]
#[serde(with = "serde_optional_bytes_32")] #[serde(with = "serde_optional_bytes_32")]
pub ca_validation_context_digest: Option<[u8; 32]>, pub parent_context_digest: Option<[u8; 32]>,
#[serde(rename = "f", default, skip_serializing_if = "Option::is_none")] #[serde(rename = "f", default, skip_serializing_if = "Option::is_none")]
#[serde(with = "serde_optional_bytes_32")] #[serde(with = "serde_optional_bytes_32")]
pub policy_fingerprint: Option<[u8; 32]>, pub policy_fingerprint: Option<[u8; 32]>,
@@ -1408,7 +1400,7 @@ pub struct ChildCertificateCacheProjection {
pub issuer_crl_sha256_hex: String, pub issuer_crl_sha256_hex: String,
#[serde(rename = "pc")] #[serde(rename = "pc")]
#[serde(with = "serde_bytes_32")] #[serde(with = "serde_bytes_32")]
pub ca_validation_context_digest: [u8; 32], pub parent_context_digest: [u8; 32],
#[serde(rename = "pf")] #[serde(rename = "pf")]
#[serde(with = "serde_bytes_32")] #[serde(with = "serde_bytes_32")]
pub validation_policy_fingerprint: [u8; 32], pub validation_policy_fingerprint: [u8; 32],
@@ -1715,7 +1707,7 @@ pub struct PublicationPointCacheProjection {
pub ta_context_digest: [u8; 32], pub ta_context_digest: [u8; 32],
#[serde(rename = "pc")] #[serde(rename = "pc")]
#[serde(with = "serde_bytes_32")] #[serde(with = "serde_bytes_32")]
pub ca_validation_context_digest: [u8; 32], pub parent_context_digest: [u8; 32],
#[serde(rename = "pf")] #[serde(rename = "pf")]
#[serde(with = "serde_bytes_32")] #[serde(with = "serde_bytes_32")]
pub validation_policy_fingerprint: [u8; 32], pub validation_policy_fingerprint: [u8; 32],
@@ -1751,7 +1743,7 @@ impl PublicationPointCacheProjection {
ca_cert_sha256: [u8; 32], ca_cert_sha256: [u8; 32],
manifest_sha256: [u8; 32], manifest_sha256: [u8; 32],
ta_context_digest: [u8; 32], ta_context_digest: [u8; 32],
ca_validation_context_digest: [u8; 32], parent_context_digest: [u8; 32],
validation_policy_fingerprint: [u8; 32], validation_policy_fingerprint: [u8; 32],
) -> StorageResult<Self> { ) -> StorageResult<Self> {
let projection = Self { let projection = Self {
@@ -1764,7 +1756,7 @@ impl PublicationPointCacheProjection {
manifest_sha256, manifest_sha256,
tal_id: vcir.tal_id.clone(), tal_id: vcir.tal_id.clone(),
ta_context_digest, ta_context_digest,
ca_validation_context_digest, parent_context_digest,
validation_policy_fingerprint, validation_policy_fingerprint,
instance_effective_not_before: vcir.last_successful_validation_time.clone(), instance_effective_not_before: vcir.last_successful_validation_time.clone(),
instance_effective_until: vcir.instance_gate.instance_effective_until.clone(), instance_effective_until: vcir.instance_gate.instance_effective_until.clone(),
@@ -1819,7 +1811,7 @@ impl PublicationPointCacheProjection {
tal_id: self.tal_id.clone(), tal_id: self.tal_id.clone(),
ca_subject_name: String::from("publication-point-cache"), ca_subject_name: String::from("publication-point-cache"),
ca_ski: hex::encode(self.ca_cert_sha256), ca_ski: hex::encode(self.ca_cert_sha256),
issuer_ski: hex::encode(self.ca_validation_context_digest), issuer_ski: hex::encode(self.parent_context_digest),
last_successful_validation_time: validation_time.clone(), last_successful_validation_time: validation_time.clone(),
current_manifest_rsync_uri: self.manifest_rsync_uri.clone(), current_manifest_rsync_uri: self.manifest_rsync_uri.clone(),
current_crl_rsync_uri: self current_crl_rsync_uri: self
@@ -2050,7 +2042,7 @@ impl RoaCacheProjection {
manifest_rsync_uri: vcir.manifest_rsync_uri.clone(), manifest_rsync_uri: vcir.manifest_rsync_uri.clone(),
instance_effective_until: vcir.instance_gate.instance_effective_until.clone(), instance_effective_until: vcir.instance_gate.instance_effective_until.clone(),
issuer_ca_sha256_hex, issuer_ca_sha256_hex,
ca_validation_context_digest: Some(context.ca_validation_context_digest), parent_context_digest: Some(context.parent_context_digest),
policy_fingerprint: Some(context.policy_fingerprint), policy_fingerprint: Some(context.policy_fingerprint),
crl_sha256_by_uri, crl_sha256_by_uri,
entries, entries,
@@ -2071,10 +2063,10 @@ impl RoaCacheProjection {
if let Some(hash) = &self.issuer_ca_sha256_hex { if let Some(hash) = &self.issuer_ca_sha256_hex {
validate_sha256_hex("roa_cache_projection.issuer_ca_sha256_hex", hash)?; validate_sha256_hex("roa_cache_projection.issuer_ca_sha256_hex", hash)?;
} }
if self.ca_validation_context_digest.is_some() != self.policy_fingerprint.is_some() { if self.parent_context_digest.is_some() != self.policy_fingerprint.is_some() {
return Err(StorageError::InvalidData { return Err(StorageError::InvalidData {
entity: "roa_cache_projection.context", entity: "roa_cache_projection.context",
detail: "ca_validation_context_digest and policy_fingerprint must be both present or both absent" detail: "parent_context_digest and policy_fingerprint must be both present or both absent"
.to_string(), .to_string(),
}); });
} }
@@ -2901,38 +2893,6 @@ fn write_publication_point_cache_projection_to_batch(
} }
impl RocksStore { impl RocksStore {
pub fn create_read_only_checkpoint(source: &Path, destination: &Path) -> StorageResult<()> {
if destination.exists() {
return Err(StorageError::InvalidData {
entity: "work_db_checkpoint",
detail: format!(
"checkpoint destination already exists: {}",
destination.display()
),
});
}
if let Some(parent) = destination.parent() {
std::fs::create_dir_all(parent)
.map_err(|error| StorageError::RocksDb(error.to_string()))?;
}
let mut options = Options::default();
let blob_mode = work_db_blob_mode_from_env();
let memory_profile = work_db_memory_profile_from_env();
configure_work_db_options(&mut options, blob_mode, memory_profile);
let db = DB::open_cf_descriptors_read_only(
&options,
source,
column_family_descriptors_for_blob_mode(blob_mode),
false,
)
.map_err(|error| StorageError::RocksDb(error.to_string()))?;
let checkpoint =
Checkpoint::new(&db).map_err(|error| StorageError::RocksDb(error.to_string()))?;
checkpoint
.create_checkpoint(destination)
.map_err(|error| StorageError::RocksDb(error.to_string()))
}
pub fn open(path: &Path) -> StorageResult<Self> { pub fn open(path: &Path) -> StorageResult<Self> {
let mut base_opts = Options::default(); let mut base_opts = Options::default();
base_opts.create_if_missing(true); base_opts.create_if_missing(true);
@@ -2971,15 +2931,6 @@ impl RocksStore {
Self::open_with_external_stores(path, None, Some(repo_bytes_path)) Self::open_with_external_stores(path, None, Some(repo_bytes_path))
} }
pub fn open_with_external_repo_bytes_read_only(
path: &Path,
repo_bytes_path: &Path,
) -> StorageResult<Self> {
let mut store = Self::open(path)?;
store.external_repo_bytes = Some(ExternalRepoBytesDb::open_read_only(repo_bytes_path)?);
Ok(store)
}
pub fn open_with_external_stores( pub fn open_with_external_stores(
path: &Path, path: &Path,
raw_store_path: Option<&Path>, raw_store_path: Option<&Path>,
@@ -3196,53 +3147,6 @@ impl RocksStore {
.collect() .collect()
} }
pub fn verify_current_repository_blobs(
&self,
batch_size: usize,
) -> StorageResult<RepositoryBlobVerificationSummary> {
if batch_size == 0 {
return Err(StorageError::InvalidData {
entity: "repository_blob_verification",
detail: "batch_size must be greater than zero".to_string(),
});
}
let cf = self.cf(CF_REPOSITORY_VIEW)?;
let prefix = repository_view_prefix("");
let mode = IteratorMode::From(prefix.as_bytes(), Direction::Forward);
let mut summary = RepositoryBlobVerificationSummary::default();
let mut batch = Vec::with_capacity(batch_size);
for item in self.db.iterator_cf(cf, mode) {
let (key, value) = item.map_err(|error| StorageError::RocksDb(error.to_string()))?;
if !key.starts_with(prefix.as_bytes()) {
break;
}
let entry = decode_cbor::<RepositoryViewEntry>(&value, "repository_view")?;
entry.validate_internal()?;
if !matches!(
entry.state,
RepositoryViewState::Present | RepositoryViewState::Replaced
) {
continue;
}
let hash = entry
.current_hash
.clone()
.ok_or(StorageError::InvalidData {
entity: "repository_blob_verification",
detail: format!("current hash missing for {}", entry.rsync_uri),
})?;
batch.push((entry.rsync_uri, hash));
if batch.len() >= batch_size {
verify_repository_blob_batch(self, &batch, &mut summary)?;
batch.clear();
}
}
if !batch.is_empty() {
verify_repository_blob_batch(self, &batch, &mut summary)?;
}
Ok(summary)
}
pub fn put_raw_by_hash_entry(&self, entry: &RawByHashEntry) -> StorageResult<()> { pub fn put_raw_by_hash_entry(&self, entry: &RawByHashEntry) -> StorageResult<()> {
entry.validate_internal()?; entry.validate_internal()?;
if let Some(raw_store) = self.external_raw_store.as_ref() { if let Some(raw_store) = self.external_raw_store.as_ref() {
@@ -3302,9 +3206,6 @@ impl RocksStore {
return Ok(()); return Ok(());
} }
if let Some(repo_bytes) = self.external_repo_bytes.as_ref() { if let Some(repo_bytes) = self.external_repo_bytes.as_ref() {
if repo_bytes.is_read_only() {
return repo_bytes.require_existing_blob_bytes_batch(blobs);
}
return repo_bytes.put_blob_bytes_batch(blobs); return repo_bytes.put_blob_bytes_batch(blobs);
} }
if let Some(raw_store) = self.external_raw_store.as_ref() { if let Some(raw_store) = self.external_raw_store.as_ref() {
@@ -4781,37 +4682,6 @@ impl RocksStore {
} }
} }
fn verify_repository_blob_batch(
store: &RocksStore,
batch: &[(String, String)],
summary: &mut RepositoryBlobVerificationSummary,
) -> StorageResult<()> {
let hashes = batch
.iter()
.map(|(_, hash)| hash.clone())
.collect::<Vec<_>>();
let blobs = store.get_blob_bytes_batch(&hashes)?;
for ((uri, expected_hash), blob) in batch.iter().zip(blobs.into_iter()) {
let bytes = blob.as_ref().ok_or(StorageError::InvalidData {
entity: "repository_blob_verification",
detail: format!("blob missing for URI {uri} (hash={expected_hash})"),
})?;
let actual_hash = hex::encode(compute_sha256_32(bytes));
if !actual_hash.eq_ignore_ascii_case(expected_hash) {
return Err(StorageError::InvalidData {
entity: "repository_blob_verification",
detail: format!(
"blob hash mismatch for URI {uri}: expected={expected_hash}, actual={actual_hash}"
),
});
}
summary.current_objects += 1;
summary.bytes_verified += bytes.len() as u64;
}
summary.batches += 1;
Ok(())
}
#[cfg(test)] #[cfg(test)]
#[path = "storage/tests.rs"] #[path = "storage/tests.rs"]
mod tests; mod tests;

View File

@@ -29,7 +29,7 @@ fn sample_child_certificate_cache_projection(
issuer_ca_sha256_hex: sha256_hex(b"issuer-ca"), issuer_ca_sha256_hex: sha256_hex(b"issuer-ca"),
issuer_crl_uri: "rsync://example.test/repo/issuer.crl".to_string(), issuer_crl_uri: "rsync://example.test/repo/issuer.crl".to_string(),
issuer_crl_sha256_hex: sha256_hex(b"issuer-crl"), issuer_crl_sha256_hex: sha256_hex(b"issuer-crl"),
ca_validation_context_digest: sha256_32(b"parent-context"), parent_context_digest: sha256_32(b"parent-context"),
validation_policy_fingerprint: sha256_32(b"policy"), validation_policy_fingerprint: sha256_32(b"policy"),
effective_not_before: pack_time(0), effective_not_before: pack_time(0),
effective_until: pack_time(24), effective_until: pack_time(24),
@@ -294,7 +294,7 @@ fn roa_cache_projection_context(vcir: &ValidatedCaInstanceResult) -> RoaCachePro
}) })
.expect("sample VCIR has a ROA output"); .expect("sample VCIR has a ROA output");
RoaCacheProjectionContext { RoaCacheProjectionContext {
ca_validation_context_digest: [0x31; 32], parent_context_digest: [0x31; 32],
policy_fingerprint: [0x32; 32], policy_fingerprint: [0x32; 32],
object_meta: vec![RoaCacheObjectMeta { object_meta: vec![RoaCacheObjectMeta {
source_object_uri: roa_output.source_object_uri.clone(), source_object_uri: roa_output.source_object_uri.clone(),
@@ -2276,121 +2276,3 @@ fn pack_file_can_lazy_load_bytes_from_external_repo_bytes_store() {
assert_eq!(file.bytes_cloned().expect("cloned repo bytes"), bytes); assert_eq!(file.bytes_cloned().expect("cloned repo bytes"), bytes);
assert_eq!(file.compute_sha256().expect("compute sha256"), file.sha256); assert_eq!(file.compute_sha256().expect("compute sha256"), file.sha256);
} }
#[test]
fn read_only_checkpoint_isolated_from_source_work_db() {
let td = tempfile::tempdir().expect("tempdir");
let source_path = td.path().join("source-work-db");
let checkpoint_path = td.path().join("checkpoint-work-db");
let uri = "rsync://example.test/repo/a.roa";
let bytes = b"checkpoint-object";
let hash = sha256_hex(bytes);
{
let source = RocksStore::open(&source_path).expect("open source");
source
.put_blob_bytes_batch(&[(hash.clone(), bytes.to_vec())])
.expect("put blob");
source
.put_repository_view_entry(&RepositoryViewEntry {
rsync_uri: uri.to_string(),
current_hash: Some(hash),
repository_source: Some("fixture".to_string()),
object_type: Some("roa".to_string()),
state: RepositoryViewState::Present,
})
.expect("put view");
}
RocksStore::create_read_only_checkpoint(&source_path, &checkpoint_path)
.expect("create checkpoint");
let checkpoint = RocksStore::open(&checkpoint_path).expect("open checkpoint");
checkpoint
.delete_repository_view_entry(uri)
.expect("delete checkpoint entry");
drop(checkpoint);
let source = RocksStore::open(&source_path).expect("reopen source");
assert!(
source
.get_repository_view_entry(uri)
.expect("get source entry")
.is_some()
);
}
#[test]
fn read_only_external_repo_bytes_rejects_writes() {
let td = tempfile::tempdir().expect("tempdir");
let path = td.path().join("repo-bytes.db");
let hash = sha256_hex(b"repo-object");
{
let writable = ExternalRepoBytesDb::open(&path).expect("open writable repo bytes");
writable
.put_blob_bytes_batch(&[(hash.clone(), b"repo-object".to_vec())])
.expect("seed repo bytes");
}
let read_only = ExternalRepoBytesDb::open_read_only(&path).expect("open read-only repo bytes");
assert_eq!(
read_only
.get_blob_bytes(&hash)
.expect("read repo bytes")
.expect("blob"),
b"repo-object"
);
assert!(
read_only
.put_blob_bytes_batch(&[(hash, b"repo-object".to_vec())])
.is_err()
);
}
#[test]
fn read_only_store_accepts_only_idempotent_existing_blob_apply() {
let td = tempfile::tempdir().expect("tempdir");
let work_db_path = td.path().join("work-db");
let repo_bytes_path = td.path().join("repo-bytes.db");
let bytes = b"repo-object".to_vec();
let hash = sha256_hex(&bytes);
{
let repo_bytes = ExternalRepoBytesDb::open(&repo_bytes_path).expect("open repo bytes");
repo_bytes
.put_blob_bytes_batch(&[(hash.clone(), bytes.clone())])
.expect("seed repo bytes");
}
let store =
RocksStore::open_with_external_repo_bytes_read_only(&work_db_path, &repo_bytes_path)
.expect("open read-only store");
store
.put_blob_bytes_batch(&[(hash.clone(), bytes)])
.expect("idempotent apply");
assert!(
store
.put_blob_bytes_batch(&[(hash, b"different".to_vec())])
.is_err()
);
}
#[test]
fn repository_blob_verification_detects_missing_external_blob() {
let td = tempfile::tempdir().expect("tempdir");
let repo_bytes_path = td.path().join("repo-bytes.db");
ExternalRepoBytesDb::open(&repo_bytes_path).expect("create repo bytes");
let store = RocksStore::open_with_external_repo_bytes_read_only(
&td.path().join("work-db"),
&repo_bytes_path,
)
.expect("open work db");
store
.put_repository_view_entry(&RepositoryViewEntry {
rsync_uri: "rsync://example.test/repo/missing.roa".to_string(),
current_hash: Some(sha256_hex(b"missing")),
repository_source: Some("fixture".to_string()),
object_type: Some("roa".to_string()),
state: RepositoryViewState::Present,
})
.expect("put view");
let error = store
.verify_current_repository_blobs(16)
.expect_err("missing blob must fail");
assert!(error.to_string().contains("missing.roa"), "{error}");
}

View File

@@ -180,7 +180,7 @@ pub struct RoaValidationCacheStats {
pub struct RoaValidationCacheInput<'a> { pub struct RoaValidationCacheInput<'a> {
enabled: bool, enabled: bool,
view: Option<&'a RoaValidationCacheView>, view: Option<&'a RoaValidationCacheView>,
ca_validation_context_digest: Option<[u8; 32]>, parent_context_digest: Option<[u8; 32]>,
policy_fingerprint: Option<[u8; 32]>, policy_fingerprint: Option<[u8; 32]>,
} }
@@ -189,7 +189,7 @@ impl<'a> RoaValidationCacheInput<'a> {
Self { Self {
enabled: false, enabled: false,
view: None, view: None,
ca_validation_context_digest: None, parent_context_digest: None,
policy_fingerprint: None, policy_fingerprint: None,
} }
} }
@@ -198,20 +198,20 @@ impl<'a> RoaValidationCacheInput<'a> {
Self { Self {
enabled: true, enabled: true,
view, view,
ca_validation_context_digest: None, parent_context_digest: None,
policy_fingerprint: None, policy_fingerprint: None,
} }
} }
pub fn enabled_with_context( pub fn enabled_with_context(
view: Option<&'a RoaValidationCacheView>, view: Option<&'a RoaValidationCacheView>,
ca_validation_context_digest: [u8; 32], parent_context_digest: [u8; 32],
policy_fingerprint: [u8; 32], policy_fingerprint: [u8; 32],
) -> Self { ) -> Self {
Self { Self {
enabled: true, enabled: true,
view, view,
ca_validation_context_digest: Some(ca_validation_context_digest), parent_context_digest: Some(parent_context_digest),
policy_fingerprint: Some(policy_fingerprint), policy_fingerprint: Some(policy_fingerprint),
} }
} }
@@ -495,7 +495,7 @@ fn evaluate_roa_cache_crl_gate(
pub struct RoaValidationCacheView { pub struct RoaValidationCacheView {
entries_by_uri: HashMap<String, CachedRoaValidationResult>, entries_by_uri: HashMap<String, CachedRoaValidationResult>,
issuer_ca_sha256_hex: Option<String>, issuer_ca_sha256_hex: Option<String>,
ca_validation_context_digest: Option<[u8; 32]>, parent_context_digest: Option<[u8; 32]>,
policy_fingerprint: Option<[u8; 32]>, policy_fingerprint: Option<[u8; 32]>,
crl_sha256_by_uri: HashMap<String, String>, crl_sha256_by_uri: HashMap<String, String>,
blocked: bool, blocked: bool,
@@ -519,7 +519,7 @@ impl RoaValidationCacheView {
let mut entries_by_uri: HashMap<String, CachedRoaValidationResult> = let mut entries_by_uri: HashMap<String, CachedRoaValidationResult> =
HashMap::with_capacity(projection.entries.len()); HashMap::with_capacity(projection.entries.len());
let issuer_ca_sha256_hex = projection.issuer_ca_sha256_hex.clone(); let issuer_ca_sha256_hex = projection.issuer_ca_sha256_hex.clone();
let ca_validation_context_digest = projection.ca_validation_context_digest; let parent_context_digest = projection.parent_context_digest;
let policy_fingerprint = projection.policy_fingerprint; let policy_fingerprint = projection.policy_fingerprint;
let crl_sha256_by_uri = projection let crl_sha256_by_uri = projection
.crl_sha256_by_uri .crl_sha256_by_uri
@@ -536,7 +536,7 @@ impl RoaValidationCacheView {
return Self { return Self {
entries_by_uri, entries_by_uri,
issuer_ca_sha256_hex, issuer_ca_sha256_hex,
ca_validation_context_digest, parent_context_digest,
policy_fingerprint, policy_fingerprint,
crl_sha256_by_uri, crl_sha256_by_uri,
blocked, blocked,
@@ -580,7 +580,7 @@ impl RoaValidationCacheView {
Self { Self {
entries_by_uri, entries_by_uri,
issuer_ca_sha256_hex, issuer_ca_sha256_hex,
ca_validation_context_digest, parent_context_digest,
policy_fingerprint, policy_fingerprint,
crl_sha256_by_uri, crl_sha256_by_uri,
blocked, blocked,
@@ -590,7 +590,7 @@ impl RoaValidationCacheView {
fn matches_current_context( fn matches_current_context(
&self, &self,
issuer_ca_der: &[u8], issuer_ca_der: &[u8],
ca_validation_context_digest: Option<[u8; 32]>, parent_context_digest: Option<[u8; 32]>,
policy_fingerprint: Option<[u8; 32]>, policy_fingerprint: Option<[u8; 32]>,
) -> bool { ) -> bool {
if self.blocked { if self.blocked {
@@ -604,8 +604,8 @@ impl RoaValidationCacheView {
return false; return false;
} }
if let Some(expected_ca_validation_context) = self.ca_validation_context_digest { if let Some(expected_parent_context) = self.parent_context_digest {
if Some(expected_ca_validation_context) != ca_validation_context_digest { if Some(expected_parent_context) != parent_context_digest {
return false; return false;
} }
} else { } else {
@@ -797,7 +797,7 @@ fn active_roa_cache_view<'a>(
let gate_started = Instant::now(); let gate_started = Instant::now();
if view.matches_current_context( if view.matches_current_context(
issuer_ca_der, issuer_ca_der,
roa_cache.ca_validation_context_digest, roa_cache.parent_context_digest,
roa_cache.policy_fingerprint, roa_cache.policy_fingerprint,
) { ) {
stats.context_gate_nanos = stats stats.context_gate_nanos = stats
@@ -3914,7 +3914,7 @@ mod tests {
OffsetDateTime::parse(value, &Rfc3339).expect("parse fixed test time") OffsetDateTime::parse(value, &Rfc3339).expect("parse fixed test time")
} }
const TEST_CA_VALIDATION_CONTEXT: [u8; 32] = [0x70; 32]; const TEST_PARENT_CONTEXT: [u8; 32] = [0x70; 32];
const TEST_POLICY_FINGERPRINT: [u8; 32] = [0x71; 32]; const TEST_POLICY_FINGERPRINT: [u8; 32] = [0x71; 32];
const TEST_CRL_URI: &str = "rsync://example.test/repo/current.crl"; const TEST_CRL_URI: &str = "rsync://example.test/repo/current.crl";
const TEST_ROA_URI: &str = "rsync://example.test/repo/a.roa"; const TEST_ROA_URI: &str = "rsync://example.test/repo/a.roa";
@@ -3933,7 +3933,7 @@ mod tests {
RoaCacheProjection::from_vcir_with_context( RoaCacheProjection::from_vcir_with_context(
vcir, vcir,
Some(&RoaCacheProjectionContext { Some(&RoaCacheProjectionContext {
ca_validation_context_digest: TEST_CA_VALIDATION_CONTEXT, parent_context_digest: TEST_PARENT_CONTEXT,
policy_fingerprint: TEST_POLICY_FINGERPRINT, policy_fingerprint: TEST_POLICY_FINGERPRINT,
object_meta: vec![RoaCacheObjectMeta { object_meta: vec![RoaCacheObjectMeta {
source_object_uri: TEST_ROA_URI.to_string(), source_object_uri: TEST_ROA_URI.to_string(),
@@ -4071,7 +4071,7 @@ mod tests {
assert!(view.matches_current_context( assert!(view.matches_current_context(
issuer_der, issuer_der,
Some(TEST_CA_VALIDATION_CONTEXT), Some(TEST_PARENT_CONTEXT),
Some(TEST_POLICY_FINGERPRINT) Some(TEST_POLICY_FINGERPRINT)
)); ));
let hit = view.lookup(&file, &mut crl_cache, issuer_der, validation_time); let hit = view.lookup(&file, &mut crl_cache, issuer_der, validation_time);
@@ -4202,10 +4202,7 @@ mod tests {
fixed_time("2026-06-08T00:00:00Z"), fixed_time("2026-06-08T00:00:00Z"),
); );
let projection = sample_roa_cache_projection(&vcir, roa_hash); let projection = sample_roa_cache_projection(&vcir, roa_hash);
assert_eq!( assert_eq!(projection.parent_context_digest, Some(TEST_PARENT_CONTEXT));
projection.ca_validation_context_digest,
Some(TEST_CA_VALIDATION_CONTEXT)
);
assert_eq!(projection.policy_fingerprint, Some(TEST_POLICY_FINGERPRINT)); assert_eq!(projection.policy_fingerprint, Some(TEST_POLICY_FINGERPRINT));
assert_eq!( assert_eq!(
projection.entries[0].outputs_effective_until_unix, projection.entries[0].outputs_effective_until_unix,
@@ -4248,7 +4245,7 @@ mod tests {
let issuer_changed = RoaValidationCacheView::from_projection(&projection, validation_time); let issuer_changed = RoaValidationCacheView::from_projection(&projection, validation_time);
assert!(!issuer_changed.matches_current_context( assert!(!issuer_changed.matches_current_context(
issuer_der, issuer_der,
Some(TEST_CA_VALIDATION_CONTEXT), Some(TEST_PARENT_CONTEXT),
Some(TEST_POLICY_FINGERPRINT) Some(TEST_POLICY_FINGERPRINT)
)); ));
@@ -4261,7 +4258,7 @@ mod tests {
)); ));
assert!(!parent_changed.matches_current_context( assert!(!parent_changed.matches_current_context(
issuer_der, issuer_der,
Some(TEST_CA_VALIDATION_CONTEXT), Some(TEST_PARENT_CONTEXT),
Some([0x98; 32]) Some([0x98; 32])
)); ));
@@ -4286,7 +4283,7 @@ mod tests {
RoaValidationCacheView::from_projection(&expired_projection, validation_time); RoaValidationCacheView::from_projection(&expired_projection, validation_time);
assert!(!expired_view.matches_current_context( assert!(!expired_view.matches_current_context(
issuer_der, issuer_der,
Some(TEST_CA_VALIDATION_CONTEXT), Some(TEST_PARENT_CONTEXT),
Some(TEST_POLICY_FINGERPRINT) Some(TEST_POLICY_FINGERPRINT)
)); ));
let mut crl_cache = sample_crl_cache(b"current-crl".to_vec()); let mut crl_cache = sample_crl_cache(b"current-crl".to_vec());
@@ -4566,7 +4563,7 @@ mod tests {
assert!(view.matches_current_context( assert!(view.matches_current_context(
issuer_der, issuer_der,
Some(TEST_CA_VALIDATION_CONTEXT), Some(TEST_PARENT_CONTEXT),
Some(TEST_POLICY_FINGERPRINT) Some(TEST_POLICY_FINGERPRINT)
)); ));
assert!(matches!( assert!(matches!(
@@ -4598,43 +4595,43 @@ mod tests {
let mut view = RoaValidationCacheView { let mut view = RoaValidationCacheView {
entries_by_uri: HashMap::new(), entries_by_uri: HashMap::new(),
issuer_ca_sha256_hex: None, issuer_ca_sha256_hex: None,
ca_validation_context_digest: Some(TEST_CA_VALIDATION_CONTEXT), parent_context_digest: Some(TEST_PARENT_CONTEXT),
policy_fingerprint: Some(TEST_POLICY_FINGERPRINT), policy_fingerprint: Some(TEST_POLICY_FINGERPRINT),
crl_sha256_by_uri: HashMap::new(), crl_sha256_by_uri: HashMap::new(),
blocked: false, blocked: false,
}; };
assert!(!view.matches_current_context( assert!(!view.matches_current_context(
b"issuer-ca", b"issuer-ca",
Some(TEST_CA_VALIDATION_CONTEXT), Some(TEST_PARENT_CONTEXT),
Some(TEST_POLICY_FINGERPRINT) Some(TEST_POLICY_FINGERPRINT)
)); ));
view.issuer_ca_sha256_hex = Some("00".repeat(32)); view.issuer_ca_sha256_hex = Some("00".repeat(32));
assert!(!view.matches_current_context( assert!(!view.matches_current_context(
b"issuer-ca", b"issuer-ca",
Some(TEST_CA_VALIDATION_CONTEXT), Some(TEST_PARENT_CONTEXT),
Some(TEST_POLICY_FINGERPRINT) Some(TEST_POLICY_FINGERPRINT)
)); ));
view.issuer_ca_sha256_hex = Some(sha256_hex(b"issuer-ca")); view.issuer_ca_sha256_hex = Some(sha256_hex(b"issuer-ca"));
assert!(view.matches_current_context( assert!(view.matches_current_context(
b"issuer-ca", b"issuer-ca",
Some(TEST_CA_VALIDATION_CONTEXT), Some(TEST_PARENT_CONTEXT),
Some(TEST_POLICY_FINGERPRINT) Some(TEST_POLICY_FINGERPRINT)
)); ));
assert!(!view.matches_current_context(b"issuer-ca", None, Some(TEST_POLICY_FINGERPRINT))); assert!(!view.matches_current_context(b"issuer-ca", None, Some(TEST_POLICY_FINGERPRINT)));
view.ca_validation_context_digest = None; view.parent_context_digest = None;
assert!(!view.matches_current_context( assert!(!view.matches_current_context(
b"issuer-ca", b"issuer-ca",
Some(TEST_CA_VALIDATION_CONTEXT), Some(TEST_PARENT_CONTEXT),
Some(TEST_POLICY_FINGERPRINT) Some(TEST_POLICY_FINGERPRINT)
)); ));
view.ca_validation_context_digest = Some(TEST_CA_VALIDATION_CONTEXT); view.parent_context_digest = Some(TEST_PARENT_CONTEXT);
view.policy_fingerprint = None; view.policy_fingerprint = None;
assert!(!view.matches_current_context( assert!(!view.matches_current_context(
b"issuer-ca", b"issuer-ca",
Some(TEST_CA_VALIDATION_CONTEXT), Some(TEST_PARENT_CONTEXT),
Some(TEST_POLICY_FINGERPRINT) Some(TEST_POLICY_FINGERPRINT)
)); ));
} }
@@ -4664,7 +4661,7 @@ mod tests {
let mut view = RoaValidationCacheView { let mut view = RoaValidationCacheView {
entries_by_uri: HashMap::new(), entries_by_uri: HashMap::new(),
issuer_ca_sha256_hex: Some(sha256_hex(issuer_der)), issuer_ca_sha256_hex: Some(sha256_hex(issuer_der)),
ca_validation_context_digest: Some(TEST_CA_VALIDATION_CONTEXT), parent_context_digest: Some(TEST_PARENT_CONTEXT),
policy_fingerprint: Some(TEST_POLICY_FINGERPRINT), policy_fingerprint: Some(TEST_POLICY_FINGERPRINT),
crl_sha256_by_uri: HashMap::from([(TEST_CRL_URI.to_string(), crl_hash_hex.clone())]), crl_sha256_by_uri: HashMap::from([(TEST_CRL_URI.to_string(), crl_hash_hex.clone())]),
blocked: false, blocked: false,
@@ -4852,7 +4849,7 @@ mod tests {
false, false,
RoaValidationCacheInput::enabled_with_context( RoaValidationCacheInput::enabled_with_context(
Some(&view), Some(&view),
TEST_CA_VALIDATION_CONTEXT, TEST_PARENT_CONTEXT,
TEST_POLICY_FINGERPRINT, TEST_POLICY_FINGERPRINT,
), ),
) { ) {

View File

@@ -10,8 +10,6 @@ use crate::validation::publication_point::PublicationPointSnapshot;
use std::borrow::Cow; use std::borrow::Cow;
use std::sync::{Arc, OnceLock}; use std::sync::{Arc, OnceLock};
pub const DEFAULT_MAX_CA_DEPTH: usize = 32;
#[derive(Clone, Debug, PartialEq, Eq)] #[derive(Clone, Debug, PartialEq, Eq)]
pub struct TreeRunConfig { pub struct TreeRunConfig {
/// Max CA instance depth to process (0 = root only). /// Max CA instance depth to process (0 = root only).
@@ -39,7 +37,7 @@ pub struct TreeRunConfig {
impl Default for TreeRunConfig { impl Default for TreeRunConfig {
fn default() -> Self { fn default() -> Self {
Self { Self {
max_depth: Some(DEFAULT_MAX_CA_DEPTH), max_depth: None,
max_instances: None, max_instances: None,
compact_audit: false, compact_audit: false,
persist_vcir: true, persist_vcir: true,
@@ -53,15 +51,6 @@ impl Default for TreeRunConfig {
} }
} }
pub(crate) fn ca_depth_is_allowed(config: &TreeRunConfig, depth: usize) -> bool {
config.max_depth.is_none_or(|max_depth| depth <= max_depth)
}
pub(crate) fn next_allowed_ca_depth(config: &TreeRunConfig, parent_depth: usize) -> Option<usize> {
let child_depth = parent_depth.checked_add(1)?;
ca_depth_is_allowed(config, child_depth).then_some(child_depth)
}
#[derive(Clone, Debug)] #[derive(Clone, Debug)]
pub enum CaCertificateRef { pub enum CaCertificateRef {
InlineDer(Vec<u8>), InlineDer(Vec<u8>),
@@ -358,9 +347,11 @@ pub fn run_tree_serial_audit_multi_root(
} }
} }
if !ca_depth_is_allowed(config, ca.depth) { if let Some(max_depth) = config.max_depth {
if ca.depth > max_depth {
continue; continue;
} }
}
let res = match runner.run_publication_point(ca) { let res = match runner.run_publication_point(ca) {
Ok(v) => v, Ok(v) => v,
@@ -402,9 +393,6 @@ pub fn run_tree_serial_audit_multi_root(
} }
publication_points.push(audit); publication_points.push(audit);
let Some(child_depth) = next_allowed_ca_depth(config, ca.depth) else {
continue;
};
let mut children = res.discovered_children; let mut children = res.discovered_children;
children.sort_by(|a, b| { children.sort_by(|a, b| {
a.handle a.handle
@@ -425,7 +413,7 @@ pub fn run_tree_serial_audit_multi_root(
for child in children { for child in children {
queue.push_back(QueuedCaInstance { queue.push_back(QueuedCaInstance {
id: next_id, id: next_id,
handle: child.handle.with_depth(child_depth), handle: child.handle.with_depth(ca.depth + 1),
parent_id: Some(node.id), parent_id: Some(node.id),
discovered_from: Some(child.discovered_from), discovered_from: Some(child.discovered_from),
}); });

View File

@@ -19,8 +19,8 @@ use crate::validation::objects::{
}; };
use crate::validation::tree::{ use crate::validation::tree::{
CaInstanceHandle, DiscoveredChildCaInstance, PublicationPointRunResult, PublicationPointRunner, CaInstanceHandle, DiscoveredChildCaInstance, PublicationPointRunResult, PublicationPointRunner,
TreeRunAuditOutput, TreeRunConfig, TreeRunError, TreeRunOutput, ca_depth_is_allowed, TreeRunAuditOutput, TreeRunConfig, TreeRunError, TreeRunOutput,
next_allowed_ca_depth, run_tree_serial_audit_multi_root, run_tree_serial_audit_multi_root,
}; };
use crate::validation::tree_runner::{ use crate::validation::tree_runner::{
FreshPublicationPointFinalizeOutput, FreshPublicationPointStage, Rpkiv1PublicationPointRunner, FreshPublicationPointFinalizeOutput, FreshPublicationPointStage, Rpkiv1PublicationPointRunner,
@@ -659,7 +659,6 @@ pub fn run_tree_parallel_phase2_audit_multi_root(
&mut finished, &mut finished,
ready, ready,
ready_queue_len_after_pop, ready_queue_len_after_pop,
config,
config.compact_audit, config.compact_audit,
); );
ready_batch_metrics.record(metrics); ready_batch_metrics.record(metrics);
@@ -917,9 +916,11 @@ fn start_queued_ca_instances(
if !visited_manifest_uris.insert(node.handle.manifest_rsync_uri.clone()) { if !visited_manifest_uris.insert(node.handle.manifest_rsync_uri.clone()) {
continue; continue;
} }
if !ca_depth_is_allowed(config, node.handle.depth) { if let Some(max_depth) = config.max_depth {
if node.handle.depth > max_depth {
continue; continue;
} }
}
*instances_started += 1; *instances_started += 1;
match repo_runtime.request_publication_point_repo(&node.handle, 0) { match repo_runtime.request_publication_point_repo(&node.handle, 0) {
Ok(RepoSyncRequestStatus::Ready { mut outcome, .. }) => { Ok(RepoSyncRequestStatus::Ready { mut outcome, .. }) => {
@@ -958,7 +959,6 @@ fn stage_ready_publication_point(
finished: &mut Vec<FinishedPublicationPoint>, finished: &mut Vec<FinishedPublicationPoint>,
ready: ReadyCaInstance, ready: ReadyCaInstance,
ready_queue_len_after_pop: usize, ready_queue_len_after_pop: usize,
config: &TreeRunConfig,
compact_audit: bool, compact_audit: bool,
) -> ReadyStageMetrics { ) -> ReadyStageMetrics {
let publication_point_started = Instant::now(); let publication_point_started = Instant::now();
@@ -991,7 +991,6 @@ fn stage_ready_publication_point(
next_id, next_id,
ca_queue, ca_queue,
&ready.node, &ready.node,
config,
result.discovered_children.clone(), result.discovered_children.clone(),
); );
metrics.child_enqueue_ms = elapsed_ms(child_enqueue_started); metrics.child_enqueue_ms = elapsed_ms(child_enqueue_started);
@@ -1058,7 +1057,6 @@ fn stage_ready_publication_point(
next_id, next_id,
ca_queue, ca_queue,
&ready.node, &ready.node,
config,
result.discovered_children.clone(), result.discovered_children.clone(),
); );
metrics.child_enqueue_ms = elapsed_ms(child_enqueue_started); metrics.child_enqueue_ms = elapsed_ms(child_enqueue_started);
@@ -1144,7 +1142,6 @@ fn stage_ready_publication_point(
next_id, next_id,
ca_queue, ca_queue,
&ready.node, &ready.node,
config,
fresh_stage.discovered_children.clone(), fresh_stage.discovered_children.clone(),
); );
metrics.child_enqueue_ms = elapsed_ms(child_enqueue_started); metrics.child_enqueue_ms = elapsed_ms(child_enqueue_started);
@@ -1178,7 +1175,7 @@ fn stage_ready_publication_point(
let roa_cache = if runner.enable_roa_validation_cache && has_roa { let roa_cache = if runner.enable_roa_validation_cache && has_roa {
RoaValidationCacheInput::enabled_with_context( RoaValidationCacheInput::enabled_with_context(
roa_cache_view.as_ref(), roa_cache_view.as_ref(),
crate::validation::tree_runner::ca_validation_context_digest_for_ca(&ready.node.handle), crate::validation::tree_runner::parent_context_digest_for_ca(&ready.node.handle),
crate::validation::tree_runner::publication_point_cache_policy_fingerprint( crate::validation::tree_runner::publication_point_cache_policy_fingerprint(
runner.policy, runner.policy,
), ),
@@ -1415,13 +1412,8 @@ fn enqueue_discovered_children(
next_id: &mut u64, next_id: &mut u64,
ca_queue: &mut VecDeque<QueuedCaInstance>, ca_queue: &mut VecDeque<QueuedCaInstance>,
parent: &QueuedCaInstance, parent: &QueuedCaInstance,
config: &TreeRunConfig,
mut children: Vec<DiscoveredChildCaInstance>, mut children: Vec<DiscoveredChildCaInstance>,
) { ) {
let Some(child_depth) = next_allowed_ca_depth(config, parent.handle.depth) else {
return;
};
children.sort_by(|a, b| { children.sort_by(|a, b| {
a.handle a.handle
.manifest_rsync_uri .manifest_rsync_uri
@@ -1436,7 +1428,7 @@ fn enqueue_discovered_children(
let _ = runtime.prefetch_discovered_children(&children); let _ = runtime.prefetch_discovered_children(&children);
} }
for child in children { for child in children {
let mut handle = child.handle.with_depth(child_depth); let mut handle = child.handle.with_depth(parent.handle.depth + 1);
handle.parent_manifest_rsync_uri = Some(parent.handle.manifest_rsync_uri.clone()); handle.parent_manifest_rsync_uri = Some(parent.handle.manifest_rsync_uri.clone());
ca_queue.push_back(QueuedCaInstance { ca_queue.push_back(QueuedCaInstance {
id: *next_id, id: *next_id,

View File

@@ -1,3 +1,5 @@
mod vcir_der;
use crate::analysis::timing::TimingHandle; use crate::analysis::timing::TimingHandle;
use crate::audit::{ use crate::audit::{
AuditObjectKind, AuditObjectResult, AuditWarning, ObjectAuditEntry, PublicationPointAudit, AuditObjectKind, AuditObjectResult, AuditWarning, ObjectAuditEntry, PublicationPointAudit,
@@ -67,7 +69,7 @@ use base64::Engine as _;
use x509_parser::prelude::FromDer; use x509_parser::prelude::FromDer;
use x509_parser::x509::SubjectPublicKeyInfo; use x509_parser::x509::SubjectPublicKeyInfo;
use crate::ccr::manifest_location::select_manifest_signed_object_location; use vcir_der::encode_access_description_der_for_vcir_ccr_projection;
const PUBLICATION_POINT_CACHE_CHILD_RESTORE_PARALLEL_MIN_CHILDREN: usize = 256; const PUBLICATION_POINT_CACHE_CHILD_RESTORE_PARALLEL_MIN_CHILDREN: usize = 256;
const PUBLICATION_POINT_CACHE_CHILD_RESTORE_MAX_WORKERS: usize = 16; const PUBLICATION_POINT_CACHE_CHILD_RESTORE_MAX_WORKERS: usize = 16;
@@ -302,8 +304,7 @@ impl<'a> Rpkiv1PublicationPointRunner<'a> {
self.finish_publication_point_cache_miss(ca, "ta_context_mismatch", lookup_started); self.finish_publication_point_cache_miss(ca, "ta_context_mismatch", lookup_started);
return None; return None;
} }
if projection.ca_validation_context_digest != current_identity.ca_validation_context_digest if projection.parent_context_digest != current_identity.parent_context_digest {
{
self.finish_publication_point_cache_miss(ca, "parent_context_mismatch", lookup_started); self.finish_publication_point_cache_miss(ca, "parent_context_mismatch", lookup_started);
return None; return None;
} }
@@ -504,7 +505,7 @@ impl<'a> Rpkiv1PublicationPointRunner<'a> {
ca_cert_sha256, ca_cert_sha256,
manifest_sha256, manifest_sha256,
ta_context_digest: ta_context_digest_for_ca(ca), ta_context_digest: ta_context_digest_for_ca(ca),
ca_validation_context_digest: ca_validation_context_digest_for_ca(ca), parent_context_digest: parent_context_digest_for_ca(ca),
policy_fingerprint: publication_point_cache_policy_fingerprint(self.policy), policy_fingerprint: publication_point_cache_policy_fingerprint(self.policy),
}) })
} }
@@ -937,7 +938,7 @@ impl<'a> Rpkiv1PublicationPointRunner<'a> {
Some(ChildCertificateValidationCacheContext { Some(ChildCertificateValidationCacheContext {
store: self.store, store: self.store,
issuer_ca_sha256: sha256_digest_32(issuer_ca_der.as_ref()), issuer_ca_sha256: sha256_digest_32(issuer_ca_der.as_ref()),
ca_validation_context_digest: ca_validation_context_digest_for_ca(ca), parent_context_digest: parent_context_digest_for_ca(ca),
policy_fingerprint: publication_point_cache_policy_fingerprint(self.policy), policy_fingerprint: publication_point_cache_policy_fingerprint(self.policy),
}) })
} else { } else {
@@ -1529,7 +1530,7 @@ impl<'a> PublicationPointRunner for Rpkiv1PublicationPointRunner<'a> {
let roa_cache = if self.enable_roa_validation_cache && has_roa { let roa_cache = if self.enable_roa_validation_cache && has_roa {
RoaValidationCacheInput::enabled_with_context( RoaValidationCacheInput::enabled_with_context(
roa_cache_view.as_ref(), roa_cache_view.as_ref(),
ca_validation_context_digest_for_ca(ca), parent_context_digest_for_ca(ca),
publication_point_cache_policy_fingerprint(self.policy), publication_point_cache_policy_fingerprint(self.policy),
) )
} else { } else {
@@ -1945,7 +1946,7 @@ struct PublicationPointCacheIdentity {
ca_cert_sha256: [u8; 32], ca_cert_sha256: [u8; 32],
manifest_sha256: [u8; 32], manifest_sha256: [u8; 32],
ta_context_digest: [u8; 32], ta_context_digest: [u8; 32],
ca_validation_context_digest: [u8; 32], parent_context_digest: [u8; 32],
policy_fingerprint: [u8; 32], policy_fingerprint: [u8; 32],
} }
@@ -1981,7 +1982,7 @@ fn ta_context_digest_for_ca(ca: &CaInstanceHandle) -> [u8; 32] {
]) ])
} }
pub(crate) fn ca_validation_context_digest_for_ca(ca: &CaInstanceHandle) -> [u8; 32] { pub(crate) fn parent_context_digest_for_ca(ca: &CaInstanceHandle) -> [u8; 32] {
hash_serialized_parts(&[ hash_serialized_parts(&[
( (
"version", "version",
@@ -2057,7 +2058,7 @@ fn pack_time_window_contains(
struct ChildCertificateValidationCacheContext<'a> { struct ChildCertificateValidationCacheContext<'a> {
store: &'a RocksStore, store: &'a RocksStore,
issuer_ca_sha256: [u8; 32], issuer_ca_sha256: [u8; 32],
ca_validation_context_digest: [u8; 32], parent_context_digest: [u8; 32],
policy_fingerprint: [u8; 32], policy_fingerprint: [u8; 32],
} }
@@ -2065,7 +2066,7 @@ fn child_certificate_cache_key_sha256_hex(
child_cert_uri: &str, child_cert_uri: &str,
child_cert_sha256: &[u8; 32], child_cert_sha256: &[u8; 32],
issuer_ca_sha256: &[u8; 32], issuer_ca_sha256: &[u8; 32],
ca_validation_context_digest: &[u8; 32], parent_context_digest: &[u8; 32],
policy_fingerprint: &[u8; 32], policy_fingerprint: &[u8; 32],
) -> String { ) -> String {
let digest = hash_serialized_parts(&[ let digest = hash_serialized_parts(&[
@@ -2073,11 +2074,7 @@ fn child_certificate_cache_key_sha256_hex(
("child_cert_uri", child_cert_uri.as_bytes().to_vec()), ("child_cert_uri", child_cert_uri.as_bytes().to_vec()),
("child_cert_sha256", child_cert_sha256.to_vec()), ("child_cert_sha256", child_cert_sha256.to_vec()),
("issuer_ca_sha256", issuer_ca_sha256.to_vec()), ("issuer_ca_sha256", issuer_ca_sha256.to_vec()),
// Keep the persisted cache-key label stable; this change only renames Rust identifiers. ("parent_context_digest", parent_context_digest.to_vec()),
(
"parent_context_digest",
ca_validation_context_digest.to_vec(),
),
("policy_fingerprint", policy_fingerprint.to_vec()), ("policy_fingerprint", policy_fingerprint.to_vec()),
]); ]);
sha256_hex_from_32(&digest) sha256_hex_from_32(&digest)
@@ -2472,7 +2469,7 @@ fn discover_children_from_fresh_snapshot_with_audit_cached_with_issuer_der<
&f.rsync_uri, &f.rsync_uri,
&f.sha256, &f.sha256,
&cache.issuer_ca_sha256, &cache.issuer_ca_sha256,
&cache.ca_validation_context_digest, &cache.parent_context_digest,
&cache.policy_fingerprint, &cache.policy_fingerprint,
)); ));
} }
@@ -3025,7 +3022,7 @@ fn discover_children_from_fresh_snapshot_with_audit_cached_with_issuer_der<
&f.rsync_uri, &f.rsync_uri,
&f.sha256, &f.sha256,
&cache.issuer_ca_sha256, &cache.issuer_ca_sha256,
&cache.ca_validation_context_digest, &cache.parent_context_digest,
&cache.policy_fingerprint, &cache.policy_fingerprint,
); );
let projection = ChildCertificateCacheProjection { let projection = ChildCertificateCacheProjection {
@@ -3043,8 +3040,7 @@ fn discover_children_from_fresh_snapshot_with_audit_cached_with_issuer_der<
), ),
issuer_crl_uri: issuer_crl_uri.clone(), issuer_crl_uri: issuer_crl_uri.clone(),
issuer_crl_sha256_hex: verified_crl.sha256_hex.clone(), issuer_crl_sha256_hex: verified_crl.sha256_hex.clone(),
ca_validation_context_digest: cache parent_context_digest: cache.parent_context_digest,
.ca_validation_context_digest,
validation_policy_fingerprint: cache.policy_fingerprint, validation_policy_fingerprint: cache.policy_fingerprint,
effective_not_before, effective_not_before,
effective_until, effective_until,
@@ -3243,7 +3239,7 @@ fn discover_children_from_fresh_snapshot_with_audit_cached_with_issuer_der<
&f.rsync_uri, &f.rsync_uri,
&f.sha256, &f.sha256,
&cache.issuer_ca_sha256, &cache.issuer_ca_sha256,
&cache.ca_validation_context_digest, &cache.parent_context_digest,
&cache.policy_fingerprint, &cache.policy_fingerprint,
); );
let projection = ChildCertificateCacheProjection { let projection = ChildCertificateCacheProjection {
@@ -3259,7 +3255,7 @@ fn discover_children_from_fresh_snapshot_with_audit_cached_with_issuer_der<
issuer_ca_sha256_hex: sha256_hex_from_32(&cache.issuer_ca_sha256), issuer_ca_sha256_hex: sha256_hex_from_32(&cache.issuer_ca_sha256),
issuer_crl_uri: issuer_crl_uri.clone(), issuer_crl_uri: issuer_crl_uri.clone(),
issuer_crl_sha256_hex: verified_crl.sha256_hex.clone(), issuer_crl_sha256_hex: verified_crl.sha256_hex.clone(),
ca_validation_context_digest: cache.ca_validation_context_digest, parent_context_digest: cache.parent_context_digest,
validation_policy_fingerprint: cache.policy_fingerprint, validation_policy_fingerprint: cache.policy_fingerprint,
effective_not_before, effective_not_before,
effective_until, effective_until,
@@ -4399,7 +4395,7 @@ fn failed_fetch_reuse_identity_for_fresh_result(
let identity = VcirFailedFetchReuseIdentity { let identity = VcirFailedFetchReuseIdentity {
current_ca_sha256, current_ca_sha256,
ta_context_digest: ta_context_digest_for_ca(ca), ta_context_digest: ta_context_digest_for_ca(ca),
ca_validation_context_digest: ca_validation_context_digest_for_ca(ca), parent_context_digest: parent_context_digest_for_ca(ca),
policy_fingerprint: publication_point_cache_policy_fingerprint(policy), policy_fingerprint: publication_point_cache_policy_fingerprint(policy),
effective_not_before: PackTime::from_utc_offset_datetime(validation_time), effective_not_before: PackTime::from_utc_offset_datetime(validation_time),
effective_until, effective_until,
@@ -4422,7 +4418,7 @@ fn failed_fetch_reuse_identity_matches_current(
}; };
cached.current_ca_sha256 == current_ca_sha256 cached.current_ca_sha256 == current_ca_sha256
&& cached.ta_context_digest == ta_context_digest_for_ca(ca) && cached.ta_context_digest == ta_context_digest_for_ca(ca)
&& cached.ca_validation_context_digest == ca_validation_context_digest_for_ca(ca) && cached.parent_context_digest == parent_context_digest_for_ca(ca)
&& cached.policy_fingerprint == publication_point_cache_policy_fingerprint(policy) && cached.policy_fingerprint == publication_point_cache_policy_fingerprint(policy)
&& cached.effective_until == vcir.instance_gate.instance_effective_until && cached.effective_until == vcir.instance_gate.instance_effective_until
&& cached.contains_validation_time(validation_time) && cached.contains_validation_time(validation_time)
@@ -5351,7 +5347,7 @@ fn persist_vcir_for_fresh_result_with_timing(
.replace_vcir_manifest_replay_meta_and_projection_action_with_failed_fetch_reuse_identity( .replace_vcir_manifest_replay_meta_and_projection_action_with_failed_fetch_reuse_identity(
&vcir, &vcir,
Some(&RoaCacheProjectionContext { Some(&RoaCacheProjectionContext {
ca_validation_context_digest: ca_validation_context_digest_for_ca(ca), parent_context_digest: parent_context_digest_for_ca(ca),
policy_fingerprint: publication_point_cache_policy_fingerprint(policy), policy_fingerprint: publication_point_cache_policy_fingerprint(policy),
object_meta: objects.roa_cache_object_meta.clone(), object_meta: objects.roa_cache_object_meta.clone(),
}), }),
@@ -5382,7 +5378,7 @@ fn build_publication_point_cache_projection_from_fresh(
ca_cert_sha256, ca_cert_sha256,
manifest_sha256, manifest_sha256,
ta_context_digest_for_ca(ca), ta_context_digest_for_ca(ca),
ca_validation_context_digest_for_ca(ca), parent_context_digest_for_ca(ca),
publication_point_cache_policy_fingerprint(policy), publication_point_cache_policy_fingerprint(policy),
) )
.map_err(|e| e.to_string()) .map_err(|e| e.to_string())
@@ -5675,10 +5671,11 @@ fn build_vcir_ccr_manifest_projection_from_fresh(
.as_ref() .as_ref()
.ok_or_else(|| "manifest EE certificate missing Subject Information Access".to_string())? .ok_or_else(|| "manifest EE certificate missing Subject Information Access".to_string())?
{ {
SubjectInfoAccess::Ee(ee_sia) => vec![select_manifest_signed_object_location( SubjectInfoAccess::Ee(ee_sia) => ee_sia
&ca.manifest_rsync_uri, .access_descriptions
&ee_sia.access_descriptions, .iter()
)?], .map(encode_access_description_der_for_vcir_ccr_projection)
.collect::<Result<Vec<_>, _>>()?,
SubjectInfoAccess::Ca(_) => { SubjectInfoAccess::Ca(_) => {
return Err( return Err(
"manifest EE certificate Subject Information Access has CA variant".to_string(), "manifest EE certificate Subject Information Access has CA variant".to_string(),

View File

@@ -1,8 +1,5 @@
use super::*; use super::*;
use crate::data_model::oid::OID_AD_SIGNED_OBJECT; use crate::data_model::rc::{AsIdOrRange, AsIdentifierChoice, AsResourceSet, ResourceCertificate};
use crate::data_model::rc::{
AccessDescription, AsIdOrRange, AsIdentifierChoice, AsResourceSet, ResourceCertificate,
};
use crate::data_model::roa::RoaAfi; use crate::data_model::roa::RoaAfi;
use crate::fetch::rsync::LocalDirRsyncFetcher; use crate::fetch::rsync::LocalDirRsyncFetcher;
use crate::fetch::rsync::{RsyncFetchError, RsyncFetcher}; use crate::fetch::rsync::{RsyncFetchError, RsyncFetcher};
@@ -606,13 +603,10 @@ fn sample_vcir_for_projection(
manifest_ee_aki: vec![0x11; 20], manifest_ee_aki: vec![0x11; 20],
manifest_number_be: vec![1], manifest_number_be: vec![1],
manifest_this_update: PackTime::from_utc_offset_datetime(now), manifest_this_update: PackTime::from_utc_offset_datetime(now),
manifest_sia_locations_der: vec![ manifest_sia_locations_der: vec![vec![
crate::ccr::manifest_location::encode_access_description_der(&AccessDescription { 0x30, 0x11, 0x06, 0x08, 0x2b, 0x06, 0x01, 0x05, 0x05, 0x07, 0x30, 0x05, 0x86, 0x05,
access_method_oid: OID_AD_SIGNED_OBJECT.to_string(), b'r', b's', b'y', b'n', b'c',
access_location: manifest_uri.clone(), ]],
})
.expect("encode signedObject"),
],
subordinate_skis: vec![vec![0x33; 20]], subordinate_skis: vec![vec![0x33; 20]],
}; };
ValidatedCaInstanceResult { ValidatedCaInstanceResult {
@@ -1258,13 +1252,12 @@ fn build_vcir_ccr_manifest_projection_from_fresh_real_snapshot_matches_manifest_
.as_ref() .as_ref()
.expect("manifest sia") .expect("manifest sia")
{ {
SubjectInfoAccess::Ee(ee_sia) => vec![ SubjectInfoAccess::Ee(ee_sia) => ee_sia
crate::ccr::manifest_location::select_manifest_signed_object_location( .access_descriptions
&pack.manifest_rsync_uri, .iter()
&ee_sia.access_descriptions, .map(encode_access_description_der_for_vcir_ccr_projection)
) .collect::<Result<Vec<_>, _>>()
.expect("select manifest signedObject"), .expect("encode locations"),
],
SubjectInfoAccess::Ca(_) => panic!("manifest ee SIA should not be CA variant"), SubjectInfoAccess::Ca(_) => panic!("manifest ee SIA should not be CA variant"),
}; };
@@ -1608,7 +1601,7 @@ fn discover_children_child_certificate_cache_reuses_successful_child_ca() {
let cache_context = ChildCertificateValidationCacheContext { let cache_context = ChildCertificateValidationCacheContext {
store: &store, store: &store,
issuer_ca_sha256: issuer.ca_certificate_sha256_32().unwrap(), issuer_ca_sha256: issuer.ca_certificate_sha256_32().unwrap(),
ca_validation_context_digest: ca_validation_context_digest_for_ca(&issuer), parent_context_digest: parent_context_digest_for_ca(&issuer),
policy_fingerprint: publication_point_cache_policy_fingerprint(&policy), policy_fingerprint: publication_point_cache_policy_fingerprint(&policy),
}; };
let validation_time = time::OffsetDateTime::now_utc(); let validation_time = time::OffsetDateTime::now_utc();
@@ -1632,7 +1625,7 @@ fn discover_children_child_certificate_cache_reuses_successful_child_ca() {
&child_file.rsync_uri, &child_file.rsync_uri,
&child_file.sha256, &child_file.sha256,
&cache_context.issuer_ca_sha256, &cache_context.issuer_ca_sha256,
&cache_context.ca_validation_context_digest, &cache_context.parent_context_digest,
&cache_context.policy_fingerprint, &cache_context.policy_fingerprint,
); );
assert!( assert!(
@@ -1731,7 +1724,7 @@ fn discover_children_child_certificate_cache_rechecks_changed_valid_crl() {
let cache_context = ChildCertificateValidationCacheContext { let cache_context = ChildCertificateValidationCacheContext {
store: &store, store: &store,
issuer_ca_sha256: issuer.ca_certificate_sha256_32().unwrap(), issuer_ca_sha256: issuer.ca_certificate_sha256_32().unwrap(),
ca_validation_context_digest: ca_validation_context_digest_for_ca(&issuer), parent_context_digest: parent_context_digest_for_ca(&issuer),
policy_fingerprint: publication_point_cache_policy_fingerprint(&policy), policy_fingerprint: publication_point_cache_policy_fingerprint(&policy),
}; };
let validation_time = time::OffsetDateTime::now_utc(); let validation_time = time::OffsetDateTime::now_utc();
@@ -1752,7 +1745,7 @@ fn discover_children_child_certificate_cache_rechecks_changed_valid_crl() {
&child_file.rsync_uri, &child_file.rsync_uri,
&child_file.sha256, &child_file.sha256,
&cache_context.issuer_ca_sha256, &cache_context.issuer_ca_sha256,
&cache_context.ca_validation_context_digest, &cache_context.parent_context_digest,
&cache_context.policy_fingerprint, &cache_context.policy_fingerprint,
); );
let projection = store let projection = store
@@ -1839,7 +1832,7 @@ fn discover_children_child_certificate_cache_misses_when_current_crl_invalid() {
let cache_context = ChildCertificateValidationCacheContext { let cache_context = ChildCertificateValidationCacheContext {
store: &store, store: &store,
issuer_ca_sha256: issuer.ca_certificate_sha256_32().unwrap(), issuer_ca_sha256: issuer.ca_certificate_sha256_32().unwrap(),
ca_validation_context_digest: ca_validation_context_digest_for_ca(&issuer), parent_context_digest: parent_context_digest_for_ca(&issuer),
policy_fingerprint: publication_point_cache_policy_fingerprint(&policy), policy_fingerprint: publication_point_cache_policy_fingerprint(&policy),
}; };
let validation_time = time::OffsetDateTime::now_utc(); let validation_time = time::OffsetDateTime::now_utc();
@@ -1914,7 +1907,7 @@ fn discover_children_child_certificate_cache_misses_when_unchanged_crl_expired()
let cache_context = ChildCertificateValidationCacheContext { let cache_context = ChildCertificateValidationCacheContext {
store: &store, store: &store,
issuer_ca_sha256: issuer.ca_certificate_sha256_32().unwrap(), issuer_ca_sha256: issuer.ca_certificate_sha256_32().unwrap(),
ca_validation_context_digest: ca_validation_context_digest_for_ca(&issuer), parent_context_digest: parent_context_digest_for_ca(&issuer),
policy_fingerprint: publication_point_cache_policy_fingerprint(&policy), policy_fingerprint: publication_point_cache_policy_fingerprint(&policy),
}; };
let validation_time = time::OffsetDateTime::now_utc(); let validation_time = time::OffsetDateTime::now_utc();
@@ -2443,7 +2436,7 @@ fn seed_publication_point_cache_projection(
ca.ca_certificate_sha256_32().unwrap(), ca.ca_certificate_sha256_32().unwrap(),
sha256_32(b"manifest-bytes"), sha256_32(b"manifest-bytes"),
ta_context_digest_for_ca(ca), ta_context_digest_for_ca(ca),
ca_validation_context_digest_for_ca(ca), parent_context_digest_for_ca(ca),
publication_point_cache_policy_fingerprint(policy), publication_point_cache_policy_fingerprint(policy),
) )
.expect("build publication point projection"); .expect("build publication point projection");

View File

@@ -0,0 +1,78 @@
use crate::data_model::rc::AccessDescription;
pub(super) fn encode_access_description_der_for_vcir_ccr_projection(
access_description: &AccessDescription,
) -> Result<Vec<u8>, String> {
let oid = encode_oid_der_for_vcir_ccr_projection(&access_description.access_method_oid)?;
let uri = encode_tlv_for_vcir_ccr_projection(
0x86,
access_description.access_location.as_bytes().to_vec(),
);
Ok(encode_sequence_for_vcir_ccr_projection(&[oid, uri]))
}
fn encode_oid_der_for_vcir_ccr_projection(oid: &str) -> Result<Vec<u8>, String> {
let arcs = oid
.split('.')
.map(|part| {
part.parse::<u64>()
.map_err(|_| format!("unsupported accessMethod OID: {oid}"))
})
.collect::<Result<Vec<_>, _>>()?;
if arcs.len() < 2 {
return Err(format!("unsupported accessMethod OID: {oid}"));
}
if arcs[0] > 2 || (arcs[0] < 2 && arcs[1] >= 40) {
return Err(format!("unsupported accessMethod OID: {oid}"));
}
let mut body = Vec::new();
body.push((arcs[0] * 40 + arcs[1]) as u8);
for arc in &arcs[2..] {
encode_base128_for_vcir_ccr_projection(*arc, &mut body);
}
Ok(encode_tlv_for_vcir_ccr_projection(0x06, body))
}
fn encode_base128_for_vcir_ccr_projection(mut value: u64, out: &mut Vec<u8>) {
let mut tmp = vec![(value & 0x7F) as u8];
value >>= 7;
while value > 0 {
tmp.push(((value & 0x7F) as u8) | 0x80);
value >>= 7;
}
tmp.reverse();
out.extend_from_slice(&tmp);
}
fn encode_sequence_for_vcir_ccr_projection(elements: &[Vec<u8>]) -> Vec<u8> {
let total_len: usize = elements.iter().map(Vec::len).sum();
let mut buf = Vec::with_capacity(total_len);
for element in elements {
buf.extend_from_slice(element);
}
encode_tlv_for_vcir_ccr_projection(0x30, buf)
}
fn encode_tlv_for_vcir_ccr_projection(tag: u8, value: Vec<u8>) -> Vec<u8> {
let mut out = Vec::with_capacity(1 + 9 + value.len());
out.push(tag);
encode_length_for_vcir_ccr_projection(value.len(), &mut out);
out.extend_from_slice(&value);
out
}
fn encode_length_for_vcir_ccr_projection(len: usize, out: &mut Vec<u8>) {
if len < 0x80 {
out.push(len as u8);
return;
}
let mut bytes = Vec::new();
let mut value = len;
while value > 0 {
bytes.push((value & 0xFF) as u8);
value >>= 8;
}
bytes.reverse();
out.push(0x80 | (bytes.len() as u8));
out.extend_from_slice(&bytes);
}

File diff suppressed because it is too large Load Diff

View File

@@ -1,139 +0,0 @@
use rpki::ccr::{CcrAccumulator, build_roa_payload_state};
use rpki::data_model::roa::{IpPrefix, RoaAfi};
use rpki::storage::{PackTime, VcirCcrManifestProjection};
use rpki::validation::objects::Vrp;
const MANIFEST_URI: &str = "rsync://example.test/repo/current.mft";
const OID_AD_SIGNED_OBJECT: &[u8] = &[0x2b, 0x06, 0x01, 0x05, 0x05, 0x07, 0x30, 0x0b];
const OID_AD_RPKI_NOTIFY: &[u8] = &[0x2b, 0x06, 0x01, 0x05, 0x05, 0x07, 0x30, 0x0d];
fn der_length(len: usize) -> Vec<u8> {
if len < 0x80 {
return vec![len as u8];
}
let mut bytes = len.to_be_bytes().to_vec();
let first = bytes
.iter()
.position(|byte| *byte != 0)
.expect("non-zero length");
bytes.drain(..first);
let mut out = vec![0x80 | bytes.len() as u8];
out.extend_from_slice(&bytes);
out
}
fn der_tlv(tag: u8, value: &[u8]) -> Vec<u8> {
let mut out = vec![tag];
out.extend(der_length(value.len()));
out.extend_from_slice(value);
out
}
fn access_description_der(access_method_oid: &[u8], uri: &str) -> Vec<u8> {
let oid = der_tlv(0x06, access_method_oid);
let location = der_tlv(0x86, uri.as_bytes());
let mut body = oid;
body.extend(location);
der_tlv(0x30, &body)
}
fn projection(locations: Vec<Vec<u8>>) -> VcirCcrManifestProjection {
VcirCcrManifestProjection {
manifest_rsync_uri: MANIFEST_URI.to_string(),
manifest_sha256: vec![0x11; 32],
manifest_size: 1000,
manifest_ee_aki: vec![0x22; 20],
manifest_number_be: vec![1],
manifest_this_update: PackTime::from_utc_offset_datetime(time::OffsetDateTime::now_utc()),
manifest_sia_locations_der: locations,
subordinate_skis: vec![vec![0x33; 20]],
}
}
#[test]
fn historical_projection_normalizes_locations_and_accounts_memory() {
let signed_object = access_description_der(OID_AD_SIGNED_OBJECT, MANIFEST_URI);
let notify = access_description_der(
OID_AD_RPKI_NOTIFY,
"https://rrdp.example.test/notification.xml",
);
let projection = projection(vec![notify, signed_object]);
let mut accumulator = CcrAccumulator::new(Vec::new());
accumulator
.append_manifest_projection(&projection)
.expect("append normalized historical projection");
accumulator
.append_manifest_projection(&projection)
.expect("duplicate matching projection is idempotent");
assert_eq!(accumulator.manifest_count(), 1);
let stats = accumulator.memory_stats();
assert_eq!(stats.manifest_count, 1);
assert_eq!(stats.locations_der_count, 1);
assert_eq!(stats.subordinate_ski_count, 1);
assert!(stats.estimated_heap_bytes > 0);
assert!(stats.btree_entry_shallow_bytes > 0);
let mut conflicting = projection;
conflicting.subordinate_skis.push(vec![0x44; 20]);
let error = accumulator
.append_manifest_projection(&conflicting)
.expect_err("same hash with different projection must fail");
assert!(error.contains("duplicate manifest hash"), "{error}");
}
#[test]
fn historical_projection_rejects_invalid_location_sets() {
let mut accumulator = CcrAccumulator::new(Vec::new());
let empty_error = accumulator
.append_manifest_projection(&projection(Vec::new()))
.expect_err("empty location set must fail");
assert!(empty_error.contains("contains 0"), "{empty_error}");
let signed_object = access_description_der(OID_AD_SIGNED_OBJECT, MANIFEST_URI);
let duplicate_error = accumulator
.append_manifest_projection(&projection(vec![signed_object.clone(), signed_object]))
.expect_err("duplicate matching locations must fail");
assert!(duplicate_error.contains("contains 2"), "{duplicate_error}");
let malformed_error = accumulator
.append_manifest_projection(&projection(vec![vec![0x30, 0x01, 0x06]]))
.expect_err("malformed location DER must fail");
assert!(
malformed_error.contains("truncated DER"),
"{malformed_error}"
);
}
#[test]
fn roa_payload_builder_encodes_large_non_byte_aligned_family() {
let mut vrps = Vec::new();
for octet in 0..=255 {
vrps.push(Vrp {
asn: 64_496,
prefix: IpPrefix {
afi: RoaAfi::Ipv4,
prefix_len: 24,
addr: [10, octet, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
},
max_length: 24,
});
}
vrps.push(Vrp {
asn: 64_496,
prefix: IpPrefix {
afi: RoaAfi::Ipv4,
prefix_len: 13,
addr: [10, 0x1f, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0],
},
max_length: 24,
});
let state = build_roa_payload_state(&vrps).expect("build ROA payload state");
assert_eq!(state.rps.len(), 1);
assert_eq!(state.rps[0].as_id, 64_496);
assert_eq!(state.rps[0].ip_addr_blocks.len(), 1);
assert!(state.rps[0].ip_addr_blocks[0].len() > 128);
}

View File

@@ -253,7 +253,6 @@ fn tree_enqueues_children_for_fresh_and_current_instance_vcir_results() {
fn tree_respects_max_depth_and_max_instances() { fn tree_respects_max_depth_and_max_instances() {
let root_manifest = "rsync://example.test/repo/root.mft"; let root_manifest = "rsync://example.test/repo/root.mft";
let child_manifest = "rsync://example.test/repo/child.mft"; let child_manifest = "rsync://example.test/repo/child.mft";
let grandchild_manifest = "rsync://example.test/repo/grandchild.mft";
let runner = MockRunner::default() let runner = MockRunner::default()
.with( .with(
@@ -302,32 +301,6 @@ fn tree_respects_max_depth_and_max_instances() {
audit: PublicationPointAudit::default(), audit: PublicationPointAudit::default(),
cir_fresh_objects: Vec::new(), cir_fresh_objects: Vec::new(),
cir_cached_objects: Vec::new(), cir_cached_objects: Vec::new(),
discovered_children: vec![discovered_child(child_manifest, grandchild_manifest)],
},
)
.with(
grandchild_manifest,
PublicationPointRunResult {
source: PublicationPointSource::Fresh,
snapshot: Some(empty_snapshot(
grandchild_manifest,
"rsync://example.test/repo/grandchild/",
)),
warnings: Vec::new(),
objects: ObjectsOutput {
vrps: Vec::new(),
aspas: Vec::new(),
router_keys: Vec::new(),
local_outputs_cache: Vec::new(),
warnings: Vec::new(),
stats: ObjectsStats::default(),
audit: Vec::new(),
roa_cache_stats: Default::default(),
roa_cache_object_meta: Vec::new(),
},
audit: PublicationPointAudit::default(),
cir_fresh_objects: Vec::new(),
cir_cached_objects: Vec::new(),
discovered_children: Vec::new(), discovered_children: Vec::new(),
}, },
); );
@@ -351,62 +324,6 @@ fn tree_respects_max_depth_and_max_instances() {
.expect("run tree depth-limited"); .expect("run tree depth-limited");
assert_eq!(out.instances_processed, 1); assert_eq!(out.instances_processed, 1);
assert_eq!(out.instances_failed, 0); assert_eq!(out.instances_failed, 0);
assert_eq!(runner.called(), vec![root_manifest]);
let out = run_tree_serial(
ca_handle(root_manifest),
&runner,
&TreeRunConfig {
max_depth: Some(1),
max_instances: None,
compact_audit: false,
persist_vcir: true,
build_ccr_accumulator: true,
enable_roa_validation_cache: false,
enable_child_certificate_validation_cache: false,
publication_point_cache_observe_only: false,
enable_publication_point_validation_cache: false,
enable_transport_request_prefetch: false,
},
)
.expect("run tree depth-one-limited");
assert_eq!(out.instances_processed, 2);
assert_eq!(out.instances_failed, 0);
assert_eq!(
runner.called(),
vec![root_manifest, root_manifest, child_manifest]
);
let out = run_tree_serial(
ca_handle(root_manifest),
&runner,
&TreeRunConfig {
max_depth: Some(2),
max_instances: None,
compact_audit: false,
persist_vcir: true,
build_ccr_accumulator: true,
enable_roa_validation_cache: false,
enable_child_certificate_validation_cache: false,
publication_point_cache_observe_only: false,
enable_publication_point_validation_cache: false,
enable_transport_request_prefetch: false,
},
)
.expect("run tree depth-two-limited");
assert_eq!(out.instances_processed, 3);
assert_eq!(out.instances_failed, 0);
assert_eq!(
runner.called(),
vec![
root_manifest,
root_manifest,
child_manifest,
root_manifest,
child_manifest,
grandchild_manifest,
]
);
let out = run_tree_serial( let out = run_tree_serial(
ca_handle(root_manifest), ca_handle(root_manifest),

View File

@@ -1,117 +1,81 @@
# RPKI Explorer # RPKI Explorer
RPKI Explorer is the web UI for browsing the outputs of the ours RP RPKI Explorer is the SPA frontend for browsing ours RP `rpki_query_service` outputs.
`rpki_query_service` — runs, repositories, publication points, objects,
validation issues and evidence exports. It is a read-only query layer: it does
not change validation logic or RP state.
## Features ## Scope
- **Overview** — latest-run health at a glance: VRP/ASPA/object/PP counts, Current MVP covers:
validation result and object type charts, top repositories, top reject
reasons, all with drill-down links.
- **Repositories** — paginated repository list, repository detail with sync
phases/terminal states, publication points and scoped object browsing.
- **Publication Points** — global PP list and PP detail (URIs, sync state,
thisUpdate/nextUpdate, objects).
- **Objects** — global object browser with **server-side filters**
(`type`, `result`, `rejected`, `q` URI substring) and cursor paging.
- **Object detail** — structured parsed projections (ROA prefixes, ASPA
providers, manifest file list, CRL revocations, certificate fields),
validation summary, on-demand *explain* (audit projection, non-authoritative),
chain edges, raw DER download, object-set export.
- **Validation** — result distribution, reject-reason distribution with
click-to-filter, paginated issue list (`/issues`).
- **Search** — unified lookup: exact object URI, SHA-256 (full or ≥8 hex
prefix), repository host/URI substring, PP URI substring — plus **VRP
lookup**: enter an IPv4/IPv6 address or CIDR prefix to list the covering
VRPs (prefix queries use covering semantics: VRP prefix must cover the whole
queried prefix with `maxLength` ≥ queried length), with an optional ASN
filter and cursor paging. ASN-only queries are not supported by the API and
the page says so.
- **Exports** — export job list with live polling and tarball download.
- **Runs** — indexed run history; pin any run as the browsing context via the
run selector in the top bar (`?run=` URL param, deep-linkable everywhere).
- **API** — service health and endpoint reference.
## Backend requirements - Overview dashboard from latest run, stats, validation reasons, and top repositories.
- Repository -> publication point -> object lazy browser.
- Object detail with live object record, validation summary, lazy parsed/chain/list tabs, and validation explain.
- Exact URI lookup, raw download action, and object / publication point export actions.
`rpki_query_service` (this repository, `src/bin/rpki_query_service.rs`) with a Known backend-dependent limits:
query-db built by `rpki_query_indexer`. Some features need extra service flags:
| Feature | Requirement | - Parsed projection, raw download, and export success require `rpki_query_service --repo-bytes-db <path>`.
| --- | --- | - Without repo bytes, the UI shows explicit unavailable/error states instead of fabricating data.
| Parsed projections, raw download, exports | `--repo-bytes-db <path>` | - VRP IP/prefix lookup is not part of this MVP and is tracked separately.
| New runs appear automatically | `--watch-run-root <path>` | - Query service does not provide CORS headers; use a same-origin proxy.
The service has **no CORS headers and no authentication** — the UI must be
served same-origin with the API behind a proxy (dev: Vite proxy; prod: reverse
proxy). VRP lookup needs runs indexed at query-db schema version 2 (`#122`);
re-index older runs with `rpki_query_indexer` (or the service
`--watch-run-root` backfill), otherwise lookup results are empty.
## Development ## Development
Start the query service (example with a local query-db): Start query service first:
```bash ```bash
rpki_2/rpki/target/debug/rpki_query_service \ rpki_2/rpki/target/llvm-cov-target/debug/rpki_query_service \
--query-db /path/to/query-db \ --query-db .scratch/rpki_explorer_m3_api/query-db \
--repo-bytes-db /path/to/repo-bytes.db \ --listen 127.0.0.1:19571
--listen 127.0.0.1:9557
``` ```
Start the Vite dev server: Start the Vite dev server with a proxy:
```bash ```bash
cd rpki_2/rpki/ui/rpki-explorer cd rpki_2/rpki/ui/rpki-explorer
npm install npm install
RPKI_EXPLORER_API_TARGET=http://127.0.0.1:9557 npm run dev RPKI_EXPLORER_API_TARGET=http://127.0.0.1:19571 npm run dev -- --port 5173
``` ```
The dev server proxies `/api/v1/*` to `RPKI_EXPLORER_API_TARGET` The Vite dev server proxies `/api/v1/*` to `RPKI_EXPLORER_API_TARGET`; default target is `http://127.0.0.1:9557`.
(default `http://127.0.0.1:9557`) and serves the UI on `http://127.0.0.1:5173`.
## Production ## Production Build Preview
```bash ```bash
cd rpki_2/rpki/ui/rpki-explorer
npm run build npm run build
RPKI_EXPLORER_API_TARGET=http://127.0.0.1:9557 npm run preview # smoke-test the build RPKI_EXPLORER_API_TARGET=http://127.0.0.1:19571 npm run preview -- --port 4173
``` ```
For real deployments serve `dist/` with any static server / reverse proxy and For real deployment, serve `dist/` through a static server or reverse proxy and route `/api/v1/*` to `rpki_query_service` on the same origin.
forward `/api/v1/*` to `rpki_query_service` on the same origin, e.g. nginx:
```nginx
location /api/v1/ { proxy_pass http://127.0.0.1:9557; }
location / { root /srv/rpki-explorer/dist; try_files $uri /index.html; }
```
`try_files ... /index.html` (SPA fallback) is required because routing is
client-side.
## Validation ## Validation
```bash ```bash
npm run typecheck # tsc project references npm run typecheck
npm run lint # eslint npm run lint
npm test # vitest unit tests (lib + api contract) npm run build
npm run build # production build npm audit --audit-level=high
npm run test:e2e # Playwright, fully mocked backend (no service needed) npm run test:e2e
``` ```
E2E tests intercept every `/api/v1/**` request with a fixture-backed mock Playwright is intentionally configured with `workers: 1`. The current query service object endpoints use lazy `report.json` scans, so parallel browser tests can create artificial backend contention and false failures.
(`tests/e2e/mockApi.ts`), so they are deterministic and need no running query
service. They start the Vite dev server automatically (`workers: 1`).
## Directory layout ## Screenshot Evidence
Milestone screenshots are stored under:
```text ```text
src/api/ fetch client, zod schemas, typed endpoint functions specs/develop/20260617/m*_playwright/
src/app/ providers, router, react-query client ```
src/components/ shell, data table, pills, pager, tabs, object table, …
src/lib/ formatting, cursor pager state machine, run context, guards ## Directory Layout
src/pages/ one file per route
src/styles/ design tokens + base/shell/components/pages CSS ```text
tests/e2e/ Playwright specs + fixture-backed API mock src/api/ query-service client and TypeScript records
src/app/ app shell and top-level navigation state
src/components/layout/ sidebar/topbar shell
src/features/overview/ latest run overview dashboard
src/features/repositories repo -> PP -> object browser
src/features/object-detail live object detail and workflows
src/styles/globals.css MVP visual system
``` ```

View File

@@ -1,14 +1,17 @@
{ {
"name": "rpki-explorer", "name": "rpki-explorer",
"version": "1.0.0", "version": "0.1.0",
"lockfileVersion": 3, "lockfileVersion": 3,
"requires": true, "requires": true,
"packages": { "packages": {
"": { "": {
"name": "rpki-explorer", "name": "rpki-explorer",
"version": "1.0.0", "version": "0.1.0",
"dependencies": { "dependencies": {
"@tanstack/react-query": "^5.81.5", "@tanstack/react-query": "^5.81.5",
"@tanstack/react-table": "^8.21.3",
"@tanstack/react-virtual": "^3.13.12",
"@xyflow/react": "^12.8.2",
"lucide-react": "^0.468.0", "lucide-react": "^0.468.0",
"react": "^19.1.0", "react": "^19.1.0",
"react-dom": "^19.1.0", "react-dom": "^19.1.0",
@@ -19,6 +22,8 @@
"devDependencies": { "devDependencies": {
"@eslint/js": "^9.29.0", "@eslint/js": "^9.29.0",
"@playwright/test": "^1.53.1", "@playwright/test": "^1.53.1",
"@testing-library/jest-dom": "^6.6.3",
"@testing-library/react": "^16.3.0",
"@types/node": "^24.0.3", "@types/node": "^24.0.3",
"@types/react": "^19.1.8", "@types/react": "^19.1.8",
"@types/react-dom": "^19.1.6", "@types/react-dom": "^19.1.6",
@@ -33,6 +38,50 @@
"vitest": "^4.1.9" "vitest": "^4.1.9"
} }
}, },
"node_modules/@adobe/css-tools": {
"version": "4.5.0",
"resolved": "https://registry.npmjs.org/@adobe/css-tools/-/css-tools-4.5.0.tgz",
"integrity": "sha512-6OzddxPio9UiWTCemp4N8cYLV2ZN1ncRnV1cVGtve7dhPOtRkleRyx32GQCYSwDYgaHU3USMm84tNsvKzRCa1Q==",
"dev": true,
"license": "MIT"
},
"node_modules/@babel/code-frame": {
"version": "7.29.7",
"resolved": "https://registry.npmjs.org/@babel/code-frame/-/code-frame-7.29.7.tgz",
"integrity": "sha512-Aup7aUOfpbAUg2ROOJN6Iw5f9DMBlzu0mIkm/malLQFN/YQgO48wCj0Kxa3sEHJvPVFg7siR+qRInwXd2qhQKw==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@babel/helper-validator-identifier": "^7.29.7",
"js-tokens": "^4.0.0",
"picocolors": "^1.1.1"
},
"engines": {
"node": ">=6.9.0"
}
},
"node_modules/@babel/helper-validator-identifier": {
"version": "7.29.7",
"resolved": "https://registry.npmjs.org/@babel/helper-validator-identifier/-/helper-validator-identifier-7.29.7.tgz",
"integrity": "sha512-qehxGkRj55h/ff8EMaJ+cYhyaKlHIxqYDn682wQD7RNp9UujOQsHog2uS0r2vzr4pW+sXf90NeeayjcNaX3fFg==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=6.9.0"
}
},
"node_modules/@babel/runtime": {
"version": "7.29.7",
"resolved": "https://registry.npmjs.org/@babel/runtime/-/runtime-7.29.7.tgz",
"integrity": "sha512-Nq8OhGWiZIZGV6hLHoyAKLLcJihP/xFeBMGJoUrxTX2psI8dCifzLhZISFb+VWS3wFMRDmCGw5R+dOySCqPLhw==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=6.9.0"
}
},
"node_modules/@emnapi/core": { "node_modules/@emnapi/core": {
"version": "1.10.0", "version": "1.10.0",
"resolved": "https://registry.npmjs.org/@emnapi/core/-/core-1.10.0.tgz", "resolved": "https://registry.npmjs.org/@emnapi/core/-/core-1.10.0.tgz",
@@ -698,6 +747,142 @@
"react": "^18 || ^19" "react": "^18 || ^19"
} }
}, },
"node_modules/@tanstack/react-table": {
"version": "8.21.3",
"resolved": "https://registry.npmjs.org/@tanstack/react-table/-/react-table-8.21.3.tgz",
"integrity": "sha512-5nNMTSETP4ykGegmVkhjcS8tTLW6Vl4axfEGQN3v0zdHYbK4UfoqfPChclTrJ4EoK9QynqAu9oUf8VEmrpZ5Ww==",
"license": "MIT",
"dependencies": {
"@tanstack/table-core": "8.21.3"
},
"engines": {
"node": ">=12"
},
"funding": {
"type": "github",
"url": "https://github.com/sponsors/tannerlinsley"
},
"peerDependencies": {
"react": ">=16.8",
"react-dom": ">=16.8"
}
},
"node_modules/@tanstack/react-virtual": {
"version": "3.14.3",
"resolved": "https://registry.npmjs.org/@tanstack/react-virtual/-/react-virtual-3.14.3.tgz",
"integrity": "sha512-k/cnHPVaOfn46hSbiY6n4Dzf4QjCGWSF40zR5QIIYUqPAjpA6TN7InfYmcMiDVQGP2iUn9xsRbAl8u1v3UmeVQ==",
"license": "MIT",
"dependencies": {
"@tanstack/virtual-core": "3.17.1"
},
"funding": {
"type": "github",
"url": "https://github.com/sponsors/tannerlinsley"
},
"peerDependencies": {
"react": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0",
"react-dom": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
}
},
"node_modules/@tanstack/table-core": {
"version": "8.21.3",
"resolved": "https://registry.npmjs.org/@tanstack/table-core/-/table-core-8.21.3.tgz",
"integrity": "sha512-ldZXEhOBb8Is7xLs01fR3YEc3DERiz5silj8tnGkFZytt1abEvl/GhUmCE0PMLaMPTa3Jk4HbKmRlHmu+gCftg==",
"license": "MIT",
"engines": {
"node": ">=12"
},
"funding": {
"type": "github",
"url": "https://github.com/sponsors/tannerlinsley"
}
},
"node_modules/@tanstack/virtual-core": {
"version": "3.17.1",
"resolved": "https://registry.npmjs.org/@tanstack/virtual-core/-/virtual-core-3.17.1.tgz",
"integrity": "sha512-VZyW2Uiml5tmBZwPGrSD3Sz73OxzljQMCmzYHsUTPEuTsERf5xwa+uWb01xEzkz3ZSYTjj8NEb/mKHvgKxyZdA==",
"license": "MIT",
"funding": {
"type": "github",
"url": "https://github.com/sponsors/tannerlinsley"
}
},
"node_modules/@testing-library/dom": {
"version": "10.4.1",
"resolved": "https://registry.npmjs.org/@testing-library/dom/-/dom-10.4.1.tgz",
"integrity": "sha512-o4PXJQidqJl82ckFaXUeoAW+XysPLauYI43Abki5hABd853iMhitooc6znOnczgbTYmEP6U6/y1ZyKAIsvMKGg==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"@babel/code-frame": "^7.10.4",
"@babel/runtime": "^7.12.5",
"@types/aria-query": "^5.0.1",
"aria-query": "5.3.0",
"dom-accessibility-api": "^0.5.9",
"lz-string": "^1.5.0",
"picocolors": "1.1.1",
"pretty-format": "^27.0.2"
},
"engines": {
"node": ">=18"
}
},
"node_modules/@testing-library/jest-dom": {
"version": "6.9.1",
"resolved": "https://registry.npmjs.org/@testing-library/jest-dom/-/jest-dom-6.9.1.tgz",
"integrity": "sha512-zIcONa+hVtVSSep9UT3jZ5rizo2BsxgyDYU7WFD5eICBE7no3881HGeb/QkGfsJs6JTkY1aQhT7rIPC7e+0nnA==",
"dev": true,
"license": "MIT",
"dependencies": {
"@adobe/css-tools": "^4.4.0",
"aria-query": "^5.0.0",
"css.escape": "^1.5.1",
"dom-accessibility-api": "^0.6.3",
"picocolors": "^1.1.1",
"redent": "^3.0.0"
},
"engines": {
"node": ">=14",
"npm": ">=6",
"yarn": ">=1"
}
},
"node_modules/@testing-library/jest-dom/node_modules/dom-accessibility-api": {
"version": "0.6.3",
"resolved": "https://registry.npmjs.org/dom-accessibility-api/-/dom-accessibility-api-0.6.3.tgz",
"integrity": "sha512-7ZgogeTnjuHbo+ct10G9Ffp0mif17idi0IyWNVA/wcwcm7NPOD/WEHVP3n7n3MhXqxoIYm8d6MuZohYWIZ4T3w==",
"dev": true,
"license": "MIT"
},
"node_modules/@testing-library/react": {
"version": "16.3.2",
"resolved": "https://registry.npmjs.org/@testing-library/react/-/react-16.3.2.tgz",
"integrity": "sha512-XU5/SytQM+ykqMnAnvB2umaJNIOsLF3PVv//1Ew4CTcpz0/BRyy/af40qqrt7SjKpDdT1saBMc42CUok5gaw+g==",
"dev": true,
"license": "MIT",
"dependencies": {
"@babel/runtime": "^7.12.5"
},
"engines": {
"node": ">=18"
},
"peerDependencies": {
"@testing-library/dom": "^10.0.0",
"@types/react": "^18.0.0 || ^19.0.0",
"@types/react-dom": "^18.0.0 || ^19.0.0",
"react": "^18.0.0 || ^19.0.0",
"react-dom": "^18.0.0 || ^19.0.0"
},
"peerDependenciesMeta": {
"@types/react": {
"optional": true
},
"@types/react-dom": {
"optional": true
}
}
},
"node_modules/@tybys/wasm-util": { "node_modules/@tybys/wasm-util": {
"version": "0.10.2", "version": "0.10.2",
"resolved": "https://registry.npmjs.org/@tybys/wasm-util/-/wasm-util-0.10.2.tgz", "resolved": "https://registry.npmjs.org/@tybys/wasm-util/-/wasm-util-0.10.2.tgz",
@@ -709,6 +894,14 @@
"tslib": "^2.4.0" "tslib": "^2.4.0"
} }
}, },
"node_modules/@types/aria-query": {
"version": "5.0.4",
"resolved": "https://registry.npmjs.org/@types/aria-query/-/aria-query-5.0.4.tgz",
"integrity": "sha512-rfT93uj5s0PRL7EzccGMs3brplhcrghnDoV26NqKhCAS1hVo+WdNsPvE/yb6ilfr5hi2MEk6d5EWJTKdxg8jVw==",
"dev": true,
"license": "MIT",
"peer": true
},
"node_modules/@types/chai": { "node_modules/@types/chai": {
"version": "5.2.3", "version": "5.2.3",
"resolved": "https://registry.npmjs.org/@types/chai/-/chai-5.2.3.tgz", "resolved": "https://registry.npmjs.org/@types/chai/-/chai-5.2.3.tgz",
@@ -732,6 +925,15 @@
"integrity": "sha512-iO90scth9WAbmgv7ogoq57O9YpKmFBbmoEoCHDB2xMBY0+/KVrqAaCDyCE16dUspeOvIxFFRI+0sEtqDqy2b4A==", "integrity": "sha512-iO90scth9WAbmgv7ogoq57O9YpKmFBbmoEoCHDB2xMBY0+/KVrqAaCDyCE16dUspeOvIxFFRI+0sEtqDqy2b4A==",
"license": "MIT" "license": "MIT"
}, },
"node_modules/@types/d3-drag": {
"version": "3.0.7",
"resolved": "https://registry.npmjs.org/@types/d3-drag/-/d3-drag-3.0.7.tgz",
"integrity": "sha512-HE3jVKlzU9AaMazNufooRJ5ZpWmLIoc90A37WU2JMmeq28w1FQqCZswHZ3xR+SuxYftzHq6WU6KJHvqxKzTxxQ==",
"license": "MIT",
"dependencies": {
"@types/d3-selection": "*"
}
},
"node_modules/@types/d3-ease": { "node_modules/@types/d3-ease": {
"version": "3.0.2", "version": "3.0.2",
"resolved": "https://registry.npmjs.org/@types/d3-ease/-/d3-ease-3.0.2.tgz", "resolved": "https://registry.npmjs.org/@types/d3-ease/-/d3-ease-3.0.2.tgz",
@@ -762,6 +964,12 @@
"@types/d3-time": "*" "@types/d3-time": "*"
} }
}, },
"node_modules/@types/d3-selection": {
"version": "3.0.11",
"resolved": "https://registry.npmjs.org/@types/d3-selection/-/d3-selection-3.0.11.tgz",
"integrity": "sha512-bhAXu23DJWsrI45xafYpkQ4NtcKMwWnAC/vKrd2l+nxMFuvOT3XMYTIj2opv8vq8AO5Yh7Qac/nSeP/3zjTK0w==",
"license": "MIT"
},
"node_modules/@types/d3-shape": { "node_modules/@types/d3-shape": {
"version": "3.1.8", "version": "3.1.8",
"resolved": "https://registry.npmjs.org/@types/d3-shape/-/d3-shape-3.1.8.tgz", "resolved": "https://registry.npmjs.org/@types/d3-shape/-/d3-shape-3.1.8.tgz",
@@ -783,6 +991,25 @@
"integrity": "sha512-Ps3T8E8dZDam6fUyNiMkekK3XUsaUEik+idO9/YjPtfj2qruF8tFBXS7XhtE4iIXBLxhmLjP3SXpLhVf21I9Lw==", "integrity": "sha512-Ps3T8E8dZDam6fUyNiMkekK3XUsaUEik+idO9/YjPtfj2qruF8tFBXS7XhtE4iIXBLxhmLjP3SXpLhVf21I9Lw==",
"license": "MIT" "license": "MIT"
}, },
"node_modules/@types/d3-transition": {
"version": "3.0.9",
"resolved": "https://registry.npmjs.org/@types/d3-transition/-/d3-transition-3.0.9.tgz",
"integrity": "sha512-uZS5shfxzO3rGlu0cC3bjmMFKsXv+SmZZcgp0KD22ts4uGXp5EVYGzu/0YdwZeKmddhcAccYtREJKkPfXkZuCg==",
"license": "MIT",
"dependencies": {
"@types/d3-selection": "*"
}
},
"node_modules/@types/d3-zoom": {
"version": "3.0.8",
"resolved": "https://registry.npmjs.org/@types/d3-zoom/-/d3-zoom-3.0.8.tgz",
"integrity": "sha512-iqMC4/YlFCSlO8+2Ii1GGGliCAY4XdeG748w5vQUbevlbDu0zSjH/+jojorQVBK/se0j6DUFNPBGSqD3YWYnDw==",
"license": "MIT",
"dependencies": {
"@types/d3-interpolate": "*",
"@types/d3-selection": "*"
}
},
"node_modules/@types/deep-eql": { "node_modules/@types/deep-eql": {
"version": "4.0.2", "version": "4.0.2",
"resolved": "https://registry.npmjs.org/@types/deep-eql/-/deep-eql-4.0.2.tgz", "resolved": "https://registry.npmjs.org/@types/deep-eql/-/deep-eql-4.0.2.tgz",
@@ -828,7 +1055,7 @@
"version": "19.2.3", "version": "19.2.3",
"resolved": "https://registry.npmjs.org/@types/react-dom/-/react-dom-19.2.3.tgz", "resolved": "https://registry.npmjs.org/@types/react-dom/-/react-dom-19.2.3.tgz",
"integrity": "sha512-jp2L/eY6fn+KgVVQAOqYItbF0VY/YApe5Mz2F0aykSO8gx31bYCZyvSeYxCHKvzHG5eZjc+zyaS5BrBWya2+kQ==", "integrity": "sha512-jp2L/eY6fn+KgVVQAOqYItbF0VY/YApe5Mz2F0aykSO8gx31bYCZyvSeYxCHKvzHG5eZjc+zyaS5BrBWya2+kQ==",
"dev": true, "devOptional": true,
"license": "MIT", "license": "MIT",
"peerDependencies": { "peerDependencies": {
"@types/react": "^19.2.0" "@types/react": "^19.2.0"
@@ -1274,6 +1501,48 @@
"url": "https://opencollective.com/vitest" "url": "https://opencollective.com/vitest"
} }
}, },
"node_modules/@xyflow/react": {
"version": "12.11.0",
"resolved": "https://registry.npmjs.org/@xyflow/react/-/react-12.11.0.tgz",
"integrity": "sha512-na4IO33FSs2OS72hASgZDmTYwFAkef7Z74uBUVrong3ARmQQHfnRUVaCFn1kTt5LbS6pK03TbYjCPGLjLFfziA==",
"license": "MIT",
"dependencies": {
"@xyflow/system": "0.0.77",
"classcat": "^5.0.3",
"zustand": "^4.4.0"
},
"peerDependencies": {
"@types/react": ">=17",
"@types/react-dom": ">=17",
"react": ">=17",
"react-dom": ">=17"
},
"peerDependenciesMeta": {
"@types/react": {
"optional": true
},
"@types/react-dom": {
"optional": true
}
}
},
"node_modules/@xyflow/system": {
"version": "0.0.77",
"resolved": "https://registry.npmjs.org/@xyflow/system/-/system-0.0.77.tgz",
"integrity": "sha512-qCDCMCQAAgUu8yHnhloHG9F5mwPX5E+Wl8McpYIOPSSXfzFJJoZcwOcsDiAjitVKIg2de1WmJbCHfpcvxprsgg==",
"license": "MIT",
"dependencies": {
"@types/d3-drag": "^3.0.7",
"@types/d3-interpolate": "^3.0.4",
"@types/d3-selection": "^3.0.10",
"@types/d3-transition": "^3.0.8",
"@types/d3-zoom": "^3.0.8",
"d3-drag": "^3.0.0",
"d3-interpolate": "^3.0.1",
"d3-selection": "^3.0.0",
"d3-zoom": "^3.0.0"
}
},
"node_modules/acorn": { "node_modules/acorn": {
"version": "8.17.0", "version": "8.17.0",
"resolved": "https://registry.npmjs.org/acorn/-/acorn-8.17.0.tgz", "resolved": "https://registry.npmjs.org/acorn/-/acorn-8.17.0.tgz",
@@ -1314,6 +1583,17 @@
"url": "https://github.com/sponsors/epoberezkin" "url": "https://github.com/sponsors/epoberezkin"
} }
}, },
"node_modules/ansi-regex": {
"version": "5.0.1",
"resolved": "https://registry.npmjs.org/ansi-regex/-/ansi-regex-5.0.1.tgz",
"integrity": "sha512-quJQXlTSUGL2LH9SUXo8VwsY4soanhgo6LNSm84E1LBcE8s3O0wpdiRzyR9z/ZZJMlMWv37qOOb9pdJlMUEKFQ==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=8"
}
},
"node_modules/ansi-styles": { "node_modules/ansi-styles": {
"version": "4.3.0", "version": "4.3.0",
"resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.3.0.tgz", "resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-4.3.0.tgz",
@@ -1337,6 +1617,16 @@
"dev": true, "dev": true,
"license": "Python-2.0" "license": "Python-2.0"
}, },
"node_modules/aria-query": {
"version": "5.3.0",
"resolved": "https://registry.npmjs.org/aria-query/-/aria-query-5.3.0.tgz",
"integrity": "sha512-b0P0sZPKtyu8HkeRAfCq0IfURZK+SuwMjY1UXGBU27wpAiTwQAIlq56IbIO+ytk/JjS1fMR14ee5WBBfKi5J6A==",
"dev": true,
"license": "Apache-2.0",
"dependencies": {
"dequal": "^2.0.3"
}
},
"node_modules/assertion-error": { "node_modules/assertion-error": {
"version": "2.0.1", "version": "2.0.1",
"resolved": "https://registry.npmjs.org/assertion-error/-/assertion-error-2.0.1.tgz", "resolved": "https://registry.npmjs.org/assertion-error/-/assertion-error-2.0.1.tgz",
@@ -1402,6 +1692,12 @@
"url": "https://github.com/chalk/chalk?sponsor=1" "url": "https://github.com/chalk/chalk?sponsor=1"
} }
}, },
"node_modules/classcat": {
"version": "5.0.5",
"resolved": "https://registry.npmjs.org/classcat/-/classcat-5.0.5.tgz",
"integrity": "sha512-JhZUT7JFcQy/EzW605k/ktHtncoo9vnyW/2GspNYwFlN1C/WmjuV/xtS04e9SOkL2sTdw0VAZ2UGCcQ9lR6p6w==",
"license": "MIT"
},
"node_modules/clsx": { "node_modules/clsx": {
"version": "2.1.1", "version": "2.1.1",
"resolved": "https://registry.npmjs.org/clsx/-/clsx-2.1.1.tgz", "resolved": "https://registry.npmjs.org/clsx/-/clsx-2.1.1.tgz",
@@ -1473,6 +1769,13 @@
"node": ">= 8" "node": ">= 8"
} }
}, },
"node_modules/css.escape": {
"version": "1.5.1",
"resolved": "https://registry.npmjs.org/css.escape/-/css.escape-1.5.1.tgz",
"integrity": "sha512-YUifsXXuknHlUsmlgyY0PKzgPOr7/FjCePfHNt0jxm83wHZi44VDMQ7/fGNkjY3/jV1MC+1CmZbaHzugyeRtpg==",
"dev": true,
"license": "MIT"
},
"node_modules/csstype": { "node_modules/csstype": {
"version": "3.2.3", "version": "3.2.3",
"resolved": "https://registry.npmjs.org/csstype/-/csstype-3.2.3.tgz", "resolved": "https://registry.npmjs.org/csstype/-/csstype-3.2.3.tgz",
@@ -1501,6 +1804,28 @@
"node": ">=12" "node": ">=12"
} }
}, },
"node_modules/d3-dispatch": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/d3-dispatch/-/d3-dispatch-3.0.1.tgz",
"integrity": "sha512-rzUyPU/S7rwUflMyLc1ETDeBj0NRuHKKAcvukozwhshr6g6c5d8zh4c2gQjY2bZ0dXeGLWc1PF174P2tVvKhfg==",
"license": "ISC",
"engines": {
"node": ">=12"
}
},
"node_modules/d3-drag": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/d3-drag/-/d3-drag-3.0.0.tgz",
"integrity": "sha512-pWbUJLdETVA8lQNJecMxoXfH6x+mO2UQo8rSmZ+QqxcbyA3hfeprFgIT//HW2nlHChWeIIMwS2Fq+gEARkhTkg==",
"license": "ISC",
"dependencies": {
"d3-dispatch": "1 - 3",
"d3-selection": "3"
},
"engines": {
"node": ">=12"
}
},
"node_modules/d3-ease": { "node_modules/d3-ease": {
"version": "3.0.1", "version": "3.0.1",
"resolved": "https://registry.npmjs.org/d3-ease/-/d3-ease-3.0.1.tgz", "resolved": "https://registry.npmjs.org/d3-ease/-/d3-ease-3.0.1.tgz",
@@ -1556,6 +1881,15 @@
"node": ">=12" "node": ">=12"
} }
}, },
"node_modules/d3-selection": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/d3-selection/-/d3-selection-3.0.0.tgz",
"integrity": "sha512-fmTRWbNMmsmWq6xJV8D19U/gw/bwrHfNXxrIN+HfZgnzqTHp9jOmKMhsTUjXOJnZOdZY9Q28y4yebKzqDKlxlQ==",
"license": "ISC",
"engines": {
"node": ">=12"
}
},
"node_modules/d3-shape": { "node_modules/d3-shape": {
"version": "3.2.0", "version": "3.2.0",
"resolved": "https://registry.npmjs.org/d3-shape/-/d3-shape-3.2.0.tgz", "resolved": "https://registry.npmjs.org/d3-shape/-/d3-shape-3.2.0.tgz",
@@ -1601,6 +1935,41 @@
"node": ">=12" "node": ">=12"
} }
}, },
"node_modules/d3-transition": {
"version": "3.0.1",
"resolved": "https://registry.npmjs.org/d3-transition/-/d3-transition-3.0.1.tgz",
"integrity": "sha512-ApKvfjsSR6tg06xrL434C0WydLr7JewBB3V+/39RMHsaXTOG0zmt/OAXeng5M5LBm0ojmxJrpomQVZ1aPvBL4w==",
"license": "ISC",
"dependencies": {
"d3-color": "1 - 3",
"d3-dispatch": "1 - 3",
"d3-ease": "1 - 3",
"d3-interpolate": "1 - 3",
"d3-timer": "1 - 3"
},
"engines": {
"node": ">=12"
},
"peerDependencies": {
"d3-selection": "2 - 3"
}
},
"node_modules/d3-zoom": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/d3-zoom/-/d3-zoom-3.0.0.tgz",
"integrity": "sha512-b8AmV3kfQaqWAuacbPuNbL6vahnOJflOhexLzMMNLga62+/nh0JzvJ0aO/5a5MVgUFGS7Hu1P9P03o3fJkDCyw==",
"license": "ISC",
"dependencies": {
"d3-dispatch": "1 - 3",
"d3-drag": "2 - 3",
"d3-interpolate": "1 - 3",
"d3-selection": "2 - 3",
"d3-transition": "2 - 3"
},
"engines": {
"node": ">=12"
}
},
"node_modules/debug": { "node_modules/debug": {
"version": "4.4.3", "version": "4.4.3",
"resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz", "resolved": "https://registry.npmjs.org/debug/-/debug-4.4.3.tgz",
@@ -1632,6 +2001,16 @@
"dev": true, "dev": true,
"license": "MIT" "license": "MIT"
}, },
"node_modules/dequal": {
"version": "2.0.3",
"resolved": "https://registry.npmjs.org/dequal/-/dequal-2.0.3.tgz",
"integrity": "sha512-0je+qPKHEMohvfRTCEo3CrPG6cAzAYgmzKyxRiYSSDkS6eGJdyVJm7WaYA5ECaAD9wLB2T4EEeymA5aFVcYXCA==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=6"
}
},
"node_modules/detect-libc": { "node_modules/detect-libc": {
"version": "2.1.2", "version": "2.1.2",
"resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz", "resolved": "https://registry.npmjs.org/detect-libc/-/detect-libc-2.1.2.tgz",
@@ -1642,6 +2021,14 @@
"node": ">=8" "node": ">=8"
} }
}, },
"node_modules/dom-accessibility-api": {
"version": "0.5.16",
"resolved": "https://registry.npmjs.org/dom-accessibility-api/-/dom-accessibility-api-0.5.16.tgz",
"integrity": "sha512-X7BJ2yElsnOJ30pZF4uIIDfBEVgF4XEBxL9Bxhy6dnrm5hkzqmsWHGTiHqRiITNhMyFLyAiWndIJP7Z1NTteDg==",
"dev": true,
"license": "MIT",
"peer": true
},
"node_modules/es-module-lexer": { "node_modules/es-module-lexer": {
"version": "2.1.0", "version": "2.1.0",
"resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-2.1.0.tgz", "resolved": "https://registry.npmjs.org/es-module-lexer/-/es-module-lexer-2.1.0.tgz",
@@ -2063,6 +2450,16 @@
"node": ">=0.8.19" "node": ">=0.8.19"
} }
}, },
"node_modules/indent-string": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/indent-string/-/indent-string-4.0.0.tgz",
"integrity": "sha512-EdDDZu4A2OyIK7Lr/2zG+w5jmbuk1DVBnEwREQvBzspBJkCEbRa8GxU1lghYcaGJCnRWibjDXlq779X1/y5xwg==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=8"
}
},
"node_modules/internmap": { "node_modules/internmap": {
"version": "2.0.3", "version": "2.0.3",
"resolved": "https://registry.npmjs.org/internmap/-/internmap-2.0.3.tgz", "resolved": "https://registry.npmjs.org/internmap/-/internmap-2.0.3.tgz",
@@ -2102,6 +2499,14 @@
"dev": true, "dev": true,
"license": "ISC" "license": "ISC"
}, },
"node_modules/js-tokens": {
"version": "4.0.0",
"resolved": "https://registry.npmjs.org/js-tokens/-/js-tokens-4.0.0.tgz",
"integrity": "sha512-RdJUflcE3cUzKiMqQgsCu06FPu9UdIJO0beYbPhHN4k6apgJtifcoCtT9bcxOpYBtpD2kCM6Sbzg4CausW/PKQ==",
"dev": true,
"license": "MIT",
"peer": true
},
"node_modules/js-yaml": { "node_modules/js-yaml": {
"version": "4.2.0", "version": "4.2.0",
"resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.2.0.tgz", "resolved": "https://registry.npmjs.org/js-yaml/-/js-yaml-4.2.0.tgz",
@@ -2475,6 +2880,17 @@
"react": "^16.5.1 || ^17.0.0 || ^18.0.0 || ^19.0.0-rc" "react": "^16.5.1 || ^17.0.0 || ^18.0.0 || ^19.0.0-rc"
} }
}, },
"node_modules/lz-string": {
"version": "1.5.0",
"resolved": "https://registry.npmjs.org/lz-string/-/lz-string-1.5.0.tgz",
"integrity": "sha512-h5bgJWpxJNswbU7qCrV0tIKQCaS3blPDrqKWx+QxzuzL1zGUzij9XCWLrSLsJPu5t+eWA/ycetzYAO5IOMcWAQ==",
"dev": true,
"license": "MIT",
"peer": true,
"bin": {
"lz-string": "bin/bin.js"
}
},
"node_modules/magic-string": { "node_modules/magic-string": {
"version": "0.30.21", "version": "0.30.21",
"resolved": "https://registry.npmjs.org/magic-string/-/magic-string-0.30.21.tgz", "resolved": "https://registry.npmjs.org/magic-string/-/magic-string-0.30.21.tgz",
@@ -2485,6 +2901,16 @@
"@jridgewell/sourcemap-codec": "^1.5.5" "@jridgewell/sourcemap-codec": "^1.5.5"
} }
}, },
"node_modules/min-indent": {
"version": "1.0.1",
"resolved": "https://registry.npmjs.org/min-indent/-/min-indent-1.0.1.tgz",
"integrity": "sha512-I9jwMn07Sy/IwOj3zVkVik2JTvgpaykDZEigL6Rx6N9LbMywwUSMtxET+7lVoDLLd3O3IXwJwvuuns8UB/HeAg==",
"dev": true,
"license": "MIT",
"engines": {
"node": ">=4"
}
},
"node_modules/minimatch": { "node_modules/minimatch": {
"version": "3.1.5", "version": "3.1.5",
"resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.1.5.tgz", "resolved": "https://registry.npmjs.org/minimatch/-/minimatch-3.1.5.tgz",
@@ -2726,6 +3152,36 @@
"node": ">= 0.8.0" "node": ">= 0.8.0"
} }
}, },
"node_modules/pretty-format": {
"version": "27.5.1",
"resolved": "https://registry.npmjs.org/pretty-format/-/pretty-format-27.5.1.tgz",
"integrity": "sha512-Qb1gy5OrP5+zDf2Bvnzdl3jsTf1qXVMazbvCoKhtKqVs4/YK4ozX4gKQJJVyNe+cajNPn0KoC0MC3FUmaHWEmQ==",
"dev": true,
"license": "MIT",
"peer": true,
"dependencies": {
"ansi-regex": "^5.0.1",
"ansi-styles": "^5.0.0",
"react-is": "^17.0.1"
},
"engines": {
"node": "^10.13.0 || ^12.13.0 || ^14.15.0 || >=15.0.0"
}
},
"node_modules/pretty-format/node_modules/ansi-styles": {
"version": "5.2.0",
"resolved": "https://registry.npmjs.org/ansi-styles/-/ansi-styles-5.2.0.tgz",
"integrity": "sha512-Cxwpt2SfTzTtXcfOlzGEee8O+c+MmUgGrNiBcXnuWxuFJHe6a5Hz7qwhwe5OgaSYI0IJvkLqWX1ASG+cJOkEiA==",
"dev": true,
"license": "MIT",
"peer": true,
"engines": {
"node": ">=10"
},
"funding": {
"url": "https://github.com/chalk/ansi-styles?sponsor=1"
}
},
"node_modules/punycode": { "node_modules/punycode": {
"version": "2.3.1", "version": "2.3.1",
"resolved": "https://registry.npmjs.org/punycode/-/punycode-2.3.1.tgz", "resolved": "https://registry.npmjs.org/punycode/-/punycode-2.3.1.tgz",
@@ -2855,6 +3311,20 @@
"react-is": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0" "react-is": "^16.8.0 || ^17.0.0 || ^18.0.0 || ^19.0.0"
} }
}, },
"node_modules/redent": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/redent/-/redent-3.0.0.tgz",
"integrity": "sha512-6tDA8g98We0zd0GvVeMT9arEOnTw9qM03L9cJXaCjrip1OO764RDBLBfrB4cwzNGDj5OA5ioymC9GkizgWJDUg==",
"dev": true,
"license": "MIT",
"dependencies": {
"indent-string": "^4.0.0",
"strip-indent": "^3.0.0"
},
"engines": {
"node": ">=8"
}
},
"node_modules/redux": { "node_modules/redux": {
"version": "5.0.1", "version": "5.0.1",
"resolved": "https://registry.npmjs.org/redux/-/redux-5.0.1.tgz", "resolved": "https://registry.npmjs.org/redux/-/redux-5.0.1.tgz",
@@ -2986,6 +3456,19 @@
"dev": true, "dev": true,
"license": "MIT" "license": "MIT"
}, },
"node_modules/strip-indent": {
"version": "3.0.0",
"resolved": "https://registry.npmjs.org/strip-indent/-/strip-indent-3.0.0.tgz",
"integrity": "sha512-laJTa3Jb+VQpaC6DseHhF7dXVqHTfJPCRDaEbid/drOhgitgYku/letMUqOXFoWV0zIIUbjpdH2t+tYj4bQMRQ==",
"dev": true,
"license": "MIT",
"dependencies": {
"min-indent": "^1.0.0"
},
"engines": {
"node": ">=8"
}
},
"node_modules/strip-json-comments": { "node_modules/strip-json-comments": {
"version": "3.1.1", "version": "3.1.1",
"resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-3.1.1.tgz", "resolved": "https://registry.npmjs.org/strip-json-comments/-/strip-json-comments-3.1.1.tgz",
@@ -3429,6 +3912,34 @@
"funding": { "funding": {
"url": "https://github.com/sponsors/colinhacks" "url": "https://github.com/sponsors/colinhacks"
} }
},
"node_modules/zustand": {
"version": "4.5.7",
"resolved": "https://registry.npmjs.org/zustand/-/zustand-4.5.7.tgz",
"integrity": "sha512-CHOUy7mu3lbD6o6LJLfllpjkzhHXSBlX8B9+qPddUsIfeF5S/UZ5q0kmCsnRqT1UHFQZchNFDDzMbQsuesHWlw==",
"license": "MIT",
"dependencies": {
"use-sync-external-store": "^1.2.2"
},
"engines": {
"node": ">=12.7.0"
},
"peerDependencies": {
"@types/react": ">=16.8",
"immer": ">=9.0.6",
"react": ">=16.8"
},
"peerDependenciesMeta": {
"@types/react": {
"optional": true
},
"immer": {
"optional": true
},
"react": {
"optional": true
}
}
} }
} }
} }

View File

@@ -1,6 +1,6 @@
{ {
"name": "rpki-explorer", "name": "rpki-explorer",
"version": "1.0.0", "version": "0.1.0",
"private": true, "private": true,
"type": "module", "type": "module",
"scripts": { "scripts": {
@@ -9,11 +9,13 @@
"typecheck": "tsc -b --pretty false", "typecheck": "tsc -b --pretty false",
"lint": "eslint .", "lint": "eslint .",
"preview": "vite preview --host 127.0.0.1", "preview": "vite preview --host 127.0.0.1",
"test": "vitest run",
"test:e2e": "playwright test" "test:e2e": "playwright test"
}, },
"dependencies": { "dependencies": {
"@tanstack/react-query": "^5.81.5", "@tanstack/react-query": "^5.81.5",
"@tanstack/react-table": "^8.21.3",
"@tanstack/react-virtual": "^3.13.12",
"@xyflow/react": "^12.8.2",
"lucide-react": "^0.468.0", "lucide-react": "^0.468.0",
"react": "^19.1.0", "react": "^19.1.0",
"react-dom": "^19.1.0", "react-dom": "^19.1.0",
@@ -24,6 +26,8 @@
"devDependencies": { "devDependencies": {
"@eslint/js": "^9.29.0", "@eslint/js": "^9.29.0",
"@playwright/test": "^1.53.1", "@playwright/test": "^1.53.1",
"@testing-library/jest-dom": "^6.6.3",
"@testing-library/react": "^16.3.0",
"@types/node": "^24.0.3", "@types/node": "^24.0.3",
"@types/react": "^19.1.8", "@types/react": "^19.1.8",
"@types/react-dom": "^19.1.6", "@types/react-dom": "^19.1.6",

View File

@@ -3,24 +3,18 @@ import { defineConfig, devices } from "@playwright/test";
export default defineConfig({ export default defineConfig({
testDir: "./tests/e2e", testDir: "./tests/e2e",
outputDir: "./test-results", outputDir: "./test-results",
fullyParallel: false, fullyParallel: true,
workers: 1, workers: 1,
reporter: [["list"], ["html", { open: "never" }]], reporter: [["list"], ["html", { open: "never" }]],
use: { use: {
baseURL: process.env.PLAYWRIGHT_BASE_URL ?? "http://127.0.0.1:5173", baseURL: process.env.PLAYWRIGHT_BASE_URL ?? "http://127.0.0.1:5173",
trace: "retain-on-failure", trace: "retain-on-failure",
screenshot: "only-on-failure", screenshot: "only-on-failure"
},
webServer: {
command: "npm run dev -- --port 5173 --strictPort",
url: "http://127.0.0.1:5173",
reuseExistingServer: !process.env.CI,
timeout: 60_000,
}, },
projects: [ projects: [
{ {
name: "chromium", name: "chromium",
use: { ...devices["Desktop Chrome"] }, use: { ...devices["Desktop Chrome"] }
}, }
], ]
}); });

View File

@@ -1,102 +0,0 @@
import { describe, expect, it } from "vitest";
import { z } from "zod";
import { withQuery } from "./client";
import {
envelopeSchema,
objectInstanceRecordSchema,
runRecordSchema,
searchResultSchema,
} from "./schemas";
describe("withQuery", () => {
it("builds a query string skipping empty values", () => {
expect(
withQuery("/api/v1/runs/latest/objects", {
limit: 50,
cursor: null,
type: "roa",
rejected: false,
q: "",
reason: undefined,
}),
).toBe("/api/v1/runs/latest/objects?limit=50&type=roa&rejected=false");
});
it("URL-encodes values", () => {
expect(withQuery("/x", { uri: "rsync://a/b.cer" })).toBe(
"/x?uri=rsync%3A%2F%2Fa%2Fb.cer",
);
});
it("returns the bare path without params", () => {
expect(withQuery("/x")).toBe("/x");
expect(withQuery("/x", {})).toBe("/x");
});
});
describe("envelope + record schemas", () => {
const objectInstance = {
objectInstanceId: "obj-1",
uri: "rsync://repo.example/ca/roa.roa",
sha256: "ab".repeat(32),
objectType: "roa",
result: "ok",
repoId: "repo-1",
ppId: "pp-1",
sourceSection: "fresh",
rejected: false,
};
it("parses a list envelope and exposes the cursor", () => {
const parsed = envelopeSchema(objectInstanceRecordSchema.array()).parse({
data: [objectInstance],
page: { nextCursor: "r1:0:50", limit: 50 },
meta: { runId: "run_0002", schemaVersion: 1 },
});
expect(parsed.data).toHaveLength(1);
expect(parsed.page?.nextCursor).toBe("r1:0:50");
});
it("accepts a null meta.runId (healthz and other run-less endpoints)", () => {
const parsed = envelopeSchema(z.object({ status: z.string() })).parse({
data: { status: "ok" },
page: null,
meta: { runId: null, schemaVersion: 1 },
});
expect(parsed.data.status).toBe("ok");
expect(parsed.meta?.runId).toBeNull();
});
it("keeps unknown fields (passthrough) for forward compatibility", () => {
const parsed = objectInstanceRecordSchema.parse({
...objectInstance,
futureField: { nested: true },
});
expect((parsed as Record<string, unknown>).futureField).toEqual({ nested: true });
});
it("rejects records missing required identity fields", () => {
expect(() =>
objectInstanceRecordSchema.parse({ uri: "rsync://x" }),
).toThrow();
});
it("parses run records with partial counts", () => {
const run = runRecordSchema.parse({
runId: "run_0002",
runSeq: 2,
counts: { objects: 10 },
});
expect(run.counts?.objects).toBe(10);
expect(run.counts?.vrps).toBeUndefined();
});
it("parses the unified search result", () => {
const result = searchResultSchema.parse({
objects: [objectInstance],
repos: [],
publicationPoints: [],
});
expect(result.objects[0]?.objectInstanceId).toBe("obj-1");
});
});

View File

@@ -1,101 +1,101 @@
/** export interface ApiEnvelope<T> {
* Low-level fetch helpers for the rpki_query_service HTTP API. data: T;
* page: { nextCursor: string | null; limit: number } | null;
* All JSON endpoints share the envelope shape: meta: { runId: string | null; schemaVersion: number };
* { "data": ..., "page": { "nextCursor": string | null, "limit": number } | null, }
* "meta": { "runId": string, "schemaVersion": number } }
* Errors are `{ "error": string }` with a non-2xx status.
*/
export class ApiError extends Error { export class ApiError extends Error {
readonly status: number; constructor(
message: string,
constructor(status: number, message: string) { readonly status: number
) {
super(message); super(message);
this.name = "ApiError"; this.name = "ApiError";
this.status = status;
} }
} }
export type QueryParams = Record< export type JsonQuery = Record<string, string | number | boolean | null | undefined>;
string,
string | number | boolean | null | undefined
>;
/** Build a URL with a query string, skipping empty values. */ export function withQuery(path: string, query?: JsonQuery): string {
export function withQuery(path: string, params?: QueryParams): string { if (!query) {
if (!params) return path; return path;
}
const search = new URLSearchParams(); const search = new URLSearchParams();
for (const [key, value] of Object.entries(params)) { for (const [key, value] of Object.entries(query)) {
if (value === undefined || value === null || value === "") continue; if (value === undefined || value === null) {
continue;
}
search.set(key, String(value)); search.set(key, String(value));
} }
const qs = search.toString(); const queryString = search.toString();
return qs ? `${path}?${qs}` : path; return queryString ? `${path}?${queryString}` : path;
} }
async function parseErrorBody(res: Response): Promise<string> { export async function getJson<T>(path: string, query?: JsonQuery): Promise<ApiEnvelope<T>> {
try { const response = await fetch(withQuery(path, query));
const body: unknown = await res.json(); const body = (await response.json().catch(() => ({}))) as { error?: string };
if ( if (!response.ok) {
body !== null && throw new ApiError(body.error ?? `Request failed: ${response.status}`, response.status);
typeof body === "object" &&
"error" in body &&
typeof (body as { error: unknown }).error === "string"
) {
return (body as { error: string }).error;
} }
} catch { return normalizeEnvelope<T>(body);
// fall through to status text
}
return `HTTP ${res.status} ${res.statusText}`.trim();
} }
/** Fetch JSON from the API, throwing ApiError on non-2xx. */ export async function postJson<T>(path: string, body?: unknown): Promise<ApiEnvelope<T>> {
export async function apiFetch<T>(path: string, init?: RequestInit): Promise<T> { const response = await fetch(path, {
let res: Response;
try {
res = await fetch(path, init);
} catch (err) {
throw new ApiError(0, err instanceof Error ? err.message : "network error");
}
if (!res.ok) {
throw new ApiError(res.status, await parseErrorBody(res));
}
return (await res.json()) as Promise<T>;
}
/** Fetch binary content (raw object bytes, export tarballs). */
export async function apiFetchBlob(path: string): Promise<Blob> {
let res: Response;
try {
res = await fetch(path);
} catch (err) {
throw new ApiError(0, err instanceof Error ? err.message : "network error");
}
if (!res.ok) {
throw new ApiError(res.status, await parseErrorBody(res));
}
return res.blob();
}
/** POST a JSON body, returning the parsed JSON response. */
export async function apiPost<T>(path: string, body: unknown): Promise<T> {
return apiFetch<T>(path, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify(body ?? {}), body: JSON.stringify(body ?? {}),
headers: { "content-type": "application/json" },
method: "POST"
}); });
const responseBody = (await response.json().catch(() => ({}))) as { error?: string };
if (!response.ok) {
throw new ApiError(responseBody.error ?? `Request failed: ${response.status}`, response.status);
}
return normalizeEnvelope<T>(responseBody);
} }
/** Trigger a browser download for a blob. */ export async function getBinary(path: string): Promise<Blob> {
export function saveBlob(blob: Blob, filename: string): void { const response = await fetch(path);
const url = URL.createObjectURL(blob); if (!response.ok) {
const anchor = document.createElement("a"); const body = (await response.json().catch(() => ({}))) as { error?: string };
anchor.href = url; throw new ApiError(body.error ?? `Request failed: ${response.status}`, response.status);
anchor.download = filename; }
document.body.appendChild(anchor); return response.blob();
anchor.click(); }
anchor.remove();
URL.revokeObjectURL(url); export function normalizeEnvelope<T>(value: unknown): ApiEnvelope<T> {
if (typeof value !== "object" || value === null) {
return {
data: value as T,
page: null,
meta: { runId: null, schemaVersion: 1 }
};
}
const record = value as {
data?: T;
page?: { nextCursor?: string | null; limit?: number } | null;
meta?: { runId?: string | null; schemaVersion?: number } | null;
};
return {
data: record.data as T,
page: record.page
? {
nextCursor: record.page.nextCursor ?? null,
limit: record.page.limit ?? 0
}
: null,
meta: {
runId: record.meta?.runId ?? null,
schemaVersion: record.meta?.schemaVersion ?? 1
}
};
}
export function normalizeApiError(error: unknown): Error {
if (error instanceof ApiError) {
return error;
}
if (error instanceof Error) {
return error;
}
return new Error("Unknown API error");
} }

View File

@@ -0,0 +1,301 @@
import { getBinary, getJson, postJson } from "./client";
export interface ServiceInfo {
service: string;
version: number;
}
export interface RunRecord {
schemaVersion: number;
runId: string;
runSeq: number | null;
runDir: string;
validationTime: string | null;
syncMode: string | null;
startedAt: string | null;
finishedAt: string | null;
wallMs: number | null;
artifactPaths: Record<string, string>;
counts: {
publicationPoints: number;
objects: number;
freshObjects: number;
cachedObjects: number;
rejectedObjects: number;
freshRejectedObjects: number;
cachedRejectedObjects: number;
trustAnchors: number;
vrps: number;
aspas: number;
warnings: number;
};
indexStatus: string;
indexError: string | null;
}
export interface RepositoryRecord {
schemaVersion: number;
runId: string;
repoId: string;
uri: string;
host: string;
transport: string;
publicationPoints: number;
objects: number;
rejectedObjects: number;
downloadBytes: number | null;
syncDurationMsTotal: number;
phases: Record<string, number>;
terminalStates: Record<string, number>;
}
export interface PublicationPointRecord {
schemaVersion: number;
runId: string;
ppId: string;
repoId: string;
nodeId: number | null;
parentNodeId: number | null;
rsyncBaseUri: string | null;
manifestRsyncUri: string | null;
publicationPointRsyncUri: string | null;
rrdpNotificationUri: string | null;
source: string | null;
repoSyncSource: string | null;
repoSyncPhase: string | null;
repoSyncDurationMs: number | null;
repoSyncError: string | null;
repoTerminalState: string | null;
thisUpdate: string | null;
nextUpdate: string | null;
verifiedAt: string | null;
objects: number;
rejectedObjects: number;
warnings: number;
}
export interface ObjectInstanceRecord {
schemaVersion: number;
runId: string;
objectInstanceId: string;
uri: string;
uriHash: string;
sha256: string;
objectType: string;
result: string;
detailSummary: string | null;
repoId: string;
ppId: string;
sourceSection: string;
rejected: boolean;
rejectReason: string | null;
}
export interface ObjectProjectionRecord {
schemaVersion?: number;
objectType?: string;
sha256?: string;
uri?: string;
parseStatus?: string;
errorSummary?: string | null;
projection?: unknown;
}
export interface ValidationIssueRecord {
stage?: string;
severity?: string;
reasonCode?: string;
summary?: string;
}
export interface ObjectValidationRecord {
objectInstanceId: string;
uri: string;
sha256: string;
objectType: string;
finalStatus: string;
auditResult: string;
detailSummary: string | null;
rejected: boolean;
rejectReason: string | null;
sourceSection: string;
explainAvailable: boolean;
authoritative: boolean;
fileValidation: {
status: string;
stage: string;
issues: ValidationIssueRecord[];
detailSummary: string | null;
};
chainValidation: {
status: string;
stage: string;
issues: ValidationIssueRecord[];
note?: string;
edgesPage?: { nextCursor: string | null; limit: number };
};
}
export interface ChainEdgeRecord {
relation: string;
fromUri: string;
toUri: string;
toObjectInstanceId: string | null;
toSha256: string | null;
status: string;
evidence: unknown;
}
export interface ValidationExplainRecord {
schemaVersion: number;
explainVersion: number;
runId: string;
objectInstanceId: string;
uri: string;
sha256: string;
objectType: string;
finalStatus: string;
auditResult: string;
detailSummary: string | null;
authoritative: boolean;
explainMode: string;
generatedAt: string;
parsevalidate: unknown;
chainvalidate: unknown;
chainEdges: ChainEdgeRecord[];
}
export interface ObjectUriIndexRecord {
runId: string;
uri: string;
sha256: string;
objectInstanceId: string;
repoId: string;
ppId: string;
}
export interface ExportJobRecord {
schemaVersion: number;
jobId: string;
runId: string;
scope: string;
repoId: string | null;
ppId: string | null;
status: string;
createdAt: string;
finishedAt: string | null;
outputPath: string | null;
objectCount: number;
bytesWritten: number;
error: string | null;
}
export type CountsByKey = Record<string, number>;
export async function getServiceInfo() {
return getJson<ServiceInfo>("/api/v1");
}
export async function getLatestRun() {
return getJson<RunRecord>("/api/v1/latest_run");
}
export async function getRunSummary(runId: string) {
return getJson<Record<string, number>>(`/api/v1/runs/${runId}/summary`);
}
export async function getStatsObjectTypes(runId: string) {
return getJson<CountsByKey>(`/api/v1/runs/${runId}/stats/object-types`);
}
export async function getStatsValidation(runId: string) {
return getJson<CountsByKey>(`/api/v1/runs/${runId}/stats/validation`);
}
export async function getStatsReasons(runId: string) {
return getJson<CountsByKey>(`/api/v1/runs/${runId}/stats/reasons`);
}
export async function getStatsDownloads(runId: string) {
return getJson<Record<string, unknown>>(`/api/v1/runs/${runId}/stats/downloads`);
}
export async function listRepos(runId: string, limit = 5, cursor?: string | null) {
return getJson<RepositoryRecord[]>(`/api/v1/runs/${runId}/repos`, { limit, cursor: cursor ?? undefined });
}
export async function listPublicationPointsForRepo(runId: string, repoId: string, limit = 50, cursor?: string | null) {
return getJson<PublicationPointRecord[]>(`/api/v1/runs/${runId}/repos/${repoId}/publication-points`, {
limit,
cursor: cursor ?? undefined
});
}
export async function listObjectsForPublicationPoint(runId: string, ppId: string, limit = 50, cursor?: string | null) {
return getJson<ObjectInstanceRecord[]>(`/api/v1/runs/${runId}/publication-points/${ppId}/objects`, {
limit,
cursor: cursor ?? undefined
});
}
export async function getObject(runId: string, objectInstanceId: string) {
return getJson<ObjectInstanceRecord>(`/api/v1/runs/${runId}/objects/${objectInstanceId}`);
}
export async function getObjectByUri(runId: string, uri: string) {
return getJson<ObjectUriIndexRecord>(`/api/v1/runs/${runId}/objects/by-uri`, { uri });
}
export async function getObjectParsed(runId: string, objectInstanceId: string) {
return getJson<ObjectProjectionRecord | null>(`/api/v1/runs/${runId}/objects/${objectInstanceId}/parsed`);
}
export async function getObjectValidation(runId: string, objectInstanceId: string) {
return getJson<ObjectValidationRecord>(`/api/v1/runs/${runId}/objects/${objectInstanceId}/validation`);
}
export async function getObjectChain(runId: string, objectInstanceId: string) {
return getJson<ChainEdgeRecord[]>(`/api/v1/runs/${runId}/objects/${objectInstanceId}/chain`);
}
export async function listManifestFiles(runId: string, objectInstanceId: string, limit = 50, cursor?: string | null) {
return getJson<unknown[]>(`/api/v1/runs/${runId}/objects/${objectInstanceId}/parsed/manifest-files`, {
limit,
cursor: cursor ?? undefined
});
}
export async function listRevokedCertificates(runId: string, objectInstanceId: string, limit = 50, cursor?: string | null) {
return getJson<unknown[]>(`/api/v1/runs/${runId}/objects/${objectInstanceId}/parsed/revoked-certs`, {
limit,
cursor: cursor ?? undefined
});
}
export async function explainObjectValidation(runId: string, objectInstanceId: string, forceRefresh = false) {
return postJson<ValidationExplainRecord>(`/api/v1/runs/${runId}/objects/${objectInstanceId}/validation/explain`, {
forceRefresh
});
}
export async function downloadRawObject(runId: string, objectInstanceId: string) {
return getBinary(`/api/v1/runs/${runId}/objects/${objectInstanceId}/raw`);
}
export async function createObjectSetExport(runId: string, objectInstanceIds: string[]) {
return postJson<ExportJobRecord>(`/api/v1/runs/${runId}/exports`, {
objectInstanceIds,
scope: "object_set"
});
}
export async function createPublicationPointExport(runId: string, ppId: string) {
return postJson<ExportJobRecord>(`/api/v1/runs/${runId}/exports`, {
ppId,
scope: "publication_point"
});
}
export async function getExportJob(runId: string, jobId: string) {
return getJson<ExportJobRecord>(`/api/v1/runs/${runId}/exports/${jobId}`);
}

View File

@@ -1,318 +0,0 @@
/**
* Zod schemas describing the rpki_query_service API contract.
*
* Schemas are intentionally `.passthrough()` so additive backend changes do
* not break the UI; fields the UI depends on are typed strictly.
*/
import { z } from "zod";
/* ------------------------------------------------------------------ */
/* Records */
/* ------------------------------------------------------------------ */
export const runCountsSchema = z
.object({
publicationPoints: z.number().optional(),
objects: z.number().optional(),
freshObjects: z.number().optional(),
cachedObjects: z.number().optional(),
rejectedObjects: z.number().optional(),
freshRejectedObjects: z.number().optional(),
cachedRejectedObjects: z.number().optional(),
trustAnchors: z.number().optional(),
vrps: z.number().optional(),
aspas: z.number().optional(),
warnings: z.number().optional(),
})
.passthrough();
export const runRecordSchema = z
.object({
schemaVersion: z.number().optional(),
runId: z.string(),
runSeq: z.number().nullish(),
runDir: z.string().optional(),
validationTime: z.string().nullish(),
syncMode: z.string().nullish(),
startedAt: z.string().nullish(),
finishedAt: z.string().nullish(),
wallMs: z.number().nullish(),
maxRssKb: z.number().nullish(),
artifactPaths: z.record(z.string()).optional(),
counts: runCountsSchema.optional(),
indexStatus: z.string().optional(),
indexError: z.string().nullish(),
})
.passthrough();
export const repositoryRecordSchema = z
.object({
repoId: z.string(),
uri: z.string(),
host: z.string(),
transport: z.string().nullish(),
publicationPoints: z.number().nullish(),
objects: z.number().nullish(),
rejectedObjects: z.number().nullish(),
downloadBytes: z.number().nullish(),
syncDurationMsTotal: z.number().nullish(),
phases: z.record(z.number()).optional(),
terminalStates: z.record(z.number()).optional(),
})
.passthrough();
export const publicationPointRecordSchema = z
.object({
ppId: z.string(),
repoId: z.string(),
nodeId: z.number().nullish(),
parentNodeId: z.number().nullish(),
rsyncBaseUri: z.string().nullish(),
manifestRsyncUri: z.string().nullish(),
publicationPointRsyncUri: z.string().nullish(),
rrdpNotificationUri: z.string().nullish(),
source: z.string().nullish(),
repoSyncSource: z.string().nullish(),
repoSyncPhase: z.string().nullish(),
repoSyncDurationMs: z.number().nullish(),
repoSyncError: z.string().nullish(),
repoTerminalState: z.string().nullish(),
thisUpdate: z.string().nullish(),
nextUpdate: z.string().nullish(),
verifiedAt: z.string().nullish(),
objects: z.number().nullish(),
rejectedObjects: z.number().nullish(),
warnings: z.number().nullish(),
})
.passthrough();
export const objectInstanceRecordSchema = z
.object({
objectInstanceId: z.string(),
uri: z.string(),
uriHash: z.string().optional(),
sha256: z.string().optional(),
objectType: z.string(),
result: z.string().nullish(),
detailSummary: z.string().nullish(),
repoId: z.string(),
ppId: z.string(),
sourceSection: z.string().nullish(),
rejected: z.boolean().optional(),
rejectReason: z.string().nullish(),
sizeBytes: z.number().nullish(),
rawAvailable: z.boolean().optional(),
})
.passthrough();
export const chainEdgeRecordSchema = z
.object({
relation: z.string(),
fromUri: z.string().nullish(),
toUri: z.string().nullish(),
toObjectInstanceId: z.string().nullish(),
toSha256: z.string().nullish(),
status: z.string().nullish(),
evidence: z.unknown().optional(),
})
.passthrough();
/** Sparse record returned by /objects/by-uri (index hit, no lazy scan). */
export const objectUriIndexRecordSchema = z
.object({
uri: z.string(),
sha256: z.string(),
objectInstanceId: z.string(),
repoId: z.string(),
ppId: z.string(),
})
.passthrough();
export const exportJobRecordSchema = z
.object({
jobId: z.string(),
runId: z.string(),
scope: z.string(),
repoId: z.string().nullish(),
ppId: z.string().nullish(),
status: z.string(),
createdAt: z.string().nullish(),
finishedAt: z.string().nullish(),
outputPath: z.string().nullish(),
objectCount: z.number().nullish(),
bytesWritten: z.number().nullish(),
error: z.string().nullish(),
})
.passthrough();
export const projectionRecordSchema = z
.object({
schemaVersion: z.number().optional(),
sha256: z.string().optional(),
objectType: z.string().optional(),
parseStatus: z.string().optional(),
errorSummary: z.string().nullish(),
projection: z.record(z.unknown()).nullish(),
})
.passthrough();
export const validationIssueSchema = z
.object({
severity: z.string().nullish(),
reasonCode: z.string().nullish(),
reasonText: z.string().nullish(),
rfcRefs: z.array(z.string()).optional(),
evidence: z.unknown().optional(),
})
.passthrough();
export const validationStageSchema = z
.object({
status: z.string().nullish(),
stage: z.string().nullish(),
mode: z.string().nullish(),
projectionStatus: z.string().nullish(),
issues: z.array(validationIssueSchema).optional(),
detailSummary: z.string().nullish(),
edgesCount: z.number().nullish(),
note: z.string().nullish(),
})
.passthrough();
export const validationSummarySchema = z
.object({
finalStatus: z.string().nullish(),
auditResult: z.string().nullish(),
detailSummary: z.string().nullish(),
parsevalidate: validationStageSchema.nullish(),
chainvalidate: validationStageSchema.nullish(),
})
.passthrough();
export const explainRecordSchema = z
.object({
finalStatus: z.string().nullish(),
auditResult: z.string().nullish(),
authoritative: z.boolean().optional(),
explainMode: z.string().nullish(),
parsevalidate: validationStageSchema.nullish(),
chainvalidate: validationStageSchema.nullish(),
chainEdges: z.array(chainEdgeRecordSchema).optional(),
})
.passthrough();
export const searchResultSchema = z
.object({
objects: z.array(objectInstanceRecordSchema),
repos: z.array(repositoryRecordSchema),
publicationPoints: z.array(publicationPointRecordSchema),
})
.passthrough();
export const vrpRecordSchema = z
.object({
runId: z.string(),
asn: z.number(),
prefix: z.string(),
maxLength: z.number(),
})
.passthrough();
export const healthSchema = z
.object({
status: z.string(),
service: z.string().optional(),
version: z.number().optional(),
latestReadyRun: z.string().nullish(),
})
.passthrough();
export const serviceInfoSchema = z
.object({
service: z.string(),
version: z.number(),
})
.passthrough();
export const manifestFileEntrySchema = z
.object({
fileName: z.string(),
hashHex: z.string(),
})
.passthrough();
export const revokedCertEntrySchema = z
.object({
serialNumberHex: z.string().nullish(),
serialNumber: z.string().nullish(),
revocationDate: z.string().nullish(),
})
.passthrough();
export const runSummarySchema = z
.object({
publicationPoints: z.number().optional(),
objects: z.number().optional(),
repos: z.number().optional(),
vrps: z.number().optional(),
aspas: z.number().optional(),
})
.passthrough();
export const statsMapSchema = z.record(z.number());
/* ------------------------------------------------------------------ */
/* Envelope */
/* ------------------------------------------------------------------ */
const pageSchema = z.object({
nextCursor: z.string().nullable(),
limit: z.number(),
});
const metaSchema = z
.object({
runId: z.string().nullish(),
schemaVersion: z.number().optional(),
})
.passthrough();
export function envelopeSchema<T extends z.ZodTypeAny>(dataSchema: T) {
return z
.object({
data: dataSchema,
page: pageSchema.nullish(),
meta: metaSchema.optional(),
})
.passthrough();
}
/* ------------------------------------------------------------------ */
/* Inferred types */
/* ------------------------------------------------------------------ */
export type RunCounts = z.infer<typeof runCountsSchema>;
export type RunRecord = z.infer<typeof runRecordSchema>;
export type RepositoryRecord = z.infer<typeof repositoryRecordSchema>;
export type PublicationPointRecord = z.infer<typeof publicationPointRecordSchema>;
export type ObjectInstanceRecord = z.infer<typeof objectInstanceRecordSchema>;
export type ChainEdgeRecord = z.infer<typeof chainEdgeRecordSchema>;
export type ExportJobRecord = z.infer<typeof exportJobRecordSchema>;
export type ProjectionRecord = z.infer<typeof projectionRecordSchema>;
export type ValidationIssue = z.infer<typeof validationIssueSchema>;
export type ValidationStage = z.infer<typeof validationStageSchema>;
export type ValidationSummary = z.infer<typeof validationSummarySchema>;
export type ExplainRecord = z.infer<typeof explainRecordSchema>;
export type SearchResult = z.infer<typeof searchResultSchema>;
export type VrpRecord = z.infer<typeof vrpRecordSchema>;
export type HealthStatus = z.infer<typeof healthSchema>;
export type ServiceInfo = z.infer<typeof serviceInfoSchema>;
export type ManifestFileEntry = z.infer<typeof manifestFileEntrySchema>;
export type RevokedCertEntry = z.infer<typeof revokedCertEntrySchema>;
export type RunSummary = z.infer<typeof runSummarySchema>;
export interface ListResult<T> {
items: T[];
nextCursor: string | null;
runId: string | undefined;
}

View File

@@ -1,472 +0,0 @@
/**
* Typed endpoint functions for the rpki_query_service API.
* Every function validates the response envelope with zod before returning.
*/
import { z } from "zod";
import { apiFetch, apiPost, withQuery, type QueryParams } from "./client";
import {
chainEdgeRecordSchema,
envelopeSchema,
exportJobRecordSchema,
explainRecordSchema,
healthSchema,
manifestFileEntrySchema,
objectInstanceRecordSchema,
projectionRecordSchema,
publicationPointRecordSchema,
repositoryRecordSchema,
revokedCertEntrySchema,
runRecordSchema,
runSummarySchema,
searchResultSchema,
serviceInfoSchema,
statsMapSchema,
validationSummarySchema,
vrpRecordSchema,
type ChainEdgeRecord,
type ExportJobRecord,
type ExplainRecord,
type HealthStatus,
type ListResult,
type ManifestFileEntry,
type ObjectInstanceRecord,
type ProjectionRecord,
type PublicationPointRecord,
type RepositoryRecord,
type RevokedCertEntry,
type RunRecord,
type RunSummary,
type SearchResult,
type ServiceInfo,
type ValidationSummary,
type VrpRecord,
} from "./schemas";
const API = "/api/v1";
function runBase(runId: string): string {
return `${API}/runs/${encodeURIComponent(runId)}`;
}
async function getData<S extends z.ZodTypeAny>(
path: string,
schema: S,
params?: QueryParams,
): Promise<z.infer<S>> {
const raw = await apiFetch<unknown>(withQuery(path, params));
return envelopeSchema(schema).parse(raw).data;
}
async function getList<S extends z.ZodTypeAny>(
path: string,
schema: S,
params?: QueryParams,
): Promise<ListResult<z.infer<S>>> {
const raw = await apiFetch<unknown>(withQuery(path, params));
const parsed = envelopeSchema(z.array(schema)).parse(raw);
return {
items: parsed.data,
nextCursor: parsed.page?.nextCursor ?? null,
runId: parsed.meta?.runId ?? undefined,
};
}
/* ------------------------------------------------------------------ */
/* Service / health */
/* ------------------------------------------------------------------ */
export function getServiceInfo(): Promise<ServiceInfo> {
return getData(`${API}`, serviceInfoSchema);
}
export function getHealth(): Promise<HealthStatus> {
return getData(`${API}/healthz`, healthSchema);
}
/* ------------------------------------------------------------------ */
/* Runs */
/* ------------------------------------------------------------------ */
export type PageParams = {
limit?: number;
cursor?: string | null;
};
export function listRuns(params: PageParams = {}): Promise<ListResult<RunRecord>> {
return getList(`${API}/runs`, runRecordSchema, params);
}
export function getLatestRun(): Promise<RunRecord> {
return getData(`${API}/latest_run`, runRecordSchema);
}
export function getRun(runId: string): Promise<RunRecord> {
return getData(runBase(runId), runRecordSchema);
}
export function getRunArtifacts(runId: string): Promise<Record<string, string>> {
return getData(`${runBase(runId)}/artifacts`, z.record(z.string()));
}
export function getRunSummary(runId: string): Promise<RunSummary> {
return getData(`${runBase(runId)}/summary`, runSummarySchema);
}
/* ------------------------------------------------------------------ */
/* Repositories */
/* ------------------------------------------------------------------ */
export function listRepos(
runId: string,
params: PageParams = {},
): Promise<ListResult<RepositoryRecord>> {
return getList(`${runBase(runId)}/repos`, repositoryRecordSchema, params);
}
export function getRepo(runId: string, repoId: string): Promise<RepositoryRecord> {
return getData(
`${runBase(runId)}/repos/${encodeURIComponent(repoId)}`,
repositoryRecordSchema,
);
}
export interface RepoStats {
publicationPoints?: number;
objects?: number;
rejectedObjects?: number;
syncDurationMsTotal?: number;
phases?: Record<string, number>;
terminalStates?: Record<string, number>;
}
export function getRepoStats(runId: string, repoId: string): Promise<RepoStats> {
return getData(
`${runBase(runId)}/repos/${encodeURIComponent(repoId)}/stats`,
z
.object({
publicationPoints: z.number().optional(),
objects: z.number().optional(),
rejectedObjects: z.number().optional(),
syncDurationMsTotal: z.number().optional(),
phases: z.record(z.number()).optional(),
terminalStates: z.record(z.number()).optional(),
})
.passthrough(),
);
}
/* ------------------------------------------------------------------ */
/* Publication points */
/* ------------------------------------------------------------------ */
export function listPublicationPoints(
runId: string,
params: PageParams = {},
): Promise<ListResult<PublicationPointRecord>> {
return getList(`${runBase(runId)}/publication-points`, publicationPointRecordSchema, params);
}
export function listRepoPublicationPoints(
runId: string,
repoId: string,
params: PageParams = {},
): Promise<ListResult<PublicationPointRecord>> {
return getList(
`${runBase(runId)}/repos/${encodeURIComponent(repoId)}/publication-points`,
publicationPointRecordSchema,
params,
);
}
export function getPublicationPoint(
runId: string,
ppId: string,
): Promise<PublicationPointRecord> {
return getData(
`${runBase(runId)}/publication-points/${encodeURIComponent(ppId)}`,
publicationPointRecordSchema,
);
}
export interface PpStats {
objects?: number;
rejectedObjects?: number;
warnings?: number;
repoSyncSource?: string;
repoSyncPhase?: string;
repoSyncDurationMs?: number;
repoTerminalState?: string;
}
export function getPublicationPointStats(runId: string, ppId: string): Promise<PpStats> {
return getData(
`${runBase(runId)}/publication-points/${encodeURIComponent(ppId)}/stats`,
z
.object({
objects: z.number().optional(),
rejectedObjects: z.number().optional(),
warnings: z.number().optional(),
repoSyncSource: z.string().optional(),
repoSyncPhase: z.string().optional(),
repoSyncDurationMs: z.number().optional(),
repoTerminalState: z.string().optional(),
})
.passthrough(),
);
}
/* ------------------------------------------------------------------ */
/* Objects */
/* ------------------------------------------------------------------ */
export interface ObjectFilters extends PageParams {
type?: string;
result?: string;
rejected?: boolean;
reason?: string;
q?: string;
}
function filterParams(filters: ObjectFilters): QueryParams {
return {
limit: filters.limit,
cursor: filters.cursor,
type: filters.type,
result: filters.result,
rejected: filters.rejected === undefined ? undefined : String(filters.rejected),
reason: filters.reason,
q: filters.q,
};
}
export function listObjects(
runId: string,
filters: ObjectFilters = {},
): Promise<ListResult<ObjectInstanceRecord>> {
return getList(`${runBase(runId)}/objects`, objectInstanceRecordSchema, filterParams(filters));
}
export function listRepoObjects(
runId: string,
repoId: string,
filters: ObjectFilters = {},
): Promise<ListResult<ObjectInstanceRecord>> {
return getList(
`${runBase(runId)}/repos/${encodeURIComponent(repoId)}/objects`,
objectInstanceRecordSchema,
filterParams(filters),
);
}
export function listPublicationPointObjects(
runId: string,
ppId: string,
filters: ObjectFilters = {},
): Promise<ListResult<ObjectInstanceRecord>> {
return getList(
`${runBase(runId)}/publication-points/${encodeURIComponent(ppId)}/objects`,
objectInstanceRecordSchema,
filterParams(filters),
);
}
export interface IssueFilters extends PageParams {
reason?: string;
type?: string;
repoId?: string;
ppId?: string;
}
export function listIssues(
runId: string,
filters: IssueFilters = {},
): Promise<ListResult<ObjectInstanceRecord>> {
return getList(`${runBase(runId)}/issues`, objectInstanceRecordSchema, {
limit: filters.limit,
cursor: filters.cursor,
reason: filters.reason,
type: filters.type,
repoId: filters.repoId,
ppId: filters.ppId,
});
}
export function getObjectByUri(
runId: string,
uri: string,
): Promise<ObjectInstanceRecord> {
return getData(`${runBase(runId)}/objects/by-uri`, objectInstanceRecordSchema, { uri });
}
export function getObject(
runId: string,
objectInstanceId: string,
): Promise<ObjectInstanceRecord> {
return getData(
`${runBase(runId)}/objects/${encodeURIComponent(objectInstanceId)}`,
objectInstanceRecordSchema,
);
}
export function getObjectProjection(
runId: string,
objectInstanceId: string,
): Promise<ProjectionRecord | null> {
return getData(
`${runBase(runId)}/objects/${encodeURIComponent(objectInstanceId)}/parsed`,
projectionRecordSchema.nullable(),
);
}
export function getObjectValidation(
runId: string,
objectInstanceId: string,
): Promise<ValidationSummary> {
return getData(
`${runBase(runId)}/objects/${encodeURIComponent(objectInstanceId)}/validation`,
validationSummarySchema,
);
}
export function getObjectChain(
runId: string,
objectInstanceId: string,
): Promise<ChainEdgeRecord[]> {
return getData(
`${runBase(runId)}/objects/${encodeURIComponent(objectInstanceId)}/chain`,
z.array(chainEdgeRecordSchema),
);
}
export function listManifestFiles(
runId: string,
objectInstanceId: string,
params: PageParams = {},
): Promise<ListResult<ManifestFileEntry>> {
return getList(
`${runBase(runId)}/objects/${encodeURIComponent(objectInstanceId)}/parsed/manifest-files`,
manifestFileEntrySchema,
params,
);
}
export function listRevokedCertificates(
runId: string,
objectInstanceId: string,
params: PageParams = {},
): Promise<ListResult<RevokedCertEntry>> {
return getList(
`${runBase(runId)}/objects/${encodeURIComponent(objectInstanceId)}/parsed/revoked-certs`,
revokedCertEntrySchema,
params,
);
}
export function explainObjectValidation(
runId: string,
objectInstanceId: string,
options: { forceRefresh?: boolean } = {},
): Promise<ExplainRecord> {
return apiPost<unknown>(
`${runBase(runId)}/objects/${encodeURIComponent(objectInstanceId)}/validation/explain`,
{ forceRefresh: options.forceRefresh ?? false },
).then((raw) => envelopeSchema(explainRecordSchema).parse(raw).data);
}
export function rawObjectUrl(runId: string, objectInstanceId: string): string {
return `${runBase(runId)}/objects/${encodeURIComponent(objectInstanceId)}/raw`;
}
/* ------------------------------------------------------------------ */
/* Search */
/* ------------------------------------------------------------------ */
export function searchRun(
runId: string,
q: string,
limit = 10,
): Promise<SearchResult> {
return getData(`${runBase(runId)}/search`, searchResultSchema, { q, limit });
}
/* ------------------------------------------------------------------ */
/* VRP lookup */
/* ------------------------------------------------------------------ */
export type VrpLookupParams = PageParams & {
/** Exact IPv4/IPv6 address (mutually exclusive with `prefix`). */
ip?: string;
/** CIDR; returns VRPs covering the whole prefix (maxLength semantics). */
prefix?: string;
/** Optional ASN filter applied on top of ip/prefix. */
asn?: number;
};
export function lookupVrps(
runId: string,
params: VrpLookupParams,
): Promise<ListResult<VrpRecord>> {
return getList(`${runBase(runId)}/vrps/lookup`, vrpRecordSchema, params);
}
/* ------------------------------------------------------------------ */
/* Stats */
/* ------------------------------------------------------------------ */
export function getStatsObjectTypes(runId: string): Promise<Record<string, number>> {
return getData(`${runBase(runId)}/stats/object-types`, statsMapSchema);
}
export function getStatsValidation(runId: string): Promise<Record<string, number>> {
return getData(`${runBase(runId)}/stats/validation`, statsMapSchema);
}
export function getStatsReasons(runId: string): Promise<Record<string, number>> {
return getData(`${runBase(runId)}/stats/reasons`, statsMapSchema);
}
export function getStatsDownloads(runId: string): Promise<Record<string, unknown>> {
return getData(`${runBase(runId)}/stats/downloads`, z.record(z.unknown()));
}
/* ------------------------------------------------------------------ */
/* Exports */
/* ------------------------------------------------------------------ */
export interface ExportRequest {
scope: "repo" | "publication_point" | "object_set";
repoId?: string;
ppId?: string;
objectInstanceIds?: string[];
}
export function createExport(
runId: string,
request: ExportRequest,
): Promise<ExportJobRecord> {
const body: Record<string, unknown> = { scope: request.scope };
if (request.repoId) body.repoId = request.repoId;
if (request.ppId) body.ppId = request.ppId;
if (request.objectInstanceIds) body.objectInstanceIds = request.objectInstanceIds;
return apiPost<unknown>(`${runBase(runId)}/exports`, body).then(
(raw) => envelopeSchema(exportJobRecordSchema).parse(raw).data,
);
}
export function listExports(
runId: string,
params: PageParams = {},
): Promise<ListResult<ExportJobRecord>> {
return getList(`${runBase(runId)}/exports`, exportJobRecordSchema, params);
}
export function getExportJob(runId: string, jobId: string): Promise<ExportJobRecord> {
return getData(
`${runBase(runId)}/exports/${encodeURIComponent(jobId)}`,
exportJobRecordSchema,
);
}
export function exportDownloadUrl(runId: string, jobId: string): string {
return `${runBase(runId)}/exports/${encodeURIComponent(jobId)}/download`;
}

View File

@@ -1,48 +1,103 @@
import { Suspense, lazy } from "react"; import { Activity, Braces, Database, FileSearch, GitBranch, Home, PackageOpen, ShieldCheck } from "lucide-react";
import { Navigate, Route, Routes } from "react-router-dom"; import type { LucideIcon } from "lucide-react";
import { Shell } from "../components/Shell"; import { useState } from "react";
import { LoadingBlock } from "../components/StateBlock"; import { Shell } from "../components/layout/Shell";
import { ObjectDetailPage } from "../features/object-detail/ObjectDetailPage";
import { OverviewPage } from "../features/overview/OverviewPage";
import { RepositoriesPage } from "../features/repositories/RepositoriesPage";
import { createRepositoryBrowserState } from "../features/repositories/repositoryBrowserState";
const OverviewPage = lazy(() => import("../pages/OverviewPage")); type ViewId = "overview" | "repositories" | "publication-points" | "objects" | "validation" | "exports" | "runs" | "api";
const RepositoriesPage = lazy(() => import("../pages/RepositoriesPage"));
const RepositoryDetailPage = lazy(() => import("../pages/RepositoryDetailPage")); const navigationItems = [
const PublicationPointsPage = lazy(() => import("../pages/PublicationPointsPage")); { id: "overview", label: "Overview", icon: Home },
const PublicationPointDetailPage = lazy( { id: "repositories", label: "Repositories", icon: Database },
() => import("../pages/PublicationPointDetailPage"), { id: "publication-points", label: "Publication Points", icon: GitBranch },
); { id: "objects", label: "Objects", icon: Braces },
const ObjectsPage = lazy(() => import("../pages/ObjectsPage")); { id: "validation", label: "Validation", icon: ShieldCheck },
const ObjectDetailPage = lazy(() => import("../pages/ObjectDetailPage")); { id: "exports", label: "Exports", icon: PackageOpen },
const ValidationPage = lazy(() => import("../pages/ValidationPage")); { id: "runs", label: "Runs", icon: Activity },
const SearchPage = lazy(() => import("../pages/SearchPage")); { id: "api", label: "API", icon: FileSearch }
const ExportsPage = lazy(() => import("../pages/ExportsPage")); ] satisfies Array<{ id: ViewId; label: string; icon: LucideIcon }>;
const RunsPage = lazy(() => import("../pages/RunsPage"));
const ApiStatusPage = lazy(() => import("../pages/ApiStatusPage")); const comingSoonCopy: Record<Exclude<ViewId, "overview" | "repositories" | "objects">, { title: string; description: string; alternative: string }> = {
const NotFoundPage = lazy(() => import("../pages/NotFoundPage")); "publication-points": {
title: "Publication Points",
description: "Dedicated publication point analytics are not implemented yet.",
alternative: "Use Repositories to drill down from repository to publication point and object rows."
},
validation: {
title: "Validation",
description: "A dedicated validation investigation workspace is planned but not available in this MVP.",
alternative: "Use Overview for reason summaries and Object Detail for per-object validation evidence."
},
exports: {
title: "Exports",
description: "A standalone export job browser is planned but not available yet.",
alternative: "Use Object Detail to trigger object or publication point export workflows."
},
runs: {
title: "Runs",
description: "Historical run navigation is not implemented in the current Explorer UI.",
alternative: "The current view always uses the latest indexed run from query service."
},
api: {
title: "API",
description: "The API reference page is planned but not implemented yet.",
alternative: "Use the project README and query service endpoints while the UI reference page is pending."
}
};
export function App() { export function App() {
const [activeView, setActiveView] = useState<ViewId>("overview");
const [selectedObjectInstanceId, setSelectedObjectInstanceId] = useState<string | null>(null);
const [repositoryBrowserState, setRepositoryBrowserState] = useState(createRepositoryBrowserState);
const openObject = (objectInstanceId: string) => {
setSelectedObjectInstanceId(objectInstanceId);
setActiveView("objects");
};
const renderActiveView = () => {
if (activeView === "objects") {
return <ObjectDetailPage initialObjectInstanceId={selectedObjectInstanceId} />;
}
if (activeView === "repositories") {
return ( return (
<Shell> <RepositoriesPage
<Suspense fallback={<LoadingBlock label="Loading page…" />}> browserState={repositoryBrowserState}
<Routes> onBrowserStateChange={setRepositoryBrowserState}
<Route path="/" element={<OverviewPage />} /> onOpenObject={openObject}
<Route path="/repositories" element={<RepositoriesPage />} />
<Route path="/repositories/:repoId" element={<RepositoryDetailPage />} />
<Route path="/publication-points" element={<PublicationPointsPage />} />
<Route
path="/publication-points/:ppId"
element={<PublicationPointDetailPage />}
/> />
<Route path="/objects" element={<ObjectsPage />} /> );
<Route path="/objects/:objectInstanceId" element={<ObjectDetailPage />} /> }
<Route path="/validation" element={<ValidationPage />} /> if (activeView === "overview") {
<Route path="/search" element={<SearchPage />} /> return <OverviewPage />;
<Route path="/exports" element={<ExportsPage />} /> }
<Route path="/runs" element={<RunsPage />} /> return <ComingSoonPage {...comingSoonCopy[activeView]} />;
<Route path="/api" element={<ApiStatusPage />} /> };
<Route path="/404" element={<NotFoundPage />} />
<Route path="*" element={<Navigate to="/404" replace />} /> return (
</Routes> <Shell activeView={activeView} navigationItems={navigationItems} onNavigate={setActiveView}>
</Suspense> {renderActiveView()}
</Shell> </Shell>
); );
} }
function ComingSoonPage({ title, description, alternative }: { title: string; description: string; alternative: string }) {
return (
<section className="page-stack" aria-labelledby="coming-soon-heading">
<div className="hero-dashboard compact-hero coming-soon-panel">
<div>
<p className="eyebrow">Planned workspace</p>
<h2 id="coming-soon-heading">{title}</h2>
<p>{description}</p>
</div>
<div className="hero-status muted-status">
<strong>Coming soon</strong>
<span>{alternative}</span>
</div>
</div>
</section>
);
}

View File

@@ -1,12 +1,7 @@
import { QueryClientProvider } from "@tanstack/react-query"; import { QueryClientProvider } from "@tanstack/react-query";
import { BrowserRouter } from "react-router-dom";
import type { PropsWithChildren } from "react"; import type { PropsWithChildren } from "react";
import { queryClient } from "./queryClient"; import { queryClient } from "./queryClient";
export function AppProviders({ children }: PropsWithChildren) { export function AppProviders({ children }: PropsWithChildren) {
return ( return <QueryClientProvider client={queryClient}>{children}</QueryClientProvider>;
<QueryClientProvider client={queryClient}>
<BrowserRouter>{children}</BrowserRouter>
</QueryClientProvider>
);
} }

View File

@@ -1,22 +1,15 @@
import { QueryClient } from "@tanstack/react-query"; import { QueryClient } from "@tanstack/react-query";
import { ApiError } from "../api/client";
export const queryClient = new QueryClient({ export const queryClient = new QueryClient({
defaultOptions: { defaultOptions: {
queries: { queries: {
staleTime: 30_000, staleTime: 10_000,
gcTime: 10 * 60_000, gcTime: 10 * 60_000,
refetchOnWindowFocus: false, retry: 1,
retry: (failureCount, error) => { refetchOnWindowFocus: false
// Never retry client errors (4xx); retry transient failures once.
if (error instanceof ApiError && error.status >= 400 && error.status < 500) {
return false;
}
return failureCount < 1;
},
}, },
mutations: { mutations: {
retry: false, retry: false
}, }
}, }
}); });

View File

@@ -1,61 +0,0 @@
import { useEffect, useRef, useState, type MouseEvent } from "react";
import { Check, Copy } from "lucide-react";
import { truncateMiddle } from "../lib/format";
/**
* Renders a long identifier (URI / hash / id) truncated with an always-available
* copy button. The full value is exposed via the `title` attribute.
*/
export function CopyableValue({
value,
max = 36,
label = "value",
}: {
value: string | null | undefined;
/** Max rendered characters before middle-truncation. */
max?: number;
/** Accessible description of what is copied, e.g. "object URI". */
label?: string;
}) {
const [copied, setCopied] = useState(false);
const timer = useRef<ReturnType<typeof setTimeout> | null>(null);
useEffect(() => {
return () => {
if (timer.current) clearTimeout(timer.current);
};
}, []);
if (!value) return <span className="text-faint">—</span>;
const onCopy = async (event: MouseEvent) => {
// Don't trigger row-level click handlers (navigation) when copying.
event.stopPropagation();
try {
await navigator.clipboard.writeText(value);
setCopied(true);
if (timer.current) clearTimeout(timer.current);
timer.current = setTimeout(() => setCopied(false), 1500);
} catch {
// Clipboard API unavailable (permissions, non-secure context) — select
// fallback is intentionally omitted; the title attribute exposes the
// full value for manual copying.
setCopied(false);
}
};
return (
<span className="copyable" title={value}>
<span className="copyable-text">{truncateMiddle(value, max)}</span>
<button
type="button"
className={`copy-btn${copied ? " copied" : ""}`}
onClick={onCopy}
aria-label={`Copy ${label}`}
aria-live="polite"
>
{copied ? <Check size={13} aria-hidden="true" /> : <Copy size={13} aria-hidden="true" />}
</button>
</span>
);
}

View File

@@ -1,48 +0,0 @@
import { ChevronLeft, ChevronRight } from "lucide-react";
/**
* Previous/Next controls for cursor-paged lists.
*/
export function CursorPagerControls({
page,
canPrev,
hasNext,
loading,
onPrev,
onNext,
itemCount,
}: {
page: number;
canPrev: boolean;
hasNext: boolean;
loading?: boolean;
onPrev: () => void;
onNext: () => void;
itemCount?: number;
}) {
return (
<div className="pager">
<span className="pager-info">
Page {page}
{itemCount !== undefined ? ` · ${itemCount} rows` : ""}
{loading ? " · loading…" : ""}
</span>
<button
type="button"
className="btn small"
onClick={onPrev}
disabled={!canPrev || loading}
>
<ChevronLeft size={13} aria-hidden="true" /> Previous
</button>
<button
type="button"
className="btn small"
onClick={onNext}
disabled={!hasNext || loading}
>
Next <ChevronRight size={13} aria-hidden="true" />
</button>
</div>
);
}

View File

@@ -1,104 +0,0 @@
import type { ReactNode } from "react";
import { EmptyBlock, ErrorBlock } from "./StateBlock";
export interface Column<T> {
key: string;
header: ReactNode;
render: (row: T) => ReactNode;
/** Right-align (numeric) cells. */
numeric?: boolean;
/** Extra class on the td/th. */
className?: string;
}
interface DataTableProps<T> {
columns: Column<T>[];
rows: T[];
rowKey: (row: T) => string;
loading?: boolean;
error?: unknown;
onRetry?: () => void;
emptyTitle?: string;
emptyHint?: string;
caption?: string;
/** Row click handler — renders rows as clickable. */
onRowClick?: (row: T) => void;
/** Number of skeleton rows while loading. */
skeletonRows?: number;
}
/**
* Data-dense table with unified loading / error / empty states.
* Renders inside a horizontal-scroll wrapper so narrow viewports never
* overflow the document.
*/
export function DataTable<T>({
columns,
rows,
rowKey,
loading = false,
error,
onRetry,
emptyTitle = "No rows",
emptyHint,
caption,
onRowClick,
skeletonRows = 6,
}: DataTableProps<T>) {
if (error !== undefined && error !== null) {
return <ErrorBlock error={error} onRetry={onRetry} />;
}
if (loading) {
return (
<div role="status" aria-label="Loading table rows">
{Array.from({ length: skeletonRows }, (_, i) => (
<div className="skeleton-row" key={i} />
))}
</div>
);
}
if (rows.length === 0) {
return <EmptyBlock title={emptyTitle} hint={emptyHint} />;
}
return (
<div className="table-wrap">
<table className="data-table">
{caption ? <caption className="sr-only">{caption}</caption> : null}
<thead>
<tr>
{columns.map((col) => (
<th
key={col.key}
scope="col"
className={`${col.numeric ? "num" : ""} ${col.className ?? ""}`.trim()}
>
{col.header}
</th>
))}
</tr>
</thead>
<tbody>
{rows.map((row) => (
<tr
key={rowKey(row)}
className={onRowClick ? "clickable" : undefined}
onClick={onRowClick ? () => onRowClick(row) : undefined}
>
{columns.map((col) => (
<td
key={col.key}
className={`${col.numeric ? "num" : ""} ${col.className ?? ""}`.trim()}
>
{col.render(row)}
</td>
))}
</tr>
))}
</tbody>
</table>
</div>
);
}

View File

@@ -1,33 +0,0 @@
import type { ReactNode } from "react";
import { Link } from "react-router-dom";
export function KpiCard({
label,
value,
sub,
tone,
to,
}: {
label: string;
value: ReactNode;
sub?: ReactNode;
tone?: "red" | "amber" | "blue";
to?: string;
}) {
const body = (
<>
<span className="kpi-label">{label}</span>
<span className="kpi-value">{value}</span>
{sub ? <span className="kpi-sub">{sub}</span> : null}
</>
);
const className = `kpi-card${tone ? ` tone-${tone}` : ""}`;
if (to) {
return (
<Link to={to} className={className}>
{body}
</Link>
);
}
return <div className={className}>{body}</div>;
}

View File

@@ -1,218 +0,0 @@
import { useEffect, useMemo, useState } from "react";
import { useNavigate } from "react-router-dom";
import { useQuery } from "@tanstack/react-query";
import type { ListResult, ObjectInstanceRecord } from "../api/schemas";
import type { ObjectFilters } from "../api/service";
import { useCursorPager } from "../lib/cursor";
import { objectTypeLabel, truncateMiddle } from "../lib/format";
import { withRunParam } from "../lib/run";
import { CopyableValue } from "./CopyableValue";
import { CursorPagerControls } from "./CursorPagerControls";
import { DataTable, type Column } from "./DataTable";
import { StatusPill } from "./StatusPill";
export interface ObjectFilterValues {
type: string;
result: string;
rejectedOnly: boolean;
q: string;
}
export const EMPTY_OBJECT_FILTERS: ObjectFilterValues = {
type: "",
result: "",
rejectedOnly: false,
q: "",
};
/** Report-stream object type vocabulary (backend also accepts mft/cer/asa aliases). */
const TYPE_OPTIONS = ["roa", "manifest", "crl", "certificate", "aspa", "gbr"];
const RESULT_OPTIONS = ["ok", "error", "skipped"];
/**
* Filter bar + paginated object table. The `fetcher` determines the scope
* (global / repo / publication point / issues). Filter state is controlled
* by the caller so standalone pages can sync it to the URL.
*/
export function ObjectsTable({
runId,
queryKeyScope,
fetcher,
filters,
onFiltersChange,
showReason = true,
}: {
runId: string;
queryKeyScope: string;
fetcher: (filters: ObjectFilters) => Promise<ListResult<ObjectInstanceRecord>>;
filters: ObjectFilterValues;
onFiltersChange: (next: ObjectFilterValues) => void;
showReason?: boolean;
}) {
const navigate = useNavigate();
const [qDraft, setQDraft] = useState(filters.q);
// Keep the draft input in sync when filters change externally (URL nav).
useEffect(() => {
setQDraft(filters.q);
}, [filters.q]);
const filterKey = JSON.stringify([runId, filters.type, filters.result, filters.rejectedOnly, filters.q]);
const pager = useCursorPager(filterKey);
const query = useQuery({
queryKey: ["objects", queryKeyScope, runId, filters, pager.cursor],
queryFn: () =>
fetcher({
limit: 50,
cursor: pager.cursor,
type: filters.type || undefined,
result: filters.result || undefined,
rejected: filters.rejectedOnly || undefined,
q: filters.q || undefined,
}),
placeholderData: (prev) => prev,
});
const columns: Column<ObjectInstanceRecord>[] = useMemo(
() => [
{
key: "type",
header: "Type",
render: (obj) => <span className="chip">{objectTypeLabel(obj.objectType)}</span>,
},
{
key: "uri",
header: "URI",
render: (obj) => <CopyableValue value={obj.uri} max={64} label="object URI" />,
},
{
key: "sha256",
header: "SHA-256",
render: (obj) => (
<span className="mono text-muted" title={obj.sha256 ?? undefined}>
{obj.sha256 ? truncateMiddle(obj.sha256, 18) : "—"}
</span>
),
},
{
key: "source",
header: "Source",
render: (obj) => <StatusPill status={obj.sourceSection} />,
},
{
key: "result",
header: "Result",
render: (obj) => <StatusPill status={obj.rejected ? "rejected" : obj.result} />,
},
...(showReason
? [
{
key: "reason",
header: "Reject reason",
render: (obj: ObjectInstanceRecord) =>
obj.rejectReason ? (
<span className="mono text-small" title={obj.rejectReason}>
{truncateMiddle(obj.rejectReason, 60)}
</span>
) : (
<span className="text-faint">—</span>
),
} satisfies Column<ObjectInstanceRecord>,
]
: []),
],
[showReason],
);
const submitQ = () => {
if (qDraft.trim() !== filters.q) {
onFiltersChange({ ...filters, q: qDraft.trim() });
}
};
return (
<div>
<div className="panel-header filter-bar" role="search">
<div className="filter-field">
<label htmlFor={`${queryKeyScope}-type`}>Type</label>
<select
id={`${queryKeyScope}-type`}
value={filters.type}
onChange={(e) => onFiltersChange({ ...filters, type: e.target.value })}
>
<option value="">all types</option>
{TYPE_OPTIONS.map((t) => (
<option key={t} value={t}>
{objectTypeLabel(t)}
</option>
))}
</select>
</div>
<div className="filter-field">
<label htmlFor={`${queryKeyScope}-result`}>Result</label>
<select
id={`${queryKeyScope}-result`}
value={filters.result}
onChange={(e) => onFiltersChange({ ...filters, result: e.target.value })}
>
<option value="">all results</option>
{RESULT_OPTIONS.map((r) => (
<option key={r} value={r}>
{r}
</option>
))}
</select>
</div>
<div className="filter-field">
<label htmlFor={`${queryKeyScope}-q`}>URI contains</label>
<input
id={`${queryKeyScope}-q`}
type="search"
className="mono"
value={qDraft}
onChange={(e) => setQDraft(e.target.value)}
onKeyDown={(e) => {
if (e.key === "Enter") submitQ();
}}
onBlur={submitQ}
placeholder="substring…"
/>
</div>
<label className="filter-check">
<input
type="checkbox"
checked={filters.rejectedOnly}
onChange={(e) => onFiltersChange({ ...filters, rejectedOnly: e.target.checked })}
/>
Rejected only
</label>
<span className="text-faint text-small" style={{ marginLeft: "auto" }}>
server-side filters
</span>
</div>
<DataTable
columns={columns}
rows={query.data?.items ?? []}
rowKey={(obj) => obj.objectInstanceId}
loading={query.isPending}
error={query.isError ? query.error : undefined}
onRetry={() => query.refetch()}
emptyTitle="No objects match"
emptyHint="Try relaxing the filters."
caption="RPKI objects"
onRowClick={(obj) =>
navigate(withRunParam(`/objects/${encodeURIComponent(obj.objectInstanceId)}`, runId))
}
/>
<CursorPagerControls
page={pager.page}
canPrev={pager.canPrev}
hasNext={query.data?.nextCursor != null}
loading={query.isFetching}
onPrev={pager.goPrev}
onNext={() => pager.goNext(query.data?.nextCursor ?? null)}
itemCount={query.data?.items.length}
/>
</div>
);
}

View File

@@ -1,21 +0,0 @@
import type { ReactNode } from "react";
export function PageHeader({
title,
subtitle,
actions,
}: {
title: ReactNode;
subtitle?: ReactNode;
actions?: ReactNode;
}) {
return (
<header className="page-header">
<div className="page-title-block">
<h1>{title}</h1>
{subtitle ? <div className="page-subtitle">{subtitle}</div> : null}
</div>
{actions ? <div className="page-actions">{actions}</div> : null}
</header>
);
}

View File

@@ -1,34 +0,0 @@
import type { ReactNode } from "react";
/** Panel with header (title + optional tools) and body. */
export function Panel({
title,
subtitle,
tools,
children,
flush = false,
className = "",
}: {
title?: ReactNode;
subtitle?: ReactNode;
tools?: ReactNode;
children: ReactNode;
/** Remove body padding (for tables that span the full width). */
flush?: boolean;
className?: string;
}) {
return (
<section className={`panel ${className}`.trim()}>
{title || tools ? (
<header className="panel-header">
<div>
{title ? <h2>{title}</h2> : null}
{subtitle ? <div className="panel-subtitle">{subtitle}</div> : null}
</div>
{tools ? <div className="panel-tools">{tools}</div> : null}
</header>
) : null}
<div className={`panel-body${flush ? " flush" : ""}`}>{children}</div>
</section>
);
}

View File

@@ -1,305 +0,0 @@
import { useQuery } from "@tanstack/react-query";
import type { ReactNode } from "react";
import { listManifestFiles, listRevokedCertificates } from "../api/service";
import type { ProjectionRecord } from "../api/schemas";
import { CopyableValue } from "./CopyableValue";
import { CursorPagerControls } from "./CursorPagerControls";
import { DataTable, type Column } from "./DataTable";
import { EmptyBlock, LoadingBlock, Notice } from "./StateBlock";
import { useCursorPager } from "../lib/cursor";
import { formatInt, formatUtc } from "../lib/format";
import { asArray, asNumber, asRecord, asString } from "../lib/projection";
function MetaRow({ label, value, mono = false }: { label: string; value: ReactNode; mono?: boolean }) {
return (
<>
<dt>{label}</dt>
<dd className={mono ? "mono" : undefined}>{value ?? <span className="text-faint">—</span>}</dd>
</>
);
}
function ManifestFiles({ runId, objectInstanceId }: { runId: string; objectInstanceId: string }) {
const pager = useCursorPager(`${runId}:${objectInstanceId}:mft`);
const query = useQuery({
queryKey: ["manifest-files", runId, objectInstanceId, pager.cursor],
queryFn: () => listManifestFiles(runId, objectInstanceId, { limit: 100, cursor: pager.cursor }),
placeholderData: (prev) => prev,
});
const columns: Column<{ fileName: string; hashHex: string }>[] = [
{ key: "file", header: "File", render: (row) => <span className="mono">{row.fileName}</span> },
{
key: "hash",
header: "Hash",
render: (row) => <CopyableValue value={row.hashHex} max={40} label="file hash" />,
},
];
if (query.isPending) return <LoadingBlock label="Loading manifest file list…" />;
if (query.isError) return <Notice kind="error">Failed to load the manifest file list.</Notice>;
return (
<>
<DataTable
columns={columns}
rows={query.data.items}
rowKey={(row) => row.fileName}
emptyTitle="Manifest has no file entries"
caption="Manifest file list"
/>
<CursorPagerControls
page={pager.page}
canPrev={pager.canPrev}
hasNext={query.data.nextCursor != null}
loading={query.isFetching}
onPrev={pager.goPrev}
onNext={() => pager.goNext(query.data.nextCursor ?? null)}
itemCount={query.data.items.length}
/>
</>
);
}
function RevokedCerts({ runId, objectInstanceId }: { runId: string; objectInstanceId: string }) {
const pager = useCursorPager(`${runId}:${objectInstanceId}:crl`);
const query = useQuery({
queryKey: ["revoked-certs", runId, objectInstanceId, pager.cursor],
queryFn: () =>
listRevokedCertificates(runId, objectInstanceId, { limit: 100, cursor: pager.cursor }),
placeholderData: (prev) => prev,
});
const columns: Column<{ serialNumberHex?: string | null; serialNumber?: string | null; revocationDate?: string | null }>[] = [
{
key: "serial",
header: "Serial number",
render: (row) => (
<CopyableValue value={row.serialNumberHex ?? row.serialNumber} max={44} label="serial number" />
),
},
{ key: "date", header: "Revocation date", render: (row) => formatUtc(row.revocationDate) },
];
if (query.isPending) return <LoadingBlock label="Loading revoked certificates…" />;
if (query.isError) return <Notice kind="error">Failed to load the revocation list.</Notice>;
const rows = query.data.items.map((row, index) => ({
...row,
_key: `${row.serialNumberHex ?? row.serialNumber ?? "entry"}-${index}`,
}));
return (
<>
<DataTable
columns={columns}
rows={rows}
rowKey={(row) => row._key}
emptyTitle="No revoked certificates"
caption="Revoked certificates"
/>
<CursorPagerControls
page={pager.page}
canPrev={pager.canPrev}
hasNext={query.data.nextCursor != null}
loading={query.isFetching}
onPrev={pager.goPrev}
onNext={() => pager.goNext(query.data.nextCursor ?? null)}
itemCount={query.data.items.length}
/>
</>
);
}
function RoaView({ projection }: { projection: Record<string, unknown> }) {
const roa = asRecord(projection.roa);
const families = asArray(roa?.ipAddressFamilies);
return (
<>
<dl className="meta-grid" style={{ marginBottom: 16 }}>
<MetaRow label="AS ID" value={asNumber(roa?.asId) ?? asString(roa?.asId)} mono />
</dl>
{families.map((family, i) => {
const fam = asRecord(family);
const addresses = asArray(fam?.addresses);
return (
<div key={i}>
<h3 className="text-small" style={{ margin: "12px 0 6px" }}>
Address family {asString(fam?.afi) ?? `#${i + 1}`}
</h3>
<DataTable
columns={[
{
key: "prefix",
header: "Prefix",
render: (row: Record<string, unknown>) => (
<span className="mono">{asString(row.prefix) ?? "—"}</span>
),
},
{
key: "maxLength",
header: "maxLength",
numeric: true,
render: (row: Record<string, unknown>) => asNumber(row.maxLength) ?? "—",
},
]}
rows={addresses.map((a) => asRecord(a) ?? {})}
rowKey={(row) => `${asString(row.prefix)}-${asNumber(row.maxLength)}`}
emptyTitle="No prefixes"
/>
</div>
);
})}
</>
);
}
function AspaView({ projection }: { projection: Record<string, unknown> }) {
const aspa = asRecord(projection.aspa);
const providers = asArray(aspa?.providerAsIds).map((p) => asNumber(p) ?? asString(p));
return (
<dl className="meta-grid">
<MetaRow label="Customer AS" value={asNumber(aspa?.customerAsId) ?? asString(aspa?.customerAsId)} mono />
<MetaRow
label="Provider ASes"
value={
providers.length ? (
<span className="chip-list">
{providers.map((p, i) => (
<span className="chip" key={i}>AS{String(p)}</span>
))}
</span>
) : (
"—"
)
}
/>
</dl>
);
}
function CertificateView({ projection }: { projection: Record<string, unknown> }) {
const cert = asRecord(projection.certificate) ?? asRecord(projection.cer) ?? projection;
const validity = asRecord(cert.validity);
const resources = asRecord(cert.resources);
const ipBlocks = asArray(resources?.ipResources ?? resources?.ipv4 ?? resources?.addresses);
const asBlocks = asArray(resources?.asResources ?? resources?.asns);
return (
<dl className="meta-grid">
<MetaRow label="Serial number" value={asString(cert.serialNumberHex ?? cert.serialNumber)} mono />
<MetaRow label="Issuer" value={asString(cert.issuer)} mono />
<MetaRow label="Subject" value={asString(cert.subject)} mono />
<MetaRow
label="Validity"
value={`${asString(validity?.notBefore) ?? "—"} → ${asString(validity?.notAfter) ?? "—"}`}
mono
/>
<MetaRow label="SKI" value={<CopyableValue value={asString(cert.ski)} max={56} label="SKI" />} />
<MetaRow label="AKI" value={<CopyableValue value={asString(cert.aki)} max={56} label="AKI" />} />
<MetaRow label="SIA" value={<CopyableValue value={asString(cert.sia)} max={72} label="SIA" />} />
<MetaRow label="AIA" value={<CopyableValue value={asString(cert.aia)} max={72} label="AIA" />} />
<MetaRow label="CRL DP" value={<CopyableValue value={asString(cert.crlDistributionPoint ?? cert.crldp)} max={72} label="CRL distribution point" />} />
<MetaRow
label="IP resources"
value={
ipBlocks.length ? (
<span className="chip-list">
{ipBlocks.slice(0, 12).map((b, i) => (
<span className="chip" key={i}>{typeof b === "string" ? b : JSON.stringify(b)}</span>
))}
{ipBlocks.length > 12 ? <span className="chip">+{ipBlocks.length - 12} more</span> : null}
</span>
) : (
"—"
)
}
/>
<MetaRow
label="AS resources"
value={
asBlocks.length ? (
<span className="chip-list">
{asBlocks.slice(0, 12).map((b, i) => (
<span className="chip" key={i}>{typeof b === "string" || typeof b === "number" ? String(b) : JSON.stringify(b)}</span>
))}
{asBlocks.length > 12 ? <span className="chip">+{asBlocks.length - 12} more</span> : null}
</span>
) : (
"—"
)
}
/>
<MetaRow label="Signature algorithm" value={asString(cert.signatureAlgorithm)} mono />
</dl>
);
}
/**
* Structured rendering of a parsed object projection, per object type.
*/
export function ProjectionView({
record,
runId,
objectInstanceId,
}: {
record: ProjectionRecord | null;
runId: string;
objectInstanceId: string;
}) {
if (!record || !record.projection) {
return (
<EmptyBlock
title="Parsed projection unavailable"
hint="The query service needs --repo-bytes-db to build parsed projections for this object."
/>
);
}
if (record.parseStatus === "error") {
return <Notice kind="error">Projection failed: {record.errorSummary ?? "unknown parse error"}</Notice>;
}
const projection = record.projection;
const type = (record.objectType ?? "").toLowerCase();
if (type === "roa") return <RoaView projection={projection} />;
if (type === "asa" || type === "aspa") return <AspaView projection={projection} />;
if (type === "mft") {
const manifest = asRecord(projection.manifest);
return (
<>
<dl className="meta-grid" style={{ marginBottom: 16 }}>
<MetaRow label="Manifest number" value={asString(manifest?.manifestNumberHex ?? manifest?.manifestNumber)} mono />
<MetaRow label="thisUpdate" value={formatUtc(asString(manifest?.thisUpdate))} />
<MetaRow label="nextUpdate" value={formatUtc(asString(manifest?.nextUpdate))} />
<MetaRow label="File hash algorithm" value={asString(manifest?.fileHashAlg)} mono />
<MetaRow label="File count" value={formatInt(asNumber(manifest?.fileCount))} />
</dl>
<ManifestFiles runId={runId} objectInstanceId={objectInstanceId} />
</>
);
}
if (type === "crl") {
const crl = asRecord(projection.crl);
return (
<>
<dl className="meta-grid" style={{ marginBottom: 16 }}>
<MetaRow label="Issuer" value={asString(crl?.issuer)} mono />
<MetaRow label="thisUpdate" value={formatUtc(asString(crl?.thisUpdate))} />
<MetaRow label="nextUpdate" value={formatUtc(asString(crl?.nextUpdate))} />
<MetaRow label="CRL number" value={asString(crl?.crlNumberHex ?? crl?.crlNumber)} mono />
<MetaRow label="AKI" value={<CopyableValue value={asString(crl?.aki)} max={56} label="AKI" />} />
</dl>
<RevokedCerts runId={runId} objectInstanceId={objectInstanceId} />
</>
);
}
if (type === "cer" || type === "ee" || type === "router_cert") {
return <CertificateView projection={projection} />;
}
return (
<pre className="json-block" data-testid="projection-json">
{JSON.stringify(projection, null, 2)}
</pre>
);
}

View File

@@ -1,176 +0,0 @@
import { useState, type FormEvent, type ReactNode } from "react";
import { NavLink, useNavigate } from "react-router-dom";
import { useQuery } from "@tanstack/react-query";
import {
Braces,
FileBox,
FolderTree,
History,
LayoutDashboard,
PackageOpen,
PanelLeftClose,
PanelLeftOpen,
Search as SearchIcon,
Server,
ShieldCheck,
} from "lucide-react";
import { getHealth, listRuns } from "../api/service";
import { LATEST_RUN, useRunParam, useSetRunParam } from "../lib/run";
import { StatusPill } from "./StatusPill";
const NAV_ITEMS = [
{ path: "/", label: "Overview", icon: LayoutDashboard, end: true },
{ path: "/repositories", label: "Repositories", icon: Server, end: false },
{ path: "/publication-points", label: "Publication Points", icon: FolderTree, end: false },
{ path: "/objects", label: "Objects", icon: FileBox, end: false },
{ path: "/validation", label: "Validation", icon: ShieldCheck, end: false },
{ path: "/exports", label: "Exports", icon: PackageOpen, end: false },
{ path: "/runs", label: "Runs", icon: History, end: false },
{ path: "/api", label: "API", icon: Braces, end: false },
];
const COLLAPSE_KEY = "rpki-explorer.sidebar-collapsed";
function ConnectionPill() {
const healthQuery = useQuery({
queryKey: ["health"],
queryFn: getHealth,
refetchInterval: 60_000,
});
if (healthQuery.isPending) {
return <StatusPill status="pending" label="connecting" />;
}
if (healthQuery.isError) {
return <StatusPill status="error" label="API unreachable" />;
}
const health = healthQuery.data;
return (
<span title={health.latestReadyRun ? `Latest indexed run: ${health.latestReadyRun}` : undefined}>
<StatusPill status="ok" label="connected" />
</span>
);
}
function RunSelector() {
const runParam = useRunParam();
const setRunParam = useSetRunParam();
const runsQuery = useQuery({
queryKey: ["runs", "selector"],
queryFn: () => listRuns({ limit: 25 }),
staleTime: 60_000,
});
const healthQuery = useQuery({ queryKey: ["health"], queryFn: getHealth });
const latestId = healthQuery.data?.latestReadyRun ?? runsQuery.data?.items[0]?.runId;
return (
<div className="topbar-run">
<label htmlFor="run-selector">Run</label>
<select
id="run-selector"
value={runParam ?? ""}
onChange={(event) => setRunParam(event.target.value || null)}
>
<option value="">{latestId ? `latest (${latestId})` : LATEST_RUN}</option>
{(runsQuery.data?.items ?? []).map((run) => (
<option key={run.runId} value={run.runId}>
{run.runId}
</option>
))}
</select>
</div>
);
}
function TopbarSearch() {
const navigate = useNavigate();
const runParam = useRunParam();
const [value, setValue] = useState("");
const onSubmit = (event: FormEvent) => {
event.preventDefault();
const q = value.trim();
if (!q) return;
const params = new URLSearchParams({ q });
if (runParam) params.set("run", runParam);
navigate(`/search?${params.toString()}`);
};
return (
<form className="topbar-search" role="search" onSubmit={onSubmit}>
<span className="search-icon" aria-hidden="true">
<SearchIcon size={14} />
</span>
<input
type="search"
value={value}
onChange={(event) => setValue(event.target.value)}
placeholder="Search URI, SHA-256, repository host…"
aria-label="Global search"
/>
</form>
);
}
export function Shell({ children }: { children: ReactNode }) {
const runParam = useRunParam();
const [collapsed, setCollapsed] = useState(
() => localStorage.getItem(COLLAPSE_KEY) === "1",
);
const toggleCollapsed = () => {
setCollapsed((prev) => {
localStorage.setItem(COLLAPSE_KEY, prev ? "0" : "1");
return !prev;
});
};
const search = runParam ? `?run=${encodeURIComponent(runParam)}` : "";
return (
<div className={`shell${collapsed ? " collapsed" : ""}`}>
<div className="shell-brand">
<span className="brand-badge" aria-hidden="true">
<ShieldCheck size={15} />
</span>
<span className="brand-name">RPKI Explorer</span>
</div>
<div className="shell-topbar">
<TopbarSearch />
<RunSelector />
<ConnectionPill />
</div>
<nav className="shell-nav" aria-label="Primary">
<button
type="button"
className="nav-toggle"
onClick={toggleCollapsed}
aria-label={collapsed ? "Expand sidebar" : "Collapse sidebar"}
>
{collapsed ? (
<PanelLeftOpen size={15} aria-hidden="true" />
) : (
<PanelLeftClose size={15} aria-hidden="true" />
)}
{!collapsed && <span>Collapse</span>}
</button>
{NAV_ITEMS.map((item) => (
<NavLink
key={item.path}
to={{ pathname: item.path, search }}
end={item.end}
className={({ isActive }) => (isActive ? "active" : undefined)}
>
<item.icon size={15} aria-hidden="true" />
<span className="nav-label">{item.label}</span>
</NavLink>
))}
<div className="nav-footer">ours RP query layer</div>
</nav>
<main className="shell-main">{children}</main>
</div>
);
}

View File

@@ -1,73 +0,0 @@
import { AlertTriangle, Inbox, RefreshCw } from "lucide-react";
import { ApiError } from "../api/client";
export function LoadingBlock({ label = "Loading…" }: { label?: string }) {
return (
<div className="state-block" role="status" aria-live="polite">
<span className="spinner" aria-hidden="true" />
<span>{label}</span>
</div>
);
}
export function ErrorBlock({
error,
onRetry,
title = "Failed to load data",
}: {
error: unknown;
onRetry?: () => void;
title?: string;
}) {
const message =
error instanceof ApiError
? `${error.message}${error.status ? ` (HTTP ${error.status})` : ""}`
: error instanceof Error
? error.message
: "Unknown error";
return (
<div className="banner error" role="alert">
<AlertTriangle size={16} aria-hidden="true" />
<div className="banner-body">
<strong>{title}.</strong> {message}
</div>
{onRetry ? (
<button type="button" className="btn small" onClick={onRetry}>
<RefreshCw size={13} aria-hidden="true" /> Retry
</button>
) : null}
</div>
);
}
export function EmptyBlock({
title = "No data",
hint,
}: {
title?: string;
hint?: string;
}) {
return (
<div className="state-block">
<Inbox size={22} aria-hidden="true" />
<span className="state-title">{title}</span>
{hint ? <span className="text-muted text-small">{hint}</span> : null}
</div>
);
}
/** Inline warning/info banner. */
export function Notice({
kind,
children,
}: {
kind: "info" | "warning" | "error";
children: React.ReactNode;
}) {
return (
<div className={`banner ${kind}`} role={kind === "error" ? "alert" : "status"}>
<AlertTriangle size={16} aria-hidden="true" />
<div className="banner-body">{children}</div>
</div>
);
}

View File

@@ -1,54 +0,0 @@
/**
* Status pill with a deterministic status → tone mapping. The label is
* always rendered as text (status never relies on color alone).
*/
type Tone = "green" | "amber" | "red" | "blue" | "slate";
const TONE_BY_STATUS: Record<string, Tone> = {
ok: "green",
valid: "green",
ready: "green",
complete: "green",
fresh: "green",
success: "green",
linked: "green",
connected: "green",
running: "blue",
building: "amber",
warnings: "amber",
warning: "amber",
cached: "amber",
partial: "amber",
stale: "amber",
error: "red",
invalid: "red",
failed: "red",
rejected: "red",
failed_no_cache: "red",
missing: "red",
skipped: "slate",
unknown: "slate",
pending: "slate",
};
export function toneForStatus(status: string | null | undefined): Tone {
if (!status) return "slate";
return TONE_BY_STATUS[status.toLowerCase()] ?? "slate";
}
export function StatusPill({
status,
label,
}: {
status: string | null | undefined;
label?: string;
}) {
const text = label ?? status ?? "unknown";
return (
<span className={`pill tone-${toneForStatus(status)}`}>
<span className="pill-dot" aria-hidden="true" />
{text}
</span>
);
}

View File

@@ -1,79 +0,0 @@
import { useRef } from "react";
import type { KeyboardEvent, ReactNode } from "react";
export interface TabItem {
id: string;
label: ReactNode;
}
/**
* Accessible tab strip with roving tabindex and arrow-key navigation.
* Controlled: the parent owns the active id (usually synced to the URL).
*/
export function Tabs({
tabs,
active,
onChange,
ariaLabel,
}: {
tabs: TabItem[];
active: string;
onChange: (id: string) => void;
ariaLabel: string;
}) {
const refs = useRef(new Map<string, HTMLButtonElement>());
const onKeyDown = (event: KeyboardEvent<HTMLDivElement>) => {
if (event.key !== "ArrowRight" && event.key !== "ArrowLeft") return;
const index = tabs.findIndex((t) => t.id === active);
if (index < 0) return;
const delta = event.key === "ArrowRight" ? 1 : -1;
const next = tabs[(index + delta + tabs.length) % tabs.length];
event.preventDefault();
onChange(next.id);
refs.current.get(next.id)?.focus();
};
return (
<div className="tabs" role="tablist" aria-label={ariaLabel} onKeyDown={onKeyDown}>
{tabs.map((tab) => {
const selected = tab.id === active;
return (
<button
key={tab.id}
ref={(el) => {
if (el) refs.current.set(tab.id, el);
else refs.current.delete(tab.id);
}}
type="button"
role="tab"
id={`tab-${tab.id}`}
aria-selected={selected}
aria-controls={`tabpanel-${tab.id}`}
tabIndex={selected ? 0 : -1}
onClick={() => onChange(tab.id)}
>
{tab.label}
</button>
);
})}
</div>
);
}
export function TabPanel({
id,
active,
children,
}: {
id: string;
active: string;
children: ReactNode;
}) {
if (id !== active) return null;
return (
<div role="tabpanel" id={`tabpanel-${id}`} aria-labelledby={`tab-${id}`}>
{children}
</div>
);
}

View File

@@ -1,44 +0,0 @@
import { CheckCircle2, XCircle } from "lucide-react";
import type { ExportJobRecord } from "../api/schemas";
import { exportDownloadUrl } from "../api/service";
import { formatInt } from "../lib/format";
/**
* Presentational status line for an async export job. Polling is the
* caller's responsibility (react-query `refetchInterval`).
*/
export function WorkflowStatus({
job,
runId,
}: {
job: ExportJobRecord;
runId: string;
}) {
if (job.status === "complete") {
return (
<span className="workflow-status" role="status">
<CheckCircle2 size={15} aria-hidden="true" color="var(--green-600)" />
<span>
Export ready · {formatInt(job.objectCount)} objects
</span>
<a className="btn small primary" href={exportDownloadUrl(runId, job.jobId)} download>
Download tar
</a>
</span>
);
}
if (job.status === "failed") {
return (
<span className="workflow-status" role="alert">
<XCircle size={15} aria-hidden="true" color="var(--red-600)" />
<span>Export failed{job.error ? `: ${job.error}` : ""}</span>
</span>
);
}
return (
<span className="workflow-status" role="status" aria-live="polite">
<span className="spinner small" aria-hidden="true" />
<span>Export running…</span>
</span>
);
}

View File

@@ -0,0 +1,88 @@
import { Check, Copy } from "lucide-react";
import { useEffect, useState } from "react";
interface CopyableValueProps {
className?: string;
displayValue?: string;
label: string;
value: string;
}
interface TooltipPosition {
left: number;
top: number;
}
export function CopyableValue({ className, displayValue, label, value }: CopyableValueProps) {
const [copyState, setCopyState] = useState<"idle" | "copied" | "failed">("idle");
const [tooltipPosition, setTooltipPosition] = useState<TooltipPosition | null>(null);
useEffect(() => {
if (copyState === "idle") {
return;
}
const timeout = window.setTimeout(() => setCopyState("idle"), 1600);
return () => window.clearTimeout(timeout);
}, [copyState]);
const showTooltip = (target: HTMLElement) => {
const rect = target.getBoundingClientRect();
setTooltipPosition({
left: Math.min(window.innerWidth - 24, Math.max(24, rect.left + rect.width / 2)),
top: rect.top > 90 ? rect.top - 10 : rect.bottom + 10
});
};
const copy = async () => {
try {
await copyText(value);
setCopyState("copied");
} catch {
setCopyState("failed");
}
};
return (
<span className={`copyable-value ${className ?? ""}`} data-copy-state={copyState} data-copyable-label={label}>
<span
className="copyable-value-text"
onBlur={() => setTooltipPosition(null)}
onFocus={(event) => showTooltip(event.currentTarget)}
onMouseEnter={(event) => showTooltip(event.currentTarget)}
onMouseLeave={() => setTooltipPosition(null)}
tabIndex={0}
title={value}
>
{displayValue ?? value}
</span>
<button aria-label={`Copy ${label}`} className="copy-icon-button" onClick={copy} title={`Copy ${label}`} type="button">
{copyState === "copied" ? <Check aria-hidden="true" size={13} /> : <Copy aria-hidden="true" size={13} />}
</button>
{tooltipPosition ? (
<span className="copyable-tooltip" role="tooltip" style={{ left: tooltipPosition.left, top: tooltipPosition.top }}>
{value}
</span>
) : null}
{copyState !== "idle" ? <span className="copy-state-text">{copyState === "copied" ? "Copied" : "Copy failed"}</span> : null}
</span>
);
}
async function copyText(value: string) {
if (navigator.clipboard) {
await navigator.clipboard.writeText(value);
return;
}
const textArea = document.createElement("textarea");
textArea.value = value;
textArea.style.position = "fixed";
textArea.style.opacity = "0";
document.body.appendChild(textArea);
textArea.focus();
textArea.select();
try {
document.execCommand("copy");
} finally {
document.body.removeChild(textArea);
}
}

View File

@@ -0,0 +1,32 @@
interface CursorPagerProps {
label?: string;
isFetching: boolean;
nextCursor: string | null;
onNext: () => void;
onPrevious: () => void;
pageNumber: number;
previousCount: number;
rangeLabel?: string;
}
export function CursorPager({
label,
isFetching,
nextCursor,
onNext,
onPrevious,
pageNumber,
previousCount,
rangeLabel
}: CursorPagerProps) {
return (
<div className="cursor-pager" aria-label="Cursor pagination">
<button disabled={isFetching || previousCount === 0} onClick={onPrevious} type="button">Previous</button>
<span>
{label ?? `Page ${pageNumber}`}
{rangeLabel ? <small>{rangeLabel}</small> : null}
</span>
<button disabled={isFetching || !nextCursor} onClick={onNext} type="button">Next</button>
</div>
);
}

View File

@@ -0,0 +1,24 @@
export interface PageState {
cursor: string | null;
previous: Array<string | null>;
}
export function advancePage(page: PageState, nextCursor: string | null): PageState {
if (!nextCursor) {
return page;
}
return {
cursor: nextCursor,
previous: [...page.previous, page.cursor]
};
}
export function previousPage(page: PageState): PageState {
if (page.previous.length === 0) {
return page;
}
return {
cursor: page.previous[page.previous.length - 1] ?? null,
previous: page.previous.slice(0, -1)
};
}

View File

@@ -0,0 +1,78 @@
import type { LucideIcon } from "lucide-react";
import { PanelLeftClose, PanelLeftOpen, Search } from "lucide-react";
import type { PropsWithChildren } from "react";
import { useState } from "react";
export interface NavigationItem<Id extends string = string> {
id: Id;
label: string;
icon: LucideIcon;
}
interface ShellProps<Id extends string> extends PropsWithChildren {
activeView: Id;
navigationItems: Array<NavigationItem<Id>>;
onNavigate: (id: Id) => void;
}
export function Shell<Id extends string>({ activeView, navigationItems, onNavigate, children }: ShellProps<Id>) {
const [isSidebarCollapsed, setIsSidebarCollapsed] = useState(false);
return (
<div className={`app-shell ${isSidebarCollapsed ? "sidebar-collapsed" : ""}`}>
<aside className="sidebar" aria-label="Primary navigation">
<div className="brand">
<div className="brand-mark">R</div>
<div className="brand-copy">
<h1 className="brand-title">RPKI Explorer</h1>
<div className="brand-subtitle">Validation Intelligence</div>
</div>
</div>
<nav className="nav-list">
{navigationItems.map((item) => (
<button
aria-current={activeView === item.id ? "page" : undefined}
aria-label={isSidebarCollapsed ? item.label : undefined}
className={`nav-item ${activeView === item.id ? "active" : ""}`}
key={item.id}
onClick={() => onNavigate(item.id)}
title={isSidebarCollapsed ? item.label : undefined}
type="button"
>
<item.icon aria-hidden="true" size={18} />
<span>{item.label}</span>
</button>
))}
</nav>
<button
aria-expanded={!isSidebarCollapsed}
aria-label={isSidebarCollapsed ? "Expand navigation" : "Collapse navigation"}
className="sidebar-toggle"
onClick={() => setIsSidebarCollapsed((collapsed) => !collapsed)}
title={isSidebarCollapsed ? "Expand navigation" : "Collapse navigation"}
type="button"
>
{isSidebarCollapsed ? <PanelLeftOpen aria-hidden="true" size={18} /> : <PanelLeftClose aria-hidden="true" size={18} />}
<span>{isSidebarCollapsed ? "Expand" : "Collapse"}</span>
</button>
</aside>
<main className="main-panel">
<header className="topbar">
<div className="run-selector readonly-run" aria-label="Current run">
<span>Run</span>
<strong>latest indexed run</strong>
</div>
<div className="topbar-actions">
<label className="search-box">
<Search aria-hidden="true" size={18} />
<span className="sr-only">Exact object URI search is not active yet</span>
<input disabled placeholder="Exact URI lookup planned in M2" />
</label>
<div className="ready-pill"><span /> UI Ready</div>
</div>
</header>
{children}
</main>
</div>
);
}

View File

@@ -0,0 +1,509 @@
import { useMutation, useQuery } from "@tanstack/react-query";
import { Check, Copy, Download, FileText, GitBranch, Loader2, Search, ShieldCheck } from "lucide-react";
import { useEffect, useMemo, useState } from "react";
import { normalizeApiError } from "../../api/client";
import { CopyableValue } from "../../components/common/CopyableValue";
import {
createObjectSetExport,
createPublicationPointExport,
downloadRawObject,
explainObjectValidation,
getLatestRun,
getObject,
getObjectByUri,
getObjectChain,
getObjectParsed,
getObjectValidation,
listManifestFiles,
listRevokedCertificates,
type ChainEdgeRecord,
type ObjectInstanceRecord,
type ObjectValidationRecord,
type ValidationExplainRecord
} from "../../api/queryService";
const tabs = ["Parsed", "Validation", "Chain", "Manifest files", "Revoked certs"] as const;
type ObjectTab = (typeof tabs)[number];
interface ObjectDetailPageProps {
initialObjectInstanceId?: string | null;
}
export function ObjectDetailPage({ initialObjectInstanceId }: ObjectDetailPageProps) {
const [selectedObjectInstanceId, setSelectedObjectInstanceId] = useState<string | null>(initialObjectInstanceId ?? null);
const [activeTab, setActiveTab] = useState<ObjectTab>("Validation");
const [copyState, setCopyState] = useState<"idle" | "copied" | "failed">("idle");
const [searchUri, setSearchUri] = useState("");
const latestRunQuery = useQuery({ queryKey: ["object-detail-latest-run"], queryFn: getLatestRun });
const runId = latestRunQuery.data?.data.runId ?? "latest";
useEffect(() => {
if (initialObjectInstanceId && initialObjectInstanceId !== selectedObjectInstanceId) {
setSelectedObjectInstanceId(initialObjectInstanceId);
setActiveTab("Validation");
}
}, [initialObjectInstanceId, selectedObjectInstanceId]);
const objectQuery = useQuery({
enabled: Boolean(selectedObjectInstanceId && latestRunQuery.data?.data.runId),
queryKey: ["object-detail-object", runId, selectedObjectInstanceId],
queryFn: () => getObject(runId, selectedObjectInstanceId ?? ""),
staleTime: 5 * 60 * 1000
});
const selectedObject = objectQuery.data?.data ?? null;
const validationQuery = useQuery({
enabled: Boolean(selectedObjectInstanceId && latestRunQuery.data?.data.runId),
queryKey: ["object-detail-validation", runId, selectedObjectInstanceId],
queryFn: () => getObjectValidation(runId, selectedObjectInstanceId ?? ""),
staleTime: 5 * 60 * 1000
});
const parsedQuery = useQuery({
enabled: activeTab === "Parsed" && Boolean(selectedObjectInstanceId && latestRunQuery.data?.data.runId),
queryKey: ["object-detail-parsed", runId, selectedObjectInstanceId],
queryFn: () => getObjectParsed(runId, selectedObjectInstanceId ?? ""),
staleTime: 5 * 60 * 1000
});
const chainQuery = useQuery({
enabled: activeTab === "Chain" && Boolean(selectedObjectInstanceId && latestRunQuery.data?.data.runId),
queryKey: ["object-detail-chain", runId, selectedObjectInstanceId],
queryFn: () => getObjectChain(runId, selectedObjectInstanceId ?? ""),
staleTime: 5 * 60 * 1000
});
const manifestFilesQuery = useQuery({
enabled: activeTab === "Manifest files" && Boolean(selectedObjectInstanceId && latestRunQuery.data?.data.runId),
retry: false,
queryKey: ["object-detail-manifest-files", runId, selectedObjectInstanceId],
queryFn: () => listManifestFiles(runId, selectedObjectInstanceId ?? "", 50),
staleTime: 5 * 60 * 1000
});
const revokedCertsQuery = useQuery({
enabled: activeTab === "Revoked certs" && Boolean(selectedObjectInstanceId && latestRunQuery.data?.data.runId),
retry: false,
queryKey: ["object-detail-revoked-certs", runId, selectedObjectInstanceId],
queryFn: () => listRevokedCertificates(runId, selectedObjectInstanceId ?? "", 50),
staleTime: 5 * 60 * 1000
});
const explainMutation = useMutation({
mutationFn: () => explainObjectValidation(runId, selectedObjectInstanceId ?? "")
});
const uriSearchMutation = useMutation({
mutationFn: (uri: string) => getObjectByUri(runId, uri),
onSuccess: (result) => {
setSelectedObjectInstanceId(result.data.objectInstanceId);
setActiveTab("Validation");
}
});
const rawDownloadMutation = useMutation({
mutationFn: () => downloadRawObject(runId, selectedObjectInstanceId ?? "")
});
const objectExportMutation = useMutation({
mutationFn: () => createObjectSetExport(runId, selectedObjectInstanceId ? [selectedObjectInstanceId] : [])
});
const ppExportMutation = useMutation({
mutationFn: () => {
if (!selectedObject?.ppId) {
throw new Error("No publication point selected");
}
return createPublicationPointExport(runId, selectedObject.ppId);
}
});
useEffect(() => {
if (copyState === "idle") {
return;
}
const timeout = window.setTimeout(() => setCopyState("idle"), 1800);
return () => window.clearTimeout(timeout);
}, [copyState]);
const copySelectedUri = async () => {
if (!selectedObject?.uri) {
return;
}
try {
await navigator.clipboard.writeText(selectedObject.uri);
setCopyState("copied");
} catch {
setCopyState("failed");
}
};
const validation = validationQuery.data?.data ?? null;
const apiError = latestRunQuery.error ?? objectQuery.error ?? validationQuery.error;
return (
<section className="page-stack object-page" aria-labelledby="object-detail-heading">
{apiError ? <ErrorBanner error={apiError} /> : null}
<div className="object-detail-only-layout">
<aside className="object-detail-card object-detail-card-expanded" aria-label="Live object detail">
<div className="object-detail-toolbar">
<form
className="search-box object-uri-search"
onSubmit={(event) => {
event.preventDefault();
if (searchUri.trim()) {
uriSearchMutation.mutate(searchUri.trim());
}
}}
>
<Search aria-hidden="true" size={17} />
<span className="sr-only">Search exact object URI</span>
<input
onChange={(event) => setSearchUri(event.target.value)}
placeholder="Exact URI lookup..."
value={searchUri}
/>
</form>
<button className="filter-pill" disabled={!selectedObject?.uri} type="button" onClick={() => selectedObject && setSearchUri(selectedObject.uri)}>Use selected URI</button>
<strong>{selectedObject ? `${labelObjectType(selectedObject.objectType)} selected` : "No object selected"}</strong>
</div>
{!selectedObjectInstanceId ? <div className="empty-state">Search an exact URI or open an object from Repository Browser.</div> : null}
{selectedObjectInstanceId && objectQuery.isFetching ? <LoadingLine label="Loading selected object..." /> : null}
<div className="object-hero">
<div>
<p className="eyebrow">Object detail · live query service</p>
<h2 id="object-detail-heading">{selectedObject ? labelObjectType(selectedObject.objectType) : "Object"} object</h2>
{selectedObject ? (
<p><CopyableValue className="inline-copyable" label="selected object URI" value={selectedObject.uri} /></p>
) : (
<p>Select an object from the publication point list.</p>
)}
</div>
<div className="object-actions">
<StatusPill state={validation?.finalStatus ?? selectedObject?.result ?? "loading"} />
<button className="ghost-button" disabled={!selectedObject?.uri} onClick={copySelectedUri} type="button">
<Copy size={16} /> {copyState === "copied" ? "Copied" : copyState === "failed" ? "Copy failed" : "Copy URI"}
</button>
<button className="ghost-button" disabled={!selectedObjectInstanceId} onClick={() => objectExportMutation.mutate()} type="button">
<FileText size={16} /> Export object
</button>
<button className="primary-button" disabled={!selectedObjectInstanceId} onClick={() => rawDownloadMutation.mutate()} type="button">
<Download size={16} /> Download raw
</button>
</div>
</div>
{selectedObject ? <ObjectMeta object={selectedObject} validation={validation} /> : <div className="empty-state">No object selected.</div>}
<WorkflowStatus
objectExport={objectExportMutation.data?.data ?? null}
objectExportError={objectExportMutation.error}
ppExport={ppExportMutation.data?.data ?? null}
ppExportError={ppExportMutation.error}
rawError={rawDownloadMutation.error}
rawPending={rawDownloadMutation.isPending}
searchError={uriSearchMutation.error}
searchPending={uriSearchMutation.isPending}
/>
<div className="object-actions inline-actions">
<button className="ghost-button" disabled={!selectedObject?.ppId} onClick={() => ppExportMutation.mutate()} type="button">
<FileText size={16} /> Export selected PP
</button>
</div>
<div className="tabs" role="tablist" aria-label="Object detail tabs" onKeyDown={(event) => {
if (event.key !== "ArrowLeft" && event.key !== "ArrowRight") {
return;
}
event.preventDefault();
const currentIndex = tabs.indexOf(activeTab);
const nextIndex = event.key === "ArrowRight"
? (currentIndex + 1) % tabs.length
: (currentIndex - 1 + tabs.length) % tabs.length;
setActiveTab(tabs[nextIndex]);
}}>
{tabs.map((tab) => (
<button
aria-controls={`object-tabpanel-${tabId(tab)}`}
aria-selected={activeTab === tab}
className={activeTab === tab ? "active" : ""}
id={`object-tab-${tabId(tab)}`}
key={tab}
onClick={() => setActiveTab(tab)}
role="tab"
tabIndex={activeTab === tab ? 0 : -1}
type="button"
>
{tab}
</button>
))}
</div>
<div
aria-labelledby={`object-tab-${tabId(activeTab)}`}
className="object-content-grid"
id={`object-tabpanel-${tabId(activeTab)}`}
role="tabpanel"
>
{activeTab === "Parsed" ? <ParsedPanel parsed={parsedQuery.data?.data ?? null} isFetching={parsedQuery.isFetching} error={parsedQuery.error} /> : null}
{activeTab === "Validation" ? (
<ValidationPanel validation={validation} isFetching={validationQuery.isFetching} explain={explainMutation.data?.data ?? null} explainPending={explainMutation.isPending} onExplain={() => explainMutation.mutate()} explainError={explainMutation.error} />
) : null}
{activeTab === "Chain" ? <ChainPanel edges={chainQuery.data?.data ?? []} isFetching={chainQuery.isFetching} error={chainQuery.error} /> : null}
{activeTab === "Manifest files" ? <ArrayPanel title="Manifest files" rows={manifestFilesQuery.data?.data ?? []} isFetching={manifestFilesQuery.isFetching} error={manifestFilesQuery.error} /> : null}
{activeTab === "Revoked certs" ? <ArrayPanel title="Revoked certificates" rows={revokedCertsQuery.data?.data ?? []} isFetching={revokedCertsQuery.isFetching} error={revokedCertsQuery.error} /> : null}
</div>
</aside>
</div>
</section>
);
}
function ObjectMeta({ object, validation }: { object: ObjectInstanceRecord; validation: ObjectValidationRecord | null }) {
return (
<div className="object-meta-grid">
<Meta label="SHA256" value={object.sha256} copyable />
<Meta label="Source" value={object.sourceSection} />
<Meta label="Repository" value={object.repoId} copyable />
<Meta label="Publication Point" value={object.ppId} copyable />
<Meta label="Audit Result" value={validation?.auditResult ?? object.result} />
<Meta label="Authoritative" value={validation ? String(validation.authoritative) : "loading"} />
</div>
);
}
function WorkflowStatus({
objectExport,
objectExportError,
ppExport,
ppExportError,
rawError,
rawPending,
searchError,
searchPending
}: {
objectExport: { jobId?: string; status?: string } | null;
objectExportError: unknown;
ppExport: { jobId?: string; status?: string } | null;
ppExportError: unknown;
rawError: unknown;
rawPending: boolean;
searchError: unknown;
searchPending: boolean;
}) {
const lines = [
searchPending ? "Searching object URI..." : null,
searchError ? `URI search failed: ${normalizeApiError(searchError).message}` : null,
rawPending ? "Downloading raw object..." : null,
rawError ? `Raw download failed: ${normalizeApiError(rawError).message}` : null,
objectExport ? `Object export job ${objectExport.jobId ?? ""} ${objectExport.status ?? ""}`.trim() : null,
objectExportError ? `Object export failed: ${normalizeApiError(objectExportError).message}` : null,
ppExport ? `PP export job ${ppExport.jobId ?? ""} ${ppExport.status ?? ""}`.trim() : null,
ppExportError ? `PP export failed: ${normalizeApiError(ppExportError).message}` : null
].filter((line): line is string => Boolean(line));
if (lines.length === 0) {
return null;
}
return (
<section className="panel workflow-status">
<PanelHeading eyebrow="Workflow" title="Search / raw / export status" />
<ul className="workflow-status-list">
{lines.map((line) => (
<li key={line}>{line}</li>
))}
</ul>
</section>
);
}
function ParsedPanel({ parsed, isFetching, error }: { parsed: unknown; isFetching: boolean; error: unknown }) {
if (isFetching) {
return <LoadingPanel title="Parsed projection" label="Loading parsed projection..." />;
}
if (error) {
return <NoticePanel title="Parsed projection" message={normalizeApiError(error).message} />;
}
if (!parsed) {
return <NoticePanel title="Parsed projection" message="Projection unavailable. The local query service was started without repo-bytes.db, so this object cannot be decoded on demand." />;
}
return (
<section className="panel parsed-panel">
<PanelHeading eyebrow="Parsed projection" title="Projection JSON" icon={<FileText className="panel-icon" size={22} />} />
<JsonPreview value={parsed} />
</section>
);
}
function ValidationPanel({ validation, isFetching, explain, explainPending, onExplain, explainError }: {
validation: ObjectValidationRecord | null;
isFetching: boolean;
explain: ValidationExplainRecord | null;
explainPending: boolean;
onExplain: () => void;
explainError: unknown;
}) {
if (isFetching) {
return <LoadingPanel title="Validation" label="Loading validation summary..." />;
}
return (
<section className="panel validation-panel">
<PanelHeading eyebrow="Validation" title="File and chain checks" icon={<ShieldCheck className="panel-icon success" size={22} />} />
<div className="check-list">
<ValidationCheck label="Final status" status={validation?.finalStatus ?? "unknown"} note={validation?.detailSummary ?? "Derived from run audit state."} />
<ValidationCheck label="File validation" status={validation?.fileValidation.status ?? "unknown"} note={issuesText(validation?.fileValidation.issues) ?? validation?.fileValidation.detailSummary ?? "No file issues recorded."} />
<ValidationCheck label="Chain validation" status={validation?.chainValidation.status ?? "unknown"} note={issuesText(validation?.chainValidation.issues) ?? validation?.chainValidation.note ?? "No chain issues recorded."} />
</div>
<div className="explain-box">
<button className="ghost-button" disabled={explainPending} onClick={onExplain} type="button">
{explainPending ? <Loader2 size={16} /> : <ShieldCheck size={16} />}
Explain validation
</button>
<span>Explain is an audit projection, not full authoritative revalidation.</span>
</div>
{explainError ? <div className="empty-state">Explain failed: {normalizeApiError(explainError).message}</div> : null}
{explain ? (
<div className="detail-list explain-result">
<Meta label="Mode" value={explain.explainMode} />
<Meta label="Authoritative" value={String(explain.authoritative)} />
<Meta label="Generated" value={explain.generatedAt} />
<Meta label="Edges" value={String(explain.chainEdges.length)} />
</div>
) : null}
</section>
);
}
function ChainPanel({ edges, isFetching, error }: { edges: ChainEdgeRecord[]; isFetching: boolean; error: unknown }) {
if (isFetching) {
return <LoadingPanel title="Certificate chain" label="Loading chain edges..." />;
}
if (error) {
return <NoticePanel title="Certificate chain" message={normalizeApiError(error).message} />;
}
return (
<section className="panel chain-panel">
<PanelHeading eyebrow="Certificate chain" title="Audit edge graph" icon={<GitBranch className="panel-icon" size={22} />} />
{edges.length === 0 ? <div className="empty-state">No chain edges recorded for this object.</div> : null}
<div className="chain-flow live-chain-flow">
{edges.map((edge) => (
<div className="chain-node" key={`${edge.relation}-${edge.toUri}`}>
<strong>{edge.relation}</strong>
<span>{edge.status}</span>
<code><CopyableValue label="chain edge URI" value={edge.toUri} /></code>
</div>
))}
</div>
</section>
);
}
function ArrayPanel({ title, rows, isFetching, error }: { title: string; rows: unknown[]; isFetching: boolean; error: unknown }) {
if (isFetching) {
return <LoadingPanel title={title} label={`Loading ${title.toLowerCase()}...`} />;
}
if (error) {
return <NoticePanel title={title} message={normalizeApiError(error).message} />;
}
return (
<section className="panel table-panel">
<PanelHeading eyebrow={title} title="Paged projection list" />
{rows.length === 0 ? <div className="empty-state">No rows returned for this object.</div> : null}
{rows.length > 0 ? <JsonPreview value={rows} /> : null}
</section>
);
}
function LoadingPanel({ title, label }: { title: string; label: string }) {
return (
<section className="panel">
<PanelHeading eyebrow={title} title={title} />
<LoadingLine label={label} />
</section>
);
}
function NoticePanel({ title, message }: { title: string; message: string }) {
return (
<section className="panel">
<PanelHeading eyebrow={title} title={title} />
<div className="empty-state">{message}</div>
</section>
);
}
function PanelHeading({ eyebrow, title, icon }: { eyebrow: string; title: string; icon?: React.ReactNode }) {
return (
<div className="panel-heading">
<div>
<p className="eyebrow">{eyebrow}</p>
<h3>{title}</h3>
</div>
{icon}
</div>
);
}
function ValidationCheck({ label, status, note }: { label: string; status: string; note: string }) {
const ok = status === "valid" || status === "ok";
return (
<article className={`check-item ${ok ? "" : "warning-check"}`}>
<Check size={16} />
<div>
<strong>{label}: {status}</strong>
<p>{note}</p>
</div>
</article>
);
}
function JsonPreview({ value }: { value: unknown }) {
const text = useMemo(() => JSON.stringify(value, null, 2), [value]);
return <pre className="json-preview">{text}</pre>;
}
function ErrorBanner({ error }: { error: unknown }) {
return (
<div className="alert-banner" role="status">
<strong>Object API unavailable</strong>
<span>{normalizeApiError(error).message}</span>
</div>
);
}
function StatusPill({ state }: { state: string }) {
const normalized = state.toLowerCase();
const className = normalized.includes("fail") || normalized.includes("reject") || normalized.includes("error") || normalized.includes("invalid")
? "warning"
: normalized.includes("cache") || normalized.includes("fallback")
? "fallback"
: "healthy";
return <span className={`status-badge ${className}`}>{state}</span>;
}
function LoadingLine({ label }: { label: string }) {
return (
<div className="loading-line">
<Loader2 size={16} />
{label}
</div>
);
}
function Meta({ copyable, label, value }: { copyable?: boolean; label: string; value: string }) {
return (
<div className="meta-item">
<span>{label}</span>
<strong>{copyable ? <CopyableValue label={label} value={value} /> : value}</strong>
</div>
);
}
function labelObjectType(type: string) {
const labels: Record<string, string> = {
certificate: "CER",
crl: "CRL",
manifest: "MFT",
roa: "ROA",
aspa: "ASPA"
};
return labels[type] ?? type.toUpperCase();
}
function tabId(tab: ObjectTab) {
return tab.toLowerCase().replaceAll(" ", "-");
}
function issuesText(issues: { summary?: string }[] | undefined) {
if (!issues || issues.length === 0) {
return null;
}
return issues.map((issue) => issue.summary).filter(Boolean).join("; ");
}

View File

@@ -0,0 +1,360 @@
import { useQuery } from "@tanstack/react-query";
import { AlertTriangle, ArrowUpRight, CheckCircle2, Clock3, ShieldCheck } from "lucide-react";
import { objectTypeRows, overviewKpis, topRepositories, validationIssues, validationSlices } from "../../test/fixtures/dashboard";
import {
getLatestRun,
getStatsObjectTypes,
getStatsReasons,
getStatsValidation,
listRepos,
type CountsByKey,
type RepositoryRecord,
type RunRecord
} from "../../api/queryService";
import { normalizeApiError } from "../../api/client";
import { CopyableValue } from "../../components/common/CopyableValue";
const validationColors = ["#0ca678", "#facc15", "#e11d48", "#94a3b8"];
const objectTypeLabels: Record<string, string> = {
aspa: "ASPA",
certificate: "CER / RC",
crl: "CRL",
gbr: "GBR",
manifest: "MFT",
roa: "ROA",
router_certificate: "Router Cert"
};
export function OverviewPage() {
const latestRunQuery = useQuery({
queryKey: ["latest-run"],
queryFn: getLatestRun
});
const liveRun = latestRunQuery.data?.data;
const runId = liveRun?.runId ?? "latest";
const enabled = Boolean(liveRun);
const objectTypesQuery = useQuery({
queryKey: ["stats-object-types", runId],
queryFn: () => getStatsObjectTypes(runId),
enabled
});
const validationQuery = useQuery({
queryKey: ["stats-validation", runId],
queryFn: () => getStatsValidation(runId),
enabled
});
const reasonsQuery = useQuery({
queryKey: ["stats-reasons", runId],
queryFn: () => getStatsReasons(runId),
enabled
});
const reposQuery = useQuery({
queryKey: ["top-repos", runId],
queryFn: () => listRepos(runId, 8),
enabled
});
const error = latestRunQuery.error ? normalizeApiError(latestRunQuery.error) : null;
const kpis = liveRun ? kpisFromRun(liveRun) : overviewKpis;
const objectRows = objectTypesQuery.data?.data ? objectRowsFromStats(objectTypesQuery.data.data) : objectTypeRows;
const validationRows = validationQuery.data?.data ? validationRowsFromStats(validationQuery.data.data) : validationSlices;
const validPercent = percent(validationQuery.data?.data?.ok ?? validationRows[0]?.value ?? 0, sumValues(validationQuery.data?.data) || 100);
const repoRows = reposQuery.data?.data ? reposFromApi(reposQuery.data.data) : reposFromFixtures();
const issueRows = reasonsQuery.data?.data ? issuesFromReasons(reasonsQuery.data.data) : validationIssues;
const statusText = liveRun ? statusForRun(liveRun) : "Static fixture";
const statusSubtext = liveRun ? `run ${liveRun.runSeq ?? liveRun.runId} · ${formatDuration(liveRun.wallMs)}` : "query service not connected";
return (
<section className="page-stack" aria-labelledby="overview-heading">
{error ? (
<div className="alert-banner" role="status">
<strong>Query service unavailable</strong>
<span>{error.message}. Showing static fallback data for layout inspection.</span>
</div>
) : null}
<div className="hero-dashboard">
<div>
<p className="eyebrow">{liveRun ? `Latest ready run · ${liveRun.syncMode ?? "unknown"} mode` : "Latest ready run · fallback"}</p>
<h2 id="overview-heading">Global RPKI validation health</h2>
<p>
{liveRun
? `Live query-service view for ${liveRun.runId}, validated at ${formatTimestamp(liveRun.validationTime)}.`
: "Static fallback showing how operators inspect validation output, repository health, and input quality."}
</p>
</div>
<div className="hero-status">
<ShieldCheck size={20} />
<div>
<strong>{statusText}</strong>
<span>{statusSubtext}</span>
</div>
</div>
</div>
<div className="metric-grid" aria-label="Overview KPI cards">
{kpis.map((metric) => (
<article className={`metric-card tone-${metric.tone}`} key={metric.label}>
<span>{metric.label}</span>
<strong>{metric.value}</strong>
<small>{metric.delta}</small>
</article>
))}
</div>
<div className="dashboard-grid">
<section className="panel validation-card">
<div className="panel-heading">
<div>
<p className="eyebrow">Validation</p>
<h3>Result distribution</h3>
</div>
<CheckCircle2 className="panel-icon success" size={22} />
</div>
<div className="donut-wrap">
<div className="donut" aria-label="Validation distribution donut" style={{ background: donutGradient(validationRows) }} />
<div className="donut-center">
<strong>{validPercent}%</strong>
<span>usable</span>
</div>
</div>
<div className="legend-list">
{validationRows.map((slice) => (
<div className="legend-row" key={slice.label}>
<span style={{ backgroundColor: slice.color }} />
<label>{slice.label}</label>
<strong>{slice.value}%</strong>
</div>
))}
</div>
</section>
<section className="panel">
<div className="panel-heading">
<div>
<p className="eyebrow">Objects</p>
<h3>Object type mix</h3>
</div>
<Clock3 className="panel-icon" size={22} />
</div>
<div className="bar-list">
{objectRows.map((row) => (
<div className="bar-row" key={row.type}>
<div className="bar-meta">
<strong>{row.type}</strong>
<span>{row.count}</span>
</div>
<div className="bar-track">
<span style={{ width: `${row.percent}%` }} />
</div>
</div>
))}
</div>
</section>
<section className="panel table-panel wide-panel">
<div className="panel-heading">
<div>
<p className="eyebrow">Repositories</p>
<h3>Top repositories by workload</h3>
</div>
<button className="ghost-button" type="button">
View all <ArrowUpRight size={16} />
</button>
</div>
<table>
<thead>
<tr>
<th>Host</th>
<th>Transport</th>
<th>Duration</th>
<th>Objects</th>
<th>Status</th>
</tr>
</thead>
<tbody>
{repoRows.map((repo) => (
<tr key={repo.id}>
<td>{repo.host}</td>
<td>{repo.transport}</td>
<td>{repo.duration}</td>
<td>{repo.objects}</td>
<td>
<span className={`status-badge ${repo.status}`}>{repo.status}</span>
</td>
</tr>
))}
</tbody>
</table>
</section>
<section className="panel issue-panel">
<div className="panel-heading">
<div>
<p className="eyebrow">Triage</p>
<h3>Recent validation issues</h3>
</div>
<AlertTriangle className="panel-icon warning" size={22} />
</div>
<div className="issue-list">
{issueRows.map((issue) => (
<article className="issue-card" key={issue.uri}>
<div>
<span className={`severity ${issue.severity}`}>{issue.severity}</span>
<strong>{issue.type}</strong>
</div>
<p>{issue.reason}</p>
<code><CopyableValue label="issue URI" value={issue.uri} /></code>
<small>{issue.repo}</small>
</article>
))}
</div>
</section>
</div>
</section>
);
}
function kpisFromRun(run: RunRecord) {
return [
{ label: "VRPs", value: formatNumber(run.counts.vrps), delta: "latest run", tone: "blue" },
{ label: "ASPAs", value: formatNumber(run.counts.aspas), delta: "latest run", tone: "cyan" },
{ label: "Objects", value: formatNumber(run.counts.objects), delta: "indexed inputs", tone: "blue" },
{ label: "Publication Points", value: formatNumber(run.counts.publicationPoints), delta: "indexed", tone: "purple" },
{ label: "Rejected Objects", value: formatNumber(run.counts.rejectedObjects), delta: "validation rejects", tone: "red" },
{ label: "Warnings", value: formatNumber(run.counts.warnings), delta: `${formatDuration(run.wallMs)} wall`, tone: "amber" }
];
}
function objectRowsFromStats(stats: CountsByKey) {
const total = sumValues(stats) || 1;
return Object.entries(stats)
.sort(([, left], [, right]) => right - left)
.slice(0, 7)
.map(([type, count]) => ({
type: objectTypeLabels[type] ?? type,
count: formatNumber(count),
percent: Math.max(1, Math.round((count / total) * 100))
}));
}
function validationRowsFromStats(stats: CountsByKey) {
const total = sumValues(stats) || 1;
return Object.entries(stats)
.sort(([, left], [, right]) => right - left)
.map(([label, count], index) => ({
label: titleCase(label),
value: Math.round((count / total) * 100),
color: validationColors[index] ?? "#94a3b8"
}));
}
function reposFromApi(repos: RepositoryRecord[]) {
return [...repos]
.sort((left, right) => right.objects - left.objects)
.slice(0, 6)
.map((repo) => ({
host: repo.host,
id: repo.repoId,
transport: repo.transport.toUpperCase(),
duration: formatDuration(repo.syncDurationMsTotal),
objects: formatNumber(repo.objects),
status: repoStatus(repo)
}));
}
function reposFromFixtures() {
return topRepositories.map((repo, index) => ({
...repo,
id: fallbackRepoId(repo, index)
}));
}
function fallbackRepoId(repo: { host: string; transport: string; duration: string; objects: string }, index: number) {
return `${repo.host}-${repo.transport}-${repo.duration}-${repo.objects}-${index}`;
}
function issuesFromReasons(reasons: CountsByKey) {
const entries = Object.entries(reasons).sort(([, left], [, right]) => right - left);
if (entries.length === 0) {
return [
{
severity: "low",
type: "OK",
reason: "No validation reject reasons reported in the latest run.",
uri: "query-service://validation/reasons",
repo: "latest run"
}
];
}
return entries.slice(0, 4).map(([reason, count], index) => ({
severity: index === 0 ? "high" : index === 1 ? "medium" : "low",
type: "Reject",
reason: `${formatNumber(count)} object(s): ${reason}`,
uri: `query-service://validation/reasons/${index + 1}`,
repo: "latest run"
}));
}
function repoStatus(repo: RepositoryRecord) {
if (repo.terminalStates.failed_no_cache || repo.phases.rrdp_failed_rsync_failed) {
return "warning";
}
if (repo.terminalStates.fallback_current_instance || repo.phases.rsync_ok) {
return "fallback";
}
return "healthy";
}
function statusForRun(run: RunRecord) {
if (run.counts.rejectedObjects > 0 || run.counts.warnings > 0) {
return "Healthy with warnings";
}
return "Healthy";
}
function donutGradient(rows: Array<{ value: number; color: string }>) {
let start = 0;
const parts = rows.map((row) => {
const end = Math.min(100, start + row.value);
const segment = `${row.color} ${start}% ${end}%`;
start = end;
return segment;
});
return `conic-gradient(${parts.join(", ")})`;
}
function sumValues(values?: CountsByKey) {
return Object.values(values ?? {}).reduce((sum, value) => sum + value, 0);
}
function percent(value: number, total: number) {
if (total <= 0) {
return 0;
}
return Math.round((value / total) * 100);
}
function formatNumber(value: number) {
return new Intl.NumberFormat("en-US").format(value);
}
function formatDuration(ms: number | null) {
if (ms === null || ms === undefined) {
return "—";
}
if (ms < 1000) {
return `${ms}ms`;
}
return `${(ms / 1000).toFixed(1)}s`;
}
function formatTimestamp(value: string | null) {
if (!value) {
return "unknown time";
}
return value.replace("T", " ").replace("Z", " UTC");
}
function titleCase(value: string) {
return value.replace(/_/g, " ").replace(/\b\w/g, (char) => char.toUpperCase());
}

View File

@@ -0,0 +1,467 @@
import { useQuery } from "@tanstack/react-query";
import { ChevronRight, Database, FileCode2, GitBranch, Loader2, PanelLeftClose, PanelRightClose } from "lucide-react";
import type { Dispatch, ReactNode, SetStateAction } from "react";
import { useMemo } from "react";
import { CopyableValue } from "../../components/common/CopyableValue";
import { CursorPager } from "../../components/common/CursorPager";
import { advancePage, previousPage, type PageState } from "../../components/common/cursorPaging";
import { createEmptyPageState, type RepositoryBrowserState } from "./repositoryBrowserState";
import {
getLatestRun,
listObjectsForPublicationPoint,
listPublicationPointsForRepo,
listRepos,
type ObjectInstanceRecord,
type PublicationPointRecord,
type RepositoryRecord
} from "../../api/queryService";
import { normalizeApiError } from "../../api/client";
const REPO_PAGE_SIZE = 50;
const PP_PAGE_SIZE = 50;
const OBJECT_PAGE_SIZE = 50;
interface RepositoriesPageProps {
browserState: RepositoryBrowserState;
onBrowserStateChange: Dispatch<SetStateAction<RepositoryBrowserState>>;
onOpenObject?: (objectInstanceId: string) => void;
}
export function RepositoriesPage({ browserState, onBrowserStateChange, onOpenObject }: RepositoriesPageProps) {
const {
selectedRepoId,
selectedPpId,
repoFilter,
ppFilter,
objectFilter,
repoPage,
ppPage,
objectPage,
isRepoPanelCollapsed,
isPpPanelCollapsed
} = browserState;
const setRepoFilter = (value: string) => onBrowserStateChange((state) => ({ ...state, repoFilter: value }));
const setPpFilter = (value: string) => onBrowserStateChange((state) => ({ ...state, ppFilter: value }));
const setObjectFilter = (value: string) => onBrowserStateChange((state) => ({ ...state, objectFilter: value }));
const setRepoPage = (updater: SetStateAction<PageState>) => {
onBrowserStateChange((state) => ({
...state,
repoPage: typeof updater === "function" ? updater(state.repoPage) : updater
}));
};
const setPpPage = (updater: SetStateAction<PageState>) => {
onBrowserStateChange((state) => ({
...state,
ppPage: typeof updater === "function" ? updater(state.ppPage) : updater
}));
};
const setObjectPage = (updater: SetStateAction<PageState>) => {
onBrowserStateChange((state) => ({
...state,
objectPage: typeof updater === "function" ? updater(state.objectPage) : updater
}));
};
const latestRunQuery = useQuery({
queryKey: ["repositories-latest-run"],
queryFn: getLatestRun
});
const runId = latestRunQuery.data?.data.runId ?? "latest";
const reposQuery = useQuery({
queryKey: ["repositories", runId, repoPage.cursor],
queryFn: () => listRepos(runId, REPO_PAGE_SIZE, repoPage.cursor),
enabled: Boolean(latestRunQuery.data?.data.runId)
});
const repos = useMemo(() => reposQuery.data?.data ?? [], [reposQuery.data?.data]);
const filteredRepos = useMemo(() => filterRepositories(repos, repoFilter), [repoFilter, repos]);
const selectedRepo = selectedRepoId ? repos.find((repo) => repo.repoId === selectedRepoId) ?? null : null;
const ppsQuery = useQuery({
queryKey: ["repository-publication-points", runId, selectedRepo?.repoId ?? "", ppPage.cursor],
queryFn: () => listPublicationPointsForRepo(runId, selectedRepo?.repoId ?? "", PP_PAGE_SIZE, ppPage.cursor),
enabled: Boolean(selectedRepo?.repoId)
});
const publicationPoints = useMemo(() => ppsQuery.data?.data ?? [], [ppsQuery.data?.data]);
const filteredPublicationPoints = useMemo(() => filterPublicationPoints(publicationPoints, ppFilter), [ppFilter, publicationPoints]);
const selectedPp = selectedPpId
? publicationPoints.find((pp) => pp.ppId === selectedPpId) ?? null
: null;
const objectsQuery = useQuery({
queryKey: ["publication-point-objects", runId, selectedPp?.ppId ?? "", objectPage.cursor],
queryFn: () => listObjectsForPublicationPoint(runId, selectedPp?.ppId ?? "", OBJECT_PAGE_SIZE, objectPage.cursor),
enabled: Boolean(selectedPp?.ppId)
});
const objects = useMemo(() => objectsQuery.data?.data ?? [], [objectsQuery.data?.data]);
const filteredObjects = useMemo(() => filterObjects(objects, objectFilter), [objectFilter, objects]);
const repoTotal = reposQuery.data?.page?.nextCursor ? undefined : repoPage.previous.length * REPO_PAGE_SIZE + repos.length;
const ppTotal = selectedRepo?.publicationPoints;
const objectTotal = selectedPp?.objects;
const error = latestRunQuery.error ?? reposQuery.error ?? ppsQuery.error ?? objectsQuery.error;
return (
<section className="page-stack repositories-page" aria-labelledby="repositories-heading">
<div className="hero-dashboard compact-hero">
<div>
<p className="eyebrow">Repository browser · live query service</p>
<h2 id="repositories-heading">Repository / publication point / object browser</h2>
<p>Browse repo trees first, then expand publication points and object rows only on demand.</p>
</div>
<div className="hero-status">
<Database size={20} />
<div>
<strong>{latestRunQuery.data?.data.runId ?? "Loading run"}</strong>
<span>{repos.length ? `${repos.length} repositories loaded` : "waiting for query service"}</span>
</div>
</div>
</div>
{error ? (
<div className="alert-banner" role="status">
<strong>Repository API unavailable</strong>
<span>{normalizeApiError(error).message}</span>
</div>
) : null}
<div className={`repo-browser-grid ${isRepoPanelCollapsed ? "repo-panel-collapsed" : ""} ${isPpPanelCollapsed ? "pp-panel-collapsed" : ""}`}>
<section className="panel list-panel collapsible-list-panel repo-list-panel" aria-label="Repositories">
<PanelTitle
icon={<Database size={18} />}
isCollapsed={isRepoPanelCollapsed}
label="Repositories"
meta={reposQuery.isFetching ? "loading" : rangeLabel(repoPage.previous.length + 1, REPO_PAGE_SIZE, repos.length, repoTotal, repoFilter, filteredRepos.length)}
onToggleCollapse={() => onBrowserStateChange((state) => ({ ...state, isRepoPanelCollapsed: !state.isRepoPanelCollapsed }))}
toggleLabel={isRepoPanelCollapsed ? "Expand repositories column" : "Collapse repositories column"}
/>
{isRepoPanelCollapsed ? (
<CollapsedPanelSummary icon={<Database size={20} />} label="Repositories" value={repoTotal ?? repos.length} />
) : (
<>
<CurrentPageFilter
label="Filter repositories on current page"
onChange={setRepoFilter}
placeholder="Filter host or URI..."
value={repoFilter}
/>
<div className="stack-list">
{filteredRepos.map((repo) => (
<article className={`stack-row ${repo.repoId === selectedRepo?.repoId ? "active" : ""}`} key={repo.repoId}>
<button
className="stack-row-select"
onClick={() => {
onBrowserStateChange((state) => ({
...state,
selectedRepoId: repo.repoId,
selectedPpId: null,
ppFilter: "",
objectFilter: "",
ppPage: createEmptyPageState(),
objectPage: createEmptyPageState()
}));
}}
type="button"
>
<span>
<strong>{repo.host}</strong>
<small>{repo.transport.toUpperCase()} · {repo.publicationPoints.toLocaleString()} publication points</small>
</span>
<em>{repo.objects.toLocaleString()}</em>
<ChevronRight size={16} />
</button>
<CopyableValue className="stack-row-copy" label="repository URI" value={repo.uri} />
</article>
))}
</div>
<CursorPager
isFetching={reposQuery.isFetching}
pageNumber={repoPage.previous.length + 1}
nextCursor={reposQuery.data?.page?.nextCursor ?? null}
onNext={() => setRepoPage((page) => advancePage(page, reposQuery.data?.page?.nextCursor ?? null))}
onPrevious={() => setRepoPage(previousPage)}
previousCount={repoPage.previous.length}
rangeLabel={rangeLabel(repoPage.previous.length + 1, REPO_PAGE_SIZE, repos.length, repoTotal, repoFilter, filteredRepos.length)}
/>
</>
)}
</section>
<section className="panel list-panel collapsible-list-panel pp-list-panel" aria-label="Publication points">
<PanelTitle
icon={<GitBranch size={18} />}
isCollapsed={isPpPanelCollapsed}
label="Publication Points"
meta={ppsQuery.isFetching ? "loading" : selectedRepo ? rangeLabel(ppPage.previous.length + 1, PP_PAGE_SIZE, publicationPoints.length, ppTotal, ppFilter, filteredPublicationPoints.length) : "select repo"}
onToggleCollapse={() => onBrowserStateChange((state) => ({ ...state, isPpPanelCollapsed: !state.isPpPanelCollapsed }))}
toggleLabel={isPpPanelCollapsed ? "Expand publication points column" : "Collapse publication points column"}
/>
{isPpPanelCollapsed ? (
<CollapsedPanelSummary icon={<GitBranch size={20} />} label="Publication Points" value={ppTotal ?? publicationPoints.length} />
) : (
<>
{!selectedRepo ? (
<div className="empty-state">Select a repository to load its publication points.</div>
) : null}
<CurrentPageFilter
disabled={!selectedRepo}
label="Filter publication points on current page"
onChange={setPpFilter}
placeholder="Filter manifest, rsync, source..."
value={ppFilter}
/>
{ppsQuery.isFetching ? <LoadingLine /> : null}
<div className="stack-list">
{filteredPublicationPoints.map((pp) => (
<article className={`stack-row ${pp.ppId === selectedPp?.ppId ? "active" : ""}`} key={pp.ppId}>
<button
className="stack-row-select"
onClick={() => {
onBrowserStateChange((state) => ({
...state,
selectedPpId: pp.ppId,
objectFilter: "",
objectPage: createEmptyPageState()
}));
}}
type="button"
>
<span>
<strong>{shortName(pp.manifestRsyncUri ?? pp.publicationPointRsyncUri ?? pp.rrdpNotificationUri ?? pp.ppId)}</strong>
<small>{pp.repoSyncSource ?? pp.source ?? "unknown"} · {pp.ppId}</small>
</span>
<em>{pp.objects.toLocaleString()}</em>
<StatusPill state={pp.repoTerminalState ?? pp.source ?? "unknown"} />
</button>
<CopyableValue className="stack-row-copy" label="publication point URI" value={pp.publicationPointRsyncUri ?? pp.rsyncBaseUri ?? pp.rrdpNotificationUri ?? pp.ppId} />
{pp.manifestRsyncUri ? <CopyableValue className="stack-row-copy secondary" label="manifest URI" value={pp.manifestRsyncUri} /> : null}
</article>
))}
</div>
<CursorPager
isFetching={ppsQuery.isFetching}
nextCursor={ppsQuery.data?.page?.nextCursor ?? null}
onNext={() => setPpPage((page) => advancePage(page, ppsQuery.data?.page?.nextCursor ?? null))}
onPrevious={() => setPpPage(previousPage)}
pageNumber={ppPage.previous.length + 1}
previousCount={ppPage.previous.length}
rangeLabel={selectedRepo ? rangeLabel(ppPage.previous.length + 1, PP_PAGE_SIZE, publicationPoints.length, ppTotal, ppFilter, filteredPublicationPoints.length) : undefined}
/>
</>
)}
</section>
<section className="panel table-panel objects-live-panel" aria-label="Objects for publication point">
<PanelTitle
icon={<FileCode2 size={18} />}
label="Objects"
meta={objectsQuery.isFetching ? "loading" : selectedPp ? rangeLabel(objectPage.previous.length + 1, OBJECT_PAGE_SIZE, objects.length, objectTotal, objectFilter, filteredObjects.length) : "select PP"}
/>
{!selectedPp ? (
<div className="empty-state">Select a publication point to load its objects.</div>
) : null}
<CurrentPageFilter
disabled={!selectedPp}
label="Filter objects on current page"
onChange={setObjectFilter}
placeholder="Filter type, URI, hash, status..."
value={objectFilter}
/>
{objectsQuery.isFetching ? <LoadingLine /> : null}
{selectedPp && !objectsQuery.isFetching ? (
<ObjectTable objects={filteredObjects} onOpenObject={onOpenObject} />
) : null}
<CursorPager
isFetching={objectsQuery.isFetching}
nextCursor={objectsQuery.data?.page?.nextCursor ?? null}
onNext={() => setObjectPage((page) => advancePage(page, objectsQuery.data?.page?.nextCursor ?? null))}
onPrevious={() => setObjectPage(previousPage)}
pageNumber={objectPage.previous.length + 1}
previousCount={objectPage.previous.length}
rangeLabel={selectedPp ? rangeLabel(objectPage.previous.length + 1, OBJECT_PAGE_SIZE, objects.length, objectTotal, objectFilter, filteredObjects.length) : undefined}
/>
</section>
</div>
</section>
);
}
function PanelTitle({
icon,
isCollapsed,
label,
meta,
onToggleCollapse,
toggleLabel
}: {
icon: ReactNode;
isCollapsed?: boolean;
label: string;
meta: string;
onToggleCollapse?: () => void;
toggleLabel?: string;
}) {
return (
<div className="panel-heading compact-heading">
<div>
<p className="eyebrow">{label}</p>
<h3>{label}</h3>
</div>
<div className="panel-heading-actions">
<span className="panel-meta">
{icon}
{meta}
</span>
{onToggleCollapse ? (
<button aria-expanded={!isCollapsed} aria-label={toggleLabel} className="panel-collapse-button" onClick={onToggleCollapse} title={toggleLabel} type="button">
{isCollapsed ? <PanelRightClose aria-hidden="true" size={16} /> : <PanelLeftClose aria-hidden="true" size={16} />}
</button>
) : null}
</div>
</div>
);
}
function CollapsedPanelSummary({ icon, label, value }: { icon: ReactNode; label: string; value: number }) {
return (
<div className="collapsed-panel-summary">
{icon}
<strong>{label}</strong>
<span>{value.toLocaleString()}</span>
</div>
);
}
function CurrentPageFilter({
disabled,
label,
onChange,
placeholder,
value
}: {
disabled?: boolean;
label: string;
onChange: (value: string) => void;
placeholder: string;
value: string;
}) {
return (
<label className="current-page-filter">
<span>{label}</span>
<input
disabled={disabled}
onChange={(event) => onChange(event.target.value)}
placeholder={placeholder}
value={value}
/>
</label>
);
}
function ObjectTable({ objects, onOpenObject }: { objects: ObjectInstanceRecord[]; onOpenObject?: (objectInstanceId: string) => void }) {
if (objects.length === 0) {
return <div className="empty-state">No objects returned for this publication point.</div>;
}
return (
<table>
<thead>
<tr>
<th>Type</th>
<th>URI</th>
<th>Hash</th>
<th>Source</th>
<th>Result</th>
<th>Action</th>
</tr>
</thead>
<tbody>
{objects.map((object) => (
<tr key={object.objectInstanceId}>
<td><span className="object-type">{object.objectType}</span></td>
<td className="uri-cell"><CopyableValue label="object URI" value={object.uri} /></td>
<td><CopyableValue label="object SHA256" value={object.sha256} displayValue={shortHash(object.sha256, 12)} /></td>
<td>{object.sourceSection}</td>
<td><StatusPill state={object.rejected ? "rejected" : object.result} /></td>
<td>
<button className="table-link-button" onClick={() => onOpenObject?.(object.objectInstanceId)} type="button">
Open
</button>
</td>
</tr>
))}
</tbody>
</table>
);
}
function StatusPill({ state }: { state: string }) {
const normalized = state.toLowerCase();
const className = normalized.includes("fail") || normalized.includes("reject") || normalized.includes("error")
? "warning"
: normalized.includes("cache") || normalized.includes("fallback")
? "fallback"
: "healthy";
return <span className={`status-badge ${className}`}>{state}</span>;
}
function LoadingLine() {
return (
<div className="loading-line">
<Loader2 size={16} />
Loading selected branch...
</div>
);
}
function shortName(uri: string) {
return uri.split("/").filter(Boolean).at(-1) ?? uri;
}
function shortHash(value: string, prefixLength: number) {
return `${value.slice(0, prefixLength)}...`;
}
function rangeLabel(pageNumber: number, pageSize: number, pageRowCount: number, total: number | undefined, filter: string, visibleRowCount = pageRowCount) {
const start = pageRowCount === 0 ? 0 : (pageNumber - 1) * pageSize + 1;
const end = pageRowCount === 0 ? 0 : start + pageRowCount - 1;
const totalText = total === undefined ? "unknown" : total.toLocaleString();
const filterText = filter.trim() ? ` · ${visibleRowCount.toLocaleString()}/${pageRowCount.toLocaleString()} matched on page` : "";
return `${start.toLocaleString()}-${end.toLocaleString()}/${totalText} shown${filterText}`;
}
function matchesNeedle(values: Array<string | number | null | undefined>, needle: string) {
const normalized = needle.trim().toLowerCase();
if (!normalized) {
return true;
}
return values.some((value) => String(value ?? "").toLowerCase().includes(normalized));
}
function filterRepositories(repos: RepositoryRecord[], filter: string) {
return repos.filter((repo) => matchesNeedle([repo.host, repo.uri, repo.transport, repo.objects, repo.rejectedObjects], filter));
}
function filterPublicationPoints(publicationPoints: PublicationPointRecord[], filter: string) {
return publicationPoints.filter((pp) => matchesNeedle([
pp.ppId,
pp.manifestRsyncUri,
pp.publicationPointRsyncUri,
pp.rsyncBaseUri,
pp.rrdpNotificationUri,
pp.source,
pp.repoSyncSource,
pp.repoTerminalState,
pp.objects,
pp.rejectedObjects
], filter));
}
function filterObjects(objects: ObjectInstanceRecord[], filter: string) {
return objects.filter((object) => matchesNeedle([
object.objectType,
object.uri,
object.sha256,
object.sourceSection,
object.result,
object.rejectReason,
object.rejected ? "rejected" : "accepted"
], filter));
}

View File

@@ -0,0 +1,33 @@
import type { PageState } from "../../components/common/cursorPaging";
export interface RepositoryBrowserState {
selectedRepoId: string | null;
selectedPpId: string | null;
repoFilter: string;
ppFilter: string;
objectFilter: string;
repoPage: PageState;
ppPage: PageState;
objectPage: PageState;
isRepoPanelCollapsed: boolean;
isPpPanelCollapsed: boolean;
}
export function createRepositoryBrowserState(): RepositoryBrowserState {
return {
selectedRepoId: null,
selectedPpId: null,
repoFilter: "",
ppFilter: "",
objectFilter: "",
repoPage: createEmptyPageState(),
ppPage: createEmptyPageState(),
objectPage: createEmptyPageState(),
isRepoPanelCollapsed: false,
isPpPanelCollapsed: false
};
}
export function createEmptyPageState(): PageState {
return { cursor: null, previous: [] };
}

View File

@@ -1,42 +0,0 @@
import { describe, expect, it } from "vitest";
import {
INITIAL_PAGER_STATE,
advancePager,
pageNumber,
retreatPager,
type PagerState,
} from "./cursor";
describe("cursor pager state machine", () => {
it("starts on page 1 with a null cursor", () => {
expect(INITIAL_PAGER_STATE.cursor).toBeNull();
expect(pageNumber(INITIAL_PAGER_STATE)).toBe(1);
});
it("advances and keeps a cursor stack for going back", () => {
let state: PagerState = INITIAL_PAGER_STATE;
state = advancePager(state, "c2");
expect(state.cursor).toBe("c2");
expect(pageNumber(state)).toBe(2);
state = advancePager(state, "c3");
expect(state.cursor).toBe("c3");
expect(pageNumber(state)).toBe(3);
state = retreatPager(state);
expect(state.cursor).toBe("c2");
expect(pageNumber(state)).toBe(2);
state = retreatPager(state);
expect(state.cursor).toBeNull();
expect(pageNumber(state)).toBe(1);
});
it("ignores advance with a null cursor (no next page)", () => {
const state = advancePager(INITIAL_PAGER_STATE, null);
expect(state).toEqual(INITIAL_PAGER_STATE);
});
it("ignores retreat on the first page", () => {
const state = retreatPager(INITIAL_PAGER_STATE);
expect(state).toEqual(INITIAL_PAGER_STATE);
});
});

View File

@@ -1,72 +0,0 @@
/** Cursor pagination state (previous-page stack) shared by all paged tables. */
import { useCallback, useEffect, useMemo, useState } from "react";
export interface CursorPager {
/** Cursor to pass to the backend for the current page (null = first page). */
cursor: string | null;
/** 1-based page number. */
page: number;
canPrev: boolean;
goNext: (nextCursor: string | null) => void;
goPrev: () => void;
reset: () => void;
}
export interface PagerState {
cursor: string | null;
/** Stack of cursors that produced the previous pages (page 1 = null). */
stack: (string | null)[];
}
export const INITIAL_PAGER_STATE: PagerState = { cursor: null, stack: [] };
/** Advance to the page identified by `nextCursor`; unchanged when null. */
export function advancePager(state: PagerState, nextCursor: string | null): PagerState {
if (!nextCursor) return state;
return { cursor: nextCursor, stack: [...state.stack, state.cursor] };
}
/** Go back one page; unchanged on the first page. */
export function retreatPager(state: PagerState): PagerState {
if (state.stack.length === 0) return state;
const stack = [...state.stack];
const cursor = stack.pop() ?? null;
return { cursor, stack };
}
export function pageNumber(state: PagerState): number {
return state.stack.length + 1;
}
/**
* Cursor pager that resets whenever `resetKey` changes (run id, filters…).
*/
export function useCursorPager(resetKey: string): CursorPager {
const [state, setState] = useState<PagerState>(INITIAL_PAGER_STATE);
useEffect(() => {
setState(INITIAL_PAGER_STATE);
}, [resetKey]);
const goNext = useCallback((nextCursor: string | null) => {
setState((s) => advancePager(s, nextCursor));
}, []);
const goPrev = useCallback(() => {
setState((s) => retreatPager(s));
}, []);
const reset = useCallback(() => setState(INITIAL_PAGER_STATE), []);
return useMemo(
() => ({
cursor: state.cursor,
page: pageNumber(state),
canPrev: state.stack.length > 0,
goNext,
goPrev,
reset,
}),
[state, goNext, goPrev, reset],
);
}

View File

@@ -1,99 +0,0 @@
import { describe, expect, it } from "vitest";
import {
formatBytes,
formatDurationMs,
formatInt,
formatPercent,
formatUtc,
objectTypeLabel,
parseTime,
truncateMiddle,
} from "./format";
describe("formatInt", () => {
it("formats with thousands separators", () => {
expect(formatInt(1234567)).toBe("1,234,567");
});
it("renders dash for nullish", () => {
expect(formatInt(null)).toBe("—");
expect(formatInt(undefined)).toBe("—");
});
});
describe("formatBytes", () => {
it("keeps small values in bytes", () => {
expect(formatBytes(512)).toBe("512 B");
});
it("scales to KiB/MiB/GiB", () => {
expect(formatBytes(2048)).toBe("2.0 KiB");
expect(formatBytes(5 * 1024 * 1024)).toBe("5.0 MiB");
expect(formatBytes(3 * 1024 ** 3)).toBe("3.0 GiB");
});
it("renders dash for nullish", () => {
expect(formatBytes(null)).toBe("—");
});
});
describe("formatDurationMs", () => {
it("sub-second values", () => {
expect(formatDurationMs(640)).toBe("640 ms");
});
it("seconds", () => {
expect(formatDurationMs(4200)).toBe("4.2s");
expect(formatDurationMs(42_000)).toBe("42s");
});
it("minutes and hours", () => {
expect(formatDurationMs(3 * 60_000 + 4_000)).toBe("3m 04s");
expect(formatDurationMs(2 * 3_600_000 + 5 * 60_000)).toBe("2h 05m");
});
});
describe("parseTime / formatUtc", () => {
it("parses RFC3339 and renders UTC", () => {
expect(formatUtc("2026-07-17T01:46:55Z")).toBe("2026-07-17 01:46:55 UTC");
});
it("passes through unparseable input", () => {
expect(formatUtc("not-a-date")).toBe("not-a-date");
expect(parseTime("not-a-date")).toBeNull();
});
it("renders dash for missing values", () => {
expect(formatUtc(null)).toBe("—");
expect(formatUtc(undefined)).toBe("—");
});
});
describe("formatPercent", () => {
it("computes one decimal", () => {
expect(formatPercent(1, 3)).toBe("33.3%");
});
it("guards zero total", () => {
expect(formatPercent(1, 0)).toBe("—");
});
});
describe("truncateMiddle", () => {
it("keeps short values intact", () => {
expect(truncateMiddle("abc", 10)).toBe("abc");
});
it("truncates with an ellipsis keeping head and tail", () => {
const out = truncateMiddle("a".repeat(100), 21);
expect(out.length).toBe(21);
expect(out).toContain("…");
expect(out.startsWith("a".repeat(10))).toBe(true);
expect(out.endsWith("a".repeat(10))).toBe(true);
});
});
describe("objectTypeLabel", () => {
it("maps known types", () => {
expect(objectTypeLabel("roa")).toBe("ROA");
expect(objectTypeLabel("mft")).toBe("Manifest");
expect(objectTypeLabel("asa")).toBe("ASPA");
});
it("uppercases unknown types", () => {
expect(objectTypeLabel("xyz")).toBe("XYZ");
});
it("renders dash for missing", () => {
expect(objectTypeLabel(null)).toBe("—");
});
});

View File

@@ -1,109 +0,0 @@
/** Formatting helpers for numbers, bytes, durations and timestamps. */
export function formatInt(value: number | null | undefined): string {
if (value === null || value === undefined || Number.isNaN(value)) return "—";
return value.toLocaleString("en-US");
}
export function formatBytes(value: number | null | undefined): string {
if (value === null || value === undefined || Number.isNaN(value)) return "—";
if (value < 1024) return `${value} B`;
const units = ["KiB", "MiB", "GiB", "TiB"];
let size = value;
let unit = "B";
for (const next of units) {
if (size < 1024) break;
size /= 1024;
unit = next;
}
return `${size >= 100 ? size.toFixed(0) : size.toFixed(1)} ${unit}`;
}
export function formatDurationMs(value: number | null | undefined): string {
if (value === null || value === undefined || Number.isNaN(value)) return "—";
if (value < 1000) return `${Math.round(value)} ms`;
const totalSeconds = Math.floor(value / 1000);
const hours = Math.floor(totalSeconds / 3600);
const minutes = Math.floor((totalSeconds % 3600) / 60);
const seconds = totalSeconds % 60;
if (hours > 0) return `${hours}h ${String(minutes).padStart(2, "0")}m`;
if (minutes > 0) return `${minutes}m ${String(seconds).padStart(2, "0")}s`;
const fraction = value / 1000;
return `${fraction >= 10 ? fraction.toFixed(0) : fraction.toFixed(1)}s`;
}
/** Parse an ISO/RFC3339 timestamp; returns null when unparseable. */
export function parseTime(value: string | null | undefined): Date | null {
if (!value) return null;
const date = new Date(value);
return Number.isNaN(date.getTime()) ? null : date;
}
/** "2026-07-17 01:46:55 UTC" — stable, timezone-explicit rendering. */
export function formatUtc(value: string | null | undefined): string {
const date = parseTime(value);
if (!date) return value ? String(value) : "—";
const pad = (n: number) => String(n).padStart(2, "0");
return (
`${date.getUTCFullYear()}-${pad(date.getUTCMonth() + 1)}-${pad(date.getUTCDate())} ` +
`${pad(date.getUTCHours())}:${pad(date.getUTCMinutes())}:${pad(date.getUTCSeconds())} UTC`
);
}
export function formatRelative(value: string | null | undefined): string {
const date = parseTime(value);
if (!date) return "—";
const deltaMs = Date.now() - date.getTime();
const future = deltaMs < 0;
const abs = Math.abs(deltaMs);
const minutes = Math.floor(abs / 60_000);
const hours = Math.floor(abs / 3_600_000);
const days = Math.floor(abs / 86_400_000);
let text: string;
if (abs < 60_000) text = "just now";
else if (minutes < 60) text = `${minutes}m`;
else if (hours < 24) text = `${hours}h`;
else text = `${days}d`;
if (text === "just now") return text;
return future ? `in ${text}` : `${text} ago`;
}
export function formatPercent(part: number, total: number): string {
if (!total) return "—";
return `${((part / total) * 100).toFixed(1)}%`;
}
/** Truncate a long identifier for display, keeping head and tail. */
export function truncateMiddle(value: string, max = 32): string {
if (value.length <= max) return value;
const head = Math.ceil((max - 1) / 2);
const tail = Math.floor((max - 1) / 2);
return `${value.slice(0, head)}…${value.slice(value.length - tail)}`;
}
/** Human label for an object type code (accepts both report-stream and projection vocabularies). */
export function objectTypeLabel(type: string | null | undefined): string {
switch ((type ?? "").toLowerCase()) {
case "roa":
return "ROA";
case "mft":
case "manifest":
return "Manifest";
case "crl":
return "CRL";
case "cer":
case "certificate":
return "Certificate";
case "asa":
case "aspa":
return "ASPA";
case "gbr":
return "Ghostbusters";
case "ee":
return "EE Certificate";
case "other":
return "Other";
default:
return type ? type.toUpperCase() : "—";
}
}

View File

@@ -1,25 +0,0 @@
import { describe, expect, it } from "vitest";
import { asArray, asNumber, asRecord, asString } from "./projection";
describe("projection guards", () => {
it("asRecord accepts plain objects only", () => {
expect(asRecord({ a: 1 })).toEqual({ a: 1 });
expect(asRecord([1])).toBeNull();
expect(asRecord("x")).toBeNull();
expect(asRecord(null)).toBeNull();
});
it("asString / asNumber narrow primitives", () => {
expect(asString("s")).toBe("s");
expect(asString(1)).toBeNull();
expect(asNumber(4)).toBe(4);
expect(asNumber("4")).toBeNull();
expect(asNumber(NaN)).toBeNull();
});
it("asArray defaults to empty", () => {
expect(asArray([1, 2])).toEqual([1, 2]);
expect(asArray(undefined)).toEqual([]);
expect(asArray({})).toEqual([]);
});
});

View File

@@ -1,18 +0,0 @@
/** Defensive accessors for loosely-typed projection payloads. */
export function asRecord(value: unknown): Record<string, unknown> | null {
return value !== null && typeof value === "object" && !Array.isArray(value)
? (value as Record<string, unknown>)
: null;
}
export function asString(value: unknown): string | null {
return typeof value === "string" ? value : null;
}
export function asNumber(value: unknown): number | null {
return typeof value === "number" && Number.isFinite(value) ? value : null;
}
export function asArray(value: unknown): unknown[] {
return Array.isArray(value) ? value : [];
}

View File

@@ -1,48 +0,0 @@
/**
* Run-context helpers. The active run is carried in the `?run=` search param;
* "latest" (the default, no param) resolves server-side to the newest
* indexed run.
*/
import { useCallback } from "react";
import { useSearchParams } from "react-router-dom";
export const LATEST_RUN = "latest";
/** The run id to use in API paths ("latest" when no explicit selection). */
export function useRunId(): string {
const [searchParams] = useSearchParams();
return searchParams.get("run") ?? LATEST_RUN;
}
/** Raw `?run=` value (null when following latest). */
export function useRunParam(): string | null {
const [searchParams] = useSearchParams();
return searchParams.get("run");
}
/**
* Returns a function that sets (or clears, for "latest") the `?run=` param
* while preserving the other current search params.
*/
export function useSetRunParam(): (runId: string | null) => void {
const [searchParams, setSearchParams] = useSearchParams();
return useCallback(
(runId: string | null) => {
const next = new URLSearchParams(searchParams);
if (runId === null || runId === LATEST_RUN) {
next.delete("run");
} else {
next.set("run", runId);
}
setSearchParams(next, { preventScrollReset: true });
},
[searchParams, setSearchParams],
);
}
/** Append the current run param to a path that already carries a query string. */
export function withRunParam(path: string, runId: string): string {
if (runId === LATEST_RUN) return path;
const sep = path.includes("?") ? "&" : "?";
return `${path}${sep}run=${encodeURIComponent(runId)}`;
}

View File

@@ -1,15 +0,0 @@
/** Resolve the active run record for the current `?run=` context. */
import { useQuery, type UseQueryResult } from "@tanstack/react-query";
import { getRun } from "../api/service";
import type { RunRecord } from "../api/schemas";
import { useRunId } from "./run";
export function useRun(): { runId: string; runQuery: UseQueryResult<RunRecord> } {
const runId = useRunId();
const runQuery = useQuery({
queryKey: ["run", runId],
queryFn: () => getRun(runId),
staleTime: 60_000,
});
return { runId, runQuery };
}

View File

@@ -1,97 +0,0 @@
import { describe, expect, it } from "vitest";
import {
classifyNetworkQuery,
parseAsn,
parseIpAddress,
parsePrefix,
} from "./vrpQuery";
describe("parseAsn", () => {
it("accepts bare and AS-prefixed numbers", () => {
expect(parseAsn("45090")).toBe(45090);
expect(parseAsn("AS45090")).toBe(45090);
expect(parseAsn("as0")).toBe(0);
expect(parseAsn(" AS64496 ")).toBe(64496);
});
it("rejects out-of-range and malformed ASNs", () => {
expect(parseAsn("4294967295")).toBe(4294967295);
expect(parseAsn("4294967296")).toBeNull();
expect(parseAsn("AS123abc")).toBeNull();
expect(parseAsn("AS")).toBeNull();
expect(parseAsn("-1")).toBeNull();
});
});
describe("parseIpAddress", () => {
it("accepts IPv4 addresses", () => {
expect(parseIpAddress("81.68.1.1")).toBe("81.68.1.1");
expect(parseIpAddress("0.0.0.0")).toBe("0.0.0.0");
expect(parseIpAddress("255.255.255.255")).toBe("255.255.255.255");
});
it("rejects IPv4 with bad octets or CIDR suffix", () => {
expect(parseIpAddress("256.0.0.1")).toBeNull();
expect(parseIpAddress("192.0.2.1/24")).toBeNull();
expect(parseIpAddress("192.0.2")).toBeNull();
});
it("accepts IPv6 addresses", () => {
expect(parseIpAddress("2001:7fa:0:1::1")).toBe("2001:7fa:0:1::1");
expect(parseIpAddress("::")).toBe("::");
expect(parseIpAddress("::1")).toBe("::1");
expect(parseIpAddress("2001:db8:0:0:0:0:0:1")).toBe("2001:db8:0:0:0:0:0:1");
expect(parseIpAddress("1:2:3:4:5:6:7::")).toBe("1:2:3:4:5:6:7::"); // "::" = one :0 group
expect(parseIpAddress("::ffff:192.0.2.1")).toBe("::ffff:192.0.2.1");
});
it("rejects malformed IPv6", () => {
expect(parseIpAddress("2001:::1")).toBeNull();
expect(parseIpAddress("2001:db8::g1")).toBeNull();
expect(parseIpAddress("1:2:3:4:5:6:7:8:9")).toBeNull();
expect(parseIpAddress("1:2:3:4:5:6:7:8::")).toBeNull(); // no room for "::"
expect(parseIpAddress("12345::")).toBeNull();
});
});
describe("parsePrefix", () => {
it("accepts IPv4 and IPv6 CIDR", () => {
expect(parsePrefix("192.0.2.0/24")).toBe("192.0.2.0/24");
expect(parsePrefix("192.0.2.1/24")).toBe("192.0.2.1/24"); // host bits: backend normalizes
expect(parsePrefix("0.0.0.0/0")).toBe("0.0.0.0/0");
expect(parsePrefix("2001:db8::/32")).toBe("2001:db8::/32");
expect(parsePrefix("2001:db8::1/128")).toBe("2001:db8::1/128");
});
it("rejects bad lengths and addresses", () => {
expect(parsePrefix("192.0.2.0/33")).toBeNull();
expect(parsePrefix("2001:db8::/129")).toBeNull();
expect(parsePrefix("192.0.2.0/")).toBeNull();
expect(parsePrefix("192.0.2.0/24/1")).toBeNull();
expect(parsePrefix("999.0.0.0/8")).toBeNull();
expect(parsePrefix("/24")).toBeNull();
});
});
describe("classifyNetworkQuery", () => {
it("prefix wins over ip, ip over asn", () => {
expect(classifyNetworkQuery("81.68.0.0/14")).toEqual({
kind: "prefix",
prefix: "81.68.0.0/14",
});
expect(classifyNetworkQuery("81.68.1.1")).toEqual({ kind: "ip", ip: "81.68.1.1" });
expect(classifyNetworkQuery("2001:7fa:0:1::1")).toEqual({
kind: "ip",
ip: "2001:7fa:0:1::1",
});
expect(classifyNetworkQuery("AS45090")).toEqual({ kind: "asn", asn: 45090 });
expect(classifyNetworkQuery("45090")).toEqual({ kind: "asn", asn: 45090 });
});
it("falls back to none for text queries", () => {
expect(classifyNetworkQuery("repo-a.example")).toEqual({ kind: "none" });
expect(classifyNetworkQuery("rsync://repo.example/ca/roa.roa")).toEqual({ kind: "none" });
expect(classifyNetworkQuery("d1488eb0e0faabba1fbe8236")).toEqual({ kind: "none" });
expect(classifyNetworkQuery("")).toEqual({ kind: "none" });
});
});

View File

@@ -1,104 +0,0 @@
/**
* Classify a free-text search query into a VRP lookup target.
*
* The backend VRP lookup (`GET /runs/{id}/vrps/lookup`) accepts exactly one of
* `ip` / `prefix`, plus an optional `asn` filter. An ASN-only query cannot be
* served by the API — the UI surfaces that explicitly instead of guessing.
*/
export type NetworkQuery =
| { kind: "ip"; ip: string }
| { kind: "prefix"; prefix: string }
| { kind: "asn"; asn: number }
| { kind: "none" };
const ASN_MAX = 4_294_967_295;
/** "AS45090" / "as45090" / "45090" → ASN, else null. */
export function parseAsn(input: string): number | null {
const match = /^(?:as)?(\d{1,10})$/i.exec(input.trim());
if (!match) return null;
const asn = Number(match[1]);
return Number.isSafeInteger(asn) && asn <= ASN_MAX ? asn : null;
}
function isIpv4Octets(addr: string): boolean {
const parts = addr.split(".");
return (
parts.length === 4 &&
parts.every((p) => /^\d{1,3}$/.test(p) && Number(p) <= 255)
);
}
function isIpv6(addr: string): boolean {
if (!/^[0-9a-fA-F:.]+$/.test(addr)) return false;
if ((addr.match(/::/g) ?? []).length > 1) return false;
const isGroup = (g: string, isLast: boolean) =>
isLast && g.includes(".")
? isIpv4Octets(g) // embedded IPv4 tail, e.g. "::ffff:192.0.2.1"
: /^[0-9a-fA-F]{1,4}$/.test(g);
const ipv4TailWeight = (groups: string[]) =>
groups.length > 0 && groups[groups.length - 1].includes(".") ? 1 : 0;
if (addr.includes("::")) {
const [head, tail] = addr.split("::");
const headGroups = head ? head.split(":") : [];
const tailGroups = tail ? tail.split(":") : [];
// "::" compresses at least one group; an IPv4 tail counts as two groups.
const effective =
headGroups.length + tailGroups.length + ipv4TailWeight(tailGroups);
if (effective > 7) return false;
return [...headGroups, ...tailGroups].every((g, i, all) =>
isGroup(g, i === all.length - 1),
);
}
const groups = addr.split(":");
const expected = groups[groups.length - 1]?.includes(".") ? 7 : 8;
if (groups.length !== expected) return false;
return groups.every((g, i) => isGroup(g, i === groups.length - 1));
}
/** Bare IPv4/IPv6 address (no "/") → normalized string, else null. */
export function parseIpAddress(input: string): string | null {
const s = input.trim();
if (s.includes("/")) return null;
if (isIpv4Octets(s)) return s;
if (s.includes(":") && isIpv6(s)) return s;
return null;
}
function parsePrefixParts(s: string): [string, number] | null {
const slash = s.indexOf("/");
if (slash < 0 || slash !== s.lastIndexOf("/")) return null;
const addr = s.slice(0, slash);
const lenRaw = s.slice(slash + 1);
if (!/^\d{1,3}$/.test(lenRaw)) return null;
return [addr, Number(lenRaw)];
}
/** "addr/len" CIDR with family-correct length → normalized string, else null. */
export function parsePrefix(input: string): string | null {
const s = input.trim();
const parts = parsePrefixParts(s);
if (!parts) return null;
const [addr, len] = parts;
if (isIpv4Octets(addr)) return len <= 32 ? `${addr}/${len}` : null;
if (addr.includes(":") && isIpv6(addr)) return len <= 128 ? `${addr}/${len}` : null;
return null;
}
/**
* Classify a query string. Precedence: CIDR prefix → bare IP → ASN → none.
* ("192.0.2.1" also parses as digits+ dots only; ASN requires pure digits, so
* no overlap. A bare number is always treated as an ASN.)
*/
export function classifyNetworkQuery(input: string): NetworkQuery {
const s = input.trim();
if (!s) return { kind: "none" };
const prefix = parsePrefix(s);
if (prefix) return { kind: "prefix", prefix };
const ip = parseIpAddress(s);
if (ip) return { kind: "ip", ip };
const asn = parseAsn(s);
if (asn !== null) return { kind: "asn", asn };
return { kind: "none" };
}

View File

@@ -1,17 +1,13 @@
import { StrictMode } from "react"; import React from "react";
import { createRoot } from "react-dom/client"; import ReactDOM from "react-dom/client";
import { AppProviders } from "./app/providers"; import { AppProviders } from "./app/providers";
import { App } from "./app/App"; import { App } from "./app/App";
import "./styles/tokens.css"; import "./styles/globals.css";
import "./styles/base.css";
import "./styles/shell.css";
import "./styles/components.css";
import "./styles/pages.css";
createRoot(document.getElementById("root")!).render( ReactDOM.createRoot(document.getElementById("root") as HTMLElement).render(
<StrictMode> <React.StrictMode>
<AppProviders> <AppProviders>
<App /> <App />
</AppProviders> </AppProviders>
</StrictMode>, </React.StrictMode>
); );

View File

@@ -1,112 +0,0 @@
import { useQuery } from "@tanstack/react-query";
import { getHealth, getServiceInfo } from "../api/service";
import { CopyableValue } from "../components/CopyableValue";
import { PageHeader } from "../components/PageHeader";
import { Panel } from "../components/Panel";
import { StatusPill } from "../components/StatusPill";
import { ErrorBlock } from "../components/StateBlock";
import { useRun } from "../lib/useRun";
const ENDPOINTS: { method: "GET" | "POST"; path: string; note: string }[] = [
{ method: "GET", path: "/api/v1/healthz", note: "Service health + latest indexed run" },
{ method: "GET", path: "/api/v1/runs", note: "Run history (cursor paged)" },
{ method: "GET", path: "/api/v1/runs/{run}", note: "Run record ('latest' accepted)" },
{ method: "GET", path: "/api/v1/runs/{run}/repos", note: "Repositories (cursor paged)" },
{ method: "GET", path: "/api/v1/runs/{run}/repos/{id}/publication-points", note: "PPs of a repository" },
{ method: "GET", path: "/api/v1/runs/{run}/publication-points/{id}/objects", note: "Objects of a PP (filters: type,result,rejected,reason,q)" },
{ method: "GET", path: "/api/v1/runs/{run}/objects", note: "All objects (streamed; same filters)" },
{ method: "GET", path: "/api/v1/runs/{run}/objects/by-uri?uri=", note: "Exact URI lookup" },
{ method: "GET", path: "/api/v1/runs/{run}/objects/{id}/parsed", note: "Parsed projection (needs repo-bytes-db)" },
{ method: "GET", path: "/api/v1/runs/{run}/objects/{id}/validation", note: "Validation summary" },
{ method: "GET", path: "/api/v1/runs/{run}/objects/{id}/chain", note: "Chain edges" },
{ method: "GET", path: "/api/v1/runs/{run}/objects/{id}/raw", note: "Raw DER bytes (needs repo-bytes-db)" },
{ method: "GET", path: "/api/v1/runs/{run}/issues", note: "Rejected/errored objects (filters: reason,type,repoId,ppId)" },
{ method: "GET", path: "/api/v1/runs/{run}/search?q=", note: "Unified search: URI / sha256 / repo host / PP URI" },
{ method: "GET", path: "/api/v1/runs/{run}/vrps/lookup?ip=|prefix=[&asn=]", note: "VRP lookup (covering semantics, cursor paged)" },
{ method: "GET", path: "/api/v1/runs/{run}/stats/{validation,object-types,reasons}", note: "Aggregated counters" },
{ method: "POST", path: "/api/v1/runs/{run}/exports", note: "Start export job (repo / publication_point / object_set)" },
{ method: "GET", path: "/api/v1/runs/{run}/exports", note: "Export job list (process memory)" },
{ method: "GET", path: "/api/v1/runs/{run}/exports/{job}/download", note: "Download export tarball" },
{ method: "POST", path: "/api/v1/runs/{run}/objects/{id}/validation/explain", note: "On-demand validation explain (audit projection)" },
];
export default function ApiStatusPage() {
const { runId, runQuery } = useRun();
const infoQuery = useQuery({ queryKey: ["service-info"], queryFn: getServiceInfo });
const healthQuery = useQuery({ queryKey: ["health"], queryFn: getHealth });
return (
<div className="page">
<PageHeader title="API" subtitle="Query service connection status and endpoint reference" />
<div className="detail-grid">
<Panel title="Service">
{infoQuery.isError ? (
<ErrorBlock error={infoQuery.error} onRetry={() => infoQuery.refetch()} title="Query service unreachable" />
) : (
<dl className="meta-grid">
<dt>Service</dt>
<dd className="mono">{infoQuery.data?.service ?? "…"}</dd>
<dt>API version</dt>
<dd className="mono">{infoQuery.data?.version ?? "…"}</dd>
<dt>Health</dt>
<dd>
{healthQuery.isError ? (
<StatusPill status="error" label="unhealthy" />
) : (
<StatusPill status={healthQuery.data?.status ?? "unknown"} />
)}
</dd>
<dt>Latest indexed run</dt>
<dd className="mono">{healthQuery.data?.latestReadyRun ?? "—"}</dd>
</dl>
)}
</Panel>
<Panel title="Explorer context">
<dl className="meta-grid">
<dt>Active run</dt>
<dd className="mono">{runQuery.data?.runId ?? runId}</dd>
<dt>Index status</dt>
<dd><StatusPill status={runQuery.data?.indexStatus} /></dd>
<dt>API base</dt>
<dd><CopyableValue value={`${window.location.origin}/api/v1`} max={64} label="API base URL" /></dd>
</dl>
</Panel>
</div>
<Panel title="Deployment notes">
<ul style={{ margin: 0, paddingLeft: 18, display: "flex", flexDirection: "column", gap: 6 }}>
<li>The query service has no CORS headers and no authentication — always front it with a same-origin proxy (this SPA) or an API gateway.</li>
<li><code>/parsed</code>, <code>/raw</code> and exports require <code>--repo-bytes-db</code>.</li>
<li>All list endpoints paginate with <code>limit</code> + <code>cursor</code>; the envelope is <code>{"{data, page, meta}"}</code>.</li>
<li>VRP IP/prefix/ASN lookup is tracked as backend feature #070 and is not available yet.</li>
</ul>
</Panel>
<Panel title="Endpoint reference" flush>
<div className="table-wrap">
<table className="data-table endpoint-table">
<thead>
<tr>
<th scope="col">Endpoint</th>
<th scope="col">Purpose</th>
</tr>
</thead>
<tbody>
{ENDPOINTS.map((endpoint) => (
<tr key={`${endpoint.method}:${endpoint.path}`}>
<td>
<span className={`method${endpoint.method === "POST" ? " post" : ""}`}>{endpoint.method}</span>
{endpoint.path}
</td>
<td>{endpoint.note}</td>
</tr>
))}
</tbody>
</table>
</div>
</Panel>
</div>
);
}

View File

@@ -1,100 +0,0 @@
import { useQuery } from "@tanstack/react-query";
import { exportDownloadUrl, listExports } from "../api/service";
import type { ExportJobRecord } from "../api/schemas";
import { DataTable, type Column } from "../components/DataTable";
import { PageHeader } from "../components/PageHeader";
import { Panel } from "../components/Panel";
import { StatusPill } from "../components/StatusPill";
import { Notice } from "../components/StateBlock";
import { formatBytes, formatInt, formatRelative, formatUtc } from "../lib/format";
import { useRun } from "../lib/useRun";
/**
* Export job history for the active run. Jobs are held in the query service
* process memory, so the list resets when the service restarts.
*/
export default function ExportsPage() {
const { runId } = useRun();
const exportsQuery = useQuery({
queryKey: ["exports", runId],
queryFn: () => listExports(runId, { limit: 50 }),
refetchInterval: (query) => {
const jobs = query.state.data?.items ?? [];
return jobs.some((job) => job.status === "running") ? 2000 : false;
},
});
const columns: Column<ExportJobRecord>[] = [
{
key: "job",
header: "Job",
render: (job) => <span className="mono text-small">{job.jobId.slice(0, 12)}…</span>,
},
{ key: "scope", header: "Scope", render: (job) => <span className="chip">{job.scope}</span> },
{
key: "target",
header: "Target",
render: (job) => (
<span className="mono text-small">{job.repoId ?? job.ppId ?? "object set"}</span>
),
},
{ key: "status", header: "Status", render: (job) => <StatusPill status={job.status} /> },
{
key: "objects",
header: "Objects",
numeric: true,
render: (job) => formatInt(job.objectCount),
},
{ key: "bytes", header: "Size", numeric: true, render: (job) => formatBytes(job.bytesWritten) },
{
key: "created",
header: "Created",
render: (job) => <span title={formatUtc(job.createdAt)}>{formatRelative(job.createdAt)}</span>,
},
{
key: "error",
header: "Error",
render: (job) =>
job.error ? <span className="text-small" title={job.error}>{job.error}</span> : <span className="text-faint">—</span>,
},
{
key: "download",
header: "",
render: (job) =>
job.status === "complete" ? (
<a className="btn small primary" href={exportDownloadUrl(runId, job.jobId)} download>
Download
</a>
) : null,
},
];
return (
<div className="page">
<PageHeader
title="Exports"
subtitle="Evidence bundles (raw object tarballs) produced by the query service"
/>
<Notice kind="info">
Export jobs live in the query service process memory and are cleared on
restart. Creating exports requires the service to run with --repo-bytes-db.
New exports can be started from an object, publication point or repository
page.
</Notice>
<Panel title="Jobs" flush>
<DataTable
columns={columns}
rows={exportsQuery.data?.items ?? []}
rowKey={(job) => job.jobId}
loading={exportsQuery.isPending}
error={exportsQuery.isError ? exportsQuery.error : undefined}
onRetry={() => exportsQuery.refetch()}
emptyTitle="No export jobs yet"
emptyHint="Start an export from an object detail page (object set) to see it here."
caption="Export jobs"
/>
</Panel>
</div>
);
}

View File

@@ -1,17 +0,0 @@
import { Link } from "react-router-dom";
export default function NotFoundPage() {
return (
<div className="page">
<div className="not-found">
<div className="not-found-code">404</div>
<p className="text-muted">This page does not exist.</p>
<p style={{ marginTop: 12 }}>
<Link className="btn" to="/">
Back to Overview
</Link>
</p>
</div>
</div>
);
}

View File

@@ -1,393 +0,0 @@
import { useState } from "react";
import { Link, useParams, useSearchParams } from "react-router-dom";
import { useMutation, useQuery } from "@tanstack/react-query";
import { Download, PackageOpen, Play, RefreshCw } from "lucide-react";
import { apiFetchBlob, saveBlob } from "../api/client";
import {
createExport,
explainObjectValidation,
getExportJob,
getObject,
getObjectChain,
getObjectProjection,
getObjectValidation,
rawObjectUrl,
} from "../api/service";
import type { ValidationIssue } from "../api/schemas";
import { CopyableValue } from "../components/CopyableValue";
import { DataTable, type Column } from "../components/DataTable";
import { PageHeader } from "../components/PageHeader";
import { Panel } from "../components/Panel";
import { ProjectionView } from "../components/ProjectionView";
import { StatusPill } from "../components/StatusPill";
import { EmptyBlock, ErrorBlock, LoadingBlock, Notice } from "../components/StateBlock";
import { TabPanel, Tabs } from "../components/Tabs";
import { WorkflowStatus } from "../components/WorkflowStatus";
import { formatBytes, formatInt, objectTypeLabel } from "../lib/format";
import { withRunParam } from "../lib/run";
import { useRun } from "../lib/useRun";
function filenameFor(uri: string | undefined, sha256: string | undefined, type: string): string {
const last = uri?.split("/").filter(Boolean).pop();
if (last && last.length <= 128) return last;
const ext = type === "mft" ? ".mft" : type === "crl" ? ".crl" : type === "roa" ? ".roa" : type === "asa" ? ".asa" : ".cer";
return `${(sha256 ?? "object").slice(0, 16)}${ext}`;
}
function IssueList({ issues }: { issues: ValidationIssue[] }) {
if (issues.length === 0) return <p className="text-muted">No issues recorded.</p>;
return (
<ul style={{ margin: 0, paddingLeft: 18, display: "flex", flexDirection: "column", gap: 6 }}>
{issues.map((issue, i) => (
<li key={i}>
<StatusPill status={issue.severity} />
{issue.reasonCode ? <code style={{ marginLeft: 8 }}>{issue.reasonCode}</code> : null}
<span style={{ marginLeft: 8 }}>{issue.reasonText ?? "—"}</span>
{issue.rfcRefs?.length ? (
<span className="text-faint text-small" style={{ marginLeft: 8 }}>
[{issue.rfcRefs.join(", ")}]
</span>
) : null}
</li>
))}
</ul>
);
}
function ValidationTab({ runId, objectInstanceId }: { runId: string; objectInstanceId: string }) {
const [forceRefresh, setForceRefresh] = useState(false);
const validationQuery = useQuery({
queryKey: ["object-validation", runId, objectInstanceId],
queryFn: () => getObjectValidation(runId, objectInstanceId),
staleTime: 5 * 60_000,
});
const explainMutation = useMutation({
mutationFn: () => explainObjectValidation(runId, objectInstanceId, { forceRefresh }),
});
if (validationQuery.isPending) return <LoadingBlock label="Loading validation summary…" />;
if (validationQuery.isError) {
return (
<ErrorBlock
error={validationQuery.error}
onRetry={() => validationQuery.refetch()}
title="Failed to load validation summary"
/>
);
}
const validation = validationQuery.data;
const explain = explainMutation.data;
return (
<div style={{ display: "flex", flexDirection: "column", gap: 16, padding: 16 }}>
<dl className="meta-grid">
<dt>Final status</dt>
<dd><StatusPill status={validation.finalStatus} /></dd>
<dt>Audit result</dt>
<dd><StatusPill status={validation.auditResult} /></dd>
<dt>Detail</dt>
<dd>{validation.detailSummary ?? "—"}</dd>
</dl>
<section>
<h3 className="text-small" style={{ marginBottom: 8 }}>File validation (parsevalidate)</h3>
<p style={{ marginBottom: 8 }}>
<StatusPill status={validation.parsevalidate?.status} />
</p>
<IssueList issues={validation.parsevalidate?.issues ?? []} />
</section>
<section>
<h3 className="text-small" style={{ marginBottom: 8 }}>Chain validation (chainvalidate)</h3>
<p style={{ marginBottom: 8 }}>
<StatusPill status={validation.chainvalidate?.status} />
</p>
<IssueList issues={validation.chainvalidate?.issues ?? []} />
</section>
<section>
<h3 className="text-small" style={{ marginBottom: 8 }}>Explain validation</h3>
<div style={{ display: "flex", gap: 12, alignItems: "center", flexWrap: "wrap", marginBottom: 8 }}>
<button
type="button"
className="btn"
onClick={() => explainMutation.mutate()}
disabled={explainMutation.isPending}
>
{explainMutation.isPending ? (
<span className="spinner small" aria-hidden="true" />
) : (
<Play size={13} aria-hidden="true" />
)}
{explain ? "Run again" : "Run explain"}
</button>
<label className="filter-check" style={{ paddingBottom: 0 }}>
<input
type="checkbox"
checked={forceRefresh}
onChange={(e) => setForceRefresh(e.target.checked)}
/>
Force refresh (bypass cached explain)
</label>
</div>
{explainMutation.isError ? (
<Notice kind="error">Explain failed: {explainMutation.error.message}</Notice>
) : null}
{explain ? (
<div style={{ display: "flex", flexDirection: "column", gap: 8 }}>
<Notice kind="info">
Explain mode: <strong>{explain.explainMode ?? "audit projection"}</strong> —{" "}
{explain.authoritative
? "authoritative revalidation."
: "not a full revalidation; derived from the run audit trail."}
</Notice>
<dl className="meta-grid">
<dt>Final status</dt>
<dd><StatusPill status={explain.finalStatus} /></dd>
<dt>Parse stage</dt>
<dd><StatusPill status={explain.parsevalidate?.status} /></dd>
<dt>Chain stage</dt>
<dd>
<StatusPill status={explain.chainvalidate?.status} />{" "}
<span className="text-muted text-small">
{formatInt(explain.chainvalidate?.edgesCount)} chain edges
</span>
</dd>
{explain.chainvalidate?.note ? (
<>
<dt>Note</dt>
<dd>{explain.chainvalidate.note}</dd>
</>
) : null}
</dl>
</div>
) : null}
</section>
</div>
);
}
function ChainTab({ runId, objectInstanceId }: { runId: string; objectInstanceId: string }) {
const chainQuery = useQuery({
queryKey: ["object-chain", runId, objectInstanceId],
queryFn: () => getObjectChain(runId, objectInstanceId),
staleTime: 5 * 60_000,
});
if (chainQuery.isPending) return <LoadingBlock label="Loading chain edges…" />;
if (chainQuery.isError) {
return <ErrorBlock error={chainQuery.error} onRetry={() => chainQuery.refetch()} title="Failed to load chain" />;
}
const edges = chainQuery.data;
if (edges.length === 0) {
return <EmptyBlock title="No chain edges" hint="No issuer/manifest relationships were recorded for this object." />;
}
const columns: Column<(typeof edges)[number]>[] = [
{ key: "relation", header: "Relation", render: (edge) => <span className="chip">{edge.relation}</span> },
{
key: "from",
header: "From URI",
render: (edge) => <CopyableValue value={edge.fromUri} max={48} label="from URI" />,
},
{
key: "to",
header: "To URI",
render: (edge) => <CopyableValue value={edge.toUri} max={48} label="to URI" />,
},
{ key: "status", header: "Status", render: (edge) => <StatusPill status={edge.status} /> },
];
return <DataTable columns={columns} rows={edges} rowKey={(e) => `${e.relation}:${e.fromUri}:${e.toUri}`} caption="Certificate chain edges" />;
}
export default function ObjectDetailPage() {
const { objectInstanceId = "" } = useParams();
const { runId } = useRun();
const [searchParams, setSearchParams] = useSearchParams();
const tab = searchParams.get("tab") ?? "parsed";
const [exportJobId, setExportJobId] = useState<string | null>(null);
const objectQuery = useQuery({
queryKey: ["object", runId, objectInstanceId],
queryFn: () => getObject(runId, objectInstanceId),
});
const projectionQuery = useQuery({
queryKey: ["object-parsed", runId, objectInstanceId],
queryFn: () => getObjectProjection(runId, objectInstanceId),
enabled: tab === "parsed",
staleTime: 5 * 60_000,
});
const rawMutation = useMutation({
mutationFn: async () => {
const blob = await apiFetchBlob(rawObjectUrl(runId, objectInstanceId));
const object = objectQuery.data;
saveBlob(blob, filenameFor(object?.uri, object?.sha256, object?.objectType ?? ""));
},
});
const exportMutation = useMutation({
mutationFn: () => createExport(runId, { scope: "object_set", objectInstanceIds: [objectInstanceId] }),
onSuccess: (job) => setExportJobId(job.jobId),
});
const exportJobQuery = useQuery({
queryKey: ["export-job", runId, exportJobId],
queryFn: () => getExportJob(runId, exportJobId!),
enabled: exportJobId !== null,
refetchInterval: (query) => (query.state.data?.status === "running" ? 2000 : false),
});
if (objectQuery.isError) {
return (
<div className="page">
<PageHeader title="Object" />
<ErrorBlock error={objectQuery.error} onRetry={() => objectQuery.refetch()} title="Failed to load object" />
</div>
);
}
if (objectQuery.isPending) {
return (
<div className="page">
<PageHeader title="Object" />
<LoadingBlock />
</div>
);
}
const object = objectQuery.data;
const setTab = (next: string) => {
const params = new URLSearchParams(searchParams);
params.set("tab", next);
setSearchParams(params, { preventScrollReset: true });
};
return (
<div className="page">
<nav className="breadcrumbs" aria-label="Breadcrumb">
<Link to={withRunParam("/objects", runId)}>Objects</Link>
<span aria-hidden="true">/</span>
<span>{objectTypeLabel(object.objectType)}</span>
</nav>
<PageHeader
title={
<span style={{ display: "inline-flex", gap: 10, alignItems: "center", flexWrap: "wrap" }}>
{objectTypeLabel(object.objectType)} object
<StatusPill status={object.rejected ? "rejected" : object.result} />
<StatusPill status={object.sourceSection} />
</span>
}
subtitle={<CopyableValue value={object.uri} max={110} label="object URI" />}
actions={
<>
<button
type="button"
className="btn"
onClick={() => rawMutation.mutate()}
disabled={rawMutation.isPending}
>
{rawMutation.isPending ? <span className="spinner small" aria-hidden="true" /> : <Download size={13} aria-hidden="true" />}
Download raw
</button>
<button
type="button"
className="btn"
onClick={() => exportMutation.mutate()}
disabled={exportMutation.isPending || exportJobQuery.data?.status === "running"}
>
{exportMutation.isPending ? <RefreshCw size={13} aria-hidden="true" /> : <PackageOpen size={13} aria-hidden="true" />}
Export object set
</button>
</>
}
/>
{rawMutation.isError ? (
<Notice kind="error">
Raw download failed: {rawMutation.error.message}. The query service needs
--repo-bytes-db to serve raw bytes.
</Notice>
) : null}
{exportMutation.isError ? (
<Notice kind="error">Export failed to start: {exportMutation.error.message}</Notice>
) : null}
{exportJobQuery.data ? <WorkflowStatus job={exportJobQuery.data} runId={runId} /> : null}
<Panel className="object-header-card">
<dl className="meta-grid">
<dt>Object type</dt>
<dd><span className="chip">{objectTypeLabel(object.objectType)}</span></dd>
<dt>URI</dt>
<dd><CopyableValue value={object.uri} max={110} label="object URI" /></dd>
<dt>SHA-256</dt>
<dd><CopyableValue value={object.sha256} max={72} label="object sha256" /></dd>
<dt>Instance ID</dt>
<dd><CopyableValue value={object.objectInstanceId} max={56} label="object instance id" /></dd>
<dt>Result</dt>
<dd><StatusPill status={object.rejected ? "rejected" : object.result} /></dd>
<dt>Reject reason</dt>
<dd>{object.rejectReason ?? <span className="text-faint">—</span>}</dd>
<dt>Detail</dt>
<dd>{object.detailSummary ?? <span className="text-faint">—</span>}</dd>
<dt>Size</dt>
<dd>{formatBytes(object.sizeBytes)}</dd>
<dt>Repository</dt>
<dd>
<Link className="mono" to={withRunParam(`/repositories/${encodeURIComponent(object.repoId)}`, runId)}>
{object.repoId}
</Link>
</dd>
<dt>Publication point</dt>
<dd>
<Link className="mono" to={withRunParam(`/publication-points/${encodeURIComponent(object.ppId)}`, runId)}>
{object.ppId}
</Link>
</dd>
</dl>
</Panel>
<Panel flush>
<Tabs
tabs={[
{ id: "parsed", label: "Parsed" },
{ id: "validation", label: "Validation" },
{ id: "chain", label: "Chain" },
]}
active={tab}
onChange={setTab}
ariaLabel="Object detail sections"
/>
<TabPanel id="parsed" active={tab}>
<div style={{ padding: 16 }}>
{projectionQuery.isPending ? (
<LoadingBlock label="Loading parsed projection…" />
) : projectionQuery.isError ? (
<ErrorBlock
error={projectionQuery.error}
onRetry={() => projectionQuery.refetch()}
title="Failed to load parsed projection"
/>
) : (
<ProjectionView
record={projectionQuery.data ?? null}
runId={runId}
objectInstanceId={objectInstanceId}
/>
)}
</div>
</TabPanel>
<TabPanel id="validation" active={tab}>
<ValidationTab runId={runId} objectInstanceId={objectInstanceId} />
</TabPanel>
<TabPanel id="chain" active={tab}>
<ChainTab runId={runId} objectInstanceId={objectInstanceId} />
</TabPanel>
</Panel>
</div>
);
}

View File

@@ -1,71 +0,0 @@
import { useMemo } from "react";
import { useSearchParams } from "react-router-dom";
import { listObjects } from "../api/service";
import { Notice } from "../components/StateBlock";
import {
EMPTY_OBJECT_FILTERS,
ObjectsTable,
type ObjectFilterValues,
} from "../components/ObjectsTable";
import { PageHeader } from "../components/PageHeader";
import { Panel } from "../components/Panel";
import { useRun } from "../lib/useRun";
function filtersFromParams(params: URLSearchParams): ObjectFilterValues {
return {
type: params.get("type") ?? "",
result: params.get("result") ?? "",
rejectedOnly: params.get("rejected") === "true",
q: params.get("q") ?? "",
};
}
export default function ObjectsPage() {
const { runId } = useRun();
const [searchParams, setSearchParams] = useSearchParams();
const filters = useMemo(() => filtersFromParams(searchParams), [searchParams]);
const setFilters = (next: ObjectFilterValues) => {
const params = new URLSearchParams(searchParams);
const write = (key: string, value: string) => {
if (value) params.set(key, value);
else params.delete(key);
};
write("type", next.type);
write("result", next.result);
write("q", next.q);
if (next.rejectedOnly) params.set("rejected", "true");
else params.delete("rejected");
setSearchParams(params, { preventScrollReset: true });
};
const hasFilters =
filters.type !== EMPTY_OBJECT_FILTERS.type ||
filters.result !== "" ||
filters.rejectedOnly ||
filters.q !== "";
return (
<div className="page">
<PageHeader
title="Objects"
subtitle="Browse every object instance indexed for the active run. Lists stream from the run report — deep paging on large runs can be slow."
/>
{!hasFilters ? (
<Notice kind="info">
This run may contain hundreds of thousands of objects. Use the filters to
narrow the scan; all filters are applied server-side.
</Notice>
) : null}
<Panel title="All objects" flush>
<ObjectsTable
runId={runId}
queryKeyScope="global"
fetcher={(f) => listObjects(runId, f)}
filters={filters}
onFiltersChange={setFilters}
/>
</Panel>
</div>
);
}

View File

@@ -1,381 +0,0 @@
import { useMemo } from "react";
import { Link, useNavigate } from "react-router-dom";
import { useQueries } from "@tanstack/react-query";
import {
Cell,
Pie,
PieChart,
ResponsiveContainer,
Bar,
BarChart,
XAxis,
YAxis,
Tooltip,
} from "recharts";
import { Clock, GitBranch, Timer } from "lucide-react";
import {
getStatsObjectTypes,
getStatsReasons,
getStatsValidation,
listRepos,
} from "../api/service";
import type { RepositoryRecord } from "../api/schemas";
import { DataTable, type Column } from "../components/DataTable";
import { KpiCard } from "../components/KpiCard";
import { PageHeader } from "../components/PageHeader";
import { Panel } from "../components/Panel";
import { StatusPill } from "../components/StatusPill";
import { ErrorBlock, LoadingBlock } from "../components/StateBlock";
import {
formatDurationMs,
formatInt,
formatPercent,
formatRelative,
formatUtc,
objectTypeLabel,
truncateMiddle,
} from "../lib/format";
import { withRunParam } from "../lib/run";
import { useRun } from "../lib/useRun";
/** Semantic colors keyed by validation result — never assigned by index. */
const RESULT_COLORS: Record<string, string> = {
ok: "#16a34a",
valid: "#16a34a",
warnings: "#d97706",
warning: "#d97706",
error: "#dc2626",
invalid: "#dc2626",
rejected: "#dc2626",
skipped: "#94a3b8",
unknown: "#94a3b8",
};
const FALLBACK_COLORS = ["#2563eb", "#0ea5e9", "#6366f1", "#8b5cf6", "#64748b"];
const TYPE_PALETTE = ["#1d4ed8", "#2563eb", "#3b82f6", "#60a5fa", "#93c5fd", "#0ea5e9", "#38bdf8", "#7dd3fc"];
function colorForResult(name: string, index: number): string {
return RESULT_COLORS[name.toLowerCase()] ?? FALLBACK_COLORS[index % FALLBACK_COLORS.length];
}
export default function OverviewPage() {
const { runId, runQuery } = useRun();
const navigate = useNavigate();
const [validationQuery, typesQuery, reasonsQuery, reposQuery] = useQueries({
queries: [
{
queryKey: ["stats", runId, "validation"],
queryFn: () => getStatsValidation(runId),
staleTime: 60_000,
},
{
queryKey: ["stats", runId, "object-types"],
queryFn: () => getStatsObjectTypes(runId),
staleTime: 60_000,
},
{
queryKey: ["stats", runId, "reasons"],
queryFn: () => getStatsReasons(runId),
staleTime: 60_000,
},
{
queryKey: ["repos", runId, "top"],
queryFn: () => listRepos(runId, { limit: 8 }),
staleTime: 60_000,
},
],
});
const run = runQuery.data;
const counts = run?.counts;
const validationData = useMemo(() => {
const map = validationQuery.data ?? {};
return Object.entries(map)
.map(([name, value]) => ({ name, value }))
.sort((a, b) => b.value - a.value);
}, [validationQuery.data]);
const validationTotal = useMemo(
() => validationData.reduce((sum, entry) => sum + entry.value, 0),
[validationData],
);
const typeData = useMemo(() => {
const map = typesQuery.data ?? {};
return Object.entries(map)
.map(([name, value]) => ({ name: objectTypeLabel(name), value }))
.sort((a, b) => b.value - a.value)
.slice(0, 8);
}, [typesQuery.data]);
const reasonRows = useMemo(() => {
const map = reasonsQuery.data ?? {};
return Object.entries(map)
.map(([reason, count]) => ({ reason, count }))
.sort((a, b) => b.count - a.count)
.slice(0, 8);
}, [reasonsQuery.data]);
const maxReason = reasonRows[0]?.count ?? 0;
if (runQuery.isError) {
return (
<div className="page">
<PageHeader title="Overview" />
<ErrorBlock
error={runQuery.error}
onRetry={() => runQuery.refetch()}
title="Failed to load the latest run"
/>
</div>
);
}
const repoColumns: Column<RepositoryRecord>[] = [
{
key: "host",
header: "Repository",
render: (repo) => (
<div>
<div className="cell-main">{repo.host}</div>
<div className="text-faint text-small mono ellipsis" style={{ maxWidth: 260 }}>
{truncateMiddle(repo.uri, 56)}
</div>
</div>
),
},
{
key: "transport",
header: "Transport",
render: (repo) => <span className="chip">{repo.transport ?? "unknown"}</span>,
},
{ key: "objects", header: "Objects", numeric: true, render: (repo) => formatInt(repo.objects) },
{
key: "rejected",
header: "Rejected",
numeric: true,
render: (repo) =>
repo.rejectedObjects ? (
<span style={{ color: "var(--red-700)", fontWeight: 600 }}>
{formatInt(repo.rejectedObjects)}
</span>
) : (
"0"
),
},
{
key: "duration",
header: "Sync time",
numeric: true,
render: (repo) => formatDurationMs(repo.syncDurationMsTotal),
},
];
return (
<div className="page">
<PageHeader
title="Overview"
subtitle={
run ? (
<span className="overview-run-strip">
<span className="run-strip-item mono">{run.runId}</span>
<span className="run-strip-item">
<Clock size={13} aria-hidden="true" />
<span title={formatUtc(run.validationTime)}>
validated {formatRelative(run.validationTime)}
</span>
</span>
{run.syncMode ? (
<span className="run-strip-item">
<GitBranch size={13} aria-hidden="true" /> {run.syncMode} sync
</span>
) : null}
<span className="run-strip-item">
<Timer size={13} aria-hidden="true" /> wall {formatDurationMs(run.wallMs)}
</span>
{run.indexStatus ? <StatusPill status={run.indexStatus} label={`index ${run.indexStatus}`} /> : null}
</span>
) : undefined
}
/>
{runQuery.isPending ? (
<LoadingBlock label="Loading latest run…" />
) : (
<>
<div className="kpi-grid">
<KpiCard label="VRPs" value={formatInt(counts?.vrps)} to={withRunParam("/objects?type=roa", runId)} />
<KpiCard label="ASPAs" value={formatInt(counts?.aspas)} to={withRunParam("/objects?type=aspa", runId)} />
<KpiCard label="Objects" value={formatInt(counts?.objects)} to={withRunParam("/objects", runId)} />
<KpiCard
label="Publication points"
value={formatInt(counts?.publicationPoints)}
to={withRunParam("/publication-points", runId)}
/>
<KpiCard
label="Rejected"
value={formatInt(counts?.rejectedObjects)}
tone="red"
to={withRunParam("/validation", runId)}
/>
<KpiCard
label="Warnings"
value={formatInt(counts?.warnings)}
tone="amber"
sub="objects with warnings"
/>
</div>
<div className="overview-charts">
<Panel title="Validation results" subtitle="Object audit results for this run">
{validationQuery.isError ? (
<ErrorBlock error={validationQuery.error} onRetry={() => validationQuery.refetch()} />
) : validationQuery.isPending ? (
<LoadingBlock />
) : validationData.length === 0 ? (
<p className="text-muted">No validation stats recorded.</p>
) : (
<>
<div className="chart-box">
<ResponsiveContainer>
<PieChart>
<Pie
data={validationData}
dataKey="value"
nameKey="name"
innerRadius="58%"
outerRadius="88%"
paddingAngle={2}
strokeWidth={0}
>
{validationData.map((entry, index) => (
<Cell key={entry.name} fill={colorForResult(entry.name, index)} />
))}
</Pie>
<Tooltip
formatter={(value, name) => [
`${formatInt(Number(value))} (${formatPercent(Number(value), validationTotal)})`,
String(name),
]}
/>
</PieChart>
</ResponsiveContainer>
</div>
<div className="chart-legend">
{validationData.map((entry, index) => (
<span className="legend-item" key={entry.name}>
<span
className="legend-swatch"
style={{ background: colorForResult(entry.name, index) }}
/>
{entry.name} · {formatInt(entry.value)} ({formatPercent(entry.value, validationTotal)})
</span>
))}
</div>
</>
)}
</Panel>
<Panel title="Object types" subtitle="Objects by parsed type">
{typesQuery.isError ? (
<ErrorBlock error={typesQuery.error} onRetry={() => typesQuery.refetch()} />
) : typesQuery.isPending ? (
<LoadingBlock />
) : typeData.length === 0 ? (
<p className="text-muted">No object type stats recorded.</p>
) : (
<div className="chart-box">
<ResponsiveContainer>
<BarChart data={typeData} layout="vertical" margin={{ left: 12, right: 24 }}>
<XAxis type="number" hide />
<YAxis
type="category"
dataKey="name"
width={104}
tickLine={false}
axisLine={false}
tick={{ fontSize: 12, fill: "var(--text-2)" }}
/>
<Tooltip formatter={(value) => formatInt(Number(value))} />
<Bar dataKey="value" radius={[0, 4, 4, 0]} barSize={16}>
{typeData.map((entry, index) => (
<Cell key={entry.name} fill={TYPE_PALETTE[index % TYPE_PALETTE.length]} />
))}
</Bar>
</BarChart>
</ResponsiveContainer>
</div>
)}
</Panel>
</div>
<div className="overview-bottom">
<Panel
title="Top repositories"
subtitle="First repositories by index order"
tools={
<Link className="btn small" to={withRunParam("/repositories", runId)}>
View all
</Link>
}
flush
>
{reposQuery.isError ? (
<div className="panel-body">
<ErrorBlock error={reposQuery.error} onRetry={() => reposQuery.refetch()} />
</div>
) : (
<DataTable
columns={repoColumns}
rows={reposQuery.data?.items ?? []}
rowKey={(repo) => repo.repoId}
loading={reposQuery.isPending}
emptyTitle="No repositories indexed"
onRowClick={(repo) =>
navigate(withRunParam(`/repositories/${encodeURIComponent(repo.repoId)}`, runId))
}
/>
)}
</Panel>
<Panel
title="Top reject reasons"
subtitle="Click a reason to inspect matching objects"
tools={
<Link className="btn small" to={withRunParam("/validation", runId)}>
Validation
</Link>
}
>
{reasonsQuery.isError ? (
<ErrorBlock error={reasonsQuery.error} onRetry={() => reasonsQuery.refetch()} />
) : reasonsQuery.isPending ? (
<LoadingBlock />
) : reasonRows.length === 0 ? (
<p className="text-muted">No rejected objects in this run.</p>
) : (
<div className="reason-list">
{reasonRows.map((row) => (
<Link
key={row.reason}
className="reason-row"
to={withRunParam(`/validation?reason=${encodeURIComponent(row.reason)}`, runId)}
title={row.reason}
>
<span className="reason-text">{row.reason}</span>
<span className="reason-count">{formatInt(row.count)}</span>
<span className="reason-bar" aria-hidden="true">
<span style={{ width: `${maxReason ? (row.count / maxReason) * 100 : 0}%` }} />
</span>
</Link>
))}
</div>
)}
</Panel>
</div>
</>
)}
</div>
);
}

View File

@@ -1,115 +0,0 @@
import { useState } from "react";
import { Link, useParams } from "react-router-dom";
import { useQuery } from "@tanstack/react-query";
import {
getPublicationPoint,
listPublicationPointObjects,
} from "../api/service";
import { CopyableValue } from "../components/CopyableValue";
import { KpiCard } from "../components/KpiCard";
import {
EMPTY_OBJECT_FILTERS,
ObjectsTable,
type ObjectFilterValues,
} from "../components/ObjectsTable";
import { PageHeader } from "../components/PageHeader";
import { Panel } from "../components/Panel";
import { StatusPill } from "../components/StatusPill";
import { ErrorBlock, LoadingBlock } from "../components/StateBlock";
import { formatDurationMs, formatInt, formatUtc } from "../lib/format";
import { withRunParam } from "../lib/run";
import { useRun } from "../lib/useRun";
export default function PublicationPointDetailPage() {
const { ppId = "" } = useParams();
const { runId } = useRun();
const [objectFilters, setObjectFilters] = useState<ObjectFilterValues>(EMPTY_OBJECT_FILTERS);
const ppQuery = useQuery({
queryKey: ["pp", runId, ppId],
queryFn: () => getPublicationPoint(runId, ppId),
});
if (ppQuery.isError) {
return (
<div className="page">
<PageHeader title="Publication point" />
<ErrorBlock error={ppQuery.error} onRetry={() => ppQuery.refetch()} title="Failed to load publication point" />
</div>
);
}
if (ppQuery.isPending) {
return (
<div className="page">
<PageHeader title="Publication point" />
<LoadingBlock />
</div>
);
}
const pp = ppQuery.data;
return (
<div className="page">
<nav className="breadcrumbs" aria-label="Breadcrumb">
<Link to={withRunParam("/publication-points", runId)}>Publication Points</Link>
<span aria-hidden="true">/</span>
<span className="mono">{pp.ppId}</span>
</nav>
<PageHeader
title={<span className="mono" style={{ fontSize: 16 }}>{pp.manifestRsyncUri ?? pp.ppId}</span>}
subtitle={pp.repoSyncError ? `Sync error: ${pp.repoSyncError}` : undefined}
actions={<StatusPill status={pp.repoTerminalState} label={`terminal ${pp.repoTerminalState ?? "unknown"}`} />}
/>
<div className="kpi-grid">
<KpiCard label="Objects" value={formatInt(pp.objects)} />
<KpiCard label="Rejected" value={formatInt(pp.rejectedObjects)} tone="red" />
<KpiCard label="Warnings" value={formatInt(pp.warnings)} tone="amber" />
<KpiCard label="Sync time" value={formatDurationMs(pp.repoSyncDurationMs)} />
</div>
<Panel title="Publication point details">
<dl className="meta-grid">
<dt>PP ID</dt>
<dd><CopyableValue value={pp.ppId} max={48} label="publication point id" /></dd>
<dt>Repository</dt>
<dd>
<Link to={withRunParam(`/repositories/${encodeURIComponent(pp.repoId)}`, runId)} className="mono">
{pp.repoId}
</Link>
</dd>
<dt>Manifest URI</dt>
<dd><CopyableValue value={pp.manifestRsyncUri} max={96} label="manifest URI" /></dd>
<dt>PP rsync URI</dt>
<dd><CopyableValue value={pp.publicationPointRsyncUri} max={96} label="publication point rsync URI" /></dd>
<dt>rsync base</dt>
<dd><CopyableValue value={pp.rsyncBaseUri} max={96} label="rsync base URI" /></dd>
<dt>RRDP notification</dt>
<dd><CopyableValue value={pp.rrdpNotificationUri} max={96} label="RRDP notification URI" /></dd>
<dt>Sync source</dt>
<dd>{pp.repoSyncSource ?? pp.source ?? "—"}</dd>
<dt>Sync phase</dt>
<dd>{pp.repoSyncPhase ?? "—"}</dd>
<dt>Terminal state</dt>
<dd><StatusPill status={pp.repoTerminalState} /></dd>
<dt>thisUpdate</dt>
<dd>{formatUtc(pp.thisUpdate)}</dd>
<dt>nextUpdate</dt>
<dd>{formatUtc(pp.nextUpdate)}</dd>
</dl>
</Panel>
<Panel title="Objects" flush>
<ObjectsTable
runId={runId}
queryKeyScope={`pp:${ppId}`}
fetcher={(filters) => listPublicationPointObjects(runId, ppId, filters)}
filters={objectFilters}
onFiltersChange={setObjectFilters}
/>
</Panel>
</div>
);
}

Some files were not shown because too many files have changed in this diff Show More