Compare commits
28 Commits
dev_1.2_xu
...
dev_1.0.0_
| Author | SHA1 | Date | |
|---|---|---|---|
| af1c2c7f88 | |||
| e45830d79f | |||
| 77fc2f1a41 | |||
| e083fe4daa | |||
| c9ef5aaf4c | |||
| 34fb9657f1 | |||
| 6edc420ce2 | |||
| cd0ba15286 | |||
| fe8b89d829 | |||
| d6d44669b4 | |||
| 557a69cbd2 | |||
| 73d8ebb5c1 | |||
| cf764c35bb | |||
| e3339533b8 | |||
| afc50364f8 | |||
| 6276d13814 | |||
| 0f3d65254e | |||
| 13516c4f73 | |||
| 68cbd3c500 | |||
| 1cc3351bef | |||
| 2a6a963ecd | |||
| 6e135b9d7a | |||
| afc31c02ab | |||
| 7be865d7f1 | |||
| a58e507f92 | |||
| cc9f3f21de | |||
| 56ae2ca4fc | |||
| bcd4829486 |
@@ -1,9 +0,0 @@
|
||||
.git
|
||||
.gitignore
|
||||
.idea
|
||||
target
|
||||
rtr-db
|
||||
tests
|
||||
specs
|
||||
scripts
|
||||
README.md
|
||||
6
.gitignore
vendored
6
.gitignore
vendored
@@ -1,5 +1,3 @@
|
||||
target/
|
||||
rtr-db/
|
||||
.idea/
|
||||
logs/
|
||||
report/
|
||||
Cargo.lock
|
||||
perf.*
|
||||
|
||||
3528
Cargo.lock
generated
3528
Cargo.lock
generated
File diff suppressed because it is too large
Load Diff
52
Cargo.toml
52
Cargo.toml
@@ -3,38 +3,34 @@ name = "rpki"
|
||||
version = "0.1.0"
|
||||
edition = "2024"
|
||||
|
||||
[[bin]]
|
||||
name = "rpki_rtr"
|
||||
path = "src/main_rtr.rs"
|
||||
[features]
|
||||
default = ["full"]
|
||||
# Full build used by the main RP implementation (includes RocksDB-backed storage).
|
||||
full = ["dep:rocksdb"]
|
||||
profile = ["dep:pprof", "dep:flate2"]
|
||||
|
||||
[dependencies]
|
||||
der-parser = "10.0.0"
|
||||
asn1-rs = "0.7.1"
|
||||
der-parser = { version = "10.0.0", features = ["serialize"] }
|
||||
hex = "0.4.3"
|
||||
base64 = "0.22.1"
|
||||
sha2 = "0.10.8"
|
||||
thiserror = "2.0.18"
|
||||
time = "0.3.45"
|
||||
ring = "0.17.14"
|
||||
x509-parser = { version = "0.18.0", features = ["verify"] }
|
||||
url = "2.5.8"
|
||||
asn1-rs = "0.7.1"
|
||||
asn1-rs-derive = "0.6.0"
|
||||
asn1 = "0.23.0"
|
||||
arc-swap = "1.7.0"
|
||||
chrono = { version = "0.4.44", features = ["serde"] }
|
||||
chrono-tz = "0.10"
|
||||
bytes = "1.11.1"
|
||||
tokio = { version = "1.49.0", features = ["full"] }
|
||||
rand = "0.10.0"
|
||||
rocksdb = { version = "0.21.0", default-features = false }
|
||||
serde = { version = "1", features = ["derive", "rc"] }
|
||||
serde_json = "1"
|
||||
base64 = "0.22"
|
||||
anyhow = "1"
|
||||
tracing = "0.1.44"
|
||||
sha2 = "0.10"
|
||||
tempfile = "3"
|
||||
tokio-rustls = "0.26"
|
||||
rustls = "0.23"
|
||||
rustls-pemfile = "2"
|
||||
rustls-pki-types = "1.14.0"
|
||||
socket2 = "0.5"
|
||||
russh = { version = "0.60.0", default-features = false, features = ["ring", "rsa"] }
|
||||
tracing-subscriber = { version = "0.3", features = ["env-filter", "fmt"] }
|
||||
serde = { version = "1.0.218", features = ["derive"] }
|
||||
serde_json = "1.0.140"
|
||||
toml = "0.8.20"
|
||||
rocksdb = { version = "0.22.0", optional = true, default-features = false, features = ["lz4"] }
|
||||
serde_cbor = "0.11.2"
|
||||
roxmltree = "0.20.0"
|
||||
quick-xml = "0.37.2"
|
||||
uuid = { version = "1.7.0", features = ["v4"] }
|
||||
reqwest = { version = "0.12.12", default-features = false, features = ["blocking", "rustls-tls"] }
|
||||
pprof = { version = "0.14.1", optional = true, features = ["flamegraph", "prost-codec"] }
|
||||
flate2 = { version = "1.0.35", optional = true }
|
||||
tempfile = "3.16.0"
|
||||
|
||||
[dev-dependencies]
|
||||
|
||||
238
README.client.md
238
README.client.md
@@ -1,238 +0,0 @@
|
||||
# RTR Client 使用说明
|
||||
|
||||
本文说明仓库内三类 RTR Client 的用途、启动方式和验证命令:
|
||||
|
||||
- `deploy/client`: 本仓库自带的 `rtr_debug_client`,适合做 RTR 协议调试、连通性检查和单次查询。
|
||||
- `deploy/frr`: FRR 作为黑盒 RTR Client,适合验证真实路由软件是否能消费 server 下发的 VRP。
|
||||
- `deploy/bird`: BIRD 作为黑盒 RTR Client,适合验证 TCP 或 SSH transport 下的 RPKI/ROA/ASPA 数据导入。
|
||||
|
||||
以下命令默认从仓库根目录执行。
|
||||
|
||||
## 目录
|
||||
|
||||
- [前置条件](#前置条件)
|
||||
- [rtr_debug_client](#rtr_debug_client)
|
||||
- [TCP](#tcp)
|
||||
- [TLS/mTLS](#tlsmtls)
|
||||
- [SSH](#ssh)
|
||||
- [多客户端压测](#多客户端压测)
|
||||
- [FRR Client](#frr-client)
|
||||
- [BIRD Client](#bird-client)
|
||||
- [TCP](#tcp-1)
|
||||
- [SSH](#ssh-1)
|
||||
- [如何判断 Client 正常](#如何判断-client-正常)
|
||||
|
||||
## 前置条件
|
||||
|
||||
先启动 RTR Server。TCP 模式是最常用的本地验证方式:
|
||||
|
||||
```bash
|
||||
./deploy/server/build-run.sh up --mode tcp
|
||||
```
|
||||
|
||||
如果要测试 TLS 或 SSH,需要先启动对应 server 模式,并准备匹配的证书或 SSH key:
|
||||
|
||||
```bash
|
||||
./deploy/server/build-run.sh up --mode tls
|
||||
./deploy/server/build-run.sh up --mode ssh
|
||||
```
|
||||
|
||||
TLS/SSH 所需的本地开发证书和密钥可以通过脚本生成:
|
||||
|
||||
```bash
|
||||
bash scripts/generate-certs.sh
|
||||
```
|
||||
|
||||
已有 `certs/` 时使用 `--force` 覆盖。脚本生成的 `certs/tls` 和 `certs/ssh` 文件名与本仓库 compose 默认配置一致。
|
||||
|
||||
Client 连接地址不要求固定在某个 Docker 网络里。`deploy/client` 的 TCP/TLS/SSH 模式分别通过 `RPKI_RTR_TCP_SERVER_ADDR`、`RPKI_RTR_TLS_SERVER_ADDR`、`RPKI_RTR_SSH_SERVER_ADDR` 指定 server 地址。`deploy/bird` 通过 `RPKI_BIRD_RPKI_HOST`/`RPKI_BIRD_RPKI_PORT` 指定 TCP server 地址,通过 `RPKI_BIRD_SSH_RPKI_HOST`/`RPKI_RTR_SSH_PORT` 指定 SSH server 地址。示例 compose 里的默认值只是为了和本仓库 server compose 直接联调。
|
||||
|
||||
`deploy/frr` 示例默认连接 `127.0.0.1:323`,可以通过 `RPKI_FRR_RPKI_HOST`/`RPKI_FRR_RPKI_PORT` 覆盖。
|
||||
|
||||
## rtr_debug_client
|
||||
|
||||
`rtr_debug_client` 是本仓库提供的调试客户端。它会发起 RTR Reset Query 或 Serial Query,并打印 server 返回的 PDU 摘要。Docker 配置默认执行:
|
||||
|
||||
```text
|
||||
<server_addr> <protocol_version> reset --keep-after-error --summary-only
|
||||
```
|
||||
|
||||
### TCP
|
||||
|
||||
```bash
|
||||
./deploy/client/build-run.sh up --mode tcp
|
||||
./deploy/client/build-run.sh logs --mode tcp
|
||||
./deploy/client/build-run.sh down --mode tcp
|
||||
```
|
||||
|
||||
默认连接 `rpki-rtr:323`,协议版本默认是 `2`。可以通过 `deploy/client/.env` 或环境变量覆盖:
|
||||
|
||||
```bash
|
||||
RPKI_RTR_TCP_SERVER_ADDR=10.0.0.12:323 \
|
||||
RPKI_RTR_PROTOCOL_VERSION=2 \
|
||||
./deploy/client/build-run.sh up --mode tcp
|
||||
```
|
||||
|
||||
### TLS/mTLS
|
||||
|
||||
```bash
|
||||
./deploy/client/build-run.sh up --mode tls
|
||||
./deploy/client/build-run.sh logs --mode tls
|
||||
./deploy/client/build-run.sh down --mode tls
|
||||
```
|
||||
|
||||
TLS 模式默认读取 `certs/tls` 下的示例证书,并使用 `localhost` 作为 server name。常用变量:
|
||||
|
||||
| 变量 | 默认值 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `RPKI_RTR_TLS_SERVER_ADDR` | `rpki-rtr:324` | TLS RTR server 地址 |
|
||||
| `RPKI_RTR_TLS_SERVER_NAME` | `localhost` | 必须匹配 server 证书 SAN dNSName |
|
||||
| `RPKI_RTR_TLS_CA_CERT_PATH` | `/app/certs/client-ca.crt` | 容器内 CA 证书路径 |
|
||||
| `RPKI_RTR_TLS_CLIENT_CERT_PATH` | `/app/certs/client-good.crt` | 容器内 client 证书路径 |
|
||||
| `RPKI_RTR_TLS_CLIENT_KEY_PATH` | `/app/certs/client-good.key` | 容器内 client 私钥路径 |
|
||||
| `RPKI_RTR_TLS_CERTS_HOST_DIR` | `../../certs/tls` | 宿主机 TLS 证书目录 |
|
||||
|
||||
### SSH
|
||||
|
||||
```bash
|
||||
./deploy/client/build-run.sh up --mode ssh
|
||||
./deploy/client/build-run.sh logs --mode ssh
|
||||
./deploy/client/build-run.sh down --mode ssh
|
||||
```
|
||||
|
||||
SSH 模式默认连接 `rpki-rtr-ssh:22`,使用 key 认证,并校验 server public key。常用变量:
|
||||
|
||||
| 变量 | 默认值 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `RPKI_RTR_SSH_SERVER_ADDR` | `rpki-rtr:22` | SSH RTR server 地址 |
|
||||
| `RPKI_RTR_SSH_USERNAME` | `rpki-rtr` | SSH 用户名 |
|
||||
| `RPKI_RTR_CLIENT_KEYS_VOLUME` | `../../certs/ssh:/app/certs:ro` | 宿主机 SSH key 目录挂载 |
|
||||
| `RPKI_RTR_CLIENT_KEY_PATH` | `/app/certs/rtr-client.key` | 容器内 client 私钥 |
|
||||
| `RPKI_RTR_SSH_SERVER_PUBKEY_PATH` | `/app/certs/ssh_host_rsa_key.pub` | 容器内 server public key |
|
||||
|
||||
### 多客户端压测
|
||||
|
||||
`--clients multi` 会启动 5 个 TCP debug client,用于观察并发连接、client report 和 server 连接统计。
|
||||
|
||||
```bash
|
||||
./deploy/client/build-run.sh up --mode tcp --clients multi
|
||||
./deploy/client/build-run.sh logs --mode tcp --clients multi
|
||||
./deploy/client/build-run.sh down --mode tcp --clients multi
|
||||
```
|
||||
|
||||
## FRR Client
|
||||
|
||||
FRR 示例用于验证标准路由软件能否通过 RTR over TCP 从 server 获取前缀验证数据。
|
||||
|
||||
启动:
|
||||
|
||||
```bash
|
||||
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml up -d
|
||||
```
|
||||
|
||||
默认连接 `127.0.0.1:323`。可以通过环境变量覆盖:
|
||||
|
||||
```bash
|
||||
RPKI_FRR_RPKI_HOST=10.0.0.12 \
|
||||
RPKI_FRR_RPKI_PORT=323 \
|
||||
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml up -d
|
||||
```
|
||||
|
||||
验证连接和数据:
|
||||
|
||||
```bash
|
||||
docker exec -it frr-rpki-client vtysh -c "show rpki configuration"
|
||||
docker exec -it frr-rpki-client vtysh -c "show rpki cache-connection"
|
||||
docker exec -it frr-rpki-client vtysh -c "show rpki prefix-table"
|
||||
```
|
||||
|
||||
停止:
|
||||
|
||||
```bash
|
||||
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml down
|
||||
```
|
||||
|
||||
FRR 容器启动时会读取 `deploy/frr/frr.conf.template`,并生成 `/etc/frr/frr.conf`。模板中的 RTR cache 配置为:
|
||||
|
||||
```text
|
||||
rpki cache tcp ${RPKI_FRR_RPKI_HOST} ${RPKI_FRR_RPKI_PORT} preference ${RPKI_FRR_RPKI_PREFERENCE}
|
||||
```
|
||||
|
||||
如果需要调整 polling、timeout、BGP router-id 等其他 FRR 配置,可以修改 `deploy/frr/frr.conf.template`,或在 compose 中挂载自己的模板和 entrypoint。
|
||||
|
||||
## BIRD Client
|
||||
|
||||
BIRD 示例用于验证 BIRD 3.x 对 RTR v2、ROA 和 ASPA 的导入情况。容器启动后会周期性输出 RPKI 协议状态和表项摘要。
|
||||
|
||||
### TCP
|
||||
|
||||
启动:
|
||||
|
||||
```bash
|
||||
docker compose --env-file deploy/bird/.env -f deploy/bird/docker-compose.yml up -d --build
|
||||
```
|
||||
|
||||
查看日志:
|
||||
|
||||
```bash
|
||||
docker logs -f bird-rpki-client
|
||||
```
|
||||
|
||||
停止:
|
||||
|
||||
```bash
|
||||
docker compose --env-file deploy/bird/.env -f deploy/bird/docker-compose.yml down
|
||||
```
|
||||
|
||||
默认连接 `rpki-rtr:323`。可以通过环境变量覆盖:
|
||||
|
||||
```bash
|
||||
RPKI_BIRD_RPKI_HOST=10.0.0.12 \
|
||||
RPKI_BIRD_RPKI_PORT=323 \
|
||||
docker compose --env-file deploy/bird/.env -f deploy/bird/docker-compose.yml up -d --build
|
||||
```
|
||||
|
||||
### SSH
|
||||
|
||||
先启动 server SSH 模式:
|
||||
|
||||
```bash
|
||||
./deploy/server/build-run.sh up --mode ssh
|
||||
```
|
||||
|
||||
再启动 BIRD SSH client:
|
||||
|
||||
```bash
|
||||
docker compose --env-file deploy/bird/.env \
|
||||
-f deploy/bird/docker-compose.yml \
|
||||
-f deploy/bird/docker-compose.ssh.yml \
|
||||
up -d --build
|
||||
```
|
||||
|
||||
查看日志:
|
||||
|
||||
```bash
|
||||
docker logs -f bird-rpki-client
|
||||
```
|
||||
|
||||
SSH 模式默认读取 `certs/ssh` 目录下的 key,并使用 `bird.conf.ssh.template` 生成运行时配置。关键默认值:
|
||||
|
||||
| 变量 | 默认值 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `RPKI_BIRD_SSH_RPKI_HOST` | `rpki-rtr` | SSH RTR server 主机名 |
|
||||
| `RPKI_RTR_SSH_PORT` | `22` | SSH RTR server 端口 |
|
||||
| `RPKI_BIRD_SSH_CERTS_HOST_DIR` | `../../certs/ssh` | 宿主机 SSH 证书和 key 目录,复用 `rtr-client.key` |
|
||||
|
||||
## 如何判断 Client 正常
|
||||
|
||||
- `rtr_debug_client`: 日志中能看到 Reset Query 成功、EndOfData 和 payload 统计。
|
||||
- FRR: `show rpki cache-connection` 显示已连接,`show rpki prefix-table` 有 VRP 条目。
|
||||
- BIRD: 日志中 RPKI protocol 状态为 `up`,ROA/ASPA 表项摘要不为空。
|
||||
|
||||
如果 client 无法连接,优先检查:
|
||||
|
||||
- server 是否按对应 transport 启动。
|
||||
- client 配置的 server 地址是否从当前容器或主机可达。
|
||||
- TLS 的 server name 是否匹配证书 SAN。
|
||||
- SSH 的用户名、client key 和 server public key 是否匹配。
|
||||
- `RPKI_RTR_PROTOCOL_VERSION` 或 BIRD 配置中的 `min version`/`max version` 是否和 server 能力一致。
|
||||
377
README.md
377
README.md
@@ -1,373 +1,28 @@
|
||||
# RPKI RTR Server
|
||||
|
||||
这是一个用 Rust 实现的 RPKI-to-Router (RTR) cache server。服务端从 CCR 文件中读取 RPKI 数据,生成 RTR cache snapshot 和 delta,并通过 RTR 协议提供给路由器或调试客户端。
|
||||
|
||||
当前代码重点覆盖:
|
||||
# 单元测试
|
||||
|
||||
- RTR server:TCP 默认启用,可选启用 TLS/mTLS 和进程内 SSH transport。
|
||||
- 数据输入:从 CCR 目录选择最新 `.ccr` 文件,解析 VRP 和 VAP/ASPA。
|
||||
- 本地策略:可选加载 SLURM 文件,对 CCR 解析出的 payload 做过滤和本地断言。
|
||||
- 持久化:使用 RocksDB 保存 RTR cache 状态。
|
||||
- 运行报告:输出 source、client、runtime 三类 JSON report。
|
||||
- 运行期管理:可选启用 HTTP Admin API,修改部分 runtime config、触发 source reload、管理 SLURM 文件。
|
||||
- 调试工具:`rtr_debug_client` 支持 TCP、TLS 和 SSH 连接 RTR server。
|
||||
```
|
||||
cargo test
|
||||
|
||||
## 目录
|
||||
|
||||
- [协议参考](#协议参考)
|
||||
- [项目结构](#项目结构)
|
||||
- [构建与测试](#构建与测试)
|
||||
- [数据输入](#数据输入)
|
||||
- [快速启动](#快速启动)
|
||||
- [前置数据生成](#前置数据生成)
|
||||
- [Docker 运行](#docker-运行)
|
||||
- [跨架构构建](#跨架构构建)
|
||||
- [传输模式](#传输模式)
|
||||
- [主要环境变量](#主要环境变量)
|
||||
- [运行报告](#运行报告)
|
||||
- [Admin API](#admin-api)
|
||||
- [调试客户端](#调试客户端)
|
||||
- [部署入口](#部署入口)
|
||||
- [开发说明](#开发说明)
|
||||
|
||||
## 协议参考
|
||||
|
||||
- RTR: [RFC 6810](https://www.rfc-editor.org/rfc/rfc6810.html), [RFC 8210](https://www.rfc-editor.org/rfc/rfc8210.html)
|
||||
- SLURM: [RFC 8416](https://www.rfc-editor.org/rfc/rfc8416.html)
|
||||
- CCR: [draft-ietf-sidrops-rpki-ccr](https://www.ietf.org/archive/id/draft-ietf-sidrops-rpki-ccr-02.html)
|
||||
|
||||
## 项目结构
|
||||
|
||||
| 路径 | 说明 |
|
||||
| --- | --- |
|
||||
| `src/main_rtr.rs` | RTR server 入口 |
|
||||
| `src/rtr/` | RTR 协议、cache、store、server transport、admin、report |
|
||||
| `src/source/` | CCR/SLURM source 加载流程 |
|
||||
| `src/slurm/` | SLURM 文件解析、合并和策略应用 |
|
||||
| `src/data_model/` | RPKI 相关数据结构 |
|
||||
| `src/bin/rtr_debug_client/` | RTR 调试客户端 |
|
||||
| `deploy/server/` | server Docker 镜像和 compose |
|
||||
| `deploy/client/` | debug client Docker 镜像和 compose |
|
||||
| `deploy/frr/`, `deploy/bird/` | FRR/BIRD 作为 RTR client 的测试部署 |
|
||||
| `tests/` | 单元和集成测试 |
|
||||
|
||||
## 构建与测试
|
||||
|
||||
```bash
|
||||
cargo build --bin rpki_rtr
|
||||
cargo build --bin rtr_debug_client
|
||||
cargo test
|
||||
# 查看输出
|
||||
cargo test -- --nocapture
|
||||
```
|
||||
|
||||
服务端 binary 名称是 `rpki_rtr`:
|
||||
# 覆盖率(cargo-llvm-cov)
|
||||
|
||||
```bash
|
||||
cargo run --bin rpki_rtr
|
||||
安装工具:
|
||||
|
||||
```
|
||||
rustup component add llvm-tools-preview
|
||||
cargo install cargo-llvm-cov --locked
|
||||
```
|
||||
|
||||
默认配置会监听 `0.0.0.0:323`,从 `./data` 查找 CCR 文件,并使用 `./rtr-db` 保存 RocksDB 数据。启动前需要确保 CCR 目录里存在可解析的 `.ccr` 文件。
|
||||
统计行覆盖率并要求 >=90%:
|
||||
|
||||
## 数据输入
|
||||
|
||||
服务端启动和刷新时会读取 `RPKI_RTR_CCR_DIR` 指向的目录:
|
||||
|
||||
1. 如果该目录下存在包含 `.ccr` 文件的子目录,先按子目录名排序选择最新子目录。
|
||||
2. 在选中的目录中按文件名排序选择最新 `.ccr` 文件。
|
||||
3. 从 CCR 中解析 VRP 和 VAP/ASPA。
|
||||
4. 如果配置了 `RPKI_RTR_SLURM_DIR`,加载其中启用的 `*.slurm` 文件并应用本地策略。
|
||||
|
||||
## 快速启动
|
||||
|
||||
### 前置数据生成
|
||||
|
||||
TLS/mTLS 或 SSH transport 需要本地证书和密钥。开发和 Docker 联调可用脚本生成 compose 默认引用的最小文件集:
|
||||
|
||||
```bash
|
||||
bash scripts/10_generate-certs.sh
|
||||
```
|
||||
./scripts/coverage.sh
|
||||
# 或
|
||||
cargo llvm-cov --fail-under-lines 90
|
||||
```
|
||||
|
||||
如需覆盖已有 `certs/` 内容:
|
||||
|
||||
```bash
|
||||
bash scripts/10_generate-certs.sh --force
|
||||
```
|
||||
|
||||
`10_generate-certs.sh` 常用参数:
|
||||
|
||||
| 参数 | 说明 |
|
||||
| --- | --- |
|
||||
| `--out-dir DIR` | 指定输出目录,默认生成到 `./certs`。 |
|
||||
| `--server-dns NAME` | 指定 TLS server 证书的 dNSName SAN,默认 `localhost`。 |
|
||||
| `--tls-only` | 只生成 `certs/tls` 下的 TLS 证书和私钥。 |
|
||||
| `--ssh-only` | 只生成 `certs/ssh` 下的 SSH host key、client key 和 `authorized_keys`。 |
|
||||
| `--force` | 覆盖已有文件。未指定时,如果目标文件已存在会直接报错退出。 |
|
||||
| `-h`, `--help` | 查看脚本帮助。 |
|
||||
|
||||
TLS server name 默认是 `localhost`。如果需要其它 DNS 名称,需要同时设置 `scripts/10_generate-certs.sh --server-dns <name>` 和 client 的 `RPKI_RTR_TLS_SERVER_NAME=<name>`。
|
||||
|
||||
### Docker 运行
|
||||
|
||||
推荐使用 `deploy/server/build-run.sh` 和 `deploy/client/build-run.sh` 分别管理 server 与 debug client:
|
||||
以下命令默认从仓库根目录执行。
|
||||
|
||||
```bash
|
||||
chmod +x deploy/server/build-run.sh deploy/client/build-run.sh
|
||||
./deploy/server/build-run.sh up --mode tcp
|
||||
./deploy/client/build-run.sh up --mode tcp
|
||||
./deploy/client/build-run.sh down
|
||||
./deploy/server/build-run.sh down
|
||||
```
|
||||
|
||||
脚本常用参数:
|
||||
|
||||
| 参数/变量 | server | client | 说明 |
|
||||
| --- | --- | --- | --- |
|
||||
| `--mode base|tcp|tls|ssh`, `MODE` | yes | yes | 选择 compose 模式,默认 `tcp`。 |
|
||||
| `--clients single|multi`, `CLIENTS` | no | yes | 单 client 或 5 client,默认 `single`。 |
|
||||
| `--no-build`, `BUILD=0` | yes | yes | 只运行已有镜像,不执行 compose build。 |
|
||||
| `--server-image TAG`, `SERVER_IMAGE` | yes | no | server 镜像名;会传给 compose 的 `RPKI_RTR_SERVER_IMAGE`。 |
|
||||
| `--client-image TAG`, `CLIENT_IMAGE` | no | yes | client 镜像名;会传给 compose 的 `RPKI_RTR_CLIENT_IMAGE`。 |
|
||||
| `--platform VALUE`, `TARGET_PLATFORM` | yes | yes | `buildx` 目标平台,默认 `linux/arm64`。 |
|
||||
| `--push`, `BUILDX_PUSH=1` | yes | yes | `buildx` 构建后推送到 registry;不设置时加载到本地 Docker。 |
|
||||
| `--skip-base-pull`, `SKIP_BASE_IMAGE_PULL=1` | yes | yes | 跳过 buildx 前的基础镜像预拉取。 |
|
||||
|
||||
支持的 action 包括 `up`、`build`、`buildx`、`rebuild`、`down`/`stop`、`restart`、`logs`、`ps`。完整帮助见:
|
||||
|
||||
```bash
|
||||
./deploy/server/build-run.sh --help
|
||||
./deploy/client/build-run.sh --help
|
||||
```
|
||||
|
||||
### 跨架构构建
|
||||
示例:x86 开发机本地构建 ARM 镜像,打包后传到 ARM 部署机。
|
||||
|
||||
```bash
|
||||
./deploy/server/build-run.sh buildx \
|
||||
--platform linux/arm64 \
|
||||
--server-image rpki-rtr:arm64
|
||||
|
||||
./deploy/client/build-run.sh buildx \
|
||||
--platform linux/arm64 \
|
||||
--client-image rpki-rtr-debug-client:arm64
|
||||
|
||||
mkdir -p dist/images
|
||||
docker save rpki-rtr:arm64 -o dist/images/rpki-rtr-arm64.tar
|
||||
docker save rpki-rtr-debug-client:arm64 -o dist/images/rpki-rtr-debug-client-arm64.tar
|
||||
|
||||
scp dist/images/*.tar user@arm-host:/tmp/
|
||||
```
|
||||
|
||||
ARM 部署机加载镜像并运行:
|
||||
|
||||
```bash
|
||||
docker load -i /tmp/rpki-rtr-arm64.tar
|
||||
docker load -i /tmp/rpki-rtr-debug-client-arm64.tar
|
||||
|
||||
SERVER_IMAGE=rpki-rtr:arm64 \
|
||||
BUILD=0 ./deploy/server/build-run.sh up --mode tcp
|
||||
|
||||
CLIENT_IMAGE=rpki-rtr-debug-client:arm64 \
|
||||
BUILD=0 ./deploy/client/build-run.sh up --mode tcp
|
||||
```
|
||||
|
||||
```bash
|
||||
docker compose -f deploy/server/docker-compose.yml up -d --build
|
||||
docker compose -f deploy/server/docker-compose.yml down
|
||||
```
|
||||
|
||||
TLS/mTLS 示例:
|
||||
|
||||
```bash
|
||||
docker compose -f deploy/server/docker-compose.tls.yml up -d --build
|
||||
docker compose -f deploy/server/docker-compose.tls.yml down
|
||||
```
|
||||
|
||||
## 传输模式
|
||||
|
||||
TCP 默认启用。TLS 和 SSH 是附加监听,不会替代 TCP。
|
||||
|
||||
| 模式 | 默认地址 | 启用方式 |
|
||||
| --- | --- | --- |
|
||||
| TCP | `0.0.0.0:323` | 默认启用 |
|
||||
| TLS/mTLS | `0.0.0.0:324` | `RPKI_RTR_ENABLE_TLS=true` |
|
||||
| SSH | `0.0.0.0:22` | `RPKI_RTR_ENABLE_SSH=true` |
|
||||
|
||||
TLS server 会加载服务端证书、私钥和 client CA。SSH server 使用 OpenSSH host key 和 authorized_keys 文件;`RPKI_RTR_SSH_AUTH_MODE` 支持 `key`、`password`、`both`。
|
||||
|
||||
## 主要环境变量
|
||||
|
||||
布尔值支持 `true/false`、`1/0`、`yes/no`、`on/off`。
|
||||
|
||||
| 变量 | 默认值 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `RPKI_RTR_ENABLE_TLS` | `false` | 是否启用 TLS/mTLS 监听 |
|
||||
| `RPKI_RTR_ENABLE_SSH` | `false` | 是否启用 SSH 监听 |
|
||||
| `RPKI_RTR_TCP_ADDR` | `0.0.0.0:323` | TCP 监听地址 |
|
||||
| `RPKI_RTR_TLS_ADDR` | `0.0.0.0:324` | TLS 监听地址 |
|
||||
| `RPKI_RTR_SSH_ADDR` | `0.0.0.0:22` | SSH 监听地址 |
|
||||
| `RPKI_RTR_SSH_PORT` | 未设置 | 仅覆盖 `RPKI_RTR_SSH_ADDR` 中的端口 |
|
||||
| `RPKI_RTR_DB_PATH` | `./rtr-db` | RocksDB 路径 |
|
||||
| `RPKI_RTR_CCR_DIR` | `./data` | CCR 输入目录 |
|
||||
| `RPKI_RTR_SLURM_DIR` | 未设置 | SLURM 目录;空值表示禁用 |
|
||||
| `RPKI_RTR_REPORT_DIR` | `./report` | JSON report 输出目录 |
|
||||
| `RPKI_RTR_MAX_DELTA` | `100` | 每个 RTR 版本保留的最大 delta 数,必须大于 0 |
|
||||
| `RPKI_RTR_PRUNE_DELTA_BY_SNAPSHOT_SIZE` | `false` | 是否按 snapshot wire size 继续裁剪 delta window |
|
||||
| `RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS` | `300` | source 刷新间隔,单位秒 |
|
||||
| `RPKI_RTR_REFRESH_INTERVAL_SECS` | 未设置 | 旧变量名,仍兼容;建议使用 `RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS` |
|
||||
| `RPKI_RTR_TIMING_REFRESH_SECS` | `3600` | RTR EndOfData `refresh` 字段 |
|
||||
| `RPKI_RTR_TIMING_RETRY_SECS` | `600` | RTR EndOfData `retry` 字段 |
|
||||
| `RPKI_RTR_TIMING_EXPIRE_SECS` | `7200` | RTR EndOfData `expire` 字段,必须大于 refresh 和 retry |
|
||||
| `RPKI_RTR_MAX_CONNECTIONS` | `512` | 最大并发 RTR 连接数 |
|
||||
| `RPKI_RTR_MAX_CONCURRENT_HANDSHAKES` | `128` | 最大并发握手数,不能大于最大连接数 |
|
||||
| `RPKI_RTR_NOTIFY_QUEUE_SIZE` | `1024` | Serial Notify 广播队列大小 |
|
||||
| `RPKI_RTR_TCP_KEEPALIVE_SECS` | `60` | TCP keepalive;设为 `0` 表示禁用 |
|
||||
| `RPKI_RTR_WARN_INSECURE_TCP` | `true` | TCP 模式是否输出安全提示 |
|
||||
| `RPKI_RTR_REQUIRE_TLS_SERVER_DNS_NAME_SAN` | `false` | 是否要求 TLS 服务端证书包含 dNSName SAN |
|
||||
| `RPKI_RTR_ENFORCE_TLS_CLIENT_SAN_IP_MATCH` | `true` | 是否校验 TLS client 证书 SAN IP 与 peer IP 匹配 |
|
||||
| `RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS` | `300` | runtime report 周期 |
|
||||
| `RPKI_RTR_REPORT_HISTORY_LIMIT` | `10` | 每类 report 滚动保留数量 |
|
||||
| `RPKI_RTR_TIMEZONE` | `Asia/Shanghai` | 日志和 report 使用的时区 |
|
||||
| `RPKI_RTR_ADMIN_ADDR` | 未设置 | Admin API 监听地址;未设置时关闭 |
|
||||
| `RPKI_RTR_ADMIN_TOKEN` | 未设置 | Admin API Bearer token;非 loopback 监听时必须设置 |
|
||||
|
||||
TLS 相关路径:
|
||||
|
||||
| 变量 | 默认值 |
|
||||
| --- | --- |
|
||||
| `RPKI_RTR_TLS_CERT_PATH` | `./certs/tls/server-dns.crt` |
|
||||
| `RPKI_RTR_TLS_KEY_PATH` | `./certs/tls/server-dns.key` |
|
||||
| `RPKI_RTR_TLS_CLIENT_CA_PATH` | `./certs/tls/client-ca.crt` |
|
||||
|
||||
SSH 相关配置:
|
||||
|
||||
| 变量 | 默认值 |
|
||||
| --- | --- |
|
||||
| `RPKI_RTR_SSH_HOST_KEY_PATH` | `./certs/ssh/ssh_host_rsa_key` |
|
||||
| `RPKI_RTR_SSH_AUTHORIZED_KEYS_PATH` | `./certs/ssh/rtr-authorized_keys` |
|
||||
| `RPKI_RTR_SSH_USERNAME` | `rpki-rtr` |
|
||||
| `RPKI_RTR_SSH_SUBSYSTEM_NAME` | `rpki-rtr` |
|
||||
| `RPKI_RTR_SSH_AUTH_MODE` | `key` |
|
||||
| `RPKI_RTR_SSH_PASSWORD` | 未设置 |
|
||||
|
||||
## 运行报告
|
||||
|
||||
服务会在 `RPKI_RTR_REPORT_DIR` 下写入 JSON report,并按 `RPKI_RTR_REPORT_HISTORY_LIMIT` 滚动保留:
|
||||
|
||||
- `rtr-source-*.json`:CCR/SLURM source、fingerprint、刷新状态、数据质量、cache 统计。
|
||||
- `rtr-clients-*.json`:client 连接数和连接方式统计。
|
||||
- `rtr-runtime-*.json`:进程状态、服务状态和当前生效的 runtime config。
|
||||
|
||||
## Admin API
|
||||
|
||||
Admin API 默认关闭。设置 `RPKI_RTR_ADMIN_ADDR` 后启用:
|
||||
|
||||
```bash
|
||||
export RPKI_RTR_ADMIN_ADDR=127.0.0.1:8323
|
||||
export RPKI_RTR_ADMIN_TOKEN=change-me
|
||||
```
|
||||
|
||||
如果监听非 loopback 地址,例如 `0.0.0.0:8323`,必须设置 `RPKI_RTR_ADMIN_TOKEN`。设置 token 后,请求需要携带:
|
||||
|
||||
```http
|
||||
Authorization: Bearer change-me
|
||||
```
|
||||
|
||||
当前实现的主要接口:
|
||||
|
||||
| 方法 | 路径 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `GET` | `/admin/rtr/health` | 健康检查和能力开关 |
|
||||
| `GET` | `/admin/rtr/config` | 查看 runtime config |
|
||||
| `POST` | `/admin/rtr/config` | 修改部分 runtime config |
|
||||
| `GET` | `/admin/rtr/logs/tail` | 读取 stdout/stderr 日志 tail |
|
||||
| `POST` | `/admin/rtr/slurm/reload` | 触发 source reload |
|
||||
| `GET` | `/admin/rtr/slurm/files` | 列出 SLURM 文件 |
|
||||
| `POST` | `/admin/rtr/slurm/files` | 新增或覆盖 SLURM 文件 |
|
||||
| `GET` | `/admin/rtr/slurm/files/{name}` | 读取 SLURM 文件 |
|
||||
| `PUT` | `/admin/rtr/slurm/files/{name}` | 新增或覆盖指定 SLURM 文件 |
|
||||
| `DELETE` | `/admin/rtr/slurm/files/{name}` | 删除 SLURM 文件 |
|
||||
| `POST` | `/admin/rtr/slurm/files/{name}/enable` | 启用 `.slurm.disabled` 文件 |
|
||||
| `POST` | `/admin/rtr/slurm/files/{name}/disable` | 禁用 `.slurm` 文件 |
|
||||
|
||||
示例:
|
||||
|
||||
```bash
|
||||
curl http://127.0.0.1:8323/admin/rtr/config \
|
||||
-H "Authorization: Bearer change-me"
|
||||
|
||||
curl -X POST http://127.0.0.1:8323/admin/rtr/config \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer change-me" \
|
||||
-d '{"max_delta": 6, "source_refresh_interval_seconds": 60}'
|
||||
```
|
||||
|
||||
## 调试客户端
|
||||
|
||||
`rtr_debug_client` 用于手动发起 Reset Query 或 Serial Query,并观察服务端返回的 PDU。
|
||||
|
||||
RTR Client 的 Docker 调试、FRR/BIRD 互通验证和常用排障步骤见 `README.client.md`。
|
||||
|
||||
TCP:
|
||||
|
||||
```bash
|
||||
cargo run --bin rtr_debug_client -- 127.0.0.1:323 1 reset
|
||||
cargo run --bin rtr_debug_client -- 127.0.0.1:323 1 serial 42 100
|
||||
```
|
||||
|
||||
TLS:
|
||||
|
||||
```bash
|
||||
cargo run --bin rtr_debug_client -- \
|
||||
127.0.0.1:324 1 reset \
|
||||
--tls \
|
||||
--ca-cert certs/tls/client-ca.crt \
|
||||
--server-name localhost \
|
||||
--client-cert certs/tls/client-good.crt \
|
||||
--client-key certs/tls/client-good.key
|
||||
```
|
||||
|
||||
SSH:
|
||||
|
||||
```bash
|
||||
cargo run --bin rtr_debug_client -- \
|
||||
127.0.0.1:22 1 reset \
|
||||
--ssh \
|
||||
--ssh-user rpki-rtr \
|
||||
--ssh-key certs/ssh/rtr-client.key \
|
||||
--ssh-server-key certs/ssh/ssh_host_rsa_key.pub
|
||||
```
|
||||
|
||||
SSH password auth 也已实现;SSH 模式下 `--ssh-key` 和 `--ssh-password` 二选一,同时必须通过 `--ssh-known-hosts` 或 `--ssh-server-key` 做 host key 校验。
|
||||
|
||||
更多参数见 `src/bin/rtr_debug_client/README.md`。
|
||||
|
||||
## 部署入口
|
||||
|
||||
| 用途 | Compose 文件 |
|
||||
| --- | --- |
|
||||
| Server | `deploy/server/docker-compose.yml` |
|
||||
| Server TCP 示例 | `deploy/server/docker-compose.tcp.yml` |
|
||||
| Server TLS 示例 | `deploy/server/docker-compose.tls.yml` |
|
||||
| Server SSH 示例 | `deploy/server/docker-compose.ssh.yml` |
|
||||
| Debug client | `deploy/client/docker-compose.yml` |
|
||||
| Debug client 多实例 | `deploy/client/docker-compose.clients.yml` |
|
||||
| FRR client | `deploy/frr/docker-compose.yml` |
|
||||
| BIRD client | `deploy/bird/docker-compose.yml` |
|
||||
|
||||
RTR Client 统一使用说明见 `README.client.md`。
|
||||
|
||||
通用操作:
|
||||
|
||||
```bash
|
||||
COMPOSE_FILE=deploy/server/docker-compose.yml
|
||||
docker compose -f "${COMPOSE_FILE}" up -d --build
|
||||
docker compose -f "${COMPOSE_FILE}" ps
|
||||
docker compose -f "${COMPOSE_FILE}" down
|
||||
```
|
||||
|
||||
## 开发说明
|
||||
|
||||
- 代码中的 RTR cache 按协议版本维护状态;v0 只包含 Route Origin,v1 包含 Route Origin 和 Router Key,v2 包含 Route Origin、Router Key 和 ASPA。
|
||||
- 当前 source pipeline 从 CCR 生成 Route Origin 和 ASPA,再可选应用 SLURM。
|
||||
- `RtrStore::save_cache_state_versioned(...)` 是 cache 状态写入 RocksDB 的核心入口,相关边界测试在 `tests/test_store_boundary.rs`。
|
||||
- Admin API 的日志 tail 依赖部署入口写出的 stdout/stderr 日志文件;容器入口脚本会把日志写到 `/app/logs`。
|
||||
|
||||
8
benchmark/ours_manifest_bench/Cargo.toml
Normal file
8
benchmark/ours_manifest_bench/Cargo.toml
Normal file
@@ -0,0 +1,8 @@
|
||||
[package]
|
||||
name = "ours-manifest-bench"
|
||||
version = "0.1.0"
|
||||
edition = "2024"
|
||||
|
||||
[dependencies]
|
||||
rpki = { path = "../..", default-features = false }
|
||||
|
||||
145
benchmark/ours_manifest_bench/src/main.rs
Normal file
145
benchmark/ours_manifest_bench/src/main.rs
Normal file
@@ -0,0 +1,145 @@
|
||||
use rpki::data_model::manifest::ManifestObject;
|
||||
use std::hint::black_box;
|
||||
use std::path::PathBuf;
|
||||
use std::time::Instant;
|
||||
|
||||
#[derive(Debug, Clone)]
|
||||
struct Config {
|
||||
sample: Option<String>,
|
||||
manifest_path: Option<PathBuf>,
|
||||
iterations: u64,
|
||||
warmup_iterations: u64,
|
||||
repeats: u32,
|
||||
}
|
||||
|
||||
fn usage_and_exit() -> ! {
|
||||
eprintln!(
|
||||
"Usage:\n ours-manifest-bench (--sample <name> | --manifest <path>) [--iterations N] [--warmup-iterations N] [--repeats N]\n\nExamples:\n cargo run --release -- --sample small-01 --iterations 20000 --warmup-iterations 2000 --repeats 3\n cargo run --release -- --manifest ../../tests/benchmark/selected_der/small-01.mft"
|
||||
);
|
||||
std::process::exit(2);
|
||||
}
|
||||
|
||||
fn parse_args() -> Config {
|
||||
let mut sample: Option<String> = None;
|
||||
let mut manifest_path: Option<PathBuf> = None;
|
||||
let mut iterations: u64 = 20_000;
|
||||
let mut warmup_iterations: u64 = 2_000;
|
||||
let mut repeats: u32 = 3;
|
||||
|
||||
let mut args = std::env::args().skip(1);
|
||||
while let Some(arg) = args.next() {
|
||||
match arg.as_str() {
|
||||
"--sample" => sample = Some(args.next().unwrap_or_else(|| usage_and_exit())),
|
||||
"--manifest" => {
|
||||
manifest_path = Some(PathBuf::from(args.next().unwrap_or_else(|| usage_and_exit())))
|
||||
}
|
||||
"--iterations" => {
|
||||
iterations = args
|
||||
.next()
|
||||
.unwrap_or_else(|| usage_and_exit())
|
||||
.parse()
|
||||
.unwrap_or_else(|_| usage_and_exit())
|
||||
}
|
||||
"--warmup-iterations" => {
|
||||
warmup_iterations = args
|
||||
.next()
|
||||
.unwrap_or_else(|| usage_and_exit())
|
||||
.parse()
|
||||
.unwrap_or_else(|_| usage_and_exit())
|
||||
}
|
||||
"--repeats" => {
|
||||
repeats = args
|
||||
.next()
|
||||
.unwrap_or_else(|| usage_and_exit())
|
||||
.parse()
|
||||
.unwrap_or_else(|_| usage_and_exit())
|
||||
}
|
||||
"-h" | "--help" => usage_and_exit(),
|
||||
_ => usage_and_exit(),
|
||||
}
|
||||
}
|
||||
|
||||
if sample.is_none() && manifest_path.is_none() {
|
||||
usage_and_exit();
|
||||
}
|
||||
if sample.is_some() && manifest_path.is_some() {
|
||||
usage_and_exit();
|
||||
}
|
||||
|
||||
Config {
|
||||
sample,
|
||||
manifest_path,
|
||||
iterations,
|
||||
warmup_iterations,
|
||||
repeats,
|
||||
}
|
||||
}
|
||||
|
||||
fn derive_manifest_path(sample: &str) -> PathBuf {
|
||||
// Assumes current working directory is `rpki/benchmark/ours_manifest_bench`.
|
||||
PathBuf::from(format!("../../tests/benchmark/selected_der/{sample}.mft"))
|
||||
}
|
||||
|
||||
fn main() {
|
||||
let cfg = parse_args();
|
||||
let manifest_path = cfg
|
||||
.manifest_path
|
||||
.clone()
|
||||
.unwrap_or_else(|| derive_manifest_path(cfg.sample.as_deref().unwrap()));
|
||||
|
||||
let bytes = std::fs::read(&manifest_path).unwrap_or_else(|e| {
|
||||
eprintln!("read manifest fixture failed: {e}; path={}", manifest_path.display());
|
||||
std::process::exit(1);
|
||||
});
|
||||
|
||||
let decoded_once = ManifestObject::decode_der(&bytes).unwrap_or_else(|e| {
|
||||
eprintln!("decode failed: {e}; path={}", manifest_path.display());
|
||||
std::process::exit(1);
|
||||
});
|
||||
let file_count = decoded_once.manifest.file_count();
|
||||
|
||||
let mut round_ns_per_op: Vec<f64> = Vec::with_capacity(cfg.repeats as usize);
|
||||
let mut round_ops_per_s: Vec<f64> = Vec::with_capacity(cfg.repeats as usize);
|
||||
|
||||
for _round in 0..cfg.repeats {
|
||||
for _ in 0..cfg.warmup_iterations {
|
||||
let obj = ManifestObject::decode_der(black_box(&bytes)).expect("warmup decode");
|
||||
black_box(obj);
|
||||
}
|
||||
|
||||
let start = Instant::now();
|
||||
for _ in 0..cfg.iterations {
|
||||
let obj = ManifestObject::decode_der(black_box(&bytes)).expect("timed decode");
|
||||
black_box(obj);
|
||||
}
|
||||
let elapsed = start.elapsed();
|
||||
|
||||
let ns_per_op = (elapsed.as_secs_f64() * 1e9) / (cfg.iterations as f64);
|
||||
let ops_per_s = (cfg.iterations as f64) / elapsed.as_secs_f64();
|
||||
round_ns_per_op.push(ns_per_op);
|
||||
round_ops_per_s.push(ops_per_s);
|
||||
}
|
||||
|
||||
let avg_ns_per_op = round_ns_per_op.iter().sum::<f64>() / (round_ns_per_op.len() as f64);
|
||||
let avg_ops_per_s = round_ops_per_s.iter().sum::<f64>() / (round_ops_per_s.len() as f64);
|
||||
|
||||
let sample_name = cfg.sample.clone().unwrap_or_else(|| {
|
||||
manifest_path
|
||||
.file_name()
|
||||
.map(|s| s.to_string_lossy().to_string())
|
||||
.unwrap_or_else(|| manifest_path.display().to_string())
|
||||
});
|
||||
let sample_name = sample_name
|
||||
.strip_suffix(".mft")
|
||||
.unwrap_or(&sample_name)
|
||||
.to_string();
|
||||
|
||||
println!("fixture: {}", manifest_path.display());
|
||||
println!();
|
||||
println!("| sample | avg ns/op | ops/s | file count |");
|
||||
println!("|---|---:|---:|---:|");
|
||||
println!(
|
||||
"| {} | {:.2} | {:.2} | {} |",
|
||||
sample_name, avg_ns_per_op, avg_ops_per_s, file_count
|
||||
);
|
||||
}
|
||||
8
benchmark/routinator_object_bench/Cargo.toml
Normal file
8
benchmark/routinator_object_bench/Cargo.toml
Normal file
@@ -0,0 +1,8 @@
|
||||
[package]
|
||||
name = "routinator-object-bench"
|
||||
version = "0.1.0"
|
||||
edition = "2024"
|
||||
publish = false
|
||||
|
||||
[dependencies]
|
||||
rpki = { version = "=0.19.1", features = ["repository"] }
|
||||
552
benchmark/routinator_object_bench/src/main.rs
Normal file
552
benchmark/routinator_object_bench/src/main.rs
Normal file
@@ -0,0 +1,552 @@
|
||||
use rpki::repository::cert::Cert;
|
||||
use rpki::repository::crl::Crl;
|
||||
use rpki::repository::manifest::Manifest;
|
||||
use rpki::repository::roa::Roa;
|
||||
use rpki::repository::aspa::Aspa;
|
||||
use rpki::repository::resources::{AsResources, IpResources};
|
||||
|
||||
use std::hint::black_box;
|
||||
use std::path::{Path, PathBuf};
|
||||
use std::time::Instant;
|
||||
|
||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)]
|
||||
enum ObjType {
|
||||
Cer,
|
||||
Crl,
|
||||
Manifest,
|
||||
Roa,
|
||||
Aspa,
|
||||
}
|
||||
|
||||
impl ObjType {
|
||||
fn parse(s: &str) -> Result<Self, String> {
|
||||
match s {
|
||||
"cer" => Ok(Self::Cer),
|
||||
"crl" => Ok(Self::Crl),
|
||||
"manifest" => Ok(Self::Manifest),
|
||||
"roa" => Ok(Self::Roa),
|
||||
"aspa" => Ok(Self::Aspa),
|
||||
_ => Err("type must be one of: cer, crl, manifest, roa, aspa".into()),
|
||||
}
|
||||
}
|
||||
|
||||
fn as_str(self) -> &'static str {
|
||||
match self {
|
||||
ObjType::Cer => "cer",
|
||||
ObjType::Crl => "crl",
|
||||
ObjType::Manifest => "manifest",
|
||||
ObjType::Roa => "roa",
|
||||
ObjType::Aspa => "aspa",
|
||||
}
|
||||
}
|
||||
|
||||
fn ext(self) -> &'static str {
|
||||
match self {
|
||||
ObjType::Cer => "cer",
|
||||
ObjType::Crl => "crl",
|
||||
ObjType::Manifest => "mft",
|
||||
ObjType::Roa => "roa",
|
||||
ObjType::Aspa => "asa",
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
struct Sample {
|
||||
obj_type: ObjType,
|
||||
name: String,
|
||||
path: PathBuf,
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
struct Config {
|
||||
dir: PathBuf,
|
||||
type_filter: Option<ObjType>,
|
||||
sample_filter: Option<String>,
|
||||
fixed_iters: Option<u64>,
|
||||
warmup_iters: u64,
|
||||
rounds: u64,
|
||||
min_round_ms: u64,
|
||||
max_adaptive_iters: u64,
|
||||
strict: bool,
|
||||
cert_inspect: bool,
|
||||
out_csv: Option<PathBuf>,
|
||||
out_md: Option<PathBuf>,
|
||||
}
|
||||
|
||||
fn usage_and_exit(err: Option<&str>) -> ! {
|
||||
if let Some(err) = err {
|
||||
eprintln!("error: {err}");
|
||||
eprintln!();
|
||||
}
|
||||
eprintln!(
|
||||
"Usage:\n\
|
||||
cargo run --release --manifest-path rpki/benchmark/routinator_object_bench/Cargo.toml -- [OPTIONS]\n\
|
||||
\n\
|
||||
Options:\n\
|
||||
--dir <PATH> Fixtures root dir (default: ../../tests/benchmark/selected_der_v2)\n\
|
||||
--type <cer|crl|manifest|roa|aspa> Filter by type\n\
|
||||
--sample <NAME> Filter by sample name (e.g. p50)\n\
|
||||
--iters <N> Fixed iterations per round (optional; otherwise adaptive)\n\
|
||||
--warmup-iters <N> Warmup iterations (default: 50)\n\
|
||||
--rounds <N> Rounds (default: 5)\n\
|
||||
--min-round-ms <MS> Adaptive: minimum round time (default: 200)\n\
|
||||
--max-iters <N> Adaptive: maximum iters (default: 1_000_000)\n\
|
||||
--strict <true|false> Strict DER where applicable (default: true)\n\
|
||||
--cert-inspect Also run Cert::inspect_ca/inspect_ee where applicable (default: false)\n\
|
||||
--out-csv <PATH> Write CSV output\n\
|
||||
--out-md <PATH> Write Markdown output\n\
|
||||
"
|
||||
);
|
||||
std::process::exit(2);
|
||||
}
|
||||
|
||||
fn parse_bool(s: &str, name: &str) -> bool {
|
||||
match s {
|
||||
"1" | "true" | "TRUE" | "yes" | "YES" => true,
|
||||
"0" | "false" | "FALSE" | "no" | "NO" => false,
|
||||
_ => usage_and_exit(Some(&format!("{name} must be true/false"))),
|
||||
}
|
||||
}
|
||||
|
||||
fn parse_u64(s: &str, name: &str) -> u64 {
|
||||
s.parse::<u64>()
|
||||
.unwrap_or_else(|_| usage_and_exit(Some(&format!("{name} must be an integer"))))
|
||||
}
|
||||
|
||||
fn default_samples_dir() -> PathBuf {
|
||||
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../tests/benchmark/selected_der_v2")
|
||||
}
|
||||
|
||||
fn parse_args() -> Config {
|
||||
let mut dir: PathBuf = default_samples_dir();
|
||||
let mut type_filter: Option<ObjType> = None;
|
||||
let mut sample_filter: Option<String> = None;
|
||||
let mut fixed_iters: Option<u64> = None;
|
||||
let mut warmup_iters: u64 = 50;
|
||||
let mut rounds: u64 = 5;
|
||||
let mut min_round_ms: u64 = 200;
|
||||
let mut max_adaptive_iters: u64 = 1_000_000;
|
||||
let mut strict: bool = true;
|
||||
let mut cert_inspect: bool = false;
|
||||
let mut out_csv: Option<PathBuf> = None;
|
||||
let mut out_md: Option<PathBuf> = None;
|
||||
|
||||
let mut args = std::env::args().skip(1);
|
||||
while let Some(arg) = args.next() {
|
||||
match arg.as_str() {
|
||||
"--dir" => dir = PathBuf::from(args.next().unwrap_or_else(|| usage_and_exit(None))),
|
||||
"--type" => {
|
||||
type_filter = Some(ObjType::parse(
|
||||
&args.next().unwrap_or_else(|| usage_and_exit(None)),
|
||||
)
|
||||
.unwrap_or_else(|e| usage_and_exit(Some(&e))))
|
||||
}
|
||||
"--sample" => {
|
||||
sample_filter = Some(args.next().unwrap_or_else(|| usage_and_exit(None)))
|
||||
}
|
||||
"--iters" => {
|
||||
fixed_iters = Some(parse_u64(
|
||||
&args.next().unwrap_or_else(|| usage_and_exit(None)),
|
||||
"--iters",
|
||||
))
|
||||
}
|
||||
"--warmup-iters" => {
|
||||
warmup_iters = parse_u64(
|
||||
&args.next().unwrap_or_else(|| usage_and_exit(None)),
|
||||
"--warmup-iters",
|
||||
)
|
||||
}
|
||||
"--rounds" => {
|
||||
rounds = parse_u64(&args.next().unwrap_or_else(|| usage_and_exit(None)), "--rounds")
|
||||
}
|
||||
"--min-round-ms" => {
|
||||
min_round_ms = parse_u64(
|
||||
&args.next().unwrap_or_else(|| usage_and_exit(None)),
|
||||
"--min-round-ms",
|
||||
)
|
||||
}
|
||||
"--max-iters" => {
|
||||
max_adaptive_iters = parse_u64(
|
||||
&args.next().unwrap_or_else(|| usage_and_exit(None)),
|
||||
"--max-iters",
|
||||
)
|
||||
}
|
||||
"--strict" => {
|
||||
strict = parse_bool(
|
||||
&args.next().unwrap_or_else(|| usage_and_exit(None)),
|
||||
"--strict",
|
||||
)
|
||||
}
|
||||
"--cert-inspect" => cert_inspect = true,
|
||||
"--out-csv" => out_csv = Some(PathBuf::from(args.next().unwrap_or_else(|| usage_and_exit(None)))),
|
||||
"--out-md" => out_md = Some(PathBuf::from(args.next().unwrap_or_else(|| usage_and_exit(None)))),
|
||||
"-h" | "--help" => usage_and_exit(None),
|
||||
_ => usage_and_exit(Some(&format!("unknown argument: {arg}"))),
|
||||
}
|
||||
}
|
||||
|
||||
if warmup_iters == 0 {
|
||||
usage_and_exit(Some("--warmup-iters must be > 0"));
|
||||
}
|
||||
if rounds == 0 {
|
||||
usage_and_exit(Some("--rounds must be > 0"));
|
||||
}
|
||||
if min_round_ms == 0 {
|
||||
usage_and_exit(Some("--min-round-ms must be > 0"));
|
||||
}
|
||||
if max_adaptive_iters == 0 {
|
||||
usage_and_exit(Some("--max-iters must be > 0"));
|
||||
}
|
||||
if let Some(n) = fixed_iters {
|
||||
if n == 0 {
|
||||
usage_and_exit(Some("--iters must be > 0"));
|
||||
}
|
||||
}
|
||||
|
||||
Config {
|
||||
dir,
|
||||
type_filter,
|
||||
sample_filter,
|
||||
fixed_iters,
|
||||
warmup_iters,
|
||||
rounds,
|
||||
min_round_ms,
|
||||
max_adaptive_iters,
|
||||
strict,
|
||||
cert_inspect,
|
||||
out_csv,
|
||||
out_md,
|
||||
}
|
||||
}
|
||||
|
||||
fn read_samples(root: &Path) -> Vec<Sample> {
|
||||
let mut out = Vec::new();
|
||||
for obj_type in [
|
||||
ObjType::Cer,
|
||||
ObjType::Crl,
|
||||
ObjType::Manifest,
|
||||
ObjType::Roa,
|
||||
ObjType::Aspa,
|
||||
] {
|
||||
let dir = root.join(obj_type.as_str());
|
||||
let rd = match std::fs::read_dir(&dir) {
|
||||
Ok(rd) => rd,
|
||||
Err(_) => continue,
|
||||
};
|
||||
for ent in rd.flatten() {
|
||||
let path = ent.path();
|
||||
if path.extension().and_then(|s| s.to_str()) != Some(obj_type.ext()) {
|
||||
continue;
|
||||
}
|
||||
let name = path
|
||||
.file_stem()
|
||||
.and_then(|s| s.to_str())
|
||||
.unwrap_or("unknown")
|
||||
.to_string();
|
||||
out.push(Sample { obj_type, name, path });
|
||||
}
|
||||
}
|
||||
out.sort_by(|a, b| a.obj_type.cmp(&b.obj_type).then_with(|| a.name.cmp(&b.name)));
|
||||
out
|
||||
}
|
||||
|
||||
fn choose_iters_adaptive<F: FnMut()>(mut op: F, min_round_ms: u64, max_iters: u64) -> u64 {
|
||||
let min_secs = (min_round_ms as f64) / 1e3;
|
||||
let mut iters: u64 = 1;
|
||||
loop {
|
||||
let start = Instant::now();
|
||||
for _ in 0..iters {
|
||||
op();
|
||||
}
|
||||
let elapsed = start.elapsed().as_secs_f64();
|
||||
if elapsed >= min_secs {
|
||||
return iters;
|
||||
}
|
||||
if iters >= max_iters {
|
||||
return iters;
|
||||
}
|
||||
iters = (iters.saturating_mul(2)).min(max_iters);
|
||||
}
|
||||
}
|
||||
|
||||
fn count_ip(res: &IpResources) -> u64 {
|
||||
if res.is_inherited() {
|
||||
return 1;
|
||||
}
|
||||
let Ok(blocks) = res.to_blocks() else {
|
||||
return 0;
|
||||
};
|
||||
blocks.iter().count() as u64
|
||||
}
|
||||
|
||||
fn count_as(res: &AsResources) -> u64 {
|
||||
if res.is_inherited() {
|
||||
return 1;
|
||||
}
|
||||
let Ok(blocks) = res.to_blocks() else {
|
||||
return 0;
|
||||
};
|
||||
blocks.iter().count() as u64
|
||||
}
|
||||
|
||||
fn complexity(obj_type: ObjType, bytes: &[u8], strict: bool, cert_inspect: bool) -> u64 {
|
||||
match obj_type {
|
||||
ObjType::Cer => {
|
||||
let cert = Cert::decode(bytes).expect("decode cert");
|
||||
if cert_inspect {
|
||||
if cert.is_ca() {
|
||||
cert.inspect_ca(strict).expect("inspect ca");
|
||||
} else {
|
||||
cert.inspect_ee(strict).expect("inspect ee");
|
||||
}
|
||||
}
|
||||
count_ip(cert.v4_resources())
|
||||
.saturating_add(count_ip(cert.v6_resources()))
|
||||
.saturating_add(count_as(cert.as_resources()))
|
||||
}
|
||||
ObjType::Crl => {
|
||||
let crl = Crl::decode(bytes).expect("decode crl");
|
||||
crl.revoked_certs().iter().count() as u64
|
||||
}
|
||||
ObjType::Manifest => {
|
||||
let mft = Manifest::decode(bytes, strict).expect("decode manifest");
|
||||
if cert_inspect {
|
||||
mft.cert().inspect_ee(strict).expect("inspect ee");
|
||||
}
|
||||
mft.content().len() as u64
|
||||
}
|
||||
ObjType::Roa => {
|
||||
let roa = Roa::decode(bytes, strict).expect("decode roa");
|
||||
if cert_inspect {
|
||||
roa.cert().inspect_ee(strict).expect("inspect ee");
|
||||
}
|
||||
roa.content().iter().count() as u64
|
||||
}
|
||||
ObjType::Aspa => {
|
||||
let asa = Aspa::decode(bytes, strict).expect("decode aspa");
|
||||
if cert_inspect {
|
||||
asa.cert().inspect_ee(strict).expect("inspect ee");
|
||||
}
|
||||
asa.content().provider_as_set().len() as u64
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
fn decode_profile(obj_type: ObjType, bytes: &[u8], strict: bool, cert_inspect: bool) {
|
||||
match obj_type {
|
||||
ObjType::Cer => {
|
||||
let cert = Cert::decode(black_box(bytes)).expect("decode cert");
|
||||
if cert_inspect {
|
||||
if cert.is_ca() {
|
||||
cert.inspect_ca(strict).expect("inspect ca");
|
||||
} else {
|
||||
cert.inspect_ee(strict).expect("inspect ee");
|
||||
}
|
||||
}
|
||||
black_box(cert);
|
||||
}
|
||||
ObjType::Crl => {
|
||||
let crl = Crl::decode(black_box(bytes)).expect("decode crl");
|
||||
black_box(crl);
|
||||
}
|
||||
ObjType::Manifest => {
|
||||
let mft = Manifest::decode(black_box(bytes), strict).expect("decode manifest");
|
||||
if cert_inspect {
|
||||
mft.cert().inspect_ee(strict).expect("inspect ee");
|
||||
}
|
||||
black_box(mft);
|
||||
}
|
||||
ObjType::Roa => {
|
||||
let roa = Roa::decode(black_box(bytes), strict).expect("decode roa");
|
||||
if cert_inspect {
|
||||
roa.cert().inspect_ee(strict).expect("inspect ee");
|
||||
}
|
||||
black_box(roa);
|
||||
}
|
||||
ObjType::Aspa => {
|
||||
let asa = Aspa::decode(black_box(bytes), strict).expect("decode aspa");
|
||||
if cert_inspect {
|
||||
asa.cert().inspect_ee(strict).expect("inspect ee");
|
||||
}
|
||||
black_box(asa);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
#[derive(Clone, Debug)]
|
||||
struct ResultRow {
|
||||
obj_type: String,
|
||||
sample: String,
|
||||
size_bytes: usize,
|
||||
complexity: u64,
|
||||
avg_ns_per_op: f64,
|
||||
ops_per_sec: f64,
|
||||
}
|
||||
|
||||
fn render_markdown(title: &str, rows: &[ResultRow]) -> String {
|
||||
let mut out = String::new();
|
||||
out.push_str(&format!("# {title}\n\n"));
|
||||
out.push_str("| type | sample | size_bytes | complexity | avg ns/op | ops/s |\n");
|
||||
out.push_str("|---|---|---:|---:|---:|---:|\n");
|
||||
for r in rows {
|
||||
out.push_str(&format!(
|
||||
"| {} | {} | {} | {} | {:.2} | {:.2} |\n",
|
||||
r.obj_type, r.sample, r.size_bytes, r.complexity, r.avg_ns_per_op, r.ops_per_sec
|
||||
));
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn render_csv(rows: &[ResultRow]) -> String {
|
||||
let mut out = String::new();
|
||||
out.push_str("type,sample,size_bytes,complexity,avg_ns_per_op,ops_per_sec\n");
|
||||
for r in rows {
|
||||
let sample = r.sample.replace('"', "\"\"");
|
||||
out.push_str(&format!(
|
||||
"{},{},{},{},{:.6},{:.6}\n",
|
||||
r.obj_type,
|
||||
format!("\"{}\"", sample),
|
||||
r.size_bytes,
|
||||
r.complexity,
|
||||
r.avg_ns_per_op,
|
||||
r.ops_per_sec
|
||||
));
|
||||
}
|
||||
out
|
||||
}
|
||||
|
||||
fn create_parent_dirs(path: &Path) {
|
||||
if let Some(parent) = path.parent() {
|
||||
std::fs::create_dir_all(parent).unwrap_or_else(|e| {
|
||||
panic!("create_dir_all {}: {e}", parent.display());
|
||||
});
|
||||
}
|
||||
}
|
||||
|
||||
fn write_text_file(path: &Path, content: &str) {
|
||||
create_parent_dirs(path);
|
||||
std::fs::write(path, content).unwrap_or_else(|e| panic!("write {}: {e}", path.display()));
|
||||
}
|
||||
|
||||
fn main() {
|
||||
let cfg = parse_args();
|
||||
let mut samples = read_samples(&cfg.dir);
|
||||
if samples.is_empty() {
|
||||
usage_and_exit(Some(&format!(
|
||||
"no samples found under: {}",
|
||||
cfg.dir.display()
|
||||
)));
|
||||
}
|
||||
|
||||
if let Some(t) = cfg.type_filter {
|
||||
samples.retain(|s| s.obj_type == t);
|
||||
if samples.is_empty() {
|
||||
usage_and_exit(Some(&format!("no sample matched --type {}", t.as_str())));
|
||||
}
|
||||
}
|
||||
if let Some(filter) = cfg.sample_filter.as_deref() {
|
||||
samples.retain(|s| s.name == filter);
|
||||
if samples.is_empty() {
|
||||
usage_and_exit(Some(&format!("no sample matched --sample {filter}")));
|
||||
}
|
||||
}
|
||||
|
||||
println!("# Routinator baseline (rpki crate) decode benchmark (selected_der_v2)");
|
||||
println!();
|
||||
println!("- dir: {}", cfg.dir.display());
|
||||
println!("- strict: {}", cfg.strict);
|
||||
println!("- cert_inspect: {}", cfg.cert_inspect);
|
||||
if let Some(t) = cfg.type_filter {
|
||||
println!("- type: {}", t.as_str());
|
||||
}
|
||||
if let Some(s) = cfg.sample_filter.as_deref() {
|
||||
println!("- sample: {}", s);
|
||||
}
|
||||
if let Some(n) = cfg.fixed_iters {
|
||||
println!("- iters: {} (fixed)", n);
|
||||
} else {
|
||||
println!(
|
||||
"- warmup: {} iters, rounds: {}, min_round: {}ms (adaptive iters, max {})",
|
||||
cfg.warmup_iters, cfg.rounds, cfg.min_round_ms, cfg.max_adaptive_iters
|
||||
);
|
||||
}
|
||||
if let Some(p) = cfg.out_csv.as_ref() {
|
||||
println!("- out_csv: {}", p.display());
|
||||
}
|
||||
if let Some(p) = cfg.out_md.as_ref() {
|
||||
println!("- out_md: {}", p.display());
|
||||
}
|
||||
println!();
|
||||
|
||||
println!("| type | sample | size_bytes | complexity | avg ns/op | ops/s |");
|
||||
println!("|---|---|---:|---:|---:|---:|");
|
||||
|
||||
let mut rows: Vec<ResultRow> = Vec::with_capacity(samples.len());
|
||||
for sample in &samples {
|
||||
let bytes = std::fs::read(&sample.path)
|
||||
.unwrap_or_else(|e| panic!("read {}: {e}", sample.path.display()));
|
||||
let size_bytes = bytes.len();
|
||||
let complexity = complexity(sample.obj_type, bytes.as_slice(), cfg.strict, cfg.cert_inspect);
|
||||
|
||||
for _ in 0..cfg.warmup_iters {
|
||||
decode_profile(sample.obj_type, bytes.as_slice(), cfg.strict, cfg.cert_inspect);
|
||||
}
|
||||
|
||||
let mut per_round_ns_per_op = Vec::with_capacity(cfg.rounds as usize);
|
||||
for _round in 0..cfg.rounds {
|
||||
let iters = if let Some(n) = cfg.fixed_iters {
|
||||
n
|
||||
} else {
|
||||
choose_iters_adaptive(
|
||||
|| decode_profile(sample.obj_type, bytes.as_slice(), cfg.strict, cfg.cert_inspect),
|
||||
cfg.min_round_ms,
|
||||
cfg.max_adaptive_iters,
|
||||
)
|
||||
};
|
||||
let start = Instant::now();
|
||||
for _ in 0..iters {
|
||||
decode_profile(sample.obj_type, bytes.as_slice(), cfg.strict, cfg.cert_inspect);
|
||||
}
|
||||
let elapsed = start.elapsed();
|
||||
let total_ns = elapsed.as_secs_f64() * 1e9;
|
||||
per_round_ns_per_op.push(total_ns / (iters as f64));
|
||||
}
|
||||
|
||||
let avg_ns = per_round_ns_per_op.iter().sum::<f64>() / (per_round_ns_per_op.len() as f64);
|
||||
let ops_per_sec = 1e9_f64 / avg_ns;
|
||||
|
||||
println!(
|
||||
"| {} | {} | {} | {} | {:.2} | {:.2} |",
|
||||
sample.obj_type.as_str(),
|
||||
sample.name,
|
||||
size_bytes,
|
||||
complexity,
|
||||
avg_ns,
|
||||
ops_per_sec
|
||||
);
|
||||
|
||||
rows.push(ResultRow {
|
||||
obj_type: sample.obj_type.as_str().to_string(),
|
||||
sample: sample.name.clone(),
|
||||
size_bytes,
|
||||
complexity,
|
||||
avg_ns_per_op: avg_ns,
|
||||
ops_per_sec,
|
||||
});
|
||||
}
|
||||
|
||||
if let Some(path) = cfg.out_md.as_ref() {
|
||||
let md = render_markdown(
|
||||
"Routinator baseline (rpki crate) decode+inspect (selected_der_v2)",
|
||||
&rows,
|
||||
);
|
||||
write_text_file(path, &md);
|
||||
eprintln!("Wrote {}", path.display());
|
||||
}
|
||||
if let Some(path) = cfg.out_csv.as_ref() {
|
||||
let csv = render_csv(&rows);
|
||||
write_text_file(path, &csv);
|
||||
eprintln!("Wrote {}", path.display());
|
||||
}
|
||||
}
|
||||
@@ -1,23 +0,0 @@
|
||||
{
|
||||
"slurmVersion": 2,
|
||||
"validationOutputFilters": {
|
||||
"prefixFilters": [
|
||||
{
|
||||
"prefix": "24.0.0.0/8",
|
||||
"comment": "Filter many VRPs in current CCR sample"
|
||||
}
|
||||
],
|
||||
"bgpsecFilters": [],
|
||||
"aspaFilters": [
|
||||
{
|
||||
"customerAsn": 80,
|
||||
"comment": "Filter one ASPA known to exist in current CCR sample"
|
||||
}
|
||||
]
|
||||
},
|
||||
"locallyAddedAssertions": {
|
||||
"prefixAssertions": [],
|
||||
"bgpsecAssertions": [],
|
||||
"aspaAssertions": []
|
||||
}
|
||||
}
|
||||
@@ -1,33 +0,0 @@
|
||||
# Build-time image knob.
|
||||
RPKI_BIRD_VERSION=3.2.1
|
||||
|
||||
# TCP mode target endpoint.
|
||||
RPKI_BIRD_RPKI_HOST=rpki-rtr
|
||||
RPKI_BIRD_RPKI_PORT=323
|
||||
|
||||
# SSH mode target endpoint.
|
||||
RPKI_BIRD_SSH_RPKI_HOST=rpki-rtr
|
||||
RPKI_RTR_SSH_PORT=22
|
||||
|
||||
# Config template paths in container.
|
||||
RPKI_BIRD_CONFIG_TEMPLATE_PATH=/config/bird.conf.template
|
||||
RPKI_BIRD_SSH_CONFIG_TEMPLATE_PATH=/config/bird.conf.ssh.template
|
||||
|
||||
# Observation and output knobs.
|
||||
RPKI_BIRD_OBSERVE_PROTO=rpki_tcp
|
||||
RPKI_BIRD_OBSERVE_MODE=interval
|
||||
RPKI_BIRD_OBSERVE_DEBOUNCE_SECS=1
|
||||
RPKI_BIRD_OBSERVE_INTERVAL=30
|
||||
RPKI_BIRD_OBSERVE_ASPA_TABLE=rtr_aspa
|
||||
RPKI_BIRD_OBSERVE_ROA4_TABLE=rtr_roa_v4
|
||||
RPKI_BIRD_OBSERVE_ROA6_TABLE=rtr_roa_v6
|
||||
RPKI_BIRD_OBSERVE_ASPA_COUNT=3
|
||||
RPKI_BIRD_OBSERVE_ROA4_COUNT=3
|
||||
RPKI_BIRD_OBSERVE_ROA6_COUNT=3
|
||||
RPKI_BIRD_SHOW_ASPA=1
|
||||
RPKI_BIRD_SHOW_ROA4=1
|
||||
RPKI_BIRD_SHOW_ROA6=1
|
||||
|
||||
# Host volume mounts.
|
||||
RPKI_BIRD_LOG_HOST_DIR=../../logs/bird
|
||||
RPKI_BIRD_SSH_CERTS_HOST_DIR=../../certs/ssh
|
||||
@@ -1,56 +0,0 @@
|
||||
FROM debian:bookworm-slim AS builder
|
||||
|
||||
ARG BIRD_VERSION=3.2.1
|
||||
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
build-essential \
|
||||
bison \
|
||||
flex \
|
||||
m4 \
|
||||
perl \
|
||||
ca-certificates \
|
||||
wget \
|
||||
xz-utils \
|
||||
libreadline-dev \
|
||||
libncurses-dev \
|
||||
libssh-dev \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /build
|
||||
|
||||
RUN wget -O bird.tar.gz "https://bird.nic.cz/download/bird-${BIRD_VERSION}.tar.gz" \
|
||||
&& tar -xzf bird.tar.gz \
|
||||
&& mv "bird-${BIRD_VERSION}" bird
|
||||
|
||||
WORKDIR /build/bird
|
||||
|
||||
RUN ./configure \
|
||||
--prefix=/usr \
|
||||
--sysconfdir=/etc/bird \
|
||||
--localstatedir=/run \
|
||||
&& make -j"$(nproc)" \
|
||||
&& make install
|
||||
|
||||
FROM debian:bookworm-slim
|
||||
|
||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||
ca-certificates \
|
||||
gettext-base \
|
||||
tzdata \
|
||||
netcat-openbsd \
|
||||
libreadline8 \
|
||||
libncurses6 \
|
||||
libtinfo6 \
|
||||
libssh-4 \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY --from=builder /usr/sbin/bird /usr/sbin/bird
|
||||
COPY --from=builder /usr/sbin/birdc /usr/sbin/birdc
|
||||
COPY --from=builder /etc/bird /etc/bird
|
||||
ENV TZ=Asia/Shanghai
|
||||
|
||||
COPY entrypoint.sh /entrypoint.sh
|
||||
RUN chmod +x /entrypoint.sh \
|
||||
&& mkdir -p /run/bird
|
||||
|
||||
ENTRYPOINT ["/entrypoint.sh"]
|
||||
@@ -1,90 +0,0 @@
|
||||
# BIRD Minimal RTR Client Config
|
||||
|
||||
This folder provides a minimal BIRD setup for black-box interop testing
|
||||
against this repository's RTR server defaults.
|
||||
|
||||
Server defaults in this repo:
|
||||
- TCP: `0.0.0.0:323`
|
||||
- SSH: `0.0.0.0:22` (when enabled on server)
|
||||
|
||||
## Files
|
||||
|
||||
- `Dockerfile`: builds a minimal BIRD2 runtime image.
|
||||
- `bird.conf.example`: sample `/etc/bird/bird.conf`.
|
||||
- `bird.conf.ssh.example`: sample SSH transport `/etc/bird/bird.conf`.
|
||||
- `entrypoint.sh`: starts BIRD in foreground mode.
|
||||
- `docker-compose.yml`: one-click local TCP test client.
|
||||
- `docker-compose.ssh.yml`: compose override for SSH transport.
|
||||
|
||||
By default, the container uses event-driven observation and prints snapshots
|
||||
only when BIRD reports RPKI-related changes.
|
||||
|
||||
## Docker quick start
|
||||
|
||||
From repository root:
|
||||
|
||||
```bash
|
||||
docker compose -f deploy/bird/docker-compose.yml up --build
|
||||
```
|
||||
|
||||
Use another terminal to inspect:
|
||||
|
||||
```bash
|
||||
docker logs -f bird-rpki-client
|
||||
```
|
||||
|
||||
If protocol state is `up`, the RTR client path is working.
|
||||
|
||||
Detached mode:
|
||||
|
||||
```bash
|
||||
docker compose -f deploy/bird/docker-compose.yml up -d --build
|
||||
docker logs -f bird-rpki-client
|
||||
```
|
||||
|
||||
Stop:
|
||||
|
||||
```bash
|
||||
docker compose -f deploy/bird/docker-compose.yml down
|
||||
```
|
||||
|
||||
## SSH quick start
|
||||
|
||||
Start server in SSH mode first:
|
||||
|
||||
```bash
|
||||
docker compose -f deploy/server/docker-compose.ssh.yml up -d --build
|
||||
```
|
||||
|
||||
Start BIRD client with SSH override:
|
||||
|
||||
```bash
|
||||
docker compose \
|
||||
-f deploy/bird/docker-compose.yml \
|
||||
-f deploy/bird/docker-compose.ssh.yml \
|
||||
up --build
|
||||
```
|
||||
|
||||
In detached mode, observe with:
|
||||
|
||||
```bash
|
||||
docker logs -f bird-rpki-client
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- This setup renders `bird.conf.template` at startup and uses `RPKI_HOST` / `RPKI_PORT`
|
||||
for the RTR endpoint.
|
||||
- `network_mode: host` expects your RTR server to be reachable at
|
||||
`host.docker.internal:323` from the container.
|
||||
- Observation is controlled by env vars:
|
||||
`OBSERVE_MODE` (`event` by default, `interval` as fallback),
|
||||
`OBSERVE_DEBOUNCE_SECS` (default `1`),
|
||||
`OBSERVE_INTERVAL` (seconds, used when `OBSERVE_MODE=interval`),
|
||||
and `OBSERVE_PROTO`.
|
||||
- SSH mode mounts `../../certs/ssh` into `/config/ssh` and expects:
|
||||
`rtr-client.key` and `ssh_host_rsa_key.pub`.
|
||||
- Entrypoint auto-generates `/run/bird/known_hosts` from
|
||||
`/config/ssh/ssh_host_rsa_key.pub` for BIRD SSH host-key verification.
|
||||
- If your environment does not support Docker host networking, switch to a
|
||||
bridge network and replace `remote` addresses accordingly.
|
||||
@@ -1,80 +0,0 @@
|
||||
# BIRD 最小化 RTR 客户端配置
|
||||
|
||||
本目录提供最小化 BIRD 配置,用于与本仓库 RTR Server 做黑盒互通测试。
|
||||
|
||||
本仓库默认 RTR 监听地址:
|
||||
- TCP: `0.0.0.0:323`
|
||||
- SSH: `0.0.0.0:22`(仅在 server SSH 模式启用时)
|
||||
|
||||
## 文件说明
|
||||
|
||||
- `Dockerfile`: 构建最小 BIRD 运行镜像(包含 SSH transport 支持)。
|
||||
- `bird.conf.example`: `/etc/bird/bird.conf` 的 TCP 示例。
|
||||
- `bird.conf.ssh.example`: `/etc/bird/bird.conf` 的 SSH transport 示例。
|
||||
- `entrypoint.sh`: 前台启动 BIRD,并周期输出协议快照。
|
||||
- `docker-compose.yml`: TCP 一键启动。
|
||||
- `docker-compose.ssh.yml`: SSH 覆盖配置。
|
||||
|
||||
容器默认每 30 秒向日志输出一次 RPKI 协议状态快照。
|
||||
|
||||
## Docker 快速启动(TCP)
|
||||
|
||||
在仓库根目录执行:
|
||||
|
||||
```bash
|
||||
docker compose -f deploy/bird/docker-compose.yml up --build
|
||||
```
|
||||
|
||||
另开终端查看日志:
|
||||
|
||||
```bash
|
||||
docker logs -f bird-rpki-client
|
||||
```
|
||||
|
||||
如果协议状态显示 `up`,说明 RTR 客户端链路正常。
|
||||
|
||||
后台模式:
|
||||
|
||||
```bash
|
||||
docker compose -f deploy/bird/docker-compose.yml up -d --build
|
||||
docker logs -f bird-rpki-client
|
||||
```
|
||||
|
||||
停止:
|
||||
|
||||
```bash
|
||||
docker compose -f deploy/bird/docker-compose.yml down
|
||||
```
|
||||
|
||||
## SSH 快速启动
|
||||
|
||||
先启动 server 的 SSH 模式:
|
||||
|
||||
```bash
|
||||
docker compose -f deploy/server/docker-compose.ssh.yml up -d --build
|
||||
```
|
||||
|
||||
再启动 BIRD SSH 客户端:
|
||||
|
||||
```bash
|
||||
docker compose \
|
||||
-f deploy/bird/docker-compose.yml \
|
||||
-f deploy/bird/docker-compose.ssh.yml \
|
||||
up --build
|
||||
```
|
||||
|
||||
查看日志:
|
||||
|
||||
```bash
|
||||
docker logs -f bird-rpki-client
|
||||
```
|
||||
|
||||
## 说明
|
||||
|
||||
- 当前 compose 使用 `network_mode: host`,默认通过 `host.docker.internal` 访问 server。
|
||||
- 观测频率由环境变量控制:`OBSERVE_INTERVAL`(秒,默认 `30`)和 `OBSERVE_PROTO`。
|
||||
- SSH 模式会将 `../../certs/ssh` 挂载到容器 `/config/ssh`,并使用:
|
||||
`rtr-client.key`、`ssh_host_rsa_key.pub`。
|
||||
- 入口脚本会基于 `/config/ssh/ssh_host_rsa_key.pub` 自动生成
|
||||
`/run/bird/known_hosts`,用于 BIRD 的 SSH 主机密钥校验。
|
||||
- 如果你运行在 Docker Desktop(非 Linux 原生 host network 场景),建议改为自定义 bridge 网络并将 `remote` 地址改成可达的 server 容器名或宿主机地址。
|
||||
@@ -1,31 +0,0 @@
|
||||
log stderr all;
|
||||
debug protocols all;
|
||||
router id 192.0.2.2;
|
||||
|
||||
roa4 table rtr_roa_v4;
|
||||
roa6 table rtr_roa_v6;
|
||||
aspa table rtr_aspa;
|
||||
|
||||
protocol device {
|
||||
}
|
||||
|
||||
protocol rpki rpki_ssh {
|
||||
roa4 { table rtr_roa_v4; };
|
||||
roa6 { table rtr_roa_v6; };
|
||||
aspa { table rtr_aspa; };
|
||||
|
||||
remote "host.docker.internal" port 22;
|
||||
|
||||
min version 2;
|
||||
max version 2;
|
||||
|
||||
refresh 3600;
|
||||
retry 600;
|
||||
expire 7200;
|
||||
|
||||
transport ssh {
|
||||
user "rpki-rtr";
|
||||
bird private key "/config/ssh/rtr-client.key";
|
||||
remote public key "/run/bird/known_hosts";
|
||||
};
|
||||
}
|
||||
@@ -1,28 +0,0 @@
|
||||
log stderr all;
|
||||
router id 192.0.2.2;
|
||||
|
||||
roa4 table rtr_roa_v4;
|
||||
roa6 table rtr_roa_v6;
|
||||
aspa table rtr_aspa;
|
||||
|
||||
protocol device {
|
||||
}
|
||||
|
||||
protocol rpki rpki_tcp {
|
||||
roa4 { table rtr_roa_v4; };
|
||||
roa6 { table rtr_roa_v6; };
|
||||
aspa { table rtr_aspa; };
|
||||
|
||||
remote "host.docker.internal" port 323;
|
||||
|
||||
min version 2;
|
||||
max version 2;
|
||||
|
||||
refresh 3600;
|
||||
retry 600;
|
||||
expire 7200;
|
||||
|
||||
transport tcp {
|
||||
authentication none;
|
||||
};
|
||||
}
|
||||
@@ -1,28 +0,0 @@
|
||||
log stderr all;
|
||||
router id 192.0.2.2;
|
||||
|
||||
roa4 table rtr_roa_v4;
|
||||
roa6 table rtr_roa_v6;
|
||||
aspa table rtr_aspa;
|
||||
|
||||
protocol device {
|
||||
}
|
||||
|
||||
protocol rpki rpki_tcp {
|
||||
roa4 { table rtr_roa_v4; };
|
||||
roa6 { table rtr_roa_v6; };
|
||||
aspa { table rtr_aspa; };
|
||||
|
||||
remote "host.docker.internal" port 323;
|
||||
|
||||
min version 2;
|
||||
max version 2;
|
||||
|
||||
refresh 3600;
|
||||
retry 600;
|
||||
expire 7200;
|
||||
|
||||
transport tcp {
|
||||
authentication none;
|
||||
};
|
||||
}
|
||||
@@ -1,30 +0,0 @@
|
||||
log stderr all;
|
||||
router id 192.0.2.2;
|
||||
|
||||
roa4 table rtr_roa_v4;
|
||||
roa6 table rtr_roa_v6;
|
||||
aspa table rtr_aspa;
|
||||
|
||||
protocol device {
|
||||
}
|
||||
|
||||
protocol rpki rpki_ssh {
|
||||
roa4 { table rtr_roa_v4; };
|
||||
roa6 { table rtr_roa_v6; };
|
||||
aspa { table rtr_aspa; };
|
||||
|
||||
remote "host.docker.internal" port 22;
|
||||
|
||||
min version 2;
|
||||
max version 2;
|
||||
|
||||
refresh 3600;
|
||||
retry 600;
|
||||
expire 7200;
|
||||
|
||||
transport ssh {
|
||||
user "rpki-rtr";
|
||||
bird private key "/config/ssh/rtr-client.key";
|
||||
remote public key "/run/bird/known_hosts";
|
||||
};
|
||||
}
|
||||
@@ -1,30 +0,0 @@
|
||||
log stderr all;
|
||||
router id 192.0.2.2;
|
||||
|
||||
roa4 table rtr_roa_v4;
|
||||
roa6 table rtr_roa_v6;
|
||||
aspa table rtr_aspa;
|
||||
|
||||
protocol device {
|
||||
}
|
||||
|
||||
protocol rpki rpki_ssh {
|
||||
roa4 { table rtr_roa_v4; };
|
||||
roa6 { table rtr_roa_v6; };
|
||||
aspa { table rtr_aspa; };
|
||||
|
||||
remote "${RPKI_HOST}" port ${RPKI_PORT};
|
||||
|
||||
min version 2;
|
||||
max version 2;
|
||||
|
||||
refresh 3600;
|
||||
retry 600;
|
||||
expire 7200;
|
||||
|
||||
transport ssh {
|
||||
user "rpki-rtr";
|
||||
bird private key "/config/ssh/rtr-client.key";
|
||||
remote public key "/run/bird/known_hosts";
|
||||
};
|
||||
}
|
||||
@@ -1,28 +0,0 @@
|
||||
log stderr all;
|
||||
router id 192.0.2.2;
|
||||
|
||||
roa4 table rtr_roa_v4;
|
||||
roa6 table rtr_roa_v6;
|
||||
aspa table rtr_aspa;
|
||||
|
||||
protocol device {
|
||||
}
|
||||
|
||||
protocol rpki rpki_tcp {
|
||||
roa4 { table rtr_roa_v4; };
|
||||
roa6 { table rtr_roa_v6; };
|
||||
aspa { table rtr_aspa; };
|
||||
|
||||
remote "${RPKI_HOST}" port ${RPKI_PORT};
|
||||
|
||||
min version 2;
|
||||
max version 2;
|
||||
|
||||
refresh 3600;
|
||||
retry 600;
|
||||
expire 7200;
|
||||
|
||||
transport tcp {
|
||||
authentication none;
|
||||
};
|
||||
}
|
||||
@@ -1,17 +0,0 @@
|
||||
services:
|
||||
bird-rpki-client:
|
||||
environment:
|
||||
BIRD_CONFIG_TEMPLATE_PATH: "${RPKI_BIRD_SSH_CONFIG_TEMPLATE_PATH:-/config/bird.conf.ssh.template}"
|
||||
RPKI_HOST: "${RPKI_BIRD_SSH_RPKI_HOST:-rpki-rtr}"
|
||||
RPKI_PORT: "${RPKI_RTR_SSH_PORT:-22}"
|
||||
OBSERVE_PROTO: "rpki_ssh"
|
||||
volumes:
|
||||
- ./bird.conf.ssh.template:/config/bird.conf.ssh.template:ro
|
||||
- ${RPKI_BIRD_SSH_CERTS_HOST_DIR:-../../certs/ssh}:/config/ssh:ro
|
||||
networks:
|
||||
- rpki_net
|
||||
|
||||
networks:
|
||||
rpki_net:
|
||||
name: rpki_net
|
||||
driver: bridge
|
||||
@@ -1,41 +0,0 @@
|
||||
services:
|
||||
bird-rpki-client:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
args:
|
||||
BIRD_VERSION: "${RPKI_BIRD_VERSION:-3.2.1}"
|
||||
container_name: bird-rpki-client
|
||||
restart: unless-stopped
|
||||
environment:
|
||||
BIRD_CONFIG_TEMPLATE_PATH: "${RPKI_BIRD_CONFIG_TEMPLATE_PATH:-/config/bird.conf.template}"
|
||||
|
||||
RPKI_HOST: "${RPKI_BIRD_RPKI_HOST:-rpki-rtr}"
|
||||
RPKI_PORT: "${RPKI_BIRD_RPKI_PORT:-323}"
|
||||
|
||||
OBSERVE_PROTO: "${RPKI_BIRD_OBSERVE_PROTO:-rpki_tcp}"
|
||||
OBSERVE_MODE: "${RPKI_BIRD_OBSERVE_MODE:-interval}"
|
||||
OBSERVE_DEBOUNCE_SECS: "${RPKI_BIRD_OBSERVE_DEBOUNCE_SECS:-1}"
|
||||
OBSERVE_INTERVAL: "${RPKI_BIRD_OBSERVE_INTERVAL:-30}"
|
||||
|
||||
OBSERVE_ASPA_TABLE: "${RPKI_BIRD_OBSERVE_ASPA_TABLE:-rtr_aspa}"
|
||||
OBSERVE_ROA4_TABLE: "${RPKI_BIRD_OBSERVE_ROA4_TABLE:-rtr_roa_v4}"
|
||||
OBSERVE_ROA6_TABLE: "${RPKI_BIRD_OBSERVE_ROA6_TABLE:-rtr_roa_v6}"
|
||||
|
||||
OBSERVE_ASPA_COUNT: "${RPKI_BIRD_OBSERVE_ASPA_COUNT:-3}"
|
||||
OBSERVE_ROA4_COUNT: "${RPKI_BIRD_OBSERVE_ROA4_COUNT:-3}"
|
||||
OBSERVE_ROA6_COUNT: "${RPKI_BIRD_OBSERVE_ROA6_COUNT:-3}"
|
||||
|
||||
SHOW_ASPA: "${RPKI_BIRD_SHOW_ASPA:-1}"
|
||||
SHOW_ROA4: "${RPKI_BIRD_SHOW_ROA4:-1}"
|
||||
SHOW_ROA6: "${RPKI_BIRD_SHOW_ROA6:-1}"
|
||||
volumes:
|
||||
- ./bird.conf.template:/config/bird.conf.template:ro
|
||||
- ${RPKI_BIRD_LOG_HOST_DIR:-../../logs/bird}:/app/logs
|
||||
networks:
|
||||
- rpki_net
|
||||
|
||||
networks:
|
||||
rpki_net:
|
||||
name: rpki_net
|
||||
driver: bridge
|
||||
@@ -1,218 +0,0 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
mkdir -p /run/bird
|
||||
|
||||
SOCK_PATH="/run/bird/bird.ctl"
|
||||
PROTO="${OBSERVE_PROTO:-rpki_tcp}"
|
||||
INTERVAL="${OBSERVE_INTERVAL:-30}"
|
||||
MODE="${OBSERVE_MODE:-event}"
|
||||
DEBOUNCE_SECS="${OBSERVE_DEBOUNCE_SECS:-1}"
|
||||
RPKI_HOST="${RPKI_HOST:-host.docker.internal}"
|
||||
RPKI_PORT="${RPKI_PORT:-323}"
|
||||
|
||||
BIRD_CONFIG_PATH="${BIRD_CONFIG_PATH:-/config/bird.conf}"
|
||||
BIRD_CONFIG_TEMPLATE_PATH="${BIRD_CONFIG_TEMPLATE_PATH:-/config/bird.conf.template}"
|
||||
|
||||
ASPA_TABLE="${OBSERVE_ASPA_TABLE:-rtr_aspa}"
|
||||
ROA4_TABLE="${OBSERVE_ROA4_TABLE:-rtr_roa_v4}"
|
||||
ROA6_TABLE="${OBSERVE_ROA6_TABLE:-rtr_roa_v6}"
|
||||
|
||||
ASPA_COUNT="${OBSERVE_ASPA_COUNT:-3}"
|
||||
ROA4_COUNT="${OBSERVE_ROA4_COUNT:-3}"
|
||||
ROA6_COUNT="${OBSERVE_ROA6_COUNT:-3}"
|
||||
|
||||
SHOW_ASPA="${SHOW_ASPA:-1}"
|
||||
SHOW_ROA4="${SHOW_ROA4:-1}"
|
||||
SHOW_ROA6="${SHOW_ROA6:-1}"
|
||||
SSH_HOST_PUBKEY_PATH="${SSH_HOST_PUBKEY_PATH:-/config/ssh/ssh_host_rsa_key.pub}"
|
||||
SSH_KNOWN_HOSTS_PATH="${SSH_KNOWN_HOSTS_PATH:-/run/bird/known_hosts}"
|
||||
|
||||
LOG_DIR="${LOG_DIR:-/app/logs}"
|
||||
LOG_NAME="${LOG_NAME:-${HOSTNAME:-bird-rpki-client}}"
|
||||
STDOUT_LOG="${LOG_DIR}/${LOG_NAME}.stdout.log"
|
||||
STDERR_LOG="${LOG_DIR}/${LOG_NAME}.stderr.log"
|
||||
|
||||
mkdir -p "$LOG_DIR"
|
||||
exec >>"$STDOUT_LOG" 2>>"$STDERR_LOG"
|
||||
|
||||
if [ -r "$BIRD_CONFIG_TEMPLATE_PATH" ]; then
|
||||
RENDERED_CONFIG_PATH="/run/bird/bird.generated.conf"
|
||||
envsubst '${RPKI_HOST} ${RPKI_PORT}' < "$BIRD_CONFIG_TEMPLATE_PATH" > "$RENDERED_CONFIG_PATH"
|
||||
BIRD_CONFIG_PATH="$RENDERED_CONFIG_PATH"
|
||||
echo "[entrypoint] rendered config : $BIRD_CONFIG_TEMPLATE_PATH -> $BIRD_CONFIG_PATH"
|
||||
fi
|
||||
|
||||
ensure_ssh_known_hosts() {
|
||||
if [ -s "$SSH_KNOWN_HOSTS_PATH" ]; then
|
||||
return
|
||||
fi
|
||||
|
||||
if [ ! -r "$SSH_HOST_PUBKEY_PATH" ]; then
|
||||
echo "[entrypoint] WARNING: SSH host key file not found: $SSH_HOST_PUBKEY_PATH"
|
||||
return
|
||||
fi
|
||||
|
||||
set -- $(awk 'NF >= 2 { print $1, $2; exit }' "$SSH_HOST_PUBKEY_PATH")
|
||||
if [ $# -ne 2 ]; then
|
||||
echo "[entrypoint] WARNING: invalid SSH host key format in $SSH_HOST_PUBKEY_PATH"
|
||||
return
|
||||
fi
|
||||
|
||||
key_type="$1"
|
||||
key_data="$2"
|
||||
|
||||
if echo "$key_type" | grep -q '^ssh-'; then
|
||||
{
|
||||
echo "$RPKI_HOST $key_type $key_data"
|
||||
echo "[$RPKI_HOST]:$RPKI_PORT $key_type $key_data"
|
||||
} > "$SSH_KNOWN_HOSTS_PATH"
|
||||
else
|
||||
cp "$SSH_HOST_PUBKEY_PATH" "$SSH_KNOWN_HOSTS_PATH"
|
||||
fi
|
||||
|
||||
chmod 600 "$SSH_KNOWN_HOSTS_PATH" || true
|
||||
echo "[entrypoint] generated known_hosts: $SSH_KNOWN_HOSTS_PATH"
|
||||
}
|
||||
|
||||
print_first_n_objects() {
|
||||
table_name="$1"
|
||||
max_objects="$2"
|
||||
|
||||
birdc -s "$SOCK_PATH" show route table "$table_name" all 2>/dev/null | awk -v max="$max_objects" '
|
||||
BEGIN {
|
||||
count = 0
|
||||
}
|
||||
|
||||
# 直接跳过空行
|
||||
/^[[:space:]]*$/ {
|
||||
next
|
||||
}
|
||||
|
||||
# 保留 birdc 的表头
|
||||
/^BIRD / {
|
||||
print
|
||||
next
|
||||
}
|
||||
|
||||
/^Table / {
|
||||
print
|
||||
next
|
||||
}
|
||||
|
||||
# 非缩进且不是表头,视为一个新对象的开始
|
||||
/^[^[:space:]]/ {
|
||||
count++
|
||||
if (count > max) {
|
||||
exit
|
||||
}
|
||||
print
|
||||
next
|
||||
}
|
||||
|
||||
# 缩进内容,只有在已进入前 max 个对象时才打印
|
||||
{
|
||||
if (count > 0 && count <= max) {
|
||||
print
|
||||
}
|
||||
}
|
||||
' || true
|
||||
}
|
||||
|
||||
print_snapshot() {
|
||||
echo "==== $(date +"%Y-%m-%dT%H:%M:%S%:z") RPKI snapshot ($PROTO) ===="
|
||||
birdc -s "$SOCK_PATH" show protocols all "$PROTO" || true
|
||||
|
||||
if [ "$SHOW_ASPA" = "1" ]; then
|
||||
echo "---- ASPA table ($ASPA_TABLE, first ${ASPA_COUNT} objects) ----"
|
||||
print_first_n_objects "$ASPA_TABLE" "$ASPA_COUNT"
|
||||
fi
|
||||
|
||||
if [ "$SHOW_ROA4" = "1" ]; then
|
||||
echo "---- ROA4 table ($ROA4_TABLE, first ${ROA4_COUNT} objects) ----"
|
||||
print_first_n_objects "$ROA4_TABLE" "$ROA4_COUNT"
|
||||
fi
|
||||
|
||||
if [ "$SHOW_ROA6" = "1" ]; then
|
||||
echo "---- ROA6 table ($ROA6_TABLE, first ${ROA6_COUNT} objects) ----"
|
||||
print_first_n_objects "$ROA6_TABLE" "$ROA6_COUNT"
|
||||
fi
|
||||
}
|
||||
|
||||
is_rpki_related_event() {
|
||||
line="$1"
|
||||
echo "$line" | grep -Eiq "$PROTO|$ASPA_TABLE|$ROA4_TABLE|$ROA6_TABLE|rpki|rtr"
|
||||
}
|
||||
|
||||
echo "[entrypoint] starting bird"
|
||||
echo "[entrypoint] config : $BIRD_CONFIG_PATH"
|
||||
echo "[entrypoint] observe proto : $PROTO"
|
||||
echo "[entrypoint] observe mode : $MODE"
|
||||
echo "[entrypoint] observe interval : $INTERVAL (used when mode=interval)"
|
||||
echo "[entrypoint] debounce secs : $DEBOUNCE_SECS (used when mode=event)"
|
||||
echo "[entrypoint] target : $RPKI_HOST:$RPKI_PORT"
|
||||
echo "[entrypoint] show aspa : $SHOW_ASPA ($ASPA_TABLE, first $ASPA_COUNT objects)"
|
||||
echo "[entrypoint] show roa4 : $SHOW_ROA4 ($ROA4_TABLE, first $ROA4_COUNT objects)"
|
||||
echo "[entrypoint] show roa6 : $SHOW_ROA6 ($ROA6_TABLE, first $ROA6_COUNT objects)"
|
||||
|
||||
if [ "$PROTO" = "rpki_ssh" ] || grep -q 'transport[[:space:]]\+ssh' "$BIRD_CONFIG_PATH"; then
|
||||
ensure_ssh_known_hosts
|
||||
fi
|
||||
|
||||
if nc -zvw3 "$RPKI_HOST" "$RPKI_PORT"; then
|
||||
echo "[entrypoint] TCP connectivity to $RPKI_HOST:$RPKI_PORT OK"
|
||||
else
|
||||
echo "[entrypoint] WARNING: cannot connect to $RPKI_HOST:$RPKI_PORT before BIRD starts"
|
||||
fi
|
||||
|
||||
bird -f -c "$BIRD_CONFIG_PATH" -s "$SOCK_PATH" &
|
||||
BIRD_PID="$!"
|
||||
|
||||
sleep 1
|
||||
|
||||
case "$INTERVAL" in
|
||||
''|*[!0-9]*)
|
||||
INTERVAL=0
|
||||
;;
|
||||
esac
|
||||
|
||||
case "$MODE" in
|
||||
event)
|
||||
case "$DEBOUNCE_SECS" in
|
||||
''|*[!0-9]*)
|
||||
DEBOUNCE_SECS=1
|
||||
;;
|
||||
esac
|
||||
|
||||
echo "[entrypoint] waiting for BIRD monitor events and only printing on RPKI-related changes"
|
||||
LAST_PRINT_TS=0
|
||||
|
||||
if birdc -s "$SOCK_PATH" monitor all 2>/dev/null | while IFS= read -r line; do
|
||||
[ -n "$line" ] || continue
|
||||
if ! is_rpki_related_event "$line"; then
|
||||
continue
|
||||
fi
|
||||
now_ts="$(date +%s)"
|
||||
if [ $((now_ts - LAST_PRINT_TS)) -lt "$DEBOUNCE_SECS" ]; then
|
||||
continue
|
||||
fi
|
||||
LAST_PRINT_TS="$now_ts"
|
||||
echo "[monitor] $line"
|
||||
print_snapshot
|
||||
done; then
|
||||
:
|
||||
else
|
||||
echo "[entrypoint] WARNING: birdc monitor failed, fallback to interval polling"
|
||||
MODE="interval"
|
||||
fi
|
||||
;;
|
||||
esac
|
||||
|
||||
if [ "$MODE" = "interval" ] && [ "$INTERVAL" -gt 0 ]; then
|
||||
while kill -0 "$BIRD_PID" 2>/dev/null; do
|
||||
print_snapshot
|
||||
sleep "$INTERVAL"
|
||||
done
|
||||
fi
|
||||
|
||||
wait "$BIRD_PID"
|
||||
@@ -1,32 +0,0 @@
|
||||
# Target RTR server addresses for client compose files
|
||||
# RPKI_RTR_CLIENT_IMAGE=rpki-rtr-debug-client-arm64:v1.2
|
||||
RPKI_RTR_CLIENT_IMAGE=rpki-rtr-debug-client-amd64:v1.2
|
||||
|
||||
# TCP example: 10.0.0.12:323
|
||||
# TLS example: rpki.example.com:324
|
||||
# SSH example: 10.0.0.12:22
|
||||
RPKI_RTR_TCP_SERVER_ADDR=rpki-rtr:323
|
||||
RPKI_RTR_TLS_SERVER_ADDR=rpki-rtr:324
|
||||
RPKI_RTR_SSH_SERVER_ADDR=rpki-rtr:22
|
||||
|
||||
# RTR protocol version used as client command second argument (supported: 0,1,2)
|
||||
RPKI_RTR_PROTOCOL_VERSION=2
|
||||
|
||||
# TLS server name used by --server-name in TLS mode
|
||||
# Must match server certificate SAN dNSName.
|
||||
RPKI_RTR_TLS_SERVER_NAME=localhost
|
||||
RPKI_RTR_TLS_CA_CERT_PATH=/app/certs/client-ca.crt
|
||||
RPKI_RTR_TLS_CLIENT_CERT_PATH=/app/certs/client-good.crt
|
||||
RPKI_RTR_TLS_CLIENT_KEY_PATH=/app/certs/client-good.key
|
||||
RPKI_RTR_TLS_CERTS_HOST_DIR=../../certs/tls
|
||||
|
||||
# Shared client logs mount on host.
|
||||
RPKI_RTR_CLIENT_LOG_HOST_DIR=../../logs/client
|
||||
|
||||
# SSH mode examples:
|
||||
# RPKI_RTR_SSH_SERVER_ADDR=10.0.0.12:2222
|
||||
RPKI_RTR_CLIENT_KEYS_VOLUME=../../certs/ssh:/app/certs:ro
|
||||
RPKI_RTR_CLIENT_KEY_PATH=/app/certs/rtr-client.key
|
||||
RPKI_RTR_SSH_SERVER_PUBKEY_PATH=/app/certs/ssh_host_rsa_key.pub
|
||||
RPKI_RTR_SSH_USERNAME=rpki-rtr
|
||||
RPKI_RTR_SSH_PASSWORD=
|
||||
@@ -1,26 +0,0 @@
|
||||
FROM rust:1.89-bookworm AS builder
|
||||
|
||||
WORKDIR /build
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends clang libclang-dev pkg-config \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY Cargo.toml Cargo.lock ./
|
||||
COPY src ./src
|
||||
|
||||
RUN cargo build --release --bin rtr_debug_client
|
||||
|
||||
FROM debian:bookworm-slim AS runtime
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends ca-certificates tzdata \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /app
|
||||
ENV TZ=Asia/Shanghai
|
||||
|
||||
COPY --from=builder /build/target/release/rtr_debug_client /usr/local/bin/rtr_debug_client
|
||||
COPY --chmod=755 deploy/client/entrypoint.sh /usr/local/bin/rtr-debug-client-entrypoint.sh
|
||||
|
||||
ENTRYPOINT ["/usr/local/bin/rtr-debug-client-entrypoint.sh"]
|
||||
@@ -1,237 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REPO_ROOT="$(cd -- "${SCRIPT_DIR}/../.." && pwd)"
|
||||
|
||||
MODE="${MODE:-tcp}"
|
||||
ACTION="up"
|
||||
DETACH="${DETACH:-1}"
|
||||
BUILD="${BUILD:-1}"
|
||||
CLIENTS="${CLIENTS:-single}"
|
||||
TARGET_PLATFORM="${TARGET_PLATFORM:-linux/arm64}"
|
||||
CLIENT_IMAGE="${CLIENT_IMAGE:-${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}}"
|
||||
BUILDX_PUSH="${BUILDX_PUSH:-0}"
|
||||
BUILDX_BUILDER="${BUILDX_BUILDER:-}"
|
||||
SKIP_BASE_IMAGE_PULL="${SKIP_BASE_IMAGE_PULL:-0}"
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
deploy/client/build-run.sh [action] [options]
|
||||
|
||||
Actions:
|
||||
up Build and run debug client (default)
|
||||
build Build debug client image
|
||||
buildx Build debug client image with docker buildx
|
||||
rebuild Rebuild image without cache, then run client
|
||||
down|stop Stop client compose stack
|
||||
restart Stop, then build and run client
|
||||
logs Follow client logs
|
||||
ps Show client compose status
|
||||
|
||||
Options:
|
||||
--mode MODE base|tcp|tls|ssh (default: tcp)
|
||||
--clients MODE single|multi (default: single)
|
||||
--platform VALUE Target platform for buildx (default: linux/arm64)
|
||||
--client-image TAG
|
||||
Client image tag for buildx and compose
|
||||
--push Push buildx image instead of loading it locally
|
||||
--builder NAME buildx builder name
|
||||
--skip-base-pull Do not pull Dockerfile base images before buildx
|
||||
--foreground Run single client in foreground
|
||||
--no-build Start without --build
|
||||
-h|--help Show this help
|
||||
EOF
|
||||
}
|
||||
|
||||
die() {
|
||||
echo "error: $*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
has_compose() {
|
||||
docker compose version >/dev/null 2>&1
|
||||
}
|
||||
|
||||
has_buildx() {
|
||||
docker buildx version >/dev/null 2>&1
|
||||
}
|
||||
|
||||
pull_base_images() {
|
||||
[[ "${SKIP_BASE_IMAGE_PULL}" == "1" ]] && return
|
||||
|
||||
local platform image
|
||||
local -a platforms images
|
||||
IFS=',' read -r -a platforms <<< "${TARGET_PLATFORM}"
|
||||
images=(rust:1.89-bookworm debian:bookworm-slim)
|
||||
|
||||
for platform in "${platforms[@]}"; do
|
||||
for image in "${images[@]}"; do
|
||||
docker pull --platform "${platform}" "${image}"
|
||||
done
|
||||
done
|
||||
}
|
||||
|
||||
compose_files() {
|
||||
if [[ "${CLIENTS}" == "multi" ]]; then
|
||||
[[ "${MODE}" == "tcp" || "${MODE}" == "base" ]] || die "--clients multi is only defined for tcp/base mode"
|
||||
printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.clients.yml"
|
||||
return
|
||||
fi
|
||||
|
||||
case "${MODE}" in
|
||||
base) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.yml" ;;
|
||||
tcp) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.tcp.yml" ;;
|
||||
tls) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.tls.yml" ;;
|
||||
ssh) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.ssh.yml" ;;
|
||||
*) die "unsupported mode '${MODE}', expected base|tcp|tls|ssh" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
compose_args() {
|
||||
local -a args files
|
||||
args=()
|
||||
[[ -f "${SCRIPT_DIR}/.env" ]] && args+=(--env-file "${SCRIPT_DIR}/.env")
|
||||
mapfile -t files < <(compose_files)
|
||||
args+=("${files[@]}")
|
||||
printf '%s\n' "${args[@]}"
|
||||
}
|
||||
|
||||
compose_run() {
|
||||
local -a args
|
||||
mapfile -t args < <(compose_args)
|
||||
docker compose "${args[@]}" "$@"
|
||||
}
|
||||
|
||||
up_client() {
|
||||
local -a up_args
|
||||
up_args=(up)
|
||||
[[ "${CLIENTS}" == "multi" || "${DETACH}" == "1" ]] && up_args+=(-d)
|
||||
[[ "${BUILD}" == "1" && "${CLIENTS}" != "multi" ]] && up_args+=(--build)
|
||||
compose_run "${up_args[@]}"
|
||||
}
|
||||
|
||||
build_client() {
|
||||
if [[ "${CLIENTS}" == "multi" ]]; then
|
||||
docker build "$@" -f "${SCRIPT_DIR}/Dockerfile" -t "${CLIENT_IMAGE}" "${REPO_ROOT}"
|
||||
else
|
||||
compose_run build "$@"
|
||||
fi
|
||||
}
|
||||
|
||||
buildx_client() {
|
||||
has_buildx || die "docker buildx is not available"
|
||||
if [[ "${TARGET_PLATFORM}" == *,* && "${BUILDX_PUSH}" != "1" ]]; then
|
||||
die "multi-platform buildx output requires --push"
|
||||
fi
|
||||
|
||||
local -a output_args builder_args
|
||||
[[ "${BUILDX_PUSH}" == "1" ]] && output_args=(--push) || output_args=(--load)
|
||||
[[ -n "${BUILDX_BUILDER}" ]] && builder_args=(--builder "${BUILDX_BUILDER}") || builder_args=()
|
||||
|
||||
pull_base_images
|
||||
|
||||
docker buildx build "${builder_args[@]}" \
|
||||
--platform "${TARGET_PLATFORM}" \
|
||||
-f "${SCRIPT_DIR}/Dockerfile" \
|
||||
-t "${CLIENT_IMAGE}" \
|
||||
"${output_args[@]}" \
|
||||
"${REPO_ROOT}"
|
||||
}
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
up|build|buildx|rebuild|down|stop|restart|logs|ps)
|
||||
ACTION="$1"
|
||||
shift
|
||||
;;
|
||||
--mode)
|
||||
[[ $# -ge 2 ]] || die "--mode requires a value"
|
||||
MODE="$2"
|
||||
shift 2
|
||||
;;
|
||||
--clients)
|
||||
[[ $# -ge 2 ]] || die "--clients requires a value"
|
||||
CLIENTS="$2"
|
||||
shift 2
|
||||
;;
|
||||
--platform)
|
||||
[[ $# -ge 2 ]] || die "--platform requires a value"
|
||||
TARGET_PLATFORM="$2"
|
||||
shift 2
|
||||
;;
|
||||
--client-image)
|
||||
[[ $# -ge 2 ]] || die "--client-image requires a value"
|
||||
CLIENT_IMAGE="$2"
|
||||
shift 2
|
||||
;;
|
||||
--push)
|
||||
BUILDX_PUSH=1
|
||||
shift
|
||||
;;
|
||||
--builder)
|
||||
[[ $# -ge 2 ]] || die "--builder requires a value"
|
||||
BUILDX_BUILDER="$2"
|
||||
shift 2
|
||||
;;
|
||||
--skip-base-pull)
|
||||
SKIP_BASE_IMAGE_PULL=1
|
||||
shift
|
||||
;;
|
||||
--foreground)
|
||||
DETACH=0
|
||||
shift
|
||||
;;
|
||||
--no-build)
|
||||
BUILD=0
|
||||
shift
|
||||
;;
|
||||
-h|--help)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
die "unknown argument '$1'"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ "${CLIENTS}" == "single" || "${CLIENTS}" == "multi" ]] || die "unsupported clients '${CLIENTS}', expected single|multi"
|
||||
export RPKI_RTR_CLIENT_IMAGE="${CLIENT_IMAGE}"
|
||||
if [[ "${ACTION}" == "buildx" ]]; then
|
||||
has_buildx || die "docker buildx is not available"
|
||||
else
|
||||
has_compose || die "docker compose is not available"
|
||||
fi
|
||||
|
||||
cd "${REPO_ROOT}"
|
||||
|
||||
case "${ACTION}" in
|
||||
up)
|
||||
up_client
|
||||
;;
|
||||
build)
|
||||
build_client
|
||||
;;
|
||||
buildx)
|
||||
buildx_client
|
||||
;;
|
||||
rebuild)
|
||||
build_client --no-cache
|
||||
up_client
|
||||
;;
|
||||
down|stop)
|
||||
compose_run down
|
||||
;;
|
||||
restart)
|
||||
compose_run down
|
||||
up_client
|
||||
;;
|
||||
logs)
|
||||
compose_run logs -f
|
||||
;;
|
||||
ps)
|
||||
compose_run ps
|
||||
;;
|
||||
esac
|
||||
@@ -1,52 +0,0 @@
|
||||
version: "3.9"
|
||||
|
||||
services:
|
||||
rtr-client-1:
|
||||
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
||||
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
||||
volumes:
|
||||
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
||||
restart: no
|
||||
networks:
|
||||
- rpki_net
|
||||
|
||||
rtr-client-2:
|
||||
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
||||
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
||||
volumes:
|
||||
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
||||
restart: no
|
||||
networks:
|
||||
- rpki_net
|
||||
|
||||
rtr-client-3:
|
||||
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
||||
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
||||
volumes:
|
||||
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
||||
restart: no
|
||||
networks:
|
||||
- rpki_net
|
||||
|
||||
rtr-client-4:
|
||||
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
||||
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
||||
volumes:
|
||||
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
||||
restart: no
|
||||
networks:
|
||||
- rpki_net
|
||||
|
||||
rtr-client-5:
|
||||
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
||||
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
||||
volumes:
|
||||
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
||||
restart: no
|
||||
networks:
|
||||
- rpki_net
|
||||
|
||||
networks:
|
||||
rpki_net:
|
||||
name: rpki_net
|
||||
driver: bridge
|
||||
@@ -1,36 +0,0 @@
|
||||
version: "3.9"
|
||||
|
||||
services:
|
||||
rtr-debug-client:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: deploy/client/Dockerfile
|
||||
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
||||
command:
|
||||
[
|
||||
"${RPKI_RTR_SSH_SERVER_ADDR:-rpki-rtr:22}",
|
||||
"${RPKI_RTR_PROTOCOL_VERSION:-2}",
|
||||
"reset",
|
||||
"--ssh",
|
||||
"--ssh-user",
|
||||
"${RPKI_RTR_SSH_USERNAME:-rpki-rtr}",
|
||||
"--ssh-password",
|
||||
"${RPKI_RTR_SSH_PASSWORD}",
|
||||
"--ssh-server-key",
|
||||
"${RPKI_RTR_SSH_SERVER_PUBKEY_PATH:-/app/certs/ssh_host_rsa_key.pub}",
|
||||
"--keep-after-error",
|
||||
"--summary-only"
|
||||
]
|
||||
volumes:
|
||||
- ${RPKI_RTR_CLIENT_KEYS_VOLUME:-../../certs/ssh:/app/certs:ro}
|
||||
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
||||
restart: no
|
||||
stdin_open: true
|
||||
tty: true
|
||||
networks:
|
||||
- rpki_net
|
||||
|
||||
networks:
|
||||
rpki_net:
|
||||
name: rpki_net
|
||||
driver: bridge
|
||||
@@ -1,36 +0,0 @@
|
||||
version: "3.9"
|
||||
|
||||
services:
|
||||
rtr-debug-client:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: deploy/client/Dockerfile
|
||||
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
||||
command:
|
||||
[
|
||||
"${RPKI_RTR_SSH_SERVER_ADDR:-rpki-rtr:22}",
|
||||
"${RPKI_RTR_PROTOCOL_VERSION:-2}",
|
||||
"reset",
|
||||
"--ssh",
|
||||
"--ssh-user",
|
||||
"${RPKI_RTR_SSH_USERNAME:-rpki-rtr}",
|
||||
"--ssh-key",
|
||||
"${RPKI_RTR_CLIENT_KEY_PATH:-/app/certs/rtr-client.key}",
|
||||
"--ssh-server-key",
|
||||
"${RPKI_RTR_SSH_SERVER_PUBKEY_PATH:-/app/certs/ssh_host_rsa_key.pub}",
|
||||
"--keep-after-error",
|
||||
"--summary-only"
|
||||
]
|
||||
volumes:
|
||||
- ${RPKI_RTR_CLIENT_KEYS_VOLUME:-../../certs/ssh:/app/certs:ro}
|
||||
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
||||
restart: no
|
||||
stdin_open: true
|
||||
tty: true
|
||||
networks:
|
||||
- rpki_net
|
||||
|
||||
networks:
|
||||
rpki_net:
|
||||
name: rpki_net
|
||||
driver: bridge
|
||||
@@ -1,19 +0,0 @@
|
||||
services:
|
||||
rtr-debug-client:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: deploy/client/Dockerfile
|
||||
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
||||
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
||||
volumes:
|
||||
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
||||
restart: no
|
||||
stdin_open: true
|
||||
tty: true
|
||||
networks:
|
||||
- rpki_net
|
||||
|
||||
networks:
|
||||
rpki_net:
|
||||
name: rpki_net
|
||||
driver: bridge
|
||||
@@ -1,38 +0,0 @@
|
||||
version: "3.9"
|
||||
|
||||
services:
|
||||
rtr-debug-client:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: deploy/client/Dockerfile
|
||||
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
||||
command:
|
||||
[
|
||||
"${RPKI_RTR_TLS_SERVER_ADDR:-rpki-rtr:324}",
|
||||
"${RPKI_RTR_PROTOCOL_VERSION:-2}",
|
||||
"reset",
|
||||
"--tls",
|
||||
"--ca-cert",
|
||||
"${RPKI_RTR_TLS_CA_CERT_PATH:-/app/certs/client-ca.crt}",
|
||||
"--server-name",
|
||||
"${RPKI_RTR_TLS_SERVER_NAME:-localhost}",
|
||||
"--client-cert",
|
||||
"${RPKI_RTR_TLS_CLIENT_CERT_PATH:-/app/certs/client-good.crt}",
|
||||
"--client-key",
|
||||
"${RPKI_RTR_TLS_CLIENT_KEY_PATH:-/app/certs/client-good.key}",
|
||||
"--keep-after-error",
|
||||
"--summary-only"
|
||||
]
|
||||
volumes:
|
||||
- ${RPKI_RTR_TLS_CERTS_HOST_DIR:-../../certs/tls}:/app/certs:ro
|
||||
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
||||
restart: no
|
||||
stdin_open: true
|
||||
tty: true
|
||||
networks:
|
||||
- rpki_net
|
||||
|
||||
networks:
|
||||
rpki_net:
|
||||
name: rpki_net
|
||||
driver: bridge
|
||||
@@ -1,19 +0,0 @@
|
||||
services:
|
||||
rtr-debug-client:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: deploy/client/Dockerfile
|
||||
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
||||
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
||||
volumes:
|
||||
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
||||
restart: no
|
||||
stdin_open: true
|
||||
tty: true
|
||||
networks:
|
||||
- rpki_net
|
||||
|
||||
networks:
|
||||
rpki_net:
|
||||
name: rpki_net
|
||||
driver: bridge
|
||||
@@ -1,10 +0,0 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
mkdir -p /app/logs
|
||||
|
||||
log_name="${HOSTNAME:-rtr-debug-client}"
|
||||
stdout_log="/app/logs/${log_name}.stdout.log"
|
||||
stderr_log="/app/logs/${log_name}.stderr.log"
|
||||
|
||||
exec /usr/local/bin/rtr_debug_client "$@" >>"$stdout_log" 2>>"$stderr_log"
|
||||
@@ -1,4 +0,0 @@
|
||||
# RTR server endpoint used by deploy/frr/frr.conf.template.
|
||||
RPKI_FRR_RPKI_HOST=127.0.0.1
|
||||
RPKI_FRR_RPKI_PORT=323
|
||||
RPKI_FRR_RPKI_PREFERENCE=1
|
||||
@@ -1,78 +0,0 @@
|
||||
# FRR Minimal RTR Client Config
|
||||
|
||||
中文文档: [README.zh.md](./README.zh.md)
|
||||
|
||||
This folder provides a minimal FRR setup for black-box interop testing
|
||||
against this repository's RTR server defaults.
|
||||
|
||||
Server defaults in this repo:
|
||||
- TCP: `0.0.0.0:323`
|
||||
- TLS: `0.0.0.0:324`
|
||||
|
||||
Reference:
|
||||
- `src/main.rs`
|
||||
|
||||
## Files
|
||||
|
||||
- `daemons.example`: sample `/etc/frr/daemons`
|
||||
- `frr.conf.example`: static sample `/etc/frr/frr.conf`
|
||||
- `frr.conf.template`: Docker template used to generate `/etc/frr/frr.conf`
|
||||
- `entrypoint.sh`: renders `frr.conf.template` from environment variables before starting FRR
|
||||
- `.env`: default Docker environment values
|
||||
|
||||
## How to apply on an FRR host
|
||||
|
||||
1. Copy `daemons.example` to `/etc/frr/daemons`.
|
||||
2. Copy `frr.conf.example` to `/etc/frr/frr.conf`.
|
||||
3. Restart FRR:
|
||||
|
||||
```bash
|
||||
sudo systemctl restart frr
|
||||
```
|
||||
|
||||
## Verify
|
||||
|
||||
```bash
|
||||
vtysh -c "show rpki configuration"
|
||||
vtysh -c "show rpki cache-server"
|
||||
vtysh -c "show rpki cache-connection"
|
||||
vtysh -c "show rpki prefix-table"
|
||||
```
|
||||
|
||||
If `show rpki cache-connection` is connected and `show rpki prefix-table`
|
||||
contains VRPs, the RTR client path is working.
|
||||
|
||||
## Docker quick start
|
||||
|
||||
From repository root:
|
||||
|
||||
```bash
|
||||
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml up -d
|
||||
docker exec -it frr-rpki-client vtysh -c "show rpki cache-connection"
|
||||
docker exec -it frr-rpki-client vtysh -c "show rpki prefix-table"
|
||||
```
|
||||
|
||||
The Docker example defaults to `127.0.0.1:323`. Override the RTR server
|
||||
endpoint with environment variables:
|
||||
|
||||
```bash
|
||||
RPKI_FRR_RPKI_HOST=10.0.0.12 \
|
||||
RPKI_FRR_RPKI_PORT=323 \
|
||||
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml up -d
|
||||
```
|
||||
|
||||
Stop:
|
||||
|
||||
```bash
|
||||
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml down
|
||||
```
|
||||
|
||||
## Notes
|
||||
|
||||
- This setup targets RTR over TCP (`rpki cache tcp`).
|
||||
- Docker mode renders `frr.conf.template` into `/etc/frr/frr.conf` at startup.
|
||||
- Keep protocol-level conformance checks in Rust tests and
|
||||
`src/bin/rtr_debug_client`.
|
||||
- `network_mode: host` is kept for the minimal FRR container example, but the
|
||||
RTR server address is controlled by `RPKI_FRR_RPKI_HOST` and
|
||||
`RPKI_FRR_RPKI_PORT`.
|
||||
@@ -1,83 +0,0 @@
|
||||
# FRR 最小化 RTR 客户端配置
|
||||
|
||||
本目录提供一个 FRR 最小配置,用于和本仓库 RTR Server 做黑盒互通测试。
|
||||
|
||||
本仓库默认 RTR 监听地址:
|
||||
- TCP:`0.0.0.0:323`
|
||||
- TLS:`0.0.0.0:324`
|
||||
|
||||
参考实现:
|
||||
- `src/main.rs`
|
||||
|
||||
## 文件说明
|
||||
|
||||
- `daemons.example`:示例 `/etc/frr/daemons`
|
||||
- `frr.conf.example`:静态示例 `/etc/frr/frr.conf`
|
||||
- `frr.conf.template`:Docker 启动时生成 `/etc/frr/frr.conf` 的模板
|
||||
- `entrypoint.sh`:启动 FRR 前根据环境变量渲染 `frr.conf.template`
|
||||
- `.env`:Docker 示例使用的默认环境变量
|
||||
|
||||
## 在 FRR 主机上应用
|
||||
|
||||
1. 复制 `daemons.example` 到 `/etc/frr/daemons`
|
||||
2. 复制 `frr.conf.example` 到 `/etc/frr/frr.conf`
|
||||
3. 重启 FRR:
|
||||
|
||||
```bash
|
||||
sudo systemctl restart frr
|
||||
```
|
||||
|
||||
## 验证命令
|
||||
|
||||
```bash
|
||||
vtysh -c "show rpki configuration"
|
||||
vtysh -c "show rpki cache-server"
|
||||
vtysh -c "show rpki cache-connection"
|
||||
vtysh -c "show rpki prefix-table"
|
||||
```
|
||||
|
||||
当 `show rpki cache-connection` 显示已连接,且 `show rpki prefix-table` 出现 VRP 时,说明 RTR 客户端链路工作正常。
|
||||
|
||||
## Docker 快速启动
|
||||
|
||||
在仓库根目录执行:
|
||||
|
||||
```bash
|
||||
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml up -d
|
||||
docker exec -it frr-rpki-client vtysh -c "show rpki cache-connection"
|
||||
docker exec -it frr-rpki-client vtysh -c "show rpki prefix-table"
|
||||
```
|
||||
|
||||
Docker 示例默认连接 `127.0.0.1:323`。如果 RTR Server 在其他地址,可以通过环境变量覆盖:
|
||||
|
||||
```bash
|
||||
RPKI_FRR_RPKI_HOST=10.0.0.12 \
|
||||
RPKI_FRR_RPKI_PORT=323 \
|
||||
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml up -d
|
||||
```
|
||||
|
||||
停止:
|
||||
|
||||
```bash
|
||||
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml down
|
||||
```
|
||||
|
||||
## 可测试范围
|
||||
|
||||
- FRR 作为 RTR Client 与本仓库 Server 的 TCP 建链能力
|
||||
- FRR 侧基础会话状态可见性(`cache-server` / `cache-connection`)
|
||||
- VRP 下发与导入是否成功(`prefix-table` 是否有条目)
|
||||
- 基于服务端数据变化触发的前缀表更新(可通过替换 `data` 后观察)
|
||||
- 黑盒互通回归:用于确认“路由器客户端视角”功能可用
|
||||
|
||||
## 不覆盖范围
|
||||
|
||||
- 不替代协议级单元测试/集成测试(PDU 细节、异常路径、边界条件)
|
||||
- 不替代 `src/bin/rtr_debug_client` 的逐报文调试能力
|
||||
- 默认示例以 TCP 为主,TLS/mTLS 需按你的证书与 FRR 配置单独扩展
|
||||
|
||||
## 说明
|
||||
|
||||
- 当前示例主要针对 RTR over TCP(`rpki cache tcp`)
|
||||
- Docker 模式会在启动时把 `frr.conf.template` 渲染为 `/etc/frr/frr.conf`
|
||||
- `network_mode: host` 仍用于最小化 FRR 容器示例,但 RTR Server 地址由 `RPKI_FRR_RPKI_HOST` 和 `RPKI_FRR_RPKI_PORT` 控制
|
||||
@@ -1,7 +0,0 @@
|
||||
# Minimal FRR daemons config for RPKI testing
|
||||
|
||||
zebra=yes
|
||||
bgpd=yes
|
||||
|
||||
# Enable bgpd RPKI module
|
||||
bgpd_options=" -A 127.0.0.1 -M rpki"
|
||||
@@ -1,16 +0,0 @@
|
||||
services:
|
||||
frr-rpki-client:
|
||||
image: quay.io/frrouting/frr:10.2.2
|
||||
container_name: frr-rpki-client
|
||||
restart: unless-stopped
|
||||
network_mode: host
|
||||
privileged: true
|
||||
environment:
|
||||
RPKI_FRR_RPKI_HOST: "${RPKI_FRR_RPKI_HOST:-127.0.0.1}"
|
||||
RPKI_FRR_RPKI_PORT: "${RPKI_FRR_RPKI_PORT:-323}"
|
||||
RPKI_FRR_RPKI_PREFERENCE: "${RPKI_FRR_RPKI_PREFERENCE:-1}"
|
||||
entrypoint: ["/bin/sh", "/config/entrypoint.sh"]
|
||||
volumes:
|
||||
- ./daemons.example:/etc/frr/daemons:ro
|
||||
- ./frr.conf.template:/config/frr.conf.template:ro
|
||||
- ./entrypoint.sh:/config/entrypoint.sh:ro
|
||||
@@ -1,24 +0,0 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
: "${RPKI_FRR_RPKI_HOST:=127.0.0.1}"
|
||||
: "${RPKI_FRR_RPKI_PORT:=323}"
|
||||
: "${RPKI_FRR_RPKI_PREFERENCE:=1}"
|
||||
: "${RPKI_FRR_CONFIG_TEMPLATE_PATH:=/config/frr.conf.template}"
|
||||
: "${RPKI_FRR_CONFIG_PATH:=/etc/frr/frr.conf}"
|
||||
|
||||
escape_sed() {
|
||||
printf '%s' "$1" | sed 's/[&|\\]/\\&/g'
|
||||
}
|
||||
|
||||
host="$(escape_sed "${RPKI_FRR_RPKI_HOST}")"
|
||||
port="$(escape_sed "${RPKI_FRR_RPKI_PORT}")"
|
||||
preference="$(escape_sed "${RPKI_FRR_RPKI_PREFERENCE}")"
|
||||
|
||||
sed \
|
||||
-e "s|\${RPKI_FRR_RPKI_HOST}|${host}|g" \
|
||||
-e "s|\${RPKI_FRR_RPKI_PORT}|${port}|g" \
|
||||
-e "s|\${RPKI_FRR_RPKI_PREFERENCE}|${preference}|g" \
|
||||
"${RPKI_FRR_CONFIG_TEMPLATE_PATH}" > "${RPKI_FRR_CONFIG_PATH}"
|
||||
|
||||
exec /usr/lib/frr/docker-start
|
||||
@@ -1,22 +0,0 @@
|
||||
frr version 10.2
|
||||
frr defaults traditional
|
||||
hostname rpki-lab
|
||||
service integrated-vtysh-config
|
||||
!
|
||||
debug rpki
|
||||
!
|
||||
rpki
|
||||
rpki polling_period 10
|
||||
rpki timeout 10
|
||||
rpki retry_interval 10
|
||||
rpki expire_interval 7200
|
||||
rpki cache tcp 127.0.0.1 323 preference 1
|
||||
exit
|
||||
!
|
||||
router bgp 65001
|
||||
bgp router-id 192.0.2.1
|
||||
!
|
||||
address-family ipv4 unicast
|
||||
exit-address-family
|
||||
!
|
||||
line vty
|
||||
@@ -1,22 +0,0 @@
|
||||
frr version 10.2
|
||||
frr defaults traditional
|
||||
hostname rpki-lab
|
||||
service integrated-vtysh-config
|
||||
!
|
||||
debug rpki
|
||||
!
|
||||
rpki
|
||||
rpki polling_period 10
|
||||
rpki timeout 10
|
||||
rpki retry_interval 10
|
||||
rpki expire_interval 7200
|
||||
rpki cache tcp ${RPKI_FRR_RPKI_HOST} ${RPKI_FRR_RPKI_PORT} preference ${RPKI_FRR_RPKI_PREFERENCE}
|
||||
exit
|
||||
!
|
||||
router bgp 65001
|
||||
bgp router-id 192.0.2.1
|
||||
!
|
||||
address-family ipv4 unicast
|
||||
exit-address-family
|
||||
!
|
||||
line vty
|
||||
@@ -1,49 +0,0 @@
|
||||
# Core runtime knobs.
|
||||
# RPKI_RTR_SERVER_IMAGE=rpki-rtr-arm64:v1.2
|
||||
RPKI_RTR_SERVER_IMAGE=rpki-rtr-amd64:v1.2
|
||||
|
||||
# Data source directories on host.
|
||||
RPKI_RTR_CCR_HOST_DIR=../../data
|
||||
RPKI_RTR_SLURM_HOST_DIR=../../data
|
||||
|
||||
# In-container data source directories.
|
||||
RPKI_RTR_CCR_DIR=/app/data
|
||||
RPKI_RTR_SLURM_DIR=/app/slurm
|
||||
RPKI_RTR_REPORT_HOST_DIR=../../report
|
||||
|
||||
# In-container runtime paths.
|
||||
RPKI_RTR_DB_PATH=/app/rtr-db
|
||||
RPKI_RTR_REPORT_DIR=/app/report
|
||||
|
||||
# Persistent directories on host.
|
||||
RPKI_RTR_DB_HOST_DIR=../../rtr-db
|
||||
RPKI_RTR_LOG_HOST_DIR=../../logs/server
|
||||
RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS=300
|
||||
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS=300
|
||||
RPKI_RTR_REPORT_HISTORY_LIMIT=10
|
||||
RPKI_RTR_TIMEZONE=Asia/Shanghai
|
||||
RPKI_RTR_MAX_DELTA=10
|
||||
RPKI_RTR_MAX_CONNECTIONS=100000
|
||||
RPKI_RTR_MAX_CONCURRENT_HANDSHAKES=128
|
||||
RPKI_RTR_ADMIN_ADDR=0.0.0.0:8323
|
||||
RPKI_RTR_ADMIN_TOKEN=qwert
|
||||
RUST_LOG=info
|
||||
|
||||
# TLS mode knobs.
|
||||
RPKI_RTR_ENFORCE_TLS_CLIENT_SAN_IP_MATCH=false
|
||||
RPKI_RTR_TLS_CERT_PATH=/app/certs/server-dns.crt
|
||||
RPKI_RTR_TLS_KEY_PATH=/app/certs/server-dns.key
|
||||
RPKI_RTR_TLS_CLIENT_CA_PATH=/app/certs/client-ca.crt
|
||||
|
||||
RPKI_RTR_TLS_CERTS_HOST_DIR=../../certs/tls
|
||||
|
||||
# SSH mode knobs.
|
||||
RPKI_RTR_SSH_HOST_PORT=2222
|
||||
RPKI_RTR_SSH_CONTAINER_PORT=22
|
||||
RPKI_RTR_SSH_AUTH_MODE=key
|
||||
RPKI_RTR_SSH_USERNAME=rpki-rtr
|
||||
RPKI_RTR_SSH_SUBSYSTEM_NAME=rpki-rtr
|
||||
RPKI_RTR_SSH_HOST_KEY_PATH=/app/certs/ssh_host_rsa_key
|
||||
RPKI_RTR_SSH_AUTHORIZED_KEYS_PATH=/app/certs/rtr-authorized_keys
|
||||
|
||||
RPKI_RTR_SSH_CERTS_HOST_DIR=../../certs/ssh
|
||||
@@ -1,99 +0,0 @@
|
||||
# Deployment (Docker Compose)
|
||||
|
||||
This project runs `src/main.rs` as a long-running server that:
|
||||
|
||||
1. loads latest `.ccr` from a configured directory,
|
||||
2. applies optional SLURM filtering,
|
||||
3. starts RTR server.
|
||||
|
||||
The container runs `rpki` directly as PID 1.
|
||||
|
||||
## Files
|
||||
|
||||
- `deploy/server/Dockerfile`
|
||||
- `deploy/server/docker-compose.yml`
|
||||
|
||||
## Runtime Paths in Container
|
||||
|
||||
- CCR directory: `/app/data`
|
||||
- RocksDB directory: `/app/rtr-db`
|
||||
- SLURM directory: `/app/slurm`
|
||||
- Report directory: `/app/report`
|
||||
- TLS cert directory (optional): `/app/certs`
|
||||
|
||||
## Path Configuration via `.env`
|
||||
|
||||
- `RPKI_RTR_CCR_HOST_DIR`: host CCR directory mounted into container
|
||||
- `RPKI_RTR_SLURM_HOST_DIR`: host SLURM directory mounted into container
|
||||
- `RPKI_RTR_CCR_DIR`: in-container CCR directory path
|
||||
- `RPKI_RTR_SLURM_DIR`: in-container SLURM directory path
|
||||
- `RPKI_RTR_DB_HOST_DIR`: host RocksDB directory
|
||||
- `RPKI_RTR_LOG_HOST_DIR`: host log directory
|
||||
- `RPKI_RTR_REPORT_HOST_DIR`: host directory receiving split RTR JSON reports
|
||||
- `RPKI_RTR_DB_PATH`: in-container RocksDB directory
|
||||
- `RPKI_RTR_REPORT_DIR`: in-container report directory
|
||||
|
||||
## Runtime Configuration via `.env`
|
||||
|
||||
- Core: `RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS`, `RPKI_RTR_MAX_DELTA`, `RPKI_RTR_MAX_CONCURRENT_HANDSHAKES`, `RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS`, `RPKI_RTR_REPORT_HISTORY_LIMIT`, `RPKI_RTR_TIMEZONE`, `RPKI_RTR_ADMIN_ADDR`, `RPKI_RTR_ADMIN_TOKEN`, `RUST_LOG`
|
||||
- TCP mode: `RPKI_RTR_MAX_CONNECTIONS`
|
||||
- TLS mode: `RPKI_RTR_ENFORCE_TLS_CLIENT_SAN_IP_MATCH`, `RPKI_RTR_TLS_CERT_PATH`, `RPKI_RTR_TLS_KEY_PATH`, `RPKI_RTR_TLS_CLIENT_CA_PATH`, `RPKI_RTR_TLS_CERTS_HOST_DIR`
|
||||
- SSH mode: `RPKI_RTR_SSH_HOST_PORT`, `RPKI_RTR_SSH_CONTAINER_PORT`, `RPKI_RTR_SSH_AUTH_MODE`, `RPKI_RTR_SSH_USERNAME`, `RPKI_RTR_SSH_SUBSYSTEM_NAME`, `RPKI_RTR_SSH_HOST_KEY_PATH`, `RPKI_RTR_SSH_AUTHORIZED_KEYS_PATH`, `RPKI_RTR_SSH_CERTS_HOST_DIR`
|
||||
|
||||
## Start
|
||||
|
||||
```bash
|
||||
docker compose -f deploy/server/docker-compose.yml up -d --build
|
||||
```
|
||||
|
||||
## Stop
|
||||
|
||||
```bash
|
||||
docker compose -f deploy/server/docker-compose.yml down
|
||||
```
|
||||
|
||||
## Logs
|
||||
|
||||
```bash
|
||||
docker compose -f deploy/server/docker-compose.yml logs -f rpki-rtr
|
||||
```
|
||||
|
||||
The admin API can also stream the redirected log file:
|
||||
|
||||
```bash
|
||||
curl -N "http://127.0.0.1:8323/admin/rtr/logs/tail?stream=stdout&lines=200" \
|
||||
-H "Authorization: Bearer $RPKI_RTR_ADMIN_TOKEN"
|
||||
```
|
||||
|
||||
It reads `/app/logs/${HOSTNAME}.stdout.log` or `.stderr.log` by default. Set
|
||||
`RPKI_RTR_LOG_DIR` and `RPKI_RTR_LOG_NAME` to override that lookup.
|
||||
|
||||
## Runtime Report
|
||||
|
||||
The server writes split JSON reports. Each report file uses a local-time
|
||||
timestamp suffix and each category keeps `RPKI_RTR_REPORT_HISTORY_LIMIT` files,
|
||||
defaulting to 10.
|
||||
|
||||
- `rtr-source-*.json`: CCR and SLURM source metadata, latest refresh status,
|
||||
data quality counts, cache snapshot counts, and delta counts. Written on
|
||||
startup and source refresh events.
|
||||
- `rtr-clients-*.json`: active client connection counts and counts by transport
|
||||
(`tcp`, `tls`, `ssh`). Written on startup, whenever the active connection
|
||||
count changes.
|
||||
- `rtr-runtime-*.json`: service start time, uptime, process RSS, and
|
||||
non-sensitive runtime configuration. Written on startup and every
|
||||
`RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS`, defaulting to 300 seconds.
|
||||
|
||||
Timestamps in logs and report JSON files use `RPKI_RTR_TIMEZONE`, which
|
||||
defaults to `Asia/Shanghai`. Use IANA timezone names such as `Asia/Shanghai`,
|
||||
`Europe/London`, `America/New_York`, or `UTC`; `Shanghai` is accepted as a
|
||||
convenience alias for `Asia/Shanghai`.
|
||||
|
||||
## Runtime Admin Config
|
||||
|
||||
The admin endpoint is disabled by default. Set `RPKI_RTR_ADMIN_ADDR` to enable
|
||||
`POST /admin/rtr/config`. If the address is not loopback, `RPKI_RTR_ADMIN_TOKEN`
|
||||
must also be set and requests must include `Authorization: Bearer <token>`.
|
||||
|
||||
The endpoint accepts partial JSON updates. See `docs/rtr-admin-api.md` for the
|
||||
complete request/response schema, examples, and runtime apply semantics.
|
||||
@@ -1,85 +0,0 @@
|
||||
FROM rust:1.89-bookworm AS builder
|
||||
|
||||
WORKDIR /build
|
||||
|
||||
ENV CARGO_BUILD_JOBS=2
|
||||
|
||||
RUN set -eux; \
|
||||
cat > /etc/apt/sources.list.d/debian.sources <<'EOF'
|
||||
Types: deb
|
||||
URIs: http://mirrors.tuna.tsinghua.edu.cn/debian
|
||||
Suites: bookworm bookworm-updates
|
||||
Components: main
|
||||
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
|
||||
|
||||
Types: deb
|
||||
URIs: http://mirrors.tuna.tsinghua.edu.cn/debian-security
|
||||
Suites: bookworm-security
|
||||
Components: main
|
||||
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
|
||||
EOF
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --fix-missing --no-install-recommends \
|
||||
-o Acquire::Retries=10 \
|
||||
-o Acquire::http::Timeout=60 \
|
||||
build-essential \
|
||||
cmake \
|
||||
pkg-config \
|
||||
clang \
|
||||
libclang-dev \
|
||||
libssl-dev \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY Cargo.toml Cargo.lock ./
|
||||
COPY src ./src
|
||||
|
||||
RUN cargo build --release --bin rpki_rtr
|
||||
|
||||
FROM debian:bookworm-slim AS runtime
|
||||
|
||||
RUN set -eux; \
|
||||
cat > /etc/apt/sources.list.d/debian.sources <<'EOF'
|
||||
Types: deb
|
||||
URIs: http://mirrors.tuna.tsinghua.edu.cn/debian
|
||||
Suites: bookworm bookworm-updates
|
||||
Components: main
|
||||
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
|
||||
|
||||
Types: deb
|
||||
URIs: http://mirrors.tuna.tsinghua.edu.cn/debian-security
|
||||
Suites: bookworm-security
|
||||
Components: main
|
||||
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
|
||||
EOF
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --fix-missing --no-install-recommends \
|
||||
-o Acquire::Retries=10 \
|
||||
-o Acquire::http::Timeout=60 \
|
||||
ca-certificates \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
COPY --from=builder /build/target/release/rpki_rtr /usr/local/bin/rpki_rtr
|
||||
COPY --chmod=755 deploy/server/entrypoint.sh /usr/local/bin/rpki-rtr-entrypoint.sh
|
||||
|
||||
RUN mkdir -p /app/data /app/rtr-db /app/certs /app/slurm /app/logs /app/report
|
||||
|
||||
ENV RPKI_RTR_ENABLE_TLS=false \
|
||||
RPKI_RTR_TCP_ADDR=0.0.0.0:323 \
|
||||
RPKI_RTR_TLS_ADDR=0.0.0.0:324 \
|
||||
RPKI_RTR_DB_PATH=/app/rtr-db \
|
||||
RPKI_RTR_CCR_DIR=/app/data \
|
||||
RPKI_RTR_SLURM_DIR=/app/slurm \
|
||||
RPKI_RTR_REPORT_DIR=/app/report \
|
||||
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS=300 \
|
||||
RPKI_RTR_REPORT_HISTORY_LIMIT=10 \
|
||||
RPKI_RTR_REFRESH_INTERVAL_SECS=300 \
|
||||
RPKI_RTR_ADMIN_ADDR="" \
|
||||
RPKI_RTR_ADMIN_TOKEN=""
|
||||
|
||||
EXPOSE 323 324
|
||||
|
||||
ENTRYPOINT ["/usr/local/bin/rpki-rtr-entrypoint.sh"]
|
||||
@@ -1,215 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||
REPO_ROOT="$(cd -- "${SCRIPT_DIR}/../.." && pwd)"
|
||||
|
||||
MODE="${MODE:-tcp}"
|
||||
ACTION="up"
|
||||
DETACH="${DETACH:-1}"
|
||||
BUILD="${BUILD:-1}"
|
||||
TARGET_PLATFORM="${TARGET_PLATFORM:-linux/arm64}"
|
||||
SERVER_IMAGE="${SERVER_IMAGE:-${RPKI_RTR_SERVER_IMAGE:-rpki-rtr:latest}}"
|
||||
BUILDX_PUSH="${BUILDX_PUSH:-0}"
|
||||
BUILDX_BUILDER="${BUILDX_BUILDER:-}"
|
||||
SKIP_BASE_IMAGE_PULL="${SKIP_BASE_IMAGE_PULL:-0}"
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
deploy/server/build-run.sh [action] [options]
|
||||
|
||||
Actions:
|
||||
up Build and start server (default)
|
||||
build Build server image with docker compose
|
||||
buildx Build server image with docker buildx
|
||||
rebuild Rebuild image without cache, then start server
|
||||
down|stop Stop server compose stack
|
||||
restart Stop, then build and start server
|
||||
logs Follow server logs
|
||||
ps Show server compose status
|
||||
|
||||
Options:
|
||||
--mode MODE base|tcp|tls|ssh (default: tcp)
|
||||
--platform VALUE Target platform for buildx (default: linux/arm64)
|
||||
--server-image TAG
|
||||
Server image tag for buildx and compose
|
||||
--push Push buildx image instead of loading it locally
|
||||
--builder NAME buildx builder name
|
||||
--skip-base-pull Do not pull Dockerfile base images before buildx
|
||||
--foreground Run server in foreground
|
||||
--no-build Start without --build
|
||||
-h|--help Show this help
|
||||
EOF
|
||||
}
|
||||
|
||||
die() {
|
||||
echo "error: $*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
has_compose() {
|
||||
docker compose version >/dev/null 2>&1
|
||||
}
|
||||
|
||||
has_buildx() {
|
||||
docker buildx version >/dev/null 2>&1
|
||||
}
|
||||
|
||||
pull_base_images() {
|
||||
[[ "${SKIP_BASE_IMAGE_PULL}" == "1" ]] && return
|
||||
|
||||
local platform image
|
||||
local -a platforms images
|
||||
IFS=',' read -r -a platforms <<< "${TARGET_PLATFORM}"
|
||||
images=(rust:1.89-bookworm debian:bookworm-slim)
|
||||
|
||||
for platform in "${platforms[@]}"; do
|
||||
for image in "${images[@]}"; do
|
||||
docker pull --platform "${platform}" "${image}"
|
||||
done
|
||||
done
|
||||
}
|
||||
|
||||
compose_files() {
|
||||
case "${MODE}" in
|
||||
base) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.yml" ;;
|
||||
tcp) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.tcp.yml" ;;
|
||||
tls) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.tls.yml" ;;
|
||||
ssh) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.ssh.yml" ;;
|
||||
*) die "unsupported mode '${MODE}', expected base|tcp|tls|ssh" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
compose_args() {
|
||||
local -a args files
|
||||
args=()
|
||||
[[ -f "${SCRIPT_DIR}/.env" ]] && args+=(--env-file "${SCRIPT_DIR}/.env")
|
||||
mapfile -t files < <(compose_files)
|
||||
args+=("${files[@]}")
|
||||
printf '%s\n' "${args[@]}"
|
||||
}
|
||||
|
||||
compose_run() {
|
||||
local -a args
|
||||
mapfile -t args < <(compose_args)
|
||||
docker compose "${args[@]}" "$@"
|
||||
}
|
||||
|
||||
up_server() {
|
||||
local -a up_args
|
||||
up_args=(up)
|
||||
[[ "${DETACH}" == "1" ]] && up_args+=(-d)
|
||||
[[ "${BUILD}" == "1" ]] && up_args+=(--build)
|
||||
compose_run "${up_args[@]}"
|
||||
}
|
||||
|
||||
buildx_server() {
|
||||
has_buildx || die "docker buildx is not available"
|
||||
if [[ "${TARGET_PLATFORM}" == *,* && "${BUILDX_PUSH}" != "1" ]]; then
|
||||
die "multi-platform buildx output requires --push"
|
||||
fi
|
||||
|
||||
local -a output_args builder_args
|
||||
[[ "${BUILDX_PUSH}" == "1" ]] && output_args=(--push) || output_args=(--load)
|
||||
[[ -n "${BUILDX_BUILDER}" ]] && builder_args=(--builder "${BUILDX_BUILDER}") || builder_args=()
|
||||
|
||||
pull_base_images
|
||||
|
||||
docker buildx build "${builder_args[@]}" \
|
||||
--platform "${TARGET_PLATFORM}" \
|
||||
-f "${SCRIPT_DIR}/Dockerfile" \
|
||||
-t "${SERVER_IMAGE}" \
|
||||
"${output_args[@]}" \
|
||||
"${REPO_ROOT}"
|
||||
}
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
up|build|buildx|rebuild|down|stop|restart|logs|ps)
|
||||
ACTION="$1"
|
||||
shift
|
||||
;;
|
||||
--mode)
|
||||
[[ $# -ge 2 ]] || die "--mode requires a value"
|
||||
MODE="$2"
|
||||
shift 2
|
||||
;;
|
||||
--platform)
|
||||
[[ $# -ge 2 ]] || die "--platform requires a value"
|
||||
TARGET_PLATFORM="$2"
|
||||
shift 2
|
||||
;;
|
||||
--server-image)
|
||||
[[ $# -ge 2 ]] || die "--server-image requires a value"
|
||||
SERVER_IMAGE="$2"
|
||||
shift 2
|
||||
;;
|
||||
--push)
|
||||
BUILDX_PUSH=1
|
||||
shift
|
||||
;;
|
||||
--builder)
|
||||
[[ $# -ge 2 ]] || die "--builder requires a value"
|
||||
BUILDX_BUILDER="$2"
|
||||
shift 2
|
||||
;;
|
||||
--skip-base-pull)
|
||||
SKIP_BASE_IMAGE_PULL=1
|
||||
shift
|
||||
;;
|
||||
--foreground)
|
||||
DETACH=0
|
||||
shift
|
||||
;;
|
||||
--no-build)
|
||||
BUILD=0
|
||||
shift
|
||||
;;
|
||||
-h|--help)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
die "unknown argument '$1'"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
export RPKI_RTR_SERVER_IMAGE="${SERVER_IMAGE}"
|
||||
if [[ "${ACTION}" == "buildx" ]]; then
|
||||
has_buildx || die "docker buildx is not available"
|
||||
else
|
||||
has_compose || die "docker compose is not available"
|
||||
fi
|
||||
|
||||
cd "${REPO_ROOT}"
|
||||
|
||||
case "${ACTION}" in
|
||||
up)
|
||||
up_server
|
||||
;;
|
||||
build)
|
||||
compose_run build
|
||||
;;
|
||||
buildx)
|
||||
buildx_server
|
||||
;;
|
||||
rebuild)
|
||||
compose_run build --no-cache
|
||||
up_server
|
||||
;;
|
||||
down|stop)
|
||||
compose_run down
|
||||
;;
|
||||
restart)
|
||||
compose_run down
|
||||
up_server
|
||||
;;
|
||||
logs)
|
||||
compose_run logs -f
|
||||
;;
|
||||
ps)
|
||||
compose_run ps
|
||||
;;
|
||||
esac
|
||||
@@ -1,54 +0,0 @@
|
||||
version: "3.9"
|
||||
|
||||
services:
|
||||
rpki-rtr:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: deploy/server/Dockerfile
|
||||
image: ${RPKI_RTR_SERVER_IMAGE:-rpki-rtr:latest}
|
||||
container_name: rpki-rtr-ssh
|
||||
restart: no
|
||||
ports:
|
||||
- "323:323"
|
||||
- "${RPKI_RTR_SSH_HOST_PORT:-2222}:${RPKI_RTR_SSH_CONTAINER_PORT:-22}"
|
||||
- "8323:8323"
|
||||
environment:
|
||||
RPKI_RTR_ENABLE_TLS: "false"
|
||||
RPKI_RTR_ENABLE_SSH: "true"
|
||||
RPKI_RTR_TCP_ADDR: "0.0.0.0:323"
|
||||
RPKI_RTR_SSH_ADDR: "0.0.0.0:${RPKI_RTR_SSH_CONTAINER_PORT:-22}"
|
||||
RPKI_RTR_SSH_HOST_KEY_PATH: "${RPKI_RTR_SSH_HOST_KEY_PATH:-/app/certs/ssh_host_rsa_key}"
|
||||
RPKI_RTR_SSH_AUTHORIZED_KEYS_PATH: "${RPKI_RTR_SSH_AUTHORIZED_KEYS_PATH:-/app/certs/rtr-authorized_keys}"
|
||||
RPKI_RTR_SSH_USERNAME: "${RPKI_RTR_SSH_USERNAME:-rpki-rtr}"
|
||||
RPKI_RTR_SSH_SUBSYSTEM_NAME: "${RPKI_RTR_SSH_SUBSYSTEM_NAME:-rpki-rtr}"
|
||||
# SSH auth mode: key | password | both
|
||||
RPKI_RTR_SSH_AUTH_MODE: "${RPKI_RTR_SSH_AUTH_MODE:-key}"
|
||||
# Optional: enable password authentication in addition to publickey
|
||||
# RPKI_RTR_SSH_PASSWORD: "test-password"
|
||||
RPKI_RTR_DB_PATH: "${RPKI_RTR_DB_PATH:-/app/rtr-db}"
|
||||
RPKI_RTR_CCR_DIR: "${RPKI_RTR_CCR_DIR:-/app/data}"
|
||||
RPKI_RTR_SLURM_DIR: "${RPKI_RTR_SLURM_DIR:-/app/slurm}"
|
||||
RPKI_RTR_REPORT_DIR: "${RPKI_RTR_REPORT_DIR:-/app/report}"
|
||||
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS: "${RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS:-300}"
|
||||
RPKI_RTR_REPORT_HISTORY_LIMIT: "${RPKI_RTR_REPORT_HISTORY_LIMIT:-10}"
|
||||
RPKI_RTR_TIMEZONE: "${RPKI_RTR_TIMEZONE:-Asia/Shanghai}"
|
||||
RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS: "${RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS:-300}"
|
||||
RPKI_RTR_MAX_DELTA: "${RPKI_RTR_MAX_DELTA:-10}"
|
||||
RPKI_RTR_MAX_CONCURRENT_HANDSHAKES: "${RPKI_RTR_MAX_CONCURRENT_HANDSHAKES:-128}"
|
||||
RPKI_RTR_ADMIN_ADDR: "${RPKI_RTR_ADMIN_ADDR:-}"
|
||||
RPKI_RTR_ADMIN_TOKEN: "${RPKI_RTR_ADMIN_TOKEN:-}"
|
||||
RUST_LOG: "${RUST_LOG:-info}"
|
||||
volumes:
|
||||
- ${RPKI_RTR_CCR_HOST_DIR:-../../data}:${RPKI_RTR_CCR_DIR:-/app/data}:ro
|
||||
- ${RPKI_RTR_DB_HOST_DIR:-../../rtr-db}:${RPKI_RTR_DB_PATH:-/app/rtr-db}
|
||||
- ${RPKI_RTR_SLURM_HOST_DIR:-../../data}:${RPKI_RTR_SLURM_DIR:-/app/slurm}
|
||||
- ${RPKI_RTR_SSH_CERTS_HOST_DIR:-../../certs/ssh}:/app/certs:ro
|
||||
- ${RPKI_RTR_LOG_HOST_DIR:-../../logs/server}:/app/logs
|
||||
- ${RPKI_RTR_REPORT_HOST_DIR:-../../report}:${RPKI_RTR_REPORT_DIR:-/app/report}
|
||||
networks:
|
||||
- rpki_net
|
||||
|
||||
networks:
|
||||
rpki_net:
|
||||
name: rpki_net
|
||||
driver: bridge
|
||||
@@ -1,44 +0,0 @@
|
||||
version: "3.9"
|
||||
|
||||
services:
|
||||
rpki-rtr:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: deploy/server/Dockerfile
|
||||
image: ${RPKI_RTR_SERVER_IMAGE:-rpki-rtr:latest}
|
||||
container_name: rpki-rtr-tcp
|
||||
restart: no
|
||||
ports:
|
||||
- "323:323"
|
||||
- "8323:8323"
|
||||
environment:
|
||||
RPKI_RTR_ENABLE_TLS: "false"
|
||||
RPKI_RTR_ENABLE_SSH: "false"
|
||||
RPKI_RTR_TCP_ADDR: "0.0.0.0:323"
|
||||
RPKI_RTR_DB_PATH: "${RPKI_RTR_DB_PATH:-/app/rtr-db}"
|
||||
RPKI_RTR_CCR_DIR: "${RPKI_RTR_CCR_DIR:-/app/data}"
|
||||
RPKI_RTR_SLURM_DIR: "${RPKI_RTR_SLURM_DIR:-/app/slurm}"
|
||||
RPKI_RTR_REPORT_DIR: "${RPKI_RTR_REPORT_DIR:-/app/report}"
|
||||
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS: "${RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS:-300}"
|
||||
RPKI_RTR_REPORT_HISTORY_LIMIT: "${RPKI_RTR_REPORT_HISTORY_LIMIT:-10}"
|
||||
RPKI_RTR_TIMEZONE: "${RPKI_RTR_TIMEZONE:-Asia/Shanghai}"
|
||||
RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS: "${RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS:-60}"
|
||||
RPKI_RTR_MAX_DELTA: "${RPKI_RTR_MAX_DELTA:-10}"
|
||||
RPKI_RTR_MAX_CONNECTIONS: "${RPKI_RTR_MAX_CONNECTIONS:-100000}"
|
||||
RPKI_RTR_MAX_CONCURRENT_HANDSHAKES: "${RPKI_RTR_MAX_CONCURRENT_HANDSHAKES:-128}"
|
||||
RPKI_RTR_ADMIN_ADDR: "${RPKI_RTR_ADMIN_ADDR:-}"
|
||||
RPKI_RTR_ADMIN_TOKEN: "${RPKI_RTR_ADMIN_TOKEN:-}"
|
||||
RUST_LOG: "${RUST_LOG:-info}"
|
||||
volumes:
|
||||
- ${RPKI_RTR_CCR_HOST_DIR:-../../data}:${RPKI_RTR_CCR_DIR:-/app/data}:ro
|
||||
- ${RPKI_RTR_DB_HOST_DIR:-../../rtr-db}:${RPKI_RTR_DB_PATH:-/app/rtr-db}
|
||||
- ${RPKI_RTR_SLURM_HOST_DIR:-../../data}:${RPKI_RTR_SLURM_DIR:-/app/slurm}
|
||||
- ${RPKI_RTR_LOG_HOST_DIR:-../../logs/server}:/app/logs
|
||||
- ${RPKI_RTR_REPORT_HOST_DIR:-../../report}:${RPKI_RTR_REPORT_DIR:-/app/report}
|
||||
networks:
|
||||
- rpki_net
|
||||
|
||||
networks:
|
||||
rpki_net:
|
||||
name: rpki_net
|
||||
driver: bridge
|
||||
@@ -1,48 +0,0 @@
|
||||
services:
|
||||
rpki-rtr:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: deploy/server/Dockerfile
|
||||
image: ${RPKI_RTR_SERVER_IMAGE:-rpki-rtr:latest}
|
||||
container_name: rpki-rtr-tls
|
||||
restart: no
|
||||
ports:
|
||||
# - "323:323"
|
||||
- "324:324"
|
||||
- "8323:8323"
|
||||
environment:
|
||||
RPKI_RTR_ENABLE_TLS: "true"
|
||||
RPKI_RTR_ENABLE_SSH: "false"
|
||||
# RPKI_RTR_TCP_ADDR: "0.0.0.0:323"
|
||||
RPKI_RTR_TLS_ADDR: "0.0.0.0:324"
|
||||
RPKI_RTR_TLS_CERT_PATH: "${RPKI_RTR_TLS_CERT_PATH:-/app/certs/server-dns.crt}"
|
||||
RPKI_RTR_TLS_KEY_PATH: "${RPKI_RTR_TLS_KEY_PATH:-/app/certs/server-dns.key}"
|
||||
RPKI_RTR_TLS_CLIENT_CA_PATH: "${RPKI_RTR_TLS_CLIENT_CA_PATH:-/app/certs/client-ca.crt}"
|
||||
RPKI_RTR_ENFORCE_TLS_CLIENT_SAN_IP_MATCH: "${RPKI_RTR_ENFORCE_TLS_CLIENT_SAN_IP_MATCH:-false}"
|
||||
RPKI_RTR_DB_PATH: "${RPKI_RTR_DB_PATH:-/app/rtr-db}"
|
||||
RPKI_RTR_CCR_DIR: "${RPKI_RTR_CCR_DIR:-/app/data}"
|
||||
RPKI_RTR_SLURM_DIR: "${RPKI_RTR_SLURM_DIR:-/app/slurm}"
|
||||
RPKI_RTR_REPORT_DIR: "${RPKI_RTR_REPORT_DIR:-/app/report}"
|
||||
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS: "${RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS:-300}"
|
||||
RPKI_RTR_REPORT_HISTORY_LIMIT: "${RPKI_RTR_REPORT_HISTORY_LIMIT:-10}"
|
||||
RPKI_RTR_TIMEZONE: "${RPKI_RTR_TIMEZONE:-Asia/Shanghai}"
|
||||
RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS: "${RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS:-300}"
|
||||
RPKI_RTR_MAX_DELTA: "${RPKI_RTR_MAX_DELTA:-10}"
|
||||
RPKI_RTR_MAX_CONCURRENT_HANDSHAKES: "${RPKI_RTR_MAX_CONCURRENT_HANDSHAKES:-128}"
|
||||
RPKI_RTR_ADMIN_ADDR: "${RPKI_RTR_ADMIN_ADDR:-}"
|
||||
RPKI_RTR_ADMIN_TOKEN: "${RPKI_RTR_ADMIN_TOKEN:-}"
|
||||
RUST_LOG: "${RUST_LOG:-info}"
|
||||
volumes:
|
||||
- ${RPKI_RTR_CCR_HOST_DIR:-../../data}:${RPKI_RTR_CCR_DIR:-/app/data}:ro
|
||||
- ${RPKI_RTR_DB_HOST_DIR:-../../rtr-db}:${RPKI_RTR_DB_PATH:-/app/rtr-db}
|
||||
- ${RPKI_RTR_SLURM_HOST_DIR:-../../data}:${RPKI_RTR_SLURM_DIR:-/app/slurm}
|
||||
- ${RPKI_RTR_TLS_CERTS_HOST_DIR:-../../certs/tls}:/app/certs:ro
|
||||
- ${RPKI_RTR_LOG_HOST_DIR:-../../logs/server}:/app/logs
|
||||
- ${RPKI_RTR_REPORT_HOST_DIR:-../../report}:${RPKI_RTR_REPORT_DIR:-/app/report}
|
||||
networks:
|
||||
- rpki_net
|
||||
|
||||
networks:
|
||||
rpki_net:
|
||||
name: rpki_net
|
||||
driver: bridge
|
||||
@@ -1,58 +0,0 @@
|
||||
version: "3.9"
|
||||
|
||||
services:
|
||||
rpki-rtr:
|
||||
build:
|
||||
context: ../..
|
||||
dockerfile: deploy/server/Dockerfile
|
||||
image: ${RPKI_RTR_SERVER_IMAGE:-rpki-rtr:latest}
|
||||
container_name: rpki-rtr
|
||||
restart: no
|
||||
ports:
|
||||
- "323:323"
|
||||
- "324:324"
|
||||
- "8323:8323"
|
||||
# SSH mode example:
|
||||
# - "22:22"
|
||||
environment:
|
||||
RPKI_RTR_ENABLE_TLS: "false"
|
||||
RPKI_RTR_TCP_ADDR: "0.0.0.0:323"
|
||||
RPKI_RTR_TLS_ADDR: "0.0.0.0:324"
|
||||
RPKI_RTR_DB_PATH: "${RPKI_RTR_DB_PATH:-/app/rtr-db}"
|
||||
RPKI_RTR_CCR_DIR: "${RPKI_RTR_CCR_DIR:-/app/data}"
|
||||
RPKI_RTR_SLURM_DIR: "${RPKI_RTR_SLURM_DIR:-/app/slurm}"
|
||||
RPKI_RTR_REPORT_DIR: "${RPKI_RTR_REPORT_DIR:-/app/report}"
|
||||
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS: "${RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS:-300}"
|
||||
RPKI_RTR_REPORT_HISTORY_LIMIT: "${RPKI_RTR_REPORT_HISTORY_LIMIT:-10}"
|
||||
RPKI_RTR_TIMEZONE: "${RPKI_RTR_TIMEZONE:-Asia/Shanghai}"
|
||||
RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS: "${RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS:-300}"
|
||||
RPKI_RTR_MAX_DELTA: "${RPKI_RTR_MAX_DELTA:-10}"
|
||||
RPKI_RTR_MAX_CONCURRENT_HANDSHAKES: "${RPKI_RTR_MAX_CONCURRENT_HANDSHAKES:-128}"
|
||||
RPKI_RTR_ADMIN_ADDR: "${RPKI_RTR_ADMIN_ADDR:-}"
|
||||
RPKI_RTR_ADMIN_TOKEN: "${RPKI_RTR_ADMIN_TOKEN:-}"
|
||||
RUST_LOG: "${RUST_LOG:-info}"
|
||||
# SSH mode example:
|
||||
# RPKI_RTR_ENABLE_SSH: "true"
|
||||
# RPKI_RTR_SSH_ADDR: "0.0.0.0:22"
|
||||
# RPKI_RTR_SSH_PORT: "22"
|
||||
# RPKI_RTR_SSH_HOST_KEY_PATH: "/app/certs/ssh_host_rsa_key"
|
||||
# RPKI_RTR_SSH_AUTHORIZED_KEYS_PATH: "/app/certs/rtr-authorized_keys"
|
||||
# RPKI_RTR_SSH_USERNAME: "rpki-rtr"
|
||||
# RPKI_RTR_SSH_SUBSYSTEM_NAME: "rpki-rtr"
|
||||
# Optional: enable password auth in addition to publickey
|
||||
# RPKI_RTR_SSH_PASSWORD: "test-password"
|
||||
volumes:
|
||||
- ${RPKI_RTR_CCR_HOST_DIR:-../../data}:${RPKI_RTR_CCR_DIR:-/app/data}:ro
|
||||
- ${RPKI_RTR_DB_HOST_DIR:-../../rtr-db}:${RPKI_RTR_DB_PATH:-/app/rtr-db}
|
||||
- ${RPKI_RTR_SLURM_HOST_DIR:-../../data}:${RPKI_RTR_SLURM_DIR:-/app/slurm}
|
||||
- ${RPKI_RTR_LOG_HOST_DIR:-../../logs/server}:/app/logs
|
||||
- ${RPKI_RTR_REPORT_HOST_DIR:-../../report}:${RPKI_RTR_REPORT_DIR:-/app/report}
|
||||
# TLS mode example:
|
||||
# - ../../certs/ssh:/app/certs:ro
|
||||
networks:
|
||||
- rpki_net
|
||||
|
||||
networks:
|
||||
rpki_net:
|
||||
name: rpki_net
|
||||
driver: bridge
|
||||
@@ -1,10 +0,0 @@
|
||||
#!/bin/sh
|
||||
set -eu
|
||||
|
||||
mkdir -p /app/logs
|
||||
|
||||
log_name="${HOSTNAME:-rpki-rtr}"
|
||||
stdout_log="/app/logs/${log_name}.stdout.log"
|
||||
stderr_log="/app/logs/${log_name}.stderr.log"
|
||||
|
||||
exec /usr/local/bin/rpki_rtr "$@" >>"$stdout_log" 2>>"$stderr_log"
|
||||
@@ -1,310 +0,0 @@
|
||||
# RTR Admin API
|
||||
|
||||
本文档描述 RTR server 当前提供的 HTTP 管理接口。
|
||||
|
||||
## 启用方式
|
||||
|
||||
Admin API 默认关闭。设置 `RPKI_RTR_ADMIN_ADDR` 后启用:
|
||||
|
||||
```env
|
||||
RPKI_RTR_ADMIN_ADDR=127.0.0.1:8323
|
||||
RPKI_RTR_ADMIN_TOKEN=change-me
|
||||
```
|
||||
|
||||
如果在 Docker 中需要从宿主机访问,容器内建议监听:
|
||||
|
||||
```env
|
||||
RPKI_RTR_ADMIN_ADDR=0.0.0.0:8323
|
||||
RPKI_RTR_ADMIN_TOKEN=change-me
|
||||
```
|
||||
|
||||
并在 compose 中映射端口:
|
||||
|
||||
```yaml
|
||||
ports:
|
||||
- "8323:8323"
|
||||
```
|
||||
|
||||
安全规则:
|
||||
|
||||
- `RPKI_RTR_ADMIN_ADDR` 为空时,不启动 Admin API。
|
||||
- 非 loopback 地址,例如 `0.0.0.0:8323`,必须配置 `RPKI_RTR_ADMIN_TOKEN`,否则 admin server 会拒绝启动。
|
||||
- 设置 token 后,请求必须带 `Authorization: Bearer <token>`。
|
||||
|
||||
## 通用 Headers
|
||||
|
||||
```http
|
||||
Authorization: Bearer change-me
|
||||
Content-Type: application/json
|
||||
```
|
||||
|
||||
GET 请求不需要 `Content-Type`。
|
||||
|
||||
## GET /admin/rtr/health
|
||||
|
||||
用于检查 Admin API 是否可达,以及当前启用了哪些管理能力。
|
||||
|
||||
```bash
|
||||
curl http://127.0.0.1:8323/admin/rtr/health \
|
||||
-H "Authorization: Bearer change-me"
|
||||
```
|
||||
|
||||
响应示例:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"config_api": true,
|
||||
"source_reload_api": true,
|
||||
"slurm_api": true,
|
||||
"logs_api": true
|
||||
}
|
||||
```
|
||||
|
||||
## GET /admin/rtr/logs/tail
|
||||
|
||||
实时读取 RTR 服务日志,效果类似 `tail -f`。默认读取 stdout 日志,先返回最近 200 行,然后持续输出新增内容。
|
||||
|
||||
```bash
|
||||
curl -N "http://127.0.0.1:8323/admin/rtr/logs/tail?stream=stdout&lines=200&follow=true" \
|
||||
-H "Authorization: Bearer change-me"
|
||||
```
|
||||
|
||||
Query 参数:
|
||||
|
||||
| 参数 | 默认值 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `stream` | `stdout` | 可选 `stdout` 或 `stderr`。 |
|
||||
| `lines` | `200` | 首次返回的历史行数,范围会限制在 `1..=5000`。 |
|
||||
| `follow` | `true` | `true` 时保持连接并持续输出新增日志;`false` 时只返回当前 tail 内容。 |
|
||||
|
||||
日志文件路径按容器 entrypoint 的规则解析:
|
||||
|
||||
- 目录:`RPKI_RTR_LOG_DIR`,默认 `/app/logs`。
|
||||
- 文件名前缀:`RPKI_RTR_LOG_NAME`,未设置时使用 `HOSTNAME`,再未设置时为 `rpki-rtr`。
|
||||
- stdout 文件:`<name>.stdout.log`。
|
||||
- stderr 文件:`<name>.stderr.log`。
|
||||
|
||||
## GET /admin/rtr/config
|
||||
|
||||
查询当前运行中的 runtime 配置。
|
||||
|
||||
```bash
|
||||
curl http://127.0.0.1:8323/admin/rtr/config \
|
||||
-H "Authorization: Bearer change-me"
|
||||
```
|
||||
|
||||
响应示例:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"config": {
|
||||
"max_delta": 10,
|
||||
"prune_delta_by_snapshot_size": true,
|
||||
"source_refresh_interval_seconds": 300,
|
||||
"runtime_report_interval_seconds": 300,
|
||||
"report_history_limit": 10,
|
||||
"timezone": "Asia/Shanghai",
|
||||
"timing": {
|
||||
"refresh": 3600,
|
||||
"retry": 600,
|
||||
"expire": 7200
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## POST /admin/rtr/config
|
||||
|
||||
运行中动态修改 RTR server 的部分 runtime 配置。请求体是 JSON object,支持部分更新,只需要传要修改的字段。
|
||||
|
||||
| 字段 | 类型 | 说明 |
|
||||
| --- | --- | --- |
|
||||
| `max_delta` | integer | 每个 RTR 版本最多保留的 delta 条数,必须 `>= 1`。 |
|
||||
| `prune_delta_by_snapshot_size` | boolean | 是否按 snapshot wire size 裁剪 delta window。 |
|
||||
| `source_refresh_interval_seconds` | integer | CCR/SLURM source refresh 间隔,单位秒,必须 `>= 1`。 |
|
||||
| `runtime_report_interval_seconds` | integer | `rtr-runtime-*.json` 周期写入间隔,单位秒,必须 `>= 1`。 |
|
||||
| `report_history_limit` | integer | 每类 report 文件滚动保留数量,必须 `>= 1`。 |
|
||||
| `timezone` | string | IANA 时区名,例如 `Asia/Shanghai`、`UTC`、`Europe/London`。 |
|
||||
| `timing.refresh` | integer | RTR EndOfData `refresh`,单位秒。 |
|
||||
| `timing.retry` | integer | RTR EndOfData `retry`,单位秒。 |
|
||||
| `timing.expire` | integer | RTR EndOfData `expire`,必须大于 `refresh` 和 `retry`。 |
|
||||
|
||||
示例:
|
||||
|
||||
```bash
|
||||
curl -X POST http://127.0.0.1:8323/admin/rtr/config \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer change-me" \
|
||||
-d '{"max_delta": 6}'
|
||||
```
|
||||
|
||||
成功响应:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"config": {
|
||||
"max_delta": 6,
|
||||
"prune_delta_by_snapshot_size": true,
|
||||
"source_refresh_interval_seconds": 300,
|
||||
"runtime_report_interval_seconds": 300,
|
||||
"report_history_limit": 10,
|
||||
"timezone": "Asia/Shanghai",
|
||||
"timing": {
|
||||
"refresh": 3600,
|
||||
"retry": 600,
|
||||
"expire": 7200
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## GET /admin/rtr/slurm/files
|
||||
|
||||
列出 `RPKI_RTR_SLURM_DIR` 下的 SLURM 文件。只返回合法的 `*.slurm` 和 `*.slurm.disabled` 文件,不返回 `.backup/`。
|
||||
|
||||
```bash
|
||||
curl http://127.0.0.1:8323/admin/rtr/slurm/files \
|
||||
-H "Authorization: Bearer change-me"
|
||||
```
|
||||
|
||||
响应示例:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"files": [
|
||||
{
|
||||
"name": "local-policy.slurm",
|
||||
"path": "/app/slurm/local-policy.slurm",
|
||||
"enabled": true,
|
||||
"size_bytes": 168,
|
||||
"modified_unix_seconds": 1782100000
|
||||
},
|
||||
{
|
||||
"name": "old-policy.slurm.disabled",
|
||||
"path": "/app/slurm/old-policy.slurm.disabled",
|
||||
"enabled": false,
|
||||
"size_bytes": 168,
|
||||
"modified_unix_seconds": 1782100100
|
||||
}
|
||||
]
|
||||
}
|
||||
```
|
||||
|
||||
## GET /admin/rtr/slurm/files/{name}
|
||||
|
||||
读取单个 SLURM 文件,前端编辑页面可以直接使用这个接口加载内容。
|
||||
|
||||
```bash
|
||||
curl http://127.0.0.1:8323/admin/rtr/slurm/files/local-policy.slurm \
|
||||
-H "Authorization: Bearer change-me"
|
||||
```
|
||||
|
||||
响应示例:
|
||||
|
||||
```json
|
||||
{
|
||||
"status": "ok",
|
||||
"file": {
|
||||
"name": "local-policy.slurm",
|
||||
"path": "/app/slurm/local-policy.slurm",
|
||||
"enabled": true,
|
||||
"size_bytes": 168,
|
||||
"modified_unix_seconds": 1782100000,
|
||||
"content": "{ \"slurmVersion\": 1, \"validationOutputFilters\": { \"prefixFilters\": [], \"bgpsecFilters\": [] }, \"locallyAddedAssertions\": { \"prefixAssertions\": [], \"bgpsecAssertions\": [] } }"
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
## POST /admin/rtr/slurm/files
|
||||
|
||||
新增或覆盖一个 SLURM 文件。文件名放在请求体中。
|
||||
|
||||
```json
|
||||
{
|
||||
"name": "local-policy.slurm",
|
||||
"content": "{ \"slurmVersion\": 1, \"validationOutputFilters\": { \"prefixFilters\": [], \"bgpsecFilters\": [] }, \"locallyAddedAssertions\": { \"prefixAssertions\": [], \"bgpsecAssertions\": [] } }",
|
||||
"reload": true
|
||||
}
|
||||
```
|
||||
|
||||
## PUT /admin/rtr/slurm/files/{name}
|
||||
|
||||
新增或覆盖指定 SLURM 文件。
|
||||
|
||||
```bash
|
||||
curl -X PUT http://127.0.0.1:8323/admin/rtr/slurm/files/local-policy.slurm \
|
||||
-H "Content-Type: application/json" \
|
||||
-H "Authorization: Bearer change-me" \
|
||||
-d '{
|
||||
"content": "{ \"slurmVersion\": 1, \"validationOutputFilters\": { \"prefixFilters\": [], \"bgpsecFilters\": [] }, \"locallyAddedAssertions\": { \"prefixAssertions\": [], \"bgpsecAssertions\": [] } }",
|
||||
"reload": true
|
||||
}'
|
||||
```
|
||||
|
||||
## DELETE /admin/rtr/slurm/files/{name}
|
||||
|
||||
删除指定 SLURM 文件。删除前会先备份。
|
||||
|
||||
```bash
|
||||
curl -X DELETE "http://127.0.0.1:8323/admin/rtr/slurm/files/local-policy.slurm?reload=true" \
|
||||
-H "Authorization: Bearer change-me"
|
||||
```
|
||||
|
||||
## POST /admin/rtr/slurm/files/{name}/disable
|
||||
|
||||
将启用文件改名为失效文件:
|
||||
|
||||
```text
|
||||
local-policy.slurm -> local-policy.slurm.disabled
|
||||
```
|
||||
|
||||
```bash
|
||||
curl -X POST "http://127.0.0.1:8323/admin/rtr/slurm/files/local-policy.slurm/disable?reload=true" \
|
||||
-H "Authorization: Bearer change-me"
|
||||
```
|
||||
|
||||
## POST /admin/rtr/slurm/files/{name}/enable
|
||||
|
||||
将失效文件改名为启用文件:
|
||||
|
||||
```text
|
||||
local-policy.slurm.disabled -> local-policy.slurm
|
||||
```
|
||||
|
||||
```bash
|
||||
curl -X POST "http://127.0.0.1:8323/admin/rtr/slurm/files/local-policy.slurm.disabled/enable?reload=true" \
|
||||
-H "Authorization: Bearer change-me"
|
||||
```
|
||||
|
||||
## POST /admin/rtr/slurm/reload
|
||||
|
||||
不修改文件,只立即触发一次 CCR/SLURM source reload。
|
||||
|
||||
```bash
|
||||
curl -X POST http://127.0.0.1:8323/admin/rtr/slurm/reload \
|
||||
-H "Authorization: Bearer change-me"
|
||||
```
|
||||
|
||||
## SLURM 文件规则
|
||||
|
||||
- `*.slurm`:启用,会被 source refresh 加载。
|
||||
- `*.slurm.disabled`:失效,不会被加载。
|
||||
- `.backup/`:接口自动生成的备份目录。
|
||||
- 文件名只能是 basename,不能包含 `/`、`\` 或 `..`。
|
||||
- 文件名只能包含 ASCII 字母、数字、`.`、`_`、`-`。
|
||||
- 写入内容必须是合法 JSON,并且能被解析为合法 SLURM 文件。
|
||||
- 单个请求体默认限制为 5 MiB。
|
||||
- 覆盖、删除前会写入 `.backup/` 备份。
|
||||
|
||||
## 错误响应
|
||||
|
||||
| HTTP 状态码 | 含义 |
|
||||
| --- | --- |
|
||||
| `400 Bad Request` | JSON 非法、配置值非法、SLURM 文件名非法、目标文件不存在、reload 失败等。 |
|
||||
| `401 Unauthorized` | 配置了 token,但请求缺少或使用了错误的 `Authorization`。 |
|
||||
| `404 Not Found` | 路径或方法不支持。 |
|
||||
| `413 Payload Too Large` | 请求体超过限制。 |
|
||||
@@ -1,203 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||
IMAGE_DIR="${ROOT_DIR}/images"
|
||||
SERVER_ENV_FILE="${ROOT_DIR}/deploy/server/.env"
|
||||
CONSOLE_ENV_FILE="${ROOT_DIR}/deploy/console/.env"
|
||||
|
||||
SERVER_TAR="${RPKI_RTR_SERVER_IMAGE_TAR:-}"
|
||||
CLIENT_TAR="${RPKI_RTR_CLIENT_IMAGE_TAR:-}"
|
||||
CONSOLE_TAR="${RPKI_RTR_CONSOLE_IMAGE_TAR:-}"
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage: scripts/00_0_install.sh [options]
|
||||
|
||||
Load offline Docker image tarballs from ./images, then run scripts/00_1_pre_check.sh.
|
||||
|
||||
Options:
|
||||
--server-tar FILE Server image tarball. Env: RPKI_RTR_SERVER_IMAGE_TAR
|
||||
--client-tar FILE Debug client image tarball. Env: RPKI_RTR_CLIENT_IMAGE_TAR
|
||||
--console-tar FILE Console image tarball. Env: RPKI_RTR_CONSOLE_IMAGE_TAR
|
||||
-h, --help Show this help.
|
||||
|
||||
Default tarballs under ./images:
|
||||
server: rpki-rtr-*.tar, excluding rpki-rtr-debug-client-*.tar
|
||||
client: rpki-rtr-debug-client-*.tar
|
||||
console: rpki-console-ui-*.tar
|
||||
EOF
|
||||
}
|
||||
|
||||
log() {
|
||||
printf '[00_install] %s\n' "$*" >&2
|
||||
}
|
||||
|
||||
stage() {
|
||||
printf '\n====== %s ======\n' "$*" >&2
|
||||
}
|
||||
|
||||
die() {
|
||||
printf '[00_install] ERROR: %s\n' "$*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
set_env_var() {
|
||||
local env_file="$1"
|
||||
local key="$2"
|
||||
local value="$3"
|
||||
local tmp_file
|
||||
|
||||
[ -f "${env_file}" ] || die "env file not found: ${env_file}"
|
||||
tmp_file="${env_file}.tmp.$$"
|
||||
|
||||
awk -v key="${key}" -v value="${value}" '
|
||||
BEGIN { done = 0 }
|
||||
$0 ~ "^[[:space:]]*#?[[:space:]]*" key "=" {
|
||||
if (!done) {
|
||||
print key "=" value
|
||||
done = 1
|
||||
}
|
||||
next
|
||||
}
|
||||
{ print }
|
||||
END {
|
||||
if (!done) {
|
||||
print key "=" value
|
||||
}
|
||||
}
|
||||
' "${env_file}" >"${tmp_file}"
|
||||
mv "${tmp_file}" "${env_file}"
|
||||
}
|
||||
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
--server-tar)
|
||||
[ "$#" -ge 2 ] || die "--server-tar requires a value"
|
||||
SERVER_TAR="$2"
|
||||
shift 2
|
||||
;;
|
||||
--client-tar)
|
||||
[ "$#" -ge 2 ] || die "--client-tar requires a value"
|
||||
CLIENT_TAR="$2"
|
||||
shift 2
|
||||
;;
|
||||
--console-tar)
|
||||
[ "$#" -ge 2 ] || die "--console-tar requires a value"
|
||||
CONSOLE_TAR="$2"
|
||||
shift 2
|
||||
;;
|
||||
-h|--help)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
die "unknown option: $1"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
resolve_tar() {
|
||||
local path="$1"
|
||||
|
||||
if [ -f "${path}" ]; then
|
||||
cd "$(dirname "${path}")" && printf '%s/%s\n' "$(pwd -P)" "$(basename "${path}")"
|
||||
return 0
|
||||
fi
|
||||
|
||||
if [ -f "${ROOT_DIR}/${path}" ]; then
|
||||
cd "$(dirname "${ROOT_DIR}/${path}")" && printf '%s/%s\n' "$(pwd -P)" "$(basename "${path}")"
|
||||
return 0
|
||||
fi
|
||||
|
||||
return 1
|
||||
}
|
||||
|
||||
load_image() {
|
||||
local label="$1"
|
||||
local tar_path="$2"
|
||||
local output image
|
||||
|
||||
[ -f "${tar_path}" ] || die "${label} image tarball not found: ${tar_path}"
|
||||
output="$(docker load -i "${tar_path}" 2>&1)" || {
|
||||
printf '%s\n' "${output}" >&2
|
||||
die "failed to load ${label} image from ${tar_path}"
|
||||
}
|
||||
printf '%s\n' "${output}" >&2
|
||||
|
||||
image="$(printf '%s\n' "${output}" | sed -n 's/^Loaded image: //p' | tail -n 1)"
|
||||
[ -n "${image}" ] || die "cannot determine ${label} image name from docker load output"
|
||||
printf '%s\n' "${image}"
|
||||
}
|
||||
|
||||
find_single_tar() {
|
||||
local label="$1"
|
||||
local include_pattern="$2"
|
||||
local exclude_pattern="${3:-}"
|
||||
local matches=()
|
||||
local file
|
||||
|
||||
while IFS= read -r -d '' file; do
|
||||
matches+=("${file}")
|
||||
done < <(
|
||||
if [ -n "${exclude_pattern}" ]; then
|
||||
find "${IMAGE_DIR}" -maxdepth 1 -type f -name "${include_pattern}" ! -name "${exclude_pattern}" -print0 | sort -z
|
||||
else
|
||||
find "${IMAGE_DIR}" -maxdepth 1 -type f -name "${include_pattern}" -print0 | sort -z
|
||||
fi
|
||||
)
|
||||
|
||||
case "${#matches[@]}" in
|
||||
0)
|
||||
die "no ${label} image tarball found under ${IMAGE_DIR} matching ${include_pattern}"
|
||||
;;
|
||||
1)
|
||||
printf '%s\n' "${matches[0]}"
|
||||
;;
|
||||
*)
|
||||
printf '[00_install] ERROR: multiple %s image tarballs found:\n' "${label}" >&2
|
||||
printf ' %s\n' "${matches[@]}" >&2
|
||||
die "please specify --${label}-tar or RPKI_RTR_${label^^}_IMAGE_TAR"
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
stage "Checking Docker"
|
||||
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
||||
[ -d "${IMAGE_DIR}" ] || die "image directory not found: ${IMAGE_DIR}"
|
||||
[ -f "${SERVER_ENV_FILE}" ] || die "server env file not found: ${SERVER_ENV_FILE}"
|
||||
|
||||
if [ -n "${SERVER_TAR}" ]; then
|
||||
SERVER_TAR="$(resolve_tar "${SERVER_TAR}")" || die "server image tarball not found: ${SERVER_TAR}"
|
||||
else
|
||||
SERVER_TAR="$(find_single_tar server 'rpki-rtr-*.tar' 'rpki-rtr-debug-client-*.tar')"
|
||||
fi
|
||||
|
||||
if [ -n "${CLIENT_TAR}" ]; then
|
||||
CLIENT_TAR="$(resolve_tar "${CLIENT_TAR}")" || die "client image tarball not found: ${CLIENT_TAR}"
|
||||
else
|
||||
CLIENT_TAR="$(find_single_tar client 'rpki-rtr-debug-client-*.tar')"
|
||||
fi
|
||||
|
||||
if [ -n "${CONSOLE_TAR}" ]; then
|
||||
CONSOLE_TAR="$(resolve_tar "${CONSOLE_TAR}")" || die "console image tarball not found: ${CONSOLE_TAR}"
|
||||
else
|
||||
CONSOLE_TAR="$(find_single_tar console 'rpki-console-ui-*.tar')"
|
||||
fi
|
||||
|
||||
stage "Loading Docker Images"
|
||||
SERVER_IMAGE="$(load_image server "${SERVER_TAR}")"
|
||||
CLIENT_IMAGE="$(load_image client "${CLIENT_TAR}")"
|
||||
CONSOLE_IMAGE="$(load_image console "${CONSOLE_TAR}")"
|
||||
|
||||
stage "Updating Deploy Env"
|
||||
set_env_var "${SERVER_ENV_FILE}" RPKI_RTR_SERVER_IMAGE "${SERVER_IMAGE}"
|
||||
set_env_var "${CONSOLE_ENV_FILE}" RPKI_CONSOLE_UI_IMAGE "${CONSOLE_IMAGE}"
|
||||
|
||||
stage "Running Pre-Check"
|
||||
RPKI_RTR_SERVER_IMAGE="${SERVER_IMAGE}" \
|
||||
RPKI_RTR_CLIENT_IMAGE="${CLIENT_IMAGE}" \
|
||||
bash "${SCRIPT_DIR}/00_1_pre_check.sh" || die "pre-check failed"
|
||||
|
||||
stage "Install Finished"
|
||||
@@ -1,155 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||
|
||||
PRECHECK_WORK_DIR="${PRECHECK_WORK_DIR:-${ROOT_DIR}/.precheck}"
|
||||
PRECHECK_ENV_FILE="${PRECHECK_ENV_FILE:-${PRECHECK_WORK_DIR}/precheck.env}"
|
||||
PRECHECK_WAIT_SECS="${PRECHECK_WAIT_SECS:-60}"
|
||||
KEEP_PRECHECK="${KEEP_PRECHECK:-0}"
|
||||
|
||||
log() {
|
||||
printf '[00_1_pre_check] %s\n' "$*"
|
||||
}
|
||||
|
||||
stage() {
|
||||
printf '\n====== %s ======\n' "$*"
|
||||
}
|
||||
|
||||
die() {
|
||||
printf '[00_1_pre_check] ERROR: %s\n' "$*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
||||
if docker compose version >/dev/null 2>&1; then
|
||||
COMPOSE_BIN=(docker compose)
|
||||
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
||||
COMPOSE_BIN=(docker-compose)
|
||||
else
|
||||
die "docker compose is not available"
|
||||
fi
|
||||
|
||||
stage "Preparing Pre-Check Data"
|
||||
bash "${SCRIPT_DIR}/00_2_pre_check_prepare.sh" || die "prepare step failed"
|
||||
|
||||
set -a
|
||||
# shellcheck disable=SC1090
|
||||
. "${PRECHECK_ENV_FILE}"
|
||||
set +a
|
||||
|
||||
ADMIN_PORT="${RPKI_RTR_ADMIN_ADDR##*:}"
|
||||
ADMIN_TOKEN="${RPKI_RTR_ADMIN_TOKEN}"
|
||||
|
||||
server_compose() {
|
||||
"${COMPOSE_BIN[@]}" --env-file "${PRECHECK_ENV_FILE}" -p "${PRECHECK_PROJECT}" -f "${PRECHECK_SERVER_COMPOSE_FILE}" "$@"
|
||||
}
|
||||
|
||||
client_compose() {
|
||||
"${COMPOSE_BIN[@]}" --env-file "${PRECHECK_ENV_FILE}" -p "${PRECHECK_PROJECT}" -f "${PRECHECK_CLIENT_COMPOSE_FILE}" "$@"
|
||||
}
|
||||
|
||||
cleanup() {
|
||||
KEEP_PRECHECK="${KEEP_PRECHECK}" PRECHECK_WORK_DIR="${PRECHECK_WORK_DIR}" PRECHECK_ENV_FILE="${PRECHECK_ENV_FILE}" \
|
||||
bash "${SCRIPT_DIR}/00_3_pre_check_cleanup.sh"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
http_get() {
|
||||
local host="$1"
|
||||
local port="$2"
|
||||
local path="$3"
|
||||
local token="$4"
|
||||
|
||||
exec 3<>"/dev/tcp/${host}/${port}" || return 1
|
||||
printf 'GET %s HTTP/1.1\r\nHost: %s:%s\r\nAuthorization: Bearer %s\r\nConnection: close\r\n\r\n' \
|
||||
"${path}" "${host}" "${port}" "${token}" >&3
|
||||
cat <&3
|
||||
exec 3>&-
|
||||
}
|
||||
|
||||
wait_for_admin_health() {
|
||||
local deadline=$((SECONDS + PRECHECK_WAIT_SECS))
|
||||
local response
|
||||
|
||||
log "waiting for server health"
|
||||
while [ "${SECONDS}" -lt "${deadline}" ]; do
|
||||
response="$(http_get 127.0.0.1 "${ADMIN_PORT}" /admin/rtr/health "${ADMIN_TOKEN}" 2>/dev/null || true)"
|
||||
if printf '%s' "${response}" | grep -q '"status"[[:space:]]*:[[:space:]]*"ok"'; then
|
||||
log "admin health check passed"
|
||||
return 0
|
||||
fi
|
||||
|
||||
if [ "$(server_compose ps -q rpki-rtr | wc -l | tr -d ' ')" = "0" ]; then
|
||||
server_compose logs rpki-rtr || true
|
||||
die "server container is not running"
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
|
||||
server_compose logs rpki-rtr || true
|
||||
die "admin health check did not pass within ${PRECHECK_WAIT_SECS}s"
|
||||
}
|
||||
|
||||
wait_for_client_eod() {
|
||||
local deadline=$((SECONDS + PRECHECK_WAIT_SECS))
|
||||
local log_file
|
||||
|
||||
log "waiting for client response"
|
||||
while [ "${SECONDS}" -lt "${deadline}" ]; do
|
||||
for log_file in "${PRECHECK_WORK_DIR}"/logs/client/*.stdout.log; do
|
||||
[ -f "${log_file}" ] || continue
|
||||
if grep -q 'EndOfData:' "${log_file}" && grep -q 'summary[[:space:]]*:' "${log_file}"; then
|
||||
log "client received EndOfData and payload summary"
|
||||
return 0
|
||||
fi
|
||||
if grep -qi 'Error Report' "${log_file}"; then
|
||||
cat "${log_file}" >&2
|
||||
die "client received RTR Error Report"
|
||||
fi
|
||||
done
|
||||
sleep 1
|
||||
done
|
||||
|
||||
find "${PRECHECK_WORK_DIR}/logs/client" -type f -maxdepth 1 -print -exec cat {} \; >&2 || true
|
||||
die "client did not receive EndOfData and summary within ${PRECHECK_WAIT_SECS}s"
|
||||
}
|
||||
|
||||
assert_no_error_logs() {
|
||||
local bad
|
||||
log "checking logs"
|
||||
bad="$(find "${PRECHECK_WORK_DIR}/logs" -type f -name '*.log' -print0 \
|
||||
| xargs -0 grep -Ein 'panic|failed|RTR admin config server exited|Error Report' 2>/dev/null || true)"
|
||||
if [ -n "${bad}" ]; then
|
||||
printf '%s\n' "${bad}" >&2
|
||||
die "error pattern found in pre-check logs"
|
||||
fi
|
||||
}
|
||||
|
||||
assert_report_written() {
|
||||
local deadline=$((SECONDS + PRECHECK_WAIT_SECS))
|
||||
|
||||
log "waiting for source report"
|
||||
while [ "${SECONDS}" -lt "${deadline}" ]; do
|
||||
if find "${PRECHECK_WORK_DIR}/report" -type f -name 'rtr-source-*.json' | grep -q .; then
|
||||
log "source report written"
|
||||
return 0
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
|
||||
die "server did not write rtr-source report within ${PRECHECK_WAIT_SECS}s"
|
||||
}
|
||||
|
||||
stage "Starting RTR Server"
|
||||
server_compose up -d --no-build rpki-rtr
|
||||
wait_for_admin_health
|
||||
assert_report_written
|
||||
|
||||
stage "Starting Debug Client"
|
||||
client_compose up -d --no-build rtr-debug-client
|
||||
wait_for_client_eod
|
||||
|
||||
assert_no_error_logs
|
||||
stage "Pre-Check Passed"
|
||||
@@ -1,54 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
OUT_PATH="${1:-}"
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage: scripts/00_2_generate_ccr.sh OUT.ccr
|
||||
|
||||
Generate a small deterministic CCR snapshot for pre-check.
|
||||
This script writes a fixed base64-encoded CCR fixture and does not require Python.
|
||||
The generated snapshot contains:
|
||||
- VRP: 10.0.0.0/24 maxLen 24 AS65001
|
||||
- VRP: 10.0.1.0/24 maxLen 24 AS65003
|
||||
- VRP: 2001:db8:1::/48 maxLen 48 AS65002
|
||||
- ASPA: customer AS65010 providers AS65011, AS65012, AS65013
|
||||
- ASPA: customer AS65020 provider AS65021
|
||||
EOF
|
||||
}
|
||||
|
||||
die() {
|
||||
printf '[00_2_generate_ccr] ERROR: %s\n' "$*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
if [ "${OUT_PATH}" = "-h" ] || [ "${OUT_PATH}" = "--help" ]; then
|
||||
usage
|
||||
exit 0
|
||||
fi
|
||||
|
||||
[ -n "${OUT_PATH}" ] || die "missing OUT.ccr"
|
||||
mkdir -p "$(dirname "${OUT_PATH}")"
|
||||
|
||||
decode_base64() {
|
||||
if command -v base64 >/dev/null 2>&1; then
|
||||
base64 -d
|
||||
return
|
||||
fi
|
||||
|
||||
if command -v openssl >/dev/null 2>&1; then
|
||||
openssl base64 -d
|
||||
return
|
||||
fi
|
||||
|
||||
die "base64 or openssl command is required to write CCR fixture"
|
||||
}
|
||||
|
||||
cat <<'EOF' | decode_base64 >"${OUT_PATH}"
|
||||
MIIBAwYLKoZIhvcNAQkQATaggfMwgfACAQAwDQYJYIZIAWUDBAIBBQAYDzIwMjYwNDAzMDAwMjAx
|
||||
WqJ9MHswVzAaAgMA/ekwEzARBAIAATALMAkDBAAKAAACARgwGgIDAP3rMBMwEQQCAAEwCzAJAwQA
|
||||
CgABAgEYMB0CAwD96jAWMBQEAgACMA4wDAMHACABDbgAAQIBMAQgsUgCHfy5AJcFp5RymOD0HuGO
|
||||
wfhoWpll90ugYECGQPSjTDBKMCYwFgIDAP3yMA8CAwD98wIDAP30AgMA/fUwDAIDAP38MAUCAwD9
|
||||
/QQgRPxVPOOUsUuuV4tZ8BG5PHNtKiaEr23Ird+yNxlWJ+w=
|
||||
EOF
|
||||
@@ -1,96 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||
|
||||
PRECHECK_WORK_DIR="${PRECHECK_WORK_DIR:-${ROOT_DIR}/.precheck}"
|
||||
PRECHECK_PROJECT="${PRECHECK_PROJECT:-rpki-precheck}"
|
||||
PRECHECK_ADMIN_TOKEN="${PRECHECK_ADMIN_TOKEN:-rpki-precheck-token}"
|
||||
PRECHECK_PROTOCOL_VERSION="${PRECHECK_PROTOCOL_VERSION:-2}"
|
||||
PRECHECK_SERVER_IMAGE="${PRECHECK_SERVER_IMAGE:-${RPKI_RTR_SERVER_IMAGE:-rpki-rtr-arm64:v1.2}}"
|
||||
PRECHECK_CLIENT_IMAGE="${PRECHECK_CLIENT_IMAGE:-${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client-arm64:v1.2}}"
|
||||
|
||||
log() {
|
||||
printf '[00_1_pre_check_prepare] %s\n' "$*"
|
||||
}
|
||||
|
||||
die() {
|
||||
printf '[00_1_pre_check_prepare] ERROR: %s\n' "$*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
abs_dir() {
|
||||
local path="$1"
|
||||
mkdir -p "${path}"
|
||||
cd "${path}" && pwd -P
|
||||
}
|
||||
|
||||
[ -f "${SCRIPT_DIR}/00_2_generate_ccr.sh" ] || die "missing ${SCRIPT_DIR}/00_2_generate_ccr.sh"
|
||||
|
||||
rm -rf "${PRECHECK_WORK_DIR}"
|
||||
mkdir -p \
|
||||
"${PRECHECK_WORK_DIR}/ccr" \
|
||||
"${PRECHECK_WORK_DIR}/slurm" \
|
||||
"${PRECHECK_WORK_DIR}/db" \
|
||||
"${PRECHECK_WORK_DIR}/logs/server" \
|
||||
"${PRECHECK_WORK_DIR}/logs/client" \
|
||||
"${PRECHECK_WORK_DIR}/report"
|
||||
|
||||
bash "${SCRIPT_DIR}/00_2_generate_ccr.sh" \
|
||||
"${PRECHECK_WORK_DIR}/ccr/20260403T000201Z-precheck.ccr"
|
||||
|
||||
cat >"${PRECHECK_WORK_DIR}/slurm/precheck.slurm" <<'EOF'
|
||||
{
|
||||
"slurmVersion": 2,
|
||||
"validationOutputFilters": {
|
||||
"prefixFilters": [],
|
||||
"bgpsecFilters": [],
|
||||
"aspaFilters": []
|
||||
},
|
||||
"locallyAddedAssertions": {
|
||||
"prefixAssertions": [
|
||||
{
|
||||
"prefix": "203.0.113.0/24",
|
||||
"maxPrefixLength": 24,
|
||||
"asn": 64496,
|
||||
"comment": "pre-check local assertion"
|
||||
}
|
||||
],
|
||||
"bgpsecAssertions": [],
|
||||
"aspaAssertions": []
|
||||
}
|
||||
}
|
||||
EOF
|
||||
|
||||
PRECHECK_WORK_DIR_ABS="$(abs_dir "${PRECHECK_WORK_DIR}")"
|
||||
cat >"${PRECHECK_WORK_DIR}/precheck.env" <<EOF
|
||||
PRECHECK_WORK_DIR=${PRECHECK_WORK_DIR_ABS}
|
||||
PRECHECK_PROJECT=${PRECHECK_PROJECT}
|
||||
RPKI_RTR_SERVER_IMAGE=${PRECHECK_SERVER_IMAGE}
|
||||
RPKI_RTR_CLIENT_IMAGE=${PRECHECK_CLIENT_IMAGE}
|
||||
PRECHECK_SERVER_COMPOSE_FILE=${ROOT_DIR}/deploy/server/docker-compose.tcp.yml
|
||||
PRECHECK_CLIENT_COMPOSE_FILE=${ROOT_DIR}/deploy/client/docker-compose.tcp.yml
|
||||
RPKI_RTR_CCR_HOST_DIR=${PRECHECK_WORK_DIR_ABS}/ccr
|
||||
RPKI_RTR_SLURM_HOST_DIR=${PRECHECK_WORK_DIR_ABS}/slurm
|
||||
RPKI_RTR_DB_HOST_DIR=${PRECHECK_WORK_DIR_ABS}/db
|
||||
RPKI_RTR_LOG_HOST_DIR=${PRECHECK_WORK_DIR_ABS}/logs/server
|
||||
RPKI_RTR_CLIENT_LOG_HOST_DIR=${PRECHECK_WORK_DIR_ABS}/logs/client
|
||||
RPKI_RTR_REPORT_HOST_DIR=${PRECHECK_WORK_DIR_ABS}/report
|
||||
RPKI_RTR_CCR_DIR=/app/data
|
||||
RPKI_RTR_SLURM_DIR=/app/slurm
|
||||
RPKI_RTR_DB_PATH=/app/rtr-db
|
||||
RPKI_RTR_REPORT_DIR=/app/report
|
||||
RPKI_RTR_ADMIN_ADDR=0.0.0.0:8323
|
||||
RPKI_RTR_ADMIN_TOKEN=${PRECHECK_ADMIN_TOKEN}
|
||||
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS=5
|
||||
RPKI_RTR_REPORT_HISTORY_LIMIT=3
|
||||
RPKI_RTR_TIMEZONE=Asia/Shanghai
|
||||
RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS=30
|
||||
RPKI_RTR_MAX_DELTA=10
|
||||
RPKI_RTR_MAX_CONNECTIONS=32
|
||||
RPKI_RTR_MAX_CONCURRENT_HANDSHAKES=8
|
||||
RPKI_RTR_TCP_SERVER_ADDR=rpki-rtr:323
|
||||
RPKI_RTR_PROTOCOL_VERSION=${PRECHECK_PROTOCOL_VERSION}
|
||||
RUST_LOG=info
|
||||
EOF
|
||||
@@ -1,56 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||
|
||||
PRECHECK_WORK_DIR="${PRECHECK_WORK_DIR:-${ROOT_DIR}/.precheck}"
|
||||
ENV_FILE="${PRECHECK_ENV_FILE:-${PRECHECK_WORK_DIR}/precheck.env}"
|
||||
KEEP_PRECHECK="${KEEP_PRECHECK:-0}"
|
||||
|
||||
if [ -f "${ENV_FILE}" ]; then
|
||||
set -a
|
||||
# shellcheck disable=SC1090
|
||||
. "${ENV_FILE}"
|
||||
set +a
|
||||
fi
|
||||
|
||||
PRECHECK_PROJECT="${PRECHECK_PROJECT:-rpki-precheck}"
|
||||
PRECHECK_SERVER_COMPOSE_FILE="${PRECHECK_SERVER_COMPOSE_FILE:-${ROOT_DIR}/deploy/server/docker-compose.tcp.yml}"
|
||||
PRECHECK_CLIENT_COMPOSE_FILE="${PRECHECK_CLIENT_COMPOSE_FILE:-${ROOT_DIR}/deploy/client/docker-compose.tcp.yml}"
|
||||
COMPOSE_BIN=()
|
||||
|
||||
log() {
|
||||
printf '[00_1_pre_check_cleanup] %s\n' "$*"
|
||||
}
|
||||
|
||||
compose_run() {
|
||||
local compose_file="$1"
|
||||
shift
|
||||
|
||||
if [ "${#COMPOSE_BIN[@]}" -eq 0 ]; then
|
||||
if docker compose version >/dev/null 2>&1; then
|
||||
COMPOSE_BIN=(docker compose)
|
||||
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
||||
COMPOSE_BIN=(docker-compose)
|
||||
else
|
||||
return 0
|
||||
fi
|
||||
fi
|
||||
|
||||
"${COMPOSE_BIN[@]}" --env-file "${ENV_FILE}" -p "${PRECHECK_PROJECT}" -f "${compose_file}" "$@"
|
||||
}
|
||||
|
||||
if command -v docker >/dev/null 2>&1; then
|
||||
log "stopping pre-check client compose stack"
|
||||
[ -f "${PRECHECK_CLIENT_COMPOSE_FILE}" ] && compose_run "${PRECHECK_CLIENT_COMPOSE_FILE}" down --remove-orphans >/dev/null 2>&1 || true
|
||||
log "stopping pre-check server compose stack"
|
||||
[ -f "${PRECHECK_SERVER_COMPOSE_FILE}" ] && compose_run "${PRECHECK_SERVER_COMPOSE_FILE}" down --remove-orphans >/dev/null 2>&1 || true
|
||||
fi
|
||||
|
||||
if [ "${KEEP_PRECHECK}" = "1" ]; then
|
||||
log "KEEP_PRECHECK=1, preserving ${PRECHECK_WORK_DIR}"
|
||||
else
|
||||
rm -rf "${PRECHECK_WORK_DIR}"
|
||||
log "removed ${PRECHECK_WORK_DIR}"
|
||||
fi
|
||||
@@ -1,221 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||
|
||||
COMMON_ENV_FILE="${ROOT_DIR}/environment.server"
|
||||
SERVER_ENV_FILE="${ROOT_DIR}/deploy/server/.env"
|
||||
CONSOLE_ENV_FILE="${ROOT_DIR}/deploy/console/.env"
|
||||
CONSOLE_COMPOSE_FILE="${ROOT_DIR}/deploy/console/docker-compose.yml"
|
||||
MODE="${RPKI_RTR_SERVER_MODE:-tcp}"
|
||||
COMPOSE_PROJECT="${RPKI_RTR_COMPOSE_PROJECT:-rpki-rtr}"
|
||||
CONSOLE_COMPOSE_PROJECT="${RPKI_CONSOLE_COMPOSE_PROJECT:-rpki-console}"
|
||||
WAIT_SECS="${RPKI_START_WAIT_SECS:-20}"
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage: scripts/01_start.sh [options]
|
||||
|
||||
Read root ./environment.server, update deploy/server/.env, then start the RTR server and console.
|
||||
|
||||
Options:
|
||||
--env FILE Common env file. Default: ./environment.server
|
||||
--mode MODE Server mode: tcp|tls|ssh|base. Default: tcp
|
||||
--wait SECS Wait timeout for container running checks. Default: 20
|
||||
-h, --help Show this help.
|
||||
EOF
|
||||
}
|
||||
|
||||
log() {
|
||||
printf '[01_start] %s\n' "$*" >&2
|
||||
}
|
||||
|
||||
stage() {
|
||||
printf '\n====== %s ======\n' "$*" >&2
|
||||
}
|
||||
|
||||
die() {
|
||||
printf '[01_start] ERROR: %s\n' "$*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
set_env_var() {
|
||||
local env_file="$1"
|
||||
local key="$2"
|
||||
local value="$3"
|
||||
local tmp_file
|
||||
|
||||
tmp_file="${env_file}.tmp.$$"
|
||||
awk -v key="${key}" -v value="${value}" '
|
||||
BEGIN { done = 0 }
|
||||
$0 ~ "^[[:space:]]*#?[[:space:]]*" key "=" {
|
||||
if (!done) {
|
||||
print key "=" value
|
||||
done = 1
|
||||
}
|
||||
next
|
||||
}
|
||||
{ print }
|
||||
END {
|
||||
if (!done) {
|
||||
print key "=" value
|
||||
}
|
||||
}
|
||||
' "${env_file}" >"${tmp_file}"
|
||||
mv "${tmp_file}" "${env_file}"
|
||||
}
|
||||
|
||||
compose_file_for_mode() {
|
||||
case "$1" in
|
||||
tcp) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.tcp.yml" ;;
|
||||
tls) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.tls.yml" ;;
|
||||
ssh) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.ssh.yml" ;;
|
||||
base) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.yml" ;;
|
||||
*) die "unsupported mode '$1', expected tcp|tls|ssh|base" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
update_env_from_common() {
|
||||
local target_env_file="$1"
|
||||
local updated_count=0
|
||||
local line key value
|
||||
|
||||
while IFS= read -r line || [ -n "${line}" ]; do
|
||||
case "${line}" in
|
||||
''|\#*) continue ;;
|
||||
*=*)
|
||||
key="${line%%=*}"
|
||||
value="${line#*=}"
|
||||
set_env_var "${target_env_file}" "${key}" "${value}"
|
||||
updated_count=$((updated_count + 1))
|
||||
;;
|
||||
esac
|
||||
done <"${COMMON_ENV_FILE}"
|
||||
|
||||
printf '%s\n' "${updated_count}"
|
||||
}
|
||||
|
||||
server_compose() {
|
||||
"${COMPOSE_BIN[@]}" --env-file "${SERVER_ENV_FILE}" -p "${COMPOSE_PROJECT}" -f "${SERVER_COMPOSE_FILE}" "$@"
|
||||
}
|
||||
|
||||
console_compose() {
|
||||
"${COMPOSE_BIN[@]}" --env-file "${CONSOLE_ENV_FILE}" -p "${CONSOLE_COMPOSE_PROJECT}" -f "${CONSOLE_COMPOSE_FILE}" "$@"
|
||||
}
|
||||
|
||||
container_running() {
|
||||
local compose_kind="$1"
|
||||
local service="$2"
|
||||
local cid state
|
||||
|
||||
if [ "${compose_kind}" = "server" ]; then
|
||||
cid="$(server_compose ps -q "${service}" 2>/dev/null || true)"
|
||||
else
|
||||
cid="$(console_compose ps -q "${service}" 2>/dev/null || true)"
|
||||
fi
|
||||
|
||||
[ -n "${cid}" ] || return 1
|
||||
state="$(docker inspect -f '{{.State.Running}}' "${cid}" 2>/dev/null || true)"
|
||||
[ "${state}" = "true" ]
|
||||
}
|
||||
|
||||
show_service_debug() {
|
||||
local compose_kind="$1"
|
||||
local service="$2"
|
||||
|
||||
log "${service} is not running, current compose status:"
|
||||
if [ "${compose_kind}" = "server" ]; then
|
||||
server_compose ps "${service}" >&2 || true
|
||||
log "${service} recent logs:"
|
||||
server_compose logs --tail=80 "${service}" >&2 || true
|
||||
else
|
||||
console_compose ps "${service}" >&2 || true
|
||||
log "${service} recent logs:"
|
||||
console_compose logs --tail=80 "${service}" >&2 || true
|
||||
fi
|
||||
}
|
||||
|
||||
wait_for_running() {
|
||||
local compose_kind="$1"
|
||||
local service="$2"
|
||||
local deadline=$((SECONDS + WAIT_SECS))
|
||||
|
||||
while [ "${SECONDS}" -lt "${deadline}" ]; do
|
||||
if container_running "${compose_kind}" "${service}"; then
|
||||
log "${service} is running"
|
||||
return 0
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
|
||||
show_service_debug "${compose_kind}" "${service}"
|
||||
die "${service} failed to reach running state after ${WAIT_SECS}s"
|
||||
}
|
||||
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
--env)
|
||||
[ "$#" -ge 2 ] || die "--env requires a value"
|
||||
COMMON_ENV_FILE="$2"
|
||||
shift 2
|
||||
;;
|
||||
--mode)
|
||||
[ "$#" -ge 2 ] || die "--mode requires a value"
|
||||
MODE="$2"
|
||||
shift 2
|
||||
;;
|
||||
--wait)
|
||||
[ "$#" -ge 2 ] || die "--wait requires a value"
|
||||
WAIT_SECS="$2"
|
||||
shift 2
|
||||
;;
|
||||
-h|--help)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
die "unknown option: $1"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
SERVER_COMPOSE_FILE="$(compose_file_for_mode "${MODE}")"
|
||||
|
||||
stage "Checking Docker"
|
||||
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
||||
if docker compose version >/dev/null 2>&1; then
|
||||
COMPOSE_BIN=(docker compose)
|
||||
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
||||
COMPOSE_BIN=(docker-compose)
|
||||
else
|
||||
die "docker compose is not available"
|
||||
fi
|
||||
|
||||
[ -f "${SERVER_ENV_FILE}" ] || die "server env file not found: ${SERVER_ENV_FILE}"
|
||||
[ -f "${SERVER_COMPOSE_FILE}" ] || die "server compose file not found: ${SERVER_COMPOSE_FILE}"
|
||||
[ -f "${CONSOLE_ENV_FILE}" ] || die "console env file not found: ${CONSOLE_ENV_FILE}"
|
||||
[ -f "${CONSOLE_COMPOSE_FILE}" ] || die "console compose file not found: ${CONSOLE_COMPOSE_FILE}"
|
||||
|
||||
if [ -f "${COMMON_ENV_FILE}" ]; then
|
||||
stage "Reading Environment"
|
||||
server_updated_count="$(update_env_from_common "${SERVER_ENV_FILE}")"
|
||||
log "updated ${server_updated_count} server env variable(s)"
|
||||
else
|
||||
stage "Reading Environment"
|
||||
log "environment.server file not found, skip env update"
|
||||
fi
|
||||
|
||||
stage "Starting RTR Server"
|
||||
log "starting docker compose service: rpki-rtr"
|
||||
server_compose up -d --no-build rpki-rtr
|
||||
log "waiting for rpki-rtr to be running"
|
||||
wait_for_running server rpki-rtr
|
||||
|
||||
stage "Starting Console"
|
||||
log "starting docker compose service: rpki-console-ui"
|
||||
console_compose up -d --no-build rpki-console-ui
|
||||
log "waiting for rpki-console-ui to be running"
|
||||
wait_for_running console rpki-console-ui
|
||||
|
||||
stage "Start Finished"
|
||||
@@ -1,184 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||
|
||||
SERVER_ENV_FILE="${ROOT_DIR}/deploy/server/.env"
|
||||
CONSOLE_ENV_FILE="${ROOT_DIR}/deploy/console/.env"
|
||||
SERVER_COMPOSE_FILE="${ROOT_DIR}/deploy/server/docker-compose.tcp.yml"
|
||||
CONSOLE_COMPOSE_FILE="${ROOT_DIR}/deploy/console/docker-compose.yml"
|
||||
SERVER_PROJECT="${RPKI_RTR_COMPOSE_PROJECT:-rpki-rtr}"
|
||||
CONSOLE_PROJECT="${RPKI_CONSOLE_COMPOSE_PROJECT:-rpki-console}"
|
||||
WAIT_SECS="${RPKI_SELF_CHECK_WAIT_SECS:-60}"
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage: scripts/02_self_check.sh [options]
|
||||
|
||||
Check whether the RTR server and console started by scripts/01_start.sh are healthy.
|
||||
|
||||
Options:
|
||||
--wait SECS Wait timeout for each check. Default: 60
|
||||
-h, --help Show this help.
|
||||
EOF
|
||||
}
|
||||
|
||||
log() {
|
||||
printf '[02_self_check] %s\n' "$*" >&2
|
||||
}
|
||||
|
||||
stage() {
|
||||
printf '\n====== %s ======\n' "$*" >&2
|
||||
}
|
||||
|
||||
die() {
|
||||
printf '[02_self_check] ERROR: %s\n' "$*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
--wait)
|
||||
[ "$#" -ge 2 ] || die "--wait requires a value"
|
||||
WAIT_SECS="$2"
|
||||
shift 2
|
||||
;;
|
||||
-h|--help)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
die "unknown option: $1"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
||||
if docker compose version >/dev/null 2>&1; then
|
||||
COMPOSE_BIN=(docker compose)
|
||||
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
||||
COMPOSE_BIN=(docker-compose)
|
||||
else
|
||||
die "docker compose is not available"
|
||||
fi
|
||||
|
||||
[ -f "${SERVER_ENV_FILE}" ] || die "server env file not found: ${SERVER_ENV_FILE}"
|
||||
[ -f "${SERVER_COMPOSE_FILE}" ] || die "server compose file not found: ${SERVER_COMPOSE_FILE}"
|
||||
[ -f "${CONSOLE_ENV_FILE}" ] || die "console env file not found: ${CONSOLE_ENV_FILE}"
|
||||
[ -f "${CONSOLE_COMPOSE_FILE}" ] || die "console compose file not found: ${CONSOLE_COMPOSE_FILE}"
|
||||
|
||||
set -a
|
||||
# shellcheck disable=SC1090
|
||||
. "${SERVER_ENV_FILE}"
|
||||
# shellcheck disable=SC1090
|
||||
. "${CONSOLE_ENV_FILE}"
|
||||
set +a
|
||||
|
||||
RTR_PORT="${RPKI_RTR_TCP_HOST_PORT:-323}"
|
||||
ADMIN_PORT="${RPKI_RTR_ADMIN_ADDR##*:}"
|
||||
ADMIN_PORT="${ADMIN_PORT:-8323}"
|
||||
ADMIN_TOKEN="${RPKI_RTR_ADMIN_TOKEN:-}"
|
||||
CONSOLE_PORT="${RPKI_CONSOLE_UI_PORT:-5173}"
|
||||
|
||||
server_compose() {
|
||||
"${COMPOSE_BIN[@]}" --env-file "${SERVER_ENV_FILE}" -p "${SERVER_PROJECT}" -f "${SERVER_COMPOSE_FILE}" "$@"
|
||||
}
|
||||
|
||||
console_compose() {
|
||||
"${COMPOSE_BIN[@]}" --env-file "${CONSOLE_ENV_FILE}" -p "${CONSOLE_PROJECT}" -f "${CONSOLE_COMPOSE_FILE}" "$@"
|
||||
}
|
||||
|
||||
container_running() {
|
||||
local compose_kind="$1"
|
||||
local service="$2"
|
||||
local cid state
|
||||
|
||||
if [ "${compose_kind}" = "server" ]; then
|
||||
cid="$(server_compose ps -q "${service}" 2>/dev/null || true)"
|
||||
else
|
||||
cid="$(console_compose ps -q "${service}" 2>/dev/null || true)"
|
||||
fi
|
||||
|
||||
[ -n "${cid}" ] || return 1
|
||||
state="$(docker inspect -f '{{.State.Running}}' "${cid}" 2>/dev/null || true)"
|
||||
[ "${state}" = "true" ]
|
||||
}
|
||||
|
||||
tcp_open() {
|
||||
local host="$1"
|
||||
local port="$2"
|
||||
|
||||
if command -v nc >/dev/null 2>&1; then
|
||||
nc -z "${host}" "${port}" >/dev/null 2>&1
|
||||
return
|
||||
fi
|
||||
|
||||
(exec 3<>"/dev/tcp/${host}/${port}") >/dev/null 2>&1
|
||||
}
|
||||
|
||||
http_get() {
|
||||
local url="$1"
|
||||
local token="${2:-}"
|
||||
|
||||
if command -v curl >/dev/null 2>&1; then
|
||||
if [ -n "${token}" ]; then
|
||||
curl -fsS -H "Authorization: Bearer ${token}" "${url}"
|
||||
else
|
||||
curl -fsS "${url}"
|
||||
fi
|
||||
return
|
||||
fi
|
||||
|
||||
die "curl command is required for HTTP health checks"
|
||||
}
|
||||
|
||||
wait_until() {
|
||||
local label="$1"
|
||||
shift
|
||||
local deadline=$((SECONDS + WAIT_SECS))
|
||||
|
||||
while [ "${SECONDS}" -lt "${deadline}" ]; do
|
||||
if "$@"; then
|
||||
log "${label}: ok"
|
||||
return 0
|
||||
fi
|
||||
sleep 1
|
||||
done
|
||||
|
||||
die "${label}: failed after ${WAIT_SECS}s"
|
||||
}
|
||||
|
||||
show_command() {
|
||||
printf '[02_self_check] check command: %s\n' "$*" >&2
|
||||
}
|
||||
|
||||
check_admin_health() {
|
||||
local body
|
||||
body="$(http_get "http://127.0.0.1:${ADMIN_PORT}/admin/rtr/health" "${ADMIN_TOKEN}" 2>/dev/null || true)"
|
||||
printf '%s' "${body}" | grep -q '"status"[[:space:]]*:[[:space:]]*"ok"'
|
||||
}
|
||||
|
||||
check_console_http() {
|
||||
http_get "http://127.0.0.1:${CONSOLE_PORT}/" >/dev/null 2>&1
|
||||
}
|
||||
|
||||
stage "Checking Containers"
|
||||
show_command "${COMPOSE_BIN[*]} --env-file ${SERVER_ENV_FILE} -p ${SERVER_PROJECT} -f ${SERVER_COMPOSE_FILE} ps -q rpki-rtr && docker inspect"
|
||||
wait_until "server container" container_running server rpki-rtr
|
||||
show_command "${COMPOSE_BIN[*]} --env-file ${CONSOLE_ENV_FILE} -p ${CONSOLE_PROJECT} -f ${CONSOLE_COMPOSE_FILE} ps -q rpki-console-ui && docker inspect"
|
||||
wait_until "console container" container_running console rpki-console-ui
|
||||
|
||||
stage "Checking Server"
|
||||
show_command "tcp connect 127.0.0.1:${RTR_PORT}"
|
||||
wait_until "RTR port ${RTR_PORT}" tcp_open 127.0.0.1 "${RTR_PORT}"
|
||||
show_command "curl -fsS -H 'Authorization: Bearer <token>' http://127.0.0.1:${ADMIN_PORT}/admin/rtr/health"
|
||||
wait_until "Admin health" check_admin_health
|
||||
|
||||
stage "Checking Console"
|
||||
show_command "tcp connect 127.0.0.1:${CONSOLE_PORT}"
|
||||
wait_until "Console port ${CONSOLE_PORT}" tcp_open 127.0.0.1 "${CONSOLE_PORT}"
|
||||
show_command "curl -fsS http://127.0.0.1:${CONSOLE_PORT}/"
|
||||
wait_until "Console HTTP" check_console_http
|
||||
|
||||
stage "Self-Check Passed"
|
||||
@@ -1,92 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||
|
||||
SERVER_ENV_FILE="${ROOT_DIR}/deploy/server/.env"
|
||||
CONSOLE_ENV_FILE="${ROOT_DIR}/deploy/console/.env"
|
||||
MODE="${RPKI_RTR_SERVER_MODE:-tcp}"
|
||||
SERVER_PROJECT="${RPKI_RTR_COMPOSE_PROJECT:-rpki-rtr}"
|
||||
CONSOLE_PROJECT="${RPKI_CONSOLE_COMPOSE_PROJECT:-rpki-console}"
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage: scripts/03_shutdown.sh [options]
|
||||
|
||||
Stop services started by scripts/01_start.sh.
|
||||
|
||||
Options:
|
||||
--mode MODE Server mode: tcp|tls|ssh|base. Default: tcp
|
||||
-h, --help Show this help.
|
||||
EOF
|
||||
}
|
||||
|
||||
log() {
|
||||
printf '[03_shutdown] %s\n' "$*" >&2
|
||||
}
|
||||
|
||||
stage() {
|
||||
printf '\n====== %s ======\n' "$*" >&2
|
||||
}
|
||||
|
||||
die() {
|
||||
printf '[03_shutdown] ERROR: %s\n' "$*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
compose_file_for_mode() {
|
||||
case "$1" in
|
||||
tcp) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.tcp.yml" ;;
|
||||
tls) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.tls.yml" ;;
|
||||
ssh) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.ssh.yml" ;;
|
||||
base) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.yml" ;;
|
||||
*) die "unsupported mode '$1', expected tcp|tls|ssh|base" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
--mode)
|
||||
[ "$#" -ge 2 ] || die "--mode requires a value"
|
||||
MODE="$2"
|
||||
shift 2
|
||||
;;
|
||||
-h|--help)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
die "unknown option: $1"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
SERVER_COMPOSE_FILE="$(compose_file_for_mode "${MODE}")"
|
||||
CONSOLE_COMPOSE_FILE="${ROOT_DIR}/deploy/console/docker-compose.yml"
|
||||
|
||||
stage "Checking Docker"
|
||||
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
||||
if docker compose version >/dev/null 2>&1; then
|
||||
COMPOSE_BIN=(docker compose)
|
||||
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
||||
COMPOSE_BIN=(docker-compose)
|
||||
else
|
||||
die "docker compose is not available"
|
||||
fi
|
||||
|
||||
stage "Stopping Console"
|
||||
if [ -f "${CONSOLE_ENV_FILE}" ] && [ -f "${CONSOLE_COMPOSE_FILE}" ]; then
|
||||
"${COMPOSE_BIN[@]}" --env-file "${CONSOLE_ENV_FILE}" -p "${CONSOLE_PROJECT}" -f "${CONSOLE_COMPOSE_FILE}" down --remove-orphans
|
||||
else
|
||||
log "console compose/env not found, skip"
|
||||
fi
|
||||
|
||||
stage "Stopping RTR Server"
|
||||
if [ -f "${SERVER_ENV_FILE}" ] && [ -f "${SERVER_COMPOSE_FILE}" ]; then
|
||||
"${COMPOSE_BIN[@]}" --env-file "${SERVER_ENV_FILE}" -p "${SERVER_PROJECT}" -f "${SERVER_COMPOSE_FILE}" down --remove-orphans
|
||||
else
|
||||
log "server compose/env not found, skip"
|
||||
fi
|
||||
|
||||
stage "Shutdown Finished"
|
||||
@@ -1,152 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||
SERVER_DIR="${ROOT_DIR}/deploy/server"
|
||||
SERVER_ENV_FILE="${SERVER_DIR}/.env"
|
||||
SERVER_COMPOSE_FILE="${SERVER_DIR}/docker-compose.tcp.yml"
|
||||
COMPOSE_PROJECT="${RPKI_RTR_COMPOSE_PROJECT:-rpki-rtr}"
|
||||
FORCE=0
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage: scripts/04_cleanup.sh [options]
|
||||
|
||||
Remove server generated directories: logs, reports, and RTR database.
|
||||
|
||||
Options:
|
||||
--force Clean even if the server container appears to be running.
|
||||
-h, --help Show this help.
|
||||
EOF
|
||||
}
|
||||
|
||||
log() {
|
||||
printf '[04_cleanup] %s\n' "$*" >&2
|
||||
}
|
||||
|
||||
stage() {
|
||||
printf '\n====== %s ======\n' "$*" >&2
|
||||
}
|
||||
|
||||
die() {
|
||||
printf '[04_cleanup] ERROR: %s\n' "$*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
env_value() {
|
||||
local key="$1"
|
||||
local default_value="$2"
|
||||
local line
|
||||
|
||||
line="$(grep -E "^[[:space:]]*${key}=" "${SERVER_ENV_FILE}" | tail -n 1 || true)"
|
||||
if [ -n "${line}" ]; then
|
||||
printf '%s\n' "${line#*=}"
|
||||
else
|
||||
printf '%s\n' "${default_value}"
|
||||
fi
|
||||
}
|
||||
|
||||
resolve_host_path() {
|
||||
local raw_path="$1"
|
||||
|
||||
case "${raw_path}" in
|
||||
/*|[A-Za-z]:/*|[A-Za-z]:\\*)
|
||||
printf '%s\n' "${raw_path}"
|
||||
;;
|
||||
*)
|
||||
(cd "${SERVER_DIR}" && cd "${raw_path}" 2>/dev/null && pwd -P) || printf '%s/%s\n' "${SERVER_DIR}" "${raw_path}"
|
||||
;;
|
||||
esac
|
||||
}
|
||||
|
||||
compose_available() {
|
||||
if docker compose version >/dev/null 2>&1; then
|
||||
COMPOSE_BIN=(docker compose)
|
||||
return 0
|
||||
fi
|
||||
|
||||
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
||||
COMPOSE_BIN=(docker-compose)
|
||||
return 0
|
||||
fi
|
||||
|
||||
return 1
|
||||
}
|
||||
|
||||
server_running() {
|
||||
local cid state
|
||||
|
||||
[ -f "${SERVER_COMPOSE_FILE}" ] || return 1
|
||||
compose_available || return 1
|
||||
|
||||
cid="$("${COMPOSE_BIN[@]}" --env-file "${SERVER_ENV_FILE}" -p "${COMPOSE_PROJECT}" -f "${SERVER_COMPOSE_FILE}" ps -q rpki-rtr 2>/dev/null || true)"
|
||||
[ -n "${cid}" ] || return 1
|
||||
|
||||
state="$(docker inspect -f '{{.State.Running}}' "${cid}" 2>/dev/null || true)"
|
||||
[ "${state}" = "true" ]
|
||||
}
|
||||
|
||||
clean_dir() {
|
||||
local label="$1"
|
||||
local path="$2"
|
||||
local root_with_slash
|
||||
|
||||
if [ ! -d "${path}" ]; then
|
||||
log "${label}: ${path} not found, skip"
|
||||
return 0
|
||||
fi
|
||||
|
||||
root_with_slash="${ROOT_DIR}/"
|
||||
case "${path}/" in
|
||||
"${root_with_slash}"*) ;;
|
||||
*)
|
||||
log "ERROR: ${label} path is outside repo root, refuse to remove: ${path}"
|
||||
return 1
|
||||
;;
|
||||
esac
|
||||
|
||||
log "${label}: removing ${path}"
|
||||
if ! rm -rf "${path}"; then
|
||||
log "ERROR: failed to remove ${label}: ${path}"
|
||||
return 1
|
||||
fi
|
||||
}
|
||||
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
--force)
|
||||
FORCE=1
|
||||
shift
|
||||
;;
|
||||
-h|--help)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
die "unknown option: $1"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
stage "Checking Server"
|
||||
[ -f "${SERVER_ENV_FILE}" ] || die "server env file not found: ${SERVER_ENV_FILE}"
|
||||
if [ "${FORCE}" -ne 1 ] && command -v docker >/dev/null 2>&1 && server_running; then
|
||||
die "rpki-rtr is still running, run scripts/03_shutdown.sh first or use --force"
|
||||
fi
|
||||
|
||||
LOG_DIR="${ROOT_DIR}/logs"
|
||||
REPORT_DIR="$(resolve_host_path "$(env_value RPKI_RTR_REPORT_HOST_DIR '../../report')")"
|
||||
DB_DIR="$(resolve_host_path "$(env_value RPKI_RTR_DB_HOST_DIR '../../rtr-db')")"
|
||||
|
||||
stage "Removing Server Files"
|
||||
failed_count=0
|
||||
clean_dir "logs" "${LOG_DIR}" || failed_count=$((failed_count + 1))
|
||||
clean_dir "report" "${REPORT_DIR}" || failed_count=$((failed_count + 1))
|
||||
clean_dir "rtr-db" "${DB_DIR}" || failed_count=$((failed_count + 1))
|
||||
|
||||
if [ "${failed_count}" -gt 0 ]; then
|
||||
die "${failed_count} cleanup item(s) failed"
|
||||
fi
|
||||
|
||||
stage "Cleanup Finished"
|
||||
@@ -1,303 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
OUT_DIR="${ROOT_DIR}/certs"
|
||||
TLS_DIR="${OUT_DIR}/tls"
|
||||
SSH_DIR="${OUT_DIR}/ssh"
|
||||
SERVER_DNS="localhost"
|
||||
FORCE=0
|
||||
DO_TLS=1
|
||||
DO_SSH=1
|
||||
|
||||
usage() {
|
||||
cat <<'USAGE'
|
||||
Generate local development TLS certificates and SSH keys for RTR Docker deploys.
|
||||
|
||||
Usage:
|
||||
scripts/10_generate-certs.sh [options]
|
||||
|
||||
Options:
|
||||
--out-dir DIR Output directory, default: ./certs
|
||||
--server-dns NAME TLS server dNSName SAN, default: localhost
|
||||
--tls-only Generate only TLS files
|
||||
--ssh-only Generate only SSH files
|
||||
--force Overwrite existing files
|
||||
-h, --help Show this help
|
||||
|
||||
Generated TLS files:
|
||||
certs/tls/client-ca.crt
|
||||
certs/tls/client-ca.key
|
||||
certs/tls/server-dns.crt
|
||||
certs/tls/server-dns.key
|
||||
certs/tls/client-good.crt
|
||||
certs/tls/client-good.key
|
||||
|
||||
Generated SSH files:
|
||||
certs/ssh/ssh_host_rsa_key
|
||||
certs/ssh/ssh_host_rsa_key.pub
|
||||
certs/ssh/rtr-client.key
|
||||
certs/ssh/rtr-client.key.pub
|
||||
certs/ssh/rtr-authorized_keys
|
||||
USAGE
|
||||
}
|
||||
|
||||
die() {
|
||||
echo "error: $*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
info() {
|
||||
echo "[generate-certs] $*"
|
||||
}
|
||||
|
||||
need_cmd() {
|
||||
command -v "$1" >/dev/null 2>&1 || die "required command not found: $1"
|
||||
}
|
||||
|
||||
run_quiet() {
|
||||
local log_file
|
||||
log_file="$(mktemp)"
|
||||
if "$@" >"${log_file}" 2>&1; then
|
||||
rm -f "${log_file}"
|
||||
return 0
|
||||
fi
|
||||
|
||||
cat "${log_file}" >&2
|
||||
rm -f "${log_file}"
|
||||
return 1
|
||||
}
|
||||
|
||||
ensure_abs_out_dir() {
|
||||
case "${OUT_DIR}" in
|
||||
/*) ;;
|
||||
[A-Za-z]:*) ;;
|
||||
*) OUT_DIR="${ROOT_DIR}/${OUT_DIR}" ;;
|
||||
esac
|
||||
TLS_DIR="${OUT_DIR}/tls"
|
||||
SSH_DIR="${OUT_DIR}/ssh"
|
||||
}
|
||||
|
||||
check_overwrite() {
|
||||
local path="$1"
|
||||
if [[ -e "${path}" && "${FORCE}" != "1" ]]; then
|
||||
die "${path} already exists; rerun with --force to overwrite"
|
||||
fi
|
||||
}
|
||||
|
||||
write_tls_config_files() {
|
||||
local tmp_dir="$1"
|
||||
|
||||
cat >"${tmp_dir}/ca.cnf" <<'EOF'
|
||||
[ req ]
|
||||
distinguished_name = req_distinguished_name
|
||||
prompt = no
|
||||
|
||||
[ req_distinguished_name ]
|
||||
CN = rpki-rtr-client-ca
|
||||
EOF
|
||||
|
||||
cat >"${tmp_dir}/server.cnf" <<EOF
|
||||
[ req ]
|
||||
distinguished_name = req_distinguished_name
|
||||
prompt = no
|
||||
|
||||
[ req_distinguished_name ]
|
||||
CN = ${SERVER_DNS}
|
||||
EOF
|
||||
|
||||
cat >"${tmp_dir}/client.cnf" <<'EOF'
|
||||
[ req ]
|
||||
distinguished_name = req_distinguished_name
|
||||
prompt = no
|
||||
|
||||
[ req_distinguished_name ]
|
||||
CN = rpki-rtr-client
|
||||
EOF
|
||||
|
||||
cat >"${tmp_dir}/server.ext" <<EOF
|
||||
basicConstraints=CA:FALSE
|
||||
keyUsage=digitalSignature,keyEncipherment
|
||||
extendedKeyUsage=serverAuth
|
||||
subjectAltName=DNS:${SERVER_DNS}
|
||||
EOF
|
||||
|
||||
cat >"${tmp_dir}/client.ext" <<'EOF'
|
||||
basicConstraints=CA:FALSE
|
||||
keyUsage=digitalSignature,keyEncipherment
|
||||
extendedKeyUsage=clientAuth
|
||||
EOF
|
||||
}
|
||||
|
||||
generate_tls() {
|
||||
need_cmd openssl
|
||||
mkdir -p "${TLS_DIR}"
|
||||
|
||||
info "generating TLS certificates in ${TLS_DIR}"
|
||||
info "TLS server dNSName SAN: ${SERVER_DNS}"
|
||||
|
||||
local files=(
|
||||
"${TLS_DIR}/client-ca.crt"
|
||||
"${TLS_DIR}/client-ca.key"
|
||||
"${TLS_DIR}/server-dns.crt"
|
||||
"${TLS_DIR}/server-dns.key"
|
||||
"${TLS_DIR}/client-good.crt"
|
||||
"${TLS_DIR}/client-good.key"
|
||||
)
|
||||
for file in "${files[@]}"; do
|
||||
check_overwrite "${file}"
|
||||
done
|
||||
|
||||
local tmp_dir
|
||||
tmp_dir="$(mktemp -d)"
|
||||
write_tls_config_files "${tmp_dir}"
|
||||
|
||||
run_quiet openssl genrsa -out "${TLS_DIR}/client-ca.key" 2048
|
||||
info "created TLS CA key: ${TLS_DIR}/client-ca.key"
|
||||
run_quiet openssl req -x509 -new -nodes \
|
||||
-config "${tmp_dir}/ca.cnf" \
|
||||
-key "${TLS_DIR}/client-ca.key" \
|
||||
-sha256 -days 3650 \
|
||||
-out "${TLS_DIR}/client-ca.crt"
|
||||
info "created TLS CA cert: ${TLS_DIR}/client-ca.crt"
|
||||
|
||||
run_quiet openssl genrsa -out "${TLS_DIR}/server-dns.key" 2048
|
||||
info "created TLS server key: ${TLS_DIR}/server-dns.key"
|
||||
run_quiet openssl req -new \
|
||||
-config "${tmp_dir}/server.cnf" \
|
||||
-key "${TLS_DIR}/server-dns.key" \
|
||||
-out "${tmp_dir}/server-dns.csr"
|
||||
run_quiet openssl x509 -req \
|
||||
-in "${tmp_dir}/server-dns.csr" \
|
||||
-CA "${TLS_DIR}/client-ca.crt" \
|
||||
-CAkey "${TLS_DIR}/client-ca.key" \
|
||||
-CAcreateserial \
|
||||
-out "${TLS_DIR}/server-dns.crt" \
|
||||
-days 3650 -sha256 \
|
||||
-extfile "${tmp_dir}/server.ext"
|
||||
info "created TLS server cert: ${TLS_DIR}/server-dns.crt"
|
||||
|
||||
run_quiet openssl genrsa -out "${TLS_DIR}/client-good.key" 2048
|
||||
info "created TLS client key: ${TLS_DIR}/client-good.key"
|
||||
run_quiet openssl req -new \
|
||||
-config "${tmp_dir}/client.cnf" \
|
||||
-key "${TLS_DIR}/client-good.key" \
|
||||
-out "${tmp_dir}/client-good.csr"
|
||||
run_quiet openssl x509 -req \
|
||||
-in "${tmp_dir}/client-good.csr" \
|
||||
-CA "${TLS_DIR}/client-ca.crt" \
|
||||
-CAkey "${TLS_DIR}/client-ca.key" \
|
||||
-CAcreateserial \
|
||||
-out "${TLS_DIR}/client-good.crt" \
|
||||
-days 3650 -sha256 \
|
||||
-extfile "${tmp_dir}/client.ext"
|
||||
info "created TLS client cert: ${TLS_DIR}/client-good.crt"
|
||||
|
||||
rm -f "${TLS_DIR}/client-ca.srl"
|
||||
rm -rf "${tmp_dir}"
|
||||
chmod 600 "${TLS_DIR}"/*.key
|
||||
chmod 644 "${TLS_DIR}"/*.crt
|
||||
}
|
||||
|
||||
generate_ssh_key() {
|
||||
local path="$1"
|
||||
local comment="$2"
|
||||
check_overwrite "${path}"
|
||||
check_overwrite "${path}.pub"
|
||||
rm -f "${path}" "${path}.pub"
|
||||
run_quiet ssh-keygen -q -t rsa -b 3072 -m PEM -N "" -C "${comment}" -f "${path}"
|
||||
}
|
||||
|
||||
generate_ssh() {
|
||||
need_cmd ssh-keygen
|
||||
mkdir -p "${SSH_DIR}"
|
||||
|
||||
info "generating SSH keys in ${SSH_DIR}"
|
||||
generate_ssh_key "${SSH_DIR}/ssh_host_rsa_key" "rpki-rtr-host"
|
||||
info "created SSH host key: ${SSH_DIR}/ssh_host_rsa_key"
|
||||
generate_ssh_key "${SSH_DIR}/rtr-client.key" "rpki-rtr-debug-client"
|
||||
info "created SSH client key: ${SSH_DIR}/rtr-client.key"
|
||||
|
||||
check_overwrite "${SSH_DIR}/rtr-authorized_keys"
|
||||
cp "${SSH_DIR}/rtr-client.key.pub" "${SSH_DIR}/rtr-authorized_keys"
|
||||
info "created SSH authorized_keys: ${SSH_DIR}/rtr-authorized_keys"
|
||||
|
||||
chmod 600 \
|
||||
"${SSH_DIR}/ssh_host_rsa_key" \
|
||||
"${SSH_DIR}/rtr-client.key"
|
||||
chmod 644 "${SSH_DIR}"/*.pub "${SSH_DIR}/rtr-authorized_keys"
|
||||
}
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--out-dir)
|
||||
[[ $# -ge 2 ]] || die "--out-dir requires a value"
|
||||
OUT_DIR="$2"
|
||||
shift 2
|
||||
;;
|
||||
--server-dns)
|
||||
[[ $# -ge 2 ]] || die "--server-dns requires a value"
|
||||
SERVER_DNS="$2"
|
||||
shift 2
|
||||
;;
|
||||
--tls-only)
|
||||
DO_TLS=1
|
||||
DO_SSH=0
|
||||
shift
|
||||
;;
|
||||
--ssh-only)
|
||||
DO_TLS=0
|
||||
DO_SSH=1
|
||||
shift
|
||||
;;
|
||||
--force)
|
||||
FORCE=1
|
||||
shift
|
||||
;;
|
||||
-h|--help)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
die "unknown option: $1"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
ensure_abs_out_dir
|
||||
|
||||
if [[ "${DO_TLS}" == "1" ]]; then
|
||||
generate_tls
|
||||
fi
|
||||
|
||||
if [[ "${DO_SSH}" == "1" ]]; then
|
||||
generate_ssh
|
||||
fi
|
||||
|
||||
info "done"
|
||||
info "generated files under ${OUT_DIR}"
|
||||
|
||||
if [[ "${DO_TLS}" == "1" ]]; then
|
||||
cat <<EOF
|
||||
|
||||
TLS files:
|
||||
${TLS_DIR}/client-ca.crt
|
||||
${TLS_DIR}/server-dns.crt
|
||||
${TLS_DIR}/server-dns.key
|
||||
${TLS_DIR}/client-good.crt
|
||||
${TLS_DIR}/client-good.key
|
||||
${TLS_DIR}/client-ca.key (local signing key; not mounted by compose)
|
||||
EOF
|
||||
fi
|
||||
|
||||
if [[ "${DO_SSH}" == "1" ]]; then
|
||||
cat <<EOF
|
||||
|
||||
SSH files:
|
||||
${SSH_DIR}/ssh_host_rsa_key
|
||||
${SSH_DIR}/ssh_host_rsa_key.pub
|
||||
${SSH_DIR}/rtr-client.key
|
||||
${SSH_DIR}/rtr-client.key.pub
|
||||
${SSH_DIR}/rtr-authorized_keys
|
||||
EOF
|
||||
fi
|
||||
@@ -1,136 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||
|
||||
COMMON_ENV_FILE="${ROOT_DIR}/environment.client"
|
||||
CLIENT_ENV_FILE="${ROOT_DIR}/deploy/client/.env"
|
||||
MODE="${RPKI_RTR_CLIENT_MODE:-tcp}"
|
||||
COMPOSE_PROJECT="${RPKI_RTR_CLIENT_COMPOSE_PROJECT:-rpki-rtr-client}"
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage: scripts/11_start_client.sh [options]
|
||||
|
||||
Read root ./environment.client, update deploy/client/.env, then start the RTR debug client.
|
||||
|
||||
Options:
|
||||
--mode MODE Client mode: tcp|tls|ssh. Default: tcp
|
||||
--env FILE Client common env file. Default: ./environment.client
|
||||
-h, --help Show this help.
|
||||
EOF
|
||||
}
|
||||
|
||||
log() {
|
||||
printf '[11_start_client] %s\n' "$*" >&2
|
||||
}
|
||||
|
||||
stage() {
|
||||
printf '\n====== %s ======\n' "$*" >&2
|
||||
}
|
||||
|
||||
die() {
|
||||
printf '[11_start_client] ERROR: %s\n' "$*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
set_env_var() {
|
||||
local env_file="$1"
|
||||
local key="$2"
|
||||
local value="$3"
|
||||
local tmp_file
|
||||
|
||||
tmp_file="${env_file}.tmp.$$"
|
||||
awk -v key="${key}" -v value="${value}" '
|
||||
BEGIN { done = 0 }
|
||||
$0 ~ "^[[:space:]]*#?[[:space:]]*" key "=" {
|
||||
if (!done) {
|
||||
print key "=" value
|
||||
done = 1
|
||||
}
|
||||
next
|
||||
}
|
||||
{ print }
|
||||
END {
|
||||
if (!done) {
|
||||
print key "=" value
|
||||
}
|
||||
}
|
||||
' "${env_file}" >"${tmp_file}"
|
||||
mv "${tmp_file}" "${env_file}"
|
||||
}
|
||||
|
||||
compose_file_for_mode() {
|
||||
case "$1" in
|
||||
tcp) printf '%s\n' "${ROOT_DIR}/deploy/client/docker-compose.tcp.yml" ;;
|
||||
tls) printf '%s\n' "${ROOT_DIR}/deploy/client/docker-compose.tls.yml" ;;
|
||||
ssh) printf '%s\n' "${ROOT_DIR}/deploy/client/docker-compose.ssh.yml" ;;
|
||||
*) die "unsupported mode '$1', expected tcp|tls|ssh" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
--mode)
|
||||
[ "$#" -ge 2 ] || die "--mode requires a value"
|
||||
MODE="$2"
|
||||
shift 2
|
||||
;;
|
||||
--env)
|
||||
[ "$#" -ge 2 ] || die "--env requires a value"
|
||||
COMMON_ENV_FILE="$2"
|
||||
shift 2
|
||||
;;
|
||||
-h|--help)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
die "unknown option: $1"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
CLIENT_COMPOSE_FILE="$(compose_file_for_mode "${MODE}")"
|
||||
|
||||
stage "Checking Docker"
|
||||
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
||||
if docker compose version >/dev/null 2>&1; then
|
||||
COMPOSE_BIN=(docker compose)
|
||||
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
||||
COMPOSE_BIN=(docker-compose)
|
||||
else
|
||||
die "docker compose is not available"
|
||||
fi
|
||||
|
||||
[ -f "${CLIENT_ENV_FILE}" ] || die "deploy client env file not found: ${CLIENT_ENV_FILE}"
|
||||
[ -f "${CLIENT_COMPOSE_FILE}" ] || die "client compose file not found: ${CLIENT_COMPOSE_FILE}"
|
||||
|
||||
stage "Reading Client Environment"
|
||||
if [ -f "${COMMON_ENV_FILE}" ]; then
|
||||
updated_count=0
|
||||
while IFS= read -r line || [ -n "${line}" ]; do
|
||||
case "${line}" in
|
||||
''|\#*) continue ;;
|
||||
*=*)
|
||||
key="${line%%=*}"
|
||||
value="${line#*=}"
|
||||
set_env_var "${CLIENT_ENV_FILE}" "${key}" "${value}"
|
||||
updated_count=$((updated_count + 1))
|
||||
;;
|
||||
esac
|
||||
done <"${COMMON_ENV_FILE}"
|
||||
log "updated ${updated_count} client env variable(s)"
|
||||
else
|
||||
log "environment.client file not found, skip env update"
|
||||
fi
|
||||
|
||||
stage "Starting RTR Debug Client"
|
||||
log "mode: ${MODE}"
|
||||
"${COMPOSE_BIN[@]}" --env-file "${CLIENT_ENV_FILE}" -p "${COMPOSE_PROJECT}" -f "${CLIENT_COMPOSE_FILE}" up -d --no-build rtr-debug-client
|
||||
|
||||
log "checking client compose status"
|
||||
"${COMPOSE_BIN[@]}" --env-file "${CLIENT_ENV_FILE}" -p "${COMPOSE_PROJECT}" -f "${CLIENT_COMPOSE_FILE}" ps rtr-debug-client
|
||||
|
||||
stage "Client Started"
|
||||
@@ -1,82 +0,0 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||
|
||||
CLIENT_ENV_FILE="${ROOT_DIR}/deploy/client/.env"
|
||||
MODE="${RPKI_RTR_CLIENT_MODE:-tcp}"
|
||||
COMPOSE_PROJECT="${RPKI_RTR_CLIENT_COMPOSE_PROJECT:-rpki-rtr-client}"
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage: scripts/12_shutdown_client.sh [options]
|
||||
|
||||
Stop the RTR debug client started by scripts/11_start_client.sh.
|
||||
|
||||
Options:
|
||||
--mode MODE Client mode: tcp|tls|ssh. Default: tcp
|
||||
-h, --help Show this help.
|
||||
EOF
|
||||
}
|
||||
|
||||
log() {
|
||||
printf '[12_shutdown_client] %s\n' "$*" >&2
|
||||
}
|
||||
|
||||
stage() {
|
||||
printf '\n====== %s ======\n' "$*" >&2
|
||||
}
|
||||
|
||||
die() {
|
||||
printf '[12_shutdown_client] ERROR: %s\n' "$*" >&2
|
||||
exit 1
|
||||
}
|
||||
|
||||
compose_file_for_mode() {
|
||||
case "$1" in
|
||||
tcp) printf '%s\n' "${ROOT_DIR}/deploy/client/docker-compose.tcp.yml" ;;
|
||||
tls) printf '%s\n' "${ROOT_DIR}/deploy/client/docker-compose.tls.yml" ;;
|
||||
ssh) printf '%s\n' "${ROOT_DIR}/deploy/client/docker-compose.ssh.yml" ;;
|
||||
*) die "unsupported mode '$1', expected tcp|tls|ssh" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
while [ "$#" -gt 0 ]; do
|
||||
case "$1" in
|
||||
--mode)
|
||||
[ "$#" -ge 2 ] || die "--mode requires a value"
|
||||
MODE="$2"
|
||||
shift 2
|
||||
;;
|
||||
-h|--help)
|
||||
usage
|
||||
exit 0
|
||||
;;
|
||||
*)
|
||||
die "unknown option: $1"
|
||||
;;
|
||||
esac
|
||||
done
|
||||
|
||||
CLIENT_COMPOSE_FILE="$(compose_file_for_mode "${MODE}")"
|
||||
|
||||
stage "Checking Docker"
|
||||
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
||||
if docker compose version >/dev/null 2>&1; then
|
||||
COMPOSE_BIN=(docker compose)
|
||||
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
||||
COMPOSE_BIN=(docker-compose)
|
||||
else
|
||||
die "docker compose is not available"
|
||||
fi
|
||||
|
||||
stage "Stopping RTR Debug Client"
|
||||
log "mode: ${MODE}"
|
||||
if [ -f "${CLIENT_ENV_FILE}" ] && [ -f "${CLIENT_COMPOSE_FILE}" ]; then
|
||||
"${COMPOSE_BIN[@]}" --env-file "${CLIENT_ENV_FILE}" -p "${COMPOSE_PROJECT}" -f "${CLIENT_COMPOSE_FILE}" down --remove-orphans
|
||||
else
|
||||
log "client compose/env not found, skip"
|
||||
fi
|
||||
|
||||
stage "Client Shutdown Finished"
|
||||
@@ -1,218 +0,0 @@
|
||||
# 脚本说明
|
||||
|
||||
`scripts/` 目录用于离线安装、启动、健康检查、关闭和清理。
|
||||
|
||||
所有脚本建议在仓库根目录执行:
|
||||
|
||||
```bash
|
||||
bash scripts/00_0_install.sh
|
||||
```
|
||||
|
||||
## 目录
|
||||
|
||||
- [推荐流程](#推荐流程)
|
||||
- [环境变量文件](#环境变量文件)
|
||||
- [00 安装](#00-安装)
|
||||
- [00 Pre-Check](#00-pre-check)
|
||||
- [01 启动 Server 和 Console](#01-启动-server-和-console)
|
||||
- [02 自检](#02-自检)
|
||||
- [03 关闭 Server 和 Console](#03-关闭-server-和-console)
|
||||
- [04 清理](#04-清理)
|
||||
- [10 生成证书](#10-生成证书)
|
||||
- [11 启动 Client](#11-启动-client)
|
||||
- [12 关闭 Client](#12-关闭-client)
|
||||
- [Windows 路径注意事项](#windows-路径注意事项)
|
||||
|
||||
## 推荐流程
|
||||
|
||||
```bash
|
||||
# 1. 加载镜像并执行 pre-check。
|
||||
bash scripts/00_0_install.sh
|
||||
|
||||
# 2. 启动真实 server 和 console。
|
||||
bash scripts/01_start.sh --mode tcp
|
||||
|
||||
# 3. 检查 server 和 console 是否正常。
|
||||
bash scripts/02_self_check.sh
|
||||
|
||||
# 4. 关闭 server 和 console。
|
||||
bash scripts/03_shutdown.sh --mode tcp
|
||||
|
||||
# 5. 清理 server 运行产生的文件。
|
||||
bash scripts/04_cleanup.sh
|
||||
```
|
||||
|
||||
client 单独启动和关闭:
|
||||
|
||||
```bash
|
||||
bash scripts/11_start_client.sh --mode tcp
|
||||
bash scripts/12_shutdown_client.sh --mode tcp
|
||||
```
|
||||
|
||||
支持的模式:
|
||||
|
||||
- server:`tcp`、`tls`、`ssh`、`base`
|
||||
- client:`tcp`、`tls`、`ssh`
|
||||
|
||||
## 环境变量文件
|
||||
|
||||
`01_start.sh` 会读取根目录下的 `environment.server`。如果文件存在,会将其中所有 `KEY=VALUE` 同步到 `deploy/server/.env`。
|
||||
|
||||
示例:
|
||||
|
||||
```env
|
||||
RPKI_RTR_CCR_HOST_DIR=/opt/rpki-rtr/data
|
||||
RPKI_RTR_SLURM_HOST_DIR=/opt/rpki-rtr/data
|
||||
RPKI_RTR_DB_HOST_DIR=/opt/rpki-rtr/rtr-db
|
||||
RPKI_RTR_LOG_HOST_DIR=/opt/rpki-rtr/logs/server
|
||||
RPKI_RTR_REPORT_HOST_DIR=/opt/rpki-rtr/report
|
||||
```
|
||||
|
||||
`11_start_client.sh` 会读取根目录下的 `environment.client`。如果文件存在,会将其中所有 `KEY=VALUE` 同步到 `deploy/client/.env`。
|
||||
|
||||
如果 `environment.server` 或 `environment.client` 不存在,对应脚本会跳过更新,直接使用已有的 deploy `.env` 文件。
|
||||
|
||||
## 00 安装
|
||||
|
||||
`00_0_install.sh` 用于从 `images/` 目录加载离线 Docker 镜像,更新 deploy `.env` 中的镜像名,然后执行 pre-check。
|
||||
|
||||
默认镜像包查找规则:
|
||||
|
||||
- server:`images/rpki-rtr-*.tar`,排除 `rpki-rtr-debug-client-*.tar`
|
||||
- client:`images/rpki-rtr-debug-client-*.tar`
|
||||
- console:`images/rpki-console-ui-*.tar`
|
||||
|
||||
也可以显式指定镜像包:
|
||||
|
||||
```bash
|
||||
bash scripts/00_0_install.sh \
|
||||
--server-tar images/rpki-rtr-amd64-v1.2.tar \
|
||||
--client-tar images/rpki-rtr-debug-client-amd64-v1.2.tar \
|
||||
--console-tar images/rpki-console-ui-amd64-v1.0.tar
|
||||
```
|
||||
|
||||
脚本会从 `docker load` 输出中解析真实镜像名,并写入:
|
||||
|
||||
- `deploy/server/.env` 的 `RPKI_RTR_SERVER_IMAGE`
|
||||
- `deploy/console/.env` 的 `RPKI_CONSOLE_UI_IMAGE`
|
||||
|
||||
## 00 Pre-Check
|
||||
|
||||
`00_1_pre_check.sh` 会在 `.precheck/` 下准备隔离测试数据,使用仓库里的 Docker Compose 文件启动 server 和 debug client,然后检查:
|
||||
|
||||
- admin health 接口返回正常。
|
||||
- server 写出了 source report。
|
||||
- debug client 收到了 EndOfData 和 payload summary。
|
||||
- pre-check 日志中没有 fatal 错误模式。
|
||||
|
||||
相关辅助脚本:
|
||||
|
||||
- `00_2_generate_ccr.sh`:生成固定的 CCR 测试文件。
|
||||
- `00_2_pre_check_prepare.sh`:创建 `.precheck/` 数据、SLURM 文件和 env 文件。
|
||||
- `00_3_pre_check_cleanup.sh`:停止 pre-check 容器并清理 `.precheck/`。
|
||||
|
||||
## 01 启动 Server 和 Console
|
||||
|
||||
`01_start.sh` 用于启动真实的 RTR server 和 console。
|
||||
|
||||
```bash
|
||||
bash scripts/01_start.sh --mode tcp
|
||||
```
|
||||
|
||||
启动后脚本会检查容器是否进入 `running` 状态。如果容器启动后退出,脚本会打印 compose 状态和最近日志,然后报错退出,不会继续打印 `Start Finished`。
|
||||
|
||||
常用参数:
|
||||
|
||||
```bash
|
||||
bash scripts/01_start.sh --mode tls
|
||||
bash scripts/01_start.sh --mode ssh
|
||||
bash scripts/01_start.sh --wait 60
|
||||
bash scripts/01_start.sh --env ./environment.server
|
||||
```
|
||||
|
||||
注意:console 不读取 `environment.server`,只使用 `deploy/console/.env`。
|
||||
|
||||
## 02 自检
|
||||
|
||||
`02_self_check.sh` 用于检查 `01_start.sh` 启动的 server 和 console 是否正常。
|
||||
|
||||
```bash
|
||||
bash scripts/02_self_check.sh
|
||||
```
|
||||
|
||||
检查内容包括:
|
||||
|
||||
- server 容器是否 running。
|
||||
- console 容器是否 running。
|
||||
- RTR TCP 端口是否可连接。
|
||||
- admin health 接口是否健康。
|
||||
- console 端口和 HTTP 根路径是否可访问。
|
||||
|
||||
如果启动较慢,可以增加等待时间:
|
||||
|
||||
```bash
|
||||
bash scripts/02_self_check.sh --wait 120
|
||||
```
|
||||
|
||||
## 03 关闭 Server 和 Console
|
||||
|
||||
`03_shutdown.sh` 用于关闭 `01_start.sh` 启动的服务。
|
||||
|
||||
```bash
|
||||
bash scripts/03_shutdown.sh --mode tcp
|
||||
```
|
||||
|
||||
脚本会先关闭 console,再关闭 RTR server。
|
||||
|
||||
## 04 清理
|
||||
|
||||
`04_cleanup.sh` 用于删除 server 运行产生的目录:
|
||||
|
||||
- `logs/`
|
||||
- `report/`
|
||||
- `rtr-db/`
|
||||
|
||||
推荐先关闭服务,再清理:
|
||||
|
||||
```bash
|
||||
bash scripts/03_shutdown.sh
|
||||
bash scripts/04_cleanup.sh
|
||||
```
|
||||
|
||||
清理前脚本会检查 `rpki-rtr` 是否仍在运行。如果仍在运行,会拒绝清理。只有明确需要强制清理时才使用:
|
||||
|
||||
```bash
|
||||
bash scripts/04_cleanup.sh --force
|
||||
```
|
||||
|
||||
清理脚本会拒绝删除仓库根目录之外的路径。如果某一项清理失败,会继续清理后续项,最后返回失败。
|
||||
|
||||
## 10 生成证书
|
||||
|
||||
`10_generate_certs.sh` 用于生成 TLS/SSH 模式需要的本地证书材料(测试用)。
|
||||
|
||||
```bash
|
||||
bash scripts/10_generate_certs.sh
|
||||
```
|
||||
|
||||
## 11 启动 Client
|
||||
|
||||
`11_start_client.sh` 用于启动 RTR debug client。
|
||||
|
||||
```bash
|
||||
bash scripts/11_start_client.sh --mode tcp
|
||||
bash scripts/11_start_client.sh --mode tls
|
||||
bash scripts/11_start_client.sh --mode ssh
|
||||
```
|
||||
|
||||
如果 `environment.client` 存在,脚本会将其中所有 `KEY=VALUE` 同步到 `deploy/client/.env`;如果不存在,则直接使用已有的 `deploy/client/.env`。
|
||||
|
||||
## 12 关闭 Client
|
||||
|
||||
`12_shutdown_client.sh` 用于关闭 `11_start_client.sh` 启动的 client。
|
||||
|
||||
```bash
|
||||
bash scripts/12_shutdown_client.sh --mode tcp
|
||||
```
|
||||
|
||||
关闭时建议使用和启动时相同的 mode。
|
||||
70
scripts/benchmark/README.md
Normal file
70
scripts/benchmark/README.md
Normal file
@@ -0,0 +1,70 @@
|
||||
# RPKI Benchmarks (Stage2, selected_der_v2)
|
||||
|
||||
This directory contains a reproducible, one-click benchmark to measure **decode + profile validate**
|
||||
performance for all supported object types and compare **OURS** against the **Routinator baseline**
|
||||
(`rpki` crate `=0.19.1` with `repository` feature).
|
||||
|
||||
## What it measures
|
||||
|
||||
Dataset:
|
||||
|
||||
- Fixtures: `rpki/tests/benchmark/selected_der_v2/`
|
||||
- Objects: `cer`, `crl`, `manifest` (`.mft`), `roa`, `aspa` (`.asa`)
|
||||
- Samples: 10 quantiles per type (`min/p01/p10/p25/p50/p75/p90/p95/p99/max`) → 50 files total
|
||||
|
||||
Metrics:
|
||||
|
||||
- **decode+validate**: `decode_der` (parse + profile validate) for each object file
|
||||
- **landing** (OURS only): `PackFile::from_bytes_compute_sha256` + CBOR encode + `RocksDB put_raw`
|
||||
- **compare**: ratio `ours_ns/op ÷ rout_ns/op` for decode+validate
|
||||
|
||||
## Default benchmark settings
|
||||
|
||||
Both OURS and Routinator baseline use the same run settings:
|
||||
|
||||
- warmup: `10` iterations
|
||||
- rounds: `3`
|
||||
- adaptive loop target: `min_round_ms=200` (with an internal max of `1_000_000` iters)
|
||||
- strict DER: `true` (baseline)
|
||||
- cert inspect: `false` (baseline)
|
||||
|
||||
You can override the settings via environment variables in the runner script:
|
||||
|
||||
- `BENCH_WARMUP_ITERS` (default `10`)
|
||||
- `BENCH_ROUNDS` (default `3`)
|
||||
- `BENCH_MIN_ROUND_MS` (default `200`)
|
||||
|
||||
## One-click run (OURS + Routinator compare)
|
||||
|
||||
From the `rpki/` crate directory:
|
||||
|
||||
```bash
|
||||
./scripts/benchmark/run_stage2_selected_der_v2_release.sh
|
||||
```
|
||||
|
||||
Outputs are written under:
|
||||
|
||||
- `rpki/target/bench/`
|
||||
- OURS decode+validate: `stage2_selected_der_v2_decode_release_<TS>.{md,csv}`
|
||||
- OURS landing: `stage2_selected_der_v2_landing_release_<TS>.{md,csv}`
|
||||
- Routinator: `stage2_selected_der_v2_routinator_decode_release_<TS>.{md,csv}`
|
||||
- Compare: `stage2_selected_der_v2_compare_ours_vs_routinator_decode_release_<TS>.{md,csv}`
|
||||
- Summary: `stage2_selected_der_v2_compare_summary_<TS>.md`
|
||||
|
||||
### Why decode and landing are separated
|
||||
|
||||
The underlying benchmark can run in `BENCH_MODE=both`, but the **landing** part writes to RocksDB
|
||||
and may trigger background work (e.g., compactions) that can **skew subsequent decode timings**.
|
||||
For a fair OURS-vs-Routinator comparison, the runner script:
|
||||
|
||||
- runs `BENCH_MODE=decode_validate` for comparison, and
|
||||
- runs `BENCH_MODE=landing` separately for landing-only numbers.
|
||||
|
||||
## Notes
|
||||
|
||||
- The Routinator baseline benchmark is implemented in-repo under:
|
||||
- `rpki/benchmark/routinator_object_bench/`
|
||||
- It pins `rpki = "=0.19.1"` in its `Cargo.toml`.
|
||||
- This benchmark is implemented as an `#[ignore]` integration test:
|
||||
- `rpki/tests/bench_stage2_decode_profile_selected_der_v2.rs`
|
||||
- The runner script invokes it with `cargo test --release ... -- --ignored --nocapture`.
|
||||
123
scripts/benchmark/run_stage2_selected_der_v2_release.sh
Executable file
123
scripts/benchmark/run_stage2_selected_der_v2_release.sh
Executable file
@@ -0,0 +1,123 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Stage2 (selected_der_v2) decode+profile validate benchmark.
|
||||
# Runs:
|
||||
# 1) OURS decode+validate benchmark and writes MD/CSV.
|
||||
# 2) OURS landing benchmark and writes MD/CSV.
|
||||
# 3) Routinator baseline decode benchmark (rpki crate =0.19.1).
|
||||
# 4) Produces a joined compare CSV/MD and a short geomean summary.
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
cd "$ROOT_DIR"
|
||||
|
||||
OUT_DIR="$ROOT_DIR/target/bench"
|
||||
mkdir -p "$OUT_DIR"
|
||||
|
||||
TS="$(date -u +%Y%m%dT%H%M%SZ)"
|
||||
|
||||
WARMUP_ITERS="${BENCH_WARMUP_ITERS:-10}"
|
||||
ROUNDS="${BENCH_ROUNDS:-3}"
|
||||
MIN_ROUND_MS="${BENCH_MIN_ROUND_MS:-200}"
|
||||
|
||||
OURS_MD="$OUT_DIR/stage2_selected_der_v2_decode_release_${TS}.md"
|
||||
OURS_CSV="$OUT_DIR/stage2_selected_der_v2_decode_release_${TS}.csv"
|
||||
|
||||
OURS_LANDING_MD="$OUT_DIR/stage2_selected_der_v2_landing_release_${TS}.md"
|
||||
OURS_LANDING_CSV="$OUT_DIR/stage2_selected_der_v2_landing_release_${TS}.csv"
|
||||
|
||||
ROUT_MD="$OUT_DIR/stage2_selected_der_v2_routinator_decode_release_${TS}.md"
|
||||
ROUT_CSV="$OUT_DIR/stage2_selected_der_v2_routinator_decode_release_${TS}.csv"
|
||||
|
||||
COMPARE_MD="$OUT_DIR/stage2_selected_der_v2_compare_ours_vs_routinator_decode_release_${TS}.md"
|
||||
COMPARE_CSV="$OUT_DIR/stage2_selected_der_v2_compare_ours_vs_routinator_decode_release_${TS}.csv"
|
||||
|
||||
SUMMARY_MD="$OUT_DIR/stage2_selected_der_v2_compare_summary_${TS}.md"
|
||||
|
||||
echo "[1/4] OURS: decode+validate benchmark (release)..." >&2
|
||||
BENCH_MODE="decode_validate" \
|
||||
BENCH_WARMUP_ITERS="$WARMUP_ITERS" \
|
||||
BENCH_ROUNDS="$ROUNDS" \
|
||||
BENCH_MIN_ROUND_MS="$MIN_ROUND_MS" \
|
||||
BENCH_OUT_MD="$OURS_MD" \
|
||||
BENCH_OUT_CSV="$OURS_CSV" \
|
||||
cargo test --release --test bench_stage2_decode_profile_selected_der_v2 -- --ignored --nocapture >/dev/null
|
||||
|
||||
echo "[2/4] OURS: landing benchmark (release)..." >&2
|
||||
BENCH_MODE="landing" \
|
||||
BENCH_WARMUP_ITERS="$WARMUP_ITERS" \
|
||||
BENCH_ROUNDS="$ROUNDS" \
|
||||
BENCH_MIN_ROUND_MS="$MIN_ROUND_MS" \
|
||||
BENCH_OUT_MD_LANDING="$OURS_LANDING_MD" \
|
||||
BENCH_OUT_CSV_LANDING="$OURS_LANDING_CSV" \
|
||||
cargo test --release --test bench_stage2_decode_profile_selected_der_v2 -- --ignored --nocapture >/dev/null
|
||||
|
||||
echo "[3/4] Routinator baseline + compare join..." >&2
|
||||
OURS_CSV="$OURS_CSV" \
|
||||
ROUT_CSV="$ROUT_CSV" \
|
||||
ROUT_MD="$ROUT_MD" \
|
||||
COMPARE_CSV="$COMPARE_CSV" \
|
||||
COMPARE_MD="$COMPARE_MD" \
|
||||
WARMUP_ITERS="$WARMUP_ITERS" \
|
||||
ROUNDS="$ROUNDS" \
|
||||
MIN_ROUND_MS="$MIN_ROUND_MS" \
|
||||
scripts/stage2_perf_compare_m4.sh >/dev/null
|
||||
|
||||
echo "[4/4] Summary (geomean ratios)..." >&2
|
||||
python3 - "$COMPARE_CSV" "$SUMMARY_MD" <<'PY'
|
||||
import csv
|
||||
import math
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from datetime import datetime, timezone
|
||||
|
||||
in_csv = Path(sys.argv[1])
|
||||
out_md = Path(sys.argv[2])
|
||||
|
||||
rows = list(csv.DictReader(in_csv.open(newline="")))
|
||||
ratios = {}
|
||||
for r in rows:
|
||||
ratios.setdefault(r["type"], []).append(float(r["ratio_ours_over_rout"]))
|
||||
|
||||
def geomean(vals):
|
||||
return math.exp(sum(math.log(v) for v in vals) / len(vals))
|
||||
|
||||
def p50(vals):
|
||||
v = sorted(vals)
|
||||
n = len(v)
|
||||
if n % 2 == 1:
|
||||
return v[n // 2]
|
||||
return (v[n // 2 - 1] + v[n // 2]) / 2.0
|
||||
|
||||
all_vals = [float(r["ratio_ours_over_rout"]) for r in rows]
|
||||
types = ["all"] + sorted(ratios.keys())
|
||||
|
||||
now = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
lines = []
|
||||
lines.append("# Stage2 selected_der_v2 compare summary (release)\n\n")
|
||||
lines.append(f"- recorded_at_utc: `{now}`\n")
|
||||
lines.append(f"- inputs_csv: `{in_csv}`\n\n")
|
||||
lines.append("| type | n | min | p50 | geomean | max | >1 count |\n")
|
||||
lines.append("|---|---:|---:|---:|---:|---:|---:|\n")
|
||||
|
||||
for t in types:
|
||||
vals = all_vals if t == "all" else ratios[t]
|
||||
vals_sorted = sorted(vals)
|
||||
lines.append(
|
||||
f"| {t} | {len(vals_sorted)} | {vals_sorted[0]:.4f} | {p50(vals_sorted):.4f} | "
|
||||
f"{geomean(vals_sorted):.4f} | {vals_sorted[-1]:.4f} | {sum(1 for v in vals_sorted if v>1.0)} |\n"
|
||||
)
|
||||
|
||||
out_md.write_text("".join(lines), encoding="utf-8")
|
||||
print(out_md)
|
||||
PY
|
||||
|
||||
echo "Done." >&2
|
||||
echo "- OURS decode MD: $OURS_MD" >&2
|
||||
echo "- OURS decode CSV: $OURS_CSV" >&2
|
||||
echo "- OURS landing MD: $OURS_LANDING_MD" >&2
|
||||
echo "- OURS landing CSV: $OURS_LANDING_CSV" >&2
|
||||
echo "- Routinator: $ROUT_MD" >&2
|
||||
echo "- Compare MD: $COMPARE_MD" >&2
|
||||
echo "- Compare CSV: $COMPARE_CSV" >&2
|
||||
echo "- Summary MD: $SUMMARY_MD" >&2
|
||||
56
scripts/cir/README.md
Normal file
56
scripts/cir/README.md
Normal file
@@ -0,0 +1,56 @@
|
||||
# CIR Scripts
|
||||
|
||||
## `cir-rsync-wrapper`
|
||||
|
||||
一个用于 CIR 黑盒 replay 的 rsync wrapper。
|
||||
|
||||
### 环境变量
|
||||
|
||||
- `REAL_RSYNC_BIN`
|
||||
- 真实 rsync 二进制路径
|
||||
- 默认优先 `/usr/bin/rsync`
|
||||
- `CIR_MIRROR_ROOT`
|
||||
- 本地镜像树根目录
|
||||
- 当命令行中出现 `rsync://...` source 时必需
|
||||
|
||||
### 语义
|
||||
|
||||
- 仅改写 `rsync://host/path` 类型参数
|
||||
- 其它参数原样透传给真实 rsync
|
||||
- 改写目标:
|
||||
- `rsync://example.net/repo/a.roa`
|
||||
- →
|
||||
- `<CIR_MIRROR_ROOT>/example.net/repo/a.roa`
|
||||
|
||||
### 兼容目标
|
||||
|
||||
- Routinator `--rsync-command`
|
||||
- `rpki-client -e rsync_prog`
|
||||
|
||||
## 其它脚本
|
||||
|
||||
- `run_cir_replay_ours.sh`
|
||||
- `run_cir_replay_routinator.sh`
|
||||
- `run_cir_replay_rpki_client.sh`
|
||||
- `run_cir_replay_matrix.sh`
|
||||
|
||||
## `cir-local-link-sync.py`
|
||||
|
||||
当 `CIR_LOCAL_LINK_MODE=1` 且 wrapper 检测到 source 已经被改写为本地 mirror 路径时,
|
||||
wrapper 不再调用真实 `rsync`,而是调用这个 helper 完成:
|
||||
|
||||
- `hardlink` 优先的本地树同步
|
||||
- 失败时回退到 copy
|
||||
- 支持 `--delete`
|
||||
|
||||
`run_cir_replay_matrix.sh` 会顺序执行:
|
||||
|
||||
- `ours`
|
||||
- Routinator
|
||||
- `rpki-client`
|
||||
|
||||
并汇总生成:
|
||||
|
||||
- `summary.json`
|
||||
- `summary.md`
|
||||
- `detail.md`
|
||||
BIN
scripts/cir/__pycache__/cir-local-link-sync.cpython-312.pyc
Normal file
BIN
scripts/cir/__pycache__/cir-local-link-sync.cpython-312.pyc
Normal file
Binary file not shown.
BIN
scripts/cir/__pycache__/cir-rsync-wrappercpython-312.pyc
Normal file
BIN
scripts/cir/__pycache__/cir-rsync-wrappercpython-312.pyc
Normal file
Binary file not shown.
BIN
scripts/cir/__pycache__/json_to_vaps_csv.cpython-312.pyc
Normal file
BIN
scripts/cir/__pycache__/json_to_vaps_csv.cpython-312.pyc
Normal file
Binary file not shown.
136
scripts/cir/cir-local-link-sync.py
Executable file
136
scripts/cir/cir-local-link-sync.py
Executable file
@@ -0,0 +1,136 @@
|
||||
#!/usr/bin/env python3
|
||||
import argparse
|
||||
import errno
|
||||
import os
|
||||
import shutil
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def _same_inode(src: Path, dst: Path) -> bool:
|
||||
try:
|
||||
src_stat = src.stat()
|
||||
dst_stat = dst.stat()
|
||||
except FileNotFoundError:
|
||||
return False
|
||||
return (src_stat.st_dev, src_stat.st_ino) == (dst_stat.st_dev, dst_stat.st_ino)
|
||||
|
||||
|
||||
def _remove_path(path: Path) -> None:
|
||||
if not path.exists() and not path.is_symlink():
|
||||
return
|
||||
if path.is_dir() and not path.is_symlink():
|
||||
shutil.rmtree(path)
|
||||
else:
|
||||
path.unlink()
|
||||
|
||||
|
||||
def _prune_empty_dirs(root: Path) -> None:
|
||||
if not root.exists():
|
||||
return
|
||||
for path in sorted((p for p in root.rglob("*") if p.is_dir()), key=lambda p: len(p.parts), reverse=True):
|
||||
try:
|
||||
path.rmdir()
|
||||
except OSError:
|
||||
pass
|
||||
|
||||
|
||||
def _link_or_copy(src: Path, dst: Path) -> str:
|
||||
dst.parent.mkdir(parents=True, exist_ok=True)
|
||||
if dst.exists() or dst.is_symlink():
|
||||
if _same_inode(src, dst):
|
||||
return "reused"
|
||||
_remove_path(dst)
|
||||
try:
|
||||
os.link(src, dst)
|
||||
return "linked"
|
||||
except OSError as err:
|
||||
if err.errno not in (errno.EXDEV, errno.EPERM, errno.EMLINK, errno.ENOTSUP, errno.EACCES):
|
||||
raise
|
||||
shutil.copy2(src, dst)
|
||||
return "copied"
|
||||
|
||||
|
||||
def _file_map(src_arg: str, dest_arg: str) -> tuple[Path, dict[str, Path]]:
|
||||
src = Path(src_arg.rstrip(os.sep))
|
||||
if not src.exists():
|
||||
raise FileNotFoundError(src)
|
||||
mapping: dict[str, Path] = {}
|
||||
if src.is_dir():
|
||||
copy_contents = src_arg.endswith(os.sep)
|
||||
if copy_contents:
|
||||
root = src
|
||||
for path in root.rglob("*"):
|
||||
if path.is_file():
|
||||
mapping[path.relative_to(root).as_posix()] = path
|
||||
else:
|
||||
root = src
|
||||
base = src.name
|
||||
for path in root.rglob("*"):
|
||||
if path.is_file():
|
||||
rel = Path(base) / path.relative_to(root)
|
||||
mapping[rel.as_posix()] = path
|
||||
else:
|
||||
dest_path = Path(dest_arg)
|
||||
if dest_arg.endswith(os.sep) or dest_path.is_dir():
|
||||
mapping[src.name] = src
|
||||
else:
|
||||
mapping[dest_path.name] = src
|
||||
return Path(dest_arg), mapping
|
||||
|
||||
|
||||
def sync_local_tree(src_arg: str, dst_arg: str, delete: bool) -> dict[str, int]:
|
||||
dst_root, mapping = _file_map(src_arg, dst_arg)
|
||||
dst_root.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
expected = {dst_root / rel for rel in mapping.keys()}
|
||||
|
||||
deleted = 0
|
||||
if delete and dst_root.exists():
|
||||
for path in sorted(dst_root.rglob("*"), key=lambda p: len(p.parts), reverse=True):
|
||||
if path.is_dir():
|
||||
continue
|
||||
if path not in expected:
|
||||
_remove_path(path)
|
||||
deleted += 1
|
||||
_prune_empty_dirs(dst_root)
|
||||
|
||||
linked = 0
|
||||
copied = 0
|
||||
reused = 0
|
||||
for rel, src in mapping.items():
|
||||
dst = dst_root / rel
|
||||
result = _link_or_copy(src, dst)
|
||||
if result == "linked":
|
||||
linked += 1
|
||||
elif result == "copied":
|
||||
copied += 1
|
||||
else:
|
||||
reused += 1
|
||||
|
||||
return {
|
||||
"files": len(mapping),
|
||||
"linked": linked,
|
||||
"copied": copied,
|
||||
"reused": reused,
|
||||
"deleted": deleted,
|
||||
}
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser(description="Sync a local CIR mirror tree using hardlinks when possible.")
|
||||
parser.add_argument("--delete", action="store_true", help="Delete target files not present in source")
|
||||
parser.add_argument("source")
|
||||
parser.add_argument("dest")
|
||||
args = parser.parse_args()
|
||||
|
||||
summary = sync_local_tree(args.source, args.dest, args.delete)
|
||||
print(
|
||||
"local-link-sync files={files} linked={linked} copied={copied} reused={reused} deleted={deleted}".format(
|
||||
**summary
|
||||
)
|
||||
)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
127
scripts/cir/cir-rsync-wrapper
Executable file
127
scripts/cir/cir-rsync-wrapper
Executable file
@@ -0,0 +1,127 @@
|
||||
#!/usr/bin/env python3
|
||||
import os
|
||||
import shutil
|
||||
import sys
|
||||
from pathlib import Path
|
||||
from urllib.parse import urlparse
|
||||
|
||||
|
||||
def real_rsync_bin() -> str:
|
||||
env = os.environ.get("REAL_RSYNC_BIN")
|
||||
if env:
|
||||
return env
|
||||
default = "/usr/bin/rsync"
|
||||
if Path(default).exists():
|
||||
return default
|
||||
found = shutil.which("rsync")
|
||||
if found:
|
||||
return found
|
||||
raise SystemExit("cir-rsync-wrapper: REAL_RSYNC_BIN is not set and rsync was not found")
|
||||
|
||||
|
||||
def rewrite_arg(arg: str, mirror_root: str | None) -> str:
|
||||
if not arg.startswith("rsync://"):
|
||||
return arg
|
||||
if not mirror_root:
|
||||
raise SystemExit(
|
||||
"cir-rsync-wrapper: CIR_MIRROR_ROOT is required when an rsync:// source is present"
|
||||
)
|
||||
parsed = urlparse(arg)
|
||||
if parsed.scheme != "rsync" or not parsed.hostname:
|
||||
raise SystemExit(f"cir-rsync-wrapper: invalid rsync URI: {arg}")
|
||||
path = parsed.path.lstrip("/")
|
||||
local = Path(mirror_root).resolve() / parsed.hostname
|
||||
if path:
|
||||
local = local / path
|
||||
local_str = str(local)
|
||||
if local.exists() and local.is_dir() and not local_str.endswith("/"):
|
||||
local_str += "/"
|
||||
elif arg.endswith("/") and not local_str.endswith("/"):
|
||||
local_str += "/"
|
||||
return local_str
|
||||
|
||||
|
||||
def filter_args(args: list[str]) -> list[str]:
|
||||
mirror_root = os.environ.get("CIR_MIRROR_ROOT")
|
||||
rewritten_any = any(arg.startswith("rsync://") for arg in args)
|
||||
out: list[str] = []
|
||||
i = 0
|
||||
while i < len(args):
|
||||
arg = args[i]
|
||||
if rewritten_any:
|
||||
if arg == "--address":
|
||||
i += 2
|
||||
continue
|
||||
if arg.startswith("--address="):
|
||||
i += 1
|
||||
continue
|
||||
if arg == "--contimeout":
|
||||
i += 2
|
||||
continue
|
||||
if arg.startswith("--contimeout="):
|
||||
i += 1
|
||||
continue
|
||||
out.append(rewrite_arg(arg, mirror_root))
|
||||
i += 1
|
||||
return out
|
||||
|
||||
|
||||
def local_link_mode_enabled() -> bool:
|
||||
value = os.environ.get("CIR_LOCAL_LINK_MODE", "")
|
||||
return value.lower() in {"1", "true", "yes", "on"}
|
||||
|
||||
|
||||
def extract_source_and_dest(args: list[str]) -> tuple[str, str]:
|
||||
expects_value = {
|
||||
"--timeout",
|
||||
"--min-size",
|
||||
"--max-size",
|
||||
"--include",
|
||||
"--exclude",
|
||||
"--compare-dest",
|
||||
}
|
||||
positionals: list[str] = []
|
||||
i = 0
|
||||
while i < len(args):
|
||||
arg = args[i]
|
||||
if arg in expects_value:
|
||||
i += 2
|
||||
continue
|
||||
if any(arg.startswith(prefix + "=") for prefix in expects_value):
|
||||
i += 1
|
||||
continue
|
||||
if arg.startswith("-"):
|
||||
i += 1
|
||||
continue
|
||||
positionals.append(arg)
|
||||
i += 1
|
||||
if len(positionals) < 2:
|
||||
raise SystemExit("cir-rsync-wrapper: expected source and destination arguments")
|
||||
return positionals[-2], positionals[-1]
|
||||
|
||||
|
||||
def maybe_exec_local_link_sync(args: list[str], rewritten_any: bool) -> None:
|
||||
if not rewritten_any or not local_link_mode_enabled():
|
||||
return
|
||||
source, dest = extract_source_and_dest(args)
|
||||
if source.startswith("rsync://"):
|
||||
raise SystemExit("cir-rsync-wrapper: expected rewritten local source for CIR_LOCAL_LINK_MODE")
|
||||
helper = Path(__file__).with_name("cir-local-link-sync.py")
|
||||
cmd = [sys.executable, str(helper)]
|
||||
if "--delete" in args:
|
||||
cmd.append("--delete")
|
||||
cmd.extend([source, dest])
|
||||
os.execv(sys.executable, cmd)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
args = sys.argv[1:]
|
||||
rewritten_any = any(arg.startswith("rsync://") for arg in args)
|
||||
rewritten = filter_args(args)
|
||||
maybe_exec_local_link_sync(rewritten, rewritten_any)
|
||||
os.execv(real_rsync_bin(), [real_rsync_bin(), *rewritten])
|
||||
return 127
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
32
scripts/cir/fetch_cir_sequence_from_remote.sh
Executable file
32
scripts/cir/fetch_cir_sequence_from_remote.sh
Executable file
@@ -0,0 +1,32 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/cir/fetch_cir_sequence_from_remote.sh \
|
||||
--ssh-target <user@host> \
|
||||
--remote-path <path> \
|
||||
--local-path <path>
|
||||
EOF
|
||||
}
|
||||
|
||||
SSH_TARGET=""
|
||||
REMOTE_PATH=""
|
||||
LOCAL_PATH=""
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--ssh-target) SSH_TARGET="$2"; shift 2 ;;
|
||||
--remote-path) REMOTE_PATH="$2"; shift 2 ;;
|
||||
--local-path) LOCAL_PATH="$2"; shift 2 ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -n "$SSH_TARGET" && -n "$REMOTE_PATH" && -n "$LOCAL_PATH" ]] || { usage >&2; exit 2; }
|
||||
|
||||
mkdir -p "$(dirname "$LOCAL_PATH")"
|
||||
rsync -a "$SSH_TARGET:$REMOTE_PATH/" "$LOCAL_PATH/"
|
||||
echo "done: $LOCAL_PATH"
|
||||
50
scripts/cir/json_to_vaps_csv.py
Executable file
50
scripts/cir/json_to_vaps_csv.py
Executable file
@@ -0,0 +1,50 @@
|
||||
#!/usr/bin/env python3
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
import argparse
|
||||
import csv
|
||||
import json
|
||||
from pathlib import Path
|
||||
|
||||
|
||||
def normalize_asn(value: str | int) -> str:
|
||||
text = str(value).strip().upper()
|
||||
if text.startswith("AS"):
|
||||
text = text[2:]
|
||||
return f"AS{int(text)}"
|
||||
|
||||
|
||||
def main() -> int:
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument("--input", required=True, type=Path)
|
||||
parser.add_argument("--csv-out", required=True, type=Path)
|
||||
args = parser.parse_args()
|
||||
|
||||
obj = json.loads(args.input.read_text(encoding="utf-8"))
|
||||
rows: list[tuple[str, str, str]] = []
|
||||
for aspa in obj.get("aspas", []):
|
||||
providers = sorted(
|
||||
{normalize_asn(item) for item in aspa.get("providers", [])},
|
||||
key=lambda s: int(s[2:]),
|
||||
)
|
||||
rows.append(
|
||||
(
|
||||
normalize_asn(aspa["customer"]),
|
||||
";".join(providers),
|
||||
str(aspa.get("ta", "")).strip().lower(),
|
||||
)
|
||||
)
|
||||
rows.sort(key=lambda row: (int(row[0][2:]), row[1], row[2]))
|
||||
|
||||
args.csv_out.parent.mkdir(parents=True, exist_ok=True)
|
||||
with args.csv_out.open("w", encoding="utf-8", newline="") as fh:
|
||||
writer = csv.writer(fh)
|
||||
writer.writerow(["Customer ASN", "Providers", "Trust Anchor"])
|
||||
writer.writerows(rows)
|
||||
print(args.csv_out)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
raise SystemExit(main())
|
||||
84
scripts/cir/run_cir_drop_sequence.sh
Executable file
84
scripts/cir/run_cir_drop_sequence.sh
Executable file
@@ -0,0 +1,84 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/cir/run_cir_drop_sequence.sh \
|
||||
--sequence-root <path> \
|
||||
[--drop-bin <path>]
|
||||
EOF
|
||||
}
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
SEQUENCE_ROOT=""
|
||||
DROP_BIN="${DROP_BIN:-$ROOT_DIR/target/release/cir_drop_report}"
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--sequence-root) SEQUENCE_ROOT="$2"; shift 2 ;;
|
||||
--drop-bin) DROP_BIN="$2"; shift 2 ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -n "$SEQUENCE_ROOT" ]] || { usage >&2; exit 2; }
|
||||
|
||||
python3 - <<'PY' "$SEQUENCE_ROOT" "$DROP_BIN"
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sequence_root = Path(sys.argv[1]).resolve()
|
||||
drop_bin = sys.argv[2]
|
||||
sequence = json.loads((sequence_root / "sequence.json").read_text(encoding="utf-8"))
|
||||
static_root = sequence_root / sequence["staticRoot"] if "staticRoot" in sequence else None
|
||||
raw_store_db = sequence_root / sequence["rawStoreDbPath"] if "rawStoreDbPath" in sequence else None
|
||||
backend_count = sum(x is not None for x in (static_root, raw_store_db))
|
||||
if backend_count != 1:
|
||||
raise SystemExit("sequence must set exactly one of staticRoot or rawStoreDbPath")
|
||||
|
||||
summaries = []
|
||||
for step in sequence["steps"]:
|
||||
step_id = step["stepId"]
|
||||
out_dir = sequence_root / "drop" / step_id
|
||||
out_dir.mkdir(parents=True, exist_ok=True)
|
||||
cmd = [
|
||||
drop_bin,
|
||||
"--cir",
|
||||
str(sequence_root / step["cirPath"]),
|
||||
"--ccr",
|
||||
str(sequence_root / step["ccrPath"]),
|
||||
"--report-json",
|
||||
str(sequence_root / step["reportPath"]),
|
||||
"--json-out",
|
||||
str(out_dir / "drop.json"),
|
||||
"--md-out",
|
||||
str(out_dir / "drop.md"),
|
||||
]
|
||||
if static_root is not None:
|
||||
cmd.extend(["--static-root", str(static_root)])
|
||||
else:
|
||||
cmd.extend(["--raw-store-db", str(raw_store_db)])
|
||||
proc = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if proc.returncode != 0:
|
||||
raise SystemExit(
|
||||
f"drop report failed for {step_id}: stdout={proc.stdout} stderr={proc.stderr}"
|
||||
)
|
||||
result = json.loads((out_dir / "drop.json").read_text(encoding="utf-8"))
|
||||
summaries.append(
|
||||
{
|
||||
"stepId": step_id,
|
||||
"droppedVrpCount": result["summary"]["droppedVrpCount"],
|
||||
"droppedObjectCount": result["summary"]["droppedObjectCount"],
|
||||
"reportPath": str(out_dir / "drop.json"),
|
||||
}
|
||||
)
|
||||
|
||||
summary = {"version": 1, "steps": summaries}
|
||||
(sequence_root / "drop-summary.json").write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
||||
PY
|
||||
|
||||
echo "done: $SEQUENCE_ROOT"
|
||||
173
scripts/cir/run_cir_record_full_delta.sh
Executable file
173
scripts/cir/run_cir_record_full_delta.sh
Executable file
@@ -0,0 +1,173 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/cir/run_cir_record_full_delta.sh \
|
||||
--out-dir <path> \
|
||||
--tal-path <path> \
|
||||
--ta-path <path> \
|
||||
--cir-tal-uri <url> \
|
||||
--payload-replay-archive <path> \
|
||||
--payload-replay-locks <path> \
|
||||
--payload-base-archive <path> \
|
||||
--payload-base-locks <path> \
|
||||
--payload-delta-archive <path> \
|
||||
--payload-delta-locks <path> \
|
||||
[--base-validation-time <rfc3339>] \
|
||||
[--delta-validation-time <rfc3339>] \
|
||||
[--max-depth <n>] \
|
||||
[--max-instances <n>] \
|
||||
[--rpki-bin <path>]
|
||||
EOF
|
||||
}
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
|
||||
OUT_DIR=""
|
||||
TAL_PATH=""
|
||||
TA_PATH=""
|
||||
CIR_TAL_URI=""
|
||||
PAYLOAD_REPLAY_ARCHIVE=""
|
||||
PAYLOAD_REPLAY_LOCKS=""
|
||||
PAYLOAD_BASE_ARCHIVE=""
|
||||
PAYLOAD_BASE_LOCKS=""
|
||||
PAYLOAD_DELTA_ARCHIVE=""
|
||||
PAYLOAD_DELTA_LOCKS=""
|
||||
BASE_VALIDATION_TIME=""
|
||||
DELTA_VALIDATION_TIME=""
|
||||
MAX_DEPTH=0
|
||||
MAX_INSTANCES=1
|
||||
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--out-dir) OUT_DIR="$2"; shift 2 ;;
|
||||
--tal-path) TAL_PATH="$2"; shift 2 ;;
|
||||
--ta-path) TA_PATH="$2"; shift 2 ;;
|
||||
--cir-tal-uri) CIR_TAL_URI="$2"; shift 2 ;;
|
||||
--payload-replay-archive) PAYLOAD_REPLAY_ARCHIVE="$2"; shift 2 ;;
|
||||
--payload-replay-locks) PAYLOAD_REPLAY_LOCKS="$2"; shift 2 ;;
|
||||
--payload-base-archive) PAYLOAD_BASE_ARCHIVE="$2"; shift 2 ;;
|
||||
--payload-base-locks) PAYLOAD_BASE_LOCKS="$2"; shift 2 ;;
|
||||
--payload-delta-archive) PAYLOAD_DELTA_ARCHIVE="$2"; shift 2 ;;
|
||||
--payload-delta-locks) PAYLOAD_DELTA_LOCKS="$2"; shift 2 ;;
|
||||
--base-validation-time) BASE_VALIDATION_TIME="$2"; shift 2 ;;
|
||||
--delta-validation-time) DELTA_VALIDATION_TIME="$2"; shift 2 ;;
|
||||
--max-depth) MAX_DEPTH="$2"; shift 2 ;;
|
||||
--max-instances) MAX_INSTANCES="$2"; shift 2 ;;
|
||||
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -n "$OUT_DIR" && -n "$TAL_PATH" && -n "$TA_PATH" && -n "$CIR_TAL_URI" && -n "$PAYLOAD_REPLAY_ARCHIVE" && -n "$PAYLOAD_REPLAY_LOCKS" && -n "$PAYLOAD_BASE_ARCHIVE" && -n "$PAYLOAD_BASE_LOCKS" && -n "$PAYLOAD_DELTA_ARCHIVE" && -n "$PAYLOAD_DELTA_LOCKS" ]] || {
|
||||
usage >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
if [[ ! -x "$RPKI_BIN" ]]; then
|
||||
(
|
||||
cd "$ROOT_DIR"
|
||||
cargo build --release --bin rpki
|
||||
)
|
||||
fi
|
||||
|
||||
resolve_validation_time() {
|
||||
local path="$1"
|
||||
python3 - <<'PY' "$path"
|
||||
import json, sys
|
||||
print(json.load(open(sys.argv[1], 'r', encoding='utf-8'))['validationTime'])
|
||||
PY
|
||||
}
|
||||
|
||||
if [[ -z "$BASE_VALIDATION_TIME" ]]; then
|
||||
BASE_VALIDATION_TIME="$(resolve_validation_time "$PAYLOAD_REPLAY_LOCKS")"
|
||||
fi
|
||||
|
||||
if [[ -z "$DELTA_VALIDATION_TIME" ]]; then
|
||||
DELTA_VALIDATION_TIME="$(resolve_validation_time "$PAYLOAD_DELTA_LOCKS")"
|
||||
fi
|
||||
|
||||
rm -rf "$OUT_DIR"
|
||||
mkdir -p "$OUT_DIR/full" "$OUT_DIR/delta-001" "$OUT_DIR/static"
|
||||
|
||||
FULL_DB="$OUT_DIR/full/db"
|
||||
DELTA_DB="$OUT_DIR/delta-001/db"
|
||||
|
||||
"$RPKI_BIN" \
|
||||
--db "$FULL_DB" \
|
||||
--tal-path "$TAL_PATH" \
|
||||
--ta-path "$TA_PATH" \
|
||||
--payload-replay-archive "$PAYLOAD_REPLAY_ARCHIVE" \
|
||||
--payload-replay-locks "$PAYLOAD_REPLAY_LOCKS" \
|
||||
--validation-time "$BASE_VALIDATION_TIME" \
|
||||
--max-depth "$MAX_DEPTH" \
|
||||
--max-instances "$MAX_INSTANCES" \
|
||||
--ccr-out "$OUT_DIR/full/result.ccr" \
|
||||
--report-json "$OUT_DIR/full/report.json" \
|
||||
--cir-enable \
|
||||
--cir-out "$OUT_DIR/full/input.cir" \
|
||||
--cir-static-root "$OUT_DIR/static" \
|
||||
--cir-tal-uri "$CIR_TAL_URI" \
|
||||
>"$OUT_DIR/full/run.stdout.log" 2>"$OUT_DIR/full/run.stderr.log"
|
||||
|
||||
"$RPKI_BIN" \
|
||||
--db "$DELTA_DB" \
|
||||
--tal-path "$TAL_PATH" \
|
||||
--ta-path "$TA_PATH" \
|
||||
--payload-base-archive "$PAYLOAD_BASE_ARCHIVE" \
|
||||
--payload-base-locks "$PAYLOAD_BASE_LOCKS" \
|
||||
--payload-delta-archive "$PAYLOAD_DELTA_ARCHIVE" \
|
||||
--payload-delta-locks "$PAYLOAD_DELTA_LOCKS" \
|
||||
--payload-base-validation-time "$BASE_VALIDATION_TIME" \
|
||||
--validation-time "$DELTA_VALIDATION_TIME" \
|
||||
--max-depth "$MAX_DEPTH" \
|
||||
--max-instances "$MAX_INSTANCES" \
|
||||
--ccr-out "$OUT_DIR/delta-001/result.ccr" \
|
||||
--report-json "$OUT_DIR/delta-001/report.json" \
|
||||
--cir-enable \
|
||||
--cir-out "$OUT_DIR/delta-001/input.cir" \
|
||||
--cir-static-root "$OUT_DIR/static" \
|
||||
--cir-tal-uri "$CIR_TAL_URI" \
|
||||
>"$OUT_DIR/delta-001/run.stdout.log" 2>"$OUT_DIR/delta-001/run.stderr.log"
|
||||
|
||||
python3 - <<'PY' "$OUT_DIR" "$BASE_VALIDATION_TIME" "$DELTA_VALIDATION_TIME"
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
out = Path(sys.argv[1])
|
||||
base_validation_time = sys.argv[2]
|
||||
delta_validation_time = sys.argv[3]
|
||||
static_files = sum(1 for _ in (out / "static").rglob("*") if _.is_file())
|
||||
summary = {
|
||||
"version": 1,
|
||||
"kind": "cir_pair",
|
||||
"baseValidationTime": base_validation_time,
|
||||
"deltaValidationTime": delta_validation_time,
|
||||
"staticRoot": "static",
|
||||
"steps": [
|
||||
{
|
||||
"kind": "full",
|
||||
"cirPath": "full/input.cir",
|
||||
"ccrPath": "full/result.ccr",
|
||||
"reportPath": "full/report.json",
|
||||
},
|
||||
{
|
||||
"kind": "delta",
|
||||
"cirPath": "delta-001/input.cir",
|
||||
"ccrPath": "delta-001/result.ccr",
|
||||
"reportPath": "delta-001/report.json",
|
||||
"previous": "full",
|
||||
},
|
||||
],
|
||||
"staticFileCount": static_files,
|
||||
}
|
||||
(out / "summary.json").write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
||||
PY
|
||||
|
||||
echo "done: $OUT_DIR"
|
||||
129
scripts/cir/run_cir_record_sequence_multi_rir_offline.sh
Executable file
129
scripts/cir/run_cir_record_sequence_multi_rir_offline.sh
Executable file
@@ -0,0 +1,129 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/cir/run_cir_record_sequence_multi_rir_offline.sh \
|
||||
[--bundle-root <path>] \
|
||||
[--rir <afrinic,apnic,arin,lacnic,ripe>] \
|
||||
[--delta-count <n>] \
|
||||
[--full-repo] \
|
||||
[--out-root <path>] \
|
||||
[--rpki-bin <path>]
|
||||
EOF
|
||||
}
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
CASE_INFO="$ROOT_DIR/scripts/payload_replay/multi_rir_case_info.py"
|
||||
SINGLE_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_record_sequence_offline.sh"
|
||||
|
||||
BUNDLE_ROOT="/home/yuyr/dev/rust_playground/routinator/bench/multi_rir_demo/runs/20260316-112341-multi-final3"
|
||||
RIRS="afrinic,apnic,arin,lacnic,ripe"
|
||||
DELTA_COUNT=2
|
||||
FULL_REPO=0
|
||||
OUT_ROOT="$ROOT_DIR/target/replay/cir_sequence_multi_rir_offline_$(date -u +%Y%m%dT%H%M%SZ)"
|
||||
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--bundle-root) BUNDLE_ROOT="$2"; shift 2 ;;
|
||||
--rir) RIRS="$2"; shift 2 ;;
|
||||
--delta-count) DELTA_COUNT="$2"; shift 2 ;;
|
||||
--full-repo) FULL_REPO=1; shift 1 ;;
|
||||
--out-root) OUT_ROOT="$2"; shift 2 ;;
|
||||
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
mkdir -p "$OUT_ROOT"
|
||||
SUMMARY_JSON="$OUT_ROOT/summary.json"
|
||||
SUMMARY_MD="$OUT_ROOT/summary.md"
|
||||
|
||||
IFS=',' read -r -a RIR_ITEMS <<< "$RIRS"
|
||||
|
||||
for rir in "${RIR_ITEMS[@]}"; do
|
||||
CASE_JSON="$(python3 "$CASE_INFO" --bundle-root "$BUNDLE_ROOT" --repo-root "$ROOT_DIR" --rir "$rir")"
|
||||
TAL_PATH="$(python3 - <<'PY' "$CASE_JSON"
|
||||
import json,sys
|
||||
print(json.loads(sys.argv[1])['tal_path'])
|
||||
PY
|
||||
)"
|
||||
TA_PATH="$(python3 - <<'PY' "$CASE_JSON"
|
||||
import json,sys
|
||||
print(json.loads(sys.argv[1])['ta_path'])
|
||||
PY
|
||||
)"
|
||||
BASE_ARCHIVE="$(python3 - <<'PY' "$CASE_JSON"
|
||||
import json,sys
|
||||
print(json.loads(sys.argv[1])['base_archive'])
|
||||
PY
|
||||
)"
|
||||
BASE_LOCKS="$(python3 - <<'PY' "$CASE_JSON"
|
||||
import json,sys
|
||||
print(json.loads(sys.argv[1])['base_locks'])
|
||||
PY
|
||||
)"
|
||||
DELTA_ARCHIVE="$(python3 - <<'PY' "$CASE_JSON"
|
||||
import json,sys
|
||||
print(json.loads(sys.argv[1])['delta_archive'])
|
||||
PY
|
||||
)"
|
||||
DELTA_LOCKS="$(python3 - <<'PY' "$CASE_JSON"
|
||||
import json,sys
|
||||
print(json.loads(sys.argv[1])['delta_locks'])
|
||||
PY
|
||||
)"
|
||||
OUT_DIR="$OUT_ROOT/$rir"
|
||||
args=(
|
||||
"$SINGLE_SCRIPT"
|
||||
--out-dir "$OUT_DIR" \
|
||||
--tal-path "$TAL_PATH" \
|
||||
--ta-path "$TA_PATH" \
|
||||
--cir-tal-uri "https://example.test/$rir.tal" \
|
||||
--payload-replay-archive "$BASE_ARCHIVE" \
|
||||
--payload-replay-locks "$BASE_LOCKS" \
|
||||
--payload-base-archive "$BASE_ARCHIVE" \
|
||||
--payload-base-locks "$BASE_LOCKS" \
|
||||
--payload-delta-archive "$DELTA_ARCHIVE" \
|
||||
--payload-delta-locks "$DELTA_LOCKS" \
|
||||
--delta-count "$DELTA_COUNT" \
|
||||
--rpki-bin "$RPKI_BIN"
|
||||
)
|
||||
if [[ "$FULL_REPO" -ne 1 ]]; then
|
||||
args+=(--max-depth 0 --max-instances 1)
|
||||
else
|
||||
args+=(--full-repo)
|
||||
fi
|
||||
"${args[@]}"
|
||||
done
|
||||
|
||||
python3 - <<'PY' "$OUT_ROOT" "$RIRS" "$SUMMARY_JSON" "$SUMMARY_MD"
|
||||
import json, sys
|
||||
from pathlib import Path
|
||||
out_root = Path(sys.argv[1])
|
||||
rirs = [item for item in sys.argv[2].split(',') if item]
|
||||
summary_json = Path(sys.argv[3])
|
||||
summary_md = Path(sys.argv[4])
|
||||
items = []
|
||||
for rir in rirs:
|
||||
root = out_root / rir
|
||||
seq = json.loads((root / "sequence.json").read_text(encoding="utf-8"))
|
||||
summ = json.loads((root / "summary.json").read_text(encoding="utf-8"))
|
||||
items.append({
|
||||
"rir": rir,
|
||||
"root": str(root),
|
||||
"stepCount": len(seq["steps"]),
|
||||
"staticFileCount": summ["staticFileCount"],
|
||||
})
|
||||
summary = {"version": 1, "rirs": items}
|
||||
summary_json.write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
||||
lines = ["# Multi-RIR Offline CIR Sequence Summary", ""]
|
||||
for item in items:
|
||||
lines.append(f"- `{item['rir']}`: `stepCount={item['stepCount']}` `staticFileCount={item['staticFileCount']}` `root={item['root']}`")
|
||||
summary_md.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||
PY
|
||||
|
||||
echo "done: $OUT_ROOT"
|
||||
206
scripts/cir/run_cir_record_sequence_offline.sh
Executable file
206
scripts/cir/run_cir_record_sequence_offline.sh
Executable file
@@ -0,0 +1,206 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/cir/run_cir_record_sequence_offline.sh \
|
||||
--out-dir <path> \
|
||||
--tal-path <path> \
|
||||
--ta-path <path> \
|
||||
--cir-tal-uri <url> \
|
||||
--payload-replay-archive <path> \
|
||||
--payload-replay-locks <path> \
|
||||
--payload-base-archive <path> \
|
||||
--payload-base-locks <path> \
|
||||
--payload-delta-archive <path> \
|
||||
--payload-delta-locks <path> \
|
||||
[--delta-count <n>] \
|
||||
[--base-validation-time <rfc3339>] \
|
||||
[--delta-validation-time <rfc3339>] \
|
||||
[--full-repo] \
|
||||
[--max-depth <n>] \
|
||||
[--max-instances <n>] \
|
||||
[--rpki-bin <path>]
|
||||
EOF
|
||||
}
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
|
||||
OUT_DIR=""
|
||||
TAL_PATH=""
|
||||
TA_PATH=""
|
||||
CIR_TAL_URI=""
|
||||
PAYLOAD_REPLAY_ARCHIVE=""
|
||||
PAYLOAD_REPLAY_LOCKS=""
|
||||
PAYLOAD_BASE_ARCHIVE=""
|
||||
PAYLOAD_BASE_LOCKS=""
|
||||
PAYLOAD_DELTA_ARCHIVE=""
|
||||
PAYLOAD_DELTA_LOCKS=""
|
||||
BASE_VALIDATION_TIME=""
|
||||
DELTA_VALIDATION_TIME=""
|
||||
DELTA_COUNT=2
|
||||
FULL_REPO=0
|
||||
MAX_DEPTH=0
|
||||
MAX_INSTANCES=1
|
||||
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--out-dir) OUT_DIR="$2"; shift 2 ;;
|
||||
--tal-path) TAL_PATH="$2"; shift 2 ;;
|
||||
--ta-path) TA_PATH="$2"; shift 2 ;;
|
||||
--cir-tal-uri) CIR_TAL_URI="$2"; shift 2 ;;
|
||||
--payload-replay-archive) PAYLOAD_REPLAY_ARCHIVE="$2"; shift 2 ;;
|
||||
--payload-replay-locks) PAYLOAD_REPLAY_LOCKS="$2"; shift 2 ;;
|
||||
--payload-base-archive) PAYLOAD_BASE_ARCHIVE="$2"; shift 2 ;;
|
||||
--payload-base-locks) PAYLOAD_BASE_LOCKS="$2"; shift 2 ;;
|
||||
--payload-delta-archive) PAYLOAD_DELTA_ARCHIVE="$2"; shift 2 ;;
|
||||
--payload-delta-locks) PAYLOAD_DELTA_LOCKS="$2"; shift 2 ;;
|
||||
--base-validation-time) BASE_VALIDATION_TIME="$2"; shift 2 ;;
|
||||
--delta-validation-time) DELTA_VALIDATION_TIME="$2"; shift 2 ;;
|
||||
--delta-count) DELTA_COUNT="$2"; shift 2 ;;
|
||||
--full-repo) FULL_REPO=1; shift 1 ;;
|
||||
--max-depth) MAX_DEPTH="$2"; shift 2 ;;
|
||||
--max-instances) MAX_INSTANCES="$2"; shift 2 ;;
|
||||
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -n "$OUT_DIR" && -n "$TAL_PATH" && -n "$TA_PATH" && -n "$CIR_TAL_URI" && -n "$PAYLOAD_REPLAY_ARCHIVE" && -n "$PAYLOAD_REPLAY_LOCKS" && -n "$PAYLOAD_BASE_ARCHIVE" && -n "$PAYLOAD_BASE_LOCKS" && -n "$PAYLOAD_DELTA_ARCHIVE" && -n "$PAYLOAD_DELTA_LOCKS" ]] || {
|
||||
usage >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
if [[ ! -x "$RPKI_BIN" ]]; then
|
||||
(
|
||||
cd "$ROOT_DIR"
|
||||
cargo build --release --bin rpki
|
||||
)
|
||||
fi
|
||||
|
||||
resolve_validation_time() {
|
||||
local path="$1"
|
||||
python3 - <<'PY' "$path"
|
||||
import json, sys
|
||||
print(json.load(open(sys.argv[1], 'r', encoding='utf-8'))['validationTime'])
|
||||
PY
|
||||
}
|
||||
|
||||
if [[ -z "$BASE_VALIDATION_TIME" ]]; then
|
||||
BASE_VALIDATION_TIME="$(resolve_validation_time "$PAYLOAD_REPLAY_LOCKS")"
|
||||
fi
|
||||
if [[ -z "$DELTA_VALIDATION_TIME" ]]; then
|
||||
DELTA_VALIDATION_TIME="$(resolve_validation_time "$PAYLOAD_DELTA_LOCKS")"
|
||||
fi
|
||||
|
||||
rm -rf "$OUT_DIR"
|
||||
mkdir -p "$OUT_DIR/static" "$OUT_DIR/full"
|
||||
|
||||
run_step() {
|
||||
local kind="$1"
|
||||
local step_dir="$2"
|
||||
local db_dir="$3"
|
||||
shift 3
|
||||
mkdir -p "$step_dir"
|
||||
local -a cmd=(
|
||||
"$RPKI_BIN"
|
||||
--db "$db_dir" \
|
||||
--tal-path "$TAL_PATH" \
|
||||
--ta-path "$TA_PATH" \
|
||||
--ccr-out "$step_dir/result.ccr" \
|
||||
--report-json "$step_dir/report.json" \
|
||||
--cir-enable \
|
||||
--cir-out "$step_dir/input.cir" \
|
||||
--cir-static-root "$OUT_DIR/static" \
|
||||
--cir-tal-uri "$CIR_TAL_URI"
|
||||
)
|
||||
if [[ "$FULL_REPO" -ne 1 ]]; then
|
||||
cmd+=(--max-depth "$MAX_DEPTH" --max-instances "$MAX_INSTANCES")
|
||||
fi
|
||||
cmd+=("$@")
|
||||
"${cmd[@]}" >"$step_dir/run.stdout.log" 2>"$step_dir/run.stderr.log"
|
||||
}
|
||||
|
||||
run_step \
|
||||
full \
|
||||
"$OUT_DIR/full" \
|
||||
"$OUT_DIR/full/db" \
|
||||
--payload-replay-archive "$PAYLOAD_REPLAY_ARCHIVE" \
|
||||
--payload-replay-locks "$PAYLOAD_REPLAY_LOCKS" \
|
||||
--validation-time "$BASE_VALIDATION_TIME"
|
||||
|
||||
for idx in $(seq 1 "$DELTA_COUNT"); do
|
||||
step_id="$(printf 'delta-%03d' "$idx")"
|
||||
run_step \
|
||||
delta \
|
||||
"$OUT_DIR/$step_id" \
|
||||
"$OUT_DIR/$step_id/db" \
|
||||
--payload-base-archive "$PAYLOAD_BASE_ARCHIVE" \
|
||||
--payload-base-locks "$PAYLOAD_BASE_LOCKS" \
|
||||
--payload-delta-archive "$PAYLOAD_DELTA_ARCHIVE" \
|
||||
--payload-delta-locks "$PAYLOAD_DELTA_LOCKS" \
|
||||
--payload-base-validation-time "$BASE_VALIDATION_TIME" \
|
||||
--validation-time "$DELTA_VALIDATION_TIME"
|
||||
done
|
||||
|
||||
python3 - <<'PY' "$OUT_DIR" "$BASE_VALIDATION_TIME" "$DELTA_VALIDATION_TIME" "$DELTA_COUNT"
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
out = Path(sys.argv[1])
|
||||
base_validation_time = sys.argv[2]
|
||||
delta_validation_time = sys.argv[3]
|
||||
delta_count = int(sys.argv[4])
|
||||
|
||||
steps = [
|
||||
{
|
||||
"stepId": "full",
|
||||
"kind": "full",
|
||||
"validationTime": base_validation_time,
|
||||
"cirPath": "full/input.cir",
|
||||
"ccrPath": "full/result.ccr",
|
||||
"reportPath": "full/report.json",
|
||||
"previousStepId": None,
|
||||
}
|
||||
]
|
||||
previous = "full"
|
||||
for idx in range(1, delta_count + 1):
|
||||
step_id = f"delta-{idx:03d}"
|
||||
steps.append(
|
||||
{
|
||||
"stepId": step_id,
|
||||
"kind": "delta",
|
||||
"validationTime": delta_validation_time,
|
||||
"cirPath": f"{step_id}/input.cir",
|
||||
"ccrPath": f"{step_id}/result.ccr",
|
||||
"reportPath": f"{step_id}/report.json",
|
||||
"previousStepId": previous,
|
||||
}
|
||||
)
|
||||
previous = step_id
|
||||
|
||||
summary = {
|
||||
"version": 1,
|
||||
"kind": "cir_sequence_offline",
|
||||
"staticRoot": "static",
|
||||
"steps": steps,
|
||||
}
|
||||
(out / "sequence.json").write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
||||
(out / "summary.json").write_text(
|
||||
json.dumps(
|
||||
{
|
||||
"version": 1,
|
||||
"stepCount": len(steps),
|
||||
"staticFileCount": sum(1 for p in (out / "static").rglob("*") if p.is_file()),
|
||||
},
|
||||
indent=2,
|
||||
),
|
||||
encoding="utf-8",
|
||||
)
|
||||
PY
|
||||
|
||||
echo "done: $OUT_DIR"
|
||||
244
scripts/cir/run_cir_record_sequence_remote.sh
Executable file
244
scripts/cir/run_cir_record_sequence_remote.sh
Executable file
@@ -0,0 +1,244 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/cir/run_cir_record_sequence_remote.sh \
|
||||
--rir <name> \
|
||||
--remote-root <path> \
|
||||
[--ssh-target <user@host>] \
|
||||
[--out-subdir <path>] \
|
||||
[--delta-count <n>] \
|
||||
[--sleep-secs <n>] \
|
||||
[--full-repo] \
|
||||
[--max-depth <n>] \
|
||||
[--max-instances <n>]
|
||||
EOF
|
||||
}
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
SSH_TARGET="${SSH_TARGET:-root@47.77.183.68}"
|
||||
RIR=""
|
||||
REMOTE_ROOT=""
|
||||
OUT_SUBDIR=""
|
||||
DELTA_COUNT=2
|
||||
SLEEP_SECS=30
|
||||
FULL_REPO=0
|
||||
MAX_DEPTH=0
|
||||
MAX_INSTANCES=1
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--rir) RIR="$2"; shift 2 ;;
|
||||
--remote-root) REMOTE_ROOT="$2"; shift 2 ;;
|
||||
--ssh-target) SSH_TARGET="$2"; shift 2 ;;
|
||||
--out-subdir) OUT_SUBDIR="$2"; shift 2 ;;
|
||||
--delta-count) DELTA_COUNT="$2"; shift 2 ;;
|
||||
--sleep-secs) SLEEP_SECS="$2"; shift 2 ;;
|
||||
--full-repo) FULL_REPO=1; shift 1 ;;
|
||||
--max-depth) MAX_DEPTH="$2"; shift 2 ;;
|
||||
--max-instances) MAX_INSTANCES="$2"; shift 2 ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -n "$RIR" && -n "$REMOTE_ROOT" ]] || { usage >&2; exit 2; }
|
||||
|
||||
case "$RIR" in
|
||||
afrinic) TAL_REL="tests/fixtures/tal/afrinic.tal"; TA_REL="tests/fixtures/ta/afrinic-ta.cer" ;;
|
||||
apnic) TAL_REL="tests/fixtures/tal/apnic-rfc7730-https.tal"; TA_REL="tests/fixtures/ta/apnic-ta.cer" ;;
|
||||
arin) TAL_REL="tests/fixtures/tal/arin.tal"; TA_REL="tests/fixtures/ta/arin-ta.cer" ;;
|
||||
lacnic) TAL_REL="tests/fixtures/tal/lacnic.tal"; TA_REL="tests/fixtures/ta/lacnic-ta.cer" ;;
|
||||
ripe) TAL_REL="tests/fixtures/tal/ripe-ncc.tal"; TA_REL="tests/fixtures/ta/ripe-ncc-ta.cer" ;;
|
||||
*) echo "unsupported rir: $RIR" >&2; exit 2 ;;
|
||||
esac
|
||||
|
||||
rsync -a --delete \
|
||||
--exclude target \
|
||||
--exclude .git \
|
||||
"$ROOT_DIR/" "$SSH_TARGET:$REMOTE_ROOT/"
|
||||
ssh "$SSH_TARGET" "mkdir -p '$REMOTE_ROOT/target/release'"
|
||||
rsync -a "$ROOT_DIR/target/release/rpki" "$SSH_TARGET:$REMOTE_ROOT/target/release/"
|
||||
|
||||
ssh "$SSH_TARGET" \
|
||||
RIR="$RIR" \
|
||||
REMOTE_ROOT="$REMOTE_ROOT" \
|
||||
OUT_SUBDIR="$OUT_SUBDIR" \
|
||||
DELTA_COUNT="$DELTA_COUNT" \
|
||||
SLEEP_SECS="$SLEEP_SECS" \
|
||||
FULL_REPO="$FULL_REPO" \
|
||||
MAX_DEPTH="$MAX_DEPTH" \
|
||||
MAX_INSTANCES="$MAX_INSTANCES" \
|
||||
TAL_REL="$TAL_REL" \
|
||||
TA_REL="$TA_REL" \
|
||||
'bash -s' <<'EOS'
|
||||
set -euo pipefail
|
||||
|
||||
cd "$REMOTE_ROOT"
|
||||
|
||||
if [[ -n "${OUT_SUBDIR}" ]]; then
|
||||
OUT="${OUT_SUBDIR}"
|
||||
else
|
||||
OUT="target/replay/cir_sequence_remote_${RIR}_$(date -u +%Y%m%dT%H%M%SZ)"
|
||||
fi
|
||||
|
||||
mkdir -p "$OUT"
|
||||
DB="$OUT/work-db"
|
||||
RAW_STORE_DB="$OUT/raw-store.db"
|
||||
ROWS="$OUT/.sequence_rows.tsv"
|
||||
: > "$ROWS"
|
||||
|
||||
write_step_timing() {
|
||||
local path="$1"
|
||||
local start_ms="$2"
|
||||
local end_ms="$3"
|
||||
local started_at="$4"
|
||||
local finished_at="$5"
|
||||
python3 - <<'PY' "$path" "$start_ms" "$end_ms" "$started_at" "$finished_at"
|
||||
import json, sys
|
||||
path, start_ms, end_ms, started_at, finished_at = sys.argv[1:]
|
||||
start_ms = int(start_ms)
|
||||
end_ms = int(end_ms)
|
||||
with open(path, "w", encoding="utf-8") as fh:
|
||||
json.dump(
|
||||
{
|
||||
"durationMs": end_ms - start_ms,
|
||||
"startedAt": started_at,
|
||||
"finishedAt": finished_at,
|
||||
},
|
||||
fh,
|
||||
indent=2,
|
||||
)
|
||||
PY
|
||||
}
|
||||
|
||||
run_step() {
|
||||
local step_id="$1"
|
||||
local kind="$2"
|
||||
local previous_step_id="$3"
|
||||
shift 3
|
||||
|
||||
local started_at_iso started_at_ms finished_at_iso finished_at_ms prefix
|
||||
started_at_iso="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||
started_at_ms="$(python3 - <<'PY'
|
||||
import time
|
||||
print(int(time.time() * 1000))
|
||||
PY
|
||||
)"
|
||||
prefix="${started_at_iso}-test"
|
||||
|
||||
local cir_out="$OUT/${prefix}.cir"
|
||||
local ccr_out="$OUT/${prefix}.ccr"
|
||||
local report_out="$OUT/${prefix}.report.json"
|
||||
local timing_out="$OUT/${prefix}.timing.json"
|
||||
local stdout_out="$OUT/${prefix}.stdout.log"
|
||||
local stderr_out="$OUT/${prefix}.stderr.log"
|
||||
|
||||
local -a cmd=(
|
||||
target/release/rpki
|
||||
--db "$DB"
|
||||
--raw-store-db "$RAW_STORE_DB"
|
||||
--tal-path "$TAL_REL"
|
||||
--ta-path "$TA_REL"
|
||||
--ccr-out "$ccr_out"
|
||||
--report-json "$report_out"
|
||||
--cir-enable
|
||||
--cir-out "$cir_out"
|
||||
--cir-tal-uri "https://example.test/${RIR}.tal"
|
||||
)
|
||||
if [[ "$FULL_REPO" -ne 1 ]]; then
|
||||
cmd+=(--max-depth "$MAX_DEPTH" --max-instances "$MAX_INSTANCES")
|
||||
fi
|
||||
cmd+=("$@")
|
||||
env RPKI_PROGRESS_LOG=1 "${cmd[@]}" >"$stdout_out" 2>"$stderr_out"
|
||||
|
||||
finished_at_ms="$(python3 - <<'PY'
|
||||
import time
|
||||
print(int(time.time() * 1000))
|
||||
PY
|
||||
)"
|
||||
finished_at_iso="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||
write_step_timing "$timing_out" "$started_at_ms" "$finished_at_ms" "$started_at_iso" "$finished_at_iso"
|
||||
|
||||
local validation_time
|
||||
validation_time="$(python3 - <<'PY' "$report_out"
|
||||
import json, sys
|
||||
print(json.load(open(sys.argv[1], 'r', encoding='utf-8'))['meta']['validation_time_rfc3339_utc'])
|
||||
PY
|
||||
)"
|
||||
|
||||
printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' \
|
||||
"$step_id" \
|
||||
"$kind" \
|
||||
"$validation_time" \
|
||||
"$(basename "$cir_out")" \
|
||||
"$(basename "$ccr_out")" \
|
||||
"$(basename "$report_out")" \
|
||||
"$(basename "$timing_out")" \
|
||||
"$(basename "$stdout_out")" \
|
||||
"$(basename "$stderr_out")" >> "$ROWS"
|
||||
}
|
||||
|
||||
run_step "full" "full" ""
|
||||
|
||||
prev="full"
|
||||
for idx in $(seq 1 "$DELTA_COUNT"); do
|
||||
sleep "$SLEEP_SECS"
|
||||
step="$(printf 'delta-%03d' "$idx")"
|
||||
run_step "$step" "delta" "$prev"
|
||||
prev="$step"
|
||||
done
|
||||
|
||||
python3 - <<'PY' "$OUT" "$ROWS" "$RIR"
|
||||
import json, sys
|
||||
from pathlib import Path
|
||||
|
||||
out = Path(sys.argv[1])
|
||||
rows = Path(sys.argv[2]).read_text(encoding='utf-8').splitlines()
|
||||
rir = sys.argv[3]
|
||||
steps = []
|
||||
for idx, row in enumerate(rows):
|
||||
step_id, kind, validation_time, cir_name, ccr_name, report_name, timing_name, stdout_name, stderr_name = row.split('\t')
|
||||
steps.append({
|
||||
"stepId": step_id,
|
||||
"kind": kind,
|
||||
"validationTime": validation_time,
|
||||
"cirPath": cir_name,
|
||||
"ccrPath": ccr_name,
|
||||
"reportPath": report_name,
|
||||
"timingPath": timing_name,
|
||||
"stdoutLogPath": stdout_name,
|
||||
"stderrLogPath": stderr_name,
|
||||
"artifactPrefix": cir_name[:-4], # strip .cir
|
||||
"previousStepId": None if idx == 0 else steps[idx - 1]["stepId"],
|
||||
})
|
||||
|
||||
(out / "sequence.json").write_text(
|
||||
json.dumps({"version": 1, "rawStoreDbPath": "raw-store.db", "steps": steps}, indent=2),
|
||||
encoding="utf-8",
|
||||
)
|
||||
|
||||
summary = {
|
||||
"version": 1,
|
||||
"rir": rir,
|
||||
"stepCount": len(steps),
|
||||
"steps": [],
|
||||
}
|
||||
for step in steps:
|
||||
timing = json.loads((out / step["timingPath"]).read_text(encoding="utf-8"))
|
||||
summary["steps"].append({
|
||||
"stepId": step["stepId"],
|
||||
"kind": step["kind"],
|
||||
"validationTime": step["validationTime"],
|
||||
"artifactPrefix": step["artifactPrefix"],
|
||||
**timing,
|
||||
})
|
||||
|
||||
(out / "summary.json").write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
||||
PY
|
||||
|
||||
rm -f "$ROWS"
|
||||
echo "$OUT"
|
||||
EOS
|
||||
72
scripts/cir/run_cir_record_sequence_remote_multi_rir.sh
Executable file
72
scripts/cir/run_cir_record_sequence_remote_multi_rir.sh
Executable file
@@ -0,0 +1,72 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/cir/run_cir_record_sequence_remote_multi_rir.sh \
|
||||
--remote-root <path> \
|
||||
[--rir <afrinic,apnic,arin,lacnic,ripe>] \
|
||||
[--ssh-target <user@host>] \
|
||||
[--out-subdir-root <path>] \
|
||||
[--delta-count <n>] \
|
||||
[--sleep-secs <n>] \
|
||||
[--full-repo] \
|
||||
[--max-depth <n>] \
|
||||
[--max-instances <n>]
|
||||
EOF
|
||||
}
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
SSH_TARGET="${SSH_TARGET:-root@47.77.183.68}"
|
||||
REMOTE_ROOT=""
|
||||
RIRS="afrinic,apnic,arin,lacnic,ripe"
|
||||
OUT_SUBDIR_ROOT=""
|
||||
DELTA_COUNT=2
|
||||
SLEEP_SECS=30
|
||||
FULL_REPO=0
|
||||
MAX_DEPTH=0
|
||||
MAX_INSTANCES=1
|
||||
SINGLE="$ROOT_DIR/scripts/cir/run_cir_record_sequence_remote.sh"
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--remote-root) REMOTE_ROOT="$2"; shift 2 ;;
|
||||
--rir) RIRS="$2"; shift 2 ;;
|
||||
--ssh-target) SSH_TARGET="$2"; shift 2 ;;
|
||||
--out-subdir-root) OUT_SUBDIR_ROOT="$2"; shift 2 ;;
|
||||
--delta-count) DELTA_COUNT="$2"; shift 2 ;;
|
||||
--sleep-secs) SLEEP_SECS="$2"; shift 2 ;;
|
||||
--full-repo) FULL_REPO=1; shift 1 ;;
|
||||
--max-depth) MAX_DEPTH="$2"; shift 2 ;;
|
||||
--max-instances) MAX_INSTANCES="$2"; shift 2 ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -n "$REMOTE_ROOT" ]] || { usage >&2; exit 2; }
|
||||
if [[ -z "$OUT_SUBDIR_ROOT" ]]; then
|
||||
OUT_SUBDIR_ROOT="target/replay/cir_sequence_remote_multi_rir_$(date -u +%Y%m%dT%H%M%SZ)"
|
||||
fi
|
||||
|
||||
IFS=',' read -r -a ITEMS <<< "$RIRS"
|
||||
for rir in "${ITEMS[@]}"; do
|
||||
args=(
|
||||
"$SINGLE"
|
||||
--rir "$rir" \
|
||||
--remote-root "$REMOTE_ROOT" \
|
||||
--ssh-target "$SSH_TARGET" \
|
||||
--out-subdir "$OUT_SUBDIR_ROOT/$rir" \
|
||||
--delta-count "$DELTA_COUNT" \
|
||||
--sleep-secs "$SLEEP_SECS" \
|
||||
)
|
||||
if [[ "$FULL_REPO" -eq 1 ]]; then
|
||||
args+=(--full-repo)
|
||||
else
|
||||
args+=(--max-depth "$MAX_DEPTH" --max-instances "$MAX_INSTANCES")
|
||||
fi
|
||||
"${args[@]}"
|
||||
done
|
||||
|
||||
echo "$OUT_SUBDIR_ROOT"
|
||||
118
scripts/cir/run_cir_record_sequence_ta_only_multi_rir.sh
Executable file
118
scripts/cir/run_cir_record_sequence_ta_only_multi_rir.sh
Executable file
@@ -0,0 +1,118 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/cir/run_cir_record_sequence_ta_only_multi_rir.sh \
|
||||
[--rir <afrinic,apnic,arin,lacnic,ripe>] \
|
||||
[--delta-count <n>] \
|
||||
[--out-root <path>] \
|
||||
[--rpki-bin <path>]
|
||||
EOF
|
||||
}
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
HELPER_BIN="${HELPER_BIN:-$ROOT_DIR/target/release/cir_ta_only_fixture}"
|
||||
MATERIALIZE_BIN="${MATERIALIZE_BIN:-$ROOT_DIR/target/release/cir_materialize}"
|
||||
EXTRACT_BIN="${EXTRACT_BIN:-$ROOT_DIR/target/release/cir_extract_inputs}"
|
||||
WRAPPER="$ROOT_DIR/scripts/cir/cir-rsync-wrapper"
|
||||
RIRS="afrinic,apnic,arin,lacnic,ripe"
|
||||
DELTA_COUNT=2
|
||||
OUT_ROOT="$ROOT_DIR/target/replay/cir_sequence_multi_rir_ta_only_$(date -u +%Y%m%dT%H%M%SZ)"
|
||||
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--rir) RIRS="$2"; shift 2 ;;
|
||||
--delta-count) DELTA_COUNT="$2"; shift 2 ;;
|
||||
--out-root) OUT_ROOT="$2"; shift 2 ;;
|
||||
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
if [[ ! -x "$HELPER_BIN" ]]; then
|
||||
(
|
||||
cd "$ROOT_DIR"
|
||||
cargo build --release --bin cir_ta_only_fixture --bin rpki --bin cir_materialize --bin cir_extract_inputs
|
||||
)
|
||||
fi
|
||||
|
||||
case_paths() {
|
||||
case "$1" in
|
||||
afrinic) echo "tests/fixtures/tal/afrinic.tal tests/fixtures/ta/afrinic-ta.cer" ;;
|
||||
apnic) echo "tests/fixtures/tal/apnic-rfc7730-https.tal tests/fixtures/ta/apnic-ta.cer" ;;
|
||||
arin) echo "tests/fixtures/tal/arin.tal tests/fixtures/ta/arin-ta.cer" ;;
|
||||
lacnic) echo "tests/fixtures/tal/lacnic.tal tests/fixtures/ta/lacnic-ta.cer" ;;
|
||||
ripe) echo "tests/fixtures/tal/ripe-ncc.tal tests/fixtures/ta/ripe-ncc-ta.cer" ;;
|
||||
*) return 1 ;;
|
||||
esac
|
||||
}
|
||||
|
||||
mkdir -p "$OUT_ROOT"
|
||||
IFS=',' read -r -a ITEMS <<< "$RIRS"
|
||||
for rir in "${ITEMS[@]}"; do
|
||||
read -r tal_rel ta_rel < <(case_paths "$rir")
|
||||
rir_root="$OUT_ROOT/$rir"
|
||||
mkdir -p "$rir_root/full" "$rir_root/static"
|
||||
"$HELPER_BIN" \
|
||||
--tal-path "$ROOT_DIR/$tal_rel" \
|
||||
--ta-path "$ROOT_DIR/$ta_rel" \
|
||||
--tal-uri "https://example.test/$rir.tal" \
|
||||
--validation-time "2026-04-09T00:00:00Z" \
|
||||
--cir-out "$rir_root/full/input.cir" \
|
||||
--static-root "$rir_root/static"
|
||||
"$EXTRACT_BIN" --cir "$rir_root/full/input.cir" --tals-dir "$rir_root/.tmp/tals" --meta-json "$rir_root/.tmp/meta.json"
|
||||
"$MATERIALIZE_BIN" --cir "$rir_root/full/input.cir" --static-root "$rir_root/static" --mirror-root "$rir_root/.tmp/mirror"
|
||||
FIRST_TAL="$(python3 - <<'PY' "$rir_root/.tmp/meta.json"
|
||||
import json,sys
|
||||
print(json.load(open(sys.argv[1]))["talFiles"][0]["path"])
|
||||
PY
|
||||
)"
|
||||
export CIR_MIRROR_ROOT="$rir_root/.tmp/mirror"
|
||||
export REAL_RSYNC_BIN=/usr/bin/rsync
|
||||
export CIR_LOCAL_LINK_MODE=1
|
||||
"$RPKI_BIN" \
|
||||
--db "$rir_root/full/db" \
|
||||
--tal-path "$FIRST_TAL" \
|
||||
--disable-rrdp \
|
||||
--rsync-command "$WRAPPER" \
|
||||
--validation-time "2026-04-09T00:00:00Z" \
|
||||
--ccr-out "$rir_root/full/result.ccr" \
|
||||
--report-json "$rir_root/full/report.json" >/dev/null 2>&1
|
||||
for idx in $(seq 1 "$DELTA_COUNT"); do
|
||||
step="$(printf 'delta-%03d' "$idx")"
|
||||
mkdir -p "$rir_root/$step"
|
||||
cp "$rir_root/full/input.cir" "$rir_root/$step/input.cir"
|
||||
cp "$rir_root/full/result.ccr" "$rir_root/$step/result.ccr"
|
||||
cp "$rir_root/full/report.json" "$rir_root/$step/report.json"
|
||||
done
|
||||
python3 - <<'PY' "$rir_root" "$DELTA_COUNT"
|
||||
import json, sys
|
||||
from pathlib import Path
|
||||
root = Path(sys.argv[1]); delta_count = int(sys.argv[2])
|
||||
steps = [{"stepId":"full","kind":"full","validationTime":"2026-04-09T00:00:00Z","cirPath":"full/input.cir","ccrPath":"full/result.ccr","reportPath":"full/report.json","previousStepId":None}]
|
||||
prev = "full"
|
||||
for i in range(1, delta_count + 1):
|
||||
step = f"delta-{i:03d}"
|
||||
steps.append({"stepId":step,"kind":"delta","validationTime":"2026-04-09T00:00:00Z","cirPath":f"{step}/input.cir","ccrPath":f"{step}/result.ccr","reportPath":f"{step}/report.json","previousStepId":prev})
|
||||
prev = step
|
||||
(root/"sequence.json").write_text(json.dumps({"version":1,"staticRoot":"static","steps":steps}, indent=2), encoding="utf-8")
|
||||
(root/"summary.json").write_text(json.dumps({"version":1,"stepCount":len(steps)}, indent=2), encoding="utf-8")
|
||||
PY
|
||||
done
|
||||
|
||||
python3 - <<'PY' "$OUT_ROOT" "$RIRS"
|
||||
import json, sys
|
||||
from pathlib import Path
|
||||
root = Path(sys.argv[1]); rirs = [x for x in sys.argv[2].split(',') if x]
|
||||
items=[]
|
||||
for rir in rirs:
|
||||
seq=json.loads((root/rir/'sequence.json').read_text())
|
||||
items.append({"rir":rir,"stepCount":len(seq['steps'])})
|
||||
(root/'summary.json').write_text(json.dumps({"version":1,"rirs":items}, indent=2), encoding='utf-8')
|
||||
PY
|
||||
|
||||
echo "done: $OUT_ROOT"
|
||||
49
scripts/cir/run_cir_record_sequence_ta_only_remote_multi_rir.sh
Executable file
49
scripts/cir/run_cir_record_sequence_ta_only_remote_multi_rir.sh
Executable file
@@ -0,0 +1,49 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/cir/run_cir_record_sequence_ta_only_remote_multi_rir.sh \
|
||||
--remote-root <path> \
|
||||
[--ssh-target <user@host>] \
|
||||
[--rir <afrinic,apnic,arin,lacnic,ripe>] \
|
||||
[--delta-count <n>]
|
||||
EOF
|
||||
}
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
SSH_TARGET="${SSH_TARGET:-root@47.77.183.68}"
|
||||
REMOTE_ROOT=""
|
||||
RIRS="afrinic,apnic,arin,lacnic,ripe"
|
||||
DELTA_COUNT=2
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--remote-root) REMOTE_ROOT="$2"; shift 2 ;;
|
||||
--ssh-target) SSH_TARGET="$2"; shift 2 ;;
|
||||
--rir) RIRS="$2"; shift 2 ;;
|
||||
--delta-count) DELTA_COUNT="$2"; shift 2 ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -n "$REMOTE_ROOT" ]] || { usage >&2; exit 2; }
|
||||
|
||||
rsync -a --delete \
|
||||
--exclude target \
|
||||
--exclude .git \
|
||||
"$ROOT_DIR/" "$SSH_TARGET:$REMOTE_ROOT/"
|
||||
ssh "$SSH_TARGET" "mkdir -p '$REMOTE_ROOT/target/release'"
|
||||
for bin in rpki cir_ta_only_fixture cir_materialize cir_extract_inputs; do
|
||||
rsync -a "$ROOT_DIR/target/release/$bin" "$SSH_TARGET:$REMOTE_ROOT/target/release/"
|
||||
done
|
||||
|
||||
ssh "$SSH_TARGET" "bash -lc '
|
||||
set -euo pipefail
|
||||
cd $REMOTE_ROOT
|
||||
OUT=target/replay/cir_sequence_remote_ta_only_\$(date -u +%Y%m%dT%H%M%SZ)
|
||||
./scripts/cir/run_cir_record_sequence_ta_only_multi_rir.sh --rir $RIRS --delta-count $DELTA_COUNT --out-root \"\$OUT\"
|
||||
echo \"\$OUT\"
|
||||
'"
|
||||
286
scripts/cir/run_cir_replay_matrix.sh
Executable file
286
scripts/cir/run_cir_replay_matrix.sh
Executable file
@@ -0,0 +1,286 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/cir/run_cir_replay_matrix.sh \
|
||||
--cir <path> \
|
||||
--static-root <path> \
|
||||
--out-dir <path> \
|
||||
--reference-ccr <path> \
|
||||
--rpki-client-build-dir <path> \
|
||||
[--keep-db] \
|
||||
[--rpki-bin <path>] \
|
||||
[--routinator-root <path>] \
|
||||
[--routinator-bin <path>] \
|
||||
[--real-rsync-bin <path>]
|
||||
EOF
|
||||
}
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
|
||||
CIR=""
|
||||
STATIC_ROOT=""
|
||||
OUT_DIR=""
|
||||
REFERENCE_CCR=""
|
||||
RPKI_CLIENT_BUILD_DIR=""
|
||||
KEEP_DB=0
|
||||
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
||||
ROUTINATOR_ROOT="${ROUTINATOR_ROOT:-/home/yuyr/dev/rust_playground/routinator}"
|
||||
ROUTINATOR_BIN="${ROUTINATOR_BIN:-$ROUTINATOR_ROOT/target/debug/routinator}"
|
||||
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
||||
OURS_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_ours.sh"
|
||||
ROUTINATOR_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_routinator.sh"
|
||||
RPKI_CLIENT_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_rpki_client.sh"
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--cir) CIR="$2"; shift 2 ;;
|
||||
--static-root) STATIC_ROOT="$2"; shift 2 ;;
|
||||
--out-dir) OUT_DIR="$2"; shift 2 ;;
|
||||
--reference-ccr) REFERENCE_CCR="$2"; shift 2 ;;
|
||||
--rpki-client-build-dir) RPKI_CLIENT_BUILD_DIR="$2"; shift 2 ;;
|
||||
--keep-db) KEEP_DB=1; shift ;;
|
||||
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
||||
--routinator-root) ROUTINATOR_ROOT="$2"; shift 2 ;;
|
||||
--routinator-bin) ROUTINATOR_BIN="$2"; shift 2 ;;
|
||||
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -n "$CIR" && -n "$STATIC_ROOT" && -n "$OUT_DIR" && -n "$REFERENCE_CCR" && -n "$RPKI_CLIENT_BUILD_DIR" ]] || {
|
||||
usage >&2
|
||||
exit 2
|
||||
}
|
||||
|
||||
mkdir -p "$OUT_DIR"
|
||||
|
||||
run_with_timing() {
|
||||
local summary_path="$1"
|
||||
local timing_path="$2"
|
||||
shift 2
|
||||
local start end status
|
||||
start="$(python3 - <<'PY'
|
||||
import time
|
||||
print(time.perf_counter_ns())
|
||||
PY
|
||||
)"
|
||||
if "$@"; then
|
||||
status=0
|
||||
else
|
||||
status=$?
|
||||
fi
|
||||
end="$(python3 - <<'PY'
|
||||
import time
|
||||
print(time.perf_counter_ns())
|
||||
PY
|
||||
)"
|
||||
python3 - <<'PY' "$summary_path" "$timing_path" "$status" "$start" "$end"
|
||||
import json, sys
|
||||
summary_path, timing_path, status, start, end = sys.argv[1:]
|
||||
duration_ms = max(0, (int(end) - int(start)) // 1_000_000)
|
||||
data = {"exitCode": int(status), "durationMs": duration_ms}
|
||||
try:
|
||||
with open(summary_path, "r", encoding="utf-8") as f:
|
||||
data["compare"] = json.load(f)
|
||||
except FileNotFoundError:
|
||||
data["compare"] = None
|
||||
with open(timing_path, "w", encoding="utf-8") as f:
|
||||
json.dump(data, f, indent=2)
|
||||
PY
|
||||
return "$status"
|
||||
}
|
||||
|
||||
OURS_OUT="$OUT_DIR/ours"
|
||||
ROUTINATOR_OUT="$OUT_DIR/routinator"
|
||||
RPKI_CLIENT_OUT="$OUT_DIR/rpki-client"
|
||||
mkdir -p "$OURS_OUT" "$ROUTINATOR_OUT" "$RPKI_CLIENT_OUT"
|
||||
|
||||
ours_cmd=(
|
||||
"$OURS_SCRIPT"
|
||||
--cir "$CIR"
|
||||
--static-root "$STATIC_ROOT"
|
||||
--out-dir "$OURS_OUT"
|
||||
--reference-ccr "$REFERENCE_CCR"
|
||||
--rpki-bin "$RPKI_BIN"
|
||||
--real-rsync-bin "$REAL_RSYNC_BIN"
|
||||
)
|
||||
routinator_cmd=(
|
||||
"$ROUTINATOR_SCRIPT"
|
||||
--cir "$CIR"
|
||||
--static-root "$STATIC_ROOT"
|
||||
--out-dir "$ROUTINATOR_OUT"
|
||||
--reference-ccr "$REFERENCE_CCR"
|
||||
--routinator-root "$ROUTINATOR_ROOT"
|
||||
--routinator-bin "$ROUTINATOR_BIN"
|
||||
--real-rsync-bin "$REAL_RSYNC_BIN"
|
||||
)
|
||||
rpki_client_cmd=(
|
||||
"$RPKI_CLIENT_SCRIPT"
|
||||
--cir "$CIR"
|
||||
--static-root "$STATIC_ROOT"
|
||||
--out-dir "$RPKI_CLIENT_OUT"
|
||||
--reference-ccr "$REFERENCE_CCR"
|
||||
--build-dir "$RPKI_CLIENT_BUILD_DIR"
|
||||
--real-rsync-bin "$REAL_RSYNC_BIN"
|
||||
)
|
||||
|
||||
if [[ "$KEEP_DB" -eq 1 ]]; then
|
||||
ours_cmd+=(--keep-db)
|
||||
routinator_cmd+=(--keep-db)
|
||||
rpki_client_cmd+=(--keep-db)
|
||||
fi
|
||||
|
||||
ours_status=0
|
||||
routinator_status=0
|
||||
rpki_client_status=0
|
||||
if run_with_timing "$OURS_OUT/compare-summary.json" "$OURS_OUT/timing.json" "${ours_cmd[@]}"; then
|
||||
:
|
||||
else
|
||||
ours_status=$?
|
||||
fi
|
||||
if run_with_timing "$ROUTINATOR_OUT/compare-summary.json" "$ROUTINATOR_OUT/timing.json" "${routinator_cmd[@]}"; then
|
||||
:
|
||||
else
|
||||
routinator_status=$?
|
||||
fi
|
||||
if run_with_timing "$RPKI_CLIENT_OUT/compare-summary.json" "$RPKI_CLIENT_OUT/timing.json" "${rpki_client_cmd[@]}"; then
|
||||
:
|
||||
else
|
||||
rpki_client_status=$?
|
||||
fi
|
||||
|
||||
SUMMARY_JSON="$OUT_DIR/summary.json"
|
||||
SUMMARY_MD="$OUT_DIR/summary.md"
|
||||
DETAIL_MD="$OUT_DIR/detail.md"
|
||||
|
||||
python3 - <<'PY' \
|
||||
"$CIR" \
|
||||
"$STATIC_ROOT" \
|
||||
"$REFERENCE_CCR" \
|
||||
"$OURS_OUT" \
|
||||
"$ROUTINATOR_OUT" \
|
||||
"$RPKI_CLIENT_OUT" \
|
||||
"$SUMMARY_JSON" \
|
||||
"$SUMMARY_MD" \
|
||||
"$DETAIL_MD"
|
||||
import json
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
cir_path, static_root, reference_ccr, ours_out, routinator_out, rpki_client_out, summary_json, summary_md, detail_md = sys.argv[1:]
|
||||
|
||||
participants = []
|
||||
all_match = True
|
||||
for name, out_dir in [
|
||||
("ours", ours_out),
|
||||
("routinator", routinator_out),
|
||||
("rpki-client", rpki_client_out),
|
||||
]:
|
||||
out = Path(out_dir)
|
||||
timing = json.loads((out / "timing.json").read_text(encoding="utf-8"))
|
||||
compare = timing.get("compare") or {}
|
||||
vrps = compare.get("vrps") or {}
|
||||
vaps = compare.get("vaps") or {}
|
||||
participant = {
|
||||
"name": name,
|
||||
"outDir": str(out),
|
||||
"tmpRoot": str(out / ".tmp"),
|
||||
"mirrorPath": str(out / ".tmp" / "mirror"),
|
||||
"timingPath": str(out / "timing.json"),
|
||||
"summaryPath": str(out / "compare-summary.json"),
|
||||
"exitCode": timing["exitCode"],
|
||||
"durationMs": timing["durationMs"],
|
||||
"vrps": vrps,
|
||||
"vaps": vaps,
|
||||
"match": bool(vrps.get("match")) and bool(vaps.get("match")) and timing["exitCode"] == 0,
|
||||
"logPaths": [str(path) for path in sorted(out.glob("*.log"))],
|
||||
}
|
||||
participants.append(participant)
|
||||
all_match = all_match and participant["match"]
|
||||
|
||||
summary = {
|
||||
"cirPath": cir_path,
|
||||
"staticRoot": static_root,
|
||||
"referenceCcr": reference_ccr,
|
||||
"participants": participants,
|
||||
"allMatch": all_match,
|
||||
}
|
||||
Path(summary_json).write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
||||
|
||||
lines = [
|
||||
"# CIR Replay Matrix Summary",
|
||||
"",
|
||||
f"- `cir`: `{cir_path}`",
|
||||
f"- `static_root`: `{static_root}`",
|
||||
f"- `reference_ccr`: `{reference_ccr}`",
|
||||
f"- `all_match`: `{all_match}`",
|
||||
"",
|
||||
"| Participant | Exit | Duration (ms) | VRP actual/ref | VRP match | VAP actual/ref | VAP match | Log |",
|
||||
"| --- | ---: | ---: | --- | --- | --- | --- | --- |",
|
||||
]
|
||||
for participant in participants:
|
||||
vrps = participant["vrps"] or {}
|
||||
vaps = participant["vaps"] or {}
|
||||
log_path = participant["logPaths"][0] if participant["logPaths"] else ""
|
||||
lines.append(
|
||||
"| {name} | {exit_code} | {duration_ms} | {vrp_actual}/{vrp_ref} | {vrp_match} | {vap_actual}/{vap_ref} | {vap_match} | `{log_path}` |".format(
|
||||
name=participant["name"],
|
||||
exit_code=participant["exitCode"],
|
||||
duration_ms=participant["durationMs"],
|
||||
vrp_actual=vrps.get("actual", "-"),
|
||||
vrp_ref=vrps.get("reference", "-"),
|
||||
vrp_match=vrps.get("match", False),
|
||||
vap_actual=vaps.get("actual", "-"),
|
||||
vap_ref=vaps.get("reference", "-"),
|
||||
vap_match=vaps.get("match", False),
|
||||
log_path=log_path,
|
||||
)
|
||||
)
|
||||
Path(summary_md).write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||
|
||||
detail_lines = [
|
||||
"# CIR Replay Matrix Detail",
|
||||
"",
|
||||
]
|
||||
for participant in participants:
|
||||
vrps = participant["vrps"] or {}
|
||||
vaps = participant["vaps"] or {}
|
||||
detail_lines.extend([
|
||||
f"## {participant['name']}",
|
||||
f"- `exit_code`: `{participant['exitCode']}`",
|
||||
f"- `duration_ms`: `{participant['durationMs']}`",
|
||||
f"- `out_dir`: `{participant['outDir']}`",
|
||||
f"- `tmp_root`: `{participant['tmpRoot']}`",
|
||||
f"- `mirror_path`: `{participant['mirrorPath']}`",
|
||||
f"- `summary_path`: `{participant['summaryPath']}`",
|
||||
f"- `timing_path`: `{participant['timingPath']}`",
|
||||
f"- `log_paths`: `{', '.join(participant['logPaths'])}`",
|
||||
f"- `vrps`: `actual={vrps.get('actual', '-')}` `reference={vrps.get('reference', '-')}` `match={vrps.get('match', False)}`",
|
||||
f"- `vaps`: `actual={vaps.get('actual', '-')}` `reference={vaps.get('reference', '-')}` `match={vaps.get('match', False)}`",
|
||||
f"- `vrps.only_in_actual`: `{vrps.get('only_in_actual', [])}`",
|
||||
f"- `vrps.only_in_reference`: `{vrps.get('only_in_reference', [])}`",
|
||||
f"- `vaps.only_in_actual`: `{vaps.get('only_in_actual', [])}`",
|
||||
f"- `vaps.only_in_reference`: `{vaps.get('only_in_reference', [])}`",
|
||||
"",
|
||||
])
|
||||
Path(detail_md).write_text("\n".join(detail_lines), encoding="utf-8")
|
||||
PY
|
||||
|
||||
if [[ "$ours_status" -ne 0 || "$routinator_status" -ne 0 || "$rpki_client_status" -ne 0 ]]; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
all_match="$(python3 - <<'PY' "$SUMMARY_JSON"
|
||||
import json,sys
|
||||
print("true" if json.load(open(sys.argv[1]))["allMatch"] else "false")
|
||||
PY
|
||||
)"
|
||||
if [[ "$all_match" != "true" ]]; then
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "done: $OUT_DIR"
|
||||
162
scripts/cir/run_cir_replay_ours.sh
Executable file
162
scripts/cir/run_cir_replay_ours.sh
Executable file
@@ -0,0 +1,162 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/cir/run_cir_replay_ours.sh \
|
||||
--cir <path> \
|
||||
[--static-root <path> | --raw-store-db <path>] \
|
||||
--out-dir <path> \
|
||||
--reference-ccr <path> \
|
||||
[--keep-db] \
|
||||
[--rpki-bin <path>] \
|
||||
[--real-rsync-bin <path>]
|
||||
EOF
|
||||
}
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
|
||||
CIR=""
|
||||
STATIC_ROOT=""
|
||||
POOL_DB=""
|
||||
RAW_STORE_DB=""
|
||||
OUT_DIR=""
|
||||
REFERENCE_CCR=""
|
||||
KEEP_DB=0
|
||||
RPKI_BIN="$ROOT_DIR/target/release/rpki"
|
||||
CIR_MATERIALIZE_BIN="$ROOT_DIR/target/release/cir_materialize"
|
||||
CIR_EXTRACT_INPUTS_BIN="$ROOT_DIR/target/release/cir_extract_inputs"
|
||||
CCR_TO_COMPARE_VIEWS_BIN="$ROOT_DIR/target/release/ccr_to_compare_views"
|
||||
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
||||
WRAPPER="$ROOT_DIR/scripts/cir/cir-rsync-wrapper"
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--cir) CIR="$2"; shift 2 ;;
|
||||
--static-root) STATIC_ROOT="$2"; shift 2 ;;
|
||||
--raw-store-db) RAW_STORE_DB="$2"; shift 2 ;;
|
||||
--out-dir) OUT_DIR="$2"; shift 2 ;;
|
||||
--reference-ccr) REFERENCE_CCR="$2"; shift 2 ;;
|
||||
--keep-db) KEEP_DB=1; shift ;;
|
||||
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
||||
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -n "$CIR" && -n "$OUT_DIR" && -n "$REFERENCE_CCR" ]] || {
|
||||
usage >&2
|
||||
exit 2
|
||||
}
|
||||
backend_count=0
|
||||
[[ -n "$STATIC_ROOT" ]] && backend_count=$((backend_count+1))
|
||||
[[ -n "$RAW_STORE_DB" ]] && backend_count=$((backend_count+1))
|
||||
[[ "$backend_count" -eq 1 ]] || { usage >&2; exit 2; }
|
||||
|
||||
mkdir -p "$OUT_DIR"
|
||||
|
||||
if [[ ! -x "$RPKI_BIN" || ! -x "$CIR_MATERIALIZE_BIN" || ! -x "$CIR_EXTRACT_INPUTS_BIN" || ! -x "$CCR_TO_COMPARE_VIEWS_BIN" ]]; then
|
||||
(
|
||||
cd "$ROOT_DIR"
|
||||
cargo build --release --bin rpki --bin cir_materialize --bin cir_extract_inputs --bin ccr_to_compare_views
|
||||
)
|
||||
fi
|
||||
|
||||
TMP_ROOT="$OUT_DIR/.tmp"
|
||||
TALS_DIR="$TMP_ROOT/tals"
|
||||
META_JSON="$TMP_ROOT/meta.json"
|
||||
MIRROR_ROOT="$TMP_ROOT/mirror"
|
||||
DB_DIR="$TMP_ROOT/work-db"
|
||||
ACTUAL_CCR="$OUT_DIR/actual.ccr"
|
||||
ACTUAL_REPORT="$OUT_DIR/report.json"
|
||||
ACTUAL_VRPS="$OUT_DIR/actual-vrps.csv"
|
||||
ACTUAL_VAPS="$OUT_DIR/actual-vaps.csv"
|
||||
REF_VRPS="$OUT_DIR/reference-vrps.csv"
|
||||
REF_VAPS="$OUT_DIR/reference-vaps.csv"
|
||||
COMPARE_JSON="$OUT_DIR/compare-summary.json"
|
||||
RUN_LOG="$OUT_DIR/run.log"
|
||||
|
||||
rm -rf "$TMP_ROOT"
|
||||
mkdir -p "$TMP_ROOT"
|
||||
|
||||
"$CIR_EXTRACT_INPUTS_BIN" --cir "$CIR" --tals-dir "$TALS_DIR" --meta-json "$META_JSON"
|
||||
materialize_cmd=("$CIR_MATERIALIZE_BIN" --cir "$CIR" --mirror-root "$MIRROR_ROOT")
|
||||
if [[ -n "$STATIC_ROOT" ]]; then
|
||||
materialize_cmd+=(--static-root "$STATIC_ROOT")
|
||||
else
|
||||
materialize_cmd+=(--raw-store-db "$RAW_STORE_DB")
|
||||
fi
|
||||
if [[ "$KEEP_DB" -eq 1 ]]; then
|
||||
materialize_cmd+=(--keep-db)
|
||||
fi
|
||||
"${materialize_cmd[@]}"
|
||||
|
||||
VALIDATION_TIME="$(python3 - <<'PY' "$META_JSON"
|
||||
import json,sys
|
||||
print(json.load(open(sys.argv[1]))["validationTime"])
|
||||
PY
|
||||
)"
|
||||
FIRST_TAL="$(python3 - <<'PY' "$META_JSON"
|
||||
import json,sys
|
||||
print(json.load(open(sys.argv[1]))["talFiles"][0]["path"])
|
||||
PY
|
||||
)"
|
||||
|
||||
export CIR_MIRROR_ROOT="$(python3 - <<'PY' "$MIRROR_ROOT"
|
||||
from pathlib import Path
|
||||
import sys
|
||||
print(Path(sys.argv[1]).resolve())
|
||||
PY
|
||||
)"
|
||||
export REAL_RSYNC_BIN="$REAL_RSYNC_BIN"
|
||||
export CIR_LOCAL_LINK_MODE=1
|
||||
|
||||
"$RPKI_BIN" \
|
||||
--db "$DB_DIR" \
|
||||
--tal-path "$FIRST_TAL" \
|
||||
--disable-rrdp \
|
||||
--rsync-command "$WRAPPER" \
|
||||
--validation-time "$VALIDATION_TIME" \
|
||||
--ccr-out "$ACTUAL_CCR" \
|
||||
--report-json "$ACTUAL_REPORT" \
|
||||
>"$RUN_LOG" 2>&1
|
||||
|
||||
"$CCR_TO_COMPARE_VIEWS_BIN" --ccr "$ACTUAL_CCR" --vrps-out "$ACTUAL_VRPS" --vaps-out "$ACTUAL_VAPS" --trust-anchor unknown
|
||||
"$CCR_TO_COMPARE_VIEWS_BIN" --ccr "$REFERENCE_CCR" --vrps-out "$REF_VRPS" --vaps-out "$REF_VAPS" --trust-anchor unknown
|
||||
|
||||
python3 - <<'PY' "$ACTUAL_VRPS" "$REF_VRPS" "$ACTUAL_VAPS" "$REF_VAPS" "$COMPARE_JSON"
|
||||
import csv, json, sys
|
||||
def rows(path):
|
||||
with open(path, newline="") as f:
|
||||
return list(csv.reader(f))[1:]
|
||||
actual_vrps = {tuple(r) for r in rows(sys.argv[1])}
|
||||
ref_vrps = {tuple(r) for r in rows(sys.argv[2])}
|
||||
actual_vaps = {tuple(r) for r in rows(sys.argv[3])}
|
||||
ref_vaps = {tuple(r) for r in rows(sys.argv[4])}
|
||||
summary = {
|
||||
"vrps": {
|
||||
"actual": len(actual_vrps),
|
||||
"reference": len(ref_vrps),
|
||||
"only_in_actual": sorted(actual_vrps - ref_vrps)[:20],
|
||||
"only_in_reference": sorted(ref_vrps - actual_vrps)[:20],
|
||||
"match": actual_vrps == ref_vrps,
|
||||
},
|
||||
"vaps": {
|
||||
"actual": len(actual_vaps),
|
||||
"reference": len(ref_vaps),
|
||||
"only_in_actual": sorted(actual_vaps - ref_vaps)[:20],
|
||||
"only_in_reference": sorted(ref_vaps - actual_vaps)[:20],
|
||||
"match": actual_vaps == ref_vaps,
|
||||
}
|
||||
}
|
||||
with open(sys.argv[5], "w") as f:
|
||||
json.dump(summary, f, indent=2)
|
||||
PY
|
||||
|
||||
if [[ "$KEEP_DB" -ne 1 ]]; then
|
||||
rm -rf "$TMP_ROOT"
|
||||
fi
|
||||
|
||||
echo "done: $OUT_DIR"
|
||||
221
scripts/cir/run_cir_replay_routinator.sh
Executable file
221
scripts/cir/run_cir_replay_routinator.sh
Executable file
@@ -0,0 +1,221 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/cir/run_cir_replay_routinator.sh \
|
||||
--cir <path> \
|
||||
[--static-root <path> | --raw-store-db <path>] \
|
||||
--out-dir <path> \
|
||||
--reference-ccr <path> \
|
||||
[--keep-db] \
|
||||
[--routinator-root <path>] \
|
||||
[--routinator-bin <path>] \
|
||||
[--real-rsync-bin <path>]
|
||||
EOF
|
||||
}
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
RPKI_DEV_ROOT="${RPKI_DEV_ROOT:-$ROOT_DIR}"
|
||||
|
||||
CIR=""
|
||||
STATIC_ROOT=""
|
||||
POOL_DB=""
|
||||
RAW_STORE_DB=""
|
||||
OUT_DIR=""
|
||||
REFERENCE_CCR=""
|
||||
KEEP_DB=0
|
||||
ROUTINATOR_ROOT="${ROUTINATOR_ROOT:-/home/yuyr/dev/rust_playground/routinator}"
|
||||
ROUTINATOR_BIN="${ROUTINATOR_BIN:-$ROUTINATOR_ROOT/target/debug/routinator}"
|
||||
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
||||
CIR_MATERIALIZE_BIN="$ROOT_DIR/target/release/cir_materialize"
|
||||
CIR_EXTRACT_INPUTS_BIN="$ROOT_DIR/target/release/cir_extract_inputs"
|
||||
CCR_TO_COMPARE_VIEWS_BIN="$ROOT_DIR/target/release/ccr_to_compare_views"
|
||||
WRAPPER="$ROOT_DIR/scripts/cir/cir-rsync-wrapper"
|
||||
JSON_TO_VAPS="$ROOT_DIR/scripts/cir/json_to_vaps_csv.py"
|
||||
FAKETIME_LIB="${FAKETIME_LIB:-$ROOT_DIR/target/tools/faketime_pkg/extracted/libfaketime/usr/lib/x86_64-linux-gnu/faketime/libfaketime.so.1}"
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--cir) CIR="$2"; shift 2 ;;
|
||||
--static-root) STATIC_ROOT="$2"; shift 2 ;;
|
||||
--raw-store-db) RAW_STORE_DB="$2"; shift 2 ;;
|
||||
--out-dir) OUT_DIR="$2"; shift 2 ;;
|
||||
--reference-ccr) REFERENCE_CCR="$2"; shift 2 ;;
|
||||
--keep-db) KEEP_DB=1; shift ;;
|
||||
--routinator-root) ROUTINATOR_ROOT="$2"; shift 2 ;;
|
||||
--routinator-bin) ROUTINATOR_BIN="$2"; shift 2 ;;
|
||||
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -n "$CIR" && -n "$OUT_DIR" && -n "$REFERENCE_CCR" ]] || {
|
||||
usage >&2
|
||||
exit 2
|
||||
}
|
||||
backend_count=0
|
||||
[[ -n "$STATIC_ROOT" ]] && backend_count=$((backend_count+1))
|
||||
[[ -n "$RAW_STORE_DB" ]] && backend_count=$((backend_count+1))
|
||||
[[ "$backend_count" -eq 1 ]] || { usage >&2; exit 2; }
|
||||
|
||||
mkdir -p "$OUT_DIR"
|
||||
if [[ ! -x "$CIR_MATERIALIZE_BIN" || ! -x "$CIR_EXTRACT_INPUTS_BIN" || ! -x "$CCR_TO_COMPARE_VIEWS_BIN" ]]; then
|
||||
(
|
||||
cd "$ROOT_DIR"
|
||||
cargo build --release --bin cir_materialize --bin cir_extract_inputs --bin ccr_to_compare_views
|
||||
)
|
||||
fi
|
||||
|
||||
TMP_ROOT="$OUT_DIR/.tmp"
|
||||
TALS_DIR="$TMP_ROOT/tals"
|
||||
META_JSON="$TMP_ROOT/meta.json"
|
||||
MIRROR_ROOT="$TMP_ROOT/mirror"
|
||||
WORK_REPO="$TMP_ROOT/repository"
|
||||
RUN_LOG="$OUT_DIR/routinator.log"
|
||||
ACTUAL_VRPS="$OUT_DIR/actual-vrps.csv"
|
||||
ACTUAL_VAPS_JSON="$OUT_DIR/actual-vaps.json"
|
||||
ACTUAL_VAPS="$OUT_DIR/actual-vaps.csv"
|
||||
REF_VRPS="$OUT_DIR/reference-vrps.csv"
|
||||
REF_VAPS="$OUT_DIR/reference-vaps.csv"
|
||||
SUMMARY_JSON="$OUT_DIR/compare-summary.json"
|
||||
|
||||
rm -rf "$TMP_ROOT"
|
||||
mkdir -p "$TMP_ROOT"
|
||||
|
||||
"$CIR_EXTRACT_INPUTS_BIN" --cir "$CIR" --tals-dir "$TALS_DIR" --meta-json "$META_JSON"
|
||||
python3 - <<'PY' "$TALS_DIR"
|
||||
from pathlib import Path
|
||||
import sys
|
||||
for tal in Path(sys.argv[1]).glob("*.tal"):
|
||||
lines = tal.read_text(encoding="utf-8").splitlines()
|
||||
rsync_uris = [line for line in lines if line.startswith("rsync://")]
|
||||
base64_lines = []
|
||||
seen_sep = False
|
||||
for line in lines:
|
||||
if seen_sep:
|
||||
if line.strip():
|
||||
base64_lines.append(line)
|
||||
elif line.strip() == "":
|
||||
seen_sep = True
|
||||
tal.write_text("\n".join(rsync_uris) + "\n\n" + "\n".join(base64_lines) + "\n", encoding="utf-8")
|
||||
PY
|
||||
materialize_cmd=("$CIR_MATERIALIZE_BIN" --cir "$CIR" --mirror-root "$MIRROR_ROOT")
|
||||
if [[ -n "$STATIC_ROOT" ]]; then
|
||||
materialize_cmd+=(--static-root "$STATIC_ROOT")
|
||||
else
|
||||
materialize_cmd+=(--raw-store-db "$RAW_STORE_DB")
|
||||
fi
|
||||
if [[ "$KEEP_DB" -eq 1 ]]; then
|
||||
materialize_cmd+=(--keep-db)
|
||||
fi
|
||||
"${materialize_cmd[@]}"
|
||||
|
||||
VALIDATION_TIME="$(python3 - <<'PY' "$META_JSON"
|
||||
import json,sys
|
||||
print(json.load(open(sys.argv[1]))["validationTime"])
|
||||
PY
|
||||
)"
|
||||
FIRST_TAL="$(python3 - <<'PY' "$META_JSON"
|
||||
import json,sys
|
||||
print(json.load(open(sys.argv[1]))["talFiles"][0]["path"])
|
||||
PY
|
||||
)"
|
||||
TA_NAME="$(basename "$FIRST_TAL" .tal)"
|
||||
FAKE_EPOCH="$(python3 - <<'PY' "$VALIDATION_TIME"
|
||||
from datetime import datetime, timezone
|
||||
import sys
|
||||
dt = datetime.fromisoformat(sys.argv[1].replace("Z", "+00:00")).astimezone(timezone.utc)
|
||||
print(int(dt.timestamp()))
|
||||
PY
|
||||
)"
|
||||
|
||||
export CIR_MIRROR_ROOT="$(python3 - <<'PY' "$MIRROR_ROOT"
|
||||
from pathlib import Path
|
||||
import sys
|
||||
print(Path(sys.argv[1]).resolve())
|
||||
PY
|
||||
)"
|
||||
export REAL_RSYNC_BIN="$REAL_RSYNC_BIN"
|
||||
export CIR_LOCAL_LINK_MODE=1
|
||||
env \
|
||||
LD_PRELOAD="$FAKETIME_LIB" \
|
||||
FAKETIME_FMT=%s \
|
||||
FAKETIME="$FAKE_EPOCH" \
|
||||
FAKETIME_DONT_FAKE_MONOTONIC=1 \
|
||||
"$ROUTINATOR_BIN" \
|
||||
--repository-dir "$WORK_REPO" \
|
||||
--disable-rrdp \
|
||||
--rsync-command "$WRAPPER" \
|
||||
--no-rir-tals \
|
||||
--extra-tals-dir "$TALS_DIR" \
|
||||
--enable-aspa \
|
||||
update --complete >"$RUN_LOG" 2>&1 || true
|
||||
|
||||
env \
|
||||
LD_PRELOAD="$FAKETIME_LIB" \
|
||||
FAKETIME_FMT=%s \
|
||||
FAKETIME="$FAKE_EPOCH" \
|
||||
FAKETIME_DONT_FAKE_MONOTONIC=1 \
|
||||
"$ROUTINATOR_BIN" \
|
||||
--repository-dir "$WORK_REPO" \
|
||||
--disable-rrdp \
|
||||
--rsync-command "$WRAPPER" \
|
||||
--no-rir-tals \
|
||||
--extra-tals-dir "$TALS_DIR" \
|
||||
--enable-aspa \
|
||||
vrps --noupdate -o "$ACTUAL_VRPS" >>"$RUN_LOG" 2>&1
|
||||
|
||||
env \
|
||||
LD_PRELOAD="$FAKETIME_LIB" \
|
||||
FAKETIME_FMT=%s \
|
||||
FAKETIME="$FAKE_EPOCH" \
|
||||
FAKETIME_DONT_FAKE_MONOTONIC=1 \
|
||||
"$ROUTINATOR_BIN" \
|
||||
--repository-dir "$WORK_REPO" \
|
||||
--disable-rrdp \
|
||||
--rsync-command "$WRAPPER" \
|
||||
--no-rir-tals \
|
||||
--extra-tals-dir "$TALS_DIR" \
|
||||
--enable-aspa \
|
||||
vrps --noupdate --format json -o "$ACTUAL_VAPS_JSON" >>"$RUN_LOG" 2>&1
|
||||
|
||||
python3 "$JSON_TO_VAPS" --input "$ACTUAL_VAPS_JSON" --csv-out "$ACTUAL_VAPS"
|
||||
"$CCR_TO_COMPARE_VIEWS_BIN" --ccr "$REFERENCE_CCR" --vrps-out "$REF_VRPS" --vaps-out "$REF_VAPS" --trust-anchor "$TA_NAME"
|
||||
|
||||
python3 - <<'PY' "$ACTUAL_VRPS" "$REF_VRPS" "$ACTUAL_VAPS" "$REF_VAPS" "$SUMMARY_JSON"
|
||||
import csv, json, sys
|
||||
def rows(path):
|
||||
with open(path, newline="") as f:
|
||||
return list(csv.reader(f))[1:]
|
||||
actual_vrps = {tuple(r) for r in rows(sys.argv[1])}
|
||||
ref_vrps = {tuple(r) for r in rows(sys.argv[2])}
|
||||
actual_vaps = {tuple(r) for r in rows(sys.argv[3])}
|
||||
ref_vaps = {tuple(r) for r in rows(sys.argv[4])}
|
||||
summary = {
|
||||
"vrps": {
|
||||
"actual": len(actual_vrps),
|
||||
"reference": len(ref_vrps),
|
||||
"match": actual_vrps == ref_vrps,
|
||||
"only_in_actual": sorted(actual_vrps - ref_vrps)[:20],
|
||||
"only_in_reference": sorted(ref_vrps - actual_vrps)[:20],
|
||||
},
|
||||
"vaps": {
|
||||
"actual": len(actual_vaps),
|
||||
"reference": len(ref_vaps),
|
||||
"match": actual_vaps == ref_vaps,
|
||||
"only_in_actual": sorted(actual_vaps - ref_vaps)[:20],
|
||||
"only_in_reference": sorted(ref_vaps - actual_vaps)[:20],
|
||||
}
|
||||
}
|
||||
with open(sys.argv[5], "w") as f:
|
||||
json.dump(summary, f, indent=2)
|
||||
PY
|
||||
|
||||
if [[ "$KEEP_DB" -ne 1 ]]; then
|
||||
rm -rf "$TMP_ROOT"
|
||||
fi
|
||||
|
||||
echo "done: $OUT_DIR"
|
||||
195
scripts/cir/run_cir_replay_rpki_client.sh
Executable file
195
scripts/cir/run_cir_replay_rpki_client.sh
Executable file
@@ -0,0 +1,195 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/cir/run_cir_replay_rpki_client.sh \
|
||||
--cir <path> \
|
||||
[--static-root <path> | --raw-store-db <path>] \
|
||||
--out-dir <path> \
|
||||
--reference-ccr <path> \
|
||||
--build-dir <path> \
|
||||
[--keep-db] \
|
||||
[--real-rsync-bin <path>]
|
||||
EOF
|
||||
}
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
CIR=""
|
||||
STATIC_ROOT=""
|
||||
POOL_DB=""
|
||||
RAW_STORE_DB=""
|
||||
OUT_DIR=""
|
||||
REFERENCE_CCR=""
|
||||
BUILD_DIR=""
|
||||
KEEP_DB=0
|
||||
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
||||
CIR_MATERIALIZE_BIN="$ROOT_DIR/target/release/cir_materialize"
|
||||
CIR_EXTRACT_INPUTS_BIN="$ROOT_DIR/target/release/cir_extract_inputs"
|
||||
CCR_TO_COMPARE_VIEWS_BIN="$ROOT_DIR/target/release/ccr_to_compare_views"
|
||||
WRAPPER="$ROOT_DIR/scripts/cir/cir-rsync-wrapper"
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--cir) CIR="$2"; shift 2 ;;
|
||||
--static-root) STATIC_ROOT="$2"; shift 2 ;;
|
||||
--raw-store-db) RAW_STORE_DB="$2"; shift 2 ;;
|
||||
--out-dir) OUT_DIR="$2"; shift 2 ;;
|
||||
--reference-ccr) REFERENCE_CCR="$2"; shift 2 ;;
|
||||
--build-dir) BUILD_DIR="$2"; shift 2 ;;
|
||||
--keep-db) KEEP_DB=1; shift ;;
|
||||
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -n "$CIR" && -n "$OUT_DIR" && -n "$REFERENCE_CCR" && -n "$BUILD_DIR" ]] || {
|
||||
usage >&2
|
||||
exit 2
|
||||
}
|
||||
backend_count=0
|
||||
[[ -n "$STATIC_ROOT" ]] && backend_count=$((backend_count+1))
|
||||
[[ -n "$RAW_STORE_DB" ]] && backend_count=$((backend_count+1))
|
||||
[[ "$backend_count" -eq 1 ]] || { usage >&2; exit 2; }
|
||||
|
||||
mkdir -p "$OUT_DIR"
|
||||
if [[ ! -x "$CIR_MATERIALIZE_BIN" || ! -x "$CIR_EXTRACT_INPUTS_BIN" || ! -x "$CCR_TO_COMPARE_VIEWS_BIN" ]]; then
|
||||
(
|
||||
cd "$ROOT_DIR"
|
||||
cargo build --release --bin cir_materialize --bin cir_extract_inputs --bin ccr_to_compare_views
|
||||
)
|
||||
fi
|
||||
|
||||
TMP_ROOT="$OUT_DIR/.tmp"
|
||||
TALS_DIR="$TMP_ROOT/tals"
|
||||
META_JSON="$TMP_ROOT/meta.json"
|
||||
MIRROR_ROOT="$TMP_ROOT/mirror"
|
||||
CACHE_DIR="$TMP_ROOT/cache"
|
||||
OUT_CCR_DIR="$TMP_ROOT/out"
|
||||
RUN_LOG="$OUT_DIR/rpki-client.log"
|
||||
ACTUAL_VRPS="$OUT_DIR/actual-vrps.csv"
|
||||
ACTUAL_VAPS="$OUT_DIR/actual-vaps.csv"
|
||||
ACTUAL_VAPS_META="$OUT_DIR/actual-vaps-meta.json"
|
||||
ACTUAL_VRPS_META="$OUT_DIR/actual-vrps-meta.json"
|
||||
REF_VRPS="$OUT_DIR/reference-vrps.csv"
|
||||
REF_VAPS="$OUT_DIR/reference-vaps.csv"
|
||||
SUMMARY_JSON="$OUT_DIR/compare-summary.json"
|
||||
|
||||
rm -rf "$TMP_ROOT"
|
||||
mkdir -p "$TMP_ROOT"
|
||||
|
||||
"$CIR_EXTRACT_INPUTS_BIN" --cir "$CIR" --tals-dir "$TALS_DIR" --meta-json "$META_JSON"
|
||||
python3 - <<'PY' "$TALS_DIR"
|
||||
from pathlib import Path
|
||||
import sys
|
||||
for tal in Path(sys.argv[1]).glob("*.tal"):
|
||||
lines = tal.read_text(encoding="utf-8").splitlines()
|
||||
rsync_uris = [line for line in lines if line.startswith("rsync://")]
|
||||
base64_lines = []
|
||||
seen_sep = False
|
||||
for line in lines:
|
||||
if seen_sep:
|
||||
if line.strip():
|
||||
base64_lines.append(line)
|
||||
elif line.strip() == "":
|
||||
seen_sep = True
|
||||
tal.write_text("\n".join(rsync_uris) + "\n\n" + "\n".join(base64_lines) + "\n", encoding="utf-8")
|
||||
PY
|
||||
materialize_cmd=("$CIR_MATERIALIZE_BIN" --cir "$CIR" --mirror-root "$MIRROR_ROOT")
|
||||
if [[ -n "$STATIC_ROOT" ]]; then
|
||||
materialize_cmd+=(--static-root "$STATIC_ROOT")
|
||||
else
|
||||
materialize_cmd+=(--raw-store-db "$RAW_STORE_DB")
|
||||
fi
|
||||
if [[ "$KEEP_DB" -eq 1 ]]; then
|
||||
materialize_cmd+=(--keep-db)
|
||||
fi
|
||||
"${materialize_cmd[@]}"
|
||||
|
||||
VALIDATION_EPOCH="$(python3 - <<'PY' "$META_JSON"
|
||||
from datetime import datetime, timezone
|
||||
import json, sys
|
||||
vt = json.load(open(sys.argv[1]))["validationTime"]
|
||||
dt = datetime.fromisoformat(vt.replace("Z", "+00:00")).astimezone(timezone.utc)
|
||||
print(int(dt.timestamp()))
|
||||
PY
|
||||
)"
|
||||
FIRST_TAL="$(python3 - <<'PY' "$META_JSON"
|
||||
import json,sys
|
||||
print(json.load(open(sys.argv[1]))["talFiles"][0]["path"])
|
||||
PY
|
||||
)"
|
||||
TA_NAME="$(basename "$FIRST_TAL" .tal)"
|
||||
|
||||
export CIR_MIRROR_ROOT="$(python3 - <<'PY' "$MIRROR_ROOT"
|
||||
from pathlib import Path
|
||||
import sys
|
||||
print(Path(sys.argv[1]).resolve())
|
||||
PY
|
||||
)"
|
||||
export REAL_RSYNC_BIN="$REAL_RSYNC_BIN"
|
||||
export CIR_LOCAL_LINK_MODE=1
|
||||
|
||||
mkdir -p "$CACHE_DIR" "$OUT_CCR_DIR"
|
||||
"$BUILD_DIR/src/rpki-client" \
|
||||
-R \
|
||||
-e "$WRAPPER" \
|
||||
-P "$VALIDATION_EPOCH" \
|
||||
-t "$FIRST_TAL" \
|
||||
-d "$CACHE_DIR" \
|
||||
"$OUT_CCR_DIR" >"$RUN_LOG" 2>&1
|
||||
|
||||
"$CCR_TO_COMPARE_VIEWS_BIN" \
|
||||
--ccr "$OUT_CCR_DIR/rpki.ccr" \
|
||||
--vrps-out "$ACTUAL_VRPS" \
|
||||
--vaps-out "$ACTUAL_VAPS" \
|
||||
--trust-anchor "$TA_NAME"
|
||||
|
||||
python3 - <<'PY' "$ACTUAL_VRPS" "$ACTUAL_VAPS" "$ACTUAL_VRPS_META" "$ACTUAL_VAPS_META"
|
||||
import csv, json, sys
|
||||
def count_rows(path):
|
||||
with open(path, newline="") as f:
|
||||
rows = list(csv.reader(f))
|
||||
return max(len(rows) - 1, 0)
|
||||
json.dump({"count": count_rows(sys.argv[1])}, open(sys.argv[3], "w"), indent=2)
|
||||
json.dump({"count": count_rows(sys.argv[2])}, open(sys.argv[4], "w"), indent=2)
|
||||
PY
|
||||
|
||||
"$CCR_TO_COMPARE_VIEWS_BIN" --ccr "$REFERENCE_CCR" --vrps-out "$REF_VRPS" --vaps-out "$REF_VAPS" --trust-anchor "$TA_NAME"
|
||||
|
||||
python3 - <<'PY' "$ACTUAL_VRPS" "$REF_VRPS" "$ACTUAL_VAPS" "$REF_VAPS" "$SUMMARY_JSON"
|
||||
import csv, json, sys
|
||||
def rows(path):
|
||||
with open(path, newline="") as f:
|
||||
return list(csv.reader(f))[1:]
|
||||
actual_vrps = {tuple(r) for r in rows(sys.argv[1])}
|
||||
ref_vrps = {tuple(r) for r in rows(sys.argv[2])}
|
||||
actual_vaps = {tuple(r) for r in rows(sys.argv[3])}
|
||||
ref_vaps = {tuple(r) for r in rows(sys.argv[4])}
|
||||
summary = {
|
||||
"vrps": {
|
||||
"actual": len(actual_vrps),
|
||||
"reference": len(ref_vrps),
|
||||
"match": actual_vrps == ref_vrps,
|
||||
"only_in_actual": sorted(actual_vrps - ref_vrps)[:20],
|
||||
"only_in_reference": sorted(ref_vrps - actual_vrps)[:20],
|
||||
},
|
||||
"vaps": {
|
||||
"actual": len(actual_vaps),
|
||||
"reference": len(ref_vaps),
|
||||
"match": actual_vaps == ref_vaps,
|
||||
"only_in_actual": sorted(actual_vaps - ref_vaps)[:20],
|
||||
"only_in_reference": sorted(ref_vaps - actual_vaps)[:20],
|
||||
}
|
||||
}
|
||||
with open(sys.argv[5], "w") as f:
|
||||
json.dump(summary, f, indent=2)
|
||||
PY
|
||||
|
||||
if [[ "$KEEP_DB" -ne 1 ]]; then
|
||||
rm -rf "$TMP_ROOT"
|
||||
fi
|
||||
|
||||
echo "done: $OUT_DIR"
|
||||
149
scripts/cir/run_cir_replay_sequence_ours.sh
Executable file
149
scripts/cir/run_cir_replay_sequence_ours.sh
Executable file
@@ -0,0 +1,149 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/cir/run_cir_replay_sequence_ours.sh \
|
||||
--sequence-root <path> \
|
||||
[--rpki-bin <path>] \
|
||||
[--real-rsync-bin <path>]
|
||||
EOF
|
||||
}
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
SEQUENCE_ROOT=""
|
||||
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
||||
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
||||
STEP_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_ours.sh"
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--sequence-root) SEQUENCE_ROOT="$2"; shift 2 ;;
|
||||
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
||||
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -n "$SEQUENCE_ROOT" ]] || { usage >&2; exit 2; }
|
||||
|
||||
SEQUENCE_ROOT="$(python3 - <<'PY' "$SEQUENCE_ROOT"
|
||||
from pathlib import Path
|
||||
import sys
|
||||
print(Path(sys.argv[1]).resolve())
|
||||
PY
|
||||
)"
|
||||
|
||||
SUMMARY_JSON="$SEQUENCE_ROOT/sequence-summary.json"
|
||||
SUMMARY_MD="$SEQUENCE_ROOT/sequence-summary.md"
|
||||
DETAIL_JSON="$SEQUENCE_ROOT/sequence-detail.json"
|
||||
|
||||
python3 - <<'PY' "$SEQUENCE_ROOT" "$SUMMARY_JSON" "$SUMMARY_MD" "$DETAIL_JSON" "$STEP_SCRIPT" "$RPKI_BIN" "$REAL_RSYNC_BIN"
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sequence_root = Path(sys.argv[1])
|
||||
summary_json = Path(sys.argv[2])
|
||||
summary_md = Path(sys.argv[3])
|
||||
detail_json = Path(sys.argv[4])
|
||||
step_script = Path(sys.argv[5])
|
||||
rpki_bin = sys.argv[6]
|
||||
real_rsync_bin = sys.argv[7]
|
||||
|
||||
sequence = json.loads((sequence_root / "sequence.json").read_text(encoding="utf-8"))
|
||||
static_root = sequence_root / sequence["staticRoot"] if "staticRoot" in sequence else None
|
||||
raw_store_db = sequence_root / sequence["rawStoreDbPath"] if "rawStoreDbPath" in sequence else None
|
||||
backend_count = sum(x is not None for x in (static_root, raw_store_db))
|
||||
if backend_count != 1:
|
||||
raise SystemExit("sequence must set exactly one of staticRoot or rawStoreDbPath")
|
||||
steps = sequence["steps"]
|
||||
|
||||
results = []
|
||||
all_match = True
|
||||
for step in steps:
|
||||
step_id = step["stepId"]
|
||||
out_dir = sequence_root / "replay-ours" / step_id
|
||||
out_dir.parent.mkdir(parents=True, exist_ok=True)
|
||||
cmd = [
|
||||
str(step_script),
|
||||
"--cir",
|
||||
str(sequence_root / step["cirPath"]),
|
||||
"--out-dir",
|
||||
str(out_dir),
|
||||
"--reference-ccr",
|
||||
str(sequence_root / step["ccrPath"]),
|
||||
"--rpki-bin",
|
||||
rpki_bin,
|
||||
"--real-rsync-bin",
|
||||
real_rsync_bin,
|
||||
]
|
||||
if static_root is not None:
|
||||
cmd.extend(["--static-root", str(static_root)])
|
||||
else:
|
||||
cmd.extend(["--raw-store-db", str(raw_store_db)])
|
||||
proc = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if proc.returncode != 0:
|
||||
raise SystemExit(
|
||||
f"ours sequence replay failed for {step_id}: stdout={proc.stdout} stderr={proc.stderr}"
|
||||
)
|
||||
compare = json.loads((out_dir / "compare-summary.json").read_text(encoding="utf-8"))
|
||||
timing = json.loads((out_dir / "timing.json").read_text(encoding="utf-8")) if (out_dir / "timing.json").exists() else {}
|
||||
record = {
|
||||
"stepId": step_id,
|
||||
"kind": step["kind"],
|
||||
"validationTime": step["validationTime"],
|
||||
"outDir": str(out_dir),
|
||||
"comparePath": str(out_dir / "compare-summary.json"),
|
||||
"timingPath": str(out_dir / "timing.json"),
|
||||
"compare": compare,
|
||||
"timing": timing,
|
||||
"match": bool(compare["vrps"]["match"]) and bool(compare["vaps"]["match"]),
|
||||
}
|
||||
all_match = all_match and record["match"]
|
||||
results.append(record)
|
||||
|
||||
summary = {
|
||||
"version": 1,
|
||||
"participant": "ours",
|
||||
"sequenceRoot": str(sequence_root),
|
||||
"stepCount": len(results),
|
||||
"allMatch": all_match,
|
||||
"steps": results,
|
||||
}
|
||||
summary_json.write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
||||
detail_json.write_text(json.dumps(results, indent=2), encoding="utf-8")
|
||||
|
||||
lines = [
|
||||
"# Ours CIR Sequence Replay Summary",
|
||||
"",
|
||||
f"- `sequence_root`: `{sequence_root}`",
|
||||
f"- `step_count`: `{len(results)}`",
|
||||
f"- `all_match`: `{all_match}`",
|
||||
"",
|
||||
"| Step | Kind | VRP actual/ref | VRP match | VAP actual/ref | VAP match | Duration (ms) |",
|
||||
"| --- | --- | --- | --- | --- | --- | ---: |",
|
||||
]
|
||||
for item in results:
|
||||
compare = item["compare"]
|
||||
timing = item.get("timing") or {}
|
||||
lines.append(
|
||||
"| {step} | {kind} | {va}/{vr} | {vm} | {aa}/{ar} | {am} | {dur} |".format(
|
||||
step=item["stepId"],
|
||||
kind=item["kind"],
|
||||
va=compare["vrps"]["actual"],
|
||||
vr=compare["vrps"]["reference"],
|
||||
vm=compare["vrps"]["match"],
|
||||
aa=compare["vaps"]["actual"],
|
||||
ar=compare["vaps"]["reference"],
|
||||
am=compare["vaps"]["match"],
|
||||
dur=timing.get("durationMs", "-"),
|
||||
)
|
||||
)
|
||||
summary_md.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||
PY
|
||||
|
||||
echo "done: $SEQUENCE_ROOT"
|
||||
132
scripts/cir/run_cir_replay_sequence_routinator.sh
Executable file
132
scripts/cir/run_cir_replay_sequence_routinator.sh
Executable file
@@ -0,0 +1,132 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/cir/run_cir_replay_sequence_routinator.sh \
|
||||
--sequence-root <path> \
|
||||
[--routinator-root <path>] \
|
||||
[--routinator-bin <path>] \
|
||||
[--real-rsync-bin <path>]
|
||||
EOF
|
||||
}
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
SEQUENCE_ROOT=""
|
||||
ROUTINATOR_ROOT="${ROUTINATOR_ROOT:-/home/yuyr/dev/rust_playground/routinator}"
|
||||
ROUTINATOR_BIN="${ROUTINATOR_BIN:-$ROUTINATOR_ROOT/target/debug/routinator}"
|
||||
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
||||
STEP_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_routinator.sh"
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--sequence-root) SEQUENCE_ROOT="$2"; shift 2 ;;
|
||||
--routinator-root) ROUTINATOR_ROOT="$2"; shift 2 ;;
|
||||
--routinator-bin) ROUTINATOR_BIN="$2"; shift 2 ;;
|
||||
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -n "$SEQUENCE_ROOT" ]] || { usage >&2; exit 2; }
|
||||
|
||||
SEQUENCE_ROOT="$(python3 - <<'PY' "$SEQUENCE_ROOT"
|
||||
from pathlib import Path
|
||||
import sys
|
||||
print(Path(sys.argv[1]).resolve())
|
||||
PY
|
||||
)"
|
||||
|
||||
SUMMARY_JSON="$SEQUENCE_ROOT/sequence-summary-routinator.json"
|
||||
SUMMARY_MD="$SEQUENCE_ROOT/sequence-summary-routinator.md"
|
||||
|
||||
python3 - <<'PY' "$SEQUENCE_ROOT" "$SUMMARY_JSON" "$SUMMARY_MD" "$STEP_SCRIPT" "$ROUTINATOR_ROOT" "$ROUTINATOR_BIN" "$REAL_RSYNC_BIN"
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sequence_root = Path(sys.argv[1])
|
||||
summary_json = Path(sys.argv[2])
|
||||
summary_md = Path(sys.argv[3])
|
||||
step_script = Path(sys.argv[4])
|
||||
routinator_root = sys.argv[5]
|
||||
routinator_bin = sys.argv[6]
|
||||
real_rsync_bin = sys.argv[7]
|
||||
|
||||
sequence = json.loads((sequence_root / "sequence.json").read_text(encoding="utf-8"))
|
||||
static_root = sequence_root / sequence["staticRoot"] if "staticRoot" in sequence else None
|
||||
raw_store_db = sequence_root / sequence["rawStoreDbPath"] if "rawStoreDbPath" in sequence else None
|
||||
backend_count = sum(x is not None for x in (static_root, raw_store_db))
|
||||
if backend_count != 1:
|
||||
raise SystemExit("sequence must set exactly one of staticRoot or rawStoreDbPath")
|
||||
steps = sequence["steps"]
|
||||
results = []
|
||||
all_match = True
|
||||
|
||||
for step in steps:
|
||||
step_id = step["stepId"]
|
||||
out_dir = sequence_root / "replay-routinator" / step_id
|
||||
out_dir.parent.mkdir(parents=True, exist_ok=True)
|
||||
cmd = [
|
||||
str(step_script),
|
||||
"--cir",
|
||||
str(sequence_root / step["cirPath"]),
|
||||
"--out-dir",
|
||||
str(out_dir),
|
||||
"--reference-ccr",
|
||||
str(sequence_root / step["ccrPath"]),
|
||||
"--routinator-root",
|
||||
routinator_root,
|
||||
"--routinator-bin",
|
||||
routinator_bin,
|
||||
"--real-rsync-bin",
|
||||
real_rsync_bin,
|
||||
]
|
||||
if static_root is not None:
|
||||
cmd.extend(["--static-root", str(static_root)])
|
||||
else:
|
||||
cmd.extend(["--raw-store-db", str(raw_store_db)])
|
||||
proc = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if proc.returncode != 0:
|
||||
raise SystemExit(
|
||||
f"routinator sequence replay failed for {step_id}: stdout={proc.stdout} stderr={proc.stderr}"
|
||||
)
|
||||
compare = json.loads((out_dir / "compare-summary.json").read_text(encoding="utf-8"))
|
||||
match = bool(compare["vrps"]["match"]) and bool(compare["vaps"]["match"])
|
||||
all_match = all_match and match
|
||||
results.append(
|
||||
{
|
||||
"stepId": step_id,
|
||||
"kind": step["kind"],
|
||||
"validationTime": step["validationTime"],
|
||||
"outDir": str(out_dir),
|
||||
"comparePath": str(out_dir / "compare-summary.json"),
|
||||
"match": match,
|
||||
"compare": compare,
|
||||
}
|
||||
)
|
||||
|
||||
summary = {
|
||||
"version": 1,
|
||||
"participant": "routinator",
|
||||
"sequenceRoot": str(sequence_root),
|
||||
"stepCount": len(results),
|
||||
"allMatch": all_match,
|
||||
"steps": results,
|
||||
}
|
||||
summary_json.write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
||||
lines = [
|
||||
"# Routinator CIR Sequence Replay Summary",
|
||||
"",
|
||||
f"- `sequence_root`: `{sequence_root}`",
|
||||
f"- `step_count`: `{len(results)}`",
|
||||
f"- `all_match`: `{all_match}`",
|
||||
"",
|
||||
]
|
||||
summary_md.write_text("\n".join(lines), encoding="utf-8")
|
||||
PY
|
||||
|
||||
echo "done: $SEQUENCE_ROOT"
|
||||
126
scripts/cir/run_cir_replay_sequence_rpki_client.sh
Executable file
126
scripts/cir/run_cir_replay_sequence_rpki_client.sh
Executable file
@@ -0,0 +1,126 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/cir/run_cir_replay_sequence_rpki_client.sh \
|
||||
--sequence-root <path> \
|
||||
--build-dir <path> \
|
||||
[--real-rsync-bin <path>]
|
||||
EOF
|
||||
}
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
SEQUENCE_ROOT=""
|
||||
BUILD_DIR=""
|
||||
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
||||
STEP_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_rpki_client.sh"
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--sequence-root) SEQUENCE_ROOT="$2"; shift 2 ;;
|
||||
--build-dir) BUILD_DIR="$2"; shift 2 ;;
|
||||
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -n "$SEQUENCE_ROOT" && -n "$BUILD_DIR" ]] || { usage >&2; exit 2; }
|
||||
|
||||
SEQUENCE_ROOT="$(python3 - <<'PY' "$SEQUENCE_ROOT"
|
||||
from pathlib import Path
|
||||
import sys
|
||||
print(Path(sys.argv[1]).resolve())
|
||||
PY
|
||||
)"
|
||||
|
||||
SUMMARY_JSON="$SEQUENCE_ROOT/sequence-summary-rpki-client.json"
|
||||
SUMMARY_MD="$SEQUENCE_ROOT/sequence-summary-rpki-client.md"
|
||||
|
||||
python3 - <<'PY' "$SEQUENCE_ROOT" "$SUMMARY_JSON" "$SUMMARY_MD" "$STEP_SCRIPT" "$BUILD_DIR" "$REAL_RSYNC_BIN"
|
||||
import json
|
||||
import subprocess
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sequence_root = Path(sys.argv[1])
|
||||
summary_json = Path(sys.argv[2])
|
||||
summary_md = Path(sys.argv[3])
|
||||
step_script = Path(sys.argv[4])
|
||||
build_dir = sys.argv[5]
|
||||
real_rsync_bin = sys.argv[6]
|
||||
|
||||
sequence = json.loads((sequence_root / "sequence.json").read_text(encoding="utf-8"))
|
||||
static_root = sequence_root / sequence["staticRoot"] if "staticRoot" in sequence else None
|
||||
raw_store_db = sequence_root / sequence["rawStoreDbPath"] if "rawStoreDbPath" in sequence else None
|
||||
backend_count = sum(x is not None for x in (static_root, raw_store_db))
|
||||
if backend_count != 1:
|
||||
raise SystemExit("sequence must set exactly one of staticRoot or rawStoreDbPath")
|
||||
steps = sequence["steps"]
|
||||
results = []
|
||||
all_match = True
|
||||
|
||||
for step in steps:
|
||||
step_id = step["stepId"]
|
||||
out_dir = sequence_root / "replay-rpki-client" / step_id
|
||||
out_dir.parent.mkdir(parents=True, exist_ok=True)
|
||||
cmd = [
|
||||
str(step_script),
|
||||
"--cir",
|
||||
str(sequence_root / step["cirPath"]),
|
||||
"--out-dir",
|
||||
str(out_dir),
|
||||
"--reference-ccr",
|
||||
str(sequence_root / step["ccrPath"]),
|
||||
"--build-dir",
|
||||
build_dir,
|
||||
"--real-rsync-bin",
|
||||
real_rsync_bin,
|
||||
]
|
||||
if static_root is not None:
|
||||
cmd.extend(["--static-root", str(static_root)])
|
||||
else:
|
||||
cmd.extend(["--raw-store-db", str(raw_store_db)])
|
||||
proc = subprocess.run(cmd, capture_output=True, text=True)
|
||||
if proc.returncode != 0:
|
||||
raise SystemExit(
|
||||
f"rpki-client sequence replay failed for {step_id}: stdout={proc.stdout} stderr={proc.stderr}"
|
||||
)
|
||||
compare = json.loads((out_dir / "compare-summary.json").read_text(encoding="utf-8"))
|
||||
match = bool(compare["vrps"]["match"]) and bool(compare["vaps"]["match"])
|
||||
all_match = all_match and match
|
||||
results.append(
|
||||
{
|
||||
"stepId": step_id,
|
||||
"kind": step["kind"],
|
||||
"validationTime": step["validationTime"],
|
||||
"outDir": str(out_dir),
|
||||
"comparePath": str(out_dir / "compare-summary.json"),
|
||||
"match": match,
|
||||
"compare": compare,
|
||||
}
|
||||
)
|
||||
|
||||
summary = {
|
||||
"version": 1,
|
||||
"participant": "rpki-client",
|
||||
"sequenceRoot": str(sequence_root),
|
||||
"stepCount": len(results),
|
||||
"allMatch": all_match,
|
||||
"steps": results,
|
||||
}
|
||||
summary_json.write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
||||
lines = [
|
||||
"# rpki-client CIR Sequence Replay Summary",
|
||||
"",
|
||||
f"- `sequence_root`: `{sequence_root}`",
|
||||
f"- `step_count`: `{len(results)}`",
|
||||
f"- `all_match`: `{all_match}`",
|
||||
"",
|
||||
]
|
||||
summary_md.write_text("\n".join(lines), encoding="utf-8")
|
||||
PY
|
||||
|
||||
echo "done: $SEQUENCE_ROOT"
|
||||
132
scripts/cir/run_cir_sequence_matrix_multi_rir.sh
Executable file
132
scripts/cir/run_cir_sequence_matrix_multi_rir.sh
Executable file
@@ -0,0 +1,132 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
Usage:
|
||||
./scripts/cir/run_cir_sequence_matrix_multi_rir.sh \
|
||||
--root <path> \
|
||||
[--rir <afrinic,apnic,arin,lacnic,ripe>] \
|
||||
[--rpki-bin <path>] \
|
||||
[--routinator-root <path>] \
|
||||
[--routinator-bin <path>] \
|
||||
[--rpki-client-build-dir <path>] \
|
||||
[--drop-bin <path>]
|
||||
EOF
|
||||
}
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
ROOT=""
|
||||
RIRS="afrinic,apnic,arin,lacnic,ripe"
|
||||
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
||||
ROUTINATOR_ROOT="${ROUTINATOR_ROOT:-/home/yuyr/dev/rust_playground/routinator}"
|
||||
ROUTINATOR_BIN="${ROUTINATOR_BIN:-$ROUTINATOR_ROOT/target/debug/routinator}"
|
||||
RPKI_CLIENT_BUILD_DIR="${RPKI_CLIENT_BUILD_DIR:-/home/yuyr/dev/rpki-client-9.7/build-m5}"
|
||||
DROP_BIN="${DROP_BIN:-$ROOT_DIR/target/release/cir_drop_report}"
|
||||
|
||||
OURS_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_sequence_ours.sh"
|
||||
ROUTINATOR_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_sequence_routinator.sh"
|
||||
RPKIC_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_sequence_rpki_client.sh"
|
||||
DROP_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_drop_sequence.sh"
|
||||
|
||||
while [[ $# -gt 0 ]]; do
|
||||
case "$1" in
|
||||
--root) ROOT="$2"; shift 2 ;;
|
||||
--rir) RIRS="$2"; shift 2 ;;
|
||||
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
||||
--routinator-root) ROUTINATOR_ROOT="$2"; shift 2 ;;
|
||||
--routinator-bin) ROUTINATOR_BIN="$2"; shift 2 ;;
|
||||
--rpki-client-build-dir) RPKI_CLIENT_BUILD_DIR="$2"; shift 2 ;;
|
||||
--drop-bin) DROP_BIN="$2"; shift 2 ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||
esac
|
||||
done
|
||||
|
||||
[[ -n "$ROOT" ]] || { usage >&2; exit 2; }
|
||||
|
||||
SUMMARY_JSON="$ROOT/final-summary.json"
|
||||
SUMMARY_MD="$ROOT/final-summary.md"
|
||||
IFS=',' read -r -a ITEMS <<< "$RIRS"
|
||||
|
||||
results=()
|
||||
for rir in "${ITEMS[@]}"; do
|
||||
seq_root="$ROOT/$rir"
|
||||
"$OURS_SCRIPT" --sequence-root "$seq_root" --rpki-bin "$RPKI_BIN"
|
||||
"$ROUTINATOR_SCRIPT" --sequence-root "$seq_root" --routinator-root "$ROUTINATOR_ROOT" --routinator-bin "$ROUTINATOR_BIN"
|
||||
"$RPKIC_SCRIPT" --sequence-root "$seq_root" --build-dir "$RPKI_CLIENT_BUILD_DIR"
|
||||
"$DROP_SCRIPT" --sequence-root "$seq_root" --drop-bin "$DROP_BIN"
|
||||
done
|
||||
|
||||
python3 - <<'PY' "$ROOT" "$RIRS" "$SUMMARY_JSON" "$SUMMARY_MD"
|
||||
import json, sys
|
||||
from pathlib import Path
|
||||
from collections import Counter
|
||||
|
||||
root = Path(sys.argv[1]).resolve()
|
||||
rirs = [item for item in sys.argv[2].split(',') if item]
|
||||
summary_json = Path(sys.argv[3])
|
||||
summary_md = Path(sys.argv[4])
|
||||
items = []
|
||||
total_steps = 0
|
||||
total_dropped_vrps = 0
|
||||
total_dropped_objects = 0
|
||||
reason_counter = Counter()
|
||||
for rir in rirs:
|
||||
seq_root = root / rir
|
||||
ours = json.loads((seq_root / "sequence-summary.json").read_text(encoding="utf-8"))
|
||||
routinator = json.loads((seq_root / "sequence-summary-routinator.json").read_text(encoding="utf-8"))
|
||||
rpki_client = json.loads((seq_root / "sequence-summary-rpki-client.json").read_text(encoding="utf-8"))
|
||||
drop = json.loads((seq_root / "drop-summary.json").read_text(encoding="utf-8"))
|
||||
step_count = len(ours["steps"])
|
||||
total_steps += step_count
|
||||
rir_dropped_vrps = 0
|
||||
rir_dropped_objects = 0
|
||||
for step in drop["steps"]:
|
||||
drop_path = Path(step["reportPath"])
|
||||
detail = json.loads(drop_path.read_text(encoding="utf-8"))
|
||||
summary = detail.get("summary", {})
|
||||
rir_dropped_vrps += int(summary.get("droppedVrpCount", 0))
|
||||
rir_dropped_objects += int(summary.get("droppedObjectCount", 0))
|
||||
total_dropped_vrps += int(summary.get("droppedVrpCount", 0))
|
||||
total_dropped_objects += int(summary.get("droppedObjectCount", 0))
|
||||
for reason, count in summary.get("droppedByReason", {}).items():
|
||||
reason_counter[reason] += int(count)
|
||||
items.append({
|
||||
"rir": rir,
|
||||
"stepCount": step_count,
|
||||
"oursAllMatch": ours["allMatch"],
|
||||
"routinatorAllMatch": routinator["allMatch"],
|
||||
"rpkiClientAllMatch": rpki_client["allMatch"],
|
||||
"dropSummary": drop["steps"],
|
||||
"droppedVrpCount": rir_dropped_vrps,
|
||||
"droppedObjectCount": rir_dropped_objects,
|
||||
})
|
||||
summary = {
|
||||
"version": 1,
|
||||
"totalStepCount": total_steps,
|
||||
"totalDroppedVrpCount": total_dropped_vrps,
|
||||
"totalDroppedObjectCount": total_dropped_objects,
|
||||
"topReasons": [{"reason": reason, "count": count} for reason, count in reason_counter.most_common(10)],
|
||||
"rirs": items,
|
||||
}
|
||||
summary_json.write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
||||
lines = ["# Multi-RIR CIR Sequence Matrix Summary", ""]
|
||||
lines.append(f"- `total_step_count`: `{total_steps}`")
|
||||
lines.append(f"- `total_dropped_vrps`: `{total_dropped_vrps}`")
|
||||
lines.append(f"- `total_dropped_objects`: `{total_dropped_objects}`")
|
||||
lines.append("")
|
||||
if reason_counter:
|
||||
lines.append("## Top Drop Reasons")
|
||||
lines.append("")
|
||||
for reason, count in reason_counter.most_common(10):
|
||||
lines.append(f"- `{reason}`: `{count}`")
|
||||
lines.append("")
|
||||
for item in items:
|
||||
lines.append(
|
||||
f"- `{item['rir']}`: `steps={item['stepCount']}` `ours={item['oursAllMatch']}` `routinator={item['routinatorAllMatch']}` `rpki-client={item['rpkiClientAllMatch']}` `drop_vrps={item['droppedVrpCount']}` `drop_objects={item['droppedObjectCount']}`"
|
||||
)
|
||||
summary_md.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||
PY
|
||||
|
||||
echo "done: $ROOT"
|
||||
82
scripts/coverage.sh
Executable file
82
scripts/coverage.sh
Executable file
@@ -0,0 +1,82 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Requires:
|
||||
# rustup component add llvm-tools-preview
|
||||
# cargo install cargo-llvm-cov --locked
|
||||
|
||||
run_out="$(mktemp)"
|
||||
text_out="$(mktemp)"
|
||||
html_out="$(mktemp)"
|
||||
|
||||
cleanup() {
|
||||
rm -f "$run_out" "$text_out" "$html_out"
|
||||
}
|
||||
trap cleanup EXIT
|
||||
|
||||
IGNORE_REGEX='src/bin/replay_bundle_capture\.rs|src/bin/replay_bundle_capture_delta\.rs|src/bin/replay_bundle_capture_sequence\.rs|src/bin/replay_bundle_record\.rs|src/bin/replay_bundle_refresh_sequence_outputs\.rs|src/bin/measure_sequence_replay\.rs|src/bin/repository_view_stats\.rs|src/bin/trace_arin_missing_vrps\.rs|src/bin/db_stats\.rs|src/bin/rrdp_state_dump\.rs|src/bin/ccr_dump\.rs|src/bin/ccr_verify\.rs|src/bin/ccr_to_routinator_csv\.rs|src/bin/ccr_to_compare_views\.rs|src/bin/cir_materialize\.rs|src/bin/cir_extract_inputs\.rs|src/bin/cir_drop_report\.rs|src/bin/cir_ta_only_fixture\.rs|src/bundle/live_capture\.rs|src/bundle/record_io\.rs|src/bundle/compare_view\.rs|src/progress_log\.rs|src/cli\.rs|src/validation/run_tree_from_tal\.rs|src/validation/from_tal\.rs|src/sync/store_projection\.rs|src/cir/materialize\.rs'
|
||||
|
||||
# Preserve colored output even though we post-process output by running under a pseudo-TTY.
|
||||
# We run tests only once, then generate both CLI text + HTML reports without rerunning tests.
|
||||
set +e
|
||||
|
||||
cargo llvm-cov clean --workspace >/dev/null 2>&1
|
||||
|
||||
# 1) Run tests once to collect coverage data (no report).
|
||||
script -q -e -c "CARGO_TERM_COLOR=always cargo llvm-cov --no-report" "$run_out" >/dev/null 2>&1
|
||||
run_status="$?"
|
||||
|
||||
# 2) CLI summary report + fail-under gate (no test rerun).
|
||||
script -q -e -c "CARGO_TERM_COLOR=always cargo llvm-cov report --fail-under-lines 90 --ignore-filename-regex '$IGNORE_REGEX'" "$text_out" >/dev/null 2>&1
|
||||
text_status="$?"
|
||||
|
||||
# 3) HTML report (no test rerun).
|
||||
script -q -e -c "CARGO_TERM_COLOR=always cargo llvm-cov report --html --ignore-filename-regex '$IGNORE_REGEX'" "$html_out" >/dev/null 2>&1
|
||||
html_status="$?"
|
||||
|
||||
set -e
|
||||
|
||||
strip_script_noise() {
|
||||
tr -d '\r' | sed '/^Script \(started\|done\) on /d'
|
||||
}
|
||||
|
||||
strip_ansi_for_parse() {
|
||||
awk '
|
||||
{
|
||||
line = $0
|
||||
gsub(/\033\[[0-9;]*[A-Za-z]/, "", line) # CSI escapes
|
||||
gsub(/\033\([A-Za-z]/, "", line) # charset escapes (e.g., ESC(B)
|
||||
gsub(/\r/, "", line)
|
||||
print line
|
||||
}
|
||||
'
|
||||
}
|
||||
|
||||
cat "$run_out" | strip_script_noise
|
||||
cat "$text_out" | strip_script_noise
|
||||
cat "$html_out" | strip_script_noise
|
||||
|
||||
cat "$run_out" | strip_ansi_for_parse | awk '
|
||||
BEGIN {
|
||||
passed=0; failed=0; ignored=0; measured=0; filtered=0;
|
||||
}
|
||||
/^test result: / {
|
||||
if (match($0, /([0-9]+) passed; ([0-9]+) failed; ([0-9]+) ignored; ([0-9]+) measured; ([0-9]+) filtered out;/, m)) {
|
||||
passed += m[1]; failed += m[2]; ignored += m[3]; measured += m[4]; filtered += m[5];
|
||||
}
|
||||
}
|
||||
END {
|
||||
executed = passed + failed;
|
||||
total = passed + failed + ignored + measured;
|
||||
printf("\nTEST SUMMARY (all suites): passed=%d failed=%d ignored=%d measured=%d filtered_out=%d executed=%d total=%d\n",
|
||||
passed, failed, ignored, measured, filtered, executed, total);
|
||||
}
|
||||
'
|
||||
|
||||
echo
|
||||
echo "HTML report: target/llvm-cov/html/index.html"
|
||||
|
||||
status="$text_status"
|
||||
if [ "$run_status" -ne 0 ]; then status="$run_status"; fi
|
||||
if [ "$html_status" -ne 0 ]; then status="$html_status"; fi
|
||||
exit "$status"
|
||||
195
scripts/manifest_perf_compare_m2.sh
Executable file
195
scripts/manifest_perf_compare_m2.sh
Executable file
@@ -0,0 +1,195 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# M2: Run per-sample decode+profile benchmark (Ours vs Routinator) on selected_der fixtures.
|
||||
#
|
||||
# Outputs:
|
||||
# - specs/develop/20260224/data/m2_manifest_decode_profile_compare.csv
|
||||
# - specs/develop/20260224/data/m2_raw.log
|
||||
#
|
||||
# Note: This script assumes Routinator benchmark repo exists at:
|
||||
# /home/yuyr/dev/rust_playground/routinator/benchmark
|
||||
#
|
||||
# It also assumes fixtures exist under:
|
||||
# rpki/tests/benchmark/selected_der/*.mft
|
||||
# routinator/benchmark/fixtures/selected_der/*.mft
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||
RPKI_DIR="$ROOT_DIR"
|
||||
OURS_BENCH_DIR="$RPKI_DIR/benchmark/ours_manifest_bench"
|
||||
|
||||
ROUT_BENCH_DIR="${ROUT_BENCH_DIR:-/home/yuyr/dev/rust_playground/routinator/benchmark}"
|
||||
ROUT_BIN="$ROUT_BENCH_DIR/target/release/routinator-manifest-benchmark"
|
||||
|
||||
DATE_TAG="${DATE_TAG:-20260224}"
|
||||
OUT_DIR="$RPKI_DIR/../specs/develop/${DATE_TAG}/data"
|
||||
OUT_CSV="${OUT_CSV:-$OUT_DIR/m2_manifest_decode_profile_compare.csv}"
|
||||
OUT_RAW="${OUT_RAW:-$OUT_DIR/m2_raw.log}"
|
||||
|
||||
REPEATS="${REPEATS:-3}"
|
||||
|
||||
# Iterations / warmups (kept moderate for interactive iteration).
|
||||
ITER_SMALL="${ITER_SMALL:-20000}"
|
||||
ITER_MEDIUM="${ITER_MEDIUM:-20000}"
|
||||
ITER_LARGE="${ITER_LARGE:-20000}"
|
||||
ITER_XLARGE="${ITER_XLARGE:-2000}"
|
||||
|
||||
WARM_SMALL="${WARM_SMALL:-2000}"
|
||||
WARM_MEDIUM="${WARM_MEDIUM:-2000}"
|
||||
WARM_LARGE="${WARM_LARGE:-2000}"
|
||||
WARM_XLARGE="${WARM_XLARGE:-200}"
|
||||
|
||||
SAMPLES=(
|
||||
small-01
|
||||
small-02
|
||||
medium-01
|
||||
medium-02
|
||||
large-01
|
||||
large-02
|
||||
xlarge-01
|
||||
xlarge-02
|
||||
)
|
||||
|
||||
mkdir -p "$OUT_DIR"
|
||||
: > "$OUT_RAW"
|
||||
|
||||
echo "sample,bucket,manifest_file_count,ours_avg_ns_per_op,ours_ops_per_s,rout_avg_ns_per_op,rout_ops_per_s,ratio_ours_over_rout,iterations,repeats,warmup" > "$OUT_CSV"
|
||||
|
||||
echo "[1/3] Build ours benchmark (release)..." | tee -a "$OUT_RAW"
|
||||
(cd "$OURS_BENCH_DIR" && cargo build --release -q)
|
||||
OURS_BIN="$OURS_BENCH_DIR/target/release/ours-manifest-bench"
|
||||
|
||||
echo "[2/3] Build routinator benchmark (release)..." | tee -a "$OUT_RAW"
|
||||
(cd "$ROUT_BENCH_DIR" && cargo build --release -q)
|
||||
|
||||
taskset_prefix=""
|
||||
if command -v taskset >/dev/null 2>&1; then
|
||||
if [[ -n "${TASKSET_CPU:-}" ]]; then
|
||||
taskset_prefix="taskset -c ${TASKSET_CPU}"
|
||||
fi
|
||||
fi
|
||||
|
||||
bucket_for() {
|
||||
local s="$1"
|
||||
case "$s" in
|
||||
small-*) echo "small" ;;
|
||||
medium-*) echo "medium" ;;
|
||||
large-*) echo "large" ;;
|
||||
xlarge-*) echo "xlarge" ;;
|
||||
*) echo "unknown" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
iters_for() {
|
||||
local b="$1"
|
||||
case "$b" in
|
||||
small) echo "$ITER_SMALL" ;;
|
||||
medium) echo "$ITER_MEDIUM" ;;
|
||||
large) echo "$ITER_LARGE" ;;
|
||||
xlarge) echo "$ITER_XLARGE" ;;
|
||||
*) echo "$ITER_MEDIUM" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
warm_for() {
|
||||
local b="$1"
|
||||
case "$b" in
|
||||
small) echo "$WARM_SMALL" ;;
|
||||
medium) echo "$WARM_MEDIUM" ;;
|
||||
large) echo "$WARM_LARGE" ;;
|
||||
xlarge) echo "$WARM_XLARGE" ;;
|
||||
*) echo "$WARM_MEDIUM" ;;
|
||||
esac
|
||||
}
|
||||
|
||||
run_ours() {
|
||||
local sample="$1"
|
||||
local iters="$2"
|
||||
local warm="$3"
|
||||
local ours_fixture="$RPKI_DIR/tests/benchmark/selected_der/${sample}.mft"
|
||||
if [[ ! -f "$ours_fixture" ]]; then
|
||||
echo "ours fixture not found: $ours_fixture" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "### ours $sample" >> "$OUT_RAW"
|
||||
local out
|
||||
out=$($taskset_prefix "$OURS_BIN" --manifest "$ours_fixture" --iterations "$iters" --warmup-iterations "$warm" --repeats "$REPEATS")
|
||||
echo "$out" >> "$OUT_RAW"
|
||||
|
||||
local line
|
||||
line=$(echo "$out" | rg "^\\| ${sample} \\|" | tail -n 1)
|
||||
if [[ -z "${line:-}" ]]; then
|
||||
echo "failed to parse ours output for $sample" >&2
|
||||
exit 1
|
||||
fi
|
||||
# Expected final row: | sample | avg ns/op | ops/s | file count |
|
||||
local avg ops cnt
|
||||
avg=$(echo "$line" | awk -F'|' '{gsub(/^ +| +$/,"",$3); print $3}')
|
||||
ops=$(echo "$line" | awk -F'|' '{gsub(/^ +| +$/,"",$4); print $4}')
|
||||
cnt=$(echo "$line" | awk -F'|' '{gsub(/^ +| +$/,"",$5); print $5}')
|
||||
echo "$avg,$ops,$cnt"
|
||||
}
|
||||
|
||||
run_rout() {
|
||||
local sample="$1"
|
||||
local iters="$2"
|
||||
local warm="$3"
|
||||
local rout_fixture="$ROUT_BENCH_DIR/fixtures/selected_der/${sample}.mft"
|
||||
if [[ ! -f "$rout_fixture" ]]; then
|
||||
echo "routinator fixture not found: $rout_fixture" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
echo "### routinator $sample" >> "$OUT_RAW"
|
||||
local out
|
||||
out=$(
|
||||
$taskset_prefix "$ROUT_BIN" \
|
||||
--target decode_only \
|
||||
--manifest "$rout_fixture" \
|
||||
--issuer "$ROUT_BENCH_DIR/fixtures/ta.cer" \
|
||||
--iterations "$iters" \
|
||||
--repeats "$REPEATS" \
|
||||
--warmup-iterations "$warm" \
|
||||
--strict false
|
||||
)
|
||||
echo "$out" >> "$OUT_RAW"
|
||||
|
||||
local avg_line cnt_line
|
||||
avg_line=$(echo "$out" | rg "^ avg:")
|
||||
cnt_line=$(echo "$out" | rg "^ manifest_file_count:")
|
||||
|
||||
local avg_ns ops_s cnt
|
||||
avg_ns=$(echo "$avg_line" | awk '{print $2}')
|
||||
ops_s=$(echo "$avg_line" | awk '{gsub(/[()]/,"",$4); print $4}')
|
||||
cnt=$(echo "$cnt_line" | awk '{print $2}')
|
||||
echo "$avg_ns,$ops_s,$cnt"
|
||||
}
|
||||
|
||||
echo "[3/3] Run per-sample benchmarks..." | tee -a "$OUT_RAW"
|
||||
for s in "${SAMPLES[@]}"; do
|
||||
b=$(bucket_for "$s")
|
||||
it=$(iters_for "$b")
|
||||
warm=$(warm_for "$b")
|
||||
|
||||
IFS=, read -r ours_avg ours_ops ours_cnt < <(run_ours "$s" "$it" "$warm")
|
||||
IFS=, read -r rout_avg rout_ops rout_cnt < <(run_rout "$s" "$it" "$warm")
|
||||
|
||||
if [[ "$ours_cnt" != "$rout_cnt" ]]; then
|
||||
echo "WARNING: file count differs for $s (ours=$ours_cnt rout=$rout_cnt)" | tee -a "$OUT_RAW"
|
||||
fi
|
||||
|
||||
ratio=$(python3 - <<PY
|
||||
o=float("$ours_avg")
|
||||
r=float("$rout_avg")
|
||||
print(f"{(o/r):.4f}" if r != 0 else "inf")
|
||||
PY
|
||||
)
|
||||
|
||||
echo "$s,$b,$ours_cnt,$ours_avg,$ours_ops,$rout_avg,$rout_ops,$ratio,$it,$REPEATS,$warm" >> "$OUT_CSV"
|
||||
echo >> "$OUT_RAW"
|
||||
done
|
||||
|
||||
echo "Done."
|
||||
echo "- CSV: $OUT_CSV"
|
||||
echo "- Raw: $OUT_RAW"
|
||||
142
scripts/manifest_perf_profile_m3.sh
Executable file
142
scripts/manifest_perf_profile_m3.sh
Executable file
@@ -0,0 +1,142 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# M3: Generate flamegraphs + top hotspots for Manifest decode+profile (Ours vs Routinator).
|
||||
#
|
||||
# Outputs under:
|
||||
# specs/develop/20260224/flamegraph/
|
||||
# specs/develop/20260224/hotspots/
|
||||
# specs/develop/20260224/perf/
|
||||
#
|
||||
# Notes:
|
||||
# - On WSL2, /usr/bin/perf is often a wrapper that fails. This script uses a real perf binary
|
||||
# from /usr/lib/linux-tools/*/perf (if present).
|
||||
# - Ours profiling uses perf + flamegraph --perfdata to avoid rebuilding the whole crate graph
|
||||
# with RocksDB.
|
||||
|
||||
ROOT_REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
RPKI_DIR="$ROOT_REPO/rpki"
|
||||
|
||||
DATE_TAG="${DATE_TAG:-20260224}"
|
||||
OUT_BASE="$ROOT_REPO/specs/develop/${DATE_TAG}"
|
||||
OUT_FLAME="$OUT_BASE/flamegraph"
|
||||
OUT_HOT="$OUT_BASE/hotspots"
|
||||
OUT_PERF="$OUT_BASE/perf"
|
||||
|
||||
RUN_TAG="${RUN_TAG:-p2}"
|
||||
|
||||
OURS_BENCH_DIR="$RPKI_DIR/benchmark/ours_manifest_bench"
|
||||
OURS_BIN="$OURS_BENCH_DIR/target/release/ours-manifest-bench"
|
||||
|
||||
ROUT_BENCH_DIR="${ROUT_BENCH_DIR:-/home/yuyr/dev/rust_playground/routinator/benchmark}"
|
||||
ROUT_BIN="$ROUT_BENCH_DIR/target/release/routinator-manifest-benchmark"
|
||||
ROUT_ISSUER="$ROUT_BENCH_DIR/fixtures/ta.cer"
|
||||
|
||||
PROFILE_HZ="${PROFILE_HZ:-99}"
|
||||
|
||||
mkdir -p "$OUT_FLAME" "$OUT_HOT" "$OUT_PERF"
|
||||
|
||||
PERF_WRAPPER_OUT="$(perf --version 2>&1 || true)"
|
||||
PERF_REAL=""
|
||||
if echo "${PERF_WRAPPER_OUT}" | grep -q "WARNING: perf not found for kernel"; then
|
||||
PERF_REAL="$(ls -1 /usr/lib/linux-tools/*/perf 2>/dev/null | head -n 1 || true)"
|
||||
else
|
||||
PERF_REAL="$(command -v perf || true)"
|
||||
fi
|
||||
|
||||
if [[ -z "${PERF_REAL}" ]]; then
|
||||
echo "ERROR: usable perf binary not found (wrapper detected and no /usr/lib/linux-tools/*/perf)." >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
SHIM_DIR="$RPKI_DIR/target/bench/tools"
|
||||
mkdir -p "$SHIM_DIR"
|
||||
cat > "$SHIM_DIR/perf" <<EOF
|
||||
#!/usr/bin/env bash
|
||||
exec "${PERF_REAL}" "\$@"
|
||||
EOF
|
||||
chmod +x "$SHIM_DIR/perf"
|
||||
export PATH="$SHIM_DIR:$PATH"
|
||||
|
||||
echo "Using perf: $PERF_REAL"
|
||||
|
||||
echo "[1/3] Build ours benchmark with frame pointers..."
|
||||
(cd "$OURS_BENCH_DIR" && RUSTFLAGS="-C force-frame-pointers=yes" cargo build --release -q)
|
||||
|
||||
echo "[2/3] Build routinator benchmark (release)..."
|
||||
(cd "$ROUT_BENCH_DIR" && cargo build --release -q)
|
||||
|
||||
taskset_prefix=""
|
||||
if command -v taskset >/dev/null 2>&1; then
|
||||
taskset_prefix="taskset -c 0"
|
||||
fi
|
||||
|
||||
profile_ours() {
|
||||
local sample="$1"
|
||||
local iters="$2"
|
||||
local warm="$3"
|
||||
local fixture="$RPKI_DIR/tests/benchmark/selected_der/${sample}.mft"
|
||||
if [[ ! -f "$fixture" ]]; then
|
||||
echo "ERROR: ours fixture not found: $fixture" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
local perfdata="$OUT_PERF/ours_${sample}_${RUN_TAG}.perf.data"
|
||||
local svg="$OUT_FLAME/ours_${sample}_${RUN_TAG}.svg"
|
||||
local tsv="$OUT_HOT/ours_${sample}_${RUN_TAG}.tsv"
|
||||
|
||||
echo "== ours $sample (iters=$iters warmup=$warm hz=$PROFILE_HZ)"
|
||||
$taskset_prefix perf record -o "$perfdata" -F "$PROFILE_HZ" -g -- \
|
||||
"$OURS_BIN" --manifest "$fixture" --iterations "$iters" --warmup-iterations "$warm" --repeats 1 >/dev/null
|
||||
|
||||
flamegraph --perfdata "$perfdata" --output "$svg" --title "ours ${sample} ManifestObject::decode_der" --deterministic >/dev/null
|
||||
|
||||
perf report -i "$perfdata" --stdio --no-children --sort symbol --percent-limit 0.5 \
|
||||
| awk '/^[[:space:]]*[0-9.]+%/ {pct=$1; sub(/%/,"",pct); $1=""; sub(/^[[:space:]]+/,""); print pct "\t" $0}' \
|
||||
> "$tsv"
|
||||
}
|
||||
|
||||
profile_routinator() {
|
||||
local sample="$1"
|
||||
local iters="$2"
|
||||
local warm="$3"
|
||||
local fixture="$ROUT_BENCH_DIR/fixtures/selected_der/${sample}.mft"
|
||||
if [[ ! -f "$fixture" ]]; then
|
||||
echo "ERROR: routinator fixture not found: $fixture" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
local svg="$OUT_FLAME/routinator_${sample}_${RUN_TAG}.svg"
|
||||
local tsv="$OUT_HOT/routinator_${sample}_${RUN_TAG}.tsv"
|
||||
|
||||
echo "== routinator $sample (iters=$iters warmup=$warm hz=$PROFILE_HZ)"
|
||||
$taskset_prefix "$ROUT_BIN" \
|
||||
--target decode_only \
|
||||
--manifest "$fixture" \
|
||||
--issuer "$ROUT_ISSUER" \
|
||||
--iterations "$iters" \
|
||||
--repeats 1 \
|
||||
--warmup-iterations "$warm" \
|
||||
--strict false \
|
||||
--profile-hz "$PROFILE_HZ" \
|
||||
--flamegraph "$svg" \
|
||||
--hotspots "$tsv" \
|
||||
>/dev/null
|
||||
}
|
||||
|
||||
echo "[3/3] Profile samples..."
|
||||
|
||||
# Choose iterations so each capture runs ~10-20s serially.
|
||||
profile_ours small-01 200000 0
|
||||
profile_routinator small-01 200000 0
|
||||
|
||||
profile_ours large-02 50000 0
|
||||
profile_routinator large-02 50000 0
|
||||
|
||||
profile_ours xlarge-02 5000 0
|
||||
profile_routinator xlarge-02 5000 0
|
||||
|
||||
echo "Done."
|
||||
echo "- Flamegraphs: $OUT_FLAME/"
|
||||
echo "- Hotspots: $OUT_HOT/"
|
||||
echo "- Perf data: $OUT_PERF/"
|
||||
97
scripts/manual_sync/README.md
Normal file
97
scripts/manual_sync/README.md
Normal file
@@ -0,0 +1,97 @@
|
||||
# Manual RRDP sync (APNIC-focused)
|
||||
|
||||
This directory contains **manual, command-line** scripts to reproduce the workflow described in:
|
||||
|
||||
- `specs/develop/20260226/apnic_rrdp_delta_analysis_after_manifest_revalidation_fix_20260227T022606Z.md`
|
||||
|
||||
They are meant for **hands-on validation / acceptance runs**, not for CI.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Rust toolchain (`cargo`)
|
||||
- `rsync` available on PATH (for rsync fallback/objects)
|
||||
- Network access (RRDP over HTTPS)
|
||||
|
||||
## What the scripts do
|
||||
|
||||
### `full_sync.sh`
|
||||
|
||||
- Creates a fresh RocksDB directory
|
||||
- Runs a **full serial** validation from a TAL URL (default: APNIC RFC7730 TAL)
|
||||
- Writes:
|
||||
- run log
|
||||
- audit report JSON
|
||||
- run meta JSON (includes durations + download_stats)
|
||||
- short summary Markdown (includes durations + download_stats)
|
||||
- RocksDB key statistics (`db_stats --exact`)
|
||||
- RRDP legacy session/serial dump (`rrdp_state_dump --view legacy-state`)
|
||||
|
||||
### `delta_sync.sh`
|
||||
|
||||
- Copies an existing “baseline snapshot DB” to a new DB directory (so the baseline is not modified)
|
||||
- Runs another validation against the copied DB (RRDP will prefer **delta** when available)
|
||||
- Produces the same artifacts as `full_sync.sh`
|
||||
- Additionally generates a Markdown **delta analysis** report by comparing:
|
||||
- base vs delta report JSON
|
||||
- base vs delta `rrdp_state_dump --view legacy-state` TSV
|
||||
- and includes a **duration comparison** (base vs delta) if the base meta JSON is available
|
||||
- delta meta JSON includes download_stats copied from delta report JSON
|
||||
|
||||
## Audit report fields (report.json)
|
||||
|
||||
The `rpki` binary writes an audit report JSON with:
|
||||
|
||||
- `format_version: 2`
|
||||
- `downloads`: per-download RRDP/rsync events (URI, timestamps, duration, ok/fail, error, bytes, objects stats)
|
||||
- `download_stats`: aggregate counters (by kind)
|
||||
|
||||
These are useful for diagnosing why a run is slow (e.g. RRDP snapshot vs delta vs rsync fallback).
|
||||
|
||||
The standalone `rrdp_state_dump` tool also supports `source`, `members`, `owners`, and `all` views.
|
||||
The manual sync scripts intentionally call `--view legacy-state` so delta analysis keeps using a stable session/serial TSV format.
|
||||
|
||||
## Meta fields (meta.json)
|
||||
|
||||
The scripts generate `*_meta.json` next to `*_report.json` and include:
|
||||
|
||||
- `durations_secs`: wall-clock duration breakdown for the script steps
|
||||
- `download_stats`: copied from `report_json.download_stats`
|
||||
|
||||
## Usage
|
||||
|
||||
Run from `rpki/`:
|
||||
|
||||
```bash
|
||||
./scripts/manual_sync/full_sync.sh
|
||||
```
|
||||
|
||||
After you have a baseline run, run delta against it:
|
||||
|
||||
```bash
|
||||
./scripts/manual_sync/delta_sync.sh target/live/manual_sync/apnic_full_db_YYYYMMDDTHHMMSSZ \
|
||||
target/live/manual_sync/apnic_full_report_YYYYMMDDTHHMMSSZ.json
|
||||
```
|
||||
|
||||
If the baseline was produced by `full_sync.sh`, the delta script will auto-discover the base meta JSON
|
||||
next to the base report (by replacing `_report.json` with `_meta.json`) and include base durations in
|
||||
the delta analysis report.
|
||||
|
||||
## Configuration (env vars)
|
||||
|
||||
Both scripts accept overrides via env vars:
|
||||
|
||||
- `TAL_URL` (default: APNIC TAL URL)
|
||||
- `HTTP_TIMEOUT_SECS` (default: 1800)
|
||||
- `RSYNC_TIMEOUT_SECS` (default: 1800)
|
||||
- `RSYNC_MIRROR_ROOT` (default: disabled; when set, passes `--rsync-mirror-root` to `rpki`)
|
||||
- `VALIDATION_TIME` (RFC3339; default: now UTC)
|
||||
- `OUT_DIR` (default: `rpki/target/live/manual_sync`)
|
||||
- `RUN_NAME` (default: auto timestamped)
|
||||
|
||||
Example:
|
||||
|
||||
```bash
|
||||
TAL_URL="https://tal.apnic.net/tal-archive/apnic-rfc7730-https.tal" \
|
||||
HTTP_TIMEOUT_SECS=1800 RSYNC_TIMEOUT_SECS=1800 \
|
||||
./scripts/manual_sync/full_sync.sh
|
||||
```
|
||||
543
scripts/manual_sync/delta_sync.sh
Executable file
543
scripts/manual_sync/delta_sync.sh
Executable file
@@ -0,0 +1,543 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Delta sync + validation starting from a baseline snapshot DB.
|
||||
#
|
||||
# This script:
|
||||
# 1) Copies BASE_DB_DIR -> DELTA_DB_DIR (so baseline is not modified)
|
||||
# 2) Runs rpki validation again (RRDP will prefer delta if available)
|
||||
# 3) Writes artifacts + a markdown delta analysis report
|
||||
#
|
||||
# Usage:
|
||||
# ./scripts/manual_sync/delta_sync.sh <base_db_dir> <base_report_json>
|
||||
#
|
||||
# Outputs under OUT_DIR (default: target/live/manual_sync):
|
||||
# - *_delta_db_* copied RocksDB directory
|
||||
# - *_delta_report_*.json audit report
|
||||
# - *_delta_run_*.log stdout/stderr log (includes summary)
|
||||
# - *_delta_db_stats_*.txt db_stats --exact output
|
||||
# - *_delta_rrdp_state_*.tsv rrdp_state_dump --view legacy-state output
|
||||
# - *_delta_analysis_*.md base vs delta comparison report
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
cd "$ROOT_DIR"
|
||||
|
||||
BASE_DB_DIR="${1:-}"
|
||||
BASE_REPORT_JSON="${2:-}"
|
||||
if [[ -z "${BASE_DB_DIR}" || -z "${BASE_REPORT_JSON}" ]]; then
|
||||
echo "Usage: $0 <base_db_dir> <base_report_json>" >&2
|
||||
exit 2
|
||||
fi
|
||||
if [[ ! -d "${BASE_DB_DIR}" ]]; then
|
||||
echo "ERROR: base_db_dir is not a directory: ${BASE_DB_DIR}" >&2
|
||||
exit 2
|
||||
fi
|
||||
if [[ ! -f "${BASE_REPORT_JSON}" ]]; then
|
||||
echo "ERROR: base_report_json does not exist: ${BASE_REPORT_JSON}" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
TAL_URL="${TAL_URL:-https://tal.apnic.net/tal-archive/apnic-rfc7730-https.tal}"
|
||||
HTTP_TIMEOUT_SECS="${HTTP_TIMEOUT_SECS:-1800}"
|
||||
RSYNC_TIMEOUT_SECS="${RSYNC_TIMEOUT_SECS:-1800}"
|
||||
RSYNC_MIRROR_ROOT="${RSYNC_MIRROR_ROOT:-}"
|
||||
VALIDATION_TIME="${VALIDATION_TIME:-}"
|
||||
|
||||
OUT_DIR="${OUT_DIR:-$ROOT_DIR/target/live/manual_sync}"
|
||||
mkdir -p "$OUT_DIR"
|
||||
|
||||
TS="$(date -u +%Y%m%dT%H%M%SZ)"
|
||||
RUN_NAME="${RUN_NAME:-apnic_delta_${TS}}"
|
||||
|
||||
DELTA_DB_DIR="${DELTA_DB_DIR:-$OUT_DIR/${RUN_NAME}_db}"
|
||||
DELTA_REPORT_JSON="${DELTA_REPORT_JSON:-$OUT_DIR/${RUN_NAME}_report.json}"
|
||||
DELTA_RUN_LOG="${DELTA_RUN_LOG:-$OUT_DIR/${RUN_NAME}_run.log}"
|
||||
|
||||
BASE_DB_STATS_TXT="${BASE_DB_STATS_TXT:-$OUT_DIR/${RUN_NAME}_base_db_stats.txt}"
|
||||
DELTA_DB_STATS_TXT="${DELTA_DB_STATS_TXT:-$OUT_DIR/${RUN_NAME}_delta_db_stats.txt}"
|
||||
|
||||
BASE_RRDP_STATE_TSV="${BASE_RRDP_STATE_TSV:-$OUT_DIR/${RUN_NAME}_base_rrdp_state.tsv}"
|
||||
DELTA_RRDP_STATE_TSV="${DELTA_RRDP_STATE_TSV:-$OUT_DIR/${RUN_NAME}_delta_rrdp_state.tsv}"
|
||||
|
||||
DELTA_ANALYSIS_MD="${DELTA_ANALYSIS_MD:-$OUT_DIR/${RUN_NAME}_delta_analysis.md}"
|
||||
DELTA_META_JSON="${DELTA_META_JSON:-$OUT_DIR/${RUN_NAME}_meta.json}"
|
||||
|
||||
# Best-effort base meta discovery (produced by `full_sync.sh`).
|
||||
BASE_META_JSON="${BASE_META_JSON:-}"
|
||||
if [[ -z "${BASE_META_JSON}" ]]; then
|
||||
guess="${BASE_REPORT_JSON%_report.json}_meta.json"
|
||||
if [[ -f "${guess}" ]]; then
|
||||
BASE_META_JSON="${guess}"
|
||||
fi
|
||||
fi
|
||||
|
||||
echo "== rpki manual delta sync ==" >&2
|
||||
echo "tal_url=$TAL_URL" >&2
|
||||
echo "base_db=$BASE_DB_DIR" >&2
|
||||
echo "base_report=$BASE_REPORT_JSON" >&2
|
||||
echo "delta_db=$DELTA_DB_DIR" >&2
|
||||
echo "delta_report=$DELTA_REPORT_JSON" >&2
|
||||
|
||||
echo "== copying base DB (baseline is not modified) ==" >&2
|
||||
cp -a "$BASE_DB_DIR" "$DELTA_DB_DIR"
|
||||
|
||||
script_start_s="$(date +%s)"
|
||||
run_start_s="$(date +%s)"
|
||||
cmd=(cargo run --release --bin rpki -- \
|
||||
--db "$DELTA_DB_DIR" \
|
||||
--tal-url "$TAL_URL" \
|
||||
--http-timeout-secs "$HTTP_TIMEOUT_SECS" \
|
||||
--rsync-timeout-secs "$RSYNC_TIMEOUT_SECS" \
|
||||
--report-json "$DELTA_REPORT_JSON")
|
||||
if [[ -n "${RSYNC_MIRROR_ROOT}" ]]; then
|
||||
cmd+=(--rsync-mirror-root "$RSYNC_MIRROR_ROOT")
|
||||
fi
|
||||
if [[ -n "${VALIDATION_TIME}" ]]; then
|
||||
cmd+=(--validation-time "$VALIDATION_TIME")
|
||||
fi
|
||||
|
||||
(
|
||||
echo "# command:"
|
||||
printf '%q ' "${cmd[@]}"
|
||||
echo
|
||||
echo
|
||||
"${cmd[@]}"
|
||||
) 2>&1 | tee "$DELTA_RUN_LOG" >/dev/null
|
||||
run_end_s="$(date +%s)"
|
||||
run_duration_s="$((run_end_s - run_start_s))"
|
||||
|
||||
echo "== db_stats (exact) ==" >&2
|
||||
db_stats_start_s="$(date +%s)"
|
||||
cargo run --release --bin db_stats -- --db "$BASE_DB_DIR" --exact 2>&1 | tee "$BASE_DB_STATS_TXT" >/dev/null
|
||||
cargo run --release --bin db_stats -- --db "$DELTA_DB_DIR" --exact 2>&1 | tee "$DELTA_DB_STATS_TXT" >/dev/null
|
||||
db_stats_end_s="$(date +%s)"
|
||||
db_stats_duration_s="$((db_stats_end_s - db_stats_start_s))"
|
||||
|
||||
echo "== rrdp_state_dump (legacy-state) ==" >&2
|
||||
state_start_s="$(date +%s)"
|
||||
cargo run --release --bin rrdp_state_dump -- --db "$BASE_DB_DIR" --view legacy-state >"$BASE_RRDP_STATE_TSV"
|
||||
cargo run --release --bin rrdp_state_dump -- --db "$DELTA_DB_DIR" --view legacy-state >"$DELTA_RRDP_STATE_TSV"
|
||||
state_end_s="$(date +%s)"
|
||||
state_duration_s="$((state_end_s - state_start_s))"
|
||||
|
||||
script_end_s="$(date +%s)"
|
||||
total_duration_s="$((script_end_s - script_start_s))"
|
||||
|
||||
echo "== delta analysis report ==" >&2
|
||||
TAL_URL="$TAL_URL" \
|
||||
BASE_DB_DIR="$BASE_DB_DIR" \
|
||||
DELTA_DB_DIR="$DELTA_DB_DIR" \
|
||||
DELTA_RUN_LOG="$DELTA_RUN_LOG" \
|
||||
VALIDATION_TIME_ARG="$VALIDATION_TIME" \
|
||||
HTTP_TIMEOUT_SECS="$HTTP_TIMEOUT_SECS" \
|
||||
RSYNC_TIMEOUT_SECS="$RSYNC_TIMEOUT_SECS" \
|
||||
RUN_DURATION_S="$run_duration_s" \
|
||||
DB_STATS_DURATION_S="$db_stats_duration_s" \
|
||||
STATE_DURATION_S="$state_duration_s" \
|
||||
TOTAL_DURATION_S="$total_duration_s" \
|
||||
python3 - "$BASE_REPORT_JSON" "$DELTA_REPORT_JSON" "$BASE_RRDP_STATE_TSV" "$DELTA_RRDP_STATE_TSV" \
|
||||
"$BASE_DB_STATS_TXT" "$DELTA_DB_STATS_TXT" "$BASE_META_JSON" "$DELTA_META_JSON" "$DELTA_ANALYSIS_MD" <<'PY'
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from collections import Counter, defaultdict
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
base_report_path = Path(sys.argv[1])
|
||||
delta_report_path = Path(sys.argv[2])
|
||||
base_state_path = Path(sys.argv[3])
|
||||
delta_state_path = Path(sys.argv[4])
|
||||
base_db_stats_path = Path(sys.argv[5])
|
||||
delta_db_stats_path = Path(sys.argv[6])
|
||||
base_meta_path_s = sys.argv[7]
|
||||
delta_meta_path = Path(sys.argv[8])
|
||||
out_md_path = Path(sys.argv[9])
|
||||
|
||||
def load_json(p: Path):
|
||||
s = p.read_text(encoding="utf-8")
|
||||
try:
|
||||
return json.loads(s)
|
||||
except json.JSONDecodeError:
|
||||
# Backwards-compat / robustness: tolerate accidental literal trailing "\\n".
|
||||
s2 = s.strip()
|
||||
if s2.endswith("\\n"):
|
||||
s2 = s2[:-2].rstrip()
|
||||
return json.loads(s2)
|
||||
|
||||
def load_optional_json(path_s: str):
|
||||
if not path_s:
|
||||
return None
|
||||
p = Path(path_s)
|
||||
if not p.exists():
|
||||
return None
|
||||
return load_json(p)
|
||||
|
||||
def parse_rrdp_state_tsv(p: Path):
|
||||
# format from `rrdp_state_dump --view legacy-state`:
|
||||
# [legacy-state]
|
||||
# notify_uri serial session_id
|
||||
# <notify_uri> <serial> <session_id>
|
||||
out = {}
|
||||
for line in p.read_text(encoding="utf-8").splitlines():
|
||||
line = line.strip()
|
||||
if not line or line.startswith("["):
|
||||
continue
|
||||
if line == "notify_uri serial session_id":
|
||||
continue
|
||||
parts = line.split(" ")
|
||||
if len(parts) != 3:
|
||||
raise SystemExit(f"invalid rrdp_state_dump line in {p}: {line!r}")
|
||||
uri, serial, session = parts
|
||||
out[uri] = (int(serial), session)
|
||||
return out
|
||||
|
||||
def parse_db_stats(p: Path):
|
||||
# lines: key=value
|
||||
out = {}
|
||||
for line in p.read_text(encoding="utf-8").splitlines():
|
||||
if "=" not in line:
|
||||
continue
|
||||
k, v = line.split("=", 1)
|
||||
k = k.strip()
|
||||
v = v.strip()
|
||||
if v.isdigit():
|
||||
out[k] = int(v)
|
||||
else:
|
||||
out[k] = v
|
||||
return out
|
||||
|
||||
def warnings_total(rep: dict) -> int:
|
||||
return len(rep["tree"]["warnings"]) + sum(len(pp["warnings"]) for pp in rep["publication_points"])
|
||||
|
||||
def report_summary(rep: dict) -> dict:
|
||||
return {
|
||||
"validation_time": rep["meta"]["validation_time_rfc3339_utc"],
|
||||
"publication_points_processed": rep["tree"]["instances_processed"],
|
||||
"publication_points_failed": rep["tree"]["instances_failed"],
|
||||
"rrdp_repos_unique": len({pp.get("rrdp_notification_uri") for pp in rep["publication_points"] if pp.get("rrdp_notification_uri")}),
|
||||
"vrps": len(rep["vrps"]),
|
||||
"aspas": len(rep["aspas"]),
|
||||
"audit_publication_points": len(rep["publication_points"]),
|
||||
"warnings_total": warnings_total(rep),
|
||||
}
|
||||
|
||||
def count_repo_sync_failed(rep: dict) -> int:
|
||||
# Best-effort heuristic (we don't currently expose a structured counter in the audit report).
|
||||
# Keep the match conservative to avoid false positives.
|
||||
def is_repo_sync_failed(msg: str) -> bool:
|
||||
m = msg.lower()
|
||||
return "repo sync failed" in m or "rrdp fetch failed" in m or "rsync fetch failed" in m
|
||||
|
||||
n = 0
|
||||
for w in rep["tree"]["warnings"]:
|
||||
if is_repo_sync_failed(w.get("message", "")):
|
||||
n += 1
|
||||
for pp in rep["publication_points"]:
|
||||
for w in pp.get("warnings", []):
|
||||
if is_repo_sync_failed(w.get("message", "")):
|
||||
n += 1
|
||||
return n
|
||||
|
||||
def pp_manifest_sha(pp: dict) -> str:
|
||||
# In our audit format, the first object is the manifest (synthetic entry) with sha256 of manifest_bytes.
|
||||
for o in pp["objects"]:
|
||||
if o["kind"] == "manifest":
|
||||
return o["sha256_hex"]
|
||||
return ""
|
||||
|
||||
def pp_objects_by_uri(rep: dict):
|
||||
m = {}
|
||||
for pp in rep["publication_points"]:
|
||||
for o in pp["objects"]:
|
||||
m[o["rsync_uri"]] = (o["sha256_hex"], o["kind"])
|
||||
return m
|
||||
|
||||
def vrp_set(rep: dict):
|
||||
return {(v["asn"], v["prefix"], v["max_length"]) for v in rep["vrps"]}
|
||||
|
||||
def rfc_refs_str(w: dict) -> str:
|
||||
refs = w.get("rfc_refs") or []
|
||||
return ", ".join(refs) if refs else ""
|
||||
|
||||
base = load_json(base_report_path)
|
||||
delta = load_json(delta_report_path)
|
||||
base_sum = report_summary(base)
|
||||
delta_sum = report_summary(delta)
|
||||
|
||||
base_db = parse_db_stats(base_db_stats_path)
|
||||
delta_db = parse_db_stats(delta_db_stats_path)
|
||||
|
||||
base_state = parse_rrdp_state_tsv(base_state_path)
|
||||
delta_state = parse_rrdp_state_tsv(delta_state_path)
|
||||
|
||||
base_meta = load_optional_json(base_meta_path_s)
|
||||
download_stats = delta.get("download_stats") or {}
|
||||
delta_meta = {
|
||||
"recorded_at_utc": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
|
||||
"tal_url": os.environ["TAL_URL"],
|
||||
"base_db_dir": os.environ["BASE_DB_DIR"],
|
||||
"delta_db_dir": os.environ["DELTA_DB_DIR"],
|
||||
"base_report_json": str(base_report_path),
|
||||
"delta_report_json": str(delta_report_path),
|
||||
"delta_run_log": os.environ["DELTA_RUN_LOG"],
|
||||
"validation_time_arg": os.environ.get("VALIDATION_TIME_ARG",""),
|
||||
"http_timeout_secs": int(os.environ["HTTP_TIMEOUT_SECS"]),
|
||||
"rsync_timeout_secs": int(os.environ["RSYNC_TIMEOUT_SECS"]),
|
||||
"durations_secs": {
|
||||
"rpki_run": int(os.environ["RUN_DURATION_S"]),
|
||||
"db_stats_exact": int(os.environ["DB_STATS_DURATION_S"]),
|
||||
"rrdp_state_dump": int(os.environ["STATE_DURATION_S"]),
|
||||
"total_script": int(os.environ["TOTAL_DURATION_S"]),
|
||||
},
|
||||
"download_stats": download_stats,
|
||||
}
|
||||
delta_meta_path.write_text(json.dumps(delta_meta, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
|
||||
now = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
|
||||
# RRDP state changes
|
||||
serial_changed = 0
|
||||
session_changed = 0
|
||||
serial_deltas = []
|
||||
for uri, (old_serial, old_sess) in base_state.items():
|
||||
if uri not in delta_state:
|
||||
continue
|
||||
new_serial, new_sess = delta_state[uri]
|
||||
if new_serial != old_serial:
|
||||
serial_changed += 1
|
||||
serial_deltas.append((uri, old_serial, new_serial, new_serial - old_serial))
|
||||
if new_sess != old_sess:
|
||||
session_changed += 1
|
||||
|
||||
serial_deltas.sort(key=lambda x: x[3], reverse=True)
|
||||
|
||||
# Publication point diffs
|
||||
base_pp = {pp["manifest_rsync_uri"]: pp for pp in base["publication_points"]}
|
||||
delta_pp = {pp["manifest_rsync_uri"]: pp for pp in delta["publication_points"]}
|
||||
|
||||
base_keys = set(base_pp.keys())
|
||||
delta_keys = set(delta_pp.keys())
|
||||
|
||||
new_pp = sorted(delta_keys - base_keys)
|
||||
missing_pp = sorted(base_keys - delta_keys)
|
||||
|
||||
updated_pp = 0
|
||||
unchanged_pp = 0
|
||||
for k in sorted(base_keys & delta_keys):
|
||||
if pp_manifest_sha(base_pp[k]) != pp_manifest_sha(delta_pp[k]):
|
||||
updated_pp += 1
|
||||
else:
|
||||
unchanged_pp += 1
|
||||
|
||||
# Cache usage + repo sync failure hints
|
||||
def source_counts(rep: dict) -> Counter:
|
||||
c = Counter()
|
||||
for pp in rep["publication_points"]:
|
||||
c[pp.get("source","")] += 1
|
||||
return c
|
||||
|
||||
base_sources = source_counts(base)
|
||||
delta_sources = source_counts(delta)
|
||||
base_repo_sync_failed = count_repo_sync_failed(base)
|
||||
delta_repo_sync_failed = count_repo_sync_failed(delta)
|
||||
|
||||
def cache_reason_counts(rep: dict) -> Counter:
|
||||
c = Counter()
|
||||
for pp in rep.get("publication_points", []):
|
||||
if pp.get("source") != "vcir_current_instance":
|
||||
continue
|
||||
# Use warning messages as "reason". If missing, emit a fallback bucket.
|
||||
ws = pp.get("warnings", [])
|
||||
if not ws:
|
||||
c["(no warnings recorded)"] += 1
|
||||
continue
|
||||
for w in ws:
|
||||
msg = w.get("message", "").strip() or "(empty warning message)"
|
||||
c[msg] += 1
|
||||
return c
|
||||
|
||||
base_cache_reasons = cache_reason_counts(base)
|
||||
delta_cache_reasons = cache_reason_counts(delta)
|
||||
|
||||
# Object change stats (by rsync URI, sha256)
|
||||
base_obj = pp_objects_by_uri(base)
|
||||
delta_obj = pp_objects_by_uri(delta)
|
||||
|
||||
kind_stats = {k: {"added": 0, "changed": 0, "removed": 0} for k in ["manifest","crl","certificate","roa","aspa","other"]}
|
||||
all_uris = set(base_obj.keys()) | set(delta_obj.keys())
|
||||
for uri in all_uris:
|
||||
b = base_obj.get(uri)
|
||||
d = delta_obj.get(uri)
|
||||
if b is None and d is not None:
|
||||
kind_stats[d[1]]["added"] += 1
|
||||
elif b is not None and d is None:
|
||||
kind_stats[b[1]]["removed"] += 1
|
||||
else:
|
||||
if b[0] != d[0]:
|
||||
kind_stats[d[1]]["changed"] += 1
|
||||
|
||||
# VRP diff
|
||||
base_v = vrp_set(base)
|
||||
delta_v = vrp_set(delta)
|
||||
added_v = delta_v - base_v
|
||||
removed_v = base_v - delta_v
|
||||
|
||||
def fmt_db_stats(db: dict) -> str:
|
||||
ordered = [
|
||||
"mode",
|
||||
"repository_view",
|
||||
"raw_by_hash",
|
||||
"vcir",
|
||||
"audit_rule_index",
|
||||
"rrdp_source",
|
||||
"rrdp_source_member",
|
||||
"rrdp_uri_owner",
|
||||
"rrdp_state",
|
||||
"raw_objects",
|
||||
"rrdp_object_index",
|
||||
"group_current_repository_view",
|
||||
"group_current_validation_state",
|
||||
"group_current_rrdp_state",
|
||||
"group_legacy_compatibility",
|
||||
"total",
|
||||
"sst_files",
|
||||
]
|
||||
out = []
|
||||
seen = set()
|
||||
for k in ordered:
|
||||
if k in db:
|
||||
out.append(f"- `{k}={db[k]}`")
|
||||
seen.add(k)
|
||||
for k in sorted(set(db) - seen):
|
||||
out.append(f"- `{k}={db[k]}`")
|
||||
return "\n".join(out) if out else "_(missing db_stats keys)_"
|
||||
|
||||
lines = []
|
||||
lines.append("# APNIC RRDP 增量同步验收(manual_sync)\n\n")
|
||||
lines.append(f"时间戳:`{now}`(UTC)\n\n")
|
||||
|
||||
lines.append("## 复现信息\n\n")
|
||||
lines.append(f"- base_report:`{base_report_path}`\n")
|
||||
lines.append(f"- delta_report:`{delta_report_path}`\n")
|
||||
lines.append(f"- base_db_stats:`{base_db_stats_path}`\n")
|
||||
lines.append(f"- delta_db_stats:`{delta_db_stats_path}`\n")
|
||||
lines.append(f"- base_rrdp_state:`{base_state_path}`\n")
|
||||
lines.append(f"- delta_rrdp_state:`{delta_state_path}`\n\n")
|
||||
|
||||
lines.append("## 运行结果概览\n\n")
|
||||
lines.append("| metric | base | delta |\n")
|
||||
lines.append("|---|---:|---:|\n")
|
||||
for k in [
|
||||
"validation_time",
|
||||
"publication_points_processed",
|
||||
"publication_points_failed",
|
||||
"rrdp_repos_unique",
|
||||
"vrps",
|
||||
"aspas",
|
||||
"audit_publication_points",
|
||||
"warnings_total",
|
||||
]:
|
||||
lines.append(f"| {k} | {base_sum[k]} | {delta_sum[k]} |\n")
|
||||
lines.append("\n")
|
||||
|
||||
def dur(meta: dict | None, key: str):
|
||||
if not meta:
|
||||
return None
|
||||
return (meta.get("durations_secs") or {}).get(key)
|
||||
|
||||
base_rpki_run = dur(base_meta, "rpki_run")
|
||||
delta_rpki_run = delta_meta["durations_secs"]["rpki_run"]
|
||||
base_total = dur(base_meta, "total_script")
|
||||
delta_total = delta_meta["durations_secs"]["total_script"]
|
||||
|
||||
lines.append("## 持续时间(seconds)\n\n")
|
||||
lines.append("| step | base | delta |\n")
|
||||
lines.append("|---|---:|---:|\n")
|
||||
lines.append(f"| rpki_run | {base_rpki_run if base_rpki_run is not None else 'unknown'} | {delta_rpki_run} |\n")
|
||||
lines.append(f"| total_script | {base_total if base_total is not None else 'unknown'} | {delta_total} |\n")
|
||||
lines.append("\n")
|
||||
if base_meta is None and base_meta_path_s:
|
||||
lines.append(f"> 注:未能读取 base meta:`{base_meta_path_s}`(文件不存在或不可读)。建议用 `full_sync.sh` 生成 baseline 以获得 base 时长对比。\n\n")
|
||||
|
||||
lines.append("RocksDB KV(`db_stats --exact`):\n\n")
|
||||
lines.append("### 基线(base)\n\n")
|
||||
lines.append(fmt_db_stats(base_db) + "\n\n")
|
||||
lines.append("### 增量(delta)\n\n")
|
||||
lines.append(fmt_db_stats(delta_db) + "\n\n")
|
||||
|
||||
lines.append("## RRDP 增量是否发生(基于 `rrdp_state` 变化)\n\n")
|
||||
lines.append(f"- repo_total(base)={len(base_state)}\n")
|
||||
lines.append(f"- repo_total(delta)={len(delta_state)}\n")
|
||||
lines.append(f"- serial_changed={serial_changed}\n")
|
||||
lines.append(f"- session_changed={session_changed}\n\n")
|
||||
if serial_deltas:
|
||||
lines.append("serial 增长最大的 10 个 RRDP repo(old -> new):\n\n")
|
||||
for uri, old, new, diff in serial_deltas[:10]:
|
||||
lines.append(f"- `{uri}`:`{old} -> {new}`(+{diff})\n")
|
||||
lines.append("\n")
|
||||
|
||||
lines.append("## 发布点(Publication Point)变化统计\n\n")
|
||||
lines.append("以 `manifest_rsync_uri` 作为发布点 key,对比 base vs delta:\n\n")
|
||||
lines.append(f"- base PP:`{len(base_keys)}`\n")
|
||||
lines.append(f"- delta PP:`{len(delta_keys)}`\n")
|
||||
lines.append(f"- `new_pp={len(new_pp)}`\n")
|
||||
lines.append(f"- `missing_pp={len(missing_pp)}`\n")
|
||||
lines.append(f"- `updated_pp={updated_pp}`\n")
|
||||
lines.append(f"- `unchanged_pp={unchanged_pp}`\n\n")
|
||||
lines.append("> 注:`new_pp/missing_pp/updated_pp` 会混入“遍历范围变化”的影响(例如 validation_time 不同、或 base 中存在更多失败 PP)。\n\n")
|
||||
|
||||
lines.append("## fail fetch / VCIR 当前实例缓存复用情况\n\n")
|
||||
lines.append(f"- repo sync failed(启发式:warning contains 'repo sync failed'/'rrdp fetch failed'/'rsync fetch failed')\n")
|
||||
lines.append(f" - base:`{base_repo_sync_failed}`\n")
|
||||
lines.append(f" - delta:`{delta_repo_sync_failed}`\n\n")
|
||||
|
||||
lines.append("- source 计数(按 `PublicationPointAudit.source`):\n\n")
|
||||
lines.append(f" - base:`{dict(base_sources)}`\n")
|
||||
lines.append(f" - delta:`{dict(delta_sources)}`\n\n")
|
||||
|
||||
def render_cache_reasons(title: str, c: Counter) -> str:
|
||||
if not c:
|
||||
return f"{title}:`0`(未使用 VCIR 当前实例缓存复用)\n\n"
|
||||
lines = []
|
||||
total = sum(c.values())
|
||||
lines.append(f"{title}:`{total}`\n\n")
|
||||
lines.append("Top reasons(按 warning message 聚合,可能一条 PP 有多条 warning):\n\n")
|
||||
for msg, n in c.most_common(10):
|
||||
lines.append(f"- `{n}` × {msg}\n")
|
||||
lines.append("\n")
|
||||
return "".join(lines)
|
||||
|
||||
lines.append(render_cache_reasons("- base `source=vcir_current_instance`", base_cache_reasons))
|
||||
lines.append(render_cache_reasons("- delta `source=vcir_current_instance`", delta_cache_reasons))
|
||||
|
||||
lines.append("## 文件变更统计(按对象类型)\n\n")
|
||||
lines.append("按 `ObjectAuditEntry.sha256_hex` 对比(同一 rsync URI 前后 hash 变化记为 `~changed`):\n\n")
|
||||
lines.append("| kind | added | changed | removed |\n")
|
||||
lines.append("|---|---:|---:|---:|\n")
|
||||
for kind in ["manifest","crl","certificate","roa","aspa","other"]:
|
||||
st = kind_stats[kind]
|
||||
lines.append(f"| {kind} | {st['added']} | {st['changed']} | {st['removed']} |\n")
|
||||
lines.append("\n")
|
||||
|
||||
lines.append("## VRP 影响(去重后集合 diff)\n\n")
|
||||
lines.append("以 `(asn, prefix, max_length)` 为 key:\n\n")
|
||||
lines.append(f"- base unique VRP:`{len(base_v)}`\n")
|
||||
lines.append(f"- delta unique VRP:`{len(delta_v)}`\n")
|
||||
lines.append(f"- `added={len(added_v)}`\n")
|
||||
lines.append(f"- `removed={len(removed_v)}`\n")
|
||||
lines.append(f"- `net={len(added_v) - len(removed_v)}`\n\n")
|
||||
|
||||
out_md_path.write_text("".join(lines), encoding="utf-8")
|
||||
print(out_md_path)
|
||||
PY
|
||||
|
||||
echo "== done ==" >&2
|
||||
echo "artifacts:" >&2
|
||||
echo "- delta db: $DELTA_DB_DIR" >&2
|
||||
echo "- delta report: $DELTA_REPORT_JSON" >&2
|
||||
echo "- delta run log: $DELTA_RUN_LOG" >&2
|
||||
echo "- delta meta json: $DELTA_META_JSON" >&2
|
||||
echo "- analysis md: $DELTA_ANALYSIS_MD" >&2
|
||||
echo "- base state tsv: $BASE_RRDP_STATE_TSV" >&2
|
||||
echo "- delta state tsv: $DELTA_RRDP_STATE_TSV" >&2
|
||||
189
scripts/manual_sync/full_sync.sh
Executable file
189
scripts/manual_sync/full_sync.sh
Executable file
@@ -0,0 +1,189 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
# Full sync + validation from a TAL URL (default: APNIC).
|
||||
#
|
||||
# Produces artifacts under OUT_DIR (default: target/live/manual_sync):
|
||||
# - *_db_* RocksDB directory
|
||||
# - *_report_*.json audit report
|
||||
# - *_run_*.log stdout/stderr log (includes summary)
|
||||
# - *_db_stats_*.txt db_stats --exact output
|
||||
# - *_rrdp_state_*.tsv rrdp_state_dump --view legacy-state output
|
||||
|
||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||
cd "$ROOT_DIR"
|
||||
|
||||
TAL_URL="${TAL_URL:-https://tal.apnic.net/tal-archive/apnic-rfc7730-https.tal}"
|
||||
HTTP_TIMEOUT_SECS="${HTTP_TIMEOUT_SECS:-1800}"
|
||||
RSYNC_TIMEOUT_SECS="${RSYNC_TIMEOUT_SECS:-1800}"
|
||||
RSYNC_MIRROR_ROOT="${RSYNC_MIRROR_ROOT:-}"
|
||||
VALIDATION_TIME="${VALIDATION_TIME:-}"
|
||||
|
||||
OUT_DIR="${OUT_DIR:-$ROOT_DIR/target/live/manual_sync}"
|
||||
mkdir -p "$OUT_DIR"
|
||||
|
||||
TS="$(date -u +%Y%m%dT%H%M%SZ)"
|
||||
RUN_NAME="${RUN_NAME:-apnic_full_${TS}}"
|
||||
|
||||
DB_DIR="${DB_DIR:-$OUT_DIR/${RUN_NAME}_db}"
|
||||
REPORT_JSON="${REPORT_JSON:-$OUT_DIR/${RUN_NAME}_report.json}"
|
||||
RUN_LOG="${RUN_LOG:-$OUT_DIR/${RUN_NAME}_run.log}"
|
||||
DB_STATS_TXT="${DB_STATS_TXT:-$OUT_DIR/${RUN_NAME}_db_stats.txt}"
|
||||
RRDP_STATE_TSV="${RRDP_STATE_TSV:-$OUT_DIR/${RUN_NAME}_rrdp_state.tsv}"
|
||||
RUN_META_JSON="${RUN_META_JSON:-$OUT_DIR/${RUN_NAME}_meta.json}"
|
||||
SUMMARY_MD="${SUMMARY_MD:-$OUT_DIR/${RUN_NAME}_summary.md}"
|
||||
|
||||
echo "== rpki manual full sync ==" >&2
|
||||
echo "tal_url=$TAL_URL" >&2
|
||||
echo "db=$DB_DIR" >&2
|
||||
echo "report_json=$REPORT_JSON" >&2
|
||||
echo "out_dir=$OUT_DIR" >&2
|
||||
|
||||
cmd=(cargo run --release --bin rpki -- \
|
||||
--db "$DB_DIR" \
|
||||
--tal-url "$TAL_URL" \
|
||||
--http-timeout-secs "$HTTP_TIMEOUT_SECS" \
|
||||
--rsync-timeout-secs "$RSYNC_TIMEOUT_SECS" \
|
||||
--report-json "$REPORT_JSON")
|
||||
|
||||
if [[ -n "${RSYNC_MIRROR_ROOT}" ]]; then
|
||||
cmd+=(--rsync-mirror-root "$RSYNC_MIRROR_ROOT")
|
||||
fi
|
||||
if [[ -n "${VALIDATION_TIME}" ]]; then
|
||||
cmd+=(--validation-time "$VALIDATION_TIME")
|
||||
fi
|
||||
|
||||
script_start_s="$(date +%s)"
|
||||
run_start_s="$(date +%s)"
|
||||
(
|
||||
echo "# command:"
|
||||
printf '%q ' "${cmd[@]}"
|
||||
echo
|
||||
echo
|
||||
"${cmd[@]}"
|
||||
) 2>&1 | tee "$RUN_LOG" >/dev/null
|
||||
run_end_s="$(date +%s)"
|
||||
run_duration_s="$((run_end_s - run_start_s))"
|
||||
|
||||
echo "== db_stats (exact) ==" >&2
|
||||
db_stats_start_s="$(date +%s)"
|
||||
cargo run --release --bin db_stats -- --db "$DB_DIR" --exact 2>&1 | tee "$DB_STATS_TXT" >/dev/null
|
||||
db_stats_end_s="$(date +%s)"
|
||||
db_stats_duration_s="$((db_stats_end_s - db_stats_start_s))"
|
||||
|
||||
echo "== rrdp_state_dump (legacy-state) ==" >&2
|
||||
state_start_s="$(date +%s)"
|
||||
cargo run --release --bin rrdp_state_dump -- --db "$DB_DIR" --view legacy-state >"$RRDP_STATE_TSV"
|
||||
state_end_s="$(date +%s)"
|
||||
state_duration_s="$((state_end_s - state_start_s))"
|
||||
|
||||
script_end_s="$(date +%s)"
|
||||
total_duration_s="$((script_end_s - script_start_s))"
|
||||
|
||||
echo "== write run meta + summary ==" >&2
|
||||
TAL_URL="$TAL_URL" \
|
||||
DB_DIR="$DB_DIR" \
|
||||
REPORT_JSON="$REPORT_JSON" \
|
||||
RUN_LOG="$RUN_LOG" \
|
||||
HTTP_TIMEOUT_SECS="$HTTP_TIMEOUT_SECS" \
|
||||
RSYNC_TIMEOUT_SECS="$RSYNC_TIMEOUT_SECS" \
|
||||
VALIDATION_TIME_ARG="$VALIDATION_TIME" \
|
||||
RUN_DURATION_S="$run_duration_s" \
|
||||
DB_STATS_DURATION_S="$db_stats_duration_s" \
|
||||
STATE_DURATION_S="$state_duration_s" \
|
||||
TOTAL_DURATION_S="$total_duration_s" \
|
||||
python3 - "$REPORT_JSON" "$RUN_META_JSON" "$SUMMARY_MD" <<'PY'
|
||||
import json
|
||||
import os
|
||||
import sys
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
|
||||
report_path = Path(sys.argv[1])
|
||||
meta_path = Path(sys.argv[2])
|
||||
summary_path = Path(sys.argv[3])
|
||||
|
||||
rep = json.loads(report_path.read_text(encoding="utf-8"))
|
||||
|
||||
now = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||
download_stats = rep.get("download_stats") or {}
|
||||
meta = {
|
||||
"recorded_at_utc": now,
|
||||
"tal_url": os.environ["TAL_URL"],
|
||||
"db_dir": os.environ["DB_DIR"],
|
||||
"report_json": os.environ["REPORT_JSON"],
|
||||
"run_log": os.environ["RUN_LOG"],
|
||||
"validation_time_rfc3339_utc": rep["meta"]["validation_time_rfc3339_utc"],
|
||||
"http_timeout_secs": int(os.environ["HTTP_TIMEOUT_SECS"]),
|
||||
"rsync_timeout_secs": int(os.environ["RSYNC_TIMEOUT_SECS"]),
|
||||
"validation_time_arg": os.environ.get("VALIDATION_TIME_ARG",""),
|
||||
"durations_secs": {
|
||||
"rpki_run": int(os.environ["RUN_DURATION_S"]),
|
||||
"db_stats_exact": int(os.environ["DB_STATS_DURATION_S"]),
|
||||
"rrdp_state_dump": int(os.environ["STATE_DURATION_S"]),
|
||||
"total_script": int(os.environ["TOTAL_DURATION_S"]),
|
||||
},
|
||||
"counts": {
|
||||
"publication_points_processed": rep["tree"]["instances_processed"],
|
||||
"publication_points_failed": rep["tree"]["instances_failed"],
|
||||
"vrps": len(rep["vrps"]),
|
||||
"aspas": len(rep["aspas"]),
|
||||
"audit_publication_points": len(rep["publication_points"]),
|
||||
},
|
||||
"download_stats": download_stats,
|
||||
}
|
||||
|
||||
meta_path.write_text(json.dumps(meta, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||
|
||||
lines = []
|
||||
lines.append("# Manual full sync summary\\n\\n")
|
||||
lines.append(f"- recorded_at_utc: `{now}`\\n")
|
||||
lines.append(f"- tal_url: `{meta['tal_url']}`\\n")
|
||||
lines.append(f"- db: `{meta['db_dir']}`\\n")
|
||||
lines.append(f"- report_json: `{meta['report_json']}`\\n")
|
||||
lines.append(f"- validation_time: `{meta['validation_time_rfc3339_utc']}`\\n\\n")
|
||||
lines.append("## Results\\n\\n")
|
||||
lines.append("| metric | value |\\n")
|
||||
lines.append("|---|---:|\\n")
|
||||
for k in ["publication_points_processed","publication_points_failed","vrps","aspas","audit_publication_points"]:
|
||||
lines.append(f"| {k} | {meta['counts'][k]} |\\n")
|
||||
lines.append("\\n")
|
||||
lines.append("## Durations (seconds)\\n\\n")
|
||||
lines.append("| step | seconds |\\n")
|
||||
lines.append("|---|---:|\\n")
|
||||
for k,v in meta["durations_secs"].items():
|
||||
lines.append(f"| {k} | {v} |\\n")
|
||||
lines.append("\\n")
|
||||
|
||||
lines.append("## Download Stats\\n\\n")
|
||||
lines.append("- raw events: `report_json.downloads`\\n")
|
||||
lines.append("- aggregated: `report_json.download_stats` (copied into meta.json)\\n\\n")
|
||||
|
||||
def fmt_u(v):
|
||||
if v is None:
|
||||
return ""
|
||||
return str(v)
|
||||
|
||||
by_kind = download_stats.get("by_kind") or {}
|
||||
lines.append("| kind | ok | fail | duration_ms_total | bytes_total | objects_count_total | objects_bytes_total |\\n")
|
||||
lines.append("|---|---:|---:|---:|---:|---:|---:|\\n")
|
||||
for kind in sorted(by_kind.keys()):
|
||||
st = by_kind[kind] or {}
|
||||
lines.append(
|
||||
f"| {kind} | {st.get('ok_total',0)} | {st.get('fail_total',0)} | {st.get('duration_ms_total',0)} | {fmt_u(st.get('bytes_total'))} | {fmt_u(st.get('objects_count_total'))} | {fmt_u(st.get('objects_bytes_total'))} |\\n"
|
||||
)
|
||||
lines.append("\\n")
|
||||
|
||||
summary_path.write_text("".join(lines), encoding="utf-8")
|
||||
print(summary_path)
|
||||
PY
|
||||
|
||||
echo "== done ==" >&2
|
||||
echo "artifacts:" >&2
|
||||
echo "- db: $DB_DIR" >&2
|
||||
echo "- report: $REPORT_JSON" >&2
|
||||
echo "- run log: $RUN_LOG" >&2
|
||||
echo "- db stats: $DB_STATS_TXT" >&2
|
||||
echo "- rrdp state: $RRDP_STATE_TSV" >&2
|
||||
echo "- meta json: $RUN_META_JSON" >&2
|
||||
echo "- summary md: $SUMMARY_MD" >&2
|
||||
268
scripts/payload_replay/README.md
Normal file
268
scripts/payload_replay/README.md
Normal file
@@ -0,0 +1,268 @@
|
||||
# Payload Replay Scripts
|
||||
|
||||
本目录提供基于本地 payload archive 的手工 replay 入口。
|
||||
|
||||
## `multi_rir_case_info.py`
|
||||
|
||||
用于从 multi-RIR bundle 中解析指定 `rir` 的输入路径、对照 CSV、fixture、以及 Routinator replay timing 基线。
|
||||
|
||||
示例:
|
||||
|
||||
```bash
|
||||
python3 scripts/payload_replay/multi_rir_case_info.py \
|
||||
--bundle-root ../../rpki/target/live/20260316-112341-multi-final3 \
|
||||
--rir afrinic
|
||||
```
|
||||
|
||||
也支持输出 shell 环境变量:
|
||||
|
||||
```bash
|
||||
python3 scripts/payload_replay/multi_rir_case_info.py \
|
||||
--bundle-root ../../rpki/target/live/20260316-112341-multi-final3 \
|
||||
--rir afrinic \
|
||||
--format env
|
||||
```
|
||||
|
||||
## `run_multi_rir_replay_case.sh`
|
||||
|
||||
统一的 multi-RIR 入口。给定 `rir` 和模式后,它会自动选择该 RIR 的:
|
||||
|
||||
- snapshot/base replay 输入
|
||||
- delta replay 输入
|
||||
- 对照 CSV
|
||||
- TAL / TA fixture
|
||||
- trust anchor 名称
|
||||
|
||||
用法:
|
||||
|
||||
```bash
|
||||
./scripts/payload_replay/run_multi_rir_replay_case.sh <rir> [describe|snapshot|delta|both]
|
||||
```
|
||||
|
||||
示例:
|
||||
|
||||
```bash
|
||||
./scripts/payload_replay/run_multi_rir_replay_case.sh afrinic describe
|
||||
./scripts/payload_replay/run_multi_rir_replay_case.sh lacnic snapshot
|
||||
./scripts/payload_replay/run_multi_rir_replay_case.sh arin delta
|
||||
./scripts/payload_replay/run_multi_rir_replay_case.sh ripe both
|
||||
```
|
||||
|
||||
|
||||
脚本会自动:
|
||||
|
||||
- 从 multi-RIR bundle 中选择指定 RIR 的 snapshot/base 与 delta 输入
|
||||
- 读取该 RIR 的 Routinator `base-replay` / `delta-replay` timing 基线
|
||||
- 优先使用 `base-locks.json.validationTime` 与 `locks-delta.json.validationTime` 作为 replay `--validation-time`;若缺失才回退到 `timings/base-replay.json` 与 `timings/delta-replay.json` 的 `startedAt`
|
||||
- 在 `target/live/multi_rir_replay_runs/<rir>/` 下生成:
|
||||
- snapshot replay 产物
|
||||
- delta replay 产物
|
||||
- per-RIR 合并 case report(含 correctness + timing compare)
|
||||
|
||||
默认 bundle 根目录为:
|
||||
|
||||
- `../../rpki/target/live/20260316-112341-multi-final3`
|
||||
|
||||
也可以通过 `BUNDLE_ROOT` 覆盖。
|
||||
|
||||
## `run_apnic_snapshot_replay_profile.sh`
|
||||
|
||||
基于 multi-RIR bundle 中的 APNIC snapshot 输入,使用当前 replay 主流程执行一次带 `--analyze` 和 `--profile-cpu` 的离线 profile。
|
||||
|
||||
```bash
|
||||
./scripts/payload_replay/run_apnic_snapshot_replay_profile.sh
|
||||
```
|
||||
|
||||
作用:
|
||||
|
||||
- 使用 `APNIC` 的 snapshot/base replay 输入
|
||||
- 自动开启:
|
||||
- `--analyze`
|
||||
- `--profile-cpu`
|
||||
- 自动记录:
|
||||
- replay wall-clock 时长
|
||||
- Routinator baseline (`base-replay`)
|
||||
- analyze 目录路径
|
||||
- 生成:
|
||||
- `report.json`
|
||||
- `run.log`
|
||||
- `meta.json`
|
||||
- `summary.md`
|
||||
- 以及 `target/live/analyze/<timestamp>/` 下的:
|
||||
- `timing.json`
|
||||
- `flamegraph.svg`
|
||||
- `pprof.pb.gz`
|
||||
|
||||
支持:
|
||||
|
||||
- `DRY_RUN=1`:只打印命令,不真正执行
|
||||
- `MAX_DEPTH` / `MAX_INSTANCES`:用于限定 replay 范围
|
||||
- `PROFILE_RUN_ROOT`:覆盖 wrapper 产物输出目录
|
||||
|
||||
## `run_apnic_replay.sh`
|
||||
|
||||
默认使用:
|
||||
|
||||
- `tests/fixtures/tal/apnic-rfc7730-https.tal`
|
||||
- `tests/fixtures/ta/apnic-ta.cer`
|
||||
- `target/live/payload_replay/payload-archive`
|
||||
- `target/live/payload_replay/locks.json`
|
||||
|
||||
运行:
|
||||
|
||||
```bash
|
||||
./scripts/payload_replay/run_apnic_replay.sh
|
||||
```
|
||||
|
||||
产物默认输出到:
|
||||
|
||||
- `target/live/payload_replay_runs/`
|
||||
|
||||
包含:
|
||||
|
||||
- replay DB 目录
|
||||
- `report.json`
|
||||
- `run.log`
|
||||
- `meta.json`
|
||||
- `summary.md`
|
||||
|
||||
## 环境变量
|
||||
|
||||
可覆盖:
|
||||
|
||||
- `TAL_PATH`
|
||||
- `TA_PATH`
|
||||
- `PAYLOAD_REPLAY_ARCHIVE`
|
||||
- `PAYLOAD_REPLAY_LOCKS`
|
||||
- `VALIDATION_TIME`
|
||||
- `MAX_DEPTH`
|
||||
- `MAX_INSTANCES`
|
||||
- `OUT_DIR`
|
||||
- `RUN_NAME`
|
||||
- `DB_DIR`
|
||||
- `REPORT_JSON`
|
||||
- `RUN_LOG`
|
||||
- `META_JSON`
|
||||
- `SUMMARY_MD`
|
||||
|
||||
## 说明
|
||||
|
||||
- 该脚本依赖 `rpki` CLI 已支持:
|
||||
- `--payload-replay-archive`
|
||||
- `--payload-replay-locks`
|
||||
- replay 模式必须搭配离线 TAL/TA 输入,不会去访问真实 RRDP / rsync 网络源。
|
||||
|
||||
## `report_to_routinator_csv.py`
|
||||
|
||||
把 `rpki` 生成的 `report.json` 转成 Routinator 风格的 VRP CSV:
|
||||
|
||||
```bash
|
||||
python3 scripts/payload_replay/report_to_routinator_csv.py \
|
||||
--report target/live/payload_replay_runs/<run>_report.json \
|
||||
--out target/live/payload_replay_runs/<run>_vrps.csv \
|
||||
--trust-anchor apnic
|
||||
```
|
||||
|
||||
输出列为:
|
||||
|
||||
- `ASN`
|
||||
- `IP Prefix`
|
||||
- `Max Length`
|
||||
- `Trust Anchor`
|
||||
|
||||
## `compare_with_routinator_record.sh`
|
||||
|
||||
把 ours 生成的 VRP CSV 与 Routinator 的 `record.csv` 做对比:
|
||||
|
||||
```bash
|
||||
./scripts/payload_replay/compare_with_routinator_record.sh \
|
||||
target/live/payload_replay_runs/<run>_vrps.csv \
|
||||
target/live/payload_replay/record.csv
|
||||
```
|
||||
|
||||
会产出:
|
||||
|
||||
- compare summary Markdown
|
||||
- `only_in_ours.csv`
|
||||
- `only_in_record.csv`
|
||||
|
||||
## `run_apnic_replay.sh` 现有额外产物
|
||||
|
||||
脚本现在除了 `report/meta/summary`,还会额外生成:
|
||||
|
||||
- `vrps.csv`
|
||||
- 若 `ROUTINATOR_RECORD_CSV` 存在,则生成:
|
||||
- compare summary
|
||||
- `only_in_ours.csv`
|
||||
- `only_in_record.csv`
|
||||
|
||||
## `run_apnic_delta_replay.sh`
|
||||
|
||||
使用 APNIC delta demo 数据集运行 base + delta replay:
|
||||
|
||||
```bash
|
||||
./scripts/payload_replay/run_apnic_delta_replay.sh
|
||||
```
|
||||
|
||||
默认输入:
|
||||
|
||||
- `target/live/apnic_delta_demo/20260315-170223-autoplay/base-payload-archive`
|
||||
- `target/live/apnic_delta_demo/20260315-170223-autoplay/base-locks.json`
|
||||
- `target/live/apnic_delta_demo/20260315-170223-autoplay/payload-delta-archive`
|
||||
- `target/live/apnic_delta_demo/20260315-170223-autoplay/locks-delta.json`
|
||||
- `tests/fixtures/tal/apnic-rfc7730-https.tal`
|
||||
- `tests/fixtures/ta/apnic-ta.cer`
|
||||
|
||||
输出目录默认:`target/live/payload_delta_replay_runs/`
|
||||
|
||||
## `run_apnic_delta_replay.sh` compare outputs
|
||||
|
||||
脚本现在在 delta replay 结束后还会额外生成:
|
||||
|
||||
- `vrps.csv`
|
||||
- compare summary Markdown
|
||||
- `only_in_ours.csv`
|
||||
- `only_in_record.csv`
|
||||
|
||||
默认 compare 输入是:
|
||||
|
||||
- `target/live/apnic_delta_demo/20260315-170223-autoplay/record-delta.csv`
|
||||
|
||||
也可以通过环境变量覆盖:
|
||||
|
||||
- `TRUST_ANCHOR`
|
||||
- `ROUTINATOR_RECORD_CSV`
|
||||
- `VRPS_CSV`
|
||||
- `COMPARE_SUMMARY_MD`
|
||||
- `ONLY_IN_OURS_CSV`
|
||||
- `ONLY_IN_RECORD_CSV`
|
||||
|
||||
## `write_multi_rir_case_report.py`
|
||||
|
||||
把某个 RIR 的 snapshot replay 与 delta replay 的 `meta.json`、compare summary 以及 Routinator timing 基线合并成一个 per-RIR Markdown/JSON 报告。
|
||||
|
||||
该脚本通常由 `run_multi_rir_replay_case.sh <rir> both` 自动调用。
|
||||
|
||||
## `run_multi_rir_replay_suite.sh`
|
||||
|
||||
顺序执行 5 个 RIR(或环境变量 `RIRS` 指定的子集)的 `both` 模式,并最终生成 multi-RIR 汇总报告。
|
||||
|
||||
```bash
|
||||
./scripts/payload_replay/run_multi_rir_replay_suite.sh
|
||||
```
|
||||
|
||||
可覆盖环境变量:
|
||||
|
||||
- `BUNDLE_ROOT`
|
||||
- `SUITE_OUT_DIR`
|
||||
- `RIRS`
|
||||
|
||||
最终输出:
|
||||
|
||||
- `<suite_out_dir>/<rir>/<rir>_case_report.md`
|
||||
- `<suite_out_dir>/multi_rir_summary.md`
|
||||
- `<suite_out_dir>/multi_rir_summary.json`
|
||||
|
||||
## `write_multi_rir_summary.py`
|
||||
|
||||
汇总 5 个 RIR 的 per-RIR case report,生成 correctness + timing 总表与几何平均比值。
|
||||
110
scripts/payload_replay/compare_with_routinator_record.sh
Executable file
110
scripts/payload_replay/compare_with_routinator_record.sh
Executable file
@@ -0,0 +1,110 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
if [[ $# -lt 2 || $# -gt 5 ]]; then
|
||||
echo "Usage: $0 <ours.csv> <record.csv> [summary.md] [only_in_ours.csv] [only_in_record.csv]" >&2
|
||||
exit 2
|
||||
fi
|
||||
|
||||
OURS_CSV="$1"
|
||||
RECORD_CSV="$2"
|
||||
SUMMARY_MD="${3:-}"
|
||||
ONLY_IN_OURS_CSV="${4:-}"
|
||||
ONLY_IN_RECORD_CSV="${5:-}"
|
||||
|
||||
if [[ -z "$SUMMARY_MD" ]]; then
|
||||
SUMMARY_MD="$(dirname "$OURS_CSV")/$(basename "$OURS_CSV" .csv)_vs_routinator_summary.md"
|
||||
fi
|
||||
if [[ -z "$ONLY_IN_OURS_CSV" ]]; then
|
||||
ONLY_IN_OURS_CSV="$(dirname "$OURS_CSV")/$(basename "$OURS_CSV" .csv)_only_in_ours.csv"
|
||||
fi
|
||||
if [[ -z "$ONLY_IN_RECORD_CSV" ]]; then
|
||||
ONLY_IN_RECORD_CSV="$(dirname "$OURS_CSV")/$(basename "$OURS_CSV" .csv)_only_in_record.csv"
|
||||
fi
|
||||
|
||||
python3 - "$OURS_CSV" "$RECORD_CSV" "$SUMMARY_MD" "$ONLY_IN_OURS_CSV" "$ONLY_IN_RECORD_CSV" <<'PY'
|
||||
import csv
|
||||
import ipaddress
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
ours_csv = Path(sys.argv[1])
|
||||
record_csv = Path(sys.argv[2])
|
||||
summary_md = Path(sys.argv[3])
|
||||
only_in_ours_csv = Path(sys.argv[4])
|
||||
only_in_record_csv = Path(sys.argv[5])
|
||||
|
||||
|
||||
def normalize_row(row: dict):
|
||||
asn = row["ASN"].strip().upper()
|
||||
prefix = row["IP Prefix"].strip()
|
||||
max_len = str(int(row["Max Length"]))
|
||||
ta = row["Trust Anchor"].strip()
|
||||
network = ipaddress.ip_network(prefix, strict=False)
|
||||
return {
|
||||
"ASN": asn,
|
||||
"IP Prefix": str(network),
|
||||
"Max Length": max_len,
|
||||
"Trust Anchor": ta,
|
||||
}
|
||||
|
||||
|
||||
def read_rows(path: Path):
|
||||
with path.open(encoding="utf-8", newline="") as f:
|
||||
rows = [normalize_row(r) for r in csv.DictReader(f)]
|
||||
return rows
|
||||
|
||||
|
||||
def row_key(row: dict):
|
||||
network = ipaddress.ip_network(row["IP Prefix"], strict=False)
|
||||
return (
|
||||
row["ASN"],
|
||||
network.version,
|
||||
int(network.network_address),
|
||||
network.prefixlen,
|
||||
int(row["Max Length"]),
|
||||
row["Trust Anchor"],
|
||||
)
|
||||
|
||||
|
||||
def write_rows(path: Path, rows):
|
||||
path.parent.mkdir(parents=True, exist_ok=True)
|
||||
with path.open("w", encoding="utf-8", newline="") as f:
|
||||
writer = csv.DictWriter(f, fieldnames=["ASN", "IP Prefix", "Max Length", "Trust Anchor"])
|
||||
writer.writeheader()
|
||||
for row in rows:
|
||||
writer.writerow(row)
|
||||
|
||||
ours = read_rows(ours_csv)
|
||||
record = read_rows(record_csv)
|
||||
ours_map = {row_key(r): r for r in ours}
|
||||
record_map = {row_key(r): r for r in record}
|
||||
only_in_ours = [ours_map[k] for k in sorted(set(ours_map) - set(record_map))]
|
||||
only_in_record = [record_map[k] for k in sorted(set(record_map) - set(ours_map))]
|
||||
intersection = len(set(ours_map) & set(record_map))
|
||||
|
||||
write_rows(only_in_ours_csv, only_in_ours)
|
||||
write_rows(only_in_record_csv, only_in_record)
|
||||
|
||||
summary_md.parent.mkdir(parents=True, exist_ok=True)
|
||||
summary = []
|
||||
summary.append("# Replay vs Routinator VRP Compare\n\n")
|
||||
summary.append(f"- ours_csv: `{ours_csv}`\n")
|
||||
summary.append(f"- record_csv: `{record_csv}`\n")
|
||||
summary.append(f"- only_in_ours_csv: `{only_in_ours_csv}`\n")
|
||||
summary.append(f"- only_in_record_csv: `{only_in_record_csv}`\n\n")
|
||||
summary.append("| metric | value |\n")
|
||||
summary.append("|---|---:|\n")
|
||||
summary.append(f"| ours_total | {len(ours_map)} |\n")
|
||||
summary.append(f"| record_total | {len(record_map)} |\n")
|
||||
summary.append(f"| intersection | {intersection} |\n")
|
||||
summary.append(f"| only_in_ours | {len(only_in_ours)} |\n")
|
||||
summary.append(f"| only_in_record | {len(only_in_record)} |\n")
|
||||
summary_md.write_text("".join(summary), encoding="utf-8")
|
||||
print(summary_md)
|
||||
PY
|
||||
|
||||
echo "== compare complete ==" >&2
|
||||
echo "- summary: $SUMMARY_MD" >&2
|
||||
echo "- only_in_ours: $ONLY_IN_OURS_CSV" >&2
|
||||
echo "- only_in_record: $ONLY_IN_RECORD_CSV" >&2
|
||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user