Compare commits
28 Commits
dev_1.2_xu
...
dev_1.0.0_
| Author | SHA1 | Date | |
|---|---|---|---|
| af1c2c7f88 | |||
| e45830d79f | |||
| 77fc2f1a41 | |||
| e083fe4daa | |||
| c9ef5aaf4c | |||
| 34fb9657f1 | |||
| 6edc420ce2 | |||
| cd0ba15286 | |||
| fe8b89d829 | |||
| d6d44669b4 | |||
| 557a69cbd2 | |||
| 73d8ebb5c1 | |||
| cf764c35bb | |||
| e3339533b8 | |||
| afc50364f8 | |||
| 6276d13814 | |||
| 0f3d65254e | |||
| 13516c4f73 | |||
| 68cbd3c500 | |||
| 1cc3351bef | |||
| 2a6a963ecd | |||
| 6e135b9d7a | |||
| afc31c02ab | |||
| 7be865d7f1 | |||
| a58e507f92 | |||
| cc9f3f21de | |||
| 56ae2ca4fc | |||
| bcd4829486 |
@@ -1,9 +0,0 @@
|
|||||||
.git
|
|
||||||
.gitignore
|
|
||||||
.idea
|
|
||||||
target
|
|
||||||
rtr-db
|
|
||||||
tests
|
|
||||||
specs
|
|
||||||
scripts
|
|
||||||
README.md
|
|
||||||
6
.gitignore
vendored
6
.gitignore
vendored
@@ -1,5 +1,3 @@
|
|||||||
target/
|
target/
|
||||||
rtr-db/
|
Cargo.lock
|
||||||
.idea/
|
perf.*
|
||||||
logs/
|
|
||||||
report/
|
|
||||||
|
|||||||
3528
Cargo.lock
generated
3528
Cargo.lock
generated
File diff suppressed because it is too large
Load Diff
52
Cargo.toml
52
Cargo.toml
@@ -3,38 +3,34 @@ name = "rpki"
|
|||||||
version = "0.1.0"
|
version = "0.1.0"
|
||||||
edition = "2024"
|
edition = "2024"
|
||||||
|
|
||||||
[[bin]]
|
[features]
|
||||||
name = "rpki_rtr"
|
default = ["full"]
|
||||||
path = "src/main_rtr.rs"
|
# Full build used by the main RP implementation (includes RocksDB-backed storage).
|
||||||
|
full = ["dep:rocksdb"]
|
||||||
|
profile = ["dep:pprof", "dep:flate2"]
|
||||||
|
|
||||||
[dependencies]
|
[dependencies]
|
||||||
der-parser = "10.0.0"
|
asn1-rs = "0.7.1"
|
||||||
|
der-parser = { version = "10.0.0", features = ["serialize"] }
|
||||||
hex = "0.4.3"
|
hex = "0.4.3"
|
||||||
|
base64 = "0.22.1"
|
||||||
|
sha2 = "0.10.8"
|
||||||
thiserror = "2.0.18"
|
thiserror = "2.0.18"
|
||||||
time = "0.3.45"
|
time = "0.3.45"
|
||||||
|
ring = "0.17.14"
|
||||||
x509-parser = { version = "0.18.0", features = ["verify"] }
|
x509-parser = { version = "0.18.0", features = ["verify"] }
|
||||||
url = "2.5.8"
|
url = "2.5.8"
|
||||||
asn1-rs = "0.7.1"
|
serde = { version = "1.0.218", features = ["derive"] }
|
||||||
asn1-rs-derive = "0.6.0"
|
serde_json = "1.0.140"
|
||||||
asn1 = "0.23.0"
|
toml = "0.8.20"
|
||||||
arc-swap = "1.7.0"
|
rocksdb = { version = "0.22.0", optional = true, default-features = false, features = ["lz4"] }
|
||||||
chrono = { version = "0.4.44", features = ["serde"] }
|
serde_cbor = "0.11.2"
|
||||||
chrono-tz = "0.10"
|
roxmltree = "0.20.0"
|
||||||
bytes = "1.11.1"
|
quick-xml = "0.37.2"
|
||||||
tokio = { version = "1.49.0", features = ["full"] }
|
uuid = { version = "1.7.0", features = ["v4"] }
|
||||||
rand = "0.10.0"
|
reqwest = { version = "0.12.12", default-features = false, features = ["blocking", "rustls-tls"] }
|
||||||
rocksdb = { version = "0.21.0", default-features = false }
|
pprof = { version = "0.14.1", optional = true, features = ["flamegraph", "prost-codec"] }
|
||||||
serde = { version = "1", features = ["derive", "rc"] }
|
flate2 = { version = "1.0.35", optional = true }
|
||||||
serde_json = "1"
|
tempfile = "3.16.0"
|
||||||
base64 = "0.22"
|
|
||||||
anyhow = "1"
|
[dev-dependencies]
|
||||||
tracing = "0.1.44"
|
|
||||||
sha2 = "0.10"
|
|
||||||
tempfile = "3"
|
|
||||||
tokio-rustls = "0.26"
|
|
||||||
rustls = "0.23"
|
|
||||||
rustls-pemfile = "2"
|
|
||||||
rustls-pki-types = "1.14.0"
|
|
||||||
socket2 = "0.5"
|
|
||||||
russh = { version = "0.60.0", default-features = false, features = ["ring", "rsa"] }
|
|
||||||
tracing-subscriber = { version = "0.3", features = ["env-filter", "fmt"] }
|
|
||||||
|
|||||||
238
README.client.md
238
README.client.md
@@ -1,238 +0,0 @@
|
|||||||
# RTR Client 使用说明
|
|
||||||
|
|
||||||
本文说明仓库内三类 RTR Client 的用途、启动方式和验证命令:
|
|
||||||
|
|
||||||
- `deploy/client`: 本仓库自带的 `rtr_debug_client`,适合做 RTR 协议调试、连通性检查和单次查询。
|
|
||||||
- `deploy/frr`: FRR 作为黑盒 RTR Client,适合验证真实路由软件是否能消费 server 下发的 VRP。
|
|
||||||
- `deploy/bird`: BIRD 作为黑盒 RTR Client,适合验证 TCP 或 SSH transport 下的 RPKI/ROA/ASPA 数据导入。
|
|
||||||
|
|
||||||
以下命令默认从仓库根目录执行。
|
|
||||||
|
|
||||||
## 目录
|
|
||||||
|
|
||||||
- [前置条件](#前置条件)
|
|
||||||
- [rtr_debug_client](#rtr_debug_client)
|
|
||||||
- [TCP](#tcp)
|
|
||||||
- [TLS/mTLS](#tlsmtls)
|
|
||||||
- [SSH](#ssh)
|
|
||||||
- [多客户端压测](#多客户端压测)
|
|
||||||
- [FRR Client](#frr-client)
|
|
||||||
- [BIRD Client](#bird-client)
|
|
||||||
- [TCP](#tcp-1)
|
|
||||||
- [SSH](#ssh-1)
|
|
||||||
- [如何判断 Client 正常](#如何判断-client-正常)
|
|
||||||
|
|
||||||
## 前置条件
|
|
||||||
|
|
||||||
先启动 RTR Server。TCP 模式是最常用的本地验证方式:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./deploy/server/build-run.sh up --mode tcp
|
|
||||||
```
|
|
||||||
|
|
||||||
如果要测试 TLS 或 SSH,需要先启动对应 server 模式,并准备匹配的证书或 SSH key:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./deploy/server/build-run.sh up --mode tls
|
|
||||||
./deploy/server/build-run.sh up --mode ssh
|
|
||||||
```
|
|
||||||
|
|
||||||
TLS/SSH 所需的本地开发证书和密钥可以通过脚本生成:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash scripts/generate-certs.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
已有 `certs/` 时使用 `--force` 覆盖。脚本生成的 `certs/tls` 和 `certs/ssh` 文件名与本仓库 compose 默认配置一致。
|
|
||||||
|
|
||||||
Client 连接地址不要求固定在某个 Docker 网络里。`deploy/client` 的 TCP/TLS/SSH 模式分别通过 `RPKI_RTR_TCP_SERVER_ADDR`、`RPKI_RTR_TLS_SERVER_ADDR`、`RPKI_RTR_SSH_SERVER_ADDR` 指定 server 地址。`deploy/bird` 通过 `RPKI_BIRD_RPKI_HOST`/`RPKI_BIRD_RPKI_PORT` 指定 TCP server 地址,通过 `RPKI_BIRD_SSH_RPKI_HOST`/`RPKI_RTR_SSH_PORT` 指定 SSH server 地址。示例 compose 里的默认值只是为了和本仓库 server compose 直接联调。
|
|
||||||
|
|
||||||
`deploy/frr` 示例默认连接 `127.0.0.1:323`,可以通过 `RPKI_FRR_RPKI_HOST`/`RPKI_FRR_RPKI_PORT` 覆盖。
|
|
||||||
|
|
||||||
## rtr_debug_client
|
|
||||||
|
|
||||||
`rtr_debug_client` 是本仓库提供的调试客户端。它会发起 RTR Reset Query 或 Serial Query,并打印 server 返回的 PDU 摘要。Docker 配置默认执行:
|
|
||||||
|
|
||||||
```text
|
|
||||||
<server_addr> <protocol_version> reset --keep-after-error --summary-only
|
|
||||||
```
|
|
||||||
|
|
||||||
### TCP
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./deploy/client/build-run.sh up --mode tcp
|
|
||||||
./deploy/client/build-run.sh logs --mode tcp
|
|
||||||
./deploy/client/build-run.sh down --mode tcp
|
|
||||||
```
|
|
||||||
|
|
||||||
默认连接 `rpki-rtr:323`,协议版本默认是 `2`。可以通过 `deploy/client/.env` 或环境变量覆盖:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
RPKI_RTR_TCP_SERVER_ADDR=10.0.0.12:323 \
|
|
||||||
RPKI_RTR_PROTOCOL_VERSION=2 \
|
|
||||||
./deploy/client/build-run.sh up --mode tcp
|
|
||||||
```
|
|
||||||
|
|
||||||
### TLS/mTLS
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./deploy/client/build-run.sh up --mode tls
|
|
||||||
./deploy/client/build-run.sh logs --mode tls
|
|
||||||
./deploy/client/build-run.sh down --mode tls
|
|
||||||
```
|
|
||||||
|
|
||||||
TLS 模式默认读取 `certs/tls` 下的示例证书,并使用 `localhost` 作为 server name。常用变量:
|
|
||||||
|
|
||||||
| 变量 | 默认值 | 说明 |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `RPKI_RTR_TLS_SERVER_ADDR` | `rpki-rtr:324` | TLS RTR server 地址 |
|
|
||||||
| `RPKI_RTR_TLS_SERVER_NAME` | `localhost` | 必须匹配 server 证书 SAN dNSName |
|
|
||||||
| `RPKI_RTR_TLS_CA_CERT_PATH` | `/app/certs/client-ca.crt` | 容器内 CA 证书路径 |
|
|
||||||
| `RPKI_RTR_TLS_CLIENT_CERT_PATH` | `/app/certs/client-good.crt` | 容器内 client 证书路径 |
|
|
||||||
| `RPKI_RTR_TLS_CLIENT_KEY_PATH` | `/app/certs/client-good.key` | 容器内 client 私钥路径 |
|
|
||||||
| `RPKI_RTR_TLS_CERTS_HOST_DIR` | `../../certs/tls` | 宿主机 TLS 证书目录 |
|
|
||||||
|
|
||||||
### SSH
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./deploy/client/build-run.sh up --mode ssh
|
|
||||||
./deploy/client/build-run.sh logs --mode ssh
|
|
||||||
./deploy/client/build-run.sh down --mode ssh
|
|
||||||
```
|
|
||||||
|
|
||||||
SSH 模式默认连接 `rpki-rtr-ssh:22`,使用 key 认证,并校验 server public key。常用变量:
|
|
||||||
|
|
||||||
| 变量 | 默认值 | 说明 |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `RPKI_RTR_SSH_SERVER_ADDR` | `rpki-rtr:22` | SSH RTR server 地址 |
|
|
||||||
| `RPKI_RTR_SSH_USERNAME` | `rpki-rtr` | SSH 用户名 |
|
|
||||||
| `RPKI_RTR_CLIENT_KEYS_VOLUME` | `../../certs/ssh:/app/certs:ro` | 宿主机 SSH key 目录挂载 |
|
|
||||||
| `RPKI_RTR_CLIENT_KEY_PATH` | `/app/certs/rtr-client.key` | 容器内 client 私钥 |
|
|
||||||
| `RPKI_RTR_SSH_SERVER_PUBKEY_PATH` | `/app/certs/ssh_host_rsa_key.pub` | 容器内 server public key |
|
|
||||||
|
|
||||||
### 多客户端压测
|
|
||||||
|
|
||||||
`--clients multi` 会启动 5 个 TCP debug client,用于观察并发连接、client report 和 server 连接统计。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./deploy/client/build-run.sh up --mode tcp --clients multi
|
|
||||||
./deploy/client/build-run.sh logs --mode tcp --clients multi
|
|
||||||
./deploy/client/build-run.sh down --mode tcp --clients multi
|
|
||||||
```
|
|
||||||
|
|
||||||
## FRR Client
|
|
||||||
|
|
||||||
FRR 示例用于验证标准路由软件能否通过 RTR over TCP 从 server 获取前缀验证数据。
|
|
||||||
|
|
||||||
启动:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
默认连接 `127.0.0.1:323`。可以通过环境变量覆盖:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
RPKI_FRR_RPKI_HOST=10.0.0.12 \
|
|
||||||
RPKI_FRR_RPKI_PORT=323 \
|
|
||||||
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
验证连接和数据:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker exec -it frr-rpki-client vtysh -c "show rpki configuration"
|
|
||||||
docker exec -it frr-rpki-client vtysh -c "show rpki cache-connection"
|
|
||||||
docker exec -it frr-rpki-client vtysh -c "show rpki prefix-table"
|
|
||||||
```
|
|
||||||
|
|
||||||
停止:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml down
|
|
||||||
```
|
|
||||||
|
|
||||||
FRR 容器启动时会读取 `deploy/frr/frr.conf.template`,并生成 `/etc/frr/frr.conf`。模板中的 RTR cache 配置为:
|
|
||||||
|
|
||||||
```text
|
|
||||||
rpki cache tcp ${RPKI_FRR_RPKI_HOST} ${RPKI_FRR_RPKI_PORT} preference ${RPKI_FRR_RPKI_PREFERENCE}
|
|
||||||
```
|
|
||||||
|
|
||||||
如果需要调整 polling、timeout、BGP router-id 等其他 FRR 配置,可以修改 `deploy/frr/frr.conf.template`,或在 compose 中挂载自己的模板和 entrypoint。
|
|
||||||
|
|
||||||
## BIRD Client
|
|
||||||
|
|
||||||
BIRD 示例用于验证 BIRD 3.x 对 RTR v2、ROA 和 ASPA 的导入情况。容器启动后会周期性输出 RPKI 协议状态和表项摘要。
|
|
||||||
|
|
||||||
### TCP
|
|
||||||
|
|
||||||
启动:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose --env-file deploy/bird/.env -f deploy/bird/docker-compose.yml up -d --build
|
|
||||||
```
|
|
||||||
|
|
||||||
查看日志:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker logs -f bird-rpki-client
|
|
||||||
```
|
|
||||||
|
|
||||||
停止:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose --env-file deploy/bird/.env -f deploy/bird/docker-compose.yml down
|
|
||||||
```
|
|
||||||
|
|
||||||
默认连接 `rpki-rtr:323`。可以通过环境变量覆盖:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
RPKI_BIRD_RPKI_HOST=10.0.0.12 \
|
|
||||||
RPKI_BIRD_RPKI_PORT=323 \
|
|
||||||
docker compose --env-file deploy/bird/.env -f deploy/bird/docker-compose.yml up -d --build
|
|
||||||
```
|
|
||||||
|
|
||||||
### SSH
|
|
||||||
|
|
||||||
先启动 server SSH 模式:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./deploy/server/build-run.sh up --mode ssh
|
|
||||||
```
|
|
||||||
|
|
||||||
再启动 BIRD SSH client:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose --env-file deploy/bird/.env \
|
|
||||||
-f deploy/bird/docker-compose.yml \
|
|
||||||
-f deploy/bird/docker-compose.ssh.yml \
|
|
||||||
up -d --build
|
|
||||||
```
|
|
||||||
|
|
||||||
查看日志:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker logs -f bird-rpki-client
|
|
||||||
```
|
|
||||||
|
|
||||||
SSH 模式默认读取 `certs/ssh` 目录下的 key,并使用 `bird.conf.ssh.template` 生成运行时配置。关键默认值:
|
|
||||||
|
|
||||||
| 变量 | 默认值 | 说明 |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `RPKI_BIRD_SSH_RPKI_HOST` | `rpki-rtr` | SSH RTR server 主机名 |
|
|
||||||
| `RPKI_RTR_SSH_PORT` | `22` | SSH RTR server 端口 |
|
|
||||||
| `RPKI_BIRD_SSH_CERTS_HOST_DIR` | `../../certs/ssh` | 宿主机 SSH 证书和 key 目录,复用 `rtr-client.key` |
|
|
||||||
|
|
||||||
## 如何判断 Client 正常
|
|
||||||
|
|
||||||
- `rtr_debug_client`: 日志中能看到 Reset Query 成功、EndOfData 和 payload 统计。
|
|
||||||
- FRR: `show rpki cache-connection` 显示已连接,`show rpki prefix-table` 有 VRP 条目。
|
|
||||||
- BIRD: 日志中 RPKI protocol 状态为 `up`,ROA/ASPA 表项摘要不为空。
|
|
||||||
|
|
||||||
如果 client 无法连接,优先检查:
|
|
||||||
|
|
||||||
- server 是否按对应 transport 启动。
|
|
||||||
- client 配置的 server 地址是否从当前容器或主机可达。
|
|
||||||
- TLS 的 server name 是否匹配证书 SAN。
|
|
||||||
- SSH 的用户名、client key 和 server public key 是否匹配。
|
|
||||||
- `RPKI_RTR_PROTOCOL_VERSION` 或 BIRD 配置中的 `min version`/`max version` 是否和 server 能力一致。
|
|
||||||
377
README.md
377
README.md
@@ -1,373 +1,28 @@
|
|||||||
# RPKI RTR Server
|
|
||||||
|
|
||||||
这是一个用 Rust 实现的 RPKI-to-Router (RTR) cache server。服务端从 CCR 文件中读取 RPKI 数据,生成 RTR cache snapshot 和 delta,并通过 RTR 协议提供给路由器或调试客户端。
|
|
||||||
|
|
||||||
当前代码重点覆盖:
|
# 单元测试
|
||||||
|
|
||||||
- RTR server:TCP 默认启用,可选启用 TLS/mTLS 和进程内 SSH transport。
|
```
|
||||||
- 数据输入:从 CCR 目录选择最新 `.ccr` 文件,解析 VRP 和 VAP/ASPA。
|
cargo test
|
||||||
- 本地策略:可选加载 SLURM 文件,对 CCR 解析出的 payload 做过滤和本地断言。
|
|
||||||
- 持久化:使用 RocksDB 保存 RTR cache 状态。
|
|
||||||
- 运行报告:输出 source、client、runtime 三类 JSON report。
|
|
||||||
- 运行期管理:可选启用 HTTP Admin API,修改部分 runtime config、触发 source reload、管理 SLURM 文件。
|
|
||||||
- 调试工具:`rtr_debug_client` 支持 TCP、TLS 和 SSH 连接 RTR server。
|
|
||||||
|
|
||||||
## 目录
|
# 查看输出
|
||||||
|
cargo test -- --nocapture
|
||||||
- [协议参考](#协议参考)
|
|
||||||
- [项目结构](#项目结构)
|
|
||||||
- [构建与测试](#构建与测试)
|
|
||||||
- [数据输入](#数据输入)
|
|
||||||
- [快速启动](#快速启动)
|
|
||||||
- [前置数据生成](#前置数据生成)
|
|
||||||
- [Docker 运行](#docker-运行)
|
|
||||||
- [跨架构构建](#跨架构构建)
|
|
||||||
- [传输模式](#传输模式)
|
|
||||||
- [主要环境变量](#主要环境变量)
|
|
||||||
- [运行报告](#运行报告)
|
|
||||||
- [Admin API](#admin-api)
|
|
||||||
- [调试客户端](#调试客户端)
|
|
||||||
- [部署入口](#部署入口)
|
|
||||||
- [开发说明](#开发说明)
|
|
||||||
|
|
||||||
## 协议参考
|
|
||||||
|
|
||||||
- RTR: [RFC 6810](https://www.rfc-editor.org/rfc/rfc6810.html), [RFC 8210](https://www.rfc-editor.org/rfc/rfc8210.html)
|
|
||||||
- SLURM: [RFC 8416](https://www.rfc-editor.org/rfc/rfc8416.html)
|
|
||||||
- CCR: [draft-ietf-sidrops-rpki-ccr](https://www.ietf.org/archive/id/draft-ietf-sidrops-rpki-ccr-02.html)
|
|
||||||
|
|
||||||
## 项目结构
|
|
||||||
|
|
||||||
| 路径 | 说明 |
|
|
||||||
| --- | --- |
|
|
||||||
| `src/main_rtr.rs` | RTR server 入口 |
|
|
||||||
| `src/rtr/` | RTR 协议、cache、store、server transport、admin、report |
|
|
||||||
| `src/source/` | CCR/SLURM source 加载流程 |
|
|
||||||
| `src/slurm/` | SLURM 文件解析、合并和策略应用 |
|
|
||||||
| `src/data_model/` | RPKI 相关数据结构 |
|
|
||||||
| `src/bin/rtr_debug_client/` | RTR 调试客户端 |
|
|
||||||
| `deploy/server/` | server Docker 镜像和 compose |
|
|
||||||
| `deploy/client/` | debug client Docker 镜像和 compose |
|
|
||||||
| `deploy/frr/`, `deploy/bird/` | FRR/BIRD 作为 RTR client 的测试部署 |
|
|
||||||
| `tests/` | 单元和集成测试 |
|
|
||||||
|
|
||||||
## 构建与测试
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo build --bin rpki_rtr
|
|
||||||
cargo build --bin rtr_debug_client
|
|
||||||
cargo test
|
|
||||||
```
|
```
|
||||||
|
|
||||||
服务端 binary 名称是 `rpki_rtr`:
|
# 覆盖率(cargo-llvm-cov)
|
||||||
|
|
||||||
```bash
|
安装工具:
|
||||||
cargo run --bin rpki_rtr
|
|
||||||
|
```
|
||||||
|
rustup component add llvm-tools-preview
|
||||||
|
cargo install cargo-llvm-cov --locked
|
||||||
```
|
```
|
||||||
|
|
||||||
默认配置会监听 `0.0.0.0:323`,从 `./data` 查找 CCR 文件,并使用 `./rtr-db` 保存 RocksDB 数据。启动前需要确保 CCR 目录里存在可解析的 `.ccr` 文件。
|
统计行覆盖率并要求 >=90%:
|
||||||
|
|
||||||
## 数据输入
|
```
|
||||||
|
./scripts/coverage.sh
|
||||||
服务端启动和刷新时会读取 `RPKI_RTR_CCR_DIR` 指向的目录:
|
# 或
|
||||||
|
cargo llvm-cov --fail-under-lines 90
|
||||||
1. 如果该目录下存在包含 `.ccr` 文件的子目录,先按子目录名排序选择最新子目录。
|
|
||||||
2. 在选中的目录中按文件名排序选择最新 `.ccr` 文件。
|
|
||||||
3. 从 CCR 中解析 VRP 和 VAP/ASPA。
|
|
||||||
4. 如果配置了 `RPKI_RTR_SLURM_DIR`,加载其中启用的 `*.slurm` 文件并应用本地策略。
|
|
||||||
|
|
||||||
## 快速启动
|
|
||||||
|
|
||||||
### 前置数据生成
|
|
||||||
|
|
||||||
TLS/mTLS 或 SSH transport 需要本地证书和密钥。开发和 Docker 联调可用脚本生成 compose 默认引用的最小文件集:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash scripts/10_generate-certs.sh
|
|
||||||
```
|
```
|
||||||
|
|
||||||
如需覆盖已有 `certs/` 内容:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash scripts/10_generate-certs.sh --force
|
|
||||||
```
|
|
||||||
|
|
||||||
`10_generate-certs.sh` 常用参数:
|
|
||||||
|
|
||||||
| 参数 | 说明 |
|
|
||||||
| --- | --- |
|
|
||||||
| `--out-dir DIR` | 指定输出目录,默认生成到 `./certs`。 |
|
|
||||||
| `--server-dns NAME` | 指定 TLS server 证书的 dNSName SAN,默认 `localhost`。 |
|
|
||||||
| `--tls-only` | 只生成 `certs/tls` 下的 TLS 证书和私钥。 |
|
|
||||||
| `--ssh-only` | 只生成 `certs/ssh` 下的 SSH host key、client key 和 `authorized_keys`。 |
|
|
||||||
| `--force` | 覆盖已有文件。未指定时,如果目标文件已存在会直接报错退出。 |
|
|
||||||
| `-h`, `--help` | 查看脚本帮助。 |
|
|
||||||
|
|
||||||
TLS server name 默认是 `localhost`。如果需要其它 DNS 名称,需要同时设置 `scripts/10_generate-certs.sh --server-dns <name>` 和 client 的 `RPKI_RTR_TLS_SERVER_NAME=<name>`。
|
|
||||||
|
|
||||||
### Docker 运行
|
|
||||||
|
|
||||||
推荐使用 `deploy/server/build-run.sh` 和 `deploy/client/build-run.sh` 分别管理 server 与 debug client:
|
|
||||||
以下命令默认从仓库根目录执行。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
chmod +x deploy/server/build-run.sh deploy/client/build-run.sh
|
|
||||||
./deploy/server/build-run.sh up --mode tcp
|
|
||||||
./deploy/client/build-run.sh up --mode tcp
|
|
||||||
./deploy/client/build-run.sh down
|
|
||||||
./deploy/server/build-run.sh down
|
|
||||||
```
|
|
||||||
|
|
||||||
脚本常用参数:
|
|
||||||
|
|
||||||
| 参数/变量 | server | client | 说明 |
|
|
||||||
| --- | --- | --- | --- |
|
|
||||||
| `--mode base|tcp|tls|ssh`, `MODE` | yes | yes | 选择 compose 模式,默认 `tcp`。 |
|
|
||||||
| `--clients single|multi`, `CLIENTS` | no | yes | 单 client 或 5 client,默认 `single`。 |
|
|
||||||
| `--no-build`, `BUILD=0` | yes | yes | 只运行已有镜像,不执行 compose build。 |
|
|
||||||
| `--server-image TAG`, `SERVER_IMAGE` | yes | no | server 镜像名;会传给 compose 的 `RPKI_RTR_SERVER_IMAGE`。 |
|
|
||||||
| `--client-image TAG`, `CLIENT_IMAGE` | no | yes | client 镜像名;会传给 compose 的 `RPKI_RTR_CLIENT_IMAGE`。 |
|
|
||||||
| `--platform VALUE`, `TARGET_PLATFORM` | yes | yes | `buildx` 目标平台,默认 `linux/arm64`。 |
|
|
||||||
| `--push`, `BUILDX_PUSH=1` | yes | yes | `buildx` 构建后推送到 registry;不设置时加载到本地 Docker。 |
|
|
||||||
| `--skip-base-pull`, `SKIP_BASE_IMAGE_PULL=1` | yes | yes | 跳过 buildx 前的基础镜像预拉取。 |
|
|
||||||
|
|
||||||
支持的 action 包括 `up`、`build`、`buildx`、`rebuild`、`down`/`stop`、`restart`、`logs`、`ps`。完整帮助见:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./deploy/server/build-run.sh --help
|
|
||||||
./deploy/client/build-run.sh --help
|
|
||||||
```
|
|
||||||
|
|
||||||
### 跨架构构建
|
|
||||||
示例:x86 开发机本地构建 ARM 镜像,打包后传到 ARM 部署机。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./deploy/server/build-run.sh buildx \
|
|
||||||
--platform linux/arm64 \
|
|
||||||
--server-image rpki-rtr:arm64
|
|
||||||
|
|
||||||
./deploy/client/build-run.sh buildx \
|
|
||||||
--platform linux/arm64 \
|
|
||||||
--client-image rpki-rtr-debug-client:arm64
|
|
||||||
|
|
||||||
mkdir -p dist/images
|
|
||||||
docker save rpki-rtr:arm64 -o dist/images/rpki-rtr-arm64.tar
|
|
||||||
docker save rpki-rtr-debug-client:arm64 -o dist/images/rpki-rtr-debug-client-arm64.tar
|
|
||||||
|
|
||||||
scp dist/images/*.tar user@arm-host:/tmp/
|
|
||||||
```
|
|
||||||
|
|
||||||
ARM 部署机加载镜像并运行:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker load -i /tmp/rpki-rtr-arm64.tar
|
|
||||||
docker load -i /tmp/rpki-rtr-debug-client-arm64.tar
|
|
||||||
|
|
||||||
SERVER_IMAGE=rpki-rtr:arm64 \
|
|
||||||
BUILD=0 ./deploy/server/build-run.sh up --mode tcp
|
|
||||||
|
|
||||||
CLIENT_IMAGE=rpki-rtr-debug-client:arm64 \
|
|
||||||
BUILD=0 ./deploy/client/build-run.sh up --mode tcp
|
|
||||||
```
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f deploy/server/docker-compose.yml up -d --build
|
|
||||||
docker compose -f deploy/server/docker-compose.yml down
|
|
||||||
```
|
|
||||||
|
|
||||||
TLS/mTLS 示例:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f deploy/server/docker-compose.tls.yml up -d --build
|
|
||||||
docker compose -f deploy/server/docker-compose.tls.yml down
|
|
||||||
```
|
|
||||||
|
|
||||||
## 传输模式
|
|
||||||
|
|
||||||
TCP 默认启用。TLS 和 SSH 是附加监听,不会替代 TCP。
|
|
||||||
|
|
||||||
| 模式 | 默认地址 | 启用方式 |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| TCP | `0.0.0.0:323` | 默认启用 |
|
|
||||||
| TLS/mTLS | `0.0.0.0:324` | `RPKI_RTR_ENABLE_TLS=true` |
|
|
||||||
| SSH | `0.0.0.0:22` | `RPKI_RTR_ENABLE_SSH=true` |
|
|
||||||
|
|
||||||
TLS server 会加载服务端证书、私钥和 client CA。SSH server 使用 OpenSSH host key 和 authorized_keys 文件;`RPKI_RTR_SSH_AUTH_MODE` 支持 `key`、`password`、`both`。
|
|
||||||
|
|
||||||
## 主要环境变量
|
|
||||||
|
|
||||||
布尔值支持 `true/false`、`1/0`、`yes/no`、`on/off`。
|
|
||||||
|
|
||||||
| 变量 | 默认值 | 说明 |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `RPKI_RTR_ENABLE_TLS` | `false` | 是否启用 TLS/mTLS 监听 |
|
|
||||||
| `RPKI_RTR_ENABLE_SSH` | `false` | 是否启用 SSH 监听 |
|
|
||||||
| `RPKI_RTR_TCP_ADDR` | `0.0.0.0:323` | TCP 监听地址 |
|
|
||||||
| `RPKI_RTR_TLS_ADDR` | `0.0.0.0:324` | TLS 监听地址 |
|
|
||||||
| `RPKI_RTR_SSH_ADDR` | `0.0.0.0:22` | SSH 监听地址 |
|
|
||||||
| `RPKI_RTR_SSH_PORT` | 未设置 | 仅覆盖 `RPKI_RTR_SSH_ADDR` 中的端口 |
|
|
||||||
| `RPKI_RTR_DB_PATH` | `./rtr-db` | RocksDB 路径 |
|
|
||||||
| `RPKI_RTR_CCR_DIR` | `./data` | CCR 输入目录 |
|
|
||||||
| `RPKI_RTR_SLURM_DIR` | 未设置 | SLURM 目录;空值表示禁用 |
|
|
||||||
| `RPKI_RTR_REPORT_DIR` | `./report` | JSON report 输出目录 |
|
|
||||||
| `RPKI_RTR_MAX_DELTA` | `100` | 每个 RTR 版本保留的最大 delta 数,必须大于 0 |
|
|
||||||
| `RPKI_RTR_PRUNE_DELTA_BY_SNAPSHOT_SIZE` | `false` | 是否按 snapshot wire size 继续裁剪 delta window |
|
|
||||||
| `RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS` | `300` | source 刷新间隔,单位秒 |
|
|
||||||
| `RPKI_RTR_REFRESH_INTERVAL_SECS` | 未设置 | 旧变量名,仍兼容;建议使用 `RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS` |
|
|
||||||
| `RPKI_RTR_TIMING_REFRESH_SECS` | `3600` | RTR EndOfData `refresh` 字段 |
|
|
||||||
| `RPKI_RTR_TIMING_RETRY_SECS` | `600` | RTR EndOfData `retry` 字段 |
|
|
||||||
| `RPKI_RTR_TIMING_EXPIRE_SECS` | `7200` | RTR EndOfData `expire` 字段,必须大于 refresh 和 retry |
|
|
||||||
| `RPKI_RTR_MAX_CONNECTIONS` | `512` | 最大并发 RTR 连接数 |
|
|
||||||
| `RPKI_RTR_MAX_CONCURRENT_HANDSHAKES` | `128` | 最大并发握手数,不能大于最大连接数 |
|
|
||||||
| `RPKI_RTR_NOTIFY_QUEUE_SIZE` | `1024` | Serial Notify 广播队列大小 |
|
|
||||||
| `RPKI_RTR_TCP_KEEPALIVE_SECS` | `60` | TCP keepalive;设为 `0` 表示禁用 |
|
|
||||||
| `RPKI_RTR_WARN_INSECURE_TCP` | `true` | TCP 模式是否输出安全提示 |
|
|
||||||
| `RPKI_RTR_REQUIRE_TLS_SERVER_DNS_NAME_SAN` | `false` | 是否要求 TLS 服务端证书包含 dNSName SAN |
|
|
||||||
| `RPKI_RTR_ENFORCE_TLS_CLIENT_SAN_IP_MATCH` | `true` | 是否校验 TLS client 证书 SAN IP 与 peer IP 匹配 |
|
|
||||||
| `RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS` | `300` | runtime report 周期 |
|
|
||||||
| `RPKI_RTR_REPORT_HISTORY_LIMIT` | `10` | 每类 report 滚动保留数量 |
|
|
||||||
| `RPKI_RTR_TIMEZONE` | `Asia/Shanghai` | 日志和 report 使用的时区 |
|
|
||||||
| `RPKI_RTR_ADMIN_ADDR` | 未设置 | Admin API 监听地址;未设置时关闭 |
|
|
||||||
| `RPKI_RTR_ADMIN_TOKEN` | 未设置 | Admin API Bearer token;非 loopback 监听时必须设置 |
|
|
||||||
|
|
||||||
TLS 相关路径:
|
|
||||||
|
|
||||||
| 变量 | 默认值 |
|
|
||||||
| --- | --- |
|
|
||||||
| `RPKI_RTR_TLS_CERT_PATH` | `./certs/tls/server-dns.crt` |
|
|
||||||
| `RPKI_RTR_TLS_KEY_PATH` | `./certs/tls/server-dns.key` |
|
|
||||||
| `RPKI_RTR_TLS_CLIENT_CA_PATH` | `./certs/tls/client-ca.crt` |
|
|
||||||
|
|
||||||
SSH 相关配置:
|
|
||||||
|
|
||||||
| 变量 | 默认值 |
|
|
||||||
| --- | --- |
|
|
||||||
| `RPKI_RTR_SSH_HOST_KEY_PATH` | `./certs/ssh/ssh_host_rsa_key` |
|
|
||||||
| `RPKI_RTR_SSH_AUTHORIZED_KEYS_PATH` | `./certs/ssh/rtr-authorized_keys` |
|
|
||||||
| `RPKI_RTR_SSH_USERNAME` | `rpki-rtr` |
|
|
||||||
| `RPKI_RTR_SSH_SUBSYSTEM_NAME` | `rpki-rtr` |
|
|
||||||
| `RPKI_RTR_SSH_AUTH_MODE` | `key` |
|
|
||||||
| `RPKI_RTR_SSH_PASSWORD` | 未设置 |
|
|
||||||
|
|
||||||
## 运行报告
|
|
||||||
|
|
||||||
服务会在 `RPKI_RTR_REPORT_DIR` 下写入 JSON report,并按 `RPKI_RTR_REPORT_HISTORY_LIMIT` 滚动保留:
|
|
||||||
|
|
||||||
- `rtr-source-*.json`:CCR/SLURM source、fingerprint、刷新状态、数据质量、cache 统计。
|
|
||||||
- `rtr-clients-*.json`:client 连接数和连接方式统计。
|
|
||||||
- `rtr-runtime-*.json`:进程状态、服务状态和当前生效的 runtime config。
|
|
||||||
|
|
||||||
## Admin API
|
|
||||||
|
|
||||||
Admin API 默认关闭。设置 `RPKI_RTR_ADMIN_ADDR` 后启用:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
export RPKI_RTR_ADMIN_ADDR=127.0.0.1:8323
|
|
||||||
export RPKI_RTR_ADMIN_TOKEN=change-me
|
|
||||||
```
|
|
||||||
|
|
||||||
如果监听非 loopback 地址,例如 `0.0.0.0:8323`,必须设置 `RPKI_RTR_ADMIN_TOKEN`。设置 token 后,请求需要携带:
|
|
||||||
|
|
||||||
```http
|
|
||||||
Authorization: Bearer change-me
|
|
||||||
```
|
|
||||||
|
|
||||||
当前实现的主要接口:
|
|
||||||
|
|
||||||
| 方法 | 路径 | 说明 |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `GET` | `/admin/rtr/health` | 健康检查和能力开关 |
|
|
||||||
| `GET` | `/admin/rtr/config` | 查看 runtime config |
|
|
||||||
| `POST` | `/admin/rtr/config` | 修改部分 runtime config |
|
|
||||||
| `GET` | `/admin/rtr/logs/tail` | 读取 stdout/stderr 日志 tail |
|
|
||||||
| `POST` | `/admin/rtr/slurm/reload` | 触发 source reload |
|
|
||||||
| `GET` | `/admin/rtr/slurm/files` | 列出 SLURM 文件 |
|
|
||||||
| `POST` | `/admin/rtr/slurm/files` | 新增或覆盖 SLURM 文件 |
|
|
||||||
| `GET` | `/admin/rtr/slurm/files/{name}` | 读取 SLURM 文件 |
|
|
||||||
| `PUT` | `/admin/rtr/slurm/files/{name}` | 新增或覆盖指定 SLURM 文件 |
|
|
||||||
| `DELETE` | `/admin/rtr/slurm/files/{name}` | 删除 SLURM 文件 |
|
|
||||||
| `POST` | `/admin/rtr/slurm/files/{name}/enable` | 启用 `.slurm.disabled` 文件 |
|
|
||||||
| `POST` | `/admin/rtr/slurm/files/{name}/disable` | 禁用 `.slurm` 文件 |
|
|
||||||
|
|
||||||
示例:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl http://127.0.0.1:8323/admin/rtr/config \
|
|
||||||
-H "Authorization: Bearer change-me"
|
|
||||||
|
|
||||||
curl -X POST http://127.0.0.1:8323/admin/rtr/config \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-H "Authorization: Bearer change-me" \
|
|
||||||
-d '{"max_delta": 6, "source_refresh_interval_seconds": 60}'
|
|
||||||
```
|
|
||||||
|
|
||||||
## 调试客户端
|
|
||||||
|
|
||||||
`rtr_debug_client` 用于手动发起 Reset Query 或 Serial Query,并观察服务端返回的 PDU。
|
|
||||||
|
|
||||||
RTR Client 的 Docker 调试、FRR/BIRD 互通验证和常用排障步骤见 `README.client.md`。
|
|
||||||
|
|
||||||
TCP:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo run --bin rtr_debug_client -- 127.0.0.1:323 1 reset
|
|
||||||
cargo run --bin rtr_debug_client -- 127.0.0.1:323 1 serial 42 100
|
|
||||||
```
|
|
||||||
|
|
||||||
TLS:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo run --bin rtr_debug_client -- \
|
|
||||||
127.0.0.1:324 1 reset \
|
|
||||||
--tls \
|
|
||||||
--ca-cert certs/tls/client-ca.crt \
|
|
||||||
--server-name localhost \
|
|
||||||
--client-cert certs/tls/client-good.crt \
|
|
||||||
--client-key certs/tls/client-good.key
|
|
||||||
```
|
|
||||||
|
|
||||||
SSH:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
cargo run --bin rtr_debug_client -- \
|
|
||||||
127.0.0.1:22 1 reset \
|
|
||||||
--ssh \
|
|
||||||
--ssh-user rpki-rtr \
|
|
||||||
--ssh-key certs/ssh/rtr-client.key \
|
|
||||||
--ssh-server-key certs/ssh/ssh_host_rsa_key.pub
|
|
||||||
```
|
|
||||||
|
|
||||||
SSH password auth 也已实现;SSH 模式下 `--ssh-key` 和 `--ssh-password` 二选一,同时必须通过 `--ssh-known-hosts` 或 `--ssh-server-key` 做 host key 校验。
|
|
||||||
|
|
||||||
更多参数见 `src/bin/rtr_debug_client/README.md`。
|
|
||||||
|
|
||||||
## 部署入口
|
|
||||||
|
|
||||||
| 用途 | Compose 文件 |
|
|
||||||
| --- | --- |
|
|
||||||
| Server | `deploy/server/docker-compose.yml` |
|
|
||||||
| Server TCP 示例 | `deploy/server/docker-compose.tcp.yml` |
|
|
||||||
| Server TLS 示例 | `deploy/server/docker-compose.tls.yml` |
|
|
||||||
| Server SSH 示例 | `deploy/server/docker-compose.ssh.yml` |
|
|
||||||
| Debug client | `deploy/client/docker-compose.yml` |
|
|
||||||
| Debug client 多实例 | `deploy/client/docker-compose.clients.yml` |
|
|
||||||
| FRR client | `deploy/frr/docker-compose.yml` |
|
|
||||||
| BIRD client | `deploy/bird/docker-compose.yml` |
|
|
||||||
|
|
||||||
RTR Client 统一使用说明见 `README.client.md`。
|
|
||||||
|
|
||||||
通用操作:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
COMPOSE_FILE=deploy/server/docker-compose.yml
|
|
||||||
docker compose -f "${COMPOSE_FILE}" up -d --build
|
|
||||||
docker compose -f "${COMPOSE_FILE}" ps
|
|
||||||
docker compose -f "${COMPOSE_FILE}" down
|
|
||||||
```
|
|
||||||
|
|
||||||
## 开发说明
|
|
||||||
|
|
||||||
- 代码中的 RTR cache 按协议版本维护状态;v0 只包含 Route Origin,v1 包含 Route Origin 和 Router Key,v2 包含 Route Origin、Router Key 和 ASPA。
|
|
||||||
- 当前 source pipeline 从 CCR 生成 Route Origin 和 ASPA,再可选应用 SLURM。
|
|
||||||
- `RtrStore::save_cache_state_versioned(...)` 是 cache 状态写入 RocksDB 的核心入口,相关边界测试在 `tests/test_store_boundary.rs`。
|
|
||||||
- Admin API 的日志 tail 依赖部署入口写出的 stdout/stderr 日志文件;容器入口脚本会把日志写到 `/app/logs`。
|
|
||||||
|
|||||||
8
benchmark/ours_manifest_bench/Cargo.toml
Normal file
8
benchmark/ours_manifest_bench/Cargo.toml
Normal file
@@ -0,0 +1,8 @@
|
|||||||
|
[package]
|
||||||
|
name = "ours-manifest-bench"
|
||||||
|
version = "0.1.0"
|
||||||
|
edition = "2024"
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
rpki = { path = "../..", default-features = false }
|
||||||
|
|
||||||
145
benchmark/ours_manifest_bench/src/main.rs
Normal file
145
benchmark/ours_manifest_bench/src/main.rs
Normal file
@@ -0,0 +1,145 @@
|
|||||||
|
use rpki::data_model::manifest::ManifestObject;
|
||||||
|
use std::hint::black_box;
|
||||||
|
use std::path::PathBuf;
|
||||||
|
use std::time::Instant;
|
||||||
|
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
struct Config {
|
||||||
|
sample: Option<String>,
|
||||||
|
manifest_path: Option<PathBuf>,
|
||||||
|
iterations: u64,
|
||||||
|
warmup_iterations: u64,
|
||||||
|
repeats: u32,
|
||||||
|
}
|
||||||
|
|
||||||
|
fn usage_and_exit() -> ! {
|
||||||
|
eprintln!(
|
||||||
|
"Usage:\n ours-manifest-bench (--sample <name> | --manifest <path>) [--iterations N] [--warmup-iterations N] [--repeats N]\n\nExamples:\n cargo run --release -- --sample small-01 --iterations 20000 --warmup-iterations 2000 --repeats 3\n cargo run --release -- --manifest ../../tests/benchmark/selected_der/small-01.mft"
|
||||||
|
);
|
||||||
|
std::process::exit(2);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn parse_args() -> Config {
|
||||||
|
let mut sample: Option<String> = None;
|
||||||
|
let mut manifest_path: Option<PathBuf> = None;
|
||||||
|
let mut iterations: u64 = 20_000;
|
||||||
|
let mut warmup_iterations: u64 = 2_000;
|
||||||
|
let mut repeats: u32 = 3;
|
||||||
|
|
||||||
|
let mut args = std::env::args().skip(1);
|
||||||
|
while let Some(arg) = args.next() {
|
||||||
|
match arg.as_str() {
|
||||||
|
"--sample" => sample = Some(args.next().unwrap_or_else(|| usage_and_exit())),
|
||||||
|
"--manifest" => {
|
||||||
|
manifest_path = Some(PathBuf::from(args.next().unwrap_or_else(|| usage_and_exit())))
|
||||||
|
}
|
||||||
|
"--iterations" => {
|
||||||
|
iterations = args
|
||||||
|
.next()
|
||||||
|
.unwrap_or_else(|| usage_and_exit())
|
||||||
|
.parse()
|
||||||
|
.unwrap_or_else(|_| usage_and_exit())
|
||||||
|
}
|
||||||
|
"--warmup-iterations" => {
|
||||||
|
warmup_iterations = args
|
||||||
|
.next()
|
||||||
|
.unwrap_or_else(|| usage_and_exit())
|
||||||
|
.parse()
|
||||||
|
.unwrap_or_else(|_| usage_and_exit())
|
||||||
|
}
|
||||||
|
"--repeats" => {
|
||||||
|
repeats = args
|
||||||
|
.next()
|
||||||
|
.unwrap_or_else(|| usage_and_exit())
|
||||||
|
.parse()
|
||||||
|
.unwrap_or_else(|_| usage_and_exit())
|
||||||
|
}
|
||||||
|
"-h" | "--help" => usage_and_exit(),
|
||||||
|
_ => usage_and_exit(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if sample.is_none() && manifest_path.is_none() {
|
||||||
|
usage_and_exit();
|
||||||
|
}
|
||||||
|
if sample.is_some() && manifest_path.is_some() {
|
||||||
|
usage_and_exit();
|
||||||
|
}
|
||||||
|
|
||||||
|
Config {
|
||||||
|
sample,
|
||||||
|
manifest_path,
|
||||||
|
iterations,
|
||||||
|
warmup_iterations,
|
||||||
|
repeats,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn derive_manifest_path(sample: &str) -> PathBuf {
|
||||||
|
// Assumes current working directory is `rpki/benchmark/ours_manifest_bench`.
|
||||||
|
PathBuf::from(format!("../../tests/benchmark/selected_der/{sample}.mft"))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn main() {
|
||||||
|
let cfg = parse_args();
|
||||||
|
let manifest_path = cfg
|
||||||
|
.manifest_path
|
||||||
|
.clone()
|
||||||
|
.unwrap_or_else(|| derive_manifest_path(cfg.sample.as_deref().unwrap()));
|
||||||
|
|
||||||
|
let bytes = std::fs::read(&manifest_path).unwrap_or_else(|e| {
|
||||||
|
eprintln!("read manifest fixture failed: {e}; path={}", manifest_path.display());
|
||||||
|
std::process::exit(1);
|
||||||
|
});
|
||||||
|
|
||||||
|
let decoded_once = ManifestObject::decode_der(&bytes).unwrap_or_else(|e| {
|
||||||
|
eprintln!("decode failed: {e}; path={}", manifest_path.display());
|
||||||
|
std::process::exit(1);
|
||||||
|
});
|
||||||
|
let file_count = decoded_once.manifest.file_count();
|
||||||
|
|
||||||
|
let mut round_ns_per_op: Vec<f64> = Vec::with_capacity(cfg.repeats as usize);
|
||||||
|
let mut round_ops_per_s: Vec<f64> = Vec::with_capacity(cfg.repeats as usize);
|
||||||
|
|
||||||
|
for _round in 0..cfg.repeats {
|
||||||
|
for _ in 0..cfg.warmup_iterations {
|
||||||
|
let obj = ManifestObject::decode_der(black_box(&bytes)).expect("warmup decode");
|
||||||
|
black_box(obj);
|
||||||
|
}
|
||||||
|
|
||||||
|
let start = Instant::now();
|
||||||
|
for _ in 0..cfg.iterations {
|
||||||
|
let obj = ManifestObject::decode_der(black_box(&bytes)).expect("timed decode");
|
||||||
|
black_box(obj);
|
||||||
|
}
|
||||||
|
let elapsed = start.elapsed();
|
||||||
|
|
||||||
|
let ns_per_op = (elapsed.as_secs_f64() * 1e9) / (cfg.iterations as f64);
|
||||||
|
let ops_per_s = (cfg.iterations as f64) / elapsed.as_secs_f64();
|
||||||
|
round_ns_per_op.push(ns_per_op);
|
||||||
|
round_ops_per_s.push(ops_per_s);
|
||||||
|
}
|
||||||
|
|
||||||
|
let avg_ns_per_op = round_ns_per_op.iter().sum::<f64>() / (round_ns_per_op.len() as f64);
|
||||||
|
let avg_ops_per_s = round_ops_per_s.iter().sum::<f64>() / (round_ops_per_s.len() as f64);
|
||||||
|
|
||||||
|
let sample_name = cfg.sample.clone().unwrap_or_else(|| {
|
||||||
|
manifest_path
|
||||||
|
.file_name()
|
||||||
|
.map(|s| s.to_string_lossy().to_string())
|
||||||
|
.unwrap_or_else(|| manifest_path.display().to_string())
|
||||||
|
});
|
||||||
|
let sample_name = sample_name
|
||||||
|
.strip_suffix(".mft")
|
||||||
|
.unwrap_or(&sample_name)
|
||||||
|
.to_string();
|
||||||
|
|
||||||
|
println!("fixture: {}", manifest_path.display());
|
||||||
|
println!();
|
||||||
|
println!("| sample | avg ns/op | ops/s | file count |");
|
||||||
|
println!("|---|---:|---:|---:|");
|
||||||
|
println!(
|
||||||
|
"| {} | {:.2} | {:.2} | {} |",
|
||||||
|
sample_name, avg_ns_per_op, avg_ops_per_s, file_count
|
||||||
|
);
|
||||||
|
}
|
||||||
8
benchmark/routinator_object_bench/Cargo.toml
Normal file
8
benchmark/routinator_object_bench/Cargo.toml
Normal file
@@ -0,0 +1,8 @@
|
|||||||
|
[package]
|
||||||
|
name = "routinator-object-bench"
|
||||||
|
version = "0.1.0"
|
||||||
|
edition = "2024"
|
||||||
|
publish = false
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
rpki = { version = "=0.19.1", features = ["repository"] }
|
||||||
552
benchmark/routinator_object_bench/src/main.rs
Normal file
552
benchmark/routinator_object_bench/src/main.rs
Normal file
@@ -0,0 +1,552 @@
|
|||||||
|
use rpki::repository::cert::Cert;
|
||||||
|
use rpki::repository::crl::Crl;
|
||||||
|
use rpki::repository::manifest::Manifest;
|
||||||
|
use rpki::repository::roa::Roa;
|
||||||
|
use rpki::repository::aspa::Aspa;
|
||||||
|
use rpki::repository::resources::{AsResources, IpResources};
|
||||||
|
|
||||||
|
use std::hint::black_box;
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
use std::time::Instant;
|
||||||
|
|
||||||
|
#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)]
|
||||||
|
enum ObjType {
|
||||||
|
Cer,
|
||||||
|
Crl,
|
||||||
|
Manifest,
|
||||||
|
Roa,
|
||||||
|
Aspa,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl ObjType {
|
||||||
|
fn parse(s: &str) -> Result<Self, String> {
|
||||||
|
match s {
|
||||||
|
"cer" => Ok(Self::Cer),
|
||||||
|
"crl" => Ok(Self::Crl),
|
||||||
|
"manifest" => Ok(Self::Manifest),
|
||||||
|
"roa" => Ok(Self::Roa),
|
||||||
|
"aspa" => Ok(Self::Aspa),
|
||||||
|
_ => Err("type must be one of: cer, crl, manifest, roa, aspa".into()),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn as_str(self) -> &'static str {
|
||||||
|
match self {
|
||||||
|
ObjType::Cer => "cer",
|
||||||
|
ObjType::Crl => "crl",
|
||||||
|
ObjType::Manifest => "manifest",
|
||||||
|
ObjType::Roa => "roa",
|
||||||
|
ObjType::Aspa => "aspa",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn ext(self) -> &'static str {
|
||||||
|
match self {
|
||||||
|
ObjType::Cer => "cer",
|
||||||
|
ObjType::Crl => "crl",
|
||||||
|
ObjType::Manifest => "mft",
|
||||||
|
ObjType::Roa => "roa",
|
||||||
|
ObjType::Aspa => "asa",
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Clone, Debug)]
|
||||||
|
struct Sample {
|
||||||
|
obj_type: ObjType,
|
||||||
|
name: String,
|
||||||
|
path: PathBuf,
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Clone, Debug)]
|
||||||
|
struct Config {
|
||||||
|
dir: PathBuf,
|
||||||
|
type_filter: Option<ObjType>,
|
||||||
|
sample_filter: Option<String>,
|
||||||
|
fixed_iters: Option<u64>,
|
||||||
|
warmup_iters: u64,
|
||||||
|
rounds: u64,
|
||||||
|
min_round_ms: u64,
|
||||||
|
max_adaptive_iters: u64,
|
||||||
|
strict: bool,
|
||||||
|
cert_inspect: bool,
|
||||||
|
out_csv: Option<PathBuf>,
|
||||||
|
out_md: Option<PathBuf>,
|
||||||
|
}
|
||||||
|
|
||||||
|
fn usage_and_exit(err: Option<&str>) -> ! {
|
||||||
|
if let Some(err) = err {
|
||||||
|
eprintln!("error: {err}");
|
||||||
|
eprintln!();
|
||||||
|
}
|
||||||
|
eprintln!(
|
||||||
|
"Usage:\n\
|
||||||
|
cargo run --release --manifest-path rpki/benchmark/routinator_object_bench/Cargo.toml -- [OPTIONS]\n\
|
||||||
|
\n\
|
||||||
|
Options:\n\
|
||||||
|
--dir <PATH> Fixtures root dir (default: ../../tests/benchmark/selected_der_v2)\n\
|
||||||
|
--type <cer|crl|manifest|roa|aspa> Filter by type\n\
|
||||||
|
--sample <NAME> Filter by sample name (e.g. p50)\n\
|
||||||
|
--iters <N> Fixed iterations per round (optional; otherwise adaptive)\n\
|
||||||
|
--warmup-iters <N> Warmup iterations (default: 50)\n\
|
||||||
|
--rounds <N> Rounds (default: 5)\n\
|
||||||
|
--min-round-ms <MS> Adaptive: minimum round time (default: 200)\n\
|
||||||
|
--max-iters <N> Adaptive: maximum iters (default: 1_000_000)\n\
|
||||||
|
--strict <true|false> Strict DER where applicable (default: true)\n\
|
||||||
|
--cert-inspect Also run Cert::inspect_ca/inspect_ee where applicable (default: false)\n\
|
||||||
|
--out-csv <PATH> Write CSV output\n\
|
||||||
|
--out-md <PATH> Write Markdown output\n\
|
||||||
|
"
|
||||||
|
);
|
||||||
|
std::process::exit(2);
|
||||||
|
}
|
||||||
|
|
||||||
|
fn parse_bool(s: &str, name: &str) -> bool {
|
||||||
|
match s {
|
||||||
|
"1" | "true" | "TRUE" | "yes" | "YES" => true,
|
||||||
|
"0" | "false" | "FALSE" | "no" | "NO" => false,
|
||||||
|
_ => usage_and_exit(Some(&format!("{name} must be true/false"))),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn parse_u64(s: &str, name: &str) -> u64 {
|
||||||
|
s.parse::<u64>()
|
||||||
|
.unwrap_or_else(|_| usage_and_exit(Some(&format!("{name} must be an integer"))))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn default_samples_dir() -> PathBuf {
|
||||||
|
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../tests/benchmark/selected_der_v2")
|
||||||
|
}
|
||||||
|
|
||||||
|
fn parse_args() -> Config {
|
||||||
|
let mut dir: PathBuf = default_samples_dir();
|
||||||
|
let mut type_filter: Option<ObjType> = None;
|
||||||
|
let mut sample_filter: Option<String> = None;
|
||||||
|
let mut fixed_iters: Option<u64> = None;
|
||||||
|
let mut warmup_iters: u64 = 50;
|
||||||
|
let mut rounds: u64 = 5;
|
||||||
|
let mut min_round_ms: u64 = 200;
|
||||||
|
let mut max_adaptive_iters: u64 = 1_000_000;
|
||||||
|
let mut strict: bool = true;
|
||||||
|
let mut cert_inspect: bool = false;
|
||||||
|
let mut out_csv: Option<PathBuf> = None;
|
||||||
|
let mut out_md: Option<PathBuf> = None;
|
||||||
|
|
||||||
|
let mut args = std::env::args().skip(1);
|
||||||
|
while let Some(arg) = args.next() {
|
||||||
|
match arg.as_str() {
|
||||||
|
"--dir" => dir = PathBuf::from(args.next().unwrap_or_else(|| usage_and_exit(None))),
|
||||||
|
"--type" => {
|
||||||
|
type_filter = Some(ObjType::parse(
|
||||||
|
&args.next().unwrap_or_else(|| usage_and_exit(None)),
|
||||||
|
)
|
||||||
|
.unwrap_or_else(|e| usage_and_exit(Some(&e))))
|
||||||
|
}
|
||||||
|
"--sample" => {
|
||||||
|
sample_filter = Some(args.next().unwrap_or_else(|| usage_and_exit(None)))
|
||||||
|
}
|
||||||
|
"--iters" => {
|
||||||
|
fixed_iters = Some(parse_u64(
|
||||||
|
&args.next().unwrap_or_else(|| usage_and_exit(None)),
|
||||||
|
"--iters",
|
||||||
|
))
|
||||||
|
}
|
||||||
|
"--warmup-iters" => {
|
||||||
|
warmup_iters = parse_u64(
|
||||||
|
&args.next().unwrap_or_else(|| usage_and_exit(None)),
|
||||||
|
"--warmup-iters",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
"--rounds" => {
|
||||||
|
rounds = parse_u64(&args.next().unwrap_or_else(|| usage_and_exit(None)), "--rounds")
|
||||||
|
}
|
||||||
|
"--min-round-ms" => {
|
||||||
|
min_round_ms = parse_u64(
|
||||||
|
&args.next().unwrap_or_else(|| usage_and_exit(None)),
|
||||||
|
"--min-round-ms",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
"--max-iters" => {
|
||||||
|
max_adaptive_iters = parse_u64(
|
||||||
|
&args.next().unwrap_or_else(|| usage_and_exit(None)),
|
||||||
|
"--max-iters",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
"--strict" => {
|
||||||
|
strict = parse_bool(
|
||||||
|
&args.next().unwrap_or_else(|| usage_and_exit(None)),
|
||||||
|
"--strict",
|
||||||
|
)
|
||||||
|
}
|
||||||
|
"--cert-inspect" => cert_inspect = true,
|
||||||
|
"--out-csv" => out_csv = Some(PathBuf::from(args.next().unwrap_or_else(|| usage_and_exit(None)))),
|
||||||
|
"--out-md" => out_md = Some(PathBuf::from(args.next().unwrap_or_else(|| usage_and_exit(None)))),
|
||||||
|
"-h" | "--help" => usage_and_exit(None),
|
||||||
|
_ => usage_and_exit(Some(&format!("unknown argument: {arg}"))),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if warmup_iters == 0 {
|
||||||
|
usage_and_exit(Some("--warmup-iters must be > 0"));
|
||||||
|
}
|
||||||
|
if rounds == 0 {
|
||||||
|
usage_and_exit(Some("--rounds must be > 0"));
|
||||||
|
}
|
||||||
|
if min_round_ms == 0 {
|
||||||
|
usage_and_exit(Some("--min-round-ms must be > 0"));
|
||||||
|
}
|
||||||
|
if max_adaptive_iters == 0 {
|
||||||
|
usage_and_exit(Some("--max-iters must be > 0"));
|
||||||
|
}
|
||||||
|
if let Some(n) = fixed_iters {
|
||||||
|
if n == 0 {
|
||||||
|
usage_and_exit(Some("--iters must be > 0"));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Config {
|
||||||
|
dir,
|
||||||
|
type_filter,
|
||||||
|
sample_filter,
|
||||||
|
fixed_iters,
|
||||||
|
warmup_iters,
|
||||||
|
rounds,
|
||||||
|
min_round_ms,
|
||||||
|
max_adaptive_iters,
|
||||||
|
strict,
|
||||||
|
cert_inspect,
|
||||||
|
out_csv,
|
||||||
|
out_md,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn read_samples(root: &Path) -> Vec<Sample> {
|
||||||
|
let mut out = Vec::new();
|
||||||
|
for obj_type in [
|
||||||
|
ObjType::Cer,
|
||||||
|
ObjType::Crl,
|
||||||
|
ObjType::Manifest,
|
||||||
|
ObjType::Roa,
|
||||||
|
ObjType::Aspa,
|
||||||
|
] {
|
||||||
|
let dir = root.join(obj_type.as_str());
|
||||||
|
let rd = match std::fs::read_dir(&dir) {
|
||||||
|
Ok(rd) => rd,
|
||||||
|
Err(_) => continue,
|
||||||
|
};
|
||||||
|
for ent in rd.flatten() {
|
||||||
|
let path = ent.path();
|
||||||
|
if path.extension().and_then(|s| s.to_str()) != Some(obj_type.ext()) {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
let name = path
|
||||||
|
.file_stem()
|
||||||
|
.and_then(|s| s.to_str())
|
||||||
|
.unwrap_or("unknown")
|
||||||
|
.to_string();
|
||||||
|
out.push(Sample { obj_type, name, path });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
out.sort_by(|a, b| a.obj_type.cmp(&b.obj_type).then_with(|| a.name.cmp(&b.name)));
|
||||||
|
out
|
||||||
|
}
|
||||||
|
|
||||||
|
fn choose_iters_adaptive<F: FnMut()>(mut op: F, min_round_ms: u64, max_iters: u64) -> u64 {
|
||||||
|
let min_secs = (min_round_ms as f64) / 1e3;
|
||||||
|
let mut iters: u64 = 1;
|
||||||
|
loop {
|
||||||
|
let start = Instant::now();
|
||||||
|
for _ in 0..iters {
|
||||||
|
op();
|
||||||
|
}
|
||||||
|
let elapsed = start.elapsed().as_secs_f64();
|
||||||
|
if elapsed >= min_secs {
|
||||||
|
return iters;
|
||||||
|
}
|
||||||
|
if iters >= max_iters {
|
||||||
|
return iters;
|
||||||
|
}
|
||||||
|
iters = (iters.saturating_mul(2)).min(max_iters);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn count_ip(res: &IpResources) -> u64 {
|
||||||
|
if res.is_inherited() {
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
let Ok(blocks) = res.to_blocks() else {
|
||||||
|
return 0;
|
||||||
|
};
|
||||||
|
blocks.iter().count() as u64
|
||||||
|
}
|
||||||
|
|
||||||
|
fn count_as(res: &AsResources) -> u64 {
|
||||||
|
if res.is_inherited() {
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
let Ok(blocks) = res.to_blocks() else {
|
||||||
|
return 0;
|
||||||
|
};
|
||||||
|
blocks.iter().count() as u64
|
||||||
|
}
|
||||||
|
|
||||||
|
fn complexity(obj_type: ObjType, bytes: &[u8], strict: bool, cert_inspect: bool) -> u64 {
|
||||||
|
match obj_type {
|
||||||
|
ObjType::Cer => {
|
||||||
|
let cert = Cert::decode(bytes).expect("decode cert");
|
||||||
|
if cert_inspect {
|
||||||
|
if cert.is_ca() {
|
||||||
|
cert.inspect_ca(strict).expect("inspect ca");
|
||||||
|
} else {
|
||||||
|
cert.inspect_ee(strict).expect("inspect ee");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
count_ip(cert.v4_resources())
|
||||||
|
.saturating_add(count_ip(cert.v6_resources()))
|
||||||
|
.saturating_add(count_as(cert.as_resources()))
|
||||||
|
}
|
||||||
|
ObjType::Crl => {
|
||||||
|
let crl = Crl::decode(bytes).expect("decode crl");
|
||||||
|
crl.revoked_certs().iter().count() as u64
|
||||||
|
}
|
||||||
|
ObjType::Manifest => {
|
||||||
|
let mft = Manifest::decode(bytes, strict).expect("decode manifest");
|
||||||
|
if cert_inspect {
|
||||||
|
mft.cert().inspect_ee(strict).expect("inspect ee");
|
||||||
|
}
|
||||||
|
mft.content().len() as u64
|
||||||
|
}
|
||||||
|
ObjType::Roa => {
|
||||||
|
let roa = Roa::decode(bytes, strict).expect("decode roa");
|
||||||
|
if cert_inspect {
|
||||||
|
roa.cert().inspect_ee(strict).expect("inspect ee");
|
||||||
|
}
|
||||||
|
roa.content().iter().count() as u64
|
||||||
|
}
|
||||||
|
ObjType::Aspa => {
|
||||||
|
let asa = Aspa::decode(bytes, strict).expect("decode aspa");
|
||||||
|
if cert_inspect {
|
||||||
|
asa.cert().inspect_ee(strict).expect("inspect ee");
|
||||||
|
}
|
||||||
|
asa.content().provider_as_set().len() as u64
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn decode_profile(obj_type: ObjType, bytes: &[u8], strict: bool, cert_inspect: bool) {
|
||||||
|
match obj_type {
|
||||||
|
ObjType::Cer => {
|
||||||
|
let cert = Cert::decode(black_box(bytes)).expect("decode cert");
|
||||||
|
if cert_inspect {
|
||||||
|
if cert.is_ca() {
|
||||||
|
cert.inspect_ca(strict).expect("inspect ca");
|
||||||
|
} else {
|
||||||
|
cert.inspect_ee(strict).expect("inspect ee");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
black_box(cert);
|
||||||
|
}
|
||||||
|
ObjType::Crl => {
|
||||||
|
let crl = Crl::decode(black_box(bytes)).expect("decode crl");
|
||||||
|
black_box(crl);
|
||||||
|
}
|
||||||
|
ObjType::Manifest => {
|
||||||
|
let mft = Manifest::decode(black_box(bytes), strict).expect("decode manifest");
|
||||||
|
if cert_inspect {
|
||||||
|
mft.cert().inspect_ee(strict).expect("inspect ee");
|
||||||
|
}
|
||||||
|
black_box(mft);
|
||||||
|
}
|
||||||
|
ObjType::Roa => {
|
||||||
|
let roa = Roa::decode(black_box(bytes), strict).expect("decode roa");
|
||||||
|
if cert_inspect {
|
||||||
|
roa.cert().inspect_ee(strict).expect("inspect ee");
|
||||||
|
}
|
||||||
|
black_box(roa);
|
||||||
|
}
|
||||||
|
ObjType::Aspa => {
|
||||||
|
let asa = Aspa::decode(black_box(bytes), strict).expect("decode aspa");
|
||||||
|
if cert_inspect {
|
||||||
|
asa.cert().inspect_ee(strict).expect("inspect ee");
|
||||||
|
}
|
||||||
|
black_box(asa);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[derive(Clone, Debug)]
|
||||||
|
struct ResultRow {
|
||||||
|
obj_type: String,
|
||||||
|
sample: String,
|
||||||
|
size_bytes: usize,
|
||||||
|
complexity: u64,
|
||||||
|
avg_ns_per_op: f64,
|
||||||
|
ops_per_sec: f64,
|
||||||
|
}
|
||||||
|
|
||||||
|
fn render_markdown(title: &str, rows: &[ResultRow]) -> String {
|
||||||
|
let mut out = String::new();
|
||||||
|
out.push_str(&format!("# {title}\n\n"));
|
||||||
|
out.push_str("| type | sample | size_bytes | complexity | avg ns/op | ops/s |\n");
|
||||||
|
out.push_str("|---|---|---:|---:|---:|---:|\n");
|
||||||
|
for r in rows {
|
||||||
|
out.push_str(&format!(
|
||||||
|
"| {} | {} | {} | {} | {:.2} | {:.2} |\n",
|
||||||
|
r.obj_type, r.sample, r.size_bytes, r.complexity, r.avg_ns_per_op, r.ops_per_sec
|
||||||
|
));
|
||||||
|
}
|
||||||
|
out
|
||||||
|
}
|
||||||
|
|
||||||
|
fn render_csv(rows: &[ResultRow]) -> String {
|
||||||
|
let mut out = String::new();
|
||||||
|
out.push_str("type,sample,size_bytes,complexity,avg_ns_per_op,ops_per_sec\n");
|
||||||
|
for r in rows {
|
||||||
|
let sample = r.sample.replace('"', "\"\"");
|
||||||
|
out.push_str(&format!(
|
||||||
|
"{},{},{},{},{:.6},{:.6}\n",
|
||||||
|
r.obj_type,
|
||||||
|
format!("\"{}\"", sample),
|
||||||
|
r.size_bytes,
|
||||||
|
r.complexity,
|
||||||
|
r.avg_ns_per_op,
|
||||||
|
r.ops_per_sec
|
||||||
|
));
|
||||||
|
}
|
||||||
|
out
|
||||||
|
}
|
||||||
|
|
||||||
|
fn create_parent_dirs(path: &Path) {
|
||||||
|
if let Some(parent) = path.parent() {
|
||||||
|
std::fs::create_dir_all(parent).unwrap_or_else(|e| {
|
||||||
|
panic!("create_dir_all {}: {e}", parent.display());
|
||||||
|
});
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn write_text_file(path: &Path, content: &str) {
|
||||||
|
create_parent_dirs(path);
|
||||||
|
std::fs::write(path, content).unwrap_or_else(|e| panic!("write {}: {e}", path.display()));
|
||||||
|
}
|
||||||
|
|
||||||
|
fn main() {
|
||||||
|
let cfg = parse_args();
|
||||||
|
let mut samples = read_samples(&cfg.dir);
|
||||||
|
if samples.is_empty() {
|
||||||
|
usage_and_exit(Some(&format!(
|
||||||
|
"no samples found under: {}",
|
||||||
|
cfg.dir.display()
|
||||||
|
)));
|
||||||
|
}
|
||||||
|
|
||||||
|
if let Some(t) = cfg.type_filter {
|
||||||
|
samples.retain(|s| s.obj_type == t);
|
||||||
|
if samples.is_empty() {
|
||||||
|
usage_and_exit(Some(&format!("no sample matched --type {}", t.as_str())));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if let Some(filter) = cfg.sample_filter.as_deref() {
|
||||||
|
samples.retain(|s| s.name == filter);
|
||||||
|
if samples.is_empty() {
|
||||||
|
usage_and_exit(Some(&format!("no sample matched --sample {filter}")));
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
println!("# Routinator baseline (rpki crate) decode benchmark (selected_der_v2)");
|
||||||
|
println!();
|
||||||
|
println!("- dir: {}", cfg.dir.display());
|
||||||
|
println!("- strict: {}", cfg.strict);
|
||||||
|
println!("- cert_inspect: {}", cfg.cert_inspect);
|
||||||
|
if let Some(t) = cfg.type_filter {
|
||||||
|
println!("- type: {}", t.as_str());
|
||||||
|
}
|
||||||
|
if let Some(s) = cfg.sample_filter.as_deref() {
|
||||||
|
println!("- sample: {}", s);
|
||||||
|
}
|
||||||
|
if let Some(n) = cfg.fixed_iters {
|
||||||
|
println!("- iters: {} (fixed)", n);
|
||||||
|
} else {
|
||||||
|
println!(
|
||||||
|
"- warmup: {} iters, rounds: {}, min_round: {}ms (adaptive iters, max {})",
|
||||||
|
cfg.warmup_iters, cfg.rounds, cfg.min_round_ms, cfg.max_adaptive_iters
|
||||||
|
);
|
||||||
|
}
|
||||||
|
if let Some(p) = cfg.out_csv.as_ref() {
|
||||||
|
println!("- out_csv: {}", p.display());
|
||||||
|
}
|
||||||
|
if let Some(p) = cfg.out_md.as_ref() {
|
||||||
|
println!("- out_md: {}", p.display());
|
||||||
|
}
|
||||||
|
println!();
|
||||||
|
|
||||||
|
println!("| type | sample | size_bytes | complexity | avg ns/op | ops/s |");
|
||||||
|
println!("|---|---|---:|---:|---:|---:|");
|
||||||
|
|
||||||
|
let mut rows: Vec<ResultRow> = Vec::with_capacity(samples.len());
|
||||||
|
for sample in &samples {
|
||||||
|
let bytes = std::fs::read(&sample.path)
|
||||||
|
.unwrap_or_else(|e| panic!("read {}: {e}", sample.path.display()));
|
||||||
|
let size_bytes = bytes.len();
|
||||||
|
let complexity = complexity(sample.obj_type, bytes.as_slice(), cfg.strict, cfg.cert_inspect);
|
||||||
|
|
||||||
|
for _ in 0..cfg.warmup_iters {
|
||||||
|
decode_profile(sample.obj_type, bytes.as_slice(), cfg.strict, cfg.cert_inspect);
|
||||||
|
}
|
||||||
|
|
||||||
|
let mut per_round_ns_per_op = Vec::with_capacity(cfg.rounds as usize);
|
||||||
|
for _round in 0..cfg.rounds {
|
||||||
|
let iters = if let Some(n) = cfg.fixed_iters {
|
||||||
|
n
|
||||||
|
} else {
|
||||||
|
choose_iters_adaptive(
|
||||||
|
|| decode_profile(sample.obj_type, bytes.as_slice(), cfg.strict, cfg.cert_inspect),
|
||||||
|
cfg.min_round_ms,
|
||||||
|
cfg.max_adaptive_iters,
|
||||||
|
)
|
||||||
|
};
|
||||||
|
let start = Instant::now();
|
||||||
|
for _ in 0..iters {
|
||||||
|
decode_profile(sample.obj_type, bytes.as_slice(), cfg.strict, cfg.cert_inspect);
|
||||||
|
}
|
||||||
|
let elapsed = start.elapsed();
|
||||||
|
let total_ns = elapsed.as_secs_f64() * 1e9;
|
||||||
|
per_round_ns_per_op.push(total_ns / (iters as f64));
|
||||||
|
}
|
||||||
|
|
||||||
|
let avg_ns = per_round_ns_per_op.iter().sum::<f64>() / (per_round_ns_per_op.len() as f64);
|
||||||
|
let ops_per_sec = 1e9_f64 / avg_ns;
|
||||||
|
|
||||||
|
println!(
|
||||||
|
"| {} | {} | {} | {} | {:.2} | {:.2} |",
|
||||||
|
sample.obj_type.as_str(),
|
||||||
|
sample.name,
|
||||||
|
size_bytes,
|
||||||
|
complexity,
|
||||||
|
avg_ns,
|
||||||
|
ops_per_sec
|
||||||
|
);
|
||||||
|
|
||||||
|
rows.push(ResultRow {
|
||||||
|
obj_type: sample.obj_type.as_str().to_string(),
|
||||||
|
sample: sample.name.clone(),
|
||||||
|
size_bytes,
|
||||||
|
complexity,
|
||||||
|
avg_ns_per_op: avg_ns,
|
||||||
|
ops_per_sec,
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
|
if let Some(path) = cfg.out_md.as_ref() {
|
||||||
|
let md = render_markdown(
|
||||||
|
"Routinator baseline (rpki crate) decode+inspect (selected_der_v2)",
|
||||||
|
&rows,
|
||||||
|
);
|
||||||
|
write_text_file(path, &md);
|
||||||
|
eprintln!("Wrote {}", path.display());
|
||||||
|
}
|
||||||
|
if let Some(path) = cfg.out_csv.as_ref() {
|
||||||
|
let csv = render_csv(&rows);
|
||||||
|
write_text_file(path, &csv);
|
||||||
|
eprintln!("Wrote {}", path.display());
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,23 +0,0 @@
|
|||||||
{
|
|
||||||
"slurmVersion": 2,
|
|
||||||
"validationOutputFilters": {
|
|
||||||
"prefixFilters": [
|
|
||||||
{
|
|
||||||
"prefix": "24.0.0.0/8",
|
|
||||||
"comment": "Filter many VRPs in current CCR sample"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"bgpsecFilters": [],
|
|
||||||
"aspaFilters": [
|
|
||||||
{
|
|
||||||
"customerAsn": 80,
|
|
||||||
"comment": "Filter one ASPA known to exist in current CCR sample"
|
|
||||||
}
|
|
||||||
]
|
|
||||||
},
|
|
||||||
"locallyAddedAssertions": {
|
|
||||||
"prefixAssertions": [],
|
|
||||||
"bgpsecAssertions": [],
|
|
||||||
"aspaAssertions": []
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,33 +0,0 @@
|
|||||||
# Build-time image knob.
|
|
||||||
RPKI_BIRD_VERSION=3.2.1
|
|
||||||
|
|
||||||
# TCP mode target endpoint.
|
|
||||||
RPKI_BIRD_RPKI_HOST=rpki-rtr
|
|
||||||
RPKI_BIRD_RPKI_PORT=323
|
|
||||||
|
|
||||||
# SSH mode target endpoint.
|
|
||||||
RPKI_BIRD_SSH_RPKI_HOST=rpki-rtr
|
|
||||||
RPKI_RTR_SSH_PORT=22
|
|
||||||
|
|
||||||
# Config template paths in container.
|
|
||||||
RPKI_BIRD_CONFIG_TEMPLATE_PATH=/config/bird.conf.template
|
|
||||||
RPKI_BIRD_SSH_CONFIG_TEMPLATE_PATH=/config/bird.conf.ssh.template
|
|
||||||
|
|
||||||
# Observation and output knobs.
|
|
||||||
RPKI_BIRD_OBSERVE_PROTO=rpki_tcp
|
|
||||||
RPKI_BIRD_OBSERVE_MODE=interval
|
|
||||||
RPKI_BIRD_OBSERVE_DEBOUNCE_SECS=1
|
|
||||||
RPKI_BIRD_OBSERVE_INTERVAL=30
|
|
||||||
RPKI_BIRD_OBSERVE_ASPA_TABLE=rtr_aspa
|
|
||||||
RPKI_BIRD_OBSERVE_ROA4_TABLE=rtr_roa_v4
|
|
||||||
RPKI_BIRD_OBSERVE_ROA6_TABLE=rtr_roa_v6
|
|
||||||
RPKI_BIRD_OBSERVE_ASPA_COUNT=3
|
|
||||||
RPKI_BIRD_OBSERVE_ROA4_COUNT=3
|
|
||||||
RPKI_BIRD_OBSERVE_ROA6_COUNT=3
|
|
||||||
RPKI_BIRD_SHOW_ASPA=1
|
|
||||||
RPKI_BIRD_SHOW_ROA4=1
|
|
||||||
RPKI_BIRD_SHOW_ROA6=1
|
|
||||||
|
|
||||||
# Host volume mounts.
|
|
||||||
RPKI_BIRD_LOG_HOST_DIR=../../logs/bird
|
|
||||||
RPKI_BIRD_SSH_CERTS_HOST_DIR=../../certs/ssh
|
|
||||||
@@ -1,56 +0,0 @@
|
|||||||
FROM debian:bookworm-slim AS builder
|
|
||||||
|
|
||||||
ARG BIRD_VERSION=3.2.1
|
|
||||||
|
|
||||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
||||||
build-essential \
|
|
||||||
bison \
|
|
||||||
flex \
|
|
||||||
m4 \
|
|
||||||
perl \
|
|
||||||
ca-certificates \
|
|
||||||
wget \
|
|
||||||
xz-utils \
|
|
||||||
libreadline-dev \
|
|
||||||
libncurses-dev \
|
|
||||||
libssh-dev \
|
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
|
||||||
|
|
||||||
WORKDIR /build
|
|
||||||
|
|
||||||
RUN wget -O bird.tar.gz "https://bird.nic.cz/download/bird-${BIRD_VERSION}.tar.gz" \
|
|
||||||
&& tar -xzf bird.tar.gz \
|
|
||||||
&& mv "bird-${BIRD_VERSION}" bird
|
|
||||||
|
|
||||||
WORKDIR /build/bird
|
|
||||||
|
|
||||||
RUN ./configure \
|
|
||||||
--prefix=/usr \
|
|
||||||
--sysconfdir=/etc/bird \
|
|
||||||
--localstatedir=/run \
|
|
||||||
&& make -j"$(nproc)" \
|
|
||||||
&& make install
|
|
||||||
|
|
||||||
FROM debian:bookworm-slim
|
|
||||||
|
|
||||||
RUN apt-get update && apt-get install -y --no-install-recommends \
|
|
||||||
ca-certificates \
|
|
||||||
gettext-base \
|
|
||||||
tzdata \
|
|
||||||
netcat-openbsd \
|
|
||||||
libreadline8 \
|
|
||||||
libncurses6 \
|
|
||||||
libtinfo6 \
|
|
||||||
libssh-4 \
|
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
|
||||||
|
|
||||||
COPY --from=builder /usr/sbin/bird /usr/sbin/bird
|
|
||||||
COPY --from=builder /usr/sbin/birdc /usr/sbin/birdc
|
|
||||||
COPY --from=builder /etc/bird /etc/bird
|
|
||||||
ENV TZ=Asia/Shanghai
|
|
||||||
|
|
||||||
COPY entrypoint.sh /entrypoint.sh
|
|
||||||
RUN chmod +x /entrypoint.sh \
|
|
||||||
&& mkdir -p /run/bird
|
|
||||||
|
|
||||||
ENTRYPOINT ["/entrypoint.sh"]
|
|
||||||
@@ -1,90 +0,0 @@
|
|||||||
# BIRD Minimal RTR Client Config
|
|
||||||
|
|
||||||
This folder provides a minimal BIRD setup for black-box interop testing
|
|
||||||
against this repository's RTR server defaults.
|
|
||||||
|
|
||||||
Server defaults in this repo:
|
|
||||||
- TCP: `0.0.0.0:323`
|
|
||||||
- SSH: `0.0.0.0:22` (when enabled on server)
|
|
||||||
|
|
||||||
## Files
|
|
||||||
|
|
||||||
- `Dockerfile`: builds a minimal BIRD2 runtime image.
|
|
||||||
- `bird.conf.example`: sample `/etc/bird/bird.conf`.
|
|
||||||
- `bird.conf.ssh.example`: sample SSH transport `/etc/bird/bird.conf`.
|
|
||||||
- `entrypoint.sh`: starts BIRD in foreground mode.
|
|
||||||
- `docker-compose.yml`: one-click local TCP test client.
|
|
||||||
- `docker-compose.ssh.yml`: compose override for SSH transport.
|
|
||||||
|
|
||||||
By default, the container uses event-driven observation and prints snapshots
|
|
||||||
only when BIRD reports RPKI-related changes.
|
|
||||||
|
|
||||||
## Docker quick start
|
|
||||||
|
|
||||||
From repository root:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f deploy/bird/docker-compose.yml up --build
|
|
||||||
```
|
|
||||||
|
|
||||||
Use another terminal to inspect:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker logs -f bird-rpki-client
|
|
||||||
```
|
|
||||||
|
|
||||||
If protocol state is `up`, the RTR client path is working.
|
|
||||||
|
|
||||||
Detached mode:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f deploy/bird/docker-compose.yml up -d --build
|
|
||||||
docker logs -f bird-rpki-client
|
|
||||||
```
|
|
||||||
|
|
||||||
Stop:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f deploy/bird/docker-compose.yml down
|
|
||||||
```
|
|
||||||
|
|
||||||
## SSH quick start
|
|
||||||
|
|
||||||
Start server in SSH mode first:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f deploy/server/docker-compose.ssh.yml up -d --build
|
|
||||||
```
|
|
||||||
|
|
||||||
Start BIRD client with SSH override:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose \
|
|
||||||
-f deploy/bird/docker-compose.yml \
|
|
||||||
-f deploy/bird/docker-compose.ssh.yml \
|
|
||||||
up --build
|
|
||||||
```
|
|
||||||
|
|
||||||
In detached mode, observe with:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker logs -f bird-rpki-client
|
|
||||||
```
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
|
|
||||||
- This setup renders `bird.conf.template` at startup and uses `RPKI_HOST` / `RPKI_PORT`
|
|
||||||
for the RTR endpoint.
|
|
||||||
- `network_mode: host` expects your RTR server to be reachable at
|
|
||||||
`host.docker.internal:323` from the container.
|
|
||||||
- Observation is controlled by env vars:
|
|
||||||
`OBSERVE_MODE` (`event` by default, `interval` as fallback),
|
|
||||||
`OBSERVE_DEBOUNCE_SECS` (default `1`),
|
|
||||||
`OBSERVE_INTERVAL` (seconds, used when `OBSERVE_MODE=interval`),
|
|
||||||
and `OBSERVE_PROTO`.
|
|
||||||
- SSH mode mounts `../../certs/ssh` into `/config/ssh` and expects:
|
|
||||||
`rtr-client.key` and `ssh_host_rsa_key.pub`.
|
|
||||||
- Entrypoint auto-generates `/run/bird/known_hosts` from
|
|
||||||
`/config/ssh/ssh_host_rsa_key.pub` for BIRD SSH host-key verification.
|
|
||||||
- If your environment does not support Docker host networking, switch to a
|
|
||||||
bridge network and replace `remote` addresses accordingly.
|
|
||||||
@@ -1,80 +0,0 @@
|
|||||||
# BIRD 最小化 RTR 客户端配置
|
|
||||||
|
|
||||||
本目录提供最小化 BIRD 配置,用于与本仓库 RTR Server 做黑盒互通测试。
|
|
||||||
|
|
||||||
本仓库默认 RTR 监听地址:
|
|
||||||
- TCP: `0.0.0.0:323`
|
|
||||||
- SSH: `0.0.0.0:22`(仅在 server SSH 模式启用时)
|
|
||||||
|
|
||||||
## 文件说明
|
|
||||||
|
|
||||||
- `Dockerfile`: 构建最小 BIRD 运行镜像(包含 SSH transport 支持)。
|
|
||||||
- `bird.conf.example`: `/etc/bird/bird.conf` 的 TCP 示例。
|
|
||||||
- `bird.conf.ssh.example`: `/etc/bird/bird.conf` 的 SSH transport 示例。
|
|
||||||
- `entrypoint.sh`: 前台启动 BIRD,并周期输出协议快照。
|
|
||||||
- `docker-compose.yml`: TCP 一键启动。
|
|
||||||
- `docker-compose.ssh.yml`: SSH 覆盖配置。
|
|
||||||
|
|
||||||
容器默认每 30 秒向日志输出一次 RPKI 协议状态快照。
|
|
||||||
|
|
||||||
## Docker 快速启动(TCP)
|
|
||||||
|
|
||||||
在仓库根目录执行:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f deploy/bird/docker-compose.yml up --build
|
|
||||||
```
|
|
||||||
|
|
||||||
另开终端查看日志:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker logs -f bird-rpki-client
|
|
||||||
```
|
|
||||||
|
|
||||||
如果协议状态显示 `up`,说明 RTR 客户端链路正常。
|
|
||||||
|
|
||||||
后台模式:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f deploy/bird/docker-compose.yml up -d --build
|
|
||||||
docker logs -f bird-rpki-client
|
|
||||||
```
|
|
||||||
|
|
||||||
停止:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f deploy/bird/docker-compose.yml down
|
|
||||||
```
|
|
||||||
|
|
||||||
## SSH 快速启动
|
|
||||||
|
|
||||||
先启动 server 的 SSH 模式:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f deploy/server/docker-compose.ssh.yml up -d --build
|
|
||||||
```
|
|
||||||
|
|
||||||
再启动 BIRD SSH 客户端:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose \
|
|
||||||
-f deploy/bird/docker-compose.yml \
|
|
||||||
-f deploy/bird/docker-compose.ssh.yml \
|
|
||||||
up --build
|
|
||||||
```
|
|
||||||
|
|
||||||
查看日志:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker logs -f bird-rpki-client
|
|
||||||
```
|
|
||||||
|
|
||||||
## 说明
|
|
||||||
|
|
||||||
- 当前 compose 使用 `network_mode: host`,默认通过 `host.docker.internal` 访问 server。
|
|
||||||
- 观测频率由环境变量控制:`OBSERVE_INTERVAL`(秒,默认 `30`)和 `OBSERVE_PROTO`。
|
|
||||||
- SSH 模式会将 `../../certs/ssh` 挂载到容器 `/config/ssh`,并使用:
|
|
||||||
`rtr-client.key`、`ssh_host_rsa_key.pub`。
|
|
||||||
- 入口脚本会基于 `/config/ssh/ssh_host_rsa_key.pub` 自动生成
|
|
||||||
`/run/bird/known_hosts`,用于 BIRD 的 SSH 主机密钥校验。
|
|
||||||
- 如果你运行在 Docker Desktop(非 Linux 原生 host network 场景),建议改为自定义 bridge 网络并将 `remote` 地址改成可达的 server 容器名或宿主机地址。
|
|
||||||
@@ -1,31 +0,0 @@
|
|||||||
log stderr all;
|
|
||||||
debug protocols all;
|
|
||||||
router id 192.0.2.2;
|
|
||||||
|
|
||||||
roa4 table rtr_roa_v4;
|
|
||||||
roa6 table rtr_roa_v6;
|
|
||||||
aspa table rtr_aspa;
|
|
||||||
|
|
||||||
protocol device {
|
|
||||||
}
|
|
||||||
|
|
||||||
protocol rpki rpki_ssh {
|
|
||||||
roa4 { table rtr_roa_v4; };
|
|
||||||
roa6 { table rtr_roa_v6; };
|
|
||||||
aspa { table rtr_aspa; };
|
|
||||||
|
|
||||||
remote "host.docker.internal" port 22;
|
|
||||||
|
|
||||||
min version 2;
|
|
||||||
max version 2;
|
|
||||||
|
|
||||||
refresh 3600;
|
|
||||||
retry 600;
|
|
||||||
expire 7200;
|
|
||||||
|
|
||||||
transport ssh {
|
|
||||||
user "rpki-rtr";
|
|
||||||
bird private key "/config/ssh/rtr-client.key";
|
|
||||||
remote public key "/run/bird/known_hosts";
|
|
||||||
};
|
|
||||||
}
|
|
||||||
@@ -1,28 +0,0 @@
|
|||||||
log stderr all;
|
|
||||||
router id 192.0.2.2;
|
|
||||||
|
|
||||||
roa4 table rtr_roa_v4;
|
|
||||||
roa6 table rtr_roa_v6;
|
|
||||||
aspa table rtr_aspa;
|
|
||||||
|
|
||||||
protocol device {
|
|
||||||
}
|
|
||||||
|
|
||||||
protocol rpki rpki_tcp {
|
|
||||||
roa4 { table rtr_roa_v4; };
|
|
||||||
roa6 { table rtr_roa_v6; };
|
|
||||||
aspa { table rtr_aspa; };
|
|
||||||
|
|
||||||
remote "host.docker.internal" port 323;
|
|
||||||
|
|
||||||
min version 2;
|
|
||||||
max version 2;
|
|
||||||
|
|
||||||
refresh 3600;
|
|
||||||
retry 600;
|
|
||||||
expire 7200;
|
|
||||||
|
|
||||||
transport tcp {
|
|
||||||
authentication none;
|
|
||||||
};
|
|
||||||
}
|
|
||||||
@@ -1,28 +0,0 @@
|
|||||||
log stderr all;
|
|
||||||
router id 192.0.2.2;
|
|
||||||
|
|
||||||
roa4 table rtr_roa_v4;
|
|
||||||
roa6 table rtr_roa_v6;
|
|
||||||
aspa table rtr_aspa;
|
|
||||||
|
|
||||||
protocol device {
|
|
||||||
}
|
|
||||||
|
|
||||||
protocol rpki rpki_tcp {
|
|
||||||
roa4 { table rtr_roa_v4; };
|
|
||||||
roa6 { table rtr_roa_v6; };
|
|
||||||
aspa { table rtr_aspa; };
|
|
||||||
|
|
||||||
remote "host.docker.internal" port 323;
|
|
||||||
|
|
||||||
min version 2;
|
|
||||||
max version 2;
|
|
||||||
|
|
||||||
refresh 3600;
|
|
||||||
retry 600;
|
|
||||||
expire 7200;
|
|
||||||
|
|
||||||
transport tcp {
|
|
||||||
authentication none;
|
|
||||||
};
|
|
||||||
}
|
|
||||||
@@ -1,30 +0,0 @@
|
|||||||
log stderr all;
|
|
||||||
router id 192.0.2.2;
|
|
||||||
|
|
||||||
roa4 table rtr_roa_v4;
|
|
||||||
roa6 table rtr_roa_v6;
|
|
||||||
aspa table rtr_aspa;
|
|
||||||
|
|
||||||
protocol device {
|
|
||||||
}
|
|
||||||
|
|
||||||
protocol rpki rpki_ssh {
|
|
||||||
roa4 { table rtr_roa_v4; };
|
|
||||||
roa6 { table rtr_roa_v6; };
|
|
||||||
aspa { table rtr_aspa; };
|
|
||||||
|
|
||||||
remote "host.docker.internal" port 22;
|
|
||||||
|
|
||||||
min version 2;
|
|
||||||
max version 2;
|
|
||||||
|
|
||||||
refresh 3600;
|
|
||||||
retry 600;
|
|
||||||
expire 7200;
|
|
||||||
|
|
||||||
transport ssh {
|
|
||||||
user "rpki-rtr";
|
|
||||||
bird private key "/config/ssh/rtr-client.key";
|
|
||||||
remote public key "/run/bird/known_hosts";
|
|
||||||
};
|
|
||||||
}
|
|
||||||
@@ -1,30 +0,0 @@
|
|||||||
log stderr all;
|
|
||||||
router id 192.0.2.2;
|
|
||||||
|
|
||||||
roa4 table rtr_roa_v4;
|
|
||||||
roa6 table rtr_roa_v6;
|
|
||||||
aspa table rtr_aspa;
|
|
||||||
|
|
||||||
protocol device {
|
|
||||||
}
|
|
||||||
|
|
||||||
protocol rpki rpki_ssh {
|
|
||||||
roa4 { table rtr_roa_v4; };
|
|
||||||
roa6 { table rtr_roa_v6; };
|
|
||||||
aspa { table rtr_aspa; };
|
|
||||||
|
|
||||||
remote "${RPKI_HOST}" port ${RPKI_PORT};
|
|
||||||
|
|
||||||
min version 2;
|
|
||||||
max version 2;
|
|
||||||
|
|
||||||
refresh 3600;
|
|
||||||
retry 600;
|
|
||||||
expire 7200;
|
|
||||||
|
|
||||||
transport ssh {
|
|
||||||
user "rpki-rtr";
|
|
||||||
bird private key "/config/ssh/rtr-client.key";
|
|
||||||
remote public key "/run/bird/known_hosts";
|
|
||||||
};
|
|
||||||
}
|
|
||||||
@@ -1,28 +0,0 @@
|
|||||||
log stderr all;
|
|
||||||
router id 192.0.2.2;
|
|
||||||
|
|
||||||
roa4 table rtr_roa_v4;
|
|
||||||
roa6 table rtr_roa_v6;
|
|
||||||
aspa table rtr_aspa;
|
|
||||||
|
|
||||||
protocol device {
|
|
||||||
}
|
|
||||||
|
|
||||||
protocol rpki rpki_tcp {
|
|
||||||
roa4 { table rtr_roa_v4; };
|
|
||||||
roa6 { table rtr_roa_v6; };
|
|
||||||
aspa { table rtr_aspa; };
|
|
||||||
|
|
||||||
remote "${RPKI_HOST}" port ${RPKI_PORT};
|
|
||||||
|
|
||||||
min version 2;
|
|
||||||
max version 2;
|
|
||||||
|
|
||||||
refresh 3600;
|
|
||||||
retry 600;
|
|
||||||
expire 7200;
|
|
||||||
|
|
||||||
transport tcp {
|
|
||||||
authentication none;
|
|
||||||
};
|
|
||||||
}
|
|
||||||
@@ -1,17 +0,0 @@
|
|||||||
services:
|
|
||||||
bird-rpki-client:
|
|
||||||
environment:
|
|
||||||
BIRD_CONFIG_TEMPLATE_PATH: "${RPKI_BIRD_SSH_CONFIG_TEMPLATE_PATH:-/config/bird.conf.ssh.template}"
|
|
||||||
RPKI_HOST: "${RPKI_BIRD_SSH_RPKI_HOST:-rpki-rtr}"
|
|
||||||
RPKI_PORT: "${RPKI_RTR_SSH_PORT:-22}"
|
|
||||||
OBSERVE_PROTO: "rpki_ssh"
|
|
||||||
volumes:
|
|
||||||
- ./bird.conf.ssh.template:/config/bird.conf.ssh.template:ro
|
|
||||||
- ${RPKI_BIRD_SSH_CERTS_HOST_DIR:-../../certs/ssh}:/config/ssh:ro
|
|
||||||
networks:
|
|
||||||
- rpki_net
|
|
||||||
|
|
||||||
networks:
|
|
||||||
rpki_net:
|
|
||||||
name: rpki_net
|
|
||||||
driver: bridge
|
|
||||||
@@ -1,41 +0,0 @@
|
|||||||
services:
|
|
||||||
bird-rpki-client:
|
|
||||||
build:
|
|
||||||
context: .
|
|
||||||
dockerfile: Dockerfile
|
|
||||||
args:
|
|
||||||
BIRD_VERSION: "${RPKI_BIRD_VERSION:-3.2.1}"
|
|
||||||
container_name: bird-rpki-client
|
|
||||||
restart: unless-stopped
|
|
||||||
environment:
|
|
||||||
BIRD_CONFIG_TEMPLATE_PATH: "${RPKI_BIRD_CONFIG_TEMPLATE_PATH:-/config/bird.conf.template}"
|
|
||||||
|
|
||||||
RPKI_HOST: "${RPKI_BIRD_RPKI_HOST:-rpki-rtr}"
|
|
||||||
RPKI_PORT: "${RPKI_BIRD_RPKI_PORT:-323}"
|
|
||||||
|
|
||||||
OBSERVE_PROTO: "${RPKI_BIRD_OBSERVE_PROTO:-rpki_tcp}"
|
|
||||||
OBSERVE_MODE: "${RPKI_BIRD_OBSERVE_MODE:-interval}"
|
|
||||||
OBSERVE_DEBOUNCE_SECS: "${RPKI_BIRD_OBSERVE_DEBOUNCE_SECS:-1}"
|
|
||||||
OBSERVE_INTERVAL: "${RPKI_BIRD_OBSERVE_INTERVAL:-30}"
|
|
||||||
|
|
||||||
OBSERVE_ASPA_TABLE: "${RPKI_BIRD_OBSERVE_ASPA_TABLE:-rtr_aspa}"
|
|
||||||
OBSERVE_ROA4_TABLE: "${RPKI_BIRD_OBSERVE_ROA4_TABLE:-rtr_roa_v4}"
|
|
||||||
OBSERVE_ROA6_TABLE: "${RPKI_BIRD_OBSERVE_ROA6_TABLE:-rtr_roa_v6}"
|
|
||||||
|
|
||||||
OBSERVE_ASPA_COUNT: "${RPKI_BIRD_OBSERVE_ASPA_COUNT:-3}"
|
|
||||||
OBSERVE_ROA4_COUNT: "${RPKI_BIRD_OBSERVE_ROA4_COUNT:-3}"
|
|
||||||
OBSERVE_ROA6_COUNT: "${RPKI_BIRD_OBSERVE_ROA6_COUNT:-3}"
|
|
||||||
|
|
||||||
SHOW_ASPA: "${RPKI_BIRD_SHOW_ASPA:-1}"
|
|
||||||
SHOW_ROA4: "${RPKI_BIRD_SHOW_ROA4:-1}"
|
|
||||||
SHOW_ROA6: "${RPKI_BIRD_SHOW_ROA6:-1}"
|
|
||||||
volumes:
|
|
||||||
- ./bird.conf.template:/config/bird.conf.template:ro
|
|
||||||
- ${RPKI_BIRD_LOG_HOST_DIR:-../../logs/bird}:/app/logs
|
|
||||||
networks:
|
|
||||||
- rpki_net
|
|
||||||
|
|
||||||
networks:
|
|
||||||
rpki_net:
|
|
||||||
name: rpki_net
|
|
||||||
driver: bridge
|
|
||||||
@@ -1,218 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
set -eu
|
|
||||||
|
|
||||||
mkdir -p /run/bird
|
|
||||||
|
|
||||||
SOCK_PATH="/run/bird/bird.ctl"
|
|
||||||
PROTO="${OBSERVE_PROTO:-rpki_tcp}"
|
|
||||||
INTERVAL="${OBSERVE_INTERVAL:-30}"
|
|
||||||
MODE="${OBSERVE_MODE:-event}"
|
|
||||||
DEBOUNCE_SECS="${OBSERVE_DEBOUNCE_SECS:-1}"
|
|
||||||
RPKI_HOST="${RPKI_HOST:-host.docker.internal}"
|
|
||||||
RPKI_PORT="${RPKI_PORT:-323}"
|
|
||||||
|
|
||||||
BIRD_CONFIG_PATH="${BIRD_CONFIG_PATH:-/config/bird.conf}"
|
|
||||||
BIRD_CONFIG_TEMPLATE_PATH="${BIRD_CONFIG_TEMPLATE_PATH:-/config/bird.conf.template}"
|
|
||||||
|
|
||||||
ASPA_TABLE="${OBSERVE_ASPA_TABLE:-rtr_aspa}"
|
|
||||||
ROA4_TABLE="${OBSERVE_ROA4_TABLE:-rtr_roa_v4}"
|
|
||||||
ROA6_TABLE="${OBSERVE_ROA6_TABLE:-rtr_roa_v6}"
|
|
||||||
|
|
||||||
ASPA_COUNT="${OBSERVE_ASPA_COUNT:-3}"
|
|
||||||
ROA4_COUNT="${OBSERVE_ROA4_COUNT:-3}"
|
|
||||||
ROA6_COUNT="${OBSERVE_ROA6_COUNT:-3}"
|
|
||||||
|
|
||||||
SHOW_ASPA="${SHOW_ASPA:-1}"
|
|
||||||
SHOW_ROA4="${SHOW_ROA4:-1}"
|
|
||||||
SHOW_ROA6="${SHOW_ROA6:-1}"
|
|
||||||
SSH_HOST_PUBKEY_PATH="${SSH_HOST_PUBKEY_PATH:-/config/ssh/ssh_host_rsa_key.pub}"
|
|
||||||
SSH_KNOWN_HOSTS_PATH="${SSH_KNOWN_HOSTS_PATH:-/run/bird/known_hosts}"
|
|
||||||
|
|
||||||
LOG_DIR="${LOG_DIR:-/app/logs}"
|
|
||||||
LOG_NAME="${LOG_NAME:-${HOSTNAME:-bird-rpki-client}}"
|
|
||||||
STDOUT_LOG="${LOG_DIR}/${LOG_NAME}.stdout.log"
|
|
||||||
STDERR_LOG="${LOG_DIR}/${LOG_NAME}.stderr.log"
|
|
||||||
|
|
||||||
mkdir -p "$LOG_DIR"
|
|
||||||
exec >>"$STDOUT_LOG" 2>>"$STDERR_LOG"
|
|
||||||
|
|
||||||
if [ -r "$BIRD_CONFIG_TEMPLATE_PATH" ]; then
|
|
||||||
RENDERED_CONFIG_PATH="/run/bird/bird.generated.conf"
|
|
||||||
envsubst '${RPKI_HOST} ${RPKI_PORT}' < "$BIRD_CONFIG_TEMPLATE_PATH" > "$RENDERED_CONFIG_PATH"
|
|
||||||
BIRD_CONFIG_PATH="$RENDERED_CONFIG_PATH"
|
|
||||||
echo "[entrypoint] rendered config : $BIRD_CONFIG_TEMPLATE_PATH -> $BIRD_CONFIG_PATH"
|
|
||||||
fi
|
|
||||||
|
|
||||||
ensure_ssh_known_hosts() {
|
|
||||||
if [ -s "$SSH_KNOWN_HOSTS_PATH" ]; then
|
|
||||||
return
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [ ! -r "$SSH_HOST_PUBKEY_PATH" ]; then
|
|
||||||
echo "[entrypoint] WARNING: SSH host key file not found: $SSH_HOST_PUBKEY_PATH"
|
|
||||||
return
|
|
||||||
fi
|
|
||||||
|
|
||||||
set -- $(awk 'NF >= 2 { print $1, $2; exit }' "$SSH_HOST_PUBKEY_PATH")
|
|
||||||
if [ $# -ne 2 ]; then
|
|
||||||
echo "[entrypoint] WARNING: invalid SSH host key format in $SSH_HOST_PUBKEY_PATH"
|
|
||||||
return
|
|
||||||
fi
|
|
||||||
|
|
||||||
key_type="$1"
|
|
||||||
key_data="$2"
|
|
||||||
|
|
||||||
if echo "$key_type" | grep -q '^ssh-'; then
|
|
||||||
{
|
|
||||||
echo "$RPKI_HOST $key_type $key_data"
|
|
||||||
echo "[$RPKI_HOST]:$RPKI_PORT $key_type $key_data"
|
|
||||||
} > "$SSH_KNOWN_HOSTS_PATH"
|
|
||||||
else
|
|
||||||
cp "$SSH_HOST_PUBKEY_PATH" "$SSH_KNOWN_HOSTS_PATH"
|
|
||||||
fi
|
|
||||||
|
|
||||||
chmod 600 "$SSH_KNOWN_HOSTS_PATH" || true
|
|
||||||
echo "[entrypoint] generated known_hosts: $SSH_KNOWN_HOSTS_PATH"
|
|
||||||
}
|
|
||||||
|
|
||||||
print_first_n_objects() {
|
|
||||||
table_name="$1"
|
|
||||||
max_objects="$2"
|
|
||||||
|
|
||||||
birdc -s "$SOCK_PATH" show route table "$table_name" all 2>/dev/null | awk -v max="$max_objects" '
|
|
||||||
BEGIN {
|
|
||||||
count = 0
|
|
||||||
}
|
|
||||||
|
|
||||||
# 直接跳过空行
|
|
||||||
/^[[:space:]]*$/ {
|
|
||||||
next
|
|
||||||
}
|
|
||||||
|
|
||||||
# 保留 birdc 的表头
|
|
||||||
/^BIRD / {
|
|
||||||
print
|
|
||||||
next
|
|
||||||
}
|
|
||||||
|
|
||||||
/^Table / {
|
|
||||||
print
|
|
||||||
next
|
|
||||||
}
|
|
||||||
|
|
||||||
# 非缩进且不是表头,视为一个新对象的开始
|
|
||||||
/^[^[:space:]]/ {
|
|
||||||
count++
|
|
||||||
if (count > max) {
|
|
||||||
exit
|
|
||||||
}
|
|
||||||
print
|
|
||||||
next
|
|
||||||
}
|
|
||||||
|
|
||||||
# 缩进内容,只有在已进入前 max 个对象时才打印
|
|
||||||
{
|
|
||||||
if (count > 0 && count <= max) {
|
|
||||||
print
|
|
||||||
}
|
|
||||||
}
|
|
||||||
' || true
|
|
||||||
}
|
|
||||||
|
|
||||||
print_snapshot() {
|
|
||||||
echo "==== $(date +"%Y-%m-%dT%H:%M:%S%:z") RPKI snapshot ($PROTO) ===="
|
|
||||||
birdc -s "$SOCK_PATH" show protocols all "$PROTO" || true
|
|
||||||
|
|
||||||
if [ "$SHOW_ASPA" = "1" ]; then
|
|
||||||
echo "---- ASPA table ($ASPA_TABLE, first ${ASPA_COUNT} objects) ----"
|
|
||||||
print_first_n_objects "$ASPA_TABLE" "$ASPA_COUNT"
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [ "$SHOW_ROA4" = "1" ]; then
|
|
||||||
echo "---- ROA4 table ($ROA4_TABLE, first ${ROA4_COUNT} objects) ----"
|
|
||||||
print_first_n_objects "$ROA4_TABLE" "$ROA4_COUNT"
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [ "$SHOW_ROA6" = "1" ]; then
|
|
||||||
echo "---- ROA6 table ($ROA6_TABLE, first ${ROA6_COUNT} objects) ----"
|
|
||||||
print_first_n_objects "$ROA6_TABLE" "$ROA6_COUNT"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
is_rpki_related_event() {
|
|
||||||
line="$1"
|
|
||||||
echo "$line" | grep -Eiq "$PROTO|$ASPA_TABLE|$ROA4_TABLE|$ROA6_TABLE|rpki|rtr"
|
|
||||||
}
|
|
||||||
|
|
||||||
echo "[entrypoint] starting bird"
|
|
||||||
echo "[entrypoint] config : $BIRD_CONFIG_PATH"
|
|
||||||
echo "[entrypoint] observe proto : $PROTO"
|
|
||||||
echo "[entrypoint] observe mode : $MODE"
|
|
||||||
echo "[entrypoint] observe interval : $INTERVAL (used when mode=interval)"
|
|
||||||
echo "[entrypoint] debounce secs : $DEBOUNCE_SECS (used when mode=event)"
|
|
||||||
echo "[entrypoint] target : $RPKI_HOST:$RPKI_PORT"
|
|
||||||
echo "[entrypoint] show aspa : $SHOW_ASPA ($ASPA_TABLE, first $ASPA_COUNT objects)"
|
|
||||||
echo "[entrypoint] show roa4 : $SHOW_ROA4 ($ROA4_TABLE, first $ROA4_COUNT objects)"
|
|
||||||
echo "[entrypoint] show roa6 : $SHOW_ROA6 ($ROA6_TABLE, first $ROA6_COUNT objects)"
|
|
||||||
|
|
||||||
if [ "$PROTO" = "rpki_ssh" ] || grep -q 'transport[[:space:]]\+ssh' "$BIRD_CONFIG_PATH"; then
|
|
||||||
ensure_ssh_known_hosts
|
|
||||||
fi
|
|
||||||
|
|
||||||
if nc -zvw3 "$RPKI_HOST" "$RPKI_PORT"; then
|
|
||||||
echo "[entrypoint] TCP connectivity to $RPKI_HOST:$RPKI_PORT OK"
|
|
||||||
else
|
|
||||||
echo "[entrypoint] WARNING: cannot connect to $RPKI_HOST:$RPKI_PORT before BIRD starts"
|
|
||||||
fi
|
|
||||||
|
|
||||||
bird -f -c "$BIRD_CONFIG_PATH" -s "$SOCK_PATH" &
|
|
||||||
BIRD_PID="$!"
|
|
||||||
|
|
||||||
sleep 1
|
|
||||||
|
|
||||||
case "$INTERVAL" in
|
|
||||||
''|*[!0-9]*)
|
|
||||||
INTERVAL=0
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
|
|
||||||
case "$MODE" in
|
|
||||||
event)
|
|
||||||
case "$DEBOUNCE_SECS" in
|
|
||||||
''|*[!0-9]*)
|
|
||||||
DEBOUNCE_SECS=1
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
|
|
||||||
echo "[entrypoint] waiting for BIRD monitor events and only printing on RPKI-related changes"
|
|
||||||
LAST_PRINT_TS=0
|
|
||||||
|
|
||||||
if birdc -s "$SOCK_PATH" monitor all 2>/dev/null | while IFS= read -r line; do
|
|
||||||
[ -n "$line" ] || continue
|
|
||||||
if ! is_rpki_related_event "$line"; then
|
|
||||||
continue
|
|
||||||
fi
|
|
||||||
now_ts="$(date +%s)"
|
|
||||||
if [ $((now_ts - LAST_PRINT_TS)) -lt "$DEBOUNCE_SECS" ]; then
|
|
||||||
continue
|
|
||||||
fi
|
|
||||||
LAST_PRINT_TS="$now_ts"
|
|
||||||
echo "[monitor] $line"
|
|
||||||
print_snapshot
|
|
||||||
done; then
|
|
||||||
:
|
|
||||||
else
|
|
||||||
echo "[entrypoint] WARNING: birdc monitor failed, fallback to interval polling"
|
|
||||||
MODE="interval"
|
|
||||||
fi
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
|
|
||||||
if [ "$MODE" = "interval" ] && [ "$INTERVAL" -gt 0 ]; then
|
|
||||||
while kill -0 "$BIRD_PID" 2>/dev/null; do
|
|
||||||
print_snapshot
|
|
||||||
sleep "$INTERVAL"
|
|
||||||
done
|
|
||||||
fi
|
|
||||||
|
|
||||||
wait "$BIRD_PID"
|
|
||||||
@@ -1,32 +0,0 @@
|
|||||||
# Target RTR server addresses for client compose files
|
|
||||||
# RPKI_RTR_CLIENT_IMAGE=rpki-rtr-debug-client-arm64:v1.2
|
|
||||||
RPKI_RTR_CLIENT_IMAGE=rpki-rtr-debug-client-amd64:v1.2
|
|
||||||
|
|
||||||
# TCP example: 10.0.0.12:323
|
|
||||||
# TLS example: rpki.example.com:324
|
|
||||||
# SSH example: 10.0.0.12:22
|
|
||||||
RPKI_RTR_TCP_SERVER_ADDR=rpki-rtr:323
|
|
||||||
RPKI_RTR_TLS_SERVER_ADDR=rpki-rtr:324
|
|
||||||
RPKI_RTR_SSH_SERVER_ADDR=rpki-rtr:22
|
|
||||||
|
|
||||||
# RTR protocol version used as client command second argument (supported: 0,1,2)
|
|
||||||
RPKI_RTR_PROTOCOL_VERSION=2
|
|
||||||
|
|
||||||
# TLS server name used by --server-name in TLS mode
|
|
||||||
# Must match server certificate SAN dNSName.
|
|
||||||
RPKI_RTR_TLS_SERVER_NAME=localhost
|
|
||||||
RPKI_RTR_TLS_CA_CERT_PATH=/app/certs/client-ca.crt
|
|
||||||
RPKI_RTR_TLS_CLIENT_CERT_PATH=/app/certs/client-good.crt
|
|
||||||
RPKI_RTR_TLS_CLIENT_KEY_PATH=/app/certs/client-good.key
|
|
||||||
RPKI_RTR_TLS_CERTS_HOST_DIR=../../certs/tls
|
|
||||||
|
|
||||||
# Shared client logs mount on host.
|
|
||||||
RPKI_RTR_CLIENT_LOG_HOST_DIR=../../logs/client
|
|
||||||
|
|
||||||
# SSH mode examples:
|
|
||||||
# RPKI_RTR_SSH_SERVER_ADDR=10.0.0.12:2222
|
|
||||||
RPKI_RTR_CLIENT_KEYS_VOLUME=../../certs/ssh:/app/certs:ro
|
|
||||||
RPKI_RTR_CLIENT_KEY_PATH=/app/certs/rtr-client.key
|
|
||||||
RPKI_RTR_SSH_SERVER_PUBKEY_PATH=/app/certs/ssh_host_rsa_key.pub
|
|
||||||
RPKI_RTR_SSH_USERNAME=rpki-rtr
|
|
||||||
RPKI_RTR_SSH_PASSWORD=
|
|
||||||
@@ -1,26 +0,0 @@
|
|||||||
FROM rust:1.89-bookworm AS builder
|
|
||||||
|
|
||||||
WORKDIR /build
|
|
||||||
|
|
||||||
RUN apt-get update \
|
|
||||||
&& apt-get install -y --no-install-recommends clang libclang-dev pkg-config \
|
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
|
||||||
|
|
||||||
COPY Cargo.toml Cargo.lock ./
|
|
||||||
COPY src ./src
|
|
||||||
|
|
||||||
RUN cargo build --release --bin rtr_debug_client
|
|
||||||
|
|
||||||
FROM debian:bookworm-slim AS runtime
|
|
||||||
|
|
||||||
RUN apt-get update \
|
|
||||||
&& apt-get install -y --no-install-recommends ca-certificates tzdata \
|
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
|
||||||
|
|
||||||
WORKDIR /app
|
|
||||||
ENV TZ=Asia/Shanghai
|
|
||||||
|
|
||||||
COPY --from=builder /build/target/release/rtr_debug_client /usr/local/bin/rtr_debug_client
|
|
||||||
COPY --chmod=755 deploy/client/entrypoint.sh /usr/local/bin/rtr-debug-client-entrypoint.sh
|
|
||||||
|
|
||||||
ENTRYPOINT ["/usr/local/bin/rtr-debug-client-entrypoint.sh"]
|
|
||||||
@@ -1,237 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
REPO_ROOT="$(cd -- "${SCRIPT_DIR}/../.." && pwd)"
|
|
||||||
|
|
||||||
MODE="${MODE:-tcp}"
|
|
||||||
ACTION="up"
|
|
||||||
DETACH="${DETACH:-1}"
|
|
||||||
BUILD="${BUILD:-1}"
|
|
||||||
CLIENTS="${CLIENTS:-single}"
|
|
||||||
TARGET_PLATFORM="${TARGET_PLATFORM:-linux/arm64}"
|
|
||||||
CLIENT_IMAGE="${CLIENT_IMAGE:-${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}}"
|
|
||||||
BUILDX_PUSH="${BUILDX_PUSH:-0}"
|
|
||||||
BUILDX_BUILDER="${BUILDX_BUILDER:-}"
|
|
||||||
SKIP_BASE_IMAGE_PULL="${SKIP_BASE_IMAGE_PULL:-0}"
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage:
|
|
||||||
deploy/client/build-run.sh [action] [options]
|
|
||||||
|
|
||||||
Actions:
|
|
||||||
up Build and run debug client (default)
|
|
||||||
build Build debug client image
|
|
||||||
buildx Build debug client image with docker buildx
|
|
||||||
rebuild Rebuild image without cache, then run client
|
|
||||||
down|stop Stop client compose stack
|
|
||||||
restart Stop, then build and run client
|
|
||||||
logs Follow client logs
|
|
||||||
ps Show client compose status
|
|
||||||
|
|
||||||
Options:
|
|
||||||
--mode MODE base|tcp|tls|ssh (default: tcp)
|
|
||||||
--clients MODE single|multi (default: single)
|
|
||||||
--platform VALUE Target platform for buildx (default: linux/arm64)
|
|
||||||
--client-image TAG
|
|
||||||
Client image tag for buildx and compose
|
|
||||||
--push Push buildx image instead of loading it locally
|
|
||||||
--builder NAME buildx builder name
|
|
||||||
--skip-base-pull Do not pull Dockerfile base images before buildx
|
|
||||||
--foreground Run single client in foreground
|
|
||||||
--no-build Start without --build
|
|
||||||
-h|--help Show this help
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
die() {
|
|
||||||
echo "error: $*" >&2
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
has_compose() {
|
|
||||||
docker compose version >/dev/null 2>&1
|
|
||||||
}
|
|
||||||
|
|
||||||
has_buildx() {
|
|
||||||
docker buildx version >/dev/null 2>&1
|
|
||||||
}
|
|
||||||
|
|
||||||
pull_base_images() {
|
|
||||||
[[ "${SKIP_BASE_IMAGE_PULL}" == "1" ]] && return
|
|
||||||
|
|
||||||
local platform image
|
|
||||||
local -a platforms images
|
|
||||||
IFS=',' read -r -a platforms <<< "${TARGET_PLATFORM}"
|
|
||||||
images=(rust:1.89-bookworm debian:bookworm-slim)
|
|
||||||
|
|
||||||
for platform in "${platforms[@]}"; do
|
|
||||||
for image in "${images[@]}"; do
|
|
||||||
docker pull --platform "${platform}" "${image}"
|
|
||||||
done
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
compose_files() {
|
|
||||||
if [[ "${CLIENTS}" == "multi" ]]; then
|
|
||||||
[[ "${MODE}" == "tcp" || "${MODE}" == "base" ]] || die "--clients multi is only defined for tcp/base mode"
|
|
||||||
printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.clients.yml"
|
|
||||||
return
|
|
||||||
fi
|
|
||||||
|
|
||||||
case "${MODE}" in
|
|
||||||
base) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.yml" ;;
|
|
||||||
tcp) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.tcp.yml" ;;
|
|
||||||
tls) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.tls.yml" ;;
|
|
||||||
ssh) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.ssh.yml" ;;
|
|
||||||
*) die "unsupported mode '${MODE}', expected base|tcp|tls|ssh" ;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
|
|
||||||
compose_args() {
|
|
||||||
local -a args files
|
|
||||||
args=()
|
|
||||||
[[ -f "${SCRIPT_DIR}/.env" ]] && args+=(--env-file "${SCRIPT_DIR}/.env")
|
|
||||||
mapfile -t files < <(compose_files)
|
|
||||||
args+=("${files[@]}")
|
|
||||||
printf '%s\n' "${args[@]}"
|
|
||||||
}
|
|
||||||
|
|
||||||
compose_run() {
|
|
||||||
local -a args
|
|
||||||
mapfile -t args < <(compose_args)
|
|
||||||
docker compose "${args[@]}" "$@"
|
|
||||||
}
|
|
||||||
|
|
||||||
up_client() {
|
|
||||||
local -a up_args
|
|
||||||
up_args=(up)
|
|
||||||
[[ "${CLIENTS}" == "multi" || "${DETACH}" == "1" ]] && up_args+=(-d)
|
|
||||||
[[ "${BUILD}" == "1" && "${CLIENTS}" != "multi" ]] && up_args+=(--build)
|
|
||||||
compose_run "${up_args[@]}"
|
|
||||||
}
|
|
||||||
|
|
||||||
build_client() {
|
|
||||||
if [[ "${CLIENTS}" == "multi" ]]; then
|
|
||||||
docker build "$@" -f "${SCRIPT_DIR}/Dockerfile" -t "${CLIENT_IMAGE}" "${REPO_ROOT}"
|
|
||||||
else
|
|
||||||
compose_run build "$@"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
buildx_client() {
|
|
||||||
has_buildx || die "docker buildx is not available"
|
|
||||||
if [[ "${TARGET_PLATFORM}" == *,* && "${BUILDX_PUSH}" != "1" ]]; then
|
|
||||||
die "multi-platform buildx output requires --push"
|
|
||||||
fi
|
|
||||||
|
|
||||||
local -a output_args builder_args
|
|
||||||
[[ "${BUILDX_PUSH}" == "1" ]] && output_args=(--push) || output_args=(--load)
|
|
||||||
[[ -n "${BUILDX_BUILDER}" ]] && builder_args=(--builder "${BUILDX_BUILDER}") || builder_args=()
|
|
||||||
|
|
||||||
pull_base_images
|
|
||||||
|
|
||||||
docker buildx build "${builder_args[@]}" \
|
|
||||||
--platform "${TARGET_PLATFORM}" \
|
|
||||||
-f "${SCRIPT_DIR}/Dockerfile" \
|
|
||||||
-t "${CLIENT_IMAGE}" \
|
|
||||||
"${output_args[@]}" \
|
|
||||||
"${REPO_ROOT}"
|
|
||||||
}
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
up|build|buildx|rebuild|down|stop|restart|logs|ps)
|
|
||||||
ACTION="$1"
|
|
||||||
shift
|
|
||||||
;;
|
|
||||||
--mode)
|
|
||||||
[[ $# -ge 2 ]] || die "--mode requires a value"
|
|
||||||
MODE="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
--clients)
|
|
||||||
[[ $# -ge 2 ]] || die "--clients requires a value"
|
|
||||||
CLIENTS="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
--platform)
|
|
||||||
[[ $# -ge 2 ]] || die "--platform requires a value"
|
|
||||||
TARGET_PLATFORM="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
--client-image)
|
|
||||||
[[ $# -ge 2 ]] || die "--client-image requires a value"
|
|
||||||
CLIENT_IMAGE="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
--push)
|
|
||||||
BUILDX_PUSH=1
|
|
||||||
shift
|
|
||||||
;;
|
|
||||||
--builder)
|
|
||||||
[[ $# -ge 2 ]] || die "--builder requires a value"
|
|
||||||
BUILDX_BUILDER="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
--skip-base-pull)
|
|
||||||
SKIP_BASE_IMAGE_PULL=1
|
|
||||||
shift
|
|
||||||
;;
|
|
||||||
--foreground)
|
|
||||||
DETACH=0
|
|
||||||
shift
|
|
||||||
;;
|
|
||||||
--no-build)
|
|
||||||
BUILD=0
|
|
||||||
shift
|
|
||||||
;;
|
|
||||||
-h|--help)
|
|
||||||
usage
|
|
||||||
exit 0
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
die "unknown argument '$1'"
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
[[ "${CLIENTS}" == "single" || "${CLIENTS}" == "multi" ]] || die "unsupported clients '${CLIENTS}', expected single|multi"
|
|
||||||
export RPKI_RTR_CLIENT_IMAGE="${CLIENT_IMAGE}"
|
|
||||||
if [[ "${ACTION}" == "buildx" ]]; then
|
|
||||||
has_buildx || die "docker buildx is not available"
|
|
||||||
else
|
|
||||||
has_compose || die "docker compose is not available"
|
|
||||||
fi
|
|
||||||
|
|
||||||
cd "${REPO_ROOT}"
|
|
||||||
|
|
||||||
case "${ACTION}" in
|
|
||||||
up)
|
|
||||||
up_client
|
|
||||||
;;
|
|
||||||
build)
|
|
||||||
build_client
|
|
||||||
;;
|
|
||||||
buildx)
|
|
||||||
buildx_client
|
|
||||||
;;
|
|
||||||
rebuild)
|
|
||||||
build_client --no-cache
|
|
||||||
up_client
|
|
||||||
;;
|
|
||||||
down|stop)
|
|
||||||
compose_run down
|
|
||||||
;;
|
|
||||||
restart)
|
|
||||||
compose_run down
|
|
||||||
up_client
|
|
||||||
;;
|
|
||||||
logs)
|
|
||||||
compose_run logs -f
|
|
||||||
;;
|
|
||||||
ps)
|
|
||||||
compose_run ps
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
@@ -1,52 +0,0 @@
|
|||||||
version: "3.9"
|
|
||||||
|
|
||||||
services:
|
|
||||||
rtr-client-1:
|
|
||||||
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
|
||||||
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
|
||||||
volumes:
|
|
||||||
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
|
||||||
restart: no
|
|
||||||
networks:
|
|
||||||
- rpki_net
|
|
||||||
|
|
||||||
rtr-client-2:
|
|
||||||
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
|
||||||
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
|
||||||
volumes:
|
|
||||||
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
|
||||||
restart: no
|
|
||||||
networks:
|
|
||||||
- rpki_net
|
|
||||||
|
|
||||||
rtr-client-3:
|
|
||||||
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
|
||||||
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
|
||||||
volumes:
|
|
||||||
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
|
||||||
restart: no
|
|
||||||
networks:
|
|
||||||
- rpki_net
|
|
||||||
|
|
||||||
rtr-client-4:
|
|
||||||
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
|
||||||
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
|
||||||
volumes:
|
|
||||||
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
|
||||||
restart: no
|
|
||||||
networks:
|
|
||||||
- rpki_net
|
|
||||||
|
|
||||||
rtr-client-5:
|
|
||||||
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
|
||||||
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
|
||||||
volumes:
|
|
||||||
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
|
||||||
restart: no
|
|
||||||
networks:
|
|
||||||
- rpki_net
|
|
||||||
|
|
||||||
networks:
|
|
||||||
rpki_net:
|
|
||||||
name: rpki_net
|
|
||||||
driver: bridge
|
|
||||||
@@ -1,36 +0,0 @@
|
|||||||
version: "3.9"
|
|
||||||
|
|
||||||
services:
|
|
||||||
rtr-debug-client:
|
|
||||||
build:
|
|
||||||
context: ../..
|
|
||||||
dockerfile: deploy/client/Dockerfile
|
|
||||||
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
|
||||||
command:
|
|
||||||
[
|
|
||||||
"${RPKI_RTR_SSH_SERVER_ADDR:-rpki-rtr:22}",
|
|
||||||
"${RPKI_RTR_PROTOCOL_VERSION:-2}",
|
|
||||||
"reset",
|
|
||||||
"--ssh",
|
|
||||||
"--ssh-user",
|
|
||||||
"${RPKI_RTR_SSH_USERNAME:-rpki-rtr}",
|
|
||||||
"--ssh-password",
|
|
||||||
"${RPKI_RTR_SSH_PASSWORD}",
|
|
||||||
"--ssh-server-key",
|
|
||||||
"${RPKI_RTR_SSH_SERVER_PUBKEY_PATH:-/app/certs/ssh_host_rsa_key.pub}",
|
|
||||||
"--keep-after-error",
|
|
||||||
"--summary-only"
|
|
||||||
]
|
|
||||||
volumes:
|
|
||||||
- ${RPKI_RTR_CLIENT_KEYS_VOLUME:-../../certs/ssh:/app/certs:ro}
|
|
||||||
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
|
||||||
restart: no
|
|
||||||
stdin_open: true
|
|
||||||
tty: true
|
|
||||||
networks:
|
|
||||||
- rpki_net
|
|
||||||
|
|
||||||
networks:
|
|
||||||
rpki_net:
|
|
||||||
name: rpki_net
|
|
||||||
driver: bridge
|
|
||||||
@@ -1,36 +0,0 @@
|
|||||||
version: "3.9"
|
|
||||||
|
|
||||||
services:
|
|
||||||
rtr-debug-client:
|
|
||||||
build:
|
|
||||||
context: ../..
|
|
||||||
dockerfile: deploy/client/Dockerfile
|
|
||||||
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
|
||||||
command:
|
|
||||||
[
|
|
||||||
"${RPKI_RTR_SSH_SERVER_ADDR:-rpki-rtr:22}",
|
|
||||||
"${RPKI_RTR_PROTOCOL_VERSION:-2}",
|
|
||||||
"reset",
|
|
||||||
"--ssh",
|
|
||||||
"--ssh-user",
|
|
||||||
"${RPKI_RTR_SSH_USERNAME:-rpki-rtr}",
|
|
||||||
"--ssh-key",
|
|
||||||
"${RPKI_RTR_CLIENT_KEY_PATH:-/app/certs/rtr-client.key}",
|
|
||||||
"--ssh-server-key",
|
|
||||||
"${RPKI_RTR_SSH_SERVER_PUBKEY_PATH:-/app/certs/ssh_host_rsa_key.pub}",
|
|
||||||
"--keep-after-error",
|
|
||||||
"--summary-only"
|
|
||||||
]
|
|
||||||
volumes:
|
|
||||||
- ${RPKI_RTR_CLIENT_KEYS_VOLUME:-../../certs/ssh:/app/certs:ro}
|
|
||||||
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
|
||||||
restart: no
|
|
||||||
stdin_open: true
|
|
||||||
tty: true
|
|
||||||
networks:
|
|
||||||
- rpki_net
|
|
||||||
|
|
||||||
networks:
|
|
||||||
rpki_net:
|
|
||||||
name: rpki_net
|
|
||||||
driver: bridge
|
|
||||||
@@ -1,19 +0,0 @@
|
|||||||
services:
|
|
||||||
rtr-debug-client:
|
|
||||||
build:
|
|
||||||
context: ../..
|
|
||||||
dockerfile: deploy/client/Dockerfile
|
|
||||||
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
|
||||||
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
|
||||||
volumes:
|
|
||||||
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
|
||||||
restart: no
|
|
||||||
stdin_open: true
|
|
||||||
tty: true
|
|
||||||
networks:
|
|
||||||
- rpki_net
|
|
||||||
|
|
||||||
networks:
|
|
||||||
rpki_net:
|
|
||||||
name: rpki_net
|
|
||||||
driver: bridge
|
|
||||||
@@ -1,38 +0,0 @@
|
|||||||
version: "3.9"
|
|
||||||
|
|
||||||
services:
|
|
||||||
rtr-debug-client:
|
|
||||||
build:
|
|
||||||
context: ../..
|
|
||||||
dockerfile: deploy/client/Dockerfile
|
|
||||||
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
|
||||||
command:
|
|
||||||
[
|
|
||||||
"${RPKI_RTR_TLS_SERVER_ADDR:-rpki-rtr:324}",
|
|
||||||
"${RPKI_RTR_PROTOCOL_VERSION:-2}",
|
|
||||||
"reset",
|
|
||||||
"--tls",
|
|
||||||
"--ca-cert",
|
|
||||||
"${RPKI_RTR_TLS_CA_CERT_PATH:-/app/certs/client-ca.crt}",
|
|
||||||
"--server-name",
|
|
||||||
"${RPKI_RTR_TLS_SERVER_NAME:-localhost}",
|
|
||||||
"--client-cert",
|
|
||||||
"${RPKI_RTR_TLS_CLIENT_CERT_PATH:-/app/certs/client-good.crt}",
|
|
||||||
"--client-key",
|
|
||||||
"${RPKI_RTR_TLS_CLIENT_KEY_PATH:-/app/certs/client-good.key}",
|
|
||||||
"--keep-after-error",
|
|
||||||
"--summary-only"
|
|
||||||
]
|
|
||||||
volumes:
|
|
||||||
- ${RPKI_RTR_TLS_CERTS_HOST_DIR:-../../certs/tls}:/app/certs:ro
|
|
||||||
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
|
||||||
restart: no
|
|
||||||
stdin_open: true
|
|
||||||
tty: true
|
|
||||||
networks:
|
|
||||||
- rpki_net
|
|
||||||
|
|
||||||
networks:
|
|
||||||
rpki_net:
|
|
||||||
name: rpki_net
|
|
||||||
driver: bridge
|
|
||||||
@@ -1,19 +0,0 @@
|
|||||||
services:
|
|
||||||
rtr-debug-client:
|
|
||||||
build:
|
|
||||||
context: ../..
|
|
||||||
dockerfile: deploy/client/Dockerfile
|
|
||||||
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
|
||||||
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
|
||||||
volumes:
|
|
||||||
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
|
||||||
restart: no
|
|
||||||
stdin_open: true
|
|
||||||
tty: true
|
|
||||||
networks:
|
|
||||||
- rpki_net
|
|
||||||
|
|
||||||
networks:
|
|
||||||
rpki_net:
|
|
||||||
name: rpki_net
|
|
||||||
driver: bridge
|
|
||||||
@@ -1,10 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
set -eu
|
|
||||||
|
|
||||||
mkdir -p /app/logs
|
|
||||||
|
|
||||||
log_name="${HOSTNAME:-rtr-debug-client}"
|
|
||||||
stdout_log="/app/logs/${log_name}.stdout.log"
|
|
||||||
stderr_log="/app/logs/${log_name}.stderr.log"
|
|
||||||
|
|
||||||
exec /usr/local/bin/rtr_debug_client "$@" >>"$stdout_log" 2>>"$stderr_log"
|
|
||||||
@@ -1,4 +0,0 @@
|
|||||||
# RTR server endpoint used by deploy/frr/frr.conf.template.
|
|
||||||
RPKI_FRR_RPKI_HOST=127.0.0.1
|
|
||||||
RPKI_FRR_RPKI_PORT=323
|
|
||||||
RPKI_FRR_RPKI_PREFERENCE=1
|
|
||||||
@@ -1,78 +0,0 @@
|
|||||||
# FRR Minimal RTR Client Config
|
|
||||||
|
|
||||||
中文文档: [README.zh.md](./README.zh.md)
|
|
||||||
|
|
||||||
This folder provides a minimal FRR setup for black-box interop testing
|
|
||||||
against this repository's RTR server defaults.
|
|
||||||
|
|
||||||
Server defaults in this repo:
|
|
||||||
- TCP: `0.0.0.0:323`
|
|
||||||
- TLS: `0.0.0.0:324`
|
|
||||||
|
|
||||||
Reference:
|
|
||||||
- `src/main.rs`
|
|
||||||
|
|
||||||
## Files
|
|
||||||
|
|
||||||
- `daemons.example`: sample `/etc/frr/daemons`
|
|
||||||
- `frr.conf.example`: static sample `/etc/frr/frr.conf`
|
|
||||||
- `frr.conf.template`: Docker template used to generate `/etc/frr/frr.conf`
|
|
||||||
- `entrypoint.sh`: renders `frr.conf.template` from environment variables before starting FRR
|
|
||||||
- `.env`: default Docker environment values
|
|
||||||
|
|
||||||
## How to apply on an FRR host
|
|
||||||
|
|
||||||
1. Copy `daemons.example` to `/etc/frr/daemons`.
|
|
||||||
2. Copy `frr.conf.example` to `/etc/frr/frr.conf`.
|
|
||||||
3. Restart FRR:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo systemctl restart frr
|
|
||||||
```
|
|
||||||
|
|
||||||
## Verify
|
|
||||||
|
|
||||||
```bash
|
|
||||||
vtysh -c "show rpki configuration"
|
|
||||||
vtysh -c "show rpki cache-server"
|
|
||||||
vtysh -c "show rpki cache-connection"
|
|
||||||
vtysh -c "show rpki prefix-table"
|
|
||||||
```
|
|
||||||
|
|
||||||
If `show rpki cache-connection` is connected and `show rpki prefix-table`
|
|
||||||
contains VRPs, the RTR client path is working.
|
|
||||||
|
|
||||||
## Docker quick start
|
|
||||||
|
|
||||||
From repository root:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml up -d
|
|
||||||
docker exec -it frr-rpki-client vtysh -c "show rpki cache-connection"
|
|
||||||
docker exec -it frr-rpki-client vtysh -c "show rpki prefix-table"
|
|
||||||
```
|
|
||||||
|
|
||||||
The Docker example defaults to `127.0.0.1:323`. Override the RTR server
|
|
||||||
endpoint with environment variables:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
RPKI_FRR_RPKI_HOST=10.0.0.12 \
|
|
||||||
RPKI_FRR_RPKI_PORT=323 \
|
|
||||||
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
Stop:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml down
|
|
||||||
```
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
|
|
||||||
- This setup targets RTR over TCP (`rpki cache tcp`).
|
|
||||||
- Docker mode renders `frr.conf.template` into `/etc/frr/frr.conf` at startup.
|
|
||||||
- Keep protocol-level conformance checks in Rust tests and
|
|
||||||
`src/bin/rtr_debug_client`.
|
|
||||||
- `network_mode: host` is kept for the minimal FRR container example, but the
|
|
||||||
RTR server address is controlled by `RPKI_FRR_RPKI_HOST` and
|
|
||||||
`RPKI_FRR_RPKI_PORT`.
|
|
||||||
@@ -1,83 +0,0 @@
|
|||||||
# FRR 最小化 RTR 客户端配置
|
|
||||||
|
|
||||||
本目录提供一个 FRR 最小配置,用于和本仓库 RTR Server 做黑盒互通测试。
|
|
||||||
|
|
||||||
本仓库默认 RTR 监听地址:
|
|
||||||
- TCP:`0.0.0.0:323`
|
|
||||||
- TLS:`0.0.0.0:324`
|
|
||||||
|
|
||||||
参考实现:
|
|
||||||
- `src/main.rs`
|
|
||||||
|
|
||||||
## 文件说明
|
|
||||||
|
|
||||||
- `daemons.example`:示例 `/etc/frr/daemons`
|
|
||||||
- `frr.conf.example`:静态示例 `/etc/frr/frr.conf`
|
|
||||||
- `frr.conf.template`:Docker 启动时生成 `/etc/frr/frr.conf` 的模板
|
|
||||||
- `entrypoint.sh`:启动 FRR 前根据环境变量渲染 `frr.conf.template`
|
|
||||||
- `.env`:Docker 示例使用的默认环境变量
|
|
||||||
|
|
||||||
## 在 FRR 主机上应用
|
|
||||||
|
|
||||||
1. 复制 `daemons.example` 到 `/etc/frr/daemons`
|
|
||||||
2. 复制 `frr.conf.example` 到 `/etc/frr/frr.conf`
|
|
||||||
3. 重启 FRR:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
sudo systemctl restart frr
|
|
||||||
```
|
|
||||||
|
|
||||||
## 验证命令
|
|
||||||
|
|
||||||
```bash
|
|
||||||
vtysh -c "show rpki configuration"
|
|
||||||
vtysh -c "show rpki cache-server"
|
|
||||||
vtysh -c "show rpki cache-connection"
|
|
||||||
vtysh -c "show rpki prefix-table"
|
|
||||||
```
|
|
||||||
|
|
||||||
当 `show rpki cache-connection` 显示已连接,且 `show rpki prefix-table` 出现 VRP 时,说明 RTR 客户端链路工作正常。
|
|
||||||
|
|
||||||
## Docker 快速启动
|
|
||||||
|
|
||||||
在仓库根目录执行:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml up -d
|
|
||||||
docker exec -it frr-rpki-client vtysh -c "show rpki cache-connection"
|
|
||||||
docker exec -it frr-rpki-client vtysh -c "show rpki prefix-table"
|
|
||||||
```
|
|
||||||
|
|
||||||
Docker 示例默认连接 `127.0.0.1:323`。如果 RTR Server 在其他地址,可以通过环境变量覆盖:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
RPKI_FRR_RPKI_HOST=10.0.0.12 \
|
|
||||||
RPKI_FRR_RPKI_PORT=323 \
|
|
||||||
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml up -d
|
|
||||||
```
|
|
||||||
|
|
||||||
停止:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml down
|
|
||||||
```
|
|
||||||
|
|
||||||
## 可测试范围
|
|
||||||
|
|
||||||
- FRR 作为 RTR Client 与本仓库 Server 的 TCP 建链能力
|
|
||||||
- FRR 侧基础会话状态可见性(`cache-server` / `cache-connection`)
|
|
||||||
- VRP 下发与导入是否成功(`prefix-table` 是否有条目)
|
|
||||||
- 基于服务端数据变化触发的前缀表更新(可通过替换 `data` 后观察)
|
|
||||||
- 黑盒互通回归:用于确认“路由器客户端视角”功能可用
|
|
||||||
|
|
||||||
## 不覆盖范围
|
|
||||||
|
|
||||||
- 不替代协议级单元测试/集成测试(PDU 细节、异常路径、边界条件)
|
|
||||||
- 不替代 `src/bin/rtr_debug_client` 的逐报文调试能力
|
|
||||||
- 默认示例以 TCP 为主,TLS/mTLS 需按你的证书与 FRR 配置单独扩展
|
|
||||||
|
|
||||||
## 说明
|
|
||||||
|
|
||||||
- 当前示例主要针对 RTR over TCP(`rpki cache tcp`)
|
|
||||||
- Docker 模式会在启动时把 `frr.conf.template` 渲染为 `/etc/frr/frr.conf`
|
|
||||||
- `network_mode: host` 仍用于最小化 FRR 容器示例,但 RTR Server 地址由 `RPKI_FRR_RPKI_HOST` 和 `RPKI_FRR_RPKI_PORT` 控制
|
|
||||||
@@ -1,7 +0,0 @@
|
|||||||
# Minimal FRR daemons config for RPKI testing
|
|
||||||
|
|
||||||
zebra=yes
|
|
||||||
bgpd=yes
|
|
||||||
|
|
||||||
# Enable bgpd RPKI module
|
|
||||||
bgpd_options=" -A 127.0.0.1 -M rpki"
|
|
||||||
@@ -1,16 +0,0 @@
|
|||||||
services:
|
|
||||||
frr-rpki-client:
|
|
||||||
image: quay.io/frrouting/frr:10.2.2
|
|
||||||
container_name: frr-rpki-client
|
|
||||||
restart: unless-stopped
|
|
||||||
network_mode: host
|
|
||||||
privileged: true
|
|
||||||
environment:
|
|
||||||
RPKI_FRR_RPKI_HOST: "${RPKI_FRR_RPKI_HOST:-127.0.0.1}"
|
|
||||||
RPKI_FRR_RPKI_PORT: "${RPKI_FRR_RPKI_PORT:-323}"
|
|
||||||
RPKI_FRR_RPKI_PREFERENCE: "${RPKI_FRR_RPKI_PREFERENCE:-1}"
|
|
||||||
entrypoint: ["/bin/sh", "/config/entrypoint.sh"]
|
|
||||||
volumes:
|
|
||||||
- ./daemons.example:/etc/frr/daemons:ro
|
|
||||||
- ./frr.conf.template:/config/frr.conf.template:ro
|
|
||||||
- ./entrypoint.sh:/config/entrypoint.sh:ro
|
|
||||||
@@ -1,24 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
set -eu
|
|
||||||
|
|
||||||
: "${RPKI_FRR_RPKI_HOST:=127.0.0.1}"
|
|
||||||
: "${RPKI_FRR_RPKI_PORT:=323}"
|
|
||||||
: "${RPKI_FRR_RPKI_PREFERENCE:=1}"
|
|
||||||
: "${RPKI_FRR_CONFIG_TEMPLATE_PATH:=/config/frr.conf.template}"
|
|
||||||
: "${RPKI_FRR_CONFIG_PATH:=/etc/frr/frr.conf}"
|
|
||||||
|
|
||||||
escape_sed() {
|
|
||||||
printf '%s' "$1" | sed 's/[&|\\]/\\&/g'
|
|
||||||
}
|
|
||||||
|
|
||||||
host="$(escape_sed "${RPKI_FRR_RPKI_HOST}")"
|
|
||||||
port="$(escape_sed "${RPKI_FRR_RPKI_PORT}")"
|
|
||||||
preference="$(escape_sed "${RPKI_FRR_RPKI_PREFERENCE}")"
|
|
||||||
|
|
||||||
sed \
|
|
||||||
-e "s|\${RPKI_FRR_RPKI_HOST}|${host}|g" \
|
|
||||||
-e "s|\${RPKI_FRR_RPKI_PORT}|${port}|g" \
|
|
||||||
-e "s|\${RPKI_FRR_RPKI_PREFERENCE}|${preference}|g" \
|
|
||||||
"${RPKI_FRR_CONFIG_TEMPLATE_PATH}" > "${RPKI_FRR_CONFIG_PATH}"
|
|
||||||
|
|
||||||
exec /usr/lib/frr/docker-start
|
|
||||||
@@ -1,22 +0,0 @@
|
|||||||
frr version 10.2
|
|
||||||
frr defaults traditional
|
|
||||||
hostname rpki-lab
|
|
||||||
service integrated-vtysh-config
|
|
||||||
!
|
|
||||||
debug rpki
|
|
||||||
!
|
|
||||||
rpki
|
|
||||||
rpki polling_period 10
|
|
||||||
rpki timeout 10
|
|
||||||
rpki retry_interval 10
|
|
||||||
rpki expire_interval 7200
|
|
||||||
rpki cache tcp 127.0.0.1 323 preference 1
|
|
||||||
exit
|
|
||||||
!
|
|
||||||
router bgp 65001
|
|
||||||
bgp router-id 192.0.2.1
|
|
||||||
!
|
|
||||||
address-family ipv4 unicast
|
|
||||||
exit-address-family
|
|
||||||
!
|
|
||||||
line vty
|
|
||||||
@@ -1,22 +0,0 @@
|
|||||||
frr version 10.2
|
|
||||||
frr defaults traditional
|
|
||||||
hostname rpki-lab
|
|
||||||
service integrated-vtysh-config
|
|
||||||
!
|
|
||||||
debug rpki
|
|
||||||
!
|
|
||||||
rpki
|
|
||||||
rpki polling_period 10
|
|
||||||
rpki timeout 10
|
|
||||||
rpki retry_interval 10
|
|
||||||
rpki expire_interval 7200
|
|
||||||
rpki cache tcp ${RPKI_FRR_RPKI_HOST} ${RPKI_FRR_RPKI_PORT} preference ${RPKI_FRR_RPKI_PREFERENCE}
|
|
||||||
exit
|
|
||||||
!
|
|
||||||
router bgp 65001
|
|
||||||
bgp router-id 192.0.2.1
|
|
||||||
!
|
|
||||||
address-family ipv4 unicast
|
|
||||||
exit-address-family
|
|
||||||
!
|
|
||||||
line vty
|
|
||||||
@@ -1,49 +0,0 @@
|
|||||||
# Core runtime knobs.
|
|
||||||
# RPKI_RTR_SERVER_IMAGE=rpki-rtr-arm64:v1.2
|
|
||||||
RPKI_RTR_SERVER_IMAGE=rpki-rtr-amd64:v1.2
|
|
||||||
|
|
||||||
# Data source directories on host.
|
|
||||||
RPKI_RTR_CCR_HOST_DIR=../../data
|
|
||||||
RPKI_RTR_SLURM_HOST_DIR=../../data
|
|
||||||
|
|
||||||
# In-container data source directories.
|
|
||||||
RPKI_RTR_CCR_DIR=/app/data
|
|
||||||
RPKI_RTR_SLURM_DIR=/app/slurm
|
|
||||||
RPKI_RTR_REPORT_HOST_DIR=../../report
|
|
||||||
|
|
||||||
# In-container runtime paths.
|
|
||||||
RPKI_RTR_DB_PATH=/app/rtr-db
|
|
||||||
RPKI_RTR_REPORT_DIR=/app/report
|
|
||||||
|
|
||||||
# Persistent directories on host.
|
|
||||||
RPKI_RTR_DB_HOST_DIR=../../rtr-db
|
|
||||||
RPKI_RTR_LOG_HOST_DIR=../../logs/server
|
|
||||||
RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS=300
|
|
||||||
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS=300
|
|
||||||
RPKI_RTR_REPORT_HISTORY_LIMIT=10
|
|
||||||
RPKI_RTR_TIMEZONE=Asia/Shanghai
|
|
||||||
RPKI_RTR_MAX_DELTA=10
|
|
||||||
RPKI_RTR_MAX_CONNECTIONS=100000
|
|
||||||
RPKI_RTR_MAX_CONCURRENT_HANDSHAKES=128
|
|
||||||
RPKI_RTR_ADMIN_ADDR=0.0.0.0:8323
|
|
||||||
RPKI_RTR_ADMIN_TOKEN=qwert
|
|
||||||
RUST_LOG=info
|
|
||||||
|
|
||||||
# TLS mode knobs.
|
|
||||||
RPKI_RTR_ENFORCE_TLS_CLIENT_SAN_IP_MATCH=false
|
|
||||||
RPKI_RTR_TLS_CERT_PATH=/app/certs/server-dns.crt
|
|
||||||
RPKI_RTR_TLS_KEY_PATH=/app/certs/server-dns.key
|
|
||||||
RPKI_RTR_TLS_CLIENT_CA_PATH=/app/certs/client-ca.crt
|
|
||||||
|
|
||||||
RPKI_RTR_TLS_CERTS_HOST_DIR=../../certs/tls
|
|
||||||
|
|
||||||
# SSH mode knobs.
|
|
||||||
RPKI_RTR_SSH_HOST_PORT=2222
|
|
||||||
RPKI_RTR_SSH_CONTAINER_PORT=22
|
|
||||||
RPKI_RTR_SSH_AUTH_MODE=key
|
|
||||||
RPKI_RTR_SSH_USERNAME=rpki-rtr
|
|
||||||
RPKI_RTR_SSH_SUBSYSTEM_NAME=rpki-rtr
|
|
||||||
RPKI_RTR_SSH_HOST_KEY_PATH=/app/certs/ssh_host_rsa_key
|
|
||||||
RPKI_RTR_SSH_AUTHORIZED_KEYS_PATH=/app/certs/rtr-authorized_keys
|
|
||||||
|
|
||||||
RPKI_RTR_SSH_CERTS_HOST_DIR=../../certs/ssh
|
|
||||||
@@ -1,99 +0,0 @@
|
|||||||
# Deployment (Docker Compose)
|
|
||||||
|
|
||||||
This project runs `src/main.rs` as a long-running server that:
|
|
||||||
|
|
||||||
1. loads latest `.ccr` from a configured directory,
|
|
||||||
2. applies optional SLURM filtering,
|
|
||||||
3. starts RTR server.
|
|
||||||
|
|
||||||
The container runs `rpki` directly as PID 1.
|
|
||||||
|
|
||||||
## Files
|
|
||||||
|
|
||||||
- `deploy/server/Dockerfile`
|
|
||||||
- `deploy/server/docker-compose.yml`
|
|
||||||
|
|
||||||
## Runtime Paths in Container
|
|
||||||
|
|
||||||
- CCR directory: `/app/data`
|
|
||||||
- RocksDB directory: `/app/rtr-db`
|
|
||||||
- SLURM directory: `/app/slurm`
|
|
||||||
- Report directory: `/app/report`
|
|
||||||
- TLS cert directory (optional): `/app/certs`
|
|
||||||
|
|
||||||
## Path Configuration via `.env`
|
|
||||||
|
|
||||||
- `RPKI_RTR_CCR_HOST_DIR`: host CCR directory mounted into container
|
|
||||||
- `RPKI_RTR_SLURM_HOST_DIR`: host SLURM directory mounted into container
|
|
||||||
- `RPKI_RTR_CCR_DIR`: in-container CCR directory path
|
|
||||||
- `RPKI_RTR_SLURM_DIR`: in-container SLURM directory path
|
|
||||||
- `RPKI_RTR_DB_HOST_DIR`: host RocksDB directory
|
|
||||||
- `RPKI_RTR_LOG_HOST_DIR`: host log directory
|
|
||||||
- `RPKI_RTR_REPORT_HOST_DIR`: host directory receiving split RTR JSON reports
|
|
||||||
- `RPKI_RTR_DB_PATH`: in-container RocksDB directory
|
|
||||||
- `RPKI_RTR_REPORT_DIR`: in-container report directory
|
|
||||||
|
|
||||||
## Runtime Configuration via `.env`
|
|
||||||
|
|
||||||
- Core: `RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS`, `RPKI_RTR_MAX_DELTA`, `RPKI_RTR_MAX_CONCURRENT_HANDSHAKES`, `RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS`, `RPKI_RTR_REPORT_HISTORY_LIMIT`, `RPKI_RTR_TIMEZONE`, `RPKI_RTR_ADMIN_ADDR`, `RPKI_RTR_ADMIN_TOKEN`, `RUST_LOG`
|
|
||||||
- TCP mode: `RPKI_RTR_MAX_CONNECTIONS`
|
|
||||||
- TLS mode: `RPKI_RTR_ENFORCE_TLS_CLIENT_SAN_IP_MATCH`, `RPKI_RTR_TLS_CERT_PATH`, `RPKI_RTR_TLS_KEY_PATH`, `RPKI_RTR_TLS_CLIENT_CA_PATH`, `RPKI_RTR_TLS_CERTS_HOST_DIR`
|
|
||||||
- SSH mode: `RPKI_RTR_SSH_HOST_PORT`, `RPKI_RTR_SSH_CONTAINER_PORT`, `RPKI_RTR_SSH_AUTH_MODE`, `RPKI_RTR_SSH_USERNAME`, `RPKI_RTR_SSH_SUBSYSTEM_NAME`, `RPKI_RTR_SSH_HOST_KEY_PATH`, `RPKI_RTR_SSH_AUTHORIZED_KEYS_PATH`, `RPKI_RTR_SSH_CERTS_HOST_DIR`
|
|
||||||
|
|
||||||
## Start
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f deploy/server/docker-compose.yml up -d --build
|
|
||||||
```
|
|
||||||
|
|
||||||
## Stop
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f deploy/server/docker-compose.yml down
|
|
||||||
```
|
|
||||||
|
|
||||||
## Logs
|
|
||||||
|
|
||||||
```bash
|
|
||||||
docker compose -f deploy/server/docker-compose.yml logs -f rpki-rtr
|
|
||||||
```
|
|
||||||
|
|
||||||
The admin API can also stream the redirected log file:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl -N "http://127.0.0.1:8323/admin/rtr/logs/tail?stream=stdout&lines=200" \
|
|
||||||
-H "Authorization: Bearer $RPKI_RTR_ADMIN_TOKEN"
|
|
||||||
```
|
|
||||||
|
|
||||||
It reads `/app/logs/${HOSTNAME}.stdout.log` or `.stderr.log` by default. Set
|
|
||||||
`RPKI_RTR_LOG_DIR` and `RPKI_RTR_LOG_NAME` to override that lookup.
|
|
||||||
|
|
||||||
## Runtime Report
|
|
||||||
|
|
||||||
The server writes split JSON reports. Each report file uses a local-time
|
|
||||||
timestamp suffix and each category keeps `RPKI_RTR_REPORT_HISTORY_LIMIT` files,
|
|
||||||
defaulting to 10.
|
|
||||||
|
|
||||||
- `rtr-source-*.json`: CCR and SLURM source metadata, latest refresh status,
|
|
||||||
data quality counts, cache snapshot counts, and delta counts. Written on
|
|
||||||
startup and source refresh events.
|
|
||||||
- `rtr-clients-*.json`: active client connection counts and counts by transport
|
|
||||||
(`tcp`, `tls`, `ssh`). Written on startup, whenever the active connection
|
|
||||||
count changes.
|
|
||||||
- `rtr-runtime-*.json`: service start time, uptime, process RSS, and
|
|
||||||
non-sensitive runtime configuration. Written on startup and every
|
|
||||||
`RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS`, defaulting to 300 seconds.
|
|
||||||
|
|
||||||
Timestamps in logs and report JSON files use `RPKI_RTR_TIMEZONE`, which
|
|
||||||
defaults to `Asia/Shanghai`. Use IANA timezone names such as `Asia/Shanghai`,
|
|
||||||
`Europe/London`, `America/New_York`, or `UTC`; `Shanghai` is accepted as a
|
|
||||||
convenience alias for `Asia/Shanghai`.
|
|
||||||
|
|
||||||
## Runtime Admin Config
|
|
||||||
|
|
||||||
The admin endpoint is disabled by default. Set `RPKI_RTR_ADMIN_ADDR` to enable
|
|
||||||
`POST /admin/rtr/config`. If the address is not loopback, `RPKI_RTR_ADMIN_TOKEN`
|
|
||||||
must also be set and requests must include `Authorization: Bearer <token>`.
|
|
||||||
|
|
||||||
The endpoint accepts partial JSON updates. See `docs/rtr-admin-api.md` for the
|
|
||||||
complete request/response schema, examples, and runtime apply semantics.
|
|
||||||
@@ -1,85 +0,0 @@
|
|||||||
FROM rust:1.89-bookworm AS builder
|
|
||||||
|
|
||||||
WORKDIR /build
|
|
||||||
|
|
||||||
ENV CARGO_BUILD_JOBS=2
|
|
||||||
|
|
||||||
RUN set -eux; \
|
|
||||||
cat > /etc/apt/sources.list.d/debian.sources <<'EOF'
|
|
||||||
Types: deb
|
|
||||||
URIs: http://mirrors.tuna.tsinghua.edu.cn/debian
|
|
||||||
Suites: bookworm bookworm-updates
|
|
||||||
Components: main
|
|
||||||
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
|
|
||||||
|
|
||||||
Types: deb
|
|
||||||
URIs: http://mirrors.tuna.tsinghua.edu.cn/debian-security
|
|
||||||
Suites: bookworm-security
|
|
||||||
Components: main
|
|
||||||
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
|
|
||||||
EOF
|
|
||||||
|
|
||||||
RUN apt-get update \
|
|
||||||
&& apt-get install -y --fix-missing --no-install-recommends \
|
|
||||||
-o Acquire::Retries=10 \
|
|
||||||
-o Acquire::http::Timeout=60 \
|
|
||||||
build-essential \
|
|
||||||
cmake \
|
|
||||||
pkg-config \
|
|
||||||
clang \
|
|
||||||
libclang-dev \
|
|
||||||
libssl-dev \
|
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
|
||||||
|
|
||||||
COPY Cargo.toml Cargo.lock ./
|
|
||||||
COPY src ./src
|
|
||||||
|
|
||||||
RUN cargo build --release --bin rpki_rtr
|
|
||||||
|
|
||||||
FROM debian:bookworm-slim AS runtime
|
|
||||||
|
|
||||||
RUN set -eux; \
|
|
||||||
cat > /etc/apt/sources.list.d/debian.sources <<'EOF'
|
|
||||||
Types: deb
|
|
||||||
URIs: http://mirrors.tuna.tsinghua.edu.cn/debian
|
|
||||||
Suites: bookworm bookworm-updates
|
|
||||||
Components: main
|
|
||||||
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
|
|
||||||
|
|
||||||
Types: deb
|
|
||||||
URIs: http://mirrors.tuna.tsinghua.edu.cn/debian-security
|
|
||||||
Suites: bookworm-security
|
|
||||||
Components: main
|
|
||||||
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
|
|
||||||
EOF
|
|
||||||
|
|
||||||
RUN apt-get update \
|
|
||||||
&& apt-get install -y --fix-missing --no-install-recommends \
|
|
||||||
-o Acquire::Retries=10 \
|
|
||||||
-o Acquire::http::Timeout=60 \
|
|
||||||
ca-certificates \
|
|
||||||
&& rm -rf /var/lib/apt/lists/*
|
|
||||||
|
|
||||||
WORKDIR /app
|
|
||||||
|
|
||||||
COPY --from=builder /build/target/release/rpki_rtr /usr/local/bin/rpki_rtr
|
|
||||||
COPY --chmod=755 deploy/server/entrypoint.sh /usr/local/bin/rpki-rtr-entrypoint.sh
|
|
||||||
|
|
||||||
RUN mkdir -p /app/data /app/rtr-db /app/certs /app/slurm /app/logs /app/report
|
|
||||||
|
|
||||||
ENV RPKI_RTR_ENABLE_TLS=false \
|
|
||||||
RPKI_RTR_TCP_ADDR=0.0.0.0:323 \
|
|
||||||
RPKI_RTR_TLS_ADDR=0.0.0.0:324 \
|
|
||||||
RPKI_RTR_DB_PATH=/app/rtr-db \
|
|
||||||
RPKI_RTR_CCR_DIR=/app/data \
|
|
||||||
RPKI_RTR_SLURM_DIR=/app/slurm \
|
|
||||||
RPKI_RTR_REPORT_DIR=/app/report \
|
|
||||||
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS=300 \
|
|
||||||
RPKI_RTR_REPORT_HISTORY_LIMIT=10 \
|
|
||||||
RPKI_RTR_REFRESH_INTERVAL_SECS=300 \
|
|
||||||
RPKI_RTR_ADMIN_ADDR="" \
|
|
||||||
RPKI_RTR_ADMIN_TOKEN=""
|
|
||||||
|
|
||||||
EXPOSE 323 324
|
|
||||||
|
|
||||||
ENTRYPOINT ["/usr/local/bin/rpki-rtr-entrypoint.sh"]
|
|
||||||
@@ -1,215 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
REPO_ROOT="$(cd -- "${SCRIPT_DIR}/../.." && pwd)"
|
|
||||||
|
|
||||||
MODE="${MODE:-tcp}"
|
|
||||||
ACTION="up"
|
|
||||||
DETACH="${DETACH:-1}"
|
|
||||||
BUILD="${BUILD:-1}"
|
|
||||||
TARGET_PLATFORM="${TARGET_PLATFORM:-linux/arm64}"
|
|
||||||
SERVER_IMAGE="${SERVER_IMAGE:-${RPKI_RTR_SERVER_IMAGE:-rpki-rtr:latest}}"
|
|
||||||
BUILDX_PUSH="${BUILDX_PUSH:-0}"
|
|
||||||
BUILDX_BUILDER="${BUILDX_BUILDER:-}"
|
|
||||||
SKIP_BASE_IMAGE_PULL="${SKIP_BASE_IMAGE_PULL:-0}"
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage:
|
|
||||||
deploy/server/build-run.sh [action] [options]
|
|
||||||
|
|
||||||
Actions:
|
|
||||||
up Build and start server (default)
|
|
||||||
build Build server image with docker compose
|
|
||||||
buildx Build server image with docker buildx
|
|
||||||
rebuild Rebuild image without cache, then start server
|
|
||||||
down|stop Stop server compose stack
|
|
||||||
restart Stop, then build and start server
|
|
||||||
logs Follow server logs
|
|
||||||
ps Show server compose status
|
|
||||||
|
|
||||||
Options:
|
|
||||||
--mode MODE base|tcp|tls|ssh (default: tcp)
|
|
||||||
--platform VALUE Target platform for buildx (default: linux/arm64)
|
|
||||||
--server-image TAG
|
|
||||||
Server image tag for buildx and compose
|
|
||||||
--push Push buildx image instead of loading it locally
|
|
||||||
--builder NAME buildx builder name
|
|
||||||
--skip-base-pull Do not pull Dockerfile base images before buildx
|
|
||||||
--foreground Run server in foreground
|
|
||||||
--no-build Start without --build
|
|
||||||
-h|--help Show this help
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
die() {
|
|
||||||
echo "error: $*" >&2
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
has_compose() {
|
|
||||||
docker compose version >/dev/null 2>&1
|
|
||||||
}
|
|
||||||
|
|
||||||
has_buildx() {
|
|
||||||
docker buildx version >/dev/null 2>&1
|
|
||||||
}
|
|
||||||
|
|
||||||
pull_base_images() {
|
|
||||||
[[ "${SKIP_BASE_IMAGE_PULL}" == "1" ]] && return
|
|
||||||
|
|
||||||
local platform image
|
|
||||||
local -a platforms images
|
|
||||||
IFS=',' read -r -a platforms <<< "${TARGET_PLATFORM}"
|
|
||||||
images=(rust:1.89-bookworm debian:bookworm-slim)
|
|
||||||
|
|
||||||
for platform in "${platforms[@]}"; do
|
|
||||||
for image in "${images[@]}"; do
|
|
||||||
docker pull --platform "${platform}" "${image}"
|
|
||||||
done
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
compose_files() {
|
|
||||||
case "${MODE}" in
|
|
||||||
base) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.yml" ;;
|
|
||||||
tcp) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.tcp.yml" ;;
|
|
||||||
tls) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.tls.yml" ;;
|
|
||||||
ssh) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.ssh.yml" ;;
|
|
||||||
*) die "unsupported mode '${MODE}', expected base|tcp|tls|ssh" ;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
|
|
||||||
compose_args() {
|
|
||||||
local -a args files
|
|
||||||
args=()
|
|
||||||
[[ -f "${SCRIPT_DIR}/.env" ]] && args+=(--env-file "${SCRIPT_DIR}/.env")
|
|
||||||
mapfile -t files < <(compose_files)
|
|
||||||
args+=("${files[@]}")
|
|
||||||
printf '%s\n' "${args[@]}"
|
|
||||||
}
|
|
||||||
|
|
||||||
compose_run() {
|
|
||||||
local -a args
|
|
||||||
mapfile -t args < <(compose_args)
|
|
||||||
docker compose "${args[@]}" "$@"
|
|
||||||
}
|
|
||||||
|
|
||||||
up_server() {
|
|
||||||
local -a up_args
|
|
||||||
up_args=(up)
|
|
||||||
[[ "${DETACH}" == "1" ]] && up_args+=(-d)
|
|
||||||
[[ "${BUILD}" == "1" ]] && up_args+=(--build)
|
|
||||||
compose_run "${up_args[@]}"
|
|
||||||
}
|
|
||||||
|
|
||||||
buildx_server() {
|
|
||||||
has_buildx || die "docker buildx is not available"
|
|
||||||
if [[ "${TARGET_PLATFORM}" == *,* && "${BUILDX_PUSH}" != "1" ]]; then
|
|
||||||
die "multi-platform buildx output requires --push"
|
|
||||||
fi
|
|
||||||
|
|
||||||
local -a output_args builder_args
|
|
||||||
[[ "${BUILDX_PUSH}" == "1" ]] && output_args=(--push) || output_args=(--load)
|
|
||||||
[[ -n "${BUILDX_BUILDER}" ]] && builder_args=(--builder "${BUILDX_BUILDER}") || builder_args=()
|
|
||||||
|
|
||||||
pull_base_images
|
|
||||||
|
|
||||||
docker buildx build "${builder_args[@]}" \
|
|
||||||
--platform "${TARGET_PLATFORM}" \
|
|
||||||
-f "${SCRIPT_DIR}/Dockerfile" \
|
|
||||||
-t "${SERVER_IMAGE}" \
|
|
||||||
"${output_args[@]}" \
|
|
||||||
"${REPO_ROOT}"
|
|
||||||
}
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
up|build|buildx|rebuild|down|stop|restart|logs|ps)
|
|
||||||
ACTION="$1"
|
|
||||||
shift
|
|
||||||
;;
|
|
||||||
--mode)
|
|
||||||
[[ $# -ge 2 ]] || die "--mode requires a value"
|
|
||||||
MODE="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
--platform)
|
|
||||||
[[ $# -ge 2 ]] || die "--platform requires a value"
|
|
||||||
TARGET_PLATFORM="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
--server-image)
|
|
||||||
[[ $# -ge 2 ]] || die "--server-image requires a value"
|
|
||||||
SERVER_IMAGE="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
--push)
|
|
||||||
BUILDX_PUSH=1
|
|
||||||
shift
|
|
||||||
;;
|
|
||||||
--builder)
|
|
||||||
[[ $# -ge 2 ]] || die "--builder requires a value"
|
|
||||||
BUILDX_BUILDER="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
--skip-base-pull)
|
|
||||||
SKIP_BASE_IMAGE_PULL=1
|
|
||||||
shift
|
|
||||||
;;
|
|
||||||
--foreground)
|
|
||||||
DETACH=0
|
|
||||||
shift
|
|
||||||
;;
|
|
||||||
--no-build)
|
|
||||||
BUILD=0
|
|
||||||
shift
|
|
||||||
;;
|
|
||||||
-h|--help)
|
|
||||||
usage
|
|
||||||
exit 0
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
die "unknown argument '$1'"
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
export RPKI_RTR_SERVER_IMAGE="${SERVER_IMAGE}"
|
|
||||||
if [[ "${ACTION}" == "buildx" ]]; then
|
|
||||||
has_buildx || die "docker buildx is not available"
|
|
||||||
else
|
|
||||||
has_compose || die "docker compose is not available"
|
|
||||||
fi
|
|
||||||
|
|
||||||
cd "${REPO_ROOT}"
|
|
||||||
|
|
||||||
case "${ACTION}" in
|
|
||||||
up)
|
|
||||||
up_server
|
|
||||||
;;
|
|
||||||
build)
|
|
||||||
compose_run build
|
|
||||||
;;
|
|
||||||
buildx)
|
|
||||||
buildx_server
|
|
||||||
;;
|
|
||||||
rebuild)
|
|
||||||
compose_run build --no-cache
|
|
||||||
up_server
|
|
||||||
;;
|
|
||||||
down|stop)
|
|
||||||
compose_run down
|
|
||||||
;;
|
|
||||||
restart)
|
|
||||||
compose_run down
|
|
||||||
up_server
|
|
||||||
;;
|
|
||||||
logs)
|
|
||||||
compose_run logs -f
|
|
||||||
;;
|
|
||||||
ps)
|
|
||||||
compose_run ps
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
@@ -1,54 +0,0 @@
|
|||||||
version: "3.9"
|
|
||||||
|
|
||||||
services:
|
|
||||||
rpki-rtr:
|
|
||||||
build:
|
|
||||||
context: ../..
|
|
||||||
dockerfile: deploy/server/Dockerfile
|
|
||||||
image: ${RPKI_RTR_SERVER_IMAGE:-rpki-rtr:latest}
|
|
||||||
container_name: rpki-rtr-ssh
|
|
||||||
restart: no
|
|
||||||
ports:
|
|
||||||
- "323:323"
|
|
||||||
- "${RPKI_RTR_SSH_HOST_PORT:-2222}:${RPKI_RTR_SSH_CONTAINER_PORT:-22}"
|
|
||||||
- "8323:8323"
|
|
||||||
environment:
|
|
||||||
RPKI_RTR_ENABLE_TLS: "false"
|
|
||||||
RPKI_RTR_ENABLE_SSH: "true"
|
|
||||||
RPKI_RTR_TCP_ADDR: "0.0.0.0:323"
|
|
||||||
RPKI_RTR_SSH_ADDR: "0.0.0.0:${RPKI_RTR_SSH_CONTAINER_PORT:-22}"
|
|
||||||
RPKI_RTR_SSH_HOST_KEY_PATH: "${RPKI_RTR_SSH_HOST_KEY_PATH:-/app/certs/ssh_host_rsa_key}"
|
|
||||||
RPKI_RTR_SSH_AUTHORIZED_KEYS_PATH: "${RPKI_RTR_SSH_AUTHORIZED_KEYS_PATH:-/app/certs/rtr-authorized_keys}"
|
|
||||||
RPKI_RTR_SSH_USERNAME: "${RPKI_RTR_SSH_USERNAME:-rpki-rtr}"
|
|
||||||
RPKI_RTR_SSH_SUBSYSTEM_NAME: "${RPKI_RTR_SSH_SUBSYSTEM_NAME:-rpki-rtr}"
|
|
||||||
# SSH auth mode: key | password | both
|
|
||||||
RPKI_RTR_SSH_AUTH_MODE: "${RPKI_RTR_SSH_AUTH_MODE:-key}"
|
|
||||||
# Optional: enable password authentication in addition to publickey
|
|
||||||
# RPKI_RTR_SSH_PASSWORD: "test-password"
|
|
||||||
RPKI_RTR_DB_PATH: "${RPKI_RTR_DB_PATH:-/app/rtr-db}"
|
|
||||||
RPKI_RTR_CCR_DIR: "${RPKI_RTR_CCR_DIR:-/app/data}"
|
|
||||||
RPKI_RTR_SLURM_DIR: "${RPKI_RTR_SLURM_DIR:-/app/slurm}"
|
|
||||||
RPKI_RTR_REPORT_DIR: "${RPKI_RTR_REPORT_DIR:-/app/report}"
|
|
||||||
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS: "${RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS:-300}"
|
|
||||||
RPKI_RTR_REPORT_HISTORY_LIMIT: "${RPKI_RTR_REPORT_HISTORY_LIMIT:-10}"
|
|
||||||
RPKI_RTR_TIMEZONE: "${RPKI_RTR_TIMEZONE:-Asia/Shanghai}"
|
|
||||||
RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS: "${RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS:-300}"
|
|
||||||
RPKI_RTR_MAX_DELTA: "${RPKI_RTR_MAX_DELTA:-10}"
|
|
||||||
RPKI_RTR_MAX_CONCURRENT_HANDSHAKES: "${RPKI_RTR_MAX_CONCURRENT_HANDSHAKES:-128}"
|
|
||||||
RPKI_RTR_ADMIN_ADDR: "${RPKI_RTR_ADMIN_ADDR:-}"
|
|
||||||
RPKI_RTR_ADMIN_TOKEN: "${RPKI_RTR_ADMIN_TOKEN:-}"
|
|
||||||
RUST_LOG: "${RUST_LOG:-info}"
|
|
||||||
volumes:
|
|
||||||
- ${RPKI_RTR_CCR_HOST_DIR:-../../data}:${RPKI_RTR_CCR_DIR:-/app/data}:ro
|
|
||||||
- ${RPKI_RTR_DB_HOST_DIR:-../../rtr-db}:${RPKI_RTR_DB_PATH:-/app/rtr-db}
|
|
||||||
- ${RPKI_RTR_SLURM_HOST_DIR:-../../data}:${RPKI_RTR_SLURM_DIR:-/app/slurm}
|
|
||||||
- ${RPKI_RTR_SSH_CERTS_HOST_DIR:-../../certs/ssh}:/app/certs:ro
|
|
||||||
- ${RPKI_RTR_LOG_HOST_DIR:-../../logs/server}:/app/logs
|
|
||||||
- ${RPKI_RTR_REPORT_HOST_DIR:-../../report}:${RPKI_RTR_REPORT_DIR:-/app/report}
|
|
||||||
networks:
|
|
||||||
- rpki_net
|
|
||||||
|
|
||||||
networks:
|
|
||||||
rpki_net:
|
|
||||||
name: rpki_net
|
|
||||||
driver: bridge
|
|
||||||
@@ -1,44 +0,0 @@
|
|||||||
version: "3.9"
|
|
||||||
|
|
||||||
services:
|
|
||||||
rpki-rtr:
|
|
||||||
build:
|
|
||||||
context: ../..
|
|
||||||
dockerfile: deploy/server/Dockerfile
|
|
||||||
image: ${RPKI_RTR_SERVER_IMAGE:-rpki-rtr:latest}
|
|
||||||
container_name: rpki-rtr-tcp
|
|
||||||
restart: no
|
|
||||||
ports:
|
|
||||||
- "323:323"
|
|
||||||
- "8323:8323"
|
|
||||||
environment:
|
|
||||||
RPKI_RTR_ENABLE_TLS: "false"
|
|
||||||
RPKI_RTR_ENABLE_SSH: "false"
|
|
||||||
RPKI_RTR_TCP_ADDR: "0.0.0.0:323"
|
|
||||||
RPKI_RTR_DB_PATH: "${RPKI_RTR_DB_PATH:-/app/rtr-db}"
|
|
||||||
RPKI_RTR_CCR_DIR: "${RPKI_RTR_CCR_DIR:-/app/data}"
|
|
||||||
RPKI_RTR_SLURM_DIR: "${RPKI_RTR_SLURM_DIR:-/app/slurm}"
|
|
||||||
RPKI_RTR_REPORT_DIR: "${RPKI_RTR_REPORT_DIR:-/app/report}"
|
|
||||||
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS: "${RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS:-300}"
|
|
||||||
RPKI_RTR_REPORT_HISTORY_LIMIT: "${RPKI_RTR_REPORT_HISTORY_LIMIT:-10}"
|
|
||||||
RPKI_RTR_TIMEZONE: "${RPKI_RTR_TIMEZONE:-Asia/Shanghai}"
|
|
||||||
RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS: "${RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS:-60}"
|
|
||||||
RPKI_RTR_MAX_DELTA: "${RPKI_RTR_MAX_DELTA:-10}"
|
|
||||||
RPKI_RTR_MAX_CONNECTIONS: "${RPKI_RTR_MAX_CONNECTIONS:-100000}"
|
|
||||||
RPKI_RTR_MAX_CONCURRENT_HANDSHAKES: "${RPKI_RTR_MAX_CONCURRENT_HANDSHAKES:-128}"
|
|
||||||
RPKI_RTR_ADMIN_ADDR: "${RPKI_RTR_ADMIN_ADDR:-}"
|
|
||||||
RPKI_RTR_ADMIN_TOKEN: "${RPKI_RTR_ADMIN_TOKEN:-}"
|
|
||||||
RUST_LOG: "${RUST_LOG:-info}"
|
|
||||||
volumes:
|
|
||||||
- ${RPKI_RTR_CCR_HOST_DIR:-../../data}:${RPKI_RTR_CCR_DIR:-/app/data}:ro
|
|
||||||
- ${RPKI_RTR_DB_HOST_DIR:-../../rtr-db}:${RPKI_RTR_DB_PATH:-/app/rtr-db}
|
|
||||||
- ${RPKI_RTR_SLURM_HOST_DIR:-../../data}:${RPKI_RTR_SLURM_DIR:-/app/slurm}
|
|
||||||
- ${RPKI_RTR_LOG_HOST_DIR:-../../logs/server}:/app/logs
|
|
||||||
- ${RPKI_RTR_REPORT_HOST_DIR:-../../report}:${RPKI_RTR_REPORT_DIR:-/app/report}
|
|
||||||
networks:
|
|
||||||
- rpki_net
|
|
||||||
|
|
||||||
networks:
|
|
||||||
rpki_net:
|
|
||||||
name: rpki_net
|
|
||||||
driver: bridge
|
|
||||||
@@ -1,48 +0,0 @@
|
|||||||
services:
|
|
||||||
rpki-rtr:
|
|
||||||
build:
|
|
||||||
context: ../..
|
|
||||||
dockerfile: deploy/server/Dockerfile
|
|
||||||
image: ${RPKI_RTR_SERVER_IMAGE:-rpki-rtr:latest}
|
|
||||||
container_name: rpki-rtr-tls
|
|
||||||
restart: no
|
|
||||||
ports:
|
|
||||||
# - "323:323"
|
|
||||||
- "324:324"
|
|
||||||
- "8323:8323"
|
|
||||||
environment:
|
|
||||||
RPKI_RTR_ENABLE_TLS: "true"
|
|
||||||
RPKI_RTR_ENABLE_SSH: "false"
|
|
||||||
# RPKI_RTR_TCP_ADDR: "0.0.0.0:323"
|
|
||||||
RPKI_RTR_TLS_ADDR: "0.0.0.0:324"
|
|
||||||
RPKI_RTR_TLS_CERT_PATH: "${RPKI_RTR_TLS_CERT_PATH:-/app/certs/server-dns.crt}"
|
|
||||||
RPKI_RTR_TLS_KEY_PATH: "${RPKI_RTR_TLS_KEY_PATH:-/app/certs/server-dns.key}"
|
|
||||||
RPKI_RTR_TLS_CLIENT_CA_PATH: "${RPKI_RTR_TLS_CLIENT_CA_PATH:-/app/certs/client-ca.crt}"
|
|
||||||
RPKI_RTR_ENFORCE_TLS_CLIENT_SAN_IP_MATCH: "${RPKI_RTR_ENFORCE_TLS_CLIENT_SAN_IP_MATCH:-false}"
|
|
||||||
RPKI_RTR_DB_PATH: "${RPKI_RTR_DB_PATH:-/app/rtr-db}"
|
|
||||||
RPKI_RTR_CCR_DIR: "${RPKI_RTR_CCR_DIR:-/app/data}"
|
|
||||||
RPKI_RTR_SLURM_DIR: "${RPKI_RTR_SLURM_DIR:-/app/slurm}"
|
|
||||||
RPKI_RTR_REPORT_DIR: "${RPKI_RTR_REPORT_DIR:-/app/report}"
|
|
||||||
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS: "${RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS:-300}"
|
|
||||||
RPKI_RTR_REPORT_HISTORY_LIMIT: "${RPKI_RTR_REPORT_HISTORY_LIMIT:-10}"
|
|
||||||
RPKI_RTR_TIMEZONE: "${RPKI_RTR_TIMEZONE:-Asia/Shanghai}"
|
|
||||||
RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS: "${RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS:-300}"
|
|
||||||
RPKI_RTR_MAX_DELTA: "${RPKI_RTR_MAX_DELTA:-10}"
|
|
||||||
RPKI_RTR_MAX_CONCURRENT_HANDSHAKES: "${RPKI_RTR_MAX_CONCURRENT_HANDSHAKES:-128}"
|
|
||||||
RPKI_RTR_ADMIN_ADDR: "${RPKI_RTR_ADMIN_ADDR:-}"
|
|
||||||
RPKI_RTR_ADMIN_TOKEN: "${RPKI_RTR_ADMIN_TOKEN:-}"
|
|
||||||
RUST_LOG: "${RUST_LOG:-info}"
|
|
||||||
volumes:
|
|
||||||
- ${RPKI_RTR_CCR_HOST_DIR:-../../data}:${RPKI_RTR_CCR_DIR:-/app/data}:ro
|
|
||||||
- ${RPKI_RTR_DB_HOST_DIR:-../../rtr-db}:${RPKI_RTR_DB_PATH:-/app/rtr-db}
|
|
||||||
- ${RPKI_RTR_SLURM_HOST_DIR:-../../data}:${RPKI_RTR_SLURM_DIR:-/app/slurm}
|
|
||||||
- ${RPKI_RTR_TLS_CERTS_HOST_DIR:-../../certs/tls}:/app/certs:ro
|
|
||||||
- ${RPKI_RTR_LOG_HOST_DIR:-../../logs/server}:/app/logs
|
|
||||||
- ${RPKI_RTR_REPORT_HOST_DIR:-../../report}:${RPKI_RTR_REPORT_DIR:-/app/report}
|
|
||||||
networks:
|
|
||||||
- rpki_net
|
|
||||||
|
|
||||||
networks:
|
|
||||||
rpki_net:
|
|
||||||
name: rpki_net
|
|
||||||
driver: bridge
|
|
||||||
@@ -1,58 +0,0 @@
|
|||||||
version: "3.9"
|
|
||||||
|
|
||||||
services:
|
|
||||||
rpki-rtr:
|
|
||||||
build:
|
|
||||||
context: ../..
|
|
||||||
dockerfile: deploy/server/Dockerfile
|
|
||||||
image: ${RPKI_RTR_SERVER_IMAGE:-rpki-rtr:latest}
|
|
||||||
container_name: rpki-rtr
|
|
||||||
restart: no
|
|
||||||
ports:
|
|
||||||
- "323:323"
|
|
||||||
- "324:324"
|
|
||||||
- "8323:8323"
|
|
||||||
# SSH mode example:
|
|
||||||
# - "22:22"
|
|
||||||
environment:
|
|
||||||
RPKI_RTR_ENABLE_TLS: "false"
|
|
||||||
RPKI_RTR_TCP_ADDR: "0.0.0.0:323"
|
|
||||||
RPKI_RTR_TLS_ADDR: "0.0.0.0:324"
|
|
||||||
RPKI_RTR_DB_PATH: "${RPKI_RTR_DB_PATH:-/app/rtr-db}"
|
|
||||||
RPKI_RTR_CCR_DIR: "${RPKI_RTR_CCR_DIR:-/app/data}"
|
|
||||||
RPKI_RTR_SLURM_DIR: "${RPKI_RTR_SLURM_DIR:-/app/slurm}"
|
|
||||||
RPKI_RTR_REPORT_DIR: "${RPKI_RTR_REPORT_DIR:-/app/report}"
|
|
||||||
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS: "${RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS:-300}"
|
|
||||||
RPKI_RTR_REPORT_HISTORY_LIMIT: "${RPKI_RTR_REPORT_HISTORY_LIMIT:-10}"
|
|
||||||
RPKI_RTR_TIMEZONE: "${RPKI_RTR_TIMEZONE:-Asia/Shanghai}"
|
|
||||||
RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS: "${RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS:-300}"
|
|
||||||
RPKI_RTR_MAX_DELTA: "${RPKI_RTR_MAX_DELTA:-10}"
|
|
||||||
RPKI_RTR_MAX_CONCURRENT_HANDSHAKES: "${RPKI_RTR_MAX_CONCURRENT_HANDSHAKES:-128}"
|
|
||||||
RPKI_RTR_ADMIN_ADDR: "${RPKI_RTR_ADMIN_ADDR:-}"
|
|
||||||
RPKI_RTR_ADMIN_TOKEN: "${RPKI_RTR_ADMIN_TOKEN:-}"
|
|
||||||
RUST_LOG: "${RUST_LOG:-info}"
|
|
||||||
# SSH mode example:
|
|
||||||
# RPKI_RTR_ENABLE_SSH: "true"
|
|
||||||
# RPKI_RTR_SSH_ADDR: "0.0.0.0:22"
|
|
||||||
# RPKI_RTR_SSH_PORT: "22"
|
|
||||||
# RPKI_RTR_SSH_HOST_KEY_PATH: "/app/certs/ssh_host_rsa_key"
|
|
||||||
# RPKI_RTR_SSH_AUTHORIZED_KEYS_PATH: "/app/certs/rtr-authorized_keys"
|
|
||||||
# RPKI_RTR_SSH_USERNAME: "rpki-rtr"
|
|
||||||
# RPKI_RTR_SSH_SUBSYSTEM_NAME: "rpki-rtr"
|
|
||||||
# Optional: enable password auth in addition to publickey
|
|
||||||
# RPKI_RTR_SSH_PASSWORD: "test-password"
|
|
||||||
volumes:
|
|
||||||
- ${RPKI_RTR_CCR_HOST_DIR:-../../data}:${RPKI_RTR_CCR_DIR:-/app/data}:ro
|
|
||||||
- ${RPKI_RTR_DB_HOST_DIR:-../../rtr-db}:${RPKI_RTR_DB_PATH:-/app/rtr-db}
|
|
||||||
- ${RPKI_RTR_SLURM_HOST_DIR:-../../data}:${RPKI_RTR_SLURM_DIR:-/app/slurm}
|
|
||||||
- ${RPKI_RTR_LOG_HOST_DIR:-../../logs/server}:/app/logs
|
|
||||||
- ${RPKI_RTR_REPORT_HOST_DIR:-../../report}:${RPKI_RTR_REPORT_DIR:-/app/report}
|
|
||||||
# TLS mode example:
|
|
||||||
# - ../../certs/ssh:/app/certs:ro
|
|
||||||
networks:
|
|
||||||
- rpki_net
|
|
||||||
|
|
||||||
networks:
|
|
||||||
rpki_net:
|
|
||||||
name: rpki_net
|
|
||||||
driver: bridge
|
|
||||||
@@ -1,10 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
set -eu
|
|
||||||
|
|
||||||
mkdir -p /app/logs
|
|
||||||
|
|
||||||
log_name="${HOSTNAME:-rpki-rtr}"
|
|
||||||
stdout_log="/app/logs/${log_name}.stdout.log"
|
|
||||||
stderr_log="/app/logs/${log_name}.stderr.log"
|
|
||||||
|
|
||||||
exec /usr/local/bin/rpki_rtr "$@" >>"$stdout_log" 2>>"$stderr_log"
|
|
||||||
@@ -1,310 +0,0 @@
|
|||||||
# RTR Admin API
|
|
||||||
|
|
||||||
本文档描述 RTR server 当前提供的 HTTP 管理接口。
|
|
||||||
|
|
||||||
## 启用方式
|
|
||||||
|
|
||||||
Admin API 默认关闭。设置 `RPKI_RTR_ADMIN_ADDR` 后启用:
|
|
||||||
|
|
||||||
```env
|
|
||||||
RPKI_RTR_ADMIN_ADDR=127.0.0.1:8323
|
|
||||||
RPKI_RTR_ADMIN_TOKEN=change-me
|
|
||||||
```
|
|
||||||
|
|
||||||
如果在 Docker 中需要从宿主机访问,容器内建议监听:
|
|
||||||
|
|
||||||
```env
|
|
||||||
RPKI_RTR_ADMIN_ADDR=0.0.0.0:8323
|
|
||||||
RPKI_RTR_ADMIN_TOKEN=change-me
|
|
||||||
```
|
|
||||||
|
|
||||||
并在 compose 中映射端口:
|
|
||||||
|
|
||||||
```yaml
|
|
||||||
ports:
|
|
||||||
- "8323:8323"
|
|
||||||
```
|
|
||||||
|
|
||||||
安全规则:
|
|
||||||
|
|
||||||
- `RPKI_RTR_ADMIN_ADDR` 为空时,不启动 Admin API。
|
|
||||||
- 非 loopback 地址,例如 `0.0.0.0:8323`,必须配置 `RPKI_RTR_ADMIN_TOKEN`,否则 admin server 会拒绝启动。
|
|
||||||
- 设置 token 后,请求必须带 `Authorization: Bearer <token>`。
|
|
||||||
|
|
||||||
## 通用 Headers
|
|
||||||
|
|
||||||
```http
|
|
||||||
Authorization: Bearer change-me
|
|
||||||
Content-Type: application/json
|
|
||||||
```
|
|
||||||
|
|
||||||
GET 请求不需要 `Content-Type`。
|
|
||||||
|
|
||||||
## GET /admin/rtr/health
|
|
||||||
|
|
||||||
用于检查 Admin API 是否可达,以及当前启用了哪些管理能力。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl http://127.0.0.1:8323/admin/rtr/health \
|
|
||||||
-H "Authorization: Bearer change-me"
|
|
||||||
```
|
|
||||||
|
|
||||||
响应示例:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"status": "ok",
|
|
||||||
"config_api": true,
|
|
||||||
"source_reload_api": true,
|
|
||||||
"slurm_api": true,
|
|
||||||
"logs_api": true
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## GET /admin/rtr/logs/tail
|
|
||||||
|
|
||||||
实时读取 RTR 服务日志,效果类似 `tail -f`。默认读取 stdout 日志,先返回最近 200 行,然后持续输出新增内容。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl -N "http://127.0.0.1:8323/admin/rtr/logs/tail?stream=stdout&lines=200&follow=true" \
|
|
||||||
-H "Authorization: Bearer change-me"
|
|
||||||
```
|
|
||||||
|
|
||||||
Query 参数:
|
|
||||||
|
|
||||||
| 参数 | 默认值 | 说明 |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `stream` | `stdout` | 可选 `stdout` 或 `stderr`。 |
|
|
||||||
| `lines` | `200` | 首次返回的历史行数,范围会限制在 `1..=5000`。 |
|
|
||||||
| `follow` | `true` | `true` 时保持连接并持续输出新增日志;`false` 时只返回当前 tail 内容。 |
|
|
||||||
|
|
||||||
日志文件路径按容器 entrypoint 的规则解析:
|
|
||||||
|
|
||||||
- 目录:`RPKI_RTR_LOG_DIR`,默认 `/app/logs`。
|
|
||||||
- 文件名前缀:`RPKI_RTR_LOG_NAME`,未设置时使用 `HOSTNAME`,再未设置时为 `rpki-rtr`。
|
|
||||||
- stdout 文件:`<name>.stdout.log`。
|
|
||||||
- stderr 文件:`<name>.stderr.log`。
|
|
||||||
|
|
||||||
## GET /admin/rtr/config
|
|
||||||
|
|
||||||
查询当前运行中的 runtime 配置。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl http://127.0.0.1:8323/admin/rtr/config \
|
|
||||||
-H "Authorization: Bearer change-me"
|
|
||||||
```
|
|
||||||
|
|
||||||
响应示例:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"status": "ok",
|
|
||||||
"config": {
|
|
||||||
"max_delta": 10,
|
|
||||||
"prune_delta_by_snapshot_size": true,
|
|
||||||
"source_refresh_interval_seconds": 300,
|
|
||||||
"runtime_report_interval_seconds": 300,
|
|
||||||
"report_history_limit": 10,
|
|
||||||
"timezone": "Asia/Shanghai",
|
|
||||||
"timing": {
|
|
||||||
"refresh": 3600,
|
|
||||||
"retry": 600,
|
|
||||||
"expire": 7200
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## POST /admin/rtr/config
|
|
||||||
|
|
||||||
运行中动态修改 RTR server 的部分 runtime 配置。请求体是 JSON object,支持部分更新,只需要传要修改的字段。
|
|
||||||
|
|
||||||
| 字段 | 类型 | 说明 |
|
|
||||||
| --- | --- | --- |
|
|
||||||
| `max_delta` | integer | 每个 RTR 版本最多保留的 delta 条数,必须 `>= 1`。 |
|
|
||||||
| `prune_delta_by_snapshot_size` | boolean | 是否按 snapshot wire size 裁剪 delta window。 |
|
|
||||||
| `source_refresh_interval_seconds` | integer | CCR/SLURM source refresh 间隔,单位秒,必须 `>= 1`。 |
|
|
||||||
| `runtime_report_interval_seconds` | integer | `rtr-runtime-*.json` 周期写入间隔,单位秒,必须 `>= 1`。 |
|
|
||||||
| `report_history_limit` | integer | 每类 report 文件滚动保留数量,必须 `>= 1`。 |
|
|
||||||
| `timezone` | string | IANA 时区名,例如 `Asia/Shanghai`、`UTC`、`Europe/London`。 |
|
|
||||||
| `timing.refresh` | integer | RTR EndOfData `refresh`,单位秒。 |
|
|
||||||
| `timing.retry` | integer | RTR EndOfData `retry`,单位秒。 |
|
|
||||||
| `timing.expire` | integer | RTR EndOfData `expire`,必须大于 `refresh` 和 `retry`。 |
|
|
||||||
|
|
||||||
示例:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl -X POST http://127.0.0.1:8323/admin/rtr/config \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-H "Authorization: Bearer change-me" \
|
|
||||||
-d '{"max_delta": 6}'
|
|
||||||
```
|
|
||||||
|
|
||||||
成功响应:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"status": "ok",
|
|
||||||
"config": {
|
|
||||||
"max_delta": 6,
|
|
||||||
"prune_delta_by_snapshot_size": true,
|
|
||||||
"source_refresh_interval_seconds": 300,
|
|
||||||
"runtime_report_interval_seconds": 300,
|
|
||||||
"report_history_limit": 10,
|
|
||||||
"timezone": "Asia/Shanghai",
|
|
||||||
"timing": {
|
|
||||||
"refresh": 3600,
|
|
||||||
"retry": 600,
|
|
||||||
"expire": 7200
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## GET /admin/rtr/slurm/files
|
|
||||||
|
|
||||||
列出 `RPKI_RTR_SLURM_DIR` 下的 SLURM 文件。只返回合法的 `*.slurm` 和 `*.slurm.disabled` 文件,不返回 `.backup/`。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl http://127.0.0.1:8323/admin/rtr/slurm/files \
|
|
||||||
-H "Authorization: Bearer change-me"
|
|
||||||
```
|
|
||||||
|
|
||||||
响应示例:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"status": "ok",
|
|
||||||
"files": [
|
|
||||||
{
|
|
||||||
"name": "local-policy.slurm",
|
|
||||||
"path": "/app/slurm/local-policy.slurm",
|
|
||||||
"enabled": true,
|
|
||||||
"size_bytes": 168,
|
|
||||||
"modified_unix_seconds": 1782100000
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"name": "old-policy.slurm.disabled",
|
|
||||||
"path": "/app/slurm/old-policy.slurm.disabled",
|
|
||||||
"enabled": false,
|
|
||||||
"size_bytes": 168,
|
|
||||||
"modified_unix_seconds": 1782100100
|
|
||||||
}
|
|
||||||
]
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## GET /admin/rtr/slurm/files/{name}
|
|
||||||
|
|
||||||
读取单个 SLURM 文件,前端编辑页面可以直接使用这个接口加载内容。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl http://127.0.0.1:8323/admin/rtr/slurm/files/local-policy.slurm \
|
|
||||||
-H "Authorization: Bearer change-me"
|
|
||||||
```
|
|
||||||
|
|
||||||
响应示例:
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"status": "ok",
|
|
||||||
"file": {
|
|
||||||
"name": "local-policy.slurm",
|
|
||||||
"path": "/app/slurm/local-policy.slurm",
|
|
||||||
"enabled": true,
|
|
||||||
"size_bytes": 168,
|
|
||||||
"modified_unix_seconds": 1782100000,
|
|
||||||
"content": "{ \"slurmVersion\": 1, \"validationOutputFilters\": { \"prefixFilters\": [], \"bgpsecFilters\": [] }, \"locallyAddedAssertions\": { \"prefixAssertions\": [], \"bgpsecAssertions\": [] } }"
|
|
||||||
}
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## POST /admin/rtr/slurm/files
|
|
||||||
|
|
||||||
新增或覆盖一个 SLURM 文件。文件名放在请求体中。
|
|
||||||
|
|
||||||
```json
|
|
||||||
{
|
|
||||||
"name": "local-policy.slurm",
|
|
||||||
"content": "{ \"slurmVersion\": 1, \"validationOutputFilters\": { \"prefixFilters\": [], \"bgpsecFilters\": [] }, \"locallyAddedAssertions\": { \"prefixAssertions\": [], \"bgpsecAssertions\": [] } }",
|
|
||||||
"reload": true
|
|
||||||
}
|
|
||||||
```
|
|
||||||
|
|
||||||
## PUT /admin/rtr/slurm/files/{name}
|
|
||||||
|
|
||||||
新增或覆盖指定 SLURM 文件。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl -X PUT http://127.0.0.1:8323/admin/rtr/slurm/files/local-policy.slurm \
|
|
||||||
-H "Content-Type: application/json" \
|
|
||||||
-H "Authorization: Bearer change-me" \
|
|
||||||
-d '{
|
|
||||||
"content": "{ \"slurmVersion\": 1, \"validationOutputFilters\": { \"prefixFilters\": [], \"bgpsecFilters\": [] }, \"locallyAddedAssertions\": { \"prefixAssertions\": [], \"bgpsecAssertions\": [] } }",
|
|
||||||
"reload": true
|
|
||||||
}'
|
|
||||||
```
|
|
||||||
|
|
||||||
## DELETE /admin/rtr/slurm/files/{name}
|
|
||||||
|
|
||||||
删除指定 SLURM 文件。删除前会先备份。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl -X DELETE "http://127.0.0.1:8323/admin/rtr/slurm/files/local-policy.slurm?reload=true" \
|
|
||||||
-H "Authorization: Bearer change-me"
|
|
||||||
```
|
|
||||||
|
|
||||||
## POST /admin/rtr/slurm/files/{name}/disable
|
|
||||||
|
|
||||||
将启用文件改名为失效文件:
|
|
||||||
|
|
||||||
```text
|
|
||||||
local-policy.slurm -> local-policy.slurm.disabled
|
|
||||||
```
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl -X POST "http://127.0.0.1:8323/admin/rtr/slurm/files/local-policy.slurm/disable?reload=true" \
|
|
||||||
-H "Authorization: Bearer change-me"
|
|
||||||
```
|
|
||||||
|
|
||||||
## POST /admin/rtr/slurm/files/{name}/enable
|
|
||||||
|
|
||||||
将失效文件改名为启用文件:
|
|
||||||
|
|
||||||
```text
|
|
||||||
local-policy.slurm.disabled -> local-policy.slurm
|
|
||||||
```
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl -X POST "http://127.0.0.1:8323/admin/rtr/slurm/files/local-policy.slurm.disabled/enable?reload=true" \
|
|
||||||
-H "Authorization: Bearer change-me"
|
|
||||||
```
|
|
||||||
|
|
||||||
## POST /admin/rtr/slurm/reload
|
|
||||||
|
|
||||||
不修改文件,只立即触发一次 CCR/SLURM source reload。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
curl -X POST http://127.0.0.1:8323/admin/rtr/slurm/reload \
|
|
||||||
-H "Authorization: Bearer change-me"
|
|
||||||
```
|
|
||||||
|
|
||||||
## SLURM 文件规则
|
|
||||||
|
|
||||||
- `*.slurm`:启用,会被 source refresh 加载。
|
|
||||||
- `*.slurm.disabled`:失效,不会被加载。
|
|
||||||
- `.backup/`:接口自动生成的备份目录。
|
|
||||||
- 文件名只能是 basename,不能包含 `/`、`\` 或 `..`。
|
|
||||||
- 文件名只能包含 ASCII 字母、数字、`.`、`_`、`-`。
|
|
||||||
- 写入内容必须是合法 JSON,并且能被解析为合法 SLURM 文件。
|
|
||||||
- 单个请求体默认限制为 5 MiB。
|
|
||||||
- 覆盖、删除前会写入 `.backup/` 备份。
|
|
||||||
|
|
||||||
## 错误响应
|
|
||||||
|
|
||||||
| HTTP 状态码 | 含义 |
|
|
||||||
| --- | --- |
|
|
||||||
| `400 Bad Request` | JSON 非法、配置值非法、SLURM 文件名非法、目标文件不存在、reload 失败等。 |
|
|
||||||
| `401 Unauthorized` | 配置了 token,但请求缺少或使用了错误的 `Authorization`。 |
|
|
||||||
| `404 Not Found` | 路径或方法不支持。 |
|
|
||||||
| `413 Payload Too Large` | 请求体超过限制。 |
|
|
||||||
@@ -1,203 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
|
||||||
IMAGE_DIR="${ROOT_DIR}/images"
|
|
||||||
SERVER_ENV_FILE="${ROOT_DIR}/deploy/server/.env"
|
|
||||||
CONSOLE_ENV_FILE="${ROOT_DIR}/deploy/console/.env"
|
|
||||||
|
|
||||||
SERVER_TAR="${RPKI_RTR_SERVER_IMAGE_TAR:-}"
|
|
||||||
CLIENT_TAR="${RPKI_RTR_CLIENT_IMAGE_TAR:-}"
|
|
||||||
CONSOLE_TAR="${RPKI_RTR_CONSOLE_IMAGE_TAR:-}"
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage: scripts/00_0_install.sh [options]
|
|
||||||
|
|
||||||
Load offline Docker image tarballs from ./images, then run scripts/00_1_pre_check.sh.
|
|
||||||
|
|
||||||
Options:
|
|
||||||
--server-tar FILE Server image tarball. Env: RPKI_RTR_SERVER_IMAGE_TAR
|
|
||||||
--client-tar FILE Debug client image tarball. Env: RPKI_RTR_CLIENT_IMAGE_TAR
|
|
||||||
--console-tar FILE Console image tarball. Env: RPKI_RTR_CONSOLE_IMAGE_TAR
|
|
||||||
-h, --help Show this help.
|
|
||||||
|
|
||||||
Default tarballs under ./images:
|
|
||||||
server: rpki-rtr-*.tar, excluding rpki-rtr-debug-client-*.tar
|
|
||||||
client: rpki-rtr-debug-client-*.tar
|
|
||||||
console: rpki-console-ui-*.tar
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
log() {
|
|
||||||
printf '[00_install] %s\n' "$*" >&2
|
|
||||||
}
|
|
||||||
|
|
||||||
stage() {
|
|
||||||
printf '\n====== %s ======\n' "$*" >&2
|
|
||||||
}
|
|
||||||
|
|
||||||
die() {
|
|
||||||
printf '[00_install] ERROR: %s\n' "$*" >&2
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
set_env_var() {
|
|
||||||
local env_file="$1"
|
|
||||||
local key="$2"
|
|
||||||
local value="$3"
|
|
||||||
local tmp_file
|
|
||||||
|
|
||||||
[ -f "${env_file}" ] || die "env file not found: ${env_file}"
|
|
||||||
tmp_file="${env_file}.tmp.$$"
|
|
||||||
|
|
||||||
awk -v key="${key}" -v value="${value}" '
|
|
||||||
BEGIN { done = 0 }
|
|
||||||
$0 ~ "^[[:space:]]*#?[[:space:]]*" key "=" {
|
|
||||||
if (!done) {
|
|
||||||
print key "=" value
|
|
||||||
done = 1
|
|
||||||
}
|
|
||||||
next
|
|
||||||
}
|
|
||||||
{ print }
|
|
||||||
END {
|
|
||||||
if (!done) {
|
|
||||||
print key "=" value
|
|
||||||
}
|
|
||||||
}
|
|
||||||
' "${env_file}" >"${tmp_file}"
|
|
||||||
mv "${tmp_file}" "${env_file}"
|
|
||||||
}
|
|
||||||
|
|
||||||
while [ "$#" -gt 0 ]; do
|
|
||||||
case "$1" in
|
|
||||||
--server-tar)
|
|
||||||
[ "$#" -ge 2 ] || die "--server-tar requires a value"
|
|
||||||
SERVER_TAR="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
--client-tar)
|
|
||||||
[ "$#" -ge 2 ] || die "--client-tar requires a value"
|
|
||||||
CLIENT_TAR="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
--console-tar)
|
|
||||||
[ "$#" -ge 2 ] || die "--console-tar requires a value"
|
|
||||||
CONSOLE_TAR="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
-h|--help)
|
|
||||||
usage
|
|
||||||
exit 0
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
die "unknown option: $1"
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
resolve_tar() {
|
|
||||||
local path="$1"
|
|
||||||
|
|
||||||
if [ -f "${path}" ]; then
|
|
||||||
cd "$(dirname "${path}")" && printf '%s/%s\n' "$(pwd -P)" "$(basename "${path}")"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [ -f "${ROOT_DIR}/${path}" ]; then
|
|
||||||
cd "$(dirname "${ROOT_DIR}/${path}")" && printf '%s/%s\n' "$(pwd -P)" "$(basename "${path}")"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
|
|
||||||
load_image() {
|
|
||||||
local label="$1"
|
|
||||||
local tar_path="$2"
|
|
||||||
local output image
|
|
||||||
|
|
||||||
[ -f "${tar_path}" ] || die "${label} image tarball not found: ${tar_path}"
|
|
||||||
output="$(docker load -i "${tar_path}" 2>&1)" || {
|
|
||||||
printf '%s\n' "${output}" >&2
|
|
||||||
die "failed to load ${label} image from ${tar_path}"
|
|
||||||
}
|
|
||||||
printf '%s\n' "${output}" >&2
|
|
||||||
|
|
||||||
image="$(printf '%s\n' "${output}" | sed -n 's/^Loaded image: //p' | tail -n 1)"
|
|
||||||
[ -n "${image}" ] || die "cannot determine ${label} image name from docker load output"
|
|
||||||
printf '%s\n' "${image}"
|
|
||||||
}
|
|
||||||
|
|
||||||
find_single_tar() {
|
|
||||||
local label="$1"
|
|
||||||
local include_pattern="$2"
|
|
||||||
local exclude_pattern="${3:-}"
|
|
||||||
local matches=()
|
|
||||||
local file
|
|
||||||
|
|
||||||
while IFS= read -r -d '' file; do
|
|
||||||
matches+=("${file}")
|
|
||||||
done < <(
|
|
||||||
if [ -n "${exclude_pattern}" ]; then
|
|
||||||
find "${IMAGE_DIR}" -maxdepth 1 -type f -name "${include_pattern}" ! -name "${exclude_pattern}" -print0 | sort -z
|
|
||||||
else
|
|
||||||
find "${IMAGE_DIR}" -maxdepth 1 -type f -name "${include_pattern}" -print0 | sort -z
|
|
||||||
fi
|
|
||||||
)
|
|
||||||
|
|
||||||
case "${#matches[@]}" in
|
|
||||||
0)
|
|
||||||
die "no ${label} image tarball found under ${IMAGE_DIR} matching ${include_pattern}"
|
|
||||||
;;
|
|
||||||
1)
|
|
||||||
printf '%s\n' "${matches[0]}"
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
printf '[00_install] ERROR: multiple %s image tarballs found:\n' "${label}" >&2
|
|
||||||
printf ' %s\n' "${matches[@]}" >&2
|
|
||||||
die "please specify --${label}-tar or RPKI_RTR_${label^^}_IMAGE_TAR"
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
|
|
||||||
stage "Checking Docker"
|
|
||||||
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
|
||||||
[ -d "${IMAGE_DIR}" ] || die "image directory not found: ${IMAGE_DIR}"
|
|
||||||
[ -f "${SERVER_ENV_FILE}" ] || die "server env file not found: ${SERVER_ENV_FILE}"
|
|
||||||
|
|
||||||
if [ -n "${SERVER_TAR}" ]; then
|
|
||||||
SERVER_TAR="$(resolve_tar "${SERVER_TAR}")" || die "server image tarball not found: ${SERVER_TAR}"
|
|
||||||
else
|
|
||||||
SERVER_TAR="$(find_single_tar server 'rpki-rtr-*.tar' 'rpki-rtr-debug-client-*.tar')"
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [ -n "${CLIENT_TAR}" ]; then
|
|
||||||
CLIENT_TAR="$(resolve_tar "${CLIENT_TAR}")" || die "client image tarball not found: ${CLIENT_TAR}"
|
|
||||||
else
|
|
||||||
CLIENT_TAR="$(find_single_tar client 'rpki-rtr-debug-client-*.tar')"
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [ -n "${CONSOLE_TAR}" ]; then
|
|
||||||
CONSOLE_TAR="$(resolve_tar "${CONSOLE_TAR}")" || die "console image tarball not found: ${CONSOLE_TAR}"
|
|
||||||
else
|
|
||||||
CONSOLE_TAR="$(find_single_tar console 'rpki-console-ui-*.tar')"
|
|
||||||
fi
|
|
||||||
|
|
||||||
stage "Loading Docker Images"
|
|
||||||
SERVER_IMAGE="$(load_image server "${SERVER_TAR}")"
|
|
||||||
CLIENT_IMAGE="$(load_image client "${CLIENT_TAR}")"
|
|
||||||
CONSOLE_IMAGE="$(load_image console "${CONSOLE_TAR}")"
|
|
||||||
|
|
||||||
stage "Updating Deploy Env"
|
|
||||||
set_env_var "${SERVER_ENV_FILE}" RPKI_RTR_SERVER_IMAGE "${SERVER_IMAGE}"
|
|
||||||
set_env_var "${CONSOLE_ENV_FILE}" RPKI_CONSOLE_UI_IMAGE "${CONSOLE_IMAGE}"
|
|
||||||
|
|
||||||
stage "Running Pre-Check"
|
|
||||||
RPKI_RTR_SERVER_IMAGE="${SERVER_IMAGE}" \
|
|
||||||
RPKI_RTR_CLIENT_IMAGE="${CLIENT_IMAGE}" \
|
|
||||||
bash "${SCRIPT_DIR}/00_1_pre_check.sh" || die "pre-check failed"
|
|
||||||
|
|
||||||
stage "Install Finished"
|
|
||||||
@@ -1,155 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
|
||||||
|
|
||||||
PRECHECK_WORK_DIR="${PRECHECK_WORK_DIR:-${ROOT_DIR}/.precheck}"
|
|
||||||
PRECHECK_ENV_FILE="${PRECHECK_ENV_FILE:-${PRECHECK_WORK_DIR}/precheck.env}"
|
|
||||||
PRECHECK_WAIT_SECS="${PRECHECK_WAIT_SECS:-60}"
|
|
||||||
KEEP_PRECHECK="${KEEP_PRECHECK:-0}"
|
|
||||||
|
|
||||||
log() {
|
|
||||||
printf '[00_1_pre_check] %s\n' "$*"
|
|
||||||
}
|
|
||||||
|
|
||||||
stage() {
|
|
||||||
printf '\n====== %s ======\n' "$*"
|
|
||||||
}
|
|
||||||
|
|
||||||
die() {
|
|
||||||
printf '[00_1_pre_check] ERROR: %s\n' "$*" >&2
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
|
||||||
if docker compose version >/dev/null 2>&1; then
|
|
||||||
COMPOSE_BIN=(docker compose)
|
|
||||||
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
|
||||||
COMPOSE_BIN=(docker-compose)
|
|
||||||
else
|
|
||||||
die "docker compose is not available"
|
|
||||||
fi
|
|
||||||
|
|
||||||
stage "Preparing Pre-Check Data"
|
|
||||||
bash "${SCRIPT_DIR}/00_2_pre_check_prepare.sh" || die "prepare step failed"
|
|
||||||
|
|
||||||
set -a
|
|
||||||
# shellcheck disable=SC1090
|
|
||||||
. "${PRECHECK_ENV_FILE}"
|
|
||||||
set +a
|
|
||||||
|
|
||||||
ADMIN_PORT="${RPKI_RTR_ADMIN_ADDR##*:}"
|
|
||||||
ADMIN_TOKEN="${RPKI_RTR_ADMIN_TOKEN}"
|
|
||||||
|
|
||||||
server_compose() {
|
|
||||||
"${COMPOSE_BIN[@]}" --env-file "${PRECHECK_ENV_FILE}" -p "${PRECHECK_PROJECT}" -f "${PRECHECK_SERVER_COMPOSE_FILE}" "$@"
|
|
||||||
}
|
|
||||||
|
|
||||||
client_compose() {
|
|
||||||
"${COMPOSE_BIN[@]}" --env-file "${PRECHECK_ENV_FILE}" -p "${PRECHECK_PROJECT}" -f "${PRECHECK_CLIENT_COMPOSE_FILE}" "$@"
|
|
||||||
}
|
|
||||||
|
|
||||||
cleanup() {
|
|
||||||
KEEP_PRECHECK="${KEEP_PRECHECK}" PRECHECK_WORK_DIR="${PRECHECK_WORK_DIR}" PRECHECK_ENV_FILE="${PRECHECK_ENV_FILE}" \
|
|
||||||
bash "${SCRIPT_DIR}/00_3_pre_check_cleanup.sh"
|
|
||||||
}
|
|
||||||
trap cleanup EXIT
|
|
||||||
|
|
||||||
http_get() {
|
|
||||||
local host="$1"
|
|
||||||
local port="$2"
|
|
||||||
local path="$3"
|
|
||||||
local token="$4"
|
|
||||||
|
|
||||||
exec 3<>"/dev/tcp/${host}/${port}" || return 1
|
|
||||||
printf 'GET %s HTTP/1.1\r\nHost: %s:%s\r\nAuthorization: Bearer %s\r\nConnection: close\r\n\r\n' \
|
|
||||||
"${path}" "${host}" "${port}" "${token}" >&3
|
|
||||||
cat <&3
|
|
||||||
exec 3>&-
|
|
||||||
}
|
|
||||||
|
|
||||||
wait_for_admin_health() {
|
|
||||||
local deadline=$((SECONDS + PRECHECK_WAIT_SECS))
|
|
||||||
local response
|
|
||||||
|
|
||||||
log "waiting for server health"
|
|
||||||
while [ "${SECONDS}" -lt "${deadline}" ]; do
|
|
||||||
response="$(http_get 127.0.0.1 "${ADMIN_PORT}" /admin/rtr/health "${ADMIN_TOKEN}" 2>/dev/null || true)"
|
|
||||||
if printf '%s' "${response}" | grep -q '"status"[[:space:]]*:[[:space:]]*"ok"'; then
|
|
||||||
log "admin health check passed"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [ "$(server_compose ps -q rpki-rtr | wc -l | tr -d ' ')" = "0" ]; then
|
|
||||||
server_compose logs rpki-rtr || true
|
|
||||||
die "server container is not running"
|
|
||||||
fi
|
|
||||||
sleep 1
|
|
||||||
done
|
|
||||||
|
|
||||||
server_compose logs rpki-rtr || true
|
|
||||||
die "admin health check did not pass within ${PRECHECK_WAIT_SECS}s"
|
|
||||||
}
|
|
||||||
|
|
||||||
wait_for_client_eod() {
|
|
||||||
local deadline=$((SECONDS + PRECHECK_WAIT_SECS))
|
|
||||||
local log_file
|
|
||||||
|
|
||||||
log "waiting for client response"
|
|
||||||
while [ "${SECONDS}" -lt "${deadline}" ]; do
|
|
||||||
for log_file in "${PRECHECK_WORK_DIR}"/logs/client/*.stdout.log; do
|
|
||||||
[ -f "${log_file}" ] || continue
|
|
||||||
if grep -q 'EndOfData:' "${log_file}" && grep -q 'summary[[:space:]]*:' "${log_file}"; then
|
|
||||||
log "client received EndOfData and payload summary"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
if grep -qi 'Error Report' "${log_file}"; then
|
|
||||||
cat "${log_file}" >&2
|
|
||||||
die "client received RTR Error Report"
|
|
||||||
fi
|
|
||||||
done
|
|
||||||
sleep 1
|
|
||||||
done
|
|
||||||
|
|
||||||
find "${PRECHECK_WORK_DIR}/logs/client" -type f -maxdepth 1 -print -exec cat {} \; >&2 || true
|
|
||||||
die "client did not receive EndOfData and summary within ${PRECHECK_WAIT_SECS}s"
|
|
||||||
}
|
|
||||||
|
|
||||||
assert_no_error_logs() {
|
|
||||||
local bad
|
|
||||||
log "checking logs"
|
|
||||||
bad="$(find "${PRECHECK_WORK_DIR}/logs" -type f -name '*.log' -print0 \
|
|
||||||
| xargs -0 grep -Ein 'panic|failed|RTR admin config server exited|Error Report' 2>/dev/null || true)"
|
|
||||||
if [ -n "${bad}" ]; then
|
|
||||||
printf '%s\n' "${bad}" >&2
|
|
||||||
die "error pattern found in pre-check logs"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
assert_report_written() {
|
|
||||||
local deadline=$((SECONDS + PRECHECK_WAIT_SECS))
|
|
||||||
|
|
||||||
log "waiting for source report"
|
|
||||||
while [ "${SECONDS}" -lt "${deadline}" ]; do
|
|
||||||
if find "${PRECHECK_WORK_DIR}/report" -type f -name 'rtr-source-*.json' | grep -q .; then
|
|
||||||
log "source report written"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
sleep 1
|
|
||||||
done
|
|
||||||
|
|
||||||
die "server did not write rtr-source report within ${PRECHECK_WAIT_SECS}s"
|
|
||||||
}
|
|
||||||
|
|
||||||
stage "Starting RTR Server"
|
|
||||||
server_compose up -d --no-build rpki-rtr
|
|
||||||
wait_for_admin_health
|
|
||||||
assert_report_written
|
|
||||||
|
|
||||||
stage "Starting Debug Client"
|
|
||||||
client_compose up -d --no-build rtr-debug-client
|
|
||||||
wait_for_client_eod
|
|
||||||
|
|
||||||
assert_no_error_logs
|
|
||||||
stage "Pre-Check Passed"
|
|
||||||
@@ -1,54 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
OUT_PATH="${1:-}"
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage: scripts/00_2_generate_ccr.sh OUT.ccr
|
|
||||||
|
|
||||||
Generate a small deterministic CCR snapshot for pre-check.
|
|
||||||
This script writes a fixed base64-encoded CCR fixture and does not require Python.
|
|
||||||
The generated snapshot contains:
|
|
||||||
- VRP: 10.0.0.0/24 maxLen 24 AS65001
|
|
||||||
- VRP: 10.0.1.0/24 maxLen 24 AS65003
|
|
||||||
- VRP: 2001:db8:1::/48 maxLen 48 AS65002
|
|
||||||
- ASPA: customer AS65010 providers AS65011, AS65012, AS65013
|
|
||||||
- ASPA: customer AS65020 provider AS65021
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
die() {
|
|
||||||
printf '[00_2_generate_ccr] ERROR: %s\n' "$*" >&2
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
if [ "${OUT_PATH}" = "-h" ] || [ "${OUT_PATH}" = "--help" ]; then
|
|
||||||
usage
|
|
||||||
exit 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
[ -n "${OUT_PATH}" ] || die "missing OUT.ccr"
|
|
||||||
mkdir -p "$(dirname "${OUT_PATH}")"
|
|
||||||
|
|
||||||
decode_base64() {
|
|
||||||
if command -v base64 >/dev/null 2>&1; then
|
|
||||||
base64 -d
|
|
||||||
return
|
|
||||||
fi
|
|
||||||
|
|
||||||
if command -v openssl >/dev/null 2>&1; then
|
|
||||||
openssl base64 -d
|
|
||||||
return
|
|
||||||
fi
|
|
||||||
|
|
||||||
die "base64 or openssl command is required to write CCR fixture"
|
|
||||||
}
|
|
||||||
|
|
||||||
cat <<'EOF' | decode_base64 >"${OUT_PATH}"
|
|
||||||
MIIBAwYLKoZIhvcNAQkQATaggfMwgfACAQAwDQYJYIZIAWUDBAIBBQAYDzIwMjYwNDAzMDAwMjAx
|
|
||||||
WqJ9MHswVzAaAgMA/ekwEzARBAIAATALMAkDBAAKAAACARgwGgIDAP3rMBMwEQQCAAEwCzAJAwQA
|
|
||||||
CgABAgEYMB0CAwD96jAWMBQEAgACMA4wDAMHACABDbgAAQIBMAQgsUgCHfy5AJcFp5RymOD0HuGO
|
|
||||||
wfhoWpll90ugYECGQPSjTDBKMCYwFgIDAP3yMA8CAwD98wIDAP30AgMA/fUwDAIDAP38MAUCAwD9
|
|
||||||
/QQgRPxVPOOUsUuuV4tZ8BG5PHNtKiaEr23Ird+yNxlWJ+w=
|
|
||||||
EOF
|
|
||||||
@@ -1,96 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
|
||||||
|
|
||||||
PRECHECK_WORK_DIR="${PRECHECK_WORK_DIR:-${ROOT_DIR}/.precheck}"
|
|
||||||
PRECHECK_PROJECT="${PRECHECK_PROJECT:-rpki-precheck}"
|
|
||||||
PRECHECK_ADMIN_TOKEN="${PRECHECK_ADMIN_TOKEN:-rpki-precheck-token}"
|
|
||||||
PRECHECK_PROTOCOL_VERSION="${PRECHECK_PROTOCOL_VERSION:-2}"
|
|
||||||
PRECHECK_SERVER_IMAGE="${PRECHECK_SERVER_IMAGE:-${RPKI_RTR_SERVER_IMAGE:-rpki-rtr-arm64:v1.2}}"
|
|
||||||
PRECHECK_CLIENT_IMAGE="${PRECHECK_CLIENT_IMAGE:-${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client-arm64:v1.2}}"
|
|
||||||
|
|
||||||
log() {
|
|
||||||
printf '[00_1_pre_check_prepare] %s\n' "$*"
|
|
||||||
}
|
|
||||||
|
|
||||||
die() {
|
|
||||||
printf '[00_1_pre_check_prepare] ERROR: %s\n' "$*" >&2
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
abs_dir() {
|
|
||||||
local path="$1"
|
|
||||||
mkdir -p "${path}"
|
|
||||||
cd "${path}" && pwd -P
|
|
||||||
}
|
|
||||||
|
|
||||||
[ -f "${SCRIPT_DIR}/00_2_generate_ccr.sh" ] || die "missing ${SCRIPT_DIR}/00_2_generate_ccr.sh"
|
|
||||||
|
|
||||||
rm -rf "${PRECHECK_WORK_DIR}"
|
|
||||||
mkdir -p \
|
|
||||||
"${PRECHECK_WORK_DIR}/ccr" \
|
|
||||||
"${PRECHECK_WORK_DIR}/slurm" \
|
|
||||||
"${PRECHECK_WORK_DIR}/db" \
|
|
||||||
"${PRECHECK_WORK_DIR}/logs/server" \
|
|
||||||
"${PRECHECK_WORK_DIR}/logs/client" \
|
|
||||||
"${PRECHECK_WORK_DIR}/report"
|
|
||||||
|
|
||||||
bash "${SCRIPT_DIR}/00_2_generate_ccr.sh" \
|
|
||||||
"${PRECHECK_WORK_DIR}/ccr/20260403T000201Z-precheck.ccr"
|
|
||||||
|
|
||||||
cat >"${PRECHECK_WORK_DIR}/slurm/precheck.slurm" <<'EOF'
|
|
||||||
{
|
|
||||||
"slurmVersion": 2,
|
|
||||||
"validationOutputFilters": {
|
|
||||||
"prefixFilters": [],
|
|
||||||
"bgpsecFilters": [],
|
|
||||||
"aspaFilters": []
|
|
||||||
},
|
|
||||||
"locallyAddedAssertions": {
|
|
||||||
"prefixAssertions": [
|
|
||||||
{
|
|
||||||
"prefix": "203.0.113.0/24",
|
|
||||||
"maxPrefixLength": 24,
|
|
||||||
"asn": 64496,
|
|
||||||
"comment": "pre-check local assertion"
|
|
||||||
}
|
|
||||||
],
|
|
||||||
"bgpsecAssertions": [],
|
|
||||||
"aspaAssertions": []
|
|
||||||
}
|
|
||||||
}
|
|
||||||
EOF
|
|
||||||
|
|
||||||
PRECHECK_WORK_DIR_ABS="$(abs_dir "${PRECHECK_WORK_DIR}")"
|
|
||||||
cat >"${PRECHECK_WORK_DIR}/precheck.env" <<EOF
|
|
||||||
PRECHECK_WORK_DIR=${PRECHECK_WORK_DIR_ABS}
|
|
||||||
PRECHECK_PROJECT=${PRECHECK_PROJECT}
|
|
||||||
RPKI_RTR_SERVER_IMAGE=${PRECHECK_SERVER_IMAGE}
|
|
||||||
RPKI_RTR_CLIENT_IMAGE=${PRECHECK_CLIENT_IMAGE}
|
|
||||||
PRECHECK_SERVER_COMPOSE_FILE=${ROOT_DIR}/deploy/server/docker-compose.tcp.yml
|
|
||||||
PRECHECK_CLIENT_COMPOSE_FILE=${ROOT_DIR}/deploy/client/docker-compose.tcp.yml
|
|
||||||
RPKI_RTR_CCR_HOST_DIR=${PRECHECK_WORK_DIR_ABS}/ccr
|
|
||||||
RPKI_RTR_SLURM_HOST_DIR=${PRECHECK_WORK_DIR_ABS}/slurm
|
|
||||||
RPKI_RTR_DB_HOST_DIR=${PRECHECK_WORK_DIR_ABS}/db
|
|
||||||
RPKI_RTR_LOG_HOST_DIR=${PRECHECK_WORK_DIR_ABS}/logs/server
|
|
||||||
RPKI_RTR_CLIENT_LOG_HOST_DIR=${PRECHECK_WORK_DIR_ABS}/logs/client
|
|
||||||
RPKI_RTR_REPORT_HOST_DIR=${PRECHECK_WORK_DIR_ABS}/report
|
|
||||||
RPKI_RTR_CCR_DIR=/app/data
|
|
||||||
RPKI_RTR_SLURM_DIR=/app/slurm
|
|
||||||
RPKI_RTR_DB_PATH=/app/rtr-db
|
|
||||||
RPKI_RTR_REPORT_DIR=/app/report
|
|
||||||
RPKI_RTR_ADMIN_ADDR=0.0.0.0:8323
|
|
||||||
RPKI_RTR_ADMIN_TOKEN=${PRECHECK_ADMIN_TOKEN}
|
|
||||||
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS=5
|
|
||||||
RPKI_RTR_REPORT_HISTORY_LIMIT=3
|
|
||||||
RPKI_RTR_TIMEZONE=Asia/Shanghai
|
|
||||||
RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS=30
|
|
||||||
RPKI_RTR_MAX_DELTA=10
|
|
||||||
RPKI_RTR_MAX_CONNECTIONS=32
|
|
||||||
RPKI_RTR_MAX_CONCURRENT_HANDSHAKES=8
|
|
||||||
RPKI_RTR_TCP_SERVER_ADDR=rpki-rtr:323
|
|
||||||
RPKI_RTR_PROTOCOL_VERSION=${PRECHECK_PROTOCOL_VERSION}
|
|
||||||
RUST_LOG=info
|
|
||||||
EOF
|
|
||||||
@@ -1,56 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
|
||||||
|
|
||||||
PRECHECK_WORK_DIR="${PRECHECK_WORK_DIR:-${ROOT_DIR}/.precheck}"
|
|
||||||
ENV_FILE="${PRECHECK_ENV_FILE:-${PRECHECK_WORK_DIR}/precheck.env}"
|
|
||||||
KEEP_PRECHECK="${KEEP_PRECHECK:-0}"
|
|
||||||
|
|
||||||
if [ -f "${ENV_FILE}" ]; then
|
|
||||||
set -a
|
|
||||||
# shellcheck disable=SC1090
|
|
||||||
. "${ENV_FILE}"
|
|
||||||
set +a
|
|
||||||
fi
|
|
||||||
|
|
||||||
PRECHECK_PROJECT="${PRECHECK_PROJECT:-rpki-precheck}"
|
|
||||||
PRECHECK_SERVER_COMPOSE_FILE="${PRECHECK_SERVER_COMPOSE_FILE:-${ROOT_DIR}/deploy/server/docker-compose.tcp.yml}"
|
|
||||||
PRECHECK_CLIENT_COMPOSE_FILE="${PRECHECK_CLIENT_COMPOSE_FILE:-${ROOT_DIR}/deploy/client/docker-compose.tcp.yml}"
|
|
||||||
COMPOSE_BIN=()
|
|
||||||
|
|
||||||
log() {
|
|
||||||
printf '[00_1_pre_check_cleanup] %s\n' "$*"
|
|
||||||
}
|
|
||||||
|
|
||||||
compose_run() {
|
|
||||||
local compose_file="$1"
|
|
||||||
shift
|
|
||||||
|
|
||||||
if [ "${#COMPOSE_BIN[@]}" -eq 0 ]; then
|
|
||||||
if docker compose version >/dev/null 2>&1; then
|
|
||||||
COMPOSE_BIN=(docker compose)
|
|
||||||
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
|
||||||
COMPOSE_BIN=(docker-compose)
|
|
||||||
else
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
|
|
||||||
"${COMPOSE_BIN[@]}" --env-file "${ENV_FILE}" -p "${PRECHECK_PROJECT}" -f "${compose_file}" "$@"
|
|
||||||
}
|
|
||||||
|
|
||||||
if command -v docker >/dev/null 2>&1; then
|
|
||||||
log "stopping pre-check client compose stack"
|
|
||||||
[ -f "${PRECHECK_CLIENT_COMPOSE_FILE}" ] && compose_run "${PRECHECK_CLIENT_COMPOSE_FILE}" down --remove-orphans >/dev/null 2>&1 || true
|
|
||||||
log "stopping pre-check server compose stack"
|
|
||||||
[ -f "${PRECHECK_SERVER_COMPOSE_FILE}" ] && compose_run "${PRECHECK_SERVER_COMPOSE_FILE}" down --remove-orphans >/dev/null 2>&1 || true
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [ "${KEEP_PRECHECK}" = "1" ]; then
|
|
||||||
log "KEEP_PRECHECK=1, preserving ${PRECHECK_WORK_DIR}"
|
|
||||||
else
|
|
||||||
rm -rf "${PRECHECK_WORK_DIR}"
|
|
||||||
log "removed ${PRECHECK_WORK_DIR}"
|
|
||||||
fi
|
|
||||||
@@ -1,221 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
|
||||||
|
|
||||||
COMMON_ENV_FILE="${ROOT_DIR}/environment.server"
|
|
||||||
SERVER_ENV_FILE="${ROOT_DIR}/deploy/server/.env"
|
|
||||||
CONSOLE_ENV_FILE="${ROOT_DIR}/deploy/console/.env"
|
|
||||||
CONSOLE_COMPOSE_FILE="${ROOT_DIR}/deploy/console/docker-compose.yml"
|
|
||||||
MODE="${RPKI_RTR_SERVER_MODE:-tcp}"
|
|
||||||
COMPOSE_PROJECT="${RPKI_RTR_COMPOSE_PROJECT:-rpki-rtr}"
|
|
||||||
CONSOLE_COMPOSE_PROJECT="${RPKI_CONSOLE_COMPOSE_PROJECT:-rpki-console}"
|
|
||||||
WAIT_SECS="${RPKI_START_WAIT_SECS:-20}"
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage: scripts/01_start.sh [options]
|
|
||||||
|
|
||||||
Read root ./environment.server, update deploy/server/.env, then start the RTR server and console.
|
|
||||||
|
|
||||||
Options:
|
|
||||||
--env FILE Common env file. Default: ./environment.server
|
|
||||||
--mode MODE Server mode: tcp|tls|ssh|base. Default: tcp
|
|
||||||
--wait SECS Wait timeout for container running checks. Default: 20
|
|
||||||
-h, --help Show this help.
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
log() {
|
|
||||||
printf '[01_start] %s\n' "$*" >&2
|
|
||||||
}
|
|
||||||
|
|
||||||
stage() {
|
|
||||||
printf '\n====== %s ======\n' "$*" >&2
|
|
||||||
}
|
|
||||||
|
|
||||||
die() {
|
|
||||||
printf '[01_start] ERROR: %s\n' "$*" >&2
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
set_env_var() {
|
|
||||||
local env_file="$1"
|
|
||||||
local key="$2"
|
|
||||||
local value="$3"
|
|
||||||
local tmp_file
|
|
||||||
|
|
||||||
tmp_file="${env_file}.tmp.$$"
|
|
||||||
awk -v key="${key}" -v value="${value}" '
|
|
||||||
BEGIN { done = 0 }
|
|
||||||
$0 ~ "^[[:space:]]*#?[[:space:]]*" key "=" {
|
|
||||||
if (!done) {
|
|
||||||
print key "=" value
|
|
||||||
done = 1
|
|
||||||
}
|
|
||||||
next
|
|
||||||
}
|
|
||||||
{ print }
|
|
||||||
END {
|
|
||||||
if (!done) {
|
|
||||||
print key "=" value
|
|
||||||
}
|
|
||||||
}
|
|
||||||
' "${env_file}" >"${tmp_file}"
|
|
||||||
mv "${tmp_file}" "${env_file}"
|
|
||||||
}
|
|
||||||
|
|
||||||
compose_file_for_mode() {
|
|
||||||
case "$1" in
|
|
||||||
tcp) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.tcp.yml" ;;
|
|
||||||
tls) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.tls.yml" ;;
|
|
||||||
ssh) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.ssh.yml" ;;
|
|
||||||
base) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.yml" ;;
|
|
||||||
*) die "unsupported mode '$1', expected tcp|tls|ssh|base" ;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
|
|
||||||
update_env_from_common() {
|
|
||||||
local target_env_file="$1"
|
|
||||||
local updated_count=0
|
|
||||||
local line key value
|
|
||||||
|
|
||||||
while IFS= read -r line || [ -n "${line}" ]; do
|
|
||||||
case "${line}" in
|
|
||||||
''|\#*) continue ;;
|
|
||||||
*=*)
|
|
||||||
key="${line%%=*}"
|
|
||||||
value="${line#*=}"
|
|
||||||
set_env_var "${target_env_file}" "${key}" "${value}"
|
|
||||||
updated_count=$((updated_count + 1))
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
done <"${COMMON_ENV_FILE}"
|
|
||||||
|
|
||||||
printf '%s\n' "${updated_count}"
|
|
||||||
}
|
|
||||||
|
|
||||||
server_compose() {
|
|
||||||
"${COMPOSE_BIN[@]}" --env-file "${SERVER_ENV_FILE}" -p "${COMPOSE_PROJECT}" -f "${SERVER_COMPOSE_FILE}" "$@"
|
|
||||||
}
|
|
||||||
|
|
||||||
console_compose() {
|
|
||||||
"${COMPOSE_BIN[@]}" --env-file "${CONSOLE_ENV_FILE}" -p "${CONSOLE_COMPOSE_PROJECT}" -f "${CONSOLE_COMPOSE_FILE}" "$@"
|
|
||||||
}
|
|
||||||
|
|
||||||
container_running() {
|
|
||||||
local compose_kind="$1"
|
|
||||||
local service="$2"
|
|
||||||
local cid state
|
|
||||||
|
|
||||||
if [ "${compose_kind}" = "server" ]; then
|
|
||||||
cid="$(server_compose ps -q "${service}" 2>/dev/null || true)"
|
|
||||||
else
|
|
||||||
cid="$(console_compose ps -q "${service}" 2>/dev/null || true)"
|
|
||||||
fi
|
|
||||||
|
|
||||||
[ -n "${cid}" ] || return 1
|
|
||||||
state="$(docker inspect -f '{{.State.Running}}' "${cid}" 2>/dev/null || true)"
|
|
||||||
[ "${state}" = "true" ]
|
|
||||||
}
|
|
||||||
|
|
||||||
show_service_debug() {
|
|
||||||
local compose_kind="$1"
|
|
||||||
local service="$2"
|
|
||||||
|
|
||||||
log "${service} is not running, current compose status:"
|
|
||||||
if [ "${compose_kind}" = "server" ]; then
|
|
||||||
server_compose ps "${service}" >&2 || true
|
|
||||||
log "${service} recent logs:"
|
|
||||||
server_compose logs --tail=80 "${service}" >&2 || true
|
|
||||||
else
|
|
||||||
console_compose ps "${service}" >&2 || true
|
|
||||||
log "${service} recent logs:"
|
|
||||||
console_compose logs --tail=80 "${service}" >&2 || true
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
wait_for_running() {
|
|
||||||
local compose_kind="$1"
|
|
||||||
local service="$2"
|
|
||||||
local deadline=$((SECONDS + WAIT_SECS))
|
|
||||||
|
|
||||||
while [ "${SECONDS}" -lt "${deadline}" ]; do
|
|
||||||
if container_running "${compose_kind}" "${service}"; then
|
|
||||||
log "${service} is running"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
sleep 1
|
|
||||||
done
|
|
||||||
|
|
||||||
show_service_debug "${compose_kind}" "${service}"
|
|
||||||
die "${service} failed to reach running state after ${WAIT_SECS}s"
|
|
||||||
}
|
|
||||||
|
|
||||||
while [ "$#" -gt 0 ]; do
|
|
||||||
case "$1" in
|
|
||||||
--env)
|
|
||||||
[ "$#" -ge 2 ] || die "--env requires a value"
|
|
||||||
COMMON_ENV_FILE="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
--mode)
|
|
||||||
[ "$#" -ge 2 ] || die "--mode requires a value"
|
|
||||||
MODE="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
--wait)
|
|
||||||
[ "$#" -ge 2 ] || die "--wait requires a value"
|
|
||||||
WAIT_SECS="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
-h|--help)
|
|
||||||
usage
|
|
||||||
exit 0
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
die "unknown option: $1"
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
SERVER_COMPOSE_FILE="$(compose_file_for_mode "${MODE}")"
|
|
||||||
|
|
||||||
stage "Checking Docker"
|
|
||||||
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
|
||||||
if docker compose version >/dev/null 2>&1; then
|
|
||||||
COMPOSE_BIN=(docker compose)
|
|
||||||
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
|
||||||
COMPOSE_BIN=(docker-compose)
|
|
||||||
else
|
|
||||||
die "docker compose is not available"
|
|
||||||
fi
|
|
||||||
|
|
||||||
[ -f "${SERVER_ENV_FILE}" ] || die "server env file not found: ${SERVER_ENV_FILE}"
|
|
||||||
[ -f "${SERVER_COMPOSE_FILE}" ] || die "server compose file not found: ${SERVER_COMPOSE_FILE}"
|
|
||||||
[ -f "${CONSOLE_ENV_FILE}" ] || die "console env file not found: ${CONSOLE_ENV_FILE}"
|
|
||||||
[ -f "${CONSOLE_COMPOSE_FILE}" ] || die "console compose file not found: ${CONSOLE_COMPOSE_FILE}"
|
|
||||||
|
|
||||||
if [ -f "${COMMON_ENV_FILE}" ]; then
|
|
||||||
stage "Reading Environment"
|
|
||||||
server_updated_count="$(update_env_from_common "${SERVER_ENV_FILE}")"
|
|
||||||
log "updated ${server_updated_count} server env variable(s)"
|
|
||||||
else
|
|
||||||
stage "Reading Environment"
|
|
||||||
log "environment.server file not found, skip env update"
|
|
||||||
fi
|
|
||||||
|
|
||||||
stage "Starting RTR Server"
|
|
||||||
log "starting docker compose service: rpki-rtr"
|
|
||||||
server_compose up -d --no-build rpki-rtr
|
|
||||||
log "waiting for rpki-rtr to be running"
|
|
||||||
wait_for_running server rpki-rtr
|
|
||||||
|
|
||||||
stage "Starting Console"
|
|
||||||
log "starting docker compose service: rpki-console-ui"
|
|
||||||
console_compose up -d --no-build rpki-console-ui
|
|
||||||
log "waiting for rpki-console-ui to be running"
|
|
||||||
wait_for_running console rpki-console-ui
|
|
||||||
|
|
||||||
stage "Start Finished"
|
|
||||||
@@ -1,184 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
|
||||||
|
|
||||||
SERVER_ENV_FILE="${ROOT_DIR}/deploy/server/.env"
|
|
||||||
CONSOLE_ENV_FILE="${ROOT_DIR}/deploy/console/.env"
|
|
||||||
SERVER_COMPOSE_FILE="${ROOT_DIR}/deploy/server/docker-compose.tcp.yml"
|
|
||||||
CONSOLE_COMPOSE_FILE="${ROOT_DIR}/deploy/console/docker-compose.yml"
|
|
||||||
SERVER_PROJECT="${RPKI_RTR_COMPOSE_PROJECT:-rpki-rtr}"
|
|
||||||
CONSOLE_PROJECT="${RPKI_CONSOLE_COMPOSE_PROJECT:-rpki-console}"
|
|
||||||
WAIT_SECS="${RPKI_SELF_CHECK_WAIT_SECS:-60}"
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage: scripts/02_self_check.sh [options]
|
|
||||||
|
|
||||||
Check whether the RTR server and console started by scripts/01_start.sh are healthy.
|
|
||||||
|
|
||||||
Options:
|
|
||||||
--wait SECS Wait timeout for each check. Default: 60
|
|
||||||
-h, --help Show this help.
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
log() {
|
|
||||||
printf '[02_self_check] %s\n' "$*" >&2
|
|
||||||
}
|
|
||||||
|
|
||||||
stage() {
|
|
||||||
printf '\n====== %s ======\n' "$*" >&2
|
|
||||||
}
|
|
||||||
|
|
||||||
die() {
|
|
||||||
printf '[02_self_check] ERROR: %s\n' "$*" >&2
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
while [ "$#" -gt 0 ]; do
|
|
||||||
case "$1" in
|
|
||||||
--wait)
|
|
||||||
[ "$#" -ge 2 ] || die "--wait requires a value"
|
|
||||||
WAIT_SECS="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
-h|--help)
|
|
||||||
usage
|
|
||||||
exit 0
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
die "unknown option: $1"
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
|
||||||
if docker compose version >/dev/null 2>&1; then
|
|
||||||
COMPOSE_BIN=(docker compose)
|
|
||||||
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
|
||||||
COMPOSE_BIN=(docker-compose)
|
|
||||||
else
|
|
||||||
die "docker compose is not available"
|
|
||||||
fi
|
|
||||||
|
|
||||||
[ -f "${SERVER_ENV_FILE}" ] || die "server env file not found: ${SERVER_ENV_FILE}"
|
|
||||||
[ -f "${SERVER_COMPOSE_FILE}" ] || die "server compose file not found: ${SERVER_COMPOSE_FILE}"
|
|
||||||
[ -f "${CONSOLE_ENV_FILE}" ] || die "console env file not found: ${CONSOLE_ENV_FILE}"
|
|
||||||
[ -f "${CONSOLE_COMPOSE_FILE}" ] || die "console compose file not found: ${CONSOLE_COMPOSE_FILE}"
|
|
||||||
|
|
||||||
set -a
|
|
||||||
# shellcheck disable=SC1090
|
|
||||||
. "${SERVER_ENV_FILE}"
|
|
||||||
# shellcheck disable=SC1090
|
|
||||||
. "${CONSOLE_ENV_FILE}"
|
|
||||||
set +a
|
|
||||||
|
|
||||||
RTR_PORT="${RPKI_RTR_TCP_HOST_PORT:-323}"
|
|
||||||
ADMIN_PORT="${RPKI_RTR_ADMIN_ADDR##*:}"
|
|
||||||
ADMIN_PORT="${ADMIN_PORT:-8323}"
|
|
||||||
ADMIN_TOKEN="${RPKI_RTR_ADMIN_TOKEN:-}"
|
|
||||||
CONSOLE_PORT="${RPKI_CONSOLE_UI_PORT:-5173}"
|
|
||||||
|
|
||||||
server_compose() {
|
|
||||||
"${COMPOSE_BIN[@]}" --env-file "${SERVER_ENV_FILE}" -p "${SERVER_PROJECT}" -f "${SERVER_COMPOSE_FILE}" "$@"
|
|
||||||
}
|
|
||||||
|
|
||||||
console_compose() {
|
|
||||||
"${COMPOSE_BIN[@]}" --env-file "${CONSOLE_ENV_FILE}" -p "${CONSOLE_PROJECT}" -f "${CONSOLE_COMPOSE_FILE}" "$@"
|
|
||||||
}
|
|
||||||
|
|
||||||
container_running() {
|
|
||||||
local compose_kind="$1"
|
|
||||||
local service="$2"
|
|
||||||
local cid state
|
|
||||||
|
|
||||||
if [ "${compose_kind}" = "server" ]; then
|
|
||||||
cid="$(server_compose ps -q "${service}" 2>/dev/null || true)"
|
|
||||||
else
|
|
||||||
cid="$(console_compose ps -q "${service}" 2>/dev/null || true)"
|
|
||||||
fi
|
|
||||||
|
|
||||||
[ -n "${cid}" ] || return 1
|
|
||||||
state="$(docker inspect -f '{{.State.Running}}' "${cid}" 2>/dev/null || true)"
|
|
||||||
[ "${state}" = "true" ]
|
|
||||||
}
|
|
||||||
|
|
||||||
tcp_open() {
|
|
||||||
local host="$1"
|
|
||||||
local port="$2"
|
|
||||||
|
|
||||||
if command -v nc >/dev/null 2>&1; then
|
|
||||||
nc -z "${host}" "${port}" >/dev/null 2>&1
|
|
||||||
return
|
|
||||||
fi
|
|
||||||
|
|
||||||
(exec 3<>"/dev/tcp/${host}/${port}") >/dev/null 2>&1
|
|
||||||
}
|
|
||||||
|
|
||||||
http_get() {
|
|
||||||
local url="$1"
|
|
||||||
local token="${2:-}"
|
|
||||||
|
|
||||||
if command -v curl >/dev/null 2>&1; then
|
|
||||||
if [ -n "${token}" ]; then
|
|
||||||
curl -fsS -H "Authorization: Bearer ${token}" "${url}"
|
|
||||||
else
|
|
||||||
curl -fsS "${url}"
|
|
||||||
fi
|
|
||||||
return
|
|
||||||
fi
|
|
||||||
|
|
||||||
die "curl command is required for HTTP health checks"
|
|
||||||
}
|
|
||||||
|
|
||||||
wait_until() {
|
|
||||||
local label="$1"
|
|
||||||
shift
|
|
||||||
local deadline=$((SECONDS + WAIT_SECS))
|
|
||||||
|
|
||||||
while [ "${SECONDS}" -lt "${deadline}" ]; do
|
|
||||||
if "$@"; then
|
|
||||||
log "${label}: ok"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
sleep 1
|
|
||||||
done
|
|
||||||
|
|
||||||
die "${label}: failed after ${WAIT_SECS}s"
|
|
||||||
}
|
|
||||||
|
|
||||||
show_command() {
|
|
||||||
printf '[02_self_check] check command: %s\n' "$*" >&2
|
|
||||||
}
|
|
||||||
|
|
||||||
check_admin_health() {
|
|
||||||
local body
|
|
||||||
body="$(http_get "http://127.0.0.1:${ADMIN_PORT}/admin/rtr/health" "${ADMIN_TOKEN}" 2>/dev/null || true)"
|
|
||||||
printf '%s' "${body}" | grep -q '"status"[[:space:]]*:[[:space:]]*"ok"'
|
|
||||||
}
|
|
||||||
|
|
||||||
check_console_http() {
|
|
||||||
http_get "http://127.0.0.1:${CONSOLE_PORT}/" >/dev/null 2>&1
|
|
||||||
}
|
|
||||||
|
|
||||||
stage "Checking Containers"
|
|
||||||
show_command "${COMPOSE_BIN[*]} --env-file ${SERVER_ENV_FILE} -p ${SERVER_PROJECT} -f ${SERVER_COMPOSE_FILE} ps -q rpki-rtr && docker inspect"
|
|
||||||
wait_until "server container" container_running server rpki-rtr
|
|
||||||
show_command "${COMPOSE_BIN[*]} --env-file ${CONSOLE_ENV_FILE} -p ${CONSOLE_PROJECT} -f ${CONSOLE_COMPOSE_FILE} ps -q rpki-console-ui && docker inspect"
|
|
||||||
wait_until "console container" container_running console rpki-console-ui
|
|
||||||
|
|
||||||
stage "Checking Server"
|
|
||||||
show_command "tcp connect 127.0.0.1:${RTR_PORT}"
|
|
||||||
wait_until "RTR port ${RTR_PORT}" tcp_open 127.0.0.1 "${RTR_PORT}"
|
|
||||||
show_command "curl -fsS -H 'Authorization: Bearer <token>' http://127.0.0.1:${ADMIN_PORT}/admin/rtr/health"
|
|
||||||
wait_until "Admin health" check_admin_health
|
|
||||||
|
|
||||||
stage "Checking Console"
|
|
||||||
show_command "tcp connect 127.0.0.1:${CONSOLE_PORT}"
|
|
||||||
wait_until "Console port ${CONSOLE_PORT}" tcp_open 127.0.0.1 "${CONSOLE_PORT}"
|
|
||||||
show_command "curl -fsS http://127.0.0.1:${CONSOLE_PORT}/"
|
|
||||||
wait_until "Console HTTP" check_console_http
|
|
||||||
|
|
||||||
stage "Self-Check Passed"
|
|
||||||
@@ -1,92 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
|
||||||
|
|
||||||
SERVER_ENV_FILE="${ROOT_DIR}/deploy/server/.env"
|
|
||||||
CONSOLE_ENV_FILE="${ROOT_DIR}/deploy/console/.env"
|
|
||||||
MODE="${RPKI_RTR_SERVER_MODE:-tcp}"
|
|
||||||
SERVER_PROJECT="${RPKI_RTR_COMPOSE_PROJECT:-rpki-rtr}"
|
|
||||||
CONSOLE_PROJECT="${RPKI_CONSOLE_COMPOSE_PROJECT:-rpki-console}"
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage: scripts/03_shutdown.sh [options]
|
|
||||||
|
|
||||||
Stop services started by scripts/01_start.sh.
|
|
||||||
|
|
||||||
Options:
|
|
||||||
--mode MODE Server mode: tcp|tls|ssh|base. Default: tcp
|
|
||||||
-h, --help Show this help.
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
log() {
|
|
||||||
printf '[03_shutdown] %s\n' "$*" >&2
|
|
||||||
}
|
|
||||||
|
|
||||||
stage() {
|
|
||||||
printf '\n====== %s ======\n' "$*" >&2
|
|
||||||
}
|
|
||||||
|
|
||||||
die() {
|
|
||||||
printf '[03_shutdown] ERROR: %s\n' "$*" >&2
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
compose_file_for_mode() {
|
|
||||||
case "$1" in
|
|
||||||
tcp) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.tcp.yml" ;;
|
|
||||||
tls) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.tls.yml" ;;
|
|
||||||
ssh) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.ssh.yml" ;;
|
|
||||||
base) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.yml" ;;
|
|
||||||
*) die "unsupported mode '$1', expected tcp|tls|ssh|base" ;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
|
|
||||||
while [ "$#" -gt 0 ]; do
|
|
||||||
case "$1" in
|
|
||||||
--mode)
|
|
||||||
[ "$#" -ge 2 ] || die "--mode requires a value"
|
|
||||||
MODE="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
-h|--help)
|
|
||||||
usage
|
|
||||||
exit 0
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
die "unknown option: $1"
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
SERVER_COMPOSE_FILE="$(compose_file_for_mode "${MODE}")"
|
|
||||||
CONSOLE_COMPOSE_FILE="${ROOT_DIR}/deploy/console/docker-compose.yml"
|
|
||||||
|
|
||||||
stage "Checking Docker"
|
|
||||||
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
|
||||||
if docker compose version >/dev/null 2>&1; then
|
|
||||||
COMPOSE_BIN=(docker compose)
|
|
||||||
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
|
||||||
COMPOSE_BIN=(docker-compose)
|
|
||||||
else
|
|
||||||
die "docker compose is not available"
|
|
||||||
fi
|
|
||||||
|
|
||||||
stage "Stopping Console"
|
|
||||||
if [ -f "${CONSOLE_ENV_FILE}" ] && [ -f "${CONSOLE_COMPOSE_FILE}" ]; then
|
|
||||||
"${COMPOSE_BIN[@]}" --env-file "${CONSOLE_ENV_FILE}" -p "${CONSOLE_PROJECT}" -f "${CONSOLE_COMPOSE_FILE}" down --remove-orphans
|
|
||||||
else
|
|
||||||
log "console compose/env not found, skip"
|
|
||||||
fi
|
|
||||||
|
|
||||||
stage "Stopping RTR Server"
|
|
||||||
if [ -f "${SERVER_ENV_FILE}" ] && [ -f "${SERVER_COMPOSE_FILE}" ]; then
|
|
||||||
"${COMPOSE_BIN[@]}" --env-file "${SERVER_ENV_FILE}" -p "${SERVER_PROJECT}" -f "${SERVER_COMPOSE_FILE}" down --remove-orphans
|
|
||||||
else
|
|
||||||
log "server compose/env not found, skip"
|
|
||||||
fi
|
|
||||||
|
|
||||||
stage "Shutdown Finished"
|
|
||||||
@@ -1,152 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
|
||||||
SERVER_DIR="${ROOT_DIR}/deploy/server"
|
|
||||||
SERVER_ENV_FILE="${SERVER_DIR}/.env"
|
|
||||||
SERVER_COMPOSE_FILE="${SERVER_DIR}/docker-compose.tcp.yml"
|
|
||||||
COMPOSE_PROJECT="${RPKI_RTR_COMPOSE_PROJECT:-rpki-rtr}"
|
|
||||||
FORCE=0
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage: scripts/04_cleanup.sh [options]
|
|
||||||
|
|
||||||
Remove server generated directories: logs, reports, and RTR database.
|
|
||||||
|
|
||||||
Options:
|
|
||||||
--force Clean even if the server container appears to be running.
|
|
||||||
-h, --help Show this help.
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
log() {
|
|
||||||
printf '[04_cleanup] %s\n' "$*" >&2
|
|
||||||
}
|
|
||||||
|
|
||||||
stage() {
|
|
||||||
printf '\n====== %s ======\n' "$*" >&2
|
|
||||||
}
|
|
||||||
|
|
||||||
die() {
|
|
||||||
printf '[04_cleanup] ERROR: %s\n' "$*" >&2
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
env_value() {
|
|
||||||
local key="$1"
|
|
||||||
local default_value="$2"
|
|
||||||
local line
|
|
||||||
|
|
||||||
line="$(grep -E "^[[:space:]]*${key}=" "${SERVER_ENV_FILE}" | tail -n 1 || true)"
|
|
||||||
if [ -n "${line}" ]; then
|
|
||||||
printf '%s\n' "${line#*=}"
|
|
||||||
else
|
|
||||||
printf '%s\n' "${default_value}"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
resolve_host_path() {
|
|
||||||
local raw_path="$1"
|
|
||||||
|
|
||||||
case "${raw_path}" in
|
|
||||||
/*|[A-Za-z]:/*|[A-Za-z]:\\*)
|
|
||||||
printf '%s\n' "${raw_path}"
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
(cd "${SERVER_DIR}" && cd "${raw_path}" 2>/dev/null && pwd -P) || printf '%s/%s\n' "${SERVER_DIR}" "${raw_path}"
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
|
|
||||||
compose_available() {
|
|
||||||
if docker compose version >/dev/null 2>&1; then
|
|
||||||
COMPOSE_BIN=(docker compose)
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
|
||||||
COMPOSE_BIN=(docker-compose)
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
|
|
||||||
server_running() {
|
|
||||||
local cid state
|
|
||||||
|
|
||||||
[ -f "${SERVER_COMPOSE_FILE}" ] || return 1
|
|
||||||
compose_available || return 1
|
|
||||||
|
|
||||||
cid="$("${COMPOSE_BIN[@]}" --env-file "${SERVER_ENV_FILE}" -p "${COMPOSE_PROJECT}" -f "${SERVER_COMPOSE_FILE}" ps -q rpki-rtr 2>/dev/null || true)"
|
|
||||||
[ -n "${cid}" ] || return 1
|
|
||||||
|
|
||||||
state="$(docker inspect -f '{{.State.Running}}' "${cid}" 2>/dev/null || true)"
|
|
||||||
[ "${state}" = "true" ]
|
|
||||||
}
|
|
||||||
|
|
||||||
clean_dir() {
|
|
||||||
local label="$1"
|
|
||||||
local path="$2"
|
|
||||||
local root_with_slash
|
|
||||||
|
|
||||||
if [ ! -d "${path}" ]; then
|
|
||||||
log "${label}: ${path} not found, skip"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
root_with_slash="${ROOT_DIR}/"
|
|
||||||
case "${path}/" in
|
|
||||||
"${root_with_slash}"*) ;;
|
|
||||||
*)
|
|
||||||
log "ERROR: ${label} path is outside repo root, refuse to remove: ${path}"
|
|
||||||
return 1
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
|
|
||||||
log "${label}: removing ${path}"
|
|
||||||
if ! rm -rf "${path}"; then
|
|
||||||
log "ERROR: failed to remove ${label}: ${path}"
|
|
||||||
return 1
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
while [ "$#" -gt 0 ]; do
|
|
||||||
case "$1" in
|
|
||||||
--force)
|
|
||||||
FORCE=1
|
|
||||||
shift
|
|
||||||
;;
|
|
||||||
-h|--help)
|
|
||||||
usage
|
|
||||||
exit 0
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
die "unknown option: $1"
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
stage "Checking Server"
|
|
||||||
[ -f "${SERVER_ENV_FILE}" ] || die "server env file not found: ${SERVER_ENV_FILE}"
|
|
||||||
if [ "${FORCE}" -ne 1 ] && command -v docker >/dev/null 2>&1 && server_running; then
|
|
||||||
die "rpki-rtr is still running, run scripts/03_shutdown.sh first or use --force"
|
|
||||||
fi
|
|
||||||
|
|
||||||
LOG_DIR="${ROOT_DIR}/logs"
|
|
||||||
REPORT_DIR="$(resolve_host_path "$(env_value RPKI_RTR_REPORT_HOST_DIR '../../report')")"
|
|
||||||
DB_DIR="$(resolve_host_path "$(env_value RPKI_RTR_DB_HOST_DIR '../../rtr-db')")"
|
|
||||||
|
|
||||||
stage "Removing Server Files"
|
|
||||||
failed_count=0
|
|
||||||
clean_dir "logs" "${LOG_DIR}" || failed_count=$((failed_count + 1))
|
|
||||||
clean_dir "report" "${REPORT_DIR}" || failed_count=$((failed_count + 1))
|
|
||||||
clean_dir "rtr-db" "${DB_DIR}" || failed_count=$((failed_count + 1))
|
|
||||||
|
|
||||||
if [ "${failed_count}" -gt 0 ]; then
|
|
||||||
die "${failed_count} cleanup item(s) failed"
|
|
||||||
fi
|
|
||||||
|
|
||||||
stage "Cleanup Finished"
|
|
||||||
@@ -1,303 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
|
||||||
OUT_DIR="${ROOT_DIR}/certs"
|
|
||||||
TLS_DIR="${OUT_DIR}/tls"
|
|
||||||
SSH_DIR="${OUT_DIR}/ssh"
|
|
||||||
SERVER_DNS="localhost"
|
|
||||||
FORCE=0
|
|
||||||
DO_TLS=1
|
|
||||||
DO_SSH=1
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'USAGE'
|
|
||||||
Generate local development TLS certificates and SSH keys for RTR Docker deploys.
|
|
||||||
|
|
||||||
Usage:
|
|
||||||
scripts/10_generate-certs.sh [options]
|
|
||||||
|
|
||||||
Options:
|
|
||||||
--out-dir DIR Output directory, default: ./certs
|
|
||||||
--server-dns NAME TLS server dNSName SAN, default: localhost
|
|
||||||
--tls-only Generate only TLS files
|
|
||||||
--ssh-only Generate only SSH files
|
|
||||||
--force Overwrite existing files
|
|
||||||
-h, --help Show this help
|
|
||||||
|
|
||||||
Generated TLS files:
|
|
||||||
certs/tls/client-ca.crt
|
|
||||||
certs/tls/client-ca.key
|
|
||||||
certs/tls/server-dns.crt
|
|
||||||
certs/tls/server-dns.key
|
|
||||||
certs/tls/client-good.crt
|
|
||||||
certs/tls/client-good.key
|
|
||||||
|
|
||||||
Generated SSH files:
|
|
||||||
certs/ssh/ssh_host_rsa_key
|
|
||||||
certs/ssh/ssh_host_rsa_key.pub
|
|
||||||
certs/ssh/rtr-client.key
|
|
||||||
certs/ssh/rtr-client.key.pub
|
|
||||||
certs/ssh/rtr-authorized_keys
|
|
||||||
USAGE
|
|
||||||
}
|
|
||||||
|
|
||||||
die() {
|
|
||||||
echo "error: $*" >&2
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
info() {
|
|
||||||
echo "[generate-certs] $*"
|
|
||||||
}
|
|
||||||
|
|
||||||
need_cmd() {
|
|
||||||
command -v "$1" >/dev/null 2>&1 || die "required command not found: $1"
|
|
||||||
}
|
|
||||||
|
|
||||||
run_quiet() {
|
|
||||||
local log_file
|
|
||||||
log_file="$(mktemp)"
|
|
||||||
if "$@" >"${log_file}" 2>&1; then
|
|
||||||
rm -f "${log_file}"
|
|
||||||
return 0
|
|
||||||
fi
|
|
||||||
|
|
||||||
cat "${log_file}" >&2
|
|
||||||
rm -f "${log_file}"
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
|
|
||||||
ensure_abs_out_dir() {
|
|
||||||
case "${OUT_DIR}" in
|
|
||||||
/*) ;;
|
|
||||||
[A-Za-z]:*) ;;
|
|
||||||
*) OUT_DIR="${ROOT_DIR}/${OUT_DIR}" ;;
|
|
||||||
esac
|
|
||||||
TLS_DIR="${OUT_DIR}/tls"
|
|
||||||
SSH_DIR="${OUT_DIR}/ssh"
|
|
||||||
}
|
|
||||||
|
|
||||||
check_overwrite() {
|
|
||||||
local path="$1"
|
|
||||||
if [[ -e "${path}" && "${FORCE}" != "1" ]]; then
|
|
||||||
die "${path} already exists; rerun with --force to overwrite"
|
|
||||||
fi
|
|
||||||
}
|
|
||||||
|
|
||||||
write_tls_config_files() {
|
|
||||||
local tmp_dir="$1"
|
|
||||||
|
|
||||||
cat >"${tmp_dir}/ca.cnf" <<'EOF'
|
|
||||||
[ req ]
|
|
||||||
distinguished_name = req_distinguished_name
|
|
||||||
prompt = no
|
|
||||||
|
|
||||||
[ req_distinguished_name ]
|
|
||||||
CN = rpki-rtr-client-ca
|
|
||||||
EOF
|
|
||||||
|
|
||||||
cat >"${tmp_dir}/server.cnf" <<EOF
|
|
||||||
[ req ]
|
|
||||||
distinguished_name = req_distinguished_name
|
|
||||||
prompt = no
|
|
||||||
|
|
||||||
[ req_distinguished_name ]
|
|
||||||
CN = ${SERVER_DNS}
|
|
||||||
EOF
|
|
||||||
|
|
||||||
cat >"${tmp_dir}/client.cnf" <<'EOF'
|
|
||||||
[ req ]
|
|
||||||
distinguished_name = req_distinguished_name
|
|
||||||
prompt = no
|
|
||||||
|
|
||||||
[ req_distinguished_name ]
|
|
||||||
CN = rpki-rtr-client
|
|
||||||
EOF
|
|
||||||
|
|
||||||
cat >"${tmp_dir}/server.ext" <<EOF
|
|
||||||
basicConstraints=CA:FALSE
|
|
||||||
keyUsage=digitalSignature,keyEncipherment
|
|
||||||
extendedKeyUsage=serverAuth
|
|
||||||
subjectAltName=DNS:${SERVER_DNS}
|
|
||||||
EOF
|
|
||||||
|
|
||||||
cat >"${tmp_dir}/client.ext" <<'EOF'
|
|
||||||
basicConstraints=CA:FALSE
|
|
||||||
keyUsage=digitalSignature,keyEncipherment
|
|
||||||
extendedKeyUsage=clientAuth
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
generate_tls() {
|
|
||||||
need_cmd openssl
|
|
||||||
mkdir -p "${TLS_DIR}"
|
|
||||||
|
|
||||||
info "generating TLS certificates in ${TLS_DIR}"
|
|
||||||
info "TLS server dNSName SAN: ${SERVER_DNS}"
|
|
||||||
|
|
||||||
local files=(
|
|
||||||
"${TLS_DIR}/client-ca.crt"
|
|
||||||
"${TLS_DIR}/client-ca.key"
|
|
||||||
"${TLS_DIR}/server-dns.crt"
|
|
||||||
"${TLS_DIR}/server-dns.key"
|
|
||||||
"${TLS_DIR}/client-good.crt"
|
|
||||||
"${TLS_DIR}/client-good.key"
|
|
||||||
)
|
|
||||||
for file in "${files[@]}"; do
|
|
||||||
check_overwrite "${file}"
|
|
||||||
done
|
|
||||||
|
|
||||||
local tmp_dir
|
|
||||||
tmp_dir="$(mktemp -d)"
|
|
||||||
write_tls_config_files "${tmp_dir}"
|
|
||||||
|
|
||||||
run_quiet openssl genrsa -out "${TLS_DIR}/client-ca.key" 2048
|
|
||||||
info "created TLS CA key: ${TLS_DIR}/client-ca.key"
|
|
||||||
run_quiet openssl req -x509 -new -nodes \
|
|
||||||
-config "${tmp_dir}/ca.cnf" \
|
|
||||||
-key "${TLS_DIR}/client-ca.key" \
|
|
||||||
-sha256 -days 3650 \
|
|
||||||
-out "${TLS_DIR}/client-ca.crt"
|
|
||||||
info "created TLS CA cert: ${TLS_DIR}/client-ca.crt"
|
|
||||||
|
|
||||||
run_quiet openssl genrsa -out "${TLS_DIR}/server-dns.key" 2048
|
|
||||||
info "created TLS server key: ${TLS_DIR}/server-dns.key"
|
|
||||||
run_quiet openssl req -new \
|
|
||||||
-config "${tmp_dir}/server.cnf" \
|
|
||||||
-key "${TLS_DIR}/server-dns.key" \
|
|
||||||
-out "${tmp_dir}/server-dns.csr"
|
|
||||||
run_quiet openssl x509 -req \
|
|
||||||
-in "${tmp_dir}/server-dns.csr" \
|
|
||||||
-CA "${TLS_DIR}/client-ca.crt" \
|
|
||||||
-CAkey "${TLS_DIR}/client-ca.key" \
|
|
||||||
-CAcreateserial \
|
|
||||||
-out "${TLS_DIR}/server-dns.crt" \
|
|
||||||
-days 3650 -sha256 \
|
|
||||||
-extfile "${tmp_dir}/server.ext"
|
|
||||||
info "created TLS server cert: ${TLS_DIR}/server-dns.crt"
|
|
||||||
|
|
||||||
run_quiet openssl genrsa -out "${TLS_DIR}/client-good.key" 2048
|
|
||||||
info "created TLS client key: ${TLS_DIR}/client-good.key"
|
|
||||||
run_quiet openssl req -new \
|
|
||||||
-config "${tmp_dir}/client.cnf" \
|
|
||||||
-key "${TLS_DIR}/client-good.key" \
|
|
||||||
-out "${tmp_dir}/client-good.csr"
|
|
||||||
run_quiet openssl x509 -req \
|
|
||||||
-in "${tmp_dir}/client-good.csr" \
|
|
||||||
-CA "${TLS_DIR}/client-ca.crt" \
|
|
||||||
-CAkey "${TLS_DIR}/client-ca.key" \
|
|
||||||
-CAcreateserial \
|
|
||||||
-out "${TLS_DIR}/client-good.crt" \
|
|
||||||
-days 3650 -sha256 \
|
|
||||||
-extfile "${tmp_dir}/client.ext"
|
|
||||||
info "created TLS client cert: ${TLS_DIR}/client-good.crt"
|
|
||||||
|
|
||||||
rm -f "${TLS_DIR}/client-ca.srl"
|
|
||||||
rm -rf "${tmp_dir}"
|
|
||||||
chmod 600 "${TLS_DIR}"/*.key
|
|
||||||
chmod 644 "${TLS_DIR}"/*.crt
|
|
||||||
}
|
|
||||||
|
|
||||||
generate_ssh_key() {
|
|
||||||
local path="$1"
|
|
||||||
local comment="$2"
|
|
||||||
check_overwrite "${path}"
|
|
||||||
check_overwrite "${path}.pub"
|
|
||||||
rm -f "${path}" "${path}.pub"
|
|
||||||
run_quiet ssh-keygen -q -t rsa -b 3072 -m PEM -N "" -C "${comment}" -f "${path}"
|
|
||||||
}
|
|
||||||
|
|
||||||
generate_ssh() {
|
|
||||||
need_cmd ssh-keygen
|
|
||||||
mkdir -p "${SSH_DIR}"
|
|
||||||
|
|
||||||
info "generating SSH keys in ${SSH_DIR}"
|
|
||||||
generate_ssh_key "${SSH_DIR}/ssh_host_rsa_key" "rpki-rtr-host"
|
|
||||||
info "created SSH host key: ${SSH_DIR}/ssh_host_rsa_key"
|
|
||||||
generate_ssh_key "${SSH_DIR}/rtr-client.key" "rpki-rtr-debug-client"
|
|
||||||
info "created SSH client key: ${SSH_DIR}/rtr-client.key"
|
|
||||||
|
|
||||||
check_overwrite "${SSH_DIR}/rtr-authorized_keys"
|
|
||||||
cp "${SSH_DIR}/rtr-client.key.pub" "${SSH_DIR}/rtr-authorized_keys"
|
|
||||||
info "created SSH authorized_keys: ${SSH_DIR}/rtr-authorized_keys"
|
|
||||||
|
|
||||||
chmod 600 \
|
|
||||||
"${SSH_DIR}/ssh_host_rsa_key" \
|
|
||||||
"${SSH_DIR}/rtr-client.key"
|
|
||||||
chmod 644 "${SSH_DIR}"/*.pub "${SSH_DIR}/rtr-authorized_keys"
|
|
||||||
}
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--out-dir)
|
|
||||||
[[ $# -ge 2 ]] || die "--out-dir requires a value"
|
|
||||||
OUT_DIR="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
--server-dns)
|
|
||||||
[[ $# -ge 2 ]] || die "--server-dns requires a value"
|
|
||||||
SERVER_DNS="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
--tls-only)
|
|
||||||
DO_TLS=1
|
|
||||||
DO_SSH=0
|
|
||||||
shift
|
|
||||||
;;
|
|
||||||
--ssh-only)
|
|
||||||
DO_TLS=0
|
|
||||||
DO_SSH=1
|
|
||||||
shift
|
|
||||||
;;
|
|
||||||
--force)
|
|
||||||
FORCE=1
|
|
||||||
shift
|
|
||||||
;;
|
|
||||||
-h|--help)
|
|
||||||
usage
|
|
||||||
exit 0
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
die "unknown option: $1"
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
ensure_abs_out_dir
|
|
||||||
|
|
||||||
if [[ "${DO_TLS}" == "1" ]]; then
|
|
||||||
generate_tls
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [[ "${DO_SSH}" == "1" ]]; then
|
|
||||||
generate_ssh
|
|
||||||
fi
|
|
||||||
|
|
||||||
info "done"
|
|
||||||
info "generated files under ${OUT_DIR}"
|
|
||||||
|
|
||||||
if [[ "${DO_TLS}" == "1" ]]; then
|
|
||||||
cat <<EOF
|
|
||||||
|
|
||||||
TLS files:
|
|
||||||
${TLS_DIR}/client-ca.crt
|
|
||||||
${TLS_DIR}/server-dns.crt
|
|
||||||
${TLS_DIR}/server-dns.key
|
|
||||||
${TLS_DIR}/client-good.crt
|
|
||||||
${TLS_DIR}/client-good.key
|
|
||||||
${TLS_DIR}/client-ca.key (local signing key; not mounted by compose)
|
|
||||||
EOF
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [[ "${DO_SSH}" == "1" ]]; then
|
|
||||||
cat <<EOF
|
|
||||||
|
|
||||||
SSH files:
|
|
||||||
${SSH_DIR}/ssh_host_rsa_key
|
|
||||||
${SSH_DIR}/ssh_host_rsa_key.pub
|
|
||||||
${SSH_DIR}/rtr-client.key
|
|
||||||
${SSH_DIR}/rtr-client.key.pub
|
|
||||||
${SSH_DIR}/rtr-authorized_keys
|
|
||||||
EOF
|
|
||||||
fi
|
|
||||||
@@ -1,136 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
|
||||||
|
|
||||||
COMMON_ENV_FILE="${ROOT_DIR}/environment.client"
|
|
||||||
CLIENT_ENV_FILE="${ROOT_DIR}/deploy/client/.env"
|
|
||||||
MODE="${RPKI_RTR_CLIENT_MODE:-tcp}"
|
|
||||||
COMPOSE_PROJECT="${RPKI_RTR_CLIENT_COMPOSE_PROJECT:-rpki-rtr-client}"
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage: scripts/11_start_client.sh [options]
|
|
||||||
|
|
||||||
Read root ./environment.client, update deploy/client/.env, then start the RTR debug client.
|
|
||||||
|
|
||||||
Options:
|
|
||||||
--mode MODE Client mode: tcp|tls|ssh. Default: tcp
|
|
||||||
--env FILE Client common env file. Default: ./environment.client
|
|
||||||
-h, --help Show this help.
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
log() {
|
|
||||||
printf '[11_start_client] %s\n' "$*" >&2
|
|
||||||
}
|
|
||||||
|
|
||||||
stage() {
|
|
||||||
printf '\n====== %s ======\n' "$*" >&2
|
|
||||||
}
|
|
||||||
|
|
||||||
die() {
|
|
||||||
printf '[11_start_client] ERROR: %s\n' "$*" >&2
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
set_env_var() {
|
|
||||||
local env_file="$1"
|
|
||||||
local key="$2"
|
|
||||||
local value="$3"
|
|
||||||
local tmp_file
|
|
||||||
|
|
||||||
tmp_file="${env_file}.tmp.$$"
|
|
||||||
awk -v key="${key}" -v value="${value}" '
|
|
||||||
BEGIN { done = 0 }
|
|
||||||
$0 ~ "^[[:space:]]*#?[[:space:]]*" key "=" {
|
|
||||||
if (!done) {
|
|
||||||
print key "=" value
|
|
||||||
done = 1
|
|
||||||
}
|
|
||||||
next
|
|
||||||
}
|
|
||||||
{ print }
|
|
||||||
END {
|
|
||||||
if (!done) {
|
|
||||||
print key "=" value
|
|
||||||
}
|
|
||||||
}
|
|
||||||
' "${env_file}" >"${tmp_file}"
|
|
||||||
mv "${tmp_file}" "${env_file}"
|
|
||||||
}
|
|
||||||
|
|
||||||
compose_file_for_mode() {
|
|
||||||
case "$1" in
|
|
||||||
tcp) printf '%s\n' "${ROOT_DIR}/deploy/client/docker-compose.tcp.yml" ;;
|
|
||||||
tls) printf '%s\n' "${ROOT_DIR}/deploy/client/docker-compose.tls.yml" ;;
|
|
||||||
ssh) printf '%s\n' "${ROOT_DIR}/deploy/client/docker-compose.ssh.yml" ;;
|
|
||||||
*) die "unsupported mode '$1', expected tcp|tls|ssh" ;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
|
|
||||||
while [ "$#" -gt 0 ]; do
|
|
||||||
case "$1" in
|
|
||||||
--mode)
|
|
||||||
[ "$#" -ge 2 ] || die "--mode requires a value"
|
|
||||||
MODE="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
--env)
|
|
||||||
[ "$#" -ge 2 ] || die "--env requires a value"
|
|
||||||
COMMON_ENV_FILE="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
-h|--help)
|
|
||||||
usage
|
|
||||||
exit 0
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
die "unknown option: $1"
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
CLIENT_COMPOSE_FILE="$(compose_file_for_mode "${MODE}")"
|
|
||||||
|
|
||||||
stage "Checking Docker"
|
|
||||||
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
|
||||||
if docker compose version >/dev/null 2>&1; then
|
|
||||||
COMPOSE_BIN=(docker compose)
|
|
||||||
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
|
||||||
COMPOSE_BIN=(docker-compose)
|
|
||||||
else
|
|
||||||
die "docker compose is not available"
|
|
||||||
fi
|
|
||||||
|
|
||||||
[ -f "${CLIENT_ENV_FILE}" ] || die "deploy client env file not found: ${CLIENT_ENV_FILE}"
|
|
||||||
[ -f "${CLIENT_COMPOSE_FILE}" ] || die "client compose file not found: ${CLIENT_COMPOSE_FILE}"
|
|
||||||
|
|
||||||
stage "Reading Client Environment"
|
|
||||||
if [ -f "${COMMON_ENV_FILE}" ]; then
|
|
||||||
updated_count=0
|
|
||||||
while IFS= read -r line || [ -n "${line}" ]; do
|
|
||||||
case "${line}" in
|
|
||||||
''|\#*) continue ;;
|
|
||||||
*=*)
|
|
||||||
key="${line%%=*}"
|
|
||||||
value="${line#*=}"
|
|
||||||
set_env_var "${CLIENT_ENV_FILE}" "${key}" "${value}"
|
|
||||||
updated_count=$((updated_count + 1))
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
done <"${COMMON_ENV_FILE}"
|
|
||||||
log "updated ${updated_count} client env variable(s)"
|
|
||||||
else
|
|
||||||
log "environment.client file not found, skip env update"
|
|
||||||
fi
|
|
||||||
|
|
||||||
stage "Starting RTR Debug Client"
|
|
||||||
log "mode: ${MODE}"
|
|
||||||
"${COMPOSE_BIN[@]}" --env-file "${CLIENT_ENV_FILE}" -p "${COMPOSE_PROJECT}" -f "${CLIENT_COMPOSE_FILE}" up -d --no-build rtr-debug-client
|
|
||||||
|
|
||||||
log "checking client compose status"
|
|
||||||
"${COMPOSE_BIN[@]}" --env-file "${CLIENT_ENV_FILE}" -p "${COMPOSE_PROJECT}" -f "${CLIENT_COMPOSE_FILE}" ps rtr-debug-client
|
|
||||||
|
|
||||||
stage "Client Started"
|
|
||||||
@@ -1,82 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
|
||||||
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
|
||||||
|
|
||||||
CLIENT_ENV_FILE="${ROOT_DIR}/deploy/client/.env"
|
|
||||||
MODE="${RPKI_RTR_CLIENT_MODE:-tcp}"
|
|
||||||
COMPOSE_PROJECT="${RPKI_RTR_CLIENT_COMPOSE_PROJECT:-rpki-rtr-client}"
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage: scripts/12_shutdown_client.sh [options]
|
|
||||||
|
|
||||||
Stop the RTR debug client started by scripts/11_start_client.sh.
|
|
||||||
|
|
||||||
Options:
|
|
||||||
--mode MODE Client mode: tcp|tls|ssh. Default: tcp
|
|
||||||
-h, --help Show this help.
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
log() {
|
|
||||||
printf '[12_shutdown_client] %s\n' "$*" >&2
|
|
||||||
}
|
|
||||||
|
|
||||||
stage() {
|
|
||||||
printf '\n====== %s ======\n' "$*" >&2
|
|
||||||
}
|
|
||||||
|
|
||||||
die() {
|
|
||||||
printf '[12_shutdown_client] ERROR: %s\n' "$*" >&2
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
compose_file_for_mode() {
|
|
||||||
case "$1" in
|
|
||||||
tcp) printf '%s\n' "${ROOT_DIR}/deploy/client/docker-compose.tcp.yml" ;;
|
|
||||||
tls) printf '%s\n' "${ROOT_DIR}/deploy/client/docker-compose.tls.yml" ;;
|
|
||||||
ssh) printf '%s\n' "${ROOT_DIR}/deploy/client/docker-compose.ssh.yml" ;;
|
|
||||||
*) die "unsupported mode '$1', expected tcp|tls|ssh" ;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
|
|
||||||
while [ "$#" -gt 0 ]; do
|
|
||||||
case "$1" in
|
|
||||||
--mode)
|
|
||||||
[ "$#" -ge 2 ] || die "--mode requires a value"
|
|
||||||
MODE="$2"
|
|
||||||
shift 2
|
|
||||||
;;
|
|
||||||
-h|--help)
|
|
||||||
usage
|
|
||||||
exit 0
|
|
||||||
;;
|
|
||||||
*)
|
|
||||||
die "unknown option: $1"
|
|
||||||
;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
CLIENT_COMPOSE_FILE="$(compose_file_for_mode "${MODE}")"
|
|
||||||
|
|
||||||
stage "Checking Docker"
|
|
||||||
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
|
||||||
if docker compose version >/dev/null 2>&1; then
|
|
||||||
COMPOSE_BIN=(docker compose)
|
|
||||||
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
|
||||||
COMPOSE_BIN=(docker-compose)
|
|
||||||
else
|
|
||||||
die "docker compose is not available"
|
|
||||||
fi
|
|
||||||
|
|
||||||
stage "Stopping RTR Debug Client"
|
|
||||||
log "mode: ${MODE}"
|
|
||||||
if [ -f "${CLIENT_ENV_FILE}" ] && [ -f "${CLIENT_COMPOSE_FILE}" ]; then
|
|
||||||
"${COMPOSE_BIN[@]}" --env-file "${CLIENT_ENV_FILE}" -p "${COMPOSE_PROJECT}" -f "${CLIENT_COMPOSE_FILE}" down --remove-orphans
|
|
||||||
else
|
|
||||||
log "client compose/env not found, skip"
|
|
||||||
fi
|
|
||||||
|
|
||||||
stage "Client Shutdown Finished"
|
|
||||||
@@ -1,218 +0,0 @@
|
|||||||
# 脚本说明
|
|
||||||
|
|
||||||
`scripts/` 目录用于离线安装、启动、健康检查、关闭和清理。
|
|
||||||
|
|
||||||
所有脚本建议在仓库根目录执行:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash scripts/00_0_install.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
## 目录
|
|
||||||
|
|
||||||
- [推荐流程](#推荐流程)
|
|
||||||
- [环境变量文件](#环境变量文件)
|
|
||||||
- [00 安装](#00-安装)
|
|
||||||
- [00 Pre-Check](#00-pre-check)
|
|
||||||
- [01 启动 Server 和 Console](#01-启动-server-和-console)
|
|
||||||
- [02 自检](#02-自检)
|
|
||||||
- [03 关闭 Server 和 Console](#03-关闭-server-和-console)
|
|
||||||
- [04 清理](#04-清理)
|
|
||||||
- [10 生成证书](#10-生成证书)
|
|
||||||
- [11 启动 Client](#11-启动-client)
|
|
||||||
- [12 关闭 Client](#12-关闭-client)
|
|
||||||
- [Windows 路径注意事项](#windows-路径注意事项)
|
|
||||||
|
|
||||||
## 推荐流程
|
|
||||||
|
|
||||||
```bash
|
|
||||||
# 1. 加载镜像并执行 pre-check。
|
|
||||||
bash scripts/00_0_install.sh
|
|
||||||
|
|
||||||
# 2. 启动真实 server 和 console。
|
|
||||||
bash scripts/01_start.sh --mode tcp
|
|
||||||
|
|
||||||
# 3. 检查 server 和 console 是否正常。
|
|
||||||
bash scripts/02_self_check.sh
|
|
||||||
|
|
||||||
# 4. 关闭 server 和 console。
|
|
||||||
bash scripts/03_shutdown.sh --mode tcp
|
|
||||||
|
|
||||||
# 5. 清理 server 运行产生的文件。
|
|
||||||
bash scripts/04_cleanup.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
client 单独启动和关闭:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash scripts/11_start_client.sh --mode tcp
|
|
||||||
bash scripts/12_shutdown_client.sh --mode tcp
|
|
||||||
```
|
|
||||||
|
|
||||||
支持的模式:
|
|
||||||
|
|
||||||
- server:`tcp`、`tls`、`ssh`、`base`
|
|
||||||
- client:`tcp`、`tls`、`ssh`
|
|
||||||
|
|
||||||
## 环境变量文件
|
|
||||||
|
|
||||||
`01_start.sh` 会读取根目录下的 `environment.server`。如果文件存在,会将其中所有 `KEY=VALUE` 同步到 `deploy/server/.env`。
|
|
||||||
|
|
||||||
示例:
|
|
||||||
|
|
||||||
```env
|
|
||||||
RPKI_RTR_CCR_HOST_DIR=/opt/rpki-rtr/data
|
|
||||||
RPKI_RTR_SLURM_HOST_DIR=/opt/rpki-rtr/data
|
|
||||||
RPKI_RTR_DB_HOST_DIR=/opt/rpki-rtr/rtr-db
|
|
||||||
RPKI_RTR_LOG_HOST_DIR=/opt/rpki-rtr/logs/server
|
|
||||||
RPKI_RTR_REPORT_HOST_DIR=/opt/rpki-rtr/report
|
|
||||||
```
|
|
||||||
|
|
||||||
`11_start_client.sh` 会读取根目录下的 `environment.client`。如果文件存在,会将其中所有 `KEY=VALUE` 同步到 `deploy/client/.env`。
|
|
||||||
|
|
||||||
如果 `environment.server` 或 `environment.client` 不存在,对应脚本会跳过更新,直接使用已有的 deploy `.env` 文件。
|
|
||||||
|
|
||||||
## 00 安装
|
|
||||||
|
|
||||||
`00_0_install.sh` 用于从 `images/` 目录加载离线 Docker 镜像,更新 deploy `.env` 中的镜像名,然后执行 pre-check。
|
|
||||||
|
|
||||||
默认镜像包查找规则:
|
|
||||||
|
|
||||||
- server:`images/rpki-rtr-*.tar`,排除 `rpki-rtr-debug-client-*.tar`
|
|
||||||
- client:`images/rpki-rtr-debug-client-*.tar`
|
|
||||||
- console:`images/rpki-console-ui-*.tar`
|
|
||||||
|
|
||||||
也可以显式指定镜像包:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash scripts/00_0_install.sh \
|
|
||||||
--server-tar images/rpki-rtr-amd64-v1.2.tar \
|
|
||||||
--client-tar images/rpki-rtr-debug-client-amd64-v1.2.tar \
|
|
||||||
--console-tar images/rpki-console-ui-amd64-v1.0.tar
|
|
||||||
```
|
|
||||||
|
|
||||||
脚本会从 `docker load` 输出中解析真实镜像名,并写入:
|
|
||||||
|
|
||||||
- `deploy/server/.env` 的 `RPKI_RTR_SERVER_IMAGE`
|
|
||||||
- `deploy/console/.env` 的 `RPKI_CONSOLE_UI_IMAGE`
|
|
||||||
|
|
||||||
## 00 Pre-Check
|
|
||||||
|
|
||||||
`00_1_pre_check.sh` 会在 `.precheck/` 下准备隔离测试数据,使用仓库里的 Docker Compose 文件启动 server 和 debug client,然后检查:
|
|
||||||
|
|
||||||
- admin health 接口返回正常。
|
|
||||||
- server 写出了 source report。
|
|
||||||
- debug client 收到了 EndOfData 和 payload summary。
|
|
||||||
- pre-check 日志中没有 fatal 错误模式。
|
|
||||||
|
|
||||||
相关辅助脚本:
|
|
||||||
|
|
||||||
- `00_2_generate_ccr.sh`:生成固定的 CCR 测试文件。
|
|
||||||
- `00_2_pre_check_prepare.sh`:创建 `.precheck/` 数据、SLURM 文件和 env 文件。
|
|
||||||
- `00_3_pre_check_cleanup.sh`:停止 pre-check 容器并清理 `.precheck/`。
|
|
||||||
|
|
||||||
## 01 启动 Server 和 Console
|
|
||||||
|
|
||||||
`01_start.sh` 用于启动真实的 RTR server 和 console。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash scripts/01_start.sh --mode tcp
|
|
||||||
```
|
|
||||||
|
|
||||||
启动后脚本会检查容器是否进入 `running` 状态。如果容器启动后退出,脚本会打印 compose 状态和最近日志,然后报错退出,不会继续打印 `Start Finished`。
|
|
||||||
|
|
||||||
常用参数:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash scripts/01_start.sh --mode tls
|
|
||||||
bash scripts/01_start.sh --mode ssh
|
|
||||||
bash scripts/01_start.sh --wait 60
|
|
||||||
bash scripts/01_start.sh --env ./environment.server
|
|
||||||
```
|
|
||||||
|
|
||||||
注意:console 不读取 `environment.server`,只使用 `deploy/console/.env`。
|
|
||||||
|
|
||||||
## 02 自检
|
|
||||||
|
|
||||||
`02_self_check.sh` 用于检查 `01_start.sh` 启动的 server 和 console 是否正常。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash scripts/02_self_check.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
检查内容包括:
|
|
||||||
|
|
||||||
- server 容器是否 running。
|
|
||||||
- console 容器是否 running。
|
|
||||||
- RTR TCP 端口是否可连接。
|
|
||||||
- admin health 接口是否健康。
|
|
||||||
- console 端口和 HTTP 根路径是否可访问。
|
|
||||||
|
|
||||||
如果启动较慢,可以增加等待时间:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash scripts/02_self_check.sh --wait 120
|
|
||||||
```
|
|
||||||
|
|
||||||
## 03 关闭 Server 和 Console
|
|
||||||
|
|
||||||
`03_shutdown.sh` 用于关闭 `01_start.sh` 启动的服务。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash scripts/03_shutdown.sh --mode tcp
|
|
||||||
```
|
|
||||||
|
|
||||||
脚本会先关闭 console,再关闭 RTR server。
|
|
||||||
|
|
||||||
## 04 清理
|
|
||||||
|
|
||||||
`04_cleanup.sh` 用于删除 server 运行产生的目录:
|
|
||||||
|
|
||||||
- `logs/`
|
|
||||||
- `report/`
|
|
||||||
- `rtr-db/`
|
|
||||||
|
|
||||||
推荐先关闭服务,再清理:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash scripts/03_shutdown.sh
|
|
||||||
bash scripts/04_cleanup.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
清理前脚本会检查 `rpki-rtr` 是否仍在运行。如果仍在运行,会拒绝清理。只有明确需要强制清理时才使用:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash scripts/04_cleanup.sh --force
|
|
||||||
```
|
|
||||||
|
|
||||||
清理脚本会拒绝删除仓库根目录之外的路径。如果某一项清理失败,会继续清理后续项,最后返回失败。
|
|
||||||
|
|
||||||
## 10 生成证书
|
|
||||||
|
|
||||||
`10_generate_certs.sh` 用于生成 TLS/SSH 模式需要的本地证书材料(测试用)。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash scripts/10_generate_certs.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
## 11 启动 Client
|
|
||||||
|
|
||||||
`11_start_client.sh` 用于启动 RTR debug client。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash scripts/11_start_client.sh --mode tcp
|
|
||||||
bash scripts/11_start_client.sh --mode tls
|
|
||||||
bash scripts/11_start_client.sh --mode ssh
|
|
||||||
```
|
|
||||||
|
|
||||||
如果 `environment.client` 存在,脚本会将其中所有 `KEY=VALUE` 同步到 `deploy/client/.env`;如果不存在,则直接使用已有的 `deploy/client/.env`。
|
|
||||||
|
|
||||||
## 12 关闭 Client
|
|
||||||
|
|
||||||
`12_shutdown_client.sh` 用于关闭 `11_start_client.sh` 启动的 client。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
bash scripts/12_shutdown_client.sh --mode tcp
|
|
||||||
```
|
|
||||||
|
|
||||||
关闭时建议使用和启动时相同的 mode。
|
|
||||||
70
scripts/benchmark/README.md
Normal file
70
scripts/benchmark/README.md
Normal file
@@ -0,0 +1,70 @@
|
|||||||
|
# RPKI Benchmarks (Stage2, selected_der_v2)
|
||||||
|
|
||||||
|
This directory contains a reproducible, one-click benchmark to measure **decode + profile validate**
|
||||||
|
performance for all supported object types and compare **OURS** against the **Routinator baseline**
|
||||||
|
(`rpki` crate `=0.19.1` with `repository` feature).
|
||||||
|
|
||||||
|
## What it measures
|
||||||
|
|
||||||
|
Dataset:
|
||||||
|
|
||||||
|
- Fixtures: `rpki/tests/benchmark/selected_der_v2/`
|
||||||
|
- Objects: `cer`, `crl`, `manifest` (`.mft`), `roa`, `aspa` (`.asa`)
|
||||||
|
- Samples: 10 quantiles per type (`min/p01/p10/p25/p50/p75/p90/p95/p99/max`) → 50 files total
|
||||||
|
|
||||||
|
Metrics:
|
||||||
|
|
||||||
|
- **decode+validate**: `decode_der` (parse + profile validate) for each object file
|
||||||
|
- **landing** (OURS only): `PackFile::from_bytes_compute_sha256` + CBOR encode + `RocksDB put_raw`
|
||||||
|
- **compare**: ratio `ours_ns/op ÷ rout_ns/op` for decode+validate
|
||||||
|
|
||||||
|
## Default benchmark settings
|
||||||
|
|
||||||
|
Both OURS and Routinator baseline use the same run settings:
|
||||||
|
|
||||||
|
- warmup: `10` iterations
|
||||||
|
- rounds: `3`
|
||||||
|
- adaptive loop target: `min_round_ms=200` (with an internal max of `1_000_000` iters)
|
||||||
|
- strict DER: `true` (baseline)
|
||||||
|
- cert inspect: `false` (baseline)
|
||||||
|
|
||||||
|
You can override the settings via environment variables in the runner script:
|
||||||
|
|
||||||
|
- `BENCH_WARMUP_ITERS` (default `10`)
|
||||||
|
- `BENCH_ROUNDS` (default `3`)
|
||||||
|
- `BENCH_MIN_ROUND_MS` (default `200`)
|
||||||
|
|
||||||
|
## One-click run (OURS + Routinator compare)
|
||||||
|
|
||||||
|
From the `rpki/` crate directory:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/benchmark/run_stage2_selected_der_v2_release.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
Outputs are written under:
|
||||||
|
|
||||||
|
- `rpki/target/bench/`
|
||||||
|
- OURS decode+validate: `stage2_selected_der_v2_decode_release_<TS>.{md,csv}`
|
||||||
|
- OURS landing: `stage2_selected_der_v2_landing_release_<TS>.{md,csv}`
|
||||||
|
- Routinator: `stage2_selected_der_v2_routinator_decode_release_<TS>.{md,csv}`
|
||||||
|
- Compare: `stage2_selected_der_v2_compare_ours_vs_routinator_decode_release_<TS>.{md,csv}`
|
||||||
|
- Summary: `stage2_selected_der_v2_compare_summary_<TS>.md`
|
||||||
|
|
||||||
|
### Why decode and landing are separated
|
||||||
|
|
||||||
|
The underlying benchmark can run in `BENCH_MODE=both`, but the **landing** part writes to RocksDB
|
||||||
|
and may trigger background work (e.g., compactions) that can **skew subsequent decode timings**.
|
||||||
|
For a fair OURS-vs-Routinator comparison, the runner script:
|
||||||
|
|
||||||
|
- runs `BENCH_MODE=decode_validate` for comparison, and
|
||||||
|
- runs `BENCH_MODE=landing` separately for landing-only numbers.
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- The Routinator baseline benchmark is implemented in-repo under:
|
||||||
|
- `rpki/benchmark/routinator_object_bench/`
|
||||||
|
- It pins `rpki = "=0.19.1"` in its `Cargo.toml`.
|
||||||
|
- This benchmark is implemented as an `#[ignore]` integration test:
|
||||||
|
- `rpki/tests/bench_stage2_decode_profile_selected_der_v2.rs`
|
||||||
|
- The runner script invokes it with `cargo test --release ... -- --ignored --nocapture`.
|
||||||
123
scripts/benchmark/run_stage2_selected_der_v2_release.sh
Executable file
123
scripts/benchmark/run_stage2_selected_der_v2_release.sh
Executable file
@@ -0,0 +1,123 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
# Stage2 (selected_der_v2) decode+profile validate benchmark.
|
||||||
|
# Runs:
|
||||||
|
# 1) OURS decode+validate benchmark and writes MD/CSV.
|
||||||
|
# 2) OURS landing benchmark and writes MD/CSV.
|
||||||
|
# 3) Routinator baseline decode benchmark (rpki crate =0.19.1).
|
||||||
|
# 4) Produces a joined compare CSV/MD and a short geomean summary.
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
cd "$ROOT_DIR"
|
||||||
|
|
||||||
|
OUT_DIR="$ROOT_DIR/target/bench"
|
||||||
|
mkdir -p "$OUT_DIR"
|
||||||
|
|
||||||
|
TS="$(date -u +%Y%m%dT%H%M%SZ)"
|
||||||
|
|
||||||
|
WARMUP_ITERS="${BENCH_WARMUP_ITERS:-10}"
|
||||||
|
ROUNDS="${BENCH_ROUNDS:-3}"
|
||||||
|
MIN_ROUND_MS="${BENCH_MIN_ROUND_MS:-200}"
|
||||||
|
|
||||||
|
OURS_MD="$OUT_DIR/stage2_selected_der_v2_decode_release_${TS}.md"
|
||||||
|
OURS_CSV="$OUT_DIR/stage2_selected_der_v2_decode_release_${TS}.csv"
|
||||||
|
|
||||||
|
OURS_LANDING_MD="$OUT_DIR/stage2_selected_der_v2_landing_release_${TS}.md"
|
||||||
|
OURS_LANDING_CSV="$OUT_DIR/stage2_selected_der_v2_landing_release_${TS}.csv"
|
||||||
|
|
||||||
|
ROUT_MD="$OUT_DIR/stage2_selected_der_v2_routinator_decode_release_${TS}.md"
|
||||||
|
ROUT_CSV="$OUT_DIR/stage2_selected_der_v2_routinator_decode_release_${TS}.csv"
|
||||||
|
|
||||||
|
COMPARE_MD="$OUT_DIR/stage2_selected_der_v2_compare_ours_vs_routinator_decode_release_${TS}.md"
|
||||||
|
COMPARE_CSV="$OUT_DIR/stage2_selected_der_v2_compare_ours_vs_routinator_decode_release_${TS}.csv"
|
||||||
|
|
||||||
|
SUMMARY_MD="$OUT_DIR/stage2_selected_der_v2_compare_summary_${TS}.md"
|
||||||
|
|
||||||
|
echo "[1/4] OURS: decode+validate benchmark (release)..." >&2
|
||||||
|
BENCH_MODE="decode_validate" \
|
||||||
|
BENCH_WARMUP_ITERS="$WARMUP_ITERS" \
|
||||||
|
BENCH_ROUNDS="$ROUNDS" \
|
||||||
|
BENCH_MIN_ROUND_MS="$MIN_ROUND_MS" \
|
||||||
|
BENCH_OUT_MD="$OURS_MD" \
|
||||||
|
BENCH_OUT_CSV="$OURS_CSV" \
|
||||||
|
cargo test --release --test bench_stage2_decode_profile_selected_der_v2 -- --ignored --nocapture >/dev/null
|
||||||
|
|
||||||
|
echo "[2/4] OURS: landing benchmark (release)..." >&2
|
||||||
|
BENCH_MODE="landing" \
|
||||||
|
BENCH_WARMUP_ITERS="$WARMUP_ITERS" \
|
||||||
|
BENCH_ROUNDS="$ROUNDS" \
|
||||||
|
BENCH_MIN_ROUND_MS="$MIN_ROUND_MS" \
|
||||||
|
BENCH_OUT_MD_LANDING="$OURS_LANDING_MD" \
|
||||||
|
BENCH_OUT_CSV_LANDING="$OURS_LANDING_CSV" \
|
||||||
|
cargo test --release --test bench_stage2_decode_profile_selected_der_v2 -- --ignored --nocapture >/dev/null
|
||||||
|
|
||||||
|
echo "[3/4] Routinator baseline + compare join..." >&2
|
||||||
|
OURS_CSV="$OURS_CSV" \
|
||||||
|
ROUT_CSV="$ROUT_CSV" \
|
||||||
|
ROUT_MD="$ROUT_MD" \
|
||||||
|
COMPARE_CSV="$COMPARE_CSV" \
|
||||||
|
COMPARE_MD="$COMPARE_MD" \
|
||||||
|
WARMUP_ITERS="$WARMUP_ITERS" \
|
||||||
|
ROUNDS="$ROUNDS" \
|
||||||
|
MIN_ROUND_MS="$MIN_ROUND_MS" \
|
||||||
|
scripts/stage2_perf_compare_m4.sh >/dev/null
|
||||||
|
|
||||||
|
echo "[4/4] Summary (geomean ratios)..." >&2
|
||||||
|
python3 - "$COMPARE_CSV" "$SUMMARY_MD" <<'PY'
|
||||||
|
import csv
|
||||||
|
import math
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
|
||||||
|
in_csv = Path(sys.argv[1])
|
||||||
|
out_md = Path(sys.argv[2])
|
||||||
|
|
||||||
|
rows = list(csv.DictReader(in_csv.open(newline="")))
|
||||||
|
ratios = {}
|
||||||
|
for r in rows:
|
||||||
|
ratios.setdefault(r["type"], []).append(float(r["ratio_ours_over_rout"]))
|
||||||
|
|
||||||
|
def geomean(vals):
|
||||||
|
return math.exp(sum(math.log(v) for v in vals) / len(vals))
|
||||||
|
|
||||||
|
def p50(vals):
|
||||||
|
v = sorted(vals)
|
||||||
|
n = len(v)
|
||||||
|
if n % 2 == 1:
|
||||||
|
return v[n // 2]
|
||||||
|
return (v[n // 2 - 1] + v[n // 2]) / 2.0
|
||||||
|
|
||||||
|
all_vals = [float(r["ratio_ours_over_rout"]) for r in rows]
|
||||||
|
types = ["all"] + sorted(ratios.keys())
|
||||||
|
|
||||||
|
now = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||||
|
lines = []
|
||||||
|
lines.append("# Stage2 selected_der_v2 compare summary (release)\n\n")
|
||||||
|
lines.append(f"- recorded_at_utc: `{now}`\n")
|
||||||
|
lines.append(f"- inputs_csv: `{in_csv}`\n\n")
|
||||||
|
lines.append("| type | n | min | p50 | geomean | max | >1 count |\n")
|
||||||
|
lines.append("|---|---:|---:|---:|---:|---:|---:|\n")
|
||||||
|
|
||||||
|
for t in types:
|
||||||
|
vals = all_vals if t == "all" else ratios[t]
|
||||||
|
vals_sorted = sorted(vals)
|
||||||
|
lines.append(
|
||||||
|
f"| {t} | {len(vals_sorted)} | {vals_sorted[0]:.4f} | {p50(vals_sorted):.4f} | "
|
||||||
|
f"{geomean(vals_sorted):.4f} | {vals_sorted[-1]:.4f} | {sum(1 for v in vals_sorted if v>1.0)} |\n"
|
||||||
|
)
|
||||||
|
|
||||||
|
out_md.write_text("".join(lines), encoding="utf-8")
|
||||||
|
print(out_md)
|
||||||
|
PY
|
||||||
|
|
||||||
|
echo "Done." >&2
|
||||||
|
echo "- OURS decode MD: $OURS_MD" >&2
|
||||||
|
echo "- OURS decode CSV: $OURS_CSV" >&2
|
||||||
|
echo "- OURS landing MD: $OURS_LANDING_MD" >&2
|
||||||
|
echo "- OURS landing CSV: $OURS_LANDING_CSV" >&2
|
||||||
|
echo "- Routinator: $ROUT_MD" >&2
|
||||||
|
echo "- Compare MD: $COMPARE_MD" >&2
|
||||||
|
echo "- Compare CSV: $COMPARE_CSV" >&2
|
||||||
|
echo "- Summary MD: $SUMMARY_MD" >&2
|
||||||
56
scripts/cir/README.md
Normal file
56
scripts/cir/README.md
Normal file
@@ -0,0 +1,56 @@
|
|||||||
|
# CIR Scripts
|
||||||
|
|
||||||
|
## `cir-rsync-wrapper`
|
||||||
|
|
||||||
|
一个用于 CIR 黑盒 replay 的 rsync wrapper。
|
||||||
|
|
||||||
|
### 环境变量
|
||||||
|
|
||||||
|
- `REAL_RSYNC_BIN`
|
||||||
|
- 真实 rsync 二进制路径
|
||||||
|
- 默认优先 `/usr/bin/rsync`
|
||||||
|
- `CIR_MIRROR_ROOT`
|
||||||
|
- 本地镜像树根目录
|
||||||
|
- 当命令行中出现 `rsync://...` source 时必需
|
||||||
|
|
||||||
|
### 语义
|
||||||
|
|
||||||
|
- 仅改写 `rsync://host/path` 类型参数
|
||||||
|
- 其它参数原样透传给真实 rsync
|
||||||
|
- 改写目标:
|
||||||
|
- `rsync://example.net/repo/a.roa`
|
||||||
|
- →
|
||||||
|
- `<CIR_MIRROR_ROOT>/example.net/repo/a.roa`
|
||||||
|
|
||||||
|
### 兼容目标
|
||||||
|
|
||||||
|
- Routinator `--rsync-command`
|
||||||
|
- `rpki-client -e rsync_prog`
|
||||||
|
|
||||||
|
## 其它脚本
|
||||||
|
|
||||||
|
- `run_cir_replay_ours.sh`
|
||||||
|
- `run_cir_replay_routinator.sh`
|
||||||
|
- `run_cir_replay_rpki_client.sh`
|
||||||
|
- `run_cir_replay_matrix.sh`
|
||||||
|
|
||||||
|
## `cir-local-link-sync.py`
|
||||||
|
|
||||||
|
当 `CIR_LOCAL_LINK_MODE=1` 且 wrapper 检测到 source 已经被改写为本地 mirror 路径时,
|
||||||
|
wrapper 不再调用真实 `rsync`,而是调用这个 helper 完成:
|
||||||
|
|
||||||
|
- `hardlink` 优先的本地树同步
|
||||||
|
- 失败时回退到 copy
|
||||||
|
- 支持 `--delete`
|
||||||
|
|
||||||
|
`run_cir_replay_matrix.sh` 会顺序执行:
|
||||||
|
|
||||||
|
- `ours`
|
||||||
|
- Routinator
|
||||||
|
- `rpki-client`
|
||||||
|
|
||||||
|
并汇总生成:
|
||||||
|
|
||||||
|
- `summary.json`
|
||||||
|
- `summary.md`
|
||||||
|
- `detail.md`
|
||||||
BIN
scripts/cir/__pycache__/cir-local-link-sync.cpython-312.pyc
Normal file
BIN
scripts/cir/__pycache__/cir-local-link-sync.cpython-312.pyc
Normal file
Binary file not shown.
BIN
scripts/cir/__pycache__/cir-rsync-wrappercpython-312.pyc
Normal file
BIN
scripts/cir/__pycache__/cir-rsync-wrappercpython-312.pyc
Normal file
Binary file not shown.
BIN
scripts/cir/__pycache__/json_to_vaps_csv.cpython-312.pyc
Normal file
BIN
scripts/cir/__pycache__/json_to_vaps_csv.cpython-312.pyc
Normal file
Binary file not shown.
136
scripts/cir/cir-local-link-sync.py
Executable file
136
scripts/cir/cir-local-link-sync.py
Executable file
@@ -0,0 +1,136 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
import argparse
|
||||||
|
import errno
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
def _same_inode(src: Path, dst: Path) -> bool:
|
||||||
|
try:
|
||||||
|
src_stat = src.stat()
|
||||||
|
dst_stat = dst.stat()
|
||||||
|
except FileNotFoundError:
|
||||||
|
return False
|
||||||
|
return (src_stat.st_dev, src_stat.st_ino) == (dst_stat.st_dev, dst_stat.st_ino)
|
||||||
|
|
||||||
|
|
||||||
|
def _remove_path(path: Path) -> None:
|
||||||
|
if not path.exists() and not path.is_symlink():
|
||||||
|
return
|
||||||
|
if path.is_dir() and not path.is_symlink():
|
||||||
|
shutil.rmtree(path)
|
||||||
|
else:
|
||||||
|
path.unlink()
|
||||||
|
|
||||||
|
|
||||||
|
def _prune_empty_dirs(root: Path) -> None:
|
||||||
|
if not root.exists():
|
||||||
|
return
|
||||||
|
for path in sorted((p for p in root.rglob("*") if p.is_dir()), key=lambda p: len(p.parts), reverse=True):
|
||||||
|
try:
|
||||||
|
path.rmdir()
|
||||||
|
except OSError:
|
||||||
|
pass
|
||||||
|
|
||||||
|
|
||||||
|
def _link_or_copy(src: Path, dst: Path) -> str:
|
||||||
|
dst.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
if dst.exists() or dst.is_symlink():
|
||||||
|
if _same_inode(src, dst):
|
||||||
|
return "reused"
|
||||||
|
_remove_path(dst)
|
||||||
|
try:
|
||||||
|
os.link(src, dst)
|
||||||
|
return "linked"
|
||||||
|
except OSError as err:
|
||||||
|
if err.errno not in (errno.EXDEV, errno.EPERM, errno.EMLINK, errno.ENOTSUP, errno.EACCES):
|
||||||
|
raise
|
||||||
|
shutil.copy2(src, dst)
|
||||||
|
return "copied"
|
||||||
|
|
||||||
|
|
||||||
|
def _file_map(src_arg: str, dest_arg: str) -> tuple[Path, dict[str, Path]]:
|
||||||
|
src = Path(src_arg.rstrip(os.sep))
|
||||||
|
if not src.exists():
|
||||||
|
raise FileNotFoundError(src)
|
||||||
|
mapping: dict[str, Path] = {}
|
||||||
|
if src.is_dir():
|
||||||
|
copy_contents = src_arg.endswith(os.sep)
|
||||||
|
if copy_contents:
|
||||||
|
root = src
|
||||||
|
for path in root.rglob("*"):
|
||||||
|
if path.is_file():
|
||||||
|
mapping[path.relative_to(root).as_posix()] = path
|
||||||
|
else:
|
||||||
|
root = src
|
||||||
|
base = src.name
|
||||||
|
for path in root.rglob("*"):
|
||||||
|
if path.is_file():
|
||||||
|
rel = Path(base) / path.relative_to(root)
|
||||||
|
mapping[rel.as_posix()] = path
|
||||||
|
else:
|
||||||
|
dest_path = Path(dest_arg)
|
||||||
|
if dest_arg.endswith(os.sep) or dest_path.is_dir():
|
||||||
|
mapping[src.name] = src
|
||||||
|
else:
|
||||||
|
mapping[dest_path.name] = src
|
||||||
|
return Path(dest_arg), mapping
|
||||||
|
|
||||||
|
|
||||||
|
def sync_local_tree(src_arg: str, dst_arg: str, delete: bool) -> dict[str, int]:
|
||||||
|
dst_root, mapping = _file_map(src_arg, dst_arg)
|
||||||
|
dst_root.mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
expected = {dst_root / rel for rel in mapping.keys()}
|
||||||
|
|
||||||
|
deleted = 0
|
||||||
|
if delete and dst_root.exists():
|
||||||
|
for path in sorted(dst_root.rglob("*"), key=lambda p: len(p.parts), reverse=True):
|
||||||
|
if path.is_dir():
|
||||||
|
continue
|
||||||
|
if path not in expected:
|
||||||
|
_remove_path(path)
|
||||||
|
deleted += 1
|
||||||
|
_prune_empty_dirs(dst_root)
|
||||||
|
|
||||||
|
linked = 0
|
||||||
|
copied = 0
|
||||||
|
reused = 0
|
||||||
|
for rel, src in mapping.items():
|
||||||
|
dst = dst_root / rel
|
||||||
|
result = _link_or_copy(src, dst)
|
||||||
|
if result == "linked":
|
||||||
|
linked += 1
|
||||||
|
elif result == "copied":
|
||||||
|
copied += 1
|
||||||
|
else:
|
||||||
|
reused += 1
|
||||||
|
|
||||||
|
return {
|
||||||
|
"files": len(mapping),
|
||||||
|
"linked": linked,
|
||||||
|
"copied": copied,
|
||||||
|
"reused": reused,
|
||||||
|
"deleted": deleted,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser(description="Sync a local CIR mirror tree using hardlinks when possible.")
|
||||||
|
parser.add_argument("--delete", action="store_true", help="Delete target files not present in source")
|
||||||
|
parser.add_argument("source")
|
||||||
|
parser.add_argument("dest")
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
summary = sync_local_tree(args.source, args.dest, args.delete)
|
||||||
|
print(
|
||||||
|
"local-link-sync files={files} linked={linked} copied={copied} reused={reused} deleted={deleted}".format(
|
||||||
|
**summary
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
127
scripts/cir/cir-rsync-wrapper
Executable file
127
scripts/cir/cir-rsync-wrapper
Executable file
@@ -0,0 +1,127 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
import os
|
||||||
|
import shutil
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
from urllib.parse import urlparse
|
||||||
|
|
||||||
|
|
||||||
|
def real_rsync_bin() -> str:
|
||||||
|
env = os.environ.get("REAL_RSYNC_BIN")
|
||||||
|
if env:
|
||||||
|
return env
|
||||||
|
default = "/usr/bin/rsync"
|
||||||
|
if Path(default).exists():
|
||||||
|
return default
|
||||||
|
found = shutil.which("rsync")
|
||||||
|
if found:
|
||||||
|
return found
|
||||||
|
raise SystemExit("cir-rsync-wrapper: REAL_RSYNC_BIN is not set and rsync was not found")
|
||||||
|
|
||||||
|
|
||||||
|
def rewrite_arg(arg: str, mirror_root: str | None) -> str:
|
||||||
|
if not arg.startswith("rsync://"):
|
||||||
|
return arg
|
||||||
|
if not mirror_root:
|
||||||
|
raise SystemExit(
|
||||||
|
"cir-rsync-wrapper: CIR_MIRROR_ROOT is required when an rsync:// source is present"
|
||||||
|
)
|
||||||
|
parsed = urlparse(arg)
|
||||||
|
if parsed.scheme != "rsync" or not parsed.hostname:
|
||||||
|
raise SystemExit(f"cir-rsync-wrapper: invalid rsync URI: {arg}")
|
||||||
|
path = parsed.path.lstrip("/")
|
||||||
|
local = Path(mirror_root).resolve() / parsed.hostname
|
||||||
|
if path:
|
||||||
|
local = local / path
|
||||||
|
local_str = str(local)
|
||||||
|
if local.exists() and local.is_dir() and not local_str.endswith("/"):
|
||||||
|
local_str += "/"
|
||||||
|
elif arg.endswith("/") and not local_str.endswith("/"):
|
||||||
|
local_str += "/"
|
||||||
|
return local_str
|
||||||
|
|
||||||
|
|
||||||
|
def filter_args(args: list[str]) -> list[str]:
|
||||||
|
mirror_root = os.environ.get("CIR_MIRROR_ROOT")
|
||||||
|
rewritten_any = any(arg.startswith("rsync://") for arg in args)
|
||||||
|
out: list[str] = []
|
||||||
|
i = 0
|
||||||
|
while i < len(args):
|
||||||
|
arg = args[i]
|
||||||
|
if rewritten_any:
|
||||||
|
if arg == "--address":
|
||||||
|
i += 2
|
||||||
|
continue
|
||||||
|
if arg.startswith("--address="):
|
||||||
|
i += 1
|
||||||
|
continue
|
||||||
|
if arg == "--contimeout":
|
||||||
|
i += 2
|
||||||
|
continue
|
||||||
|
if arg.startswith("--contimeout="):
|
||||||
|
i += 1
|
||||||
|
continue
|
||||||
|
out.append(rewrite_arg(arg, mirror_root))
|
||||||
|
i += 1
|
||||||
|
return out
|
||||||
|
|
||||||
|
|
||||||
|
def local_link_mode_enabled() -> bool:
|
||||||
|
value = os.environ.get("CIR_LOCAL_LINK_MODE", "")
|
||||||
|
return value.lower() in {"1", "true", "yes", "on"}
|
||||||
|
|
||||||
|
|
||||||
|
def extract_source_and_dest(args: list[str]) -> tuple[str, str]:
|
||||||
|
expects_value = {
|
||||||
|
"--timeout",
|
||||||
|
"--min-size",
|
||||||
|
"--max-size",
|
||||||
|
"--include",
|
||||||
|
"--exclude",
|
||||||
|
"--compare-dest",
|
||||||
|
}
|
||||||
|
positionals: list[str] = []
|
||||||
|
i = 0
|
||||||
|
while i < len(args):
|
||||||
|
arg = args[i]
|
||||||
|
if arg in expects_value:
|
||||||
|
i += 2
|
||||||
|
continue
|
||||||
|
if any(arg.startswith(prefix + "=") for prefix in expects_value):
|
||||||
|
i += 1
|
||||||
|
continue
|
||||||
|
if arg.startswith("-"):
|
||||||
|
i += 1
|
||||||
|
continue
|
||||||
|
positionals.append(arg)
|
||||||
|
i += 1
|
||||||
|
if len(positionals) < 2:
|
||||||
|
raise SystemExit("cir-rsync-wrapper: expected source and destination arguments")
|
||||||
|
return positionals[-2], positionals[-1]
|
||||||
|
|
||||||
|
|
||||||
|
def maybe_exec_local_link_sync(args: list[str], rewritten_any: bool) -> None:
|
||||||
|
if not rewritten_any or not local_link_mode_enabled():
|
||||||
|
return
|
||||||
|
source, dest = extract_source_and_dest(args)
|
||||||
|
if source.startswith("rsync://"):
|
||||||
|
raise SystemExit("cir-rsync-wrapper: expected rewritten local source for CIR_LOCAL_LINK_MODE")
|
||||||
|
helper = Path(__file__).with_name("cir-local-link-sync.py")
|
||||||
|
cmd = [sys.executable, str(helper)]
|
||||||
|
if "--delete" in args:
|
||||||
|
cmd.append("--delete")
|
||||||
|
cmd.extend([source, dest])
|
||||||
|
os.execv(sys.executable, cmd)
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
args = sys.argv[1:]
|
||||||
|
rewritten_any = any(arg.startswith("rsync://") for arg in args)
|
||||||
|
rewritten = filter_args(args)
|
||||||
|
maybe_exec_local_link_sync(rewritten, rewritten_any)
|
||||||
|
os.execv(real_rsync_bin(), [real_rsync_bin(), *rewritten])
|
||||||
|
return 127
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
32
scripts/cir/fetch_cir_sequence_from_remote.sh
Executable file
32
scripts/cir/fetch_cir_sequence_from_remote.sh
Executable file
@@ -0,0 +1,32 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage:
|
||||||
|
./scripts/cir/fetch_cir_sequence_from_remote.sh \
|
||||||
|
--ssh-target <user@host> \
|
||||||
|
--remote-path <path> \
|
||||||
|
--local-path <path>
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
SSH_TARGET=""
|
||||||
|
REMOTE_PATH=""
|
||||||
|
LOCAL_PATH=""
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--ssh-target) SSH_TARGET="$2"; shift 2 ;;
|
||||||
|
--remote-path) REMOTE_PATH="$2"; shift 2 ;;
|
||||||
|
--local-path) LOCAL_PATH="$2"; shift 2 ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
[[ -n "$SSH_TARGET" && -n "$REMOTE_PATH" && -n "$LOCAL_PATH" ]] || { usage >&2; exit 2; }
|
||||||
|
|
||||||
|
mkdir -p "$(dirname "$LOCAL_PATH")"
|
||||||
|
rsync -a "$SSH_TARGET:$REMOTE_PATH/" "$LOCAL_PATH/"
|
||||||
|
echo "done: $LOCAL_PATH"
|
||||||
50
scripts/cir/json_to_vaps_csv.py
Executable file
50
scripts/cir/json_to_vaps_csv.py
Executable file
@@ -0,0 +1,50 @@
|
|||||||
|
#!/usr/bin/env python3
|
||||||
|
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import argparse
|
||||||
|
import csv
|
||||||
|
import json
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_asn(value: str | int) -> str:
|
||||||
|
text = str(value).strip().upper()
|
||||||
|
if text.startswith("AS"):
|
||||||
|
text = text[2:]
|
||||||
|
return f"AS{int(text)}"
|
||||||
|
|
||||||
|
|
||||||
|
def main() -> int:
|
||||||
|
parser = argparse.ArgumentParser()
|
||||||
|
parser.add_argument("--input", required=True, type=Path)
|
||||||
|
parser.add_argument("--csv-out", required=True, type=Path)
|
||||||
|
args = parser.parse_args()
|
||||||
|
|
||||||
|
obj = json.loads(args.input.read_text(encoding="utf-8"))
|
||||||
|
rows: list[tuple[str, str, str]] = []
|
||||||
|
for aspa in obj.get("aspas", []):
|
||||||
|
providers = sorted(
|
||||||
|
{normalize_asn(item) for item in aspa.get("providers", [])},
|
||||||
|
key=lambda s: int(s[2:]),
|
||||||
|
)
|
||||||
|
rows.append(
|
||||||
|
(
|
||||||
|
normalize_asn(aspa["customer"]),
|
||||||
|
";".join(providers),
|
||||||
|
str(aspa.get("ta", "")).strip().lower(),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
rows.sort(key=lambda row: (int(row[0][2:]), row[1], row[2]))
|
||||||
|
|
||||||
|
args.csv_out.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with args.csv_out.open("w", encoding="utf-8", newline="") as fh:
|
||||||
|
writer = csv.writer(fh)
|
||||||
|
writer.writerow(["Customer ASN", "Providers", "Trust Anchor"])
|
||||||
|
writer.writerows(rows)
|
||||||
|
print(args.csv_out)
|
||||||
|
return 0
|
||||||
|
|
||||||
|
|
||||||
|
if __name__ == "__main__":
|
||||||
|
raise SystemExit(main())
|
||||||
84
scripts/cir/run_cir_drop_sequence.sh
Executable file
84
scripts/cir/run_cir_drop_sequence.sh
Executable file
@@ -0,0 +1,84 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage:
|
||||||
|
./scripts/cir/run_cir_drop_sequence.sh \
|
||||||
|
--sequence-root <path> \
|
||||||
|
[--drop-bin <path>]
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
SEQUENCE_ROOT=""
|
||||||
|
DROP_BIN="${DROP_BIN:-$ROOT_DIR/target/release/cir_drop_report}"
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--sequence-root) SEQUENCE_ROOT="$2"; shift 2 ;;
|
||||||
|
--drop-bin) DROP_BIN="$2"; shift 2 ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
[[ -n "$SEQUENCE_ROOT" ]] || { usage >&2; exit 2; }
|
||||||
|
|
||||||
|
python3 - <<'PY' "$SEQUENCE_ROOT" "$DROP_BIN"
|
||||||
|
import json
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
sequence_root = Path(sys.argv[1]).resolve()
|
||||||
|
drop_bin = sys.argv[2]
|
||||||
|
sequence = json.loads((sequence_root / "sequence.json").read_text(encoding="utf-8"))
|
||||||
|
static_root = sequence_root / sequence["staticRoot"] if "staticRoot" in sequence else None
|
||||||
|
raw_store_db = sequence_root / sequence["rawStoreDbPath"] if "rawStoreDbPath" in sequence else None
|
||||||
|
backend_count = sum(x is not None for x in (static_root, raw_store_db))
|
||||||
|
if backend_count != 1:
|
||||||
|
raise SystemExit("sequence must set exactly one of staticRoot or rawStoreDbPath")
|
||||||
|
|
||||||
|
summaries = []
|
||||||
|
for step in sequence["steps"]:
|
||||||
|
step_id = step["stepId"]
|
||||||
|
out_dir = sequence_root / "drop" / step_id
|
||||||
|
out_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
cmd = [
|
||||||
|
drop_bin,
|
||||||
|
"--cir",
|
||||||
|
str(sequence_root / step["cirPath"]),
|
||||||
|
"--ccr",
|
||||||
|
str(sequence_root / step["ccrPath"]),
|
||||||
|
"--report-json",
|
||||||
|
str(sequence_root / step["reportPath"]),
|
||||||
|
"--json-out",
|
||||||
|
str(out_dir / "drop.json"),
|
||||||
|
"--md-out",
|
||||||
|
str(out_dir / "drop.md"),
|
||||||
|
]
|
||||||
|
if static_root is not None:
|
||||||
|
cmd.extend(["--static-root", str(static_root)])
|
||||||
|
else:
|
||||||
|
cmd.extend(["--raw-store-db", str(raw_store_db)])
|
||||||
|
proc = subprocess.run(cmd, capture_output=True, text=True)
|
||||||
|
if proc.returncode != 0:
|
||||||
|
raise SystemExit(
|
||||||
|
f"drop report failed for {step_id}: stdout={proc.stdout} stderr={proc.stderr}"
|
||||||
|
)
|
||||||
|
result = json.loads((out_dir / "drop.json").read_text(encoding="utf-8"))
|
||||||
|
summaries.append(
|
||||||
|
{
|
||||||
|
"stepId": step_id,
|
||||||
|
"droppedVrpCount": result["summary"]["droppedVrpCount"],
|
||||||
|
"droppedObjectCount": result["summary"]["droppedObjectCount"],
|
||||||
|
"reportPath": str(out_dir / "drop.json"),
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
summary = {"version": 1, "steps": summaries}
|
||||||
|
(sequence_root / "drop-summary.json").write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
||||||
|
PY
|
||||||
|
|
||||||
|
echo "done: $SEQUENCE_ROOT"
|
||||||
173
scripts/cir/run_cir_record_full_delta.sh
Executable file
173
scripts/cir/run_cir_record_full_delta.sh
Executable file
@@ -0,0 +1,173 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage:
|
||||||
|
./scripts/cir/run_cir_record_full_delta.sh \
|
||||||
|
--out-dir <path> \
|
||||||
|
--tal-path <path> \
|
||||||
|
--ta-path <path> \
|
||||||
|
--cir-tal-uri <url> \
|
||||||
|
--payload-replay-archive <path> \
|
||||||
|
--payload-replay-locks <path> \
|
||||||
|
--payload-base-archive <path> \
|
||||||
|
--payload-base-locks <path> \
|
||||||
|
--payload-delta-archive <path> \
|
||||||
|
--payload-delta-locks <path> \
|
||||||
|
[--base-validation-time <rfc3339>] \
|
||||||
|
[--delta-validation-time <rfc3339>] \
|
||||||
|
[--max-depth <n>] \
|
||||||
|
[--max-instances <n>] \
|
||||||
|
[--rpki-bin <path>]
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
|
||||||
|
OUT_DIR=""
|
||||||
|
TAL_PATH=""
|
||||||
|
TA_PATH=""
|
||||||
|
CIR_TAL_URI=""
|
||||||
|
PAYLOAD_REPLAY_ARCHIVE=""
|
||||||
|
PAYLOAD_REPLAY_LOCKS=""
|
||||||
|
PAYLOAD_BASE_ARCHIVE=""
|
||||||
|
PAYLOAD_BASE_LOCKS=""
|
||||||
|
PAYLOAD_DELTA_ARCHIVE=""
|
||||||
|
PAYLOAD_DELTA_LOCKS=""
|
||||||
|
BASE_VALIDATION_TIME=""
|
||||||
|
DELTA_VALIDATION_TIME=""
|
||||||
|
MAX_DEPTH=0
|
||||||
|
MAX_INSTANCES=1
|
||||||
|
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--out-dir) OUT_DIR="$2"; shift 2 ;;
|
||||||
|
--tal-path) TAL_PATH="$2"; shift 2 ;;
|
||||||
|
--ta-path) TA_PATH="$2"; shift 2 ;;
|
||||||
|
--cir-tal-uri) CIR_TAL_URI="$2"; shift 2 ;;
|
||||||
|
--payload-replay-archive) PAYLOAD_REPLAY_ARCHIVE="$2"; shift 2 ;;
|
||||||
|
--payload-replay-locks) PAYLOAD_REPLAY_LOCKS="$2"; shift 2 ;;
|
||||||
|
--payload-base-archive) PAYLOAD_BASE_ARCHIVE="$2"; shift 2 ;;
|
||||||
|
--payload-base-locks) PAYLOAD_BASE_LOCKS="$2"; shift 2 ;;
|
||||||
|
--payload-delta-archive) PAYLOAD_DELTA_ARCHIVE="$2"; shift 2 ;;
|
||||||
|
--payload-delta-locks) PAYLOAD_DELTA_LOCKS="$2"; shift 2 ;;
|
||||||
|
--base-validation-time) BASE_VALIDATION_TIME="$2"; shift 2 ;;
|
||||||
|
--delta-validation-time) DELTA_VALIDATION_TIME="$2"; shift 2 ;;
|
||||||
|
--max-depth) MAX_DEPTH="$2"; shift 2 ;;
|
||||||
|
--max-instances) MAX_INSTANCES="$2"; shift 2 ;;
|
||||||
|
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
[[ -n "$OUT_DIR" && -n "$TAL_PATH" && -n "$TA_PATH" && -n "$CIR_TAL_URI" && -n "$PAYLOAD_REPLAY_ARCHIVE" && -n "$PAYLOAD_REPLAY_LOCKS" && -n "$PAYLOAD_BASE_ARCHIVE" && -n "$PAYLOAD_BASE_LOCKS" && -n "$PAYLOAD_DELTA_ARCHIVE" && -n "$PAYLOAD_DELTA_LOCKS" ]] || {
|
||||||
|
usage >&2
|
||||||
|
exit 2
|
||||||
|
}
|
||||||
|
|
||||||
|
if [[ ! -x "$RPKI_BIN" ]]; then
|
||||||
|
(
|
||||||
|
cd "$ROOT_DIR"
|
||||||
|
cargo build --release --bin rpki
|
||||||
|
)
|
||||||
|
fi
|
||||||
|
|
||||||
|
resolve_validation_time() {
|
||||||
|
local path="$1"
|
||||||
|
python3 - <<'PY' "$path"
|
||||||
|
import json, sys
|
||||||
|
print(json.load(open(sys.argv[1], 'r', encoding='utf-8'))['validationTime'])
|
||||||
|
PY
|
||||||
|
}
|
||||||
|
|
||||||
|
if [[ -z "$BASE_VALIDATION_TIME" ]]; then
|
||||||
|
BASE_VALIDATION_TIME="$(resolve_validation_time "$PAYLOAD_REPLAY_LOCKS")"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ -z "$DELTA_VALIDATION_TIME" ]]; then
|
||||||
|
DELTA_VALIDATION_TIME="$(resolve_validation_time "$PAYLOAD_DELTA_LOCKS")"
|
||||||
|
fi
|
||||||
|
|
||||||
|
rm -rf "$OUT_DIR"
|
||||||
|
mkdir -p "$OUT_DIR/full" "$OUT_DIR/delta-001" "$OUT_DIR/static"
|
||||||
|
|
||||||
|
FULL_DB="$OUT_DIR/full/db"
|
||||||
|
DELTA_DB="$OUT_DIR/delta-001/db"
|
||||||
|
|
||||||
|
"$RPKI_BIN" \
|
||||||
|
--db "$FULL_DB" \
|
||||||
|
--tal-path "$TAL_PATH" \
|
||||||
|
--ta-path "$TA_PATH" \
|
||||||
|
--payload-replay-archive "$PAYLOAD_REPLAY_ARCHIVE" \
|
||||||
|
--payload-replay-locks "$PAYLOAD_REPLAY_LOCKS" \
|
||||||
|
--validation-time "$BASE_VALIDATION_TIME" \
|
||||||
|
--max-depth "$MAX_DEPTH" \
|
||||||
|
--max-instances "$MAX_INSTANCES" \
|
||||||
|
--ccr-out "$OUT_DIR/full/result.ccr" \
|
||||||
|
--report-json "$OUT_DIR/full/report.json" \
|
||||||
|
--cir-enable \
|
||||||
|
--cir-out "$OUT_DIR/full/input.cir" \
|
||||||
|
--cir-static-root "$OUT_DIR/static" \
|
||||||
|
--cir-tal-uri "$CIR_TAL_URI" \
|
||||||
|
>"$OUT_DIR/full/run.stdout.log" 2>"$OUT_DIR/full/run.stderr.log"
|
||||||
|
|
||||||
|
"$RPKI_BIN" \
|
||||||
|
--db "$DELTA_DB" \
|
||||||
|
--tal-path "$TAL_PATH" \
|
||||||
|
--ta-path "$TA_PATH" \
|
||||||
|
--payload-base-archive "$PAYLOAD_BASE_ARCHIVE" \
|
||||||
|
--payload-base-locks "$PAYLOAD_BASE_LOCKS" \
|
||||||
|
--payload-delta-archive "$PAYLOAD_DELTA_ARCHIVE" \
|
||||||
|
--payload-delta-locks "$PAYLOAD_DELTA_LOCKS" \
|
||||||
|
--payload-base-validation-time "$BASE_VALIDATION_TIME" \
|
||||||
|
--validation-time "$DELTA_VALIDATION_TIME" \
|
||||||
|
--max-depth "$MAX_DEPTH" \
|
||||||
|
--max-instances "$MAX_INSTANCES" \
|
||||||
|
--ccr-out "$OUT_DIR/delta-001/result.ccr" \
|
||||||
|
--report-json "$OUT_DIR/delta-001/report.json" \
|
||||||
|
--cir-enable \
|
||||||
|
--cir-out "$OUT_DIR/delta-001/input.cir" \
|
||||||
|
--cir-static-root "$OUT_DIR/static" \
|
||||||
|
--cir-tal-uri "$CIR_TAL_URI" \
|
||||||
|
>"$OUT_DIR/delta-001/run.stdout.log" 2>"$OUT_DIR/delta-001/run.stderr.log"
|
||||||
|
|
||||||
|
python3 - <<'PY' "$OUT_DIR" "$BASE_VALIDATION_TIME" "$DELTA_VALIDATION_TIME"
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
out = Path(sys.argv[1])
|
||||||
|
base_validation_time = sys.argv[2]
|
||||||
|
delta_validation_time = sys.argv[3]
|
||||||
|
static_files = sum(1 for _ in (out / "static").rglob("*") if _.is_file())
|
||||||
|
summary = {
|
||||||
|
"version": 1,
|
||||||
|
"kind": "cir_pair",
|
||||||
|
"baseValidationTime": base_validation_time,
|
||||||
|
"deltaValidationTime": delta_validation_time,
|
||||||
|
"staticRoot": "static",
|
||||||
|
"steps": [
|
||||||
|
{
|
||||||
|
"kind": "full",
|
||||||
|
"cirPath": "full/input.cir",
|
||||||
|
"ccrPath": "full/result.ccr",
|
||||||
|
"reportPath": "full/report.json",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"kind": "delta",
|
||||||
|
"cirPath": "delta-001/input.cir",
|
||||||
|
"ccrPath": "delta-001/result.ccr",
|
||||||
|
"reportPath": "delta-001/report.json",
|
||||||
|
"previous": "full",
|
||||||
|
},
|
||||||
|
],
|
||||||
|
"staticFileCount": static_files,
|
||||||
|
}
|
||||||
|
(out / "summary.json").write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
||||||
|
PY
|
||||||
|
|
||||||
|
echo "done: $OUT_DIR"
|
||||||
129
scripts/cir/run_cir_record_sequence_multi_rir_offline.sh
Executable file
129
scripts/cir/run_cir_record_sequence_multi_rir_offline.sh
Executable file
@@ -0,0 +1,129 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage:
|
||||||
|
./scripts/cir/run_cir_record_sequence_multi_rir_offline.sh \
|
||||||
|
[--bundle-root <path>] \
|
||||||
|
[--rir <afrinic,apnic,arin,lacnic,ripe>] \
|
||||||
|
[--delta-count <n>] \
|
||||||
|
[--full-repo] \
|
||||||
|
[--out-root <path>] \
|
||||||
|
[--rpki-bin <path>]
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
CASE_INFO="$ROOT_DIR/scripts/payload_replay/multi_rir_case_info.py"
|
||||||
|
SINGLE_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_record_sequence_offline.sh"
|
||||||
|
|
||||||
|
BUNDLE_ROOT="/home/yuyr/dev/rust_playground/routinator/bench/multi_rir_demo/runs/20260316-112341-multi-final3"
|
||||||
|
RIRS="afrinic,apnic,arin,lacnic,ripe"
|
||||||
|
DELTA_COUNT=2
|
||||||
|
FULL_REPO=0
|
||||||
|
OUT_ROOT="$ROOT_DIR/target/replay/cir_sequence_multi_rir_offline_$(date -u +%Y%m%dT%H%M%SZ)"
|
||||||
|
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--bundle-root) BUNDLE_ROOT="$2"; shift 2 ;;
|
||||||
|
--rir) RIRS="$2"; shift 2 ;;
|
||||||
|
--delta-count) DELTA_COUNT="$2"; shift 2 ;;
|
||||||
|
--full-repo) FULL_REPO=1; shift 1 ;;
|
||||||
|
--out-root) OUT_ROOT="$2"; shift 2 ;;
|
||||||
|
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
mkdir -p "$OUT_ROOT"
|
||||||
|
SUMMARY_JSON="$OUT_ROOT/summary.json"
|
||||||
|
SUMMARY_MD="$OUT_ROOT/summary.md"
|
||||||
|
|
||||||
|
IFS=',' read -r -a RIR_ITEMS <<< "$RIRS"
|
||||||
|
|
||||||
|
for rir in "${RIR_ITEMS[@]}"; do
|
||||||
|
CASE_JSON="$(python3 "$CASE_INFO" --bundle-root "$BUNDLE_ROOT" --repo-root "$ROOT_DIR" --rir "$rir")"
|
||||||
|
TAL_PATH="$(python3 - <<'PY' "$CASE_JSON"
|
||||||
|
import json,sys
|
||||||
|
print(json.loads(sys.argv[1])['tal_path'])
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
TA_PATH="$(python3 - <<'PY' "$CASE_JSON"
|
||||||
|
import json,sys
|
||||||
|
print(json.loads(sys.argv[1])['ta_path'])
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
BASE_ARCHIVE="$(python3 - <<'PY' "$CASE_JSON"
|
||||||
|
import json,sys
|
||||||
|
print(json.loads(sys.argv[1])['base_archive'])
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
BASE_LOCKS="$(python3 - <<'PY' "$CASE_JSON"
|
||||||
|
import json,sys
|
||||||
|
print(json.loads(sys.argv[1])['base_locks'])
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
DELTA_ARCHIVE="$(python3 - <<'PY' "$CASE_JSON"
|
||||||
|
import json,sys
|
||||||
|
print(json.loads(sys.argv[1])['delta_archive'])
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
DELTA_LOCKS="$(python3 - <<'PY' "$CASE_JSON"
|
||||||
|
import json,sys
|
||||||
|
print(json.loads(sys.argv[1])['delta_locks'])
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
OUT_DIR="$OUT_ROOT/$rir"
|
||||||
|
args=(
|
||||||
|
"$SINGLE_SCRIPT"
|
||||||
|
--out-dir "$OUT_DIR" \
|
||||||
|
--tal-path "$TAL_PATH" \
|
||||||
|
--ta-path "$TA_PATH" \
|
||||||
|
--cir-tal-uri "https://example.test/$rir.tal" \
|
||||||
|
--payload-replay-archive "$BASE_ARCHIVE" \
|
||||||
|
--payload-replay-locks "$BASE_LOCKS" \
|
||||||
|
--payload-base-archive "$BASE_ARCHIVE" \
|
||||||
|
--payload-base-locks "$BASE_LOCKS" \
|
||||||
|
--payload-delta-archive "$DELTA_ARCHIVE" \
|
||||||
|
--payload-delta-locks "$DELTA_LOCKS" \
|
||||||
|
--delta-count "$DELTA_COUNT" \
|
||||||
|
--rpki-bin "$RPKI_BIN"
|
||||||
|
)
|
||||||
|
if [[ "$FULL_REPO" -ne 1 ]]; then
|
||||||
|
args+=(--max-depth 0 --max-instances 1)
|
||||||
|
else
|
||||||
|
args+=(--full-repo)
|
||||||
|
fi
|
||||||
|
"${args[@]}"
|
||||||
|
done
|
||||||
|
|
||||||
|
python3 - <<'PY' "$OUT_ROOT" "$RIRS" "$SUMMARY_JSON" "$SUMMARY_MD"
|
||||||
|
import json, sys
|
||||||
|
from pathlib import Path
|
||||||
|
out_root = Path(sys.argv[1])
|
||||||
|
rirs = [item for item in sys.argv[2].split(',') if item]
|
||||||
|
summary_json = Path(sys.argv[3])
|
||||||
|
summary_md = Path(sys.argv[4])
|
||||||
|
items = []
|
||||||
|
for rir in rirs:
|
||||||
|
root = out_root / rir
|
||||||
|
seq = json.loads((root / "sequence.json").read_text(encoding="utf-8"))
|
||||||
|
summ = json.loads((root / "summary.json").read_text(encoding="utf-8"))
|
||||||
|
items.append({
|
||||||
|
"rir": rir,
|
||||||
|
"root": str(root),
|
||||||
|
"stepCount": len(seq["steps"]),
|
||||||
|
"staticFileCount": summ["staticFileCount"],
|
||||||
|
})
|
||||||
|
summary = {"version": 1, "rirs": items}
|
||||||
|
summary_json.write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
||||||
|
lines = ["# Multi-RIR Offline CIR Sequence Summary", ""]
|
||||||
|
for item in items:
|
||||||
|
lines.append(f"- `{item['rir']}`: `stepCount={item['stepCount']}` `staticFileCount={item['staticFileCount']}` `root={item['root']}`")
|
||||||
|
summary_md.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||||
|
PY
|
||||||
|
|
||||||
|
echo "done: $OUT_ROOT"
|
||||||
206
scripts/cir/run_cir_record_sequence_offline.sh
Executable file
206
scripts/cir/run_cir_record_sequence_offline.sh
Executable file
@@ -0,0 +1,206 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage:
|
||||||
|
./scripts/cir/run_cir_record_sequence_offline.sh \
|
||||||
|
--out-dir <path> \
|
||||||
|
--tal-path <path> \
|
||||||
|
--ta-path <path> \
|
||||||
|
--cir-tal-uri <url> \
|
||||||
|
--payload-replay-archive <path> \
|
||||||
|
--payload-replay-locks <path> \
|
||||||
|
--payload-base-archive <path> \
|
||||||
|
--payload-base-locks <path> \
|
||||||
|
--payload-delta-archive <path> \
|
||||||
|
--payload-delta-locks <path> \
|
||||||
|
[--delta-count <n>] \
|
||||||
|
[--base-validation-time <rfc3339>] \
|
||||||
|
[--delta-validation-time <rfc3339>] \
|
||||||
|
[--full-repo] \
|
||||||
|
[--max-depth <n>] \
|
||||||
|
[--max-instances <n>] \
|
||||||
|
[--rpki-bin <path>]
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
|
||||||
|
OUT_DIR=""
|
||||||
|
TAL_PATH=""
|
||||||
|
TA_PATH=""
|
||||||
|
CIR_TAL_URI=""
|
||||||
|
PAYLOAD_REPLAY_ARCHIVE=""
|
||||||
|
PAYLOAD_REPLAY_LOCKS=""
|
||||||
|
PAYLOAD_BASE_ARCHIVE=""
|
||||||
|
PAYLOAD_BASE_LOCKS=""
|
||||||
|
PAYLOAD_DELTA_ARCHIVE=""
|
||||||
|
PAYLOAD_DELTA_LOCKS=""
|
||||||
|
BASE_VALIDATION_TIME=""
|
||||||
|
DELTA_VALIDATION_TIME=""
|
||||||
|
DELTA_COUNT=2
|
||||||
|
FULL_REPO=0
|
||||||
|
MAX_DEPTH=0
|
||||||
|
MAX_INSTANCES=1
|
||||||
|
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--out-dir) OUT_DIR="$2"; shift 2 ;;
|
||||||
|
--tal-path) TAL_PATH="$2"; shift 2 ;;
|
||||||
|
--ta-path) TA_PATH="$2"; shift 2 ;;
|
||||||
|
--cir-tal-uri) CIR_TAL_URI="$2"; shift 2 ;;
|
||||||
|
--payload-replay-archive) PAYLOAD_REPLAY_ARCHIVE="$2"; shift 2 ;;
|
||||||
|
--payload-replay-locks) PAYLOAD_REPLAY_LOCKS="$2"; shift 2 ;;
|
||||||
|
--payload-base-archive) PAYLOAD_BASE_ARCHIVE="$2"; shift 2 ;;
|
||||||
|
--payload-base-locks) PAYLOAD_BASE_LOCKS="$2"; shift 2 ;;
|
||||||
|
--payload-delta-archive) PAYLOAD_DELTA_ARCHIVE="$2"; shift 2 ;;
|
||||||
|
--payload-delta-locks) PAYLOAD_DELTA_LOCKS="$2"; shift 2 ;;
|
||||||
|
--base-validation-time) BASE_VALIDATION_TIME="$2"; shift 2 ;;
|
||||||
|
--delta-validation-time) DELTA_VALIDATION_TIME="$2"; shift 2 ;;
|
||||||
|
--delta-count) DELTA_COUNT="$2"; shift 2 ;;
|
||||||
|
--full-repo) FULL_REPO=1; shift 1 ;;
|
||||||
|
--max-depth) MAX_DEPTH="$2"; shift 2 ;;
|
||||||
|
--max-instances) MAX_INSTANCES="$2"; shift 2 ;;
|
||||||
|
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
[[ -n "$OUT_DIR" && -n "$TAL_PATH" && -n "$TA_PATH" && -n "$CIR_TAL_URI" && -n "$PAYLOAD_REPLAY_ARCHIVE" && -n "$PAYLOAD_REPLAY_LOCKS" && -n "$PAYLOAD_BASE_ARCHIVE" && -n "$PAYLOAD_BASE_LOCKS" && -n "$PAYLOAD_DELTA_ARCHIVE" && -n "$PAYLOAD_DELTA_LOCKS" ]] || {
|
||||||
|
usage >&2
|
||||||
|
exit 2
|
||||||
|
}
|
||||||
|
|
||||||
|
if [[ ! -x "$RPKI_BIN" ]]; then
|
||||||
|
(
|
||||||
|
cd "$ROOT_DIR"
|
||||||
|
cargo build --release --bin rpki
|
||||||
|
)
|
||||||
|
fi
|
||||||
|
|
||||||
|
resolve_validation_time() {
|
||||||
|
local path="$1"
|
||||||
|
python3 - <<'PY' "$path"
|
||||||
|
import json, sys
|
||||||
|
print(json.load(open(sys.argv[1], 'r', encoding='utf-8'))['validationTime'])
|
||||||
|
PY
|
||||||
|
}
|
||||||
|
|
||||||
|
if [[ -z "$BASE_VALIDATION_TIME" ]]; then
|
||||||
|
BASE_VALIDATION_TIME="$(resolve_validation_time "$PAYLOAD_REPLAY_LOCKS")"
|
||||||
|
fi
|
||||||
|
if [[ -z "$DELTA_VALIDATION_TIME" ]]; then
|
||||||
|
DELTA_VALIDATION_TIME="$(resolve_validation_time "$PAYLOAD_DELTA_LOCKS")"
|
||||||
|
fi
|
||||||
|
|
||||||
|
rm -rf "$OUT_DIR"
|
||||||
|
mkdir -p "$OUT_DIR/static" "$OUT_DIR/full"
|
||||||
|
|
||||||
|
run_step() {
|
||||||
|
local kind="$1"
|
||||||
|
local step_dir="$2"
|
||||||
|
local db_dir="$3"
|
||||||
|
shift 3
|
||||||
|
mkdir -p "$step_dir"
|
||||||
|
local -a cmd=(
|
||||||
|
"$RPKI_BIN"
|
||||||
|
--db "$db_dir" \
|
||||||
|
--tal-path "$TAL_PATH" \
|
||||||
|
--ta-path "$TA_PATH" \
|
||||||
|
--ccr-out "$step_dir/result.ccr" \
|
||||||
|
--report-json "$step_dir/report.json" \
|
||||||
|
--cir-enable \
|
||||||
|
--cir-out "$step_dir/input.cir" \
|
||||||
|
--cir-static-root "$OUT_DIR/static" \
|
||||||
|
--cir-tal-uri "$CIR_TAL_URI"
|
||||||
|
)
|
||||||
|
if [[ "$FULL_REPO" -ne 1 ]]; then
|
||||||
|
cmd+=(--max-depth "$MAX_DEPTH" --max-instances "$MAX_INSTANCES")
|
||||||
|
fi
|
||||||
|
cmd+=("$@")
|
||||||
|
"${cmd[@]}" >"$step_dir/run.stdout.log" 2>"$step_dir/run.stderr.log"
|
||||||
|
}
|
||||||
|
|
||||||
|
run_step \
|
||||||
|
full \
|
||||||
|
"$OUT_DIR/full" \
|
||||||
|
"$OUT_DIR/full/db" \
|
||||||
|
--payload-replay-archive "$PAYLOAD_REPLAY_ARCHIVE" \
|
||||||
|
--payload-replay-locks "$PAYLOAD_REPLAY_LOCKS" \
|
||||||
|
--validation-time "$BASE_VALIDATION_TIME"
|
||||||
|
|
||||||
|
for idx in $(seq 1 "$DELTA_COUNT"); do
|
||||||
|
step_id="$(printf 'delta-%03d' "$idx")"
|
||||||
|
run_step \
|
||||||
|
delta \
|
||||||
|
"$OUT_DIR/$step_id" \
|
||||||
|
"$OUT_DIR/$step_id/db" \
|
||||||
|
--payload-base-archive "$PAYLOAD_BASE_ARCHIVE" \
|
||||||
|
--payload-base-locks "$PAYLOAD_BASE_LOCKS" \
|
||||||
|
--payload-delta-archive "$PAYLOAD_DELTA_ARCHIVE" \
|
||||||
|
--payload-delta-locks "$PAYLOAD_DELTA_LOCKS" \
|
||||||
|
--payload-base-validation-time "$BASE_VALIDATION_TIME" \
|
||||||
|
--validation-time "$DELTA_VALIDATION_TIME"
|
||||||
|
done
|
||||||
|
|
||||||
|
python3 - <<'PY' "$OUT_DIR" "$BASE_VALIDATION_TIME" "$DELTA_VALIDATION_TIME" "$DELTA_COUNT"
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
out = Path(sys.argv[1])
|
||||||
|
base_validation_time = sys.argv[2]
|
||||||
|
delta_validation_time = sys.argv[3]
|
||||||
|
delta_count = int(sys.argv[4])
|
||||||
|
|
||||||
|
steps = [
|
||||||
|
{
|
||||||
|
"stepId": "full",
|
||||||
|
"kind": "full",
|
||||||
|
"validationTime": base_validation_time,
|
||||||
|
"cirPath": "full/input.cir",
|
||||||
|
"ccrPath": "full/result.ccr",
|
||||||
|
"reportPath": "full/report.json",
|
||||||
|
"previousStepId": None,
|
||||||
|
}
|
||||||
|
]
|
||||||
|
previous = "full"
|
||||||
|
for idx in range(1, delta_count + 1):
|
||||||
|
step_id = f"delta-{idx:03d}"
|
||||||
|
steps.append(
|
||||||
|
{
|
||||||
|
"stepId": step_id,
|
||||||
|
"kind": "delta",
|
||||||
|
"validationTime": delta_validation_time,
|
||||||
|
"cirPath": f"{step_id}/input.cir",
|
||||||
|
"ccrPath": f"{step_id}/result.ccr",
|
||||||
|
"reportPath": f"{step_id}/report.json",
|
||||||
|
"previousStepId": previous,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
previous = step_id
|
||||||
|
|
||||||
|
summary = {
|
||||||
|
"version": 1,
|
||||||
|
"kind": "cir_sequence_offline",
|
||||||
|
"staticRoot": "static",
|
||||||
|
"steps": steps,
|
||||||
|
}
|
||||||
|
(out / "sequence.json").write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
||||||
|
(out / "summary.json").write_text(
|
||||||
|
json.dumps(
|
||||||
|
{
|
||||||
|
"version": 1,
|
||||||
|
"stepCount": len(steps),
|
||||||
|
"staticFileCount": sum(1 for p in (out / "static").rglob("*") if p.is_file()),
|
||||||
|
},
|
||||||
|
indent=2,
|
||||||
|
),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
PY
|
||||||
|
|
||||||
|
echo "done: $OUT_DIR"
|
||||||
244
scripts/cir/run_cir_record_sequence_remote.sh
Executable file
244
scripts/cir/run_cir_record_sequence_remote.sh
Executable file
@@ -0,0 +1,244 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage:
|
||||||
|
./scripts/cir/run_cir_record_sequence_remote.sh \
|
||||||
|
--rir <name> \
|
||||||
|
--remote-root <path> \
|
||||||
|
[--ssh-target <user@host>] \
|
||||||
|
[--out-subdir <path>] \
|
||||||
|
[--delta-count <n>] \
|
||||||
|
[--sleep-secs <n>] \
|
||||||
|
[--full-repo] \
|
||||||
|
[--max-depth <n>] \
|
||||||
|
[--max-instances <n>]
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
SSH_TARGET="${SSH_TARGET:-root@47.77.183.68}"
|
||||||
|
RIR=""
|
||||||
|
REMOTE_ROOT=""
|
||||||
|
OUT_SUBDIR=""
|
||||||
|
DELTA_COUNT=2
|
||||||
|
SLEEP_SECS=30
|
||||||
|
FULL_REPO=0
|
||||||
|
MAX_DEPTH=0
|
||||||
|
MAX_INSTANCES=1
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--rir) RIR="$2"; shift 2 ;;
|
||||||
|
--remote-root) REMOTE_ROOT="$2"; shift 2 ;;
|
||||||
|
--ssh-target) SSH_TARGET="$2"; shift 2 ;;
|
||||||
|
--out-subdir) OUT_SUBDIR="$2"; shift 2 ;;
|
||||||
|
--delta-count) DELTA_COUNT="$2"; shift 2 ;;
|
||||||
|
--sleep-secs) SLEEP_SECS="$2"; shift 2 ;;
|
||||||
|
--full-repo) FULL_REPO=1; shift 1 ;;
|
||||||
|
--max-depth) MAX_DEPTH="$2"; shift 2 ;;
|
||||||
|
--max-instances) MAX_INSTANCES="$2"; shift 2 ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
[[ -n "$RIR" && -n "$REMOTE_ROOT" ]] || { usage >&2; exit 2; }
|
||||||
|
|
||||||
|
case "$RIR" in
|
||||||
|
afrinic) TAL_REL="tests/fixtures/tal/afrinic.tal"; TA_REL="tests/fixtures/ta/afrinic-ta.cer" ;;
|
||||||
|
apnic) TAL_REL="tests/fixtures/tal/apnic-rfc7730-https.tal"; TA_REL="tests/fixtures/ta/apnic-ta.cer" ;;
|
||||||
|
arin) TAL_REL="tests/fixtures/tal/arin.tal"; TA_REL="tests/fixtures/ta/arin-ta.cer" ;;
|
||||||
|
lacnic) TAL_REL="tests/fixtures/tal/lacnic.tal"; TA_REL="tests/fixtures/ta/lacnic-ta.cer" ;;
|
||||||
|
ripe) TAL_REL="tests/fixtures/tal/ripe-ncc.tal"; TA_REL="tests/fixtures/ta/ripe-ncc-ta.cer" ;;
|
||||||
|
*) echo "unsupported rir: $RIR" >&2; exit 2 ;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
rsync -a --delete \
|
||||||
|
--exclude target \
|
||||||
|
--exclude .git \
|
||||||
|
"$ROOT_DIR/" "$SSH_TARGET:$REMOTE_ROOT/"
|
||||||
|
ssh "$SSH_TARGET" "mkdir -p '$REMOTE_ROOT/target/release'"
|
||||||
|
rsync -a "$ROOT_DIR/target/release/rpki" "$SSH_TARGET:$REMOTE_ROOT/target/release/"
|
||||||
|
|
||||||
|
ssh "$SSH_TARGET" \
|
||||||
|
RIR="$RIR" \
|
||||||
|
REMOTE_ROOT="$REMOTE_ROOT" \
|
||||||
|
OUT_SUBDIR="$OUT_SUBDIR" \
|
||||||
|
DELTA_COUNT="$DELTA_COUNT" \
|
||||||
|
SLEEP_SECS="$SLEEP_SECS" \
|
||||||
|
FULL_REPO="$FULL_REPO" \
|
||||||
|
MAX_DEPTH="$MAX_DEPTH" \
|
||||||
|
MAX_INSTANCES="$MAX_INSTANCES" \
|
||||||
|
TAL_REL="$TAL_REL" \
|
||||||
|
TA_REL="$TA_REL" \
|
||||||
|
'bash -s' <<'EOS'
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
cd "$REMOTE_ROOT"
|
||||||
|
|
||||||
|
if [[ -n "${OUT_SUBDIR}" ]]; then
|
||||||
|
OUT="${OUT_SUBDIR}"
|
||||||
|
else
|
||||||
|
OUT="target/replay/cir_sequence_remote_${RIR}_$(date -u +%Y%m%dT%H%M%SZ)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
mkdir -p "$OUT"
|
||||||
|
DB="$OUT/work-db"
|
||||||
|
RAW_STORE_DB="$OUT/raw-store.db"
|
||||||
|
ROWS="$OUT/.sequence_rows.tsv"
|
||||||
|
: > "$ROWS"
|
||||||
|
|
||||||
|
write_step_timing() {
|
||||||
|
local path="$1"
|
||||||
|
local start_ms="$2"
|
||||||
|
local end_ms="$3"
|
||||||
|
local started_at="$4"
|
||||||
|
local finished_at="$5"
|
||||||
|
python3 - <<'PY' "$path" "$start_ms" "$end_ms" "$started_at" "$finished_at"
|
||||||
|
import json, sys
|
||||||
|
path, start_ms, end_ms, started_at, finished_at = sys.argv[1:]
|
||||||
|
start_ms = int(start_ms)
|
||||||
|
end_ms = int(end_ms)
|
||||||
|
with open(path, "w", encoding="utf-8") as fh:
|
||||||
|
json.dump(
|
||||||
|
{
|
||||||
|
"durationMs": end_ms - start_ms,
|
||||||
|
"startedAt": started_at,
|
||||||
|
"finishedAt": finished_at,
|
||||||
|
},
|
||||||
|
fh,
|
||||||
|
indent=2,
|
||||||
|
)
|
||||||
|
PY
|
||||||
|
}
|
||||||
|
|
||||||
|
run_step() {
|
||||||
|
local step_id="$1"
|
||||||
|
local kind="$2"
|
||||||
|
local previous_step_id="$3"
|
||||||
|
shift 3
|
||||||
|
|
||||||
|
local started_at_iso started_at_ms finished_at_iso finished_at_ms prefix
|
||||||
|
started_at_iso="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||||
|
started_at_ms="$(python3 - <<'PY'
|
||||||
|
import time
|
||||||
|
print(int(time.time() * 1000))
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
prefix="${started_at_iso}-test"
|
||||||
|
|
||||||
|
local cir_out="$OUT/${prefix}.cir"
|
||||||
|
local ccr_out="$OUT/${prefix}.ccr"
|
||||||
|
local report_out="$OUT/${prefix}.report.json"
|
||||||
|
local timing_out="$OUT/${prefix}.timing.json"
|
||||||
|
local stdout_out="$OUT/${prefix}.stdout.log"
|
||||||
|
local stderr_out="$OUT/${prefix}.stderr.log"
|
||||||
|
|
||||||
|
local -a cmd=(
|
||||||
|
target/release/rpki
|
||||||
|
--db "$DB"
|
||||||
|
--raw-store-db "$RAW_STORE_DB"
|
||||||
|
--tal-path "$TAL_REL"
|
||||||
|
--ta-path "$TA_REL"
|
||||||
|
--ccr-out "$ccr_out"
|
||||||
|
--report-json "$report_out"
|
||||||
|
--cir-enable
|
||||||
|
--cir-out "$cir_out"
|
||||||
|
--cir-tal-uri "https://example.test/${RIR}.tal"
|
||||||
|
)
|
||||||
|
if [[ "$FULL_REPO" -ne 1 ]]; then
|
||||||
|
cmd+=(--max-depth "$MAX_DEPTH" --max-instances "$MAX_INSTANCES")
|
||||||
|
fi
|
||||||
|
cmd+=("$@")
|
||||||
|
env RPKI_PROGRESS_LOG=1 "${cmd[@]}" >"$stdout_out" 2>"$stderr_out"
|
||||||
|
|
||||||
|
finished_at_ms="$(python3 - <<'PY'
|
||||||
|
import time
|
||||||
|
print(int(time.time() * 1000))
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
finished_at_iso="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||||
|
write_step_timing "$timing_out" "$started_at_ms" "$finished_at_ms" "$started_at_iso" "$finished_at_iso"
|
||||||
|
|
||||||
|
local validation_time
|
||||||
|
validation_time="$(python3 - <<'PY' "$report_out"
|
||||||
|
import json, sys
|
||||||
|
print(json.load(open(sys.argv[1], 'r', encoding='utf-8'))['meta']['validation_time_rfc3339_utc'])
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
|
||||||
|
printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' \
|
||||||
|
"$step_id" \
|
||||||
|
"$kind" \
|
||||||
|
"$validation_time" \
|
||||||
|
"$(basename "$cir_out")" \
|
||||||
|
"$(basename "$ccr_out")" \
|
||||||
|
"$(basename "$report_out")" \
|
||||||
|
"$(basename "$timing_out")" \
|
||||||
|
"$(basename "$stdout_out")" \
|
||||||
|
"$(basename "$stderr_out")" >> "$ROWS"
|
||||||
|
}
|
||||||
|
|
||||||
|
run_step "full" "full" ""
|
||||||
|
|
||||||
|
prev="full"
|
||||||
|
for idx in $(seq 1 "$DELTA_COUNT"); do
|
||||||
|
sleep "$SLEEP_SECS"
|
||||||
|
step="$(printf 'delta-%03d' "$idx")"
|
||||||
|
run_step "$step" "delta" "$prev"
|
||||||
|
prev="$step"
|
||||||
|
done
|
||||||
|
|
||||||
|
python3 - <<'PY' "$OUT" "$ROWS" "$RIR"
|
||||||
|
import json, sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
out = Path(sys.argv[1])
|
||||||
|
rows = Path(sys.argv[2]).read_text(encoding='utf-8').splitlines()
|
||||||
|
rir = sys.argv[3]
|
||||||
|
steps = []
|
||||||
|
for idx, row in enumerate(rows):
|
||||||
|
step_id, kind, validation_time, cir_name, ccr_name, report_name, timing_name, stdout_name, stderr_name = row.split('\t')
|
||||||
|
steps.append({
|
||||||
|
"stepId": step_id,
|
||||||
|
"kind": kind,
|
||||||
|
"validationTime": validation_time,
|
||||||
|
"cirPath": cir_name,
|
||||||
|
"ccrPath": ccr_name,
|
||||||
|
"reportPath": report_name,
|
||||||
|
"timingPath": timing_name,
|
||||||
|
"stdoutLogPath": stdout_name,
|
||||||
|
"stderrLogPath": stderr_name,
|
||||||
|
"artifactPrefix": cir_name[:-4], # strip .cir
|
||||||
|
"previousStepId": None if idx == 0 else steps[idx - 1]["stepId"],
|
||||||
|
})
|
||||||
|
|
||||||
|
(out / "sequence.json").write_text(
|
||||||
|
json.dumps({"version": 1, "rawStoreDbPath": "raw-store.db", "steps": steps}, indent=2),
|
||||||
|
encoding="utf-8",
|
||||||
|
)
|
||||||
|
|
||||||
|
summary = {
|
||||||
|
"version": 1,
|
||||||
|
"rir": rir,
|
||||||
|
"stepCount": len(steps),
|
||||||
|
"steps": [],
|
||||||
|
}
|
||||||
|
for step in steps:
|
||||||
|
timing = json.loads((out / step["timingPath"]).read_text(encoding="utf-8"))
|
||||||
|
summary["steps"].append({
|
||||||
|
"stepId": step["stepId"],
|
||||||
|
"kind": step["kind"],
|
||||||
|
"validationTime": step["validationTime"],
|
||||||
|
"artifactPrefix": step["artifactPrefix"],
|
||||||
|
**timing,
|
||||||
|
})
|
||||||
|
|
||||||
|
(out / "summary.json").write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
||||||
|
PY
|
||||||
|
|
||||||
|
rm -f "$ROWS"
|
||||||
|
echo "$OUT"
|
||||||
|
EOS
|
||||||
72
scripts/cir/run_cir_record_sequence_remote_multi_rir.sh
Executable file
72
scripts/cir/run_cir_record_sequence_remote_multi_rir.sh
Executable file
@@ -0,0 +1,72 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage:
|
||||||
|
./scripts/cir/run_cir_record_sequence_remote_multi_rir.sh \
|
||||||
|
--remote-root <path> \
|
||||||
|
[--rir <afrinic,apnic,arin,lacnic,ripe>] \
|
||||||
|
[--ssh-target <user@host>] \
|
||||||
|
[--out-subdir-root <path>] \
|
||||||
|
[--delta-count <n>] \
|
||||||
|
[--sleep-secs <n>] \
|
||||||
|
[--full-repo] \
|
||||||
|
[--max-depth <n>] \
|
||||||
|
[--max-instances <n>]
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
SSH_TARGET="${SSH_TARGET:-root@47.77.183.68}"
|
||||||
|
REMOTE_ROOT=""
|
||||||
|
RIRS="afrinic,apnic,arin,lacnic,ripe"
|
||||||
|
OUT_SUBDIR_ROOT=""
|
||||||
|
DELTA_COUNT=2
|
||||||
|
SLEEP_SECS=30
|
||||||
|
FULL_REPO=0
|
||||||
|
MAX_DEPTH=0
|
||||||
|
MAX_INSTANCES=1
|
||||||
|
SINGLE="$ROOT_DIR/scripts/cir/run_cir_record_sequence_remote.sh"
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--remote-root) REMOTE_ROOT="$2"; shift 2 ;;
|
||||||
|
--rir) RIRS="$2"; shift 2 ;;
|
||||||
|
--ssh-target) SSH_TARGET="$2"; shift 2 ;;
|
||||||
|
--out-subdir-root) OUT_SUBDIR_ROOT="$2"; shift 2 ;;
|
||||||
|
--delta-count) DELTA_COUNT="$2"; shift 2 ;;
|
||||||
|
--sleep-secs) SLEEP_SECS="$2"; shift 2 ;;
|
||||||
|
--full-repo) FULL_REPO=1; shift 1 ;;
|
||||||
|
--max-depth) MAX_DEPTH="$2"; shift 2 ;;
|
||||||
|
--max-instances) MAX_INSTANCES="$2"; shift 2 ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
[[ -n "$REMOTE_ROOT" ]] || { usage >&2; exit 2; }
|
||||||
|
if [[ -z "$OUT_SUBDIR_ROOT" ]]; then
|
||||||
|
OUT_SUBDIR_ROOT="target/replay/cir_sequence_remote_multi_rir_$(date -u +%Y%m%dT%H%M%SZ)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
IFS=',' read -r -a ITEMS <<< "$RIRS"
|
||||||
|
for rir in "${ITEMS[@]}"; do
|
||||||
|
args=(
|
||||||
|
"$SINGLE"
|
||||||
|
--rir "$rir" \
|
||||||
|
--remote-root "$REMOTE_ROOT" \
|
||||||
|
--ssh-target "$SSH_TARGET" \
|
||||||
|
--out-subdir "$OUT_SUBDIR_ROOT/$rir" \
|
||||||
|
--delta-count "$DELTA_COUNT" \
|
||||||
|
--sleep-secs "$SLEEP_SECS" \
|
||||||
|
)
|
||||||
|
if [[ "$FULL_REPO" -eq 1 ]]; then
|
||||||
|
args+=(--full-repo)
|
||||||
|
else
|
||||||
|
args+=(--max-depth "$MAX_DEPTH" --max-instances "$MAX_INSTANCES")
|
||||||
|
fi
|
||||||
|
"${args[@]}"
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "$OUT_SUBDIR_ROOT"
|
||||||
118
scripts/cir/run_cir_record_sequence_ta_only_multi_rir.sh
Executable file
118
scripts/cir/run_cir_record_sequence_ta_only_multi_rir.sh
Executable file
@@ -0,0 +1,118 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage:
|
||||||
|
./scripts/cir/run_cir_record_sequence_ta_only_multi_rir.sh \
|
||||||
|
[--rir <afrinic,apnic,arin,lacnic,ripe>] \
|
||||||
|
[--delta-count <n>] \
|
||||||
|
[--out-root <path>] \
|
||||||
|
[--rpki-bin <path>]
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
HELPER_BIN="${HELPER_BIN:-$ROOT_DIR/target/release/cir_ta_only_fixture}"
|
||||||
|
MATERIALIZE_BIN="${MATERIALIZE_BIN:-$ROOT_DIR/target/release/cir_materialize}"
|
||||||
|
EXTRACT_BIN="${EXTRACT_BIN:-$ROOT_DIR/target/release/cir_extract_inputs}"
|
||||||
|
WRAPPER="$ROOT_DIR/scripts/cir/cir-rsync-wrapper"
|
||||||
|
RIRS="afrinic,apnic,arin,lacnic,ripe"
|
||||||
|
DELTA_COUNT=2
|
||||||
|
OUT_ROOT="$ROOT_DIR/target/replay/cir_sequence_multi_rir_ta_only_$(date -u +%Y%m%dT%H%M%SZ)"
|
||||||
|
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--rir) RIRS="$2"; shift 2 ;;
|
||||||
|
--delta-count) DELTA_COUNT="$2"; shift 2 ;;
|
||||||
|
--out-root) OUT_ROOT="$2"; shift 2 ;;
|
||||||
|
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
if [[ ! -x "$HELPER_BIN" ]]; then
|
||||||
|
(
|
||||||
|
cd "$ROOT_DIR"
|
||||||
|
cargo build --release --bin cir_ta_only_fixture --bin rpki --bin cir_materialize --bin cir_extract_inputs
|
||||||
|
)
|
||||||
|
fi
|
||||||
|
|
||||||
|
case_paths() {
|
||||||
|
case "$1" in
|
||||||
|
afrinic) echo "tests/fixtures/tal/afrinic.tal tests/fixtures/ta/afrinic-ta.cer" ;;
|
||||||
|
apnic) echo "tests/fixtures/tal/apnic-rfc7730-https.tal tests/fixtures/ta/apnic-ta.cer" ;;
|
||||||
|
arin) echo "tests/fixtures/tal/arin.tal tests/fixtures/ta/arin-ta.cer" ;;
|
||||||
|
lacnic) echo "tests/fixtures/tal/lacnic.tal tests/fixtures/ta/lacnic-ta.cer" ;;
|
||||||
|
ripe) echo "tests/fixtures/tal/ripe-ncc.tal tests/fixtures/ta/ripe-ncc-ta.cer" ;;
|
||||||
|
*) return 1 ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
mkdir -p "$OUT_ROOT"
|
||||||
|
IFS=',' read -r -a ITEMS <<< "$RIRS"
|
||||||
|
for rir in "${ITEMS[@]}"; do
|
||||||
|
read -r tal_rel ta_rel < <(case_paths "$rir")
|
||||||
|
rir_root="$OUT_ROOT/$rir"
|
||||||
|
mkdir -p "$rir_root/full" "$rir_root/static"
|
||||||
|
"$HELPER_BIN" \
|
||||||
|
--tal-path "$ROOT_DIR/$tal_rel" \
|
||||||
|
--ta-path "$ROOT_DIR/$ta_rel" \
|
||||||
|
--tal-uri "https://example.test/$rir.tal" \
|
||||||
|
--validation-time "2026-04-09T00:00:00Z" \
|
||||||
|
--cir-out "$rir_root/full/input.cir" \
|
||||||
|
--static-root "$rir_root/static"
|
||||||
|
"$EXTRACT_BIN" --cir "$rir_root/full/input.cir" --tals-dir "$rir_root/.tmp/tals" --meta-json "$rir_root/.tmp/meta.json"
|
||||||
|
"$MATERIALIZE_BIN" --cir "$rir_root/full/input.cir" --static-root "$rir_root/static" --mirror-root "$rir_root/.tmp/mirror"
|
||||||
|
FIRST_TAL="$(python3 - <<'PY' "$rir_root/.tmp/meta.json"
|
||||||
|
import json,sys
|
||||||
|
print(json.load(open(sys.argv[1]))["talFiles"][0]["path"])
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
export CIR_MIRROR_ROOT="$rir_root/.tmp/mirror"
|
||||||
|
export REAL_RSYNC_BIN=/usr/bin/rsync
|
||||||
|
export CIR_LOCAL_LINK_MODE=1
|
||||||
|
"$RPKI_BIN" \
|
||||||
|
--db "$rir_root/full/db" \
|
||||||
|
--tal-path "$FIRST_TAL" \
|
||||||
|
--disable-rrdp \
|
||||||
|
--rsync-command "$WRAPPER" \
|
||||||
|
--validation-time "2026-04-09T00:00:00Z" \
|
||||||
|
--ccr-out "$rir_root/full/result.ccr" \
|
||||||
|
--report-json "$rir_root/full/report.json" >/dev/null 2>&1
|
||||||
|
for idx in $(seq 1 "$DELTA_COUNT"); do
|
||||||
|
step="$(printf 'delta-%03d' "$idx")"
|
||||||
|
mkdir -p "$rir_root/$step"
|
||||||
|
cp "$rir_root/full/input.cir" "$rir_root/$step/input.cir"
|
||||||
|
cp "$rir_root/full/result.ccr" "$rir_root/$step/result.ccr"
|
||||||
|
cp "$rir_root/full/report.json" "$rir_root/$step/report.json"
|
||||||
|
done
|
||||||
|
python3 - <<'PY' "$rir_root" "$DELTA_COUNT"
|
||||||
|
import json, sys
|
||||||
|
from pathlib import Path
|
||||||
|
root = Path(sys.argv[1]); delta_count = int(sys.argv[2])
|
||||||
|
steps = [{"stepId":"full","kind":"full","validationTime":"2026-04-09T00:00:00Z","cirPath":"full/input.cir","ccrPath":"full/result.ccr","reportPath":"full/report.json","previousStepId":None}]
|
||||||
|
prev = "full"
|
||||||
|
for i in range(1, delta_count + 1):
|
||||||
|
step = f"delta-{i:03d}"
|
||||||
|
steps.append({"stepId":step,"kind":"delta","validationTime":"2026-04-09T00:00:00Z","cirPath":f"{step}/input.cir","ccrPath":f"{step}/result.ccr","reportPath":f"{step}/report.json","previousStepId":prev})
|
||||||
|
prev = step
|
||||||
|
(root/"sequence.json").write_text(json.dumps({"version":1,"staticRoot":"static","steps":steps}, indent=2), encoding="utf-8")
|
||||||
|
(root/"summary.json").write_text(json.dumps({"version":1,"stepCount":len(steps)}, indent=2), encoding="utf-8")
|
||||||
|
PY
|
||||||
|
done
|
||||||
|
|
||||||
|
python3 - <<'PY' "$OUT_ROOT" "$RIRS"
|
||||||
|
import json, sys
|
||||||
|
from pathlib import Path
|
||||||
|
root = Path(sys.argv[1]); rirs = [x for x in sys.argv[2].split(',') if x]
|
||||||
|
items=[]
|
||||||
|
for rir in rirs:
|
||||||
|
seq=json.loads((root/rir/'sequence.json').read_text())
|
||||||
|
items.append({"rir":rir,"stepCount":len(seq['steps'])})
|
||||||
|
(root/'summary.json').write_text(json.dumps({"version":1,"rirs":items}, indent=2), encoding='utf-8')
|
||||||
|
PY
|
||||||
|
|
||||||
|
echo "done: $OUT_ROOT"
|
||||||
49
scripts/cir/run_cir_record_sequence_ta_only_remote_multi_rir.sh
Executable file
49
scripts/cir/run_cir_record_sequence_ta_only_remote_multi_rir.sh
Executable file
@@ -0,0 +1,49 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage:
|
||||||
|
./scripts/cir/run_cir_record_sequence_ta_only_remote_multi_rir.sh \
|
||||||
|
--remote-root <path> \
|
||||||
|
[--ssh-target <user@host>] \
|
||||||
|
[--rir <afrinic,apnic,arin,lacnic,ripe>] \
|
||||||
|
[--delta-count <n>]
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
SSH_TARGET="${SSH_TARGET:-root@47.77.183.68}"
|
||||||
|
REMOTE_ROOT=""
|
||||||
|
RIRS="afrinic,apnic,arin,lacnic,ripe"
|
||||||
|
DELTA_COUNT=2
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--remote-root) REMOTE_ROOT="$2"; shift 2 ;;
|
||||||
|
--ssh-target) SSH_TARGET="$2"; shift 2 ;;
|
||||||
|
--rir) RIRS="$2"; shift 2 ;;
|
||||||
|
--delta-count) DELTA_COUNT="$2"; shift 2 ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
[[ -n "$REMOTE_ROOT" ]] || { usage >&2; exit 2; }
|
||||||
|
|
||||||
|
rsync -a --delete \
|
||||||
|
--exclude target \
|
||||||
|
--exclude .git \
|
||||||
|
"$ROOT_DIR/" "$SSH_TARGET:$REMOTE_ROOT/"
|
||||||
|
ssh "$SSH_TARGET" "mkdir -p '$REMOTE_ROOT/target/release'"
|
||||||
|
for bin in rpki cir_ta_only_fixture cir_materialize cir_extract_inputs; do
|
||||||
|
rsync -a "$ROOT_DIR/target/release/$bin" "$SSH_TARGET:$REMOTE_ROOT/target/release/"
|
||||||
|
done
|
||||||
|
|
||||||
|
ssh "$SSH_TARGET" "bash -lc '
|
||||||
|
set -euo pipefail
|
||||||
|
cd $REMOTE_ROOT
|
||||||
|
OUT=target/replay/cir_sequence_remote_ta_only_\$(date -u +%Y%m%dT%H%M%SZ)
|
||||||
|
./scripts/cir/run_cir_record_sequence_ta_only_multi_rir.sh --rir $RIRS --delta-count $DELTA_COUNT --out-root \"\$OUT\"
|
||||||
|
echo \"\$OUT\"
|
||||||
|
'"
|
||||||
286
scripts/cir/run_cir_replay_matrix.sh
Executable file
286
scripts/cir/run_cir_replay_matrix.sh
Executable file
@@ -0,0 +1,286 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage:
|
||||||
|
./scripts/cir/run_cir_replay_matrix.sh \
|
||||||
|
--cir <path> \
|
||||||
|
--static-root <path> \
|
||||||
|
--out-dir <path> \
|
||||||
|
--reference-ccr <path> \
|
||||||
|
--rpki-client-build-dir <path> \
|
||||||
|
[--keep-db] \
|
||||||
|
[--rpki-bin <path>] \
|
||||||
|
[--routinator-root <path>] \
|
||||||
|
[--routinator-bin <path>] \
|
||||||
|
[--real-rsync-bin <path>]
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
|
||||||
|
CIR=""
|
||||||
|
STATIC_ROOT=""
|
||||||
|
OUT_DIR=""
|
||||||
|
REFERENCE_CCR=""
|
||||||
|
RPKI_CLIENT_BUILD_DIR=""
|
||||||
|
KEEP_DB=0
|
||||||
|
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
||||||
|
ROUTINATOR_ROOT="${ROUTINATOR_ROOT:-/home/yuyr/dev/rust_playground/routinator}"
|
||||||
|
ROUTINATOR_BIN="${ROUTINATOR_BIN:-$ROUTINATOR_ROOT/target/debug/routinator}"
|
||||||
|
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
||||||
|
OURS_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_ours.sh"
|
||||||
|
ROUTINATOR_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_routinator.sh"
|
||||||
|
RPKI_CLIENT_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_rpki_client.sh"
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--cir) CIR="$2"; shift 2 ;;
|
||||||
|
--static-root) STATIC_ROOT="$2"; shift 2 ;;
|
||||||
|
--out-dir) OUT_DIR="$2"; shift 2 ;;
|
||||||
|
--reference-ccr) REFERENCE_CCR="$2"; shift 2 ;;
|
||||||
|
--rpki-client-build-dir) RPKI_CLIENT_BUILD_DIR="$2"; shift 2 ;;
|
||||||
|
--keep-db) KEEP_DB=1; shift ;;
|
||||||
|
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
||||||
|
--routinator-root) ROUTINATOR_ROOT="$2"; shift 2 ;;
|
||||||
|
--routinator-bin) ROUTINATOR_BIN="$2"; shift 2 ;;
|
||||||
|
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
[[ -n "$CIR" && -n "$STATIC_ROOT" && -n "$OUT_DIR" && -n "$REFERENCE_CCR" && -n "$RPKI_CLIENT_BUILD_DIR" ]] || {
|
||||||
|
usage >&2
|
||||||
|
exit 2
|
||||||
|
}
|
||||||
|
|
||||||
|
mkdir -p "$OUT_DIR"
|
||||||
|
|
||||||
|
run_with_timing() {
|
||||||
|
local summary_path="$1"
|
||||||
|
local timing_path="$2"
|
||||||
|
shift 2
|
||||||
|
local start end status
|
||||||
|
start="$(python3 - <<'PY'
|
||||||
|
import time
|
||||||
|
print(time.perf_counter_ns())
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
if "$@"; then
|
||||||
|
status=0
|
||||||
|
else
|
||||||
|
status=$?
|
||||||
|
fi
|
||||||
|
end="$(python3 - <<'PY'
|
||||||
|
import time
|
||||||
|
print(time.perf_counter_ns())
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
python3 - <<'PY' "$summary_path" "$timing_path" "$status" "$start" "$end"
|
||||||
|
import json, sys
|
||||||
|
summary_path, timing_path, status, start, end = sys.argv[1:]
|
||||||
|
duration_ms = max(0, (int(end) - int(start)) // 1_000_000)
|
||||||
|
data = {"exitCode": int(status), "durationMs": duration_ms}
|
||||||
|
try:
|
||||||
|
with open(summary_path, "r", encoding="utf-8") as f:
|
||||||
|
data["compare"] = json.load(f)
|
||||||
|
except FileNotFoundError:
|
||||||
|
data["compare"] = None
|
||||||
|
with open(timing_path, "w", encoding="utf-8") as f:
|
||||||
|
json.dump(data, f, indent=2)
|
||||||
|
PY
|
||||||
|
return "$status"
|
||||||
|
}
|
||||||
|
|
||||||
|
OURS_OUT="$OUT_DIR/ours"
|
||||||
|
ROUTINATOR_OUT="$OUT_DIR/routinator"
|
||||||
|
RPKI_CLIENT_OUT="$OUT_DIR/rpki-client"
|
||||||
|
mkdir -p "$OURS_OUT" "$ROUTINATOR_OUT" "$RPKI_CLIENT_OUT"
|
||||||
|
|
||||||
|
ours_cmd=(
|
||||||
|
"$OURS_SCRIPT"
|
||||||
|
--cir "$CIR"
|
||||||
|
--static-root "$STATIC_ROOT"
|
||||||
|
--out-dir "$OURS_OUT"
|
||||||
|
--reference-ccr "$REFERENCE_CCR"
|
||||||
|
--rpki-bin "$RPKI_BIN"
|
||||||
|
--real-rsync-bin "$REAL_RSYNC_BIN"
|
||||||
|
)
|
||||||
|
routinator_cmd=(
|
||||||
|
"$ROUTINATOR_SCRIPT"
|
||||||
|
--cir "$CIR"
|
||||||
|
--static-root "$STATIC_ROOT"
|
||||||
|
--out-dir "$ROUTINATOR_OUT"
|
||||||
|
--reference-ccr "$REFERENCE_CCR"
|
||||||
|
--routinator-root "$ROUTINATOR_ROOT"
|
||||||
|
--routinator-bin "$ROUTINATOR_BIN"
|
||||||
|
--real-rsync-bin "$REAL_RSYNC_BIN"
|
||||||
|
)
|
||||||
|
rpki_client_cmd=(
|
||||||
|
"$RPKI_CLIENT_SCRIPT"
|
||||||
|
--cir "$CIR"
|
||||||
|
--static-root "$STATIC_ROOT"
|
||||||
|
--out-dir "$RPKI_CLIENT_OUT"
|
||||||
|
--reference-ccr "$REFERENCE_CCR"
|
||||||
|
--build-dir "$RPKI_CLIENT_BUILD_DIR"
|
||||||
|
--real-rsync-bin "$REAL_RSYNC_BIN"
|
||||||
|
)
|
||||||
|
|
||||||
|
if [[ "$KEEP_DB" -eq 1 ]]; then
|
||||||
|
ours_cmd+=(--keep-db)
|
||||||
|
routinator_cmd+=(--keep-db)
|
||||||
|
rpki_client_cmd+=(--keep-db)
|
||||||
|
fi
|
||||||
|
|
||||||
|
ours_status=0
|
||||||
|
routinator_status=0
|
||||||
|
rpki_client_status=0
|
||||||
|
if run_with_timing "$OURS_OUT/compare-summary.json" "$OURS_OUT/timing.json" "${ours_cmd[@]}"; then
|
||||||
|
:
|
||||||
|
else
|
||||||
|
ours_status=$?
|
||||||
|
fi
|
||||||
|
if run_with_timing "$ROUTINATOR_OUT/compare-summary.json" "$ROUTINATOR_OUT/timing.json" "${routinator_cmd[@]}"; then
|
||||||
|
:
|
||||||
|
else
|
||||||
|
routinator_status=$?
|
||||||
|
fi
|
||||||
|
if run_with_timing "$RPKI_CLIENT_OUT/compare-summary.json" "$RPKI_CLIENT_OUT/timing.json" "${rpki_client_cmd[@]}"; then
|
||||||
|
:
|
||||||
|
else
|
||||||
|
rpki_client_status=$?
|
||||||
|
fi
|
||||||
|
|
||||||
|
SUMMARY_JSON="$OUT_DIR/summary.json"
|
||||||
|
SUMMARY_MD="$OUT_DIR/summary.md"
|
||||||
|
DETAIL_MD="$OUT_DIR/detail.md"
|
||||||
|
|
||||||
|
python3 - <<'PY' \
|
||||||
|
"$CIR" \
|
||||||
|
"$STATIC_ROOT" \
|
||||||
|
"$REFERENCE_CCR" \
|
||||||
|
"$OURS_OUT" \
|
||||||
|
"$ROUTINATOR_OUT" \
|
||||||
|
"$RPKI_CLIENT_OUT" \
|
||||||
|
"$SUMMARY_JSON" \
|
||||||
|
"$SUMMARY_MD" \
|
||||||
|
"$DETAIL_MD"
|
||||||
|
import json
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
cir_path, static_root, reference_ccr, ours_out, routinator_out, rpki_client_out, summary_json, summary_md, detail_md = sys.argv[1:]
|
||||||
|
|
||||||
|
participants = []
|
||||||
|
all_match = True
|
||||||
|
for name, out_dir in [
|
||||||
|
("ours", ours_out),
|
||||||
|
("routinator", routinator_out),
|
||||||
|
("rpki-client", rpki_client_out),
|
||||||
|
]:
|
||||||
|
out = Path(out_dir)
|
||||||
|
timing = json.loads((out / "timing.json").read_text(encoding="utf-8"))
|
||||||
|
compare = timing.get("compare") or {}
|
||||||
|
vrps = compare.get("vrps") or {}
|
||||||
|
vaps = compare.get("vaps") or {}
|
||||||
|
participant = {
|
||||||
|
"name": name,
|
||||||
|
"outDir": str(out),
|
||||||
|
"tmpRoot": str(out / ".tmp"),
|
||||||
|
"mirrorPath": str(out / ".tmp" / "mirror"),
|
||||||
|
"timingPath": str(out / "timing.json"),
|
||||||
|
"summaryPath": str(out / "compare-summary.json"),
|
||||||
|
"exitCode": timing["exitCode"],
|
||||||
|
"durationMs": timing["durationMs"],
|
||||||
|
"vrps": vrps,
|
||||||
|
"vaps": vaps,
|
||||||
|
"match": bool(vrps.get("match")) and bool(vaps.get("match")) and timing["exitCode"] == 0,
|
||||||
|
"logPaths": [str(path) for path in sorted(out.glob("*.log"))],
|
||||||
|
}
|
||||||
|
participants.append(participant)
|
||||||
|
all_match = all_match and participant["match"]
|
||||||
|
|
||||||
|
summary = {
|
||||||
|
"cirPath": cir_path,
|
||||||
|
"staticRoot": static_root,
|
||||||
|
"referenceCcr": reference_ccr,
|
||||||
|
"participants": participants,
|
||||||
|
"allMatch": all_match,
|
||||||
|
}
|
||||||
|
Path(summary_json).write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
||||||
|
|
||||||
|
lines = [
|
||||||
|
"# CIR Replay Matrix Summary",
|
||||||
|
"",
|
||||||
|
f"- `cir`: `{cir_path}`",
|
||||||
|
f"- `static_root`: `{static_root}`",
|
||||||
|
f"- `reference_ccr`: `{reference_ccr}`",
|
||||||
|
f"- `all_match`: `{all_match}`",
|
||||||
|
"",
|
||||||
|
"| Participant | Exit | Duration (ms) | VRP actual/ref | VRP match | VAP actual/ref | VAP match | Log |",
|
||||||
|
"| --- | ---: | ---: | --- | --- | --- | --- | --- |",
|
||||||
|
]
|
||||||
|
for participant in participants:
|
||||||
|
vrps = participant["vrps"] or {}
|
||||||
|
vaps = participant["vaps"] or {}
|
||||||
|
log_path = participant["logPaths"][0] if participant["logPaths"] else ""
|
||||||
|
lines.append(
|
||||||
|
"| {name} | {exit_code} | {duration_ms} | {vrp_actual}/{vrp_ref} | {vrp_match} | {vap_actual}/{vap_ref} | {vap_match} | `{log_path}` |".format(
|
||||||
|
name=participant["name"],
|
||||||
|
exit_code=participant["exitCode"],
|
||||||
|
duration_ms=participant["durationMs"],
|
||||||
|
vrp_actual=vrps.get("actual", "-"),
|
||||||
|
vrp_ref=vrps.get("reference", "-"),
|
||||||
|
vrp_match=vrps.get("match", False),
|
||||||
|
vap_actual=vaps.get("actual", "-"),
|
||||||
|
vap_ref=vaps.get("reference", "-"),
|
||||||
|
vap_match=vaps.get("match", False),
|
||||||
|
log_path=log_path,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
Path(summary_md).write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||||
|
|
||||||
|
detail_lines = [
|
||||||
|
"# CIR Replay Matrix Detail",
|
||||||
|
"",
|
||||||
|
]
|
||||||
|
for participant in participants:
|
||||||
|
vrps = participant["vrps"] or {}
|
||||||
|
vaps = participant["vaps"] or {}
|
||||||
|
detail_lines.extend([
|
||||||
|
f"## {participant['name']}",
|
||||||
|
f"- `exit_code`: `{participant['exitCode']}`",
|
||||||
|
f"- `duration_ms`: `{participant['durationMs']}`",
|
||||||
|
f"- `out_dir`: `{participant['outDir']}`",
|
||||||
|
f"- `tmp_root`: `{participant['tmpRoot']}`",
|
||||||
|
f"- `mirror_path`: `{participant['mirrorPath']}`",
|
||||||
|
f"- `summary_path`: `{participant['summaryPath']}`",
|
||||||
|
f"- `timing_path`: `{participant['timingPath']}`",
|
||||||
|
f"- `log_paths`: `{', '.join(participant['logPaths'])}`",
|
||||||
|
f"- `vrps`: `actual={vrps.get('actual', '-')}` `reference={vrps.get('reference', '-')}` `match={vrps.get('match', False)}`",
|
||||||
|
f"- `vaps`: `actual={vaps.get('actual', '-')}` `reference={vaps.get('reference', '-')}` `match={vaps.get('match', False)}`",
|
||||||
|
f"- `vrps.only_in_actual`: `{vrps.get('only_in_actual', [])}`",
|
||||||
|
f"- `vrps.only_in_reference`: `{vrps.get('only_in_reference', [])}`",
|
||||||
|
f"- `vaps.only_in_actual`: `{vaps.get('only_in_actual', [])}`",
|
||||||
|
f"- `vaps.only_in_reference`: `{vaps.get('only_in_reference', [])}`",
|
||||||
|
"",
|
||||||
|
])
|
||||||
|
Path(detail_md).write_text("\n".join(detail_lines), encoding="utf-8")
|
||||||
|
PY
|
||||||
|
|
||||||
|
if [[ "$ours_status" -ne 0 || "$routinator_status" -ne 0 || "$rpki_client_status" -ne 0 ]]; then
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
all_match="$(python3 - <<'PY' "$SUMMARY_JSON"
|
||||||
|
import json,sys
|
||||||
|
print("true" if json.load(open(sys.argv[1]))["allMatch"] else "false")
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
if [[ "$all_match" != "true" ]]; then
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "done: $OUT_DIR"
|
||||||
162
scripts/cir/run_cir_replay_ours.sh
Executable file
162
scripts/cir/run_cir_replay_ours.sh
Executable file
@@ -0,0 +1,162 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage:
|
||||||
|
./scripts/cir/run_cir_replay_ours.sh \
|
||||||
|
--cir <path> \
|
||||||
|
[--static-root <path> | --raw-store-db <path>] \
|
||||||
|
--out-dir <path> \
|
||||||
|
--reference-ccr <path> \
|
||||||
|
[--keep-db] \
|
||||||
|
[--rpki-bin <path>] \
|
||||||
|
[--real-rsync-bin <path>]
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
|
||||||
|
CIR=""
|
||||||
|
STATIC_ROOT=""
|
||||||
|
POOL_DB=""
|
||||||
|
RAW_STORE_DB=""
|
||||||
|
OUT_DIR=""
|
||||||
|
REFERENCE_CCR=""
|
||||||
|
KEEP_DB=0
|
||||||
|
RPKI_BIN="$ROOT_DIR/target/release/rpki"
|
||||||
|
CIR_MATERIALIZE_BIN="$ROOT_DIR/target/release/cir_materialize"
|
||||||
|
CIR_EXTRACT_INPUTS_BIN="$ROOT_DIR/target/release/cir_extract_inputs"
|
||||||
|
CCR_TO_COMPARE_VIEWS_BIN="$ROOT_DIR/target/release/ccr_to_compare_views"
|
||||||
|
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
||||||
|
WRAPPER="$ROOT_DIR/scripts/cir/cir-rsync-wrapper"
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--cir) CIR="$2"; shift 2 ;;
|
||||||
|
--static-root) STATIC_ROOT="$2"; shift 2 ;;
|
||||||
|
--raw-store-db) RAW_STORE_DB="$2"; shift 2 ;;
|
||||||
|
--out-dir) OUT_DIR="$2"; shift 2 ;;
|
||||||
|
--reference-ccr) REFERENCE_CCR="$2"; shift 2 ;;
|
||||||
|
--keep-db) KEEP_DB=1; shift ;;
|
||||||
|
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
||||||
|
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
[[ -n "$CIR" && -n "$OUT_DIR" && -n "$REFERENCE_CCR" ]] || {
|
||||||
|
usage >&2
|
||||||
|
exit 2
|
||||||
|
}
|
||||||
|
backend_count=0
|
||||||
|
[[ -n "$STATIC_ROOT" ]] && backend_count=$((backend_count+1))
|
||||||
|
[[ -n "$RAW_STORE_DB" ]] && backend_count=$((backend_count+1))
|
||||||
|
[[ "$backend_count" -eq 1 ]] || { usage >&2; exit 2; }
|
||||||
|
|
||||||
|
mkdir -p "$OUT_DIR"
|
||||||
|
|
||||||
|
if [[ ! -x "$RPKI_BIN" || ! -x "$CIR_MATERIALIZE_BIN" || ! -x "$CIR_EXTRACT_INPUTS_BIN" || ! -x "$CCR_TO_COMPARE_VIEWS_BIN" ]]; then
|
||||||
|
(
|
||||||
|
cd "$ROOT_DIR"
|
||||||
|
cargo build --release --bin rpki --bin cir_materialize --bin cir_extract_inputs --bin ccr_to_compare_views
|
||||||
|
)
|
||||||
|
fi
|
||||||
|
|
||||||
|
TMP_ROOT="$OUT_DIR/.tmp"
|
||||||
|
TALS_DIR="$TMP_ROOT/tals"
|
||||||
|
META_JSON="$TMP_ROOT/meta.json"
|
||||||
|
MIRROR_ROOT="$TMP_ROOT/mirror"
|
||||||
|
DB_DIR="$TMP_ROOT/work-db"
|
||||||
|
ACTUAL_CCR="$OUT_DIR/actual.ccr"
|
||||||
|
ACTUAL_REPORT="$OUT_DIR/report.json"
|
||||||
|
ACTUAL_VRPS="$OUT_DIR/actual-vrps.csv"
|
||||||
|
ACTUAL_VAPS="$OUT_DIR/actual-vaps.csv"
|
||||||
|
REF_VRPS="$OUT_DIR/reference-vrps.csv"
|
||||||
|
REF_VAPS="$OUT_DIR/reference-vaps.csv"
|
||||||
|
COMPARE_JSON="$OUT_DIR/compare-summary.json"
|
||||||
|
RUN_LOG="$OUT_DIR/run.log"
|
||||||
|
|
||||||
|
rm -rf "$TMP_ROOT"
|
||||||
|
mkdir -p "$TMP_ROOT"
|
||||||
|
|
||||||
|
"$CIR_EXTRACT_INPUTS_BIN" --cir "$CIR" --tals-dir "$TALS_DIR" --meta-json "$META_JSON"
|
||||||
|
materialize_cmd=("$CIR_MATERIALIZE_BIN" --cir "$CIR" --mirror-root "$MIRROR_ROOT")
|
||||||
|
if [[ -n "$STATIC_ROOT" ]]; then
|
||||||
|
materialize_cmd+=(--static-root "$STATIC_ROOT")
|
||||||
|
else
|
||||||
|
materialize_cmd+=(--raw-store-db "$RAW_STORE_DB")
|
||||||
|
fi
|
||||||
|
if [[ "$KEEP_DB" -eq 1 ]]; then
|
||||||
|
materialize_cmd+=(--keep-db)
|
||||||
|
fi
|
||||||
|
"${materialize_cmd[@]}"
|
||||||
|
|
||||||
|
VALIDATION_TIME="$(python3 - <<'PY' "$META_JSON"
|
||||||
|
import json,sys
|
||||||
|
print(json.load(open(sys.argv[1]))["validationTime"])
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
FIRST_TAL="$(python3 - <<'PY' "$META_JSON"
|
||||||
|
import json,sys
|
||||||
|
print(json.load(open(sys.argv[1]))["talFiles"][0]["path"])
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
|
||||||
|
export CIR_MIRROR_ROOT="$(python3 - <<'PY' "$MIRROR_ROOT"
|
||||||
|
from pathlib import Path
|
||||||
|
import sys
|
||||||
|
print(Path(sys.argv[1]).resolve())
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
export REAL_RSYNC_BIN="$REAL_RSYNC_BIN"
|
||||||
|
export CIR_LOCAL_LINK_MODE=1
|
||||||
|
|
||||||
|
"$RPKI_BIN" \
|
||||||
|
--db "$DB_DIR" \
|
||||||
|
--tal-path "$FIRST_TAL" \
|
||||||
|
--disable-rrdp \
|
||||||
|
--rsync-command "$WRAPPER" \
|
||||||
|
--validation-time "$VALIDATION_TIME" \
|
||||||
|
--ccr-out "$ACTUAL_CCR" \
|
||||||
|
--report-json "$ACTUAL_REPORT" \
|
||||||
|
>"$RUN_LOG" 2>&1
|
||||||
|
|
||||||
|
"$CCR_TO_COMPARE_VIEWS_BIN" --ccr "$ACTUAL_CCR" --vrps-out "$ACTUAL_VRPS" --vaps-out "$ACTUAL_VAPS" --trust-anchor unknown
|
||||||
|
"$CCR_TO_COMPARE_VIEWS_BIN" --ccr "$REFERENCE_CCR" --vrps-out "$REF_VRPS" --vaps-out "$REF_VAPS" --trust-anchor unknown
|
||||||
|
|
||||||
|
python3 - <<'PY' "$ACTUAL_VRPS" "$REF_VRPS" "$ACTUAL_VAPS" "$REF_VAPS" "$COMPARE_JSON"
|
||||||
|
import csv, json, sys
|
||||||
|
def rows(path):
|
||||||
|
with open(path, newline="") as f:
|
||||||
|
return list(csv.reader(f))[1:]
|
||||||
|
actual_vrps = {tuple(r) for r in rows(sys.argv[1])}
|
||||||
|
ref_vrps = {tuple(r) for r in rows(sys.argv[2])}
|
||||||
|
actual_vaps = {tuple(r) for r in rows(sys.argv[3])}
|
||||||
|
ref_vaps = {tuple(r) for r in rows(sys.argv[4])}
|
||||||
|
summary = {
|
||||||
|
"vrps": {
|
||||||
|
"actual": len(actual_vrps),
|
||||||
|
"reference": len(ref_vrps),
|
||||||
|
"only_in_actual": sorted(actual_vrps - ref_vrps)[:20],
|
||||||
|
"only_in_reference": sorted(ref_vrps - actual_vrps)[:20],
|
||||||
|
"match": actual_vrps == ref_vrps,
|
||||||
|
},
|
||||||
|
"vaps": {
|
||||||
|
"actual": len(actual_vaps),
|
||||||
|
"reference": len(ref_vaps),
|
||||||
|
"only_in_actual": sorted(actual_vaps - ref_vaps)[:20],
|
||||||
|
"only_in_reference": sorted(ref_vaps - actual_vaps)[:20],
|
||||||
|
"match": actual_vaps == ref_vaps,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
with open(sys.argv[5], "w") as f:
|
||||||
|
json.dump(summary, f, indent=2)
|
||||||
|
PY
|
||||||
|
|
||||||
|
if [[ "$KEEP_DB" -ne 1 ]]; then
|
||||||
|
rm -rf "$TMP_ROOT"
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "done: $OUT_DIR"
|
||||||
221
scripts/cir/run_cir_replay_routinator.sh
Executable file
221
scripts/cir/run_cir_replay_routinator.sh
Executable file
@@ -0,0 +1,221 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage:
|
||||||
|
./scripts/cir/run_cir_replay_routinator.sh \
|
||||||
|
--cir <path> \
|
||||||
|
[--static-root <path> | --raw-store-db <path>] \
|
||||||
|
--out-dir <path> \
|
||||||
|
--reference-ccr <path> \
|
||||||
|
[--keep-db] \
|
||||||
|
[--routinator-root <path>] \
|
||||||
|
[--routinator-bin <path>] \
|
||||||
|
[--real-rsync-bin <path>]
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
RPKI_DEV_ROOT="${RPKI_DEV_ROOT:-$ROOT_DIR}"
|
||||||
|
|
||||||
|
CIR=""
|
||||||
|
STATIC_ROOT=""
|
||||||
|
POOL_DB=""
|
||||||
|
RAW_STORE_DB=""
|
||||||
|
OUT_DIR=""
|
||||||
|
REFERENCE_CCR=""
|
||||||
|
KEEP_DB=0
|
||||||
|
ROUTINATOR_ROOT="${ROUTINATOR_ROOT:-/home/yuyr/dev/rust_playground/routinator}"
|
||||||
|
ROUTINATOR_BIN="${ROUTINATOR_BIN:-$ROUTINATOR_ROOT/target/debug/routinator}"
|
||||||
|
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
||||||
|
CIR_MATERIALIZE_BIN="$ROOT_DIR/target/release/cir_materialize"
|
||||||
|
CIR_EXTRACT_INPUTS_BIN="$ROOT_DIR/target/release/cir_extract_inputs"
|
||||||
|
CCR_TO_COMPARE_VIEWS_BIN="$ROOT_DIR/target/release/ccr_to_compare_views"
|
||||||
|
WRAPPER="$ROOT_DIR/scripts/cir/cir-rsync-wrapper"
|
||||||
|
JSON_TO_VAPS="$ROOT_DIR/scripts/cir/json_to_vaps_csv.py"
|
||||||
|
FAKETIME_LIB="${FAKETIME_LIB:-$ROOT_DIR/target/tools/faketime_pkg/extracted/libfaketime/usr/lib/x86_64-linux-gnu/faketime/libfaketime.so.1}"
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--cir) CIR="$2"; shift 2 ;;
|
||||||
|
--static-root) STATIC_ROOT="$2"; shift 2 ;;
|
||||||
|
--raw-store-db) RAW_STORE_DB="$2"; shift 2 ;;
|
||||||
|
--out-dir) OUT_DIR="$2"; shift 2 ;;
|
||||||
|
--reference-ccr) REFERENCE_CCR="$2"; shift 2 ;;
|
||||||
|
--keep-db) KEEP_DB=1; shift ;;
|
||||||
|
--routinator-root) ROUTINATOR_ROOT="$2"; shift 2 ;;
|
||||||
|
--routinator-bin) ROUTINATOR_BIN="$2"; shift 2 ;;
|
||||||
|
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
[[ -n "$CIR" && -n "$OUT_DIR" && -n "$REFERENCE_CCR" ]] || {
|
||||||
|
usage >&2
|
||||||
|
exit 2
|
||||||
|
}
|
||||||
|
backend_count=0
|
||||||
|
[[ -n "$STATIC_ROOT" ]] && backend_count=$((backend_count+1))
|
||||||
|
[[ -n "$RAW_STORE_DB" ]] && backend_count=$((backend_count+1))
|
||||||
|
[[ "$backend_count" -eq 1 ]] || { usage >&2; exit 2; }
|
||||||
|
|
||||||
|
mkdir -p "$OUT_DIR"
|
||||||
|
if [[ ! -x "$CIR_MATERIALIZE_BIN" || ! -x "$CIR_EXTRACT_INPUTS_BIN" || ! -x "$CCR_TO_COMPARE_VIEWS_BIN" ]]; then
|
||||||
|
(
|
||||||
|
cd "$ROOT_DIR"
|
||||||
|
cargo build --release --bin cir_materialize --bin cir_extract_inputs --bin ccr_to_compare_views
|
||||||
|
)
|
||||||
|
fi
|
||||||
|
|
||||||
|
TMP_ROOT="$OUT_DIR/.tmp"
|
||||||
|
TALS_DIR="$TMP_ROOT/tals"
|
||||||
|
META_JSON="$TMP_ROOT/meta.json"
|
||||||
|
MIRROR_ROOT="$TMP_ROOT/mirror"
|
||||||
|
WORK_REPO="$TMP_ROOT/repository"
|
||||||
|
RUN_LOG="$OUT_DIR/routinator.log"
|
||||||
|
ACTUAL_VRPS="$OUT_DIR/actual-vrps.csv"
|
||||||
|
ACTUAL_VAPS_JSON="$OUT_DIR/actual-vaps.json"
|
||||||
|
ACTUAL_VAPS="$OUT_DIR/actual-vaps.csv"
|
||||||
|
REF_VRPS="$OUT_DIR/reference-vrps.csv"
|
||||||
|
REF_VAPS="$OUT_DIR/reference-vaps.csv"
|
||||||
|
SUMMARY_JSON="$OUT_DIR/compare-summary.json"
|
||||||
|
|
||||||
|
rm -rf "$TMP_ROOT"
|
||||||
|
mkdir -p "$TMP_ROOT"
|
||||||
|
|
||||||
|
"$CIR_EXTRACT_INPUTS_BIN" --cir "$CIR" --tals-dir "$TALS_DIR" --meta-json "$META_JSON"
|
||||||
|
python3 - <<'PY' "$TALS_DIR"
|
||||||
|
from pathlib import Path
|
||||||
|
import sys
|
||||||
|
for tal in Path(sys.argv[1]).glob("*.tal"):
|
||||||
|
lines = tal.read_text(encoding="utf-8").splitlines()
|
||||||
|
rsync_uris = [line for line in lines if line.startswith("rsync://")]
|
||||||
|
base64_lines = []
|
||||||
|
seen_sep = False
|
||||||
|
for line in lines:
|
||||||
|
if seen_sep:
|
||||||
|
if line.strip():
|
||||||
|
base64_lines.append(line)
|
||||||
|
elif line.strip() == "":
|
||||||
|
seen_sep = True
|
||||||
|
tal.write_text("\n".join(rsync_uris) + "\n\n" + "\n".join(base64_lines) + "\n", encoding="utf-8")
|
||||||
|
PY
|
||||||
|
materialize_cmd=("$CIR_MATERIALIZE_BIN" --cir "$CIR" --mirror-root "$MIRROR_ROOT")
|
||||||
|
if [[ -n "$STATIC_ROOT" ]]; then
|
||||||
|
materialize_cmd+=(--static-root "$STATIC_ROOT")
|
||||||
|
else
|
||||||
|
materialize_cmd+=(--raw-store-db "$RAW_STORE_DB")
|
||||||
|
fi
|
||||||
|
if [[ "$KEEP_DB" -eq 1 ]]; then
|
||||||
|
materialize_cmd+=(--keep-db)
|
||||||
|
fi
|
||||||
|
"${materialize_cmd[@]}"
|
||||||
|
|
||||||
|
VALIDATION_TIME="$(python3 - <<'PY' "$META_JSON"
|
||||||
|
import json,sys
|
||||||
|
print(json.load(open(sys.argv[1]))["validationTime"])
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
FIRST_TAL="$(python3 - <<'PY' "$META_JSON"
|
||||||
|
import json,sys
|
||||||
|
print(json.load(open(sys.argv[1]))["talFiles"][0]["path"])
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
TA_NAME="$(basename "$FIRST_TAL" .tal)"
|
||||||
|
FAKE_EPOCH="$(python3 - <<'PY' "$VALIDATION_TIME"
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
import sys
|
||||||
|
dt = datetime.fromisoformat(sys.argv[1].replace("Z", "+00:00")).astimezone(timezone.utc)
|
||||||
|
print(int(dt.timestamp()))
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
|
||||||
|
export CIR_MIRROR_ROOT="$(python3 - <<'PY' "$MIRROR_ROOT"
|
||||||
|
from pathlib import Path
|
||||||
|
import sys
|
||||||
|
print(Path(sys.argv[1]).resolve())
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
export REAL_RSYNC_BIN="$REAL_RSYNC_BIN"
|
||||||
|
export CIR_LOCAL_LINK_MODE=1
|
||||||
|
env \
|
||||||
|
LD_PRELOAD="$FAKETIME_LIB" \
|
||||||
|
FAKETIME_FMT=%s \
|
||||||
|
FAKETIME="$FAKE_EPOCH" \
|
||||||
|
FAKETIME_DONT_FAKE_MONOTONIC=1 \
|
||||||
|
"$ROUTINATOR_BIN" \
|
||||||
|
--repository-dir "$WORK_REPO" \
|
||||||
|
--disable-rrdp \
|
||||||
|
--rsync-command "$WRAPPER" \
|
||||||
|
--no-rir-tals \
|
||||||
|
--extra-tals-dir "$TALS_DIR" \
|
||||||
|
--enable-aspa \
|
||||||
|
update --complete >"$RUN_LOG" 2>&1 || true
|
||||||
|
|
||||||
|
env \
|
||||||
|
LD_PRELOAD="$FAKETIME_LIB" \
|
||||||
|
FAKETIME_FMT=%s \
|
||||||
|
FAKETIME="$FAKE_EPOCH" \
|
||||||
|
FAKETIME_DONT_FAKE_MONOTONIC=1 \
|
||||||
|
"$ROUTINATOR_BIN" \
|
||||||
|
--repository-dir "$WORK_REPO" \
|
||||||
|
--disable-rrdp \
|
||||||
|
--rsync-command "$WRAPPER" \
|
||||||
|
--no-rir-tals \
|
||||||
|
--extra-tals-dir "$TALS_DIR" \
|
||||||
|
--enable-aspa \
|
||||||
|
vrps --noupdate -o "$ACTUAL_VRPS" >>"$RUN_LOG" 2>&1
|
||||||
|
|
||||||
|
env \
|
||||||
|
LD_PRELOAD="$FAKETIME_LIB" \
|
||||||
|
FAKETIME_FMT=%s \
|
||||||
|
FAKETIME="$FAKE_EPOCH" \
|
||||||
|
FAKETIME_DONT_FAKE_MONOTONIC=1 \
|
||||||
|
"$ROUTINATOR_BIN" \
|
||||||
|
--repository-dir "$WORK_REPO" \
|
||||||
|
--disable-rrdp \
|
||||||
|
--rsync-command "$WRAPPER" \
|
||||||
|
--no-rir-tals \
|
||||||
|
--extra-tals-dir "$TALS_DIR" \
|
||||||
|
--enable-aspa \
|
||||||
|
vrps --noupdate --format json -o "$ACTUAL_VAPS_JSON" >>"$RUN_LOG" 2>&1
|
||||||
|
|
||||||
|
python3 "$JSON_TO_VAPS" --input "$ACTUAL_VAPS_JSON" --csv-out "$ACTUAL_VAPS"
|
||||||
|
"$CCR_TO_COMPARE_VIEWS_BIN" --ccr "$REFERENCE_CCR" --vrps-out "$REF_VRPS" --vaps-out "$REF_VAPS" --trust-anchor "$TA_NAME"
|
||||||
|
|
||||||
|
python3 - <<'PY' "$ACTUAL_VRPS" "$REF_VRPS" "$ACTUAL_VAPS" "$REF_VAPS" "$SUMMARY_JSON"
|
||||||
|
import csv, json, sys
|
||||||
|
def rows(path):
|
||||||
|
with open(path, newline="") as f:
|
||||||
|
return list(csv.reader(f))[1:]
|
||||||
|
actual_vrps = {tuple(r) for r in rows(sys.argv[1])}
|
||||||
|
ref_vrps = {tuple(r) for r in rows(sys.argv[2])}
|
||||||
|
actual_vaps = {tuple(r) for r in rows(sys.argv[3])}
|
||||||
|
ref_vaps = {tuple(r) for r in rows(sys.argv[4])}
|
||||||
|
summary = {
|
||||||
|
"vrps": {
|
||||||
|
"actual": len(actual_vrps),
|
||||||
|
"reference": len(ref_vrps),
|
||||||
|
"match": actual_vrps == ref_vrps,
|
||||||
|
"only_in_actual": sorted(actual_vrps - ref_vrps)[:20],
|
||||||
|
"only_in_reference": sorted(ref_vrps - actual_vrps)[:20],
|
||||||
|
},
|
||||||
|
"vaps": {
|
||||||
|
"actual": len(actual_vaps),
|
||||||
|
"reference": len(ref_vaps),
|
||||||
|
"match": actual_vaps == ref_vaps,
|
||||||
|
"only_in_actual": sorted(actual_vaps - ref_vaps)[:20],
|
||||||
|
"only_in_reference": sorted(ref_vaps - actual_vaps)[:20],
|
||||||
|
}
|
||||||
|
}
|
||||||
|
with open(sys.argv[5], "w") as f:
|
||||||
|
json.dump(summary, f, indent=2)
|
||||||
|
PY
|
||||||
|
|
||||||
|
if [[ "$KEEP_DB" -ne 1 ]]; then
|
||||||
|
rm -rf "$TMP_ROOT"
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "done: $OUT_DIR"
|
||||||
195
scripts/cir/run_cir_replay_rpki_client.sh
Executable file
195
scripts/cir/run_cir_replay_rpki_client.sh
Executable file
@@ -0,0 +1,195 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage:
|
||||||
|
./scripts/cir/run_cir_replay_rpki_client.sh \
|
||||||
|
--cir <path> \
|
||||||
|
[--static-root <path> | --raw-store-db <path>] \
|
||||||
|
--out-dir <path> \
|
||||||
|
--reference-ccr <path> \
|
||||||
|
--build-dir <path> \
|
||||||
|
[--keep-db] \
|
||||||
|
[--real-rsync-bin <path>]
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
CIR=""
|
||||||
|
STATIC_ROOT=""
|
||||||
|
POOL_DB=""
|
||||||
|
RAW_STORE_DB=""
|
||||||
|
OUT_DIR=""
|
||||||
|
REFERENCE_CCR=""
|
||||||
|
BUILD_DIR=""
|
||||||
|
KEEP_DB=0
|
||||||
|
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
||||||
|
CIR_MATERIALIZE_BIN="$ROOT_DIR/target/release/cir_materialize"
|
||||||
|
CIR_EXTRACT_INPUTS_BIN="$ROOT_DIR/target/release/cir_extract_inputs"
|
||||||
|
CCR_TO_COMPARE_VIEWS_BIN="$ROOT_DIR/target/release/ccr_to_compare_views"
|
||||||
|
WRAPPER="$ROOT_DIR/scripts/cir/cir-rsync-wrapper"
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--cir) CIR="$2"; shift 2 ;;
|
||||||
|
--static-root) STATIC_ROOT="$2"; shift 2 ;;
|
||||||
|
--raw-store-db) RAW_STORE_DB="$2"; shift 2 ;;
|
||||||
|
--out-dir) OUT_DIR="$2"; shift 2 ;;
|
||||||
|
--reference-ccr) REFERENCE_CCR="$2"; shift 2 ;;
|
||||||
|
--build-dir) BUILD_DIR="$2"; shift 2 ;;
|
||||||
|
--keep-db) KEEP_DB=1; shift ;;
|
||||||
|
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
[[ -n "$CIR" && -n "$OUT_DIR" && -n "$REFERENCE_CCR" && -n "$BUILD_DIR" ]] || {
|
||||||
|
usage >&2
|
||||||
|
exit 2
|
||||||
|
}
|
||||||
|
backend_count=0
|
||||||
|
[[ -n "$STATIC_ROOT" ]] && backend_count=$((backend_count+1))
|
||||||
|
[[ -n "$RAW_STORE_DB" ]] && backend_count=$((backend_count+1))
|
||||||
|
[[ "$backend_count" -eq 1 ]] || { usage >&2; exit 2; }
|
||||||
|
|
||||||
|
mkdir -p "$OUT_DIR"
|
||||||
|
if [[ ! -x "$CIR_MATERIALIZE_BIN" || ! -x "$CIR_EXTRACT_INPUTS_BIN" || ! -x "$CCR_TO_COMPARE_VIEWS_BIN" ]]; then
|
||||||
|
(
|
||||||
|
cd "$ROOT_DIR"
|
||||||
|
cargo build --release --bin cir_materialize --bin cir_extract_inputs --bin ccr_to_compare_views
|
||||||
|
)
|
||||||
|
fi
|
||||||
|
|
||||||
|
TMP_ROOT="$OUT_DIR/.tmp"
|
||||||
|
TALS_DIR="$TMP_ROOT/tals"
|
||||||
|
META_JSON="$TMP_ROOT/meta.json"
|
||||||
|
MIRROR_ROOT="$TMP_ROOT/mirror"
|
||||||
|
CACHE_DIR="$TMP_ROOT/cache"
|
||||||
|
OUT_CCR_DIR="$TMP_ROOT/out"
|
||||||
|
RUN_LOG="$OUT_DIR/rpki-client.log"
|
||||||
|
ACTUAL_VRPS="$OUT_DIR/actual-vrps.csv"
|
||||||
|
ACTUAL_VAPS="$OUT_DIR/actual-vaps.csv"
|
||||||
|
ACTUAL_VAPS_META="$OUT_DIR/actual-vaps-meta.json"
|
||||||
|
ACTUAL_VRPS_META="$OUT_DIR/actual-vrps-meta.json"
|
||||||
|
REF_VRPS="$OUT_DIR/reference-vrps.csv"
|
||||||
|
REF_VAPS="$OUT_DIR/reference-vaps.csv"
|
||||||
|
SUMMARY_JSON="$OUT_DIR/compare-summary.json"
|
||||||
|
|
||||||
|
rm -rf "$TMP_ROOT"
|
||||||
|
mkdir -p "$TMP_ROOT"
|
||||||
|
|
||||||
|
"$CIR_EXTRACT_INPUTS_BIN" --cir "$CIR" --tals-dir "$TALS_DIR" --meta-json "$META_JSON"
|
||||||
|
python3 - <<'PY' "$TALS_DIR"
|
||||||
|
from pathlib import Path
|
||||||
|
import sys
|
||||||
|
for tal in Path(sys.argv[1]).glob("*.tal"):
|
||||||
|
lines = tal.read_text(encoding="utf-8").splitlines()
|
||||||
|
rsync_uris = [line for line in lines if line.startswith("rsync://")]
|
||||||
|
base64_lines = []
|
||||||
|
seen_sep = False
|
||||||
|
for line in lines:
|
||||||
|
if seen_sep:
|
||||||
|
if line.strip():
|
||||||
|
base64_lines.append(line)
|
||||||
|
elif line.strip() == "":
|
||||||
|
seen_sep = True
|
||||||
|
tal.write_text("\n".join(rsync_uris) + "\n\n" + "\n".join(base64_lines) + "\n", encoding="utf-8")
|
||||||
|
PY
|
||||||
|
materialize_cmd=("$CIR_MATERIALIZE_BIN" --cir "$CIR" --mirror-root "$MIRROR_ROOT")
|
||||||
|
if [[ -n "$STATIC_ROOT" ]]; then
|
||||||
|
materialize_cmd+=(--static-root "$STATIC_ROOT")
|
||||||
|
else
|
||||||
|
materialize_cmd+=(--raw-store-db "$RAW_STORE_DB")
|
||||||
|
fi
|
||||||
|
if [[ "$KEEP_DB" -eq 1 ]]; then
|
||||||
|
materialize_cmd+=(--keep-db)
|
||||||
|
fi
|
||||||
|
"${materialize_cmd[@]}"
|
||||||
|
|
||||||
|
VALIDATION_EPOCH="$(python3 - <<'PY' "$META_JSON"
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
import json, sys
|
||||||
|
vt = json.load(open(sys.argv[1]))["validationTime"]
|
||||||
|
dt = datetime.fromisoformat(vt.replace("Z", "+00:00")).astimezone(timezone.utc)
|
||||||
|
print(int(dt.timestamp()))
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
FIRST_TAL="$(python3 - <<'PY' "$META_JSON"
|
||||||
|
import json,sys
|
||||||
|
print(json.load(open(sys.argv[1]))["talFiles"][0]["path"])
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
TA_NAME="$(basename "$FIRST_TAL" .tal)"
|
||||||
|
|
||||||
|
export CIR_MIRROR_ROOT="$(python3 - <<'PY' "$MIRROR_ROOT"
|
||||||
|
from pathlib import Path
|
||||||
|
import sys
|
||||||
|
print(Path(sys.argv[1]).resolve())
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
export REAL_RSYNC_BIN="$REAL_RSYNC_BIN"
|
||||||
|
export CIR_LOCAL_LINK_MODE=1
|
||||||
|
|
||||||
|
mkdir -p "$CACHE_DIR" "$OUT_CCR_DIR"
|
||||||
|
"$BUILD_DIR/src/rpki-client" \
|
||||||
|
-R \
|
||||||
|
-e "$WRAPPER" \
|
||||||
|
-P "$VALIDATION_EPOCH" \
|
||||||
|
-t "$FIRST_TAL" \
|
||||||
|
-d "$CACHE_DIR" \
|
||||||
|
"$OUT_CCR_DIR" >"$RUN_LOG" 2>&1
|
||||||
|
|
||||||
|
"$CCR_TO_COMPARE_VIEWS_BIN" \
|
||||||
|
--ccr "$OUT_CCR_DIR/rpki.ccr" \
|
||||||
|
--vrps-out "$ACTUAL_VRPS" \
|
||||||
|
--vaps-out "$ACTUAL_VAPS" \
|
||||||
|
--trust-anchor "$TA_NAME"
|
||||||
|
|
||||||
|
python3 - <<'PY' "$ACTUAL_VRPS" "$ACTUAL_VAPS" "$ACTUAL_VRPS_META" "$ACTUAL_VAPS_META"
|
||||||
|
import csv, json, sys
|
||||||
|
def count_rows(path):
|
||||||
|
with open(path, newline="") as f:
|
||||||
|
rows = list(csv.reader(f))
|
||||||
|
return max(len(rows) - 1, 0)
|
||||||
|
json.dump({"count": count_rows(sys.argv[1])}, open(sys.argv[3], "w"), indent=2)
|
||||||
|
json.dump({"count": count_rows(sys.argv[2])}, open(sys.argv[4], "w"), indent=2)
|
||||||
|
PY
|
||||||
|
|
||||||
|
"$CCR_TO_COMPARE_VIEWS_BIN" --ccr "$REFERENCE_CCR" --vrps-out "$REF_VRPS" --vaps-out "$REF_VAPS" --trust-anchor "$TA_NAME"
|
||||||
|
|
||||||
|
python3 - <<'PY' "$ACTUAL_VRPS" "$REF_VRPS" "$ACTUAL_VAPS" "$REF_VAPS" "$SUMMARY_JSON"
|
||||||
|
import csv, json, sys
|
||||||
|
def rows(path):
|
||||||
|
with open(path, newline="") as f:
|
||||||
|
return list(csv.reader(f))[1:]
|
||||||
|
actual_vrps = {tuple(r) for r in rows(sys.argv[1])}
|
||||||
|
ref_vrps = {tuple(r) for r in rows(sys.argv[2])}
|
||||||
|
actual_vaps = {tuple(r) for r in rows(sys.argv[3])}
|
||||||
|
ref_vaps = {tuple(r) for r in rows(sys.argv[4])}
|
||||||
|
summary = {
|
||||||
|
"vrps": {
|
||||||
|
"actual": len(actual_vrps),
|
||||||
|
"reference": len(ref_vrps),
|
||||||
|
"match": actual_vrps == ref_vrps,
|
||||||
|
"only_in_actual": sorted(actual_vrps - ref_vrps)[:20],
|
||||||
|
"only_in_reference": sorted(ref_vrps - actual_vrps)[:20],
|
||||||
|
},
|
||||||
|
"vaps": {
|
||||||
|
"actual": len(actual_vaps),
|
||||||
|
"reference": len(ref_vaps),
|
||||||
|
"match": actual_vaps == ref_vaps,
|
||||||
|
"only_in_actual": sorted(actual_vaps - ref_vaps)[:20],
|
||||||
|
"only_in_reference": sorted(ref_vaps - actual_vaps)[:20],
|
||||||
|
}
|
||||||
|
}
|
||||||
|
with open(sys.argv[5], "w") as f:
|
||||||
|
json.dump(summary, f, indent=2)
|
||||||
|
PY
|
||||||
|
|
||||||
|
if [[ "$KEEP_DB" -ne 1 ]]; then
|
||||||
|
rm -rf "$TMP_ROOT"
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "done: $OUT_DIR"
|
||||||
149
scripts/cir/run_cir_replay_sequence_ours.sh
Executable file
149
scripts/cir/run_cir_replay_sequence_ours.sh
Executable file
@@ -0,0 +1,149 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage:
|
||||||
|
./scripts/cir/run_cir_replay_sequence_ours.sh \
|
||||||
|
--sequence-root <path> \
|
||||||
|
[--rpki-bin <path>] \
|
||||||
|
[--real-rsync-bin <path>]
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
SEQUENCE_ROOT=""
|
||||||
|
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
||||||
|
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
||||||
|
STEP_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_ours.sh"
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--sequence-root) SEQUENCE_ROOT="$2"; shift 2 ;;
|
||||||
|
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
||||||
|
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
[[ -n "$SEQUENCE_ROOT" ]] || { usage >&2; exit 2; }
|
||||||
|
|
||||||
|
SEQUENCE_ROOT="$(python3 - <<'PY' "$SEQUENCE_ROOT"
|
||||||
|
from pathlib import Path
|
||||||
|
import sys
|
||||||
|
print(Path(sys.argv[1]).resolve())
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
|
||||||
|
SUMMARY_JSON="$SEQUENCE_ROOT/sequence-summary.json"
|
||||||
|
SUMMARY_MD="$SEQUENCE_ROOT/sequence-summary.md"
|
||||||
|
DETAIL_JSON="$SEQUENCE_ROOT/sequence-detail.json"
|
||||||
|
|
||||||
|
python3 - <<'PY' "$SEQUENCE_ROOT" "$SUMMARY_JSON" "$SUMMARY_MD" "$DETAIL_JSON" "$STEP_SCRIPT" "$RPKI_BIN" "$REAL_RSYNC_BIN"
|
||||||
|
import json
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
sequence_root = Path(sys.argv[1])
|
||||||
|
summary_json = Path(sys.argv[2])
|
||||||
|
summary_md = Path(sys.argv[3])
|
||||||
|
detail_json = Path(sys.argv[4])
|
||||||
|
step_script = Path(sys.argv[5])
|
||||||
|
rpki_bin = sys.argv[6]
|
||||||
|
real_rsync_bin = sys.argv[7]
|
||||||
|
|
||||||
|
sequence = json.loads((sequence_root / "sequence.json").read_text(encoding="utf-8"))
|
||||||
|
static_root = sequence_root / sequence["staticRoot"] if "staticRoot" in sequence else None
|
||||||
|
raw_store_db = sequence_root / sequence["rawStoreDbPath"] if "rawStoreDbPath" in sequence else None
|
||||||
|
backend_count = sum(x is not None for x in (static_root, raw_store_db))
|
||||||
|
if backend_count != 1:
|
||||||
|
raise SystemExit("sequence must set exactly one of staticRoot or rawStoreDbPath")
|
||||||
|
steps = sequence["steps"]
|
||||||
|
|
||||||
|
results = []
|
||||||
|
all_match = True
|
||||||
|
for step in steps:
|
||||||
|
step_id = step["stepId"]
|
||||||
|
out_dir = sequence_root / "replay-ours" / step_id
|
||||||
|
out_dir.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
cmd = [
|
||||||
|
str(step_script),
|
||||||
|
"--cir",
|
||||||
|
str(sequence_root / step["cirPath"]),
|
||||||
|
"--out-dir",
|
||||||
|
str(out_dir),
|
||||||
|
"--reference-ccr",
|
||||||
|
str(sequence_root / step["ccrPath"]),
|
||||||
|
"--rpki-bin",
|
||||||
|
rpki_bin,
|
||||||
|
"--real-rsync-bin",
|
||||||
|
real_rsync_bin,
|
||||||
|
]
|
||||||
|
if static_root is not None:
|
||||||
|
cmd.extend(["--static-root", str(static_root)])
|
||||||
|
else:
|
||||||
|
cmd.extend(["--raw-store-db", str(raw_store_db)])
|
||||||
|
proc = subprocess.run(cmd, capture_output=True, text=True)
|
||||||
|
if proc.returncode != 0:
|
||||||
|
raise SystemExit(
|
||||||
|
f"ours sequence replay failed for {step_id}: stdout={proc.stdout} stderr={proc.stderr}"
|
||||||
|
)
|
||||||
|
compare = json.loads((out_dir / "compare-summary.json").read_text(encoding="utf-8"))
|
||||||
|
timing = json.loads((out_dir / "timing.json").read_text(encoding="utf-8")) if (out_dir / "timing.json").exists() else {}
|
||||||
|
record = {
|
||||||
|
"stepId": step_id,
|
||||||
|
"kind": step["kind"],
|
||||||
|
"validationTime": step["validationTime"],
|
||||||
|
"outDir": str(out_dir),
|
||||||
|
"comparePath": str(out_dir / "compare-summary.json"),
|
||||||
|
"timingPath": str(out_dir / "timing.json"),
|
||||||
|
"compare": compare,
|
||||||
|
"timing": timing,
|
||||||
|
"match": bool(compare["vrps"]["match"]) and bool(compare["vaps"]["match"]),
|
||||||
|
}
|
||||||
|
all_match = all_match and record["match"]
|
||||||
|
results.append(record)
|
||||||
|
|
||||||
|
summary = {
|
||||||
|
"version": 1,
|
||||||
|
"participant": "ours",
|
||||||
|
"sequenceRoot": str(sequence_root),
|
||||||
|
"stepCount": len(results),
|
||||||
|
"allMatch": all_match,
|
||||||
|
"steps": results,
|
||||||
|
}
|
||||||
|
summary_json.write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
||||||
|
detail_json.write_text(json.dumps(results, indent=2), encoding="utf-8")
|
||||||
|
|
||||||
|
lines = [
|
||||||
|
"# Ours CIR Sequence Replay Summary",
|
||||||
|
"",
|
||||||
|
f"- `sequence_root`: `{sequence_root}`",
|
||||||
|
f"- `step_count`: `{len(results)}`",
|
||||||
|
f"- `all_match`: `{all_match}`",
|
||||||
|
"",
|
||||||
|
"| Step | Kind | VRP actual/ref | VRP match | VAP actual/ref | VAP match | Duration (ms) |",
|
||||||
|
"| --- | --- | --- | --- | --- | --- | ---: |",
|
||||||
|
]
|
||||||
|
for item in results:
|
||||||
|
compare = item["compare"]
|
||||||
|
timing = item.get("timing") or {}
|
||||||
|
lines.append(
|
||||||
|
"| {step} | {kind} | {va}/{vr} | {vm} | {aa}/{ar} | {am} | {dur} |".format(
|
||||||
|
step=item["stepId"],
|
||||||
|
kind=item["kind"],
|
||||||
|
va=compare["vrps"]["actual"],
|
||||||
|
vr=compare["vrps"]["reference"],
|
||||||
|
vm=compare["vrps"]["match"],
|
||||||
|
aa=compare["vaps"]["actual"],
|
||||||
|
ar=compare["vaps"]["reference"],
|
||||||
|
am=compare["vaps"]["match"],
|
||||||
|
dur=timing.get("durationMs", "-"),
|
||||||
|
)
|
||||||
|
)
|
||||||
|
summary_md.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||||
|
PY
|
||||||
|
|
||||||
|
echo "done: $SEQUENCE_ROOT"
|
||||||
132
scripts/cir/run_cir_replay_sequence_routinator.sh
Executable file
132
scripts/cir/run_cir_replay_sequence_routinator.sh
Executable file
@@ -0,0 +1,132 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage:
|
||||||
|
./scripts/cir/run_cir_replay_sequence_routinator.sh \
|
||||||
|
--sequence-root <path> \
|
||||||
|
[--routinator-root <path>] \
|
||||||
|
[--routinator-bin <path>] \
|
||||||
|
[--real-rsync-bin <path>]
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
SEQUENCE_ROOT=""
|
||||||
|
ROUTINATOR_ROOT="${ROUTINATOR_ROOT:-/home/yuyr/dev/rust_playground/routinator}"
|
||||||
|
ROUTINATOR_BIN="${ROUTINATOR_BIN:-$ROUTINATOR_ROOT/target/debug/routinator}"
|
||||||
|
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
||||||
|
STEP_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_routinator.sh"
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--sequence-root) SEQUENCE_ROOT="$2"; shift 2 ;;
|
||||||
|
--routinator-root) ROUTINATOR_ROOT="$2"; shift 2 ;;
|
||||||
|
--routinator-bin) ROUTINATOR_BIN="$2"; shift 2 ;;
|
||||||
|
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
[[ -n "$SEQUENCE_ROOT" ]] || { usage >&2; exit 2; }
|
||||||
|
|
||||||
|
SEQUENCE_ROOT="$(python3 - <<'PY' "$SEQUENCE_ROOT"
|
||||||
|
from pathlib import Path
|
||||||
|
import sys
|
||||||
|
print(Path(sys.argv[1]).resolve())
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
|
||||||
|
SUMMARY_JSON="$SEQUENCE_ROOT/sequence-summary-routinator.json"
|
||||||
|
SUMMARY_MD="$SEQUENCE_ROOT/sequence-summary-routinator.md"
|
||||||
|
|
||||||
|
python3 - <<'PY' "$SEQUENCE_ROOT" "$SUMMARY_JSON" "$SUMMARY_MD" "$STEP_SCRIPT" "$ROUTINATOR_ROOT" "$ROUTINATOR_BIN" "$REAL_RSYNC_BIN"
|
||||||
|
import json
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
sequence_root = Path(sys.argv[1])
|
||||||
|
summary_json = Path(sys.argv[2])
|
||||||
|
summary_md = Path(sys.argv[3])
|
||||||
|
step_script = Path(sys.argv[4])
|
||||||
|
routinator_root = sys.argv[5]
|
||||||
|
routinator_bin = sys.argv[6]
|
||||||
|
real_rsync_bin = sys.argv[7]
|
||||||
|
|
||||||
|
sequence = json.loads((sequence_root / "sequence.json").read_text(encoding="utf-8"))
|
||||||
|
static_root = sequence_root / sequence["staticRoot"] if "staticRoot" in sequence else None
|
||||||
|
raw_store_db = sequence_root / sequence["rawStoreDbPath"] if "rawStoreDbPath" in sequence else None
|
||||||
|
backend_count = sum(x is not None for x in (static_root, raw_store_db))
|
||||||
|
if backend_count != 1:
|
||||||
|
raise SystemExit("sequence must set exactly one of staticRoot or rawStoreDbPath")
|
||||||
|
steps = sequence["steps"]
|
||||||
|
results = []
|
||||||
|
all_match = True
|
||||||
|
|
||||||
|
for step in steps:
|
||||||
|
step_id = step["stepId"]
|
||||||
|
out_dir = sequence_root / "replay-routinator" / step_id
|
||||||
|
out_dir.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
cmd = [
|
||||||
|
str(step_script),
|
||||||
|
"--cir",
|
||||||
|
str(sequence_root / step["cirPath"]),
|
||||||
|
"--out-dir",
|
||||||
|
str(out_dir),
|
||||||
|
"--reference-ccr",
|
||||||
|
str(sequence_root / step["ccrPath"]),
|
||||||
|
"--routinator-root",
|
||||||
|
routinator_root,
|
||||||
|
"--routinator-bin",
|
||||||
|
routinator_bin,
|
||||||
|
"--real-rsync-bin",
|
||||||
|
real_rsync_bin,
|
||||||
|
]
|
||||||
|
if static_root is not None:
|
||||||
|
cmd.extend(["--static-root", str(static_root)])
|
||||||
|
else:
|
||||||
|
cmd.extend(["--raw-store-db", str(raw_store_db)])
|
||||||
|
proc = subprocess.run(cmd, capture_output=True, text=True)
|
||||||
|
if proc.returncode != 0:
|
||||||
|
raise SystemExit(
|
||||||
|
f"routinator sequence replay failed for {step_id}: stdout={proc.stdout} stderr={proc.stderr}"
|
||||||
|
)
|
||||||
|
compare = json.loads((out_dir / "compare-summary.json").read_text(encoding="utf-8"))
|
||||||
|
match = bool(compare["vrps"]["match"]) and bool(compare["vaps"]["match"])
|
||||||
|
all_match = all_match and match
|
||||||
|
results.append(
|
||||||
|
{
|
||||||
|
"stepId": step_id,
|
||||||
|
"kind": step["kind"],
|
||||||
|
"validationTime": step["validationTime"],
|
||||||
|
"outDir": str(out_dir),
|
||||||
|
"comparePath": str(out_dir / "compare-summary.json"),
|
||||||
|
"match": match,
|
||||||
|
"compare": compare,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
summary = {
|
||||||
|
"version": 1,
|
||||||
|
"participant": "routinator",
|
||||||
|
"sequenceRoot": str(sequence_root),
|
||||||
|
"stepCount": len(results),
|
||||||
|
"allMatch": all_match,
|
||||||
|
"steps": results,
|
||||||
|
}
|
||||||
|
summary_json.write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
||||||
|
lines = [
|
||||||
|
"# Routinator CIR Sequence Replay Summary",
|
||||||
|
"",
|
||||||
|
f"- `sequence_root`: `{sequence_root}`",
|
||||||
|
f"- `step_count`: `{len(results)}`",
|
||||||
|
f"- `all_match`: `{all_match}`",
|
||||||
|
"",
|
||||||
|
]
|
||||||
|
summary_md.write_text("\n".join(lines), encoding="utf-8")
|
||||||
|
PY
|
||||||
|
|
||||||
|
echo "done: $SEQUENCE_ROOT"
|
||||||
126
scripts/cir/run_cir_replay_sequence_rpki_client.sh
Executable file
126
scripts/cir/run_cir_replay_sequence_rpki_client.sh
Executable file
@@ -0,0 +1,126 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage:
|
||||||
|
./scripts/cir/run_cir_replay_sequence_rpki_client.sh \
|
||||||
|
--sequence-root <path> \
|
||||||
|
--build-dir <path> \
|
||||||
|
[--real-rsync-bin <path>]
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
SEQUENCE_ROOT=""
|
||||||
|
BUILD_DIR=""
|
||||||
|
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
||||||
|
STEP_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_rpki_client.sh"
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--sequence-root) SEQUENCE_ROOT="$2"; shift 2 ;;
|
||||||
|
--build-dir) BUILD_DIR="$2"; shift 2 ;;
|
||||||
|
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
[[ -n "$SEQUENCE_ROOT" && -n "$BUILD_DIR" ]] || { usage >&2; exit 2; }
|
||||||
|
|
||||||
|
SEQUENCE_ROOT="$(python3 - <<'PY' "$SEQUENCE_ROOT"
|
||||||
|
from pathlib import Path
|
||||||
|
import sys
|
||||||
|
print(Path(sys.argv[1]).resolve())
|
||||||
|
PY
|
||||||
|
)"
|
||||||
|
|
||||||
|
SUMMARY_JSON="$SEQUENCE_ROOT/sequence-summary-rpki-client.json"
|
||||||
|
SUMMARY_MD="$SEQUENCE_ROOT/sequence-summary-rpki-client.md"
|
||||||
|
|
||||||
|
python3 - <<'PY' "$SEQUENCE_ROOT" "$SUMMARY_JSON" "$SUMMARY_MD" "$STEP_SCRIPT" "$BUILD_DIR" "$REAL_RSYNC_BIN"
|
||||||
|
import json
|
||||||
|
import subprocess
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
sequence_root = Path(sys.argv[1])
|
||||||
|
summary_json = Path(sys.argv[2])
|
||||||
|
summary_md = Path(sys.argv[3])
|
||||||
|
step_script = Path(sys.argv[4])
|
||||||
|
build_dir = sys.argv[5]
|
||||||
|
real_rsync_bin = sys.argv[6]
|
||||||
|
|
||||||
|
sequence = json.loads((sequence_root / "sequence.json").read_text(encoding="utf-8"))
|
||||||
|
static_root = sequence_root / sequence["staticRoot"] if "staticRoot" in sequence else None
|
||||||
|
raw_store_db = sequence_root / sequence["rawStoreDbPath"] if "rawStoreDbPath" in sequence else None
|
||||||
|
backend_count = sum(x is not None for x in (static_root, raw_store_db))
|
||||||
|
if backend_count != 1:
|
||||||
|
raise SystemExit("sequence must set exactly one of staticRoot or rawStoreDbPath")
|
||||||
|
steps = sequence["steps"]
|
||||||
|
results = []
|
||||||
|
all_match = True
|
||||||
|
|
||||||
|
for step in steps:
|
||||||
|
step_id = step["stepId"]
|
||||||
|
out_dir = sequence_root / "replay-rpki-client" / step_id
|
||||||
|
out_dir.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
cmd = [
|
||||||
|
str(step_script),
|
||||||
|
"--cir",
|
||||||
|
str(sequence_root / step["cirPath"]),
|
||||||
|
"--out-dir",
|
||||||
|
str(out_dir),
|
||||||
|
"--reference-ccr",
|
||||||
|
str(sequence_root / step["ccrPath"]),
|
||||||
|
"--build-dir",
|
||||||
|
build_dir,
|
||||||
|
"--real-rsync-bin",
|
||||||
|
real_rsync_bin,
|
||||||
|
]
|
||||||
|
if static_root is not None:
|
||||||
|
cmd.extend(["--static-root", str(static_root)])
|
||||||
|
else:
|
||||||
|
cmd.extend(["--raw-store-db", str(raw_store_db)])
|
||||||
|
proc = subprocess.run(cmd, capture_output=True, text=True)
|
||||||
|
if proc.returncode != 0:
|
||||||
|
raise SystemExit(
|
||||||
|
f"rpki-client sequence replay failed for {step_id}: stdout={proc.stdout} stderr={proc.stderr}"
|
||||||
|
)
|
||||||
|
compare = json.loads((out_dir / "compare-summary.json").read_text(encoding="utf-8"))
|
||||||
|
match = bool(compare["vrps"]["match"]) and bool(compare["vaps"]["match"])
|
||||||
|
all_match = all_match and match
|
||||||
|
results.append(
|
||||||
|
{
|
||||||
|
"stepId": step_id,
|
||||||
|
"kind": step["kind"],
|
||||||
|
"validationTime": step["validationTime"],
|
||||||
|
"outDir": str(out_dir),
|
||||||
|
"comparePath": str(out_dir / "compare-summary.json"),
|
||||||
|
"match": match,
|
||||||
|
"compare": compare,
|
||||||
|
}
|
||||||
|
)
|
||||||
|
|
||||||
|
summary = {
|
||||||
|
"version": 1,
|
||||||
|
"participant": "rpki-client",
|
||||||
|
"sequenceRoot": str(sequence_root),
|
||||||
|
"stepCount": len(results),
|
||||||
|
"allMatch": all_match,
|
||||||
|
"steps": results,
|
||||||
|
}
|
||||||
|
summary_json.write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
||||||
|
lines = [
|
||||||
|
"# rpki-client CIR Sequence Replay Summary",
|
||||||
|
"",
|
||||||
|
f"- `sequence_root`: `{sequence_root}`",
|
||||||
|
f"- `step_count`: `{len(results)}`",
|
||||||
|
f"- `all_match`: `{all_match}`",
|
||||||
|
"",
|
||||||
|
]
|
||||||
|
summary_md.write_text("\n".join(lines), encoding="utf-8")
|
||||||
|
PY
|
||||||
|
|
||||||
|
echo "done: $SEQUENCE_ROOT"
|
||||||
132
scripts/cir/run_cir_sequence_matrix_multi_rir.sh
Executable file
132
scripts/cir/run_cir_sequence_matrix_multi_rir.sh
Executable file
@@ -0,0 +1,132 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage:
|
||||||
|
./scripts/cir/run_cir_sequence_matrix_multi_rir.sh \
|
||||||
|
--root <path> \
|
||||||
|
[--rir <afrinic,apnic,arin,lacnic,ripe>] \
|
||||||
|
[--rpki-bin <path>] \
|
||||||
|
[--routinator-root <path>] \
|
||||||
|
[--routinator-bin <path>] \
|
||||||
|
[--rpki-client-build-dir <path>] \
|
||||||
|
[--drop-bin <path>]
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
ROOT=""
|
||||||
|
RIRS="afrinic,apnic,arin,lacnic,ripe"
|
||||||
|
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
||||||
|
ROUTINATOR_ROOT="${ROUTINATOR_ROOT:-/home/yuyr/dev/rust_playground/routinator}"
|
||||||
|
ROUTINATOR_BIN="${ROUTINATOR_BIN:-$ROUTINATOR_ROOT/target/debug/routinator}"
|
||||||
|
RPKI_CLIENT_BUILD_DIR="${RPKI_CLIENT_BUILD_DIR:-/home/yuyr/dev/rpki-client-9.7/build-m5}"
|
||||||
|
DROP_BIN="${DROP_BIN:-$ROOT_DIR/target/release/cir_drop_report}"
|
||||||
|
|
||||||
|
OURS_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_sequence_ours.sh"
|
||||||
|
ROUTINATOR_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_sequence_routinator.sh"
|
||||||
|
RPKIC_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_sequence_rpki_client.sh"
|
||||||
|
DROP_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_drop_sequence.sh"
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--root) ROOT="$2"; shift 2 ;;
|
||||||
|
--rir) RIRS="$2"; shift 2 ;;
|
||||||
|
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
||||||
|
--routinator-root) ROUTINATOR_ROOT="$2"; shift 2 ;;
|
||||||
|
--routinator-bin) ROUTINATOR_BIN="$2"; shift 2 ;;
|
||||||
|
--rpki-client-build-dir) RPKI_CLIENT_BUILD_DIR="$2"; shift 2 ;;
|
||||||
|
--drop-bin) DROP_BIN="$2"; shift 2 ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
[[ -n "$ROOT" ]] || { usage >&2; exit 2; }
|
||||||
|
|
||||||
|
SUMMARY_JSON="$ROOT/final-summary.json"
|
||||||
|
SUMMARY_MD="$ROOT/final-summary.md"
|
||||||
|
IFS=',' read -r -a ITEMS <<< "$RIRS"
|
||||||
|
|
||||||
|
results=()
|
||||||
|
for rir in "${ITEMS[@]}"; do
|
||||||
|
seq_root="$ROOT/$rir"
|
||||||
|
"$OURS_SCRIPT" --sequence-root "$seq_root" --rpki-bin "$RPKI_BIN"
|
||||||
|
"$ROUTINATOR_SCRIPT" --sequence-root "$seq_root" --routinator-root "$ROUTINATOR_ROOT" --routinator-bin "$ROUTINATOR_BIN"
|
||||||
|
"$RPKIC_SCRIPT" --sequence-root "$seq_root" --build-dir "$RPKI_CLIENT_BUILD_DIR"
|
||||||
|
"$DROP_SCRIPT" --sequence-root "$seq_root" --drop-bin "$DROP_BIN"
|
||||||
|
done
|
||||||
|
|
||||||
|
python3 - <<'PY' "$ROOT" "$RIRS" "$SUMMARY_JSON" "$SUMMARY_MD"
|
||||||
|
import json, sys
|
||||||
|
from pathlib import Path
|
||||||
|
from collections import Counter
|
||||||
|
|
||||||
|
root = Path(sys.argv[1]).resolve()
|
||||||
|
rirs = [item for item in sys.argv[2].split(',') if item]
|
||||||
|
summary_json = Path(sys.argv[3])
|
||||||
|
summary_md = Path(sys.argv[4])
|
||||||
|
items = []
|
||||||
|
total_steps = 0
|
||||||
|
total_dropped_vrps = 0
|
||||||
|
total_dropped_objects = 0
|
||||||
|
reason_counter = Counter()
|
||||||
|
for rir in rirs:
|
||||||
|
seq_root = root / rir
|
||||||
|
ours = json.loads((seq_root / "sequence-summary.json").read_text(encoding="utf-8"))
|
||||||
|
routinator = json.loads((seq_root / "sequence-summary-routinator.json").read_text(encoding="utf-8"))
|
||||||
|
rpki_client = json.loads((seq_root / "sequence-summary-rpki-client.json").read_text(encoding="utf-8"))
|
||||||
|
drop = json.loads((seq_root / "drop-summary.json").read_text(encoding="utf-8"))
|
||||||
|
step_count = len(ours["steps"])
|
||||||
|
total_steps += step_count
|
||||||
|
rir_dropped_vrps = 0
|
||||||
|
rir_dropped_objects = 0
|
||||||
|
for step in drop["steps"]:
|
||||||
|
drop_path = Path(step["reportPath"])
|
||||||
|
detail = json.loads(drop_path.read_text(encoding="utf-8"))
|
||||||
|
summary = detail.get("summary", {})
|
||||||
|
rir_dropped_vrps += int(summary.get("droppedVrpCount", 0))
|
||||||
|
rir_dropped_objects += int(summary.get("droppedObjectCount", 0))
|
||||||
|
total_dropped_vrps += int(summary.get("droppedVrpCount", 0))
|
||||||
|
total_dropped_objects += int(summary.get("droppedObjectCount", 0))
|
||||||
|
for reason, count in summary.get("droppedByReason", {}).items():
|
||||||
|
reason_counter[reason] += int(count)
|
||||||
|
items.append({
|
||||||
|
"rir": rir,
|
||||||
|
"stepCount": step_count,
|
||||||
|
"oursAllMatch": ours["allMatch"],
|
||||||
|
"routinatorAllMatch": routinator["allMatch"],
|
||||||
|
"rpkiClientAllMatch": rpki_client["allMatch"],
|
||||||
|
"dropSummary": drop["steps"],
|
||||||
|
"droppedVrpCount": rir_dropped_vrps,
|
||||||
|
"droppedObjectCount": rir_dropped_objects,
|
||||||
|
})
|
||||||
|
summary = {
|
||||||
|
"version": 1,
|
||||||
|
"totalStepCount": total_steps,
|
||||||
|
"totalDroppedVrpCount": total_dropped_vrps,
|
||||||
|
"totalDroppedObjectCount": total_dropped_objects,
|
||||||
|
"topReasons": [{"reason": reason, "count": count} for reason, count in reason_counter.most_common(10)],
|
||||||
|
"rirs": items,
|
||||||
|
}
|
||||||
|
summary_json.write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
||||||
|
lines = ["# Multi-RIR CIR Sequence Matrix Summary", ""]
|
||||||
|
lines.append(f"- `total_step_count`: `{total_steps}`")
|
||||||
|
lines.append(f"- `total_dropped_vrps`: `{total_dropped_vrps}`")
|
||||||
|
lines.append(f"- `total_dropped_objects`: `{total_dropped_objects}`")
|
||||||
|
lines.append("")
|
||||||
|
if reason_counter:
|
||||||
|
lines.append("## Top Drop Reasons")
|
||||||
|
lines.append("")
|
||||||
|
for reason, count in reason_counter.most_common(10):
|
||||||
|
lines.append(f"- `{reason}`: `{count}`")
|
||||||
|
lines.append("")
|
||||||
|
for item in items:
|
||||||
|
lines.append(
|
||||||
|
f"- `{item['rir']}`: `steps={item['stepCount']}` `ours={item['oursAllMatch']}` `routinator={item['routinatorAllMatch']}` `rpki-client={item['rpkiClientAllMatch']}` `drop_vrps={item['droppedVrpCount']}` `drop_objects={item['droppedObjectCount']}`"
|
||||||
|
)
|
||||||
|
summary_md.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
||||||
|
PY
|
||||||
|
|
||||||
|
echo "done: $ROOT"
|
||||||
82
scripts/coverage.sh
Executable file
82
scripts/coverage.sh
Executable file
@@ -0,0 +1,82 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
# Requires:
|
||||||
|
# rustup component add llvm-tools-preview
|
||||||
|
# cargo install cargo-llvm-cov --locked
|
||||||
|
|
||||||
|
run_out="$(mktemp)"
|
||||||
|
text_out="$(mktemp)"
|
||||||
|
html_out="$(mktemp)"
|
||||||
|
|
||||||
|
cleanup() {
|
||||||
|
rm -f "$run_out" "$text_out" "$html_out"
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
|
||||||
|
IGNORE_REGEX='src/bin/replay_bundle_capture\.rs|src/bin/replay_bundle_capture_delta\.rs|src/bin/replay_bundle_capture_sequence\.rs|src/bin/replay_bundle_record\.rs|src/bin/replay_bundle_refresh_sequence_outputs\.rs|src/bin/measure_sequence_replay\.rs|src/bin/repository_view_stats\.rs|src/bin/trace_arin_missing_vrps\.rs|src/bin/db_stats\.rs|src/bin/rrdp_state_dump\.rs|src/bin/ccr_dump\.rs|src/bin/ccr_verify\.rs|src/bin/ccr_to_routinator_csv\.rs|src/bin/ccr_to_compare_views\.rs|src/bin/cir_materialize\.rs|src/bin/cir_extract_inputs\.rs|src/bin/cir_drop_report\.rs|src/bin/cir_ta_only_fixture\.rs|src/bundle/live_capture\.rs|src/bundle/record_io\.rs|src/bundle/compare_view\.rs|src/progress_log\.rs|src/cli\.rs|src/validation/run_tree_from_tal\.rs|src/validation/from_tal\.rs|src/sync/store_projection\.rs|src/cir/materialize\.rs'
|
||||||
|
|
||||||
|
# Preserve colored output even though we post-process output by running under a pseudo-TTY.
|
||||||
|
# We run tests only once, then generate both CLI text + HTML reports without rerunning tests.
|
||||||
|
set +e
|
||||||
|
|
||||||
|
cargo llvm-cov clean --workspace >/dev/null 2>&1
|
||||||
|
|
||||||
|
# 1) Run tests once to collect coverage data (no report).
|
||||||
|
script -q -e -c "CARGO_TERM_COLOR=always cargo llvm-cov --no-report" "$run_out" >/dev/null 2>&1
|
||||||
|
run_status="$?"
|
||||||
|
|
||||||
|
# 2) CLI summary report + fail-under gate (no test rerun).
|
||||||
|
script -q -e -c "CARGO_TERM_COLOR=always cargo llvm-cov report --fail-under-lines 90 --ignore-filename-regex '$IGNORE_REGEX'" "$text_out" >/dev/null 2>&1
|
||||||
|
text_status="$?"
|
||||||
|
|
||||||
|
# 3) HTML report (no test rerun).
|
||||||
|
script -q -e -c "CARGO_TERM_COLOR=always cargo llvm-cov report --html --ignore-filename-regex '$IGNORE_REGEX'" "$html_out" >/dev/null 2>&1
|
||||||
|
html_status="$?"
|
||||||
|
|
||||||
|
set -e
|
||||||
|
|
||||||
|
strip_script_noise() {
|
||||||
|
tr -d '\r' | sed '/^Script \(started\|done\) on /d'
|
||||||
|
}
|
||||||
|
|
||||||
|
strip_ansi_for_parse() {
|
||||||
|
awk '
|
||||||
|
{
|
||||||
|
line = $0
|
||||||
|
gsub(/\033\[[0-9;]*[A-Za-z]/, "", line) # CSI escapes
|
||||||
|
gsub(/\033\([A-Za-z]/, "", line) # charset escapes (e.g., ESC(B)
|
||||||
|
gsub(/\r/, "", line)
|
||||||
|
print line
|
||||||
|
}
|
||||||
|
'
|
||||||
|
}
|
||||||
|
|
||||||
|
cat "$run_out" | strip_script_noise
|
||||||
|
cat "$text_out" | strip_script_noise
|
||||||
|
cat "$html_out" | strip_script_noise
|
||||||
|
|
||||||
|
cat "$run_out" | strip_ansi_for_parse | awk '
|
||||||
|
BEGIN {
|
||||||
|
passed=0; failed=0; ignored=0; measured=0; filtered=0;
|
||||||
|
}
|
||||||
|
/^test result: / {
|
||||||
|
if (match($0, /([0-9]+) passed; ([0-9]+) failed; ([0-9]+) ignored; ([0-9]+) measured; ([0-9]+) filtered out;/, m)) {
|
||||||
|
passed += m[1]; failed += m[2]; ignored += m[3]; measured += m[4]; filtered += m[5];
|
||||||
|
}
|
||||||
|
}
|
||||||
|
END {
|
||||||
|
executed = passed + failed;
|
||||||
|
total = passed + failed + ignored + measured;
|
||||||
|
printf("\nTEST SUMMARY (all suites): passed=%d failed=%d ignored=%d measured=%d filtered_out=%d executed=%d total=%d\n",
|
||||||
|
passed, failed, ignored, measured, filtered, executed, total);
|
||||||
|
}
|
||||||
|
'
|
||||||
|
|
||||||
|
echo
|
||||||
|
echo "HTML report: target/llvm-cov/html/index.html"
|
||||||
|
|
||||||
|
status="$text_status"
|
||||||
|
if [ "$run_status" -ne 0 ]; then status="$run_status"; fi
|
||||||
|
if [ "$html_status" -ne 0 ]; then status="$html_status"; fi
|
||||||
|
exit "$status"
|
||||||
195
scripts/manifest_perf_compare_m2.sh
Executable file
195
scripts/manifest_perf_compare_m2.sh
Executable file
@@ -0,0 +1,195 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
# M2: Run per-sample decode+profile benchmark (Ours vs Routinator) on selected_der fixtures.
|
||||||
|
#
|
||||||
|
# Outputs:
|
||||||
|
# - specs/develop/20260224/data/m2_manifest_decode_profile_compare.csv
|
||||||
|
# - specs/develop/20260224/data/m2_raw.log
|
||||||
|
#
|
||||||
|
# Note: This script assumes Routinator benchmark repo exists at:
|
||||||
|
# /home/yuyr/dev/rust_playground/routinator/benchmark
|
||||||
|
#
|
||||||
|
# It also assumes fixtures exist under:
|
||||||
|
# rpki/tests/benchmark/selected_der/*.mft
|
||||||
|
# routinator/benchmark/fixtures/selected_der/*.mft
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||||
|
RPKI_DIR="$ROOT_DIR"
|
||||||
|
OURS_BENCH_DIR="$RPKI_DIR/benchmark/ours_manifest_bench"
|
||||||
|
|
||||||
|
ROUT_BENCH_DIR="${ROUT_BENCH_DIR:-/home/yuyr/dev/rust_playground/routinator/benchmark}"
|
||||||
|
ROUT_BIN="$ROUT_BENCH_DIR/target/release/routinator-manifest-benchmark"
|
||||||
|
|
||||||
|
DATE_TAG="${DATE_TAG:-20260224}"
|
||||||
|
OUT_DIR="$RPKI_DIR/../specs/develop/${DATE_TAG}/data"
|
||||||
|
OUT_CSV="${OUT_CSV:-$OUT_DIR/m2_manifest_decode_profile_compare.csv}"
|
||||||
|
OUT_RAW="${OUT_RAW:-$OUT_DIR/m2_raw.log}"
|
||||||
|
|
||||||
|
REPEATS="${REPEATS:-3}"
|
||||||
|
|
||||||
|
# Iterations / warmups (kept moderate for interactive iteration).
|
||||||
|
ITER_SMALL="${ITER_SMALL:-20000}"
|
||||||
|
ITER_MEDIUM="${ITER_MEDIUM:-20000}"
|
||||||
|
ITER_LARGE="${ITER_LARGE:-20000}"
|
||||||
|
ITER_XLARGE="${ITER_XLARGE:-2000}"
|
||||||
|
|
||||||
|
WARM_SMALL="${WARM_SMALL:-2000}"
|
||||||
|
WARM_MEDIUM="${WARM_MEDIUM:-2000}"
|
||||||
|
WARM_LARGE="${WARM_LARGE:-2000}"
|
||||||
|
WARM_XLARGE="${WARM_XLARGE:-200}"
|
||||||
|
|
||||||
|
SAMPLES=(
|
||||||
|
small-01
|
||||||
|
small-02
|
||||||
|
medium-01
|
||||||
|
medium-02
|
||||||
|
large-01
|
||||||
|
large-02
|
||||||
|
xlarge-01
|
||||||
|
xlarge-02
|
||||||
|
)
|
||||||
|
|
||||||
|
mkdir -p "$OUT_DIR"
|
||||||
|
: > "$OUT_RAW"
|
||||||
|
|
||||||
|
echo "sample,bucket,manifest_file_count,ours_avg_ns_per_op,ours_ops_per_s,rout_avg_ns_per_op,rout_ops_per_s,ratio_ours_over_rout,iterations,repeats,warmup" > "$OUT_CSV"
|
||||||
|
|
||||||
|
echo "[1/3] Build ours benchmark (release)..." | tee -a "$OUT_RAW"
|
||||||
|
(cd "$OURS_BENCH_DIR" && cargo build --release -q)
|
||||||
|
OURS_BIN="$OURS_BENCH_DIR/target/release/ours-manifest-bench"
|
||||||
|
|
||||||
|
echo "[2/3] Build routinator benchmark (release)..." | tee -a "$OUT_RAW"
|
||||||
|
(cd "$ROUT_BENCH_DIR" && cargo build --release -q)
|
||||||
|
|
||||||
|
taskset_prefix=""
|
||||||
|
if command -v taskset >/dev/null 2>&1; then
|
||||||
|
if [[ -n "${TASKSET_CPU:-}" ]]; then
|
||||||
|
taskset_prefix="taskset -c ${TASKSET_CPU}"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
bucket_for() {
|
||||||
|
local s="$1"
|
||||||
|
case "$s" in
|
||||||
|
small-*) echo "small" ;;
|
||||||
|
medium-*) echo "medium" ;;
|
||||||
|
large-*) echo "large" ;;
|
||||||
|
xlarge-*) echo "xlarge" ;;
|
||||||
|
*) echo "unknown" ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
iters_for() {
|
||||||
|
local b="$1"
|
||||||
|
case "$b" in
|
||||||
|
small) echo "$ITER_SMALL" ;;
|
||||||
|
medium) echo "$ITER_MEDIUM" ;;
|
||||||
|
large) echo "$ITER_LARGE" ;;
|
||||||
|
xlarge) echo "$ITER_XLARGE" ;;
|
||||||
|
*) echo "$ITER_MEDIUM" ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
warm_for() {
|
||||||
|
local b="$1"
|
||||||
|
case "$b" in
|
||||||
|
small) echo "$WARM_SMALL" ;;
|
||||||
|
medium) echo "$WARM_MEDIUM" ;;
|
||||||
|
large) echo "$WARM_LARGE" ;;
|
||||||
|
xlarge) echo "$WARM_XLARGE" ;;
|
||||||
|
*) echo "$WARM_MEDIUM" ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
run_ours() {
|
||||||
|
local sample="$1"
|
||||||
|
local iters="$2"
|
||||||
|
local warm="$3"
|
||||||
|
local ours_fixture="$RPKI_DIR/tests/benchmark/selected_der/${sample}.mft"
|
||||||
|
if [[ ! -f "$ours_fixture" ]]; then
|
||||||
|
echo "ours fixture not found: $ours_fixture" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "### ours $sample" >> "$OUT_RAW"
|
||||||
|
local out
|
||||||
|
out=$($taskset_prefix "$OURS_BIN" --manifest "$ours_fixture" --iterations "$iters" --warmup-iterations "$warm" --repeats "$REPEATS")
|
||||||
|
echo "$out" >> "$OUT_RAW"
|
||||||
|
|
||||||
|
local line
|
||||||
|
line=$(echo "$out" | rg "^\\| ${sample} \\|" | tail -n 1)
|
||||||
|
if [[ -z "${line:-}" ]]; then
|
||||||
|
echo "failed to parse ours output for $sample" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
# Expected final row: | sample | avg ns/op | ops/s | file count |
|
||||||
|
local avg ops cnt
|
||||||
|
avg=$(echo "$line" | awk -F'|' '{gsub(/^ +| +$/,"",$3); print $3}')
|
||||||
|
ops=$(echo "$line" | awk -F'|' '{gsub(/^ +| +$/,"",$4); print $4}')
|
||||||
|
cnt=$(echo "$line" | awk -F'|' '{gsub(/^ +| +$/,"",$5); print $5}')
|
||||||
|
echo "$avg,$ops,$cnt"
|
||||||
|
}
|
||||||
|
|
||||||
|
run_rout() {
|
||||||
|
local sample="$1"
|
||||||
|
local iters="$2"
|
||||||
|
local warm="$3"
|
||||||
|
local rout_fixture="$ROUT_BENCH_DIR/fixtures/selected_der/${sample}.mft"
|
||||||
|
if [[ ! -f "$rout_fixture" ]]; then
|
||||||
|
echo "routinator fixture not found: $rout_fixture" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "### routinator $sample" >> "$OUT_RAW"
|
||||||
|
local out
|
||||||
|
out=$(
|
||||||
|
$taskset_prefix "$ROUT_BIN" \
|
||||||
|
--target decode_only \
|
||||||
|
--manifest "$rout_fixture" \
|
||||||
|
--issuer "$ROUT_BENCH_DIR/fixtures/ta.cer" \
|
||||||
|
--iterations "$iters" \
|
||||||
|
--repeats "$REPEATS" \
|
||||||
|
--warmup-iterations "$warm" \
|
||||||
|
--strict false
|
||||||
|
)
|
||||||
|
echo "$out" >> "$OUT_RAW"
|
||||||
|
|
||||||
|
local avg_line cnt_line
|
||||||
|
avg_line=$(echo "$out" | rg "^ avg:")
|
||||||
|
cnt_line=$(echo "$out" | rg "^ manifest_file_count:")
|
||||||
|
|
||||||
|
local avg_ns ops_s cnt
|
||||||
|
avg_ns=$(echo "$avg_line" | awk '{print $2}')
|
||||||
|
ops_s=$(echo "$avg_line" | awk '{gsub(/[()]/,"",$4); print $4}')
|
||||||
|
cnt=$(echo "$cnt_line" | awk '{print $2}')
|
||||||
|
echo "$avg_ns,$ops_s,$cnt"
|
||||||
|
}
|
||||||
|
|
||||||
|
echo "[3/3] Run per-sample benchmarks..." | tee -a "$OUT_RAW"
|
||||||
|
for s in "${SAMPLES[@]}"; do
|
||||||
|
b=$(bucket_for "$s")
|
||||||
|
it=$(iters_for "$b")
|
||||||
|
warm=$(warm_for "$b")
|
||||||
|
|
||||||
|
IFS=, read -r ours_avg ours_ops ours_cnt < <(run_ours "$s" "$it" "$warm")
|
||||||
|
IFS=, read -r rout_avg rout_ops rout_cnt < <(run_rout "$s" "$it" "$warm")
|
||||||
|
|
||||||
|
if [[ "$ours_cnt" != "$rout_cnt" ]]; then
|
||||||
|
echo "WARNING: file count differs for $s (ours=$ours_cnt rout=$rout_cnt)" | tee -a "$OUT_RAW"
|
||||||
|
fi
|
||||||
|
|
||||||
|
ratio=$(python3 - <<PY
|
||||||
|
o=float("$ours_avg")
|
||||||
|
r=float("$rout_avg")
|
||||||
|
print(f"{(o/r):.4f}" if r != 0 else "inf")
|
||||||
|
PY
|
||||||
|
)
|
||||||
|
|
||||||
|
echo "$s,$b,$ours_cnt,$ours_avg,$ours_ops,$rout_avg,$rout_ops,$ratio,$it,$REPEATS,$warm" >> "$OUT_CSV"
|
||||||
|
echo >> "$OUT_RAW"
|
||||||
|
done
|
||||||
|
|
||||||
|
echo "Done."
|
||||||
|
echo "- CSV: $OUT_CSV"
|
||||||
|
echo "- Raw: $OUT_RAW"
|
||||||
142
scripts/manifest_perf_profile_m3.sh
Executable file
142
scripts/manifest_perf_profile_m3.sh
Executable file
@@ -0,0 +1,142 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
# M3: Generate flamegraphs + top hotspots for Manifest decode+profile (Ours vs Routinator).
|
||||||
|
#
|
||||||
|
# Outputs under:
|
||||||
|
# specs/develop/20260224/flamegraph/
|
||||||
|
# specs/develop/20260224/hotspots/
|
||||||
|
# specs/develop/20260224/perf/
|
||||||
|
#
|
||||||
|
# Notes:
|
||||||
|
# - On WSL2, /usr/bin/perf is often a wrapper that fails. This script uses a real perf binary
|
||||||
|
# from /usr/lib/linux-tools/*/perf (if present).
|
||||||
|
# - Ours profiling uses perf + flamegraph --perfdata to avoid rebuilding the whole crate graph
|
||||||
|
# with RocksDB.
|
||||||
|
|
||||||
|
ROOT_REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
RPKI_DIR="$ROOT_REPO/rpki"
|
||||||
|
|
||||||
|
DATE_TAG="${DATE_TAG:-20260224}"
|
||||||
|
OUT_BASE="$ROOT_REPO/specs/develop/${DATE_TAG}"
|
||||||
|
OUT_FLAME="$OUT_BASE/flamegraph"
|
||||||
|
OUT_HOT="$OUT_BASE/hotspots"
|
||||||
|
OUT_PERF="$OUT_BASE/perf"
|
||||||
|
|
||||||
|
RUN_TAG="${RUN_TAG:-p2}"
|
||||||
|
|
||||||
|
OURS_BENCH_DIR="$RPKI_DIR/benchmark/ours_manifest_bench"
|
||||||
|
OURS_BIN="$OURS_BENCH_DIR/target/release/ours-manifest-bench"
|
||||||
|
|
||||||
|
ROUT_BENCH_DIR="${ROUT_BENCH_DIR:-/home/yuyr/dev/rust_playground/routinator/benchmark}"
|
||||||
|
ROUT_BIN="$ROUT_BENCH_DIR/target/release/routinator-manifest-benchmark"
|
||||||
|
ROUT_ISSUER="$ROUT_BENCH_DIR/fixtures/ta.cer"
|
||||||
|
|
||||||
|
PROFILE_HZ="${PROFILE_HZ:-99}"
|
||||||
|
|
||||||
|
mkdir -p "$OUT_FLAME" "$OUT_HOT" "$OUT_PERF"
|
||||||
|
|
||||||
|
PERF_WRAPPER_OUT="$(perf --version 2>&1 || true)"
|
||||||
|
PERF_REAL=""
|
||||||
|
if echo "${PERF_WRAPPER_OUT}" | grep -q "WARNING: perf not found for kernel"; then
|
||||||
|
PERF_REAL="$(ls -1 /usr/lib/linux-tools/*/perf 2>/dev/null | head -n 1 || true)"
|
||||||
|
else
|
||||||
|
PERF_REAL="$(command -v perf || true)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ -z "${PERF_REAL}" ]]; then
|
||||||
|
echo "ERROR: usable perf binary not found (wrapper detected and no /usr/lib/linux-tools/*/perf)." >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
SHIM_DIR="$RPKI_DIR/target/bench/tools"
|
||||||
|
mkdir -p "$SHIM_DIR"
|
||||||
|
cat > "$SHIM_DIR/perf" <<EOF
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
exec "${PERF_REAL}" "\$@"
|
||||||
|
EOF
|
||||||
|
chmod +x "$SHIM_DIR/perf"
|
||||||
|
export PATH="$SHIM_DIR:$PATH"
|
||||||
|
|
||||||
|
echo "Using perf: $PERF_REAL"
|
||||||
|
|
||||||
|
echo "[1/3] Build ours benchmark with frame pointers..."
|
||||||
|
(cd "$OURS_BENCH_DIR" && RUSTFLAGS="-C force-frame-pointers=yes" cargo build --release -q)
|
||||||
|
|
||||||
|
echo "[2/3] Build routinator benchmark (release)..."
|
||||||
|
(cd "$ROUT_BENCH_DIR" && cargo build --release -q)
|
||||||
|
|
||||||
|
taskset_prefix=""
|
||||||
|
if command -v taskset >/dev/null 2>&1; then
|
||||||
|
taskset_prefix="taskset -c 0"
|
||||||
|
fi
|
||||||
|
|
||||||
|
profile_ours() {
|
||||||
|
local sample="$1"
|
||||||
|
local iters="$2"
|
||||||
|
local warm="$3"
|
||||||
|
local fixture="$RPKI_DIR/tests/benchmark/selected_der/${sample}.mft"
|
||||||
|
if [[ ! -f "$fixture" ]]; then
|
||||||
|
echo "ERROR: ours fixture not found: $fixture" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
local perfdata="$OUT_PERF/ours_${sample}_${RUN_TAG}.perf.data"
|
||||||
|
local svg="$OUT_FLAME/ours_${sample}_${RUN_TAG}.svg"
|
||||||
|
local tsv="$OUT_HOT/ours_${sample}_${RUN_TAG}.tsv"
|
||||||
|
|
||||||
|
echo "== ours $sample (iters=$iters warmup=$warm hz=$PROFILE_HZ)"
|
||||||
|
$taskset_prefix perf record -o "$perfdata" -F "$PROFILE_HZ" -g -- \
|
||||||
|
"$OURS_BIN" --manifest "$fixture" --iterations "$iters" --warmup-iterations "$warm" --repeats 1 >/dev/null
|
||||||
|
|
||||||
|
flamegraph --perfdata "$perfdata" --output "$svg" --title "ours ${sample} ManifestObject::decode_der" --deterministic >/dev/null
|
||||||
|
|
||||||
|
perf report -i "$perfdata" --stdio --no-children --sort symbol --percent-limit 0.5 \
|
||||||
|
| awk '/^[[:space:]]*[0-9.]+%/ {pct=$1; sub(/%/,"",pct); $1=""; sub(/^[[:space:]]+/,""); print pct "\t" $0}' \
|
||||||
|
> "$tsv"
|
||||||
|
}
|
||||||
|
|
||||||
|
profile_routinator() {
|
||||||
|
local sample="$1"
|
||||||
|
local iters="$2"
|
||||||
|
local warm="$3"
|
||||||
|
local fixture="$ROUT_BENCH_DIR/fixtures/selected_der/${sample}.mft"
|
||||||
|
if [[ ! -f "$fixture" ]]; then
|
||||||
|
echo "ERROR: routinator fixture not found: $fixture" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
|
local svg="$OUT_FLAME/routinator_${sample}_${RUN_TAG}.svg"
|
||||||
|
local tsv="$OUT_HOT/routinator_${sample}_${RUN_TAG}.tsv"
|
||||||
|
|
||||||
|
echo "== routinator $sample (iters=$iters warmup=$warm hz=$PROFILE_HZ)"
|
||||||
|
$taskset_prefix "$ROUT_BIN" \
|
||||||
|
--target decode_only \
|
||||||
|
--manifest "$fixture" \
|
||||||
|
--issuer "$ROUT_ISSUER" \
|
||||||
|
--iterations "$iters" \
|
||||||
|
--repeats 1 \
|
||||||
|
--warmup-iterations "$warm" \
|
||||||
|
--strict false \
|
||||||
|
--profile-hz "$PROFILE_HZ" \
|
||||||
|
--flamegraph "$svg" \
|
||||||
|
--hotspots "$tsv" \
|
||||||
|
>/dev/null
|
||||||
|
}
|
||||||
|
|
||||||
|
echo "[3/3] Profile samples..."
|
||||||
|
|
||||||
|
# Choose iterations so each capture runs ~10-20s serially.
|
||||||
|
profile_ours small-01 200000 0
|
||||||
|
profile_routinator small-01 200000 0
|
||||||
|
|
||||||
|
profile_ours large-02 50000 0
|
||||||
|
profile_routinator large-02 50000 0
|
||||||
|
|
||||||
|
profile_ours xlarge-02 5000 0
|
||||||
|
profile_routinator xlarge-02 5000 0
|
||||||
|
|
||||||
|
echo "Done."
|
||||||
|
echo "- Flamegraphs: $OUT_FLAME/"
|
||||||
|
echo "- Hotspots: $OUT_HOT/"
|
||||||
|
echo "- Perf data: $OUT_PERF/"
|
||||||
97
scripts/manual_sync/README.md
Normal file
97
scripts/manual_sync/README.md
Normal file
@@ -0,0 +1,97 @@
|
|||||||
|
# Manual RRDP sync (APNIC-focused)
|
||||||
|
|
||||||
|
This directory contains **manual, command-line** scripts to reproduce the workflow described in:
|
||||||
|
|
||||||
|
- `specs/develop/20260226/apnic_rrdp_delta_analysis_after_manifest_revalidation_fix_20260227T022606Z.md`
|
||||||
|
|
||||||
|
They are meant for **hands-on validation / acceptance runs**, not for CI.
|
||||||
|
|
||||||
|
## Prerequisites
|
||||||
|
|
||||||
|
- Rust toolchain (`cargo`)
|
||||||
|
- `rsync` available on PATH (for rsync fallback/objects)
|
||||||
|
- Network access (RRDP over HTTPS)
|
||||||
|
|
||||||
|
## What the scripts do
|
||||||
|
|
||||||
|
### `full_sync.sh`
|
||||||
|
|
||||||
|
- Creates a fresh RocksDB directory
|
||||||
|
- Runs a **full serial** validation from a TAL URL (default: APNIC RFC7730 TAL)
|
||||||
|
- Writes:
|
||||||
|
- run log
|
||||||
|
- audit report JSON
|
||||||
|
- run meta JSON (includes durations + download_stats)
|
||||||
|
- short summary Markdown (includes durations + download_stats)
|
||||||
|
- RocksDB key statistics (`db_stats --exact`)
|
||||||
|
- RRDP legacy session/serial dump (`rrdp_state_dump --view legacy-state`)
|
||||||
|
|
||||||
|
### `delta_sync.sh`
|
||||||
|
|
||||||
|
- Copies an existing “baseline snapshot DB” to a new DB directory (so the baseline is not modified)
|
||||||
|
- Runs another validation against the copied DB (RRDP will prefer **delta** when available)
|
||||||
|
- Produces the same artifacts as `full_sync.sh`
|
||||||
|
- Additionally generates a Markdown **delta analysis** report by comparing:
|
||||||
|
- base vs delta report JSON
|
||||||
|
- base vs delta `rrdp_state_dump --view legacy-state` TSV
|
||||||
|
- and includes a **duration comparison** (base vs delta) if the base meta JSON is available
|
||||||
|
- delta meta JSON includes download_stats copied from delta report JSON
|
||||||
|
|
||||||
|
## Audit report fields (report.json)
|
||||||
|
|
||||||
|
The `rpki` binary writes an audit report JSON with:
|
||||||
|
|
||||||
|
- `format_version: 2`
|
||||||
|
- `downloads`: per-download RRDP/rsync events (URI, timestamps, duration, ok/fail, error, bytes, objects stats)
|
||||||
|
- `download_stats`: aggregate counters (by kind)
|
||||||
|
|
||||||
|
These are useful for diagnosing why a run is slow (e.g. RRDP snapshot vs delta vs rsync fallback).
|
||||||
|
|
||||||
|
The standalone `rrdp_state_dump` tool also supports `source`, `members`, `owners`, and `all` views.
|
||||||
|
The manual sync scripts intentionally call `--view legacy-state` so delta analysis keeps using a stable session/serial TSV format.
|
||||||
|
|
||||||
|
## Meta fields (meta.json)
|
||||||
|
|
||||||
|
The scripts generate `*_meta.json` next to `*_report.json` and include:
|
||||||
|
|
||||||
|
- `durations_secs`: wall-clock duration breakdown for the script steps
|
||||||
|
- `download_stats`: copied from `report_json.download_stats`
|
||||||
|
|
||||||
|
## Usage
|
||||||
|
|
||||||
|
Run from `rpki/`:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/manual_sync/full_sync.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
After you have a baseline run, run delta against it:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/manual_sync/delta_sync.sh target/live/manual_sync/apnic_full_db_YYYYMMDDTHHMMSSZ \
|
||||||
|
target/live/manual_sync/apnic_full_report_YYYYMMDDTHHMMSSZ.json
|
||||||
|
```
|
||||||
|
|
||||||
|
If the baseline was produced by `full_sync.sh`, the delta script will auto-discover the base meta JSON
|
||||||
|
next to the base report (by replacing `_report.json` with `_meta.json`) and include base durations in
|
||||||
|
the delta analysis report.
|
||||||
|
|
||||||
|
## Configuration (env vars)
|
||||||
|
|
||||||
|
Both scripts accept overrides via env vars:
|
||||||
|
|
||||||
|
- `TAL_URL` (default: APNIC TAL URL)
|
||||||
|
- `HTTP_TIMEOUT_SECS` (default: 1800)
|
||||||
|
- `RSYNC_TIMEOUT_SECS` (default: 1800)
|
||||||
|
- `RSYNC_MIRROR_ROOT` (default: disabled; when set, passes `--rsync-mirror-root` to `rpki`)
|
||||||
|
- `VALIDATION_TIME` (RFC3339; default: now UTC)
|
||||||
|
- `OUT_DIR` (default: `rpki/target/live/manual_sync`)
|
||||||
|
- `RUN_NAME` (default: auto timestamped)
|
||||||
|
|
||||||
|
Example:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
TAL_URL="https://tal.apnic.net/tal-archive/apnic-rfc7730-https.tal" \
|
||||||
|
HTTP_TIMEOUT_SECS=1800 RSYNC_TIMEOUT_SECS=1800 \
|
||||||
|
./scripts/manual_sync/full_sync.sh
|
||||||
|
```
|
||||||
543
scripts/manual_sync/delta_sync.sh
Executable file
543
scripts/manual_sync/delta_sync.sh
Executable file
@@ -0,0 +1,543 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
# Delta sync + validation starting from a baseline snapshot DB.
|
||||||
|
#
|
||||||
|
# This script:
|
||||||
|
# 1) Copies BASE_DB_DIR -> DELTA_DB_DIR (so baseline is not modified)
|
||||||
|
# 2) Runs rpki validation again (RRDP will prefer delta if available)
|
||||||
|
# 3) Writes artifacts + a markdown delta analysis report
|
||||||
|
#
|
||||||
|
# Usage:
|
||||||
|
# ./scripts/manual_sync/delta_sync.sh <base_db_dir> <base_report_json>
|
||||||
|
#
|
||||||
|
# Outputs under OUT_DIR (default: target/live/manual_sync):
|
||||||
|
# - *_delta_db_* copied RocksDB directory
|
||||||
|
# - *_delta_report_*.json audit report
|
||||||
|
# - *_delta_run_*.log stdout/stderr log (includes summary)
|
||||||
|
# - *_delta_db_stats_*.txt db_stats --exact output
|
||||||
|
# - *_delta_rrdp_state_*.tsv rrdp_state_dump --view legacy-state output
|
||||||
|
# - *_delta_analysis_*.md base vs delta comparison report
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
cd "$ROOT_DIR"
|
||||||
|
|
||||||
|
BASE_DB_DIR="${1:-}"
|
||||||
|
BASE_REPORT_JSON="${2:-}"
|
||||||
|
if [[ -z "${BASE_DB_DIR}" || -z "${BASE_REPORT_JSON}" ]]; then
|
||||||
|
echo "Usage: $0 <base_db_dir> <base_report_json>" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
if [[ ! -d "${BASE_DB_DIR}" ]]; then
|
||||||
|
echo "ERROR: base_db_dir is not a directory: ${BASE_DB_DIR}" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
if [[ ! -f "${BASE_REPORT_JSON}" ]]; then
|
||||||
|
echo "ERROR: base_report_json does not exist: ${BASE_REPORT_JSON}" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
TAL_URL="${TAL_URL:-https://tal.apnic.net/tal-archive/apnic-rfc7730-https.tal}"
|
||||||
|
HTTP_TIMEOUT_SECS="${HTTP_TIMEOUT_SECS:-1800}"
|
||||||
|
RSYNC_TIMEOUT_SECS="${RSYNC_TIMEOUT_SECS:-1800}"
|
||||||
|
RSYNC_MIRROR_ROOT="${RSYNC_MIRROR_ROOT:-}"
|
||||||
|
VALIDATION_TIME="${VALIDATION_TIME:-}"
|
||||||
|
|
||||||
|
OUT_DIR="${OUT_DIR:-$ROOT_DIR/target/live/manual_sync}"
|
||||||
|
mkdir -p "$OUT_DIR"
|
||||||
|
|
||||||
|
TS="$(date -u +%Y%m%dT%H%M%SZ)"
|
||||||
|
RUN_NAME="${RUN_NAME:-apnic_delta_${TS}}"
|
||||||
|
|
||||||
|
DELTA_DB_DIR="${DELTA_DB_DIR:-$OUT_DIR/${RUN_NAME}_db}"
|
||||||
|
DELTA_REPORT_JSON="${DELTA_REPORT_JSON:-$OUT_DIR/${RUN_NAME}_report.json}"
|
||||||
|
DELTA_RUN_LOG="${DELTA_RUN_LOG:-$OUT_DIR/${RUN_NAME}_run.log}"
|
||||||
|
|
||||||
|
BASE_DB_STATS_TXT="${BASE_DB_STATS_TXT:-$OUT_DIR/${RUN_NAME}_base_db_stats.txt}"
|
||||||
|
DELTA_DB_STATS_TXT="${DELTA_DB_STATS_TXT:-$OUT_DIR/${RUN_NAME}_delta_db_stats.txt}"
|
||||||
|
|
||||||
|
BASE_RRDP_STATE_TSV="${BASE_RRDP_STATE_TSV:-$OUT_DIR/${RUN_NAME}_base_rrdp_state.tsv}"
|
||||||
|
DELTA_RRDP_STATE_TSV="${DELTA_RRDP_STATE_TSV:-$OUT_DIR/${RUN_NAME}_delta_rrdp_state.tsv}"
|
||||||
|
|
||||||
|
DELTA_ANALYSIS_MD="${DELTA_ANALYSIS_MD:-$OUT_DIR/${RUN_NAME}_delta_analysis.md}"
|
||||||
|
DELTA_META_JSON="${DELTA_META_JSON:-$OUT_DIR/${RUN_NAME}_meta.json}"
|
||||||
|
|
||||||
|
# Best-effort base meta discovery (produced by `full_sync.sh`).
|
||||||
|
BASE_META_JSON="${BASE_META_JSON:-}"
|
||||||
|
if [[ -z "${BASE_META_JSON}" ]]; then
|
||||||
|
guess="${BASE_REPORT_JSON%_report.json}_meta.json"
|
||||||
|
if [[ -f "${guess}" ]]; then
|
||||||
|
BASE_META_JSON="${guess}"
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
echo "== rpki manual delta sync ==" >&2
|
||||||
|
echo "tal_url=$TAL_URL" >&2
|
||||||
|
echo "base_db=$BASE_DB_DIR" >&2
|
||||||
|
echo "base_report=$BASE_REPORT_JSON" >&2
|
||||||
|
echo "delta_db=$DELTA_DB_DIR" >&2
|
||||||
|
echo "delta_report=$DELTA_REPORT_JSON" >&2
|
||||||
|
|
||||||
|
echo "== copying base DB (baseline is not modified) ==" >&2
|
||||||
|
cp -a "$BASE_DB_DIR" "$DELTA_DB_DIR"
|
||||||
|
|
||||||
|
script_start_s="$(date +%s)"
|
||||||
|
run_start_s="$(date +%s)"
|
||||||
|
cmd=(cargo run --release --bin rpki -- \
|
||||||
|
--db "$DELTA_DB_DIR" \
|
||||||
|
--tal-url "$TAL_URL" \
|
||||||
|
--http-timeout-secs "$HTTP_TIMEOUT_SECS" \
|
||||||
|
--rsync-timeout-secs "$RSYNC_TIMEOUT_SECS" \
|
||||||
|
--report-json "$DELTA_REPORT_JSON")
|
||||||
|
if [[ -n "${RSYNC_MIRROR_ROOT}" ]]; then
|
||||||
|
cmd+=(--rsync-mirror-root "$RSYNC_MIRROR_ROOT")
|
||||||
|
fi
|
||||||
|
if [[ -n "${VALIDATION_TIME}" ]]; then
|
||||||
|
cmd+=(--validation-time "$VALIDATION_TIME")
|
||||||
|
fi
|
||||||
|
|
||||||
|
(
|
||||||
|
echo "# command:"
|
||||||
|
printf '%q ' "${cmd[@]}"
|
||||||
|
echo
|
||||||
|
echo
|
||||||
|
"${cmd[@]}"
|
||||||
|
) 2>&1 | tee "$DELTA_RUN_LOG" >/dev/null
|
||||||
|
run_end_s="$(date +%s)"
|
||||||
|
run_duration_s="$((run_end_s - run_start_s))"
|
||||||
|
|
||||||
|
echo "== db_stats (exact) ==" >&2
|
||||||
|
db_stats_start_s="$(date +%s)"
|
||||||
|
cargo run --release --bin db_stats -- --db "$BASE_DB_DIR" --exact 2>&1 | tee "$BASE_DB_STATS_TXT" >/dev/null
|
||||||
|
cargo run --release --bin db_stats -- --db "$DELTA_DB_DIR" --exact 2>&1 | tee "$DELTA_DB_STATS_TXT" >/dev/null
|
||||||
|
db_stats_end_s="$(date +%s)"
|
||||||
|
db_stats_duration_s="$((db_stats_end_s - db_stats_start_s))"
|
||||||
|
|
||||||
|
echo "== rrdp_state_dump (legacy-state) ==" >&2
|
||||||
|
state_start_s="$(date +%s)"
|
||||||
|
cargo run --release --bin rrdp_state_dump -- --db "$BASE_DB_DIR" --view legacy-state >"$BASE_RRDP_STATE_TSV"
|
||||||
|
cargo run --release --bin rrdp_state_dump -- --db "$DELTA_DB_DIR" --view legacy-state >"$DELTA_RRDP_STATE_TSV"
|
||||||
|
state_end_s="$(date +%s)"
|
||||||
|
state_duration_s="$((state_end_s - state_start_s))"
|
||||||
|
|
||||||
|
script_end_s="$(date +%s)"
|
||||||
|
total_duration_s="$((script_end_s - script_start_s))"
|
||||||
|
|
||||||
|
echo "== delta analysis report ==" >&2
|
||||||
|
TAL_URL="$TAL_URL" \
|
||||||
|
BASE_DB_DIR="$BASE_DB_DIR" \
|
||||||
|
DELTA_DB_DIR="$DELTA_DB_DIR" \
|
||||||
|
DELTA_RUN_LOG="$DELTA_RUN_LOG" \
|
||||||
|
VALIDATION_TIME_ARG="$VALIDATION_TIME" \
|
||||||
|
HTTP_TIMEOUT_SECS="$HTTP_TIMEOUT_SECS" \
|
||||||
|
RSYNC_TIMEOUT_SECS="$RSYNC_TIMEOUT_SECS" \
|
||||||
|
RUN_DURATION_S="$run_duration_s" \
|
||||||
|
DB_STATS_DURATION_S="$db_stats_duration_s" \
|
||||||
|
STATE_DURATION_S="$state_duration_s" \
|
||||||
|
TOTAL_DURATION_S="$total_duration_s" \
|
||||||
|
python3 - "$BASE_REPORT_JSON" "$DELTA_REPORT_JSON" "$BASE_RRDP_STATE_TSV" "$DELTA_RRDP_STATE_TSV" \
|
||||||
|
"$BASE_DB_STATS_TXT" "$DELTA_DB_STATS_TXT" "$BASE_META_JSON" "$DELTA_META_JSON" "$DELTA_ANALYSIS_MD" <<'PY'
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from collections import Counter, defaultdict
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
base_report_path = Path(sys.argv[1])
|
||||||
|
delta_report_path = Path(sys.argv[2])
|
||||||
|
base_state_path = Path(sys.argv[3])
|
||||||
|
delta_state_path = Path(sys.argv[4])
|
||||||
|
base_db_stats_path = Path(sys.argv[5])
|
||||||
|
delta_db_stats_path = Path(sys.argv[6])
|
||||||
|
base_meta_path_s = sys.argv[7]
|
||||||
|
delta_meta_path = Path(sys.argv[8])
|
||||||
|
out_md_path = Path(sys.argv[9])
|
||||||
|
|
||||||
|
def load_json(p: Path):
|
||||||
|
s = p.read_text(encoding="utf-8")
|
||||||
|
try:
|
||||||
|
return json.loads(s)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
# Backwards-compat / robustness: tolerate accidental literal trailing "\\n".
|
||||||
|
s2 = s.strip()
|
||||||
|
if s2.endswith("\\n"):
|
||||||
|
s2 = s2[:-2].rstrip()
|
||||||
|
return json.loads(s2)
|
||||||
|
|
||||||
|
def load_optional_json(path_s: str):
|
||||||
|
if not path_s:
|
||||||
|
return None
|
||||||
|
p = Path(path_s)
|
||||||
|
if not p.exists():
|
||||||
|
return None
|
||||||
|
return load_json(p)
|
||||||
|
|
||||||
|
def parse_rrdp_state_tsv(p: Path):
|
||||||
|
# format from `rrdp_state_dump --view legacy-state`:
|
||||||
|
# [legacy-state]
|
||||||
|
# notify_uri serial session_id
|
||||||
|
# <notify_uri> <serial> <session_id>
|
||||||
|
out = {}
|
||||||
|
for line in p.read_text(encoding="utf-8").splitlines():
|
||||||
|
line = line.strip()
|
||||||
|
if not line or line.startswith("["):
|
||||||
|
continue
|
||||||
|
if line == "notify_uri serial session_id":
|
||||||
|
continue
|
||||||
|
parts = line.split(" ")
|
||||||
|
if len(parts) != 3:
|
||||||
|
raise SystemExit(f"invalid rrdp_state_dump line in {p}: {line!r}")
|
||||||
|
uri, serial, session = parts
|
||||||
|
out[uri] = (int(serial), session)
|
||||||
|
return out
|
||||||
|
|
||||||
|
def parse_db_stats(p: Path):
|
||||||
|
# lines: key=value
|
||||||
|
out = {}
|
||||||
|
for line in p.read_text(encoding="utf-8").splitlines():
|
||||||
|
if "=" not in line:
|
||||||
|
continue
|
||||||
|
k, v = line.split("=", 1)
|
||||||
|
k = k.strip()
|
||||||
|
v = v.strip()
|
||||||
|
if v.isdigit():
|
||||||
|
out[k] = int(v)
|
||||||
|
else:
|
||||||
|
out[k] = v
|
||||||
|
return out
|
||||||
|
|
||||||
|
def warnings_total(rep: dict) -> int:
|
||||||
|
return len(rep["tree"]["warnings"]) + sum(len(pp["warnings"]) for pp in rep["publication_points"])
|
||||||
|
|
||||||
|
def report_summary(rep: dict) -> dict:
|
||||||
|
return {
|
||||||
|
"validation_time": rep["meta"]["validation_time_rfc3339_utc"],
|
||||||
|
"publication_points_processed": rep["tree"]["instances_processed"],
|
||||||
|
"publication_points_failed": rep["tree"]["instances_failed"],
|
||||||
|
"rrdp_repos_unique": len({pp.get("rrdp_notification_uri") for pp in rep["publication_points"] if pp.get("rrdp_notification_uri")}),
|
||||||
|
"vrps": len(rep["vrps"]),
|
||||||
|
"aspas": len(rep["aspas"]),
|
||||||
|
"audit_publication_points": len(rep["publication_points"]),
|
||||||
|
"warnings_total": warnings_total(rep),
|
||||||
|
}
|
||||||
|
|
||||||
|
def count_repo_sync_failed(rep: dict) -> int:
|
||||||
|
# Best-effort heuristic (we don't currently expose a structured counter in the audit report).
|
||||||
|
# Keep the match conservative to avoid false positives.
|
||||||
|
def is_repo_sync_failed(msg: str) -> bool:
|
||||||
|
m = msg.lower()
|
||||||
|
return "repo sync failed" in m or "rrdp fetch failed" in m or "rsync fetch failed" in m
|
||||||
|
|
||||||
|
n = 0
|
||||||
|
for w in rep["tree"]["warnings"]:
|
||||||
|
if is_repo_sync_failed(w.get("message", "")):
|
||||||
|
n += 1
|
||||||
|
for pp in rep["publication_points"]:
|
||||||
|
for w in pp.get("warnings", []):
|
||||||
|
if is_repo_sync_failed(w.get("message", "")):
|
||||||
|
n += 1
|
||||||
|
return n
|
||||||
|
|
||||||
|
def pp_manifest_sha(pp: dict) -> str:
|
||||||
|
# In our audit format, the first object is the manifest (synthetic entry) with sha256 of manifest_bytes.
|
||||||
|
for o in pp["objects"]:
|
||||||
|
if o["kind"] == "manifest":
|
||||||
|
return o["sha256_hex"]
|
||||||
|
return ""
|
||||||
|
|
||||||
|
def pp_objects_by_uri(rep: dict):
|
||||||
|
m = {}
|
||||||
|
for pp in rep["publication_points"]:
|
||||||
|
for o in pp["objects"]:
|
||||||
|
m[o["rsync_uri"]] = (o["sha256_hex"], o["kind"])
|
||||||
|
return m
|
||||||
|
|
||||||
|
def vrp_set(rep: dict):
|
||||||
|
return {(v["asn"], v["prefix"], v["max_length"]) for v in rep["vrps"]}
|
||||||
|
|
||||||
|
def rfc_refs_str(w: dict) -> str:
|
||||||
|
refs = w.get("rfc_refs") or []
|
||||||
|
return ", ".join(refs) if refs else ""
|
||||||
|
|
||||||
|
base = load_json(base_report_path)
|
||||||
|
delta = load_json(delta_report_path)
|
||||||
|
base_sum = report_summary(base)
|
||||||
|
delta_sum = report_summary(delta)
|
||||||
|
|
||||||
|
base_db = parse_db_stats(base_db_stats_path)
|
||||||
|
delta_db = parse_db_stats(delta_db_stats_path)
|
||||||
|
|
||||||
|
base_state = parse_rrdp_state_tsv(base_state_path)
|
||||||
|
delta_state = parse_rrdp_state_tsv(delta_state_path)
|
||||||
|
|
||||||
|
base_meta = load_optional_json(base_meta_path_s)
|
||||||
|
download_stats = delta.get("download_stats") or {}
|
||||||
|
delta_meta = {
|
||||||
|
"recorded_at_utc": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
|
||||||
|
"tal_url": os.environ["TAL_URL"],
|
||||||
|
"base_db_dir": os.environ["BASE_DB_DIR"],
|
||||||
|
"delta_db_dir": os.environ["DELTA_DB_DIR"],
|
||||||
|
"base_report_json": str(base_report_path),
|
||||||
|
"delta_report_json": str(delta_report_path),
|
||||||
|
"delta_run_log": os.environ["DELTA_RUN_LOG"],
|
||||||
|
"validation_time_arg": os.environ.get("VALIDATION_TIME_ARG",""),
|
||||||
|
"http_timeout_secs": int(os.environ["HTTP_TIMEOUT_SECS"]),
|
||||||
|
"rsync_timeout_secs": int(os.environ["RSYNC_TIMEOUT_SECS"]),
|
||||||
|
"durations_secs": {
|
||||||
|
"rpki_run": int(os.environ["RUN_DURATION_S"]),
|
||||||
|
"db_stats_exact": int(os.environ["DB_STATS_DURATION_S"]),
|
||||||
|
"rrdp_state_dump": int(os.environ["STATE_DURATION_S"]),
|
||||||
|
"total_script": int(os.environ["TOTAL_DURATION_S"]),
|
||||||
|
},
|
||||||
|
"download_stats": download_stats,
|
||||||
|
}
|
||||||
|
delta_meta_path.write_text(json.dumps(delta_meta, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
|
||||||
|
now = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||||
|
|
||||||
|
# RRDP state changes
|
||||||
|
serial_changed = 0
|
||||||
|
session_changed = 0
|
||||||
|
serial_deltas = []
|
||||||
|
for uri, (old_serial, old_sess) in base_state.items():
|
||||||
|
if uri not in delta_state:
|
||||||
|
continue
|
||||||
|
new_serial, new_sess = delta_state[uri]
|
||||||
|
if new_serial != old_serial:
|
||||||
|
serial_changed += 1
|
||||||
|
serial_deltas.append((uri, old_serial, new_serial, new_serial - old_serial))
|
||||||
|
if new_sess != old_sess:
|
||||||
|
session_changed += 1
|
||||||
|
|
||||||
|
serial_deltas.sort(key=lambda x: x[3], reverse=True)
|
||||||
|
|
||||||
|
# Publication point diffs
|
||||||
|
base_pp = {pp["manifest_rsync_uri"]: pp for pp in base["publication_points"]}
|
||||||
|
delta_pp = {pp["manifest_rsync_uri"]: pp for pp in delta["publication_points"]}
|
||||||
|
|
||||||
|
base_keys = set(base_pp.keys())
|
||||||
|
delta_keys = set(delta_pp.keys())
|
||||||
|
|
||||||
|
new_pp = sorted(delta_keys - base_keys)
|
||||||
|
missing_pp = sorted(base_keys - delta_keys)
|
||||||
|
|
||||||
|
updated_pp = 0
|
||||||
|
unchanged_pp = 0
|
||||||
|
for k in sorted(base_keys & delta_keys):
|
||||||
|
if pp_manifest_sha(base_pp[k]) != pp_manifest_sha(delta_pp[k]):
|
||||||
|
updated_pp += 1
|
||||||
|
else:
|
||||||
|
unchanged_pp += 1
|
||||||
|
|
||||||
|
# Cache usage + repo sync failure hints
|
||||||
|
def source_counts(rep: dict) -> Counter:
|
||||||
|
c = Counter()
|
||||||
|
for pp in rep["publication_points"]:
|
||||||
|
c[pp.get("source","")] += 1
|
||||||
|
return c
|
||||||
|
|
||||||
|
base_sources = source_counts(base)
|
||||||
|
delta_sources = source_counts(delta)
|
||||||
|
base_repo_sync_failed = count_repo_sync_failed(base)
|
||||||
|
delta_repo_sync_failed = count_repo_sync_failed(delta)
|
||||||
|
|
||||||
|
def cache_reason_counts(rep: dict) -> Counter:
|
||||||
|
c = Counter()
|
||||||
|
for pp in rep.get("publication_points", []):
|
||||||
|
if pp.get("source") != "vcir_current_instance":
|
||||||
|
continue
|
||||||
|
# Use warning messages as "reason". If missing, emit a fallback bucket.
|
||||||
|
ws = pp.get("warnings", [])
|
||||||
|
if not ws:
|
||||||
|
c["(no warnings recorded)"] += 1
|
||||||
|
continue
|
||||||
|
for w in ws:
|
||||||
|
msg = w.get("message", "").strip() or "(empty warning message)"
|
||||||
|
c[msg] += 1
|
||||||
|
return c
|
||||||
|
|
||||||
|
base_cache_reasons = cache_reason_counts(base)
|
||||||
|
delta_cache_reasons = cache_reason_counts(delta)
|
||||||
|
|
||||||
|
# Object change stats (by rsync URI, sha256)
|
||||||
|
base_obj = pp_objects_by_uri(base)
|
||||||
|
delta_obj = pp_objects_by_uri(delta)
|
||||||
|
|
||||||
|
kind_stats = {k: {"added": 0, "changed": 0, "removed": 0} for k in ["manifest","crl","certificate","roa","aspa","other"]}
|
||||||
|
all_uris = set(base_obj.keys()) | set(delta_obj.keys())
|
||||||
|
for uri in all_uris:
|
||||||
|
b = base_obj.get(uri)
|
||||||
|
d = delta_obj.get(uri)
|
||||||
|
if b is None and d is not None:
|
||||||
|
kind_stats[d[1]]["added"] += 1
|
||||||
|
elif b is not None and d is None:
|
||||||
|
kind_stats[b[1]]["removed"] += 1
|
||||||
|
else:
|
||||||
|
if b[0] != d[0]:
|
||||||
|
kind_stats[d[1]]["changed"] += 1
|
||||||
|
|
||||||
|
# VRP diff
|
||||||
|
base_v = vrp_set(base)
|
||||||
|
delta_v = vrp_set(delta)
|
||||||
|
added_v = delta_v - base_v
|
||||||
|
removed_v = base_v - delta_v
|
||||||
|
|
||||||
|
def fmt_db_stats(db: dict) -> str:
|
||||||
|
ordered = [
|
||||||
|
"mode",
|
||||||
|
"repository_view",
|
||||||
|
"raw_by_hash",
|
||||||
|
"vcir",
|
||||||
|
"audit_rule_index",
|
||||||
|
"rrdp_source",
|
||||||
|
"rrdp_source_member",
|
||||||
|
"rrdp_uri_owner",
|
||||||
|
"rrdp_state",
|
||||||
|
"raw_objects",
|
||||||
|
"rrdp_object_index",
|
||||||
|
"group_current_repository_view",
|
||||||
|
"group_current_validation_state",
|
||||||
|
"group_current_rrdp_state",
|
||||||
|
"group_legacy_compatibility",
|
||||||
|
"total",
|
||||||
|
"sst_files",
|
||||||
|
]
|
||||||
|
out = []
|
||||||
|
seen = set()
|
||||||
|
for k in ordered:
|
||||||
|
if k in db:
|
||||||
|
out.append(f"- `{k}={db[k]}`")
|
||||||
|
seen.add(k)
|
||||||
|
for k in sorted(set(db) - seen):
|
||||||
|
out.append(f"- `{k}={db[k]}`")
|
||||||
|
return "\n".join(out) if out else "_(missing db_stats keys)_"
|
||||||
|
|
||||||
|
lines = []
|
||||||
|
lines.append("# APNIC RRDP 增量同步验收(manual_sync)\n\n")
|
||||||
|
lines.append(f"时间戳:`{now}`(UTC)\n\n")
|
||||||
|
|
||||||
|
lines.append("## 复现信息\n\n")
|
||||||
|
lines.append(f"- base_report:`{base_report_path}`\n")
|
||||||
|
lines.append(f"- delta_report:`{delta_report_path}`\n")
|
||||||
|
lines.append(f"- base_db_stats:`{base_db_stats_path}`\n")
|
||||||
|
lines.append(f"- delta_db_stats:`{delta_db_stats_path}`\n")
|
||||||
|
lines.append(f"- base_rrdp_state:`{base_state_path}`\n")
|
||||||
|
lines.append(f"- delta_rrdp_state:`{delta_state_path}`\n\n")
|
||||||
|
|
||||||
|
lines.append("## 运行结果概览\n\n")
|
||||||
|
lines.append("| metric | base | delta |\n")
|
||||||
|
lines.append("|---|---:|---:|\n")
|
||||||
|
for k in [
|
||||||
|
"validation_time",
|
||||||
|
"publication_points_processed",
|
||||||
|
"publication_points_failed",
|
||||||
|
"rrdp_repos_unique",
|
||||||
|
"vrps",
|
||||||
|
"aspas",
|
||||||
|
"audit_publication_points",
|
||||||
|
"warnings_total",
|
||||||
|
]:
|
||||||
|
lines.append(f"| {k} | {base_sum[k]} | {delta_sum[k]} |\n")
|
||||||
|
lines.append("\n")
|
||||||
|
|
||||||
|
def dur(meta: dict | None, key: str):
|
||||||
|
if not meta:
|
||||||
|
return None
|
||||||
|
return (meta.get("durations_secs") or {}).get(key)
|
||||||
|
|
||||||
|
base_rpki_run = dur(base_meta, "rpki_run")
|
||||||
|
delta_rpki_run = delta_meta["durations_secs"]["rpki_run"]
|
||||||
|
base_total = dur(base_meta, "total_script")
|
||||||
|
delta_total = delta_meta["durations_secs"]["total_script"]
|
||||||
|
|
||||||
|
lines.append("## 持续时间(seconds)\n\n")
|
||||||
|
lines.append("| step | base | delta |\n")
|
||||||
|
lines.append("|---|---:|---:|\n")
|
||||||
|
lines.append(f"| rpki_run | {base_rpki_run if base_rpki_run is not None else 'unknown'} | {delta_rpki_run} |\n")
|
||||||
|
lines.append(f"| total_script | {base_total if base_total is not None else 'unknown'} | {delta_total} |\n")
|
||||||
|
lines.append("\n")
|
||||||
|
if base_meta is None and base_meta_path_s:
|
||||||
|
lines.append(f"> 注:未能读取 base meta:`{base_meta_path_s}`(文件不存在或不可读)。建议用 `full_sync.sh` 生成 baseline 以获得 base 时长对比。\n\n")
|
||||||
|
|
||||||
|
lines.append("RocksDB KV(`db_stats --exact`):\n\n")
|
||||||
|
lines.append("### 基线(base)\n\n")
|
||||||
|
lines.append(fmt_db_stats(base_db) + "\n\n")
|
||||||
|
lines.append("### 增量(delta)\n\n")
|
||||||
|
lines.append(fmt_db_stats(delta_db) + "\n\n")
|
||||||
|
|
||||||
|
lines.append("## RRDP 增量是否发生(基于 `rrdp_state` 变化)\n\n")
|
||||||
|
lines.append(f"- repo_total(base)={len(base_state)}\n")
|
||||||
|
lines.append(f"- repo_total(delta)={len(delta_state)}\n")
|
||||||
|
lines.append(f"- serial_changed={serial_changed}\n")
|
||||||
|
lines.append(f"- session_changed={session_changed}\n\n")
|
||||||
|
if serial_deltas:
|
||||||
|
lines.append("serial 增长最大的 10 个 RRDP repo(old -> new):\n\n")
|
||||||
|
for uri, old, new, diff in serial_deltas[:10]:
|
||||||
|
lines.append(f"- `{uri}`:`{old} -> {new}`(+{diff})\n")
|
||||||
|
lines.append("\n")
|
||||||
|
|
||||||
|
lines.append("## 发布点(Publication Point)变化统计\n\n")
|
||||||
|
lines.append("以 `manifest_rsync_uri` 作为发布点 key,对比 base vs delta:\n\n")
|
||||||
|
lines.append(f"- base PP:`{len(base_keys)}`\n")
|
||||||
|
lines.append(f"- delta PP:`{len(delta_keys)}`\n")
|
||||||
|
lines.append(f"- `new_pp={len(new_pp)}`\n")
|
||||||
|
lines.append(f"- `missing_pp={len(missing_pp)}`\n")
|
||||||
|
lines.append(f"- `updated_pp={updated_pp}`\n")
|
||||||
|
lines.append(f"- `unchanged_pp={unchanged_pp}`\n\n")
|
||||||
|
lines.append("> 注:`new_pp/missing_pp/updated_pp` 会混入“遍历范围变化”的影响(例如 validation_time 不同、或 base 中存在更多失败 PP)。\n\n")
|
||||||
|
|
||||||
|
lines.append("## fail fetch / VCIR 当前实例缓存复用情况\n\n")
|
||||||
|
lines.append(f"- repo sync failed(启发式:warning contains 'repo sync failed'/'rrdp fetch failed'/'rsync fetch failed')\n")
|
||||||
|
lines.append(f" - base:`{base_repo_sync_failed}`\n")
|
||||||
|
lines.append(f" - delta:`{delta_repo_sync_failed}`\n\n")
|
||||||
|
|
||||||
|
lines.append("- source 计数(按 `PublicationPointAudit.source`):\n\n")
|
||||||
|
lines.append(f" - base:`{dict(base_sources)}`\n")
|
||||||
|
lines.append(f" - delta:`{dict(delta_sources)}`\n\n")
|
||||||
|
|
||||||
|
def render_cache_reasons(title: str, c: Counter) -> str:
|
||||||
|
if not c:
|
||||||
|
return f"{title}:`0`(未使用 VCIR 当前实例缓存复用)\n\n"
|
||||||
|
lines = []
|
||||||
|
total = sum(c.values())
|
||||||
|
lines.append(f"{title}:`{total}`\n\n")
|
||||||
|
lines.append("Top reasons(按 warning message 聚合,可能一条 PP 有多条 warning):\n\n")
|
||||||
|
for msg, n in c.most_common(10):
|
||||||
|
lines.append(f"- `{n}` × {msg}\n")
|
||||||
|
lines.append("\n")
|
||||||
|
return "".join(lines)
|
||||||
|
|
||||||
|
lines.append(render_cache_reasons("- base `source=vcir_current_instance`", base_cache_reasons))
|
||||||
|
lines.append(render_cache_reasons("- delta `source=vcir_current_instance`", delta_cache_reasons))
|
||||||
|
|
||||||
|
lines.append("## 文件变更统计(按对象类型)\n\n")
|
||||||
|
lines.append("按 `ObjectAuditEntry.sha256_hex` 对比(同一 rsync URI 前后 hash 变化记为 `~changed`):\n\n")
|
||||||
|
lines.append("| kind | added | changed | removed |\n")
|
||||||
|
lines.append("|---|---:|---:|---:|\n")
|
||||||
|
for kind in ["manifest","crl","certificate","roa","aspa","other"]:
|
||||||
|
st = kind_stats[kind]
|
||||||
|
lines.append(f"| {kind} | {st['added']} | {st['changed']} | {st['removed']} |\n")
|
||||||
|
lines.append("\n")
|
||||||
|
|
||||||
|
lines.append("## VRP 影响(去重后集合 diff)\n\n")
|
||||||
|
lines.append("以 `(asn, prefix, max_length)` 为 key:\n\n")
|
||||||
|
lines.append(f"- base unique VRP:`{len(base_v)}`\n")
|
||||||
|
lines.append(f"- delta unique VRP:`{len(delta_v)}`\n")
|
||||||
|
lines.append(f"- `added={len(added_v)}`\n")
|
||||||
|
lines.append(f"- `removed={len(removed_v)}`\n")
|
||||||
|
lines.append(f"- `net={len(added_v) - len(removed_v)}`\n\n")
|
||||||
|
|
||||||
|
out_md_path.write_text("".join(lines), encoding="utf-8")
|
||||||
|
print(out_md_path)
|
||||||
|
PY
|
||||||
|
|
||||||
|
echo "== done ==" >&2
|
||||||
|
echo "artifacts:" >&2
|
||||||
|
echo "- delta db: $DELTA_DB_DIR" >&2
|
||||||
|
echo "- delta report: $DELTA_REPORT_JSON" >&2
|
||||||
|
echo "- delta run log: $DELTA_RUN_LOG" >&2
|
||||||
|
echo "- delta meta json: $DELTA_META_JSON" >&2
|
||||||
|
echo "- analysis md: $DELTA_ANALYSIS_MD" >&2
|
||||||
|
echo "- base state tsv: $BASE_RRDP_STATE_TSV" >&2
|
||||||
|
echo "- delta state tsv: $DELTA_RRDP_STATE_TSV" >&2
|
||||||
189
scripts/manual_sync/full_sync.sh
Executable file
189
scripts/manual_sync/full_sync.sh
Executable file
@@ -0,0 +1,189 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
# Full sync + validation from a TAL URL (default: APNIC).
|
||||||
|
#
|
||||||
|
# Produces artifacts under OUT_DIR (default: target/live/manual_sync):
|
||||||
|
# - *_db_* RocksDB directory
|
||||||
|
# - *_report_*.json audit report
|
||||||
|
# - *_run_*.log stdout/stderr log (includes summary)
|
||||||
|
# - *_db_stats_*.txt db_stats --exact output
|
||||||
|
# - *_rrdp_state_*.tsv rrdp_state_dump --view legacy-state output
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
||||||
|
cd "$ROOT_DIR"
|
||||||
|
|
||||||
|
TAL_URL="${TAL_URL:-https://tal.apnic.net/tal-archive/apnic-rfc7730-https.tal}"
|
||||||
|
HTTP_TIMEOUT_SECS="${HTTP_TIMEOUT_SECS:-1800}"
|
||||||
|
RSYNC_TIMEOUT_SECS="${RSYNC_TIMEOUT_SECS:-1800}"
|
||||||
|
RSYNC_MIRROR_ROOT="${RSYNC_MIRROR_ROOT:-}"
|
||||||
|
VALIDATION_TIME="${VALIDATION_TIME:-}"
|
||||||
|
|
||||||
|
OUT_DIR="${OUT_DIR:-$ROOT_DIR/target/live/manual_sync}"
|
||||||
|
mkdir -p "$OUT_DIR"
|
||||||
|
|
||||||
|
TS="$(date -u +%Y%m%dT%H%M%SZ)"
|
||||||
|
RUN_NAME="${RUN_NAME:-apnic_full_${TS}}"
|
||||||
|
|
||||||
|
DB_DIR="${DB_DIR:-$OUT_DIR/${RUN_NAME}_db}"
|
||||||
|
REPORT_JSON="${REPORT_JSON:-$OUT_DIR/${RUN_NAME}_report.json}"
|
||||||
|
RUN_LOG="${RUN_LOG:-$OUT_DIR/${RUN_NAME}_run.log}"
|
||||||
|
DB_STATS_TXT="${DB_STATS_TXT:-$OUT_DIR/${RUN_NAME}_db_stats.txt}"
|
||||||
|
RRDP_STATE_TSV="${RRDP_STATE_TSV:-$OUT_DIR/${RUN_NAME}_rrdp_state.tsv}"
|
||||||
|
RUN_META_JSON="${RUN_META_JSON:-$OUT_DIR/${RUN_NAME}_meta.json}"
|
||||||
|
SUMMARY_MD="${SUMMARY_MD:-$OUT_DIR/${RUN_NAME}_summary.md}"
|
||||||
|
|
||||||
|
echo "== rpki manual full sync ==" >&2
|
||||||
|
echo "tal_url=$TAL_URL" >&2
|
||||||
|
echo "db=$DB_DIR" >&2
|
||||||
|
echo "report_json=$REPORT_JSON" >&2
|
||||||
|
echo "out_dir=$OUT_DIR" >&2
|
||||||
|
|
||||||
|
cmd=(cargo run --release --bin rpki -- \
|
||||||
|
--db "$DB_DIR" \
|
||||||
|
--tal-url "$TAL_URL" \
|
||||||
|
--http-timeout-secs "$HTTP_TIMEOUT_SECS" \
|
||||||
|
--rsync-timeout-secs "$RSYNC_TIMEOUT_SECS" \
|
||||||
|
--report-json "$REPORT_JSON")
|
||||||
|
|
||||||
|
if [[ -n "${RSYNC_MIRROR_ROOT}" ]]; then
|
||||||
|
cmd+=(--rsync-mirror-root "$RSYNC_MIRROR_ROOT")
|
||||||
|
fi
|
||||||
|
if [[ -n "${VALIDATION_TIME}" ]]; then
|
||||||
|
cmd+=(--validation-time "$VALIDATION_TIME")
|
||||||
|
fi
|
||||||
|
|
||||||
|
script_start_s="$(date +%s)"
|
||||||
|
run_start_s="$(date +%s)"
|
||||||
|
(
|
||||||
|
echo "# command:"
|
||||||
|
printf '%q ' "${cmd[@]}"
|
||||||
|
echo
|
||||||
|
echo
|
||||||
|
"${cmd[@]}"
|
||||||
|
) 2>&1 | tee "$RUN_LOG" >/dev/null
|
||||||
|
run_end_s="$(date +%s)"
|
||||||
|
run_duration_s="$((run_end_s - run_start_s))"
|
||||||
|
|
||||||
|
echo "== db_stats (exact) ==" >&2
|
||||||
|
db_stats_start_s="$(date +%s)"
|
||||||
|
cargo run --release --bin db_stats -- --db "$DB_DIR" --exact 2>&1 | tee "$DB_STATS_TXT" >/dev/null
|
||||||
|
db_stats_end_s="$(date +%s)"
|
||||||
|
db_stats_duration_s="$((db_stats_end_s - db_stats_start_s))"
|
||||||
|
|
||||||
|
echo "== rrdp_state_dump (legacy-state) ==" >&2
|
||||||
|
state_start_s="$(date +%s)"
|
||||||
|
cargo run --release --bin rrdp_state_dump -- --db "$DB_DIR" --view legacy-state >"$RRDP_STATE_TSV"
|
||||||
|
state_end_s="$(date +%s)"
|
||||||
|
state_duration_s="$((state_end_s - state_start_s))"
|
||||||
|
|
||||||
|
script_end_s="$(date +%s)"
|
||||||
|
total_duration_s="$((script_end_s - script_start_s))"
|
||||||
|
|
||||||
|
echo "== write run meta + summary ==" >&2
|
||||||
|
TAL_URL="$TAL_URL" \
|
||||||
|
DB_DIR="$DB_DIR" \
|
||||||
|
REPORT_JSON="$REPORT_JSON" \
|
||||||
|
RUN_LOG="$RUN_LOG" \
|
||||||
|
HTTP_TIMEOUT_SECS="$HTTP_TIMEOUT_SECS" \
|
||||||
|
RSYNC_TIMEOUT_SECS="$RSYNC_TIMEOUT_SECS" \
|
||||||
|
VALIDATION_TIME_ARG="$VALIDATION_TIME" \
|
||||||
|
RUN_DURATION_S="$run_duration_s" \
|
||||||
|
DB_STATS_DURATION_S="$db_stats_duration_s" \
|
||||||
|
STATE_DURATION_S="$state_duration_s" \
|
||||||
|
TOTAL_DURATION_S="$total_duration_s" \
|
||||||
|
python3 - "$REPORT_JSON" "$RUN_META_JSON" "$SUMMARY_MD" <<'PY'
|
||||||
|
import json
|
||||||
|
import os
|
||||||
|
import sys
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
report_path = Path(sys.argv[1])
|
||||||
|
meta_path = Path(sys.argv[2])
|
||||||
|
summary_path = Path(sys.argv[3])
|
||||||
|
|
||||||
|
rep = json.loads(report_path.read_text(encoding="utf-8"))
|
||||||
|
|
||||||
|
now = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
||||||
|
download_stats = rep.get("download_stats") or {}
|
||||||
|
meta = {
|
||||||
|
"recorded_at_utc": now,
|
||||||
|
"tal_url": os.environ["TAL_URL"],
|
||||||
|
"db_dir": os.environ["DB_DIR"],
|
||||||
|
"report_json": os.environ["REPORT_JSON"],
|
||||||
|
"run_log": os.environ["RUN_LOG"],
|
||||||
|
"validation_time_rfc3339_utc": rep["meta"]["validation_time_rfc3339_utc"],
|
||||||
|
"http_timeout_secs": int(os.environ["HTTP_TIMEOUT_SECS"]),
|
||||||
|
"rsync_timeout_secs": int(os.environ["RSYNC_TIMEOUT_SECS"]),
|
||||||
|
"validation_time_arg": os.environ.get("VALIDATION_TIME_ARG",""),
|
||||||
|
"durations_secs": {
|
||||||
|
"rpki_run": int(os.environ["RUN_DURATION_S"]),
|
||||||
|
"db_stats_exact": int(os.environ["DB_STATS_DURATION_S"]),
|
||||||
|
"rrdp_state_dump": int(os.environ["STATE_DURATION_S"]),
|
||||||
|
"total_script": int(os.environ["TOTAL_DURATION_S"]),
|
||||||
|
},
|
||||||
|
"counts": {
|
||||||
|
"publication_points_processed": rep["tree"]["instances_processed"],
|
||||||
|
"publication_points_failed": rep["tree"]["instances_failed"],
|
||||||
|
"vrps": len(rep["vrps"]),
|
||||||
|
"aspas": len(rep["aspas"]),
|
||||||
|
"audit_publication_points": len(rep["publication_points"]),
|
||||||
|
},
|
||||||
|
"download_stats": download_stats,
|
||||||
|
}
|
||||||
|
|
||||||
|
meta_path.write_text(json.dumps(meta, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
||||||
|
|
||||||
|
lines = []
|
||||||
|
lines.append("# Manual full sync summary\\n\\n")
|
||||||
|
lines.append(f"- recorded_at_utc: `{now}`\\n")
|
||||||
|
lines.append(f"- tal_url: `{meta['tal_url']}`\\n")
|
||||||
|
lines.append(f"- db: `{meta['db_dir']}`\\n")
|
||||||
|
lines.append(f"- report_json: `{meta['report_json']}`\\n")
|
||||||
|
lines.append(f"- validation_time: `{meta['validation_time_rfc3339_utc']}`\\n\\n")
|
||||||
|
lines.append("## Results\\n\\n")
|
||||||
|
lines.append("| metric | value |\\n")
|
||||||
|
lines.append("|---|---:|\\n")
|
||||||
|
for k in ["publication_points_processed","publication_points_failed","vrps","aspas","audit_publication_points"]:
|
||||||
|
lines.append(f"| {k} | {meta['counts'][k]} |\\n")
|
||||||
|
lines.append("\\n")
|
||||||
|
lines.append("## Durations (seconds)\\n\\n")
|
||||||
|
lines.append("| step | seconds |\\n")
|
||||||
|
lines.append("|---|---:|\\n")
|
||||||
|
for k,v in meta["durations_secs"].items():
|
||||||
|
lines.append(f"| {k} | {v} |\\n")
|
||||||
|
lines.append("\\n")
|
||||||
|
|
||||||
|
lines.append("## Download Stats\\n\\n")
|
||||||
|
lines.append("- raw events: `report_json.downloads`\\n")
|
||||||
|
lines.append("- aggregated: `report_json.download_stats` (copied into meta.json)\\n\\n")
|
||||||
|
|
||||||
|
def fmt_u(v):
|
||||||
|
if v is None:
|
||||||
|
return ""
|
||||||
|
return str(v)
|
||||||
|
|
||||||
|
by_kind = download_stats.get("by_kind") or {}
|
||||||
|
lines.append("| kind | ok | fail | duration_ms_total | bytes_total | objects_count_total | objects_bytes_total |\\n")
|
||||||
|
lines.append("|---|---:|---:|---:|---:|---:|---:|\\n")
|
||||||
|
for kind in sorted(by_kind.keys()):
|
||||||
|
st = by_kind[kind] or {}
|
||||||
|
lines.append(
|
||||||
|
f"| {kind} | {st.get('ok_total',0)} | {st.get('fail_total',0)} | {st.get('duration_ms_total',0)} | {fmt_u(st.get('bytes_total'))} | {fmt_u(st.get('objects_count_total'))} | {fmt_u(st.get('objects_bytes_total'))} |\\n"
|
||||||
|
)
|
||||||
|
lines.append("\\n")
|
||||||
|
|
||||||
|
summary_path.write_text("".join(lines), encoding="utf-8")
|
||||||
|
print(summary_path)
|
||||||
|
PY
|
||||||
|
|
||||||
|
echo "== done ==" >&2
|
||||||
|
echo "artifacts:" >&2
|
||||||
|
echo "- db: $DB_DIR" >&2
|
||||||
|
echo "- report: $REPORT_JSON" >&2
|
||||||
|
echo "- run log: $RUN_LOG" >&2
|
||||||
|
echo "- db stats: $DB_STATS_TXT" >&2
|
||||||
|
echo "- rrdp state: $RRDP_STATE_TSV" >&2
|
||||||
|
echo "- meta json: $RUN_META_JSON" >&2
|
||||||
|
echo "- summary md: $SUMMARY_MD" >&2
|
||||||
268
scripts/payload_replay/README.md
Normal file
268
scripts/payload_replay/README.md
Normal file
@@ -0,0 +1,268 @@
|
|||||||
|
# Payload Replay Scripts
|
||||||
|
|
||||||
|
本目录提供基于本地 payload archive 的手工 replay 入口。
|
||||||
|
|
||||||
|
## `multi_rir_case_info.py`
|
||||||
|
|
||||||
|
用于从 multi-RIR bundle 中解析指定 `rir` 的输入路径、对照 CSV、fixture、以及 Routinator replay timing 基线。
|
||||||
|
|
||||||
|
示例:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/payload_replay/multi_rir_case_info.py \
|
||||||
|
--bundle-root ../../rpki/target/live/20260316-112341-multi-final3 \
|
||||||
|
--rir afrinic
|
||||||
|
```
|
||||||
|
|
||||||
|
也支持输出 shell 环境变量:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/payload_replay/multi_rir_case_info.py \
|
||||||
|
--bundle-root ../../rpki/target/live/20260316-112341-multi-final3 \
|
||||||
|
--rir afrinic \
|
||||||
|
--format env
|
||||||
|
```
|
||||||
|
|
||||||
|
## `run_multi_rir_replay_case.sh`
|
||||||
|
|
||||||
|
统一的 multi-RIR 入口。给定 `rir` 和模式后,它会自动选择该 RIR 的:
|
||||||
|
|
||||||
|
- snapshot/base replay 输入
|
||||||
|
- delta replay 输入
|
||||||
|
- 对照 CSV
|
||||||
|
- TAL / TA fixture
|
||||||
|
- trust anchor 名称
|
||||||
|
|
||||||
|
用法:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/payload_replay/run_multi_rir_replay_case.sh <rir> [describe|snapshot|delta|both]
|
||||||
|
```
|
||||||
|
|
||||||
|
示例:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/payload_replay/run_multi_rir_replay_case.sh afrinic describe
|
||||||
|
./scripts/payload_replay/run_multi_rir_replay_case.sh lacnic snapshot
|
||||||
|
./scripts/payload_replay/run_multi_rir_replay_case.sh arin delta
|
||||||
|
./scripts/payload_replay/run_multi_rir_replay_case.sh ripe both
|
||||||
|
```
|
||||||
|
|
||||||
|
|
||||||
|
脚本会自动:
|
||||||
|
|
||||||
|
- 从 multi-RIR bundle 中选择指定 RIR 的 snapshot/base 与 delta 输入
|
||||||
|
- 读取该 RIR 的 Routinator `base-replay` / `delta-replay` timing 基线
|
||||||
|
- 优先使用 `base-locks.json.validationTime` 与 `locks-delta.json.validationTime` 作为 replay `--validation-time`;若缺失才回退到 `timings/base-replay.json` 与 `timings/delta-replay.json` 的 `startedAt`
|
||||||
|
- 在 `target/live/multi_rir_replay_runs/<rir>/` 下生成:
|
||||||
|
- snapshot replay 产物
|
||||||
|
- delta replay 产物
|
||||||
|
- per-RIR 合并 case report(含 correctness + timing compare)
|
||||||
|
|
||||||
|
默认 bundle 根目录为:
|
||||||
|
|
||||||
|
- `../../rpki/target/live/20260316-112341-multi-final3`
|
||||||
|
|
||||||
|
也可以通过 `BUNDLE_ROOT` 覆盖。
|
||||||
|
|
||||||
|
## `run_apnic_snapshot_replay_profile.sh`
|
||||||
|
|
||||||
|
基于 multi-RIR bundle 中的 APNIC snapshot 输入,使用当前 replay 主流程执行一次带 `--analyze` 和 `--profile-cpu` 的离线 profile。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/payload_replay/run_apnic_snapshot_replay_profile.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
作用:
|
||||||
|
|
||||||
|
- 使用 `APNIC` 的 snapshot/base replay 输入
|
||||||
|
- 自动开启:
|
||||||
|
- `--analyze`
|
||||||
|
- `--profile-cpu`
|
||||||
|
- 自动记录:
|
||||||
|
- replay wall-clock 时长
|
||||||
|
- Routinator baseline (`base-replay`)
|
||||||
|
- analyze 目录路径
|
||||||
|
- 生成:
|
||||||
|
- `report.json`
|
||||||
|
- `run.log`
|
||||||
|
- `meta.json`
|
||||||
|
- `summary.md`
|
||||||
|
- 以及 `target/live/analyze/<timestamp>/` 下的:
|
||||||
|
- `timing.json`
|
||||||
|
- `flamegraph.svg`
|
||||||
|
- `pprof.pb.gz`
|
||||||
|
|
||||||
|
支持:
|
||||||
|
|
||||||
|
- `DRY_RUN=1`:只打印命令,不真正执行
|
||||||
|
- `MAX_DEPTH` / `MAX_INSTANCES`:用于限定 replay 范围
|
||||||
|
- `PROFILE_RUN_ROOT`:覆盖 wrapper 产物输出目录
|
||||||
|
|
||||||
|
## `run_apnic_replay.sh`
|
||||||
|
|
||||||
|
默认使用:
|
||||||
|
|
||||||
|
- `tests/fixtures/tal/apnic-rfc7730-https.tal`
|
||||||
|
- `tests/fixtures/ta/apnic-ta.cer`
|
||||||
|
- `target/live/payload_replay/payload-archive`
|
||||||
|
- `target/live/payload_replay/locks.json`
|
||||||
|
|
||||||
|
运行:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/payload_replay/run_apnic_replay.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
产物默认输出到:
|
||||||
|
|
||||||
|
- `target/live/payload_replay_runs/`
|
||||||
|
|
||||||
|
包含:
|
||||||
|
|
||||||
|
- replay DB 目录
|
||||||
|
- `report.json`
|
||||||
|
- `run.log`
|
||||||
|
- `meta.json`
|
||||||
|
- `summary.md`
|
||||||
|
|
||||||
|
## 环境变量
|
||||||
|
|
||||||
|
可覆盖:
|
||||||
|
|
||||||
|
- `TAL_PATH`
|
||||||
|
- `TA_PATH`
|
||||||
|
- `PAYLOAD_REPLAY_ARCHIVE`
|
||||||
|
- `PAYLOAD_REPLAY_LOCKS`
|
||||||
|
- `VALIDATION_TIME`
|
||||||
|
- `MAX_DEPTH`
|
||||||
|
- `MAX_INSTANCES`
|
||||||
|
- `OUT_DIR`
|
||||||
|
- `RUN_NAME`
|
||||||
|
- `DB_DIR`
|
||||||
|
- `REPORT_JSON`
|
||||||
|
- `RUN_LOG`
|
||||||
|
- `META_JSON`
|
||||||
|
- `SUMMARY_MD`
|
||||||
|
|
||||||
|
## 说明
|
||||||
|
|
||||||
|
- 该脚本依赖 `rpki` CLI 已支持:
|
||||||
|
- `--payload-replay-archive`
|
||||||
|
- `--payload-replay-locks`
|
||||||
|
- replay 模式必须搭配离线 TAL/TA 输入,不会去访问真实 RRDP / rsync 网络源。
|
||||||
|
|
||||||
|
## `report_to_routinator_csv.py`
|
||||||
|
|
||||||
|
把 `rpki` 生成的 `report.json` 转成 Routinator 风格的 VRP CSV:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python3 scripts/payload_replay/report_to_routinator_csv.py \
|
||||||
|
--report target/live/payload_replay_runs/<run>_report.json \
|
||||||
|
--out target/live/payload_replay_runs/<run>_vrps.csv \
|
||||||
|
--trust-anchor apnic
|
||||||
|
```
|
||||||
|
|
||||||
|
输出列为:
|
||||||
|
|
||||||
|
- `ASN`
|
||||||
|
- `IP Prefix`
|
||||||
|
- `Max Length`
|
||||||
|
- `Trust Anchor`
|
||||||
|
|
||||||
|
## `compare_with_routinator_record.sh`
|
||||||
|
|
||||||
|
把 ours 生成的 VRP CSV 与 Routinator 的 `record.csv` 做对比:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/payload_replay/compare_with_routinator_record.sh \
|
||||||
|
target/live/payload_replay_runs/<run>_vrps.csv \
|
||||||
|
target/live/payload_replay/record.csv
|
||||||
|
```
|
||||||
|
|
||||||
|
会产出:
|
||||||
|
|
||||||
|
- compare summary Markdown
|
||||||
|
- `only_in_ours.csv`
|
||||||
|
- `only_in_record.csv`
|
||||||
|
|
||||||
|
## `run_apnic_replay.sh` 现有额外产物
|
||||||
|
|
||||||
|
脚本现在除了 `report/meta/summary`,还会额外生成:
|
||||||
|
|
||||||
|
- `vrps.csv`
|
||||||
|
- 若 `ROUTINATOR_RECORD_CSV` 存在,则生成:
|
||||||
|
- compare summary
|
||||||
|
- `only_in_ours.csv`
|
||||||
|
- `only_in_record.csv`
|
||||||
|
|
||||||
|
## `run_apnic_delta_replay.sh`
|
||||||
|
|
||||||
|
使用 APNIC delta demo 数据集运行 base + delta replay:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/payload_replay/run_apnic_delta_replay.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
默认输入:
|
||||||
|
|
||||||
|
- `target/live/apnic_delta_demo/20260315-170223-autoplay/base-payload-archive`
|
||||||
|
- `target/live/apnic_delta_demo/20260315-170223-autoplay/base-locks.json`
|
||||||
|
- `target/live/apnic_delta_demo/20260315-170223-autoplay/payload-delta-archive`
|
||||||
|
- `target/live/apnic_delta_demo/20260315-170223-autoplay/locks-delta.json`
|
||||||
|
- `tests/fixtures/tal/apnic-rfc7730-https.tal`
|
||||||
|
- `tests/fixtures/ta/apnic-ta.cer`
|
||||||
|
|
||||||
|
输出目录默认:`target/live/payload_delta_replay_runs/`
|
||||||
|
|
||||||
|
## `run_apnic_delta_replay.sh` compare outputs
|
||||||
|
|
||||||
|
脚本现在在 delta replay 结束后还会额外生成:
|
||||||
|
|
||||||
|
- `vrps.csv`
|
||||||
|
- compare summary Markdown
|
||||||
|
- `only_in_ours.csv`
|
||||||
|
- `only_in_record.csv`
|
||||||
|
|
||||||
|
默认 compare 输入是:
|
||||||
|
|
||||||
|
- `target/live/apnic_delta_demo/20260315-170223-autoplay/record-delta.csv`
|
||||||
|
|
||||||
|
也可以通过环境变量覆盖:
|
||||||
|
|
||||||
|
- `TRUST_ANCHOR`
|
||||||
|
- `ROUTINATOR_RECORD_CSV`
|
||||||
|
- `VRPS_CSV`
|
||||||
|
- `COMPARE_SUMMARY_MD`
|
||||||
|
- `ONLY_IN_OURS_CSV`
|
||||||
|
- `ONLY_IN_RECORD_CSV`
|
||||||
|
|
||||||
|
## `write_multi_rir_case_report.py`
|
||||||
|
|
||||||
|
把某个 RIR 的 snapshot replay 与 delta replay 的 `meta.json`、compare summary 以及 Routinator timing 基线合并成一个 per-RIR Markdown/JSON 报告。
|
||||||
|
|
||||||
|
该脚本通常由 `run_multi_rir_replay_case.sh <rir> both` 自动调用。
|
||||||
|
|
||||||
|
## `run_multi_rir_replay_suite.sh`
|
||||||
|
|
||||||
|
顺序执行 5 个 RIR(或环境变量 `RIRS` 指定的子集)的 `both` 模式,并最终生成 multi-RIR 汇总报告。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./scripts/payload_replay/run_multi_rir_replay_suite.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
可覆盖环境变量:
|
||||||
|
|
||||||
|
- `BUNDLE_ROOT`
|
||||||
|
- `SUITE_OUT_DIR`
|
||||||
|
- `RIRS`
|
||||||
|
|
||||||
|
最终输出:
|
||||||
|
|
||||||
|
- `<suite_out_dir>/<rir>/<rir>_case_report.md`
|
||||||
|
- `<suite_out_dir>/multi_rir_summary.md`
|
||||||
|
- `<suite_out_dir>/multi_rir_summary.json`
|
||||||
|
|
||||||
|
## `write_multi_rir_summary.py`
|
||||||
|
|
||||||
|
汇总 5 个 RIR 的 per-RIR case report,生成 correctness + timing 总表与几何平均比值。
|
||||||
110
scripts/payload_replay/compare_with_routinator_record.sh
Executable file
110
scripts/payload_replay/compare_with_routinator_record.sh
Executable file
@@ -0,0 +1,110 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
if [[ $# -lt 2 || $# -gt 5 ]]; then
|
||||||
|
echo "Usage: $0 <ours.csv> <record.csv> [summary.md] [only_in_ours.csv] [only_in_record.csv]" >&2
|
||||||
|
exit 2
|
||||||
|
fi
|
||||||
|
|
||||||
|
OURS_CSV="$1"
|
||||||
|
RECORD_CSV="$2"
|
||||||
|
SUMMARY_MD="${3:-}"
|
||||||
|
ONLY_IN_OURS_CSV="${4:-}"
|
||||||
|
ONLY_IN_RECORD_CSV="${5:-}"
|
||||||
|
|
||||||
|
if [[ -z "$SUMMARY_MD" ]]; then
|
||||||
|
SUMMARY_MD="$(dirname "$OURS_CSV")/$(basename "$OURS_CSV" .csv)_vs_routinator_summary.md"
|
||||||
|
fi
|
||||||
|
if [[ -z "$ONLY_IN_OURS_CSV" ]]; then
|
||||||
|
ONLY_IN_OURS_CSV="$(dirname "$OURS_CSV")/$(basename "$OURS_CSV" .csv)_only_in_ours.csv"
|
||||||
|
fi
|
||||||
|
if [[ -z "$ONLY_IN_RECORD_CSV" ]]; then
|
||||||
|
ONLY_IN_RECORD_CSV="$(dirname "$OURS_CSV")/$(basename "$OURS_CSV" .csv)_only_in_record.csv"
|
||||||
|
fi
|
||||||
|
|
||||||
|
python3 - "$OURS_CSV" "$RECORD_CSV" "$SUMMARY_MD" "$ONLY_IN_OURS_CSV" "$ONLY_IN_RECORD_CSV" <<'PY'
|
||||||
|
import csv
|
||||||
|
import ipaddress
|
||||||
|
import sys
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
ours_csv = Path(sys.argv[1])
|
||||||
|
record_csv = Path(sys.argv[2])
|
||||||
|
summary_md = Path(sys.argv[3])
|
||||||
|
only_in_ours_csv = Path(sys.argv[4])
|
||||||
|
only_in_record_csv = Path(sys.argv[5])
|
||||||
|
|
||||||
|
|
||||||
|
def normalize_row(row: dict):
|
||||||
|
asn = row["ASN"].strip().upper()
|
||||||
|
prefix = row["IP Prefix"].strip()
|
||||||
|
max_len = str(int(row["Max Length"]))
|
||||||
|
ta = row["Trust Anchor"].strip()
|
||||||
|
network = ipaddress.ip_network(prefix, strict=False)
|
||||||
|
return {
|
||||||
|
"ASN": asn,
|
||||||
|
"IP Prefix": str(network),
|
||||||
|
"Max Length": max_len,
|
||||||
|
"Trust Anchor": ta,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def read_rows(path: Path):
|
||||||
|
with path.open(encoding="utf-8", newline="") as f:
|
||||||
|
rows = [normalize_row(r) for r in csv.DictReader(f)]
|
||||||
|
return rows
|
||||||
|
|
||||||
|
|
||||||
|
def row_key(row: dict):
|
||||||
|
network = ipaddress.ip_network(row["IP Prefix"], strict=False)
|
||||||
|
return (
|
||||||
|
row["ASN"],
|
||||||
|
network.version,
|
||||||
|
int(network.network_address),
|
||||||
|
network.prefixlen,
|
||||||
|
int(row["Max Length"]),
|
||||||
|
row["Trust Anchor"],
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
|
def write_rows(path: Path, rows):
|
||||||
|
path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
with path.open("w", encoding="utf-8", newline="") as f:
|
||||||
|
writer = csv.DictWriter(f, fieldnames=["ASN", "IP Prefix", "Max Length", "Trust Anchor"])
|
||||||
|
writer.writeheader()
|
||||||
|
for row in rows:
|
||||||
|
writer.writerow(row)
|
||||||
|
|
||||||
|
ours = read_rows(ours_csv)
|
||||||
|
record = read_rows(record_csv)
|
||||||
|
ours_map = {row_key(r): r for r in ours}
|
||||||
|
record_map = {row_key(r): r for r in record}
|
||||||
|
only_in_ours = [ours_map[k] for k in sorted(set(ours_map) - set(record_map))]
|
||||||
|
only_in_record = [record_map[k] for k in sorted(set(record_map) - set(ours_map))]
|
||||||
|
intersection = len(set(ours_map) & set(record_map))
|
||||||
|
|
||||||
|
write_rows(only_in_ours_csv, only_in_ours)
|
||||||
|
write_rows(only_in_record_csv, only_in_record)
|
||||||
|
|
||||||
|
summary_md.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
summary = []
|
||||||
|
summary.append("# Replay vs Routinator VRP Compare\n\n")
|
||||||
|
summary.append(f"- ours_csv: `{ours_csv}`\n")
|
||||||
|
summary.append(f"- record_csv: `{record_csv}`\n")
|
||||||
|
summary.append(f"- only_in_ours_csv: `{only_in_ours_csv}`\n")
|
||||||
|
summary.append(f"- only_in_record_csv: `{only_in_record_csv}`\n\n")
|
||||||
|
summary.append("| metric | value |\n")
|
||||||
|
summary.append("|---|---:|\n")
|
||||||
|
summary.append(f"| ours_total | {len(ours_map)} |\n")
|
||||||
|
summary.append(f"| record_total | {len(record_map)} |\n")
|
||||||
|
summary.append(f"| intersection | {intersection} |\n")
|
||||||
|
summary.append(f"| only_in_ours | {len(only_in_ours)} |\n")
|
||||||
|
summary.append(f"| only_in_record | {len(only_in_record)} |\n")
|
||||||
|
summary_md.write_text("".join(summary), encoding="utf-8")
|
||||||
|
print(summary_md)
|
||||||
|
PY
|
||||||
|
|
||||||
|
echo "== compare complete ==" >&2
|
||||||
|
echo "- summary: $SUMMARY_MD" >&2
|
||||||
|
echo "- only_in_ours: $ONLY_IN_OURS_CSV" >&2
|
||||||
|
echo "- only_in_record: $ONLY_IN_RECORD_CSV" >&2
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user