Compare commits
42 Commits
dev_1.0.0_
...
dev_1.2_xu
| Author | SHA1 | Date | |
|---|---|---|---|
|
4f6597476e | ||
|
|
5fb79a7c8d | ||
|
|
490b79f22c | ||
|
|
e2bdcd3820 | ||
|
|
b5cbc7fbee | ||
|
|
cc3518897f | ||
|
|
b105b3b7d0 | ||
|
|
303bdffd97 | ||
|
|
3c61fbf152 | ||
|
|
b70bad4721 | ||
|
|
fa19fc8722 | ||
|
|
13f29843db | ||
|
|
cdf9372929 | ||
|
|
a11f2bc864 | ||
|
|
ad76745619 | ||
|
|
9dc69e5f31 | ||
|
|
ff711a2fe1 | ||
|
|
4c6b441753 | ||
|
|
a4d674190a | ||
|
|
b79e25ffea | ||
|
|
84866c1d86 | ||
|
|
ddeff71618 | ||
|
|
ca0cf4800a | ||
|
|
897d168ceb | ||
|
|
19225edaa7 | ||
|
|
6e4b59a208 | ||
|
|
88c11e7a97 | ||
|
|
8a65f639cb | ||
|
|
cef9495534 | ||
|
|
b60d579a38 | ||
|
|
99250f8aa9 | ||
|
|
17c1a02c90 | ||
|
|
c1d3112a45 | ||
|
|
23cdad095d | ||
|
|
b483c2af8b | ||
|
|
cd0330e8ae | ||
|
|
8d6173f605 | ||
|
|
7432fbcac4 | ||
|
|
03c0ab0ec7 | ||
|
|
9cbea4e2d0 | ||
|
|
251dea8e5e | ||
|
f5b7da921a |
9
.dockerignore
Normal file
9
.dockerignore
Normal file
@@ -0,0 +1,9 @@
|
|||||||
|
.git
|
||||||
|
.gitignore
|
||||||
|
.idea
|
||||||
|
target
|
||||||
|
rtr-db
|
||||||
|
tests
|
||||||
|
specs
|
||||||
|
scripts
|
||||||
|
README.md
|
||||||
6
.gitignore
vendored
6
.gitignore
vendored
@@ -1,3 +1,5 @@
|
|||||||
target/
|
target/
|
||||||
Cargo.lock
|
rtr-db/
|
||||||
perf.*
|
.idea/
|
||||||
|
logs/
|
||||||
|
report/
|
||||||
|
|||||||
3528
Cargo.lock
generated
Normal file
3528
Cargo.lock
generated
Normal file
File diff suppressed because it is too large
Load Diff
52
Cargo.toml
52
Cargo.toml
@@ -3,34 +3,38 @@ name = "rpki"
|
|||||||
version = "0.1.0"
|
version = "0.1.0"
|
||||||
edition = "2024"
|
edition = "2024"
|
||||||
|
|
||||||
[features]
|
[[bin]]
|
||||||
default = ["full"]
|
name = "rpki_rtr"
|
||||||
# Full build used by the main RP implementation (includes RocksDB-backed storage).
|
path = "src/main_rtr.rs"
|
||||||
full = ["dep:rocksdb"]
|
|
||||||
profile = ["dep:pprof", "dep:flate2"]
|
|
||||||
|
|
||||||
[dependencies]
|
[dependencies]
|
||||||
asn1-rs = "0.7.1"
|
der-parser = "10.0.0"
|
||||||
der-parser = { version = "10.0.0", features = ["serialize"] }
|
|
||||||
hex = "0.4.3"
|
hex = "0.4.3"
|
||||||
base64 = "0.22.1"
|
|
||||||
sha2 = "0.10.8"
|
|
||||||
thiserror = "2.0.18"
|
thiserror = "2.0.18"
|
||||||
time = "0.3.45"
|
time = "0.3.45"
|
||||||
ring = "0.17.14"
|
|
||||||
x509-parser = { version = "0.18.0", features = ["verify"] }
|
x509-parser = { version = "0.18.0", features = ["verify"] }
|
||||||
url = "2.5.8"
|
url = "2.5.8"
|
||||||
serde = { version = "1.0.218", features = ["derive"] }
|
asn1-rs = "0.7.1"
|
||||||
serde_json = "1.0.140"
|
asn1-rs-derive = "0.6.0"
|
||||||
toml = "0.8.20"
|
asn1 = "0.23.0"
|
||||||
rocksdb = { version = "0.22.0", optional = true, default-features = false, features = ["lz4"] }
|
arc-swap = "1.7.0"
|
||||||
serde_cbor = "0.11.2"
|
chrono = { version = "0.4.44", features = ["serde"] }
|
||||||
roxmltree = "0.20.0"
|
chrono-tz = "0.10"
|
||||||
quick-xml = "0.37.2"
|
bytes = "1.11.1"
|
||||||
uuid = { version = "1.7.0", features = ["v4"] }
|
tokio = { version = "1.49.0", features = ["full"] }
|
||||||
reqwest = { version = "0.12.12", default-features = false, features = ["blocking", "rustls-tls"] }
|
rand = "0.10.0"
|
||||||
pprof = { version = "0.14.1", optional = true, features = ["flamegraph", "prost-codec"] }
|
rocksdb = { version = "0.21.0", default-features = false }
|
||||||
flate2 = { version = "1.0.35", optional = true }
|
serde = { version = "1", features = ["derive", "rc"] }
|
||||||
tempfile = "3.16.0"
|
serde_json = "1"
|
||||||
|
base64 = "0.22"
|
||||||
[dev-dependencies]
|
anyhow = "1"
|
||||||
|
tracing = "0.1.44"
|
||||||
|
sha2 = "0.10"
|
||||||
|
tempfile = "3"
|
||||||
|
tokio-rustls = "0.26"
|
||||||
|
rustls = "0.23"
|
||||||
|
rustls-pemfile = "2"
|
||||||
|
rustls-pki-types = "1.14.0"
|
||||||
|
socket2 = "0.5"
|
||||||
|
russh = { version = "0.60.0", default-features = false, features = ["ring", "rsa"] }
|
||||||
|
tracing-subscriber = { version = "0.3", features = ["env-filter", "fmt"] }
|
||||||
|
|||||||
238
README.client.md
Normal file
238
README.client.md
Normal file
@@ -0,0 +1,238 @@
|
|||||||
|
# RTR Client 使用说明
|
||||||
|
|
||||||
|
本文说明仓库内三类 RTR Client 的用途、启动方式和验证命令：
|
||||||
|
|
||||||
|
- `deploy/client`: 本仓库自带的 `rtr_debug_client`，适合做 RTR 协议调试、连通性检查和单次查询。
|
||||||
|
- `deploy/frr`: FRR 作为黑盒 RTR Client，适合验证真实路由软件是否能消费 server 下发的 VRP。
|
||||||
|
- `deploy/bird`: BIRD 作为黑盒 RTR Client，适合验证 TCP 或 SSH transport 下的 RPKI/ROA/ASPA 数据导入。
|
||||||
|
|
||||||
|
以下命令默认从仓库根目录执行。
|
||||||
|
|
||||||
|
## 目录
|
||||||
|
|
||||||
|
- [前置条件](#前置条件)
|
||||||
|
- [rtr_debug_client](#rtr_debug_client)
|
||||||
|
- [TCP](#tcp)
|
||||||
|
- [TLS/mTLS](#tlsmtls)
|
||||||
|
- [SSH](#ssh)
|
||||||
|
- [多客户端压测](#多客户端压测)
|
||||||
|
- [FRR Client](#frr-client)
|
||||||
|
- [BIRD Client](#bird-client)
|
||||||
|
- [TCP](#tcp-1)
|
||||||
|
- [SSH](#ssh-1)
|
||||||
|
- [如何判断 Client 正常](#如何判断-client-正常)
|
||||||
|
|
||||||
|
## 前置条件
|
||||||
|
|
||||||
|
先启动 RTR Server。TCP 模式是最常用的本地验证方式：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./deploy/server/build-run.sh up --mode tcp
|
||||||
|
```
|
||||||
|
|
||||||
|
如果要测试 TLS 或 SSH，需要先启动对应 server 模式，并准备匹配的证书或 SSH key：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./deploy/server/build-run.sh up --mode tls
|
||||||
|
./deploy/server/build-run.sh up --mode ssh
|
||||||
|
```
|
||||||
|
|
||||||
|
TLS/SSH 所需的本地开发证书和密钥可以通过脚本生成：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash scripts/generate-certs.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
已有 `certs/` 时使用 `--force` 覆盖。脚本生成的 `certs/tls` 和 `certs/ssh` 文件名与本仓库 compose 默认配置一致。
|
||||||
|
|
||||||
|
Client 连接地址不要求固定在某个 Docker 网络里。`deploy/client` 的 TCP/TLS/SSH 模式分别通过 `RPKI_RTR_TCP_SERVER_ADDR`、`RPKI_RTR_TLS_SERVER_ADDR`、`RPKI_RTR_SSH_SERVER_ADDR` 指定 server 地址。`deploy/bird` 通过 `RPKI_BIRD_RPKI_HOST`/`RPKI_BIRD_RPKI_PORT` 指定 TCP server 地址，通过 `RPKI_BIRD_SSH_RPKI_HOST`/`RPKI_RTR_SSH_PORT` 指定 SSH server 地址。示例 compose 里的默认值只是为了和本仓库 server compose 直接联调。
|
||||||
|
|
||||||
|
`deploy/frr` 示例默认连接 `127.0.0.1:323`，可以通过 `RPKI_FRR_RPKI_HOST`/`RPKI_FRR_RPKI_PORT` 覆盖。
|
||||||
|
|
||||||
|
## rtr_debug_client
|
||||||
|
|
||||||
|
`rtr_debug_client` 是本仓库提供的调试客户端。它会发起 RTR Reset Query 或 Serial Query，并打印 server 返回的 PDU 摘要。Docker 配置默认执行：
|
||||||
|
|
||||||
|
```text
|
||||||
|
<server_addr> <protocol_version> reset --keep-after-error --summary-only
|
||||||
|
```
|
||||||
|
|
||||||
|
### TCP
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./deploy/client/build-run.sh up --mode tcp
|
||||||
|
./deploy/client/build-run.sh logs --mode tcp
|
||||||
|
./deploy/client/build-run.sh down --mode tcp
|
||||||
|
```
|
||||||
|
|
||||||
|
默认连接 `rpki-rtr:323`，协议版本默认是 `2`。可以通过 `deploy/client/.env` 或环境变量覆盖：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
RPKI_RTR_TCP_SERVER_ADDR=10.0.0.12:323 \
|
||||||
|
RPKI_RTR_PROTOCOL_VERSION=2 \
|
||||||
|
./deploy/client/build-run.sh up --mode tcp
|
||||||
|
```
|
||||||
|
|
||||||
|
### TLS/mTLS
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./deploy/client/build-run.sh up --mode tls
|
||||||
|
./deploy/client/build-run.sh logs --mode tls
|
||||||
|
./deploy/client/build-run.sh down --mode tls
|
||||||
|
```
|
||||||
|
|
||||||
|
TLS 模式默认读取 `certs/tls` 下的示例证书，并使用 `localhost` 作为 server name。常用变量：
|
||||||
|
|
||||||
|
| 变量 | 默认值 | 说明 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `RPKI_RTR_TLS_SERVER_ADDR` | `rpki-rtr:324` | TLS RTR server 地址 |
|
||||||
|
| `RPKI_RTR_TLS_SERVER_NAME` | `localhost` | 必须匹配 server 证书 SAN dNSName |
|
||||||
|
| `RPKI_RTR_TLS_CA_CERT_PATH` | `/app/certs/client-ca.crt` | 容器内 CA 证书路径 |
|
||||||
|
| `RPKI_RTR_TLS_CLIENT_CERT_PATH` | `/app/certs/client-good.crt` | 容器内 client 证书路径 |
|
||||||
|
| `RPKI_RTR_TLS_CLIENT_KEY_PATH` | `/app/certs/client-good.key` | 容器内 client 私钥路径 |
|
||||||
|
| `RPKI_RTR_TLS_CERTS_HOST_DIR` | `../../certs/tls` | 宿主机 TLS 证书目录 |
|
||||||
|
|
||||||
|
### SSH
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./deploy/client/build-run.sh up --mode ssh
|
||||||
|
./deploy/client/build-run.sh logs --mode ssh
|
||||||
|
./deploy/client/build-run.sh down --mode ssh
|
||||||
|
```
|
||||||
|
|
||||||
|
SSH 模式默认连接 `rpki-rtr-ssh:22`，使用 key 认证，并校验 server public key。常用变量：
|
||||||
|
|
||||||
|
| 变量 | 默认值 | 说明 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `RPKI_RTR_SSH_SERVER_ADDR` | `rpki-rtr:22` | SSH RTR server 地址 |
|
||||||
|
| `RPKI_RTR_SSH_USERNAME` | `rpki-rtr` | SSH 用户名 |
|
||||||
|
| `RPKI_RTR_CLIENT_KEYS_VOLUME` | `../../certs/ssh:/app/certs:ro` | 宿主机 SSH key 目录挂载 |
|
||||||
|
| `RPKI_RTR_CLIENT_KEY_PATH` | `/app/certs/rtr-client.key` | 容器内 client 私钥 |
|
||||||
|
| `RPKI_RTR_SSH_SERVER_PUBKEY_PATH` | `/app/certs/ssh_host_rsa_key.pub` | 容器内 server public key |
|
||||||
|
|
||||||
|
### 多客户端压测
|
||||||
|
|
||||||
|
`--clients multi` 会启动 5 个 TCP debug client，用于观察并发连接、client report 和 server 连接统计。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./deploy/client/build-run.sh up --mode tcp --clients multi
|
||||||
|
./deploy/client/build-run.sh logs --mode tcp --clients multi
|
||||||
|
./deploy/client/build-run.sh down --mode tcp --clients multi
|
||||||
|
```
|
||||||
|
|
||||||
|
## FRR Client
|
||||||
|
|
||||||
|
FRR 示例用于验证标准路由软件能否通过 RTR over TCP 从 server 获取前缀验证数据。
|
||||||
|
|
||||||
|
启动：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
默认连接 `127.0.0.1:323`。可以通过环境变量覆盖：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
RPKI_FRR_RPKI_HOST=10.0.0.12 \
|
||||||
|
RPKI_FRR_RPKI_PORT=323 \
|
||||||
|
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
验证连接和数据：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker exec -it frr-rpki-client vtysh -c "show rpki configuration"
|
||||||
|
docker exec -it frr-rpki-client vtysh -c "show rpki cache-connection"
|
||||||
|
docker exec -it frr-rpki-client vtysh -c "show rpki prefix-table"
|
||||||
|
```
|
||||||
|
|
||||||
|
停止：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml down
|
||||||
|
```
|
||||||
|
|
||||||
|
FRR 容器启动时会读取 `deploy/frr/frr.conf.template`，并生成 `/etc/frr/frr.conf`。模板中的 RTR cache 配置为：
|
||||||
|
|
||||||
|
```text
|
||||||
|
rpki cache tcp ${RPKI_FRR_RPKI_HOST} ${RPKI_FRR_RPKI_PORT} preference ${RPKI_FRR_RPKI_PREFERENCE}
|
||||||
|
```
|
||||||
|
|
||||||
|
如果需要调整 polling、timeout、BGP router-id 等其他 FRR 配置，可以修改 `deploy/frr/frr.conf.template`，或在 compose 中挂载自己的模板和 entrypoint。
|
||||||
|
|
||||||
|
## BIRD Client
|
||||||
|
|
||||||
|
BIRD 示例用于验证 BIRD 3.x 对 RTR v2、ROA 和 ASPA 的导入情况。容器启动后会周期性输出 RPKI 协议状态和表项摘要。
|
||||||
|
|
||||||
|
### TCP
|
||||||
|
|
||||||
|
启动：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose --env-file deploy/bird/.env -f deploy/bird/docker-compose.yml up -d --build
|
||||||
|
```
|
||||||
|
|
||||||
|
查看日志：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker logs -f bird-rpki-client
|
||||||
|
```
|
||||||
|
|
||||||
|
停止：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose --env-file deploy/bird/.env -f deploy/bird/docker-compose.yml down
|
||||||
|
```
|
||||||
|
|
||||||
|
默认连接 `rpki-rtr:323`。可以通过环境变量覆盖：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
RPKI_BIRD_RPKI_HOST=10.0.0.12 \
|
||||||
|
RPKI_BIRD_RPKI_PORT=323 \
|
||||||
|
docker compose --env-file deploy/bird/.env -f deploy/bird/docker-compose.yml up -d --build
|
||||||
|
```
|
||||||
|
|
||||||
|
### SSH
|
||||||
|
|
||||||
|
先启动 server SSH 模式：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./deploy/server/build-run.sh up --mode ssh
|
||||||
|
```
|
||||||
|
|
||||||
|
再启动 BIRD SSH client：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose --env-file deploy/bird/.env \
|
||||||
|
-f deploy/bird/docker-compose.yml \
|
||||||
|
-f deploy/bird/docker-compose.ssh.yml \
|
||||||
|
up -d --build
|
||||||
|
```
|
||||||
|
|
||||||
|
查看日志：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker logs -f bird-rpki-client
|
||||||
|
```
|
||||||
|
|
||||||
|
SSH 模式默认读取 `certs/ssh` 目录下的 key，并使用 `bird.conf.ssh.template` 生成运行时配置。关键默认值：
|
||||||
|
|
||||||
|
| 变量 | 默认值 | 说明 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `RPKI_BIRD_SSH_RPKI_HOST` | `rpki-rtr` | SSH RTR server 主机名 |
|
||||||
|
| `RPKI_RTR_SSH_PORT` | `22` | SSH RTR server 端口 |
|
||||||
|
| `RPKI_BIRD_SSH_CERTS_HOST_DIR` | `../../certs/ssh` | 宿主机 SSH 证书和 key 目录，复用 `rtr-client.key` |
|
||||||
|
|
||||||
|
## 如何判断 Client 正常
|
||||||
|
|
||||||
|
- `rtr_debug_client`: 日志中能看到 Reset Query 成功、EndOfData 和 payload 统计。
|
||||||
|
- FRR: `show rpki cache-connection` 显示已连接，`show rpki prefix-table` 有 VRP 条目。
|
||||||
|
- BIRD: 日志中 RPKI protocol 状态为 `up`，ROA/ASPA 表项摘要不为空。
|
||||||
|
|
||||||
|
如果 client 无法连接，优先检查：
|
||||||
|
|
||||||
|
- server 是否按对应 transport 启动。
|
||||||
|
- client 配置的 server 地址是否从当前容器或主机可达。
|
||||||
|
- TLS 的 server name 是否匹配证书 SAN。
|
||||||
|
- SSH 的用户名、client key 和 server public key 是否匹配。
|
||||||
|
- `RPKI_RTR_PROTOCOL_VERSION` 或 BIRD 配置中的 `min version`/`max version` 是否和 server 能力一致。
|
||||||
377
README.md
377
README.md
@@ -1,28 +1,373 @@
|
|||||||
|
# RPKI RTR Server
|
||||||
|
|
||||||
|
这是一个用 Rust 实现的 RPKI-to-Router (RTR) cache server。服务端从 CCR 文件中读取 RPKI 数据，生成 RTR cache snapshot 和 delta，并通过 RTR 协议提供给路由器或调试客户端。
|
||||||
|
|
||||||
# 单元测试
|
当前代码重点覆盖：
|
||||||
|
|
||||||
```
|
- RTR server：TCP 默认启用，可选启用 TLS/mTLS 和进程内 SSH transport。
|
||||||
|
- 数据输入：从 CCR 目录选择最新 `.ccr` 文件，解析 VRP 和 VAP/ASPA。
|
||||||
|
- 本地策略：可选加载 SLURM 文件，对 CCR 解析出的 payload 做过滤和本地断言。
|
||||||
|
- 持久化：使用 RocksDB 保存 RTR cache 状态。
|
||||||
|
- 运行报告：输出 source、client、runtime 三类 JSON report。
|
||||||
|
- 运行期管理：可选启用 HTTP Admin API，修改部分 runtime config、触发 source reload、管理 SLURM 文件。
|
||||||
|
- 调试工具：`rtr_debug_client` 支持 TCP、TLS 和 SSH 连接 RTR server。
|
||||||
|
|
||||||
|
## 目录
|
||||||
|
|
||||||
|
- [协议参考](#协议参考)
|
||||||
|
- [项目结构](#项目结构)
|
||||||
|
- [构建与测试](#构建与测试)
|
||||||
|
- [数据输入](#数据输入)
|
||||||
|
- [快速启动](#快速启动)
|
||||||
|
- [前置数据生成](#前置数据生成)
|
||||||
|
- [Docker 运行](#docker-运行)
|
||||||
|
- [跨架构构建](#跨架构构建)
|
||||||
|
- [传输模式](#传输模式)
|
||||||
|
- [主要环境变量](#主要环境变量)
|
||||||
|
- [运行报告](#运行报告)
|
||||||
|
- [Admin API](#admin-api)
|
||||||
|
- [调试客户端](#调试客户端)
|
||||||
|
- [部署入口](#部署入口)
|
||||||
|
- [开发说明](#开发说明)
|
||||||
|
|
||||||
|
## 协议参考
|
||||||
|
|
||||||
|
- RTR: [RFC 6810](https://www.rfc-editor.org/rfc/rfc6810.html), [RFC 8210](https://www.rfc-editor.org/rfc/rfc8210.html)
|
||||||
|
- SLURM: [RFC 8416](https://www.rfc-editor.org/rfc/rfc8416.html)
|
||||||
|
- CCR: [draft-ietf-sidrops-rpki-ccr](https://www.ietf.org/archive/id/draft-ietf-sidrops-rpki-ccr-02.html)
|
||||||
|
|
||||||
|
## 项目结构
|
||||||
|
|
||||||
|
| 路径 | 说明 |
|
||||||
|
| --- | --- |
|
||||||
|
| `src/main_rtr.rs` | RTR server 入口 |
|
||||||
|
| `src/rtr/` | RTR 协议、cache、store、server transport、admin、report |
|
||||||
|
| `src/source/` | CCR/SLURM source 加载流程 |
|
||||||
|
| `src/slurm/` | SLURM 文件解析、合并和策略应用 |
|
||||||
|
| `src/data_model/` | RPKI 相关数据结构 |
|
||||||
|
| `src/bin/rtr_debug_client/` | RTR 调试客户端 |
|
||||||
|
| `deploy/server/` | server Docker 镜像和 compose |
|
||||||
|
| `deploy/client/` | debug client Docker 镜像和 compose |
|
||||||
|
| `deploy/frr/`, `deploy/bird/` | FRR/BIRD 作为 RTR client 的测试部署 |
|
||||||
|
| `tests/` | 单元和集成测试 |
|
||||||
|
|
||||||
|
## 构建与测试
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo build --bin rpki_rtr
|
||||||
|
cargo build --bin rtr_debug_client
|
||||||
cargo test
|
cargo test
|
||||||
|
|
||||||
# 查看输出
|
|
||||||
cargo test -- --nocapture
|
|
||||||
```
|
```
|
||||||
|
|
||||||
# 覆盖率（cargo-llvm-cov）
|
服务端 binary 名称是 `rpki_rtr`：
|
||||||
|
|
||||||
安装工具：
|
```bash
|
||||||
|
cargo run --bin rpki_rtr
|
||||||
```
|
|
||||||
rustup component add llvm-tools-preview
|
|
||||||
cargo install cargo-llvm-cov --locked
|
|
||||||
```
|
```
|
||||||
|
|
||||||
统计行覆盖率并要求 >=90%：
|
默认配置会监听 `0.0.0.0:323`，从 `./data` 查找 CCR 文件，并使用 `./rtr-db` 保存 RocksDB 数据。启动前需要确保 CCR 目录里存在可解析的 `.ccr` 文件。
|
||||||
|
|
||||||
```
|
## 数据输入
|
||||||
./scripts/coverage.sh
|
|
||||||
# 或
|
服务端启动和刷新时会读取 `RPKI_RTR_CCR_DIR` 指向的目录：
|
||||||
cargo llvm-cov --fail-under-lines 90
|
|
||||||
|
1. 如果该目录下存在包含 `.ccr` 文件的子目录，先按子目录名排序选择最新子目录。
|
||||||
|
2. 在选中的目录中按文件名排序选择最新 `.ccr` 文件。
|
||||||
|
3. 从 CCR 中解析 VRP 和 VAP/ASPA。
|
||||||
|
4. 如果配置了 `RPKI_RTR_SLURM_DIR`，加载其中启用的 `*.slurm` 文件并应用本地策略。
|
||||||
|
|
||||||
|
## 快速启动
|
||||||
|
|
||||||
|
### 前置数据生成
|
||||||
|
|
||||||
|
TLS/mTLS 或 SSH transport 需要本地证书和密钥。开发和 Docker 联调可用脚本生成 compose 默认引用的最小文件集：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash scripts/10_generate-certs.sh
|
||||||
```
|
```
|
||||||
|
|
||||||
|
如需覆盖已有 `certs/` 内容：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash scripts/10_generate-certs.sh --force
|
||||||
|
```
|
||||||
|
|
||||||
|
`10_generate-certs.sh` 常用参数：
|
||||||
|
|
||||||
|
| 参数 | 说明 |
|
||||||
|
| --- | --- |
|
||||||
|
| `--out-dir DIR` | 指定输出目录，默认生成到 `./certs`。 |
|
||||||
|
| `--server-dns NAME` | 指定 TLS server 证书的 dNSName SAN，默认 `localhost`。 |
|
||||||
|
| `--tls-only` | 只生成 `certs/tls` 下的 TLS 证书和私钥。 |
|
||||||
|
| `--ssh-only` | 只生成 `certs/ssh` 下的 SSH host key、client key 和 `authorized_keys`。 |
|
||||||
|
| `--force` | 覆盖已有文件。未指定时，如果目标文件已存在会直接报错退出。 |
|
||||||
|
| `-h`, `--help` | 查看脚本帮助。 |
|
||||||
|
|
||||||
|
TLS server name 默认是 `localhost`。如果需要其它 DNS 名称，需要同时设置 `scripts/10_generate-certs.sh --server-dns <name>` 和 client 的 `RPKI_RTR_TLS_SERVER_NAME=<name>`。
|
||||||
|
|
||||||
|
### Docker 运行
|
||||||
|
|
||||||
|
推荐使用 `deploy/server/build-run.sh` 和 `deploy/client/build-run.sh` 分别管理 server 与 debug client：
|
||||||
|
以下命令默认从仓库根目录执行。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
chmod +x deploy/server/build-run.sh deploy/client/build-run.sh
|
||||||
|
./deploy/server/build-run.sh up --mode tcp
|
||||||
|
./deploy/client/build-run.sh up --mode tcp
|
||||||
|
./deploy/client/build-run.sh down
|
||||||
|
./deploy/server/build-run.sh down
|
||||||
|
```
|
||||||
|
|
||||||
|
脚本常用参数：
|
||||||
|
|
||||||
|
| 参数/变量 | server | client | 说明 |
|
||||||
|
| --- | --- | --- | --- |
|
||||||
|
| `--mode base|tcp|tls|ssh`, `MODE` | yes | yes | 选择 compose 模式，默认 `tcp`。 |
|
||||||
|
| `--clients single|multi`, `CLIENTS` | no | yes | 单 client 或 5 client，默认 `single`。 |
|
||||||
|
| `--no-build`, `BUILD=0` | yes | yes | 只运行已有镜像，不执行 compose build。 |
|
||||||
|
| `--server-image TAG`, `SERVER_IMAGE` | yes | no | server 镜像名；会传给 compose 的 `RPKI_RTR_SERVER_IMAGE`。 |
|
||||||
|
| `--client-image TAG`, `CLIENT_IMAGE` | no | yes | client 镜像名；会传给 compose 的 `RPKI_RTR_CLIENT_IMAGE`。 |
|
||||||
|
| `--platform VALUE`, `TARGET_PLATFORM` | yes | yes | `buildx` 目标平台，默认 `linux/arm64`。 |
|
||||||
|
| `--push`, `BUILDX_PUSH=1` | yes | yes | `buildx` 构建后推送到 registry；不设置时加载到本地 Docker。 |
|
||||||
|
| `--skip-base-pull`, `SKIP_BASE_IMAGE_PULL=1` | yes | yes | 跳过 buildx 前的基础镜像预拉取。 |
|
||||||
|
|
||||||
|
支持的 action 包括 `up`、`build`、`buildx`、`rebuild`、`down`/`stop`、`restart`、`logs`、`ps`。完整帮助见：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./deploy/server/build-run.sh --help
|
||||||
|
./deploy/client/build-run.sh --help
|
||||||
|
```
|
||||||
|
|
||||||
|
### 跨架构构建
|
||||||
|
示例：x86 开发机本地构建 ARM 镜像，打包后传到 ARM 部署机。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
./deploy/server/build-run.sh buildx \
|
||||||
|
--platform linux/arm64 \
|
||||||
|
--server-image rpki-rtr:arm64
|
||||||
|
|
||||||
|
./deploy/client/build-run.sh buildx \
|
||||||
|
--platform linux/arm64 \
|
||||||
|
--client-image rpki-rtr-debug-client:arm64
|
||||||
|
|
||||||
|
mkdir -p dist/images
|
||||||
|
docker save rpki-rtr:arm64 -o dist/images/rpki-rtr-arm64.tar
|
||||||
|
docker save rpki-rtr-debug-client:arm64 -o dist/images/rpki-rtr-debug-client-arm64.tar
|
||||||
|
|
||||||
|
scp dist/images/*.tar user@arm-host:/tmp/
|
||||||
|
```
|
||||||
|
|
||||||
|
ARM 部署机加载镜像并运行：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker load -i /tmp/rpki-rtr-arm64.tar
|
||||||
|
docker load -i /tmp/rpki-rtr-debug-client-arm64.tar
|
||||||
|
|
||||||
|
SERVER_IMAGE=rpki-rtr:arm64 \
|
||||||
|
BUILD=0 ./deploy/server/build-run.sh up --mode tcp
|
||||||
|
|
||||||
|
CLIENT_IMAGE=rpki-rtr-debug-client:arm64 \
|
||||||
|
BUILD=0 ./deploy/client/build-run.sh up --mode tcp
|
||||||
|
```
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f deploy/server/docker-compose.yml up -d --build
|
||||||
|
docker compose -f deploy/server/docker-compose.yml down
|
||||||
|
```
|
||||||
|
|
||||||
|
TLS/mTLS 示例：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f deploy/server/docker-compose.tls.yml up -d --build
|
||||||
|
docker compose -f deploy/server/docker-compose.tls.yml down
|
||||||
|
```
|
||||||
|
|
||||||
|
## 传输模式
|
||||||
|
|
||||||
|
TCP 默认启用。TLS 和 SSH 是附加监听，不会替代 TCP。
|
||||||
|
|
||||||
|
| 模式 | 默认地址 | 启用方式 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| TCP | `0.0.0.0:323` | 默认启用 |
|
||||||
|
| TLS/mTLS | `0.0.0.0:324` | `RPKI_RTR_ENABLE_TLS=true` |
|
||||||
|
| SSH | `0.0.0.0:22` | `RPKI_RTR_ENABLE_SSH=true` |
|
||||||
|
|
||||||
|
TLS server 会加载服务端证书、私钥和 client CA。SSH server 使用 OpenSSH host key 和 authorized_keys 文件；`RPKI_RTR_SSH_AUTH_MODE` 支持 `key`、`password`、`both`。
|
||||||
|
|
||||||
|
## 主要环境变量
|
||||||
|
|
||||||
|
布尔值支持 `true/false`、`1/0`、`yes/no`、`on/off`。
|
||||||
|
|
||||||
|
| 变量 | 默认值 | 说明 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `RPKI_RTR_ENABLE_TLS` | `false` | 是否启用 TLS/mTLS 监听 |
|
||||||
|
| `RPKI_RTR_ENABLE_SSH` | `false` | 是否启用 SSH 监听 |
|
||||||
|
| `RPKI_RTR_TCP_ADDR` | `0.0.0.0:323` | TCP 监听地址 |
|
||||||
|
| `RPKI_RTR_TLS_ADDR` | `0.0.0.0:324` | TLS 监听地址 |
|
||||||
|
| `RPKI_RTR_SSH_ADDR` | `0.0.0.0:22` | SSH 监听地址 |
|
||||||
|
| `RPKI_RTR_SSH_PORT` | 未设置 | 仅覆盖 `RPKI_RTR_SSH_ADDR` 中的端口 |
|
||||||
|
| `RPKI_RTR_DB_PATH` | `./rtr-db` | RocksDB 路径 |
|
||||||
|
| `RPKI_RTR_CCR_DIR` | `./data` | CCR 输入目录 |
|
||||||
|
| `RPKI_RTR_SLURM_DIR` | 未设置 | SLURM 目录；空值表示禁用 |
|
||||||
|
| `RPKI_RTR_REPORT_DIR` | `./report` | JSON report 输出目录 |
|
||||||
|
| `RPKI_RTR_MAX_DELTA` | `100` | 每个 RTR 版本保留的最大 delta 数，必须大于 0 |
|
||||||
|
| `RPKI_RTR_PRUNE_DELTA_BY_SNAPSHOT_SIZE` | `false` | 是否按 snapshot wire size 继续裁剪 delta window |
|
||||||
|
| `RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS` | `300` | source 刷新间隔，单位秒 |
|
||||||
|
| `RPKI_RTR_REFRESH_INTERVAL_SECS` | 未设置 | 旧变量名，仍兼容；建议使用 `RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS` |
|
||||||
|
| `RPKI_RTR_TIMING_REFRESH_SECS` | `3600` | RTR EndOfData `refresh` 字段 |
|
||||||
|
| `RPKI_RTR_TIMING_RETRY_SECS` | `600` | RTR EndOfData `retry` 字段 |
|
||||||
|
| `RPKI_RTR_TIMING_EXPIRE_SECS` | `7200` | RTR EndOfData `expire` 字段，必须大于 refresh 和 retry |
|
||||||
|
| `RPKI_RTR_MAX_CONNECTIONS` | `512` | 最大并发 RTR 连接数 |
|
||||||
|
| `RPKI_RTR_MAX_CONCURRENT_HANDSHAKES` | `128` | 最大并发握手数，不能大于最大连接数 |
|
||||||
|
| `RPKI_RTR_NOTIFY_QUEUE_SIZE` | `1024` | Serial Notify 广播队列大小 |
|
||||||
|
| `RPKI_RTR_TCP_KEEPALIVE_SECS` | `60` | TCP keepalive；设为 `0` 表示禁用 |
|
||||||
|
| `RPKI_RTR_WARN_INSECURE_TCP` | `true` | TCP 模式是否输出安全提示 |
|
||||||
|
| `RPKI_RTR_REQUIRE_TLS_SERVER_DNS_NAME_SAN` | `false` | 是否要求 TLS 服务端证书包含 dNSName SAN |
|
||||||
|
| `RPKI_RTR_ENFORCE_TLS_CLIENT_SAN_IP_MATCH` | `true` | 是否校验 TLS client 证书 SAN IP 与 peer IP 匹配 |
|
||||||
|
| `RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS` | `300` | runtime report 周期 |
|
||||||
|
| `RPKI_RTR_REPORT_HISTORY_LIMIT` | `10` | 每类 report 滚动保留数量 |
|
||||||
|
| `RPKI_RTR_TIMEZONE` | `Asia/Shanghai` | 日志和 report 使用的时区 |
|
||||||
|
| `RPKI_RTR_ADMIN_ADDR` | 未设置 | Admin API 监听地址；未设置时关闭 |
|
||||||
|
| `RPKI_RTR_ADMIN_TOKEN` | 未设置 | Admin API Bearer token；非 loopback 监听时必须设置 |
|
||||||
|
|
||||||
|
TLS 相关路径：
|
||||||
|
|
||||||
|
| 变量 | 默认值 |
|
||||||
|
| --- | --- |
|
||||||
|
| `RPKI_RTR_TLS_CERT_PATH` | `./certs/tls/server-dns.crt` |
|
||||||
|
| `RPKI_RTR_TLS_KEY_PATH` | `./certs/tls/server-dns.key` |
|
||||||
|
| `RPKI_RTR_TLS_CLIENT_CA_PATH` | `./certs/tls/client-ca.crt` |
|
||||||
|
|
||||||
|
SSH 相关配置：
|
||||||
|
|
||||||
|
| 变量 | 默认值 |
|
||||||
|
| --- | --- |
|
||||||
|
| `RPKI_RTR_SSH_HOST_KEY_PATH` | `./certs/ssh/ssh_host_rsa_key` |
|
||||||
|
| `RPKI_RTR_SSH_AUTHORIZED_KEYS_PATH` | `./certs/ssh/rtr-authorized_keys` |
|
||||||
|
| `RPKI_RTR_SSH_USERNAME` | `rpki-rtr` |
|
||||||
|
| `RPKI_RTR_SSH_SUBSYSTEM_NAME` | `rpki-rtr` |
|
||||||
|
| `RPKI_RTR_SSH_AUTH_MODE` | `key` |
|
||||||
|
| `RPKI_RTR_SSH_PASSWORD` | 未设置 |
|
||||||
|
|
||||||
|
## 运行报告
|
||||||
|
|
||||||
|
服务会在 `RPKI_RTR_REPORT_DIR` 下写入 JSON report，并按 `RPKI_RTR_REPORT_HISTORY_LIMIT` 滚动保留：
|
||||||
|
|
||||||
|
- `rtr-source-*.json`：CCR/SLURM source、fingerprint、刷新状态、数据质量、cache 统计。
|
||||||
|
- `rtr-clients-*.json`：client 连接数和连接方式统计。
|
||||||
|
- `rtr-runtime-*.json`：进程状态、服务状态和当前生效的 runtime config。
|
||||||
|
|
||||||
|
## Admin API
|
||||||
|
|
||||||
|
Admin API 默认关闭。设置 `RPKI_RTR_ADMIN_ADDR` 后启用：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
export RPKI_RTR_ADMIN_ADDR=127.0.0.1:8323
|
||||||
|
export RPKI_RTR_ADMIN_TOKEN=change-me
|
||||||
|
```
|
||||||
|
|
||||||
|
如果监听非 loopback 地址，例如 `0.0.0.0:8323`，必须设置 `RPKI_RTR_ADMIN_TOKEN`。设置 token 后，请求需要携带：
|
||||||
|
|
||||||
|
```http
|
||||||
|
Authorization: Bearer change-me
|
||||||
|
```
|
||||||
|
|
||||||
|
当前实现的主要接口：
|
||||||
|
|
||||||
|
| 方法 | 路径 | 说明 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `GET` | `/admin/rtr/health` | 健康检查和能力开关 |
|
||||||
|
| `GET` | `/admin/rtr/config` | 查看 runtime config |
|
||||||
|
| `POST` | `/admin/rtr/config` | 修改部分 runtime config |
|
||||||
|
| `GET` | `/admin/rtr/logs/tail` | 读取 stdout/stderr 日志 tail |
|
||||||
|
| `POST` | `/admin/rtr/slurm/reload` | 触发 source reload |
|
||||||
|
| `GET` | `/admin/rtr/slurm/files` | 列出 SLURM 文件 |
|
||||||
|
| `POST` | `/admin/rtr/slurm/files` | 新增或覆盖 SLURM 文件 |
|
||||||
|
| `GET` | `/admin/rtr/slurm/files/{name}` | 读取 SLURM 文件 |
|
||||||
|
| `PUT` | `/admin/rtr/slurm/files/{name}` | 新增或覆盖指定 SLURM 文件 |
|
||||||
|
| `DELETE` | `/admin/rtr/slurm/files/{name}` | 删除 SLURM 文件 |
|
||||||
|
| `POST` | `/admin/rtr/slurm/files/{name}/enable` | 启用 `.slurm.disabled` 文件 |
|
||||||
|
| `POST` | `/admin/rtr/slurm/files/{name}/disable` | 禁用 `.slurm` 文件 |
|
||||||
|
|
||||||
|
示例：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://127.0.0.1:8323/admin/rtr/config \
|
||||||
|
-H "Authorization: Bearer change-me"
|
||||||
|
|
||||||
|
curl -X POST http://127.0.0.1:8323/admin/rtr/config \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-H "Authorization: Bearer change-me" \
|
||||||
|
-d '{"max_delta": 6, "source_refresh_interval_seconds": 60}'
|
||||||
|
```
|
||||||
|
|
||||||
|
## 调试客户端
|
||||||
|
|
||||||
|
`rtr_debug_client` 用于手动发起 Reset Query 或 Serial Query，并观察服务端返回的 PDU。
|
||||||
|
|
||||||
|
RTR Client 的 Docker 调试、FRR/BIRD 互通验证和常用排障步骤见 `README.client.md`。
|
||||||
|
|
||||||
|
TCP：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo run --bin rtr_debug_client -- 127.0.0.1:323 1 reset
|
||||||
|
cargo run --bin rtr_debug_client -- 127.0.0.1:323 1 serial 42 100
|
||||||
|
```
|
||||||
|
|
||||||
|
TLS：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo run --bin rtr_debug_client -- \
|
||||||
|
127.0.0.1:324 1 reset \
|
||||||
|
--tls \
|
||||||
|
--ca-cert certs/tls/client-ca.crt \
|
||||||
|
--server-name localhost \
|
||||||
|
--client-cert certs/tls/client-good.crt \
|
||||||
|
--client-key certs/tls/client-good.key
|
||||||
|
```
|
||||||
|
|
||||||
|
SSH：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cargo run --bin rtr_debug_client -- \
|
||||||
|
127.0.0.1:22 1 reset \
|
||||||
|
--ssh \
|
||||||
|
--ssh-user rpki-rtr \
|
||||||
|
--ssh-key certs/ssh/rtr-client.key \
|
||||||
|
--ssh-server-key certs/ssh/ssh_host_rsa_key.pub
|
||||||
|
```
|
||||||
|
|
||||||
|
SSH password auth 也已实现；SSH 模式下 `--ssh-key` 和 `--ssh-password` 二选一，同时必须通过 `--ssh-known-hosts` 或 `--ssh-server-key` 做 host key 校验。
|
||||||
|
|
||||||
|
更多参数见 `src/bin/rtr_debug_client/README.md`。
|
||||||
|
|
||||||
|
## 部署入口
|
||||||
|
|
||||||
|
| 用途 | Compose 文件 |
|
||||||
|
| --- | --- |
|
||||||
|
| Server | `deploy/server/docker-compose.yml` |
|
||||||
|
| Server TCP 示例 | `deploy/server/docker-compose.tcp.yml` |
|
||||||
|
| Server TLS 示例 | `deploy/server/docker-compose.tls.yml` |
|
||||||
|
| Server SSH 示例 | `deploy/server/docker-compose.ssh.yml` |
|
||||||
|
| Debug client | `deploy/client/docker-compose.yml` |
|
||||||
|
| Debug client 多实例 | `deploy/client/docker-compose.clients.yml` |
|
||||||
|
| FRR client | `deploy/frr/docker-compose.yml` |
|
||||||
|
| BIRD client | `deploy/bird/docker-compose.yml` |
|
||||||
|
|
||||||
|
RTR Client 统一使用说明见 `README.client.md`。
|
||||||
|
|
||||||
|
通用操作：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
COMPOSE_FILE=deploy/server/docker-compose.yml
|
||||||
|
docker compose -f "${COMPOSE_FILE}" up -d --build
|
||||||
|
docker compose -f "${COMPOSE_FILE}" ps
|
||||||
|
docker compose -f "${COMPOSE_FILE}" down
|
||||||
|
```
|
||||||
|
|
||||||
|
## 开发说明
|
||||||
|
|
||||||
|
- 代码中的 RTR cache 按协议版本维护状态；v0 只包含 Route Origin，v1 包含 Route Origin 和 Router Key，v2 包含 Route Origin、Router Key 和 ASPA。
|
||||||
|
- 当前 source pipeline 从 CCR 生成 Route Origin 和 ASPA，再可选应用 SLURM。
|
||||||
|
- `RtrStore::save_cache_state_versioned(...)` 是 cache 状态写入 RocksDB 的核心入口，相关边界测试在 `tests/test_store_boundary.rs`。
|
||||||
|
- Admin API 的日志 tail 依赖部署入口写出的 stdout/stderr 日志文件；容器入口脚本会把日志写到 `/app/logs`。
|
||||||
|
|||||||
@@ -1,8 +0,0 @@
|
|||||||
[package]
|
|
||||||
name = "ours-manifest-bench"
|
|
||||||
version = "0.1.0"
|
|
||||||
edition = "2024"
|
|
||||||
|
|
||||||
[dependencies]
|
|
||||||
rpki = { path = "../..", default-features = false }
|
|
||||||
|
|
||||||
@@ -1,145 +0,0 @@
|
|||||||
use rpki::data_model::manifest::ManifestObject;
|
|
||||||
use std::hint::black_box;
|
|
||||||
use std::path::PathBuf;
|
|
||||||
use std::time::Instant;
|
|
||||||
|
|
||||||
#[derive(Debug, Clone)]
|
|
||||||
struct Config {
|
|
||||||
sample: Option<String>,
|
|
||||||
manifest_path: Option<PathBuf>,
|
|
||||||
iterations: u64,
|
|
||||||
warmup_iterations: u64,
|
|
||||||
repeats: u32,
|
|
||||||
}
|
|
||||||
|
|
||||||
fn usage_and_exit() -> ! {
|
|
||||||
eprintln!(
|
|
||||||
"Usage:\n ours-manifest-bench (--sample <name> | --manifest <path>) [--iterations N] [--warmup-iterations N] [--repeats N]\n\nExamples:\n cargo run --release -- --sample small-01 --iterations 20000 --warmup-iterations 2000 --repeats 3\n cargo run --release -- --manifest ../../tests/benchmark/selected_der/small-01.mft"
|
|
||||||
);
|
|
||||||
std::process::exit(2);
|
|
||||||
}
|
|
||||||
|
|
||||||
fn parse_args() -> Config {
|
|
||||||
let mut sample: Option<String> = None;
|
|
||||||
let mut manifest_path: Option<PathBuf> = None;
|
|
||||||
let mut iterations: u64 = 20_000;
|
|
||||||
let mut warmup_iterations: u64 = 2_000;
|
|
||||||
let mut repeats: u32 = 3;
|
|
||||||
|
|
||||||
let mut args = std::env::args().skip(1);
|
|
||||||
while let Some(arg) = args.next() {
|
|
||||||
match arg.as_str() {
|
|
||||||
"--sample" => sample = Some(args.next().unwrap_or_else(|| usage_and_exit())),
|
|
||||||
"--manifest" => {
|
|
||||||
manifest_path = Some(PathBuf::from(args.next().unwrap_or_else(|| usage_and_exit())))
|
|
||||||
}
|
|
||||||
"--iterations" => {
|
|
||||||
iterations = args
|
|
||||||
.next()
|
|
||||||
.unwrap_or_else(|| usage_and_exit())
|
|
||||||
.parse()
|
|
||||||
.unwrap_or_else(|_| usage_and_exit())
|
|
||||||
}
|
|
||||||
"--warmup-iterations" => {
|
|
||||||
warmup_iterations = args
|
|
||||||
.next()
|
|
||||||
.unwrap_or_else(|| usage_and_exit())
|
|
||||||
.parse()
|
|
||||||
.unwrap_or_else(|_| usage_and_exit())
|
|
||||||
}
|
|
||||||
"--repeats" => {
|
|
||||||
repeats = args
|
|
||||||
.next()
|
|
||||||
.unwrap_or_else(|| usage_and_exit())
|
|
||||||
.parse()
|
|
||||||
.unwrap_or_else(|_| usage_and_exit())
|
|
||||||
}
|
|
||||||
"-h" | "--help" => usage_and_exit(),
|
|
||||||
_ => usage_and_exit(),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if sample.is_none() && manifest_path.is_none() {
|
|
||||||
usage_and_exit();
|
|
||||||
}
|
|
||||||
if sample.is_some() && manifest_path.is_some() {
|
|
||||||
usage_and_exit();
|
|
||||||
}
|
|
||||||
|
|
||||||
Config {
|
|
||||||
sample,
|
|
||||||
manifest_path,
|
|
||||||
iterations,
|
|
||||||
warmup_iterations,
|
|
||||||
repeats,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn derive_manifest_path(sample: &str) -> PathBuf {
|
|
||||||
// Assumes current working directory is `rpki/benchmark/ours_manifest_bench`.
|
|
||||||
PathBuf::from(format!("../../tests/benchmark/selected_der/{sample}.mft"))
|
|
||||||
}
|
|
||||||
|
|
||||||
fn main() {
|
|
||||||
let cfg = parse_args();
|
|
||||||
let manifest_path = cfg
|
|
||||||
.manifest_path
|
|
||||||
.clone()
|
|
||||||
.unwrap_or_else(|| derive_manifest_path(cfg.sample.as_deref().unwrap()));
|
|
||||||
|
|
||||||
let bytes = std::fs::read(&manifest_path).unwrap_or_else(|e| {
|
|
||||||
eprintln!("read manifest fixture failed: {e}; path={}", manifest_path.display());
|
|
||||||
std::process::exit(1);
|
|
||||||
});
|
|
||||||
|
|
||||||
let decoded_once = ManifestObject::decode_der(&bytes).unwrap_or_else(|e| {
|
|
||||||
eprintln!("decode failed: {e}; path={}", manifest_path.display());
|
|
||||||
std::process::exit(1);
|
|
||||||
});
|
|
||||||
let file_count = decoded_once.manifest.file_count();
|
|
||||||
|
|
||||||
let mut round_ns_per_op: Vec<f64> = Vec::with_capacity(cfg.repeats as usize);
|
|
||||||
let mut round_ops_per_s: Vec<f64> = Vec::with_capacity(cfg.repeats as usize);
|
|
||||||
|
|
||||||
for _round in 0..cfg.repeats {
|
|
||||||
for _ in 0..cfg.warmup_iterations {
|
|
||||||
let obj = ManifestObject::decode_der(black_box(&bytes)).expect("warmup decode");
|
|
||||||
black_box(obj);
|
|
||||||
}
|
|
||||||
|
|
||||||
let start = Instant::now();
|
|
||||||
for _ in 0..cfg.iterations {
|
|
||||||
let obj = ManifestObject::decode_der(black_box(&bytes)).expect("timed decode");
|
|
||||||
black_box(obj);
|
|
||||||
}
|
|
||||||
let elapsed = start.elapsed();
|
|
||||||
|
|
||||||
let ns_per_op = (elapsed.as_secs_f64() * 1e9) / (cfg.iterations as f64);
|
|
||||||
let ops_per_s = (cfg.iterations as f64) / elapsed.as_secs_f64();
|
|
||||||
round_ns_per_op.push(ns_per_op);
|
|
||||||
round_ops_per_s.push(ops_per_s);
|
|
||||||
}
|
|
||||||
|
|
||||||
let avg_ns_per_op = round_ns_per_op.iter().sum::<f64>() / (round_ns_per_op.len() as f64);
|
|
||||||
let avg_ops_per_s = round_ops_per_s.iter().sum::<f64>() / (round_ops_per_s.len() as f64);
|
|
||||||
|
|
||||||
let sample_name = cfg.sample.clone().unwrap_or_else(|| {
|
|
||||||
manifest_path
|
|
||||||
.file_name()
|
|
||||||
.map(|s| s.to_string_lossy().to_string())
|
|
||||||
.unwrap_or_else(|| manifest_path.display().to_string())
|
|
||||||
});
|
|
||||||
let sample_name = sample_name
|
|
||||||
.strip_suffix(".mft")
|
|
||||||
.unwrap_or(&sample_name)
|
|
||||||
.to_string();
|
|
||||||
|
|
||||||
println!("fixture: {}", manifest_path.display());
|
|
||||||
println!();
|
|
||||||
println!("| sample | avg ns/op | ops/s | file count |");
|
|
||||||
println!("|---|---:|---:|---:|");
|
|
||||||
println!(
|
|
||||||
"| {} | {:.2} | {:.2} | {} |",
|
|
||||||
sample_name, avg_ns_per_op, avg_ops_per_s, file_count
|
|
||||||
);
|
|
||||||
}
|
|
||||||
@@ -1,8 +0,0 @@
|
|||||||
[package]
|
|
||||||
name = "routinator-object-bench"
|
|
||||||
version = "0.1.0"
|
|
||||||
edition = "2024"
|
|
||||||
publish = false
|
|
||||||
|
|
||||||
[dependencies]
|
|
||||||
rpki = { version = "=0.19.1", features = ["repository"] }
|
|
||||||
@@ -1,552 +0,0 @@
|
|||||||
use rpki::repository::cert::Cert;
|
|
||||||
use rpki::repository::crl::Crl;
|
|
||||||
use rpki::repository::manifest::Manifest;
|
|
||||||
use rpki::repository::roa::Roa;
|
|
||||||
use rpki::repository::aspa::Aspa;
|
|
||||||
use rpki::repository::resources::{AsResources, IpResources};
|
|
||||||
|
|
||||||
use std::hint::black_box;
|
|
||||||
use std::path::{Path, PathBuf};
|
|
||||||
use std::time::Instant;
|
|
||||||
|
|
||||||
#[derive(Clone, Copy, Debug, PartialEq, Eq, PartialOrd, Ord)]
|
|
||||||
enum ObjType {
|
|
||||||
Cer,
|
|
||||||
Crl,
|
|
||||||
Manifest,
|
|
||||||
Roa,
|
|
||||||
Aspa,
|
|
||||||
}
|
|
||||||
|
|
||||||
impl ObjType {
|
|
||||||
fn parse(s: &str) -> Result<Self, String> {
|
|
||||||
match s {
|
|
||||||
"cer" => Ok(Self::Cer),
|
|
||||||
"crl" => Ok(Self::Crl),
|
|
||||||
"manifest" => Ok(Self::Manifest),
|
|
||||||
"roa" => Ok(Self::Roa),
|
|
||||||
"aspa" => Ok(Self::Aspa),
|
|
||||||
_ => Err("type must be one of: cer, crl, manifest, roa, aspa".into()),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn as_str(self) -> &'static str {
|
|
||||||
match self {
|
|
||||||
ObjType::Cer => "cer",
|
|
||||||
ObjType::Crl => "crl",
|
|
||||||
ObjType::Manifest => "manifest",
|
|
||||||
ObjType::Roa => "roa",
|
|
||||||
ObjType::Aspa => "aspa",
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn ext(self) -> &'static str {
|
|
||||||
match self {
|
|
||||||
ObjType::Cer => "cer",
|
|
||||||
ObjType::Crl => "crl",
|
|
||||||
ObjType::Manifest => "mft",
|
|
||||||
ObjType::Roa => "roa",
|
|
||||||
ObjType::Aspa => "asa",
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Clone, Debug)]
|
|
||||||
struct Sample {
|
|
||||||
obj_type: ObjType,
|
|
||||||
name: String,
|
|
||||||
path: PathBuf,
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Clone, Debug)]
|
|
||||||
struct Config {
|
|
||||||
dir: PathBuf,
|
|
||||||
type_filter: Option<ObjType>,
|
|
||||||
sample_filter: Option<String>,
|
|
||||||
fixed_iters: Option<u64>,
|
|
||||||
warmup_iters: u64,
|
|
||||||
rounds: u64,
|
|
||||||
min_round_ms: u64,
|
|
||||||
max_adaptive_iters: u64,
|
|
||||||
strict: bool,
|
|
||||||
cert_inspect: bool,
|
|
||||||
out_csv: Option<PathBuf>,
|
|
||||||
out_md: Option<PathBuf>,
|
|
||||||
}
|
|
||||||
|
|
||||||
fn usage_and_exit(err: Option<&str>) -> ! {
|
|
||||||
if let Some(err) = err {
|
|
||||||
eprintln!("error: {err}");
|
|
||||||
eprintln!();
|
|
||||||
}
|
|
||||||
eprintln!(
|
|
||||||
"Usage:\n\
|
|
||||||
cargo run --release --manifest-path rpki/benchmark/routinator_object_bench/Cargo.toml -- [OPTIONS]\n\
|
|
||||||
\n\
|
|
||||||
Options:\n\
|
|
||||||
--dir <PATH> Fixtures root dir (default: ../../tests/benchmark/selected_der_v2)\n\
|
|
||||||
--type <cer|crl|manifest|roa|aspa> Filter by type\n\
|
|
||||||
--sample <NAME> Filter by sample name (e.g. p50)\n\
|
|
||||||
--iters <N> Fixed iterations per round (optional; otherwise adaptive)\n\
|
|
||||||
--warmup-iters <N> Warmup iterations (default: 50)\n\
|
|
||||||
--rounds <N> Rounds (default: 5)\n\
|
|
||||||
--min-round-ms <MS> Adaptive: minimum round time (default: 200)\n\
|
|
||||||
--max-iters <N> Adaptive: maximum iters (default: 1_000_000)\n\
|
|
||||||
--strict <true|false> Strict DER where applicable (default: true)\n\
|
|
||||||
--cert-inspect Also run Cert::inspect_ca/inspect_ee where applicable (default: false)\n\
|
|
||||||
--out-csv <PATH> Write CSV output\n\
|
|
||||||
--out-md <PATH> Write Markdown output\n\
|
|
||||||
"
|
|
||||||
);
|
|
||||||
std::process::exit(2);
|
|
||||||
}
|
|
||||||
|
|
||||||
fn parse_bool(s: &str, name: &str) -> bool {
|
|
||||||
match s {
|
|
||||||
"1" | "true" | "TRUE" | "yes" | "YES" => true,
|
|
||||||
"0" | "false" | "FALSE" | "no" | "NO" => false,
|
|
||||||
_ => usage_and_exit(Some(&format!("{name} must be true/false"))),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn parse_u64(s: &str, name: &str) -> u64 {
|
|
||||||
s.parse::<u64>()
|
|
||||||
.unwrap_or_else(|_| usage_and_exit(Some(&format!("{name} must be an integer"))))
|
|
||||||
}
|
|
||||||
|
|
||||||
fn default_samples_dir() -> PathBuf {
|
|
||||||
PathBuf::from(env!("CARGO_MANIFEST_DIR")).join("../../tests/benchmark/selected_der_v2")
|
|
||||||
}
|
|
||||||
|
|
||||||
fn parse_args() -> Config {
|
|
||||||
let mut dir: PathBuf = default_samples_dir();
|
|
||||||
let mut type_filter: Option<ObjType> = None;
|
|
||||||
let mut sample_filter: Option<String> = None;
|
|
||||||
let mut fixed_iters: Option<u64> = None;
|
|
||||||
let mut warmup_iters: u64 = 50;
|
|
||||||
let mut rounds: u64 = 5;
|
|
||||||
let mut min_round_ms: u64 = 200;
|
|
||||||
let mut max_adaptive_iters: u64 = 1_000_000;
|
|
||||||
let mut strict: bool = true;
|
|
||||||
let mut cert_inspect: bool = false;
|
|
||||||
let mut out_csv: Option<PathBuf> = None;
|
|
||||||
let mut out_md: Option<PathBuf> = None;
|
|
||||||
|
|
||||||
let mut args = std::env::args().skip(1);
|
|
||||||
while let Some(arg) = args.next() {
|
|
||||||
match arg.as_str() {
|
|
||||||
"--dir" => dir = PathBuf::from(args.next().unwrap_or_else(|| usage_and_exit(None))),
|
|
||||||
"--type" => {
|
|
||||||
type_filter = Some(ObjType::parse(
|
|
||||||
&args.next().unwrap_or_else(|| usage_and_exit(None)),
|
|
||||||
)
|
|
||||||
.unwrap_or_else(|e| usage_and_exit(Some(&e))))
|
|
||||||
}
|
|
||||||
"--sample" => {
|
|
||||||
sample_filter = Some(args.next().unwrap_or_else(|| usage_and_exit(None)))
|
|
||||||
}
|
|
||||||
"--iters" => {
|
|
||||||
fixed_iters = Some(parse_u64(
|
|
||||||
&args.next().unwrap_or_else(|| usage_and_exit(None)),
|
|
||||||
"--iters",
|
|
||||||
))
|
|
||||||
}
|
|
||||||
"--warmup-iters" => {
|
|
||||||
warmup_iters = parse_u64(
|
|
||||||
&args.next().unwrap_or_else(|| usage_and_exit(None)),
|
|
||||||
"--warmup-iters",
|
|
||||||
)
|
|
||||||
}
|
|
||||||
"--rounds" => {
|
|
||||||
rounds = parse_u64(&args.next().unwrap_or_else(|| usage_and_exit(None)), "--rounds")
|
|
||||||
}
|
|
||||||
"--min-round-ms" => {
|
|
||||||
min_round_ms = parse_u64(
|
|
||||||
&args.next().unwrap_or_else(|| usage_and_exit(None)),
|
|
||||||
"--min-round-ms",
|
|
||||||
)
|
|
||||||
}
|
|
||||||
"--max-iters" => {
|
|
||||||
max_adaptive_iters = parse_u64(
|
|
||||||
&args.next().unwrap_or_else(|| usage_and_exit(None)),
|
|
||||||
"--max-iters",
|
|
||||||
)
|
|
||||||
}
|
|
||||||
"--strict" => {
|
|
||||||
strict = parse_bool(
|
|
||||||
&args.next().unwrap_or_else(|| usage_and_exit(None)),
|
|
||||||
"--strict",
|
|
||||||
)
|
|
||||||
}
|
|
||||||
"--cert-inspect" => cert_inspect = true,
|
|
||||||
"--out-csv" => out_csv = Some(PathBuf::from(args.next().unwrap_or_else(|| usage_and_exit(None)))),
|
|
||||||
"--out-md" => out_md = Some(PathBuf::from(args.next().unwrap_or_else(|| usage_and_exit(None)))),
|
|
||||||
"-h" | "--help" => usage_and_exit(None),
|
|
||||||
_ => usage_and_exit(Some(&format!("unknown argument: {arg}"))),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
if warmup_iters == 0 {
|
|
||||||
usage_and_exit(Some("--warmup-iters must be > 0"));
|
|
||||||
}
|
|
||||||
if rounds == 0 {
|
|
||||||
usage_and_exit(Some("--rounds must be > 0"));
|
|
||||||
}
|
|
||||||
if min_round_ms == 0 {
|
|
||||||
usage_and_exit(Some("--min-round-ms must be > 0"));
|
|
||||||
}
|
|
||||||
if max_adaptive_iters == 0 {
|
|
||||||
usage_and_exit(Some("--max-iters must be > 0"));
|
|
||||||
}
|
|
||||||
if let Some(n) = fixed_iters {
|
|
||||||
if n == 0 {
|
|
||||||
usage_and_exit(Some("--iters must be > 0"));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Config {
|
|
||||||
dir,
|
|
||||||
type_filter,
|
|
||||||
sample_filter,
|
|
||||||
fixed_iters,
|
|
||||||
warmup_iters,
|
|
||||||
rounds,
|
|
||||||
min_round_ms,
|
|
||||||
max_adaptive_iters,
|
|
||||||
strict,
|
|
||||||
cert_inspect,
|
|
||||||
out_csv,
|
|
||||||
out_md,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn read_samples(root: &Path) -> Vec<Sample> {
|
|
||||||
let mut out = Vec::new();
|
|
||||||
for obj_type in [
|
|
||||||
ObjType::Cer,
|
|
||||||
ObjType::Crl,
|
|
||||||
ObjType::Manifest,
|
|
||||||
ObjType::Roa,
|
|
||||||
ObjType::Aspa,
|
|
||||||
] {
|
|
||||||
let dir = root.join(obj_type.as_str());
|
|
||||||
let rd = match std::fs::read_dir(&dir) {
|
|
||||||
Ok(rd) => rd,
|
|
||||||
Err(_) => continue,
|
|
||||||
};
|
|
||||||
for ent in rd.flatten() {
|
|
||||||
let path = ent.path();
|
|
||||||
if path.extension().and_then(|s| s.to_str()) != Some(obj_type.ext()) {
|
|
||||||
continue;
|
|
||||||
}
|
|
||||||
let name = path
|
|
||||||
.file_stem()
|
|
||||||
.and_then(|s| s.to_str())
|
|
||||||
.unwrap_or("unknown")
|
|
||||||
.to_string();
|
|
||||||
out.push(Sample { obj_type, name, path });
|
|
||||||
}
|
|
||||||
}
|
|
||||||
out.sort_by(|a, b| a.obj_type.cmp(&b.obj_type).then_with(|| a.name.cmp(&b.name)));
|
|
||||||
out
|
|
||||||
}
|
|
||||||
|
|
||||||
fn choose_iters_adaptive<F: FnMut()>(mut op: F, min_round_ms: u64, max_iters: u64) -> u64 {
|
|
||||||
let min_secs = (min_round_ms as f64) / 1e3;
|
|
||||||
let mut iters: u64 = 1;
|
|
||||||
loop {
|
|
||||||
let start = Instant::now();
|
|
||||||
for _ in 0..iters {
|
|
||||||
op();
|
|
||||||
}
|
|
||||||
let elapsed = start.elapsed().as_secs_f64();
|
|
||||||
if elapsed >= min_secs {
|
|
||||||
return iters;
|
|
||||||
}
|
|
||||||
if iters >= max_iters {
|
|
||||||
return iters;
|
|
||||||
}
|
|
||||||
iters = (iters.saturating_mul(2)).min(max_iters);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn count_ip(res: &IpResources) -> u64 {
|
|
||||||
if res.is_inherited() {
|
|
||||||
return 1;
|
|
||||||
}
|
|
||||||
let Ok(blocks) = res.to_blocks() else {
|
|
||||||
return 0;
|
|
||||||
};
|
|
||||||
blocks.iter().count() as u64
|
|
||||||
}
|
|
||||||
|
|
||||||
fn count_as(res: &AsResources) -> u64 {
|
|
||||||
if res.is_inherited() {
|
|
||||||
return 1;
|
|
||||||
}
|
|
||||||
let Ok(blocks) = res.to_blocks() else {
|
|
||||||
return 0;
|
|
||||||
};
|
|
||||||
blocks.iter().count() as u64
|
|
||||||
}
|
|
||||||
|
|
||||||
fn complexity(obj_type: ObjType, bytes: &[u8], strict: bool, cert_inspect: bool) -> u64 {
|
|
||||||
match obj_type {
|
|
||||||
ObjType::Cer => {
|
|
||||||
let cert = Cert::decode(bytes).expect("decode cert");
|
|
||||||
if cert_inspect {
|
|
||||||
if cert.is_ca() {
|
|
||||||
cert.inspect_ca(strict).expect("inspect ca");
|
|
||||||
} else {
|
|
||||||
cert.inspect_ee(strict).expect("inspect ee");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
count_ip(cert.v4_resources())
|
|
||||||
.saturating_add(count_ip(cert.v6_resources()))
|
|
||||||
.saturating_add(count_as(cert.as_resources()))
|
|
||||||
}
|
|
||||||
ObjType::Crl => {
|
|
||||||
let crl = Crl::decode(bytes).expect("decode crl");
|
|
||||||
crl.revoked_certs().iter().count() as u64
|
|
||||||
}
|
|
||||||
ObjType::Manifest => {
|
|
||||||
let mft = Manifest::decode(bytes, strict).expect("decode manifest");
|
|
||||||
if cert_inspect {
|
|
||||||
mft.cert().inspect_ee(strict).expect("inspect ee");
|
|
||||||
}
|
|
||||||
mft.content().len() as u64
|
|
||||||
}
|
|
||||||
ObjType::Roa => {
|
|
||||||
let roa = Roa::decode(bytes, strict).expect("decode roa");
|
|
||||||
if cert_inspect {
|
|
||||||
roa.cert().inspect_ee(strict).expect("inspect ee");
|
|
||||||
}
|
|
||||||
roa.content().iter().count() as u64
|
|
||||||
}
|
|
||||||
ObjType::Aspa => {
|
|
||||||
let asa = Aspa::decode(bytes, strict).expect("decode aspa");
|
|
||||||
if cert_inspect {
|
|
||||||
asa.cert().inspect_ee(strict).expect("inspect ee");
|
|
||||||
}
|
|
||||||
asa.content().provider_as_set().len() as u64
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn decode_profile(obj_type: ObjType, bytes: &[u8], strict: bool, cert_inspect: bool) {
|
|
||||||
match obj_type {
|
|
||||||
ObjType::Cer => {
|
|
||||||
let cert = Cert::decode(black_box(bytes)).expect("decode cert");
|
|
||||||
if cert_inspect {
|
|
||||||
if cert.is_ca() {
|
|
||||||
cert.inspect_ca(strict).expect("inspect ca");
|
|
||||||
} else {
|
|
||||||
cert.inspect_ee(strict).expect("inspect ee");
|
|
||||||
}
|
|
||||||
}
|
|
||||||
black_box(cert);
|
|
||||||
}
|
|
||||||
ObjType::Crl => {
|
|
||||||
let crl = Crl::decode(black_box(bytes)).expect("decode crl");
|
|
||||||
black_box(crl);
|
|
||||||
}
|
|
||||||
ObjType::Manifest => {
|
|
||||||
let mft = Manifest::decode(black_box(bytes), strict).expect("decode manifest");
|
|
||||||
if cert_inspect {
|
|
||||||
mft.cert().inspect_ee(strict).expect("inspect ee");
|
|
||||||
}
|
|
||||||
black_box(mft);
|
|
||||||
}
|
|
||||||
ObjType::Roa => {
|
|
||||||
let roa = Roa::decode(black_box(bytes), strict).expect("decode roa");
|
|
||||||
if cert_inspect {
|
|
||||||
roa.cert().inspect_ee(strict).expect("inspect ee");
|
|
||||||
}
|
|
||||||
black_box(roa);
|
|
||||||
}
|
|
||||||
ObjType::Aspa => {
|
|
||||||
let asa = Aspa::decode(black_box(bytes), strict).expect("decode aspa");
|
|
||||||
if cert_inspect {
|
|
||||||
asa.cert().inspect_ee(strict).expect("inspect ee");
|
|
||||||
}
|
|
||||||
black_box(asa);
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
#[derive(Clone, Debug)]
|
|
||||||
struct ResultRow {
|
|
||||||
obj_type: String,
|
|
||||||
sample: String,
|
|
||||||
size_bytes: usize,
|
|
||||||
complexity: u64,
|
|
||||||
avg_ns_per_op: f64,
|
|
||||||
ops_per_sec: f64,
|
|
||||||
}
|
|
||||||
|
|
||||||
fn render_markdown(title: &str, rows: &[ResultRow]) -> String {
|
|
||||||
let mut out = String::new();
|
|
||||||
out.push_str(&format!("# {title}\n\n"));
|
|
||||||
out.push_str("| type | sample | size_bytes | complexity | avg ns/op | ops/s |\n");
|
|
||||||
out.push_str("|---|---|---:|---:|---:|---:|\n");
|
|
||||||
for r in rows {
|
|
||||||
out.push_str(&format!(
|
|
||||||
"| {} | {} | {} | {} | {:.2} | {:.2} |\n",
|
|
||||||
r.obj_type, r.sample, r.size_bytes, r.complexity, r.avg_ns_per_op, r.ops_per_sec
|
|
||||||
));
|
|
||||||
}
|
|
||||||
out
|
|
||||||
}
|
|
||||||
|
|
||||||
fn render_csv(rows: &[ResultRow]) -> String {
|
|
||||||
let mut out = String::new();
|
|
||||||
out.push_str("type,sample,size_bytes,complexity,avg_ns_per_op,ops_per_sec\n");
|
|
||||||
for r in rows {
|
|
||||||
let sample = r.sample.replace('"', "\"\"");
|
|
||||||
out.push_str(&format!(
|
|
||||||
"{},{},{},{},{:.6},{:.6}\n",
|
|
||||||
r.obj_type,
|
|
||||||
format!("\"{}\"", sample),
|
|
||||||
r.size_bytes,
|
|
||||||
r.complexity,
|
|
||||||
r.avg_ns_per_op,
|
|
||||||
r.ops_per_sec
|
|
||||||
));
|
|
||||||
}
|
|
||||||
out
|
|
||||||
}
|
|
||||||
|
|
||||||
fn create_parent_dirs(path: &Path) {
|
|
||||||
if let Some(parent) = path.parent() {
|
|
||||||
std::fs::create_dir_all(parent).unwrap_or_else(|e| {
|
|
||||||
panic!("create_dir_all {}: {e}", parent.display());
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
fn write_text_file(path: &Path, content: &str) {
|
|
||||||
create_parent_dirs(path);
|
|
||||||
std::fs::write(path, content).unwrap_or_else(|e| panic!("write {}: {e}", path.display()));
|
|
||||||
}
|
|
||||||
|
|
||||||
fn main() {
|
|
||||||
let cfg = parse_args();
|
|
||||||
let mut samples = read_samples(&cfg.dir);
|
|
||||||
if samples.is_empty() {
|
|
||||||
usage_and_exit(Some(&format!(
|
|
||||||
"no samples found under: {}",
|
|
||||||
cfg.dir.display()
|
|
||||||
)));
|
|
||||||
}
|
|
||||||
|
|
||||||
if let Some(t) = cfg.type_filter {
|
|
||||||
samples.retain(|s| s.obj_type == t);
|
|
||||||
if samples.is_empty() {
|
|
||||||
usage_and_exit(Some(&format!("no sample matched --type {}", t.as_str())));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
if let Some(filter) = cfg.sample_filter.as_deref() {
|
|
||||||
samples.retain(|s| s.name == filter);
|
|
||||||
if samples.is_empty() {
|
|
||||||
usage_and_exit(Some(&format!("no sample matched --sample {filter}")));
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
println!("# Routinator baseline (rpki crate) decode benchmark (selected_der_v2)");
|
|
||||||
println!();
|
|
||||||
println!("- dir: {}", cfg.dir.display());
|
|
||||||
println!("- strict: {}", cfg.strict);
|
|
||||||
println!("- cert_inspect: {}", cfg.cert_inspect);
|
|
||||||
if let Some(t) = cfg.type_filter {
|
|
||||||
println!("- type: {}", t.as_str());
|
|
||||||
}
|
|
||||||
if let Some(s) = cfg.sample_filter.as_deref() {
|
|
||||||
println!("- sample: {}", s);
|
|
||||||
}
|
|
||||||
if let Some(n) = cfg.fixed_iters {
|
|
||||||
println!("- iters: {} (fixed)", n);
|
|
||||||
} else {
|
|
||||||
println!(
|
|
||||||
"- warmup: {} iters, rounds: {}, min_round: {}ms (adaptive iters, max {})",
|
|
||||||
cfg.warmup_iters, cfg.rounds, cfg.min_round_ms, cfg.max_adaptive_iters
|
|
||||||
);
|
|
||||||
}
|
|
||||||
if let Some(p) = cfg.out_csv.as_ref() {
|
|
||||||
println!("- out_csv: {}", p.display());
|
|
||||||
}
|
|
||||||
if let Some(p) = cfg.out_md.as_ref() {
|
|
||||||
println!("- out_md: {}", p.display());
|
|
||||||
}
|
|
||||||
println!();
|
|
||||||
|
|
||||||
println!("| type | sample | size_bytes | complexity | avg ns/op | ops/s |");
|
|
||||||
println!("|---|---|---:|---:|---:|---:|");
|
|
||||||
|
|
||||||
let mut rows: Vec<ResultRow> = Vec::with_capacity(samples.len());
|
|
||||||
for sample in &samples {
|
|
||||||
let bytes = std::fs::read(&sample.path)
|
|
||||||
.unwrap_or_else(|e| panic!("read {}: {e}", sample.path.display()));
|
|
||||||
let size_bytes = bytes.len();
|
|
||||||
let complexity = complexity(sample.obj_type, bytes.as_slice(), cfg.strict, cfg.cert_inspect);
|
|
||||||
|
|
||||||
for _ in 0..cfg.warmup_iters {
|
|
||||||
decode_profile(sample.obj_type, bytes.as_slice(), cfg.strict, cfg.cert_inspect);
|
|
||||||
}
|
|
||||||
|
|
||||||
let mut per_round_ns_per_op = Vec::with_capacity(cfg.rounds as usize);
|
|
||||||
for _round in 0..cfg.rounds {
|
|
||||||
let iters = if let Some(n) = cfg.fixed_iters {
|
|
||||||
n
|
|
||||||
} else {
|
|
||||||
choose_iters_adaptive(
|
|
||||||
|| decode_profile(sample.obj_type, bytes.as_slice(), cfg.strict, cfg.cert_inspect),
|
|
||||||
cfg.min_round_ms,
|
|
||||||
cfg.max_adaptive_iters,
|
|
||||||
)
|
|
||||||
};
|
|
||||||
let start = Instant::now();
|
|
||||||
for _ in 0..iters {
|
|
||||||
decode_profile(sample.obj_type, bytes.as_slice(), cfg.strict, cfg.cert_inspect);
|
|
||||||
}
|
|
||||||
let elapsed = start.elapsed();
|
|
||||||
let total_ns = elapsed.as_secs_f64() * 1e9;
|
|
||||||
per_round_ns_per_op.push(total_ns / (iters as f64));
|
|
||||||
}
|
|
||||||
|
|
||||||
let avg_ns = per_round_ns_per_op.iter().sum::<f64>() / (per_round_ns_per_op.len() as f64);
|
|
||||||
let ops_per_sec = 1e9_f64 / avg_ns;
|
|
||||||
|
|
||||||
println!(
|
|
||||||
"| {} | {} | {} | {} | {:.2} | {:.2} |",
|
|
||||||
sample.obj_type.as_str(),
|
|
||||||
sample.name,
|
|
||||||
size_bytes,
|
|
||||||
complexity,
|
|
||||||
avg_ns,
|
|
||||||
ops_per_sec
|
|
||||||
);
|
|
||||||
|
|
||||||
rows.push(ResultRow {
|
|
||||||
obj_type: sample.obj_type.as_str().to_string(),
|
|
||||||
sample: sample.name.clone(),
|
|
||||||
size_bytes,
|
|
||||||
complexity,
|
|
||||||
avg_ns_per_op: avg_ns,
|
|
||||||
ops_per_sec,
|
|
||||||
});
|
|
||||||
}
|
|
||||||
|
|
||||||
if let Some(path) = cfg.out_md.as_ref() {
|
|
||||||
let md = render_markdown(
|
|
||||||
"Routinator baseline (rpki crate) decode+inspect (selected_der_v2)",
|
|
||||||
&rows,
|
|
||||||
);
|
|
||||||
write_text_file(path, &md);
|
|
||||||
eprintln!("Wrote {}", path.display());
|
|
||||||
}
|
|
||||||
if let Some(path) = cfg.out_csv.as_ref() {
|
|
||||||
let csv = render_csv(&rows);
|
|
||||||
write_text_file(path, &csv);
|
|
||||||
eprintln!("Wrote {}", path.display());
|
|
||||||
}
|
|
||||||
}
|
|
||||||
23
data/example.slurm
Normal file
23
data/example.slurm
Normal file
@@ -0,0 +1,23 @@
|
|||||||
|
{
|
||||||
|
"slurmVersion": 2,
|
||||||
|
"validationOutputFilters": {
|
||||||
|
"prefixFilters": [
|
||||||
|
{
|
||||||
|
"prefix": "24.0.0.0/8",
|
||||||
|
"comment": "Filter many VRPs in current CCR sample"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"bgpsecFilters": [],
|
||||||
|
"aspaFilters": [
|
||||||
|
{
|
||||||
|
"customerAsn": 80,
|
||||||
|
"comment": "Filter one ASPA known to exist in current CCR sample"
|
||||||
|
}
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"locallyAddedAssertions": {
|
||||||
|
"prefixAssertions": [],
|
||||||
|
"bgpsecAssertions": [],
|
||||||
|
"aspaAssertions": []
|
||||||
|
}
|
||||||
|
}
|
||||||
33
deploy/bird/.env
Normal file
33
deploy/bird/.env
Normal file
@@ -0,0 +1,33 @@
|
|||||||
|
# Build-time image knob.
|
||||||
|
RPKI_BIRD_VERSION=3.2.1
|
||||||
|
|
||||||
|
# TCP mode target endpoint.
|
||||||
|
RPKI_BIRD_RPKI_HOST=rpki-rtr
|
||||||
|
RPKI_BIRD_RPKI_PORT=323
|
||||||
|
|
||||||
|
# SSH mode target endpoint.
|
||||||
|
RPKI_BIRD_SSH_RPKI_HOST=rpki-rtr
|
||||||
|
RPKI_RTR_SSH_PORT=22
|
||||||
|
|
||||||
|
# Config template paths in container.
|
||||||
|
RPKI_BIRD_CONFIG_TEMPLATE_PATH=/config/bird.conf.template
|
||||||
|
RPKI_BIRD_SSH_CONFIG_TEMPLATE_PATH=/config/bird.conf.ssh.template
|
||||||
|
|
||||||
|
# Observation and output knobs.
|
||||||
|
RPKI_BIRD_OBSERVE_PROTO=rpki_tcp
|
||||||
|
RPKI_BIRD_OBSERVE_MODE=interval
|
||||||
|
RPKI_BIRD_OBSERVE_DEBOUNCE_SECS=1
|
||||||
|
RPKI_BIRD_OBSERVE_INTERVAL=30
|
||||||
|
RPKI_BIRD_OBSERVE_ASPA_TABLE=rtr_aspa
|
||||||
|
RPKI_BIRD_OBSERVE_ROA4_TABLE=rtr_roa_v4
|
||||||
|
RPKI_BIRD_OBSERVE_ROA6_TABLE=rtr_roa_v6
|
||||||
|
RPKI_BIRD_OBSERVE_ASPA_COUNT=3
|
||||||
|
RPKI_BIRD_OBSERVE_ROA4_COUNT=3
|
||||||
|
RPKI_BIRD_OBSERVE_ROA6_COUNT=3
|
||||||
|
RPKI_BIRD_SHOW_ASPA=1
|
||||||
|
RPKI_BIRD_SHOW_ROA4=1
|
||||||
|
RPKI_BIRD_SHOW_ROA6=1
|
||||||
|
|
||||||
|
# Host volume mounts.
|
||||||
|
RPKI_BIRD_LOG_HOST_DIR=../../logs/bird
|
||||||
|
RPKI_BIRD_SSH_CERTS_HOST_DIR=../../certs/ssh
|
||||||
56
deploy/bird/Dockerfile
Normal file
56
deploy/bird/Dockerfile
Normal file
@@ -0,0 +1,56 @@
|
|||||||
|
FROM debian:bookworm-slim AS builder
|
||||||
|
|
||||||
|
ARG BIRD_VERSION=3.2.1
|
||||||
|
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
|
build-essential \
|
||||||
|
bison \
|
||||||
|
flex \
|
||||||
|
m4 \
|
||||||
|
perl \
|
||||||
|
ca-certificates \
|
||||||
|
wget \
|
||||||
|
xz-utils \
|
||||||
|
libreadline-dev \
|
||||||
|
libncurses-dev \
|
||||||
|
libssh-dev \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
WORKDIR /build
|
||||||
|
|
||||||
|
RUN wget -O bird.tar.gz "https://bird.nic.cz/download/bird-${BIRD_VERSION}.tar.gz" \
|
||||||
|
&& tar -xzf bird.tar.gz \
|
||||||
|
&& mv "bird-${BIRD_VERSION}" bird
|
||||||
|
|
||||||
|
WORKDIR /build/bird
|
||||||
|
|
||||||
|
RUN ./configure \
|
||||||
|
--prefix=/usr \
|
||||||
|
--sysconfdir=/etc/bird \
|
||||||
|
--localstatedir=/run \
|
||||||
|
&& make -j"$(nproc)" \
|
||||||
|
&& make install
|
||||||
|
|
||||||
|
FROM debian:bookworm-slim
|
||||||
|
|
||||||
|
RUN apt-get update && apt-get install -y --no-install-recommends \
|
||||||
|
ca-certificates \
|
||||||
|
gettext-base \
|
||||||
|
tzdata \
|
||||||
|
netcat-openbsd \
|
||||||
|
libreadline8 \
|
||||||
|
libncurses6 \
|
||||||
|
libtinfo6 \
|
||||||
|
libssh-4 \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
COPY --from=builder /usr/sbin/bird /usr/sbin/bird
|
||||||
|
COPY --from=builder /usr/sbin/birdc /usr/sbin/birdc
|
||||||
|
COPY --from=builder /etc/bird /etc/bird
|
||||||
|
ENV TZ=Asia/Shanghai
|
||||||
|
|
||||||
|
COPY entrypoint.sh /entrypoint.sh
|
||||||
|
RUN chmod +x /entrypoint.sh \
|
||||||
|
&& mkdir -p /run/bird
|
||||||
|
|
||||||
|
ENTRYPOINT ["/entrypoint.sh"]
|
||||||
90
deploy/bird/README.md
Normal file
90
deploy/bird/README.md
Normal file
@@ -0,0 +1,90 @@
|
|||||||
|
# BIRD Minimal RTR Client Config
|
||||||
|
|
||||||
|
This folder provides a minimal BIRD setup for black-box interop testing
|
||||||
|
against this repository's RTR server defaults.
|
||||||
|
|
||||||
|
Server defaults in this repo:
|
||||||
|
- TCP: `0.0.0.0:323`
|
||||||
|
- SSH: `0.0.0.0:22` (when enabled on server)
|
||||||
|
|
||||||
|
## Files
|
||||||
|
|
||||||
|
- `Dockerfile`: builds a minimal BIRD2 runtime image.
|
||||||
|
- `bird.conf.example`: sample `/etc/bird/bird.conf`.
|
||||||
|
- `bird.conf.ssh.example`: sample SSH transport `/etc/bird/bird.conf`.
|
||||||
|
- `entrypoint.sh`: starts BIRD in foreground mode.
|
||||||
|
- `docker-compose.yml`: one-click local TCP test client.
|
||||||
|
- `docker-compose.ssh.yml`: compose override for SSH transport.
|
||||||
|
|
||||||
|
By default, the container uses event-driven observation and prints snapshots
|
||||||
|
only when BIRD reports RPKI-related changes.
|
||||||
|
|
||||||
|
## Docker quick start
|
||||||
|
|
||||||
|
From repository root:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f deploy/bird/docker-compose.yml up --build
|
||||||
|
```
|
||||||
|
|
||||||
|
Use another terminal to inspect:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker logs -f bird-rpki-client
|
||||||
|
```
|
||||||
|
|
||||||
|
If protocol state is `up`, the RTR client path is working.
|
||||||
|
|
||||||
|
Detached mode:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f deploy/bird/docker-compose.yml up -d --build
|
||||||
|
docker logs -f bird-rpki-client
|
||||||
|
```
|
||||||
|
|
||||||
|
Stop:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f deploy/bird/docker-compose.yml down
|
||||||
|
```
|
||||||
|
|
||||||
|
## SSH quick start
|
||||||
|
|
||||||
|
Start server in SSH mode first:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f deploy/server/docker-compose.ssh.yml up -d --build
|
||||||
|
```
|
||||||
|
|
||||||
|
Start BIRD client with SSH override:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose \
|
||||||
|
-f deploy/bird/docker-compose.yml \
|
||||||
|
-f deploy/bird/docker-compose.ssh.yml \
|
||||||
|
up --build
|
||||||
|
```
|
||||||
|
|
||||||
|
In detached mode, observe with:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker logs -f bird-rpki-client
|
||||||
|
```
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- This setup renders `bird.conf.template` at startup and uses `RPKI_HOST` / `RPKI_PORT`
|
||||||
|
for the RTR endpoint.
|
||||||
|
- `network_mode: host` expects your RTR server to be reachable at
|
||||||
|
`host.docker.internal:323` from the container.
|
||||||
|
- Observation is controlled by env vars:
|
||||||
|
`OBSERVE_MODE` (`event` by default, `interval` as fallback),
|
||||||
|
`OBSERVE_DEBOUNCE_SECS` (default `1`),
|
||||||
|
`OBSERVE_INTERVAL` (seconds, used when `OBSERVE_MODE=interval`),
|
||||||
|
and `OBSERVE_PROTO`.
|
||||||
|
- SSH mode mounts `../../certs/ssh` into `/config/ssh` and expects:
|
||||||
|
`rtr-client.key` and `ssh_host_rsa_key.pub`.
|
||||||
|
- Entrypoint auto-generates `/run/bird/known_hosts` from
|
||||||
|
`/config/ssh/ssh_host_rsa_key.pub` for BIRD SSH host-key verification.
|
||||||
|
- If your environment does not support Docker host networking, switch to a
|
||||||
|
bridge network and replace `remote` addresses accordingly.
|
||||||
80
deploy/bird/README.zh.md
Normal file
80
deploy/bird/README.zh.md
Normal file
@@ -0,0 +1,80 @@
|
|||||||
|
# BIRD 最小化 RTR 客户端配置
|
||||||
|
|
||||||
|
本目录提供最小化 BIRD 配置，用于与本仓库 RTR Server 做黑盒互通测试。
|
||||||
|
|
||||||
|
本仓库默认 RTR 监听地址：
|
||||||
|
- TCP: `0.0.0.0:323`
|
||||||
|
- SSH: `0.0.0.0:22`（仅在 server SSH 模式启用时）
|
||||||
|
|
||||||
|
## 文件说明
|
||||||
|
|
||||||
|
- `Dockerfile`: 构建最小 BIRD 运行镜像（包含 SSH transport 支持）。
|
||||||
|
- `bird.conf.example`: `/etc/bird/bird.conf` 的 TCP 示例。
|
||||||
|
- `bird.conf.ssh.example`: `/etc/bird/bird.conf` 的 SSH transport 示例。
|
||||||
|
- `entrypoint.sh`: 前台启动 BIRD，并周期输出协议快照。
|
||||||
|
- `docker-compose.yml`: TCP 一键启动。
|
||||||
|
- `docker-compose.ssh.yml`: SSH 覆盖配置。
|
||||||
|
|
||||||
|
容器默认每 30 秒向日志输出一次 RPKI 协议状态快照。
|
||||||
|
|
||||||
|
## Docker 快速启动（TCP）
|
||||||
|
|
||||||
|
在仓库根目录执行：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f deploy/bird/docker-compose.yml up --build
|
||||||
|
```
|
||||||
|
|
||||||
|
另开终端查看日志：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker logs -f bird-rpki-client
|
||||||
|
```
|
||||||
|
|
||||||
|
如果协议状态显示 `up`，说明 RTR 客户端链路正常。
|
||||||
|
|
||||||
|
后台模式：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f deploy/bird/docker-compose.yml up -d --build
|
||||||
|
docker logs -f bird-rpki-client
|
||||||
|
```
|
||||||
|
|
||||||
|
停止：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f deploy/bird/docker-compose.yml down
|
||||||
|
```
|
||||||
|
|
||||||
|
## SSH 快速启动
|
||||||
|
|
||||||
|
先启动 server 的 SSH 模式：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f deploy/server/docker-compose.ssh.yml up -d --build
|
||||||
|
```
|
||||||
|
|
||||||
|
再启动 BIRD SSH 客户端：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose \
|
||||||
|
-f deploy/bird/docker-compose.yml \
|
||||||
|
-f deploy/bird/docker-compose.ssh.yml \
|
||||||
|
up --build
|
||||||
|
```
|
||||||
|
|
||||||
|
查看日志：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker logs -f bird-rpki-client
|
||||||
|
```
|
||||||
|
|
||||||
|
## 说明
|
||||||
|
|
||||||
|
- 当前 compose 使用 `network_mode: host`，默认通过 `host.docker.internal` 访问 server。
|
||||||
|
- 观测频率由环境变量控制：`OBSERVE_INTERVAL`（秒，默认 `30`）和 `OBSERVE_PROTO`。
|
||||||
|
- SSH 模式会将 `../../certs/ssh` 挂载到容器 `/config/ssh`，并使用：
|
||||||
|
`rtr-client.key`、`ssh_host_rsa_key.pub`。
|
||||||
|
- 入口脚本会基于 `/config/ssh/ssh_host_rsa_key.pub` 自动生成
|
||||||
|
`/run/bird/known_hosts`，用于 BIRD 的 SSH 主机密钥校验。
|
||||||
|
- 如果你运行在 Docker Desktop（非 Linux 原生 host network 场景），建议改为自定义 bridge 网络并将 `remote` 地址改成可达的 server 容器名或宿主机地址。
|
||||||
31
deploy/bird/_debug_ssh.conf
Normal file
31
deploy/bird/_debug_ssh.conf
Normal file
@@ -0,0 +1,31 @@
|
|||||||
|
log stderr all;
|
||||||
|
debug protocols all;
|
||||||
|
router id 192.0.2.2;
|
||||||
|
|
||||||
|
roa4 table rtr_roa_v4;
|
||||||
|
roa6 table rtr_roa_v6;
|
||||||
|
aspa table rtr_aspa;
|
||||||
|
|
||||||
|
protocol device {
|
||||||
|
}
|
||||||
|
|
||||||
|
protocol rpki rpki_ssh {
|
||||||
|
roa4 { table rtr_roa_v4; };
|
||||||
|
roa6 { table rtr_roa_v6; };
|
||||||
|
aspa { table rtr_aspa; };
|
||||||
|
|
||||||
|
remote "host.docker.internal" port 22;
|
||||||
|
|
||||||
|
min version 2;
|
||||||
|
max version 2;
|
||||||
|
|
||||||
|
refresh 3600;
|
||||||
|
retry 600;
|
||||||
|
expire 7200;
|
||||||
|
|
||||||
|
transport ssh {
|
||||||
|
user "rpki-rtr";
|
||||||
|
bird private key "/config/ssh/rtr-client.key";
|
||||||
|
remote public key "/run/bird/known_hosts";
|
||||||
|
};
|
||||||
|
}
|
||||||
28
deploy/bird/bird.conf
Normal file
28
deploy/bird/bird.conf
Normal file
@@ -0,0 +1,28 @@
|
|||||||
|
log stderr all;
|
||||||
|
router id 192.0.2.2;
|
||||||
|
|
||||||
|
roa4 table rtr_roa_v4;
|
||||||
|
roa6 table rtr_roa_v6;
|
||||||
|
aspa table rtr_aspa;
|
||||||
|
|
||||||
|
protocol device {
|
||||||
|
}
|
||||||
|
|
||||||
|
protocol rpki rpki_tcp {
|
||||||
|
roa4 { table rtr_roa_v4; };
|
||||||
|
roa6 { table rtr_roa_v6; };
|
||||||
|
aspa { table rtr_aspa; };
|
||||||
|
|
||||||
|
remote "host.docker.internal" port 323;
|
||||||
|
|
||||||
|
min version 2;
|
||||||
|
max version 2;
|
||||||
|
|
||||||
|
refresh 3600;
|
||||||
|
retry 600;
|
||||||
|
expire 7200;
|
||||||
|
|
||||||
|
transport tcp {
|
||||||
|
authentication none;
|
||||||
|
};
|
||||||
|
}
|
||||||
28
deploy/bird/bird.conf.example
Normal file
28
deploy/bird/bird.conf.example
Normal file
@@ -0,0 +1,28 @@
|
|||||||
|
log stderr all;
|
||||||
|
router id 192.0.2.2;
|
||||||
|
|
||||||
|
roa4 table rtr_roa_v4;
|
||||||
|
roa6 table rtr_roa_v6;
|
||||||
|
aspa table rtr_aspa;
|
||||||
|
|
||||||
|
protocol device {
|
||||||
|
}
|
||||||
|
|
||||||
|
protocol rpki rpki_tcp {
|
||||||
|
roa4 { table rtr_roa_v4; };
|
||||||
|
roa6 { table rtr_roa_v6; };
|
||||||
|
aspa { table rtr_aspa; };
|
||||||
|
|
||||||
|
remote "host.docker.internal" port 323;
|
||||||
|
|
||||||
|
min version 2;
|
||||||
|
max version 2;
|
||||||
|
|
||||||
|
refresh 3600;
|
||||||
|
retry 600;
|
||||||
|
expire 7200;
|
||||||
|
|
||||||
|
transport tcp {
|
||||||
|
authentication none;
|
||||||
|
};
|
||||||
|
}
|
||||||
30
deploy/bird/bird.conf.ssh.example
Normal file
30
deploy/bird/bird.conf.ssh.example
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
log stderr all;
|
||||||
|
router id 192.0.2.2;
|
||||||
|
|
||||||
|
roa4 table rtr_roa_v4;
|
||||||
|
roa6 table rtr_roa_v6;
|
||||||
|
aspa table rtr_aspa;
|
||||||
|
|
||||||
|
protocol device {
|
||||||
|
}
|
||||||
|
|
||||||
|
protocol rpki rpki_ssh {
|
||||||
|
roa4 { table rtr_roa_v4; };
|
||||||
|
roa6 { table rtr_roa_v6; };
|
||||||
|
aspa { table rtr_aspa; };
|
||||||
|
|
||||||
|
remote "host.docker.internal" port 22;
|
||||||
|
|
||||||
|
min version 2;
|
||||||
|
max version 2;
|
||||||
|
|
||||||
|
refresh 3600;
|
||||||
|
retry 600;
|
||||||
|
expire 7200;
|
||||||
|
|
||||||
|
transport ssh {
|
||||||
|
user "rpki-rtr";
|
||||||
|
bird private key "/config/ssh/rtr-client.key";
|
||||||
|
remote public key "/run/bird/known_hosts";
|
||||||
|
};
|
||||||
|
}
|
||||||
30
deploy/bird/bird.conf.ssh.template
Normal file
30
deploy/bird/bird.conf.ssh.template
Normal file
@@ -0,0 +1,30 @@
|
|||||||
|
log stderr all;
|
||||||
|
router id 192.0.2.2;
|
||||||
|
|
||||||
|
roa4 table rtr_roa_v4;
|
||||||
|
roa6 table rtr_roa_v6;
|
||||||
|
aspa table rtr_aspa;
|
||||||
|
|
||||||
|
protocol device {
|
||||||
|
}
|
||||||
|
|
||||||
|
protocol rpki rpki_ssh {
|
||||||
|
roa4 { table rtr_roa_v4; };
|
||||||
|
roa6 { table rtr_roa_v6; };
|
||||||
|
aspa { table rtr_aspa; };
|
||||||
|
|
||||||
|
remote "${RPKI_HOST}" port ${RPKI_PORT};
|
||||||
|
|
||||||
|
min version 2;
|
||||||
|
max version 2;
|
||||||
|
|
||||||
|
refresh 3600;
|
||||||
|
retry 600;
|
||||||
|
expire 7200;
|
||||||
|
|
||||||
|
transport ssh {
|
||||||
|
user "rpki-rtr";
|
||||||
|
bird private key "/config/ssh/rtr-client.key";
|
||||||
|
remote public key "/run/bird/known_hosts";
|
||||||
|
};
|
||||||
|
}
|
||||||
28
deploy/bird/bird.conf.template
Normal file
28
deploy/bird/bird.conf.template
Normal file
@@ -0,0 +1,28 @@
|
|||||||
|
log stderr all;
|
||||||
|
router id 192.0.2.2;
|
||||||
|
|
||||||
|
roa4 table rtr_roa_v4;
|
||||||
|
roa6 table rtr_roa_v6;
|
||||||
|
aspa table rtr_aspa;
|
||||||
|
|
||||||
|
protocol device {
|
||||||
|
}
|
||||||
|
|
||||||
|
protocol rpki rpki_tcp {
|
||||||
|
roa4 { table rtr_roa_v4; };
|
||||||
|
roa6 { table rtr_roa_v6; };
|
||||||
|
aspa { table rtr_aspa; };
|
||||||
|
|
||||||
|
remote "${RPKI_HOST}" port ${RPKI_PORT};
|
||||||
|
|
||||||
|
min version 2;
|
||||||
|
max version 2;
|
||||||
|
|
||||||
|
refresh 3600;
|
||||||
|
retry 600;
|
||||||
|
expire 7200;
|
||||||
|
|
||||||
|
transport tcp {
|
||||||
|
authentication none;
|
||||||
|
};
|
||||||
|
}
|
||||||
17
deploy/bird/docker-compose.ssh.yml
Normal file
17
deploy/bird/docker-compose.ssh.yml
Normal file
@@ -0,0 +1,17 @@
|
|||||||
|
services:
|
||||||
|
bird-rpki-client:
|
||||||
|
environment:
|
||||||
|
BIRD_CONFIG_TEMPLATE_PATH: "${RPKI_BIRD_SSH_CONFIG_TEMPLATE_PATH:-/config/bird.conf.ssh.template}"
|
||||||
|
RPKI_HOST: "${RPKI_BIRD_SSH_RPKI_HOST:-rpki-rtr}"
|
||||||
|
RPKI_PORT: "${RPKI_RTR_SSH_PORT:-22}"
|
||||||
|
OBSERVE_PROTO: "rpki_ssh"
|
||||||
|
volumes:
|
||||||
|
- ./bird.conf.ssh.template:/config/bird.conf.ssh.template:ro
|
||||||
|
- ${RPKI_BIRD_SSH_CERTS_HOST_DIR:-../../certs/ssh}:/config/ssh:ro
|
||||||
|
networks:
|
||||||
|
- rpki_net
|
||||||
|
|
||||||
|
networks:
|
||||||
|
rpki_net:
|
||||||
|
name: rpki_net
|
||||||
|
driver: bridge
|
||||||
41
deploy/bird/docker-compose.yml
Normal file
41
deploy/bird/docker-compose.yml
Normal file
@@ -0,0 +1,41 @@
|
|||||||
|
services:
|
||||||
|
bird-rpki-client:
|
||||||
|
build:
|
||||||
|
context: .
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
args:
|
||||||
|
BIRD_VERSION: "${RPKI_BIRD_VERSION:-3.2.1}"
|
||||||
|
container_name: bird-rpki-client
|
||||||
|
restart: unless-stopped
|
||||||
|
environment:
|
||||||
|
BIRD_CONFIG_TEMPLATE_PATH: "${RPKI_BIRD_CONFIG_TEMPLATE_PATH:-/config/bird.conf.template}"
|
||||||
|
|
||||||
|
RPKI_HOST: "${RPKI_BIRD_RPKI_HOST:-rpki-rtr}"
|
||||||
|
RPKI_PORT: "${RPKI_BIRD_RPKI_PORT:-323}"
|
||||||
|
|
||||||
|
OBSERVE_PROTO: "${RPKI_BIRD_OBSERVE_PROTO:-rpki_tcp}"
|
||||||
|
OBSERVE_MODE: "${RPKI_BIRD_OBSERVE_MODE:-interval}"
|
||||||
|
OBSERVE_DEBOUNCE_SECS: "${RPKI_BIRD_OBSERVE_DEBOUNCE_SECS:-1}"
|
||||||
|
OBSERVE_INTERVAL: "${RPKI_BIRD_OBSERVE_INTERVAL:-30}"
|
||||||
|
|
||||||
|
OBSERVE_ASPA_TABLE: "${RPKI_BIRD_OBSERVE_ASPA_TABLE:-rtr_aspa}"
|
||||||
|
OBSERVE_ROA4_TABLE: "${RPKI_BIRD_OBSERVE_ROA4_TABLE:-rtr_roa_v4}"
|
||||||
|
OBSERVE_ROA6_TABLE: "${RPKI_BIRD_OBSERVE_ROA6_TABLE:-rtr_roa_v6}"
|
||||||
|
|
||||||
|
OBSERVE_ASPA_COUNT: "${RPKI_BIRD_OBSERVE_ASPA_COUNT:-3}"
|
||||||
|
OBSERVE_ROA4_COUNT: "${RPKI_BIRD_OBSERVE_ROA4_COUNT:-3}"
|
||||||
|
OBSERVE_ROA6_COUNT: "${RPKI_BIRD_OBSERVE_ROA6_COUNT:-3}"
|
||||||
|
|
||||||
|
SHOW_ASPA: "${RPKI_BIRD_SHOW_ASPA:-1}"
|
||||||
|
SHOW_ROA4: "${RPKI_BIRD_SHOW_ROA4:-1}"
|
||||||
|
SHOW_ROA6: "${RPKI_BIRD_SHOW_ROA6:-1}"
|
||||||
|
volumes:
|
||||||
|
- ./bird.conf.template:/config/bird.conf.template:ro
|
||||||
|
- ${RPKI_BIRD_LOG_HOST_DIR:-../../logs/bird}:/app/logs
|
||||||
|
networks:
|
||||||
|
- rpki_net
|
||||||
|
|
||||||
|
networks:
|
||||||
|
rpki_net:
|
||||||
|
name: rpki_net
|
||||||
|
driver: bridge
|
||||||
218
deploy/bird/entrypoint.sh
Normal file
218
deploy/bird/entrypoint.sh
Normal file
@@ -0,0 +1,218 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
mkdir -p /run/bird
|
||||||
|
|
||||||
|
SOCK_PATH="/run/bird/bird.ctl"
|
||||||
|
PROTO="${OBSERVE_PROTO:-rpki_tcp}"
|
||||||
|
INTERVAL="${OBSERVE_INTERVAL:-30}"
|
||||||
|
MODE="${OBSERVE_MODE:-event}"
|
||||||
|
DEBOUNCE_SECS="${OBSERVE_DEBOUNCE_SECS:-1}"
|
||||||
|
RPKI_HOST="${RPKI_HOST:-host.docker.internal}"
|
||||||
|
RPKI_PORT="${RPKI_PORT:-323}"
|
||||||
|
|
||||||
|
BIRD_CONFIG_PATH="${BIRD_CONFIG_PATH:-/config/bird.conf}"
|
||||||
|
BIRD_CONFIG_TEMPLATE_PATH="${BIRD_CONFIG_TEMPLATE_PATH:-/config/bird.conf.template}"
|
||||||
|
|
||||||
|
ASPA_TABLE="${OBSERVE_ASPA_TABLE:-rtr_aspa}"
|
||||||
|
ROA4_TABLE="${OBSERVE_ROA4_TABLE:-rtr_roa_v4}"
|
||||||
|
ROA6_TABLE="${OBSERVE_ROA6_TABLE:-rtr_roa_v6}"
|
||||||
|
|
||||||
|
ASPA_COUNT="${OBSERVE_ASPA_COUNT:-3}"
|
||||||
|
ROA4_COUNT="${OBSERVE_ROA4_COUNT:-3}"
|
||||||
|
ROA6_COUNT="${OBSERVE_ROA6_COUNT:-3}"
|
||||||
|
|
||||||
|
SHOW_ASPA="${SHOW_ASPA:-1}"
|
||||||
|
SHOW_ROA4="${SHOW_ROA4:-1}"
|
||||||
|
SHOW_ROA6="${SHOW_ROA6:-1}"
|
||||||
|
SSH_HOST_PUBKEY_PATH="${SSH_HOST_PUBKEY_PATH:-/config/ssh/ssh_host_rsa_key.pub}"
|
||||||
|
SSH_KNOWN_HOSTS_PATH="${SSH_KNOWN_HOSTS_PATH:-/run/bird/known_hosts}"
|
||||||
|
|
||||||
|
LOG_DIR="${LOG_DIR:-/app/logs}"
|
||||||
|
LOG_NAME="${LOG_NAME:-${HOSTNAME:-bird-rpki-client}}"
|
||||||
|
STDOUT_LOG="${LOG_DIR}/${LOG_NAME}.stdout.log"
|
||||||
|
STDERR_LOG="${LOG_DIR}/${LOG_NAME}.stderr.log"
|
||||||
|
|
||||||
|
mkdir -p "$LOG_DIR"
|
||||||
|
exec >>"$STDOUT_LOG" 2>>"$STDERR_LOG"
|
||||||
|
|
||||||
|
if [ -r "$BIRD_CONFIG_TEMPLATE_PATH" ]; then
|
||||||
|
RENDERED_CONFIG_PATH="/run/bird/bird.generated.conf"
|
||||||
|
envsubst '${RPKI_HOST} ${RPKI_PORT}' < "$BIRD_CONFIG_TEMPLATE_PATH" > "$RENDERED_CONFIG_PATH"
|
||||||
|
BIRD_CONFIG_PATH="$RENDERED_CONFIG_PATH"
|
||||||
|
echo "[entrypoint] rendered config : $BIRD_CONFIG_TEMPLATE_PATH -> $BIRD_CONFIG_PATH"
|
||||||
|
fi
|
||||||
|
|
||||||
|
ensure_ssh_known_hosts() {
|
||||||
|
if [ -s "$SSH_KNOWN_HOSTS_PATH" ]; then
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ ! -r "$SSH_HOST_PUBKEY_PATH" ]; then
|
||||||
|
echo "[entrypoint] WARNING: SSH host key file not found: $SSH_HOST_PUBKEY_PATH"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
set -- $(awk 'NF >= 2 { print $1, $2; exit }' "$SSH_HOST_PUBKEY_PATH")
|
||||||
|
if [ $# -ne 2 ]; then
|
||||||
|
echo "[entrypoint] WARNING: invalid SSH host key format in $SSH_HOST_PUBKEY_PATH"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
key_type="$1"
|
||||||
|
key_data="$2"
|
||||||
|
|
||||||
|
if echo "$key_type" | grep -q '^ssh-'; then
|
||||||
|
{
|
||||||
|
echo "$RPKI_HOST $key_type $key_data"
|
||||||
|
echo "[$RPKI_HOST]:$RPKI_PORT $key_type $key_data"
|
||||||
|
} > "$SSH_KNOWN_HOSTS_PATH"
|
||||||
|
else
|
||||||
|
cp "$SSH_HOST_PUBKEY_PATH" "$SSH_KNOWN_HOSTS_PATH"
|
||||||
|
fi
|
||||||
|
|
||||||
|
chmod 600 "$SSH_KNOWN_HOSTS_PATH" || true
|
||||||
|
echo "[entrypoint] generated known_hosts: $SSH_KNOWN_HOSTS_PATH"
|
||||||
|
}
|
||||||
|
|
||||||
|
print_first_n_objects() {
|
||||||
|
table_name="$1"
|
||||||
|
max_objects="$2"
|
||||||
|
|
||||||
|
birdc -s "$SOCK_PATH" show route table "$table_name" all 2>/dev/null | awk -v max="$max_objects" '
|
||||||
|
BEGIN {
|
||||||
|
count = 0
|
||||||
|
}
|
||||||
|
|
||||||
|
# 直接跳过空行
|
||||||
|
/^[[:space:]]*$/ {
|
||||||
|
next
|
||||||
|
}
|
||||||
|
|
||||||
|
# 保留 birdc 的表头
|
||||||
|
/^BIRD / {
|
||||||
|
print
|
||||||
|
next
|
||||||
|
}
|
||||||
|
|
||||||
|
/^Table / {
|
||||||
|
print
|
||||||
|
next
|
||||||
|
}
|
||||||
|
|
||||||
|
# 非缩进且不是表头，视为一个新对象的开始
|
||||||
|
/^[^[:space:]]/ {
|
||||||
|
count++
|
||||||
|
if (count > max) {
|
||||||
|
exit
|
||||||
|
}
|
||||||
|
print
|
||||||
|
next
|
||||||
|
}
|
||||||
|
|
||||||
|
# 缩进内容，只有在已进入前 max 个对象时才打印
|
||||||
|
{
|
||||||
|
if (count > 0 && count <= max) {
|
||||||
|
print
|
||||||
|
}
|
||||||
|
}
|
||||||
|
' || true
|
||||||
|
}
|
||||||
|
|
||||||
|
print_snapshot() {
|
||||||
|
echo "==== $(date +"%Y-%m-%dT%H:%M:%S%:z") RPKI snapshot ($PROTO) ===="
|
||||||
|
birdc -s "$SOCK_PATH" show protocols all "$PROTO" || true
|
||||||
|
|
||||||
|
if [ "$SHOW_ASPA" = "1" ]; then
|
||||||
|
echo "---- ASPA table ($ASPA_TABLE, first ${ASPA_COUNT} objects) ----"
|
||||||
|
print_first_n_objects "$ASPA_TABLE" "$ASPA_COUNT"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "$SHOW_ROA4" = "1" ]; then
|
||||||
|
echo "---- ROA4 table ($ROA4_TABLE, first ${ROA4_COUNT} objects) ----"
|
||||||
|
print_first_n_objects "$ROA4_TABLE" "$ROA4_COUNT"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "$SHOW_ROA6" = "1" ]; then
|
||||||
|
echo "---- ROA6 table ($ROA6_TABLE, first ${ROA6_COUNT} objects) ----"
|
||||||
|
print_first_n_objects "$ROA6_TABLE" "$ROA6_COUNT"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
is_rpki_related_event() {
|
||||||
|
line="$1"
|
||||||
|
echo "$line" | grep -Eiq "$PROTO|$ASPA_TABLE|$ROA4_TABLE|$ROA6_TABLE|rpki|rtr"
|
||||||
|
}
|
||||||
|
|
||||||
|
echo "[entrypoint] starting bird"
|
||||||
|
echo "[entrypoint] config : $BIRD_CONFIG_PATH"
|
||||||
|
echo "[entrypoint] observe proto : $PROTO"
|
||||||
|
echo "[entrypoint] observe mode : $MODE"
|
||||||
|
echo "[entrypoint] observe interval : $INTERVAL (used when mode=interval)"
|
||||||
|
echo "[entrypoint] debounce secs : $DEBOUNCE_SECS (used when mode=event)"
|
||||||
|
echo "[entrypoint] target : $RPKI_HOST:$RPKI_PORT"
|
||||||
|
echo "[entrypoint] show aspa : $SHOW_ASPA ($ASPA_TABLE, first $ASPA_COUNT objects)"
|
||||||
|
echo "[entrypoint] show roa4 : $SHOW_ROA4 ($ROA4_TABLE, first $ROA4_COUNT objects)"
|
||||||
|
echo "[entrypoint] show roa6 : $SHOW_ROA6 ($ROA6_TABLE, first $ROA6_COUNT objects)"
|
||||||
|
|
||||||
|
if [ "$PROTO" = "rpki_ssh" ] || grep -q 'transport[[:space:]]\+ssh' "$BIRD_CONFIG_PATH"; then
|
||||||
|
ensure_ssh_known_hosts
|
||||||
|
fi
|
||||||
|
|
||||||
|
if nc -zvw3 "$RPKI_HOST" "$RPKI_PORT"; then
|
||||||
|
echo "[entrypoint] TCP connectivity to $RPKI_HOST:$RPKI_PORT OK"
|
||||||
|
else
|
||||||
|
echo "[entrypoint] WARNING: cannot connect to $RPKI_HOST:$RPKI_PORT before BIRD starts"
|
||||||
|
fi
|
||||||
|
|
||||||
|
bird -f -c "$BIRD_CONFIG_PATH" -s "$SOCK_PATH" &
|
||||||
|
BIRD_PID="$!"
|
||||||
|
|
||||||
|
sleep 1
|
||||||
|
|
||||||
|
case "$INTERVAL" in
|
||||||
|
''|*[!0-9]*)
|
||||||
|
INTERVAL=0
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
case "$MODE" in
|
||||||
|
event)
|
||||||
|
case "$DEBOUNCE_SECS" in
|
||||||
|
''|*[!0-9]*)
|
||||||
|
DEBOUNCE_SECS=1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
echo "[entrypoint] waiting for BIRD monitor events and only printing on RPKI-related changes"
|
||||||
|
LAST_PRINT_TS=0
|
||||||
|
|
||||||
|
if birdc -s "$SOCK_PATH" monitor all 2>/dev/null | while IFS= read -r line; do
|
||||||
|
[ -n "$line" ] || continue
|
||||||
|
if ! is_rpki_related_event "$line"; then
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
now_ts="$(date +%s)"
|
||||||
|
if [ $((now_ts - LAST_PRINT_TS)) -lt "$DEBOUNCE_SECS" ]; then
|
||||||
|
continue
|
||||||
|
fi
|
||||||
|
LAST_PRINT_TS="$now_ts"
|
||||||
|
echo "[monitor] $line"
|
||||||
|
print_snapshot
|
||||||
|
done; then
|
||||||
|
:
|
||||||
|
else
|
||||||
|
echo "[entrypoint] WARNING: birdc monitor failed, fallback to interval polling"
|
||||||
|
MODE="interval"
|
||||||
|
fi
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
if [ "$MODE" = "interval" ] && [ "$INTERVAL" -gt 0 ]; then
|
||||||
|
while kill -0 "$BIRD_PID" 2>/dev/null; do
|
||||||
|
print_snapshot
|
||||||
|
sleep "$INTERVAL"
|
||||||
|
done
|
||||||
|
fi
|
||||||
|
|
||||||
|
wait "$BIRD_PID"
|
||||||
32
deploy/client/.env
Normal file
32
deploy/client/.env
Normal file
@@ -0,0 +1,32 @@
|
|||||||
|
# Target RTR server addresses for client compose files
|
||||||
|
# RPKI_RTR_CLIENT_IMAGE=rpki-rtr-debug-client-arm64:v1.2
|
||||||
|
RPKI_RTR_CLIENT_IMAGE=rpki-rtr-debug-client-amd64:v1.2
|
||||||
|
|
||||||
|
# TCP example: 10.0.0.12:323
|
||||||
|
# TLS example: rpki.example.com:324
|
||||||
|
# SSH example: 10.0.0.12:22
|
||||||
|
RPKI_RTR_TCP_SERVER_ADDR=rpki-rtr:323
|
||||||
|
RPKI_RTR_TLS_SERVER_ADDR=rpki-rtr:324
|
||||||
|
RPKI_RTR_SSH_SERVER_ADDR=rpki-rtr:22
|
||||||
|
|
||||||
|
# RTR protocol version used as client command second argument (supported: 0,1,2)
|
||||||
|
RPKI_RTR_PROTOCOL_VERSION=2
|
||||||
|
|
||||||
|
# TLS server name used by --server-name in TLS mode
|
||||||
|
# Must match server certificate SAN dNSName.
|
||||||
|
RPKI_RTR_TLS_SERVER_NAME=localhost
|
||||||
|
RPKI_RTR_TLS_CA_CERT_PATH=/app/certs/client-ca.crt
|
||||||
|
RPKI_RTR_TLS_CLIENT_CERT_PATH=/app/certs/client-good.crt
|
||||||
|
RPKI_RTR_TLS_CLIENT_KEY_PATH=/app/certs/client-good.key
|
||||||
|
RPKI_RTR_TLS_CERTS_HOST_DIR=../../certs/tls
|
||||||
|
|
||||||
|
# Shared client logs mount on host.
|
||||||
|
RPKI_RTR_CLIENT_LOG_HOST_DIR=../../logs/client
|
||||||
|
|
||||||
|
# SSH mode examples:
|
||||||
|
# RPKI_RTR_SSH_SERVER_ADDR=10.0.0.12:2222
|
||||||
|
RPKI_RTR_CLIENT_KEYS_VOLUME=../../certs/ssh:/app/certs:ro
|
||||||
|
RPKI_RTR_CLIENT_KEY_PATH=/app/certs/rtr-client.key
|
||||||
|
RPKI_RTR_SSH_SERVER_PUBKEY_PATH=/app/certs/ssh_host_rsa_key.pub
|
||||||
|
RPKI_RTR_SSH_USERNAME=rpki-rtr
|
||||||
|
RPKI_RTR_SSH_PASSWORD=
|
||||||
26
deploy/client/Dockerfile
Normal file
26
deploy/client/Dockerfile
Normal file
@@ -0,0 +1,26 @@
|
|||||||
|
FROM rust:1.89-bookworm AS builder
|
||||||
|
|
||||||
|
WORKDIR /build
|
||||||
|
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends clang libclang-dev pkg-config \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
COPY Cargo.toml Cargo.lock ./
|
||||||
|
COPY src ./src
|
||||||
|
|
||||||
|
RUN cargo build --release --bin rtr_debug_client
|
||||||
|
|
||||||
|
FROM debian:bookworm-slim AS runtime
|
||||||
|
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends ca-certificates tzdata \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
ENV TZ=Asia/Shanghai
|
||||||
|
|
||||||
|
COPY --from=builder /build/target/release/rtr_debug_client /usr/local/bin/rtr_debug_client
|
||||||
|
COPY --chmod=755 deploy/client/entrypoint.sh /usr/local/bin/rtr-debug-client-entrypoint.sh
|
||||||
|
|
||||||
|
ENTRYPOINT ["/usr/local/bin/rtr-debug-client-entrypoint.sh"]
|
||||||
237
deploy/client/build-run.sh
Normal file
237
deploy/client/build-run.sh
Normal file
@@ -0,0 +1,237 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
REPO_ROOT="$(cd -- "${SCRIPT_DIR}/../.." && pwd)"
|
||||||
|
|
||||||
|
MODE="${MODE:-tcp}"
|
||||||
|
ACTION="up"
|
||||||
|
DETACH="${DETACH:-1}"
|
||||||
|
BUILD="${BUILD:-1}"
|
||||||
|
CLIENTS="${CLIENTS:-single}"
|
||||||
|
TARGET_PLATFORM="${TARGET_PLATFORM:-linux/arm64}"
|
||||||
|
CLIENT_IMAGE="${CLIENT_IMAGE:-${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}}"
|
||||||
|
BUILDX_PUSH="${BUILDX_PUSH:-0}"
|
||||||
|
BUILDX_BUILDER="${BUILDX_BUILDER:-}"
|
||||||
|
SKIP_BASE_IMAGE_PULL="${SKIP_BASE_IMAGE_PULL:-0}"
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage:
|
||||||
|
deploy/client/build-run.sh [action] [options]
|
||||||
|
|
||||||
|
Actions:
|
||||||
|
up Build and run debug client (default)
|
||||||
|
build Build debug client image
|
||||||
|
buildx Build debug client image with docker buildx
|
||||||
|
rebuild Rebuild image without cache, then run client
|
||||||
|
down|stop Stop client compose stack
|
||||||
|
restart Stop, then build and run client
|
||||||
|
logs Follow client logs
|
||||||
|
ps Show client compose status
|
||||||
|
|
||||||
|
Options:
|
||||||
|
--mode MODE base|tcp|tls|ssh (default: tcp)
|
||||||
|
--clients MODE single|multi (default: single)
|
||||||
|
--platform VALUE Target platform for buildx (default: linux/arm64)
|
||||||
|
--client-image TAG
|
||||||
|
Client image tag for buildx and compose
|
||||||
|
--push Push buildx image instead of loading it locally
|
||||||
|
--builder NAME buildx builder name
|
||||||
|
--skip-base-pull Do not pull Dockerfile base images before buildx
|
||||||
|
--foreground Run single client in foreground
|
||||||
|
--no-build Start without --build
|
||||||
|
-h|--help Show this help
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
die() {
|
||||||
|
echo "error: $*" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
has_compose() {
|
||||||
|
docker compose version >/dev/null 2>&1
|
||||||
|
}
|
||||||
|
|
||||||
|
has_buildx() {
|
||||||
|
docker buildx version >/dev/null 2>&1
|
||||||
|
}
|
||||||
|
|
||||||
|
pull_base_images() {
|
||||||
|
[[ "${SKIP_BASE_IMAGE_PULL}" == "1" ]] && return
|
||||||
|
|
||||||
|
local platform image
|
||||||
|
local -a platforms images
|
||||||
|
IFS=',' read -r -a platforms <<< "${TARGET_PLATFORM}"
|
||||||
|
images=(rust:1.89-bookworm debian:bookworm-slim)
|
||||||
|
|
||||||
|
for platform in "${platforms[@]}"; do
|
||||||
|
for image in "${images[@]}"; do
|
||||||
|
docker pull --platform "${platform}" "${image}"
|
||||||
|
done
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
compose_files() {
|
||||||
|
if [[ "${CLIENTS}" == "multi" ]]; then
|
||||||
|
[[ "${MODE}" == "tcp" || "${MODE}" == "base" ]] || die "--clients multi is only defined for tcp/base mode"
|
||||||
|
printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.clients.yml"
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
case "${MODE}" in
|
||||||
|
base) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.yml" ;;
|
||||||
|
tcp) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.tcp.yml" ;;
|
||||||
|
tls) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.tls.yml" ;;
|
||||||
|
ssh) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.ssh.yml" ;;
|
||||||
|
*) die "unsupported mode '${MODE}', expected base|tcp|tls|ssh" ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
compose_args() {
|
||||||
|
local -a args files
|
||||||
|
args=()
|
||||||
|
[[ -f "${SCRIPT_DIR}/.env" ]] && args+=(--env-file "${SCRIPT_DIR}/.env")
|
||||||
|
mapfile -t files < <(compose_files)
|
||||||
|
args+=("${files[@]}")
|
||||||
|
printf '%s\n' "${args[@]}"
|
||||||
|
}
|
||||||
|
|
||||||
|
compose_run() {
|
||||||
|
local -a args
|
||||||
|
mapfile -t args < <(compose_args)
|
||||||
|
docker compose "${args[@]}" "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
up_client() {
|
||||||
|
local -a up_args
|
||||||
|
up_args=(up)
|
||||||
|
[[ "${CLIENTS}" == "multi" || "${DETACH}" == "1" ]] && up_args+=(-d)
|
||||||
|
[[ "${BUILD}" == "1" && "${CLIENTS}" != "multi" ]] && up_args+=(--build)
|
||||||
|
compose_run "${up_args[@]}"
|
||||||
|
}
|
||||||
|
|
||||||
|
build_client() {
|
||||||
|
if [[ "${CLIENTS}" == "multi" ]]; then
|
||||||
|
docker build "$@" -f "${SCRIPT_DIR}/Dockerfile" -t "${CLIENT_IMAGE}" "${REPO_ROOT}"
|
||||||
|
else
|
||||||
|
compose_run build "$@"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
buildx_client() {
|
||||||
|
has_buildx || die "docker buildx is not available"
|
||||||
|
if [[ "${TARGET_PLATFORM}" == *,* && "${BUILDX_PUSH}" != "1" ]]; then
|
||||||
|
die "multi-platform buildx output requires --push"
|
||||||
|
fi
|
||||||
|
|
||||||
|
local -a output_args builder_args
|
||||||
|
[[ "${BUILDX_PUSH}" == "1" ]] && output_args=(--push) || output_args=(--load)
|
||||||
|
[[ -n "${BUILDX_BUILDER}" ]] && builder_args=(--builder "${BUILDX_BUILDER}") || builder_args=()
|
||||||
|
|
||||||
|
pull_base_images
|
||||||
|
|
||||||
|
docker buildx build "${builder_args[@]}" \
|
||||||
|
--platform "${TARGET_PLATFORM}" \
|
||||||
|
-f "${SCRIPT_DIR}/Dockerfile" \
|
||||||
|
-t "${CLIENT_IMAGE}" \
|
||||||
|
"${output_args[@]}" \
|
||||||
|
"${REPO_ROOT}"
|
||||||
|
}
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
up|build|buildx|rebuild|down|stop|restart|logs|ps)
|
||||||
|
ACTION="$1"
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--mode)
|
||||||
|
[[ $# -ge 2 ]] || die "--mode requires a value"
|
||||||
|
MODE="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--clients)
|
||||||
|
[[ $# -ge 2 ]] || die "--clients requires a value"
|
||||||
|
CLIENTS="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--platform)
|
||||||
|
[[ $# -ge 2 ]] || die "--platform requires a value"
|
||||||
|
TARGET_PLATFORM="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--client-image)
|
||||||
|
[[ $# -ge 2 ]] || die "--client-image requires a value"
|
||||||
|
CLIENT_IMAGE="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--push)
|
||||||
|
BUILDX_PUSH=1
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--builder)
|
||||||
|
[[ $# -ge 2 ]] || die "--builder requires a value"
|
||||||
|
BUILDX_BUILDER="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--skip-base-pull)
|
||||||
|
SKIP_BASE_IMAGE_PULL=1
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--foreground)
|
||||||
|
DETACH=0
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--no-build)
|
||||||
|
BUILD=0
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
-h|--help)
|
||||||
|
usage
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
die "unknown argument '$1'"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
[[ "${CLIENTS}" == "single" || "${CLIENTS}" == "multi" ]] || die "unsupported clients '${CLIENTS}', expected single|multi"
|
||||||
|
export RPKI_RTR_CLIENT_IMAGE="${CLIENT_IMAGE}"
|
||||||
|
if [[ "${ACTION}" == "buildx" ]]; then
|
||||||
|
has_buildx || die "docker buildx is not available"
|
||||||
|
else
|
||||||
|
has_compose || die "docker compose is not available"
|
||||||
|
fi
|
||||||
|
|
||||||
|
cd "${REPO_ROOT}"
|
||||||
|
|
||||||
|
case "${ACTION}" in
|
||||||
|
up)
|
||||||
|
up_client
|
||||||
|
;;
|
||||||
|
build)
|
||||||
|
build_client
|
||||||
|
;;
|
||||||
|
buildx)
|
||||||
|
buildx_client
|
||||||
|
;;
|
||||||
|
rebuild)
|
||||||
|
build_client --no-cache
|
||||||
|
up_client
|
||||||
|
;;
|
||||||
|
down|stop)
|
||||||
|
compose_run down
|
||||||
|
;;
|
||||||
|
restart)
|
||||||
|
compose_run down
|
||||||
|
up_client
|
||||||
|
;;
|
||||||
|
logs)
|
||||||
|
compose_run logs -f
|
||||||
|
;;
|
||||||
|
ps)
|
||||||
|
compose_run ps
|
||||||
|
;;
|
||||||
|
esac
|
||||||
52
deploy/client/docker-compose.clients.yml
Normal file
52
deploy/client/docker-compose.clients.yml
Normal file
@@ -0,0 +1,52 @@
|
|||||||
|
version: "3.9"
|
||||||
|
|
||||||
|
services:
|
||||||
|
rtr-client-1:
|
||||||
|
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
||||||
|
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
||||||
|
volumes:
|
||||||
|
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
||||||
|
restart: no
|
||||||
|
networks:
|
||||||
|
- rpki_net
|
||||||
|
|
||||||
|
rtr-client-2:
|
||||||
|
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
||||||
|
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
||||||
|
volumes:
|
||||||
|
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
||||||
|
restart: no
|
||||||
|
networks:
|
||||||
|
- rpki_net
|
||||||
|
|
||||||
|
rtr-client-3:
|
||||||
|
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
||||||
|
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
||||||
|
volumes:
|
||||||
|
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
||||||
|
restart: no
|
||||||
|
networks:
|
||||||
|
- rpki_net
|
||||||
|
|
||||||
|
rtr-client-4:
|
||||||
|
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
||||||
|
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
||||||
|
volumes:
|
||||||
|
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
||||||
|
restart: no
|
||||||
|
networks:
|
||||||
|
- rpki_net
|
||||||
|
|
||||||
|
rtr-client-5:
|
||||||
|
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
||||||
|
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
||||||
|
volumes:
|
||||||
|
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
||||||
|
restart: no
|
||||||
|
networks:
|
||||||
|
- rpki_net
|
||||||
|
|
||||||
|
networks:
|
||||||
|
rpki_net:
|
||||||
|
name: rpki_net
|
||||||
|
driver: bridge
|
||||||
36
deploy/client/docker-compose.ssh.password.yml
Normal file
36
deploy/client/docker-compose.ssh.password.yml
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
version: "3.9"
|
||||||
|
|
||||||
|
services:
|
||||||
|
rtr-debug-client:
|
||||||
|
build:
|
||||||
|
context: ../..
|
||||||
|
dockerfile: deploy/client/Dockerfile
|
||||||
|
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
||||||
|
command:
|
||||||
|
[
|
||||||
|
"${RPKI_RTR_SSH_SERVER_ADDR:-rpki-rtr:22}",
|
||||||
|
"${RPKI_RTR_PROTOCOL_VERSION:-2}",
|
||||||
|
"reset",
|
||||||
|
"--ssh",
|
||||||
|
"--ssh-user",
|
||||||
|
"${RPKI_RTR_SSH_USERNAME:-rpki-rtr}",
|
||||||
|
"--ssh-password",
|
||||||
|
"${RPKI_RTR_SSH_PASSWORD}",
|
||||||
|
"--ssh-server-key",
|
||||||
|
"${RPKI_RTR_SSH_SERVER_PUBKEY_PATH:-/app/certs/ssh_host_rsa_key.pub}",
|
||||||
|
"--keep-after-error",
|
||||||
|
"--summary-only"
|
||||||
|
]
|
||||||
|
volumes:
|
||||||
|
- ${RPKI_RTR_CLIENT_KEYS_VOLUME:-../../certs/ssh:/app/certs:ro}
|
||||||
|
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
||||||
|
restart: no
|
||||||
|
stdin_open: true
|
||||||
|
tty: true
|
||||||
|
networks:
|
||||||
|
- rpki_net
|
||||||
|
|
||||||
|
networks:
|
||||||
|
rpki_net:
|
||||||
|
name: rpki_net
|
||||||
|
driver: bridge
|
||||||
36
deploy/client/docker-compose.ssh.yml
Normal file
36
deploy/client/docker-compose.ssh.yml
Normal file
@@ -0,0 +1,36 @@
|
|||||||
|
version: "3.9"
|
||||||
|
|
||||||
|
services:
|
||||||
|
rtr-debug-client:
|
||||||
|
build:
|
||||||
|
context: ../..
|
||||||
|
dockerfile: deploy/client/Dockerfile
|
||||||
|
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
||||||
|
command:
|
||||||
|
[
|
||||||
|
"${RPKI_RTR_SSH_SERVER_ADDR:-rpki-rtr:22}",
|
||||||
|
"${RPKI_RTR_PROTOCOL_VERSION:-2}",
|
||||||
|
"reset",
|
||||||
|
"--ssh",
|
||||||
|
"--ssh-user",
|
||||||
|
"${RPKI_RTR_SSH_USERNAME:-rpki-rtr}",
|
||||||
|
"--ssh-key",
|
||||||
|
"${RPKI_RTR_CLIENT_KEY_PATH:-/app/certs/rtr-client.key}",
|
||||||
|
"--ssh-server-key",
|
||||||
|
"${RPKI_RTR_SSH_SERVER_PUBKEY_PATH:-/app/certs/ssh_host_rsa_key.pub}",
|
||||||
|
"--keep-after-error",
|
||||||
|
"--summary-only"
|
||||||
|
]
|
||||||
|
volumes:
|
||||||
|
- ${RPKI_RTR_CLIENT_KEYS_VOLUME:-../../certs/ssh:/app/certs:ro}
|
||||||
|
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
||||||
|
restart: no
|
||||||
|
stdin_open: true
|
||||||
|
tty: true
|
||||||
|
networks:
|
||||||
|
- rpki_net
|
||||||
|
|
||||||
|
networks:
|
||||||
|
rpki_net:
|
||||||
|
name: rpki_net
|
||||||
|
driver: bridge
|
||||||
19
deploy/client/docker-compose.tcp.yml
Normal file
19
deploy/client/docker-compose.tcp.yml
Normal file
@@ -0,0 +1,19 @@
|
|||||||
|
services:
|
||||||
|
rtr-debug-client:
|
||||||
|
build:
|
||||||
|
context: ../..
|
||||||
|
dockerfile: deploy/client/Dockerfile
|
||||||
|
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
||||||
|
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
||||||
|
volumes:
|
||||||
|
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
||||||
|
restart: no
|
||||||
|
stdin_open: true
|
||||||
|
tty: true
|
||||||
|
networks:
|
||||||
|
- rpki_net
|
||||||
|
|
||||||
|
networks:
|
||||||
|
rpki_net:
|
||||||
|
name: rpki_net
|
||||||
|
driver: bridge
|
||||||
38
deploy/client/docker-compose.tls.yml
Normal file
38
deploy/client/docker-compose.tls.yml
Normal file
@@ -0,0 +1,38 @@
|
|||||||
|
version: "3.9"
|
||||||
|
|
||||||
|
services:
|
||||||
|
rtr-debug-client:
|
||||||
|
build:
|
||||||
|
context: ../..
|
||||||
|
dockerfile: deploy/client/Dockerfile
|
||||||
|
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
||||||
|
command:
|
||||||
|
[
|
||||||
|
"${RPKI_RTR_TLS_SERVER_ADDR:-rpki-rtr:324}",
|
||||||
|
"${RPKI_RTR_PROTOCOL_VERSION:-2}",
|
||||||
|
"reset",
|
||||||
|
"--tls",
|
||||||
|
"--ca-cert",
|
||||||
|
"${RPKI_RTR_TLS_CA_CERT_PATH:-/app/certs/client-ca.crt}",
|
||||||
|
"--server-name",
|
||||||
|
"${RPKI_RTR_TLS_SERVER_NAME:-localhost}",
|
||||||
|
"--client-cert",
|
||||||
|
"${RPKI_RTR_TLS_CLIENT_CERT_PATH:-/app/certs/client-good.crt}",
|
||||||
|
"--client-key",
|
||||||
|
"${RPKI_RTR_TLS_CLIENT_KEY_PATH:-/app/certs/client-good.key}",
|
||||||
|
"--keep-after-error",
|
||||||
|
"--summary-only"
|
||||||
|
]
|
||||||
|
volumes:
|
||||||
|
- ${RPKI_RTR_TLS_CERTS_HOST_DIR:-../../certs/tls}:/app/certs:ro
|
||||||
|
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
||||||
|
restart: no
|
||||||
|
stdin_open: true
|
||||||
|
tty: true
|
||||||
|
networks:
|
||||||
|
- rpki_net
|
||||||
|
|
||||||
|
networks:
|
||||||
|
rpki_net:
|
||||||
|
name: rpki_net
|
||||||
|
driver: bridge
|
||||||
19
deploy/client/docker-compose.yml
Normal file
19
deploy/client/docker-compose.yml
Normal file
@@ -0,0 +1,19 @@
|
|||||||
|
services:
|
||||||
|
rtr-debug-client:
|
||||||
|
build:
|
||||||
|
context: ../..
|
||||||
|
dockerfile: deploy/client/Dockerfile
|
||||||
|
image: ${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client:latest}
|
||||||
|
command: ["${RPKI_RTR_TCP_SERVER_ADDR:-rpki-rtr:323}", "${RPKI_RTR_PROTOCOL_VERSION:-2}", "reset", "--keep-after-error", "--summary-only"]
|
||||||
|
volumes:
|
||||||
|
- ${RPKI_RTR_CLIENT_LOG_HOST_DIR:-../../logs/client}:/app/logs
|
||||||
|
restart: no
|
||||||
|
stdin_open: true
|
||||||
|
tty: true
|
||||||
|
networks:
|
||||||
|
- rpki_net
|
||||||
|
|
||||||
|
networks:
|
||||||
|
rpki_net:
|
||||||
|
name: rpki_net
|
||||||
|
driver: bridge
|
||||||
10
deploy/client/entrypoint.sh
Normal file
10
deploy/client/entrypoint.sh
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
mkdir -p /app/logs
|
||||||
|
|
||||||
|
log_name="${HOSTNAME:-rtr-debug-client}"
|
||||||
|
stdout_log="/app/logs/${log_name}.stdout.log"
|
||||||
|
stderr_log="/app/logs/${log_name}.stderr.log"
|
||||||
|
|
||||||
|
exec /usr/local/bin/rtr_debug_client "$@" >>"$stdout_log" 2>>"$stderr_log"
|
||||||
4
deploy/frr/.env
Normal file
4
deploy/frr/.env
Normal file
@@ -0,0 +1,4 @@
|
|||||||
|
# RTR server endpoint used by deploy/frr/frr.conf.template.
|
||||||
|
RPKI_FRR_RPKI_HOST=127.0.0.1
|
||||||
|
RPKI_FRR_RPKI_PORT=323
|
||||||
|
RPKI_FRR_RPKI_PREFERENCE=1
|
||||||
78
deploy/frr/README.md
Normal file
78
deploy/frr/README.md
Normal file
@@ -0,0 +1,78 @@
|
|||||||
|
# FRR Minimal RTR Client Config
|
||||||
|
|
||||||
|
中文文档: [README.zh.md](./README.zh.md)
|
||||||
|
|
||||||
|
This folder provides a minimal FRR setup for black-box interop testing
|
||||||
|
against this repository's RTR server defaults.
|
||||||
|
|
||||||
|
Server defaults in this repo:
|
||||||
|
- TCP: `0.0.0.0:323`
|
||||||
|
- TLS: `0.0.0.0:324`
|
||||||
|
|
||||||
|
Reference:
|
||||||
|
- `src/main.rs`
|
||||||
|
|
||||||
|
## Files
|
||||||
|
|
||||||
|
- `daemons.example`: sample `/etc/frr/daemons`
|
||||||
|
- `frr.conf.example`: static sample `/etc/frr/frr.conf`
|
||||||
|
- `frr.conf.template`: Docker template used to generate `/etc/frr/frr.conf`
|
||||||
|
- `entrypoint.sh`: renders `frr.conf.template` from environment variables before starting FRR
|
||||||
|
- `.env`: default Docker environment values
|
||||||
|
|
||||||
|
## How to apply on an FRR host
|
||||||
|
|
||||||
|
1. Copy `daemons.example` to `/etc/frr/daemons`.
|
||||||
|
2. Copy `frr.conf.example` to `/etc/frr/frr.conf`.
|
||||||
|
3. Restart FRR:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo systemctl restart frr
|
||||||
|
```
|
||||||
|
|
||||||
|
## Verify
|
||||||
|
|
||||||
|
```bash
|
||||||
|
vtysh -c "show rpki configuration"
|
||||||
|
vtysh -c "show rpki cache-server"
|
||||||
|
vtysh -c "show rpki cache-connection"
|
||||||
|
vtysh -c "show rpki prefix-table"
|
||||||
|
```
|
||||||
|
|
||||||
|
If `show rpki cache-connection` is connected and `show rpki prefix-table`
|
||||||
|
contains VRPs, the RTR client path is working.
|
||||||
|
|
||||||
|
## Docker quick start
|
||||||
|
|
||||||
|
From repository root:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml up -d
|
||||||
|
docker exec -it frr-rpki-client vtysh -c "show rpki cache-connection"
|
||||||
|
docker exec -it frr-rpki-client vtysh -c "show rpki prefix-table"
|
||||||
|
```
|
||||||
|
|
||||||
|
The Docker example defaults to `127.0.0.1:323`. Override the RTR server
|
||||||
|
endpoint with environment variables:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
RPKI_FRR_RPKI_HOST=10.0.0.12 \
|
||||||
|
RPKI_FRR_RPKI_PORT=323 \
|
||||||
|
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
Stop:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml down
|
||||||
|
```
|
||||||
|
|
||||||
|
## Notes
|
||||||
|
|
||||||
|
- This setup targets RTR over TCP (`rpki cache tcp`).
|
||||||
|
- Docker mode renders `frr.conf.template` into `/etc/frr/frr.conf` at startup.
|
||||||
|
- Keep protocol-level conformance checks in Rust tests and
|
||||||
|
`src/bin/rtr_debug_client`.
|
||||||
|
- `network_mode: host` is kept for the minimal FRR container example, but the
|
||||||
|
RTR server address is controlled by `RPKI_FRR_RPKI_HOST` and
|
||||||
|
`RPKI_FRR_RPKI_PORT`.
|
||||||
83
deploy/frr/README.zh.md
Normal file
83
deploy/frr/README.zh.md
Normal file
@@ -0,0 +1,83 @@
|
|||||||
|
# FRR 最小化 RTR 客户端配置
|
||||||
|
|
||||||
|
本目录提供一个 FRR 最小配置，用于和本仓库 RTR Server 做黑盒互通测试。
|
||||||
|
|
||||||
|
本仓库默认 RTR 监听地址：
|
||||||
|
- TCP：`0.0.0.0:323`
|
||||||
|
- TLS：`0.0.0.0:324`
|
||||||
|
|
||||||
|
参考实现：
|
||||||
|
- `src/main.rs`
|
||||||
|
|
||||||
|
## 文件说明
|
||||||
|
|
||||||
|
- `daemons.example`：示例 `/etc/frr/daemons`
|
||||||
|
- `frr.conf.example`：静态示例 `/etc/frr/frr.conf`
|
||||||
|
- `frr.conf.template`：Docker 启动时生成 `/etc/frr/frr.conf` 的模板
|
||||||
|
- `entrypoint.sh`：启动 FRR 前根据环境变量渲染 `frr.conf.template`
|
||||||
|
- `.env`：Docker 示例使用的默认环境变量
|
||||||
|
|
||||||
|
## 在 FRR 主机上应用
|
||||||
|
|
||||||
|
1. 复制 `daemons.example` 到 `/etc/frr/daemons`
|
||||||
|
2. 复制 `frr.conf.example` 到 `/etc/frr/frr.conf`
|
||||||
|
3. 重启 FRR：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
sudo systemctl restart frr
|
||||||
|
```
|
||||||
|
|
||||||
|
## 验证命令
|
||||||
|
|
||||||
|
```bash
|
||||||
|
vtysh -c "show rpki configuration"
|
||||||
|
vtysh -c "show rpki cache-server"
|
||||||
|
vtysh -c "show rpki cache-connection"
|
||||||
|
vtysh -c "show rpki prefix-table"
|
||||||
|
```
|
||||||
|
|
||||||
|
当 `show rpki cache-connection` 显示已连接，且 `show rpki prefix-table` 出现 VRP 时，说明 RTR 客户端链路工作正常。
|
||||||
|
|
||||||
|
## Docker 快速启动
|
||||||
|
|
||||||
|
在仓库根目录执行：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml up -d
|
||||||
|
docker exec -it frr-rpki-client vtysh -c "show rpki cache-connection"
|
||||||
|
docker exec -it frr-rpki-client vtysh -c "show rpki prefix-table"
|
||||||
|
```
|
||||||
|
|
||||||
|
Docker 示例默认连接 `127.0.0.1:323`。如果 RTR Server 在其他地址，可以通过环境变量覆盖：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
RPKI_FRR_RPKI_HOST=10.0.0.12 \
|
||||||
|
RPKI_FRR_RPKI_PORT=323 \
|
||||||
|
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml up -d
|
||||||
|
```
|
||||||
|
|
||||||
|
停止：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose --env-file deploy/frr/.env -f deploy/frr/docker-compose.yml down
|
||||||
|
```
|
||||||
|
|
||||||
|
## 可测试范围
|
||||||
|
|
||||||
|
- FRR 作为 RTR Client 与本仓库 Server 的 TCP 建链能力
|
||||||
|
- FRR 侧基础会话状态可见性（`cache-server` / `cache-connection`）
|
||||||
|
- VRP 下发与导入是否成功（`prefix-table` 是否有条目）
|
||||||
|
- 基于服务端数据变化触发的前缀表更新（可通过替换 `data` 后观察）
|
||||||
|
- 黑盒互通回归：用于确认“路由器客户端视角”功能可用
|
||||||
|
|
||||||
|
## 不覆盖范围
|
||||||
|
|
||||||
|
- 不替代协议级单元测试/集成测试（PDU 细节、异常路径、边界条件）
|
||||||
|
- 不替代 `src/bin/rtr_debug_client` 的逐报文调试能力
|
||||||
|
- 默认示例以 TCP 为主，TLS/mTLS 需按你的证书与 FRR 配置单独扩展
|
||||||
|
|
||||||
|
## 说明
|
||||||
|
|
||||||
|
- 当前示例主要针对 RTR over TCP（`rpki cache tcp`）
|
||||||
|
- Docker 模式会在启动时把 `frr.conf.template` 渲染为 `/etc/frr/frr.conf`
|
||||||
|
- `network_mode: host` 仍用于最小化 FRR 容器示例，但 RTR Server 地址由 `RPKI_FRR_RPKI_HOST` 和 `RPKI_FRR_RPKI_PORT` 控制
|
||||||
7
deploy/frr/daemons.example
Normal file
7
deploy/frr/daemons.example
Normal file
@@ -0,0 +1,7 @@
|
|||||||
|
# Minimal FRR daemons config for RPKI testing
|
||||||
|
|
||||||
|
zebra=yes
|
||||||
|
bgpd=yes
|
||||||
|
|
||||||
|
# Enable bgpd RPKI module
|
||||||
|
bgpd_options=" -A 127.0.0.1 -M rpki"
|
||||||
16
deploy/frr/docker-compose.yml
Normal file
16
deploy/frr/docker-compose.yml
Normal file
@@ -0,0 +1,16 @@
|
|||||||
|
services:
|
||||||
|
frr-rpki-client:
|
||||||
|
image: quay.io/frrouting/frr:10.2.2
|
||||||
|
container_name: frr-rpki-client
|
||||||
|
restart: unless-stopped
|
||||||
|
network_mode: host
|
||||||
|
privileged: true
|
||||||
|
environment:
|
||||||
|
RPKI_FRR_RPKI_HOST: "${RPKI_FRR_RPKI_HOST:-127.0.0.1}"
|
||||||
|
RPKI_FRR_RPKI_PORT: "${RPKI_FRR_RPKI_PORT:-323}"
|
||||||
|
RPKI_FRR_RPKI_PREFERENCE: "${RPKI_FRR_RPKI_PREFERENCE:-1}"
|
||||||
|
entrypoint: ["/bin/sh", "/config/entrypoint.sh"]
|
||||||
|
volumes:
|
||||||
|
- ./daemons.example:/etc/frr/daemons:ro
|
||||||
|
- ./frr.conf.template:/config/frr.conf.template:ro
|
||||||
|
- ./entrypoint.sh:/config/entrypoint.sh:ro
|
||||||
24
deploy/frr/entrypoint.sh
Normal file
24
deploy/frr/entrypoint.sh
Normal file
@@ -0,0 +1,24 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
: "${RPKI_FRR_RPKI_HOST:=127.0.0.1}"
|
||||||
|
: "${RPKI_FRR_RPKI_PORT:=323}"
|
||||||
|
: "${RPKI_FRR_RPKI_PREFERENCE:=1}"
|
||||||
|
: "${RPKI_FRR_CONFIG_TEMPLATE_PATH:=/config/frr.conf.template}"
|
||||||
|
: "${RPKI_FRR_CONFIG_PATH:=/etc/frr/frr.conf}"
|
||||||
|
|
||||||
|
escape_sed() {
|
||||||
|
printf '%s' "$1" | sed 's/[&|\\]/\\&/g'
|
||||||
|
}
|
||||||
|
|
||||||
|
host="$(escape_sed "${RPKI_FRR_RPKI_HOST}")"
|
||||||
|
port="$(escape_sed "${RPKI_FRR_RPKI_PORT}")"
|
||||||
|
preference="$(escape_sed "${RPKI_FRR_RPKI_PREFERENCE}")"
|
||||||
|
|
||||||
|
sed \
|
||||||
|
-e "s|\${RPKI_FRR_RPKI_HOST}|${host}|g" \
|
||||||
|
-e "s|\${RPKI_FRR_RPKI_PORT}|${port}|g" \
|
||||||
|
-e "s|\${RPKI_FRR_RPKI_PREFERENCE}|${preference}|g" \
|
||||||
|
"${RPKI_FRR_CONFIG_TEMPLATE_PATH}" > "${RPKI_FRR_CONFIG_PATH}"
|
||||||
|
|
||||||
|
exec /usr/lib/frr/docker-start
|
||||||
22
deploy/frr/frr.conf.example
Normal file
22
deploy/frr/frr.conf.example
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
frr version 10.2
|
||||||
|
frr defaults traditional
|
||||||
|
hostname rpki-lab
|
||||||
|
service integrated-vtysh-config
|
||||||
|
!
|
||||||
|
debug rpki
|
||||||
|
!
|
||||||
|
rpki
|
||||||
|
rpki polling_period 10
|
||||||
|
rpki timeout 10
|
||||||
|
rpki retry_interval 10
|
||||||
|
rpki expire_interval 7200
|
||||||
|
rpki cache tcp 127.0.0.1 323 preference 1
|
||||||
|
exit
|
||||||
|
!
|
||||||
|
router bgp 65001
|
||||||
|
bgp router-id 192.0.2.1
|
||||||
|
!
|
||||||
|
address-family ipv4 unicast
|
||||||
|
exit-address-family
|
||||||
|
!
|
||||||
|
line vty
|
||||||
22
deploy/frr/frr.conf.template
Normal file
22
deploy/frr/frr.conf.template
Normal file
@@ -0,0 +1,22 @@
|
|||||||
|
frr version 10.2
|
||||||
|
frr defaults traditional
|
||||||
|
hostname rpki-lab
|
||||||
|
service integrated-vtysh-config
|
||||||
|
!
|
||||||
|
debug rpki
|
||||||
|
!
|
||||||
|
rpki
|
||||||
|
rpki polling_period 10
|
||||||
|
rpki timeout 10
|
||||||
|
rpki retry_interval 10
|
||||||
|
rpki expire_interval 7200
|
||||||
|
rpki cache tcp ${RPKI_FRR_RPKI_HOST} ${RPKI_FRR_RPKI_PORT} preference ${RPKI_FRR_RPKI_PREFERENCE}
|
||||||
|
exit
|
||||||
|
!
|
||||||
|
router bgp 65001
|
||||||
|
bgp router-id 192.0.2.1
|
||||||
|
!
|
||||||
|
address-family ipv4 unicast
|
||||||
|
exit-address-family
|
||||||
|
!
|
||||||
|
line vty
|
||||||
49
deploy/server/.env
Normal file
49
deploy/server/.env
Normal file
@@ -0,0 +1,49 @@
|
|||||||
|
# Core runtime knobs.
|
||||||
|
# RPKI_RTR_SERVER_IMAGE=rpki-rtr-arm64:v1.2
|
||||||
|
RPKI_RTR_SERVER_IMAGE=rpki-rtr-amd64:v1.2
|
||||||
|
|
||||||
|
# Data source directories on host.
|
||||||
|
RPKI_RTR_CCR_HOST_DIR=../../data
|
||||||
|
RPKI_RTR_SLURM_HOST_DIR=../../data
|
||||||
|
|
||||||
|
# In-container data source directories.
|
||||||
|
RPKI_RTR_CCR_DIR=/app/data
|
||||||
|
RPKI_RTR_SLURM_DIR=/app/slurm
|
||||||
|
RPKI_RTR_REPORT_HOST_DIR=../../report
|
||||||
|
|
||||||
|
# In-container runtime paths.
|
||||||
|
RPKI_RTR_DB_PATH=/app/rtr-db
|
||||||
|
RPKI_RTR_REPORT_DIR=/app/report
|
||||||
|
|
||||||
|
# Persistent directories on host.
|
||||||
|
RPKI_RTR_DB_HOST_DIR=../../rtr-db
|
||||||
|
RPKI_RTR_LOG_HOST_DIR=../../logs/server
|
||||||
|
RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS=300
|
||||||
|
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS=300
|
||||||
|
RPKI_RTR_REPORT_HISTORY_LIMIT=10
|
||||||
|
RPKI_RTR_TIMEZONE=Asia/Shanghai
|
||||||
|
RPKI_RTR_MAX_DELTA=10
|
||||||
|
RPKI_RTR_MAX_CONNECTIONS=100000
|
||||||
|
RPKI_RTR_MAX_CONCURRENT_HANDSHAKES=128
|
||||||
|
RPKI_RTR_ADMIN_ADDR=0.0.0.0:8323
|
||||||
|
RPKI_RTR_ADMIN_TOKEN=qwert
|
||||||
|
RUST_LOG=info
|
||||||
|
|
||||||
|
# TLS mode knobs.
|
||||||
|
RPKI_RTR_ENFORCE_TLS_CLIENT_SAN_IP_MATCH=false
|
||||||
|
RPKI_RTR_TLS_CERT_PATH=/app/certs/server-dns.crt
|
||||||
|
RPKI_RTR_TLS_KEY_PATH=/app/certs/server-dns.key
|
||||||
|
RPKI_RTR_TLS_CLIENT_CA_PATH=/app/certs/client-ca.crt
|
||||||
|
|
||||||
|
RPKI_RTR_TLS_CERTS_HOST_DIR=../../certs/tls
|
||||||
|
|
||||||
|
# SSH mode knobs.
|
||||||
|
RPKI_RTR_SSH_HOST_PORT=2222
|
||||||
|
RPKI_RTR_SSH_CONTAINER_PORT=22
|
||||||
|
RPKI_RTR_SSH_AUTH_MODE=key
|
||||||
|
RPKI_RTR_SSH_USERNAME=rpki-rtr
|
||||||
|
RPKI_RTR_SSH_SUBSYSTEM_NAME=rpki-rtr
|
||||||
|
RPKI_RTR_SSH_HOST_KEY_PATH=/app/certs/ssh_host_rsa_key
|
||||||
|
RPKI_RTR_SSH_AUTHORIZED_KEYS_PATH=/app/certs/rtr-authorized_keys
|
||||||
|
|
||||||
|
RPKI_RTR_SSH_CERTS_HOST_DIR=../../certs/ssh
|
||||||
99
deploy/server/DEPLOYMENT.md
Normal file
99
deploy/server/DEPLOYMENT.md
Normal file
@@ -0,0 +1,99 @@
|
|||||||
|
# Deployment (Docker Compose)
|
||||||
|
|
||||||
|
This project runs `src/main.rs` as a long-running server that:
|
||||||
|
|
||||||
|
1. loads latest `.ccr` from a configured directory,
|
||||||
|
2. applies optional SLURM filtering,
|
||||||
|
3. starts RTR server.
|
||||||
|
|
||||||
|
The container runs `rpki` directly as PID 1.
|
||||||
|
|
||||||
|
## Files
|
||||||
|
|
||||||
|
- `deploy/server/Dockerfile`
|
||||||
|
- `deploy/server/docker-compose.yml`
|
||||||
|
|
||||||
|
## Runtime Paths in Container
|
||||||
|
|
||||||
|
- CCR directory: `/app/data`
|
||||||
|
- RocksDB directory: `/app/rtr-db`
|
||||||
|
- SLURM directory: `/app/slurm`
|
||||||
|
- Report directory: `/app/report`
|
||||||
|
- TLS cert directory (optional): `/app/certs`
|
||||||
|
|
||||||
|
## Path Configuration via `.env`
|
||||||
|
|
||||||
|
- `RPKI_RTR_CCR_HOST_DIR`: host CCR directory mounted into container
|
||||||
|
- `RPKI_RTR_SLURM_HOST_DIR`: host SLURM directory mounted into container
|
||||||
|
- `RPKI_RTR_CCR_DIR`: in-container CCR directory path
|
||||||
|
- `RPKI_RTR_SLURM_DIR`: in-container SLURM directory path
|
||||||
|
- `RPKI_RTR_DB_HOST_DIR`: host RocksDB directory
|
||||||
|
- `RPKI_RTR_LOG_HOST_DIR`: host log directory
|
||||||
|
- `RPKI_RTR_REPORT_HOST_DIR`: host directory receiving split RTR JSON reports
|
||||||
|
- `RPKI_RTR_DB_PATH`: in-container RocksDB directory
|
||||||
|
- `RPKI_RTR_REPORT_DIR`: in-container report directory
|
||||||
|
|
||||||
|
## Runtime Configuration via `.env`
|
||||||
|
|
||||||
|
- Core: `RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS`, `RPKI_RTR_MAX_DELTA`, `RPKI_RTR_MAX_CONCURRENT_HANDSHAKES`, `RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS`, `RPKI_RTR_REPORT_HISTORY_LIMIT`, `RPKI_RTR_TIMEZONE`, `RPKI_RTR_ADMIN_ADDR`, `RPKI_RTR_ADMIN_TOKEN`, `RUST_LOG`
|
||||||
|
- TCP mode: `RPKI_RTR_MAX_CONNECTIONS`
|
||||||
|
- TLS mode: `RPKI_RTR_ENFORCE_TLS_CLIENT_SAN_IP_MATCH`, `RPKI_RTR_TLS_CERT_PATH`, `RPKI_RTR_TLS_KEY_PATH`, `RPKI_RTR_TLS_CLIENT_CA_PATH`, `RPKI_RTR_TLS_CERTS_HOST_DIR`
|
||||||
|
- SSH mode: `RPKI_RTR_SSH_HOST_PORT`, `RPKI_RTR_SSH_CONTAINER_PORT`, `RPKI_RTR_SSH_AUTH_MODE`, `RPKI_RTR_SSH_USERNAME`, `RPKI_RTR_SSH_SUBSYSTEM_NAME`, `RPKI_RTR_SSH_HOST_KEY_PATH`, `RPKI_RTR_SSH_AUTHORIZED_KEYS_PATH`, `RPKI_RTR_SSH_CERTS_HOST_DIR`
|
||||||
|
|
||||||
|
## Start
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f deploy/server/docker-compose.yml up -d --build
|
||||||
|
```
|
||||||
|
|
||||||
|
## Stop
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f deploy/server/docker-compose.yml down
|
||||||
|
```
|
||||||
|
|
||||||
|
## Logs
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose -f deploy/server/docker-compose.yml logs -f rpki-rtr
|
||||||
|
```
|
||||||
|
|
||||||
|
The admin API can also stream the redirected log file:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -N "http://127.0.0.1:8323/admin/rtr/logs/tail?stream=stdout&lines=200" \
|
||||||
|
-H "Authorization: Bearer $RPKI_RTR_ADMIN_TOKEN"
|
||||||
|
```
|
||||||
|
|
||||||
|
It reads `/app/logs/${HOSTNAME}.stdout.log` or `.stderr.log` by default. Set
|
||||||
|
`RPKI_RTR_LOG_DIR` and `RPKI_RTR_LOG_NAME` to override that lookup.
|
||||||
|
|
||||||
|
## Runtime Report
|
||||||
|
|
||||||
|
The server writes split JSON reports. Each report file uses a local-time
|
||||||
|
timestamp suffix and each category keeps `RPKI_RTR_REPORT_HISTORY_LIMIT` files,
|
||||||
|
defaulting to 10.
|
||||||
|
|
||||||
|
- `rtr-source-*.json`: CCR and SLURM source metadata, latest refresh status,
|
||||||
|
data quality counts, cache snapshot counts, and delta counts. Written on
|
||||||
|
startup and source refresh events.
|
||||||
|
- `rtr-clients-*.json`: active client connection counts and counts by transport
|
||||||
|
(`tcp`, `tls`, `ssh`). Written on startup, whenever the active connection
|
||||||
|
count changes.
|
||||||
|
- `rtr-runtime-*.json`: service start time, uptime, process RSS, and
|
||||||
|
non-sensitive runtime configuration. Written on startup and every
|
||||||
|
`RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS`, defaulting to 300 seconds.
|
||||||
|
|
||||||
|
Timestamps in logs and report JSON files use `RPKI_RTR_TIMEZONE`, which
|
||||||
|
defaults to `Asia/Shanghai`. Use IANA timezone names such as `Asia/Shanghai`,
|
||||||
|
`Europe/London`, `America/New_York`, or `UTC`; `Shanghai` is accepted as a
|
||||||
|
convenience alias for `Asia/Shanghai`.
|
||||||
|
|
||||||
|
## Runtime Admin Config
|
||||||
|
|
||||||
|
The admin endpoint is disabled by default. Set `RPKI_RTR_ADMIN_ADDR` to enable
|
||||||
|
`POST /admin/rtr/config`. If the address is not loopback, `RPKI_RTR_ADMIN_TOKEN`
|
||||||
|
must also be set and requests must include `Authorization: Bearer <token>`.
|
||||||
|
|
||||||
|
The endpoint accepts partial JSON updates. See `docs/rtr-admin-api.md` for the
|
||||||
|
complete request/response schema, examples, and runtime apply semantics.
|
||||||
85
deploy/server/Dockerfile
Normal file
85
deploy/server/Dockerfile
Normal file
@@ -0,0 +1,85 @@
|
|||||||
|
FROM rust:1.89-bookworm AS builder
|
||||||
|
|
||||||
|
WORKDIR /build
|
||||||
|
|
||||||
|
ENV CARGO_BUILD_JOBS=2
|
||||||
|
|
||||||
|
RUN set -eux; \
|
||||||
|
cat > /etc/apt/sources.list.d/debian.sources <<'EOF'
|
||||||
|
Types: deb
|
||||||
|
URIs: http://mirrors.tuna.tsinghua.edu.cn/debian
|
||||||
|
Suites: bookworm bookworm-updates
|
||||||
|
Components: main
|
||||||
|
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
|
||||||
|
|
||||||
|
Types: deb
|
||||||
|
URIs: http://mirrors.tuna.tsinghua.edu.cn/debian-security
|
||||||
|
Suites: bookworm-security
|
||||||
|
Components: main
|
||||||
|
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
|
||||||
|
EOF
|
||||||
|
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --fix-missing --no-install-recommends \
|
||||||
|
-o Acquire::Retries=10 \
|
||||||
|
-o Acquire::http::Timeout=60 \
|
||||||
|
build-essential \
|
||||||
|
cmake \
|
||||||
|
pkg-config \
|
||||||
|
clang \
|
||||||
|
libclang-dev \
|
||||||
|
libssl-dev \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
COPY Cargo.toml Cargo.lock ./
|
||||||
|
COPY src ./src
|
||||||
|
|
||||||
|
RUN cargo build --release --bin rpki_rtr
|
||||||
|
|
||||||
|
FROM debian:bookworm-slim AS runtime
|
||||||
|
|
||||||
|
RUN set -eux; \
|
||||||
|
cat > /etc/apt/sources.list.d/debian.sources <<'EOF'
|
||||||
|
Types: deb
|
||||||
|
URIs: http://mirrors.tuna.tsinghua.edu.cn/debian
|
||||||
|
Suites: bookworm bookworm-updates
|
||||||
|
Components: main
|
||||||
|
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
|
||||||
|
|
||||||
|
Types: deb
|
||||||
|
URIs: http://mirrors.tuna.tsinghua.edu.cn/debian-security
|
||||||
|
Suites: bookworm-security
|
||||||
|
Components: main
|
||||||
|
Signed-By: /usr/share/keyrings/debian-archive-keyring.gpg
|
||||||
|
EOF
|
||||||
|
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --fix-missing --no-install-recommends \
|
||||||
|
-o Acquire::Retries=10 \
|
||||||
|
-o Acquire::http::Timeout=60 \
|
||||||
|
ca-certificates \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
COPY --from=builder /build/target/release/rpki_rtr /usr/local/bin/rpki_rtr
|
||||||
|
COPY --chmod=755 deploy/server/entrypoint.sh /usr/local/bin/rpki-rtr-entrypoint.sh
|
||||||
|
|
||||||
|
RUN mkdir -p /app/data /app/rtr-db /app/certs /app/slurm /app/logs /app/report
|
||||||
|
|
||||||
|
ENV RPKI_RTR_ENABLE_TLS=false \
|
||||||
|
RPKI_RTR_TCP_ADDR=0.0.0.0:323 \
|
||||||
|
RPKI_RTR_TLS_ADDR=0.0.0.0:324 \
|
||||||
|
RPKI_RTR_DB_PATH=/app/rtr-db \
|
||||||
|
RPKI_RTR_CCR_DIR=/app/data \
|
||||||
|
RPKI_RTR_SLURM_DIR=/app/slurm \
|
||||||
|
RPKI_RTR_REPORT_DIR=/app/report \
|
||||||
|
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS=300 \
|
||||||
|
RPKI_RTR_REPORT_HISTORY_LIMIT=10 \
|
||||||
|
RPKI_RTR_REFRESH_INTERVAL_SECS=300 \
|
||||||
|
RPKI_RTR_ADMIN_ADDR="" \
|
||||||
|
RPKI_RTR_ADMIN_TOKEN=""
|
||||||
|
|
||||||
|
EXPOSE 323 324
|
||||||
|
|
||||||
|
ENTRYPOINT ["/usr/local/bin/rpki-rtr-entrypoint.sh"]
|
||||||
215
deploy/server/build-run.sh
Normal file
215
deploy/server/build-run.sh
Normal file
@@ -0,0 +1,215 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
REPO_ROOT="$(cd -- "${SCRIPT_DIR}/../.." && pwd)"
|
||||||
|
|
||||||
|
MODE="${MODE:-tcp}"
|
||||||
|
ACTION="up"
|
||||||
|
DETACH="${DETACH:-1}"
|
||||||
|
BUILD="${BUILD:-1}"
|
||||||
|
TARGET_PLATFORM="${TARGET_PLATFORM:-linux/arm64}"
|
||||||
|
SERVER_IMAGE="${SERVER_IMAGE:-${RPKI_RTR_SERVER_IMAGE:-rpki-rtr:latest}}"
|
||||||
|
BUILDX_PUSH="${BUILDX_PUSH:-0}"
|
||||||
|
BUILDX_BUILDER="${BUILDX_BUILDER:-}"
|
||||||
|
SKIP_BASE_IMAGE_PULL="${SKIP_BASE_IMAGE_PULL:-0}"
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage:
|
||||||
|
deploy/server/build-run.sh [action] [options]
|
||||||
|
|
||||||
|
Actions:
|
||||||
|
up Build and start server (default)
|
||||||
|
build Build server image with docker compose
|
||||||
|
buildx Build server image with docker buildx
|
||||||
|
rebuild Rebuild image without cache, then start server
|
||||||
|
down|stop Stop server compose stack
|
||||||
|
restart Stop, then build and start server
|
||||||
|
logs Follow server logs
|
||||||
|
ps Show server compose status
|
||||||
|
|
||||||
|
Options:
|
||||||
|
--mode MODE base|tcp|tls|ssh (default: tcp)
|
||||||
|
--platform VALUE Target platform for buildx (default: linux/arm64)
|
||||||
|
--server-image TAG
|
||||||
|
Server image tag for buildx and compose
|
||||||
|
--push Push buildx image instead of loading it locally
|
||||||
|
--builder NAME buildx builder name
|
||||||
|
--skip-base-pull Do not pull Dockerfile base images before buildx
|
||||||
|
--foreground Run server in foreground
|
||||||
|
--no-build Start without --build
|
||||||
|
-h|--help Show this help
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
die() {
|
||||||
|
echo "error: $*" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
has_compose() {
|
||||||
|
docker compose version >/dev/null 2>&1
|
||||||
|
}
|
||||||
|
|
||||||
|
has_buildx() {
|
||||||
|
docker buildx version >/dev/null 2>&1
|
||||||
|
}
|
||||||
|
|
||||||
|
pull_base_images() {
|
||||||
|
[[ "${SKIP_BASE_IMAGE_PULL}" == "1" ]] && return
|
||||||
|
|
||||||
|
local platform image
|
||||||
|
local -a platforms images
|
||||||
|
IFS=',' read -r -a platforms <<< "${TARGET_PLATFORM}"
|
||||||
|
images=(rust:1.89-bookworm debian:bookworm-slim)
|
||||||
|
|
||||||
|
for platform in "${platforms[@]}"; do
|
||||||
|
for image in "${images[@]}"; do
|
||||||
|
docker pull --platform "${platform}" "${image}"
|
||||||
|
done
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
compose_files() {
|
||||||
|
case "${MODE}" in
|
||||||
|
base) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.yml" ;;
|
||||||
|
tcp) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.tcp.yml" ;;
|
||||||
|
tls) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.tls.yml" ;;
|
||||||
|
ssh) printf '%s\n' -f "${SCRIPT_DIR}/docker-compose.ssh.yml" ;;
|
||||||
|
*) die "unsupported mode '${MODE}', expected base|tcp|tls|ssh" ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
compose_args() {
|
||||||
|
local -a args files
|
||||||
|
args=()
|
||||||
|
[[ -f "${SCRIPT_DIR}/.env" ]] && args+=(--env-file "${SCRIPT_DIR}/.env")
|
||||||
|
mapfile -t files < <(compose_files)
|
||||||
|
args+=("${files[@]}")
|
||||||
|
printf '%s\n' "${args[@]}"
|
||||||
|
}
|
||||||
|
|
||||||
|
compose_run() {
|
||||||
|
local -a args
|
||||||
|
mapfile -t args < <(compose_args)
|
||||||
|
docker compose "${args[@]}" "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
up_server() {
|
||||||
|
local -a up_args
|
||||||
|
up_args=(up)
|
||||||
|
[[ "${DETACH}" == "1" ]] && up_args+=(-d)
|
||||||
|
[[ "${BUILD}" == "1" ]] && up_args+=(--build)
|
||||||
|
compose_run "${up_args[@]}"
|
||||||
|
}
|
||||||
|
|
||||||
|
buildx_server() {
|
||||||
|
has_buildx || die "docker buildx is not available"
|
||||||
|
if [[ "${TARGET_PLATFORM}" == *,* && "${BUILDX_PUSH}" != "1" ]]; then
|
||||||
|
die "multi-platform buildx output requires --push"
|
||||||
|
fi
|
||||||
|
|
||||||
|
local -a output_args builder_args
|
||||||
|
[[ "${BUILDX_PUSH}" == "1" ]] && output_args=(--push) || output_args=(--load)
|
||||||
|
[[ -n "${BUILDX_BUILDER}" ]] && builder_args=(--builder "${BUILDX_BUILDER}") || builder_args=()
|
||||||
|
|
||||||
|
pull_base_images
|
||||||
|
|
||||||
|
docker buildx build "${builder_args[@]}" \
|
||||||
|
--platform "${TARGET_PLATFORM}" \
|
||||||
|
-f "${SCRIPT_DIR}/Dockerfile" \
|
||||||
|
-t "${SERVER_IMAGE}" \
|
||||||
|
"${output_args[@]}" \
|
||||||
|
"${REPO_ROOT}"
|
||||||
|
}
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
up|build|buildx|rebuild|down|stop|restart|logs|ps)
|
||||||
|
ACTION="$1"
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--mode)
|
||||||
|
[[ $# -ge 2 ]] || die "--mode requires a value"
|
||||||
|
MODE="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--platform)
|
||||||
|
[[ $# -ge 2 ]] || die "--platform requires a value"
|
||||||
|
TARGET_PLATFORM="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--server-image)
|
||||||
|
[[ $# -ge 2 ]] || die "--server-image requires a value"
|
||||||
|
SERVER_IMAGE="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--push)
|
||||||
|
BUILDX_PUSH=1
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--builder)
|
||||||
|
[[ $# -ge 2 ]] || die "--builder requires a value"
|
||||||
|
BUILDX_BUILDER="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--skip-base-pull)
|
||||||
|
SKIP_BASE_IMAGE_PULL=1
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--foreground)
|
||||||
|
DETACH=0
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--no-build)
|
||||||
|
BUILD=0
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
-h|--help)
|
||||||
|
usage
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
die "unknown argument '$1'"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
export RPKI_RTR_SERVER_IMAGE="${SERVER_IMAGE}"
|
||||||
|
if [[ "${ACTION}" == "buildx" ]]; then
|
||||||
|
has_buildx || die "docker buildx is not available"
|
||||||
|
else
|
||||||
|
has_compose || die "docker compose is not available"
|
||||||
|
fi
|
||||||
|
|
||||||
|
cd "${REPO_ROOT}"
|
||||||
|
|
||||||
|
case "${ACTION}" in
|
||||||
|
up)
|
||||||
|
up_server
|
||||||
|
;;
|
||||||
|
build)
|
||||||
|
compose_run build
|
||||||
|
;;
|
||||||
|
buildx)
|
||||||
|
buildx_server
|
||||||
|
;;
|
||||||
|
rebuild)
|
||||||
|
compose_run build --no-cache
|
||||||
|
up_server
|
||||||
|
;;
|
||||||
|
down|stop)
|
||||||
|
compose_run down
|
||||||
|
;;
|
||||||
|
restart)
|
||||||
|
compose_run down
|
||||||
|
up_server
|
||||||
|
;;
|
||||||
|
logs)
|
||||||
|
compose_run logs -f
|
||||||
|
;;
|
||||||
|
ps)
|
||||||
|
compose_run ps
|
||||||
|
;;
|
||||||
|
esac
|
||||||
54
deploy/server/docker-compose.ssh.yml
Normal file
54
deploy/server/docker-compose.ssh.yml
Normal file
@@ -0,0 +1,54 @@
|
|||||||
|
version: "3.9"
|
||||||
|
|
||||||
|
services:
|
||||||
|
rpki-rtr:
|
||||||
|
build:
|
||||||
|
context: ../..
|
||||||
|
dockerfile: deploy/server/Dockerfile
|
||||||
|
image: ${RPKI_RTR_SERVER_IMAGE:-rpki-rtr:latest}
|
||||||
|
container_name: rpki-rtr-ssh
|
||||||
|
restart: no
|
||||||
|
ports:
|
||||||
|
- "323:323"
|
||||||
|
- "${RPKI_RTR_SSH_HOST_PORT:-2222}:${RPKI_RTR_SSH_CONTAINER_PORT:-22}"
|
||||||
|
- "8323:8323"
|
||||||
|
environment:
|
||||||
|
RPKI_RTR_ENABLE_TLS: "false"
|
||||||
|
RPKI_RTR_ENABLE_SSH: "true"
|
||||||
|
RPKI_RTR_TCP_ADDR: "0.0.0.0:323"
|
||||||
|
RPKI_RTR_SSH_ADDR: "0.0.0.0:${RPKI_RTR_SSH_CONTAINER_PORT:-22}"
|
||||||
|
RPKI_RTR_SSH_HOST_KEY_PATH: "${RPKI_RTR_SSH_HOST_KEY_PATH:-/app/certs/ssh_host_rsa_key}"
|
||||||
|
RPKI_RTR_SSH_AUTHORIZED_KEYS_PATH: "${RPKI_RTR_SSH_AUTHORIZED_KEYS_PATH:-/app/certs/rtr-authorized_keys}"
|
||||||
|
RPKI_RTR_SSH_USERNAME: "${RPKI_RTR_SSH_USERNAME:-rpki-rtr}"
|
||||||
|
RPKI_RTR_SSH_SUBSYSTEM_NAME: "${RPKI_RTR_SSH_SUBSYSTEM_NAME:-rpki-rtr}"
|
||||||
|
# SSH auth mode: key | password | both
|
||||||
|
RPKI_RTR_SSH_AUTH_MODE: "${RPKI_RTR_SSH_AUTH_MODE:-key}"
|
||||||
|
# Optional: enable password authentication in addition to publickey
|
||||||
|
# RPKI_RTR_SSH_PASSWORD: "test-password"
|
||||||
|
RPKI_RTR_DB_PATH: "${RPKI_RTR_DB_PATH:-/app/rtr-db}"
|
||||||
|
RPKI_RTR_CCR_DIR: "${RPKI_RTR_CCR_DIR:-/app/data}"
|
||||||
|
RPKI_RTR_SLURM_DIR: "${RPKI_RTR_SLURM_DIR:-/app/slurm}"
|
||||||
|
RPKI_RTR_REPORT_DIR: "${RPKI_RTR_REPORT_DIR:-/app/report}"
|
||||||
|
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS: "${RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS:-300}"
|
||||||
|
RPKI_RTR_REPORT_HISTORY_LIMIT: "${RPKI_RTR_REPORT_HISTORY_LIMIT:-10}"
|
||||||
|
RPKI_RTR_TIMEZONE: "${RPKI_RTR_TIMEZONE:-Asia/Shanghai}"
|
||||||
|
RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS: "${RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS:-300}"
|
||||||
|
RPKI_RTR_MAX_DELTA: "${RPKI_RTR_MAX_DELTA:-10}"
|
||||||
|
RPKI_RTR_MAX_CONCURRENT_HANDSHAKES: "${RPKI_RTR_MAX_CONCURRENT_HANDSHAKES:-128}"
|
||||||
|
RPKI_RTR_ADMIN_ADDR: "${RPKI_RTR_ADMIN_ADDR:-}"
|
||||||
|
RPKI_RTR_ADMIN_TOKEN: "${RPKI_RTR_ADMIN_TOKEN:-}"
|
||||||
|
RUST_LOG: "${RUST_LOG:-info}"
|
||||||
|
volumes:
|
||||||
|
- ${RPKI_RTR_CCR_HOST_DIR:-../../data}:${RPKI_RTR_CCR_DIR:-/app/data}:ro
|
||||||
|
- ${RPKI_RTR_DB_HOST_DIR:-../../rtr-db}:${RPKI_RTR_DB_PATH:-/app/rtr-db}
|
||||||
|
- ${RPKI_RTR_SLURM_HOST_DIR:-../../data}:${RPKI_RTR_SLURM_DIR:-/app/slurm}
|
||||||
|
- ${RPKI_RTR_SSH_CERTS_HOST_DIR:-../../certs/ssh}:/app/certs:ro
|
||||||
|
- ${RPKI_RTR_LOG_HOST_DIR:-../../logs/server}:/app/logs
|
||||||
|
- ${RPKI_RTR_REPORT_HOST_DIR:-../../report}:${RPKI_RTR_REPORT_DIR:-/app/report}
|
||||||
|
networks:
|
||||||
|
- rpki_net
|
||||||
|
|
||||||
|
networks:
|
||||||
|
rpki_net:
|
||||||
|
name: rpki_net
|
||||||
|
driver: bridge
|
||||||
44
deploy/server/docker-compose.tcp.yml
Normal file
44
deploy/server/docker-compose.tcp.yml
Normal file
@@ -0,0 +1,44 @@
|
|||||||
|
version: "3.9"
|
||||||
|
|
||||||
|
services:
|
||||||
|
rpki-rtr:
|
||||||
|
build:
|
||||||
|
context: ../..
|
||||||
|
dockerfile: deploy/server/Dockerfile
|
||||||
|
image: ${RPKI_RTR_SERVER_IMAGE:-rpki-rtr:latest}
|
||||||
|
container_name: rpki-rtr-tcp
|
||||||
|
restart: no
|
||||||
|
ports:
|
||||||
|
- "323:323"
|
||||||
|
- "8323:8323"
|
||||||
|
environment:
|
||||||
|
RPKI_RTR_ENABLE_TLS: "false"
|
||||||
|
RPKI_RTR_ENABLE_SSH: "false"
|
||||||
|
RPKI_RTR_TCP_ADDR: "0.0.0.0:323"
|
||||||
|
RPKI_RTR_DB_PATH: "${RPKI_RTR_DB_PATH:-/app/rtr-db}"
|
||||||
|
RPKI_RTR_CCR_DIR: "${RPKI_RTR_CCR_DIR:-/app/data}"
|
||||||
|
RPKI_RTR_SLURM_DIR: "${RPKI_RTR_SLURM_DIR:-/app/slurm}"
|
||||||
|
RPKI_RTR_REPORT_DIR: "${RPKI_RTR_REPORT_DIR:-/app/report}"
|
||||||
|
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS: "${RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS:-300}"
|
||||||
|
RPKI_RTR_REPORT_HISTORY_LIMIT: "${RPKI_RTR_REPORT_HISTORY_LIMIT:-10}"
|
||||||
|
RPKI_RTR_TIMEZONE: "${RPKI_RTR_TIMEZONE:-Asia/Shanghai}"
|
||||||
|
RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS: "${RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS:-60}"
|
||||||
|
RPKI_RTR_MAX_DELTA: "${RPKI_RTR_MAX_DELTA:-10}"
|
||||||
|
RPKI_RTR_MAX_CONNECTIONS: "${RPKI_RTR_MAX_CONNECTIONS:-100000}"
|
||||||
|
RPKI_RTR_MAX_CONCURRENT_HANDSHAKES: "${RPKI_RTR_MAX_CONCURRENT_HANDSHAKES:-128}"
|
||||||
|
RPKI_RTR_ADMIN_ADDR: "${RPKI_RTR_ADMIN_ADDR:-}"
|
||||||
|
RPKI_RTR_ADMIN_TOKEN: "${RPKI_RTR_ADMIN_TOKEN:-}"
|
||||||
|
RUST_LOG: "${RUST_LOG:-info}"
|
||||||
|
volumes:
|
||||||
|
- ${RPKI_RTR_CCR_HOST_DIR:-../../data}:${RPKI_RTR_CCR_DIR:-/app/data}:ro
|
||||||
|
- ${RPKI_RTR_DB_HOST_DIR:-../../rtr-db}:${RPKI_RTR_DB_PATH:-/app/rtr-db}
|
||||||
|
- ${RPKI_RTR_SLURM_HOST_DIR:-../../data}:${RPKI_RTR_SLURM_DIR:-/app/slurm}
|
||||||
|
- ${RPKI_RTR_LOG_HOST_DIR:-../../logs/server}:/app/logs
|
||||||
|
- ${RPKI_RTR_REPORT_HOST_DIR:-../../report}:${RPKI_RTR_REPORT_DIR:-/app/report}
|
||||||
|
networks:
|
||||||
|
- rpki_net
|
||||||
|
|
||||||
|
networks:
|
||||||
|
rpki_net:
|
||||||
|
name: rpki_net
|
||||||
|
driver: bridge
|
||||||
48
deploy/server/docker-compose.tls.yml
Normal file
48
deploy/server/docker-compose.tls.yml
Normal file
@@ -0,0 +1,48 @@
|
|||||||
|
services:
|
||||||
|
rpki-rtr:
|
||||||
|
build:
|
||||||
|
context: ../..
|
||||||
|
dockerfile: deploy/server/Dockerfile
|
||||||
|
image: ${RPKI_RTR_SERVER_IMAGE:-rpki-rtr:latest}
|
||||||
|
container_name: rpki-rtr-tls
|
||||||
|
restart: no
|
||||||
|
ports:
|
||||||
|
# - "323:323"
|
||||||
|
- "324:324"
|
||||||
|
- "8323:8323"
|
||||||
|
environment:
|
||||||
|
RPKI_RTR_ENABLE_TLS: "true"
|
||||||
|
RPKI_RTR_ENABLE_SSH: "false"
|
||||||
|
# RPKI_RTR_TCP_ADDR: "0.0.0.0:323"
|
||||||
|
RPKI_RTR_TLS_ADDR: "0.0.0.0:324"
|
||||||
|
RPKI_RTR_TLS_CERT_PATH: "${RPKI_RTR_TLS_CERT_PATH:-/app/certs/server-dns.crt}"
|
||||||
|
RPKI_RTR_TLS_KEY_PATH: "${RPKI_RTR_TLS_KEY_PATH:-/app/certs/server-dns.key}"
|
||||||
|
RPKI_RTR_TLS_CLIENT_CA_PATH: "${RPKI_RTR_TLS_CLIENT_CA_PATH:-/app/certs/client-ca.crt}"
|
||||||
|
RPKI_RTR_ENFORCE_TLS_CLIENT_SAN_IP_MATCH: "${RPKI_RTR_ENFORCE_TLS_CLIENT_SAN_IP_MATCH:-false}"
|
||||||
|
RPKI_RTR_DB_PATH: "${RPKI_RTR_DB_PATH:-/app/rtr-db}"
|
||||||
|
RPKI_RTR_CCR_DIR: "${RPKI_RTR_CCR_DIR:-/app/data}"
|
||||||
|
RPKI_RTR_SLURM_DIR: "${RPKI_RTR_SLURM_DIR:-/app/slurm}"
|
||||||
|
RPKI_RTR_REPORT_DIR: "${RPKI_RTR_REPORT_DIR:-/app/report}"
|
||||||
|
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS: "${RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS:-300}"
|
||||||
|
RPKI_RTR_REPORT_HISTORY_LIMIT: "${RPKI_RTR_REPORT_HISTORY_LIMIT:-10}"
|
||||||
|
RPKI_RTR_TIMEZONE: "${RPKI_RTR_TIMEZONE:-Asia/Shanghai}"
|
||||||
|
RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS: "${RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS:-300}"
|
||||||
|
RPKI_RTR_MAX_DELTA: "${RPKI_RTR_MAX_DELTA:-10}"
|
||||||
|
RPKI_RTR_MAX_CONCURRENT_HANDSHAKES: "${RPKI_RTR_MAX_CONCURRENT_HANDSHAKES:-128}"
|
||||||
|
RPKI_RTR_ADMIN_ADDR: "${RPKI_RTR_ADMIN_ADDR:-}"
|
||||||
|
RPKI_RTR_ADMIN_TOKEN: "${RPKI_RTR_ADMIN_TOKEN:-}"
|
||||||
|
RUST_LOG: "${RUST_LOG:-info}"
|
||||||
|
volumes:
|
||||||
|
- ${RPKI_RTR_CCR_HOST_DIR:-../../data}:${RPKI_RTR_CCR_DIR:-/app/data}:ro
|
||||||
|
- ${RPKI_RTR_DB_HOST_DIR:-../../rtr-db}:${RPKI_RTR_DB_PATH:-/app/rtr-db}
|
||||||
|
- ${RPKI_RTR_SLURM_HOST_DIR:-../../data}:${RPKI_RTR_SLURM_DIR:-/app/slurm}
|
||||||
|
- ${RPKI_RTR_TLS_CERTS_HOST_DIR:-../../certs/tls}:/app/certs:ro
|
||||||
|
- ${RPKI_RTR_LOG_HOST_DIR:-../../logs/server}:/app/logs
|
||||||
|
- ${RPKI_RTR_REPORT_HOST_DIR:-../../report}:${RPKI_RTR_REPORT_DIR:-/app/report}
|
||||||
|
networks:
|
||||||
|
- rpki_net
|
||||||
|
|
||||||
|
networks:
|
||||||
|
rpki_net:
|
||||||
|
name: rpki_net
|
||||||
|
driver: bridge
|
||||||
58
deploy/server/docker-compose.yml
Normal file
58
deploy/server/docker-compose.yml
Normal file
@@ -0,0 +1,58 @@
|
|||||||
|
version: "3.9"
|
||||||
|
|
||||||
|
services:
|
||||||
|
rpki-rtr:
|
||||||
|
build:
|
||||||
|
context: ../..
|
||||||
|
dockerfile: deploy/server/Dockerfile
|
||||||
|
image: ${RPKI_RTR_SERVER_IMAGE:-rpki-rtr:latest}
|
||||||
|
container_name: rpki-rtr
|
||||||
|
restart: no
|
||||||
|
ports:
|
||||||
|
- "323:323"
|
||||||
|
- "324:324"
|
||||||
|
- "8323:8323"
|
||||||
|
# SSH mode example:
|
||||||
|
# - "22:22"
|
||||||
|
environment:
|
||||||
|
RPKI_RTR_ENABLE_TLS: "false"
|
||||||
|
RPKI_RTR_TCP_ADDR: "0.0.0.0:323"
|
||||||
|
RPKI_RTR_TLS_ADDR: "0.0.0.0:324"
|
||||||
|
RPKI_RTR_DB_PATH: "${RPKI_RTR_DB_PATH:-/app/rtr-db}"
|
||||||
|
RPKI_RTR_CCR_DIR: "${RPKI_RTR_CCR_DIR:-/app/data}"
|
||||||
|
RPKI_RTR_SLURM_DIR: "${RPKI_RTR_SLURM_DIR:-/app/slurm}"
|
||||||
|
RPKI_RTR_REPORT_DIR: "${RPKI_RTR_REPORT_DIR:-/app/report}"
|
||||||
|
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS: "${RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS:-300}"
|
||||||
|
RPKI_RTR_REPORT_HISTORY_LIMIT: "${RPKI_RTR_REPORT_HISTORY_LIMIT:-10}"
|
||||||
|
RPKI_RTR_TIMEZONE: "${RPKI_RTR_TIMEZONE:-Asia/Shanghai}"
|
||||||
|
RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS: "${RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS:-300}"
|
||||||
|
RPKI_RTR_MAX_DELTA: "${RPKI_RTR_MAX_DELTA:-10}"
|
||||||
|
RPKI_RTR_MAX_CONCURRENT_HANDSHAKES: "${RPKI_RTR_MAX_CONCURRENT_HANDSHAKES:-128}"
|
||||||
|
RPKI_RTR_ADMIN_ADDR: "${RPKI_RTR_ADMIN_ADDR:-}"
|
||||||
|
RPKI_RTR_ADMIN_TOKEN: "${RPKI_RTR_ADMIN_TOKEN:-}"
|
||||||
|
RUST_LOG: "${RUST_LOG:-info}"
|
||||||
|
# SSH mode example:
|
||||||
|
# RPKI_RTR_ENABLE_SSH: "true"
|
||||||
|
# RPKI_RTR_SSH_ADDR: "0.0.0.0:22"
|
||||||
|
# RPKI_RTR_SSH_PORT: "22"
|
||||||
|
# RPKI_RTR_SSH_HOST_KEY_PATH: "/app/certs/ssh_host_rsa_key"
|
||||||
|
# RPKI_RTR_SSH_AUTHORIZED_KEYS_PATH: "/app/certs/rtr-authorized_keys"
|
||||||
|
# RPKI_RTR_SSH_USERNAME: "rpki-rtr"
|
||||||
|
# RPKI_RTR_SSH_SUBSYSTEM_NAME: "rpki-rtr"
|
||||||
|
# Optional: enable password auth in addition to publickey
|
||||||
|
# RPKI_RTR_SSH_PASSWORD: "test-password"
|
||||||
|
volumes:
|
||||||
|
- ${RPKI_RTR_CCR_HOST_DIR:-../../data}:${RPKI_RTR_CCR_DIR:-/app/data}:ro
|
||||||
|
- ${RPKI_RTR_DB_HOST_DIR:-../../rtr-db}:${RPKI_RTR_DB_PATH:-/app/rtr-db}
|
||||||
|
- ${RPKI_RTR_SLURM_HOST_DIR:-../../data}:${RPKI_RTR_SLURM_DIR:-/app/slurm}
|
||||||
|
- ${RPKI_RTR_LOG_HOST_DIR:-../../logs/server}:/app/logs
|
||||||
|
- ${RPKI_RTR_REPORT_HOST_DIR:-../../report}:${RPKI_RTR_REPORT_DIR:-/app/report}
|
||||||
|
# TLS mode example:
|
||||||
|
# - ../../certs/ssh:/app/certs:ro
|
||||||
|
networks:
|
||||||
|
- rpki_net
|
||||||
|
|
||||||
|
networks:
|
||||||
|
rpki_net:
|
||||||
|
name: rpki_net
|
||||||
|
driver: bridge
|
||||||
10
deploy/server/entrypoint.sh
Normal file
10
deploy/server/entrypoint.sh
Normal file
@@ -0,0 +1,10 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
mkdir -p /app/logs
|
||||||
|
|
||||||
|
log_name="${HOSTNAME:-rpki-rtr}"
|
||||||
|
stdout_log="/app/logs/${log_name}.stdout.log"
|
||||||
|
stderr_log="/app/logs/${log_name}.stderr.log"
|
||||||
|
|
||||||
|
exec /usr/local/bin/rpki_rtr "$@" >>"$stdout_log" 2>>"$stderr_log"
|
||||||
310
docs/rtr-admin-api.md
Normal file
310
docs/rtr-admin-api.md
Normal file
@@ -0,0 +1,310 @@
|
|||||||
|
# RTR Admin API
|
||||||
|
|
||||||
|
本文档描述 RTR server 当前提供的 HTTP 管理接口。
|
||||||
|
|
||||||
|
## 启用方式
|
||||||
|
|
||||||
|
Admin API 默认关闭。设置 `RPKI_RTR_ADMIN_ADDR` 后启用：
|
||||||
|
|
||||||
|
```env
|
||||||
|
RPKI_RTR_ADMIN_ADDR=127.0.0.1:8323
|
||||||
|
RPKI_RTR_ADMIN_TOKEN=change-me
|
||||||
|
```
|
||||||
|
|
||||||
|
如果在 Docker 中需要从宿主机访问，容器内建议监听：
|
||||||
|
|
||||||
|
```env
|
||||||
|
RPKI_RTR_ADMIN_ADDR=0.0.0.0:8323
|
||||||
|
RPKI_RTR_ADMIN_TOKEN=change-me
|
||||||
|
```
|
||||||
|
|
||||||
|
并在 compose 中映射端口：
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
ports:
|
||||||
|
- "8323:8323"
|
||||||
|
```
|
||||||
|
|
||||||
|
安全规则：
|
||||||
|
|
||||||
|
- `RPKI_RTR_ADMIN_ADDR` 为空时，不启动 Admin API。
|
||||||
|
- 非 loopback 地址，例如 `0.0.0.0:8323`，必须配置 `RPKI_RTR_ADMIN_TOKEN`，否则 admin server 会拒绝启动。
|
||||||
|
- 设置 token 后，请求必须带 `Authorization: Bearer <token>`。
|
||||||
|
|
||||||
|
## 通用 Headers
|
||||||
|
|
||||||
|
```http
|
||||||
|
Authorization: Bearer change-me
|
||||||
|
Content-Type: application/json
|
||||||
|
```
|
||||||
|
|
||||||
|
GET 请求不需要 `Content-Type`。
|
||||||
|
|
||||||
|
## GET /admin/rtr/health
|
||||||
|
|
||||||
|
用于检查 Admin API 是否可达，以及当前启用了哪些管理能力。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://127.0.0.1:8323/admin/rtr/health \
|
||||||
|
-H "Authorization: Bearer change-me"
|
||||||
|
```
|
||||||
|
|
||||||
|
响应示例：
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "ok",
|
||||||
|
"config_api": true,
|
||||||
|
"source_reload_api": true,
|
||||||
|
"slurm_api": true,
|
||||||
|
"logs_api": true
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## GET /admin/rtr/logs/tail
|
||||||
|
|
||||||
|
实时读取 RTR 服务日志，效果类似 `tail -f`。默认读取 stdout 日志，先返回最近 200 行，然后持续输出新增内容。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -N "http://127.0.0.1:8323/admin/rtr/logs/tail?stream=stdout&lines=200&follow=true" \
|
||||||
|
-H "Authorization: Bearer change-me"
|
||||||
|
```
|
||||||
|
|
||||||
|
Query 参数：
|
||||||
|
|
||||||
|
| 参数 | 默认值 | 说明 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `stream` | `stdout` | 可选 `stdout` 或 `stderr`。 |
|
||||||
|
| `lines` | `200` | 首次返回的历史行数，范围会限制在 `1..=5000`。 |
|
||||||
|
| `follow` | `true` | `true` 时保持连接并持续输出新增日志；`false` 时只返回当前 tail 内容。 |
|
||||||
|
|
||||||
|
日志文件路径按容器 entrypoint 的规则解析：
|
||||||
|
|
||||||
|
- 目录：`RPKI_RTR_LOG_DIR`，默认 `/app/logs`。
|
||||||
|
- 文件名前缀：`RPKI_RTR_LOG_NAME`，未设置时使用 `HOSTNAME`，再未设置时为 `rpki-rtr`。
|
||||||
|
- stdout 文件：`<name>.stdout.log`。
|
||||||
|
- stderr 文件：`<name>.stderr.log`。
|
||||||
|
|
||||||
|
## GET /admin/rtr/config
|
||||||
|
|
||||||
|
查询当前运行中的 runtime 配置。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://127.0.0.1:8323/admin/rtr/config \
|
||||||
|
-H "Authorization: Bearer change-me"
|
||||||
|
```
|
||||||
|
|
||||||
|
响应示例：
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "ok",
|
||||||
|
"config": {
|
||||||
|
"max_delta": 10,
|
||||||
|
"prune_delta_by_snapshot_size": true,
|
||||||
|
"source_refresh_interval_seconds": 300,
|
||||||
|
"runtime_report_interval_seconds": 300,
|
||||||
|
"report_history_limit": 10,
|
||||||
|
"timezone": "Asia/Shanghai",
|
||||||
|
"timing": {
|
||||||
|
"refresh": 3600,
|
||||||
|
"retry": 600,
|
||||||
|
"expire": 7200
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## POST /admin/rtr/config
|
||||||
|
|
||||||
|
运行中动态修改 RTR server 的部分 runtime 配置。请求体是 JSON object，支持部分更新，只需要传要修改的字段。
|
||||||
|
|
||||||
|
| 字段 | 类型 | 说明 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| `max_delta` | integer | 每个 RTR 版本最多保留的 delta 条数，必须 `>= 1`。 |
|
||||||
|
| `prune_delta_by_snapshot_size` | boolean | 是否按 snapshot wire size 裁剪 delta window。 |
|
||||||
|
| `source_refresh_interval_seconds` | integer | CCR/SLURM source refresh 间隔，单位秒，必须 `>= 1`。 |
|
||||||
|
| `runtime_report_interval_seconds` | integer | `rtr-runtime-*.json` 周期写入间隔，单位秒，必须 `>= 1`。 |
|
||||||
|
| `report_history_limit` | integer | 每类 report 文件滚动保留数量，必须 `>= 1`。 |
|
||||||
|
| `timezone` | string | IANA 时区名，例如 `Asia/Shanghai`、`UTC`、`Europe/London`。 |
|
||||||
|
| `timing.refresh` | integer | RTR EndOfData `refresh`，单位秒。 |
|
||||||
|
| `timing.retry` | integer | RTR EndOfData `retry`，单位秒。 |
|
||||||
|
| `timing.expire` | integer | RTR EndOfData `expire`，必须大于 `refresh` 和 `retry`。 |
|
||||||
|
|
||||||
|
示例：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://127.0.0.1:8323/admin/rtr/config \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-H "Authorization: Bearer change-me" \
|
||||||
|
-d '{"max_delta": 6}'
|
||||||
|
```
|
||||||
|
|
||||||
|
成功响应：
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "ok",
|
||||||
|
"config": {
|
||||||
|
"max_delta": 6,
|
||||||
|
"prune_delta_by_snapshot_size": true,
|
||||||
|
"source_refresh_interval_seconds": 300,
|
||||||
|
"runtime_report_interval_seconds": 300,
|
||||||
|
"report_history_limit": 10,
|
||||||
|
"timezone": "Asia/Shanghai",
|
||||||
|
"timing": {
|
||||||
|
"refresh": 3600,
|
||||||
|
"retry": 600,
|
||||||
|
"expire": 7200
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## GET /admin/rtr/slurm/files
|
||||||
|
|
||||||
|
列出 `RPKI_RTR_SLURM_DIR` 下的 SLURM 文件。只返回合法的 `*.slurm` 和 `*.slurm.disabled` 文件，不返回 `.backup/`。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://127.0.0.1:8323/admin/rtr/slurm/files \
|
||||||
|
-H "Authorization: Bearer change-me"
|
||||||
|
```
|
||||||
|
|
||||||
|
响应示例：
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "ok",
|
||||||
|
"files": [
|
||||||
|
{
|
||||||
|
"name": "local-policy.slurm",
|
||||||
|
"path": "/app/slurm/local-policy.slurm",
|
||||||
|
"enabled": true,
|
||||||
|
"size_bytes": 168,
|
||||||
|
"modified_unix_seconds": 1782100000
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"name": "old-policy.slurm.disabled",
|
||||||
|
"path": "/app/slurm/old-policy.slurm.disabled",
|
||||||
|
"enabled": false,
|
||||||
|
"size_bytes": 168,
|
||||||
|
"modified_unix_seconds": 1782100100
|
||||||
|
}
|
||||||
|
]
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## GET /admin/rtr/slurm/files/{name}
|
||||||
|
|
||||||
|
读取单个 SLURM 文件，前端编辑页面可以直接使用这个接口加载内容。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://127.0.0.1:8323/admin/rtr/slurm/files/local-policy.slurm \
|
||||||
|
-H "Authorization: Bearer change-me"
|
||||||
|
```
|
||||||
|
|
||||||
|
响应示例：
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"status": "ok",
|
||||||
|
"file": {
|
||||||
|
"name": "local-policy.slurm",
|
||||||
|
"path": "/app/slurm/local-policy.slurm",
|
||||||
|
"enabled": true,
|
||||||
|
"size_bytes": 168,
|
||||||
|
"modified_unix_seconds": 1782100000,
|
||||||
|
"content": "{ \"slurmVersion\": 1, \"validationOutputFilters\": { \"prefixFilters\": [], \"bgpsecFilters\": [] }, \"locallyAddedAssertions\": { \"prefixAssertions\": [], \"bgpsecAssertions\": [] } }"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## POST /admin/rtr/slurm/files
|
||||||
|
|
||||||
|
新增或覆盖一个 SLURM 文件。文件名放在请求体中。
|
||||||
|
|
||||||
|
```json
|
||||||
|
{
|
||||||
|
"name": "local-policy.slurm",
|
||||||
|
"content": "{ \"slurmVersion\": 1, \"validationOutputFilters\": { \"prefixFilters\": [], \"bgpsecFilters\": [] }, \"locallyAddedAssertions\": { \"prefixAssertions\": [], \"bgpsecAssertions\": [] } }",
|
||||||
|
"reload": true
|
||||||
|
}
|
||||||
|
```
|
||||||
|
|
||||||
|
## PUT /admin/rtr/slurm/files/{name}
|
||||||
|
|
||||||
|
新增或覆盖指定 SLURM 文件。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X PUT http://127.0.0.1:8323/admin/rtr/slurm/files/local-policy.slurm \
|
||||||
|
-H "Content-Type: application/json" \
|
||||||
|
-H "Authorization: Bearer change-me" \
|
||||||
|
-d '{
|
||||||
|
"content": "{ \"slurmVersion\": 1, \"validationOutputFilters\": { \"prefixFilters\": [], \"bgpsecFilters\": [] }, \"locallyAddedAssertions\": { \"prefixAssertions\": [], \"bgpsecAssertions\": [] } }",
|
||||||
|
"reload": true
|
||||||
|
}'
|
||||||
|
```
|
||||||
|
|
||||||
|
## DELETE /admin/rtr/slurm/files/{name}
|
||||||
|
|
||||||
|
删除指定 SLURM 文件。删除前会先备份。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X DELETE "http://127.0.0.1:8323/admin/rtr/slurm/files/local-policy.slurm?reload=true" \
|
||||||
|
-H "Authorization: Bearer change-me"
|
||||||
|
```
|
||||||
|
|
||||||
|
## POST /admin/rtr/slurm/files/{name}/disable
|
||||||
|
|
||||||
|
将启用文件改名为失效文件：
|
||||||
|
|
||||||
|
```text
|
||||||
|
local-policy.slurm -> local-policy.slurm.disabled
|
||||||
|
```
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST "http://127.0.0.1:8323/admin/rtr/slurm/files/local-policy.slurm/disable?reload=true" \
|
||||||
|
-H "Authorization: Bearer change-me"
|
||||||
|
```
|
||||||
|
|
||||||
|
## POST /admin/rtr/slurm/files/{name}/enable
|
||||||
|
|
||||||
|
将失效文件改名为启用文件：
|
||||||
|
|
||||||
|
```text
|
||||||
|
local-policy.slurm.disabled -> local-policy.slurm
|
||||||
|
```
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST "http://127.0.0.1:8323/admin/rtr/slurm/files/local-policy.slurm.disabled/enable?reload=true" \
|
||||||
|
-H "Authorization: Bearer change-me"
|
||||||
|
```
|
||||||
|
|
||||||
|
## POST /admin/rtr/slurm/reload
|
||||||
|
|
||||||
|
不修改文件，只立即触发一次 CCR/SLURM source reload。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://127.0.0.1:8323/admin/rtr/slurm/reload \
|
||||||
|
-H "Authorization: Bearer change-me"
|
||||||
|
```
|
||||||
|
|
||||||
|
## SLURM 文件规则
|
||||||
|
|
||||||
|
- `*.slurm`：启用，会被 source refresh 加载。
|
||||||
|
- `*.slurm.disabled`：失效，不会被加载。
|
||||||
|
- `.backup/`：接口自动生成的备份目录。
|
||||||
|
- 文件名只能是 basename，不能包含 `/`、`\` 或 `..`。
|
||||||
|
- 文件名只能包含 ASCII 字母、数字、`.`、`_`、`-`。
|
||||||
|
- 写入内容必须是合法 JSON，并且能被解析为合法 SLURM 文件。
|
||||||
|
- 单个请求体默认限制为 5 MiB。
|
||||||
|
- 覆盖、删除前会写入 `.backup/` 备份。
|
||||||
|
|
||||||
|
## 错误响应
|
||||||
|
|
||||||
|
| HTTP 状态码 | 含义 |
|
||||||
|
| --- | --- |
|
||||||
|
| `400 Bad Request` | JSON 非法、配置值非法、SLURM 文件名非法、目标文件不存在、reload 失败等。 |
|
||||||
|
| `401 Unauthorized` | 配置了 token，但请求缺少或使用了错误的 `Authorization`。 |
|
||||||
|
| `404 Not Found` | 路径或方法不支持。 |
|
||||||
|
| `413 Payload Too Large` | 请求体超过限制。 |
|
||||||
203
scripts/00_0_install.sh
Normal file
203
scripts/00_0_install.sh
Normal file
@@ -0,0 +1,203 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||||
|
IMAGE_DIR="${ROOT_DIR}/images"
|
||||||
|
SERVER_ENV_FILE="${ROOT_DIR}/deploy/server/.env"
|
||||||
|
CONSOLE_ENV_FILE="${ROOT_DIR}/deploy/console/.env"
|
||||||
|
|
||||||
|
SERVER_TAR="${RPKI_RTR_SERVER_IMAGE_TAR:-}"
|
||||||
|
CLIENT_TAR="${RPKI_RTR_CLIENT_IMAGE_TAR:-}"
|
||||||
|
CONSOLE_TAR="${RPKI_RTR_CONSOLE_IMAGE_TAR:-}"
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage: scripts/00_0_install.sh [options]
|
||||||
|
|
||||||
|
Load offline Docker image tarballs from ./images, then run scripts/00_1_pre_check.sh.
|
||||||
|
|
||||||
|
Options:
|
||||||
|
--server-tar FILE Server image tarball. Env: RPKI_RTR_SERVER_IMAGE_TAR
|
||||||
|
--client-tar FILE Debug client image tarball. Env: RPKI_RTR_CLIENT_IMAGE_TAR
|
||||||
|
--console-tar FILE Console image tarball. Env: RPKI_RTR_CONSOLE_IMAGE_TAR
|
||||||
|
-h, --help Show this help.
|
||||||
|
|
||||||
|
Default tarballs under ./images:
|
||||||
|
server: rpki-rtr-*.tar, excluding rpki-rtr-debug-client-*.tar
|
||||||
|
client: rpki-rtr-debug-client-*.tar
|
||||||
|
console: rpki-console-ui-*.tar
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
log() {
|
||||||
|
printf '[00_install] %s\n' "$*" >&2
|
||||||
|
}
|
||||||
|
|
||||||
|
stage() {
|
||||||
|
printf '\n====== %s ======\n' "$*" >&2
|
||||||
|
}
|
||||||
|
|
||||||
|
die() {
|
||||||
|
printf '[00_install] ERROR: %s\n' "$*" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
set_env_var() {
|
||||||
|
local env_file="$1"
|
||||||
|
local key="$2"
|
||||||
|
local value="$3"
|
||||||
|
local tmp_file
|
||||||
|
|
||||||
|
[ -f "${env_file}" ] || die "env file not found: ${env_file}"
|
||||||
|
tmp_file="${env_file}.tmp.$$"
|
||||||
|
|
||||||
|
awk -v key="${key}" -v value="${value}" '
|
||||||
|
BEGIN { done = 0 }
|
||||||
|
$0 ~ "^[[:space:]]*#?[[:space:]]*" key "=" {
|
||||||
|
if (!done) {
|
||||||
|
print key "=" value
|
||||||
|
done = 1
|
||||||
|
}
|
||||||
|
next
|
||||||
|
}
|
||||||
|
{ print }
|
||||||
|
END {
|
||||||
|
if (!done) {
|
||||||
|
print key "=" value
|
||||||
|
}
|
||||||
|
}
|
||||||
|
' "${env_file}" >"${tmp_file}"
|
||||||
|
mv "${tmp_file}" "${env_file}"
|
||||||
|
}
|
||||||
|
|
||||||
|
while [ "$#" -gt 0 ]; do
|
||||||
|
case "$1" in
|
||||||
|
--server-tar)
|
||||||
|
[ "$#" -ge 2 ] || die "--server-tar requires a value"
|
||||||
|
SERVER_TAR="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--client-tar)
|
||||||
|
[ "$#" -ge 2 ] || die "--client-tar requires a value"
|
||||||
|
CLIENT_TAR="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--console-tar)
|
||||||
|
[ "$#" -ge 2 ] || die "--console-tar requires a value"
|
||||||
|
CONSOLE_TAR="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
-h|--help)
|
||||||
|
usage
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
die "unknown option: $1"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
resolve_tar() {
|
||||||
|
local path="$1"
|
||||||
|
|
||||||
|
if [ -f "${path}" ]; then
|
||||||
|
cd "$(dirname "${path}")" && printf '%s/%s\n' "$(pwd -P)" "$(basename "${path}")"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -f "${ROOT_DIR}/${path}" ]; then
|
||||||
|
cd "$(dirname "${ROOT_DIR}/${path}")" && printf '%s/%s\n' "$(pwd -P)" "$(basename "${path}")"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
load_image() {
|
||||||
|
local label="$1"
|
||||||
|
local tar_path="$2"
|
||||||
|
local output image
|
||||||
|
|
||||||
|
[ -f "${tar_path}" ] || die "${label} image tarball not found: ${tar_path}"
|
||||||
|
output="$(docker load -i "${tar_path}" 2>&1)" || {
|
||||||
|
printf '%s\n' "${output}" >&2
|
||||||
|
die "failed to load ${label} image from ${tar_path}"
|
||||||
|
}
|
||||||
|
printf '%s\n' "${output}" >&2
|
||||||
|
|
||||||
|
image="$(printf '%s\n' "${output}" | sed -n 's/^Loaded image: //p' | tail -n 1)"
|
||||||
|
[ -n "${image}" ] || die "cannot determine ${label} image name from docker load output"
|
||||||
|
printf '%s\n' "${image}"
|
||||||
|
}
|
||||||
|
|
||||||
|
find_single_tar() {
|
||||||
|
local label="$1"
|
||||||
|
local include_pattern="$2"
|
||||||
|
local exclude_pattern="${3:-}"
|
||||||
|
local matches=()
|
||||||
|
local file
|
||||||
|
|
||||||
|
while IFS= read -r -d '' file; do
|
||||||
|
matches+=("${file}")
|
||||||
|
done < <(
|
||||||
|
if [ -n "${exclude_pattern}" ]; then
|
||||||
|
find "${IMAGE_DIR}" -maxdepth 1 -type f -name "${include_pattern}" ! -name "${exclude_pattern}" -print0 | sort -z
|
||||||
|
else
|
||||||
|
find "${IMAGE_DIR}" -maxdepth 1 -type f -name "${include_pattern}" -print0 | sort -z
|
||||||
|
fi
|
||||||
|
)
|
||||||
|
|
||||||
|
case "${#matches[@]}" in
|
||||||
|
0)
|
||||||
|
die "no ${label} image tarball found under ${IMAGE_DIR} matching ${include_pattern}"
|
||||||
|
;;
|
||||||
|
1)
|
||||||
|
printf '%s\n' "${matches[0]}"
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
printf '[00_install] ERROR: multiple %s image tarballs found:\n' "${label}" >&2
|
||||||
|
printf ' %s\n' "${matches[@]}" >&2
|
||||||
|
die "please specify --${label}-tar or RPKI_RTR_${label^^}_IMAGE_TAR"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
stage "Checking Docker"
|
||||||
|
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
||||||
|
[ -d "${IMAGE_DIR}" ] || die "image directory not found: ${IMAGE_DIR}"
|
||||||
|
[ -f "${SERVER_ENV_FILE}" ] || die "server env file not found: ${SERVER_ENV_FILE}"
|
||||||
|
|
||||||
|
if [ -n "${SERVER_TAR}" ]; then
|
||||||
|
SERVER_TAR="$(resolve_tar "${SERVER_TAR}")" || die "server image tarball not found: ${SERVER_TAR}"
|
||||||
|
else
|
||||||
|
SERVER_TAR="$(find_single_tar server 'rpki-rtr-*.tar' 'rpki-rtr-debug-client-*.tar')"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -n "${CLIENT_TAR}" ]; then
|
||||||
|
CLIENT_TAR="$(resolve_tar "${CLIENT_TAR}")" || die "client image tarball not found: ${CLIENT_TAR}"
|
||||||
|
else
|
||||||
|
CLIENT_TAR="$(find_single_tar client 'rpki-rtr-debug-client-*.tar')"
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ -n "${CONSOLE_TAR}" ]; then
|
||||||
|
CONSOLE_TAR="$(resolve_tar "${CONSOLE_TAR}")" || die "console image tarball not found: ${CONSOLE_TAR}"
|
||||||
|
else
|
||||||
|
CONSOLE_TAR="$(find_single_tar console 'rpki-console-ui-*.tar')"
|
||||||
|
fi
|
||||||
|
|
||||||
|
stage "Loading Docker Images"
|
||||||
|
SERVER_IMAGE="$(load_image server "${SERVER_TAR}")"
|
||||||
|
CLIENT_IMAGE="$(load_image client "${CLIENT_TAR}")"
|
||||||
|
CONSOLE_IMAGE="$(load_image console "${CONSOLE_TAR}")"
|
||||||
|
|
||||||
|
stage "Updating Deploy Env"
|
||||||
|
set_env_var "${SERVER_ENV_FILE}" RPKI_RTR_SERVER_IMAGE "${SERVER_IMAGE}"
|
||||||
|
set_env_var "${CONSOLE_ENV_FILE}" RPKI_CONSOLE_UI_IMAGE "${CONSOLE_IMAGE}"
|
||||||
|
|
||||||
|
stage "Running Pre-Check"
|
||||||
|
RPKI_RTR_SERVER_IMAGE="${SERVER_IMAGE}" \
|
||||||
|
RPKI_RTR_CLIENT_IMAGE="${CLIENT_IMAGE}" \
|
||||||
|
bash "${SCRIPT_DIR}/00_1_pre_check.sh" || die "pre-check failed"
|
||||||
|
|
||||||
|
stage "Install Finished"
|
||||||
155
scripts/00_1_pre_check.sh
Normal file
155
scripts/00_1_pre_check.sh
Normal file
@@ -0,0 +1,155 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||||
|
|
||||||
|
PRECHECK_WORK_DIR="${PRECHECK_WORK_DIR:-${ROOT_DIR}/.precheck}"
|
||||||
|
PRECHECK_ENV_FILE="${PRECHECK_ENV_FILE:-${PRECHECK_WORK_DIR}/precheck.env}"
|
||||||
|
PRECHECK_WAIT_SECS="${PRECHECK_WAIT_SECS:-60}"
|
||||||
|
KEEP_PRECHECK="${KEEP_PRECHECK:-0}"
|
||||||
|
|
||||||
|
log() {
|
||||||
|
printf '[00_1_pre_check] %s\n' "$*"
|
||||||
|
}
|
||||||
|
|
||||||
|
stage() {
|
||||||
|
printf '\n====== %s ======\n' "$*"
|
||||||
|
}
|
||||||
|
|
||||||
|
die() {
|
||||||
|
printf '[00_1_pre_check] ERROR: %s\n' "$*" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
||||||
|
if docker compose version >/dev/null 2>&1; then
|
||||||
|
COMPOSE_BIN=(docker compose)
|
||||||
|
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
||||||
|
COMPOSE_BIN=(docker-compose)
|
||||||
|
else
|
||||||
|
die "docker compose is not available"
|
||||||
|
fi
|
||||||
|
|
||||||
|
stage "Preparing Pre-Check Data"
|
||||||
|
bash "${SCRIPT_DIR}/00_2_pre_check_prepare.sh" || die "prepare step failed"
|
||||||
|
|
||||||
|
set -a
|
||||||
|
# shellcheck disable=SC1090
|
||||||
|
. "${PRECHECK_ENV_FILE}"
|
||||||
|
set +a
|
||||||
|
|
||||||
|
ADMIN_PORT="${RPKI_RTR_ADMIN_ADDR##*:}"
|
||||||
|
ADMIN_TOKEN="${RPKI_RTR_ADMIN_TOKEN}"
|
||||||
|
|
||||||
|
server_compose() {
|
||||||
|
"${COMPOSE_BIN[@]}" --env-file "${PRECHECK_ENV_FILE}" -p "${PRECHECK_PROJECT}" -f "${PRECHECK_SERVER_COMPOSE_FILE}" "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
client_compose() {
|
||||||
|
"${COMPOSE_BIN[@]}" --env-file "${PRECHECK_ENV_FILE}" -p "${PRECHECK_PROJECT}" -f "${PRECHECK_CLIENT_COMPOSE_FILE}" "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
cleanup() {
|
||||||
|
KEEP_PRECHECK="${KEEP_PRECHECK}" PRECHECK_WORK_DIR="${PRECHECK_WORK_DIR}" PRECHECK_ENV_FILE="${PRECHECK_ENV_FILE}" \
|
||||||
|
bash "${SCRIPT_DIR}/00_3_pre_check_cleanup.sh"
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
|
||||||
|
http_get() {
|
||||||
|
local host="$1"
|
||||||
|
local port="$2"
|
||||||
|
local path="$3"
|
||||||
|
local token="$4"
|
||||||
|
|
||||||
|
exec 3<>"/dev/tcp/${host}/${port}" || return 1
|
||||||
|
printf 'GET %s HTTP/1.1\r\nHost: %s:%s\r\nAuthorization: Bearer %s\r\nConnection: close\r\n\r\n' \
|
||||||
|
"${path}" "${host}" "${port}" "${token}" >&3
|
||||||
|
cat <&3
|
||||||
|
exec 3>&-
|
||||||
|
}
|
||||||
|
|
||||||
|
wait_for_admin_health() {
|
||||||
|
local deadline=$((SECONDS + PRECHECK_WAIT_SECS))
|
||||||
|
local response
|
||||||
|
|
||||||
|
log "waiting for server health"
|
||||||
|
while [ "${SECONDS}" -lt "${deadline}" ]; do
|
||||||
|
response="$(http_get 127.0.0.1 "${ADMIN_PORT}" /admin/rtr/health "${ADMIN_TOKEN}" 2>/dev/null || true)"
|
||||||
|
if printf '%s' "${response}" | grep -q '"status"[[:space:]]*:[[:space:]]*"ok"'; then
|
||||||
|
log "admin health check passed"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "$(server_compose ps -q rpki-rtr | wc -l | tr -d ' ')" = "0" ]; then
|
||||||
|
server_compose logs rpki-rtr || true
|
||||||
|
die "server container is not running"
|
||||||
|
fi
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
|
||||||
|
server_compose logs rpki-rtr || true
|
||||||
|
die "admin health check did not pass within ${PRECHECK_WAIT_SECS}s"
|
||||||
|
}
|
||||||
|
|
||||||
|
wait_for_client_eod() {
|
||||||
|
local deadline=$((SECONDS + PRECHECK_WAIT_SECS))
|
||||||
|
local log_file
|
||||||
|
|
||||||
|
log "waiting for client response"
|
||||||
|
while [ "${SECONDS}" -lt "${deadline}" ]; do
|
||||||
|
for log_file in "${PRECHECK_WORK_DIR}"/logs/client/*.stdout.log; do
|
||||||
|
[ -f "${log_file}" ] || continue
|
||||||
|
if grep -q 'EndOfData:' "${log_file}" && grep -q 'summary[[:space:]]*:' "${log_file}"; then
|
||||||
|
log "client received EndOfData and payload summary"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
if grep -qi 'Error Report' "${log_file}"; then
|
||||||
|
cat "${log_file}" >&2
|
||||||
|
die "client received RTR Error Report"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
|
||||||
|
find "${PRECHECK_WORK_DIR}/logs/client" -type f -maxdepth 1 -print -exec cat {} \; >&2 || true
|
||||||
|
die "client did not receive EndOfData and summary within ${PRECHECK_WAIT_SECS}s"
|
||||||
|
}
|
||||||
|
|
||||||
|
assert_no_error_logs() {
|
||||||
|
local bad
|
||||||
|
log "checking logs"
|
||||||
|
bad="$(find "${PRECHECK_WORK_DIR}/logs" -type f -name '*.log' -print0 \
|
||||||
|
| xargs -0 grep -Ein 'panic|failed|RTR admin config server exited|Error Report' 2>/dev/null || true)"
|
||||||
|
if [ -n "${bad}" ]; then
|
||||||
|
printf '%s\n' "${bad}" >&2
|
||||||
|
die "error pattern found in pre-check logs"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
assert_report_written() {
|
||||||
|
local deadline=$((SECONDS + PRECHECK_WAIT_SECS))
|
||||||
|
|
||||||
|
log "waiting for source report"
|
||||||
|
while [ "${SECONDS}" -lt "${deadline}" ]; do
|
||||||
|
if find "${PRECHECK_WORK_DIR}/report" -type f -name 'rtr-source-*.json' | grep -q .; then
|
||||||
|
log "source report written"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
|
||||||
|
die "server did not write rtr-source report within ${PRECHECK_WAIT_SECS}s"
|
||||||
|
}
|
||||||
|
|
||||||
|
stage "Starting RTR Server"
|
||||||
|
server_compose up -d --no-build rpki-rtr
|
||||||
|
wait_for_admin_health
|
||||||
|
assert_report_written
|
||||||
|
|
||||||
|
stage "Starting Debug Client"
|
||||||
|
client_compose up -d --no-build rtr-debug-client
|
||||||
|
wait_for_client_eod
|
||||||
|
|
||||||
|
assert_no_error_logs
|
||||||
|
stage "Pre-Check Passed"
|
||||||
54
scripts/00_2_generate_ccr.sh
Normal file
54
scripts/00_2_generate_ccr.sh
Normal file
@@ -0,0 +1,54 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
OUT_PATH="${1:-}"
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage: scripts/00_2_generate_ccr.sh OUT.ccr
|
||||||
|
|
||||||
|
Generate a small deterministic CCR snapshot for pre-check.
|
||||||
|
This script writes a fixed base64-encoded CCR fixture and does not require Python.
|
||||||
|
The generated snapshot contains:
|
||||||
|
- VRP: 10.0.0.0/24 maxLen 24 AS65001
|
||||||
|
- VRP: 10.0.1.0/24 maxLen 24 AS65003
|
||||||
|
- VRP: 2001:db8:1::/48 maxLen 48 AS65002
|
||||||
|
- ASPA: customer AS65010 providers AS65011, AS65012, AS65013
|
||||||
|
- ASPA: customer AS65020 provider AS65021
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
die() {
|
||||||
|
printf '[00_2_generate_ccr] ERROR: %s\n' "$*" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
if [ "${OUT_PATH}" = "-h" ] || [ "${OUT_PATH}" = "--help" ]; then
|
||||||
|
usage
|
||||||
|
exit 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
[ -n "${OUT_PATH}" ] || die "missing OUT.ccr"
|
||||||
|
mkdir -p "$(dirname "${OUT_PATH}")"
|
||||||
|
|
||||||
|
decode_base64() {
|
||||||
|
if command -v base64 >/dev/null 2>&1; then
|
||||||
|
base64 -d
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
if command -v openssl >/dev/null 2>&1; then
|
||||||
|
openssl base64 -d
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
die "base64 or openssl command is required to write CCR fixture"
|
||||||
|
}
|
||||||
|
|
||||||
|
cat <<'EOF' | decode_base64 >"${OUT_PATH}"
|
||||||
|
MIIBAwYLKoZIhvcNAQkQATaggfMwgfACAQAwDQYJYIZIAWUDBAIBBQAYDzIwMjYwNDAzMDAwMjAx
|
||||||
|
WqJ9MHswVzAaAgMA/ekwEzARBAIAATALMAkDBAAKAAACARgwGgIDAP3rMBMwEQQCAAEwCzAJAwQA
|
||||||
|
CgABAgEYMB0CAwD96jAWMBQEAgACMA4wDAMHACABDbgAAQIBMAQgsUgCHfy5AJcFp5RymOD0HuGO
|
||||||
|
wfhoWpll90ugYECGQPSjTDBKMCYwFgIDAP3yMA8CAwD98wIDAP30AgMA/fUwDAIDAP38MAUCAwD9
|
||||||
|
/QQgRPxVPOOUsUuuV4tZ8BG5PHNtKiaEr23Ird+yNxlWJ+w=
|
||||||
|
EOF
|
||||||
96
scripts/00_2_pre_check_prepare.sh
Normal file
96
scripts/00_2_pre_check_prepare.sh
Normal file
@@ -0,0 +1,96 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||||
|
|
||||||
|
PRECHECK_WORK_DIR="${PRECHECK_WORK_DIR:-${ROOT_DIR}/.precheck}"
|
||||||
|
PRECHECK_PROJECT="${PRECHECK_PROJECT:-rpki-precheck}"
|
||||||
|
PRECHECK_ADMIN_TOKEN="${PRECHECK_ADMIN_TOKEN:-rpki-precheck-token}"
|
||||||
|
PRECHECK_PROTOCOL_VERSION="${PRECHECK_PROTOCOL_VERSION:-2}"
|
||||||
|
PRECHECK_SERVER_IMAGE="${PRECHECK_SERVER_IMAGE:-${RPKI_RTR_SERVER_IMAGE:-rpki-rtr-arm64:v1.2}}"
|
||||||
|
PRECHECK_CLIENT_IMAGE="${PRECHECK_CLIENT_IMAGE:-${RPKI_RTR_CLIENT_IMAGE:-rpki-rtr-debug-client-arm64:v1.2}}"
|
||||||
|
|
||||||
|
log() {
|
||||||
|
printf '[00_1_pre_check_prepare] %s\n' "$*"
|
||||||
|
}
|
||||||
|
|
||||||
|
die() {
|
||||||
|
printf '[00_1_pre_check_prepare] ERROR: %s\n' "$*" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
abs_dir() {
|
||||||
|
local path="$1"
|
||||||
|
mkdir -p "${path}"
|
||||||
|
cd "${path}" && pwd -P
|
||||||
|
}
|
||||||
|
|
||||||
|
[ -f "${SCRIPT_DIR}/00_2_generate_ccr.sh" ] || die "missing ${SCRIPT_DIR}/00_2_generate_ccr.sh"
|
||||||
|
|
||||||
|
rm -rf "${PRECHECK_WORK_DIR}"
|
||||||
|
mkdir -p \
|
||||||
|
"${PRECHECK_WORK_DIR}/ccr" \
|
||||||
|
"${PRECHECK_WORK_DIR}/slurm" \
|
||||||
|
"${PRECHECK_WORK_DIR}/db" \
|
||||||
|
"${PRECHECK_WORK_DIR}/logs/server" \
|
||||||
|
"${PRECHECK_WORK_DIR}/logs/client" \
|
||||||
|
"${PRECHECK_WORK_DIR}/report"
|
||||||
|
|
||||||
|
bash "${SCRIPT_DIR}/00_2_generate_ccr.sh" \
|
||||||
|
"${PRECHECK_WORK_DIR}/ccr/20260403T000201Z-precheck.ccr"
|
||||||
|
|
||||||
|
cat >"${PRECHECK_WORK_DIR}/slurm/precheck.slurm" <<'EOF'
|
||||||
|
{
|
||||||
|
"slurmVersion": 2,
|
||||||
|
"validationOutputFilters": {
|
||||||
|
"prefixFilters": [],
|
||||||
|
"bgpsecFilters": [],
|
||||||
|
"aspaFilters": []
|
||||||
|
},
|
||||||
|
"locallyAddedAssertions": {
|
||||||
|
"prefixAssertions": [
|
||||||
|
{
|
||||||
|
"prefix": "203.0.113.0/24",
|
||||||
|
"maxPrefixLength": 24,
|
||||||
|
"asn": 64496,
|
||||||
|
"comment": "pre-check local assertion"
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"bgpsecAssertions": [],
|
||||||
|
"aspaAssertions": []
|
||||||
|
}
|
||||||
|
}
|
||||||
|
EOF
|
||||||
|
|
||||||
|
PRECHECK_WORK_DIR_ABS="$(abs_dir "${PRECHECK_WORK_DIR}")"
|
||||||
|
cat >"${PRECHECK_WORK_DIR}/precheck.env" <<EOF
|
||||||
|
PRECHECK_WORK_DIR=${PRECHECK_WORK_DIR_ABS}
|
||||||
|
PRECHECK_PROJECT=${PRECHECK_PROJECT}
|
||||||
|
RPKI_RTR_SERVER_IMAGE=${PRECHECK_SERVER_IMAGE}
|
||||||
|
RPKI_RTR_CLIENT_IMAGE=${PRECHECK_CLIENT_IMAGE}
|
||||||
|
PRECHECK_SERVER_COMPOSE_FILE=${ROOT_DIR}/deploy/server/docker-compose.tcp.yml
|
||||||
|
PRECHECK_CLIENT_COMPOSE_FILE=${ROOT_DIR}/deploy/client/docker-compose.tcp.yml
|
||||||
|
RPKI_RTR_CCR_HOST_DIR=${PRECHECK_WORK_DIR_ABS}/ccr
|
||||||
|
RPKI_RTR_SLURM_HOST_DIR=${PRECHECK_WORK_DIR_ABS}/slurm
|
||||||
|
RPKI_RTR_DB_HOST_DIR=${PRECHECK_WORK_DIR_ABS}/db
|
||||||
|
RPKI_RTR_LOG_HOST_DIR=${PRECHECK_WORK_DIR_ABS}/logs/server
|
||||||
|
RPKI_RTR_CLIENT_LOG_HOST_DIR=${PRECHECK_WORK_DIR_ABS}/logs/client
|
||||||
|
RPKI_RTR_REPORT_HOST_DIR=${PRECHECK_WORK_DIR_ABS}/report
|
||||||
|
RPKI_RTR_CCR_DIR=/app/data
|
||||||
|
RPKI_RTR_SLURM_DIR=/app/slurm
|
||||||
|
RPKI_RTR_DB_PATH=/app/rtr-db
|
||||||
|
RPKI_RTR_REPORT_DIR=/app/report
|
||||||
|
RPKI_RTR_ADMIN_ADDR=0.0.0.0:8323
|
||||||
|
RPKI_RTR_ADMIN_TOKEN=${PRECHECK_ADMIN_TOKEN}
|
||||||
|
RPKI_RTR_RUNTIME_REPORT_INTERVAL_SECS=5
|
||||||
|
RPKI_RTR_REPORT_HISTORY_LIMIT=3
|
||||||
|
RPKI_RTR_TIMEZONE=Asia/Shanghai
|
||||||
|
RPKI_RTR_SOURCE_REFRESH_INTERVAL_SECS=30
|
||||||
|
RPKI_RTR_MAX_DELTA=10
|
||||||
|
RPKI_RTR_MAX_CONNECTIONS=32
|
||||||
|
RPKI_RTR_MAX_CONCURRENT_HANDSHAKES=8
|
||||||
|
RPKI_RTR_TCP_SERVER_ADDR=rpki-rtr:323
|
||||||
|
RPKI_RTR_PROTOCOL_VERSION=${PRECHECK_PROTOCOL_VERSION}
|
||||||
|
RUST_LOG=info
|
||||||
|
EOF
|
||||||
56
scripts/00_3_pre_check_cleanup.sh
Normal file
56
scripts/00_3_pre_check_cleanup.sh
Normal file
@@ -0,0 +1,56 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||||
|
|
||||||
|
PRECHECK_WORK_DIR="${PRECHECK_WORK_DIR:-${ROOT_DIR}/.precheck}"
|
||||||
|
ENV_FILE="${PRECHECK_ENV_FILE:-${PRECHECK_WORK_DIR}/precheck.env}"
|
||||||
|
KEEP_PRECHECK="${KEEP_PRECHECK:-0}"
|
||||||
|
|
||||||
|
if [ -f "${ENV_FILE}" ]; then
|
||||||
|
set -a
|
||||||
|
# shellcheck disable=SC1090
|
||||||
|
. "${ENV_FILE}"
|
||||||
|
set +a
|
||||||
|
fi
|
||||||
|
|
||||||
|
PRECHECK_PROJECT="${PRECHECK_PROJECT:-rpki-precheck}"
|
||||||
|
PRECHECK_SERVER_COMPOSE_FILE="${PRECHECK_SERVER_COMPOSE_FILE:-${ROOT_DIR}/deploy/server/docker-compose.tcp.yml}"
|
||||||
|
PRECHECK_CLIENT_COMPOSE_FILE="${PRECHECK_CLIENT_COMPOSE_FILE:-${ROOT_DIR}/deploy/client/docker-compose.tcp.yml}"
|
||||||
|
COMPOSE_BIN=()
|
||||||
|
|
||||||
|
log() {
|
||||||
|
printf '[00_1_pre_check_cleanup] %s\n' "$*"
|
||||||
|
}
|
||||||
|
|
||||||
|
compose_run() {
|
||||||
|
local compose_file="$1"
|
||||||
|
shift
|
||||||
|
|
||||||
|
if [ "${#COMPOSE_BIN[@]}" -eq 0 ]; then
|
||||||
|
if docker compose version >/dev/null 2>&1; then
|
||||||
|
COMPOSE_BIN=(docker compose)
|
||||||
|
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
||||||
|
COMPOSE_BIN=(docker-compose)
|
||||||
|
else
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
|
"${COMPOSE_BIN[@]}" --env-file "${ENV_FILE}" -p "${PRECHECK_PROJECT}" -f "${compose_file}" "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
if command -v docker >/dev/null 2>&1; then
|
||||||
|
log "stopping pre-check client compose stack"
|
||||||
|
[ -f "${PRECHECK_CLIENT_COMPOSE_FILE}" ] && compose_run "${PRECHECK_CLIENT_COMPOSE_FILE}" down --remove-orphans >/dev/null 2>&1 || true
|
||||||
|
log "stopping pre-check server compose stack"
|
||||||
|
[ -f "${PRECHECK_SERVER_COMPOSE_FILE}" ] && compose_run "${PRECHECK_SERVER_COMPOSE_FILE}" down --remove-orphans >/dev/null 2>&1 || true
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [ "${KEEP_PRECHECK}" = "1" ]; then
|
||||||
|
log "KEEP_PRECHECK=1, preserving ${PRECHECK_WORK_DIR}"
|
||||||
|
else
|
||||||
|
rm -rf "${PRECHECK_WORK_DIR}"
|
||||||
|
log "removed ${PRECHECK_WORK_DIR}"
|
||||||
|
fi
|
||||||
221
scripts/01_start.sh
Normal file
221
scripts/01_start.sh
Normal file
@@ -0,0 +1,221 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||||
|
|
||||||
|
COMMON_ENV_FILE="${ROOT_DIR}/environment.server"
|
||||||
|
SERVER_ENV_FILE="${ROOT_DIR}/deploy/server/.env"
|
||||||
|
CONSOLE_ENV_FILE="${ROOT_DIR}/deploy/console/.env"
|
||||||
|
CONSOLE_COMPOSE_FILE="${ROOT_DIR}/deploy/console/docker-compose.yml"
|
||||||
|
MODE="${RPKI_RTR_SERVER_MODE:-tcp}"
|
||||||
|
COMPOSE_PROJECT="${RPKI_RTR_COMPOSE_PROJECT:-rpki-rtr}"
|
||||||
|
CONSOLE_COMPOSE_PROJECT="${RPKI_CONSOLE_COMPOSE_PROJECT:-rpki-console}"
|
||||||
|
WAIT_SECS="${RPKI_START_WAIT_SECS:-20}"
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage: scripts/01_start.sh [options]
|
||||||
|
|
||||||
|
Read root ./environment.server, update deploy/server/.env, then start the RTR server and console.
|
||||||
|
|
||||||
|
Options:
|
||||||
|
--env FILE Common env file. Default: ./environment.server
|
||||||
|
--mode MODE Server mode: tcp|tls|ssh|base. Default: tcp
|
||||||
|
--wait SECS Wait timeout for container running checks. Default: 20
|
||||||
|
-h, --help Show this help.
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
log() {
|
||||||
|
printf '[01_start] %s\n' "$*" >&2
|
||||||
|
}
|
||||||
|
|
||||||
|
stage() {
|
||||||
|
printf '\n====== %s ======\n' "$*" >&2
|
||||||
|
}
|
||||||
|
|
||||||
|
die() {
|
||||||
|
printf '[01_start] ERROR: %s\n' "$*" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
set_env_var() {
|
||||||
|
local env_file="$1"
|
||||||
|
local key="$2"
|
||||||
|
local value="$3"
|
||||||
|
local tmp_file
|
||||||
|
|
||||||
|
tmp_file="${env_file}.tmp.$$"
|
||||||
|
awk -v key="${key}" -v value="${value}" '
|
||||||
|
BEGIN { done = 0 }
|
||||||
|
$0 ~ "^[[:space:]]*#?[[:space:]]*" key "=" {
|
||||||
|
if (!done) {
|
||||||
|
print key "=" value
|
||||||
|
done = 1
|
||||||
|
}
|
||||||
|
next
|
||||||
|
}
|
||||||
|
{ print }
|
||||||
|
END {
|
||||||
|
if (!done) {
|
||||||
|
print key "=" value
|
||||||
|
}
|
||||||
|
}
|
||||||
|
' "${env_file}" >"${tmp_file}"
|
||||||
|
mv "${tmp_file}" "${env_file}"
|
||||||
|
}
|
||||||
|
|
||||||
|
compose_file_for_mode() {
|
||||||
|
case "$1" in
|
||||||
|
tcp) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.tcp.yml" ;;
|
||||||
|
tls) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.tls.yml" ;;
|
||||||
|
ssh) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.ssh.yml" ;;
|
||||||
|
base) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.yml" ;;
|
||||||
|
*) die "unsupported mode '$1', expected tcp|tls|ssh|base" ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
update_env_from_common() {
|
||||||
|
local target_env_file="$1"
|
||||||
|
local updated_count=0
|
||||||
|
local line key value
|
||||||
|
|
||||||
|
while IFS= read -r line || [ -n "${line}" ]; do
|
||||||
|
case "${line}" in
|
||||||
|
''|\#*) continue ;;
|
||||||
|
*=*)
|
||||||
|
key="${line%%=*}"
|
||||||
|
value="${line#*=}"
|
||||||
|
set_env_var "${target_env_file}" "${key}" "${value}"
|
||||||
|
updated_count=$((updated_count + 1))
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done <"${COMMON_ENV_FILE}"
|
||||||
|
|
||||||
|
printf '%s\n' "${updated_count}"
|
||||||
|
}
|
||||||
|
|
||||||
|
server_compose() {
|
||||||
|
"${COMPOSE_BIN[@]}" --env-file "${SERVER_ENV_FILE}" -p "${COMPOSE_PROJECT}" -f "${SERVER_COMPOSE_FILE}" "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
console_compose() {
|
||||||
|
"${COMPOSE_BIN[@]}" --env-file "${CONSOLE_ENV_FILE}" -p "${CONSOLE_COMPOSE_PROJECT}" -f "${CONSOLE_COMPOSE_FILE}" "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
container_running() {
|
||||||
|
local compose_kind="$1"
|
||||||
|
local service="$2"
|
||||||
|
local cid state
|
||||||
|
|
||||||
|
if [ "${compose_kind}" = "server" ]; then
|
||||||
|
cid="$(server_compose ps -q "${service}" 2>/dev/null || true)"
|
||||||
|
else
|
||||||
|
cid="$(console_compose ps -q "${service}" 2>/dev/null || true)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
[ -n "${cid}" ] || return 1
|
||||||
|
state="$(docker inspect -f '{{.State.Running}}' "${cid}" 2>/dev/null || true)"
|
||||||
|
[ "${state}" = "true" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
show_service_debug() {
|
||||||
|
local compose_kind="$1"
|
||||||
|
local service="$2"
|
||||||
|
|
||||||
|
log "${service} is not running, current compose status:"
|
||||||
|
if [ "${compose_kind}" = "server" ]; then
|
||||||
|
server_compose ps "${service}" >&2 || true
|
||||||
|
log "${service} recent logs:"
|
||||||
|
server_compose logs --tail=80 "${service}" >&2 || true
|
||||||
|
else
|
||||||
|
console_compose ps "${service}" >&2 || true
|
||||||
|
log "${service} recent logs:"
|
||||||
|
console_compose logs --tail=80 "${service}" >&2 || true
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
wait_for_running() {
|
||||||
|
local compose_kind="$1"
|
||||||
|
local service="$2"
|
||||||
|
local deadline=$((SECONDS + WAIT_SECS))
|
||||||
|
|
||||||
|
while [ "${SECONDS}" -lt "${deadline}" ]; do
|
||||||
|
if container_running "${compose_kind}" "${service}"; then
|
||||||
|
log "${service} is running"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
|
||||||
|
show_service_debug "${compose_kind}" "${service}"
|
||||||
|
die "${service} failed to reach running state after ${WAIT_SECS}s"
|
||||||
|
}
|
||||||
|
|
||||||
|
while [ "$#" -gt 0 ]; do
|
||||||
|
case "$1" in
|
||||||
|
--env)
|
||||||
|
[ "$#" -ge 2 ] || die "--env requires a value"
|
||||||
|
COMMON_ENV_FILE="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--mode)
|
||||||
|
[ "$#" -ge 2 ] || die "--mode requires a value"
|
||||||
|
MODE="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--wait)
|
||||||
|
[ "$#" -ge 2 ] || die "--wait requires a value"
|
||||||
|
WAIT_SECS="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
-h|--help)
|
||||||
|
usage
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
die "unknown option: $1"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
SERVER_COMPOSE_FILE="$(compose_file_for_mode "${MODE}")"
|
||||||
|
|
||||||
|
stage "Checking Docker"
|
||||||
|
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
||||||
|
if docker compose version >/dev/null 2>&1; then
|
||||||
|
COMPOSE_BIN=(docker compose)
|
||||||
|
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
||||||
|
COMPOSE_BIN=(docker-compose)
|
||||||
|
else
|
||||||
|
die "docker compose is not available"
|
||||||
|
fi
|
||||||
|
|
||||||
|
[ -f "${SERVER_ENV_FILE}" ] || die "server env file not found: ${SERVER_ENV_FILE}"
|
||||||
|
[ -f "${SERVER_COMPOSE_FILE}" ] || die "server compose file not found: ${SERVER_COMPOSE_FILE}"
|
||||||
|
[ -f "${CONSOLE_ENV_FILE}" ] || die "console env file not found: ${CONSOLE_ENV_FILE}"
|
||||||
|
[ -f "${CONSOLE_COMPOSE_FILE}" ] || die "console compose file not found: ${CONSOLE_COMPOSE_FILE}"
|
||||||
|
|
||||||
|
if [ -f "${COMMON_ENV_FILE}" ]; then
|
||||||
|
stage "Reading Environment"
|
||||||
|
server_updated_count="$(update_env_from_common "${SERVER_ENV_FILE}")"
|
||||||
|
log "updated ${server_updated_count} server env variable(s)"
|
||||||
|
else
|
||||||
|
stage "Reading Environment"
|
||||||
|
log "environment.server file not found, skip env update"
|
||||||
|
fi
|
||||||
|
|
||||||
|
stage "Starting RTR Server"
|
||||||
|
log "starting docker compose service: rpki-rtr"
|
||||||
|
server_compose up -d --no-build rpki-rtr
|
||||||
|
log "waiting for rpki-rtr to be running"
|
||||||
|
wait_for_running server rpki-rtr
|
||||||
|
|
||||||
|
stage "Starting Console"
|
||||||
|
log "starting docker compose service: rpki-console-ui"
|
||||||
|
console_compose up -d --no-build rpki-console-ui
|
||||||
|
log "waiting for rpki-console-ui to be running"
|
||||||
|
wait_for_running console rpki-console-ui
|
||||||
|
|
||||||
|
stage "Start Finished"
|
||||||
184
scripts/02_self_check.sh
Normal file
184
scripts/02_self_check.sh
Normal file
@@ -0,0 +1,184 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||||
|
|
||||||
|
SERVER_ENV_FILE="${ROOT_DIR}/deploy/server/.env"
|
||||||
|
CONSOLE_ENV_FILE="${ROOT_DIR}/deploy/console/.env"
|
||||||
|
SERVER_COMPOSE_FILE="${ROOT_DIR}/deploy/server/docker-compose.tcp.yml"
|
||||||
|
CONSOLE_COMPOSE_FILE="${ROOT_DIR}/deploy/console/docker-compose.yml"
|
||||||
|
SERVER_PROJECT="${RPKI_RTR_COMPOSE_PROJECT:-rpki-rtr}"
|
||||||
|
CONSOLE_PROJECT="${RPKI_CONSOLE_COMPOSE_PROJECT:-rpki-console}"
|
||||||
|
WAIT_SECS="${RPKI_SELF_CHECK_WAIT_SECS:-60}"
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage: scripts/02_self_check.sh [options]
|
||||||
|
|
||||||
|
Check whether the RTR server and console started by scripts/01_start.sh are healthy.
|
||||||
|
|
||||||
|
Options:
|
||||||
|
--wait SECS Wait timeout for each check. Default: 60
|
||||||
|
-h, --help Show this help.
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
log() {
|
||||||
|
printf '[02_self_check] %s\n' "$*" >&2
|
||||||
|
}
|
||||||
|
|
||||||
|
stage() {
|
||||||
|
printf '\n====== %s ======\n' "$*" >&2
|
||||||
|
}
|
||||||
|
|
||||||
|
die() {
|
||||||
|
printf '[02_self_check] ERROR: %s\n' "$*" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
while [ "$#" -gt 0 ]; do
|
||||||
|
case "$1" in
|
||||||
|
--wait)
|
||||||
|
[ "$#" -ge 2 ] || die "--wait requires a value"
|
||||||
|
WAIT_SECS="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
-h|--help)
|
||||||
|
usage
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
die "unknown option: $1"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
||||||
|
if docker compose version >/dev/null 2>&1; then
|
||||||
|
COMPOSE_BIN=(docker compose)
|
||||||
|
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
||||||
|
COMPOSE_BIN=(docker-compose)
|
||||||
|
else
|
||||||
|
die "docker compose is not available"
|
||||||
|
fi
|
||||||
|
|
||||||
|
[ -f "${SERVER_ENV_FILE}" ] || die "server env file not found: ${SERVER_ENV_FILE}"
|
||||||
|
[ -f "${SERVER_COMPOSE_FILE}" ] || die "server compose file not found: ${SERVER_COMPOSE_FILE}"
|
||||||
|
[ -f "${CONSOLE_ENV_FILE}" ] || die "console env file not found: ${CONSOLE_ENV_FILE}"
|
||||||
|
[ -f "${CONSOLE_COMPOSE_FILE}" ] || die "console compose file not found: ${CONSOLE_COMPOSE_FILE}"
|
||||||
|
|
||||||
|
set -a
|
||||||
|
# shellcheck disable=SC1090
|
||||||
|
. "${SERVER_ENV_FILE}"
|
||||||
|
# shellcheck disable=SC1090
|
||||||
|
. "${CONSOLE_ENV_FILE}"
|
||||||
|
set +a
|
||||||
|
|
||||||
|
RTR_PORT="${RPKI_RTR_TCP_HOST_PORT:-323}"
|
||||||
|
ADMIN_PORT="${RPKI_RTR_ADMIN_ADDR##*:}"
|
||||||
|
ADMIN_PORT="${ADMIN_PORT:-8323}"
|
||||||
|
ADMIN_TOKEN="${RPKI_RTR_ADMIN_TOKEN:-}"
|
||||||
|
CONSOLE_PORT="${RPKI_CONSOLE_UI_PORT:-5173}"
|
||||||
|
|
||||||
|
server_compose() {
|
||||||
|
"${COMPOSE_BIN[@]}" --env-file "${SERVER_ENV_FILE}" -p "${SERVER_PROJECT}" -f "${SERVER_COMPOSE_FILE}" "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
console_compose() {
|
||||||
|
"${COMPOSE_BIN[@]}" --env-file "${CONSOLE_ENV_FILE}" -p "${CONSOLE_PROJECT}" -f "${CONSOLE_COMPOSE_FILE}" "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
container_running() {
|
||||||
|
local compose_kind="$1"
|
||||||
|
local service="$2"
|
||||||
|
local cid state
|
||||||
|
|
||||||
|
if [ "${compose_kind}" = "server" ]; then
|
||||||
|
cid="$(server_compose ps -q "${service}" 2>/dev/null || true)"
|
||||||
|
else
|
||||||
|
cid="$(console_compose ps -q "${service}" 2>/dev/null || true)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
[ -n "${cid}" ] || return 1
|
||||||
|
state="$(docker inspect -f '{{.State.Running}}' "${cid}" 2>/dev/null || true)"
|
||||||
|
[ "${state}" = "true" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
tcp_open() {
|
||||||
|
local host="$1"
|
||||||
|
local port="$2"
|
||||||
|
|
||||||
|
if command -v nc >/dev/null 2>&1; then
|
||||||
|
nc -z "${host}" "${port}" >/dev/null 2>&1
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
(exec 3<>"/dev/tcp/${host}/${port}") >/dev/null 2>&1
|
||||||
|
}
|
||||||
|
|
||||||
|
http_get() {
|
||||||
|
local url="$1"
|
||||||
|
local token="${2:-}"
|
||||||
|
|
||||||
|
if command -v curl >/dev/null 2>&1; then
|
||||||
|
if [ -n "${token}" ]; then
|
||||||
|
curl -fsS -H "Authorization: Bearer ${token}" "${url}"
|
||||||
|
else
|
||||||
|
curl -fsS "${url}"
|
||||||
|
fi
|
||||||
|
return
|
||||||
|
fi
|
||||||
|
|
||||||
|
die "curl command is required for HTTP health checks"
|
||||||
|
}
|
||||||
|
|
||||||
|
wait_until() {
|
||||||
|
local label="$1"
|
||||||
|
shift
|
||||||
|
local deadline=$((SECONDS + WAIT_SECS))
|
||||||
|
|
||||||
|
while [ "${SECONDS}" -lt "${deadline}" ]; do
|
||||||
|
if "$@"; then
|
||||||
|
log "${label}: ok"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
|
||||||
|
die "${label}: failed after ${WAIT_SECS}s"
|
||||||
|
}
|
||||||
|
|
||||||
|
show_command() {
|
||||||
|
printf '[02_self_check] check command: %s\n' "$*" >&2
|
||||||
|
}
|
||||||
|
|
||||||
|
check_admin_health() {
|
||||||
|
local body
|
||||||
|
body="$(http_get "http://127.0.0.1:${ADMIN_PORT}/admin/rtr/health" "${ADMIN_TOKEN}" 2>/dev/null || true)"
|
||||||
|
printf '%s' "${body}" | grep -q '"status"[[:space:]]*:[[:space:]]*"ok"'
|
||||||
|
}
|
||||||
|
|
||||||
|
check_console_http() {
|
||||||
|
http_get "http://127.0.0.1:${CONSOLE_PORT}/" >/dev/null 2>&1
|
||||||
|
}
|
||||||
|
|
||||||
|
stage "Checking Containers"
|
||||||
|
show_command "${COMPOSE_BIN[*]} --env-file ${SERVER_ENV_FILE} -p ${SERVER_PROJECT} -f ${SERVER_COMPOSE_FILE} ps -q rpki-rtr && docker inspect"
|
||||||
|
wait_until "server container" container_running server rpki-rtr
|
||||||
|
show_command "${COMPOSE_BIN[*]} --env-file ${CONSOLE_ENV_FILE} -p ${CONSOLE_PROJECT} -f ${CONSOLE_COMPOSE_FILE} ps -q rpki-console-ui && docker inspect"
|
||||||
|
wait_until "console container" container_running console rpki-console-ui
|
||||||
|
|
||||||
|
stage "Checking Server"
|
||||||
|
show_command "tcp connect 127.0.0.1:${RTR_PORT}"
|
||||||
|
wait_until "RTR port ${RTR_PORT}" tcp_open 127.0.0.1 "${RTR_PORT}"
|
||||||
|
show_command "curl -fsS -H 'Authorization: Bearer <token>' http://127.0.0.1:${ADMIN_PORT}/admin/rtr/health"
|
||||||
|
wait_until "Admin health" check_admin_health
|
||||||
|
|
||||||
|
stage "Checking Console"
|
||||||
|
show_command "tcp connect 127.0.0.1:${CONSOLE_PORT}"
|
||||||
|
wait_until "Console port ${CONSOLE_PORT}" tcp_open 127.0.0.1 "${CONSOLE_PORT}"
|
||||||
|
show_command "curl -fsS http://127.0.0.1:${CONSOLE_PORT}/"
|
||||||
|
wait_until "Console HTTP" check_console_http
|
||||||
|
|
||||||
|
stage "Self-Check Passed"
|
||||||
92
scripts/03_shutdown.sh
Normal file
92
scripts/03_shutdown.sh
Normal file
@@ -0,0 +1,92 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||||
|
|
||||||
|
SERVER_ENV_FILE="${ROOT_DIR}/deploy/server/.env"
|
||||||
|
CONSOLE_ENV_FILE="${ROOT_DIR}/deploy/console/.env"
|
||||||
|
MODE="${RPKI_RTR_SERVER_MODE:-tcp}"
|
||||||
|
SERVER_PROJECT="${RPKI_RTR_COMPOSE_PROJECT:-rpki-rtr}"
|
||||||
|
CONSOLE_PROJECT="${RPKI_CONSOLE_COMPOSE_PROJECT:-rpki-console}"
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage: scripts/03_shutdown.sh [options]
|
||||||
|
|
||||||
|
Stop services started by scripts/01_start.sh.
|
||||||
|
|
||||||
|
Options:
|
||||||
|
--mode MODE Server mode: tcp|tls|ssh|base. Default: tcp
|
||||||
|
-h, --help Show this help.
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
log() {
|
||||||
|
printf '[03_shutdown] %s\n' "$*" >&2
|
||||||
|
}
|
||||||
|
|
||||||
|
stage() {
|
||||||
|
printf '\n====== %s ======\n' "$*" >&2
|
||||||
|
}
|
||||||
|
|
||||||
|
die() {
|
||||||
|
printf '[03_shutdown] ERROR: %s\n' "$*" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
compose_file_for_mode() {
|
||||||
|
case "$1" in
|
||||||
|
tcp) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.tcp.yml" ;;
|
||||||
|
tls) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.tls.yml" ;;
|
||||||
|
ssh) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.ssh.yml" ;;
|
||||||
|
base) printf '%s\n' "${ROOT_DIR}/deploy/server/docker-compose.yml" ;;
|
||||||
|
*) die "unsupported mode '$1', expected tcp|tls|ssh|base" ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
while [ "$#" -gt 0 ]; do
|
||||||
|
case "$1" in
|
||||||
|
--mode)
|
||||||
|
[ "$#" -ge 2 ] || die "--mode requires a value"
|
||||||
|
MODE="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
-h|--help)
|
||||||
|
usage
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
die "unknown option: $1"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
SERVER_COMPOSE_FILE="$(compose_file_for_mode "${MODE}")"
|
||||||
|
CONSOLE_COMPOSE_FILE="${ROOT_DIR}/deploy/console/docker-compose.yml"
|
||||||
|
|
||||||
|
stage "Checking Docker"
|
||||||
|
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
||||||
|
if docker compose version >/dev/null 2>&1; then
|
||||||
|
COMPOSE_BIN=(docker compose)
|
||||||
|
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
||||||
|
COMPOSE_BIN=(docker-compose)
|
||||||
|
else
|
||||||
|
die "docker compose is not available"
|
||||||
|
fi
|
||||||
|
|
||||||
|
stage "Stopping Console"
|
||||||
|
if [ -f "${CONSOLE_ENV_FILE}" ] && [ -f "${CONSOLE_COMPOSE_FILE}" ]; then
|
||||||
|
"${COMPOSE_BIN[@]}" --env-file "${CONSOLE_ENV_FILE}" -p "${CONSOLE_PROJECT}" -f "${CONSOLE_COMPOSE_FILE}" down --remove-orphans
|
||||||
|
else
|
||||||
|
log "console compose/env not found, skip"
|
||||||
|
fi
|
||||||
|
|
||||||
|
stage "Stopping RTR Server"
|
||||||
|
if [ -f "${SERVER_ENV_FILE}" ] && [ -f "${SERVER_COMPOSE_FILE}" ]; then
|
||||||
|
"${COMPOSE_BIN[@]}" --env-file "${SERVER_ENV_FILE}" -p "${SERVER_PROJECT}" -f "${SERVER_COMPOSE_FILE}" down --remove-orphans
|
||||||
|
else
|
||||||
|
log "server compose/env not found, skip"
|
||||||
|
fi
|
||||||
|
|
||||||
|
stage "Shutdown Finished"
|
||||||
152
scripts/04_cleanup.sh
Normal file
152
scripts/04_cleanup.sh
Normal file
@@ -0,0 +1,152 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||||
|
SERVER_DIR="${ROOT_DIR}/deploy/server"
|
||||||
|
SERVER_ENV_FILE="${SERVER_DIR}/.env"
|
||||||
|
SERVER_COMPOSE_FILE="${SERVER_DIR}/docker-compose.tcp.yml"
|
||||||
|
COMPOSE_PROJECT="${RPKI_RTR_COMPOSE_PROJECT:-rpki-rtr}"
|
||||||
|
FORCE=0
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage: scripts/04_cleanup.sh [options]
|
||||||
|
|
||||||
|
Remove server generated directories: logs, reports, and RTR database.
|
||||||
|
|
||||||
|
Options:
|
||||||
|
--force Clean even if the server container appears to be running.
|
||||||
|
-h, --help Show this help.
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
log() {
|
||||||
|
printf '[04_cleanup] %s\n' "$*" >&2
|
||||||
|
}
|
||||||
|
|
||||||
|
stage() {
|
||||||
|
printf '\n====== %s ======\n' "$*" >&2
|
||||||
|
}
|
||||||
|
|
||||||
|
die() {
|
||||||
|
printf '[04_cleanup] ERROR: %s\n' "$*" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
env_value() {
|
||||||
|
local key="$1"
|
||||||
|
local default_value="$2"
|
||||||
|
local line
|
||||||
|
|
||||||
|
line="$(grep -E "^[[:space:]]*${key}=" "${SERVER_ENV_FILE}" | tail -n 1 || true)"
|
||||||
|
if [ -n "${line}" ]; then
|
||||||
|
printf '%s\n' "${line#*=}"
|
||||||
|
else
|
||||||
|
printf '%s\n' "${default_value}"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
resolve_host_path() {
|
||||||
|
local raw_path="$1"
|
||||||
|
|
||||||
|
case "${raw_path}" in
|
||||||
|
/*|[A-Za-z]:/*|[A-Za-z]:\\*)
|
||||||
|
printf '%s\n' "${raw_path}"
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
(cd "${SERVER_DIR}" && cd "${raw_path}" 2>/dev/null && pwd -P) || printf '%s/%s\n' "${SERVER_DIR}" "${raw_path}"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
compose_available() {
|
||||||
|
if docker compose version >/dev/null 2>&1; then
|
||||||
|
COMPOSE_BIN=(docker compose)
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
||||||
|
COMPOSE_BIN=(docker-compose)
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
server_running() {
|
||||||
|
local cid state
|
||||||
|
|
||||||
|
[ -f "${SERVER_COMPOSE_FILE}" ] || return 1
|
||||||
|
compose_available || return 1
|
||||||
|
|
||||||
|
cid="$("${COMPOSE_BIN[@]}" --env-file "${SERVER_ENV_FILE}" -p "${COMPOSE_PROJECT}" -f "${SERVER_COMPOSE_FILE}" ps -q rpki-rtr 2>/dev/null || true)"
|
||||||
|
[ -n "${cid}" ] || return 1
|
||||||
|
|
||||||
|
state="$(docker inspect -f '{{.State.Running}}' "${cid}" 2>/dev/null || true)"
|
||||||
|
[ "${state}" = "true" ]
|
||||||
|
}
|
||||||
|
|
||||||
|
clean_dir() {
|
||||||
|
local label="$1"
|
||||||
|
local path="$2"
|
||||||
|
local root_with_slash
|
||||||
|
|
||||||
|
if [ ! -d "${path}" ]; then
|
||||||
|
log "${label}: ${path} not found, skip"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
root_with_slash="${ROOT_DIR}/"
|
||||||
|
case "${path}/" in
|
||||||
|
"${root_with_slash}"*) ;;
|
||||||
|
*)
|
||||||
|
log "ERROR: ${label} path is outside repo root, refuse to remove: ${path}"
|
||||||
|
return 1
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
|
||||||
|
log "${label}: removing ${path}"
|
||||||
|
if ! rm -rf "${path}"; then
|
||||||
|
log "ERROR: failed to remove ${label}: ${path}"
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
while [ "$#" -gt 0 ]; do
|
||||||
|
case "$1" in
|
||||||
|
--force)
|
||||||
|
FORCE=1
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
-h|--help)
|
||||||
|
usage
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
die "unknown option: $1"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
stage "Checking Server"
|
||||||
|
[ -f "${SERVER_ENV_FILE}" ] || die "server env file not found: ${SERVER_ENV_FILE}"
|
||||||
|
if [ "${FORCE}" -ne 1 ] && command -v docker >/dev/null 2>&1 && server_running; then
|
||||||
|
die "rpki-rtr is still running, run scripts/03_shutdown.sh first or use --force"
|
||||||
|
fi
|
||||||
|
|
||||||
|
LOG_DIR="${ROOT_DIR}/logs"
|
||||||
|
REPORT_DIR="$(resolve_host_path "$(env_value RPKI_RTR_REPORT_HOST_DIR '../../report')")"
|
||||||
|
DB_DIR="$(resolve_host_path "$(env_value RPKI_RTR_DB_HOST_DIR '../../rtr-db')")"
|
||||||
|
|
||||||
|
stage "Removing Server Files"
|
||||||
|
failed_count=0
|
||||||
|
clean_dir "logs" "${LOG_DIR}" || failed_count=$((failed_count + 1))
|
||||||
|
clean_dir "report" "${REPORT_DIR}" || failed_count=$((failed_count + 1))
|
||||||
|
clean_dir "rtr-db" "${DB_DIR}" || failed_count=$((failed_count + 1))
|
||||||
|
|
||||||
|
if [ "${failed_count}" -gt 0 ]; then
|
||||||
|
die "${failed_count} cleanup item(s) failed"
|
||||||
|
fi
|
||||||
|
|
||||||
|
stage "Cleanup Finished"
|
||||||
303
scripts/10_generate_certs.sh
Normal file
303
scripts/10_generate_certs.sh
Normal file
@@ -0,0 +1,303 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||||
|
OUT_DIR="${ROOT_DIR}/certs"
|
||||||
|
TLS_DIR="${OUT_DIR}/tls"
|
||||||
|
SSH_DIR="${OUT_DIR}/ssh"
|
||||||
|
SERVER_DNS="localhost"
|
||||||
|
FORCE=0
|
||||||
|
DO_TLS=1
|
||||||
|
DO_SSH=1
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'USAGE'
|
||||||
|
Generate local development TLS certificates and SSH keys for RTR Docker deploys.
|
||||||
|
|
||||||
|
Usage:
|
||||||
|
scripts/10_generate-certs.sh [options]
|
||||||
|
|
||||||
|
Options:
|
||||||
|
--out-dir DIR Output directory, default: ./certs
|
||||||
|
--server-dns NAME TLS server dNSName SAN, default: localhost
|
||||||
|
--tls-only Generate only TLS files
|
||||||
|
--ssh-only Generate only SSH files
|
||||||
|
--force Overwrite existing files
|
||||||
|
-h, --help Show this help
|
||||||
|
|
||||||
|
Generated TLS files:
|
||||||
|
certs/tls/client-ca.crt
|
||||||
|
certs/tls/client-ca.key
|
||||||
|
certs/tls/server-dns.crt
|
||||||
|
certs/tls/server-dns.key
|
||||||
|
certs/tls/client-good.crt
|
||||||
|
certs/tls/client-good.key
|
||||||
|
|
||||||
|
Generated SSH files:
|
||||||
|
certs/ssh/ssh_host_rsa_key
|
||||||
|
certs/ssh/ssh_host_rsa_key.pub
|
||||||
|
certs/ssh/rtr-client.key
|
||||||
|
certs/ssh/rtr-client.key.pub
|
||||||
|
certs/ssh/rtr-authorized_keys
|
||||||
|
USAGE
|
||||||
|
}
|
||||||
|
|
||||||
|
die() {
|
||||||
|
echo "error: $*" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
info() {
|
||||||
|
echo "[generate-certs] $*"
|
||||||
|
}
|
||||||
|
|
||||||
|
need_cmd() {
|
||||||
|
command -v "$1" >/dev/null 2>&1 || die "required command not found: $1"
|
||||||
|
}
|
||||||
|
|
||||||
|
run_quiet() {
|
||||||
|
local log_file
|
||||||
|
log_file="$(mktemp)"
|
||||||
|
if "$@" >"${log_file}" 2>&1; then
|
||||||
|
rm -f "${log_file}"
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
|
||||||
|
cat "${log_file}" >&2
|
||||||
|
rm -f "${log_file}"
|
||||||
|
return 1
|
||||||
|
}
|
||||||
|
|
||||||
|
ensure_abs_out_dir() {
|
||||||
|
case "${OUT_DIR}" in
|
||||||
|
/*) ;;
|
||||||
|
[A-Za-z]:*) ;;
|
||||||
|
*) OUT_DIR="${ROOT_DIR}/${OUT_DIR}" ;;
|
||||||
|
esac
|
||||||
|
TLS_DIR="${OUT_DIR}/tls"
|
||||||
|
SSH_DIR="${OUT_DIR}/ssh"
|
||||||
|
}
|
||||||
|
|
||||||
|
check_overwrite() {
|
||||||
|
local path="$1"
|
||||||
|
if [[ -e "${path}" && "${FORCE}" != "1" ]]; then
|
||||||
|
die "${path} already exists; rerun with --force to overwrite"
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
|
write_tls_config_files() {
|
||||||
|
local tmp_dir="$1"
|
||||||
|
|
||||||
|
cat >"${tmp_dir}/ca.cnf" <<'EOF'
|
||||||
|
[ req ]
|
||||||
|
distinguished_name = req_distinguished_name
|
||||||
|
prompt = no
|
||||||
|
|
||||||
|
[ req_distinguished_name ]
|
||||||
|
CN = rpki-rtr-client-ca
|
||||||
|
EOF
|
||||||
|
|
||||||
|
cat >"${tmp_dir}/server.cnf" <<EOF
|
||||||
|
[ req ]
|
||||||
|
distinguished_name = req_distinguished_name
|
||||||
|
prompt = no
|
||||||
|
|
||||||
|
[ req_distinguished_name ]
|
||||||
|
CN = ${SERVER_DNS}
|
||||||
|
EOF
|
||||||
|
|
||||||
|
cat >"${tmp_dir}/client.cnf" <<'EOF'
|
||||||
|
[ req ]
|
||||||
|
distinguished_name = req_distinguished_name
|
||||||
|
prompt = no
|
||||||
|
|
||||||
|
[ req_distinguished_name ]
|
||||||
|
CN = rpki-rtr-client
|
||||||
|
EOF
|
||||||
|
|
||||||
|
cat >"${tmp_dir}/server.ext" <<EOF
|
||||||
|
basicConstraints=CA:FALSE
|
||||||
|
keyUsage=digitalSignature,keyEncipherment
|
||||||
|
extendedKeyUsage=serverAuth
|
||||||
|
subjectAltName=DNS:${SERVER_DNS}
|
||||||
|
EOF
|
||||||
|
|
||||||
|
cat >"${tmp_dir}/client.ext" <<'EOF'
|
||||||
|
basicConstraints=CA:FALSE
|
||||||
|
keyUsage=digitalSignature,keyEncipherment
|
||||||
|
extendedKeyUsage=clientAuth
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
generate_tls() {
|
||||||
|
need_cmd openssl
|
||||||
|
mkdir -p "${TLS_DIR}"
|
||||||
|
|
||||||
|
info "generating TLS certificates in ${TLS_DIR}"
|
||||||
|
info "TLS server dNSName SAN: ${SERVER_DNS}"
|
||||||
|
|
||||||
|
local files=(
|
||||||
|
"${TLS_DIR}/client-ca.crt"
|
||||||
|
"${TLS_DIR}/client-ca.key"
|
||||||
|
"${TLS_DIR}/server-dns.crt"
|
||||||
|
"${TLS_DIR}/server-dns.key"
|
||||||
|
"${TLS_DIR}/client-good.crt"
|
||||||
|
"${TLS_DIR}/client-good.key"
|
||||||
|
)
|
||||||
|
for file in "${files[@]}"; do
|
||||||
|
check_overwrite "${file}"
|
||||||
|
done
|
||||||
|
|
||||||
|
local tmp_dir
|
||||||
|
tmp_dir="$(mktemp -d)"
|
||||||
|
write_tls_config_files "${tmp_dir}"
|
||||||
|
|
||||||
|
run_quiet openssl genrsa -out "${TLS_DIR}/client-ca.key" 2048
|
||||||
|
info "created TLS CA key: ${TLS_DIR}/client-ca.key"
|
||||||
|
run_quiet openssl req -x509 -new -nodes \
|
||||||
|
-config "${tmp_dir}/ca.cnf" \
|
||||||
|
-key "${TLS_DIR}/client-ca.key" \
|
||||||
|
-sha256 -days 3650 \
|
||||||
|
-out "${TLS_DIR}/client-ca.crt"
|
||||||
|
info "created TLS CA cert: ${TLS_DIR}/client-ca.crt"
|
||||||
|
|
||||||
|
run_quiet openssl genrsa -out "${TLS_DIR}/server-dns.key" 2048
|
||||||
|
info "created TLS server key: ${TLS_DIR}/server-dns.key"
|
||||||
|
run_quiet openssl req -new \
|
||||||
|
-config "${tmp_dir}/server.cnf" \
|
||||||
|
-key "${TLS_DIR}/server-dns.key" \
|
||||||
|
-out "${tmp_dir}/server-dns.csr"
|
||||||
|
run_quiet openssl x509 -req \
|
||||||
|
-in "${tmp_dir}/server-dns.csr" \
|
||||||
|
-CA "${TLS_DIR}/client-ca.crt" \
|
||||||
|
-CAkey "${TLS_DIR}/client-ca.key" \
|
||||||
|
-CAcreateserial \
|
||||||
|
-out "${TLS_DIR}/server-dns.crt" \
|
||||||
|
-days 3650 -sha256 \
|
||||||
|
-extfile "${tmp_dir}/server.ext"
|
||||||
|
info "created TLS server cert: ${TLS_DIR}/server-dns.crt"
|
||||||
|
|
||||||
|
run_quiet openssl genrsa -out "${TLS_DIR}/client-good.key" 2048
|
||||||
|
info "created TLS client key: ${TLS_DIR}/client-good.key"
|
||||||
|
run_quiet openssl req -new \
|
||||||
|
-config "${tmp_dir}/client.cnf" \
|
||||||
|
-key "${TLS_DIR}/client-good.key" \
|
||||||
|
-out "${tmp_dir}/client-good.csr"
|
||||||
|
run_quiet openssl x509 -req \
|
||||||
|
-in "${tmp_dir}/client-good.csr" \
|
||||||
|
-CA "${TLS_DIR}/client-ca.crt" \
|
||||||
|
-CAkey "${TLS_DIR}/client-ca.key" \
|
||||||
|
-CAcreateserial \
|
||||||
|
-out "${TLS_DIR}/client-good.crt" \
|
||||||
|
-days 3650 -sha256 \
|
||||||
|
-extfile "${tmp_dir}/client.ext"
|
||||||
|
info "created TLS client cert: ${TLS_DIR}/client-good.crt"
|
||||||
|
|
||||||
|
rm -f "${TLS_DIR}/client-ca.srl"
|
||||||
|
rm -rf "${tmp_dir}"
|
||||||
|
chmod 600 "${TLS_DIR}"/*.key
|
||||||
|
chmod 644 "${TLS_DIR}"/*.crt
|
||||||
|
}
|
||||||
|
|
||||||
|
generate_ssh_key() {
|
||||||
|
local path="$1"
|
||||||
|
local comment="$2"
|
||||||
|
check_overwrite "${path}"
|
||||||
|
check_overwrite "${path}.pub"
|
||||||
|
rm -f "${path}" "${path}.pub"
|
||||||
|
run_quiet ssh-keygen -q -t rsa -b 3072 -m PEM -N "" -C "${comment}" -f "${path}"
|
||||||
|
}
|
||||||
|
|
||||||
|
generate_ssh() {
|
||||||
|
need_cmd ssh-keygen
|
||||||
|
mkdir -p "${SSH_DIR}"
|
||||||
|
|
||||||
|
info "generating SSH keys in ${SSH_DIR}"
|
||||||
|
generate_ssh_key "${SSH_DIR}/ssh_host_rsa_key" "rpki-rtr-host"
|
||||||
|
info "created SSH host key: ${SSH_DIR}/ssh_host_rsa_key"
|
||||||
|
generate_ssh_key "${SSH_DIR}/rtr-client.key" "rpki-rtr-debug-client"
|
||||||
|
info "created SSH client key: ${SSH_DIR}/rtr-client.key"
|
||||||
|
|
||||||
|
check_overwrite "${SSH_DIR}/rtr-authorized_keys"
|
||||||
|
cp "${SSH_DIR}/rtr-client.key.pub" "${SSH_DIR}/rtr-authorized_keys"
|
||||||
|
info "created SSH authorized_keys: ${SSH_DIR}/rtr-authorized_keys"
|
||||||
|
|
||||||
|
chmod 600 \
|
||||||
|
"${SSH_DIR}/ssh_host_rsa_key" \
|
||||||
|
"${SSH_DIR}/rtr-client.key"
|
||||||
|
chmod 644 "${SSH_DIR}"/*.pub "${SSH_DIR}/rtr-authorized_keys"
|
||||||
|
}
|
||||||
|
|
||||||
|
while [[ $# -gt 0 ]]; do
|
||||||
|
case "$1" in
|
||||||
|
--out-dir)
|
||||||
|
[[ $# -ge 2 ]] || die "--out-dir requires a value"
|
||||||
|
OUT_DIR="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--server-dns)
|
||||||
|
[[ $# -ge 2 ]] || die "--server-dns requires a value"
|
||||||
|
SERVER_DNS="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--tls-only)
|
||||||
|
DO_TLS=1
|
||||||
|
DO_SSH=0
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--ssh-only)
|
||||||
|
DO_TLS=0
|
||||||
|
DO_SSH=1
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
--force)
|
||||||
|
FORCE=1
|
||||||
|
shift
|
||||||
|
;;
|
||||||
|
-h|--help)
|
||||||
|
usage
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
die "unknown option: $1"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
ensure_abs_out_dir
|
||||||
|
|
||||||
|
if [[ "${DO_TLS}" == "1" ]]; then
|
||||||
|
generate_tls
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "${DO_SSH}" == "1" ]]; then
|
||||||
|
generate_ssh
|
||||||
|
fi
|
||||||
|
|
||||||
|
info "done"
|
||||||
|
info "generated files under ${OUT_DIR}"
|
||||||
|
|
||||||
|
if [[ "${DO_TLS}" == "1" ]]; then
|
||||||
|
cat <<EOF
|
||||||
|
|
||||||
|
TLS files:
|
||||||
|
${TLS_DIR}/client-ca.crt
|
||||||
|
${TLS_DIR}/server-dns.crt
|
||||||
|
${TLS_DIR}/server-dns.key
|
||||||
|
${TLS_DIR}/client-good.crt
|
||||||
|
${TLS_DIR}/client-good.key
|
||||||
|
${TLS_DIR}/client-ca.key (local signing key; not mounted by compose)
|
||||||
|
EOF
|
||||||
|
fi
|
||||||
|
|
||||||
|
if [[ "${DO_SSH}" == "1" ]]; then
|
||||||
|
cat <<EOF
|
||||||
|
|
||||||
|
SSH files:
|
||||||
|
${SSH_DIR}/ssh_host_rsa_key
|
||||||
|
${SSH_DIR}/ssh_host_rsa_key.pub
|
||||||
|
${SSH_DIR}/rtr-client.key
|
||||||
|
${SSH_DIR}/rtr-client.key.pub
|
||||||
|
${SSH_DIR}/rtr-authorized_keys
|
||||||
|
EOF
|
||||||
|
fi
|
||||||
136
scripts/11_start_client.sh
Normal file
136
scripts/11_start_client.sh
Normal file
@@ -0,0 +1,136 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||||
|
|
||||||
|
COMMON_ENV_FILE="${ROOT_DIR}/environment.client"
|
||||||
|
CLIENT_ENV_FILE="${ROOT_DIR}/deploy/client/.env"
|
||||||
|
MODE="${RPKI_RTR_CLIENT_MODE:-tcp}"
|
||||||
|
COMPOSE_PROJECT="${RPKI_RTR_CLIENT_COMPOSE_PROJECT:-rpki-rtr-client}"
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage: scripts/11_start_client.sh [options]
|
||||||
|
|
||||||
|
Read root ./environment.client, update deploy/client/.env, then start the RTR debug client.
|
||||||
|
|
||||||
|
Options:
|
||||||
|
--mode MODE Client mode: tcp|tls|ssh. Default: tcp
|
||||||
|
--env FILE Client common env file. Default: ./environment.client
|
||||||
|
-h, --help Show this help.
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
log() {
|
||||||
|
printf '[11_start_client] %s\n' "$*" >&2
|
||||||
|
}
|
||||||
|
|
||||||
|
stage() {
|
||||||
|
printf '\n====== %s ======\n' "$*" >&2
|
||||||
|
}
|
||||||
|
|
||||||
|
die() {
|
||||||
|
printf '[11_start_client] ERROR: %s\n' "$*" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
set_env_var() {
|
||||||
|
local env_file="$1"
|
||||||
|
local key="$2"
|
||||||
|
local value="$3"
|
||||||
|
local tmp_file
|
||||||
|
|
||||||
|
tmp_file="${env_file}.tmp.$$"
|
||||||
|
awk -v key="${key}" -v value="${value}" '
|
||||||
|
BEGIN { done = 0 }
|
||||||
|
$0 ~ "^[[:space:]]*#?[[:space:]]*" key "=" {
|
||||||
|
if (!done) {
|
||||||
|
print key "=" value
|
||||||
|
done = 1
|
||||||
|
}
|
||||||
|
next
|
||||||
|
}
|
||||||
|
{ print }
|
||||||
|
END {
|
||||||
|
if (!done) {
|
||||||
|
print key "=" value
|
||||||
|
}
|
||||||
|
}
|
||||||
|
' "${env_file}" >"${tmp_file}"
|
||||||
|
mv "${tmp_file}" "${env_file}"
|
||||||
|
}
|
||||||
|
|
||||||
|
compose_file_for_mode() {
|
||||||
|
case "$1" in
|
||||||
|
tcp) printf '%s\n' "${ROOT_DIR}/deploy/client/docker-compose.tcp.yml" ;;
|
||||||
|
tls) printf '%s\n' "${ROOT_DIR}/deploy/client/docker-compose.tls.yml" ;;
|
||||||
|
ssh) printf '%s\n' "${ROOT_DIR}/deploy/client/docker-compose.ssh.yml" ;;
|
||||||
|
*) die "unsupported mode '$1', expected tcp|tls|ssh" ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
while [ "$#" -gt 0 ]; do
|
||||||
|
case "$1" in
|
||||||
|
--mode)
|
||||||
|
[ "$#" -ge 2 ] || die "--mode requires a value"
|
||||||
|
MODE="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
--env)
|
||||||
|
[ "$#" -ge 2 ] || die "--env requires a value"
|
||||||
|
COMMON_ENV_FILE="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
-h|--help)
|
||||||
|
usage
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
die "unknown option: $1"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
CLIENT_COMPOSE_FILE="$(compose_file_for_mode "${MODE}")"
|
||||||
|
|
||||||
|
stage "Checking Docker"
|
||||||
|
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
||||||
|
if docker compose version >/dev/null 2>&1; then
|
||||||
|
COMPOSE_BIN=(docker compose)
|
||||||
|
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
||||||
|
COMPOSE_BIN=(docker-compose)
|
||||||
|
else
|
||||||
|
die "docker compose is not available"
|
||||||
|
fi
|
||||||
|
|
||||||
|
[ -f "${CLIENT_ENV_FILE}" ] || die "deploy client env file not found: ${CLIENT_ENV_FILE}"
|
||||||
|
[ -f "${CLIENT_COMPOSE_FILE}" ] || die "client compose file not found: ${CLIENT_COMPOSE_FILE}"
|
||||||
|
|
||||||
|
stage "Reading Client Environment"
|
||||||
|
if [ -f "${COMMON_ENV_FILE}" ]; then
|
||||||
|
updated_count=0
|
||||||
|
while IFS= read -r line || [ -n "${line}" ]; do
|
||||||
|
case "${line}" in
|
||||||
|
''|\#*) continue ;;
|
||||||
|
*=*)
|
||||||
|
key="${line%%=*}"
|
||||||
|
value="${line#*=}"
|
||||||
|
set_env_var "${CLIENT_ENV_FILE}" "${key}" "${value}"
|
||||||
|
updated_count=$((updated_count + 1))
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done <"${COMMON_ENV_FILE}"
|
||||||
|
log "updated ${updated_count} client env variable(s)"
|
||||||
|
else
|
||||||
|
log "environment.client file not found, skip env update"
|
||||||
|
fi
|
||||||
|
|
||||||
|
stage "Starting RTR Debug Client"
|
||||||
|
log "mode: ${MODE}"
|
||||||
|
"${COMPOSE_BIN[@]}" --env-file "${CLIENT_ENV_FILE}" -p "${COMPOSE_PROJECT}" -f "${CLIENT_COMPOSE_FILE}" up -d --no-build rtr-debug-client
|
||||||
|
|
||||||
|
log "checking client compose status"
|
||||||
|
"${COMPOSE_BIN[@]}" --env-file "${CLIENT_ENV_FILE}" -p "${COMPOSE_PROJECT}" -f "${CLIENT_COMPOSE_FILE}" ps rtr-debug-client
|
||||||
|
|
||||||
|
stage "Client Started"
|
||||||
82
scripts/12_shutdown_client.sh
Normal file
82
scripts/12_shutdown_client.sh
Normal file
@@ -0,0 +1,82 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
|
||||||
|
ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
|
||||||
|
|
||||||
|
CLIENT_ENV_FILE="${ROOT_DIR}/deploy/client/.env"
|
||||||
|
MODE="${RPKI_RTR_CLIENT_MODE:-tcp}"
|
||||||
|
COMPOSE_PROJECT="${RPKI_RTR_CLIENT_COMPOSE_PROJECT:-rpki-rtr-client}"
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
Usage: scripts/12_shutdown_client.sh [options]
|
||||||
|
|
||||||
|
Stop the RTR debug client started by scripts/11_start_client.sh.
|
||||||
|
|
||||||
|
Options:
|
||||||
|
--mode MODE Client mode: tcp|tls|ssh. Default: tcp
|
||||||
|
-h, --help Show this help.
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
log() {
|
||||||
|
printf '[12_shutdown_client] %s\n' "$*" >&2
|
||||||
|
}
|
||||||
|
|
||||||
|
stage() {
|
||||||
|
printf '\n====== %s ======\n' "$*" >&2
|
||||||
|
}
|
||||||
|
|
||||||
|
die() {
|
||||||
|
printf '[12_shutdown_client] ERROR: %s\n' "$*" >&2
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
compose_file_for_mode() {
|
||||||
|
case "$1" in
|
||||||
|
tcp) printf '%s\n' "${ROOT_DIR}/deploy/client/docker-compose.tcp.yml" ;;
|
||||||
|
tls) printf '%s\n' "${ROOT_DIR}/deploy/client/docker-compose.tls.yml" ;;
|
||||||
|
ssh) printf '%s\n' "${ROOT_DIR}/deploy/client/docker-compose.ssh.yml" ;;
|
||||||
|
*) die "unsupported mode '$1', expected tcp|tls|ssh" ;;
|
||||||
|
esac
|
||||||
|
}
|
||||||
|
|
||||||
|
while [ "$#" -gt 0 ]; do
|
||||||
|
case "$1" in
|
||||||
|
--mode)
|
||||||
|
[ "$#" -ge 2 ] || die "--mode requires a value"
|
||||||
|
MODE="$2"
|
||||||
|
shift 2
|
||||||
|
;;
|
||||||
|
-h|--help)
|
||||||
|
usage
|
||||||
|
exit 0
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
die "unknown option: $1"
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
CLIENT_COMPOSE_FILE="$(compose_file_for_mode "${MODE}")"
|
||||||
|
|
||||||
|
stage "Checking Docker"
|
||||||
|
command -v docker >/dev/null 2>&1 || die "docker command not found"
|
||||||
|
if docker compose version >/dev/null 2>&1; then
|
||||||
|
COMPOSE_BIN=(docker compose)
|
||||||
|
elif command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then
|
||||||
|
COMPOSE_BIN=(docker-compose)
|
||||||
|
else
|
||||||
|
die "docker compose is not available"
|
||||||
|
fi
|
||||||
|
|
||||||
|
stage "Stopping RTR Debug Client"
|
||||||
|
log "mode: ${MODE}"
|
||||||
|
if [ -f "${CLIENT_ENV_FILE}" ] && [ -f "${CLIENT_COMPOSE_FILE}" ]; then
|
||||||
|
"${COMPOSE_BIN[@]}" --env-file "${CLIENT_ENV_FILE}" -p "${COMPOSE_PROJECT}" -f "${CLIENT_COMPOSE_FILE}" down --remove-orphans
|
||||||
|
else
|
||||||
|
log "client compose/env not found, skip"
|
||||||
|
fi
|
||||||
|
|
||||||
|
stage "Client Shutdown Finished"
|
||||||
218
scripts/README.md
Normal file
218
scripts/README.md
Normal file
@@ -0,0 +1,218 @@
|
|||||||
|
# 脚本说明
|
||||||
|
|
||||||
|
`scripts/` 目录用于离线安装、启动、健康检查、关闭和清理。
|
||||||
|
|
||||||
|
所有脚本建议在仓库根目录执行：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash scripts/00_0_install.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
## 目录
|
||||||
|
|
||||||
|
- [推荐流程](#推荐流程)
|
||||||
|
- [环境变量文件](#环境变量文件)
|
||||||
|
- [00 安装](#00-安装)
|
||||||
|
- [00 Pre-Check](#00-pre-check)
|
||||||
|
- [01 启动 Server 和 Console](#01-启动-server-和-console)
|
||||||
|
- [02 自检](#02-自检)
|
||||||
|
- [03 关闭 Server 和 Console](#03-关闭-server-和-console)
|
||||||
|
- [04 清理](#04-清理)
|
||||||
|
- [10 生成证书](#10-生成证书)
|
||||||
|
- [11 启动 Client](#11-启动-client)
|
||||||
|
- [12 关闭 Client](#12-关闭-client)
|
||||||
|
- [Windows 路径注意事项](#windows-路径注意事项)
|
||||||
|
|
||||||
|
## 推荐流程
|
||||||
|
|
||||||
|
```bash
|
||||||
|
# 1. 加载镜像并执行 pre-check。
|
||||||
|
bash scripts/00_0_install.sh
|
||||||
|
|
||||||
|
# 2. 启动真实 server 和 console。
|
||||||
|
bash scripts/01_start.sh --mode tcp
|
||||||
|
|
||||||
|
# 3. 检查 server 和 console 是否正常。
|
||||||
|
bash scripts/02_self_check.sh
|
||||||
|
|
||||||
|
# 4. 关闭 server 和 console。
|
||||||
|
bash scripts/03_shutdown.sh --mode tcp
|
||||||
|
|
||||||
|
# 5. 清理 server 运行产生的文件。
|
||||||
|
bash scripts/04_cleanup.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
client 单独启动和关闭：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash scripts/11_start_client.sh --mode tcp
|
||||||
|
bash scripts/12_shutdown_client.sh --mode tcp
|
||||||
|
```
|
||||||
|
|
||||||
|
支持的模式：
|
||||||
|
|
||||||
|
- server：`tcp`、`tls`、`ssh`、`base`
|
||||||
|
- client：`tcp`、`tls`、`ssh`
|
||||||
|
|
||||||
|
## 环境变量文件
|
||||||
|
|
||||||
|
`01_start.sh` 会读取根目录下的 `environment.server`。如果文件存在，会将其中所有 `KEY=VALUE` 同步到 `deploy/server/.env`。
|
||||||
|
|
||||||
|
示例：
|
||||||
|
|
||||||
|
```env
|
||||||
|
RPKI_RTR_CCR_HOST_DIR=/opt/rpki-rtr/data
|
||||||
|
RPKI_RTR_SLURM_HOST_DIR=/opt/rpki-rtr/data
|
||||||
|
RPKI_RTR_DB_HOST_DIR=/opt/rpki-rtr/rtr-db
|
||||||
|
RPKI_RTR_LOG_HOST_DIR=/opt/rpki-rtr/logs/server
|
||||||
|
RPKI_RTR_REPORT_HOST_DIR=/opt/rpki-rtr/report
|
||||||
|
```
|
||||||
|
|
||||||
|
`11_start_client.sh` 会读取根目录下的 `environment.client`。如果文件存在，会将其中所有 `KEY=VALUE` 同步到 `deploy/client/.env`。
|
||||||
|
|
||||||
|
如果 `environment.server` 或 `environment.client` 不存在，对应脚本会跳过更新，直接使用已有的 deploy `.env` 文件。
|
||||||
|
|
||||||
|
## 00 安装
|
||||||
|
|
||||||
|
`00_0_install.sh` 用于从 `images/` 目录加载离线 Docker 镜像，更新 deploy `.env` 中的镜像名，然后执行 pre-check。
|
||||||
|
|
||||||
|
默认镜像包查找规则：
|
||||||
|
|
||||||
|
- server：`images/rpki-rtr-*.tar`，排除 `rpki-rtr-debug-client-*.tar`
|
||||||
|
- client：`images/rpki-rtr-debug-client-*.tar`
|
||||||
|
- console：`images/rpki-console-ui-*.tar`
|
||||||
|
|
||||||
|
也可以显式指定镜像包：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash scripts/00_0_install.sh \
|
||||||
|
--server-tar images/rpki-rtr-amd64-v1.2.tar \
|
||||||
|
--client-tar images/rpki-rtr-debug-client-amd64-v1.2.tar \
|
||||||
|
--console-tar images/rpki-console-ui-amd64-v1.0.tar
|
||||||
|
```
|
||||||
|
|
||||||
|
脚本会从 `docker load` 输出中解析真实镜像名，并写入：
|
||||||
|
|
||||||
|
- `deploy/server/.env` 的 `RPKI_RTR_SERVER_IMAGE`
|
||||||
|
- `deploy/console/.env` 的 `RPKI_CONSOLE_UI_IMAGE`
|
||||||
|
|
||||||
|
## 00 Pre-Check
|
||||||
|
|
||||||
|
`00_1_pre_check.sh` 会在 `.precheck/` 下准备隔离测试数据，使用仓库里的 Docker Compose 文件启动 server 和 debug client，然后检查：
|
||||||
|
|
||||||
|
- admin health 接口返回正常。
|
||||||
|
- server 写出了 source report。
|
||||||
|
- debug client 收到了 EndOfData 和 payload summary。
|
||||||
|
- pre-check 日志中没有 fatal 错误模式。
|
||||||
|
|
||||||
|
相关辅助脚本：
|
||||||
|
|
||||||
|
- `00_2_generate_ccr.sh`：生成固定的 CCR 测试文件。
|
||||||
|
- `00_2_pre_check_prepare.sh`：创建 `.precheck/` 数据、SLURM 文件和 env 文件。
|
||||||
|
- `00_3_pre_check_cleanup.sh`：停止 pre-check 容器并清理 `.precheck/`。
|
||||||
|
|
||||||
|
## 01 启动 Server 和 Console
|
||||||
|
|
||||||
|
`01_start.sh` 用于启动真实的 RTR server 和 console。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash scripts/01_start.sh --mode tcp
|
||||||
|
```
|
||||||
|
|
||||||
|
启动后脚本会检查容器是否进入 `running` 状态。如果容器启动后退出，脚本会打印 compose 状态和最近日志，然后报错退出，不会继续打印 `Start Finished`。
|
||||||
|
|
||||||
|
常用参数：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash scripts/01_start.sh --mode tls
|
||||||
|
bash scripts/01_start.sh --mode ssh
|
||||||
|
bash scripts/01_start.sh --wait 60
|
||||||
|
bash scripts/01_start.sh --env ./environment.server
|
||||||
|
```
|
||||||
|
|
||||||
|
注意：console 不读取 `environment.server`，只使用 `deploy/console/.env`。
|
||||||
|
|
||||||
|
## 02 自检
|
||||||
|
|
||||||
|
`02_self_check.sh` 用于检查 `01_start.sh` 启动的 server 和 console 是否正常。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash scripts/02_self_check.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
检查内容包括：
|
||||||
|
|
||||||
|
- server 容器是否 running。
|
||||||
|
- console 容器是否 running。
|
||||||
|
- RTR TCP 端口是否可连接。
|
||||||
|
- admin health 接口是否健康。
|
||||||
|
- console 端口和 HTTP 根路径是否可访问。
|
||||||
|
|
||||||
|
如果启动较慢，可以增加等待时间：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash scripts/02_self_check.sh --wait 120
|
||||||
|
```
|
||||||
|
|
||||||
|
## 03 关闭 Server 和 Console
|
||||||
|
|
||||||
|
`03_shutdown.sh` 用于关闭 `01_start.sh` 启动的服务。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash scripts/03_shutdown.sh --mode tcp
|
||||||
|
```
|
||||||
|
|
||||||
|
脚本会先关闭 console，再关闭 RTR server。
|
||||||
|
|
||||||
|
## 04 清理
|
||||||
|
|
||||||
|
`04_cleanup.sh` 用于删除 server 运行产生的目录：
|
||||||
|
|
||||||
|
- `logs/`
|
||||||
|
- `report/`
|
||||||
|
- `rtr-db/`
|
||||||
|
|
||||||
|
推荐先关闭服务，再清理：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash scripts/03_shutdown.sh
|
||||||
|
bash scripts/04_cleanup.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
清理前脚本会检查 `rpki-rtr` 是否仍在运行。如果仍在运行，会拒绝清理。只有明确需要强制清理时才使用：
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash scripts/04_cleanup.sh --force
|
||||||
|
```
|
||||||
|
|
||||||
|
清理脚本会拒绝删除仓库根目录之外的路径。如果某一项清理失败，会继续清理后续项，最后返回失败。
|
||||||
|
|
||||||
|
## 10 生成证书
|
||||||
|
|
||||||
|
`10_generate_certs.sh` 用于生成 TLS/SSH 模式需要的本地证书材料（测试用）。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash scripts/10_generate_certs.sh
|
||||||
|
```
|
||||||
|
|
||||||
|
## 11 启动 Client
|
||||||
|
|
||||||
|
`11_start_client.sh` 用于启动 RTR debug client。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash scripts/11_start_client.sh --mode tcp
|
||||||
|
bash scripts/11_start_client.sh --mode tls
|
||||||
|
bash scripts/11_start_client.sh --mode ssh
|
||||||
|
```
|
||||||
|
|
||||||
|
如果 `environment.client` 存在，脚本会将其中所有 `KEY=VALUE` 同步到 `deploy/client/.env`；如果不存在，则直接使用已有的 `deploy/client/.env`。
|
||||||
|
|
||||||
|
## 12 关闭 Client
|
||||||
|
|
||||||
|
`12_shutdown_client.sh` 用于关闭 `11_start_client.sh` 启动的 client。
|
||||||
|
|
||||||
|
```bash
|
||||||
|
bash scripts/12_shutdown_client.sh --mode tcp
|
||||||
|
```
|
||||||
|
|
||||||
|
关闭时建议使用和启动时相同的 mode。
|
||||||
@@ -1,70 +0,0 @@
|
|||||||
# RPKI Benchmarks (Stage2, selected_der_v2)
|
|
||||||
|
|
||||||
This directory contains a reproducible, one-click benchmark to measure **decode + profile validate**
|
|
||||||
performance for all supported object types and compare **OURS** against the **Routinator baseline**
|
|
||||||
(`rpki` crate `=0.19.1` with `repository` feature).
|
|
||||||
|
|
||||||
## What it measures
|
|
||||||
|
|
||||||
Dataset:
|
|
||||||
|
|
||||||
- Fixtures: `rpki/tests/benchmark/selected_der_v2/`
|
|
||||||
- Objects: `cer`, `crl`, `manifest` (`.mft`), `roa`, `aspa` (`.asa`)
|
|
||||||
- Samples: 10 quantiles per type (`min/p01/p10/p25/p50/p75/p90/p95/p99/max`) → 50 files total
|
|
||||||
|
|
||||||
Metrics:
|
|
||||||
|
|
||||||
- **decode+validate**: `decode_der` (parse + profile validate) for each object file
|
|
||||||
- **landing** (OURS only): `PackFile::from_bytes_compute_sha256` + CBOR encode + `RocksDB put_raw`
|
|
||||||
- **compare**: ratio `ours_ns/op ÷ rout_ns/op` for decode+validate
|
|
||||||
|
|
||||||
## Default benchmark settings
|
|
||||||
|
|
||||||
Both OURS and Routinator baseline use the same run settings:
|
|
||||||
|
|
||||||
- warmup: `10` iterations
|
|
||||||
- rounds: `3`
|
|
||||||
- adaptive loop target: `min_round_ms=200` (with an internal max of `1_000_000` iters)
|
|
||||||
- strict DER: `true` (baseline)
|
|
||||||
- cert inspect: `false` (baseline)
|
|
||||||
|
|
||||||
You can override the settings via environment variables in the runner script:
|
|
||||||
|
|
||||||
- `BENCH_WARMUP_ITERS` (default `10`)
|
|
||||||
- `BENCH_ROUNDS` (default `3`)
|
|
||||||
- `BENCH_MIN_ROUND_MS` (default `200`)
|
|
||||||
|
|
||||||
## One-click run (OURS + Routinator compare)
|
|
||||||
|
|
||||||
From the `rpki/` crate directory:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./scripts/benchmark/run_stage2_selected_der_v2_release.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
Outputs are written under:
|
|
||||||
|
|
||||||
- `rpki/target/bench/`
|
|
||||||
- OURS decode+validate: `stage2_selected_der_v2_decode_release_<TS>.{md,csv}`
|
|
||||||
- OURS landing: `stage2_selected_der_v2_landing_release_<TS>.{md,csv}`
|
|
||||||
- Routinator: `stage2_selected_der_v2_routinator_decode_release_<TS>.{md,csv}`
|
|
||||||
- Compare: `stage2_selected_der_v2_compare_ours_vs_routinator_decode_release_<TS>.{md,csv}`
|
|
||||||
- Summary: `stage2_selected_der_v2_compare_summary_<TS>.md`
|
|
||||||
|
|
||||||
### Why decode and landing are separated
|
|
||||||
|
|
||||||
The underlying benchmark can run in `BENCH_MODE=both`, but the **landing** part writes to RocksDB
|
|
||||||
and may trigger background work (e.g., compactions) that can **skew subsequent decode timings**.
|
|
||||||
For a fair OURS-vs-Routinator comparison, the runner script:
|
|
||||||
|
|
||||||
- runs `BENCH_MODE=decode_validate` for comparison, and
|
|
||||||
- runs `BENCH_MODE=landing` separately for landing-only numbers.
|
|
||||||
|
|
||||||
## Notes
|
|
||||||
|
|
||||||
- The Routinator baseline benchmark is implemented in-repo under:
|
|
||||||
- `rpki/benchmark/routinator_object_bench/`
|
|
||||||
- It pins `rpki = "=0.19.1"` in its `Cargo.toml`.
|
|
||||||
- This benchmark is implemented as an `#[ignore]` integration test:
|
|
||||||
- `rpki/tests/bench_stage2_decode_profile_selected_der_v2.rs`
|
|
||||||
- The runner script invokes it with `cargo test --release ... -- --ignored --nocapture`.
|
|
||||||
@@ -1,123 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
# Stage2 (selected_der_v2) decode+profile validate benchmark.
|
|
||||||
# Runs:
|
|
||||||
# 1) OURS decode+validate benchmark and writes MD/CSV.
|
|
||||||
# 2) OURS landing benchmark and writes MD/CSV.
|
|
||||||
# 3) Routinator baseline decode benchmark (rpki crate =0.19.1).
|
|
||||||
# 4) Produces a joined compare CSV/MD and a short geomean summary.
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
cd "$ROOT_DIR"
|
|
||||||
|
|
||||||
OUT_DIR="$ROOT_DIR/target/bench"
|
|
||||||
mkdir -p "$OUT_DIR"
|
|
||||||
|
|
||||||
TS="$(date -u +%Y%m%dT%H%M%SZ)"
|
|
||||||
|
|
||||||
WARMUP_ITERS="${BENCH_WARMUP_ITERS:-10}"
|
|
||||||
ROUNDS="${BENCH_ROUNDS:-3}"
|
|
||||||
MIN_ROUND_MS="${BENCH_MIN_ROUND_MS:-200}"
|
|
||||||
|
|
||||||
OURS_MD="$OUT_DIR/stage2_selected_der_v2_decode_release_${TS}.md"
|
|
||||||
OURS_CSV="$OUT_DIR/stage2_selected_der_v2_decode_release_${TS}.csv"
|
|
||||||
|
|
||||||
OURS_LANDING_MD="$OUT_DIR/stage2_selected_der_v2_landing_release_${TS}.md"
|
|
||||||
OURS_LANDING_CSV="$OUT_DIR/stage2_selected_der_v2_landing_release_${TS}.csv"
|
|
||||||
|
|
||||||
ROUT_MD="$OUT_DIR/stage2_selected_der_v2_routinator_decode_release_${TS}.md"
|
|
||||||
ROUT_CSV="$OUT_DIR/stage2_selected_der_v2_routinator_decode_release_${TS}.csv"
|
|
||||||
|
|
||||||
COMPARE_MD="$OUT_DIR/stage2_selected_der_v2_compare_ours_vs_routinator_decode_release_${TS}.md"
|
|
||||||
COMPARE_CSV="$OUT_DIR/stage2_selected_der_v2_compare_ours_vs_routinator_decode_release_${TS}.csv"
|
|
||||||
|
|
||||||
SUMMARY_MD="$OUT_DIR/stage2_selected_der_v2_compare_summary_${TS}.md"
|
|
||||||
|
|
||||||
echo "[1/4] OURS: decode+validate benchmark (release)..." >&2
|
|
||||||
BENCH_MODE="decode_validate" \
|
|
||||||
BENCH_WARMUP_ITERS="$WARMUP_ITERS" \
|
|
||||||
BENCH_ROUNDS="$ROUNDS" \
|
|
||||||
BENCH_MIN_ROUND_MS="$MIN_ROUND_MS" \
|
|
||||||
BENCH_OUT_MD="$OURS_MD" \
|
|
||||||
BENCH_OUT_CSV="$OURS_CSV" \
|
|
||||||
cargo test --release --test bench_stage2_decode_profile_selected_der_v2 -- --ignored --nocapture >/dev/null
|
|
||||||
|
|
||||||
echo "[2/4] OURS: landing benchmark (release)..." >&2
|
|
||||||
BENCH_MODE="landing" \
|
|
||||||
BENCH_WARMUP_ITERS="$WARMUP_ITERS" \
|
|
||||||
BENCH_ROUNDS="$ROUNDS" \
|
|
||||||
BENCH_MIN_ROUND_MS="$MIN_ROUND_MS" \
|
|
||||||
BENCH_OUT_MD_LANDING="$OURS_LANDING_MD" \
|
|
||||||
BENCH_OUT_CSV_LANDING="$OURS_LANDING_CSV" \
|
|
||||||
cargo test --release --test bench_stage2_decode_profile_selected_der_v2 -- --ignored --nocapture >/dev/null
|
|
||||||
|
|
||||||
echo "[3/4] Routinator baseline + compare join..." >&2
|
|
||||||
OURS_CSV="$OURS_CSV" \
|
|
||||||
ROUT_CSV="$ROUT_CSV" \
|
|
||||||
ROUT_MD="$ROUT_MD" \
|
|
||||||
COMPARE_CSV="$COMPARE_CSV" \
|
|
||||||
COMPARE_MD="$COMPARE_MD" \
|
|
||||||
WARMUP_ITERS="$WARMUP_ITERS" \
|
|
||||||
ROUNDS="$ROUNDS" \
|
|
||||||
MIN_ROUND_MS="$MIN_ROUND_MS" \
|
|
||||||
scripts/stage2_perf_compare_m4.sh >/dev/null
|
|
||||||
|
|
||||||
echo "[4/4] Summary (geomean ratios)..." >&2
|
|
||||||
python3 - "$COMPARE_CSV" "$SUMMARY_MD" <<'PY'
|
|
||||||
import csv
|
|
||||||
import math
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
|
|
||||||
in_csv = Path(sys.argv[1])
|
|
||||||
out_md = Path(sys.argv[2])
|
|
||||||
|
|
||||||
rows = list(csv.DictReader(in_csv.open(newline="")))
|
|
||||||
ratios = {}
|
|
||||||
for r in rows:
|
|
||||||
ratios.setdefault(r["type"], []).append(float(r["ratio_ours_over_rout"]))
|
|
||||||
|
|
||||||
def geomean(vals):
|
|
||||||
return math.exp(sum(math.log(v) for v in vals) / len(vals))
|
|
||||||
|
|
||||||
def p50(vals):
|
|
||||||
v = sorted(vals)
|
|
||||||
n = len(v)
|
|
||||||
if n % 2 == 1:
|
|
||||||
return v[n // 2]
|
|
||||||
return (v[n // 2 - 1] + v[n // 2]) / 2.0
|
|
||||||
|
|
||||||
all_vals = [float(r["ratio_ours_over_rout"]) for r in rows]
|
|
||||||
types = ["all"] + sorted(ratios.keys())
|
|
||||||
|
|
||||||
now = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
|
||||||
lines = []
|
|
||||||
lines.append("# Stage2 selected_der_v2 compare summary (release)\n\n")
|
|
||||||
lines.append(f"- recorded_at_utc: `{now}`\n")
|
|
||||||
lines.append(f"- inputs_csv: `{in_csv}`\n\n")
|
|
||||||
lines.append("| type | n | min | p50 | geomean | max | >1 count |\n")
|
|
||||||
lines.append("|---|---:|---:|---:|---:|---:|---:|\n")
|
|
||||||
|
|
||||||
for t in types:
|
|
||||||
vals = all_vals if t == "all" else ratios[t]
|
|
||||||
vals_sorted = sorted(vals)
|
|
||||||
lines.append(
|
|
||||||
f"| {t} | {len(vals_sorted)} | {vals_sorted[0]:.4f} | {p50(vals_sorted):.4f} | "
|
|
||||||
f"{geomean(vals_sorted):.4f} | {vals_sorted[-1]:.4f} | {sum(1 for v in vals_sorted if v>1.0)} |\n"
|
|
||||||
)
|
|
||||||
|
|
||||||
out_md.write_text("".join(lines), encoding="utf-8")
|
|
||||||
print(out_md)
|
|
||||||
PY
|
|
||||||
|
|
||||||
echo "Done." >&2
|
|
||||||
echo "- OURS decode MD: $OURS_MD" >&2
|
|
||||||
echo "- OURS decode CSV: $OURS_CSV" >&2
|
|
||||||
echo "- OURS landing MD: $OURS_LANDING_MD" >&2
|
|
||||||
echo "- OURS landing CSV: $OURS_LANDING_CSV" >&2
|
|
||||||
echo "- Routinator: $ROUT_MD" >&2
|
|
||||||
echo "- Compare MD: $COMPARE_MD" >&2
|
|
||||||
echo "- Compare CSV: $COMPARE_CSV" >&2
|
|
||||||
echo "- Summary MD: $SUMMARY_MD" >&2
|
|
||||||
@@ -1,56 +0,0 @@
|
|||||||
# CIR Scripts
|
|
||||||
|
|
||||||
## `cir-rsync-wrapper`
|
|
||||||
|
|
||||||
一个用于 CIR 黑盒 replay 的 rsync wrapper。
|
|
||||||
|
|
||||||
### 环境变量
|
|
||||||
|
|
||||||
- `REAL_RSYNC_BIN`
|
|
||||||
- 真实 rsync 二进制路径
|
|
||||||
- 默认优先 `/usr/bin/rsync`
|
|
||||||
- `CIR_MIRROR_ROOT`
|
|
||||||
- 本地镜像树根目录
|
|
||||||
- 当命令行中出现 `rsync://...` source 时必需
|
|
||||||
|
|
||||||
### 语义
|
|
||||||
|
|
||||||
- 仅改写 `rsync://host/path` 类型参数
|
|
||||||
- 其它参数原样透传给真实 rsync
|
|
||||||
- 改写目标：
|
|
||||||
- `rsync://example.net/repo/a.roa`
|
|
||||||
- →
|
|
||||||
- `<CIR_MIRROR_ROOT>/example.net/repo/a.roa`
|
|
||||||
|
|
||||||
### 兼容目标
|
|
||||||
|
|
||||||
- Routinator `--rsync-command`
|
|
||||||
- `rpki-client -e rsync_prog`
|
|
||||||
|
|
||||||
## 其它脚本
|
|
||||||
|
|
||||||
- `run_cir_replay_ours.sh`
|
|
||||||
- `run_cir_replay_routinator.sh`
|
|
||||||
- `run_cir_replay_rpki_client.sh`
|
|
||||||
- `run_cir_replay_matrix.sh`
|
|
||||||
|
|
||||||
## `cir-local-link-sync.py`
|
|
||||||
|
|
||||||
当 `CIR_LOCAL_LINK_MODE=1` 且 wrapper 检测到 source 已经被改写为本地 mirror 路径时，
|
|
||||||
wrapper 不再调用真实 `rsync`，而是调用这个 helper 完成：
|
|
||||||
|
|
||||||
- `hardlink` 优先的本地树同步
|
|
||||||
- 失败时回退到 copy
|
|
||||||
- 支持 `--delete`
|
|
||||||
|
|
||||||
`run_cir_replay_matrix.sh` 会顺序执行：
|
|
||||||
|
|
||||||
- `ours`
|
|
||||||
- Routinator
|
|
||||||
- `rpki-client`
|
|
||||||
|
|
||||||
并汇总生成：
|
|
||||||
|
|
||||||
- `summary.json`
|
|
||||||
- `summary.md`
|
|
||||||
- `detail.md`
|
|
||||||
Binary file not shown.
Binary file not shown.
Binary file not shown.
@@ -1,136 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
import argparse
|
|
||||||
import errno
|
|
||||||
import os
|
|
||||||
import shutil
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
|
|
||||||
def _same_inode(src: Path, dst: Path) -> bool:
|
|
||||||
try:
|
|
||||||
src_stat = src.stat()
|
|
||||||
dst_stat = dst.stat()
|
|
||||||
except FileNotFoundError:
|
|
||||||
return False
|
|
||||||
return (src_stat.st_dev, src_stat.st_ino) == (dst_stat.st_dev, dst_stat.st_ino)
|
|
||||||
|
|
||||||
|
|
||||||
def _remove_path(path: Path) -> None:
|
|
||||||
if not path.exists() and not path.is_symlink():
|
|
||||||
return
|
|
||||||
if path.is_dir() and not path.is_symlink():
|
|
||||||
shutil.rmtree(path)
|
|
||||||
else:
|
|
||||||
path.unlink()
|
|
||||||
|
|
||||||
|
|
||||||
def _prune_empty_dirs(root: Path) -> None:
|
|
||||||
if not root.exists():
|
|
||||||
return
|
|
||||||
for path in sorted((p for p in root.rglob("*") if p.is_dir()), key=lambda p: len(p.parts), reverse=True):
|
|
||||||
try:
|
|
||||||
path.rmdir()
|
|
||||||
except OSError:
|
|
||||||
pass
|
|
||||||
|
|
||||||
|
|
||||||
def _link_or_copy(src: Path, dst: Path) -> str:
|
|
||||||
dst.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
if dst.exists() or dst.is_symlink():
|
|
||||||
if _same_inode(src, dst):
|
|
||||||
return "reused"
|
|
||||||
_remove_path(dst)
|
|
||||||
try:
|
|
||||||
os.link(src, dst)
|
|
||||||
return "linked"
|
|
||||||
except OSError as err:
|
|
||||||
if err.errno not in (errno.EXDEV, errno.EPERM, errno.EMLINK, errno.ENOTSUP, errno.EACCES):
|
|
||||||
raise
|
|
||||||
shutil.copy2(src, dst)
|
|
||||||
return "copied"
|
|
||||||
|
|
||||||
|
|
||||||
def _file_map(src_arg: str, dest_arg: str) -> tuple[Path, dict[str, Path]]:
|
|
||||||
src = Path(src_arg.rstrip(os.sep))
|
|
||||||
if not src.exists():
|
|
||||||
raise FileNotFoundError(src)
|
|
||||||
mapping: dict[str, Path] = {}
|
|
||||||
if src.is_dir():
|
|
||||||
copy_contents = src_arg.endswith(os.sep)
|
|
||||||
if copy_contents:
|
|
||||||
root = src
|
|
||||||
for path in root.rglob("*"):
|
|
||||||
if path.is_file():
|
|
||||||
mapping[path.relative_to(root).as_posix()] = path
|
|
||||||
else:
|
|
||||||
root = src
|
|
||||||
base = src.name
|
|
||||||
for path in root.rglob("*"):
|
|
||||||
if path.is_file():
|
|
||||||
rel = Path(base) / path.relative_to(root)
|
|
||||||
mapping[rel.as_posix()] = path
|
|
||||||
else:
|
|
||||||
dest_path = Path(dest_arg)
|
|
||||||
if dest_arg.endswith(os.sep) or dest_path.is_dir():
|
|
||||||
mapping[src.name] = src
|
|
||||||
else:
|
|
||||||
mapping[dest_path.name] = src
|
|
||||||
return Path(dest_arg), mapping
|
|
||||||
|
|
||||||
|
|
||||||
def sync_local_tree(src_arg: str, dst_arg: str, delete: bool) -> dict[str, int]:
|
|
||||||
dst_root, mapping = _file_map(src_arg, dst_arg)
|
|
||||||
dst_root.mkdir(parents=True, exist_ok=True)
|
|
||||||
|
|
||||||
expected = {dst_root / rel for rel in mapping.keys()}
|
|
||||||
|
|
||||||
deleted = 0
|
|
||||||
if delete and dst_root.exists():
|
|
||||||
for path in sorted(dst_root.rglob("*"), key=lambda p: len(p.parts), reverse=True):
|
|
||||||
if path.is_dir():
|
|
||||||
continue
|
|
||||||
if path not in expected:
|
|
||||||
_remove_path(path)
|
|
||||||
deleted += 1
|
|
||||||
_prune_empty_dirs(dst_root)
|
|
||||||
|
|
||||||
linked = 0
|
|
||||||
copied = 0
|
|
||||||
reused = 0
|
|
||||||
for rel, src in mapping.items():
|
|
||||||
dst = dst_root / rel
|
|
||||||
result = _link_or_copy(src, dst)
|
|
||||||
if result == "linked":
|
|
||||||
linked += 1
|
|
||||||
elif result == "copied":
|
|
||||||
copied += 1
|
|
||||||
else:
|
|
||||||
reused += 1
|
|
||||||
|
|
||||||
return {
|
|
||||||
"files": len(mapping),
|
|
||||||
"linked": linked,
|
|
||||||
"copied": copied,
|
|
||||||
"reused": reused,
|
|
||||||
"deleted": deleted,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
|
||||||
parser = argparse.ArgumentParser(description="Sync a local CIR mirror tree using hardlinks when possible.")
|
|
||||||
parser.add_argument("--delete", action="store_true", help="Delete target files not present in source")
|
|
||||||
parser.add_argument("source")
|
|
||||||
parser.add_argument("dest")
|
|
||||||
args = parser.parse_args()
|
|
||||||
|
|
||||||
summary = sync_local_tree(args.source, args.dest, args.delete)
|
|
||||||
print(
|
|
||||||
"local-link-sync files={files} linked={linked} copied={copied} reused={reused} deleted={deleted}".format(
|
|
||||||
**summary
|
|
||||||
)
|
|
||||||
)
|
|
||||||
return 0
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
raise SystemExit(main())
|
|
||||||
@@ -1,127 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
import os
|
|
||||||
import shutil
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
from urllib.parse import urlparse
|
|
||||||
|
|
||||||
|
|
||||||
def real_rsync_bin() -> str:
|
|
||||||
env = os.environ.get("REAL_RSYNC_BIN")
|
|
||||||
if env:
|
|
||||||
return env
|
|
||||||
default = "/usr/bin/rsync"
|
|
||||||
if Path(default).exists():
|
|
||||||
return default
|
|
||||||
found = shutil.which("rsync")
|
|
||||||
if found:
|
|
||||||
return found
|
|
||||||
raise SystemExit("cir-rsync-wrapper: REAL_RSYNC_BIN is not set and rsync was not found")
|
|
||||||
|
|
||||||
|
|
||||||
def rewrite_arg(arg: str, mirror_root: str | None) -> str:
|
|
||||||
if not arg.startswith("rsync://"):
|
|
||||||
return arg
|
|
||||||
if not mirror_root:
|
|
||||||
raise SystemExit(
|
|
||||||
"cir-rsync-wrapper: CIR_MIRROR_ROOT is required when an rsync:// source is present"
|
|
||||||
)
|
|
||||||
parsed = urlparse(arg)
|
|
||||||
if parsed.scheme != "rsync" or not parsed.hostname:
|
|
||||||
raise SystemExit(f"cir-rsync-wrapper: invalid rsync URI: {arg}")
|
|
||||||
path = parsed.path.lstrip("/")
|
|
||||||
local = Path(mirror_root).resolve() / parsed.hostname
|
|
||||||
if path:
|
|
||||||
local = local / path
|
|
||||||
local_str = str(local)
|
|
||||||
if local.exists() and local.is_dir() and not local_str.endswith("/"):
|
|
||||||
local_str += "/"
|
|
||||||
elif arg.endswith("/") and not local_str.endswith("/"):
|
|
||||||
local_str += "/"
|
|
||||||
return local_str
|
|
||||||
|
|
||||||
|
|
||||||
def filter_args(args: list[str]) -> list[str]:
|
|
||||||
mirror_root = os.environ.get("CIR_MIRROR_ROOT")
|
|
||||||
rewritten_any = any(arg.startswith("rsync://") for arg in args)
|
|
||||||
out: list[str] = []
|
|
||||||
i = 0
|
|
||||||
while i < len(args):
|
|
||||||
arg = args[i]
|
|
||||||
if rewritten_any:
|
|
||||||
if arg == "--address":
|
|
||||||
i += 2
|
|
||||||
continue
|
|
||||||
if arg.startswith("--address="):
|
|
||||||
i += 1
|
|
||||||
continue
|
|
||||||
if arg == "--contimeout":
|
|
||||||
i += 2
|
|
||||||
continue
|
|
||||||
if arg.startswith("--contimeout="):
|
|
||||||
i += 1
|
|
||||||
continue
|
|
||||||
out.append(rewrite_arg(arg, mirror_root))
|
|
||||||
i += 1
|
|
||||||
return out
|
|
||||||
|
|
||||||
|
|
||||||
def local_link_mode_enabled() -> bool:
|
|
||||||
value = os.environ.get("CIR_LOCAL_LINK_MODE", "")
|
|
||||||
return value.lower() in {"1", "true", "yes", "on"}
|
|
||||||
|
|
||||||
|
|
||||||
def extract_source_and_dest(args: list[str]) -> tuple[str, str]:
|
|
||||||
expects_value = {
|
|
||||||
"--timeout",
|
|
||||||
"--min-size",
|
|
||||||
"--max-size",
|
|
||||||
"--include",
|
|
||||||
"--exclude",
|
|
||||||
"--compare-dest",
|
|
||||||
}
|
|
||||||
positionals: list[str] = []
|
|
||||||
i = 0
|
|
||||||
while i < len(args):
|
|
||||||
arg = args[i]
|
|
||||||
if arg in expects_value:
|
|
||||||
i += 2
|
|
||||||
continue
|
|
||||||
if any(arg.startswith(prefix + "=") for prefix in expects_value):
|
|
||||||
i += 1
|
|
||||||
continue
|
|
||||||
if arg.startswith("-"):
|
|
||||||
i += 1
|
|
||||||
continue
|
|
||||||
positionals.append(arg)
|
|
||||||
i += 1
|
|
||||||
if len(positionals) < 2:
|
|
||||||
raise SystemExit("cir-rsync-wrapper: expected source and destination arguments")
|
|
||||||
return positionals[-2], positionals[-1]
|
|
||||||
|
|
||||||
|
|
||||||
def maybe_exec_local_link_sync(args: list[str], rewritten_any: bool) -> None:
|
|
||||||
if not rewritten_any or not local_link_mode_enabled():
|
|
||||||
return
|
|
||||||
source, dest = extract_source_and_dest(args)
|
|
||||||
if source.startswith("rsync://"):
|
|
||||||
raise SystemExit("cir-rsync-wrapper: expected rewritten local source for CIR_LOCAL_LINK_MODE")
|
|
||||||
helper = Path(__file__).with_name("cir-local-link-sync.py")
|
|
||||||
cmd = [sys.executable, str(helper)]
|
|
||||||
if "--delete" in args:
|
|
||||||
cmd.append("--delete")
|
|
||||||
cmd.extend([source, dest])
|
|
||||||
os.execv(sys.executable, cmd)
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
|
||||||
args = sys.argv[1:]
|
|
||||||
rewritten_any = any(arg.startswith("rsync://") for arg in args)
|
|
||||||
rewritten = filter_args(args)
|
|
||||||
maybe_exec_local_link_sync(rewritten, rewritten_any)
|
|
||||||
os.execv(real_rsync_bin(), [real_rsync_bin(), *rewritten])
|
|
||||||
return 127
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
raise SystemExit(main())
|
|
||||||
@@ -1,32 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage:
|
|
||||||
./scripts/cir/fetch_cir_sequence_from_remote.sh \
|
|
||||||
--ssh-target <user@host> \
|
|
||||||
--remote-path <path> \
|
|
||||||
--local-path <path>
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
SSH_TARGET=""
|
|
||||||
REMOTE_PATH=""
|
|
||||||
LOCAL_PATH=""
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--ssh-target) SSH_TARGET="$2"; shift 2 ;;
|
|
||||||
--remote-path) REMOTE_PATH="$2"; shift 2 ;;
|
|
||||||
--local-path) LOCAL_PATH="$2"; shift 2 ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
[[ -n "$SSH_TARGET" && -n "$REMOTE_PATH" && -n "$LOCAL_PATH" ]] || { usage >&2; exit 2; }
|
|
||||||
|
|
||||||
mkdir -p "$(dirname "$LOCAL_PATH")"
|
|
||||||
rsync -a "$SSH_TARGET:$REMOTE_PATH/" "$LOCAL_PATH/"
|
|
||||||
echo "done: $LOCAL_PATH"
|
|
||||||
@@ -1,50 +0,0 @@
|
|||||||
#!/usr/bin/env python3
|
|
||||||
|
|
||||||
from __future__ import annotations
|
|
||||||
|
|
||||||
import argparse
|
|
||||||
import csv
|
|
||||||
import json
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
|
|
||||||
def normalize_asn(value: str | int) -> str:
|
|
||||||
text = str(value).strip().upper()
|
|
||||||
if text.startswith("AS"):
|
|
||||||
text = text[2:]
|
|
||||||
return f"AS{int(text)}"
|
|
||||||
|
|
||||||
|
|
||||||
def main() -> int:
|
|
||||||
parser = argparse.ArgumentParser()
|
|
||||||
parser.add_argument("--input", required=True, type=Path)
|
|
||||||
parser.add_argument("--csv-out", required=True, type=Path)
|
|
||||||
args = parser.parse_args()
|
|
||||||
|
|
||||||
obj = json.loads(args.input.read_text(encoding="utf-8"))
|
|
||||||
rows: list[tuple[str, str, str]] = []
|
|
||||||
for aspa in obj.get("aspas", []):
|
|
||||||
providers = sorted(
|
|
||||||
{normalize_asn(item) for item in aspa.get("providers", [])},
|
|
||||||
key=lambda s: int(s[2:]),
|
|
||||||
)
|
|
||||||
rows.append(
|
|
||||||
(
|
|
||||||
normalize_asn(aspa["customer"]),
|
|
||||||
";".join(providers),
|
|
||||||
str(aspa.get("ta", "")).strip().lower(),
|
|
||||||
)
|
|
||||||
)
|
|
||||||
rows.sort(key=lambda row: (int(row[0][2:]), row[1], row[2]))
|
|
||||||
|
|
||||||
args.csv_out.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
with args.csv_out.open("w", encoding="utf-8", newline="") as fh:
|
|
||||||
writer = csv.writer(fh)
|
|
||||||
writer.writerow(["Customer ASN", "Providers", "Trust Anchor"])
|
|
||||||
writer.writerows(rows)
|
|
||||||
print(args.csv_out)
|
|
||||||
return 0
|
|
||||||
|
|
||||||
|
|
||||||
if __name__ == "__main__":
|
|
||||||
raise SystemExit(main())
|
|
||||||
@@ -1,84 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage:
|
|
||||||
./scripts/cir/run_cir_drop_sequence.sh \
|
|
||||||
--sequence-root <path> \
|
|
||||||
[--drop-bin <path>]
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
SEQUENCE_ROOT=""
|
|
||||||
DROP_BIN="${DROP_BIN:-$ROOT_DIR/target/release/cir_drop_report}"
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--sequence-root) SEQUENCE_ROOT="$2"; shift 2 ;;
|
|
||||||
--drop-bin) DROP_BIN="$2"; shift 2 ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
[[ -n "$SEQUENCE_ROOT" ]] || { usage >&2; exit 2; }
|
|
||||||
|
|
||||||
python3 - <<'PY' "$SEQUENCE_ROOT" "$DROP_BIN"
|
|
||||||
import json
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
sequence_root = Path(sys.argv[1]).resolve()
|
|
||||||
drop_bin = sys.argv[2]
|
|
||||||
sequence = json.loads((sequence_root / "sequence.json").read_text(encoding="utf-8"))
|
|
||||||
static_root = sequence_root / sequence["staticRoot"] if "staticRoot" in sequence else None
|
|
||||||
raw_store_db = sequence_root / sequence["rawStoreDbPath"] if "rawStoreDbPath" in sequence else None
|
|
||||||
backend_count = sum(x is not None for x in (static_root, raw_store_db))
|
|
||||||
if backend_count != 1:
|
|
||||||
raise SystemExit("sequence must set exactly one of staticRoot or rawStoreDbPath")
|
|
||||||
|
|
||||||
summaries = []
|
|
||||||
for step in sequence["steps"]:
|
|
||||||
step_id = step["stepId"]
|
|
||||||
out_dir = sequence_root / "drop" / step_id
|
|
||||||
out_dir.mkdir(parents=True, exist_ok=True)
|
|
||||||
cmd = [
|
|
||||||
drop_bin,
|
|
||||||
"--cir",
|
|
||||||
str(sequence_root / step["cirPath"]),
|
|
||||||
"--ccr",
|
|
||||||
str(sequence_root / step["ccrPath"]),
|
|
||||||
"--report-json",
|
|
||||||
str(sequence_root / step["reportPath"]),
|
|
||||||
"--json-out",
|
|
||||||
str(out_dir / "drop.json"),
|
|
||||||
"--md-out",
|
|
||||||
str(out_dir / "drop.md"),
|
|
||||||
]
|
|
||||||
if static_root is not None:
|
|
||||||
cmd.extend(["--static-root", str(static_root)])
|
|
||||||
else:
|
|
||||||
cmd.extend(["--raw-store-db", str(raw_store_db)])
|
|
||||||
proc = subprocess.run(cmd, capture_output=True, text=True)
|
|
||||||
if proc.returncode != 0:
|
|
||||||
raise SystemExit(
|
|
||||||
f"drop report failed for {step_id}: stdout={proc.stdout} stderr={proc.stderr}"
|
|
||||||
)
|
|
||||||
result = json.loads((out_dir / "drop.json").read_text(encoding="utf-8"))
|
|
||||||
summaries.append(
|
|
||||||
{
|
|
||||||
"stepId": step_id,
|
|
||||||
"droppedVrpCount": result["summary"]["droppedVrpCount"],
|
|
||||||
"droppedObjectCount": result["summary"]["droppedObjectCount"],
|
|
||||||
"reportPath": str(out_dir / "drop.json"),
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
summary = {"version": 1, "steps": summaries}
|
|
||||||
(sequence_root / "drop-summary.json").write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
|
||||||
PY
|
|
||||||
|
|
||||||
echo "done: $SEQUENCE_ROOT"
|
|
||||||
@@ -1,173 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage:
|
|
||||||
./scripts/cir/run_cir_record_full_delta.sh \
|
|
||||||
--out-dir <path> \
|
|
||||||
--tal-path <path> \
|
|
||||||
--ta-path <path> \
|
|
||||||
--cir-tal-uri <url> \
|
|
||||||
--payload-replay-archive <path> \
|
|
||||||
--payload-replay-locks <path> \
|
|
||||||
--payload-base-archive <path> \
|
|
||||||
--payload-base-locks <path> \
|
|
||||||
--payload-delta-archive <path> \
|
|
||||||
--payload-delta-locks <path> \
|
|
||||||
[--base-validation-time <rfc3339>] \
|
|
||||||
[--delta-validation-time <rfc3339>] \
|
|
||||||
[--max-depth <n>] \
|
|
||||||
[--max-instances <n>] \
|
|
||||||
[--rpki-bin <path>]
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
|
|
||||||
OUT_DIR=""
|
|
||||||
TAL_PATH=""
|
|
||||||
TA_PATH=""
|
|
||||||
CIR_TAL_URI=""
|
|
||||||
PAYLOAD_REPLAY_ARCHIVE=""
|
|
||||||
PAYLOAD_REPLAY_LOCKS=""
|
|
||||||
PAYLOAD_BASE_ARCHIVE=""
|
|
||||||
PAYLOAD_BASE_LOCKS=""
|
|
||||||
PAYLOAD_DELTA_ARCHIVE=""
|
|
||||||
PAYLOAD_DELTA_LOCKS=""
|
|
||||||
BASE_VALIDATION_TIME=""
|
|
||||||
DELTA_VALIDATION_TIME=""
|
|
||||||
MAX_DEPTH=0
|
|
||||||
MAX_INSTANCES=1
|
|
||||||
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--out-dir) OUT_DIR="$2"; shift 2 ;;
|
|
||||||
--tal-path) TAL_PATH="$2"; shift 2 ;;
|
|
||||||
--ta-path) TA_PATH="$2"; shift 2 ;;
|
|
||||||
--cir-tal-uri) CIR_TAL_URI="$2"; shift 2 ;;
|
|
||||||
--payload-replay-archive) PAYLOAD_REPLAY_ARCHIVE="$2"; shift 2 ;;
|
|
||||||
--payload-replay-locks) PAYLOAD_REPLAY_LOCKS="$2"; shift 2 ;;
|
|
||||||
--payload-base-archive) PAYLOAD_BASE_ARCHIVE="$2"; shift 2 ;;
|
|
||||||
--payload-base-locks) PAYLOAD_BASE_LOCKS="$2"; shift 2 ;;
|
|
||||||
--payload-delta-archive) PAYLOAD_DELTA_ARCHIVE="$2"; shift 2 ;;
|
|
||||||
--payload-delta-locks) PAYLOAD_DELTA_LOCKS="$2"; shift 2 ;;
|
|
||||||
--base-validation-time) BASE_VALIDATION_TIME="$2"; shift 2 ;;
|
|
||||||
--delta-validation-time) DELTA_VALIDATION_TIME="$2"; shift 2 ;;
|
|
||||||
--max-depth) MAX_DEPTH="$2"; shift 2 ;;
|
|
||||||
--max-instances) MAX_INSTANCES="$2"; shift 2 ;;
|
|
||||||
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
[[ -n "$OUT_DIR" && -n "$TAL_PATH" && -n "$TA_PATH" && -n "$CIR_TAL_URI" && -n "$PAYLOAD_REPLAY_ARCHIVE" && -n "$PAYLOAD_REPLAY_LOCKS" && -n "$PAYLOAD_BASE_ARCHIVE" && -n "$PAYLOAD_BASE_LOCKS" && -n "$PAYLOAD_DELTA_ARCHIVE" && -n "$PAYLOAD_DELTA_LOCKS" ]] || {
|
|
||||||
usage >&2
|
|
||||||
exit 2
|
|
||||||
}
|
|
||||||
|
|
||||||
if [[ ! -x "$RPKI_BIN" ]]; then
|
|
||||||
(
|
|
||||||
cd "$ROOT_DIR"
|
|
||||||
cargo build --release --bin rpki
|
|
||||||
)
|
|
||||||
fi
|
|
||||||
|
|
||||||
resolve_validation_time() {
|
|
||||||
local path="$1"
|
|
||||||
python3 - <<'PY' "$path"
|
|
||||||
import json, sys
|
|
||||||
print(json.load(open(sys.argv[1], 'r', encoding='utf-8'))['validationTime'])
|
|
||||||
PY
|
|
||||||
}
|
|
||||||
|
|
||||||
if [[ -z "$BASE_VALIDATION_TIME" ]]; then
|
|
||||||
BASE_VALIDATION_TIME="$(resolve_validation_time "$PAYLOAD_REPLAY_LOCKS")"
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [[ -z "$DELTA_VALIDATION_TIME" ]]; then
|
|
||||||
DELTA_VALIDATION_TIME="$(resolve_validation_time "$PAYLOAD_DELTA_LOCKS")"
|
|
||||||
fi
|
|
||||||
|
|
||||||
rm -rf "$OUT_DIR"
|
|
||||||
mkdir -p "$OUT_DIR/full" "$OUT_DIR/delta-001" "$OUT_DIR/static"
|
|
||||||
|
|
||||||
FULL_DB="$OUT_DIR/full/db"
|
|
||||||
DELTA_DB="$OUT_DIR/delta-001/db"
|
|
||||||
|
|
||||||
"$RPKI_BIN" \
|
|
||||||
--db "$FULL_DB" \
|
|
||||||
--tal-path "$TAL_PATH" \
|
|
||||||
--ta-path "$TA_PATH" \
|
|
||||||
--payload-replay-archive "$PAYLOAD_REPLAY_ARCHIVE" \
|
|
||||||
--payload-replay-locks "$PAYLOAD_REPLAY_LOCKS" \
|
|
||||||
--validation-time "$BASE_VALIDATION_TIME" \
|
|
||||||
--max-depth "$MAX_DEPTH" \
|
|
||||||
--max-instances "$MAX_INSTANCES" \
|
|
||||||
--ccr-out "$OUT_DIR/full/result.ccr" \
|
|
||||||
--report-json "$OUT_DIR/full/report.json" \
|
|
||||||
--cir-enable \
|
|
||||||
--cir-out "$OUT_DIR/full/input.cir" \
|
|
||||||
--cir-static-root "$OUT_DIR/static" \
|
|
||||||
--cir-tal-uri "$CIR_TAL_URI" \
|
|
||||||
>"$OUT_DIR/full/run.stdout.log" 2>"$OUT_DIR/full/run.stderr.log"
|
|
||||||
|
|
||||||
"$RPKI_BIN" \
|
|
||||||
--db "$DELTA_DB" \
|
|
||||||
--tal-path "$TAL_PATH" \
|
|
||||||
--ta-path "$TA_PATH" \
|
|
||||||
--payload-base-archive "$PAYLOAD_BASE_ARCHIVE" \
|
|
||||||
--payload-base-locks "$PAYLOAD_BASE_LOCKS" \
|
|
||||||
--payload-delta-archive "$PAYLOAD_DELTA_ARCHIVE" \
|
|
||||||
--payload-delta-locks "$PAYLOAD_DELTA_LOCKS" \
|
|
||||||
--payload-base-validation-time "$BASE_VALIDATION_TIME" \
|
|
||||||
--validation-time "$DELTA_VALIDATION_TIME" \
|
|
||||||
--max-depth "$MAX_DEPTH" \
|
|
||||||
--max-instances "$MAX_INSTANCES" \
|
|
||||||
--ccr-out "$OUT_DIR/delta-001/result.ccr" \
|
|
||||||
--report-json "$OUT_DIR/delta-001/report.json" \
|
|
||||||
--cir-enable \
|
|
||||||
--cir-out "$OUT_DIR/delta-001/input.cir" \
|
|
||||||
--cir-static-root "$OUT_DIR/static" \
|
|
||||||
--cir-tal-uri "$CIR_TAL_URI" \
|
|
||||||
>"$OUT_DIR/delta-001/run.stdout.log" 2>"$OUT_DIR/delta-001/run.stderr.log"
|
|
||||||
|
|
||||||
python3 - <<'PY' "$OUT_DIR" "$BASE_VALIDATION_TIME" "$DELTA_VALIDATION_TIME"
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
out = Path(sys.argv[1])
|
|
||||||
base_validation_time = sys.argv[2]
|
|
||||||
delta_validation_time = sys.argv[3]
|
|
||||||
static_files = sum(1 for _ in (out / "static").rglob("*") if _.is_file())
|
|
||||||
summary = {
|
|
||||||
"version": 1,
|
|
||||||
"kind": "cir_pair",
|
|
||||||
"baseValidationTime": base_validation_time,
|
|
||||||
"deltaValidationTime": delta_validation_time,
|
|
||||||
"staticRoot": "static",
|
|
||||||
"steps": [
|
|
||||||
{
|
|
||||||
"kind": "full",
|
|
||||||
"cirPath": "full/input.cir",
|
|
||||||
"ccrPath": "full/result.ccr",
|
|
||||||
"reportPath": "full/report.json",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"kind": "delta",
|
|
||||||
"cirPath": "delta-001/input.cir",
|
|
||||||
"ccrPath": "delta-001/result.ccr",
|
|
||||||
"reportPath": "delta-001/report.json",
|
|
||||||
"previous": "full",
|
|
||||||
},
|
|
||||||
],
|
|
||||||
"staticFileCount": static_files,
|
|
||||||
}
|
|
||||||
(out / "summary.json").write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
|
||||||
PY
|
|
||||||
|
|
||||||
echo "done: $OUT_DIR"
|
|
||||||
@@ -1,129 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage:
|
|
||||||
./scripts/cir/run_cir_record_sequence_multi_rir_offline.sh \
|
|
||||||
[--bundle-root <path>] \
|
|
||||||
[--rir <afrinic,apnic,arin,lacnic,ripe>] \
|
|
||||||
[--delta-count <n>] \
|
|
||||||
[--full-repo] \
|
|
||||||
[--out-root <path>] \
|
|
||||||
[--rpki-bin <path>]
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
CASE_INFO="$ROOT_DIR/scripts/payload_replay/multi_rir_case_info.py"
|
|
||||||
SINGLE_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_record_sequence_offline.sh"
|
|
||||||
|
|
||||||
BUNDLE_ROOT="/home/yuyr/dev/rust_playground/routinator/bench/multi_rir_demo/runs/20260316-112341-multi-final3"
|
|
||||||
RIRS="afrinic,apnic,arin,lacnic,ripe"
|
|
||||||
DELTA_COUNT=2
|
|
||||||
FULL_REPO=0
|
|
||||||
OUT_ROOT="$ROOT_DIR/target/replay/cir_sequence_multi_rir_offline_$(date -u +%Y%m%dT%H%M%SZ)"
|
|
||||||
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--bundle-root) BUNDLE_ROOT="$2"; shift 2 ;;
|
|
||||||
--rir) RIRS="$2"; shift 2 ;;
|
|
||||||
--delta-count) DELTA_COUNT="$2"; shift 2 ;;
|
|
||||||
--full-repo) FULL_REPO=1; shift 1 ;;
|
|
||||||
--out-root) OUT_ROOT="$2"; shift 2 ;;
|
|
||||||
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
mkdir -p "$OUT_ROOT"
|
|
||||||
SUMMARY_JSON="$OUT_ROOT/summary.json"
|
|
||||||
SUMMARY_MD="$OUT_ROOT/summary.md"
|
|
||||||
|
|
||||||
IFS=',' read -r -a RIR_ITEMS <<< "$RIRS"
|
|
||||||
|
|
||||||
for rir in "${RIR_ITEMS[@]}"; do
|
|
||||||
CASE_JSON="$(python3 "$CASE_INFO" --bundle-root "$BUNDLE_ROOT" --repo-root "$ROOT_DIR" --rir "$rir")"
|
|
||||||
TAL_PATH="$(python3 - <<'PY' "$CASE_JSON"
|
|
||||||
import json,sys
|
|
||||||
print(json.loads(sys.argv[1])['tal_path'])
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
TA_PATH="$(python3 - <<'PY' "$CASE_JSON"
|
|
||||||
import json,sys
|
|
||||||
print(json.loads(sys.argv[1])['ta_path'])
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
BASE_ARCHIVE="$(python3 - <<'PY' "$CASE_JSON"
|
|
||||||
import json,sys
|
|
||||||
print(json.loads(sys.argv[1])['base_archive'])
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
BASE_LOCKS="$(python3 - <<'PY' "$CASE_JSON"
|
|
||||||
import json,sys
|
|
||||||
print(json.loads(sys.argv[1])['base_locks'])
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
DELTA_ARCHIVE="$(python3 - <<'PY' "$CASE_JSON"
|
|
||||||
import json,sys
|
|
||||||
print(json.loads(sys.argv[1])['delta_archive'])
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
DELTA_LOCKS="$(python3 - <<'PY' "$CASE_JSON"
|
|
||||||
import json,sys
|
|
||||||
print(json.loads(sys.argv[1])['delta_locks'])
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
OUT_DIR="$OUT_ROOT/$rir"
|
|
||||||
args=(
|
|
||||||
"$SINGLE_SCRIPT"
|
|
||||||
--out-dir "$OUT_DIR" \
|
|
||||||
--tal-path "$TAL_PATH" \
|
|
||||||
--ta-path "$TA_PATH" \
|
|
||||||
--cir-tal-uri "https://example.test/$rir.tal" \
|
|
||||||
--payload-replay-archive "$BASE_ARCHIVE" \
|
|
||||||
--payload-replay-locks "$BASE_LOCKS" \
|
|
||||||
--payload-base-archive "$BASE_ARCHIVE" \
|
|
||||||
--payload-base-locks "$BASE_LOCKS" \
|
|
||||||
--payload-delta-archive "$DELTA_ARCHIVE" \
|
|
||||||
--payload-delta-locks "$DELTA_LOCKS" \
|
|
||||||
--delta-count "$DELTA_COUNT" \
|
|
||||||
--rpki-bin "$RPKI_BIN"
|
|
||||||
)
|
|
||||||
if [[ "$FULL_REPO" -ne 1 ]]; then
|
|
||||||
args+=(--max-depth 0 --max-instances 1)
|
|
||||||
else
|
|
||||||
args+=(--full-repo)
|
|
||||||
fi
|
|
||||||
"${args[@]}"
|
|
||||||
done
|
|
||||||
|
|
||||||
python3 - <<'PY' "$OUT_ROOT" "$RIRS" "$SUMMARY_JSON" "$SUMMARY_MD"
|
|
||||||
import json, sys
|
|
||||||
from pathlib import Path
|
|
||||||
out_root = Path(sys.argv[1])
|
|
||||||
rirs = [item for item in sys.argv[2].split(',') if item]
|
|
||||||
summary_json = Path(sys.argv[3])
|
|
||||||
summary_md = Path(sys.argv[4])
|
|
||||||
items = []
|
|
||||||
for rir in rirs:
|
|
||||||
root = out_root / rir
|
|
||||||
seq = json.loads((root / "sequence.json").read_text(encoding="utf-8"))
|
|
||||||
summ = json.loads((root / "summary.json").read_text(encoding="utf-8"))
|
|
||||||
items.append({
|
|
||||||
"rir": rir,
|
|
||||||
"root": str(root),
|
|
||||||
"stepCount": len(seq["steps"]),
|
|
||||||
"staticFileCount": summ["staticFileCount"],
|
|
||||||
})
|
|
||||||
summary = {"version": 1, "rirs": items}
|
|
||||||
summary_json.write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
|
||||||
lines = ["# Multi-RIR Offline CIR Sequence Summary", ""]
|
|
||||||
for item in items:
|
|
||||||
lines.append(f"- `{item['rir']}`: `stepCount={item['stepCount']}` `staticFileCount={item['staticFileCount']}` `root={item['root']}`")
|
|
||||||
summary_md.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
|
||||||
PY
|
|
||||||
|
|
||||||
echo "done: $OUT_ROOT"
|
|
||||||
@@ -1,206 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage:
|
|
||||||
./scripts/cir/run_cir_record_sequence_offline.sh \
|
|
||||||
--out-dir <path> \
|
|
||||||
--tal-path <path> \
|
|
||||||
--ta-path <path> \
|
|
||||||
--cir-tal-uri <url> \
|
|
||||||
--payload-replay-archive <path> \
|
|
||||||
--payload-replay-locks <path> \
|
|
||||||
--payload-base-archive <path> \
|
|
||||||
--payload-base-locks <path> \
|
|
||||||
--payload-delta-archive <path> \
|
|
||||||
--payload-delta-locks <path> \
|
|
||||||
[--delta-count <n>] \
|
|
||||||
[--base-validation-time <rfc3339>] \
|
|
||||||
[--delta-validation-time <rfc3339>] \
|
|
||||||
[--full-repo] \
|
|
||||||
[--max-depth <n>] \
|
|
||||||
[--max-instances <n>] \
|
|
||||||
[--rpki-bin <path>]
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
|
|
||||||
OUT_DIR=""
|
|
||||||
TAL_PATH=""
|
|
||||||
TA_PATH=""
|
|
||||||
CIR_TAL_URI=""
|
|
||||||
PAYLOAD_REPLAY_ARCHIVE=""
|
|
||||||
PAYLOAD_REPLAY_LOCKS=""
|
|
||||||
PAYLOAD_BASE_ARCHIVE=""
|
|
||||||
PAYLOAD_BASE_LOCKS=""
|
|
||||||
PAYLOAD_DELTA_ARCHIVE=""
|
|
||||||
PAYLOAD_DELTA_LOCKS=""
|
|
||||||
BASE_VALIDATION_TIME=""
|
|
||||||
DELTA_VALIDATION_TIME=""
|
|
||||||
DELTA_COUNT=2
|
|
||||||
FULL_REPO=0
|
|
||||||
MAX_DEPTH=0
|
|
||||||
MAX_INSTANCES=1
|
|
||||||
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--out-dir) OUT_DIR="$2"; shift 2 ;;
|
|
||||||
--tal-path) TAL_PATH="$2"; shift 2 ;;
|
|
||||||
--ta-path) TA_PATH="$2"; shift 2 ;;
|
|
||||||
--cir-tal-uri) CIR_TAL_URI="$2"; shift 2 ;;
|
|
||||||
--payload-replay-archive) PAYLOAD_REPLAY_ARCHIVE="$2"; shift 2 ;;
|
|
||||||
--payload-replay-locks) PAYLOAD_REPLAY_LOCKS="$2"; shift 2 ;;
|
|
||||||
--payload-base-archive) PAYLOAD_BASE_ARCHIVE="$2"; shift 2 ;;
|
|
||||||
--payload-base-locks) PAYLOAD_BASE_LOCKS="$2"; shift 2 ;;
|
|
||||||
--payload-delta-archive) PAYLOAD_DELTA_ARCHIVE="$2"; shift 2 ;;
|
|
||||||
--payload-delta-locks) PAYLOAD_DELTA_LOCKS="$2"; shift 2 ;;
|
|
||||||
--base-validation-time) BASE_VALIDATION_TIME="$2"; shift 2 ;;
|
|
||||||
--delta-validation-time) DELTA_VALIDATION_TIME="$2"; shift 2 ;;
|
|
||||||
--delta-count) DELTA_COUNT="$2"; shift 2 ;;
|
|
||||||
--full-repo) FULL_REPO=1; shift 1 ;;
|
|
||||||
--max-depth) MAX_DEPTH="$2"; shift 2 ;;
|
|
||||||
--max-instances) MAX_INSTANCES="$2"; shift 2 ;;
|
|
||||||
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
[[ -n "$OUT_DIR" && -n "$TAL_PATH" && -n "$TA_PATH" && -n "$CIR_TAL_URI" && -n "$PAYLOAD_REPLAY_ARCHIVE" && -n "$PAYLOAD_REPLAY_LOCKS" && -n "$PAYLOAD_BASE_ARCHIVE" && -n "$PAYLOAD_BASE_LOCKS" && -n "$PAYLOAD_DELTA_ARCHIVE" && -n "$PAYLOAD_DELTA_LOCKS" ]] || {
|
|
||||||
usage >&2
|
|
||||||
exit 2
|
|
||||||
}
|
|
||||||
|
|
||||||
if [[ ! -x "$RPKI_BIN" ]]; then
|
|
||||||
(
|
|
||||||
cd "$ROOT_DIR"
|
|
||||||
cargo build --release --bin rpki
|
|
||||||
)
|
|
||||||
fi
|
|
||||||
|
|
||||||
resolve_validation_time() {
|
|
||||||
local path="$1"
|
|
||||||
python3 - <<'PY' "$path"
|
|
||||||
import json, sys
|
|
||||||
print(json.load(open(sys.argv[1], 'r', encoding='utf-8'))['validationTime'])
|
|
||||||
PY
|
|
||||||
}
|
|
||||||
|
|
||||||
if [[ -z "$BASE_VALIDATION_TIME" ]]; then
|
|
||||||
BASE_VALIDATION_TIME="$(resolve_validation_time "$PAYLOAD_REPLAY_LOCKS")"
|
|
||||||
fi
|
|
||||||
if [[ -z "$DELTA_VALIDATION_TIME" ]]; then
|
|
||||||
DELTA_VALIDATION_TIME="$(resolve_validation_time "$PAYLOAD_DELTA_LOCKS")"
|
|
||||||
fi
|
|
||||||
|
|
||||||
rm -rf "$OUT_DIR"
|
|
||||||
mkdir -p "$OUT_DIR/static" "$OUT_DIR/full"
|
|
||||||
|
|
||||||
run_step() {
|
|
||||||
local kind="$1"
|
|
||||||
local step_dir="$2"
|
|
||||||
local db_dir="$3"
|
|
||||||
shift 3
|
|
||||||
mkdir -p "$step_dir"
|
|
||||||
local -a cmd=(
|
|
||||||
"$RPKI_BIN"
|
|
||||||
--db "$db_dir" \
|
|
||||||
--tal-path "$TAL_PATH" \
|
|
||||||
--ta-path "$TA_PATH" \
|
|
||||||
--ccr-out "$step_dir/result.ccr" \
|
|
||||||
--report-json "$step_dir/report.json" \
|
|
||||||
--cir-enable \
|
|
||||||
--cir-out "$step_dir/input.cir" \
|
|
||||||
--cir-static-root "$OUT_DIR/static" \
|
|
||||||
--cir-tal-uri "$CIR_TAL_URI"
|
|
||||||
)
|
|
||||||
if [[ "$FULL_REPO" -ne 1 ]]; then
|
|
||||||
cmd+=(--max-depth "$MAX_DEPTH" --max-instances "$MAX_INSTANCES")
|
|
||||||
fi
|
|
||||||
cmd+=("$@")
|
|
||||||
"${cmd[@]}" >"$step_dir/run.stdout.log" 2>"$step_dir/run.stderr.log"
|
|
||||||
}
|
|
||||||
|
|
||||||
run_step \
|
|
||||||
full \
|
|
||||||
"$OUT_DIR/full" \
|
|
||||||
"$OUT_DIR/full/db" \
|
|
||||||
--payload-replay-archive "$PAYLOAD_REPLAY_ARCHIVE" \
|
|
||||||
--payload-replay-locks "$PAYLOAD_REPLAY_LOCKS" \
|
|
||||||
--validation-time "$BASE_VALIDATION_TIME"
|
|
||||||
|
|
||||||
for idx in $(seq 1 "$DELTA_COUNT"); do
|
|
||||||
step_id="$(printf 'delta-%03d' "$idx")"
|
|
||||||
run_step \
|
|
||||||
delta \
|
|
||||||
"$OUT_DIR/$step_id" \
|
|
||||||
"$OUT_DIR/$step_id/db" \
|
|
||||||
--payload-base-archive "$PAYLOAD_BASE_ARCHIVE" \
|
|
||||||
--payload-base-locks "$PAYLOAD_BASE_LOCKS" \
|
|
||||||
--payload-delta-archive "$PAYLOAD_DELTA_ARCHIVE" \
|
|
||||||
--payload-delta-locks "$PAYLOAD_DELTA_LOCKS" \
|
|
||||||
--payload-base-validation-time "$BASE_VALIDATION_TIME" \
|
|
||||||
--validation-time "$DELTA_VALIDATION_TIME"
|
|
||||||
done
|
|
||||||
|
|
||||||
python3 - <<'PY' "$OUT_DIR" "$BASE_VALIDATION_TIME" "$DELTA_VALIDATION_TIME" "$DELTA_COUNT"
|
|
||||||
import json
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
out = Path(sys.argv[1])
|
|
||||||
base_validation_time = sys.argv[2]
|
|
||||||
delta_validation_time = sys.argv[3]
|
|
||||||
delta_count = int(sys.argv[4])
|
|
||||||
|
|
||||||
steps = [
|
|
||||||
{
|
|
||||||
"stepId": "full",
|
|
||||||
"kind": "full",
|
|
||||||
"validationTime": base_validation_time,
|
|
||||||
"cirPath": "full/input.cir",
|
|
||||||
"ccrPath": "full/result.ccr",
|
|
||||||
"reportPath": "full/report.json",
|
|
||||||
"previousStepId": None,
|
|
||||||
}
|
|
||||||
]
|
|
||||||
previous = "full"
|
|
||||||
for idx in range(1, delta_count + 1):
|
|
||||||
step_id = f"delta-{idx:03d}"
|
|
||||||
steps.append(
|
|
||||||
{
|
|
||||||
"stepId": step_id,
|
|
||||||
"kind": "delta",
|
|
||||||
"validationTime": delta_validation_time,
|
|
||||||
"cirPath": f"{step_id}/input.cir",
|
|
||||||
"ccrPath": f"{step_id}/result.ccr",
|
|
||||||
"reportPath": f"{step_id}/report.json",
|
|
||||||
"previousStepId": previous,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
previous = step_id
|
|
||||||
|
|
||||||
summary = {
|
|
||||||
"version": 1,
|
|
||||||
"kind": "cir_sequence_offline",
|
|
||||||
"staticRoot": "static",
|
|
||||||
"steps": steps,
|
|
||||||
}
|
|
||||||
(out / "sequence.json").write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
|
||||||
(out / "summary.json").write_text(
|
|
||||||
json.dumps(
|
|
||||||
{
|
|
||||||
"version": 1,
|
|
||||||
"stepCount": len(steps),
|
|
||||||
"staticFileCount": sum(1 for p in (out / "static").rglob("*") if p.is_file()),
|
|
||||||
},
|
|
||||||
indent=2,
|
|
||||||
),
|
|
||||||
encoding="utf-8",
|
|
||||||
)
|
|
||||||
PY
|
|
||||||
|
|
||||||
echo "done: $OUT_DIR"
|
|
||||||
@@ -1,244 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage:
|
|
||||||
./scripts/cir/run_cir_record_sequence_remote.sh \
|
|
||||||
--rir <name> \
|
|
||||||
--remote-root <path> \
|
|
||||||
[--ssh-target <user@host>] \
|
|
||||||
[--out-subdir <path>] \
|
|
||||||
[--delta-count <n>] \
|
|
||||||
[--sleep-secs <n>] \
|
|
||||||
[--full-repo] \
|
|
||||||
[--max-depth <n>] \
|
|
||||||
[--max-instances <n>]
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
SSH_TARGET="${SSH_TARGET:-root@47.77.183.68}"
|
|
||||||
RIR=""
|
|
||||||
REMOTE_ROOT=""
|
|
||||||
OUT_SUBDIR=""
|
|
||||||
DELTA_COUNT=2
|
|
||||||
SLEEP_SECS=30
|
|
||||||
FULL_REPO=0
|
|
||||||
MAX_DEPTH=0
|
|
||||||
MAX_INSTANCES=1
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--rir) RIR="$2"; shift 2 ;;
|
|
||||||
--remote-root) REMOTE_ROOT="$2"; shift 2 ;;
|
|
||||||
--ssh-target) SSH_TARGET="$2"; shift 2 ;;
|
|
||||||
--out-subdir) OUT_SUBDIR="$2"; shift 2 ;;
|
|
||||||
--delta-count) DELTA_COUNT="$2"; shift 2 ;;
|
|
||||||
--sleep-secs) SLEEP_SECS="$2"; shift 2 ;;
|
|
||||||
--full-repo) FULL_REPO=1; shift 1 ;;
|
|
||||||
--max-depth) MAX_DEPTH="$2"; shift 2 ;;
|
|
||||||
--max-instances) MAX_INSTANCES="$2"; shift 2 ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
[[ -n "$RIR" && -n "$REMOTE_ROOT" ]] || { usage >&2; exit 2; }
|
|
||||||
|
|
||||||
case "$RIR" in
|
|
||||||
afrinic) TAL_REL="tests/fixtures/tal/afrinic.tal"; TA_REL="tests/fixtures/ta/afrinic-ta.cer" ;;
|
|
||||||
apnic) TAL_REL="tests/fixtures/tal/apnic-rfc7730-https.tal"; TA_REL="tests/fixtures/ta/apnic-ta.cer" ;;
|
|
||||||
arin) TAL_REL="tests/fixtures/tal/arin.tal"; TA_REL="tests/fixtures/ta/arin-ta.cer" ;;
|
|
||||||
lacnic) TAL_REL="tests/fixtures/tal/lacnic.tal"; TA_REL="tests/fixtures/ta/lacnic-ta.cer" ;;
|
|
||||||
ripe) TAL_REL="tests/fixtures/tal/ripe-ncc.tal"; TA_REL="tests/fixtures/ta/ripe-ncc-ta.cer" ;;
|
|
||||||
*) echo "unsupported rir: $RIR" >&2; exit 2 ;;
|
|
||||||
esac
|
|
||||||
|
|
||||||
rsync -a --delete \
|
|
||||||
--exclude target \
|
|
||||||
--exclude .git \
|
|
||||||
"$ROOT_DIR/" "$SSH_TARGET:$REMOTE_ROOT/"
|
|
||||||
ssh "$SSH_TARGET" "mkdir -p '$REMOTE_ROOT/target/release'"
|
|
||||||
rsync -a "$ROOT_DIR/target/release/rpki" "$SSH_TARGET:$REMOTE_ROOT/target/release/"
|
|
||||||
|
|
||||||
ssh "$SSH_TARGET" \
|
|
||||||
RIR="$RIR" \
|
|
||||||
REMOTE_ROOT="$REMOTE_ROOT" \
|
|
||||||
OUT_SUBDIR="$OUT_SUBDIR" \
|
|
||||||
DELTA_COUNT="$DELTA_COUNT" \
|
|
||||||
SLEEP_SECS="$SLEEP_SECS" \
|
|
||||||
FULL_REPO="$FULL_REPO" \
|
|
||||||
MAX_DEPTH="$MAX_DEPTH" \
|
|
||||||
MAX_INSTANCES="$MAX_INSTANCES" \
|
|
||||||
TAL_REL="$TAL_REL" \
|
|
||||||
TA_REL="$TA_REL" \
|
|
||||||
'bash -s' <<'EOS'
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
cd "$REMOTE_ROOT"
|
|
||||||
|
|
||||||
if [[ -n "${OUT_SUBDIR}" ]]; then
|
|
||||||
OUT="${OUT_SUBDIR}"
|
|
||||||
else
|
|
||||||
OUT="target/replay/cir_sequence_remote_${RIR}_$(date -u +%Y%m%dT%H%M%SZ)"
|
|
||||||
fi
|
|
||||||
|
|
||||||
mkdir -p "$OUT"
|
|
||||||
DB="$OUT/work-db"
|
|
||||||
RAW_STORE_DB="$OUT/raw-store.db"
|
|
||||||
ROWS="$OUT/.sequence_rows.tsv"
|
|
||||||
: > "$ROWS"
|
|
||||||
|
|
||||||
write_step_timing() {
|
|
||||||
local path="$1"
|
|
||||||
local start_ms="$2"
|
|
||||||
local end_ms="$3"
|
|
||||||
local started_at="$4"
|
|
||||||
local finished_at="$5"
|
|
||||||
python3 - <<'PY' "$path" "$start_ms" "$end_ms" "$started_at" "$finished_at"
|
|
||||||
import json, sys
|
|
||||||
path, start_ms, end_ms, started_at, finished_at = sys.argv[1:]
|
|
||||||
start_ms = int(start_ms)
|
|
||||||
end_ms = int(end_ms)
|
|
||||||
with open(path, "w", encoding="utf-8") as fh:
|
|
||||||
json.dump(
|
|
||||||
{
|
|
||||||
"durationMs": end_ms - start_ms,
|
|
||||||
"startedAt": started_at,
|
|
||||||
"finishedAt": finished_at,
|
|
||||||
},
|
|
||||||
fh,
|
|
||||||
indent=2,
|
|
||||||
)
|
|
||||||
PY
|
|
||||||
}
|
|
||||||
|
|
||||||
run_step() {
|
|
||||||
local step_id="$1"
|
|
||||||
local kind="$2"
|
|
||||||
local previous_step_id="$3"
|
|
||||||
shift 3
|
|
||||||
|
|
||||||
local started_at_iso started_at_ms finished_at_iso finished_at_ms prefix
|
|
||||||
started_at_iso="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
|
||||||
started_at_ms="$(python3 - <<'PY'
|
|
||||||
import time
|
|
||||||
print(int(time.time() * 1000))
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
prefix="${started_at_iso}-test"
|
|
||||||
|
|
||||||
local cir_out="$OUT/${prefix}.cir"
|
|
||||||
local ccr_out="$OUT/${prefix}.ccr"
|
|
||||||
local report_out="$OUT/${prefix}.report.json"
|
|
||||||
local timing_out="$OUT/${prefix}.timing.json"
|
|
||||||
local stdout_out="$OUT/${prefix}.stdout.log"
|
|
||||||
local stderr_out="$OUT/${prefix}.stderr.log"
|
|
||||||
|
|
||||||
local -a cmd=(
|
|
||||||
target/release/rpki
|
|
||||||
--db "$DB"
|
|
||||||
--raw-store-db "$RAW_STORE_DB"
|
|
||||||
--tal-path "$TAL_REL"
|
|
||||||
--ta-path "$TA_REL"
|
|
||||||
--ccr-out "$ccr_out"
|
|
||||||
--report-json "$report_out"
|
|
||||||
--cir-enable
|
|
||||||
--cir-out "$cir_out"
|
|
||||||
--cir-tal-uri "https://example.test/${RIR}.tal"
|
|
||||||
)
|
|
||||||
if [[ "$FULL_REPO" -ne 1 ]]; then
|
|
||||||
cmd+=(--max-depth "$MAX_DEPTH" --max-instances "$MAX_INSTANCES")
|
|
||||||
fi
|
|
||||||
cmd+=("$@")
|
|
||||||
env RPKI_PROGRESS_LOG=1 "${cmd[@]}" >"$stdout_out" 2>"$stderr_out"
|
|
||||||
|
|
||||||
finished_at_ms="$(python3 - <<'PY'
|
|
||||||
import time
|
|
||||||
print(int(time.time() * 1000))
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
finished_at_iso="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
|
||||||
write_step_timing "$timing_out" "$started_at_ms" "$finished_at_ms" "$started_at_iso" "$finished_at_iso"
|
|
||||||
|
|
||||||
local validation_time
|
|
||||||
validation_time="$(python3 - <<'PY' "$report_out"
|
|
||||||
import json, sys
|
|
||||||
print(json.load(open(sys.argv[1], 'r', encoding='utf-8'))['meta']['validation_time_rfc3339_utc'])
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
|
|
||||||
printf '%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\t%s\n' \
|
|
||||||
"$step_id" \
|
|
||||||
"$kind" \
|
|
||||||
"$validation_time" \
|
|
||||||
"$(basename "$cir_out")" \
|
|
||||||
"$(basename "$ccr_out")" \
|
|
||||||
"$(basename "$report_out")" \
|
|
||||||
"$(basename "$timing_out")" \
|
|
||||||
"$(basename "$stdout_out")" \
|
|
||||||
"$(basename "$stderr_out")" >> "$ROWS"
|
|
||||||
}
|
|
||||||
|
|
||||||
run_step "full" "full" ""
|
|
||||||
|
|
||||||
prev="full"
|
|
||||||
for idx in $(seq 1 "$DELTA_COUNT"); do
|
|
||||||
sleep "$SLEEP_SECS"
|
|
||||||
step="$(printf 'delta-%03d' "$idx")"
|
|
||||||
run_step "$step" "delta" "$prev"
|
|
||||||
prev="$step"
|
|
||||||
done
|
|
||||||
|
|
||||||
python3 - <<'PY' "$OUT" "$ROWS" "$RIR"
|
|
||||||
import json, sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
out = Path(sys.argv[1])
|
|
||||||
rows = Path(sys.argv[2]).read_text(encoding='utf-8').splitlines()
|
|
||||||
rir = sys.argv[3]
|
|
||||||
steps = []
|
|
||||||
for idx, row in enumerate(rows):
|
|
||||||
step_id, kind, validation_time, cir_name, ccr_name, report_name, timing_name, stdout_name, stderr_name = row.split('\t')
|
|
||||||
steps.append({
|
|
||||||
"stepId": step_id,
|
|
||||||
"kind": kind,
|
|
||||||
"validationTime": validation_time,
|
|
||||||
"cirPath": cir_name,
|
|
||||||
"ccrPath": ccr_name,
|
|
||||||
"reportPath": report_name,
|
|
||||||
"timingPath": timing_name,
|
|
||||||
"stdoutLogPath": stdout_name,
|
|
||||||
"stderrLogPath": stderr_name,
|
|
||||||
"artifactPrefix": cir_name[:-4], # strip .cir
|
|
||||||
"previousStepId": None if idx == 0 else steps[idx - 1]["stepId"],
|
|
||||||
})
|
|
||||||
|
|
||||||
(out / "sequence.json").write_text(
|
|
||||||
json.dumps({"version": 1, "rawStoreDbPath": "raw-store.db", "steps": steps}, indent=2),
|
|
||||||
encoding="utf-8",
|
|
||||||
)
|
|
||||||
|
|
||||||
summary = {
|
|
||||||
"version": 1,
|
|
||||||
"rir": rir,
|
|
||||||
"stepCount": len(steps),
|
|
||||||
"steps": [],
|
|
||||||
}
|
|
||||||
for step in steps:
|
|
||||||
timing = json.loads((out / step["timingPath"]).read_text(encoding="utf-8"))
|
|
||||||
summary["steps"].append({
|
|
||||||
"stepId": step["stepId"],
|
|
||||||
"kind": step["kind"],
|
|
||||||
"validationTime": step["validationTime"],
|
|
||||||
"artifactPrefix": step["artifactPrefix"],
|
|
||||||
**timing,
|
|
||||||
})
|
|
||||||
|
|
||||||
(out / "summary.json").write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
|
||||||
PY
|
|
||||||
|
|
||||||
rm -f "$ROWS"
|
|
||||||
echo "$OUT"
|
|
||||||
EOS
|
|
||||||
@@ -1,72 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage:
|
|
||||||
./scripts/cir/run_cir_record_sequence_remote_multi_rir.sh \
|
|
||||||
--remote-root <path> \
|
|
||||||
[--rir <afrinic,apnic,arin,lacnic,ripe>] \
|
|
||||||
[--ssh-target <user@host>] \
|
|
||||||
[--out-subdir-root <path>] \
|
|
||||||
[--delta-count <n>] \
|
|
||||||
[--sleep-secs <n>] \
|
|
||||||
[--full-repo] \
|
|
||||||
[--max-depth <n>] \
|
|
||||||
[--max-instances <n>]
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
SSH_TARGET="${SSH_TARGET:-root@47.77.183.68}"
|
|
||||||
REMOTE_ROOT=""
|
|
||||||
RIRS="afrinic,apnic,arin,lacnic,ripe"
|
|
||||||
OUT_SUBDIR_ROOT=""
|
|
||||||
DELTA_COUNT=2
|
|
||||||
SLEEP_SECS=30
|
|
||||||
FULL_REPO=0
|
|
||||||
MAX_DEPTH=0
|
|
||||||
MAX_INSTANCES=1
|
|
||||||
SINGLE="$ROOT_DIR/scripts/cir/run_cir_record_sequence_remote.sh"
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--remote-root) REMOTE_ROOT="$2"; shift 2 ;;
|
|
||||||
--rir) RIRS="$2"; shift 2 ;;
|
|
||||||
--ssh-target) SSH_TARGET="$2"; shift 2 ;;
|
|
||||||
--out-subdir-root) OUT_SUBDIR_ROOT="$2"; shift 2 ;;
|
|
||||||
--delta-count) DELTA_COUNT="$2"; shift 2 ;;
|
|
||||||
--sleep-secs) SLEEP_SECS="$2"; shift 2 ;;
|
|
||||||
--full-repo) FULL_REPO=1; shift 1 ;;
|
|
||||||
--max-depth) MAX_DEPTH="$2"; shift 2 ;;
|
|
||||||
--max-instances) MAX_INSTANCES="$2"; shift 2 ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
[[ -n "$REMOTE_ROOT" ]] || { usage >&2; exit 2; }
|
|
||||||
if [[ -z "$OUT_SUBDIR_ROOT" ]]; then
|
|
||||||
OUT_SUBDIR_ROOT="target/replay/cir_sequence_remote_multi_rir_$(date -u +%Y%m%dT%H%M%SZ)"
|
|
||||||
fi
|
|
||||||
|
|
||||||
IFS=',' read -r -a ITEMS <<< "$RIRS"
|
|
||||||
for rir in "${ITEMS[@]}"; do
|
|
||||||
args=(
|
|
||||||
"$SINGLE"
|
|
||||||
--rir "$rir" \
|
|
||||||
--remote-root "$REMOTE_ROOT" \
|
|
||||||
--ssh-target "$SSH_TARGET" \
|
|
||||||
--out-subdir "$OUT_SUBDIR_ROOT/$rir" \
|
|
||||||
--delta-count "$DELTA_COUNT" \
|
|
||||||
--sleep-secs "$SLEEP_SECS" \
|
|
||||||
)
|
|
||||||
if [[ "$FULL_REPO" -eq 1 ]]; then
|
|
||||||
args+=(--full-repo)
|
|
||||||
else
|
|
||||||
args+=(--max-depth "$MAX_DEPTH" --max-instances "$MAX_INSTANCES")
|
|
||||||
fi
|
|
||||||
"${args[@]}"
|
|
||||||
done
|
|
||||||
|
|
||||||
echo "$OUT_SUBDIR_ROOT"
|
|
||||||
@@ -1,118 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage:
|
|
||||||
./scripts/cir/run_cir_record_sequence_ta_only_multi_rir.sh \
|
|
||||||
[--rir <afrinic,apnic,arin,lacnic,ripe>] \
|
|
||||||
[--delta-count <n>] \
|
|
||||||
[--out-root <path>] \
|
|
||||||
[--rpki-bin <path>]
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
HELPER_BIN="${HELPER_BIN:-$ROOT_DIR/target/release/cir_ta_only_fixture}"
|
|
||||||
MATERIALIZE_BIN="${MATERIALIZE_BIN:-$ROOT_DIR/target/release/cir_materialize}"
|
|
||||||
EXTRACT_BIN="${EXTRACT_BIN:-$ROOT_DIR/target/release/cir_extract_inputs}"
|
|
||||||
WRAPPER="$ROOT_DIR/scripts/cir/cir-rsync-wrapper"
|
|
||||||
RIRS="afrinic,apnic,arin,lacnic,ripe"
|
|
||||||
DELTA_COUNT=2
|
|
||||||
OUT_ROOT="$ROOT_DIR/target/replay/cir_sequence_multi_rir_ta_only_$(date -u +%Y%m%dT%H%M%SZ)"
|
|
||||||
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--rir) RIRS="$2"; shift 2 ;;
|
|
||||||
--delta-count) DELTA_COUNT="$2"; shift 2 ;;
|
|
||||||
--out-root) OUT_ROOT="$2"; shift 2 ;;
|
|
||||||
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
if [[ ! -x "$HELPER_BIN" ]]; then
|
|
||||||
(
|
|
||||||
cd "$ROOT_DIR"
|
|
||||||
cargo build --release --bin cir_ta_only_fixture --bin rpki --bin cir_materialize --bin cir_extract_inputs
|
|
||||||
)
|
|
||||||
fi
|
|
||||||
|
|
||||||
case_paths() {
|
|
||||||
case "$1" in
|
|
||||||
afrinic) echo "tests/fixtures/tal/afrinic.tal tests/fixtures/ta/afrinic-ta.cer" ;;
|
|
||||||
apnic) echo "tests/fixtures/tal/apnic-rfc7730-https.tal tests/fixtures/ta/apnic-ta.cer" ;;
|
|
||||||
arin) echo "tests/fixtures/tal/arin.tal tests/fixtures/ta/arin-ta.cer" ;;
|
|
||||||
lacnic) echo "tests/fixtures/tal/lacnic.tal tests/fixtures/ta/lacnic-ta.cer" ;;
|
|
||||||
ripe) echo "tests/fixtures/tal/ripe-ncc.tal tests/fixtures/ta/ripe-ncc-ta.cer" ;;
|
|
||||||
*) return 1 ;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
|
|
||||||
mkdir -p "$OUT_ROOT"
|
|
||||||
IFS=',' read -r -a ITEMS <<< "$RIRS"
|
|
||||||
for rir in "${ITEMS[@]}"; do
|
|
||||||
read -r tal_rel ta_rel < <(case_paths "$rir")
|
|
||||||
rir_root="$OUT_ROOT/$rir"
|
|
||||||
mkdir -p "$rir_root/full" "$rir_root/static"
|
|
||||||
"$HELPER_BIN" \
|
|
||||||
--tal-path "$ROOT_DIR/$tal_rel" \
|
|
||||||
--ta-path "$ROOT_DIR/$ta_rel" \
|
|
||||||
--tal-uri "https://example.test/$rir.tal" \
|
|
||||||
--validation-time "2026-04-09T00:00:00Z" \
|
|
||||||
--cir-out "$rir_root/full/input.cir" \
|
|
||||||
--static-root "$rir_root/static"
|
|
||||||
"$EXTRACT_BIN" --cir "$rir_root/full/input.cir" --tals-dir "$rir_root/.tmp/tals" --meta-json "$rir_root/.tmp/meta.json"
|
|
||||||
"$MATERIALIZE_BIN" --cir "$rir_root/full/input.cir" --static-root "$rir_root/static" --mirror-root "$rir_root/.tmp/mirror"
|
|
||||||
FIRST_TAL="$(python3 - <<'PY' "$rir_root/.tmp/meta.json"
|
|
||||||
import json,sys
|
|
||||||
print(json.load(open(sys.argv[1]))["talFiles"][0]["path"])
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
export CIR_MIRROR_ROOT="$rir_root/.tmp/mirror"
|
|
||||||
export REAL_RSYNC_BIN=/usr/bin/rsync
|
|
||||||
export CIR_LOCAL_LINK_MODE=1
|
|
||||||
"$RPKI_BIN" \
|
|
||||||
--db "$rir_root/full/db" \
|
|
||||||
--tal-path "$FIRST_TAL" \
|
|
||||||
--disable-rrdp \
|
|
||||||
--rsync-command "$WRAPPER" \
|
|
||||||
--validation-time "2026-04-09T00:00:00Z" \
|
|
||||||
--ccr-out "$rir_root/full/result.ccr" \
|
|
||||||
--report-json "$rir_root/full/report.json" >/dev/null 2>&1
|
|
||||||
for idx in $(seq 1 "$DELTA_COUNT"); do
|
|
||||||
step="$(printf 'delta-%03d' "$idx")"
|
|
||||||
mkdir -p "$rir_root/$step"
|
|
||||||
cp "$rir_root/full/input.cir" "$rir_root/$step/input.cir"
|
|
||||||
cp "$rir_root/full/result.ccr" "$rir_root/$step/result.ccr"
|
|
||||||
cp "$rir_root/full/report.json" "$rir_root/$step/report.json"
|
|
||||||
done
|
|
||||||
python3 - <<'PY' "$rir_root" "$DELTA_COUNT"
|
|
||||||
import json, sys
|
|
||||||
from pathlib import Path
|
|
||||||
root = Path(sys.argv[1]); delta_count = int(sys.argv[2])
|
|
||||||
steps = [{"stepId":"full","kind":"full","validationTime":"2026-04-09T00:00:00Z","cirPath":"full/input.cir","ccrPath":"full/result.ccr","reportPath":"full/report.json","previousStepId":None}]
|
|
||||||
prev = "full"
|
|
||||||
for i in range(1, delta_count + 1):
|
|
||||||
step = f"delta-{i:03d}"
|
|
||||||
steps.append({"stepId":step,"kind":"delta","validationTime":"2026-04-09T00:00:00Z","cirPath":f"{step}/input.cir","ccrPath":f"{step}/result.ccr","reportPath":f"{step}/report.json","previousStepId":prev})
|
|
||||||
prev = step
|
|
||||||
(root/"sequence.json").write_text(json.dumps({"version":1,"staticRoot":"static","steps":steps}, indent=2), encoding="utf-8")
|
|
||||||
(root/"summary.json").write_text(json.dumps({"version":1,"stepCount":len(steps)}, indent=2), encoding="utf-8")
|
|
||||||
PY
|
|
||||||
done
|
|
||||||
|
|
||||||
python3 - <<'PY' "$OUT_ROOT" "$RIRS"
|
|
||||||
import json, sys
|
|
||||||
from pathlib import Path
|
|
||||||
root = Path(sys.argv[1]); rirs = [x for x in sys.argv[2].split(',') if x]
|
|
||||||
items=[]
|
|
||||||
for rir in rirs:
|
|
||||||
seq=json.loads((root/rir/'sequence.json').read_text())
|
|
||||||
items.append({"rir":rir,"stepCount":len(seq['steps'])})
|
|
||||||
(root/'summary.json').write_text(json.dumps({"version":1,"rirs":items}, indent=2), encoding='utf-8')
|
|
||||||
PY
|
|
||||||
|
|
||||||
echo "done: $OUT_ROOT"
|
|
||||||
@@ -1,49 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage:
|
|
||||||
./scripts/cir/run_cir_record_sequence_ta_only_remote_multi_rir.sh \
|
|
||||||
--remote-root <path> \
|
|
||||||
[--ssh-target <user@host>] \
|
|
||||||
[--rir <afrinic,apnic,arin,lacnic,ripe>] \
|
|
||||||
[--delta-count <n>]
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
SSH_TARGET="${SSH_TARGET:-root@47.77.183.68}"
|
|
||||||
REMOTE_ROOT=""
|
|
||||||
RIRS="afrinic,apnic,arin,lacnic,ripe"
|
|
||||||
DELTA_COUNT=2
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--remote-root) REMOTE_ROOT="$2"; shift 2 ;;
|
|
||||||
--ssh-target) SSH_TARGET="$2"; shift 2 ;;
|
|
||||||
--rir) RIRS="$2"; shift 2 ;;
|
|
||||||
--delta-count) DELTA_COUNT="$2"; shift 2 ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
[[ -n "$REMOTE_ROOT" ]] || { usage >&2; exit 2; }
|
|
||||||
|
|
||||||
rsync -a --delete \
|
|
||||||
--exclude target \
|
|
||||||
--exclude .git \
|
|
||||||
"$ROOT_DIR/" "$SSH_TARGET:$REMOTE_ROOT/"
|
|
||||||
ssh "$SSH_TARGET" "mkdir -p '$REMOTE_ROOT/target/release'"
|
|
||||||
for bin in rpki cir_ta_only_fixture cir_materialize cir_extract_inputs; do
|
|
||||||
rsync -a "$ROOT_DIR/target/release/$bin" "$SSH_TARGET:$REMOTE_ROOT/target/release/"
|
|
||||||
done
|
|
||||||
|
|
||||||
ssh "$SSH_TARGET" "bash -lc '
|
|
||||||
set -euo pipefail
|
|
||||||
cd $REMOTE_ROOT
|
|
||||||
OUT=target/replay/cir_sequence_remote_ta_only_\$(date -u +%Y%m%dT%H%M%SZ)
|
|
||||||
./scripts/cir/run_cir_record_sequence_ta_only_multi_rir.sh --rir $RIRS --delta-count $DELTA_COUNT --out-root \"\$OUT\"
|
|
||||||
echo \"\$OUT\"
|
|
||||||
'"
|
|
||||||
@@ -1,286 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage:
|
|
||||||
./scripts/cir/run_cir_replay_matrix.sh \
|
|
||||||
--cir <path> \
|
|
||||||
--static-root <path> \
|
|
||||||
--out-dir <path> \
|
|
||||||
--reference-ccr <path> \
|
|
||||||
--rpki-client-build-dir <path> \
|
|
||||||
[--keep-db] \
|
|
||||||
[--rpki-bin <path>] \
|
|
||||||
[--routinator-root <path>] \
|
|
||||||
[--routinator-bin <path>] \
|
|
||||||
[--real-rsync-bin <path>]
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
|
|
||||||
CIR=""
|
|
||||||
STATIC_ROOT=""
|
|
||||||
OUT_DIR=""
|
|
||||||
REFERENCE_CCR=""
|
|
||||||
RPKI_CLIENT_BUILD_DIR=""
|
|
||||||
KEEP_DB=0
|
|
||||||
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
|
||||||
ROUTINATOR_ROOT="${ROUTINATOR_ROOT:-/home/yuyr/dev/rust_playground/routinator}"
|
|
||||||
ROUTINATOR_BIN="${ROUTINATOR_BIN:-$ROUTINATOR_ROOT/target/debug/routinator}"
|
|
||||||
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
|
||||||
OURS_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_ours.sh"
|
|
||||||
ROUTINATOR_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_routinator.sh"
|
|
||||||
RPKI_CLIENT_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_rpki_client.sh"
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--cir) CIR="$2"; shift 2 ;;
|
|
||||||
--static-root) STATIC_ROOT="$2"; shift 2 ;;
|
|
||||||
--out-dir) OUT_DIR="$2"; shift 2 ;;
|
|
||||||
--reference-ccr) REFERENCE_CCR="$2"; shift 2 ;;
|
|
||||||
--rpki-client-build-dir) RPKI_CLIENT_BUILD_DIR="$2"; shift 2 ;;
|
|
||||||
--keep-db) KEEP_DB=1; shift ;;
|
|
||||||
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
|
||||||
--routinator-root) ROUTINATOR_ROOT="$2"; shift 2 ;;
|
|
||||||
--routinator-bin) ROUTINATOR_BIN="$2"; shift 2 ;;
|
|
||||||
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
[[ -n "$CIR" && -n "$STATIC_ROOT" && -n "$OUT_DIR" && -n "$REFERENCE_CCR" && -n "$RPKI_CLIENT_BUILD_DIR" ]] || {
|
|
||||||
usage >&2
|
|
||||||
exit 2
|
|
||||||
}
|
|
||||||
|
|
||||||
mkdir -p "$OUT_DIR"
|
|
||||||
|
|
||||||
run_with_timing() {
|
|
||||||
local summary_path="$1"
|
|
||||||
local timing_path="$2"
|
|
||||||
shift 2
|
|
||||||
local start end status
|
|
||||||
start="$(python3 - <<'PY'
|
|
||||||
import time
|
|
||||||
print(time.perf_counter_ns())
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
if "$@"; then
|
|
||||||
status=0
|
|
||||||
else
|
|
||||||
status=$?
|
|
||||||
fi
|
|
||||||
end="$(python3 - <<'PY'
|
|
||||||
import time
|
|
||||||
print(time.perf_counter_ns())
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
python3 - <<'PY' "$summary_path" "$timing_path" "$status" "$start" "$end"
|
|
||||||
import json, sys
|
|
||||||
summary_path, timing_path, status, start, end = sys.argv[1:]
|
|
||||||
duration_ms = max(0, (int(end) - int(start)) // 1_000_000)
|
|
||||||
data = {"exitCode": int(status), "durationMs": duration_ms}
|
|
||||||
try:
|
|
||||||
with open(summary_path, "r", encoding="utf-8") as f:
|
|
||||||
data["compare"] = json.load(f)
|
|
||||||
except FileNotFoundError:
|
|
||||||
data["compare"] = None
|
|
||||||
with open(timing_path, "w", encoding="utf-8") as f:
|
|
||||||
json.dump(data, f, indent=2)
|
|
||||||
PY
|
|
||||||
return "$status"
|
|
||||||
}
|
|
||||||
|
|
||||||
OURS_OUT="$OUT_DIR/ours"
|
|
||||||
ROUTINATOR_OUT="$OUT_DIR/routinator"
|
|
||||||
RPKI_CLIENT_OUT="$OUT_DIR/rpki-client"
|
|
||||||
mkdir -p "$OURS_OUT" "$ROUTINATOR_OUT" "$RPKI_CLIENT_OUT"
|
|
||||||
|
|
||||||
ours_cmd=(
|
|
||||||
"$OURS_SCRIPT"
|
|
||||||
--cir "$CIR"
|
|
||||||
--static-root "$STATIC_ROOT"
|
|
||||||
--out-dir "$OURS_OUT"
|
|
||||||
--reference-ccr "$REFERENCE_CCR"
|
|
||||||
--rpki-bin "$RPKI_BIN"
|
|
||||||
--real-rsync-bin "$REAL_RSYNC_BIN"
|
|
||||||
)
|
|
||||||
routinator_cmd=(
|
|
||||||
"$ROUTINATOR_SCRIPT"
|
|
||||||
--cir "$CIR"
|
|
||||||
--static-root "$STATIC_ROOT"
|
|
||||||
--out-dir "$ROUTINATOR_OUT"
|
|
||||||
--reference-ccr "$REFERENCE_CCR"
|
|
||||||
--routinator-root "$ROUTINATOR_ROOT"
|
|
||||||
--routinator-bin "$ROUTINATOR_BIN"
|
|
||||||
--real-rsync-bin "$REAL_RSYNC_BIN"
|
|
||||||
)
|
|
||||||
rpki_client_cmd=(
|
|
||||||
"$RPKI_CLIENT_SCRIPT"
|
|
||||||
--cir "$CIR"
|
|
||||||
--static-root "$STATIC_ROOT"
|
|
||||||
--out-dir "$RPKI_CLIENT_OUT"
|
|
||||||
--reference-ccr "$REFERENCE_CCR"
|
|
||||||
--build-dir "$RPKI_CLIENT_BUILD_DIR"
|
|
||||||
--real-rsync-bin "$REAL_RSYNC_BIN"
|
|
||||||
)
|
|
||||||
|
|
||||||
if [[ "$KEEP_DB" -eq 1 ]]; then
|
|
||||||
ours_cmd+=(--keep-db)
|
|
||||||
routinator_cmd+=(--keep-db)
|
|
||||||
rpki_client_cmd+=(--keep-db)
|
|
||||||
fi
|
|
||||||
|
|
||||||
ours_status=0
|
|
||||||
routinator_status=0
|
|
||||||
rpki_client_status=0
|
|
||||||
if run_with_timing "$OURS_OUT/compare-summary.json" "$OURS_OUT/timing.json" "${ours_cmd[@]}"; then
|
|
||||||
:
|
|
||||||
else
|
|
||||||
ours_status=$?
|
|
||||||
fi
|
|
||||||
if run_with_timing "$ROUTINATOR_OUT/compare-summary.json" "$ROUTINATOR_OUT/timing.json" "${routinator_cmd[@]}"; then
|
|
||||||
:
|
|
||||||
else
|
|
||||||
routinator_status=$?
|
|
||||||
fi
|
|
||||||
if run_with_timing "$RPKI_CLIENT_OUT/compare-summary.json" "$RPKI_CLIENT_OUT/timing.json" "${rpki_client_cmd[@]}"; then
|
|
||||||
:
|
|
||||||
else
|
|
||||||
rpki_client_status=$?
|
|
||||||
fi
|
|
||||||
|
|
||||||
SUMMARY_JSON="$OUT_DIR/summary.json"
|
|
||||||
SUMMARY_MD="$OUT_DIR/summary.md"
|
|
||||||
DETAIL_MD="$OUT_DIR/detail.md"
|
|
||||||
|
|
||||||
python3 - <<'PY' \
|
|
||||||
"$CIR" \
|
|
||||||
"$STATIC_ROOT" \
|
|
||||||
"$REFERENCE_CCR" \
|
|
||||||
"$OURS_OUT" \
|
|
||||||
"$ROUTINATOR_OUT" \
|
|
||||||
"$RPKI_CLIENT_OUT" \
|
|
||||||
"$SUMMARY_JSON" \
|
|
||||||
"$SUMMARY_MD" \
|
|
||||||
"$DETAIL_MD"
|
|
||||||
import json
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
cir_path, static_root, reference_ccr, ours_out, routinator_out, rpki_client_out, summary_json, summary_md, detail_md = sys.argv[1:]
|
|
||||||
|
|
||||||
participants = []
|
|
||||||
all_match = True
|
|
||||||
for name, out_dir in [
|
|
||||||
("ours", ours_out),
|
|
||||||
("routinator", routinator_out),
|
|
||||||
("rpki-client", rpki_client_out),
|
|
||||||
]:
|
|
||||||
out = Path(out_dir)
|
|
||||||
timing = json.loads((out / "timing.json").read_text(encoding="utf-8"))
|
|
||||||
compare = timing.get("compare") or {}
|
|
||||||
vrps = compare.get("vrps") or {}
|
|
||||||
vaps = compare.get("vaps") or {}
|
|
||||||
participant = {
|
|
||||||
"name": name,
|
|
||||||
"outDir": str(out),
|
|
||||||
"tmpRoot": str(out / ".tmp"),
|
|
||||||
"mirrorPath": str(out / ".tmp" / "mirror"),
|
|
||||||
"timingPath": str(out / "timing.json"),
|
|
||||||
"summaryPath": str(out / "compare-summary.json"),
|
|
||||||
"exitCode": timing["exitCode"],
|
|
||||||
"durationMs": timing["durationMs"],
|
|
||||||
"vrps": vrps,
|
|
||||||
"vaps": vaps,
|
|
||||||
"match": bool(vrps.get("match")) and bool(vaps.get("match")) and timing["exitCode"] == 0,
|
|
||||||
"logPaths": [str(path) for path in sorted(out.glob("*.log"))],
|
|
||||||
}
|
|
||||||
participants.append(participant)
|
|
||||||
all_match = all_match and participant["match"]
|
|
||||||
|
|
||||||
summary = {
|
|
||||||
"cirPath": cir_path,
|
|
||||||
"staticRoot": static_root,
|
|
||||||
"referenceCcr": reference_ccr,
|
|
||||||
"participants": participants,
|
|
||||||
"allMatch": all_match,
|
|
||||||
}
|
|
||||||
Path(summary_json).write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
|
||||||
|
|
||||||
lines = [
|
|
||||||
"# CIR Replay Matrix Summary",
|
|
||||||
"",
|
|
||||||
f"- `cir`: `{cir_path}`",
|
|
||||||
f"- `static_root`: `{static_root}`",
|
|
||||||
f"- `reference_ccr`: `{reference_ccr}`",
|
|
||||||
f"- `all_match`: `{all_match}`",
|
|
||||||
"",
|
|
||||||
"| Participant | Exit | Duration (ms) | VRP actual/ref | VRP match | VAP actual/ref | VAP match | Log |",
|
|
||||||
"| --- | ---: | ---: | --- | --- | --- | --- | --- |",
|
|
||||||
]
|
|
||||||
for participant in participants:
|
|
||||||
vrps = participant["vrps"] or {}
|
|
||||||
vaps = participant["vaps"] or {}
|
|
||||||
log_path = participant["logPaths"][0] if participant["logPaths"] else ""
|
|
||||||
lines.append(
|
|
||||||
"| {name} | {exit_code} | {duration_ms} | {vrp_actual}/{vrp_ref} | {vrp_match} | {vap_actual}/{vap_ref} | {vap_match} | `{log_path}` |".format(
|
|
||||||
name=participant["name"],
|
|
||||||
exit_code=participant["exitCode"],
|
|
||||||
duration_ms=participant["durationMs"],
|
|
||||||
vrp_actual=vrps.get("actual", "-"),
|
|
||||||
vrp_ref=vrps.get("reference", "-"),
|
|
||||||
vrp_match=vrps.get("match", False),
|
|
||||||
vap_actual=vaps.get("actual", "-"),
|
|
||||||
vap_ref=vaps.get("reference", "-"),
|
|
||||||
vap_match=vaps.get("match", False),
|
|
||||||
log_path=log_path,
|
|
||||||
)
|
|
||||||
)
|
|
||||||
Path(summary_md).write_text("\n".join(lines) + "\n", encoding="utf-8")
|
|
||||||
|
|
||||||
detail_lines = [
|
|
||||||
"# CIR Replay Matrix Detail",
|
|
||||||
"",
|
|
||||||
]
|
|
||||||
for participant in participants:
|
|
||||||
vrps = participant["vrps"] or {}
|
|
||||||
vaps = participant["vaps"] or {}
|
|
||||||
detail_lines.extend([
|
|
||||||
f"## {participant['name']}",
|
|
||||||
f"- `exit_code`: `{participant['exitCode']}`",
|
|
||||||
f"- `duration_ms`: `{participant['durationMs']}`",
|
|
||||||
f"- `out_dir`: `{participant['outDir']}`",
|
|
||||||
f"- `tmp_root`: `{participant['tmpRoot']}`",
|
|
||||||
f"- `mirror_path`: `{participant['mirrorPath']}`",
|
|
||||||
f"- `summary_path`: `{participant['summaryPath']}`",
|
|
||||||
f"- `timing_path`: `{participant['timingPath']}`",
|
|
||||||
f"- `log_paths`: `{', '.join(participant['logPaths'])}`",
|
|
||||||
f"- `vrps`: `actual={vrps.get('actual', '-')}` `reference={vrps.get('reference', '-')}` `match={vrps.get('match', False)}`",
|
|
||||||
f"- `vaps`: `actual={vaps.get('actual', '-')}` `reference={vaps.get('reference', '-')}` `match={vaps.get('match', False)}`",
|
|
||||||
f"- `vrps.only_in_actual`: `{vrps.get('only_in_actual', [])}`",
|
|
||||||
f"- `vrps.only_in_reference`: `{vrps.get('only_in_reference', [])}`",
|
|
||||||
f"- `vaps.only_in_actual`: `{vaps.get('only_in_actual', [])}`",
|
|
||||||
f"- `vaps.only_in_reference`: `{vaps.get('only_in_reference', [])}`",
|
|
||||||
"",
|
|
||||||
])
|
|
||||||
Path(detail_md).write_text("\n".join(detail_lines), encoding="utf-8")
|
|
||||||
PY
|
|
||||||
|
|
||||||
if [[ "$ours_status" -ne 0 || "$routinator_status" -ne 0 || "$rpki_client_status" -ne 0 ]]; then
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
all_match="$(python3 - <<'PY' "$SUMMARY_JSON"
|
|
||||||
import json,sys
|
|
||||||
print("true" if json.load(open(sys.argv[1]))["allMatch"] else "false")
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
if [[ "$all_match" != "true" ]]; then
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
echo "done: $OUT_DIR"
|
|
||||||
@@ -1,162 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage:
|
|
||||||
./scripts/cir/run_cir_replay_ours.sh \
|
|
||||||
--cir <path> \
|
|
||||||
[--static-root <path> | --raw-store-db <path>] \
|
|
||||||
--out-dir <path> \
|
|
||||||
--reference-ccr <path> \
|
|
||||||
[--keep-db] \
|
|
||||||
[--rpki-bin <path>] \
|
|
||||||
[--real-rsync-bin <path>]
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
|
|
||||||
CIR=""
|
|
||||||
STATIC_ROOT=""
|
|
||||||
POOL_DB=""
|
|
||||||
RAW_STORE_DB=""
|
|
||||||
OUT_DIR=""
|
|
||||||
REFERENCE_CCR=""
|
|
||||||
KEEP_DB=0
|
|
||||||
RPKI_BIN="$ROOT_DIR/target/release/rpki"
|
|
||||||
CIR_MATERIALIZE_BIN="$ROOT_DIR/target/release/cir_materialize"
|
|
||||||
CIR_EXTRACT_INPUTS_BIN="$ROOT_DIR/target/release/cir_extract_inputs"
|
|
||||||
CCR_TO_COMPARE_VIEWS_BIN="$ROOT_DIR/target/release/ccr_to_compare_views"
|
|
||||||
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
|
||||||
WRAPPER="$ROOT_DIR/scripts/cir/cir-rsync-wrapper"
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--cir) CIR="$2"; shift 2 ;;
|
|
||||||
--static-root) STATIC_ROOT="$2"; shift 2 ;;
|
|
||||||
--raw-store-db) RAW_STORE_DB="$2"; shift 2 ;;
|
|
||||||
--out-dir) OUT_DIR="$2"; shift 2 ;;
|
|
||||||
--reference-ccr) REFERENCE_CCR="$2"; shift 2 ;;
|
|
||||||
--keep-db) KEEP_DB=1; shift ;;
|
|
||||||
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
|
||||||
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
[[ -n "$CIR" && -n "$OUT_DIR" && -n "$REFERENCE_CCR" ]] || {
|
|
||||||
usage >&2
|
|
||||||
exit 2
|
|
||||||
}
|
|
||||||
backend_count=0
|
|
||||||
[[ -n "$STATIC_ROOT" ]] && backend_count=$((backend_count+1))
|
|
||||||
[[ -n "$RAW_STORE_DB" ]] && backend_count=$((backend_count+1))
|
|
||||||
[[ "$backend_count" -eq 1 ]] || { usage >&2; exit 2; }
|
|
||||||
|
|
||||||
mkdir -p "$OUT_DIR"
|
|
||||||
|
|
||||||
if [[ ! -x "$RPKI_BIN" || ! -x "$CIR_MATERIALIZE_BIN" || ! -x "$CIR_EXTRACT_INPUTS_BIN" || ! -x "$CCR_TO_COMPARE_VIEWS_BIN" ]]; then
|
|
||||||
(
|
|
||||||
cd "$ROOT_DIR"
|
|
||||||
cargo build --release --bin rpki --bin cir_materialize --bin cir_extract_inputs --bin ccr_to_compare_views
|
|
||||||
)
|
|
||||||
fi
|
|
||||||
|
|
||||||
TMP_ROOT="$OUT_DIR/.tmp"
|
|
||||||
TALS_DIR="$TMP_ROOT/tals"
|
|
||||||
META_JSON="$TMP_ROOT/meta.json"
|
|
||||||
MIRROR_ROOT="$TMP_ROOT/mirror"
|
|
||||||
DB_DIR="$TMP_ROOT/work-db"
|
|
||||||
ACTUAL_CCR="$OUT_DIR/actual.ccr"
|
|
||||||
ACTUAL_REPORT="$OUT_DIR/report.json"
|
|
||||||
ACTUAL_VRPS="$OUT_DIR/actual-vrps.csv"
|
|
||||||
ACTUAL_VAPS="$OUT_DIR/actual-vaps.csv"
|
|
||||||
REF_VRPS="$OUT_DIR/reference-vrps.csv"
|
|
||||||
REF_VAPS="$OUT_DIR/reference-vaps.csv"
|
|
||||||
COMPARE_JSON="$OUT_DIR/compare-summary.json"
|
|
||||||
RUN_LOG="$OUT_DIR/run.log"
|
|
||||||
|
|
||||||
rm -rf "$TMP_ROOT"
|
|
||||||
mkdir -p "$TMP_ROOT"
|
|
||||||
|
|
||||||
"$CIR_EXTRACT_INPUTS_BIN" --cir "$CIR" --tals-dir "$TALS_DIR" --meta-json "$META_JSON"
|
|
||||||
materialize_cmd=("$CIR_MATERIALIZE_BIN" --cir "$CIR" --mirror-root "$MIRROR_ROOT")
|
|
||||||
if [[ -n "$STATIC_ROOT" ]]; then
|
|
||||||
materialize_cmd+=(--static-root "$STATIC_ROOT")
|
|
||||||
else
|
|
||||||
materialize_cmd+=(--raw-store-db "$RAW_STORE_DB")
|
|
||||||
fi
|
|
||||||
if [[ "$KEEP_DB" -eq 1 ]]; then
|
|
||||||
materialize_cmd+=(--keep-db)
|
|
||||||
fi
|
|
||||||
"${materialize_cmd[@]}"
|
|
||||||
|
|
||||||
VALIDATION_TIME="$(python3 - <<'PY' "$META_JSON"
|
|
||||||
import json,sys
|
|
||||||
print(json.load(open(sys.argv[1]))["validationTime"])
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
FIRST_TAL="$(python3 - <<'PY' "$META_JSON"
|
|
||||||
import json,sys
|
|
||||||
print(json.load(open(sys.argv[1]))["talFiles"][0]["path"])
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
|
|
||||||
export CIR_MIRROR_ROOT="$(python3 - <<'PY' "$MIRROR_ROOT"
|
|
||||||
from pathlib import Path
|
|
||||||
import sys
|
|
||||||
print(Path(sys.argv[1]).resolve())
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
export REAL_RSYNC_BIN="$REAL_RSYNC_BIN"
|
|
||||||
export CIR_LOCAL_LINK_MODE=1
|
|
||||||
|
|
||||||
"$RPKI_BIN" \
|
|
||||||
--db "$DB_DIR" \
|
|
||||||
--tal-path "$FIRST_TAL" \
|
|
||||||
--disable-rrdp \
|
|
||||||
--rsync-command "$WRAPPER" \
|
|
||||||
--validation-time "$VALIDATION_TIME" \
|
|
||||||
--ccr-out "$ACTUAL_CCR" \
|
|
||||||
--report-json "$ACTUAL_REPORT" \
|
|
||||||
>"$RUN_LOG" 2>&1
|
|
||||||
|
|
||||||
"$CCR_TO_COMPARE_VIEWS_BIN" --ccr "$ACTUAL_CCR" --vrps-out "$ACTUAL_VRPS" --vaps-out "$ACTUAL_VAPS" --trust-anchor unknown
|
|
||||||
"$CCR_TO_COMPARE_VIEWS_BIN" --ccr "$REFERENCE_CCR" --vrps-out "$REF_VRPS" --vaps-out "$REF_VAPS" --trust-anchor unknown
|
|
||||||
|
|
||||||
python3 - <<'PY' "$ACTUAL_VRPS" "$REF_VRPS" "$ACTUAL_VAPS" "$REF_VAPS" "$COMPARE_JSON"
|
|
||||||
import csv, json, sys
|
|
||||||
def rows(path):
|
|
||||||
with open(path, newline="") as f:
|
|
||||||
return list(csv.reader(f))[1:]
|
|
||||||
actual_vrps = {tuple(r) for r in rows(sys.argv[1])}
|
|
||||||
ref_vrps = {tuple(r) for r in rows(sys.argv[2])}
|
|
||||||
actual_vaps = {tuple(r) for r in rows(sys.argv[3])}
|
|
||||||
ref_vaps = {tuple(r) for r in rows(sys.argv[4])}
|
|
||||||
summary = {
|
|
||||||
"vrps": {
|
|
||||||
"actual": len(actual_vrps),
|
|
||||||
"reference": len(ref_vrps),
|
|
||||||
"only_in_actual": sorted(actual_vrps - ref_vrps)[:20],
|
|
||||||
"only_in_reference": sorted(ref_vrps - actual_vrps)[:20],
|
|
||||||
"match": actual_vrps == ref_vrps,
|
|
||||||
},
|
|
||||||
"vaps": {
|
|
||||||
"actual": len(actual_vaps),
|
|
||||||
"reference": len(ref_vaps),
|
|
||||||
"only_in_actual": sorted(actual_vaps - ref_vaps)[:20],
|
|
||||||
"only_in_reference": sorted(ref_vaps - actual_vaps)[:20],
|
|
||||||
"match": actual_vaps == ref_vaps,
|
|
||||||
}
|
|
||||||
}
|
|
||||||
with open(sys.argv[5], "w") as f:
|
|
||||||
json.dump(summary, f, indent=2)
|
|
||||||
PY
|
|
||||||
|
|
||||||
if [[ "$KEEP_DB" -ne 1 ]]; then
|
|
||||||
rm -rf "$TMP_ROOT"
|
|
||||||
fi
|
|
||||||
|
|
||||||
echo "done: $OUT_DIR"
|
|
||||||
@@ -1,221 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage:
|
|
||||||
./scripts/cir/run_cir_replay_routinator.sh \
|
|
||||||
--cir <path> \
|
|
||||||
[--static-root <path> | --raw-store-db <path>] \
|
|
||||||
--out-dir <path> \
|
|
||||||
--reference-ccr <path> \
|
|
||||||
[--keep-db] \
|
|
||||||
[--routinator-root <path>] \
|
|
||||||
[--routinator-bin <path>] \
|
|
||||||
[--real-rsync-bin <path>]
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
RPKI_DEV_ROOT="${RPKI_DEV_ROOT:-$ROOT_DIR}"
|
|
||||||
|
|
||||||
CIR=""
|
|
||||||
STATIC_ROOT=""
|
|
||||||
POOL_DB=""
|
|
||||||
RAW_STORE_DB=""
|
|
||||||
OUT_DIR=""
|
|
||||||
REFERENCE_CCR=""
|
|
||||||
KEEP_DB=0
|
|
||||||
ROUTINATOR_ROOT="${ROUTINATOR_ROOT:-/home/yuyr/dev/rust_playground/routinator}"
|
|
||||||
ROUTINATOR_BIN="${ROUTINATOR_BIN:-$ROUTINATOR_ROOT/target/debug/routinator}"
|
|
||||||
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
|
||||||
CIR_MATERIALIZE_BIN="$ROOT_DIR/target/release/cir_materialize"
|
|
||||||
CIR_EXTRACT_INPUTS_BIN="$ROOT_DIR/target/release/cir_extract_inputs"
|
|
||||||
CCR_TO_COMPARE_VIEWS_BIN="$ROOT_DIR/target/release/ccr_to_compare_views"
|
|
||||||
WRAPPER="$ROOT_DIR/scripts/cir/cir-rsync-wrapper"
|
|
||||||
JSON_TO_VAPS="$ROOT_DIR/scripts/cir/json_to_vaps_csv.py"
|
|
||||||
FAKETIME_LIB="${FAKETIME_LIB:-$ROOT_DIR/target/tools/faketime_pkg/extracted/libfaketime/usr/lib/x86_64-linux-gnu/faketime/libfaketime.so.1}"
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--cir) CIR="$2"; shift 2 ;;
|
|
||||||
--static-root) STATIC_ROOT="$2"; shift 2 ;;
|
|
||||||
--raw-store-db) RAW_STORE_DB="$2"; shift 2 ;;
|
|
||||||
--out-dir) OUT_DIR="$2"; shift 2 ;;
|
|
||||||
--reference-ccr) REFERENCE_CCR="$2"; shift 2 ;;
|
|
||||||
--keep-db) KEEP_DB=1; shift ;;
|
|
||||||
--routinator-root) ROUTINATOR_ROOT="$2"; shift 2 ;;
|
|
||||||
--routinator-bin) ROUTINATOR_BIN="$2"; shift 2 ;;
|
|
||||||
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
[[ -n "$CIR" && -n "$OUT_DIR" && -n "$REFERENCE_CCR" ]] || {
|
|
||||||
usage >&2
|
|
||||||
exit 2
|
|
||||||
}
|
|
||||||
backend_count=0
|
|
||||||
[[ -n "$STATIC_ROOT" ]] && backend_count=$((backend_count+1))
|
|
||||||
[[ -n "$RAW_STORE_DB" ]] && backend_count=$((backend_count+1))
|
|
||||||
[[ "$backend_count" -eq 1 ]] || { usage >&2; exit 2; }
|
|
||||||
|
|
||||||
mkdir -p "$OUT_DIR"
|
|
||||||
if [[ ! -x "$CIR_MATERIALIZE_BIN" || ! -x "$CIR_EXTRACT_INPUTS_BIN" || ! -x "$CCR_TO_COMPARE_VIEWS_BIN" ]]; then
|
|
||||||
(
|
|
||||||
cd "$ROOT_DIR"
|
|
||||||
cargo build --release --bin cir_materialize --bin cir_extract_inputs --bin ccr_to_compare_views
|
|
||||||
)
|
|
||||||
fi
|
|
||||||
|
|
||||||
TMP_ROOT="$OUT_DIR/.tmp"
|
|
||||||
TALS_DIR="$TMP_ROOT/tals"
|
|
||||||
META_JSON="$TMP_ROOT/meta.json"
|
|
||||||
MIRROR_ROOT="$TMP_ROOT/mirror"
|
|
||||||
WORK_REPO="$TMP_ROOT/repository"
|
|
||||||
RUN_LOG="$OUT_DIR/routinator.log"
|
|
||||||
ACTUAL_VRPS="$OUT_DIR/actual-vrps.csv"
|
|
||||||
ACTUAL_VAPS_JSON="$OUT_DIR/actual-vaps.json"
|
|
||||||
ACTUAL_VAPS="$OUT_DIR/actual-vaps.csv"
|
|
||||||
REF_VRPS="$OUT_DIR/reference-vrps.csv"
|
|
||||||
REF_VAPS="$OUT_DIR/reference-vaps.csv"
|
|
||||||
SUMMARY_JSON="$OUT_DIR/compare-summary.json"
|
|
||||||
|
|
||||||
rm -rf "$TMP_ROOT"
|
|
||||||
mkdir -p "$TMP_ROOT"
|
|
||||||
|
|
||||||
"$CIR_EXTRACT_INPUTS_BIN" --cir "$CIR" --tals-dir "$TALS_DIR" --meta-json "$META_JSON"
|
|
||||||
python3 - <<'PY' "$TALS_DIR"
|
|
||||||
from pathlib import Path
|
|
||||||
import sys
|
|
||||||
for tal in Path(sys.argv[1]).glob("*.tal"):
|
|
||||||
lines = tal.read_text(encoding="utf-8").splitlines()
|
|
||||||
rsync_uris = [line for line in lines if line.startswith("rsync://")]
|
|
||||||
base64_lines = []
|
|
||||||
seen_sep = False
|
|
||||||
for line in lines:
|
|
||||||
if seen_sep:
|
|
||||||
if line.strip():
|
|
||||||
base64_lines.append(line)
|
|
||||||
elif line.strip() == "":
|
|
||||||
seen_sep = True
|
|
||||||
tal.write_text("\n".join(rsync_uris) + "\n\n" + "\n".join(base64_lines) + "\n", encoding="utf-8")
|
|
||||||
PY
|
|
||||||
materialize_cmd=("$CIR_MATERIALIZE_BIN" --cir "$CIR" --mirror-root "$MIRROR_ROOT")
|
|
||||||
if [[ -n "$STATIC_ROOT" ]]; then
|
|
||||||
materialize_cmd+=(--static-root "$STATIC_ROOT")
|
|
||||||
else
|
|
||||||
materialize_cmd+=(--raw-store-db "$RAW_STORE_DB")
|
|
||||||
fi
|
|
||||||
if [[ "$KEEP_DB" -eq 1 ]]; then
|
|
||||||
materialize_cmd+=(--keep-db)
|
|
||||||
fi
|
|
||||||
"${materialize_cmd[@]}"
|
|
||||||
|
|
||||||
VALIDATION_TIME="$(python3 - <<'PY' "$META_JSON"
|
|
||||||
import json,sys
|
|
||||||
print(json.load(open(sys.argv[1]))["validationTime"])
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
FIRST_TAL="$(python3 - <<'PY' "$META_JSON"
|
|
||||||
import json,sys
|
|
||||||
print(json.load(open(sys.argv[1]))["talFiles"][0]["path"])
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
TA_NAME="$(basename "$FIRST_TAL" .tal)"
|
|
||||||
FAKE_EPOCH="$(python3 - <<'PY' "$VALIDATION_TIME"
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
import sys
|
|
||||||
dt = datetime.fromisoformat(sys.argv[1].replace("Z", "+00:00")).astimezone(timezone.utc)
|
|
||||||
print(int(dt.timestamp()))
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
|
|
||||||
export CIR_MIRROR_ROOT="$(python3 - <<'PY' "$MIRROR_ROOT"
|
|
||||||
from pathlib import Path
|
|
||||||
import sys
|
|
||||||
print(Path(sys.argv[1]).resolve())
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
export REAL_RSYNC_BIN="$REAL_RSYNC_BIN"
|
|
||||||
export CIR_LOCAL_LINK_MODE=1
|
|
||||||
env \
|
|
||||||
LD_PRELOAD="$FAKETIME_LIB" \
|
|
||||||
FAKETIME_FMT=%s \
|
|
||||||
FAKETIME="$FAKE_EPOCH" \
|
|
||||||
FAKETIME_DONT_FAKE_MONOTONIC=1 \
|
|
||||||
"$ROUTINATOR_BIN" \
|
|
||||||
--repository-dir "$WORK_REPO" \
|
|
||||||
--disable-rrdp \
|
|
||||||
--rsync-command "$WRAPPER" \
|
|
||||||
--no-rir-tals \
|
|
||||||
--extra-tals-dir "$TALS_DIR" \
|
|
||||||
--enable-aspa \
|
|
||||||
update --complete >"$RUN_LOG" 2>&1 || true
|
|
||||||
|
|
||||||
env \
|
|
||||||
LD_PRELOAD="$FAKETIME_LIB" \
|
|
||||||
FAKETIME_FMT=%s \
|
|
||||||
FAKETIME="$FAKE_EPOCH" \
|
|
||||||
FAKETIME_DONT_FAKE_MONOTONIC=1 \
|
|
||||||
"$ROUTINATOR_BIN" \
|
|
||||||
--repository-dir "$WORK_REPO" \
|
|
||||||
--disable-rrdp \
|
|
||||||
--rsync-command "$WRAPPER" \
|
|
||||||
--no-rir-tals \
|
|
||||||
--extra-tals-dir "$TALS_DIR" \
|
|
||||||
--enable-aspa \
|
|
||||||
vrps --noupdate -o "$ACTUAL_VRPS" >>"$RUN_LOG" 2>&1
|
|
||||||
|
|
||||||
env \
|
|
||||||
LD_PRELOAD="$FAKETIME_LIB" \
|
|
||||||
FAKETIME_FMT=%s \
|
|
||||||
FAKETIME="$FAKE_EPOCH" \
|
|
||||||
FAKETIME_DONT_FAKE_MONOTONIC=1 \
|
|
||||||
"$ROUTINATOR_BIN" \
|
|
||||||
--repository-dir "$WORK_REPO" \
|
|
||||||
--disable-rrdp \
|
|
||||||
--rsync-command "$WRAPPER" \
|
|
||||||
--no-rir-tals \
|
|
||||||
--extra-tals-dir "$TALS_DIR" \
|
|
||||||
--enable-aspa \
|
|
||||||
vrps --noupdate --format json -o "$ACTUAL_VAPS_JSON" >>"$RUN_LOG" 2>&1
|
|
||||||
|
|
||||||
python3 "$JSON_TO_VAPS" --input "$ACTUAL_VAPS_JSON" --csv-out "$ACTUAL_VAPS"
|
|
||||||
"$CCR_TO_COMPARE_VIEWS_BIN" --ccr "$REFERENCE_CCR" --vrps-out "$REF_VRPS" --vaps-out "$REF_VAPS" --trust-anchor "$TA_NAME"
|
|
||||||
|
|
||||||
python3 - <<'PY' "$ACTUAL_VRPS" "$REF_VRPS" "$ACTUAL_VAPS" "$REF_VAPS" "$SUMMARY_JSON"
|
|
||||||
import csv, json, sys
|
|
||||||
def rows(path):
|
|
||||||
with open(path, newline="") as f:
|
|
||||||
return list(csv.reader(f))[1:]
|
|
||||||
actual_vrps = {tuple(r) for r in rows(sys.argv[1])}
|
|
||||||
ref_vrps = {tuple(r) for r in rows(sys.argv[2])}
|
|
||||||
actual_vaps = {tuple(r) for r in rows(sys.argv[3])}
|
|
||||||
ref_vaps = {tuple(r) for r in rows(sys.argv[4])}
|
|
||||||
summary = {
|
|
||||||
"vrps": {
|
|
||||||
"actual": len(actual_vrps),
|
|
||||||
"reference": len(ref_vrps),
|
|
||||||
"match": actual_vrps == ref_vrps,
|
|
||||||
"only_in_actual": sorted(actual_vrps - ref_vrps)[:20],
|
|
||||||
"only_in_reference": sorted(ref_vrps - actual_vrps)[:20],
|
|
||||||
},
|
|
||||||
"vaps": {
|
|
||||||
"actual": len(actual_vaps),
|
|
||||||
"reference": len(ref_vaps),
|
|
||||||
"match": actual_vaps == ref_vaps,
|
|
||||||
"only_in_actual": sorted(actual_vaps - ref_vaps)[:20],
|
|
||||||
"only_in_reference": sorted(ref_vaps - actual_vaps)[:20],
|
|
||||||
}
|
|
||||||
}
|
|
||||||
with open(sys.argv[5], "w") as f:
|
|
||||||
json.dump(summary, f, indent=2)
|
|
||||||
PY
|
|
||||||
|
|
||||||
if [[ "$KEEP_DB" -ne 1 ]]; then
|
|
||||||
rm -rf "$TMP_ROOT"
|
|
||||||
fi
|
|
||||||
|
|
||||||
echo "done: $OUT_DIR"
|
|
||||||
@@ -1,195 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage:
|
|
||||||
./scripts/cir/run_cir_replay_rpki_client.sh \
|
|
||||||
--cir <path> \
|
|
||||||
[--static-root <path> | --raw-store-db <path>] \
|
|
||||||
--out-dir <path> \
|
|
||||||
--reference-ccr <path> \
|
|
||||||
--build-dir <path> \
|
|
||||||
[--keep-db] \
|
|
||||||
[--real-rsync-bin <path>]
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
CIR=""
|
|
||||||
STATIC_ROOT=""
|
|
||||||
POOL_DB=""
|
|
||||||
RAW_STORE_DB=""
|
|
||||||
OUT_DIR=""
|
|
||||||
REFERENCE_CCR=""
|
|
||||||
BUILD_DIR=""
|
|
||||||
KEEP_DB=0
|
|
||||||
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
|
||||||
CIR_MATERIALIZE_BIN="$ROOT_DIR/target/release/cir_materialize"
|
|
||||||
CIR_EXTRACT_INPUTS_BIN="$ROOT_DIR/target/release/cir_extract_inputs"
|
|
||||||
CCR_TO_COMPARE_VIEWS_BIN="$ROOT_DIR/target/release/ccr_to_compare_views"
|
|
||||||
WRAPPER="$ROOT_DIR/scripts/cir/cir-rsync-wrapper"
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--cir) CIR="$2"; shift 2 ;;
|
|
||||||
--static-root) STATIC_ROOT="$2"; shift 2 ;;
|
|
||||||
--raw-store-db) RAW_STORE_DB="$2"; shift 2 ;;
|
|
||||||
--out-dir) OUT_DIR="$2"; shift 2 ;;
|
|
||||||
--reference-ccr) REFERENCE_CCR="$2"; shift 2 ;;
|
|
||||||
--build-dir) BUILD_DIR="$2"; shift 2 ;;
|
|
||||||
--keep-db) KEEP_DB=1; shift ;;
|
|
||||||
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
[[ -n "$CIR" && -n "$OUT_DIR" && -n "$REFERENCE_CCR" && -n "$BUILD_DIR" ]] || {
|
|
||||||
usage >&2
|
|
||||||
exit 2
|
|
||||||
}
|
|
||||||
backend_count=0
|
|
||||||
[[ -n "$STATIC_ROOT" ]] && backend_count=$((backend_count+1))
|
|
||||||
[[ -n "$RAW_STORE_DB" ]] && backend_count=$((backend_count+1))
|
|
||||||
[[ "$backend_count" -eq 1 ]] || { usage >&2; exit 2; }
|
|
||||||
|
|
||||||
mkdir -p "$OUT_DIR"
|
|
||||||
if [[ ! -x "$CIR_MATERIALIZE_BIN" || ! -x "$CIR_EXTRACT_INPUTS_BIN" || ! -x "$CCR_TO_COMPARE_VIEWS_BIN" ]]; then
|
|
||||||
(
|
|
||||||
cd "$ROOT_DIR"
|
|
||||||
cargo build --release --bin cir_materialize --bin cir_extract_inputs --bin ccr_to_compare_views
|
|
||||||
)
|
|
||||||
fi
|
|
||||||
|
|
||||||
TMP_ROOT="$OUT_DIR/.tmp"
|
|
||||||
TALS_DIR="$TMP_ROOT/tals"
|
|
||||||
META_JSON="$TMP_ROOT/meta.json"
|
|
||||||
MIRROR_ROOT="$TMP_ROOT/mirror"
|
|
||||||
CACHE_DIR="$TMP_ROOT/cache"
|
|
||||||
OUT_CCR_DIR="$TMP_ROOT/out"
|
|
||||||
RUN_LOG="$OUT_DIR/rpki-client.log"
|
|
||||||
ACTUAL_VRPS="$OUT_DIR/actual-vrps.csv"
|
|
||||||
ACTUAL_VAPS="$OUT_DIR/actual-vaps.csv"
|
|
||||||
ACTUAL_VAPS_META="$OUT_DIR/actual-vaps-meta.json"
|
|
||||||
ACTUAL_VRPS_META="$OUT_DIR/actual-vrps-meta.json"
|
|
||||||
REF_VRPS="$OUT_DIR/reference-vrps.csv"
|
|
||||||
REF_VAPS="$OUT_DIR/reference-vaps.csv"
|
|
||||||
SUMMARY_JSON="$OUT_DIR/compare-summary.json"
|
|
||||||
|
|
||||||
rm -rf "$TMP_ROOT"
|
|
||||||
mkdir -p "$TMP_ROOT"
|
|
||||||
|
|
||||||
"$CIR_EXTRACT_INPUTS_BIN" --cir "$CIR" --tals-dir "$TALS_DIR" --meta-json "$META_JSON"
|
|
||||||
python3 - <<'PY' "$TALS_DIR"
|
|
||||||
from pathlib import Path
|
|
||||||
import sys
|
|
||||||
for tal in Path(sys.argv[1]).glob("*.tal"):
|
|
||||||
lines = tal.read_text(encoding="utf-8").splitlines()
|
|
||||||
rsync_uris = [line for line in lines if line.startswith("rsync://")]
|
|
||||||
base64_lines = []
|
|
||||||
seen_sep = False
|
|
||||||
for line in lines:
|
|
||||||
if seen_sep:
|
|
||||||
if line.strip():
|
|
||||||
base64_lines.append(line)
|
|
||||||
elif line.strip() == "":
|
|
||||||
seen_sep = True
|
|
||||||
tal.write_text("\n".join(rsync_uris) + "\n\n" + "\n".join(base64_lines) + "\n", encoding="utf-8")
|
|
||||||
PY
|
|
||||||
materialize_cmd=("$CIR_MATERIALIZE_BIN" --cir "$CIR" --mirror-root "$MIRROR_ROOT")
|
|
||||||
if [[ -n "$STATIC_ROOT" ]]; then
|
|
||||||
materialize_cmd+=(--static-root "$STATIC_ROOT")
|
|
||||||
else
|
|
||||||
materialize_cmd+=(--raw-store-db "$RAW_STORE_DB")
|
|
||||||
fi
|
|
||||||
if [[ "$KEEP_DB" -eq 1 ]]; then
|
|
||||||
materialize_cmd+=(--keep-db)
|
|
||||||
fi
|
|
||||||
"${materialize_cmd[@]}"
|
|
||||||
|
|
||||||
VALIDATION_EPOCH="$(python3 - <<'PY' "$META_JSON"
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
import json, sys
|
|
||||||
vt = json.load(open(sys.argv[1]))["validationTime"]
|
|
||||||
dt = datetime.fromisoformat(vt.replace("Z", "+00:00")).astimezone(timezone.utc)
|
|
||||||
print(int(dt.timestamp()))
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
FIRST_TAL="$(python3 - <<'PY' "$META_JSON"
|
|
||||||
import json,sys
|
|
||||||
print(json.load(open(sys.argv[1]))["talFiles"][0]["path"])
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
TA_NAME="$(basename "$FIRST_TAL" .tal)"
|
|
||||||
|
|
||||||
export CIR_MIRROR_ROOT="$(python3 - <<'PY' "$MIRROR_ROOT"
|
|
||||||
from pathlib import Path
|
|
||||||
import sys
|
|
||||||
print(Path(sys.argv[1]).resolve())
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
export REAL_RSYNC_BIN="$REAL_RSYNC_BIN"
|
|
||||||
export CIR_LOCAL_LINK_MODE=1
|
|
||||||
|
|
||||||
mkdir -p "$CACHE_DIR" "$OUT_CCR_DIR"
|
|
||||||
"$BUILD_DIR/src/rpki-client" \
|
|
||||||
-R \
|
|
||||||
-e "$WRAPPER" \
|
|
||||||
-P "$VALIDATION_EPOCH" \
|
|
||||||
-t "$FIRST_TAL" \
|
|
||||||
-d "$CACHE_DIR" \
|
|
||||||
"$OUT_CCR_DIR" >"$RUN_LOG" 2>&1
|
|
||||||
|
|
||||||
"$CCR_TO_COMPARE_VIEWS_BIN" \
|
|
||||||
--ccr "$OUT_CCR_DIR/rpki.ccr" \
|
|
||||||
--vrps-out "$ACTUAL_VRPS" \
|
|
||||||
--vaps-out "$ACTUAL_VAPS" \
|
|
||||||
--trust-anchor "$TA_NAME"
|
|
||||||
|
|
||||||
python3 - <<'PY' "$ACTUAL_VRPS" "$ACTUAL_VAPS" "$ACTUAL_VRPS_META" "$ACTUAL_VAPS_META"
|
|
||||||
import csv, json, sys
|
|
||||||
def count_rows(path):
|
|
||||||
with open(path, newline="") as f:
|
|
||||||
rows = list(csv.reader(f))
|
|
||||||
return max(len(rows) - 1, 0)
|
|
||||||
json.dump({"count": count_rows(sys.argv[1])}, open(sys.argv[3], "w"), indent=2)
|
|
||||||
json.dump({"count": count_rows(sys.argv[2])}, open(sys.argv[4], "w"), indent=2)
|
|
||||||
PY
|
|
||||||
|
|
||||||
"$CCR_TO_COMPARE_VIEWS_BIN" --ccr "$REFERENCE_CCR" --vrps-out "$REF_VRPS" --vaps-out "$REF_VAPS" --trust-anchor "$TA_NAME"
|
|
||||||
|
|
||||||
python3 - <<'PY' "$ACTUAL_VRPS" "$REF_VRPS" "$ACTUAL_VAPS" "$REF_VAPS" "$SUMMARY_JSON"
|
|
||||||
import csv, json, sys
|
|
||||||
def rows(path):
|
|
||||||
with open(path, newline="") as f:
|
|
||||||
return list(csv.reader(f))[1:]
|
|
||||||
actual_vrps = {tuple(r) for r in rows(sys.argv[1])}
|
|
||||||
ref_vrps = {tuple(r) for r in rows(sys.argv[2])}
|
|
||||||
actual_vaps = {tuple(r) for r in rows(sys.argv[3])}
|
|
||||||
ref_vaps = {tuple(r) for r in rows(sys.argv[4])}
|
|
||||||
summary = {
|
|
||||||
"vrps": {
|
|
||||||
"actual": len(actual_vrps),
|
|
||||||
"reference": len(ref_vrps),
|
|
||||||
"match": actual_vrps == ref_vrps,
|
|
||||||
"only_in_actual": sorted(actual_vrps - ref_vrps)[:20],
|
|
||||||
"only_in_reference": sorted(ref_vrps - actual_vrps)[:20],
|
|
||||||
},
|
|
||||||
"vaps": {
|
|
||||||
"actual": len(actual_vaps),
|
|
||||||
"reference": len(ref_vaps),
|
|
||||||
"match": actual_vaps == ref_vaps,
|
|
||||||
"only_in_actual": sorted(actual_vaps - ref_vaps)[:20],
|
|
||||||
"only_in_reference": sorted(ref_vaps - actual_vaps)[:20],
|
|
||||||
}
|
|
||||||
}
|
|
||||||
with open(sys.argv[5], "w") as f:
|
|
||||||
json.dump(summary, f, indent=2)
|
|
||||||
PY
|
|
||||||
|
|
||||||
if [[ "$KEEP_DB" -ne 1 ]]; then
|
|
||||||
rm -rf "$TMP_ROOT"
|
|
||||||
fi
|
|
||||||
|
|
||||||
echo "done: $OUT_DIR"
|
|
||||||
@@ -1,149 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage:
|
|
||||||
./scripts/cir/run_cir_replay_sequence_ours.sh \
|
|
||||||
--sequence-root <path> \
|
|
||||||
[--rpki-bin <path>] \
|
|
||||||
[--real-rsync-bin <path>]
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
SEQUENCE_ROOT=""
|
|
||||||
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
|
||||||
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
|
||||||
STEP_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_ours.sh"
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--sequence-root) SEQUENCE_ROOT="$2"; shift 2 ;;
|
|
||||||
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
|
||||||
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
[[ -n "$SEQUENCE_ROOT" ]] || { usage >&2; exit 2; }
|
|
||||||
|
|
||||||
SEQUENCE_ROOT="$(python3 - <<'PY' "$SEQUENCE_ROOT"
|
|
||||||
from pathlib import Path
|
|
||||||
import sys
|
|
||||||
print(Path(sys.argv[1]).resolve())
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
|
|
||||||
SUMMARY_JSON="$SEQUENCE_ROOT/sequence-summary.json"
|
|
||||||
SUMMARY_MD="$SEQUENCE_ROOT/sequence-summary.md"
|
|
||||||
DETAIL_JSON="$SEQUENCE_ROOT/sequence-detail.json"
|
|
||||||
|
|
||||||
python3 - <<'PY' "$SEQUENCE_ROOT" "$SUMMARY_JSON" "$SUMMARY_MD" "$DETAIL_JSON" "$STEP_SCRIPT" "$RPKI_BIN" "$REAL_RSYNC_BIN"
|
|
||||||
import json
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
sequence_root = Path(sys.argv[1])
|
|
||||||
summary_json = Path(sys.argv[2])
|
|
||||||
summary_md = Path(sys.argv[3])
|
|
||||||
detail_json = Path(sys.argv[4])
|
|
||||||
step_script = Path(sys.argv[5])
|
|
||||||
rpki_bin = sys.argv[6]
|
|
||||||
real_rsync_bin = sys.argv[7]
|
|
||||||
|
|
||||||
sequence = json.loads((sequence_root / "sequence.json").read_text(encoding="utf-8"))
|
|
||||||
static_root = sequence_root / sequence["staticRoot"] if "staticRoot" in sequence else None
|
|
||||||
raw_store_db = sequence_root / sequence["rawStoreDbPath"] if "rawStoreDbPath" in sequence else None
|
|
||||||
backend_count = sum(x is not None for x in (static_root, raw_store_db))
|
|
||||||
if backend_count != 1:
|
|
||||||
raise SystemExit("sequence must set exactly one of staticRoot or rawStoreDbPath")
|
|
||||||
steps = sequence["steps"]
|
|
||||||
|
|
||||||
results = []
|
|
||||||
all_match = True
|
|
||||||
for step in steps:
|
|
||||||
step_id = step["stepId"]
|
|
||||||
out_dir = sequence_root / "replay-ours" / step_id
|
|
||||||
out_dir.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
cmd = [
|
|
||||||
str(step_script),
|
|
||||||
"--cir",
|
|
||||||
str(sequence_root / step["cirPath"]),
|
|
||||||
"--out-dir",
|
|
||||||
str(out_dir),
|
|
||||||
"--reference-ccr",
|
|
||||||
str(sequence_root / step["ccrPath"]),
|
|
||||||
"--rpki-bin",
|
|
||||||
rpki_bin,
|
|
||||||
"--real-rsync-bin",
|
|
||||||
real_rsync_bin,
|
|
||||||
]
|
|
||||||
if static_root is not None:
|
|
||||||
cmd.extend(["--static-root", str(static_root)])
|
|
||||||
else:
|
|
||||||
cmd.extend(["--raw-store-db", str(raw_store_db)])
|
|
||||||
proc = subprocess.run(cmd, capture_output=True, text=True)
|
|
||||||
if proc.returncode != 0:
|
|
||||||
raise SystemExit(
|
|
||||||
f"ours sequence replay failed for {step_id}: stdout={proc.stdout} stderr={proc.stderr}"
|
|
||||||
)
|
|
||||||
compare = json.loads((out_dir / "compare-summary.json").read_text(encoding="utf-8"))
|
|
||||||
timing = json.loads((out_dir / "timing.json").read_text(encoding="utf-8")) if (out_dir / "timing.json").exists() else {}
|
|
||||||
record = {
|
|
||||||
"stepId": step_id,
|
|
||||||
"kind": step["kind"],
|
|
||||||
"validationTime": step["validationTime"],
|
|
||||||
"outDir": str(out_dir),
|
|
||||||
"comparePath": str(out_dir / "compare-summary.json"),
|
|
||||||
"timingPath": str(out_dir / "timing.json"),
|
|
||||||
"compare": compare,
|
|
||||||
"timing": timing,
|
|
||||||
"match": bool(compare["vrps"]["match"]) and bool(compare["vaps"]["match"]),
|
|
||||||
}
|
|
||||||
all_match = all_match and record["match"]
|
|
||||||
results.append(record)
|
|
||||||
|
|
||||||
summary = {
|
|
||||||
"version": 1,
|
|
||||||
"participant": "ours",
|
|
||||||
"sequenceRoot": str(sequence_root),
|
|
||||||
"stepCount": len(results),
|
|
||||||
"allMatch": all_match,
|
|
||||||
"steps": results,
|
|
||||||
}
|
|
||||||
summary_json.write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
|
||||||
detail_json.write_text(json.dumps(results, indent=2), encoding="utf-8")
|
|
||||||
|
|
||||||
lines = [
|
|
||||||
"# Ours CIR Sequence Replay Summary",
|
|
||||||
"",
|
|
||||||
f"- `sequence_root`: `{sequence_root}`",
|
|
||||||
f"- `step_count`: `{len(results)}`",
|
|
||||||
f"- `all_match`: `{all_match}`",
|
|
||||||
"",
|
|
||||||
"| Step | Kind | VRP actual/ref | VRP match | VAP actual/ref | VAP match | Duration (ms) |",
|
|
||||||
"| --- | --- | --- | --- | --- | --- | ---: |",
|
|
||||||
]
|
|
||||||
for item in results:
|
|
||||||
compare = item["compare"]
|
|
||||||
timing = item.get("timing") or {}
|
|
||||||
lines.append(
|
|
||||||
"| {step} | {kind} | {va}/{vr} | {vm} | {aa}/{ar} | {am} | {dur} |".format(
|
|
||||||
step=item["stepId"],
|
|
||||||
kind=item["kind"],
|
|
||||||
va=compare["vrps"]["actual"],
|
|
||||||
vr=compare["vrps"]["reference"],
|
|
||||||
vm=compare["vrps"]["match"],
|
|
||||||
aa=compare["vaps"]["actual"],
|
|
||||||
ar=compare["vaps"]["reference"],
|
|
||||||
am=compare["vaps"]["match"],
|
|
||||||
dur=timing.get("durationMs", "-"),
|
|
||||||
)
|
|
||||||
)
|
|
||||||
summary_md.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
|
||||||
PY
|
|
||||||
|
|
||||||
echo "done: $SEQUENCE_ROOT"
|
|
||||||
@@ -1,132 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage:
|
|
||||||
./scripts/cir/run_cir_replay_sequence_routinator.sh \
|
|
||||||
--sequence-root <path> \
|
|
||||||
[--routinator-root <path>] \
|
|
||||||
[--routinator-bin <path>] \
|
|
||||||
[--real-rsync-bin <path>]
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
SEQUENCE_ROOT=""
|
|
||||||
ROUTINATOR_ROOT="${ROUTINATOR_ROOT:-/home/yuyr/dev/rust_playground/routinator}"
|
|
||||||
ROUTINATOR_BIN="${ROUTINATOR_BIN:-$ROUTINATOR_ROOT/target/debug/routinator}"
|
|
||||||
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
|
||||||
STEP_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_routinator.sh"
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--sequence-root) SEQUENCE_ROOT="$2"; shift 2 ;;
|
|
||||||
--routinator-root) ROUTINATOR_ROOT="$2"; shift 2 ;;
|
|
||||||
--routinator-bin) ROUTINATOR_BIN="$2"; shift 2 ;;
|
|
||||||
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
[[ -n "$SEQUENCE_ROOT" ]] || { usage >&2; exit 2; }
|
|
||||||
|
|
||||||
SEQUENCE_ROOT="$(python3 - <<'PY' "$SEQUENCE_ROOT"
|
|
||||||
from pathlib import Path
|
|
||||||
import sys
|
|
||||||
print(Path(sys.argv[1]).resolve())
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
|
|
||||||
SUMMARY_JSON="$SEQUENCE_ROOT/sequence-summary-routinator.json"
|
|
||||||
SUMMARY_MD="$SEQUENCE_ROOT/sequence-summary-routinator.md"
|
|
||||||
|
|
||||||
python3 - <<'PY' "$SEQUENCE_ROOT" "$SUMMARY_JSON" "$SUMMARY_MD" "$STEP_SCRIPT" "$ROUTINATOR_ROOT" "$ROUTINATOR_BIN" "$REAL_RSYNC_BIN"
|
|
||||||
import json
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
sequence_root = Path(sys.argv[1])
|
|
||||||
summary_json = Path(sys.argv[2])
|
|
||||||
summary_md = Path(sys.argv[3])
|
|
||||||
step_script = Path(sys.argv[4])
|
|
||||||
routinator_root = sys.argv[5]
|
|
||||||
routinator_bin = sys.argv[6]
|
|
||||||
real_rsync_bin = sys.argv[7]
|
|
||||||
|
|
||||||
sequence = json.loads((sequence_root / "sequence.json").read_text(encoding="utf-8"))
|
|
||||||
static_root = sequence_root / sequence["staticRoot"] if "staticRoot" in sequence else None
|
|
||||||
raw_store_db = sequence_root / sequence["rawStoreDbPath"] if "rawStoreDbPath" in sequence else None
|
|
||||||
backend_count = sum(x is not None for x in (static_root, raw_store_db))
|
|
||||||
if backend_count != 1:
|
|
||||||
raise SystemExit("sequence must set exactly one of staticRoot or rawStoreDbPath")
|
|
||||||
steps = sequence["steps"]
|
|
||||||
results = []
|
|
||||||
all_match = True
|
|
||||||
|
|
||||||
for step in steps:
|
|
||||||
step_id = step["stepId"]
|
|
||||||
out_dir = sequence_root / "replay-routinator" / step_id
|
|
||||||
out_dir.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
cmd = [
|
|
||||||
str(step_script),
|
|
||||||
"--cir",
|
|
||||||
str(sequence_root / step["cirPath"]),
|
|
||||||
"--out-dir",
|
|
||||||
str(out_dir),
|
|
||||||
"--reference-ccr",
|
|
||||||
str(sequence_root / step["ccrPath"]),
|
|
||||||
"--routinator-root",
|
|
||||||
routinator_root,
|
|
||||||
"--routinator-bin",
|
|
||||||
routinator_bin,
|
|
||||||
"--real-rsync-bin",
|
|
||||||
real_rsync_bin,
|
|
||||||
]
|
|
||||||
if static_root is not None:
|
|
||||||
cmd.extend(["--static-root", str(static_root)])
|
|
||||||
else:
|
|
||||||
cmd.extend(["--raw-store-db", str(raw_store_db)])
|
|
||||||
proc = subprocess.run(cmd, capture_output=True, text=True)
|
|
||||||
if proc.returncode != 0:
|
|
||||||
raise SystemExit(
|
|
||||||
f"routinator sequence replay failed for {step_id}: stdout={proc.stdout} stderr={proc.stderr}"
|
|
||||||
)
|
|
||||||
compare = json.loads((out_dir / "compare-summary.json").read_text(encoding="utf-8"))
|
|
||||||
match = bool(compare["vrps"]["match"]) and bool(compare["vaps"]["match"])
|
|
||||||
all_match = all_match and match
|
|
||||||
results.append(
|
|
||||||
{
|
|
||||||
"stepId": step_id,
|
|
||||||
"kind": step["kind"],
|
|
||||||
"validationTime": step["validationTime"],
|
|
||||||
"outDir": str(out_dir),
|
|
||||||
"comparePath": str(out_dir / "compare-summary.json"),
|
|
||||||
"match": match,
|
|
||||||
"compare": compare,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
summary = {
|
|
||||||
"version": 1,
|
|
||||||
"participant": "routinator",
|
|
||||||
"sequenceRoot": str(sequence_root),
|
|
||||||
"stepCount": len(results),
|
|
||||||
"allMatch": all_match,
|
|
||||||
"steps": results,
|
|
||||||
}
|
|
||||||
summary_json.write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
|
||||||
lines = [
|
|
||||||
"# Routinator CIR Sequence Replay Summary",
|
|
||||||
"",
|
|
||||||
f"- `sequence_root`: `{sequence_root}`",
|
|
||||||
f"- `step_count`: `{len(results)}`",
|
|
||||||
f"- `all_match`: `{all_match}`",
|
|
||||||
"",
|
|
||||||
]
|
|
||||||
summary_md.write_text("\n".join(lines), encoding="utf-8")
|
|
||||||
PY
|
|
||||||
|
|
||||||
echo "done: $SEQUENCE_ROOT"
|
|
||||||
@@ -1,126 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage:
|
|
||||||
./scripts/cir/run_cir_replay_sequence_rpki_client.sh \
|
|
||||||
--sequence-root <path> \
|
|
||||||
--build-dir <path> \
|
|
||||||
[--real-rsync-bin <path>]
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
SEQUENCE_ROOT=""
|
|
||||||
BUILD_DIR=""
|
|
||||||
REAL_RSYNC_BIN="${REAL_RSYNC_BIN:-/usr/bin/rsync}"
|
|
||||||
STEP_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_rpki_client.sh"
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--sequence-root) SEQUENCE_ROOT="$2"; shift 2 ;;
|
|
||||||
--build-dir) BUILD_DIR="$2"; shift 2 ;;
|
|
||||||
--real-rsync-bin) REAL_RSYNC_BIN="$2"; shift 2 ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
[[ -n "$SEQUENCE_ROOT" && -n "$BUILD_DIR" ]] || { usage >&2; exit 2; }
|
|
||||||
|
|
||||||
SEQUENCE_ROOT="$(python3 - <<'PY' "$SEQUENCE_ROOT"
|
|
||||||
from pathlib import Path
|
|
||||||
import sys
|
|
||||||
print(Path(sys.argv[1]).resolve())
|
|
||||||
PY
|
|
||||||
)"
|
|
||||||
|
|
||||||
SUMMARY_JSON="$SEQUENCE_ROOT/sequence-summary-rpki-client.json"
|
|
||||||
SUMMARY_MD="$SEQUENCE_ROOT/sequence-summary-rpki-client.md"
|
|
||||||
|
|
||||||
python3 - <<'PY' "$SEQUENCE_ROOT" "$SUMMARY_JSON" "$SUMMARY_MD" "$STEP_SCRIPT" "$BUILD_DIR" "$REAL_RSYNC_BIN"
|
|
||||||
import json
|
|
||||||
import subprocess
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
sequence_root = Path(sys.argv[1])
|
|
||||||
summary_json = Path(sys.argv[2])
|
|
||||||
summary_md = Path(sys.argv[3])
|
|
||||||
step_script = Path(sys.argv[4])
|
|
||||||
build_dir = sys.argv[5]
|
|
||||||
real_rsync_bin = sys.argv[6]
|
|
||||||
|
|
||||||
sequence = json.loads((sequence_root / "sequence.json").read_text(encoding="utf-8"))
|
|
||||||
static_root = sequence_root / sequence["staticRoot"] if "staticRoot" in sequence else None
|
|
||||||
raw_store_db = sequence_root / sequence["rawStoreDbPath"] if "rawStoreDbPath" in sequence else None
|
|
||||||
backend_count = sum(x is not None for x in (static_root, raw_store_db))
|
|
||||||
if backend_count != 1:
|
|
||||||
raise SystemExit("sequence must set exactly one of staticRoot or rawStoreDbPath")
|
|
||||||
steps = sequence["steps"]
|
|
||||||
results = []
|
|
||||||
all_match = True
|
|
||||||
|
|
||||||
for step in steps:
|
|
||||||
step_id = step["stepId"]
|
|
||||||
out_dir = sequence_root / "replay-rpki-client" / step_id
|
|
||||||
out_dir.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
cmd = [
|
|
||||||
str(step_script),
|
|
||||||
"--cir",
|
|
||||||
str(sequence_root / step["cirPath"]),
|
|
||||||
"--out-dir",
|
|
||||||
str(out_dir),
|
|
||||||
"--reference-ccr",
|
|
||||||
str(sequence_root / step["ccrPath"]),
|
|
||||||
"--build-dir",
|
|
||||||
build_dir,
|
|
||||||
"--real-rsync-bin",
|
|
||||||
real_rsync_bin,
|
|
||||||
]
|
|
||||||
if static_root is not None:
|
|
||||||
cmd.extend(["--static-root", str(static_root)])
|
|
||||||
else:
|
|
||||||
cmd.extend(["--raw-store-db", str(raw_store_db)])
|
|
||||||
proc = subprocess.run(cmd, capture_output=True, text=True)
|
|
||||||
if proc.returncode != 0:
|
|
||||||
raise SystemExit(
|
|
||||||
f"rpki-client sequence replay failed for {step_id}: stdout={proc.stdout} stderr={proc.stderr}"
|
|
||||||
)
|
|
||||||
compare = json.loads((out_dir / "compare-summary.json").read_text(encoding="utf-8"))
|
|
||||||
match = bool(compare["vrps"]["match"]) and bool(compare["vaps"]["match"])
|
|
||||||
all_match = all_match and match
|
|
||||||
results.append(
|
|
||||||
{
|
|
||||||
"stepId": step_id,
|
|
||||||
"kind": step["kind"],
|
|
||||||
"validationTime": step["validationTime"],
|
|
||||||
"outDir": str(out_dir),
|
|
||||||
"comparePath": str(out_dir / "compare-summary.json"),
|
|
||||||
"match": match,
|
|
||||||
"compare": compare,
|
|
||||||
}
|
|
||||||
)
|
|
||||||
|
|
||||||
summary = {
|
|
||||||
"version": 1,
|
|
||||||
"participant": "rpki-client",
|
|
||||||
"sequenceRoot": str(sequence_root),
|
|
||||||
"stepCount": len(results),
|
|
||||||
"allMatch": all_match,
|
|
||||||
"steps": results,
|
|
||||||
}
|
|
||||||
summary_json.write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
|
||||||
lines = [
|
|
||||||
"# rpki-client CIR Sequence Replay Summary",
|
|
||||||
"",
|
|
||||||
f"- `sequence_root`: `{sequence_root}`",
|
|
||||||
f"- `step_count`: `{len(results)}`",
|
|
||||||
f"- `all_match`: `{all_match}`",
|
|
||||||
"",
|
|
||||||
]
|
|
||||||
summary_md.write_text("\n".join(lines), encoding="utf-8")
|
|
||||||
PY
|
|
||||||
|
|
||||||
echo "done: $SEQUENCE_ROOT"
|
|
||||||
@@ -1,132 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
usage() {
|
|
||||||
cat <<'EOF'
|
|
||||||
Usage:
|
|
||||||
./scripts/cir/run_cir_sequence_matrix_multi_rir.sh \
|
|
||||||
--root <path> \
|
|
||||||
[--rir <afrinic,apnic,arin,lacnic,ripe>] \
|
|
||||||
[--rpki-bin <path>] \
|
|
||||||
[--routinator-root <path>] \
|
|
||||||
[--routinator-bin <path>] \
|
|
||||||
[--rpki-client-build-dir <path>] \
|
|
||||||
[--drop-bin <path>]
|
|
||||||
EOF
|
|
||||||
}
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
ROOT=""
|
|
||||||
RIRS="afrinic,apnic,arin,lacnic,ripe"
|
|
||||||
RPKI_BIN="${RPKI_BIN:-$ROOT_DIR/target/release/rpki}"
|
|
||||||
ROUTINATOR_ROOT="${ROUTINATOR_ROOT:-/home/yuyr/dev/rust_playground/routinator}"
|
|
||||||
ROUTINATOR_BIN="${ROUTINATOR_BIN:-$ROUTINATOR_ROOT/target/debug/routinator}"
|
|
||||||
RPKI_CLIENT_BUILD_DIR="${RPKI_CLIENT_BUILD_DIR:-/home/yuyr/dev/rpki-client-9.7/build-m5}"
|
|
||||||
DROP_BIN="${DROP_BIN:-$ROOT_DIR/target/release/cir_drop_report}"
|
|
||||||
|
|
||||||
OURS_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_sequence_ours.sh"
|
|
||||||
ROUTINATOR_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_sequence_routinator.sh"
|
|
||||||
RPKIC_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_replay_sequence_rpki_client.sh"
|
|
||||||
DROP_SCRIPT="$ROOT_DIR/scripts/cir/run_cir_drop_sequence.sh"
|
|
||||||
|
|
||||||
while [[ $# -gt 0 ]]; do
|
|
||||||
case "$1" in
|
|
||||||
--root) ROOT="$2"; shift 2 ;;
|
|
||||||
--rir) RIRS="$2"; shift 2 ;;
|
|
||||||
--rpki-bin) RPKI_BIN="$2"; shift 2 ;;
|
|
||||||
--routinator-root) ROUTINATOR_ROOT="$2"; shift 2 ;;
|
|
||||||
--routinator-bin) ROUTINATOR_BIN="$2"; shift 2 ;;
|
|
||||||
--rpki-client-build-dir) RPKI_CLIENT_BUILD_DIR="$2"; shift 2 ;;
|
|
||||||
--drop-bin) DROP_BIN="$2"; shift 2 ;;
|
|
||||||
-h|--help) usage; exit 0 ;;
|
|
||||||
*) echo "unknown argument: $1" >&2; usage; exit 2 ;;
|
|
||||||
esac
|
|
||||||
done
|
|
||||||
|
|
||||||
[[ -n "$ROOT" ]] || { usage >&2; exit 2; }
|
|
||||||
|
|
||||||
SUMMARY_JSON="$ROOT/final-summary.json"
|
|
||||||
SUMMARY_MD="$ROOT/final-summary.md"
|
|
||||||
IFS=',' read -r -a ITEMS <<< "$RIRS"
|
|
||||||
|
|
||||||
results=()
|
|
||||||
for rir in "${ITEMS[@]}"; do
|
|
||||||
seq_root="$ROOT/$rir"
|
|
||||||
"$OURS_SCRIPT" --sequence-root "$seq_root" --rpki-bin "$RPKI_BIN"
|
|
||||||
"$ROUTINATOR_SCRIPT" --sequence-root "$seq_root" --routinator-root "$ROUTINATOR_ROOT" --routinator-bin "$ROUTINATOR_BIN"
|
|
||||||
"$RPKIC_SCRIPT" --sequence-root "$seq_root" --build-dir "$RPKI_CLIENT_BUILD_DIR"
|
|
||||||
"$DROP_SCRIPT" --sequence-root "$seq_root" --drop-bin "$DROP_BIN"
|
|
||||||
done
|
|
||||||
|
|
||||||
python3 - <<'PY' "$ROOT" "$RIRS" "$SUMMARY_JSON" "$SUMMARY_MD"
|
|
||||||
import json, sys
|
|
||||||
from pathlib import Path
|
|
||||||
from collections import Counter
|
|
||||||
|
|
||||||
root = Path(sys.argv[1]).resolve()
|
|
||||||
rirs = [item for item in sys.argv[2].split(',') if item]
|
|
||||||
summary_json = Path(sys.argv[3])
|
|
||||||
summary_md = Path(sys.argv[4])
|
|
||||||
items = []
|
|
||||||
total_steps = 0
|
|
||||||
total_dropped_vrps = 0
|
|
||||||
total_dropped_objects = 0
|
|
||||||
reason_counter = Counter()
|
|
||||||
for rir in rirs:
|
|
||||||
seq_root = root / rir
|
|
||||||
ours = json.loads((seq_root / "sequence-summary.json").read_text(encoding="utf-8"))
|
|
||||||
routinator = json.loads((seq_root / "sequence-summary-routinator.json").read_text(encoding="utf-8"))
|
|
||||||
rpki_client = json.loads((seq_root / "sequence-summary-rpki-client.json").read_text(encoding="utf-8"))
|
|
||||||
drop = json.loads((seq_root / "drop-summary.json").read_text(encoding="utf-8"))
|
|
||||||
step_count = len(ours["steps"])
|
|
||||||
total_steps += step_count
|
|
||||||
rir_dropped_vrps = 0
|
|
||||||
rir_dropped_objects = 0
|
|
||||||
for step in drop["steps"]:
|
|
||||||
drop_path = Path(step["reportPath"])
|
|
||||||
detail = json.loads(drop_path.read_text(encoding="utf-8"))
|
|
||||||
summary = detail.get("summary", {})
|
|
||||||
rir_dropped_vrps += int(summary.get("droppedVrpCount", 0))
|
|
||||||
rir_dropped_objects += int(summary.get("droppedObjectCount", 0))
|
|
||||||
total_dropped_vrps += int(summary.get("droppedVrpCount", 0))
|
|
||||||
total_dropped_objects += int(summary.get("droppedObjectCount", 0))
|
|
||||||
for reason, count in summary.get("droppedByReason", {}).items():
|
|
||||||
reason_counter[reason] += int(count)
|
|
||||||
items.append({
|
|
||||||
"rir": rir,
|
|
||||||
"stepCount": step_count,
|
|
||||||
"oursAllMatch": ours["allMatch"],
|
|
||||||
"routinatorAllMatch": routinator["allMatch"],
|
|
||||||
"rpkiClientAllMatch": rpki_client["allMatch"],
|
|
||||||
"dropSummary": drop["steps"],
|
|
||||||
"droppedVrpCount": rir_dropped_vrps,
|
|
||||||
"droppedObjectCount": rir_dropped_objects,
|
|
||||||
})
|
|
||||||
summary = {
|
|
||||||
"version": 1,
|
|
||||||
"totalStepCount": total_steps,
|
|
||||||
"totalDroppedVrpCount": total_dropped_vrps,
|
|
||||||
"totalDroppedObjectCount": total_dropped_objects,
|
|
||||||
"topReasons": [{"reason": reason, "count": count} for reason, count in reason_counter.most_common(10)],
|
|
||||||
"rirs": items,
|
|
||||||
}
|
|
||||||
summary_json.write_text(json.dumps(summary, indent=2), encoding="utf-8")
|
|
||||||
lines = ["# Multi-RIR CIR Sequence Matrix Summary", ""]
|
|
||||||
lines.append(f"- `total_step_count`: `{total_steps}`")
|
|
||||||
lines.append(f"- `total_dropped_vrps`: `{total_dropped_vrps}`")
|
|
||||||
lines.append(f"- `total_dropped_objects`: `{total_dropped_objects}`")
|
|
||||||
lines.append("")
|
|
||||||
if reason_counter:
|
|
||||||
lines.append("## Top Drop Reasons")
|
|
||||||
lines.append("")
|
|
||||||
for reason, count in reason_counter.most_common(10):
|
|
||||||
lines.append(f"- `{reason}`: `{count}`")
|
|
||||||
lines.append("")
|
|
||||||
for item in items:
|
|
||||||
lines.append(
|
|
||||||
f"- `{item['rir']}`: `steps={item['stepCount']}` `ours={item['oursAllMatch']}` `routinator={item['routinatorAllMatch']}` `rpki-client={item['rpkiClientAllMatch']}` `drop_vrps={item['droppedVrpCount']}` `drop_objects={item['droppedObjectCount']}`"
|
|
||||||
)
|
|
||||||
summary_md.write_text("\n".join(lines) + "\n", encoding="utf-8")
|
|
||||||
PY
|
|
||||||
|
|
||||||
echo "done: $ROOT"
|
|
||||||
@@ -1,82 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
# Requires:
|
|
||||||
# rustup component add llvm-tools-preview
|
|
||||||
# cargo install cargo-llvm-cov --locked
|
|
||||||
|
|
||||||
run_out="$(mktemp)"
|
|
||||||
text_out="$(mktemp)"
|
|
||||||
html_out="$(mktemp)"
|
|
||||||
|
|
||||||
cleanup() {
|
|
||||||
rm -f "$run_out" "$text_out" "$html_out"
|
|
||||||
}
|
|
||||||
trap cleanup EXIT
|
|
||||||
|
|
||||||
IGNORE_REGEX='src/bin/replay_bundle_capture\.rs|src/bin/replay_bundle_capture_delta\.rs|src/bin/replay_bundle_capture_sequence\.rs|src/bin/replay_bundle_record\.rs|src/bin/replay_bundle_refresh_sequence_outputs\.rs|src/bin/measure_sequence_replay\.rs|src/bin/repository_view_stats\.rs|src/bin/trace_arin_missing_vrps\.rs|src/bin/db_stats\.rs|src/bin/rrdp_state_dump\.rs|src/bin/ccr_dump\.rs|src/bin/ccr_verify\.rs|src/bin/ccr_to_routinator_csv\.rs|src/bin/ccr_to_compare_views\.rs|src/bin/cir_materialize\.rs|src/bin/cir_extract_inputs\.rs|src/bin/cir_drop_report\.rs|src/bin/cir_ta_only_fixture\.rs|src/bundle/live_capture\.rs|src/bundle/record_io\.rs|src/bundle/compare_view\.rs|src/progress_log\.rs|src/cli\.rs|src/validation/run_tree_from_tal\.rs|src/validation/from_tal\.rs|src/sync/store_projection\.rs|src/cir/materialize\.rs'
|
|
||||||
|
|
||||||
# Preserve colored output even though we post-process output by running under a pseudo-TTY.
|
|
||||||
# We run tests only once, then generate both CLI text + HTML reports without rerunning tests.
|
|
||||||
set +e
|
|
||||||
|
|
||||||
cargo llvm-cov clean --workspace >/dev/null 2>&1
|
|
||||||
|
|
||||||
# 1) Run tests once to collect coverage data (no report).
|
|
||||||
script -q -e -c "CARGO_TERM_COLOR=always cargo llvm-cov --no-report" "$run_out" >/dev/null 2>&1
|
|
||||||
run_status="$?"
|
|
||||||
|
|
||||||
# 2) CLI summary report + fail-under gate (no test rerun).
|
|
||||||
script -q -e -c "CARGO_TERM_COLOR=always cargo llvm-cov report --fail-under-lines 90 --ignore-filename-regex '$IGNORE_REGEX'" "$text_out" >/dev/null 2>&1
|
|
||||||
text_status="$?"
|
|
||||||
|
|
||||||
# 3) HTML report (no test rerun).
|
|
||||||
script -q -e -c "CARGO_TERM_COLOR=always cargo llvm-cov report --html --ignore-filename-regex '$IGNORE_REGEX'" "$html_out" >/dev/null 2>&1
|
|
||||||
html_status="$?"
|
|
||||||
|
|
||||||
set -e
|
|
||||||
|
|
||||||
strip_script_noise() {
|
|
||||||
tr -d '\r' | sed '/^Script \(started\|done\) on /d'
|
|
||||||
}
|
|
||||||
|
|
||||||
strip_ansi_for_parse() {
|
|
||||||
awk '
|
|
||||||
{
|
|
||||||
line = $0
|
|
||||||
gsub(/\033\[[0-9;]*[A-Za-z]/, "", line) # CSI escapes
|
|
||||||
gsub(/\033\([A-Za-z]/, "", line) # charset escapes (e.g., ESC(B)
|
|
||||||
gsub(/\r/, "", line)
|
|
||||||
print line
|
|
||||||
}
|
|
||||||
'
|
|
||||||
}
|
|
||||||
|
|
||||||
cat "$run_out" | strip_script_noise
|
|
||||||
cat "$text_out" | strip_script_noise
|
|
||||||
cat "$html_out" | strip_script_noise
|
|
||||||
|
|
||||||
cat "$run_out" | strip_ansi_for_parse | awk '
|
|
||||||
BEGIN {
|
|
||||||
passed=0; failed=0; ignored=0; measured=0; filtered=0;
|
|
||||||
}
|
|
||||||
/^test result: / {
|
|
||||||
if (match($0, /([0-9]+) passed; ([0-9]+) failed; ([0-9]+) ignored; ([0-9]+) measured; ([0-9]+) filtered out;/, m)) {
|
|
||||||
passed += m[1]; failed += m[2]; ignored += m[3]; measured += m[4]; filtered += m[5];
|
|
||||||
}
|
|
||||||
}
|
|
||||||
END {
|
|
||||||
executed = passed + failed;
|
|
||||||
total = passed + failed + ignored + measured;
|
|
||||||
printf("\nTEST SUMMARY (all suites): passed=%d failed=%d ignored=%d measured=%d filtered_out=%d executed=%d total=%d\n",
|
|
||||||
passed, failed, ignored, measured, filtered, executed, total);
|
|
||||||
}
|
|
||||||
'
|
|
||||||
|
|
||||||
echo
|
|
||||||
echo "HTML report: target/llvm-cov/html/index.html"
|
|
||||||
|
|
||||||
status="$text_status"
|
|
||||||
if [ "$run_status" -ne 0 ]; then status="$run_status"; fi
|
|
||||||
if [ "$html_status" -ne 0 ]; then status="$html_status"; fi
|
|
||||||
exit "$status"
|
|
||||||
@@ -1,195 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
# M2: Run per-sample decode+profile benchmark (Ours vs Routinator) on selected_der fixtures.
|
|
||||||
#
|
|
||||||
# Outputs:
|
|
||||||
# - specs/develop/20260224/data/m2_manifest_decode_profile_compare.csv
|
|
||||||
# - specs/develop/20260224/data/m2_raw.log
|
|
||||||
#
|
|
||||||
# Note: This script assumes Routinator benchmark repo exists at:
|
|
||||||
# /home/yuyr/dev/rust_playground/routinator/benchmark
|
|
||||||
#
|
|
||||||
# It also assumes fixtures exist under:
|
|
||||||
# rpki/tests/benchmark/selected_der/*.mft
|
|
||||||
# routinator/benchmark/fixtures/selected_der/*.mft
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
|
||||||
RPKI_DIR="$ROOT_DIR"
|
|
||||||
OURS_BENCH_DIR="$RPKI_DIR/benchmark/ours_manifest_bench"
|
|
||||||
|
|
||||||
ROUT_BENCH_DIR="${ROUT_BENCH_DIR:-/home/yuyr/dev/rust_playground/routinator/benchmark}"
|
|
||||||
ROUT_BIN="$ROUT_BENCH_DIR/target/release/routinator-manifest-benchmark"
|
|
||||||
|
|
||||||
DATE_TAG="${DATE_TAG:-20260224}"
|
|
||||||
OUT_DIR="$RPKI_DIR/../specs/develop/${DATE_TAG}/data"
|
|
||||||
OUT_CSV="${OUT_CSV:-$OUT_DIR/m2_manifest_decode_profile_compare.csv}"
|
|
||||||
OUT_RAW="${OUT_RAW:-$OUT_DIR/m2_raw.log}"
|
|
||||||
|
|
||||||
REPEATS="${REPEATS:-3}"
|
|
||||||
|
|
||||||
# Iterations / warmups (kept moderate for interactive iteration).
|
|
||||||
ITER_SMALL="${ITER_SMALL:-20000}"
|
|
||||||
ITER_MEDIUM="${ITER_MEDIUM:-20000}"
|
|
||||||
ITER_LARGE="${ITER_LARGE:-20000}"
|
|
||||||
ITER_XLARGE="${ITER_XLARGE:-2000}"
|
|
||||||
|
|
||||||
WARM_SMALL="${WARM_SMALL:-2000}"
|
|
||||||
WARM_MEDIUM="${WARM_MEDIUM:-2000}"
|
|
||||||
WARM_LARGE="${WARM_LARGE:-2000}"
|
|
||||||
WARM_XLARGE="${WARM_XLARGE:-200}"
|
|
||||||
|
|
||||||
SAMPLES=(
|
|
||||||
small-01
|
|
||||||
small-02
|
|
||||||
medium-01
|
|
||||||
medium-02
|
|
||||||
large-01
|
|
||||||
large-02
|
|
||||||
xlarge-01
|
|
||||||
xlarge-02
|
|
||||||
)
|
|
||||||
|
|
||||||
mkdir -p "$OUT_DIR"
|
|
||||||
: > "$OUT_RAW"
|
|
||||||
|
|
||||||
echo "sample,bucket,manifest_file_count,ours_avg_ns_per_op,ours_ops_per_s,rout_avg_ns_per_op,rout_ops_per_s,ratio_ours_over_rout,iterations,repeats,warmup" > "$OUT_CSV"
|
|
||||||
|
|
||||||
echo "[1/3] Build ours benchmark (release)..." | tee -a "$OUT_RAW"
|
|
||||||
(cd "$OURS_BENCH_DIR" && cargo build --release -q)
|
|
||||||
OURS_BIN="$OURS_BENCH_DIR/target/release/ours-manifest-bench"
|
|
||||||
|
|
||||||
echo "[2/3] Build routinator benchmark (release)..." | tee -a "$OUT_RAW"
|
|
||||||
(cd "$ROUT_BENCH_DIR" && cargo build --release -q)
|
|
||||||
|
|
||||||
taskset_prefix=""
|
|
||||||
if command -v taskset >/dev/null 2>&1; then
|
|
||||||
if [[ -n "${TASKSET_CPU:-}" ]]; then
|
|
||||||
taskset_prefix="taskset -c ${TASKSET_CPU}"
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
|
|
||||||
bucket_for() {
|
|
||||||
local s="$1"
|
|
||||||
case "$s" in
|
|
||||||
small-*) echo "small" ;;
|
|
||||||
medium-*) echo "medium" ;;
|
|
||||||
large-*) echo "large" ;;
|
|
||||||
xlarge-*) echo "xlarge" ;;
|
|
||||||
*) echo "unknown" ;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
|
|
||||||
iters_for() {
|
|
||||||
local b="$1"
|
|
||||||
case "$b" in
|
|
||||||
small) echo "$ITER_SMALL" ;;
|
|
||||||
medium) echo "$ITER_MEDIUM" ;;
|
|
||||||
large) echo "$ITER_LARGE" ;;
|
|
||||||
xlarge) echo "$ITER_XLARGE" ;;
|
|
||||||
*) echo "$ITER_MEDIUM" ;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
|
|
||||||
warm_for() {
|
|
||||||
local b="$1"
|
|
||||||
case "$b" in
|
|
||||||
small) echo "$WARM_SMALL" ;;
|
|
||||||
medium) echo "$WARM_MEDIUM" ;;
|
|
||||||
large) echo "$WARM_LARGE" ;;
|
|
||||||
xlarge) echo "$WARM_XLARGE" ;;
|
|
||||||
*) echo "$WARM_MEDIUM" ;;
|
|
||||||
esac
|
|
||||||
}
|
|
||||||
|
|
||||||
run_ours() {
|
|
||||||
local sample="$1"
|
|
||||||
local iters="$2"
|
|
||||||
local warm="$3"
|
|
||||||
local ours_fixture="$RPKI_DIR/tests/benchmark/selected_der/${sample}.mft"
|
|
||||||
if [[ ! -f "$ours_fixture" ]]; then
|
|
||||||
echo "ours fixture not found: $ours_fixture" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
echo "### ours $sample" >> "$OUT_RAW"
|
|
||||||
local out
|
|
||||||
out=$($taskset_prefix "$OURS_BIN" --manifest "$ours_fixture" --iterations "$iters" --warmup-iterations "$warm" --repeats "$REPEATS")
|
|
||||||
echo "$out" >> "$OUT_RAW"
|
|
||||||
|
|
||||||
local line
|
|
||||||
line=$(echo "$out" | rg "^\\| ${sample} \\|" | tail -n 1)
|
|
||||||
if [[ -z "${line:-}" ]]; then
|
|
||||||
echo "failed to parse ours output for $sample" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
# Expected final row: | sample | avg ns/op | ops/s | file count |
|
|
||||||
local avg ops cnt
|
|
||||||
avg=$(echo "$line" | awk -F'|' '{gsub(/^ +| +$/,"",$3); print $3}')
|
|
||||||
ops=$(echo "$line" | awk -F'|' '{gsub(/^ +| +$/,"",$4); print $4}')
|
|
||||||
cnt=$(echo "$line" | awk -F'|' '{gsub(/^ +| +$/,"",$5); print $5}')
|
|
||||||
echo "$avg,$ops,$cnt"
|
|
||||||
}
|
|
||||||
|
|
||||||
run_rout() {
|
|
||||||
local sample="$1"
|
|
||||||
local iters="$2"
|
|
||||||
local warm="$3"
|
|
||||||
local rout_fixture="$ROUT_BENCH_DIR/fixtures/selected_der/${sample}.mft"
|
|
||||||
if [[ ! -f "$rout_fixture" ]]; then
|
|
||||||
echo "routinator fixture not found: $rout_fixture" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
echo "### routinator $sample" >> "$OUT_RAW"
|
|
||||||
local out
|
|
||||||
out=$(
|
|
||||||
$taskset_prefix "$ROUT_BIN" \
|
|
||||||
--target decode_only \
|
|
||||||
--manifest "$rout_fixture" \
|
|
||||||
--issuer "$ROUT_BENCH_DIR/fixtures/ta.cer" \
|
|
||||||
--iterations "$iters" \
|
|
||||||
--repeats "$REPEATS" \
|
|
||||||
--warmup-iterations "$warm" \
|
|
||||||
--strict false
|
|
||||||
)
|
|
||||||
echo "$out" >> "$OUT_RAW"
|
|
||||||
|
|
||||||
local avg_line cnt_line
|
|
||||||
avg_line=$(echo "$out" | rg "^ avg:")
|
|
||||||
cnt_line=$(echo "$out" | rg "^ manifest_file_count:")
|
|
||||||
|
|
||||||
local avg_ns ops_s cnt
|
|
||||||
avg_ns=$(echo "$avg_line" | awk '{print $2}')
|
|
||||||
ops_s=$(echo "$avg_line" | awk '{gsub(/[()]/,"",$4); print $4}')
|
|
||||||
cnt=$(echo "$cnt_line" | awk '{print $2}')
|
|
||||||
echo "$avg_ns,$ops_s,$cnt"
|
|
||||||
}
|
|
||||||
|
|
||||||
echo "[3/3] Run per-sample benchmarks..." | tee -a "$OUT_RAW"
|
|
||||||
for s in "${SAMPLES[@]}"; do
|
|
||||||
b=$(bucket_for "$s")
|
|
||||||
it=$(iters_for "$b")
|
|
||||||
warm=$(warm_for "$b")
|
|
||||||
|
|
||||||
IFS=, read -r ours_avg ours_ops ours_cnt < <(run_ours "$s" "$it" "$warm")
|
|
||||||
IFS=, read -r rout_avg rout_ops rout_cnt < <(run_rout "$s" "$it" "$warm")
|
|
||||||
|
|
||||||
if [[ "$ours_cnt" != "$rout_cnt" ]]; then
|
|
||||||
echo "WARNING: file count differs for $s (ours=$ours_cnt rout=$rout_cnt)" | tee -a "$OUT_RAW"
|
|
||||||
fi
|
|
||||||
|
|
||||||
ratio=$(python3 - <<PY
|
|
||||||
o=float("$ours_avg")
|
|
||||||
r=float("$rout_avg")
|
|
||||||
print(f"{(o/r):.4f}" if r != 0 else "inf")
|
|
||||||
PY
|
|
||||||
)
|
|
||||||
|
|
||||||
echo "$s,$b,$ours_cnt,$ours_avg,$ours_ops,$rout_avg,$rout_ops,$ratio,$it,$REPEATS,$warm" >> "$OUT_CSV"
|
|
||||||
echo >> "$OUT_RAW"
|
|
||||||
done
|
|
||||||
|
|
||||||
echo "Done."
|
|
||||||
echo "- CSV: $OUT_CSV"
|
|
||||||
echo "- Raw: $OUT_RAW"
|
|
||||||
@@ -1,142 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
# M3: Generate flamegraphs + top hotspots for Manifest decode+profile (Ours vs Routinator).
|
|
||||||
#
|
|
||||||
# Outputs under:
|
|
||||||
# specs/develop/20260224/flamegraph/
|
|
||||||
# specs/develop/20260224/hotspots/
|
|
||||||
# specs/develop/20260224/perf/
|
|
||||||
#
|
|
||||||
# Notes:
|
|
||||||
# - On WSL2, /usr/bin/perf is often a wrapper that fails. This script uses a real perf binary
|
|
||||||
# from /usr/lib/linux-tools/*/perf (if present).
|
|
||||||
# - Ours profiling uses perf + flamegraph --perfdata to avoid rebuilding the whole crate graph
|
|
||||||
# with RocksDB.
|
|
||||||
|
|
||||||
ROOT_REPO="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
RPKI_DIR="$ROOT_REPO/rpki"
|
|
||||||
|
|
||||||
DATE_TAG="${DATE_TAG:-20260224}"
|
|
||||||
OUT_BASE="$ROOT_REPO/specs/develop/${DATE_TAG}"
|
|
||||||
OUT_FLAME="$OUT_BASE/flamegraph"
|
|
||||||
OUT_HOT="$OUT_BASE/hotspots"
|
|
||||||
OUT_PERF="$OUT_BASE/perf"
|
|
||||||
|
|
||||||
RUN_TAG="${RUN_TAG:-p2}"
|
|
||||||
|
|
||||||
OURS_BENCH_DIR="$RPKI_DIR/benchmark/ours_manifest_bench"
|
|
||||||
OURS_BIN="$OURS_BENCH_DIR/target/release/ours-manifest-bench"
|
|
||||||
|
|
||||||
ROUT_BENCH_DIR="${ROUT_BENCH_DIR:-/home/yuyr/dev/rust_playground/routinator/benchmark}"
|
|
||||||
ROUT_BIN="$ROUT_BENCH_DIR/target/release/routinator-manifest-benchmark"
|
|
||||||
ROUT_ISSUER="$ROUT_BENCH_DIR/fixtures/ta.cer"
|
|
||||||
|
|
||||||
PROFILE_HZ="${PROFILE_HZ:-99}"
|
|
||||||
|
|
||||||
mkdir -p "$OUT_FLAME" "$OUT_HOT" "$OUT_PERF"
|
|
||||||
|
|
||||||
PERF_WRAPPER_OUT="$(perf --version 2>&1 || true)"
|
|
||||||
PERF_REAL=""
|
|
||||||
if echo "${PERF_WRAPPER_OUT}" | grep -q "WARNING: perf not found for kernel"; then
|
|
||||||
PERF_REAL="$(ls -1 /usr/lib/linux-tools/*/perf 2>/dev/null | head -n 1 || true)"
|
|
||||||
else
|
|
||||||
PERF_REAL="$(command -v perf || true)"
|
|
||||||
fi
|
|
||||||
|
|
||||||
if [[ -z "${PERF_REAL}" ]]; then
|
|
||||||
echo "ERROR: usable perf binary not found (wrapper detected and no /usr/lib/linux-tools/*/perf)." >&2
|
|
||||||
exit 2
|
|
||||||
fi
|
|
||||||
|
|
||||||
SHIM_DIR="$RPKI_DIR/target/bench/tools"
|
|
||||||
mkdir -p "$SHIM_DIR"
|
|
||||||
cat > "$SHIM_DIR/perf" <<EOF
|
|
||||||
#!/usr/bin/env bash
|
|
||||||
exec "${PERF_REAL}" "\$@"
|
|
||||||
EOF
|
|
||||||
chmod +x "$SHIM_DIR/perf"
|
|
||||||
export PATH="$SHIM_DIR:$PATH"
|
|
||||||
|
|
||||||
echo "Using perf: $PERF_REAL"
|
|
||||||
|
|
||||||
echo "[1/3] Build ours benchmark with frame pointers..."
|
|
||||||
(cd "$OURS_BENCH_DIR" && RUSTFLAGS="-C force-frame-pointers=yes" cargo build --release -q)
|
|
||||||
|
|
||||||
echo "[2/3] Build routinator benchmark (release)..."
|
|
||||||
(cd "$ROUT_BENCH_DIR" && cargo build --release -q)
|
|
||||||
|
|
||||||
taskset_prefix=""
|
|
||||||
if command -v taskset >/dev/null 2>&1; then
|
|
||||||
taskset_prefix="taskset -c 0"
|
|
||||||
fi
|
|
||||||
|
|
||||||
profile_ours() {
|
|
||||||
local sample="$1"
|
|
||||||
local iters="$2"
|
|
||||||
local warm="$3"
|
|
||||||
local fixture="$RPKI_DIR/tests/benchmark/selected_der/${sample}.mft"
|
|
||||||
if [[ ! -f "$fixture" ]]; then
|
|
||||||
echo "ERROR: ours fixture not found: $fixture" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
local perfdata="$OUT_PERF/ours_${sample}_${RUN_TAG}.perf.data"
|
|
||||||
local svg="$OUT_FLAME/ours_${sample}_${RUN_TAG}.svg"
|
|
||||||
local tsv="$OUT_HOT/ours_${sample}_${RUN_TAG}.tsv"
|
|
||||||
|
|
||||||
echo "== ours $sample (iters=$iters warmup=$warm hz=$PROFILE_HZ)"
|
|
||||||
$taskset_prefix perf record -o "$perfdata" -F "$PROFILE_HZ" -g -- \
|
|
||||||
"$OURS_BIN" --manifest "$fixture" --iterations "$iters" --warmup-iterations "$warm" --repeats 1 >/dev/null
|
|
||||||
|
|
||||||
flamegraph --perfdata "$perfdata" --output "$svg" --title "ours ${sample} ManifestObject::decode_der" --deterministic >/dev/null
|
|
||||||
|
|
||||||
perf report -i "$perfdata" --stdio --no-children --sort symbol --percent-limit 0.5 \
|
|
||||||
| awk '/^[[:space:]]*[0-9.]+%/ {pct=$1; sub(/%/,"",pct); $1=""; sub(/^[[:space:]]+/,""); print pct "\t" $0}' \
|
|
||||||
> "$tsv"
|
|
||||||
}
|
|
||||||
|
|
||||||
profile_routinator() {
|
|
||||||
local sample="$1"
|
|
||||||
local iters="$2"
|
|
||||||
local warm="$3"
|
|
||||||
local fixture="$ROUT_BENCH_DIR/fixtures/selected_der/${sample}.mft"
|
|
||||||
if [[ ! -f "$fixture" ]]; then
|
|
||||||
echo "ERROR: routinator fixture not found: $fixture" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
local svg="$OUT_FLAME/routinator_${sample}_${RUN_TAG}.svg"
|
|
||||||
local tsv="$OUT_HOT/routinator_${sample}_${RUN_TAG}.tsv"
|
|
||||||
|
|
||||||
echo "== routinator $sample (iters=$iters warmup=$warm hz=$PROFILE_HZ)"
|
|
||||||
$taskset_prefix "$ROUT_BIN" \
|
|
||||||
--target decode_only \
|
|
||||||
--manifest "$fixture" \
|
|
||||||
--issuer "$ROUT_ISSUER" \
|
|
||||||
--iterations "$iters" \
|
|
||||||
--repeats 1 \
|
|
||||||
--warmup-iterations "$warm" \
|
|
||||||
--strict false \
|
|
||||||
--profile-hz "$PROFILE_HZ" \
|
|
||||||
--flamegraph "$svg" \
|
|
||||||
--hotspots "$tsv" \
|
|
||||||
>/dev/null
|
|
||||||
}
|
|
||||||
|
|
||||||
echo "[3/3] Profile samples..."
|
|
||||||
|
|
||||||
# Choose iterations so each capture runs ~10-20s serially.
|
|
||||||
profile_ours small-01 200000 0
|
|
||||||
profile_routinator small-01 200000 0
|
|
||||||
|
|
||||||
profile_ours large-02 50000 0
|
|
||||||
profile_routinator large-02 50000 0
|
|
||||||
|
|
||||||
profile_ours xlarge-02 5000 0
|
|
||||||
profile_routinator xlarge-02 5000 0
|
|
||||||
|
|
||||||
echo "Done."
|
|
||||||
echo "- Flamegraphs: $OUT_FLAME/"
|
|
||||||
echo "- Hotspots: $OUT_HOT/"
|
|
||||||
echo "- Perf data: $OUT_PERF/"
|
|
||||||
@@ -1,97 +0,0 @@
|
|||||||
# Manual RRDP sync (APNIC-focused)
|
|
||||||
|
|
||||||
This directory contains **manual, command-line** scripts to reproduce the workflow described in:
|
|
||||||
|
|
||||||
- `specs/develop/20260226/apnic_rrdp_delta_analysis_after_manifest_revalidation_fix_20260227T022606Z.md`
|
|
||||||
|
|
||||||
They are meant for **hands-on validation / acceptance runs**, not for CI.
|
|
||||||
|
|
||||||
## Prerequisites
|
|
||||||
|
|
||||||
- Rust toolchain (`cargo`)
|
|
||||||
- `rsync` available on PATH (for rsync fallback/objects)
|
|
||||||
- Network access (RRDP over HTTPS)
|
|
||||||
|
|
||||||
## What the scripts do
|
|
||||||
|
|
||||||
### `full_sync.sh`
|
|
||||||
|
|
||||||
- Creates a fresh RocksDB directory
|
|
||||||
- Runs a **full serial** validation from a TAL URL (default: APNIC RFC7730 TAL)
|
|
||||||
- Writes:
|
|
||||||
- run log
|
|
||||||
- audit report JSON
|
|
||||||
- run meta JSON (includes durations + download_stats)
|
|
||||||
- short summary Markdown (includes durations + download_stats)
|
|
||||||
- RocksDB key statistics (`db_stats --exact`)
|
|
||||||
- RRDP legacy session/serial dump (`rrdp_state_dump --view legacy-state`)
|
|
||||||
|
|
||||||
### `delta_sync.sh`
|
|
||||||
|
|
||||||
- Copies an existing “baseline snapshot DB” to a new DB directory (so the baseline is not modified)
|
|
||||||
- Runs another validation against the copied DB (RRDP will prefer **delta** when available)
|
|
||||||
- Produces the same artifacts as `full_sync.sh`
|
|
||||||
- Additionally generates a Markdown **delta analysis** report by comparing:
|
|
||||||
- base vs delta report JSON
|
|
||||||
- base vs delta `rrdp_state_dump --view legacy-state` TSV
|
|
||||||
- and includes a **duration comparison** (base vs delta) if the base meta JSON is available
|
|
||||||
- delta meta JSON includes download_stats copied from delta report JSON
|
|
||||||
|
|
||||||
## Audit report fields (report.json)
|
|
||||||
|
|
||||||
The `rpki` binary writes an audit report JSON with:
|
|
||||||
|
|
||||||
- `format_version: 2`
|
|
||||||
- `downloads`: per-download RRDP/rsync events (URI, timestamps, duration, ok/fail, error, bytes, objects stats)
|
|
||||||
- `download_stats`: aggregate counters (by kind)
|
|
||||||
|
|
||||||
These are useful for diagnosing why a run is slow (e.g. RRDP snapshot vs delta vs rsync fallback).
|
|
||||||
|
|
||||||
The standalone `rrdp_state_dump` tool also supports `source`, `members`, `owners`, and `all` views.
|
|
||||||
The manual sync scripts intentionally call `--view legacy-state` so delta analysis keeps using a stable session/serial TSV format.
|
|
||||||
|
|
||||||
## Meta fields (meta.json)
|
|
||||||
|
|
||||||
The scripts generate `*_meta.json` next to `*_report.json` and include:
|
|
||||||
|
|
||||||
- `durations_secs`: wall-clock duration breakdown for the script steps
|
|
||||||
- `download_stats`: copied from `report_json.download_stats`
|
|
||||||
|
|
||||||
## Usage
|
|
||||||
|
|
||||||
Run from `rpki/`:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./scripts/manual_sync/full_sync.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
After you have a baseline run, run delta against it:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./scripts/manual_sync/delta_sync.sh target/live/manual_sync/apnic_full_db_YYYYMMDDTHHMMSSZ \
|
|
||||||
target/live/manual_sync/apnic_full_report_YYYYMMDDTHHMMSSZ.json
|
|
||||||
```
|
|
||||||
|
|
||||||
If the baseline was produced by `full_sync.sh`, the delta script will auto-discover the base meta JSON
|
|
||||||
next to the base report (by replacing `_report.json` with `_meta.json`) and include base durations in
|
|
||||||
the delta analysis report.
|
|
||||||
|
|
||||||
## Configuration (env vars)
|
|
||||||
|
|
||||||
Both scripts accept overrides via env vars:
|
|
||||||
|
|
||||||
- `TAL_URL` (default: APNIC TAL URL)
|
|
||||||
- `HTTP_TIMEOUT_SECS` (default: 1800)
|
|
||||||
- `RSYNC_TIMEOUT_SECS` (default: 1800)
|
|
||||||
- `RSYNC_MIRROR_ROOT` (default: disabled; when set, passes `--rsync-mirror-root` to `rpki`)
|
|
||||||
- `VALIDATION_TIME` (RFC3339; default: now UTC)
|
|
||||||
- `OUT_DIR` (default: `rpki/target/live/manual_sync`)
|
|
||||||
- `RUN_NAME` (default: auto timestamped)
|
|
||||||
|
|
||||||
Example:
|
|
||||||
|
|
||||||
```bash
|
|
||||||
TAL_URL="https://tal.apnic.net/tal-archive/apnic-rfc7730-https.tal" \
|
|
||||||
HTTP_TIMEOUT_SECS=1800 RSYNC_TIMEOUT_SECS=1800 \
|
|
||||||
./scripts/manual_sync/full_sync.sh
|
|
||||||
```
|
|
||||||
@@ -1,543 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
# Delta sync + validation starting from a baseline snapshot DB.
|
|
||||||
#
|
|
||||||
# This script:
|
|
||||||
# 1) Copies BASE_DB_DIR -> DELTA_DB_DIR (so baseline is not modified)
|
|
||||||
# 2) Runs rpki validation again (RRDP will prefer delta if available)
|
|
||||||
# 3) Writes artifacts + a markdown delta analysis report
|
|
||||||
#
|
|
||||||
# Usage:
|
|
||||||
# ./scripts/manual_sync/delta_sync.sh <base_db_dir> <base_report_json>
|
|
||||||
#
|
|
||||||
# Outputs under OUT_DIR (default: target/live/manual_sync):
|
|
||||||
# - *_delta_db_* copied RocksDB directory
|
|
||||||
# - *_delta_report_*.json audit report
|
|
||||||
# - *_delta_run_*.log stdout/stderr log (includes summary)
|
|
||||||
# - *_delta_db_stats_*.txt db_stats --exact output
|
|
||||||
# - *_delta_rrdp_state_*.tsv rrdp_state_dump --view legacy-state output
|
|
||||||
# - *_delta_analysis_*.md base vs delta comparison report
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
cd "$ROOT_DIR"
|
|
||||||
|
|
||||||
BASE_DB_DIR="${1:-}"
|
|
||||||
BASE_REPORT_JSON="${2:-}"
|
|
||||||
if [[ -z "${BASE_DB_DIR}" || -z "${BASE_REPORT_JSON}" ]]; then
|
|
||||||
echo "Usage: $0 <base_db_dir> <base_report_json>" >&2
|
|
||||||
exit 2
|
|
||||||
fi
|
|
||||||
if [[ ! -d "${BASE_DB_DIR}" ]]; then
|
|
||||||
echo "ERROR: base_db_dir is not a directory: ${BASE_DB_DIR}" >&2
|
|
||||||
exit 2
|
|
||||||
fi
|
|
||||||
if [[ ! -f "${BASE_REPORT_JSON}" ]]; then
|
|
||||||
echo "ERROR: base_report_json does not exist: ${BASE_REPORT_JSON}" >&2
|
|
||||||
exit 2
|
|
||||||
fi
|
|
||||||
|
|
||||||
TAL_URL="${TAL_URL:-https://tal.apnic.net/tal-archive/apnic-rfc7730-https.tal}"
|
|
||||||
HTTP_TIMEOUT_SECS="${HTTP_TIMEOUT_SECS:-1800}"
|
|
||||||
RSYNC_TIMEOUT_SECS="${RSYNC_TIMEOUT_SECS:-1800}"
|
|
||||||
RSYNC_MIRROR_ROOT="${RSYNC_MIRROR_ROOT:-}"
|
|
||||||
VALIDATION_TIME="${VALIDATION_TIME:-}"
|
|
||||||
|
|
||||||
OUT_DIR="${OUT_DIR:-$ROOT_DIR/target/live/manual_sync}"
|
|
||||||
mkdir -p "$OUT_DIR"
|
|
||||||
|
|
||||||
TS="$(date -u +%Y%m%dT%H%M%SZ)"
|
|
||||||
RUN_NAME="${RUN_NAME:-apnic_delta_${TS}}"
|
|
||||||
|
|
||||||
DELTA_DB_DIR="${DELTA_DB_DIR:-$OUT_DIR/${RUN_NAME}_db}"
|
|
||||||
DELTA_REPORT_JSON="${DELTA_REPORT_JSON:-$OUT_DIR/${RUN_NAME}_report.json}"
|
|
||||||
DELTA_RUN_LOG="${DELTA_RUN_LOG:-$OUT_DIR/${RUN_NAME}_run.log}"
|
|
||||||
|
|
||||||
BASE_DB_STATS_TXT="${BASE_DB_STATS_TXT:-$OUT_DIR/${RUN_NAME}_base_db_stats.txt}"
|
|
||||||
DELTA_DB_STATS_TXT="${DELTA_DB_STATS_TXT:-$OUT_DIR/${RUN_NAME}_delta_db_stats.txt}"
|
|
||||||
|
|
||||||
BASE_RRDP_STATE_TSV="${BASE_RRDP_STATE_TSV:-$OUT_DIR/${RUN_NAME}_base_rrdp_state.tsv}"
|
|
||||||
DELTA_RRDP_STATE_TSV="${DELTA_RRDP_STATE_TSV:-$OUT_DIR/${RUN_NAME}_delta_rrdp_state.tsv}"
|
|
||||||
|
|
||||||
DELTA_ANALYSIS_MD="${DELTA_ANALYSIS_MD:-$OUT_DIR/${RUN_NAME}_delta_analysis.md}"
|
|
||||||
DELTA_META_JSON="${DELTA_META_JSON:-$OUT_DIR/${RUN_NAME}_meta.json}"
|
|
||||||
|
|
||||||
# Best-effort base meta discovery (produced by `full_sync.sh`).
|
|
||||||
BASE_META_JSON="${BASE_META_JSON:-}"
|
|
||||||
if [[ -z "${BASE_META_JSON}" ]]; then
|
|
||||||
guess="${BASE_REPORT_JSON%_report.json}_meta.json"
|
|
||||||
if [[ -f "${guess}" ]]; then
|
|
||||||
BASE_META_JSON="${guess}"
|
|
||||||
fi
|
|
||||||
fi
|
|
||||||
|
|
||||||
echo "== rpki manual delta sync ==" >&2
|
|
||||||
echo "tal_url=$TAL_URL" >&2
|
|
||||||
echo "base_db=$BASE_DB_DIR" >&2
|
|
||||||
echo "base_report=$BASE_REPORT_JSON" >&2
|
|
||||||
echo "delta_db=$DELTA_DB_DIR" >&2
|
|
||||||
echo "delta_report=$DELTA_REPORT_JSON" >&2
|
|
||||||
|
|
||||||
echo "== copying base DB (baseline is not modified) ==" >&2
|
|
||||||
cp -a "$BASE_DB_DIR" "$DELTA_DB_DIR"
|
|
||||||
|
|
||||||
script_start_s="$(date +%s)"
|
|
||||||
run_start_s="$(date +%s)"
|
|
||||||
cmd=(cargo run --release --bin rpki -- \
|
|
||||||
--db "$DELTA_DB_DIR" \
|
|
||||||
--tal-url "$TAL_URL" \
|
|
||||||
--http-timeout-secs "$HTTP_TIMEOUT_SECS" \
|
|
||||||
--rsync-timeout-secs "$RSYNC_TIMEOUT_SECS" \
|
|
||||||
--report-json "$DELTA_REPORT_JSON")
|
|
||||||
if [[ -n "${RSYNC_MIRROR_ROOT}" ]]; then
|
|
||||||
cmd+=(--rsync-mirror-root "$RSYNC_MIRROR_ROOT")
|
|
||||||
fi
|
|
||||||
if [[ -n "${VALIDATION_TIME}" ]]; then
|
|
||||||
cmd+=(--validation-time "$VALIDATION_TIME")
|
|
||||||
fi
|
|
||||||
|
|
||||||
(
|
|
||||||
echo "# command:"
|
|
||||||
printf '%q ' "${cmd[@]}"
|
|
||||||
echo
|
|
||||||
echo
|
|
||||||
"${cmd[@]}"
|
|
||||||
) 2>&1 | tee "$DELTA_RUN_LOG" >/dev/null
|
|
||||||
run_end_s="$(date +%s)"
|
|
||||||
run_duration_s="$((run_end_s - run_start_s))"
|
|
||||||
|
|
||||||
echo "== db_stats (exact) ==" >&2
|
|
||||||
db_stats_start_s="$(date +%s)"
|
|
||||||
cargo run --release --bin db_stats -- --db "$BASE_DB_DIR" --exact 2>&1 | tee "$BASE_DB_STATS_TXT" >/dev/null
|
|
||||||
cargo run --release --bin db_stats -- --db "$DELTA_DB_DIR" --exact 2>&1 | tee "$DELTA_DB_STATS_TXT" >/dev/null
|
|
||||||
db_stats_end_s="$(date +%s)"
|
|
||||||
db_stats_duration_s="$((db_stats_end_s - db_stats_start_s))"
|
|
||||||
|
|
||||||
echo "== rrdp_state_dump (legacy-state) ==" >&2
|
|
||||||
state_start_s="$(date +%s)"
|
|
||||||
cargo run --release --bin rrdp_state_dump -- --db "$BASE_DB_DIR" --view legacy-state >"$BASE_RRDP_STATE_TSV"
|
|
||||||
cargo run --release --bin rrdp_state_dump -- --db "$DELTA_DB_DIR" --view legacy-state >"$DELTA_RRDP_STATE_TSV"
|
|
||||||
state_end_s="$(date +%s)"
|
|
||||||
state_duration_s="$((state_end_s - state_start_s))"
|
|
||||||
|
|
||||||
script_end_s="$(date +%s)"
|
|
||||||
total_duration_s="$((script_end_s - script_start_s))"
|
|
||||||
|
|
||||||
echo "== delta analysis report ==" >&2
|
|
||||||
TAL_URL="$TAL_URL" \
|
|
||||||
BASE_DB_DIR="$BASE_DB_DIR" \
|
|
||||||
DELTA_DB_DIR="$DELTA_DB_DIR" \
|
|
||||||
DELTA_RUN_LOG="$DELTA_RUN_LOG" \
|
|
||||||
VALIDATION_TIME_ARG="$VALIDATION_TIME" \
|
|
||||||
HTTP_TIMEOUT_SECS="$HTTP_TIMEOUT_SECS" \
|
|
||||||
RSYNC_TIMEOUT_SECS="$RSYNC_TIMEOUT_SECS" \
|
|
||||||
RUN_DURATION_S="$run_duration_s" \
|
|
||||||
DB_STATS_DURATION_S="$db_stats_duration_s" \
|
|
||||||
STATE_DURATION_S="$state_duration_s" \
|
|
||||||
TOTAL_DURATION_S="$total_duration_s" \
|
|
||||||
python3 - "$BASE_REPORT_JSON" "$DELTA_REPORT_JSON" "$BASE_RRDP_STATE_TSV" "$DELTA_RRDP_STATE_TSV" \
|
|
||||||
"$BASE_DB_STATS_TXT" "$DELTA_DB_STATS_TXT" "$BASE_META_JSON" "$DELTA_META_JSON" "$DELTA_ANALYSIS_MD" <<'PY'
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import sys
|
|
||||||
from collections import Counter, defaultdict
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
base_report_path = Path(sys.argv[1])
|
|
||||||
delta_report_path = Path(sys.argv[2])
|
|
||||||
base_state_path = Path(sys.argv[3])
|
|
||||||
delta_state_path = Path(sys.argv[4])
|
|
||||||
base_db_stats_path = Path(sys.argv[5])
|
|
||||||
delta_db_stats_path = Path(sys.argv[6])
|
|
||||||
base_meta_path_s = sys.argv[7]
|
|
||||||
delta_meta_path = Path(sys.argv[8])
|
|
||||||
out_md_path = Path(sys.argv[9])
|
|
||||||
|
|
||||||
def load_json(p: Path):
|
|
||||||
s = p.read_text(encoding="utf-8")
|
|
||||||
try:
|
|
||||||
return json.loads(s)
|
|
||||||
except json.JSONDecodeError:
|
|
||||||
# Backwards-compat / robustness: tolerate accidental literal trailing "\\n".
|
|
||||||
s2 = s.strip()
|
|
||||||
if s2.endswith("\\n"):
|
|
||||||
s2 = s2[:-2].rstrip()
|
|
||||||
return json.loads(s2)
|
|
||||||
|
|
||||||
def load_optional_json(path_s: str):
|
|
||||||
if not path_s:
|
|
||||||
return None
|
|
||||||
p = Path(path_s)
|
|
||||||
if not p.exists():
|
|
||||||
return None
|
|
||||||
return load_json(p)
|
|
||||||
|
|
||||||
def parse_rrdp_state_tsv(p: Path):
|
|
||||||
# format from `rrdp_state_dump --view legacy-state`:
|
|
||||||
# [legacy-state]
|
|
||||||
# notify_uri serial session_id
|
|
||||||
# <notify_uri> <serial> <session_id>
|
|
||||||
out = {}
|
|
||||||
for line in p.read_text(encoding="utf-8").splitlines():
|
|
||||||
line = line.strip()
|
|
||||||
if not line or line.startswith("["):
|
|
||||||
continue
|
|
||||||
if line == "notify_uri serial session_id":
|
|
||||||
continue
|
|
||||||
parts = line.split(" ")
|
|
||||||
if len(parts) != 3:
|
|
||||||
raise SystemExit(f"invalid rrdp_state_dump line in {p}: {line!r}")
|
|
||||||
uri, serial, session = parts
|
|
||||||
out[uri] = (int(serial), session)
|
|
||||||
return out
|
|
||||||
|
|
||||||
def parse_db_stats(p: Path):
|
|
||||||
# lines: key=value
|
|
||||||
out = {}
|
|
||||||
for line in p.read_text(encoding="utf-8").splitlines():
|
|
||||||
if "=" not in line:
|
|
||||||
continue
|
|
||||||
k, v = line.split("=", 1)
|
|
||||||
k = k.strip()
|
|
||||||
v = v.strip()
|
|
||||||
if v.isdigit():
|
|
||||||
out[k] = int(v)
|
|
||||||
else:
|
|
||||||
out[k] = v
|
|
||||||
return out
|
|
||||||
|
|
||||||
def warnings_total(rep: dict) -> int:
|
|
||||||
return len(rep["tree"]["warnings"]) + sum(len(pp["warnings"]) for pp in rep["publication_points"])
|
|
||||||
|
|
||||||
def report_summary(rep: dict) -> dict:
|
|
||||||
return {
|
|
||||||
"validation_time": rep["meta"]["validation_time_rfc3339_utc"],
|
|
||||||
"publication_points_processed": rep["tree"]["instances_processed"],
|
|
||||||
"publication_points_failed": rep["tree"]["instances_failed"],
|
|
||||||
"rrdp_repos_unique": len({pp.get("rrdp_notification_uri") for pp in rep["publication_points"] if pp.get("rrdp_notification_uri")}),
|
|
||||||
"vrps": len(rep["vrps"]),
|
|
||||||
"aspas": len(rep["aspas"]),
|
|
||||||
"audit_publication_points": len(rep["publication_points"]),
|
|
||||||
"warnings_total": warnings_total(rep),
|
|
||||||
}
|
|
||||||
|
|
||||||
def count_repo_sync_failed(rep: dict) -> int:
|
|
||||||
# Best-effort heuristic (we don't currently expose a structured counter in the audit report).
|
|
||||||
# Keep the match conservative to avoid false positives.
|
|
||||||
def is_repo_sync_failed(msg: str) -> bool:
|
|
||||||
m = msg.lower()
|
|
||||||
return "repo sync failed" in m or "rrdp fetch failed" in m or "rsync fetch failed" in m
|
|
||||||
|
|
||||||
n = 0
|
|
||||||
for w in rep["tree"]["warnings"]:
|
|
||||||
if is_repo_sync_failed(w.get("message", "")):
|
|
||||||
n += 1
|
|
||||||
for pp in rep["publication_points"]:
|
|
||||||
for w in pp.get("warnings", []):
|
|
||||||
if is_repo_sync_failed(w.get("message", "")):
|
|
||||||
n += 1
|
|
||||||
return n
|
|
||||||
|
|
||||||
def pp_manifest_sha(pp: dict) -> str:
|
|
||||||
# In our audit format, the first object is the manifest (synthetic entry) with sha256 of manifest_bytes.
|
|
||||||
for o in pp["objects"]:
|
|
||||||
if o["kind"] == "manifest":
|
|
||||||
return o["sha256_hex"]
|
|
||||||
return ""
|
|
||||||
|
|
||||||
def pp_objects_by_uri(rep: dict):
|
|
||||||
m = {}
|
|
||||||
for pp in rep["publication_points"]:
|
|
||||||
for o in pp["objects"]:
|
|
||||||
m[o["rsync_uri"]] = (o["sha256_hex"], o["kind"])
|
|
||||||
return m
|
|
||||||
|
|
||||||
def vrp_set(rep: dict):
|
|
||||||
return {(v["asn"], v["prefix"], v["max_length"]) for v in rep["vrps"]}
|
|
||||||
|
|
||||||
def rfc_refs_str(w: dict) -> str:
|
|
||||||
refs = w.get("rfc_refs") or []
|
|
||||||
return ", ".join(refs) if refs else ""
|
|
||||||
|
|
||||||
base = load_json(base_report_path)
|
|
||||||
delta = load_json(delta_report_path)
|
|
||||||
base_sum = report_summary(base)
|
|
||||||
delta_sum = report_summary(delta)
|
|
||||||
|
|
||||||
base_db = parse_db_stats(base_db_stats_path)
|
|
||||||
delta_db = parse_db_stats(delta_db_stats_path)
|
|
||||||
|
|
||||||
base_state = parse_rrdp_state_tsv(base_state_path)
|
|
||||||
delta_state = parse_rrdp_state_tsv(delta_state_path)
|
|
||||||
|
|
||||||
base_meta = load_optional_json(base_meta_path_s)
|
|
||||||
download_stats = delta.get("download_stats") or {}
|
|
||||||
delta_meta = {
|
|
||||||
"recorded_at_utc": datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"),
|
|
||||||
"tal_url": os.environ["TAL_URL"],
|
|
||||||
"base_db_dir": os.environ["BASE_DB_DIR"],
|
|
||||||
"delta_db_dir": os.environ["DELTA_DB_DIR"],
|
|
||||||
"base_report_json": str(base_report_path),
|
|
||||||
"delta_report_json": str(delta_report_path),
|
|
||||||
"delta_run_log": os.environ["DELTA_RUN_LOG"],
|
|
||||||
"validation_time_arg": os.environ.get("VALIDATION_TIME_ARG",""),
|
|
||||||
"http_timeout_secs": int(os.environ["HTTP_TIMEOUT_SECS"]),
|
|
||||||
"rsync_timeout_secs": int(os.environ["RSYNC_TIMEOUT_SECS"]),
|
|
||||||
"durations_secs": {
|
|
||||||
"rpki_run": int(os.environ["RUN_DURATION_S"]),
|
|
||||||
"db_stats_exact": int(os.environ["DB_STATS_DURATION_S"]),
|
|
||||||
"rrdp_state_dump": int(os.environ["STATE_DURATION_S"]),
|
|
||||||
"total_script": int(os.environ["TOTAL_DURATION_S"]),
|
|
||||||
},
|
|
||||||
"download_stats": download_stats,
|
|
||||||
}
|
|
||||||
delta_meta_path.write_text(json.dumps(delta_meta, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
|
||||||
|
|
||||||
now = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
|
||||||
|
|
||||||
# RRDP state changes
|
|
||||||
serial_changed = 0
|
|
||||||
session_changed = 0
|
|
||||||
serial_deltas = []
|
|
||||||
for uri, (old_serial, old_sess) in base_state.items():
|
|
||||||
if uri not in delta_state:
|
|
||||||
continue
|
|
||||||
new_serial, new_sess = delta_state[uri]
|
|
||||||
if new_serial != old_serial:
|
|
||||||
serial_changed += 1
|
|
||||||
serial_deltas.append((uri, old_serial, new_serial, new_serial - old_serial))
|
|
||||||
if new_sess != old_sess:
|
|
||||||
session_changed += 1
|
|
||||||
|
|
||||||
serial_deltas.sort(key=lambda x: x[3], reverse=True)
|
|
||||||
|
|
||||||
# Publication point diffs
|
|
||||||
base_pp = {pp["manifest_rsync_uri"]: pp for pp in base["publication_points"]}
|
|
||||||
delta_pp = {pp["manifest_rsync_uri"]: pp for pp in delta["publication_points"]}
|
|
||||||
|
|
||||||
base_keys = set(base_pp.keys())
|
|
||||||
delta_keys = set(delta_pp.keys())
|
|
||||||
|
|
||||||
new_pp = sorted(delta_keys - base_keys)
|
|
||||||
missing_pp = sorted(base_keys - delta_keys)
|
|
||||||
|
|
||||||
updated_pp = 0
|
|
||||||
unchanged_pp = 0
|
|
||||||
for k in sorted(base_keys & delta_keys):
|
|
||||||
if pp_manifest_sha(base_pp[k]) != pp_manifest_sha(delta_pp[k]):
|
|
||||||
updated_pp += 1
|
|
||||||
else:
|
|
||||||
unchanged_pp += 1
|
|
||||||
|
|
||||||
# Cache usage + repo sync failure hints
|
|
||||||
def source_counts(rep: dict) -> Counter:
|
|
||||||
c = Counter()
|
|
||||||
for pp in rep["publication_points"]:
|
|
||||||
c[pp.get("source","")] += 1
|
|
||||||
return c
|
|
||||||
|
|
||||||
base_sources = source_counts(base)
|
|
||||||
delta_sources = source_counts(delta)
|
|
||||||
base_repo_sync_failed = count_repo_sync_failed(base)
|
|
||||||
delta_repo_sync_failed = count_repo_sync_failed(delta)
|
|
||||||
|
|
||||||
def cache_reason_counts(rep: dict) -> Counter:
|
|
||||||
c = Counter()
|
|
||||||
for pp in rep.get("publication_points", []):
|
|
||||||
if pp.get("source") != "vcir_current_instance":
|
|
||||||
continue
|
|
||||||
# Use warning messages as "reason". If missing, emit a fallback bucket.
|
|
||||||
ws = pp.get("warnings", [])
|
|
||||||
if not ws:
|
|
||||||
c["(no warnings recorded)"] += 1
|
|
||||||
continue
|
|
||||||
for w in ws:
|
|
||||||
msg = w.get("message", "").strip() or "(empty warning message)"
|
|
||||||
c[msg] += 1
|
|
||||||
return c
|
|
||||||
|
|
||||||
base_cache_reasons = cache_reason_counts(base)
|
|
||||||
delta_cache_reasons = cache_reason_counts(delta)
|
|
||||||
|
|
||||||
# Object change stats (by rsync URI, sha256)
|
|
||||||
base_obj = pp_objects_by_uri(base)
|
|
||||||
delta_obj = pp_objects_by_uri(delta)
|
|
||||||
|
|
||||||
kind_stats = {k: {"added": 0, "changed": 0, "removed": 0} for k in ["manifest","crl","certificate","roa","aspa","other"]}
|
|
||||||
all_uris = set(base_obj.keys()) | set(delta_obj.keys())
|
|
||||||
for uri in all_uris:
|
|
||||||
b = base_obj.get(uri)
|
|
||||||
d = delta_obj.get(uri)
|
|
||||||
if b is None and d is not None:
|
|
||||||
kind_stats[d[1]]["added"] += 1
|
|
||||||
elif b is not None and d is None:
|
|
||||||
kind_stats[b[1]]["removed"] += 1
|
|
||||||
else:
|
|
||||||
if b[0] != d[0]:
|
|
||||||
kind_stats[d[1]]["changed"] += 1
|
|
||||||
|
|
||||||
# VRP diff
|
|
||||||
base_v = vrp_set(base)
|
|
||||||
delta_v = vrp_set(delta)
|
|
||||||
added_v = delta_v - base_v
|
|
||||||
removed_v = base_v - delta_v
|
|
||||||
|
|
||||||
def fmt_db_stats(db: dict) -> str:
|
|
||||||
ordered = [
|
|
||||||
"mode",
|
|
||||||
"repository_view",
|
|
||||||
"raw_by_hash",
|
|
||||||
"vcir",
|
|
||||||
"audit_rule_index",
|
|
||||||
"rrdp_source",
|
|
||||||
"rrdp_source_member",
|
|
||||||
"rrdp_uri_owner",
|
|
||||||
"rrdp_state",
|
|
||||||
"raw_objects",
|
|
||||||
"rrdp_object_index",
|
|
||||||
"group_current_repository_view",
|
|
||||||
"group_current_validation_state",
|
|
||||||
"group_current_rrdp_state",
|
|
||||||
"group_legacy_compatibility",
|
|
||||||
"total",
|
|
||||||
"sst_files",
|
|
||||||
]
|
|
||||||
out = []
|
|
||||||
seen = set()
|
|
||||||
for k in ordered:
|
|
||||||
if k in db:
|
|
||||||
out.append(f"- `{k}={db[k]}`")
|
|
||||||
seen.add(k)
|
|
||||||
for k in sorted(set(db) - seen):
|
|
||||||
out.append(f"- `{k}={db[k]}`")
|
|
||||||
return "\n".join(out) if out else "_(missing db_stats keys)_"
|
|
||||||
|
|
||||||
lines = []
|
|
||||||
lines.append("# APNIC RRDP 增量同步验收（manual_sync）\n\n")
|
|
||||||
lines.append(f"时间戳：`{now}`（UTC）\n\n")
|
|
||||||
|
|
||||||
lines.append("## 复现信息\n\n")
|
|
||||||
lines.append(f"- base_report：`{base_report_path}`\n")
|
|
||||||
lines.append(f"- delta_report：`{delta_report_path}`\n")
|
|
||||||
lines.append(f"- base_db_stats：`{base_db_stats_path}`\n")
|
|
||||||
lines.append(f"- delta_db_stats：`{delta_db_stats_path}`\n")
|
|
||||||
lines.append(f"- base_rrdp_state：`{base_state_path}`\n")
|
|
||||||
lines.append(f"- delta_rrdp_state：`{delta_state_path}`\n\n")
|
|
||||||
|
|
||||||
lines.append("## 运行结果概览\n\n")
|
|
||||||
lines.append("| metric | base | delta |\n")
|
|
||||||
lines.append("|---|---:|---:|\n")
|
|
||||||
for k in [
|
|
||||||
"validation_time",
|
|
||||||
"publication_points_processed",
|
|
||||||
"publication_points_failed",
|
|
||||||
"rrdp_repos_unique",
|
|
||||||
"vrps",
|
|
||||||
"aspas",
|
|
||||||
"audit_publication_points",
|
|
||||||
"warnings_total",
|
|
||||||
]:
|
|
||||||
lines.append(f"| {k} | {base_sum[k]} | {delta_sum[k]} |\n")
|
|
||||||
lines.append("\n")
|
|
||||||
|
|
||||||
def dur(meta: dict | None, key: str):
|
|
||||||
if not meta:
|
|
||||||
return None
|
|
||||||
return (meta.get("durations_secs") or {}).get(key)
|
|
||||||
|
|
||||||
base_rpki_run = dur(base_meta, "rpki_run")
|
|
||||||
delta_rpki_run = delta_meta["durations_secs"]["rpki_run"]
|
|
||||||
base_total = dur(base_meta, "total_script")
|
|
||||||
delta_total = delta_meta["durations_secs"]["total_script"]
|
|
||||||
|
|
||||||
lines.append("## 持续时间（seconds）\n\n")
|
|
||||||
lines.append("| step | base | delta |\n")
|
|
||||||
lines.append("|---|---:|---:|\n")
|
|
||||||
lines.append(f"| rpki_run | {base_rpki_run if base_rpki_run is not None else 'unknown'} | {delta_rpki_run} |\n")
|
|
||||||
lines.append(f"| total_script | {base_total if base_total is not None else 'unknown'} | {delta_total} |\n")
|
|
||||||
lines.append("\n")
|
|
||||||
if base_meta is None and base_meta_path_s:
|
|
||||||
lines.append(f"> 注：未能读取 base meta：`{base_meta_path_s}`（文件不存在或不可读）。建议用 `full_sync.sh` 生成 baseline 以获得 base 时长对比。\n\n")
|
|
||||||
|
|
||||||
lines.append("RocksDB KV（`db_stats --exact`）：\n\n")
|
|
||||||
lines.append("### 基线（base）\n\n")
|
|
||||||
lines.append(fmt_db_stats(base_db) + "\n\n")
|
|
||||||
lines.append("### 增量（delta）\n\n")
|
|
||||||
lines.append(fmt_db_stats(delta_db) + "\n\n")
|
|
||||||
|
|
||||||
lines.append("## RRDP 增量是否发生（基于 `rrdp_state` 变化）\n\n")
|
|
||||||
lines.append(f"- repo_total(base)={len(base_state)}\n")
|
|
||||||
lines.append(f"- repo_total(delta)={len(delta_state)}\n")
|
|
||||||
lines.append(f"- serial_changed={serial_changed}\n")
|
|
||||||
lines.append(f"- session_changed={session_changed}\n\n")
|
|
||||||
if serial_deltas:
|
|
||||||
lines.append("serial 增长最大的 10 个 RRDP repo（old -> new）：\n\n")
|
|
||||||
for uri, old, new, diff in serial_deltas[:10]:
|
|
||||||
lines.append(f"- `{uri}`：`{old} -> {new}`（+{diff}）\n")
|
|
||||||
lines.append("\n")
|
|
||||||
|
|
||||||
lines.append("## 发布点（Publication Point）变化统计\n\n")
|
|
||||||
lines.append("以 `manifest_rsync_uri` 作为发布点 key，对比 base vs delta：\n\n")
|
|
||||||
lines.append(f"- base PP：`{len(base_keys)}`\n")
|
|
||||||
lines.append(f"- delta PP：`{len(delta_keys)}`\n")
|
|
||||||
lines.append(f"- `new_pp={len(new_pp)}`\n")
|
|
||||||
lines.append(f"- `missing_pp={len(missing_pp)}`\n")
|
|
||||||
lines.append(f"- `updated_pp={updated_pp}`\n")
|
|
||||||
lines.append(f"- `unchanged_pp={unchanged_pp}`\n\n")
|
|
||||||
lines.append("> 注：`new_pp/missing_pp/updated_pp` 会混入“遍历范围变化”的影响（例如 validation_time 不同、或 base 中存在更多失败 PP）。\n\n")
|
|
||||||
|
|
||||||
lines.append("## fail fetch / VCIR 当前实例缓存复用情况\n\n")
|
|
||||||
lines.append(f"- repo sync failed（启发式：warning contains 'repo sync failed'/'rrdp fetch failed'/'rsync fetch failed'）\n")
|
|
||||||
lines.append(f" - base：`{base_repo_sync_failed}`\n")
|
|
||||||
lines.append(f" - delta：`{delta_repo_sync_failed}`\n\n")
|
|
||||||
|
|
||||||
lines.append("- source 计数（按 `PublicationPointAudit.source`）：\n\n")
|
|
||||||
lines.append(f" - base：`{dict(base_sources)}`\n")
|
|
||||||
lines.append(f" - delta：`{dict(delta_sources)}`\n\n")
|
|
||||||
|
|
||||||
def render_cache_reasons(title: str, c: Counter) -> str:
|
|
||||||
if not c:
|
|
||||||
return f"{title}：`0`（未使用 VCIR 当前实例缓存复用）\n\n"
|
|
||||||
lines = []
|
|
||||||
total = sum(c.values())
|
|
||||||
lines.append(f"{title}：`{total}`\n\n")
|
|
||||||
lines.append("Top reasons（按 warning message 聚合，可能一条 PP 有多条 warning）：\n\n")
|
|
||||||
for msg, n in c.most_common(10):
|
|
||||||
lines.append(f"- `{n}` × {msg}\n")
|
|
||||||
lines.append("\n")
|
|
||||||
return "".join(lines)
|
|
||||||
|
|
||||||
lines.append(render_cache_reasons("- base `source=vcir_current_instance`", base_cache_reasons))
|
|
||||||
lines.append(render_cache_reasons("- delta `source=vcir_current_instance`", delta_cache_reasons))
|
|
||||||
|
|
||||||
lines.append("## 文件变更统计（按对象类型）\n\n")
|
|
||||||
lines.append("按 `ObjectAuditEntry.sha256_hex` 对比（同一 rsync URI 前后 hash 变化记为 `~changed`）：\n\n")
|
|
||||||
lines.append("| kind | added | changed | removed |\n")
|
|
||||||
lines.append("|---|---:|---:|---:|\n")
|
|
||||||
for kind in ["manifest","crl","certificate","roa","aspa","other"]:
|
|
||||||
st = kind_stats[kind]
|
|
||||||
lines.append(f"| {kind} | {st['added']} | {st['changed']} | {st['removed']} |\n")
|
|
||||||
lines.append("\n")
|
|
||||||
|
|
||||||
lines.append("## VRP 影响（去重后集合 diff）\n\n")
|
|
||||||
lines.append("以 `(asn, prefix, max_length)` 为 key：\n\n")
|
|
||||||
lines.append(f"- base unique VRP：`{len(base_v)}`\n")
|
|
||||||
lines.append(f"- delta unique VRP：`{len(delta_v)}`\n")
|
|
||||||
lines.append(f"- `added={len(added_v)}`\n")
|
|
||||||
lines.append(f"- `removed={len(removed_v)}`\n")
|
|
||||||
lines.append(f"- `net={len(added_v) - len(removed_v)}`\n\n")
|
|
||||||
|
|
||||||
out_md_path.write_text("".join(lines), encoding="utf-8")
|
|
||||||
print(out_md_path)
|
|
||||||
PY
|
|
||||||
|
|
||||||
echo "== done ==" >&2
|
|
||||||
echo "artifacts:" >&2
|
|
||||||
echo "- delta db: $DELTA_DB_DIR" >&2
|
|
||||||
echo "- delta report: $DELTA_REPORT_JSON" >&2
|
|
||||||
echo "- delta run log: $DELTA_RUN_LOG" >&2
|
|
||||||
echo "- delta meta json: $DELTA_META_JSON" >&2
|
|
||||||
echo "- analysis md: $DELTA_ANALYSIS_MD" >&2
|
|
||||||
echo "- base state tsv: $BASE_RRDP_STATE_TSV" >&2
|
|
||||||
echo "- delta state tsv: $DELTA_RRDP_STATE_TSV" >&2
|
|
||||||
@@ -1,189 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
# Full sync + validation from a TAL URL (default: APNIC).
|
|
||||||
#
|
|
||||||
# Produces artifacts under OUT_DIR (default: target/live/manual_sync):
|
|
||||||
# - *_db_* RocksDB directory
|
|
||||||
# - *_report_*.json audit report
|
|
||||||
# - *_run_*.log stdout/stderr log (includes summary)
|
|
||||||
# - *_db_stats_*.txt db_stats --exact output
|
|
||||||
# - *_rrdp_state_*.tsv rrdp_state_dump --view legacy-state output
|
|
||||||
|
|
||||||
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
|
|
||||||
cd "$ROOT_DIR"
|
|
||||||
|
|
||||||
TAL_URL="${TAL_URL:-https://tal.apnic.net/tal-archive/apnic-rfc7730-https.tal}"
|
|
||||||
HTTP_TIMEOUT_SECS="${HTTP_TIMEOUT_SECS:-1800}"
|
|
||||||
RSYNC_TIMEOUT_SECS="${RSYNC_TIMEOUT_SECS:-1800}"
|
|
||||||
RSYNC_MIRROR_ROOT="${RSYNC_MIRROR_ROOT:-}"
|
|
||||||
VALIDATION_TIME="${VALIDATION_TIME:-}"
|
|
||||||
|
|
||||||
OUT_DIR="${OUT_DIR:-$ROOT_DIR/target/live/manual_sync}"
|
|
||||||
mkdir -p "$OUT_DIR"
|
|
||||||
|
|
||||||
TS="$(date -u +%Y%m%dT%H%M%SZ)"
|
|
||||||
RUN_NAME="${RUN_NAME:-apnic_full_${TS}}"
|
|
||||||
|
|
||||||
DB_DIR="${DB_DIR:-$OUT_DIR/${RUN_NAME}_db}"
|
|
||||||
REPORT_JSON="${REPORT_JSON:-$OUT_DIR/${RUN_NAME}_report.json}"
|
|
||||||
RUN_LOG="${RUN_LOG:-$OUT_DIR/${RUN_NAME}_run.log}"
|
|
||||||
DB_STATS_TXT="${DB_STATS_TXT:-$OUT_DIR/${RUN_NAME}_db_stats.txt}"
|
|
||||||
RRDP_STATE_TSV="${RRDP_STATE_TSV:-$OUT_DIR/${RUN_NAME}_rrdp_state.tsv}"
|
|
||||||
RUN_META_JSON="${RUN_META_JSON:-$OUT_DIR/${RUN_NAME}_meta.json}"
|
|
||||||
SUMMARY_MD="${SUMMARY_MD:-$OUT_DIR/${RUN_NAME}_summary.md}"
|
|
||||||
|
|
||||||
echo "== rpki manual full sync ==" >&2
|
|
||||||
echo "tal_url=$TAL_URL" >&2
|
|
||||||
echo "db=$DB_DIR" >&2
|
|
||||||
echo "report_json=$REPORT_JSON" >&2
|
|
||||||
echo "out_dir=$OUT_DIR" >&2
|
|
||||||
|
|
||||||
cmd=(cargo run --release --bin rpki -- \
|
|
||||||
--db "$DB_DIR" \
|
|
||||||
--tal-url "$TAL_URL" \
|
|
||||||
--http-timeout-secs "$HTTP_TIMEOUT_SECS" \
|
|
||||||
--rsync-timeout-secs "$RSYNC_TIMEOUT_SECS" \
|
|
||||||
--report-json "$REPORT_JSON")
|
|
||||||
|
|
||||||
if [[ -n "${RSYNC_MIRROR_ROOT}" ]]; then
|
|
||||||
cmd+=(--rsync-mirror-root "$RSYNC_MIRROR_ROOT")
|
|
||||||
fi
|
|
||||||
if [[ -n "${VALIDATION_TIME}" ]]; then
|
|
||||||
cmd+=(--validation-time "$VALIDATION_TIME")
|
|
||||||
fi
|
|
||||||
|
|
||||||
script_start_s="$(date +%s)"
|
|
||||||
run_start_s="$(date +%s)"
|
|
||||||
(
|
|
||||||
echo "# command:"
|
|
||||||
printf '%q ' "${cmd[@]}"
|
|
||||||
echo
|
|
||||||
echo
|
|
||||||
"${cmd[@]}"
|
|
||||||
) 2>&1 | tee "$RUN_LOG" >/dev/null
|
|
||||||
run_end_s="$(date +%s)"
|
|
||||||
run_duration_s="$((run_end_s - run_start_s))"
|
|
||||||
|
|
||||||
echo "== db_stats (exact) ==" >&2
|
|
||||||
db_stats_start_s="$(date +%s)"
|
|
||||||
cargo run --release --bin db_stats -- --db "$DB_DIR" --exact 2>&1 | tee "$DB_STATS_TXT" >/dev/null
|
|
||||||
db_stats_end_s="$(date +%s)"
|
|
||||||
db_stats_duration_s="$((db_stats_end_s - db_stats_start_s))"
|
|
||||||
|
|
||||||
echo "== rrdp_state_dump (legacy-state) ==" >&2
|
|
||||||
state_start_s="$(date +%s)"
|
|
||||||
cargo run --release --bin rrdp_state_dump -- --db "$DB_DIR" --view legacy-state >"$RRDP_STATE_TSV"
|
|
||||||
state_end_s="$(date +%s)"
|
|
||||||
state_duration_s="$((state_end_s - state_start_s))"
|
|
||||||
|
|
||||||
script_end_s="$(date +%s)"
|
|
||||||
total_duration_s="$((script_end_s - script_start_s))"
|
|
||||||
|
|
||||||
echo "== write run meta + summary ==" >&2
|
|
||||||
TAL_URL="$TAL_URL" \
|
|
||||||
DB_DIR="$DB_DIR" \
|
|
||||||
REPORT_JSON="$REPORT_JSON" \
|
|
||||||
RUN_LOG="$RUN_LOG" \
|
|
||||||
HTTP_TIMEOUT_SECS="$HTTP_TIMEOUT_SECS" \
|
|
||||||
RSYNC_TIMEOUT_SECS="$RSYNC_TIMEOUT_SECS" \
|
|
||||||
VALIDATION_TIME_ARG="$VALIDATION_TIME" \
|
|
||||||
RUN_DURATION_S="$run_duration_s" \
|
|
||||||
DB_STATS_DURATION_S="$db_stats_duration_s" \
|
|
||||||
STATE_DURATION_S="$state_duration_s" \
|
|
||||||
TOTAL_DURATION_S="$total_duration_s" \
|
|
||||||
python3 - "$REPORT_JSON" "$RUN_META_JSON" "$SUMMARY_MD" <<'PY'
|
|
||||||
import json
|
|
||||||
import os
|
|
||||||
import sys
|
|
||||||
from datetime import datetime, timezone
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
report_path = Path(sys.argv[1])
|
|
||||||
meta_path = Path(sys.argv[2])
|
|
||||||
summary_path = Path(sys.argv[3])
|
|
||||||
|
|
||||||
rep = json.loads(report_path.read_text(encoding="utf-8"))
|
|
||||||
|
|
||||||
now = datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ")
|
|
||||||
download_stats = rep.get("download_stats") or {}
|
|
||||||
meta = {
|
|
||||||
"recorded_at_utc": now,
|
|
||||||
"tal_url": os.environ["TAL_URL"],
|
|
||||||
"db_dir": os.environ["DB_DIR"],
|
|
||||||
"report_json": os.environ["REPORT_JSON"],
|
|
||||||
"run_log": os.environ["RUN_LOG"],
|
|
||||||
"validation_time_rfc3339_utc": rep["meta"]["validation_time_rfc3339_utc"],
|
|
||||||
"http_timeout_secs": int(os.environ["HTTP_TIMEOUT_SECS"]),
|
|
||||||
"rsync_timeout_secs": int(os.environ["RSYNC_TIMEOUT_SECS"]),
|
|
||||||
"validation_time_arg": os.environ.get("VALIDATION_TIME_ARG",""),
|
|
||||||
"durations_secs": {
|
|
||||||
"rpki_run": int(os.environ["RUN_DURATION_S"]),
|
|
||||||
"db_stats_exact": int(os.environ["DB_STATS_DURATION_S"]),
|
|
||||||
"rrdp_state_dump": int(os.environ["STATE_DURATION_S"]),
|
|
||||||
"total_script": int(os.environ["TOTAL_DURATION_S"]),
|
|
||||||
},
|
|
||||||
"counts": {
|
|
||||||
"publication_points_processed": rep["tree"]["instances_processed"],
|
|
||||||
"publication_points_failed": rep["tree"]["instances_failed"],
|
|
||||||
"vrps": len(rep["vrps"]),
|
|
||||||
"aspas": len(rep["aspas"]),
|
|
||||||
"audit_publication_points": len(rep["publication_points"]),
|
|
||||||
},
|
|
||||||
"download_stats": download_stats,
|
|
||||||
}
|
|
||||||
|
|
||||||
meta_path.write_text(json.dumps(meta, ensure_ascii=False, indent=2) + "\n", encoding="utf-8")
|
|
||||||
|
|
||||||
lines = []
|
|
||||||
lines.append("# Manual full sync summary\\n\\n")
|
|
||||||
lines.append(f"- recorded_at_utc: `{now}`\\n")
|
|
||||||
lines.append(f"- tal_url: `{meta['tal_url']}`\\n")
|
|
||||||
lines.append(f"- db: `{meta['db_dir']}`\\n")
|
|
||||||
lines.append(f"- report_json: `{meta['report_json']}`\\n")
|
|
||||||
lines.append(f"- validation_time: `{meta['validation_time_rfc3339_utc']}`\\n\\n")
|
|
||||||
lines.append("## Results\\n\\n")
|
|
||||||
lines.append("| metric | value |\\n")
|
|
||||||
lines.append("|---|---:|\\n")
|
|
||||||
for k in ["publication_points_processed","publication_points_failed","vrps","aspas","audit_publication_points"]:
|
|
||||||
lines.append(f"| {k} | {meta['counts'][k]} |\\n")
|
|
||||||
lines.append("\\n")
|
|
||||||
lines.append("## Durations (seconds)\\n\\n")
|
|
||||||
lines.append("| step | seconds |\\n")
|
|
||||||
lines.append("|---|---:|\\n")
|
|
||||||
for k,v in meta["durations_secs"].items():
|
|
||||||
lines.append(f"| {k} | {v} |\\n")
|
|
||||||
lines.append("\\n")
|
|
||||||
|
|
||||||
lines.append("## Download Stats\\n\\n")
|
|
||||||
lines.append("- raw events: `report_json.downloads`\\n")
|
|
||||||
lines.append("- aggregated: `report_json.download_stats` (copied into meta.json)\\n\\n")
|
|
||||||
|
|
||||||
def fmt_u(v):
|
|
||||||
if v is None:
|
|
||||||
return ""
|
|
||||||
return str(v)
|
|
||||||
|
|
||||||
by_kind = download_stats.get("by_kind") or {}
|
|
||||||
lines.append("| kind | ok | fail | duration_ms_total | bytes_total | objects_count_total | objects_bytes_total |\\n")
|
|
||||||
lines.append("|---|---:|---:|---:|---:|---:|---:|\\n")
|
|
||||||
for kind in sorted(by_kind.keys()):
|
|
||||||
st = by_kind[kind] or {}
|
|
||||||
lines.append(
|
|
||||||
f"| {kind} | {st.get('ok_total',0)} | {st.get('fail_total',0)} | {st.get('duration_ms_total',0)} | {fmt_u(st.get('bytes_total'))} | {fmt_u(st.get('objects_count_total'))} | {fmt_u(st.get('objects_bytes_total'))} |\\n"
|
|
||||||
)
|
|
||||||
lines.append("\\n")
|
|
||||||
|
|
||||||
summary_path.write_text("".join(lines), encoding="utf-8")
|
|
||||||
print(summary_path)
|
|
||||||
PY
|
|
||||||
|
|
||||||
echo "== done ==" >&2
|
|
||||||
echo "artifacts:" >&2
|
|
||||||
echo "- db: $DB_DIR" >&2
|
|
||||||
echo "- report: $REPORT_JSON" >&2
|
|
||||||
echo "- run log: $RUN_LOG" >&2
|
|
||||||
echo "- db stats: $DB_STATS_TXT" >&2
|
|
||||||
echo "- rrdp state: $RRDP_STATE_TSV" >&2
|
|
||||||
echo "- meta json: $RUN_META_JSON" >&2
|
|
||||||
echo "- summary md: $SUMMARY_MD" >&2
|
|
||||||
@@ -1,268 +0,0 @@
|
|||||||
# Payload Replay Scripts
|
|
||||||
|
|
||||||
本目录提供基于本地 payload archive 的手工 replay 入口。
|
|
||||||
|
|
||||||
## `multi_rir_case_info.py`
|
|
||||||
|
|
||||||
用于从 multi-RIR bundle 中解析指定 `rir` 的输入路径、对照 CSV、fixture、以及 Routinator replay timing 基线。
|
|
||||||
|
|
||||||
示例：
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python3 scripts/payload_replay/multi_rir_case_info.py \
|
|
||||||
--bundle-root ../../rpki/target/live/20260316-112341-multi-final3 \
|
|
||||||
--rir afrinic
|
|
||||||
```
|
|
||||||
|
|
||||||
也支持输出 shell 环境变量：
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python3 scripts/payload_replay/multi_rir_case_info.py \
|
|
||||||
--bundle-root ../../rpki/target/live/20260316-112341-multi-final3 \
|
|
||||||
--rir afrinic \
|
|
||||||
--format env
|
|
||||||
```
|
|
||||||
|
|
||||||
## `run_multi_rir_replay_case.sh`
|
|
||||||
|
|
||||||
统一的 multi-RIR 入口。给定 `rir` 和模式后，它会自动选择该 RIR 的：
|
|
||||||
|
|
||||||
- snapshot/base replay 输入
|
|
||||||
- delta replay 输入
|
|
||||||
- 对照 CSV
|
|
||||||
- TAL / TA fixture
|
|
||||||
- trust anchor 名称
|
|
||||||
|
|
||||||
用法：
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./scripts/payload_replay/run_multi_rir_replay_case.sh <rir> [describe|snapshot|delta|both]
|
|
||||||
```
|
|
||||||
|
|
||||||
示例：
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./scripts/payload_replay/run_multi_rir_replay_case.sh afrinic describe
|
|
||||||
./scripts/payload_replay/run_multi_rir_replay_case.sh lacnic snapshot
|
|
||||||
./scripts/payload_replay/run_multi_rir_replay_case.sh arin delta
|
|
||||||
./scripts/payload_replay/run_multi_rir_replay_case.sh ripe both
|
|
||||||
```
|
|
||||||
|
|
||||||
|
|
||||||
脚本会自动：
|
|
||||||
|
|
||||||
- 从 multi-RIR bundle 中选择指定 RIR 的 snapshot/base 与 delta 输入
|
|
||||||
- 读取该 RIR 的 Routinator `base-replay` / `delta-replay` timing 基线
|
|
||||||
- 优先使用 `base-locks.json.validationTime` 与 `locks-delta.json.validationTime` 作为 replay `--validation-time`；若缺失才回退到 `timings/base-replay.json` 与 `timings/delta-replay.json` 的 `startedAt`
|
|
||||||
- 在 `target/live/multi_rir_replay_runs/<rir>/` 下生成：
|
|
||||||
- snapshot replay 产物
|
|
||||||
- delta replay 产物
|
|
||||||
- per-RIR 合并 case report（含 correctness + timing compare）
|
|
||||||
|
|
||||||
默认 bundle 根目录为：
|
|
||||||
|
|
||||||
- `../../rpki/target/live/20260316-112341-multi-final3`
|
|
||||||
|
|
||||||
也可以通过 `BUNDLE_ROOT` 覆盖。
|
|
||||||
|
|
||||||
## `run_apnic_snapshot_replay_profile.sh`
|
|
||||||
|
|
||||||
基于 multi-RIR bundle 中的 APNIC snapshot 输入，使用当前 replay 主流程执行一次带 `--analyze` 和 `--profile-cpu` 的离线 profile。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./scripts/payload_replay/run_apnic_snapshot_replay_profile.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
作用：
|
|
||||||
|
|
||||||
- 使用 `APNIC` 的 snapshot/base replay 输入
|
|
||||||
- 自动开启：
|
|
||||||
- `--analyze`
|
|
||||||
- `--profile-cpu`
|
|
||||||
- 自动记录：
|
|
||||||
- replay wall-clock 时长
|
|
||||||
- Routinator baseline (`base-replay`)
|
|
||||||
- analyze 目录路径
|
|
||||||
- 生成：
|
|
||||||
- `report.json`
|
|
||||||
- `run.log`
|
|
||||||
- `meta.json`
|
|
||||||
- `summary.md`
|
|
||||||
- 以及 `target/live/analyze/<timestamp>/` 下的：
|
|
||||||
- `timing.json`
|
|
||||||
- `flamegraph.svg`
|
|
||||||
- `pprof.pb.gz`
|
|
||||||
|
|
||||||
支持：
|
|
||||||
|
|
||||||
- `DRY_RUN=1`：只打印命令，不真正执行
|
|
||||||
- `MAX_DEPTH` / `MAX_INSTANCES`：用于限定 replay 范围
|
|
||||||
- `PROFILE_RUN_ROOT`：覆盖 wrapper 产物输出目录
|
|
||||||
|
|
||||||
## `run_apnic_replay.sh`
|
|
||||||
|
|
||||||
默认使用：
|
|
||||||
|
|
||||||
- `tests/fixtures/tal/apnic-rfc7730-https.tal`
|
|
||||||
- `tests/fixtures/ta/apnic-ta.cer`
|
|
||||||
- `target/live/payload_replay/payload-archive`
|
|
||||||
- `target/live/payload_replay/locks.json`
|
|
||||||
|
|
||||||
运行：
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./scripts/payload_replay/run_apnic_replay.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
产物默认输出到：
|
|
||||||
|
|
||||||
- `target/live/payload_replay_runs/`
|
|
||||||
|
|
||||||
包含：
|
|
||||||
|
|
||||||
- replay DB 目录
|
|
||||||
- `report.json`
|
|
||||||
- `run.log`
|
|
||||||
- `meta.json`
|
|
||||||
- `summary.md`
|
|
||||||
|
|
||||||
## 环境变量
|
|
||||||
|
|
||||||
可覆盖：
|
|
||||||
|
|
||||||
- `TAL_PATH`
|
|
||||||
- `TA_PATH`
|
|
||||||
- `PAYLOAD_REPLAY_ARCHIVE`
|
|
||||||
- `PAYLOAD_REPLAY_LOCKS`
|
|
||||||
- `VALIDATION_TIME`
|
|
||||||
- `MAX_DEPTH`
|
|
||||||
- `MAX_INSTANCES`
|
|
||||||
- `OUT_DIR`
|
|
||||||
- `RUN_NAME`
|
|
||||||
- `DB_DIR`
|
|
||||||
- `REPORT_JSON`
|
|
||||||
- `RUN_LOG`
|
|
||||||
- `META_JSON`
|
|
||||||
- `SUMMARY_MD`
|
|
||||||
|
|
||||||
## 说明
|
|
||||||
|
|
||||||
- 该脚本依赖 `rpki` CLI 已支持：
|
|
||||||
- `--payload-replay-archive`
|
|
||||||
- `--payload-replay-locks`
|
|
||||||
- replay 模式必须搭配离线 TAL/TA 输入，不会去访问真实 RRDP / rsync 网络源。
|
|
||||||
|
|
||||||
## `report_to_routinator_csv.py`
|
|
||||||
|
|
||||||
把 `rpki` 生成的 `report.json` 转成 Routinator 风格的 VRP CSV：
|
|
||||||
|
|
||||||
```bash
|
|
||||||
python3 scripts/payload_replay/report_to_routinator_csv.py \
|
|
||||||
--report target/live/payload_replay_runs/<run>_report.json \
|
|
||||||
--out target/live/payload_replay_runs/<run>_vrps.csv \
|
|
||||||
--trust-anchor apnic
|
|
||||||
```
|
|
||||||
|
|
||||||
输出列为：
|
|
||||||
|
|
||||||
- `ASN`
|
|
||||||
- `IP Prefix`
|
|
||||||
- `Max Length`
|
|
||||||
- `Trust Anchor`
|
|
||||||
|
|
||||||
## `compare_with_routinator_record.sh`
|
|
||||||
|
|
||||||
把 ours 生成的 VRP CSV 与 Routinator 的 `record.csv` 做对比：
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./scripts/payload_replay/compare_with_routinator_record.sh \
|
|
||||||
target/live/payload_replay_runs/<run>_vrps.csv \
|
|
||||||
target/live/payload_replay/record.csv
|
|
||||||
```
|
|
||||||
|
|
||||||
会产出：
|
|
||||||
|
|
||||||
- compare summary Markdown
|
|
||||||
- `only_in_ours.csv`
|
|
||||||
- `only_in_record.csv`
|
|
||||||
|
|
||||||
## `run_apnic_replay.sh` 现有额外产物
|
|
||||||
|
|
||||||
脚本现在除了 `report/meta/summary`，还会额外生成：
|
|
||||||
|
|
||||||
- `vrps.csv`
|
|
||||||
- 若 `ROUTINATOR_RECORD_CSV` 存在，则生成：
|
|
||||||
- compare summary
|
|
||||||
- `only_in_ours.csv`
|
|
||||||
- `only_in_record.csv`
|
|
||||||
|
|
||||||
## `run_apnic_delta_replay.sh`
|
|
||||||
|
|
||||||
使用 APNIC delta demo 数据集运行 base + delta replay：
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./scripts/payload_replay/run_apnic_delta_replay.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
默认输入：
|
|
||||||
|
|
||||||
- `target/live/apnic_delta_demo/20260315-170223-autoplay/base-payload-archive`
|
|
||||||
- `target/live/apnic_delta_demo/20260315-170223-autoplay/base-locks.json`
|
|
||||||
- `target/live/apnic_delta_demo/20260315-170223-autoplay/payload-delta-archive`
|
|
||||||
- `target/live/apnic_delta_demo/20260315-170223-autoplay/locks-delta.json`
|
|
||||||
- `tests/fixtures/tal/apnic-rfc7730-https.tal`
|
|
||||||
- `tests/fixtures/ta/apnic-ta.cer`
|
|
||||||
|
|
||||||
输出目录默认：`target/live/payload_delta_replay_runs/`
|
|
||||||
|
|
||||||
## `run_apnic_delta_replay.sh` compare outputs
|
|
||||||
|
|
||||||
脚本现在在 delta replay 结束后还会额外生成：
|
|
||||||
|
|
||||||
- `vrps.csv`
|
|
||||||
- compare summary Markdown
|
|
||||||
- `only_in_ours.csv`
|
|
||||||
- `only_in_record.csv`
|
|
||||||
|
|
||||||
默认 compare 输入是：
|
|
||||||
|
|
||||||
- `target/live/apnic_delta_demo/20260315-170223-autoplay/record-delta.csv`
|
|
||||||
|
|
||||||
也可以通过环境变量覆盖：
|
|
||||||
|
|
||||||
- `TRUST_ANCHOR`
|
|
||||||
- `ROUTINATOR_RECORD_CSV`
|
|
||||||
- `VRPS_CSV`
|
|
||||||
- `COMPARE_SUMMARY_MD`
|
|
||||||
- `ONLY_IN_OURS_CSV`
|
|
||||||
- `ONLY_IN_RECORD_CSV`
|
|
||||||
|
|
||||||
## `write_multi_rir_case_report.py`
|
|
||||||
|
|
||||||
把某个 RIR 的 snapshot replay 与 delta replay 的 `meta.json`、compare summary 以及 Routinator timing 基线合并成一个 per-RIR Markdown/JSON 报告。
|
|
||||||
|
|
||||||
该脚本通常由 `run_multi_rir_replay_case.sh <rir> both` 自动调用。
|
|
||||||
|
|
||||||
## `run_multi_rir_replay_suite.sh`
|
|
||||||
|
|
||||||
顺序执行 5 个 RIR（或环境变量 `RIRS` 指定的子集）的 `both` 模式，并最终生成 multi-RIR 汇总报告。
|
|
||||||
|
|
||||||
```bash
|
|
||||||
./scripts/payload_replay/run_multi_rir_replay_suite.sh
|
|
||||||
```
|
|
||||||
|
|
||||||
可覆盖环境变量：
|
|
||||||
|
|
||||||
- `BUNDLE_ROOT`
|
|
||||||
- `SUITE_OUT_DIR`
|
|
||||||
- `RIRS`
|
|
||||||
|
|
||||||
最终输出：
|
|
||||||
|
|
||||||
- `<suite_out_dir>/<rir>/<rir>_case_report.md`
|
|
||||||
- `<suite_out_dir>/multi_rir_summary.md`
|
|
||||||
- `<suite_out_dir>/multi_rir_summary.json`
|
|
||||||
|
|
||||||
## `write_multi_rir_summary.py`
|
|
||||||
|
|
||||||
汇总 5 个 RIR 的 per-RIR case report，生成 correctness + timing 总表与几何平均比值。
|
|
||||||
@@ -1,110 +0,0 @@
|
|||||||
#!/usr/bin/env bash
|
|
||||||
set -euo pipefail
|
|
||||||
|
|
||||||
if [[ $# -lt 2 || $# -gt 5 ]]; then
|
|
||||||
echo "Usage: $0 <ours.csv> <record.csv> [summary.md] [only_in_ours.csv] [only_in_record.csv]" >&2
|
|
||||||
exit 2
|
|
||||||
fi
|
|
||||||
|
|
||||||
OURS_CSV="$1"
|
|
||||||
RECORD_CSV="$2"
|
|
||||||
SUMMARY_MD="${3:-}"
|
|
||||||
ONLY_IN_OURS_CSV="${4:-}"
|
|
||||||
ONLY_IN_RECORD_CSV="${5:-}"
|
|
||||||
|
|
||||||
if [[ -z "$SUMMARY_MD" ]]; then
|
|
||||||
SUMMARY_MD="$(dirname "$OURS_CSV")/$(basename "$OURS_CSV" .csv)_vs_routinator_summary.md"
|
|
||||||
fi
|
|
||||||
if [[ -z "$ONLY_IN_OURS_CSV" ]]; then
|
|
||||||
ONLY_IN_OURS_CSV="$(dirname "$OURS_CSV")/$(basename "$OURS_CSV" .csv)_only_in_ours.csv"
|
|
||||||
fi
|
|
||||||
if [[ -z "$ONLY_IN_RECORD_CSV" ]]; then
|
|
||||||
ONLY_IN_RECORD_CSV="$(dirname "$OURS_CSV")/$(basename "$OURS_CSV" .csv)_only_in_record.csv"
|
|
||||||
fi
|
|
||||||
|
|
||||||
python3 - "$OURS_CSV" "$RECORD_CSV" "$SUMMARY_MD" "$ONLY_IN_OURS_CSV" "$ONLY_IN_RECORD_CSV" <<'PY'
|
|
||||||
import csv
|
|
||||||
import ipaddress
|
|
||||||
import sys
|
|
||||||
from pathlib import Path
|
|
||||||
|
|
||||||
ours_csv = Path(sys.argv[1])
|
|
||||||
record_csv = Path(sys.argv[2])
|
|
||||||
summary_md = Path(sys.argv[3])
|
|
||||||
only_in_ours_csv = Path(sys.argv[4])
|
|
||||||
only_in_record_csv = Path(sys.argv[5])
|
|
||||||
|
|
||||||
|
|
||||||
def normalize_row(row: dict):
|
|
||||||
asn = row["ASN"].strip().upper()
|
|
||||||
prefix = row["IP Prefix"].strip()
|
|
||||||
max_len = str(int(row["Max Length"]))
|
|
||||||
ta = row["Trust Anchor"].strip()
|
|
||||||
network = ipaddress.ip_network(prefix, strict=False)
|
|
||||||
return {
|
|
||||||
"ASN": asn,
|
|
||||||
"IP Prefix": str(network),
|
|
||||||
"Max Length": max_len,
|
|
||||||
"Trust Anchor": ta,
|
|
||||||
}
|
|
||||||
|
|
||||||
|
|
||||||
def read_rows(path: Path):
|
|
||||||
with path.open(encoding="utf-8", newline="") as f:
|
|
||||||
rows = [normalize_row(r) for r in csv.DictReader(f)]
|
|
||||||
return rows
|
|
||||||
|
|
||||||
|
|
||||||
def row_key(row: dict):
|
|
||||||
network = ipaddress.ip_network(row["IP Prefix"], strict=False)
|
|
||||||
return (
|
|
||||||
row["ASN"],
|
|
||||||
network.version,
|
|
||||||
int(network.network_address),
|
|
||||||
network.prefixlen,
|
|
||||||
int(row["Max Length"]),
|
|
||||||
row["Trust Anchor"],
|
|
||||||
)
|
|
||||||
|
|
||||||
|
|
||||||
def write_rows(path: Path, rows):
|
|
||||||
path.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
with path.open("w", encoding="utf-8", newline="") as f:
|
|
||||||
writer = csv.DictWriter(f, fieldnames=["ASN", "IP Prefix", "Max Length", "Trust Anchor"])
|
|
||||||
writer.writeheader()
|
|
||||||
for row in rows:
|
|
||||||
writer.writerow(row)
|
|
||||||
|
|
||||||
ours = read_rows(ours_csv)
|
|
||||||
record = read_rows(record_csv)
|
|
||||||
ours_map = {row_key(r): r for r in ours}
|
|
||||||
record_map = {row_key(r): r for r in record}
|
|
||||||
only_in_ours = [ours_map[k] for k in sorted(set(ours_map) - set(record_map))]
|
|
||||||
only_in_record = [record_map[k] for k in sorted(set(record_map) - set(ours_map))]
|
|
||||||
intersection = len(set(ours_map) & set(record_map))
|
|
||||||
|
|
||||||
write_rows(only_in_ours_csv, only_in_ours)
|
|
||||||
write_rows(only_in_record_csv, only_in_record)
|
|
||||||
|
|
||||||
summary_md.parent.mkdir(parents=True, exist_ok=True)
|
|
||||||
summary = []
|
|
||||||
summary.append("# Replay vs Routinator VRP Compare\n\n")
|
|
||||||
summary.append(f"- ours_csv: `{ours_csv}`\n")
|
|
||||||
summary.append(f"- record_csv: `{record_csv}`\n")
|
|
||||||
summary.append(f"- only_in_ours_csv: `{only_in_ours_csv}`\n")
|
|
||||||
summary.append(f"- only_in_record_csv: `{only_in_record_csv}`\n\n")
|
|
||||||
summary.append("| metric | value |\n")
|
|
||||||
summary.append("|---|---:|\n")
|
|
||||||
summary.append(f"| ours_total | {len(ours_map)} |\n")
|
|
||||||
summary.append(f"| record_total | {len(record_map)} |\n")
|
|
||||||
summary.append(f"| intersection | {intersection} |\n")
|
|
||||||
summary.append(f"| only_in_ours | {len(only_in_ours)} |\n")
|
|
||||||
summary.append(f"| only_in_record | {len(only_in_record)} |\n")
|
|
||||||
summary_md.write_text("".join(summary), encoding="utf-8")
|
|
||||||
print(summary_md)
|
|
||||||
PY
|
|
||||||
|
|
||||||
echo "== compare complete ==" >&2
|
|
||||||
echo "- summary: $SUMMARY_MD" >&2
|
|
||||||
echo "- only_in_ours: $ONLY_IN_OURS_CSV" >&2
|
|
||||||
echo "- only_in_record: $ONLY_IN_RECORD_CSV" >&2
|
|
||||||
Some files were not shown because too many files have changed in this diff Show More
Reference in New Issue
Block a user