NASP/rpki
75
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

20260713_2 落地升级环境变量优先级契约（#114 主工作树收口）

Browse Source
This commit is contained in:
yuyr 2026-07-21 15:55:15 +08:00
parent daf0764cd0
commit 654b6d1c56
8 changed files with 554 additions and 69 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
deploy/docker-installer/.env.example
View File

@ -2,6 +2,8 @@
# `build_docker_installer_package.sh` rewrites package-specific placeholders # `build_docker_installer_package.sh` rewrites package-specific placeholders
# when producing amd64/arm64 release packages. # when producing amd64/arm64 release packages.
# 中文说明见 docs/README.zh-CN.md。English guide: docs/README.en.md # 中文说明见 docs/README.zh-CN.md。English guide: docs/README.en.md
# Every assignment is a required non-empty configuration key with a package
# default. During upgrade: .env.example < reused old .env < command environment.
# Package metadata and architecture guardrails. # Package metadata and architecture guardrails.
PACKAGE_ARCH=__PACKAGE_ARCH__ PACKAGE_ARCH=__PACKAGE_ARCH__

2
deploy/docker-installer/docs/README.en.md
View File

@ -80,7 +80,7 @@ Supported values:
- `rfc6487`: switch back to the original RFC 6487 resource containment behavior. - `rfc6487`: switch back to the original RFC 6487 resource containment behavior.
The installer / soak runner passes `--resource-validation-mode` to every `rpki` child start. The installer / soak runner passes `--resource-validation-mode` to every `rpki` child start.
When an upgraded deployment reuses an older `.env` that does not contain this key, the installer appends the new default `validation-update-03` automatically. When an upgraded deployment reuses an older `.env` that does not contain this key, the new package `.env.example` default `validation-update-03` remains in the final `.env`; an empty final value fails before Docker operations.
## Architecture Guardrails ## Architecture Guardrails

2
deploy/docker-installer/docs/README.zh-CN.md
View File

@ -80,7 +80,7 @@ RESOURCE_VALIDATION_MODE=validation-update-03
- `rfc6487`：切换为 RFC 6487 原始资源包含判定语义。 - `rfc6487`：切换为 RFC 6487 原始资源包含判定语义。
installer / soak runner 每次启动 `rpki` 子进程时都会显式传入 `--resource-validation-mode`。 installer / soak runner 每次启动 `rpki` 子进程时都会显式传入 `--resource-validation-mode`。
如果升级时复用旧 `.env` 且缺少该变量，installer 会自动补入默认值 `validation-update-03`，不会破坏旧 `.env` 复用流程。 如果升级时复用旧 `.env` 且缺少该变量，新包 `.env.example` 中的默认值 `validation-update-03` 会保留在最终 `.env` 中；最终值为空时升级会在 Docker 操作前失败。
## 架构检查 ## 架构检查

21
deploy/docker-installer/docs/operations.en.md
View File

@ -89,9 +89,26 @@ Extract the new package into a new directory and explicitly reuse the existing `
./scripts/upgrade.sh --reuse-env-from /path/to/old-installer/.env ./scripts/upgrade.sh --reuse-env-from /path/to/old-installer/.env
``` ```
If the new package directory already has a `.env`, the upgrade script keeps it. Configuration is merged in this order, with a key on the right overriding the same key on the left:
If an older `.env` does not contain `METRICS_IMAGE` or `METRICS_PLATFORM`, the install/upgrade flow backfills package-matched defaults automatically. ```text
new .env.example < old .env < environment variables on this upgrade command
```
For example, override the next-run interval and Grafana password for this upgrade:
```bash
INTERVAL_SECS=300 GRAFANA_ADMIN_PASSWORD='new-password' \
./scripts/upgrade.sh --reuse-env-from /path/to/old-installer/.env
```
Before any image is loaded or container is started, upgrade verifies that every key declared by the new template is present and non-empty in the final `.env`. An explicit empty assignment fails, for example `INTERVAL_SECS= ./scripts/upgrade.sh ...`. Removed keys from an old `.env` are not copied, and the script reports key names only, never password-like values.
The installer reads `.env` as Compose-compatible dotenv assignments. Do not put shell command substitutions or environment references such as `$HOME` in the file; use concrete values or command-level overrides instead.
If the new package directory already has a `.env`, the upgrade script keeps it. In that case `--reuse-env-from` is not imported again, but environment variables from the current command still override the existing `.env`.
When an older `.env` lacks a key introduced by the new package, the new `.env.example` default remains in the final `.env`. Upgrade fails before Docker operations if any final declared value is empty.
Upgrade does not delete: Upgrade does not delete:

21
deploy/docker-installer/docs/operations.zh-CN.md
View File

@ -89,9 +89,26 @@ curl -s "http://127.0.0.1:${METRICS_PORT:-9556}/metrics" | grep '^ours_rp_rtr_'
./scripts/upgrade.sh --reuse-env-from /path/to/old-installer/.env ./scripts/upgrade.sh --reuse-env-from /path/to/old-installer/.env
``` ```
如果新目录已经存在 `.env`，升级脚本会保留它，不覆盖。 配置按以下优先级合并，右侧同名键覆盖左侧：
如果旧 `.env` 缺失 `METRICS_IMAGE` 或 `METRICS_PLATFORM`，升级/安装链路会按当前包架构自动补齐默认值。 ```text
新包 .env.example < 旧 .env < 本次 upgrade 命令的环境变量
```
例如，临时调整下轮触发间隔和 Grafana 密码：
```bash
INTERVAL_SECS=300 GRAFANA_ADMIN_PASSWORD='new-password' \
./scripts/upgrade.sh --reuse-env-from /path/to/old-installer/.env
```
升级会在加载镜像或启动容器前检查：新模板声明的所有变量在最终 `.env` 中都必须存在且非空。显式传入空值会失败，例如 `INTERVAL_SECS= ./scripts/upgrade.sh ...`。旧 `.env` 中已不在新模板内的键不会复制，脚本只输出相关键名而不会输出密码等值。
installer 按与 Compose 一致的 dotenv 赋值格式读取 `.env`。不要在文件中使用 Shell 命令替换或 `$HOME` 一类环境变量引用；请使用确定的值，或在执行 upgrade 时通过命令环境变量覆盖。
如果新目录已经存在 `.env`，升级脚本会保留它，不覆盖；此时 `--reuse-env-from` 不会再次导入旧配置，但本次命令的环境变量仍会覆盖已存在的 `.env`。
旧 `.env` 缺少新版本新增的键时，新包 `.env.example` 的默认值会保留在最终 `.env` 中；任一最终值为空则升级会在 Docker 操作前失败。
升级不会删除以下数据： 升级不会删除以下数据：

270
deploy/docker-installer/scripts/common.sh
View File

@ -6,6 +6,11 @@ ENV_FILE="${ENV_FILE:-$INSTALLER_ROOT/.env}"
ENV_EXAMPLE="$INSTALLER_ROOT/.env.example" ENV_EXAMPLE="$INSTALLER_ROOT/.env.example"
COMPOSE_FILE="$INSTALLER_ROOT/compose/docker-compose.yml" COMPOSE_FILE="$INSTALLER_ROOT/compose/docker-compose.yml"
MANIFEST_FILE="${MANIFEST_FILE:-$INSTALLER_ROOT/PACKAGE-MANIFEST.env}" MANIFEST_FILE="${MANIFEST_FILE:-$INSTALLER_ROOT/PACKAGE-MANIFEST.env}"
declare -a ENV_TEMPLATE_KEYS=()
declare -A ENV_TEMPLATE_KEY_SET=()
declare -A INVOCATION_ENV_OVERRIDES=()
declare -A REUSED_ENV_KEYS=()
declare -A ENV_PARSED_VALUES=()
log() { log() {
printf '[ours-rp-installer] %s\n' "$*" printf '[ours-rp-installer] %s\n' "$*"
@ -48,20 +53,246 @@ env_file_has_key() {
[[ -f "$env_path" ]] && grep -Eq "^${key}=" "$env_path" [[ -f "$env_path" ]] && grep -Eq "^${key}=" "$env_path"
} }
persist_env_default() { load_env_template_keys() {
local env_path="$1"
local line
local key
[[ -f "$env_path" ]] || die "missing environment template: $env_path"
ENV_TEMPLATE_KEYS=()
ENV_TEMPLATE_KEY_SET=()
while IFS= read -r line || [[ -n "$line" ]]; do
line="${line%$'\r'}"
[[ "$line" =~ ^[[:space:]]*$ || "$line" =~ ^[[:space:]]*# ]] && continue
[[ "$line" =~ ^([A-Za-z_][A-Za-z0-9_]*)= ]] || die "invalid environment assignment in $env_path: $line"
key="${BASH_REMATCH[1]}"
[[ -z "${ENV_TEMPLATE_KEY_SET[$key]+x}" ]] || die "duplicate environment key in $env_path: $key"
ENV_TEMPLATE_KEYS+=("$key")
ENV_TEMPLATE_KEY_SET["$key"]=1
done < "$env_path"
(( ${#ENV_TEMPLATE_KEYS[@]} > 0 )) || die "no environment assignments found in $env_path"
}
template_has_env_key() {
local key="$1"
[[ -n "${ENV_TEMPLATE_KEY_SET[$key]+x}" ]]
}
env_file_assignment_keys() {
local env_path="$1"
local line
[[ -f "$env_path" ]] || die "missing environment file: $env_path"
while IFS= read -r line || [[ -n "$line" ]]; do
line="${line%$'\r'}"
[[ "$line" =~ ^([A-Za-z_][A-Za-z0-9_]*)= ]] || continue
printf '%s\n' "${BASH_REMATCH[1]}"
done < "$env_path"
}
dotenv_decode_value() {
local raw_value="$1"
local quote
local body
local decoded=""
local character
local next_character
local index=0
[[ -n "$raw_value" ]] || {
printf '\n'
return 0
}
quote="${raw_value:0:1}"
case "$quote" in
"'")
[[ "${raw_value: -1}" == "'" && ${#raw_value} -ge 2 ]] || return 1
body="${raw_value:1:${#raw_value}-2}"
while (( index < ${#body} )); do
character="${body:index:1}"
if [[ "$character" == "\\" && $((index + 1)) -lt ${#body} ]]; then
next_character="${body:index + 1:1}"
case "$next_character" in
"'")
decoded+="$next_character"
((index += 2))
continue
;;
esac
fi
decoded+="$character"
((index += 1))
done
;;
'"')
[[ "${raw_value: -1}" == '"' && ${#raw_value} -ge 2 ]] || return 1
body="${raw_value:1:${#raw_value}-2}"
while (( index < ${#body} )); do
character="${body:index:1}"
if [[ "$character" == '$' && $((index + 1)) -lt ${#body} && "${body:index + 1:1}" == '$' ]]; then
decoded+='$'
((index += 2))
continue
fi
if [[ "$character" == "\\" && $((index + 1)) -lt ${#body} ]]; then
next_character="${body:index + 1:1}"
case "$next_character" in
'"'|\\|'$'|'`')
decoded+="$next_character"
((index += 2))
continue
;;
n)
decoded+=$'\n'
((index += 2))
continue
;;
r)
decoded+=$'\r'
((index += 2))
continue
;;
t)
decoded+=$'\t'
((index += 2))
continue
;;
esac
fi
decoded+="$character"
((index += 1))
done
;;
*)
decoded="$raw_value"
;;
esac
printf '%s' "$decoded"
}
parse_env_file() {
local env_path="$1"
local line
local parsed_key
local raw_value
local parsed_value
[[ -f "$env_path" ]] || die "missing environment file: $env_path"
ENV_PARSED_VALUES=()
while IFS= read -r line || [[ -n "$line" ]]; do
line="${line%$'\r'}"
[[ "$line" =~ ^[[:space:]]*$ || "$line" =~ ^[[:space:]]*# ]] && continue
[[ "$line" =~ ^([A-Za-z_][A-Za-z0-9_]*)=(.*)$ ]] || return 2
parsed_key="${BASH_REMATCH[1]}"
raw_value="${BASH_REMATCH[2]}"
parsed_value="$(dotenv_decode_value "$raw_value")" || return 2
ENV_PARSED_VALUES["$parsed_key"]="$parsed_value"
done < "$env_path"
}
env_file_value() {
local env_path="$1"
local key="$2"
parse_env_file "$env_path"
[[ -n "${ENV_PARSED_VALUES[$key]+x}" ]] || return 3
printf '%s' "${ENV_PARSED_VALUES[$key]}"
}
env_file_set_value() {
local env_path="$1" local env_path="$1"
local key="$2" local key="$2"
local value="$3" local value="$3"
local tmp_env local tmp_env
if env_file_has_key "$env_path" "$key"; then local encoded_value
[[ "$value" != *$'\n'* && "$value" != *$'\r'* ]] || die "environment value for $key contains an unsupported newline"
encoded_value="$(env_file_encode_value "$value")"
if ! env_file_has_key "$env_path" "$key"; then
printf '%s=%s\n' "$key" "$encoded_value" >> "$env_path"
return 0 return 0
fi fi
tmp_env="$(mktemp)" tmp_env="$(mktemp)"
cat "$env_path" > "$tmp_env" ENV_FILE_REPLACEMENT="$key=$encoded_value" awk -v key="$key" '
printf '%s=%s\n' "$key" "$value" >> "$tmp_env" BEGIN { done=0 }
$0 ~ "^" key "=" { print ENVIRON["ENV_FILE_REPLACEMENT"]; done=1; next }
{ print }
END { if (!done) print ENVIRON["ENV_FILE_REPLACEMENT"] }
' "$env_path" > "$tmp_env"
mv "$tmp_env" "$env_path" mv "$tmp_env" "$env_path"
} }
env_file_encode_value() {
local value="$1"
local escaped="$value"
[[ "$value" != *$'\n'* && "$value" != *$'\r'* ]] || die "environment value contains an unsupported newline"
escaped="${escaped//\\/\\\\}"
escaped="${escaped//\"/\\\"}"
escaped="${escaped//\$/\$\$}"
printf '"%s"' "$escaped"
}
load_env_file() {
local env_path="$1"
local key
parse_env_file "$env_path" || die "unable to parse environment file: $env_path"
for key in "${!ENV_PARSED_VALUES[@]}"; do
printf -v "$key" '%s' "${ENV_PARSED_VALUES[$key]}"
export "$key"
done
}
validate_env_file_contract() {
local env_path="$1"
local invalid_keys=""
local key
[[ -f "$env_path" ]] || die "missing environment file: $env_path"
(( ${#ENV_TEMPLATE_KEYS[@]} > 0 )) || die "environment template keys were not loaded"
if ! parse_env_file "$env_path"; then
die "unable to parse environment file: $env_path"
fi
for key in "${ENV_TEMPLATE_KEYS[@]}"; do
if [[ -z "${ENV_PARSED_VALUES[$key]+x}" || -z "${ENV_PARSED_VALUES[$key]}" ]]; then
invalid_keys+="$key"$'\n'
fi
done
[[ -z "$invalid_keys" ]] || die "environment contract requires non-empty values in $env_path: ${invalid_keys//$'\n'/,}"
}
capture_invocation_env_overrides() {
local key
(( ${#ENV_TEMPLATE_KEYS[@]} > 0 )) || die "environment template keys were not loaded"
INVOCATION_ENV_OVERRIDES=()
for key in "${ENV_TEMPLATE_KEYS[@]}"; do
if [[ -v "$key" ]]; then
INVOCATION_ENV_OVERRIDES["$key"]="${!key}"
fi
done
}
log_env_key_group() {
local group="$1"
shift
local rendered="none"
if (( $# > 0 )); then
local IFS=,
rendered="$*"
fi
log "$group=$rendered"
}
env_flag_enabled() { env_flag_enabled() {
case "${1:-0}" in case "${1:-0}" in
1|true|TRUE|yes|YES|on|ON) 1|true|TRUE|yes|YES|on|ON)
@ -156,15 +387,15 @@ assert_arch_compatibility() {
load_env() { load_env() {
load_manifest load_manifest
[[ -f "$ENV_EXAMPLE" ]] || die "missing $ENV_EXAMPLE"
load_env_template_keys "$ENV_EXAMPLE"
validate_env_file_contract "$ENV_EXAMPLE"
if [[ ! -f "$ENV_FILE" ]]; then if [[ ! -f "$ENV_FILE" ]]; then
[[ -f "$ENV_EXAMPLE" ]] || die "missing $ENV_EXAMPLE"
cp "$ENV_EXAMPLE" "$ENV_FILE" cp "$ENV_EXAMPLE" "$ENV_FILE"
log "created .env from .env.example" log "created .env from .env.example"
fi fi
set -a validate_env_file_contract "$ENV_FILE"
# shellcheck disable=SC1090 load_env_file "$ENV_FILE"
source "$ENV_FILE"
set +a
if [[ -n "${MANIFEST_PACKAGE_ARCH_RAW:-}" ]]; then if [[ -n "${MANIFEST_PACKAGE_ARCH_RAW:-}" ]]; then
manifest_package_arch="$(normalize_arch "$MANIFEST_PACKAGE_ARCH_RAW")" || die "unsupported manifest package arch: $MANIFEST_PACKAGE_ARCH_RAW" manifest_package_arch="$(normalize_arch "$MANIFEST_PACKAGE_ARCH_RAW")" || die "unsupported manifest package arch: $MANIFEST_PACKAGE_ARCH_RAW"
@ -190,27 +421,6 @@ load_env() {
PACKAGE_PLATFORM="${PACKAGE_PLATFORM:-$(platform_for_arch "$PACKAGE_ARCH")}" PACKAGE_PLATFORM="${PACKAGE_PLATFORM:-$(platform_for_arch "$PACKAGE_ARCH")}"
fi fi
HOST_DATA_DIR="${HOST_DATA_DIR:-/var/lib/ours-rp-${PACKAGE_ARCH}-installer}"
COMPOSE_PROJECT_NAME="${COMPOSE_PROJECT_NAME:-ours-rp-${PACKAGE_ARCH}-installer}"
RPKI_IMAGE="${RPKI_IMAGE:-ours-rp-runtime-${PACKAGE_ARCH}:${source_commit_short:-unknown}}"
RPKI_PLATFORM="${RPKI_PLATFORM:-$PACKAGE_PLATFORM}"
METRICS_IMAGE="${METRICS_IMAGE:-ours-rp-metrics-${PACKAGE_ARCH}:${source_commit_short:-unknown}}"
METRICS_PLATFORM="${METRICS_PLATFORM:-$PACKAGE_PLATFORM}"
RTR_REPORT_DIR="${RTR_REPORT_DIR:-$HOST_DATA_DIR/empty-rtr-report}"
RTR_REPORT_CONTAINER_DIR="${RTR_REPORT_CONTAINER_DIR:-/var/lib/ours-rp/rtr-report}"
MONITOR_PLATFORM="${MONITOR_PLATFORM:-$PACKAGE_PLATFORM}"
PROMETHEUS_IMAGE="${PROMETHEUS_IMAGE:-prom/prometheus:v2.55.1}"
GRAFANA_IMAGE="${GRAFANA_IMAGE:-grafana/grafana:11.3.1}"
METRICS_INSTANCE="${METRICS_INSTANCE:-${PACKAGE_ARCH}-installer}"
ALLOW_CROSS_ARCH="${ALLOW_CROSS_ARCH:-0}"
RESOURCE_VALIDATION_MODE="${RESOURCE_VALIDATION_MODE:-validation-update-03}"
FIRST_RUN_WAIT_TIMEOUT_SECS="${FIRST_RUN_WAIT_TIMEOUT_SECS:-7200}"
persist_env_default "$ENV_FILE" "METRICS_IMAGE" "$METRICS_IMAGE"
persist_env_default "$ENV_FILE" "METRICS_PLATFORM" "$METRICS_PLATFORM"
persist_env_default "$ENV_FILE" "RTR_REPORT_DIR" "$RTR_REPORT_DIR"
persist_env_default "$ENV_FILE" "RTR_REPORT_CONTAINER_DIR" "$RTR_REPORT_CONTAINER_DIR"
persist_env_default "$ENV_FILE" "RESOURCE_VALIDATION_MODE" "$RESOURCE_VALIDATION_MODE"
} }
compose_cmd() { compose_cmd() {

99
deploy/docker-installer/scripts/upgrade.sh
View File

@ -13,6 +13,8 @@ Usage: ./scripts/upgrade.sh [--reuse-env-from /path/to/.env] [--keep-reused-imag
By default, --reuse-env-from creates the new .env from this package's By default, --reuse-env-from creates the new .env from this package's
.env.example first, then overlays existing user settings from the old .env. .env.example first, then overlays existing user settings from the old .env.
For declared configuration keys, explicit invocation variables have highest
priority, for example: INTERVAL_SECS=300 ./scripts/upgrade.sh ...
Image tags are intentionally kept from the new package so the upgraded service Image tags are intentionally kept from the new package so the upgraded service
actually runs the new packaged runtime and monitor images. Use actually runs the new packaged runtime and monitor images. Use
--keep-reused-image only when you intentionally want to keep previous image tags. --keep-reused-image only when you intentionally want to keep previous image tags.
@ -39,56 +41,80 @@ while [[ $# -gt 0 ]]; do
esac esac
done done
env_get_key() {
local env_path="$1"
local key="$2"
awk -F= -v key="$key" '$1 == key {sub(/^[^=]*=/, ""); print; exit}' "$env_path"
}
env_set_key() {
local env_path="$1"
local key="$2"
local value="$3"
local tmp_env
if grep -q "^${key}=" "$env_path"; then
tmp_env="$(mktemp)"
awk -v key="$key" -v value="$value" '
BEGIN { done=0 }
$0 ~ "^" key "=" { print key "=" value; done=1; next }
{ print }
END { if (!done) print key "=" value }
' "$env_path" > "$tmp_env"
mv "$tmp_env" "$env_path"
else
printf '%s=%s\n' "$key" "$value" >> "$env_path"
fi
}
overlay_reused_env() { overlay_reused_env() {
local source_env="$1" local source_env="$1"
local target_env="$2" local target_env="$2"
local key local key
local value local value
while IFS='=' read -r key _; do local -a reused_keys=()
[[ -n "$key" ]] || continue local -a ignored_removed_keys=()
[[ "$key" =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]] || continue local -a preserved_package_keys=()
REUSED_ENV_KEYS=()
parse_env_file "$source_env" || die "unable to parse reused environment: $source_env"
while IFS= read -r key; do
if ! template_has_env_key "$key"; then
ignored_removed_keys+=("$key")
continue
fi
case "$key" in case "$key" in
PACKAGE_ARCH|PACKAGE_PLATFORM|RPKI_PLATFORM|METRICS_PLATFORM|MONITOR_PLATFORM) PACKAGE_ARCH|PACKAGE_PLATFORM|RPKI_PLATFORM|METRICS_PLATFORM|MONITOR_PLATFORM)
preserved_package_keys+=("$key")
continue continue
;; ;;
RPKI_IMAGE|METRICS_IMAGE|PROMETHEUS_IMAGE|GRAFANA_IMAGE) RPKI_IMAGE|METRICS_IMAGE|PROMETHEUS_IMAGE|GRAFANA_IMAGE)
[[ "$UPDATE_PACKAGE_IMAGE" == "0" ]] || continue if [[ "$UPDATE_PACKAGE_IMAGE" != "0" ]]; then
preserved_package_keys+=("$key")
continue
fi
;; ;;
esac esac
value="$(env_get_key "$source_env" "$key")" [[ -n "${ENV_PARSED_VALUES[$key]+x}" ]] || die "unable to read $key from reused environment: $source_env"
env_set_key "$target_env" "$key" "$value" value="${ENV_PARSED_VALUES[$key]}"
done < <(grep -E '^[A-Za-z_][A-Za-z0-9_]*=' "$source_env" || true) env_file_set_value "$target_env" "$key" "$value"
REUSED_ENV_KEYS["$key"]=1
reused_keys+=("$key")
done < <(env_file_assignment_keys "$source_env")
log_env_key_group "reused_keys" "${reused_keys[@]}"
log_env_key_group "ignored_removed_old_keys" "${ignored_removed_keys[@]}"
log_env_key_group "preserved_package_keys" "${preserved_package_keys[@]}"
} }
apply_invocation_env_overrides() {
local target_env="$1"
local key
local -a overridden_keys=()
for key in "${ENV_TEMPLATE_KEYS[@]}"; do
[[ -n "${INVOCATION_ENV_OVERRIDES[$key]+x}" ]] || continue
env_file_set_value "$target_env" "$key" "${INVOCATION_ENV_OVERRIDES[$key]}"
overridden_keys+=("$key")
done
log_env_key_group "command_override_keys" "${overridden_keys[@]}"
}
log_template_only_keys() {
local key
local -a template_only_keys=()
for key in "${ENV_TEMPLATE_KEYS[@]}"; do
[[ -n "${REUSED_ENV_KEYS[$key]+x}" ]] && continue
[[ -n "${INVOCATION_ENV_OVERRIDES[$key]+x}" ]] && continue
template_only_keys+=("$key")
done
log_env_key_group "template_only_keys" "${template_only_keys[@]}"
}
load_env_template_keys "$ENV_EXAMPLE"
validate_env_file_contract "$ENV_EXAMPLE"
capture_invocation_env_overrides
REUSED_ENV_KEYS=()
if [[ -n "$REUSE_ENV_FROM" ]]; then if [[ -n "$REUSE_ENV_FROM" ]]; then
[[ -f "$REUSE_ENV_FROM" ]] || die "missing reuse env file: $REUSE_ENV_FROM" [[ -f "$REUSE_ENV_FROM" ]] || die "missing reuse env file: $REUSE_ENV_FROM"
if [[ ! -f "$ENV_FILE" ]]; then if [[ ! -f "$ENV_FILE" ]]; then
[[ -f "$ENV_EXAMPLE" ]] || die "missing $ENV_EXAMPLE"
cp "$ENV_EXAMPLE" "$ENV_FILE" cp "$ENV_EXAMPLE" "$ENV_FILE"
overlay_reused_env "$REUSE_ENV_FROM" "$ENV_FILE" overlay_reused_env "$REUSE_ENV_FROM" "$ENV_FILE"
log "created new package env from .env.example and overlaid user settings from $REUSE_ENV_FROM" log "created new package env from .env.example and overlaid user settings from $REUSE_ENV_FROM"
@ -97,6 +123,13 @@ if [[ -n "$REUSE_ENV_FROM" ]]; then
fi fi
fi fi
if [[ ! -f "$ENV_FILE" ]]; then
cp "$ENV_EXAMPLE" "$ENV_FILE"
log "created new package env from .env.example"
fi
apply_invocation_env_overrides "$ENV_FILE"
log_template_only_keys
validate_env_file_contract "$ENV_FILE"
load_env load_env
assert_arch_compatibility assert_arch_compatibility
create_data_dirs create_data_dirs

206
deploy/docker-installer/tests/upgrade_env_contract_test.sh Normal file
View File

@ -0,0 +1,206 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
INSTALLER_SOURCE="$(cd "$SCRIPT_DIR/.." && pwd)"
TEST_ROOT="$(mktemp -d "${TMPDIR:-/tmp}/ours-rp-installer-env-contract.XXXXXX")"
cleanup() {
rm -rf "$TEST_ROOT"
}
trap cleanup EXIT
fail() {
printf 'FAIL: %s\n' "$*" >&2
exit 1
}
assert_equals() {
local expected="$1"
local actual="$2"
local message="$3"
[[ "$actual" == "$expected" ]] || fail "$message (expected=$expected actual=$actual)"
}
assert_not_contains() {
local needle="$1"
local path="$2"
! grep -Fq "$needle" "$path" || fail "unexpected value in $path: $needle"
}
expect_exit() {
local expected_status="$1"
local output_path="$2"
shift 2
local actual_status
set +e
"$@" >"$output_path" 2>&1
actual_status=$?
set -e
[[ "$actual_status" == "$expected_status" ]] || {
cat "$output_path" >&2 || true
fail "unexpected exit status (expected=$expected_status actual=$actual_status)"
}
}
host_arch() {
case "$(uname -m)" in
x86_64) printf 'amd64\n' ;;
aarch64) printf 'arm64\n' ;;
*) fail "unsupported test host architecture: $(uname -m)" ;;
esac
}
env_value() {
local env_path="$1"
local key="$2"
(
# shellcheck disable=SC1090
source "$INSTALLER_SOURCE/scripts/common.sh"
env_file_value "$env_path" "$key"
)
}
create_fixture() {
local name="$1"
local fixture="$TEST_ROOT/$name"
local arch
arch="$(host_arch)"
mkdir -p "$fixture"
cp -a "$INSTALLER_SOURCE"/. "$fixture"/
rm -rf "$fixture/tests"
mkdir -p "$fixture/images" "$fixture/fake-bin"
sed -i \
-e "s|__PACKAGE_ARCH__|$arch|g" \
-e "s|__PACKAGE_PLATFORM__|linux/$arch|g" \
-e "s|__RUNTIME_IMAGE__|ours-rp-runtime-$arch:new|g" \
-e "s|__METRICS_IMAGE__|ours-rp-metrics-$arch:new|g" \
-e "s|__HOST_DATA_DIR__|$fixture/data|g" \
"$fixture/.env.example"
cat > "$fixture/PACKAGE-MANIFEST.env" <<EOF
PACKAGE_ARCH=$arch
PACKAGE_PLATFORM=linux/$arch
EOF
cat > "$fixture/fake-bin/docker" <<'EOF'
#!/usr/bin/env bash
printf '%s\n' "$*" >> "$FAKE_DOCKER_LOG"
if [[ "${1:-}" == "compose" && "${2:-}" == "version" ]]; then
exit 0
fi
exit 99
EOF
chmod +x "$fixture/fake-bin/docker"
for command in jq rsync curl gzip tar; do
cat > "$fixture/fake-bin/$command" <<'EOF'
#!/usr/bin/env bash
exit 0
EOF
chmod +x "$fixture/fake-bin/$command"
done
printf '%s\n' "$fixture"
}
run_upgrade() {
local fixture="$1"
local old_env="$2"
shift 2
(
cd "$fixture"
export PATH="$fixture/fake-bin:$PATH"
export FAKE_DOCKER_LOG="$fixture/fake-docker.log"
"$@" ./scripts/upgrade.sh --reuse-env-from "$old_env"
)
}
run_upgrade_without_reuse() {
local fixture="$1"
(
cd "$fixture"
export PATH="$fixture/fake-bin:$PATH"
export FAKE_DOCKER_LOG="$fixture/fake-docker.log"
./scripts/upgrade.sh
)
}
new_fixture_with_old_env() {
local name="$1"
local fixture
fixture="$(create_fixture "$name")"
cat > "$fixture/old.env" <<'EOF'
INTERVAL_SECS=120
RPKI_IMAGE=ours-rp-runtime-old:old
REMOVED_LEGACY_KEY=legacy-value
EOF
printf '%s\n' "$fixture"
}
fixture="$(create_fixture template-defaults)"
expect_exit 99 "$fixture/template.log" run_upgrade_without_reuse "$fixture"
assert_equals "600" "$(env_value "$fixture/.env" INTERVAL_SECS)" "template default must be retained without reuse"
fixture="$(new_fixture_with_old_env default-old)"
expect_exit 99 "$fixture/default.log" run_upgrade "$fixture" "$fixture/old.env"
assert_equals "120" "$(env_value "$fixture/.env" INTERVAL_SECS)" "old value must override template default"
assert_equals "ours-rp-runtime-$(host_arch):new" "$(env_value "$fixture/.env" RPKI_IMAGE)" "standard upgrade must retain package image"
assert_equals "10" "$(env_value "$fixture/.env" METRICS_POLL_SECS)" "template default must fill a key absent from old environment"
! grep -q '^REMOVED_LEGACY_KEY=' "$fixture/.env" || fail "removed old key was copied into target environment"
grep -q 'reused_keys=INTERVAL_SECS' "$fixture/default.log" || fail "reused key summary missing"
grep -q 'ignored_removed_old_keys=REMOVED_LEGACY_KEY' "$fixture/default.log" || fail "removed key summary missing"
grep -q 'template_only_keys=' "$fixture/default.log" || fail "template key summary missing"
fixture="$(new_fixture_with_old_env invocation-overrides)"
expect_exit 99 "$fixture/override.log" run_upgrade "$fixture" "$fixture/old.env" env INTERVAL_SECS=300 GRAFANA_ADMIN_PASSWORD='secret with space'
assert_equals "300" "$(env_value "$fixture/.env" INTERVAL_SECS)" "invocation value must override old value"
assert_equals "secret with space" "$(env_value "$fixture/.env" GRAFANA_ADMIN_PASSWORD)" "invocation value must preserve shell-sensitive content"
grep -q 'command_override_keys=INTERVAL_SECS,GRAFANA_ADMIN_PASSWORD' "$fixture/override.log" || fail "override key summary missing"
assert_not_contains 'secret with space' "$fixture/override.log"
fixture="$(new_fixture_with_old_env invocation-special-characters)"
special_password="O'Reilly"
special_password+=' $literal `tick` slash\'
expect_exit 99 "$fixture/special.log" run_upgrade "$fixture" "$fixture/old.env" env GRAFANA_ADMIN_PASSWORD="$special_password"
assert_equals "$special_password" "$(env_value "$fixture/.env" GRAFANA_ADMIN_PASSWORD)" "invocation value with dotenv-sensitive characters must round-trip"
assert_not_contains "$special_password" "$fixture/special.log"
fixture="$(create_fixture existing-target-env)"
cp "$fixture/.env.example" "$fixture/.env"
sed -i 's/^INTERVAL_SECS=.*/INTERVAL_SECS=180/' "$fixture/.env"
cat > "$fixture/old.env" <<'EOF'
INTERVAL_SECS=120
EOF
expect_exit 99 "$fixture/existing.log" run_upgrade "$fixture" "$fixture/old.env" env INTERVAL_SECS=300
assert_equals "300" "$(env_value "$fixture/.env" INTERVAL_SECS)" "invocation value must override an existing target environment"
fixture="$(new_fixture_with_old_env explicit-empty)"
expect_exit 1 "$fixture/empty.log" run_upgrade "$fixture" "$fixture/old.env" env INTERVAL_SECS=
[[ ! -s "$fixture/fake-docker.log" ]] || fail "explicit empty value reached Docker"
grep -q 'INTERVAL_SECS' "$fixture/empty.log" || fail "empty-value error did not identify key"
fixture="$(create_fixture empty-old-value)"
cat > "$fixture/old.env" <<'EOF'
INTERVAL_SECS=
EOF
expect_exit 1 "$fixture/old-empty.log" run_upgrade "$fixture" "$fixture/old.env"
[[ ! -s "$fixture/fake-docker.log" ]] || fail "empty reused value reached Docker"
grep -q 'INTERVAL_SECS' "$fixture/old-empty.log" || fail "empty reused value error did not identify key"
fixture="$(create_fixture empty-template)"
sed -i 's/^INTERVAL_SECS=.*/INTERVAL_SECS=/' "$fixture/.env.example"
cat > "$fixture/old.env" <<'EOF'
INTERVAL_SECS=120
EOF
expect_exit 1 "$fixture/template-empty.log" run_upgrade "$fixture" "$fixture/old.env"
[[ ! -s "$fixture/fake-docker.log" ]] || fail "empty template default reached Docker"
grep -q 'INTERVAL_SECS' "$fixture/template-empty.log" || fail "empty template error did not identify key"
fixture="$(new_fixture_with_old_env package-guard)"
arch="$(host_arch)"
other_arch="amd64"
[[ "$arch" == "amd64" ]] && other_arch="arm64"
expect_exit 1 "$fixture/arch.log" run_upgrade "$fixture" "$fixture/old.env" env PACKAGE_ARCH="$other_arch"
[[ ! -s "$fixture/fake-docker.log" ]] || fail "package architecture mismatch reached Docker"
grep -q 'mismatches manifest PACKAGE_ARCH' "$fixture/arch.log" || fail "package architecture guard did not reject override"
printf 'PASS: Docker installer upgrade environment contract\n'