diff --git a/deploy/docker-installer/.env.example b/deploy/docker-installer/.env.example index 78c7b11..932f0ef 100644 --- a/deploy/docker-installer/.env.example +++ b/deploy/docker-installer/.env.example @@ -2,6 +2,8 @@ # `build_docker_installer_package.sh` rewrites package-specific placeholders # when producing amd64/arm64 release packages. # 中文说明见 docs/README.zh-CN.md。English guide: docs/README.en.md +# Every assignment is a required non-empty configuration key with a package +# default. During upgrade: .env.example < reused old .env < command environment. # Package metadata and architecture guardrails. PACKAGE_ARCH=__PACKAGE_ARCH__ diff --git a/deploy/docker-installer/docs/README.en.md b/deploy/docker-installer/docs/README.en.md index 0dba1e4..0f11509 100644 --- a/deploy/docker-installer/docs/README.en.md +++ b/deploy/docker-installer/docs/README.en.md @@ -80,7 +80,7 @@ Supported values: - `rfc6487`: switch back to the original RFC 6487 resource containment behavior. The installer / soak runner passes `--resource-validation-mode` to every `rpki` child start. -When an upgraded deployment reuses an older `.env` that does not contain this key, the installer appends the new default `validation-update-03` automatically. +When an upgraded deployment reuses an older `.env` that does not contain this key, the new package `.env.example` default `validation-update-03` remains in the final `.env`; an empty final value fails before Docker operations. ## Architecture Guardrails diff --git a/deploy/docker-installer/docs/README.zh-CN.md b/deploy/docker-installer/docs/README.zh-CN.md index 7b05f9b..b627e40 100644 --- a/deploy/docker-installer/docs/README.zh-CN.md +++ b/deploy/docker-installer/docs/README.zh-CN.md @@ -80,7 +80,7 @@ RESOURCE_VALIDATION_MODE=validation-update-03 - `rfc6487`：切换为 RFC 6487 原始资源包含判定语义。 installer / soak runner 每次启动 `rpki` 子进程时都会显式传入 `--resource-validation-mode`。 -如果升级时复用旧 `.env` 且缺少该变量，installer 会自动补入默认值 `validation-update-03`，不会破坏旧 `.env` 复用流程。 +如果升级时复用旧 `.env` 且缺少该变量，新包 `.env.example` 中的默认值 `validation-update-03` 会保留在最终 `.env` 中；最终值为空时升级会在 Docker 操作前失败。 ## 架构检查 diff --git a/deploy/docker-installer/docs/operations.en.md b/deploy/docker-installer/docs/operations.en.md index f13525a..e8f3131 100644 --- a/deploy/docker-installer/docs/operations.en.md +++ b/deploy/docker-installer/docs/operations.en.md @@ -89,9 +89,26 @@ Extract the new package into a new directory and explicitly reuse the existing ` ./scripts/upgrade.sh --reuse-env-from /path/to/old-installer/.env ``` -If the new package directory already has a `.env`, the upgrade script keeps it. +Configuration is merged in this order, with a key on the right overriding the same key on the left: -If an older `.env` does not contain `METRICS_IMAGE` or `METRICS_PLATFORM`, the install/upgrade flow backfills package-matched defaults automatically. +```text +new .env.example < old .env < environment variables on this upgrade command +``` + +For example, override the next-run interval and Grafana password for this upgrade: + +```bash +INTERVAL_SECS=300 GRAFANA_ADMIN_PASSWORD='new-password' \ + ./scripts/upgrade.sh --reuse-env-from /path/to/old-installer/.env +``` + +Before any image is loaded or container is started, upgrade verifies that every key declared by the new template is present and non-empty in the final `.env`. An explicit empty assignment fails, for example `INTERVAL_SECS= ./scripts/upgrade.sh ...`. Removed keys from an old `.env` are not copied, and the script reports key names only, never password-like values. + +The installer reads `.env` as Compose-compatible dotenv assignments. Do not put shell command substitutions or environment references such as `$HOME` in the file; use concrete values or command-level overrides instead. + +If the new package directory already has a `.env`, the upgrade script keeps it. In that case `--reuse-env-from` is not imported again, but environment variables from the current command still override the existing `.env`. + +When an older `.env` lacks a key introduced by the new package, the new `.env.example` default remains in the final `.env`. Upgrade fails before Docker operations if any final declared value is empty. Upgrade does not delete: diff --git a/deploy/docker-installer/docs/operations.zh-CN.md b/deploy/docker-installer/docs/operations.zh-CN.md index 9c76def..4235e7a 100644 --- a/deploy/docker-installer/docs/operations.zh-CN.md +++ b/deploy/docker-installer/docs/operations.zh-CN.md @@ -89,9 +89,26 @@ curl -s "http://127.0.0.1:${METRICS_PORT:-9556}/metrics" | grep '^ours_rp_rtr_' ./scripts/upgrade.sh --reuse-env-from /path/to/old-installer/.env ``` -如果新目录已经存在 `.env`，升级脚本会保留它，不覆盖。 +配置按以下优先级合并，右侧同名键覆盖左侧： -如果旧 `.env` 缺失 `METRICS_IMAGE` 或 `METRICS_PLATFORM`，升级/安装链路会按当前包架构自动补齐默认值。 +```text +新包 .env.example < 旧 .env < 本次 upgrade 命令的环境变量 +``` + +例如，临时调整下轮触发间隔和 Grafana 密码： + +```bash +INTERVAL_SECS=300 GRAFANA_ADMIN_PASSWORD='new-password' \ + ./scripts/upgrade.sh --reuse-env-from /path/to/old-installer/.env +``` + +升级会在加载镜像或启动容器前检查：新模板声明的所有变量在最终 `.env` 中都必须存在且非空。显式传入空值会失败，例如 `INTERVAL_SECS= ./scripts/upgrade.sh ...`。旧 `.env` 中已不在新模板内的键不会复制，脚本只输出相关键名而不会输出密码等值。 + +installer 按与 Compose 一致的 dotenv 赋值格式读取 `.env`。不要在文件中使用 Shell 命令替换或 `$HOME` 一类环境变量引用；请使用确定的值，或在执行 upgrade 时通过命令环境变量覆盖。 + +如果新目录已经存在 `.env`，升级脚本会保留它，不覆盖；此时 `--reuse-env-from` 不会再次导入旧配置，但本次命令的环境变量仍会覆盖已存在的 `.env`。 + +旧 `.env` 缺少新版本新增的键时，新包 `.env.example` 的默认值会保留在最终 `.env` 中；任一最终值为空则升级会在 Docker 操作前失败。 升级不会删除以下数据： diff --git a/deploy/docker-installer/scripts/common.sh b/deploy/docker-installer/scripts/common.sh index 1a42039..163f28e 100644 --- a/deploy/docker-installer/scripts/common.sh +++ b/deploy/docker-installer/scripts/common.sh @@ -6,6 +6,11 @@ ENV_FILE="${ENV_FILE:-$INSTALLER_ROOT/.env}" ENV_EXAMPLE="$INSTALLER_ROOT/.env.example" COMPOSE_FILE="$INSTALLER_ROOT/compose/docker-compose.yml" MANIFEST_FILE="${MANIFEST_FILE:-$INSTALLER_ROOT/PACKAGE-MANIFEST.env}" +declare -a ENV_TEMPLATE_KEYS=() +declare -A ENV_TEMPLATE_KEY_SET=() +declare -A INVOCATION_ENV_OVERRIDES=() +declare -A REUSED_ENV_KEYS=() +declare -A ENV_PARSED_VALUES=() log() { printf '[ours-rp-installer] %s\n' "$*" @@ -48,20 +53,246 @@ env_file_has_key() { [[ -f "$env_path" ]] && grep -Eq "^${key}=" "$env_path" } -persist_env_default() { +load_env_template_keys() { + local env_path="$1" + local line + local key + + [[ -f "$env_path" ]] || die "missing environment template: $env_path" + ENV_TEMPLATE_KEYS=() + ENV_TEMPLATE_KEY_SET=() + + while IFS= read -r line || [[ -n "$line" ]]; do + line="${line%$'\r'}" + [[ "$line" =~ ^[[:space:]]*$ || "$line" =~ ^[[:space:]]*# ]] && continue + [[ "$line" =~ ^([A-Za-z_][A-Za-z0-9_]*)= ]] || die "invalid environment assignment in $env_path: $line" + key="${BASH_REMATCH[1]}" + [[ -z "${ENV_TEMPLATE_KEY_SET[$key]+x}" ]] || die "duplicate environment key in $env_path: $key" + ENV_TEMPLATE_KEYS+=("$key") + ENV_TEMPLATE_KEY_SET["$key"]=1 + done < "$env_path" + + (( ${#ENV_TEMPLATE_KEYS[@]} > 0 )) || die "no environment assignments found in $env_path" +} + +template_has_env_key() { + local key="$1" + [[ -n "${ENV_TEMPLATE_KEY_SET[$key]+x}" ]] +} + +env_file_assignment_keys() { + local env_path="$1" + local line + + [[ -f "$env_path" ]] || die "missing environment file: $env_path" + while IFS= read -r line || [[ -n "$line" ]]; do + line="${line%$'\r'}" + [[ "$line" =~ ^([A-Za-z_][A-Za-z0-9_]*)= ]] || continue + printf '%s\n' "${BASH_REMATCH[1]}" + done < "$env_path" +} + +dotenv_decode_value() { + local raw_value="$1" + local quote + local body + local decoded="" + local character + local next_character + local index=0 + + [[ -n "$raw_value" ]] || { + printf '\n' + return 0 + } + + quote="${raw_value:0:1}" + case "$quote" in + "'") + [[ "${raw_value: -1}" == "'" && ${#raw_value} -ge 2 ]] || return 1 + body="${raw_value:1:${#raw_value}-2}" + while (( index < ${#body} )); do + character="${body:index:1}" + if [[ "$character" == "\\" && $((index + 1)) -lt ${#body} ]]; then + next_character="${body:index + 1:1}" + case "$next_character" in + "'") + decoded+="$next_character" + ((index += 2)) + continue + ;; + esac + fi + decoded+="$character" + ((index += 1)) + done + ;; + '"') + [[ "${raw_value: -1}" == '"' && ${#raw_value} -ge 2 ]] || return 1 + body="${raw_value:1:${#raw_value}-2}" + while (( index < ${#body} )); do + character="${body:index:1}" + if [[ "$character" == '$' && $((index + 1)) -lt ${#body} && "${body:index + 1:1}" == '$' ]]; then + decoded+='$' + ((index += 2)) + continue + fi + if [[ "$character" == "\\" && $((index + 1)) -lt ${#body} ]]; then + next_character="${body:index + 1:1}" + case "$next_character" in + '"'|\\|'$'|'`') + decoded+="$next_character" + ((index += 2)) + continue + ;; + n) + decoded+=$'\n' + ((index += 2)) + continue + ;; + r) + decoded+=$'\r' + ((index += 2)) + continue + ;; + t) + decoded+=$'\t' + ((index += 2)) + continue + ;; + esac + fi + decoded+="$character" + ((index += 1)) + done + ;; + *) + decoded="$raw_value" + ;; + esac + + printf '%s' "$decoded" +} + +parse_env_file() { + local env_path="$1" + local line + local parsed_key + local raw_value + local parsed_value + + [[ -f "$env_path" ]] || die "missing environment file: $env_path" + ENV_PARSED_VALUES=() + while IFS= read -r line || [[ -n "$line" ]]; do + line="${line%$'\r'}" + [[ "$line" =~ ^[[:space:]]*$ || "$line" =~ ^[[:space:]]*# ]] && continue + [[ "$line" =~ ^([A-Za-z_][A-Za-z0-9_]*)=(.*)$ ]] || return 2 + parsed_key="${BASH_REMATCH[1]}" + raw_value="${BASH_REMATCH[2]}" + parsed_value="$(dotenv_decode_value "$raw_value")" || return 2 + ENV_PARSED_VALUES["$parsed_key"]="$parsed_value" + done < "$env_path" +} + +env_file_value() { + local env_path="$1" + local key="$2" + + parse_env_file "$env_path" + + [[ -n "${ENV_PARSED_VALUES[$key]+x}" ]] || return 3 + printf '%s' "${ENV_PARSED_VALUES[$key]}" +} + +env_file_set_value() { local env_path="$1" local key="$2" local value="$3" local tmp_env - if env_file_has_key "$env_path" "$key"; then + local encoded_value + + [[ "$value" != *$'\n'* && "$value" != *$'\r'* ]] || die "environment value for $key contains an unsupported newline" + encoded_value="$(env_file_encode_value "$value")" + if ! env_file_has_key "$env_path" "$key"; then + printf '%s=%s\n' "$key" "$encoded_value" >> "$env_path" return 0 fi tmp_env="$(mktemp)" - cat "$env_path" > "$tmp_env" - printf '%s=%s\n' "$key" "$value" >> "$tmp_env" + ENV_FILE_REPLACEMENT="$key=$encoded_value" awk -v key="$key" ' + BEGIN { done=0 } + $0 ~ "^" key "=" { print ENVIRON["ENV_FILE_REPLACEMENT"]; done=1; next } + { print } + END { if (!done) print ENVIRON["ENV_FILE_REPLACEMENT"] } + ' "$env_path" > "$tmp_env" mv "$tmp_env" "$env_path" } +env_file_encode_value() { + local value="$1" + local escaped="$value" + + [[ "$value" != *$'\n'* && "$value" != *$'\r'* ]] || die "environment value contains an unsupported newline" + escaped="${escaped//\\/\\\\}" + escaped="${escaped//\"/\\\"}" + escaped="${escaped//\$/\$\$}" + printf '"%s"' "$escaped" +} + +load_env_file() { + local env_path="$1" + local key + + parse_env_file "$env_path" || die "unable to parse environment file: $env_path" + for key in "${!ENV_PARSED_VALUES[@]}"; do + printf -v "$key" '%s' "${ENV_PARSED_VALUES[$key]}" + export "$key" + done +} + +validate_env_file_contract() { + local env_path="$1" + local invalid_keys="" + local key + + [[ -f "$env_path" ]] || die "missing environment file: $env_path" + (( ${#ENV_TEMPLATE_KEYS[@]} > 0 )) || die "environment template keys were not loaded" + + if ! parse_env_file "$env_path"; then + die "unable to parse environment file: $env_path" + fi + + for key in "${ENV_TEMPLATE_KEYS[@]}"; do + if [[ -z "${ENV_PARSED_VALUES[$key]+x}" || -z "${ENV_PARSED_VALUES[$key]}" ]]; then + invalid_keys+="$key"$'\n' + fi + done + + [[ -z "$invalid_keys" ]] || die "environment contract requires non-empty values in $env_path: ${invalid_keys//$'\n'/,}" +} + +capture_invocation_env_overrides() { + local key + + (( ${#ENV_TEMPLATE_KEYS[@]} > 0 )) || die "environment template keys were not loaded" + INVOCATION_ENV_OVERRIDES=() + for key in "${ENV_TEMPLATE_KEYS[@]}"; do + if [[ -v "$key" ]]; then + INVOCATION_ENV_OVERRIDES["$key"]="${!key}" + fi + done +} + +log_env_key_group() { + local group="$1" + shift + local rendered="none" + if (( $# > 0 )); then + local IFS=, + rendered="$*" + fi + log "$group=$rendered" +} + env_flag_enabled() { case "${1:-0}" in 1|true|TRUE|yes|YES|on|ON) @@ -156,15 +387,15 @@ assert_arch_compatibility() { load_env() { load_manifest + [[ -f "$ENV_EXAMPLE" ]] || die "missing $ENV_EXAMPLE" + load_env_template_keys "$ENV_EXAMPLE" + validate_env_file_contract "$ENV_EXAMPLE" if [[ ! -f "$ENV_FILE" ]]; then - [[ -f "$ENV_EXAMPLE" ]] || die "missing $ENV_EXAMPLE" cp "$ENV_EXAMPLE" "$ENV_FILE" log "created .env from .env.example" fi - set -a - # shellcheck disable=SC1090 - source "$ENV_FILE" - set +a + validate_env_file_contract "$ENV_FILE" + load_env_file "$ENV_FILE" if [[ -n "${MANIFEST_PACKAGE_ARCH_RAW:-}" ]]; then manifest_package_arch="$(normalize_arch "$MANIFEST_PACKAGE_ARCH_RAW")" || die "unsupported manifest package arch: $MANIFEST_PACKAGE_ARCH_RAW" @@ -190,27 +421,6 @@ load_env() { PACKAGE_PLATFORM="${PACKAGE_PLATFORM:-$(platform_for_arch "$PACKAGE_ARCH")}" fi - HOST_DATA_DIR="${HOST_DATA_DIR:-/var/lib/ours-rp-${PACKAGE_ARCH}-installer}" - COMPOSE_PROJECT_NAME="${COMPOSE_PROJECT_NAME:-ours-rp-${PACKAGE_ARCH}-installer}" - RPKI_IMAGE="${RPKI_IMAGE:-ours-rp-runtime-${PACKAGE_ARCH}:${source_commit_short:-unknown}}" - RPKI_PLATFORM="${RPKI_PLATFORM:-$PACKAGE_PLATFORM}" - METRICS_IMAGE="${METRICS_IMAGE:-ours-rp-metrics-${PACKAGE_ARCH}:${source_commit_short:-unknown}}" - METRICS_PLATFORM="${METRICS_PLATFORM:-$PACKAGE_PLATFORM}" - RTR_REPORT_DIR="${RTR_REPORT_DIR:-$HOST_DATA_DIR/empty-rtr-report}" - RTR_REPORT_CONTAINER_DIR="${RTR_REPORT_CONTAINER_DIR:-/var/lib/ours-rp/rtr-report}" - MONITOR_PLATFORM="${MONITOR_PLATFORM:-$PACKAGE_PLATFORM}" - PROMETHEUS_IMAGE="${PROMETHEUS_IMAGE:-prom/prometheus:v2.55.1}" - GRAFANA_IMAGE="${GRAFANA_IMAGE:-grafana/grafana:11.3.1}" - METRICS_INSTANCE="${METRICS_INSTANCE:-${PACKAGE_ARCH}-installer}" - ALLOW_CROSS_ARCH="${ALLOW_CROSS_ARCH:-0}" - RESOURCE_VALIDATION_MODE="${RESOURCE_VALIDATION_MODE:-validation-update-03}" - FIRST_RUN_WAIT_TIMEOUT_SECS="${FIRST_RUN_WAIT_TIMEOUT_SECS:-7200}" - - persist_env_default "$ENV_FILE" "METRICS_IMAGE" "$METRICS_IMAGE" - persist_env_default "$ENV_FILE" "METRICS_PLATFORM" "$METRICS_PLATFORM" - persist_env_default "$ENV_FILE" "RTR_REPORT_DIR" "$RTR_REPORT_DIR" - persist_env_default "$ENV_FILE" "RTR_REPORT_CONTAINER_DIR" "$RTR_REPORT_CONTAINER_DIR" - persist_env_default "$ENV_FILE" "RESOURCE_VALIDATION_MODE" "$RESOURCE_VALIDATION_MODE" } compose_cmd() { diff --git a/deploy/docker-installer/scripts/upgrade.sh b/deploy/docker-installer/scripts/upgrade.sh index 8d1932d..528038b 100755 --- a/deploy/docker-installer/scripts/upgrade.sh +++ b/deploy/docker-installer/scripts/upgrade.sh @@ -13,6 +13,8 @@ Usage: ./scripts/upgrade.sh [--reuse-env-from /path/to/.env] [--keep-reused-imag By default, --reuse-env-from creates the new .env from this package's .env.example first, then overlays existing user settings from the old .env. +For declared configuration keys, explicit invocation variables have highest +priority, for example: INTERVAL_SECS=300 ./scripts/upgrade.sh ... Image tags are intentionally kept from the new package so the upgraded service actually runs the new packaged runtime and monitor images. Use --keep-reused-image only when you intentionally want to keep previous image tags. @@ -39,56 +41,80 @@ while [[ $# -gt 0 ]]; do esac done -env_get_key() { - local env_path="$1" - local key="$2" - awk -F= -v key="$key" '$1 == key {sub(/^[^=]*=/, ""); print; exit}' "$env_path" -} - -env_set_key() { - local env_path="$1" - local key="$2" - local value="$3" - local tmp_env - if grep -q "^${key}=" "$env_path"; then - tmp_env="$(mktemp)" - awk -v key="$key" -v value="$value" ' - BEGIN { done=0 } - $0 ~ "^" key "=" { print key "=" value; done=1; next } - { print } - END { if (!done) print key "=" value } - ' "$env_path" > "$tmp_env" - mv "$tmp_env" "$env_path" - else - printf '%s=%s\n' "$key" "$value" >> "$env_path" - fi -} - overlay_reused_env() { local source_env="$1" local target_env="$2" local key local value - while IFS='=' read -r key _; do - [[ -n "$key" ]] || continue - [[ "$key" =~ ^[A-Za-z_][A-Za-z0-9_]*$ ]] || continue + local -a reused_keys=() + local -a ignored_removed_keys=() + local -a preserved_package_keys=() + + REUSED_ENV_KEYS=() + parse_env_file "$source_env" || die "unable to parse reused environment: $source_env" + while IFS= read -r key; do + if ! template_has_env_key "$key"; then + ignored_removed_keys+=("$key") + continue + fi case "$key" in PACKAGE_ARCH|PACKAGE_PLATFORM|RPKI_PLATFORM|METRICS_PLATFORM|MONITOR_PLATFORM) + preserved_package_keys+=("$key") continue ;; RPKI_IMAGE|METRICS_IMAGE|PROMETHEUS_IMAGE|GRAFANA_IMAGE) - [[ "$UPDATE_PACKAGE_IMAGE" == "0" ]] || continue + if [[ "$UPDATE_PACKAGE_IMAGE" != "0" ]]; then + preserved_package_keys+=("$key") + continue + fi ;; esac - value="$(env_get_key "$source_env" "$key")" - env_set_key "$target_env" "$key" "$value" - done < <(grep -E '^[A-Za-z_][A-Za-z0-9_]*=' "$source_env" || true) + [[ -n "${ENV_PARSED_VALUES[$key]+x}" ]] || die "unable to read $key from reused environment: $source_env" + value="${ENV_PARSED_VALUES[$key]}" + env_file_set_value "$target_env" "$key" "$value" + REUSED_ENV_KEYS["$key"]=1 + reused_keys+=("$key") + done < <(env_file_assignment_keys "$source_env") + + log_env_key_group "reused_keys" "${reused_keys[@]}" + log_env_key_group "ignored_removed_old_keys" "${ignored_removed_keys[@]}" + log_env_key_group "preserved_package_keys" "${preserved_package_keys[@]}" } +apply_invocation_env_overrides() { + local target_env="$1" + local key + local -a overridden_keys=() + + for key in "${ENV_TEMPLATE_KEYS[@]}"; do + [[ -n "${INVOCATION_ENV_OVERRIDES[$key]+x}" ]] || continue + env_file_set_value "$target_env" "$key" "${INVOCATION_ENV_OVERRIDES[$key]}" + overridden_keys+=("$key") + done + + log_env_key_group "command_override_keys" "${overridden_keys[@]}" +} + +log_template_only_keys() { + local key + local -a template_only_keys=() + + for key in "${ENV_TEMPLATE_KEYS[@]}"; do + [[ -n "${REUSED_ENV_KEYS[$key]+x}" ]] && continue + [[ -n "${INVOCATION_ENV_OVERRIDES[$key]+x}" ]] && continue + template_only_keys+=("$key") + done + log_env_key_group "template_only_keys" "${template_only_keys[@]}" +} + +load_env_template_keys "$ENV_EXAMPLE" +validate_env_file_contract "$ENV_EXAMPLE" +capture_invocation_env_overrides +REUSED_ENV_KEYS=() + if [[ -n "$REUSE_ENV_FROM" ]]; then [[ -f "$REUSE_ENV_FROM" ]] || die "missing reuse env file: $REUSE_ENV_FROM" if [[ ! -f "$ENV_FILE" ]]; then - [[ -f "$ENV_EXAMPLE" ]] || die "missing $ENV_EXAMPLE" cp "$ENV_EXAMPLE" "$ENV_FILE" overlay_reused_env "$REUSE_ENV_FROM" "$ENV_FILE" log "created new package env from .env.example and overlaid user settings from $REUSE_ENV_FROM" @@ -97,6 +123,13 @@ if [[ -n "$REUSE_ENV_FROM" ]]; then fi fi +if [[ ! -f "$ENV_FILE" ]]; then + cp "$ENV_EXAMPLE" "$ENV_FILE" + log "created new package env from .env.example" +fi +apply_invocation_env_overrides "$ENV_FILE" +log_template_only_keys +validate_env_file_contract "$ENV_FILE" load_env assert_arch_compatibility create_data_dirs diff --git a/deploy/docker-installer/tests/upgrade_env_contract_test.sh b/deploy/docker-installer/tests/upgrade_env_contract_test.sh new file mode 100644 index 0000000..00d8468 --- /dev/null +++ b/deploy/docker-installer/tests/upgrade_env_contract_test.sh @@ -0,0 +1,206 @@ +#!/usr/bin/env bash +set -euo pipefail + +SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)" +INSTALLER_SOURCE="$(cd "$SCRIPT_DIR/.." && pwd)" +TEST_ROOT="$(mktemp -d "${TMPDIR:-/tmp}/ours-rp-installer-env-contract.XXXXXX")" + +cleanup() { + rm -rf "$TEST_ROOT" +} +trap cleanup EXIT + +fail() { + printf 'FAIL: %s\n' "$*" >&2 + exit 1 +} + +assert_equals() { + local expected="$1" + local actual="$2" + local message="$3" + [[ "$actual" == "$expected" ]] || fail "$message (expected=$expected actual=$actual)" +} + +assert_not_contains() { + local needle="$1" + local path="$2" + ! grep -Fq "$needle" "$path" || fail "unexpected value in $path: $needle" +} + +expect_exit() { + local expected_status="$1" + local output_path="$2" + shift 2 + local actual_status + + set +e + "$@" >"$output_path" 2>&1 + actual_status=$? + set -e + [[ "$actual_status" == "$expected_status" ]] || { + cat "$output_path" >&2 || true + fail "unexpected exit status (expected=$expected_status actual=$actual_status)" + } +} + +host_arch() { + case "$(uname -m)" in + x86_64) printf 'amd64\n' ;; + aarch64) printf 'arm64\n' ;; + *) fail "unsupported test host architecture: $(uname -m)" ;; + esac +} + +env_value() { + local env_path="$1" + local key="$2" + ( + # shellcheck disable=SC1090 + source "$INSTALLER_SOURCE/scripts/common.sh" + env_file_value "$env_path" "$key" + ) +} + +create_fixture() { + local name="$1" + local fixture="$TEST_ROOT/$name" + local arch + arch="$(host_arch)" + + mkdir -p "$fixture" + cp -a "$INSTALLER_SOURCE"/. "$fixture"/ + rm -rf "$fixture/tests" + mkdir -p "$fixture/images" "$fixture/fake-bin" + sed -i \ + -e "s|__PACKAGE_ARCH__|$arch|g" \ + -e "s|__PACKAGE_PLATFORM__|linux/$arch|g" \ + -e "s|__RUNTIME_IMAGE__|ours-rp-runtime-$arch:new|g" \ + -e "s|__METRICS_IMAGE__|ours-rp-metrics-$arch:new|g" \ + -e "s|__HOST_DATA_DIR__|$fixture/data|g" \ + "$fixture/.env.example" + cat > "$fixture/PACKAGE-MANIFEST.env" < "$fixture/fake-bin/docker" <<'EOF' +#!/usr/bin/env bash +printf '%s\n' "$*" >> "$FAKE_DOCKER_LOG" +if [[ "${1:-}" == "compose" && "${2:-}" == "version" ]]; then + exit 0 +fi +exit 99 +EOF + chmod +x "$fixture/fake-bin/docker" + for command in jq rsync curl gzip tar; do + cat > "$fixture/fake-bin/$command" <<'EOF' +#!/usr/bin/env bash +exit 0 +EOF + chmod +x "$fixture/fake-bin/$command" + done + printf '%s\n' "$fixture" +} + +run_upgrade() { + local fixture="$1" + local old_env="$2" + shift 2 + ( + cd "$fixture" + export PATH="$fixture/fake-bin:$PATH" + export FAKE_DOCKER_LOG="$fixture/fake-docker.log" + "$@" ./scripts/upgrade.sh --reuse-env-from "$old_env" + ) +} + +run_upgrade_without_reuse() { + local fixture="$1" + ( + cd "$fixture" + export PATH="$fixture/fake-bin:$PATH" + export FAKE_DOCKER_LOG="$fixture/fake-docker.log" + ./scripts/upgrade.sh + ) +} + +new_fixture_with_old_env() { + local name="$1" + local fixture + fixture="$(create_fixture "$name")" + cat > "$fixture/old.env" <<'EOF' +INTERVAL_SECS=120 +RPKI_IMAGE=ours-rp-runtime-old:old +REMOVED_LEGACY_KEY=legacy-value +EOF + printf '%s\n' "$fixture" +} + +fixture="$(create_fixture template-defaults)" +expect_exit 99 "$fixture/template.log" run_upgrade_without_reuse "$fixture" +assert_equals "600" "$(env_value "$fixture/.env" INTERVAL_SECS)" "template default must be retained without reuse" + +fixture="$(new_fixture_with_old_env default-old)" +expect_exit 99 "$fixture/default.log" run_upgrade "$fixture" "$fixture/old.env" +assert_equals "120" "$(env_value "$fixture/.env" INTERVAL_SECS)" "old value must override template default" +assert_equals "ours-rp-runtime-$(host_arch):new" "$(env_value "$fixture/.env" RPKI_IMAGE)" "standard upgrade must retain package image" +assert_equals "10" "$(env_value "$fixture/.env" METRICS_POLL_SECS)" "template default must fill a key absent from old environment" +! grep -q '^REMOVED_LEGACY_KEY=' "$fixture/.env" || fail "removed old key was copied into target environment" +grep -q 'reused_keys=INTERVAL_SECS' "$fixture/default.log" || fail "reused key summary missing" +grep -q 'ignored_removed_old_keys=REMOVED_LEGACY_KEY' "$fixture/default.log" || fail "removed key summary missing" +grep -q 'template_only_keys=' "$fixture/default.log" || fail "template key summary missing" + +fixture="$(new_fixture_with_old_env invocation-overrides)" +expect_exit 99 "$fixture/override.log" run_upgrade "$fixture" "$fixture/old.env" env INTERVAL_SECS=300 GRAFANA_ADMIN_PASSWORD='secret with space' +assert_equals "300" "$(env_value "$fixture/.env" INTERVAL_SECS)" "invocation value must override old value" +assert_equals "secret with space" "$(env_value "$fixture/.env" GRAFANA_ADMIN_PASSWORD)" "invocation value must preserve shell-sensitive content" +grep -q 'command_override_keys=INTERVAL_SECS,GRAFANA_ADMIN_PASSWORD' "$fixture/override.log" || fail "override key summary missing" +assert_not_contains 'secret with space' "$fixture/override.log" + +fixture="$(new_fixture_with_old_env invocation-special-characters)" +special_password="O'Reilly" +special_password+=' $literal `tick` slash\' +expect_exit 99 "$fixture/special.log" run_upgrade "$fixture" "$fixture/old.env" env GRAFANA_ADMIN_PASSWORD="$special_password" +assert_equals "$special_password" "$(env_value "$fixture/.env" GRAFANA_ADMIN_PASSWORD)" "invocation value with dotenv-sensitive characters must round-trip" +assert_not_contains "$special_password" "$fixture/special.log" + +fixture="$(create_fixture existing-target-env)" +cp "$fixture/.env.example" "$fixture/.env" +sed -i 's/^INTERVAL_SECS=.*/INTERVAL_SECS=180/' "$fixture/.env" +cat > "$fixture/old.env" <<'EOF' +INTERVAL_SECS=120 +EOF +expect_exit 99 "$fixture/existing.log" run_upgrade "$fixture" "$fixture/old.env" env INTERVAL_SECS=300 +assert_equals "300" "$(env_value "$fixture/.env" INTERVAL_SECS)" "invocation value must override an existing target environment" + +fixture="$(new_fixture_with_old_env explicit-empty)" +expect_exit 1 "$fixture/empty.log" run_upgrade "$fixture" "$fixture/old.env" env INTERVAL_SECS= +[[ ! -s "$fixture/fake-docker.log" ]] || fail "explicit empty value reached Docker" +grep -q 'INTERVAL_SECS' "$fixture/empty.log" || fail "empty-value error did not identify key" + +fixture="$(create_fixture empty-old-value)" +cat > "$fixture/old.env" <<'EOF' +INTERVAL_SECS= +EOF +expect_exit 1 "$fixture/old-empty.log" run_upgrade "$fixture" "$fixture/old.env" +[[ ! -s "$fixture/fake-docker.log" ]] || fail "empty reused value reached Docker" +grep -q 'INTERVAL_SECS' "$fixture/old-empty.log" || fail "empty reused value error did not identify key" + +fixture="$(create_fixture empty-template)" +sed -i 's/^INTERVAL_SECS=.*/INTERVAL_SECS=/' "$fixture/.env.example" +cat > "$fixture/old.env" <<'EOF' +INTERVAL_SECS=120 +EOF +expect_exit 1 "$fixture/template-empty.log" run_upgrade "$fixture" "$fixture/old.env" +[[ ! -s "$fixture/fake-docker.log" ]] || fail "empty template default reached Docker" +grep -q 'INTERVAL_SECS' "$fixture/template-empty.log" || fail "empty template error did not identify key" + +fixture="$(new_fixture_with_old_env package-guard)" +arch="$(host_arch)" +other_arch="amd64" +[[ "$arch" == "amd64" ]] && other_arch="arm64" +expect_exit 1 "$fixture/arch.log" run_upgrade "$fixture" "$fixture/old.env" env PACKAGE_ARCH="$other_arch" +[[ ! -s "$fixture/fake-docker.log" ]] || fail "package architecture mismatch reached Docker" +grep -q 'mismatches manifest PACKAGE_ARCH' "$fixture/arch.log" || fail "package architecture guard did not reject override" + +printf 'PASS: Docker installer upgrade environment contract\n'