20 Commits

Author SHA1 Message Date
debe91fc8b [#50] 完成arm服务器单台机器上部署 server & client,没有使用docker swarm 2025-12-05 11:51:56 +08:00
a6e1bf7fe7 [#50] x86机器使用qemu模拟方式安装部署arm版本安装包,当前未精简版,只有简单的node exporter功能,删减日志,GPU指标功能,仍使用Swarm组网 2025-12-01 15:27:12 +08:00
4c45166b44 [#50] 完成在x86_64平台上使用buildx来构建arm镜像;重新构建了core/web/prom/grafana以及cpu node镜像;跑通swarm test for arm; 暂时移除dcgm和log路径功能 2025-11-24 16:31:39 +08:00
2caf0fa214 [#49] 优化swarm test支持自动reboot和verify 2025-11-20 15:21:18 +08:00
d4e0dc1511 [#49] swarm test重启测试通过 2025-11-19 17:26:26 +08:00
1d38304936 [#49] 移除bind/ftp,改成使用docker 自带域名解析,实现重启后IP变化仍正常访问,完成swarm test通过 2025-11-18 15:10:38 +08:00
5b617f62a8 [#43] 区分server&client 不同compose project;修改require compose监测 2025-11-17 12:17:11 +08:00
69e7a3e2b8 [#47] 增加cpu bundle镜像构建 2025-11-14 16:43:34 +08:00
b402fdf960 [#37] 删除旧部署构建目录;跑通src/sys/tests; 增加一键构建 server pkg和client gpu pkg 2025-11-14 15:07:24 +08:00
fff90826a4 [#41] 增加一键从源码构建 server_pkg和client_pkg 2025-11-13 15:02:33 +08:00
d0411e6b97 [#41] 优化gpu bundle构建优化 2025-11-13 10:28:37 +08:00
06131a268a [#40] log目录供宿主机其他程序可写 2025-11-12 15:06:37 +08:00
df1f519355 [#39] 增加新docker compose部署方式,验证通过gpu部署;剩余fluent-bit logs目录权限问题 以及 gpu bundle打包优化问题 2025-11-12 12:07:04 +08:00
6837d96035 [#39] 增加使用busybox warmup接入overlay网络 2025-11-10 12:17:10 +08:00
dac180f12b [#37] 修复dcgm exporter启动 2025-11-07 17:29:06 +08:00
1819fb9c46 [#37] 修复alert镜像用户 2025-11-07 12:21:41 +08:00
7548e46d1f [#37] 增加gpu bundle node镜像构建 2025-11-07 10:23:59 +08:00
0b9268332f [#37] 修复log时间戳测试问题 2025-11-06 17:20:40 +08:00
d1fad4a05a [#37] 增加sys/swarm_tests(cpu) ;单独构建的node bundle镜像 2025-11-06 16:43:14 +08:00
94b3e910b3 [#37] 增加部署时自动检测空闲端口;增加es 水位检测和临时应急处理 2025-11-05 16:21:34 +08:00
181 changed files with 11825 additions and 1553 deletions

150
build/README.md Normal file
View File

@@ -0,0 +1,150 @@
# ARGUS 统一构建脚本使用说明(build/build_images.sh)
本目录提供单一入口脚本 `build/build_images.sh`,覆盖常见三类场景:
- 系统集成测试(src/sys/tests)
- Swarm 系统集成测试(src/sys/swarm_tests)
- 构建离线安装包(deployment_new:Server/Client‑GPU)
文档还说明 UID/GID 取值规则、镜像 tag 策略、常用参数与重试机制。
## 环境前置
- Docker Engine ≥ 20.10(建议 ≥ 23.x/24.x)
- Docker Compose v2(`docker compose` 子命令)
- 可选:内网构建镜像源(`--intranet`)
## UID/GID 规则(用于容器内用户/卷属主)
- 非 pkg 构建(core/master/metric/web/alert/sys/gpu_bundle/cpu_bundle):
- 读取 `configs/build_user.local.conf` → `configs/build_user.conf`;
- 可被环境变量覆盖:`ARGUS_BUILD_UID`、`ARGUS_BUILD_GID`;
- pkg 构建(`--only server_pkg`、`--only client_pkg`):
- 读取 `configs/build_user.pkg.conf`(优先)→ `build_user.local.conf` → `build_user.conf`;
- 可被环境变量覆盖;
- CPU bundle 明确走“非 pkg”链(不读取 `build_user.pkg.conf`)。
- 说明:仅依赖 UID/GID 的 Docker 层会因参数变动而自动重建,不同构建剖面不会“打错包”。
## 镜像 tag 策略
- 非 pkg 构建:默认输出 `:latest`。
- `--only server_pkg`:所有镜像直接输出为 `:<VERSION>`(不覆盖 `:latest`)。
- `--only client_pkg`:GPU bundle 仅输出 `:<VERSION>`(不覆盖 `:latest`)。
- `--only cpu_bundle`:默认仅输出 `:<VERSION>`;可加 `--tag-latest` 同时打 `:latest` 以兼容 swarm_tests 默认 compose。
## 不加 --only 的默认构建目标
不指定 `--only` 时,脚本会构建“基础镜像集合”(不含 bundle 与安装包):
- core:`argus-elasticsearch:latest`、`argus-kibana:latest`、`argus-bind9:latest`
- master:`argus-master:latest`(非 offline)
- metric:`argus-metric-ftp:latest`、`argus-metric-prometheus:latest`、`argus-metric-grafana:latest`
- web:`argus-web-frontend:latest`、`argus-web-proxy:latest`
- alert:`argus-alertmanager:latest`
- sys:`argus-sys-node:latest`、`argus-sys-metric-test-node:latest`、`argus-sys-metric-test-gpu-node:latest`
说明:默认 tag 为 `:latest`;UID/GID 走“非 pkg”链(`build_user.local.conf → build_user.conf`,可被环境变量覆盖)。
## 通用参数
- `--intranet`:使用内网构建参数(各 Dockerfile 中按需启用)。
- `--no-cache`:禁用 Docker 层缓存。
- `--only <list>`:逗号分隔目标,例:`--only core,master,metric,web,alert`。
- `--version YYMMDD`:bundle/pkg 的日期标签(必填于 cpu_bundle/gpu_bundle/server_pkg/client_pkg)。
- `--client-semver X.Y.Z`:all‑in‑one‑full 客户端语义化版本(可选)。
- `--cuda VER`:GPU bundle CUDA 基镜版本(默认 12.2.2)。
- `--tag-latest`:CPU bundle 构建时同时打 `:latest`。
## 自动重试
- 构建单镜像失败会自动重试(默认 3 次,间隔 5s)。
- 最后一次自动使用 `DOCKER_BUILDKIT=0` 再试,缓解 “failed to receive status: context canceled”。
- 可调:`ARGUS_BUILD_RETRIES`、`ARGUS_BUILD_RETRY_DELAY` 环境变量。
---
## 场景一:系统集成测试(src/sys/tests)
构建用于系统级端到端测试的镜像(默认 `:latest`)。
示例:
```
# 构建核心与周边
./build/build_images.sh --only core,master,metric,web,alert,sys
```
产出:
- 本地镜像:`argus-elasticsearch:latest`、`argus-kibana:latest`、`argus-master:latest`、`argus-metric-ftp:latest`、`argus-metric-prometheus:latest`、`argus-metric-grafana:latest`、`argus-alertmanager:latest`、`argus-web-frontend:latest`、`argus-web-proxy:latest`、`argus-sys-node:latest` 等。
说明:
- UID/GID 读取 `build_user.local.conf → build_user.conf`(或环境变量覆盖)。
- sys/tests 的执行见 `src/sys/tests/README.md`。
---
## 场景二:Swarm 系统集成测试(src/sys/swarm_tests)
需要服务端镜像 + CPU 节点 bundle 镜像。
步骤:
1) 构建服务端镜像(默认 `:latest`)
```
./build/build_images.sh --only core,master,metric,web,alert
```
2) 构建 CPU bundle(直接 FROM ubuntu:22.04)
```
# 仅版本 tag 输出
./build/build_images.sh --only cpu_bundle --version 20251114
# 若要兼容 swarm_tests 默认 latest:
./build/build_images.sh --only cpu_bundle --version 20251114 --tag-latest
```
3) 运行 Swarm 测试
```
cd src/sys/swarm_tests
# 如未打 latest,可先指定:
export NODE_BUNDLE_IMAGE_TAG=argus-sys-metric-test-node-bundle:20251114
./scripts/01_server_up.sh
./scripts/02_wait_ready.sh
./scripts/03_nodes_up.sh
./scripts/04_metric_verify.sh # 验证 Prometheus/Grafana/nodes.json 与日志通路
./scripts/99_down.sh # 结束
```
产出:
- 本地镜像:`argus-*:latest` 与 `argus-sys-metric-test-node-bundle:20251114`(或 latest)。
- `swarm_tests/private-*`:运行态持久化文件。
说明:
- CPU bundle 构建用户走“非 pkg”链(local.conf → conf)。
- `04_metric_verify.sh` 已内置 Fluent Bit 启动与配置修正逻辑,偶发未就绪可重跑一次即通过。
---
## 场景三:构建离线安装包(deployment_new)
Server 与 Client‑GPU 安装包均采用“版本直出”,只输出 `:<VERSION>` 标签,不改动 `:latest`。
1) Server 包
```
./build/build_images.sh --only server_pkg --version 20251114
```
产出:
- 本地镜像:`argus-<模块>:20251114`(不触碰 latest)。
- 安装包:`deployment_new/artifact/server/20251114/` 与 `server_20251114.tar.gz`
- 包内包含:逐镜像 tar.gz、compose/.env.example、scripts(config/install/selfcheck/diagnose 等)、docs、manifest/checksums。
2) Client‑GPU 包
```
# 同步构建 GPU bundle(仅 :<VERSION>,不触碰 latest),并生成客户端包
./build/build_images.sh --only client_pkg --version 20251114 \\
--client-semver 1.44.0 --cuda 12.2.2
```
产出:
- 本地镜像:`argus-sys-metric-test-node-bundle-gpu:20251114`
- 安装包:`deployment_new/artifact/client_gpu/20251114/` 与 `client_gpu_20251114.tar.gz`
- 包内包含:GPU bundle 镜像 tar.gz、busybox.tar、compose/.env.example、scripts(config/install/uninstall)、docs、manifest/checksums。
说明:
- pkg 构建使用 `configs/build_user.pkg.conf` 的 UID/GID(可被环境覆盖)。
- 包内 `.env.example` 的 `PKG_VERSION=<VERSION>` 与镜像 tag 严格一致。
---
## 常见问题(FAQ)
- 构建报 `failed to receive status: context canceled`?
- 已内置单镜像多次重试,最后一次禁用 BuildKit;建议加 `--intranet` 与 `--no-cache` 重试,或 `docker builder prune -f` 后再试。
- 先跑非 pkg(latest),再跑 pkg(version)会不会“打错包”?
- 不会。涉及 UID/GID 的层因参数变化会重建,其它层按缓存命中复用,最终 pkg 产物的属主与运行账户按 `build_user.pkg.conf` 生效。
- swarm_tests 默认拉取 `:latest`,我只构建了 `:<VERSION>` 的 CPU bundle 怎么办?
- 在运行前 `export NODE_BUNDLE_IMAGE_TAG=argus-sys-metric-test-node-bundle:<VERSION>`,或在构建时加 `--tag-latest`。
---
如需进一步自动化(例如生成 BUILD_SUMMARY.txt 汇总镜像 digest 与构建参数),可在 pkg 产出阶段追加,我可以按需补齐。

View File

@@ -12,7 +12,11 @@ Options:
--master-offline Build master offline image (requires src/master/offline_wheels.tar.gz) --master-offline Build master offline image (requires src/master/offline_wheels.tar.gz)
--metric Build metric module images (ftp, prometheus, grafana, test nodes) --metric Build metric module images (ftp, prometheus, grafana, test nodes)
--no-cache Build all images without using Docker layer cache --no-cache Build all images without using Docker layer cache
--only LIST Comma-separated targets to build: core,master,metric,web,alert,sys,all --only LIST Comma-separated targets to build: core,master,metric,web,alert,sys,gpu_bundle,cpu_bundle,server_pkg,client_pkg,all
--version DATE Date tag used by gpu_bundle/server_pkg/client_pkg (e.g. 20251112)
--client-semver X.Y.Z Override client semver used in all-in-one-full artifact (optional)
--cuda VER CUDA runtime version for NVIDIA base (default: 12.2.2)
--tag-latest Also tag bundle image as :latest (for cpu_bundle only; default off)
-h, --help Show this help message -h, --help Show this help message
Examples: Examples:
@@ -32,8 +36,18 @@ build_metric=true
build_web=true build_web=true
build_alert=true build_alert=true
build_sys=true build_sys=true
build_gpu_bundle=false
build_cpu_bundle=false
build_server_pkg=false
build_client_pkg=false
no_cache=false no_cache=false
bundle_date=""
client_semver=""
cuda_ver="12.2.2"
DEFAULT_IMAGE_TAG="latest"
tag_latest=false
while [[ $# -gt 0 ]]; do while [[ $# -gt 0 ]]; do
case $1 in case $1 in
--intranet) --intranet)
@@ -63,7 +77,7 @@ while [[ $# -gt 0 ]]; do
fi fi
sel="$2"; shift 2 sel="$2"; shift 2
# reset all, then enable selected # reset all, then enable selected
build_core=false; build_master=false; build_metric=false; build_web=false; build_alert=false; build_sys=false build_core=false; build_master=false; build_metric=false; build_web=false; build_alert=false; build_sys=false; build_gpu_bundle=false; build_cpu_bundle=false; build_server_pkg=false; build_client_pkg=false
IFS=',' read -ra parts <<< "$sel" IFS=',' read -ra parts <<< "$sel"
for p in "${parts[@]}"; do for p in "${parts[@]}"; do
case "$p" in case "$p" in
@@ -73,11 +87,31 @@ while [[ $# -gt 0 ]]; do
web) build_web=true ;; web) build_web=true ;;
alert) build_alert=true ;; alert) build_alert=true ;;
sys) build_sys=true ;; sys) build_sys=true ;;
gpu_bundle) build_gpu_bundle=true ;;
cpu_bundle) build_cpu_bundle=true ;;
server_pkg) build_server_pkg=true; build_core=true; build_master=true; build_metric=true; build_web=true; build_alert=true ;;
client_pkg) build_client_pkg=true ;;
all) build_core=true; build_master=true; build_metric=true; build_web=true; build_alert=true; build_sys=true ;; all) build_core=true; build_master=true; build_metric=true; build_web=true; build_alert=true; build_sys=true ;;
*) echo "Unknown --only target: $p" >&2; exit 1 ;; *) echo "Unknown --only target: $p" >&2; exit 1 ;;
esac esac
done done
;; ;;
--version)
if [[ -z ${2:-} ]]; then echo "--version requires a value like 20251112" >&2; exit 1; fi
bundle_date="$2"; shift 2
;;
--client-semver)
if [[ -z ${2:-} ]]; then echo "--client-semver requires a value like 1.43.0" >&2; exit 1; fi
client_semver="$2"; shift 2
;;
--cuda)
if [[ -z ${2:-} ]]; then echo "--cuda requires a value like 12.2.2" >&2; exit 1; fi
cuda_ver="$2"; shift 2
;;
--tag-latest)
tag_latest=true
shift
;;
-h|--help) -h|--help)
show_help show_help
exit 0 exit 0
@@ -101,6 +135,16 @@ fi
cd "$root" cd "$root"
# Set default image tag policy before building
if [[ "$build_server_pkg" == true ]]; then
DEFAULT_IMAGE_TAG="${bundle_date:-latest}"
fi
# Select build user profile for pkg vs default
if [[ "$build_server_pkg" == true || "$build_client_pkg" == true ]]; then
export ARGUS_BUILD_PROFILE=pkg
fi
load_build_user load_build_user
build_args+=("--build-arg" "ARGUS_BUILD_UID=${ARGUS_BUILD_UID}" "--build-arg" "ARGUS_BUILD_GID=${ARGUS_BUILD_GID}") build_args+=("--build-arg" "ARGUS_BUILD_UID=${ARGUS_BUILD_UID}" "--build-arg" "ARGUS_BUILD_GID=${ARGUS_BUILD_GID}")
@@ -163,13 +207,31 @@ build_image() {
echo " Tag: $tag" echo " Tag: $tag"
echo " Context: $context" echo " Context: $context"
if docker build "${build_args[@]}" "${extra_args[@]}" -f "$dockerfile_path" -t "$tag" "$context"; then local tries=${ARGUS_BUILD_RETRIES:-3}
local delay=${ARGUS_BUILD_RETRY_DELAY:-5}
local attempt=1
while (( attempt <= tries )); do
local prefix=""
if (( attempt == tries )); then
# final attempt: disable BuildKit to avoid docker/dockerfile front-end pulls
prefix="DOCKER_BUILDKIT=0"
echo " Attempt ${attempt}/${tries} (fallback: DOCKER_BUILDKIT=0)"
else
echo " Attempt ${attempt}/${tries}"
fi
if eval $prefix docker build "${build_args[@]}" "${extra_args[@]}" -f "$dockerfile_path" -t "$tag" "$context"; then
echo "✅ $image_name image built successfully" echo "✅ $image_name image built successfully"
return 0 return 0
else
echo "❌ Failed to build $image_name image"
return 1
fi fi
echo "⚠️ Build failed for $image_name (attempt ${attempt}/${tries})."
if (( attempt < tries )); then
echo " Retrying in ${delay}s..."
sleep "$delay"
fi
attempt=$((attempt+1))
done
echo "❌ Failed to build $image_name image after ${tries} attempts"
return 1
} }
pull_base_image() { pull_base_image() {
@@ -203,25 +265,381 @@ pull_base_image() {
images_built=() images_built=()
build_failed=false build_failed=false
build_gpu_bundle_image() {
local date_tag="$1" # e.g. 20251112
local cuda_ver_local="$2" # e.g. 12.2.2
local client_ver="$3" # semver like 1.43.0
if [[ -z "$date_tag" ]]; then
echo "❌ gpu_bundle requires --version YYMMDD (e.g. 20251112)" >&2
return 1
fi
# sanitize cuda version (trim trailing dots like '12.2.')
while [[ "$cuda_ver_local" == *"." ]]; do cuda_ver_local="${cuda_ver_local%.}"; done
# Resolve effective CUDA base tag
local resolve_cuda_base_tag
resolve_cuda_base_tag() {
local want="$1" # can be 12, 12.2 or 12.2.2
local major minor patch
if [[ "$want" =~ ^([0-9]+)\.([0-9]+)\.([0-9]+)$ ]]; then
major="${BASH_REMATCH[1]}"; minor="${BASH_REMATCH[2]}"; patch="${BASH_REMATCH[3]}"
echo "nvidia/cuda:${major}.${minor}.${patch}-runtime-ubuntu22.04"; return 0
elif [[ "$want" =~ ^([0-9]+)\.([0-9]+)$ ]]; then
major="${BASH_REMATCH[1]}"; minor="${BASH_REMATCH[2]}"
# try to find best local patch for major.minor
local best
best=$(docker images --format '{{.Repository}}:{{.Tag}}' nvidia/cuda 2>/dev/null | \
grep -E "^nvidia/cuda:${major}\.${minor}\\.[0-9]+-runtime-ubuntu22\.04$" | \
sed -E 's#^nvidia/cuda:([0-9]+\.[0-9]+\.)([0-9]+)-runtime-ubuntu22\.04$#\1\2#g' | \
sort -V | tail -n1 || true)
if [[ -n "$best" ]]; then
echo "nvidia/cuda:${best}-runtime-ubuntu22.04"; return 0
fi
# fallback patch if none local
echo "nvidia/cuda:${major}.${minor}.2-runtime-ubuntu22.04"; return 0
elif [[ "$want" =~ ^([0-9]+)$ ]]; then
major="${BASH_REMATCH[1]}"
# try to find best local for this major
local best
best=$(docker images --format '{{.Repository}}:{{.Tag}}' nvidia/cuda 2>/dev/null | \
grep -E "^nvidia/cuda:${major}\\.[0-9]+\\.[0-9]+-runtime-ubuntu22\.04$" | \
sed -E 's#^nvidia/cuda:([0-9]+\.[0-9]+\.[0-9]+)-runtime-ubuntu22\.04$#\1#g' | \
sort -V | tail -n1 || true)
if [[ -n "$best" ]]; then
echo "nvidia/cuda:${best}-runtime-ubuntu22.04"; return 0
fi
echo "nvidia/cuda:${major}.2.2-runtime-ubuntu22.04"; return 0
else
# invalid format, fallback to default
echo "nvidia/cuda:12.2.2-runtime-ubuntu22.04"; return 0
fi
}
local base_image
base_image=$(resolve_cuda_base_tag "$cuda_ver_local")
echo
echo "🔧 Preparing one-click GPU bundle build"
echo " CUDA runtime base: ${base_image}"
echo " Bundle tag : ${date_tag}"
# 1) Ensure NVIDIA base image (skip pull if local)
if ! pull_base_image "$base_image"; then
# try once more with default if resolution failed
if ! pull_base_image "nvidia/cuda:12.2.2-runtime-ubuntu22.04"; then
return 1
else
base_image="nvidia/cuda:12.2.2-runtime-ubuntu22.04"
fi
fi
# 2) Build latest argus-agent from source
echo "\n🛠 Building argus-agent from src/agent"
pushd "$root/src/agent" >/dev/null
if ! bash scripts/build_binary.sh; then
echo "❌ argus-agent build failed" >&2
popd >/dev/null
return 1
fi
if [[ ! -f "dist/argus-agent" ]]; then
echo "❌ argus-agent binary missing after build" >&2
popd >/dev/null
return 1
fi
popd >/dev/null
# 3) Inject agent into all-in-one-full plugin and package artifact
local aio_root="$root/src/metric/client-plugins/all-in-one-full"
local agent_bin_src="$root/src/agent/dist/argus-agent"
local agent_bin_dst="$aio_root/plugins/argus-agent/bin/argus-agent"
echo "\n📦 Updating all-in-one-full agent binary → $agent_bin_dst"
cp -f "$agent_bin_src" "$agent_bin_dst"
chmod +x "$agent_bin_dst" || true
pushd "$aio_root" >/dev/null
local prev_version
prev_version="$(cat config/VERSION 2>/dev/null || echo "1.0.0")"
local use_version="$prev_version"
if [[ -n "$client_semver" ]]; then
echo "${client_semver}" > config/VERSION
use_version="$client_semver"
fi
echo " Packaging all-in-one-full artifact version: $use_version"
if ! bash scripts/package_artifact.sh --force; then
echo "❌ package_artifact.sh failed" >&2
# restore VERSION if changed
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
return 1
fi
local artifact_dir="$aio_root/artifact/$use_version"
local artifact_tar
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
if [[ -z "$artifact_tar" ]]; then
echo " No argus-metric_*.tar.gz found; invoking publish_artifact.sh to assemble..."
local owner="$(id -u):$(id -g)"
if ! bash scripts/publish_artifact.sh "$use_version" --output-dir "$artifact_dir" --owner "$owner"; then
echo "❌ publish_artifact.sh failed" >&2
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
return 1
fi
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
fi
if [[ -z "$artifact_tar" ]]; then
echo "❌ artifact tar not found under $artifact_dir" >&2
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
return 1
fi
# restore VERSION if changed (keep filesystem clean)
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
# 4) Stage docker build context
local bundle_ctx="$root/src/bundle/gpu-node-bundle/.build-$date_tag"
echo "\n🧰 Staging docker build context: $bundle_ctx"
rm -rf "$bundle_ctx"
mkdir -p "$bundle_ctx/bundle" "$bundle_ctx/private"
cp "$root/src/bundle/gpu-node-bundle/Dockerfile" "$bundle_ctx/"
cp "$root/src/bundle/gpu-node-bundle/node-bootstrap.sh" "$bundle_ctx/"
cp "$root/src/bundle/gpu-node-bundle/health-watcher.sh" "$bundle_ctx/"
# bundle tar
cp "$artifact_tar" "$bundle_ctx/bundle/"
# offline fluent-bit assets (optional but useful)
if [[ -d "$root/src/log/fluent-bit/build/etc" ]]; then
cp -r "$root/src/log/fluent-bit/build/etc" "$bundle_ctx/private/"
fi
if [[ -d "$root/src/log/fluent-bit/build/packages" ]]; then
cp -r "$root/src/log/fluent-bit/build/packages" "$bundle_ctx/private/"
fi
if [[ -f "$root/src/log/fluent-bit/build/start-fluent-bit.sh" ]]; then
cp "$root/src/log/fluent-bit/build/start-fluent-bit.sh" "$bundle_ctx/private/"
fi
# 5) Build the final bundle image (directly from NVIDIA base)
local image_tag="argus-sys-metric-test-node-bundle-gpu:${date_tag}"
echo "\n🔄 Building GPU Bundle image"
if build_image "GPU Bundle" "$bundle_ctx/Dockerfile" "$image_tag" "$bundle_ctx" \
--build-arg CUDA_VER="$(echo "$base_image" | sed -E 's#^nvidia/cuda:([0-9]+\.[0-9]+\.[0-9]+)-runtime-ubuntu22\.04$#\1#')" \
--build-arg CLIENT_VER="$use_version" \
--build-arg BUNDLE_DATE="$date_tag"; then
images_built+=("$image_tag")
# In non-pkg mode, also tag latest for convenience
if [[ "${ARGUS_PKG_BUILD:-0}" != "1" ]]; then
docker tag "$image_tag" argus-sys-metric-test-node-bundle-gpu:latest >/dev/null 2>&1 || true
fi
return 0
else
return 1
fi
}
# Tag helper: ensure :<date_tag> exists for a list of repos
ensure_version_tags() {
local date_tag="$1"; shift
local repos=("$@")
for repo in "${repos[@]}"; do
if docker image inspect "$repo:$date_tag" >/dev/null 2>&1; then
:
elif docker image inspect "$repo:latest" >/dev/null 2>&1; then
docker tag "$repo:latest" "$repo:$date_tag" || true
else
echo "❌ missing image for tagging: $repo (need :latest or :$date_tag)" >&2
return 1
fi
done
return 0
}
# Build server package after images are built
build_server_pkg_bundle() {
local date_tag="$1"
if [[ -z "$date_tag" ]]; then
echo "❌ server_pkg requires --version YYMMDD" >&2
return 1
fi
local repos=(
argus-bind9 argus-master argus-elasticsearch argus-kibana \
argus-metric-ftp argus-metric-prometheus argus-metric-grafana \
argus-alertmanager argus-web-frontend argus-web-proxy
)
echo "\n🔖 Verifying server images with :$date_tag and collecting digests"
for repo in "${repos[@]}"; do
if ! docker image inspect "$repo:$date_tag" >/dev/null 2>&1; then
echo "❌ required image missing: $repo:$date_tag (build phase should have produced it)" >&2
return 1
fi
done
# Optional: show digests
for repo in "${repos[@]}"; do
local digest
digest=$(docker images --digests --format '{{.Repository}}:{{.Tag}} {{.Digest}}' | awk -v r="$repo:$date_tag" '$1==r{print $2}' | head -n1)
printf ' • %s@%s\n' "$repo:$date_tag" "${digest:-<none>}"
done
echo "\n📦 Building server package via deployment_new/build/make_server_package.sh --version $date_tag"
if ! "$root/deployment_new/build/make_server_package.sh" --version "$date_tag"; then
echo "❌ make_server_package.sh failed" >&2
return 1
fi
return 0
}
# Build client package: ensure gpu bundle image exists, then package client_gpu
build_client_pkg_bundle() {
local date_tag="$1"
local semver="$2"
local cuda="$3"
if [[ -z "$date_tag" ]]; then
echo "❌ client_pkg requires --version YYMMDD" >&2
return 1
fi
local bundle_tag="argus-sys-metric-test-node-bundle-gpu:${date_tag}"
if ! docker image inspect "$bundle_tag" >/dev/null 2>&1; then
echo "\n🧩 GPU bundle image $bundle_tag missing; building it first..."
ARGUS_PKG_BUILD=1
export ARGUS_PKG_BUILD
if ! build_gpu_bundle_image "$date_tag" "$cuda" "$semver"; then
return 1
fi
else
echo "\n✅ Using existing GPU bundle image: $bundle_tag"
fi
echo "\n📦 Building client GPU package via deployment_new/build/make_client_gpu_package.sh --version $date_tag --image $bundle_tag"
if ! "$root/deployment_new/build/make_client_gpu_package.sh" --version "$date_tag" --image "$bundle_tag"; then
echo "❌ make_client_gpu_package.sh failed" >&2
return 1
fi
return 0
}
# Build CPU bundle image directly FROM ubuntu:22.04 (no intermediate base)
build_cpu_bundle_image() {
local date_tag="$1" # e.g. 20251113
local client_ver_in="$2" # semver like 1.43.0 (optional)
local want_tag_latest="$3" # true/false
if [[ -z "$date_tag" ]]; then
echo "❌ cpu_bundle requires --version YYMMDD" >&2
return 1
fi
echo "\n🔧 Preparing one-click CPU bundle build"
echo " Base: ubuntu:22.04"
echo " Bundle tag: ${date_tag}"
# 1) Build latest argus-agent from source
echo "\n🛠 Building argus-agent from src/agent"
pushd "$root/src/agent" >/dev/null
if ! bash scripts/build_binary.sh; then
echo "❌ argus-agent build failed" >&2
popd >/dev/null
return 1
fi
if [[ ! -f "dist/argus-agent" ]]; then
echo "❌ argus-agent binary missing after build" >&2
popd >/dev/null
return 1
fi
popd >/dev/null
# 2) Inject agent into all-in-one-full plugin and package artifact
local aio_root="$root/src/metric/client-plugins/all-in-one-full"
local agent_bin_src="$root/src/agent/dist/argus-agent"
local agent_bin_dst="$aio_root/plugins/argus-agent/bin/argus-agent"
echo "\n📦 Updating all-in-one-full agent binary → $agent_bin_dst"
cp -f "$agent_bin_src" "$agent_bin_dst"
chmod +x "$agent_bin_dst" || true
pushd "$aio_root" >/dev/null
local prev_version use_version
prev_version="$(cat config/VERSION 2>/dev/null || echo "1.0.0")"
use_version="$prev_version"
if [[ -n "$client_ver_in" ]]; then
echo "$client_ver_in" > config/VERSION
use_version="$client_ver_in"
fi
echo " Packaging all-in-one-full artifact: version=$use_version"
if ! bash scripts/package_artifact.sh --force; then
echo "❌ package_artifact.sh failed" >&2
[[ -n "$client_ver_in" ]] && echo "$prev_version" > config/VERSION
popd >/dev/null
return 1
fi
local artifact_dir="$aio_root/artifact/$use_version"
local artifact_tar
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
if [[ -z "$artifact_tar" ]]; then
echo " No argus-metric_*.tar.gz found; invoking publish_artifact.sh ..."
local owner="$(id -u):$(id -g)"
if ! bash scripts/publish_artifact.sh "$use_version" --output-dir "$artifact_dir" --owner "$owner"; then
echo "❌ publish_artifact.sh failed" >&2
[[ -n "$client_ver_in" ]] && echo "$prev_version" > config/VERSION
popd >/dev/null
return 1
fi
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
fi
[[ -n "$client_ver_in" ]] && echo "$prev_version" > config/VERSION
popd >/dev/null
# 3) Stage docker build context
local bundle_ctx="$root/src/bundle/cpu-node-bundle/.build-$date_tag"
echo "\n🧰 Staging docker build context: $bundle_ctx"
rm -rf "$bundle_ctx"
mkdir -p "$bundle_ctx/bundle" "$bundle_ctx/private"
cp "$root/src/bundle/cpu-node-bundle/Dockerfile" "$bundle_ctx/"
cp "$root/src/bundle/cpu-node-bundle/node-bootstrap.sh" "$bundle_ctx/"
cp "$root/src/bundle/cpu-node-bundle/health-watcher.sh" "$bundle_ctx/"
# bundle tar
cp "$artifact_tar" "$bundle_ctx/bundle/"
# offline fluent-bit assets
if [[ -d "$root/src/log/fluent-bit/build/etc" ]]; then
cp -r "$root/src/log/fluent-bit/build/etc" "$bundle_ctx/private/"
fi
if [[ -d "$root/src/log/fluent-bit/build/packages" ]]; then
cp -r "$root/src/log/fluent-bit/build/packages" "$bundle_ctx/private/"
fi
if [[ -f "$root/src/log/fluent-bit/build/start-fluent-bit.sh" ]]; then
cp "$root/src/log/fluent-bit/build/start-fluent-bit.sh" "$bundle_ctx/private/"
fi
# 4) Build final bundle image
local image_tag="argus-sys-metric-test-node-bundle:${date_tag}"
echo "\n🔄 Building CPU Bundle image"
if build_image "CPU Bundle" "$bundle_ctx/Dockerfile" "$image_tag" "$bundle_ctx"; then
images_built+=("$image_tag")
if [[ "$want_tag_latest" == "true" ]]; then
docker tag "$image_tag" argus-sys-metric-test-node-bundle:latest >/dev/null 2>&1 || true
fi
return 0
else
return 1
fi
}
if [[ "$build_core" == true ]]; then if [[ "$build_core" == true ]]; then
if build_image "Elasticsearch" "src/log/elasticsearch/build/Dockerfile" "argus-elasticsearch:latest"; then if build_image "Elasticsearch" "src/log/elasticsearch/build/Dockerfile" "argus-elasticsearch:${DEFAULT_IMAGE_TAG}"; then
images_built+=("argus-elasticsearch:latest") images_built+=("argus-elasticsearch:${DEFAULT_IMAGE_TAG}")
else else
build_failed=true build_failed=true
fi fi
echo "" echo ""
if build_image "Kibana" "src/log/kibana/build/Dockerfile" "argus-kibana:latest"; then if build_image "Kibana" "src/log/kibana/build/Dockerfile" "argus-kibana:${DEFAULT_IMAGE_TAG}"; then
images_built+=("argus-kibana:latest") images_built+=("argus-kibana:${DEFAULT_IMAGE_TAG}")
else else
build_failed=true build_failed=true
fi fi
echo "" echo ""
if build_image "BIND9" "src/bind/build/Dockerfile" "argus-bind9:latest"; then if build_image "BIND9" "src/bind/build/Dockerfile" "argus-bind9:${DEFAULT_IMAGE_TAG}"; then
images_built+=("argus-bind9:latest") images_built+=("argus-bind9:${DEFAULT_IMAGE_TAG}")
else else
build_failed=true build_failed=true
fi fi
@@ -233,7 +651,7 @@ if [[ "$build_master" == true ]]; then
echo "" echo ""
echo "🔄 Building Master image..." echo "🔄 Building Master image..."
pushd "$master_root" >/dev/null pushd "$master_root" >/dev/null
master_args=("--tag" "argus-master:latest") master_args=("--tag" "argus-master:${DEFAULT_IMAGE_TAG}")
if [[ "$use_intranet" == true ]]; then if [[ "$use_intranet" == true ]]; then
master_args+=("--intranet") master_args+=("--intranet")
fi fi
@@ -247,7 +665,7 @@ if [[ "$build_master" == true ]]; then
if [[ "$build_master_offline" == true ]]; then if [[ "$build_master_offline" == true ]]; then
images_built+=("argus-master:offline") images_built+=("argus-master:offline")
else else
images_built+=("argus-master:latest") images_built+=("argus-master:${DEFAULT_IMAGE_TAG}")
fi fi
else else
build_failed=true build_failed=true
@@ -272,9 +690,9 @@ if [[ "$build_metric" == true ]]; then
done done
metric_builds=( metric_builds=(
"Metric FTP|src/metric/ftp/build/Dockerfile|argus-metric-ftp:latest|src/metric/ftp/build" "Metric FTP|src/metric/ftp/build/Dockerfile|argus-metric-ftp:${DEFAULT_IMAGE_TAG}|src/metric/ftp/build"
"Metric Prometheus|src/metric/prometheus/build/Dockerfile|argus-metric-prometheus:latest|src/metric/prometheus/build" "Metric Prometheus|src/metric/prometheus/build/Dockerfile|argus-metric-prometheus:${DEFAULT_IMAGE_TAG}|src/metric/prometheus/build"
"Metric Grafana|src/metric/grafana/build/Dockerfile|argus-metric-grafana:latest|src/metric/grafana/build" "Metric Grafana|src/metric/grafana/build/Dockerfile|argus-metric-grafana:${DEFAULT_IMAGE_TAG}|src/metric/grafana/build"
) )
for build_spec in "${metric_builds[@]}"; do for build_spec in "${metric_builds[@]}"; do
@@ -346,8 +764,8 @@ if [[ "$build_web" == true || "$build_alert" == true ]]; then
if [[ "$build_web" == true ]]; then if [[ "$build_web" == true ]]; then
web_builds=( web_builds=(
"Web Frontend|src/web/build_tools/frontend/Dockerfile|argus-web-frontend:latest|." "Web Frontend|src/web/build_tools/frontend/Dockerfile|argus-web-frontend:${DEFAULT_IMAGE_TAG}|."
"Web Proxy|src/web/build_tools/proxy/Dockerfile|argus-web-proxy:latest|." "Web Proxy|src/web/build_tools/proxy/Dockerfile|argus-web-proxy:${DEFAULT_IMAGE_TAG}|."
) )
for build_spec in "${web_builds[@]}"; do for build_spec in "${web_builds[@]}"; do
IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec" IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec"
@@ -362,7 +780,7 @@ if [[ "$build_web" == true || "$build_alert" == true ]]; then
if [[ "$build_alert" == true ]]; then if [[ "$build_alert" == true ]]; then
alert_builds=( alert_builds=(
"Alertmanager|src/alert/alertmanager/build/Dockerfile|argus-alertmanager:latest|." "Alertmanager|src/alert/alertmanager/build/Dockerfile|argus-alertmanager:${DEFAULT_IMAGE_TAG}|."
) )
for build_spec in "${alert_builds[@]}"; do for build_spec in "${alert_builds[@]}"; do
IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec" IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec"
@@ -376,6 +794,49 @@ if [[ "$build_web" == true || "$build_alert" == true ]]; then
fi fi
fi fi
# =======================================
# One-click GPU bundle (direct NVIDIA base)
# =======================================
if [[ "$build_gpu_bundle" == true ]]; then
echo ""
echo "Building one-click GPU bundle image..."
if ! build_gpu_bundle_image "$bundle_date" "$cuda_ver" "$client_semver"; then
build_failed=true
fi
fi
# =======================================
# One-click CPU bundle (from ubuntu:22.04)
# =======================================
if [[ "$build_cpu_bundle" == true ]]; then
echo ""
echo "Building one-click CPU bundle image..."
if ! build_cpu_bundle_image "${bundle_date}" "${client_semver}" "${tag_latest}"; then
build_failed=true
fi
fi
# =======================================
# One-click Server/Client packaging
# =======================================
if [[ "$build_server_pkg" == true ]]; then
echo ""
echo "🧳 Building one-click Server package..."
if ! build_server_pkg_bundle "${bundle_date}"; then
build_failed=true
fi
fi
if [[ "$build_client_pkg" == true ]]; then
echo ""
echo "🧳 Building one-click Client-GPU package..."
if ! build_client_pkg_bundle "${bundle_date}" "${client_semver}" "${cuda_ver}"; then
build_failed=true
fi
fi
echo "=======================================" echo "======================================="
echo "📦 Build Summary" echo "📦 Build Summary"
echo "=======================================" echo "======================================="

979
build/build_images_for_arm.sh Executable file
View File

@@ -0,0 +1,979 @@
#!/usr/bin/env bash
set -euo pipefail
export ARGUS_TARGET_ARCH="arm64"
ARGUS_BUILDX_BUILDER="${ARGUS_BUILDX_BUILDER:-mybuilder}"
# 自动加载 HTTP/HTTPS 代理配置(仅在变量未预先设置时)
if [[ -z "${HTTP_PROXY:-}" && -z "${http_proxy:-}" ]]; then
if [[ -f /home/yuyr/.source_http_proxy.sh ]]; then
# shellcheck disable=SC1090
source /home/yuyr/.source_http_proxy.sh || true
fi
fi
# 自动准备并切换到指定的 buildx builder(用于 x86_64 上构建 ARM 镜像)
if command -v docker >/dev/null 2>&1; then
if docker buildx ls >/dev/null 2>&1; then
# 若指定的 builder 不存在,则自动创建(带代理环境变量)
if ! docker buildx ls | awk '{print $1}' | grep -qx "${ARGUS_BUILDX_BUILDER}"; then
echo "🔧 Creating buildx builder '${ARGUS_BUILDX_BUILDER}' for ARM builds..."
create_args=(create --name "${ARGUS_BUILDX_BUILDER}" --driver docker-container)
if [[ -n "${HTTP_PROXY:-}" ]]; then
create_args+=(--driver-opt "env.HTTP_PROXY=${HTTP_PROXY}" --driver-opt "env.http_proxy=${HTTP_PROXY}")
fi
if [[ -n "${HTTPS_PROXY:-}" ]]; then
create_args+=(--driver-opt "env.HTTPS_PROXY=${HTTPS_PROXY}" --driver-opt "env.https_proxy=${HTTPS_PROXY}")
fi
if [[ -n "${NO_PROXY:-}" ]]; then
create_args+=(--driver-opt "env.NO_PROXY=${NO_PROXY}" --driver-opt "env.no_proxy=${NO_PROXY}")
fi
docker buildx "${create_args[@]}" --bootstrap >/dev/null 2>&1 || true
fi
docker buildx use "${ARGUS_BUILDX_BUILDER}" >/dev/null 2>&1 || true
fi
fi
show_help() {
cat <<'EOF'
ARGUS Unified Build System - Image Build Tool
Usage: $0 [OPTIONS]
Options:
--intranet Use intranet mirror for log/bind builds
--master-offline Build master offline image (requires src/master/offline_wheels.tar.gz)
--metric Build metric module images (ftp, prometheus, grafana, test nodes)
--no-cache Build all images without using Docker layer cache
--only LIST Comma-separated targets to build: core,master,metric,web,alert,sys,gpu_bundle,cpu_bundle,server_pkg,client_pkg,arm_server_pkg,arm_client_pkg,all
--version DATE Date tag used by gpu_bundle/server_pkg/client_pkg/arm_server_pkg/arm_client_pkg (e.g. 20251112)
--client-semver X.Y.Z Override client semver used in all-in-one-full artifact (optional)
--cuda VER CUDA runtime version for NVIDIA base (default: 12.2.2)
--tag-latest Also tag bundle image as :latest (for cpu_bundle only; default off)
-h, --help Show this help message
Examples:
$0 # Build with default sources
$0 --intranet # Build with intranet mirror
$0 --master-offline # Additionally build argus-master:offline
$0 --metric # Additionally build metric module images
$0 --intranet --master-offline --metric
EOF
}
use_intranet=false
build_core=true
build_master=true
build_master_offline=false
build_metric=true
build_web=true
build_alert=true
build_sys=true
build_gpu_bundle=false
build_cpu_bundle=false
build_server_pkg=false
build_client_pkg=false
build_server_pkg_arm=false
build_client_pkg_arm=false
no_cache=false
bundle_date=""
client_semver=""
cuda_ver="12.2.2"
DEFAULT_IMAGE_TAG="latest"
tag_latest=false
while [[ $# -gt 0 ]]; do
case $1 in
--intranet)
use_intranet=true
shift
;;
--master)
build_master=true
shift
;;
--master-offline)
build_master=true
build_master_offline=true
shift
;;
--metric)
build_metric=true
shift
;;
--no-cache)
no_cache=true
shift
;;
--only)
if [[ -z ${2:-} ]]; then
echo "--only requires a target list" >&2; exit 1
fi
sel="$2"; shift 2
# reset all, then enable selected
build_core=false; build_master=false; build_metric=false; build_web=false; build_alert=false; build_sys=false; build_gpu_bundle=false; build_cpu_bundle=false; build_server_pkg=false; build_client_pkg=false; build_server_pkg_arm=false; build_client_pkg_arm=false
IFS=',' read -ra parts <<< "$sel"
for p in "${parts[@]}"; do
case "$p" in
core) build_core=true ;;
master) build_master=true ;;
metric) build_metric=true ;;
web) build_web=true ;;
alert) build_alert=true ;;
sys) build_sys=true ;;
gpu_bundle) build_gpu_bundle=true ;;
cpu_bundle) build_cpu_bundle=true ;;
# 在 ARM 构建脚本中,server_pkg 视作 arm_server_pkg 的别名,避免误用 x86 打包逻辑
server_pkg) build_server_pkg_arm=true; build_core=true; build_master=true; build_metric=true; build_web=true; build_alert=true ;;
# 在 ARM 构建脚本中,client_pkg 视作 arm_client_pkg 的别名,避免误用 x86 打包逻辑
client_pkg) build_client_pkg_arm=true; build_sys=true ;;
arm_server_pkg) build_server_pkg_arm=true; build_core=true; build_master=true; build_metric=true; build_web=true; build_alert=true ;;
arm_client_pkg) build_client_pkg_arm=true; build_sys=true ;;
all) build_core=true; build_master=true; build_metric=true; build_web=true; build_alert=true; build_sys=true ;;
*) echo "Unknown --only target: $p" >&2; exit 1 ;;
esac
done
;;
--version)
if [[ -z ${2:-} ]]; then echo "--version requires a value like 20251112" >&2; exit 1; fi
bundle_date="$2"; shift 2
;;
--client-semver)
if [[ -z ${2:-} ]]; then echo "--client-semver requires a value like 1.43.0" >&2; exit 1; fi
client_semver="$2"; shift 2
;;
--cuda)
if [[ -z ${2:-} ]]; then echo "--cuda requires a value like 12.2.2" >&2; exit 1; fi
cuda_ver="$2"; shift 2
;;
--tag-latest)
tag_latest=true
shift
;;
-h|--help)
show_help
exit 0
;;
*)
echo "Unknown option: $1" >&2
show_help
exit 1
;;
esac
done
root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
. "$root/scripts/common/build_user.sh"
declare -a build_args=()
if [[ "$use_intranet" == true ]]; then
build_args+=("--build-arg" "USE_INTRANET=true")
fi
cd "$root"
# Set default image tag policy before building
if [[ "$build_server_pkg" == true || "$build_client_pkg" == true || "$build_server_pkg_arm" == true || "$build_client_pkg_arm" == true ]]; then
DEFAULT_IMAGE_TAG="${bundle_date:-latest}"
fi
# Select build user profile for pkg vs default
if [[ "$build_server_pkg" == true || "$build_client_pkg" == true || "$build_server_pkg_arm" == true || "$build_client_pkg_arm" == true ]]; then
export ARGUS_BUILD_PROFILE=pkg
fi
load_build_user
build_args+=("--build-arg" "ARGUS_BUILD_UID=${ARGUS_BUILD_UID}" "--build-arg" "ARGUS_BUILD_GID=${ARGUS_BUILD_GID}")
if [[ "$no_cache" == true ]]; then
build_args+=("--no-cache")
fi
master_root="$root/src/master"
master_offline_tar="$master_root/offline_wheels.tar.gz"
master_offline_dir="$master_root/offline_wheels"
if [[ "$build_master_offline" == true ]]; then
if [[ ! -f "$master_offline_tar" ]]; then
echo "❌ offline wheels tar not found: $master_offline_tar" >&2
echo " 请提前准备好 offline_wheels.tar.gz 后再执行 --master-offline" >&2
exit 1
fi
echo "📦 Preparing offline wheels for master (extracting $master_offline_tar)"
rm -rf "$master_offline_dir"
mkdir -p "$master_offline_dir"
tar -xzf "$master_offline_tar" -C "$master_root"
has_wheel=$(find "$master_offline_dir" -maxdepth 1 -type f -name '*.whl' -print -quit)
if [[ -z "$has_wheel" ]]; then
echo "❌ offline_wheels extraction failed或无 wheel: $master_offline_dir" >&2
exit 1
fi
# ARM 构建下,offline 模式仍通过 Dockerfile 中的 USE_OFFLINE/USE_INTRANET 参数控制
build_args+=("--build-arg" "USE_OFFLINE=1" "--build-arg" "USE_INTRANET=true")
fi
echo "======================================="
echo "ARGUS Unified Build System"
echo "======================================="
if [[ "$use_intranet" == true ]]; then
echo "🌐 Mode: Intranet (Using internal mirror: 10.68.64.1)"
else
echo "🌐 Mode: Public (Using default package sources)"
fi
echo "👤 Build user UID:GID -> ${ARGUS_BUILD_UID}:${ARGUS_BUILD_GID}"
echo "📁 Build context: $root"
echo ""
build_image() {
local image_name=$1
local dockerfile_path=$2
local tag=$3
local context="."
shift 3
if [[ $# -gt 0 ]]; then
context=$1
shift
fi
local extra_args=("$@")
# ARM 专用:如果存在带 .arm64 后缀的 Dockerfile,则优先使用
local dockerfile_for_arch="$dockerfile_path"
if [[ -n "${ARGUS_TARGET_ARCH:-}" && "$ARGUS_TARGET_ARCH" == "arm64" ]]; then
if [[ -f "${dockerfile_path}.arm64" ]]; then
dockerfile_for_arch="${dockerfile_path}.arm64"
fi
fi
echo "🔄 Building $image_name image..."
echo " Dockerfile: $dockerfile_for_arch"
echo " Tag: $tag"
echo " Context: $context"
local tries=${ARGUS_BUILD_RETRIES:-3}
local delay=${ARGUS_BUILD_RETRY_DELAY:-5}
local attempt=1
# 在非 ARM 主机上构建 ARM 镜像时,使用 buildx+--platform=linux/arm64
local use_buildx=false
if [[ "${ARGUS_TARGET_ARCH:-}" == "arm64" && "$(uname -m)" != "aarch64" ]]; then
use_buildx=true
fi
while (( attempt <= tries )); do
echo " Attempt ${attempt}/${tries}"
if [[ "$use_buildx" == true ]]; then
# 通过 buildx 在 x86_64 等非 ARM 主机上构建 ARM64 镜像
if docker buildx build \
--builder "${ARGUS_BUILDX_BUILDER}" \
--platform=linux/arm64 \
"${build_args[@]}" "${extra_args[@]}" \
-f "$dockerfile_for_arch" \
-t "$tag" \
"$context" \
--load; then
echo "✅ $image_name image built successfully (via buildx, platform=linux/arm64)"
return 0
fi
else
# 在 ARM 主机上直接使用 docker build(保留原有 DOCKER_BUILDKIT 回退行为)
local prefix=""
if (( attempt == tries )); then
prefix="DOCKER_BUILDKIT=0"
echo " (final attempt with DOCKER_BUILDKIT=0)"
fi
if eval $prefix docker build "${build_args[@]}" "${extra_args[@]}" -f "$dockerfile_for_arch" -t "$tag" "$context"; then
echo "✅ $image_name image built successfully"
return 0
fi
fi
echo "⚠️ Build failed for $image_name (attempt ${attempt}/${tries})."
if (( attempt < tries )); then
echo " Retrying in ${delay}s..."
sleep "$delay"
fi
attempt=$((attempt+1))
done
echo "❌ Failed to build $image_name image after ${tries} attempts"
return 1
}
pull_base_image() {
local image_ref=$1
local attempts=${2:-3}
local delay=${3:-5}
# If the image already exists locally, skip pulling.
if docker image inspect "$image_ref" >/dev/null 2>&1; then
echo " Local image present; skip pull: $image_ref"
return 0
fi
for ((i=1; i<=attempts; i++)); do
echo " Pulling base image ($i/$attempts): $image_ref"
if docker pull "$image_ref" >/dev/null; then
echo " Base image ready: $image_ref"
return 0
fi
echo " Pull failed: $image_ref"
if (( i < attempts )); then
echo " Retrying in ${delay}s..."
sleep "$delay"
fi
done
echo "❌ Unable to pull base image after ${attempts} attempts: $image_ref"
return 1
}
images_built=()
build_failed=false
build_gpu_bundle_image() {
local date_tag="$1" # e.g. 20251112
local cuda_ver_local="$2" # e.g. 12.2.2
local client_ver="$3" # semver like 1.43.0
if [[ -z "$date_tag" ]]; then
echo "❌ gpu_bundle requires --version YYMMDD (e.g. 20251112)" >&2
return 1
fi
# sanitize cuda version (trim trailing dots like '12.2.')
while [[ "$cuda_ver_local" == *"." ]]; do cuda_ver_local="${cuda_ver_local%.}"; done
# Resolve effective CUDA base tag
local resolve_cuda_base_tag
resolve_cuda_base_tag() {
local want="$1" # can be 12, 12.2 or 12.2.2
local major minor patch
if [[ "$want" =~ ^([0-9]+)\.([0-9]+)\.([0-9]+)$ ]]; then
major="${BASH_REMATCH[1]}"; minor="${BASH_REMATCH[2]}"; patch="${BASH_REMATCH[3]}"
echo "nvidia/cuda:${major}.${minor}.${patch}-runtime-ubuntu22.04"; return 0
elif [[ "$want" =~ ^([0-9]+)\.([0-9]+)$ ]]; then
major="${BASH_REMATCH[1]}"; minor="${BASH_REMATCH[2]}"
# try to find best local patch for major.minor
local best
best=$(docker images --format '{{.Repository}}:{{.Tag}}' nvidia/cuda 2>/dev/null | \
grep -E "^nvidia/cuda:${major}\.${minor}\\.[0-9]+-runtime-ubuntu22\.04$" | \
sed -E 's#^nvidia/cuda:([0-9]+\.[0-9]+\.)([0-9]+)-runtime-ubuntu22\.04$#\1\2#g' | \
sort -V | tail -n1 || true)
if [[ -n "$best" ]]; then
echo "nvidia/cuda:${best}-runtime-ubuntu22.04"; return 0
fi
# fallback patch if none local
echo "nvidia/cuda:${major}.${minor}.2-runtime-ubuntu22.04"; return 0
elif [[ "$want" =~ ^([0-9]+)$ ]]; then
major="${BASH_REMATCH[1]}"
# try to find best local for this major
local best
best=$(docker images --format '{{.Repository}}:{{.Tag}}' nvidia/cuda 2>/dev/null | \
grep -E "^nvidia/cuda:${major}\\.[0-9]+\\.[0-9]+-runtime-ubuntu22\.04$" | \
sed -E 's#^nvidia/cuda:([0-9]+\.[0-9]+\.[0-9]+)-runtime-ubuntu22\.04$#\1#g' | \
sort -V | tail -n1 || true)
if [[ -n "$best" ]]; then
echo "nvidia/cuda:${best}-runtime-ubuntu22.04"; return 0
fi
echo "nvidia/cuda:${major}.2.2-runtime-ubuntu22.04"; return 0
else
# invalid format, fallback to default
echo "nvidia/cuda:12.2.2-runtime-ubuntu22.04"; return 0
fi
}
local base_image
base_image=$(resolve_cuda_base_tag "$cuda_ver_local")
echo
echo "🔧 Preparing one-click GPU bundle build"
echo " CUDA runtime base: ${base_image}"
echo " Bundle tag : ${date_tag}"
# 1) Ensure NVIDIA base image (skip pull if local)
if ! pull_base_image "$base_image"; then
# try once more with default if resolution failed
if ! pull_base_image "nvidia/cuda:12.2.2-runtime-ubuntu22.04"; then
return 1
else
base_image="nvidia/cuda:12.2.2-runtime-ubuntu22.04"
fi
fi
# 2) Build latest argus-agent from source
echo "\n🛠 Building argus-agent from src/agent"
pushd "$root/src/agent" >/dev/null
if ! bash scripts/build_binary.sh; then
echo "❌ argus-agent build failed" >&2
popd >/dev/null
return 1
fi
if [[ ! -f "dist/argus-agent" ]]; then
echo "❌ argus-agent binary missing after build" >&2
popd >/dev/null
return 1
fi
popd >/dev/null
# 3) Inject agent into all-in-one-full plugin and package artifact
local aio_root="$root/src/metric/client-plugins/all-in-one-full"
local agent_bin_src="$root/src/agent/dist/argus-agent"
local agent_bin_dst="$aio_root/plugins/argus-agent/bin/argus-agent"
echo "\n📦 Updating all-in-one-full agent binary → $agent_bin_dst"
cp -f "$agent_bin_src" "$agent_bin_dst"
chmod +x "$agent_bin_dst" || true
pushd "$aio_root" >/dev/null
local prev_version
prev_version="$(cat config/VERSION 2>/dev/null || echo "1.0.0")"
local use_version="$prev_version"
if [[ -n "$client_semver" ]]; then
echo "${client_semver}" > config/VERSION
use_version="$client_semver"
fi
echo " Packaging all-in-one-full artifact version: $use_version"
if ! bash scripts/package_artifact.sh --force; then
echo "❌ package_artifact.sh failed" >&2
# restore VERSION if changed
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
return 1
fi
local artifact_dir="$aio_root/artifact/$use_version"
local artifact_tar
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
if [[ -z "$artifact_tar" ]]; then
echo " No argus-metric_*.tar.gz found; invoking publish_artifact.sh to assemble..."
local owner="$(id -u):$(id -g)"
if ! bash scripts/publish_artifact.sh "$use_version" --output-dir "$artifact_dir" --owner "$owner"; then
echo "❌ publish_artifact.sh failed" >&2
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
return 1
fi
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
fi
if [[ -z "$artifact_tar" ]]; then
echo "❌ artifact tar not found under $artifact_dir" >&2
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
return 1
fi
# restore VERSION if changed (keep filesystem clean)
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
# 4) Stage docker build context
local bundle_ctx="$root/src/bundle/gpu-node-bundle/.build-$date_tag"
echo "\n🧰 Staging docker build context: $bundle_ctx"
rm -rf "$bundle_ctx"
mkdir -p "$bundle_ctx/bundle" "$bundle_ctx/private"
cp "$root/src/bundle/gpu-node-bundle/Dockerfile" "$bundle_ctx/"
cp "$root/src/bundle/gpu-node-bundle/node-bootstrap.sh" "$bundle_ctx/"
cp "$root/src/bundle/gpu-node-bundle/health-watcher.sh" "$bundle_ctx/"
# bundle tar
cp "$artifact_tar" "$bundle_ctx/bundle/"
# offline fluent-bit assets (optional but useful)
if [[ -d "$root/src/log/fluent-bit/build/etc" ]]; then
cp -r "$root/src/log/fluent-bit/build/etc" "$bundle_ctx/private/"
fi
if [[ -d "$root/src/log/fluent-bit/build/packages" ]]; then
cp -r "$root/src/log/fluent-bit/build/packages" "$bundle_ctx/private/"
fi
if [[ -f "$root/src/log/fluent-bit/build/start-fluent-bit.sh" ]]; then
cp "$root/src/log/fluent-bit/build/start-fluent-bit.sh" "$bundle_ctx/private/"
fi
# 5) Build the final bundle image (directly from NVIDIA base)
local image_tag="argus-sys-metric-test-node-bundle-gpu-arm64:${date_tag}"
echo "\n🔄 Building GPU Bundle image"
if build_image "GPU Bundle" "$bundle_ctx/Dockerfile" "$image_tag" "$bundle_ctx" \
--build-arg CUDA_VER="$(echo "$base_image" | sed -E 's#^nvidia/cuda:([0-9]+\.[0-9]+\.[0-9]+)-runtime-ubuntu22\.04$#\1#')" \
--build-arg CLIENT_VER="$use_version" \
--build-arg BUNDLE_DATE="$date_tag"; then
images_built+=("$image_tag")
# In non-pkg mode, also tag latest for convenience
if [[ "${ARGUS_PKG_BUILD:-0}" != "1" ]]; then
docker tag "$image_tag" argus-sys-metric-test-node-bundle-gpu-arm64:latest >/dev/null 2>&1 || true
fi
return 0
else
return 1
fi
}
# Tag helper: ensure :<date_tag> exists for a list of repos
ensure_version_tags() {
local date_tag="$1"; shift
local repos=("$@")
for repo in "${repos[@]}"; do
if docker image inspect "$repo:$date_tag" >/dev/null 2>&1; then
:
elif docker image inspect "$repo:latest" >/dev/null 2>&1; then
docker tag "$repo:latest" "$repo:$date_tag" || true
else
echo "❌ missing image for tagging: $repo (need :latest or :$date_tag)" >&2
return 1
fi
done
return 0
}
# Build server package after images are built
build_server_pkg_bundle() {
local date_tag="$1"
if [[ -z "$date_tag" ]]; then
echo "❌ server_pkg requires --version YYMMDD" >&2
return 1
fi
local repos=(
argus-bind9-arm64 argus-master-arm64 argus-elasticsearch-arm64 argus-kibana-arm64 \
argus-metric-ftp-arm64 argus-metric-prometheus-arm64 argus-metric-grafana-arm64 \
argus-alertmanager-arm64 argus-web-frontend-arm64 argus-web-proxy-arm64
)
echo "\n🔖 Verifying server images with :$date_tag and collecting digests"
for repo in "${repos[@]}"; do
if ! docker image inspect "$repo:$date_tag" >/dev/null 2>&1; then
echo "❌ required image missing: $repo:$date_tag (build phase should have produced it)" >&2
return 1
fi
done
# Optional: show digests
for repo in "${repos[@]}"; do
local digest
digest=$(docker images --digests --format '{{.Repository}}:{{.Tag}} {{.Digest}}' | awk -v r="$repo:$date_tag" '$1==r{print $2}' | head -n1)
printf ' • %s@%s\n' "$repo:$date_tag" "${digest:-<none>}"
done
echo "\n📦 Building server package via deployment_new/build/make_server_package.sh --version $date_tag"
if ! "$root/deployment_new/build/make_server_package.sh" --version "$date_tag"; then
echo "❌ make_server_package.sh failed" >&2
return 1
fi
return 0
}
# Build client package: ensure gpu bundle image exists, then package client_gpu
build_client_pkg_bundle() {
local date_tag="$1"
local semver="$2"
local cuda="$3"
if [[ -z "$date_tag" ]]; then
echo "❌ client_pkg requires --version YYMMDD" >&2
return 1
fi
local bundle_tag="argus-sys-metric-test-node-bundle-gpu-arm64:${date_tag}"
if ! docker image inspect "$bundle_tag" >/dev/null 2>&1; then
echo "\n🧩 GPU bundle image $bundle_tag missing; building it first..."
ARGUS_PKG_BUILD=1
export ARGUS_PKG_BUILD
if ! build_gpu_bundle_image "$date_tag" "$cuda" "$semver"; then
return 1
fi
else
echo "\n✅ Using existing GPU bundle image: $bundle_tag"
fi
echo "\n📦 Building client GPU package via deployment_new/build/make_client_gpu_package.sh --version $date_tag --image $bundle_tag"
if ! "$root/deployment_new/build/make_client_gpu_package.sh" --version "$date_tag" --image "$bundle_tag"; then
echo "❌ make_client_gpu_package.sh failed" >&2
return 1
fi
return 0
}
# Build CPU bundle image directly FROM ubuntu:22.04 (no intermediate base)
build_cpu_bundle_image() {
local date_tag="$1" # e.g. 20251113
local client_ver_in="$2" # semver like 1.43.0 (optional)
local want_tag_latest="$3" # true/false
if [[ -z "$date_tag" ]]; then
echo "❌ cpu_bundle requires --version YYMMDD" >&2
return 1
fi
echo "\n🔧 Preparing one-click CPU bundle build"
echo " Base: ubuntu:22.04"
echo " Bundle tag: ${date_tag}"
# 1) Build latest argus-agent from source
echo "\n🛠 Building argus-agent from src/agent"
pushd "$root/src/agent" >/dev/null
if ! bash scripts/build_binary.sh; then
echo "❌ argus-agent build failed" >&2
popd >/dev/null
return 1
fi
if [[ ! -f "dist/argus-agent" ]]; then
echo "❌ argus-agent binary missing after build" >&2
popd >/dev/null
return 1
fi
popd >/dev/null
# 2) Inject agent into all-in-one-full plugin and package artifact
local aio_root="$root/src/metric/client-plugins/all-in-one-full"
local agent_bin_src="$root/src/agent/dist/argus-agent"
local agent_bin_dst="$aio_root/plugins/argus-agent/bin/argus-agent"
echo "\n📦 Updating all-in-one-full agent binary → $agent_bin_dst"
cp -f "$agent_bin_src" "$agent_bin_dst"
chmod +x "$agent_bin_dst" || true
pushd "$aio_root" >/dev/null
local prev_version use_version
prev_version="$(cat config/VERSION 2>/dev/null || echo "1.0.0")"
use_version="$prev_version"
if [[ -n "$client_ver_in" ]]; then
echo "$client_ver_in" > config/VERSION
use_version="$client_ver_in"
fi
echo " Packaging all-in-one-full artifact: version=$use_version"
if ! bash scripts/package_artifact.sh --force; then
echo "❌ package_artifact.sh failed" >&2
[[ -n "$client_ver_in" ]] && echo "$prev_version" > config/VERSION
popd >/dev/null
return 1
fi
local artifact_dir="$aio_root/artifact/$use_version"
local artifact_tar
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
if [[ -z "$artifact_tar" ]]; then
echo " No argus-metric_*.tar.gz found; invoking publish_artifact.sh ..."
local owner="$(id -u):$(id -g)"
if ! bash scripts/publish_artifact.sh "$use_version" --output-dir "$artifact_dir" --owner "$owner"; then
echo "❌ publish_artifact.sh failed" >&2
[[ -n "$client_ver_in" ]] && echo "$prev_version" > config/VERSION
popd >/dev/null
return 1
fi
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
fi
[[ -n "$client_ver_in" ]] && echo "$prev_version" > config/VERSION
popd >/dev/null
# 3) Stage docker build context
local bundle_ctx="$root/src/bundle/cpu-node-bundle/.build-$date_tag"
echo "\n🧰 Staging docker build context: $bundle_ctx"
rm -rf "$bundle_ctx"
mkdir -p "$bundle_ctx/bundle" "$bundle_ctx/private"
cp "$root/src/bundle/cpu-node-bundle/Dockerfile" "$bundle_ctx/"
cp "$root/src/bundle/cpu-node-bundle/node-bootstrap.sh" "$bundle_ctx/"
cp "$root/src/bundle/cpu-node-bundle/health-watcher.sh" "$bundle_ctx/"
# bundle tar
cp "$artifact_tar" "$bundle_ctx/bundle/"
# offline fluent-bit assets
if [[ -d "$root/src/log/fluent-bit/build/etc" ]]; then
cp -r "$root/src/log/fluent-bit/build/etc" "$bundle_ctx/private/"
fi
if [[ -d "$root/src/log/fluent-bit/build/packages" ]]; then
cp -r "$root/src/log/fluent-bit/build/packages" "$bundle_ctx/private/"
fi
if [[ -f "$root/src/log/fluent-bit/build/start-fluent-bit.sh" ]]; then
cp "$root/src/log/fluent-bit/build/start-fluent-bit.sh" "$bundle_ctx/private/"
fi
# 4) Build final bundle image
local image_tag="argus-sys-metric-test-node-bundle-arm64:${date_tag}"
echo "\n🔄 Building CPU Bundle image"
if build_image "CPU Bundle" "$bundle_ctx/Dockerfile" "$image_tag" "$bundle_ctx"; then
images_built+=("$image_tag")
# 为兼容现有 compose/部署,额外打无后缀别名
docker tag "$image_tag" "argus-sys-metric-test-node-bundle:${date_tag}" >/dev/null 2>&1 || true
if [[ "$want_tag_latest" == "true" ]]; then
docker tag "$image_tag" argus-sys-metric-test-node-bundle-arm64:latest >/dev/null 2>&1 || true
docker tag "$image_tag" argus-sys-metric-test-node-bundle:latest >/dev/null 2>&1 || true
fi
return 0
else
return 1
fi
}
if [[ "$build_core" == true ]]; then
if build_image "Elasticsearch" "src/log/elasticsearch/build/Dockerfile" "argus-elasticsearch-arm64:${DEFAULT_IMAGE_TAG}"; then
images_built+=("argus-elasticsearch-arm64:${DEFAULT_IMAGE_TAG}")
else
build_failed=true
fi
echo ""
if build_image "Kibana" "src/log/kibana/build/Dockerfile" "argus-kibana-arm64:${DEFAULT_IMAGE_TAG}"; then
images_built+=("argus-kibana-arm64:${DEFAULT_IMAGE_TAG}")
else
build_failed=true
fi
echo ""
if build_image "BIND9" "src/bind/build/Dockerfile" "argus-bind9-arm64:${DEFAULT_IMAGE_TAG}"; then
images_built+=("argus-bind9-arm64:${DEFAULT_IMAGE_TAG}")
else
build_failed=true
fi
fi
echo ""
if [[ "$build_master" == true ]]; then
echo ""
echo "🔄 Building Master image..."
# 复用通用 build_image 函数,通过 buildx 构建 ARM64 master 镜像
if build_image "Master" "src/master/Dockerfile" "argus-master-arm64:${DEFAULT_IMAGE_TAG}" "."; then
images_built+=("argus-master-arm64:${DEFAULT_IMAGE_TAG}")
else
build_failed=true
fi
fi
if [[ "$build_metric" == true ]]; then
echo ""
echo "Building Metric module images..."
metric_base_images=(
"ubuntu:22.04"
"prom/prometheus:v3.5.0"
"grafana/grafana:11.1.0"
)
for base_image in "${metric_base_images[@]}"; do
if ! pull_base_image "$base_image"; then
build_failed=true
fi
done
metric_builds=(
"Metric FTP|src/metric/ftp/build/Dockerfile|argus-metric-ftp-arm64:${DEFAULT_IMAGE_TAG}|src/metric/ftp/build"
"Metric Prometheus|src/metric/prometheus/build/Dockerfile|argus-metric-prometheus-arm64:${DEFAULT_IMAGE_TAG}|src/metric/prometheus/build"
"Metric Grafana|src/metric/grafana/build/Dockerfile|argus-metric-grafana-arm64:${DEFAULT_IMAGE_TAG}|src/metric/grafana/build"
"Metric Prometheus Targets Updater|src/metric/prometheus/build/Dockerfile.targets-updater|argus-metric-prometheus-targets-updater-arm64:${DEFAULT_IMAGE_TAG}|src/metric/prometheus/build"
)
for build_spec in "${metric_builds[@]}"; do
IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec"
if build_image "$image_label" "$dockerfile_path" "$image_tag" "$build_context"; then
images_built+=("$image_tag")
else
build_failed=true
fi
echo ""
done
fi
# =======================================
# Sys (system tests) node images
# =======================================
if [[ "$build_sys" == true ]]; then
echo ""
echo "Building Sys node images..."
sys_base_images=(
"ubuntu:22.04"
)
# GPU 相关镜像目前仅在 x86_64 上支持;ARM 上不拉取 nvidia/cuda 基础镜像
if [[ "${ARGUS_TARGET_ARCH:-}" != "arm64" ]]; then
sys_base_images+=("nvidia/cuda:12.2.2-runtime-ubuntu22.04")
fi
for base_image in "${sys_base_images[@]}"; do
if ! pull_base_image "$base_image"; then
build_failed=true
fi
done
sys_builds=(
"Sys Node|src/sys/build/node/Dockerfile|argus-sys-node-arm64:latest|."
"Sys Metric Test Node|src/sys/build/arm-cpu-node/Dockerfile|argus-sys-metric-test-node-arm64:latest|."
)
# GPU 测试节点镜像仅在 x86_64 路径构建,ARM 版本暂不支持 DCGM/GPU
if [[ "${ARGUS_TARGET_ARCH:-}" != "arm64" ]]; then
sys_builds+=("Sys Metric Test GPU Node|src/sys/build/test-gpu-node/Dockerfile|argus-sys-metric-test-gpu-node:latest|.")
fi
for build_spec in "${sys_builds[@]}"; do
IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec"
if build_image "$image_label" "$dockerfile_path" "$image_tag" "$build_context"; then
images_built+=("$image_tag")
# 与历史 NODE_BUNDLE_IMAGE_TAG 保持兼容:为 ARM CPU 节点镜像打 bundle 别名
if [[ "$image_tag" == "argus-sys-metric-test-node-arm64:latest" ]]; then
docker tag "$image_tag" argus-sys-metric-test-node-bundle-arm64:latest >/dev/null 2>&1 || true
docker tag "$image_tag" argus-sys-metric-test-node-bundle:latest >/dev/null 2>&1 || true
fi
else
build_failed=true
fi
echo ""
done
fi
# =======================================
# Web & Alert module images
# =======================================
if [[ "$build_web" == true || "$build_alert" == true ]]; then
echo ""
echo "Building Web and Alert module images..."
# Pre-pull commonly used base images for stability
web_alert_base_images=(
"node:20"
"ubuntu:24.04"
)
for base_image in "${web_alert_base_images[@]}"; do
if ! pull_base_image "$base_image"; then
build_failed=true
fi
done
if [[ "$build_web" == true ]]; then
web_builds=(
"Web Frontend|src/web/build_tools/frontend/Dockerfile|argus-web-frontend-arm64:${DEFAULT_IMAGE_TAG}|."
"Web Proxy|src/web/build_tools/proxy/Dockerfile|argus-web-proxy-arm64:${DEFAULT_IMAGE_TAG}|."
)
for build_spec in "${web_builds[@]}"; do
IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec"
if build_image "$image_label" "$dockerfile_path" "$image_tag" "$build_context"; then
images_built+=("$image_tag")
else
build_failed=true
fi
echo ""
done
fi
if [[ "$build_alert" == true ]]; then
alert_builds=(
"Alertmanager|src/alert/alertmanager/build/Dockerfile|argus-alertmanager-arm64:${DEFAULT_IMAGE_TAG}|."
)
for build_spec in "${alert_builds[@]}"; do
IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec"
if build_image "$image_label" "$dockerfile_path" "$image_tag" "$build_context" --build-arg ALERTMANAGER_ARCH=arm64; then
images_built+=("$image_tag")
else
build_failed=true
fi
echo ""
done
fi
fi
# =======================================
# One-click GPU bundle (direct NVIDIA base)
# =======================================
if [[ "$build_gpu_bundle" == true ]]; then
echo ""
echo "Building one-click GPU bundle image..."
if ! build_gpu_bundle_image "$bundle_date" "$cuda_ver" "$client_semver"; then
build_failed=true
fi
fi
# =======================================
# One-click CPU bundle (from ubuntu:22.04)
# =======================================
if [[ "$build_cpu_bundle" == true ]]; then
echo ""
echo "Building one-click CPU bundle image..."
if ! build_cpu_bundle_image "${bundle_date}" "${client_semver}" "${tag_latest}"; then
build_failed=true
fi
fi
# =======================================
# One-click Server/Client packaging
# =======================================
if [[ "$build_server_pkg" == true ]]; then
echo ""
echo "🧳 Building one-click Server package..."
if ! build_server_pkg_bundle "${bundle_date}"; then
build_failed=true
fi
fi
if [[ "$build_client_pkg" == true ]]; then
echo ""
echo "🧳 Building one-click Client-GPU package..."
if ! build_client_pkg_bundle "${bundle_date}" "${client_semver}" "${cuda_ver}"; then
build_failed=true
fi
fi
# ARM 专用:Server/Client 部署包
if [[ "$build_server_pkg_arm" == true ]]; then
if [[ -z "$bundle_date" ]]; then
echo "❌ arm_server_pkg requires --version YYYYMMDD" >&2
build_failed=true
else
echo ""
echo "🧳 Building ARM Server package..."
if ! "$root/deployment_new/build/make_arm_server_package.sh" --version "$bundle_date"; then
build_failed=true
fi
fi
fi
if [[ "$build_client_pkg_arm" == true ]]; then
if [[ -z "$bundle_date" ]]; then
echo "❌ arm_client_pkg requires --version YYYYMMDD" >&2
build_failed=true
else
echo ""
echo "🧳 Building ARM Client-CPU package..."
cpu_repo="argus-sys-metric-test-node-arm64"
cpu_image_arg="$cpu_repo:latest"
# 若已存在 :<version>,优先使用;否则退回 latest 交由打包脚本打 tag
if docker image inspect "${cpu_repo}:${bundle_date}" >/dev/null 2>&1; then
cpu_image_arg="${cpu_repo}:${bundle_date}"
elif ! docker image inspect "${cpu_repo}:latest" >/dev/null 2>&1; then
echo "❌ required CPU node image missing: ${cpu_repo}:{${bundle_date},latest}" >&2
build_failed=true
fi
if [[ "$build_failed" != "true" ]]; then
if ! "$root/deployment_new/build/make_arm_client_package.sh" --version "$bundle_date" --image "$cpu_image_arg"; then
build_failed=true
fi
fi
fi
fi
echo "======================================="
echo "📦 Build Summary"
echo "======================================="
if [[ ${#images_built[@]} -gt 0 ]]; then
echo "✅ Successfully built images:"
for image in "${images_built[@]}"; do
echo " • $image"
done
fi
if [[ "$build_failed" == true ]]; then
echo ""
echo "❌ Some images failed to build. Please check the errors above."
exit 1
fi
if [[ "$use_intranet" == true ]]; then
echo ""
echo "🌐 Built with intranet mirror configuration"
fi
if [[ "$build_master_offline" == true ]]; then
echo ""
echo "🧳 Master offline wheels 已解压到 $master_offline_dir"
fi
echo ""
echo "🚀 Next steps:"
echo " ./build/save_images.sh --compress # 导出镜像"
echo " cd src/master/tests && MASTER_IMAGE_TAG=argus-master:offline ./scripts/00_e2e_test.sh"
echo ""

875
build/build_images_for_x64.sh Executable file
View File

@@ -0,0 +1,875 @@
#!/usr/bin/env bash
set -euo pipefail
# ARGUS x86_64 Image Build Entry
# 本脚本用于在 x86_64 平台上构建 Argus 镜像,
# 逻辑与历史版本的 build/build_images.sh 保持一致。
show_help() {
cat <<'EOF'
ARGUS Unified Build System - Image Build Tool
Usage: $0 [OPTIONS]
Options:
--intranet Use intranet mirror for log/bind builds
--master-offline Build master offline image (requires src/master/offline_wheels.tar.gz)
--metric Build metric module images (ftp, prometheus, grafana, test nodes)
--no-cache Build all images without using Docker layer cache
--only LIST Comma-separated targets to build: core,master,metric,web,alert,sys,gpu_bundle,cpu_bundle,server_pkg,client_pkg,all
--version DATE Date tag used by gpu_bundle/server_pkg/client_pkg (e.g. 20251112)
--client-semver X.Y.Z Override client semver used in all-in-one-full artifact (optional)
--cuda VER CUDA runtime version for NVIDIA base (default: 12.2.2)
--tag-latest Also tag bundle image as :latest (for cpu_bundle only; default off)
-h, --help Show this help message
Examples:
$0 # Build with default sources
$0 --intranet # Build with intranet mirror
$0 --master-offline # Additionally build argus-master:offline
$0 --metric # Additionally build metric module images
$0 --intranet --master-offline --metric
EOF
}
use_intranet=false
build_core=true
build_master=true
build_master_offline=false
build_metric=true
build_web=true
build_alert=true
build_sys=true
build_gpu_bundle=false
build_cpu_bundle=false
build_server_pkg=false
build_client_pkg=false
no_cache=false
bundle_date=""
client_semver=""
cuda_ver="12.2.2"
DEFAULT_IMAGE_TAG="latest"
tag_latest=false
while [[ $# -gt 0 ]]; do
case $1 in
--intranet)
use_intranet=true
shift
;;
--master)
build_master=true
shift
;;
--master-offline)
build_master=true
build_master_offline=true
shift
;;
--metric)
build_metric=true
shift
;;
--no-cache)
no_cache=true
shift
;;
--only)
if [[ -z ${2:-} ]]; then
echo "--only requires a target list" >&2; exit 1
fi
sel="$2"; shift 2
# reset all, then enable selected
build_core=false; build_master=false; build_metric=false; build_web=false; build_alert=false; build_sys=false; build_gpu_bundle=false; build_cpu_bundle=false; build_server_pkg=false; build_client_pkg=false
IFS=',' read -ra parts <<< "$sel"
for p in "${parts[@]}"; do
case "$p" in
core) build_core=true ;;
master) build_master=true ;;
metric) build_metric=true ;;
web) build_web=true ;;
alert) build_alert=true ;;
sys) build_sys=true ;;
gpu_bundle) build_gpu_bundle=true ;;
cpu_bundle) build_cpu_bundle=true ;;
server_pkg) build_server_pkg=true; build_core=true; build_master=true; build_metric=true; build_web=true; build_alert=true ;;
client_pkg) build_client_pkg=true ;;
all) build_core=true; build_master=true; build_metric=true; build_web=true; build_alert=true; build_sys=true ;;
*) echo "Unknown --only target: $p" >&2; exit 1 ;;
esac
done
;;
--version)
if [[ -z ${2:-} ]]; then echo "--version requires a value like 20251112" >&2; exit 1; fi
bundle_date="$2"; shift 2
;;
--client-semver)
if [[ -z ${2:-} ]]; then echo "--client-semver requires a value like 1.43.0" >&2; exit 1; fi
client_semver="$2"; shift 2
;;
--cuda)
if [[ -z ${2:-} ]]; then echo "--cuda requires a value like 12.2.2" >&2; exit 1; fi
cuda_ver="$2"; shift 2
;;
--tag-latest)
tag_latest=true
shift
;;
-h|--help)
show_help
exit 0
;;
*)
echo "Unknown option: $1" >&2
show_help
exit 1
;;
esac
done
root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
. "$root/scripts/common/build_user.sh"
declare -a build_args=()
if [[ "$use_intranet" == true ]]; then
build_args+=("--build-arg" "USE_INTRANET=true")
fi
cd "$root"
# Set default image tag policy before building
if [[ "$build_server_pkg" == true ]]; then
DEFAULT_IMAGE_TAG="${bundle_date:-latest}"
fi
# Select build user profile for pkg vs default
if [[ "$build_server_pkg" == true || "$build_client_pkg" == true ]]; then
export ARGUS_BUILD_PROFILE=pkg
fi
load_build_user
build_args+=("--build-arg" "ARGUS_BUILD_UID=${ARGUS_BUILD_UID}" "--build-arg" "ARGUS_BUILD_GID=${ARGUS_BUILD_GID}")
if [[ "$no_cache" == true ]]; then
build_args+=("--no-cache")
fi
master_root="$root/src/master"
master_offline_tar="$master_root/offline_wheels.tar.gz"
master_offline_dir="$master_root/offline_wheels"
if [[ "$build_master_offline" == true ]]; then
if [[ ! -f "$master_offline_tar" ]]; then
echo "❌ offline wheels tar not found: $master_offline_tar" >&2
echo " 请提前准备好 offline_wheels.tar.gz 后再执行 --master-offline" >&2
exit 1
fi
echo "📦 Preparing offline wheels for master (extracting $master_offline_tar)"
rm -rf "$master_offline_dir"
mkdir -p "$master_offline_dir"
tar -xzf "$master_offline_tar" -C "$master_root"
has_wheel=$(find "$master_offline_dir" -maxdepth 1 -type f -name '*.whl' -print -quit)
if [[ -z "$has_wheel" ]]; then
echo "❌ offline_wheels extraction failed或无 wheel: $master_offline_dir" >&2
exit 1
fi
fi
echo "======================================="
echo "ARGUS Unified Build System"
echo "======================================="
if [[ "$use_intranet" == true ]]; then
echo "🌐 Mode: Intranet (Using internal mirror: 10.68.64.1)"
else
echo "🌐 Mode: Public (Using default package sources)"
fi
echo "👤 Build user UID:GID -> ${ARGUS_BUILD_UID}:${ARGUS_BUILD_GID}"
echo "📁 Build context: $root"
echo ""
build_image() {
local image_name=$1
local dockerfile_path=$2
local tag=$3
local context="."
shift 3
if [[ $# -gt 0 ]]; then
context=$1
shift
fi
local extra_args=("$@")
echo "🔄 Building $image_name image..."
echo " Dockerfile: $dockerfile_path"
echo " Tag: $tag"
echo " Context: $context"
local tries=${ARGUS_BUILD_RETRIES:-3}
local delay=${ARGUS_BUILD_RETRY_DELAY:-5}
local attempt=1
while (( attempt <= tries )); do
local prefix=""
if (( attempt == tries )); then
# final attempt: disable BuildKit to avoid docker/dockerfile front-end pulls
prefix="DOCKER_BUILDKIT=0"
echo " Attempt ${attempt}/${tries} (fallback: DOCKER_BUILDKIT=0)"
else
echo " Attempt ${attempt}/${tries}"
fi
if eval $prefix docker build "${build_args[@]}" "${extra_args[@]}" -f "$dockerfile_path" -t "$tag" "$context"; then
echo "✅ $image_name image built successfully"
return 0
fi
echo "⚠️ Build failed for $image_name (attempt ${attempt}/${tries})."
if (( attempt < tries )); then
echo " Retrying in ${delay}s..."
sleep "$delay"
fi
attempt=$((attempt+1))
done
echo "❌ Failed to build $image_name image after ${tries} attempts"
return 1
}
pull_base_image() {
local image_ref=$1
local attempts=${2:-3}
local delay=${3:-5}
# If the image already exists locally, skip pulling.
if docker image inspect "$image_ref" >/dev/null 2>&1; then
echo " Local image present; skip pull: $image_ref"
return 0
fi
for ((i=1; i<=attempts; i++)); do
echo " Pulling base image ($i/$attempts): $image_ref"
if docker pull "$image_ref" >/dev/null; then
echo " Base image ready: $image_ref"
return 0
fi
echo " Pull failed: $image_ref"
if (( i < attempts )); then
echo " Retrying in ${delay}s..."
sleep "$delay"
fi
done
echo "❌ Unable to pull base image after ${attempts} attempts: $image_ref"
return 1
}
images_built=()
build_failed=false
build_gpu_bundle_image() {
local date_tag="$1" # e.g. 20251112
local cuda_ver_local="$2" # e.g. 12.2.2
local client_ver="$3" # semver like 1.43.0
if [[ -z "$date_tag" ]]; then
echo "❌ gpu_bundle requires --version YYMMDD (e.g. 20251112)" >&2
return 1
fi
# sanitize cuda version (trim trailing dots like '12.2.')
while [[ "$cuda_ver_local" == *"." ]]; do cuda_ver_local="${cuda_ver_local%.}"; done
# Resolve effective CUDA base tag
local resolve_cuda_base_tag
resolve_cuda_base_tag() {
local want="$1" # can be 12, 12.2 or 12.2.2
local major minor patch
if [[ "$want" =~ ^([0-9]+)\.([0-9]+)\.([0-9]+)$ ]]; then
major="${BASH_REMATCH[1]}"; minor="${BASH_REMATCH[2]}"; patch="${BASH_REMATCH[3]}"
echo "nvidia/cuda:${major}.${minor}.${patch}-runtime-ubuntu22.04"; return 0
elif [[ "$want" =~ ^([0-9]+)\.([0-9]+)$ ]]; then
major="${BASH_REMATCH[1]}"; minor="${BASH_REMATCH[2]}"
# try to find best local patch for major.minor
local best
best=$(docker images --format '{{.Repository}}:{{.Tag}}' nvidia/cuda 2>/dev/null | \
grep -E "^nvidia/cuda:${major}\.${minor}\\.[0-9]+-runtime-ubuntu22\.04$" | \
sed -E 's#^nvidia/cuda:([0-9]+\.[0-9]+\.)([0-9]+)-runtime-ubuntu22\.04$#\1\2#g' | \
sort -V | tail -n1 || true)
if [[ -n "$best" ]]; then
echo "nvidia/cuda:${best}-runtime-ubuntu22.04"; return 0
fi
# fallback patch if none local
echo "nvidia/cuda:${major}.${minor}.2-runtime-ubuntu22.04"; return 0
elif [[ "$want" =~ ^([0-9]+)$ ]]; then
major="${BASH_REMATCH[1]}"
# try to find best local for this major
local best
best=$(docker images --format '{{.Repository}}:{{.Tag}}' nvidia/cuda 2>/dev/null | \
grep -E "^nvidia/cuda:${major}\\.[0-9]+\\.[0-9]+-runtime-ubuntu22\.04$" | \
sed -E 's#^nvidia/cuda:([0-9]+\.[0-9]+\.[0-9]+)-runtime-ubuntu22\.04$#\1#g' | \
sort -V | tail -n1 || true)
if [[ -n "$best" ]]; then
echo "nvidia/cuda:${best}-runtime-ubuntu22.04"; return 0
fi
echo "nvidia/cuda:${major}.2.2-runtime-ubuntu22.04"; return 0
else
# invalid format, fallback to default
echo "nvidia/cuda:12.2.2-runtime-ubuntu22.04"; return 0
fi
}
local base_image
base_image=$(resolve_cuda_base_tag "$cuda_ver_local")
echo
echo "🔧 Preparing one-click GPU bundle build"
echo " CUDA runtime base: ${base_image}"
echo " Bundle tag : ${date_tag}"
# 1) Ensure NVIDIA base image (skip pull if local)
if ! pull_base_image "$base_image"; then
# try once more with default if resolution failed
if ! pull_base_image "nvidia/cuda:12.2.2-runtime-ubuntu22.04"; then
return 1
else
base_image="nvidia/cuda:12.2.2-runtime-ubuntu22.04"
fi
fi
# 2) Build latest argus-agent from source
echo "\n🛠 Building argus-agent from src/agent"
pushd "$root/src/agent" >/dev/null
if ! bash scripts/build_binary.sh; then
echo "❌ argus-agent build failed" >&2
popd >/dev/null
return 1
fi
if [[ ! -f "dist/argus-agent" ]]; then
echo "❌ argus-agent binary missing after build" >&2
popd >/dev/null
return 1
fi
popd >/dev/null
# 3) Inject agent into all-in-one-full plugin and package artifact
local aio_root="$root/src/metric/client-plugins/all-in-one-full"
local agent_bin_src="$root/src/agent/dist/argus-agent"
local agent_bin_dst="$aio_root/plugins/argus-agent/bin/argus-agent"
echo "\n📦 Updating all-in-one-full agent binary → $agent_bin_dst"
cp -f "$agent_bin_src" "$agent_bin_dst"
chmod +x "$agent_bin_dst" || true
pushd "$aio_root" >/dev/null
local prev_version
prev_version="$(cat config/VERSION 2>/dev/null || echo "1.0.0")"
local use_version="$prev_version"
if [[ -n "$client_semver" ]]; then
echo "${client_semver}" > config/VERSION
use_version="$client_semver"
fi
echo " Packaging all-in-one-full artifact version: $use_version"
if ! bash scripts/package_artifact.sh --force; then
echo "❌ package_artifact.sh failed" >&2
# restore VERSION if changed
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
return 1
fi
local artifact_dir="$aio_root/artifact/$use_version"
local artifact_tar
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
if [[ -z "$artifact_tar" ]]; then
echo " No argus-metric_*.tar.gz found; invoking publish_artifact.sh to assemble..."
local owner="$(id -u):$(id -g)"
if ! bash scripts/publish_artifact.sh "$use_version" --output-dir "$artifact_dir" --owner "$owner"; then
echo "❌ publish_artifact.sh failed" >&2
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
return 1
fi
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
fi
if [[ -z "$artifact_tar" ]]; then
echo "❌ artifact tar not found under $artifact_dir" >&2
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
return 1
fi
# restore VERSION if changed (keep filesystem clean)
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
# 4) Stage docker build context
local bundle_ctx="$root/src/bundle/gpu-node-bundle/.build-$date_tag"
echo "\n🧰 Staging docker build context: $bundle_ctx"
rm -rf "$bundle_ctx"
mkdir -p "$bundle_ctx/bundle" "$bundle_ctx/private"
cp "$root/src/bundle/gpu-node-bundle/Dockerfile" "$bundle_ctx/"
cp "$root/src/bundle/gpu-node-bundle/node-bootstrap.sh" "$bundle_ctx/"
cp "$root/src/bundle/gpu-node-bundle/health-watcher.sh" "$bundle_ctx/"
# bundle tar
cp "$artifact_tar" "$bundle_ctx/bundle/"
# offline fluent-bit assets (optional but useful)
if [[ -d "$root/src/log/fluent-bit/build/etc" ]]; then
cp -r "$root/src/log/fluent-bit/build/etc" "$bundle_ctx/private/"
fi
if [[ -d "$root/src/log/fluent-bit/build/packages" ]]; then
cp -r "$root/src/log/fluent-bit/build/packages" "$bundle_ctx/private/"
fi
if [[ -f "$root/src/log/fluent-bit/build/start-fluent-bit.sh" ]]; then
cp "$root/src/log/fluent-bit/build/start-fluent-bit.sh" "$bundle_ctx/private/"
fi
# 5) Build the final bundle image (directly from NVIDIA base)
local image_tag="argus-sys-metric-test-node-bundle-gpu:${date_tag}"
echo "\n🔄 Building GPU Bundle image"
if build_image "GPU Bundle" "$bundle_ctx/Dockerfile" "$image_tag" "$bundle_ctx" \
--build-arg CUDA_VER="$(echo "$base_image" | sed -E 's#^nvidia/cuda:([0-9]+\.[0-9]+\.[0-9]+)-runtime-ubuntu22\.04$#\1#')" \
--build-arg CLIENT_VER="$use_version" \
--build-arg BUNDLE_DATE="$date_tag"; then
images_built+=("$image_tag")
# In non-pkg mode, also tag latest for convenience
if [[ "${ARGUS_PKG_BUILD:-0}" != "1" ]]; then
docker tag "$image_tag" argus-sys-metric-test-node-bundle-gpu:latest >/dev/null 2>&1 || true
fi
return 0
else
return 1
fi
}
# Tag helper: ensure :<date_tag> exists for a list of repos
ensure_version_tags() {
local date_tag="$1"; shift
local repos=("$@")
for repo in "${repos[@]}"; do
if docker image inspect "$repo:$date_tag" >/dev/null 2>&1; then
:
elif docker image inspect "$repo:latest" >/dev/null 2>&1; then
docker tag "$repo:latest" "$repo:$date_tag" || true
else
echo "❌ missing image for tagging: $repo (need :latest or :$date_tag)" >&2
return 1
fi
done
return 0
}
# Build server package after images are built
build_server_pkg_bundle() {
local date_tag="$1"
if [[ -z "$date_tag" ]]; then
echo "❌ server_pkg requires --version YYMMDD" >&2
return 1
fi
local repos=(
argus-bind9 argus-master argus-elasticsearch argus-kibana \
argus-metric-ftp argus-metric-prometheus argus-metric-grafana \
argus-alertmanager argus-web-frontend argus-web-proxy
)
echo "\n🔖 Verifying server images with :$date_tag and collecting digests"
for repo in "${repos[@]}"; do
if ! docker image inspect "$repo:$date_tag" >/dev/null 2>&1; then
echo "❌ required image missing: $repo:$date_tag (build phase should have produced it)" >&2
return 1
fi
done
# Optional: show digests
for repo in "${repos[@]}"; do
local digest
digest=$(docker images --digests --format '{{.Repository}}:{{.Tag}} {{.Digest}}' | awk -v r="$repo:$date_tag" '$1==r{print $2}' | head -n1)
printf ' • %s@%s\n' "$repo:$date_tag" "${digest:-<none>}"
done
echo "\n📦 Building server package via deployment_new/build/make_server_package.sh --version $date_tag"
if ! "$root/deployment_new/build/make_server_package.sh" --version "$date_tag"; then
echo "❌ make_server_package.sh failed" >&2
return 1
fi
return 0
}
# Build client package: ensure gpu bundle image exists, then package client_gpu
build_client_pkg_bundle() {
local date_tag="$1"
local semver="$2"
local cuda="$3"
if [[ -z "$date_tag" ]]; then
echo "❌ client_pkg requires --version YYMMDD" >&2
return 1
fi
local bundle_tag="argus-sys-metric-test-node-bundle-gpu:${date_tag}"
if ! docker image inspect "$bundle_tag" >/dev/null 2>&1; then
echo "\n🧩 GPU bundle image $bundle_tag missing; building it first..."
ARGUS_PKG_BUILD=1
export ARGUS_PKG_BUILD
if ! build_gpu_bundle_image "$date_tag" "$cuda" "$semver"; then
return 1
fi
else
echo "\n✅ Using existing GPU bundle image: $bundle_tag"
fi
echo "\n📦 Building client GPU package via deployment_new/build/make_client_gpu_package.sh --version $date_tag --image $bundle_tag"
if ! "$root/deployment_new/build/make_client_gpu_package.sh" --version "$date_tag" --image "$bundle_tag"; then
echo "❌ make_client_gpu_package.sh failed" >&2
return 1
fi
return 0
}
# Build CPU bundle image directly FROM ubuntu:22.04 (no intermediate base)
build_cpu_bundle_image() {
local date_tag="$1" # e.g. 20251113
local client_ver_in="$2" # semver like 1.43.0 (optional)
local want_tag_latest="$3" # true/false
if [[ -z "$date_tag" ]]; then
echo "❌ cpu_bundle requires --version YYMMDD" >&2
return 1
fi
echo "\n🔧 Preparing one-click CPU bundle build"
echo " Base: ubuntu:22.04"
echo " Bundle tag: ${date_tag}"
# 1) Build latest argus-agent from source
echo "\n🛠 Building argus-agent from src/agent"
pushd "$root/src/agent" >/dev/null
if ! bash scripts/build_binary.sh; then
echo "❌ argus-agent build failed" >&2
popd >/dev/null
return 1
fi
if [[ ! -f "dist/argus-agent" ]]; then
echo "❌ argus-agent binary missing after build" >&2
popd >/dev/null
return 1
fi
popd >/dev/null
# 2) Inject agent into all-in-one-full plugin and package artifact
local aio_root="$root/src/metric/client-plugins/all-in-one-full"
local agent_bin_src="$root/src/agent/dist/argus-agent"
local agent_bin_dst="$aio_root/plugins/argus-agent/bin/argus-agent"
echo "\n📦 Updating all-in-one-full agent binary → $agent_bin_dst"
cp -f "$agent_bin_src" "$agent_bin_dst"
chmod +x "$agent_bin_dst" || true
pushd "$aio_root" >/dev/null
local prev_version use_version
prev_version="$(cat config/VERSION 2>/dev/null || echo "1.0.0")"
use_version="$prev_version"
if [[ -n "$client_ver_in" ]]; then
echo "$client_ver_in" > config/VERSION
use_version="$client_ver_in"
fi
echo " Packaging all-in-one-full artifact: version=$use_version"
if ! bash scripts/package_artifact.sh --force; then
echo "❌ package_artifact.sh failed" >&2
[[ -n "$client_ver_in" ]] && echo "$prev_version" > config/VERSION
popd >/dev/null
return 1
fi
local artifact_dir="$aio_root/artifact/$use_version"
local artifact_tar
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
if [[ -z "$artifact_tar" ]]; then
echo " No argus-metric_*.tar.gz found; invoking publish_artifact.sh ..."
local owner="$(id -u):$(id -g)"
if ! bash scripts/publish_artifact.sh "$use_version" --output-dir "$artifact_dir" --owner "$owner"; then
echo "❌ publish_artifact.sh failed" >&2
[[ -n "$client_ver_in" ]] && echo "$prev_version" > config/VERSION
popd >/dev/null
return 1
fi
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
fi
[[ -n "$client_ver_in" ]] && echo "$prev_version" > config/VERSION
popd >/dev/null
# 3) Stage docker build context
local bundle_ctx="$root/src/bundle/cpu-node-bundle/.build-$date_tag"
echo "\n🧰 Staging docker build context: $bundle_ctx"
rm -rf "$bundle_ctx"
mkdir -p "$bundle_ctx/bundle" "$bundle_ctx/private"
cp "$root/src/bundle/cpu-node-bundle/Dockerfile" "$bundle_ctx/"
cp "$root/src/bundle/cpu-node-bundle/node-bootstrap.sh" "$bundle_ctx/"
cp "$root/src/bundle/cpu-node-bundle/health-watcher.sh" "$bundle_ctx/"
# bundle tar
cp "$artifact_tar" "$bundle_ctx/bundle/"
# offline fluent-bit assets
if [[ -d "$root/src/log/fluent-bit/build/etc" ]]; then
cp -r "$root/src/log/fluent-bit/build/etc" "$bundle_ctx/private/"
fi
if [[ -d "$root/src/log/fluent-bit/build/packages" ]]; then
cp -r "$root/src/log/fluent-bit/build/packages" "$bundle_ctx/private/"
fi
if [[ -f "$root/src/log/fluent-bit/build/start-fluent-bit.sh" ]]; then
cp "$root/src/log/fluent-bit/build/start-fluent-bit.sh" "$bundle_ctx/private/"
fi
# 4) Build final bundle image
local image_tag="argus-sys-metric-test-node-bundle:${date_tag}"
echo "\n🔄 Building CPU Bundle image"
if build_image "CPU Bundle" "$bundle_ctx/Dockerfile" "$image_tag" "$bundle_ctx"; then
images_built+=("$image_tag")
if [[ "$want_tag_latest" == "true" ]]; then
docker tag "$image_tag" argus-sys-metric-test-node-bundle:latest >/dev/null 2>&1 || true
fi
return 0
else
return 1
fi
}
if [[ "$build_core" == true ]]; then
if build_image "Elasticsearch" "src/log/elasticsearch/build/Dockerfile" "argus-elasticsearch:${DEFAULT_IMAGE_TAG}"; then
images_built+=("argus-elasticsearch:${DEFAULT_IMAGE_TAG}")
else
build_failed=true
fi
echo ""
if build_image "Kibana" "src/log/kibana/build/Dockerfile" "argus-kibana:${DEFAULT_IMAGE_TAG}"; then
images_built+=("argus-kibana:${DEFAULT_IMAGE_TAG}")
else
build_failed=true
fi
echo ""
if build_image "BIND9" "src/bind/build/Dockerfile" "argus-bind9:${DEFAULT_IMAGE_TAG}"; then
images_built+=("argus-bind9:${DEFAULT_IMAGE_TAG}")
else
build_failed=true
fi
fi
echo ""
if [[ "$build_master" == true ]]; then
echo ""
echo "🔄 Building Master image..."
pushd "$master_root" >/dev/null
master_args=("--tag" "argus-master:${DEFAULT_IMAGE_TAG}")
if [[ "$use_intranet" == true ]]; then
master_args+=("--intranet")
fi
if [[ "$build_master_offline" == true ]]; then
master_args+=("--offline")
fi
if [[ "$no_cache" == true ]]; then
master_args+=("--no-cache")
fi
if ./scripts/build_images.sh "${master_args[@]}"; then
if [[ "$build_master_offline" == true ]]; then
images_built+=("argus-master:offline")
else
images_built+=("argus-master:${DEFAULT_IMAGE_TAG}")
fi
else
build_failed=true
fi
popd >/dev/null
fi
if [[ "$build_metric" == true ]]; then
echo ""
echo "Building Metric module images..."
metric_base_images=(
"ubuntu:22.04"
"ubuntu/prometheus:3-24.04_stable"
"grafana/grafana:11.1.0"
)
for base_image in "${metric_base_images[@]}"; do
if ! pull_base_image "$base_image"; then
build_failed=true
fi
done
metric_builds=(
"Metric FTP|src/metric/ftp/build/Dockerfile|argus-metric-ftp:${DEFAULT_IMAGE_TAG}|src/metric/ftp/build"
"Metric Prometheus|src/metric/prometheus/build/Dockerfile|argus-metric-prometheus:${DEFAULT_IMAGE_TAG}|src/metric/prometheus/build"
"Metric Grafana|src/metric/grafana/build/Dockerfile|argus-metric-grafana:${DEFAULT_IMAGE_TAG}|src/metric/grafana/build"
"Metric Prometheus Targets Updater|src/metric/prometheus/build/Dockerfile.targets-updater|argus-metric-prometheus-targets-updater:${DEFAULT_IMAGE_TAG}|src/metric/prometheus/build"
)
for build_spec in "${metric_builds[@]}"; do
IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec"
if build_image "$image_label" "$dockerfile_path" "$image_tag" "$build_context"; then
images_built+=("$image_tag")
else
build_failed=true
fi
echo ""
done
fi
# =======================================
# Sys (system tests) node images
# =======================================
if [[ "$build_sys" == true ]]; then
echo ""
echo "Building Sys node images..."
sys_base_images=(
"ubuntu:22.04"
"nvidia/cuda:12.2.2-runtime-ubuntu22.04"
)
for base_image in "${sys_base_images[@]}"; do
if ! pull_base_image "$base_image"; then
build_failed=true
fi
done
sys_builds=(
"Sys Node|src/sys/build/node/Dockerfile|argus-sys-node:latest|."
"Sys Metric Test Node|src/sys/build/test-node/Dockerfile|argus-sys-metric-test-node:latest|."
"Sys Metric Test GPU Node|src/sys/build/test-gpu-node/Dockerfile|argus-sys-metric-test-gpu-node:latest|."
)
for build_spec in "${sys_builds[@]}"; do
IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec"
if build_image "$image_label" "$dockerfile_path" "$image_tag" "$build_context"; then
images_built+=("$image_tag")
else
build_failed=true
fi
echo ""
done
fi
# =======================================
# Web & Alert module images
# =======================================
if [[ "$build_web" == true || "$build_alert" == true ]]; then
echo ""
echo "Building Web and Alert module images..."
# Pre-pull commonly used base images for stability
web_alert_base_images=(
"node:20"
"ubuntu:24.04"
)
for base_image in "${web_alert_base_images[@]}"; do
if ! pull_base_image "$base_image"; then
build_failed=true
fi
done
if [[ "$build_web" == true ]]; then
web_builds=(
"Web Frontend|src/web/build_tools/frontend/Dockerfile|argus-web-frontend:${DEFAULT_IMAGE_TAG}|."
"Web Proxy|src/web/build_tools/proxy/Dockerfile|argus-web-proxy:${DEFAULT_IMAGE_TAG}|."
)
for build_spec in "${web_builds[@]}"; do
IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec"
if build_image "$image_label" "$dockerfile_path" "$image_tag" "$build_context"; then
images_built+=("$image_tag")
else
build_failed=true
fi
echo ""
done
fi
if [[ "$build_alert" == true ]]; then
alert_builds=(
"Alertmanager|src/alert/alertmanager/build/Dockerfile|argus-alertmanager:${DEFAULT_IMAGE_TAG}|."
)
for build_spec in "${alert_builds[@]}"; do
IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec"
if build_image "$image_label" "$dockerfile_path" "$image_tag" "$build_context"; then
images_built+=("$image_tag")
else
build_failed=true
fi
echo ""
done
fi
fi
# =======================================
# One-click GPU bundle (direct NVIDIA base)
# =======================================
if [[ "$build_gpu_bundle" == true ]]; then
echo ""
echo "Building one-click GPU bundle image..."
if ! build_gpu_bundle_image "$bundle_date" "$cuda_ver" "$client_semver"; then
build_failed=true
fi
fi
# =======================================
# One-click CPU bundle (from ubuntu:22.04)
# =======================================
if [[ "$build_cpu_bundle" == true ]]; then
echo ""
echo "Building one-click CPU bundle image..."
if ! build_cpu_bundle_image "${bundle_date}" "${client_semver}" "${tag_latest}"; then
build_failed=true
fi
fi
# =======================================
# One-click Server/Client packaging
# =======================================
if [[ "$build_server_pkg" == true ]]; then
echo ""
echo "🧳 Building one-click Server package..."
if ! build_server_pkg_bundle "${bundle_date}"; then
build_failed=true
fi
fi
if [[ "$build_client_pkg" == true ]]; then
echo ""
echo "🧳 Building one-click Client-GPU package..."
if ! build_client_pkg_bundle "${bundle_date}" "${client_semver}" "${cuda_ver}"; then
build_failed=true
fi
fi
echo "======================================="
echo "📦 Build Summary"
echo "======================================="
if [[ ${#images_built[@]} -gt 0 ]]; then
echo "✅ Successfully built images:"
for image in "${images_built[@]}"; do
echo " • $image"
done
fi
if [[ "$build_failed" == true ]]; then
echo ""
echo "❌ Some images failed to build. Please check the errors above."
exit 1
fi
if [[ "$use_intranet" == true ]]; then
echo ""
echo "🌐 Built with intranet mirror configuration"
fi
if [[ "$build_master_offline" == true ]]; then
echo ""
echo "🧳 Master offline wheels 已解压到 $master_offline_dir"
fi
echo ""
echo "🚀 Next steps:"
echo " ./build/save_images.sh --compress # 导出镜像"
echo " cd src/master/tests && MASTER_IMAGE_TAG=argus-master:offline ./scripts/00_e2e_test.sh"
echo ""

View File

@@ -0,0 +1,6 @@
# Default build-time UID/GID for Argus images
# Override by creating configs/build_user.local.conf with the same format.
# Syntax: KEY=VALUE, supports UID/GID only. Whitespace and lines starting with # are ignored.
UID=2133
GID=2015

View File

@@ -1,16 +0,0 @@
# Deployment Build Toolkit
This folder provides scripts to produce offline server/client packages and publish the client package to FTP.
Commands
- build_server_package.sh [--version YYYYMMDD]
- build_client_package.sh [--version YYYYMMDD]
- publish_client.sh --version YYYYMMDD --server <host> --user ftpuser --password <pass> [--port 21]
Outputs
- deployment/artifact/server/<YYYYMMDD>/
- deployment/artifact/client/<YYYYMMDD>/
Notes
- Server package contains docker images (single all-images.tar.gz), compose/, scripts/, docs/, private/ skeleton.
- Client package reuses all-in-one-full artifact, repacked as argus-metric_<YYYYMMDD>.tar.gz (compatible with setup.sh).

View File

@@ -1,90 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
BUILD_DIR="$ROOT_DIR/deployment/build"
ART_ROOT="$ROOT_DIR/deployment/artifact"
. "$BUILD_DIR/common.sh"
usage() { cat <<'EOF'
Build Argus Client Offline Package
Usage: build_client_package.sh [--version YYYYMMDD] [--out DIR]
Produces: deployment/artifact/client/<YYYYMMDD>/argus-metric_<YYYYMMDD>.tar.gz
EOF
}
VERSION="$(today_version)"
OUT_DIR=""
while [[ $# -gt 0 ]]; do
case "$1" in
--version) VERSION="$2"; shift 2;;
--out) OUT_DIR="$2"; shift 2;;
-h|--help) usage; exit 0;;
*) err "unknown arg: $1"; usage; exit 1;;
esac
done
PKG_DIR="${OUT_DIR:-$ART_ROOT/client/$VERSION}"
make_dir "$PKG_DIR"
log "Packaging client from all-in-one-full artifact"
PLUGIN_DIR="$ROOT_DIR/src/metric/client-plugins/all-in-one-full"
require_cmd bash tar gzip
(cd "$PLUGIN_DIR" && bash scripts/package_artifact.sh --force)
# pick latest artifact dir
ART_BASE="$PLUGIN_DIR/artifact"
latest_dir=$(ls -1dt "$ART_BASE"/*/ 2>/dev/null | head -n1 || true)
[[ -n "$latest_dir" ]] || { err "no client artifact found in $ART_BASE"; exit 1; }
tmpdir=$(mktemp -d)
trap 'rm -rf "$tmpdir"' EXIT
# Filter-only copy: keep install_order files + scripts + deps + version.json
mkdir -p "$tmpdir/src"
cp -f "$latest_dir/version.json" "$tmpdir/src/version.json"
if command -v jq >/dev/null 2>&1; then
mapfile -t files < <(jq -r '.install_order[]' "$latest_dir/version.json")
else
files=( $(grep -E '"install_order"' -A10 "$latest_dir/version.json" | sed -n 's/\s*"\(.*\.tar\.gz\)".*/\1/p') )
fi
for f in "${files[@]}"; do
[[ -f "$latest_dir/$f" ]] && cp -f "$latest_dir/$f" "$tmpdir/src/$f"
done
for aux in install.sh uninstall.sh check_health.sh check_version.sh sync_dns.sh restart_unhealthy.sh config.env; do
[[ -f "$latest_dir/$aux" ]] && cp -f "$latest_dir/$aux" "$tmpdir/src/$aux";
done
if [[ -d "$latest_dir/deps" ]]; then
mkdir -p "$tmpdir/src/deps" && rsync -a "$latest_dir/deps/" "$tmpdir/src/deps/" >/dev/null 2>&1 || cp -r "$latest_dir/deps/." "$tmpdir/src/deps/";
fi
out_name="argus-metric_$(echo "$VERSION" | sed 's/\./_/g').tar.gz"
(cd "$tmpdir/src" && tar -czf "$PKG_DIR/$out_name" .)
log "Client package ready: $PKG_DIR/$out_name"
echo "$VERSION" > "$PKG_DIR/LATEST_VERSION"
# include publish helper and setup.sh for convenience (place first)
PUBLISH_TPL="$BUILD_DIR/templates/client/publish.sh"
if [[ -f "$PUBLISH_TPL" ]]; then
cp "$PUBLISH_TPL" "$PKG_DIR/publish.sh"
fi
# also place a copy of setup.sh alongside
SETUP_SRC="$ROOT_DIR/src/metric/client-plugins/all-in-one-full/scripts/setup.sh"
[[ -f "$SETUP_SRC" ]] && cp "$SETUP_SRC" "$PKG_DIR/setup.sh" || true
# docs for end users (this may overwrite file modes), then fix execute bits
CLIENT_DOC_DIR="$BUILD_DIR/templates/client"
if [[ -d "$CLIENT_DOC_DIR" ]]; then
rsync -a "$CLIENT_DOC_DIR/" "$PKG_DIR/" >/dev/null 2>&1 || cp -r "$CLIENT_DOC_DIR/." "$PKG_DIR/"
fi
# ensure helpers are executable
chmod +x "$PKG_DIR/publish.sh" "$PKG_DIR/setup.sh" 2>/dev/null || true
exit 0

View File

@@ -1,138 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
BUILD_DIR="$ROOT_DIR/deployment/build"
ART_ROOT="$ROOT_DIR/deployment/artifact"
. "$BUILD_DIR/common.sh"
usage() { cat <<'EOF'
Build Argus Server Offline Package
Usage: build_server_package.sh [--version YYYYMMDD] [--out DIR] [--resave-image]
Outputs into deployment/artifact/server/<YYYYMMDD>/ by default.
EOF
}
VERSION="$(today_version)"
OUT_DIR=""
RESAVE_IMAGE=false
while [[ $# -gt 0 ]]; do
case "$1" in
--version) VERSION="$2"; shift 2;;
--out) OUT_DIR="$2"; shift 2;;
--resave-image) RESAVE_IMAGE=true; shift;;
-h|--help) usage; exit 0;;
*) err "unknown arg: $1"; usage; exit 1;;
esac
done
PKG_DIR="${OUT_DIR:-$ART_ROOT/server/$VERSION}"
STAGE="$(mktemp -d)"
trap 'rm -rf "$STAGE"' EXIT
log "Version: $VERSION"
log "Staging: $STAGE"
# 1) Layout
make_dir "$STAGE/images"
make_dir "$STAGE/compose"
make_dir "$STAGE/scripts"
make_dir "$STAGE/docs"
make_dir "$STAGE/private/argus"
# 2) Compose: derive from sys/tests by removing test-only services
SRC_COMPOSE="$ROOT_DIR/src/sys/tests/docker-compose.yml"
[[ -f "$SRC_COMPOSE" ]] || { err "missing $SRC_COMPOSE"; exit 1; }
# 2.1 filter out test services
tmp_compose1="$STAGE/compose/docker-compose.filtered.yml"
awk -f "$BUILD_DIR/templates/docker-compose.filter.awk" -v remove="node-a,node-b,test-node,test-gpu-node" "$SRC_COMPOSE" > "$tmp_compose1"
# 2.2 transform to external overlay network (remove sysnet and per-service blocks)
awk -f "$BUILD_DIR/templates/docker-compose.overlay.awk" "$tmp_compose1" > "$STAGE/compose/docker-compose.yml"
cp "$BUILD_DIR/templates/.env.example" "$STAGE/compose/.env.example"
# fix relative private path to match package layout (compose/ and private/ are siblings)
sed -i 's#\./private/#../private/#g' "$STAGE/compose/docker-compose.yml"
# also handle bind mount form without trailing slash
sed -i -E 's#- \./private:/private#- ../private:/private#g' "$STAGE/compose/docker-compose.yml"
# drop timezone file bind which may not exist on target distros (e.g. NixOS)
sed -i '/\/etc\/timezone:\/etc\/timezone:ro/d' "$STAGE/compose/docker-compose.yml"
# sanity-check: ensure test services are absent and external network present
if grep -E '^(\s{2})(node-a|node-b|test-node|test-gpu-node):\s*$' -n "$STAGE/compose/docker-compose.yml" >/dev/null; then
err "compose filter failed: test services still present"; exit 1;
fi
if ! grep -q '^networks:' "$STAGE/compose/docker-compose.yml" || ! grep -q 'argus-sys-net:' "$STAGE/compose/docker-compose.yml"; then
err "compose overlay transform failed: external network missing"; exit 1;
fi
# 3) Images (reuse if already exported unless --resave-image)
existing_images_tar="$PKG_DIR/images/all-images.tar.gz"
if [[ "$RESAVE_IMAGE" == false && -f "$existing_images_tar" ]]; then
log "Reusing existing images tar: $existing_images_tar"
cp "$existing_images_tar" "$STAGE/images/"
elif [[ "$RESAVE_IMAGE" == false ]]; then
# Try cross-version reuse from latest server_*.tar.gz
latest_pkg=$(ls -1t "$ART_ROOT/server"/server_*.tar.gz 2>/dev/null | head -n1 || true)
if [[ -n "$latest_pkg" ]]; then
log "Reusing images from: $latest_pkg"
mkdir -p "$STAGE/images"
# extract matching file regardless of top-level dir
if tar -xzf "$latest_pkg" -C "$STAGE" --wildcards '*/images/all-images.tar.gz' 2>/dev/null; then
# locate and move
found=$(find "$STAGE" -type f -path '*/images/all-images.tar.gz' | head -n1 || true)
if [[ -n "$found" ]]; then
mv "$found" "$STAGE/images/all-images.tar.gz"
# cleanup leftover extracted dir
dir_to_clean=$(dirname "$found")
rm -rf "${dir_to_clean%/images}" 2>/dev/null || true
fi
fi
fi
fi
# If still not present, save from local docker daemon
if [[ ! -f "$STAGE/images/all-images.tar.gz" ]]; then
require_cmd docker gzip
images=(
argus-bind9:latest
argus-master:latest
argus-elasticsearch:latest
argus-kibana:latest
argus-metric-ftp:latest
argus-metric-prometheus:latest
argus-metric-grafana:latest
argus-alertmanager:latest
argus-web-frontend:latest
argus-web-proxy:latest
)
log "Saving images: ${#images[@]}"
tarfile="$STAGE/images/all-images.tar"
docker save -o "$tarfile" "${images[@]}"
gzip -f "$tarfile"
fi
# 4) Scripts & Docs
copy_tree "$BUILD_DIR/templates/scripts" "$STAGE/scripts"
copy_tree "$BUILD_DIR/templates/docs" "$STAGE/docs"
# 5) Manifests
gen_manifest "$STAGE" "$STAGE/manifest.txt"
checksum_dir "$STAGE" "$STAGE/checksums.txt"
# 6) Move to artifact
make_dir "$PKG_DIR"
rsync -a "$STAGE/" "$PKG_DIR/" 2>/dev/null || cp -r "$STAGE/." "$PKG_DIR/"
log "Server package ready: $PKG_DIR"
echo "$VERSION" > "$PKG_DIR/version.json"
# 7) Create distributable tarball
OUT_TAR_DIR="$(dirname "$PKG_DIR")"
OUT_TAR="$OUT_TAR_DIR/server_${VERSION}.tar.gz"
log "Creating tarball: $OUT_TAR"
(cd "$PKG_DIR/.." && tar -czf "$OUT_TAR" "$(basename "$PKG_DIR")")
log "Tarball ready: $OUT_TAR"
exit 0

View File

@@ -1,32 +0,0 @@
# UID/GID for service processes
ARGUS_BUILD_UID=1000
ARGUS_BUILD_GID=1000
# Host ports (adjust if occupied)
MASTER_PORT=32300
ES_HTTP_PORT=9200
KIBANA_PORT=5601
NODE_A_PORT=2020
NODE_B_PORT=2021
PROMETHEUS_PORT=9090
GRAFANA_PORT=3000
ALERTMANAGER_PORT=9093
WEB_PROXY_PORT_8080=8080
WEB_PROXY_PORT_8081=8081
WEB_PROXY_PORT_8082=8082
WEB_PROXY_PORT_8083=8083
WEB_PROXY_PORT_8084=8084
WEB_PROXY_PORT_8085=8085
# FTP
FTP_PORT=21
FTP_DATA_PORT=20
FTP_PASSIVE_HOST_RANGE=21100-21110
FTP_PASSWORD=ZGClab1234!
FTP_DOMAIN=ftp.metric.argus.com
# GPU profile disabled by default
ENABLE_GPU=false
# External overlay network (Swarm attachable)
OVERLAY_NET_NAME=argus-sys-net

View File

@@ -1,44 +0,0 @@
# Argus Metric 客户端安装指南(容器内普通用户场景)
## 准备与连通性检查
- FTP 连接(需要账号密码,默认 `ftpuser / ZGClab1234!`)
- `curl -u ftpuser:ZGClab1234! -I ftp://<FTP_IP>:21/LATEST_VERSION`
- `curl -u ftpuser:ZGClab1234! -s ftp://<FTP_IP>:21/ | head`
- 下载安装脚本
- `curl -u ftpuser:ZGClab1234! -fsSL ftp://<FTP_IP>:21/setup.sh -o /tmp/setup.sh`
- `chmod +x /tmp/setup.sh`
## 元数据与主机名
- Agent 需要元数据(env/user/instance)与 Master 地址:
- 方式A:hostname 形如 `env-user-instance-xxx`(推荐)
- 方式B:导出环境变量:
- `export AGENT_ENV=dev`
- `export AGENT_USER=<your_user>`
- `export AGENT_INSTANCE=<node_id>`
- Master 地址:
- `export MASTER_ENDPOINT=http://master.argus.com:3000`
> 安装脚本会在开头检查上述条件,缺失时会终止并给出修复提示。
## 执行安装
- 以 root 运行(容器内如为非 root 用户请切换为 root):
- `sudo /tmp/setup.sh --server <FTP_IP> --user ftpuser --password 'ZGClab1234!' --port 21`
- 如需自定义安装根目录:`--install-dir /opt/argus-metric`
提示(容器接入 overlay 网络时):
- 在执行 setup 前,先将容器内 DNS 指向 Bind9 的 overlay IP:
- `echo "nameserver <BIND_OVERLAY_IP>" > /etc/resolv.conf`
- 这样 `master.argus.com`、`es.log.argus.com` 等域名即可解析;首次下载 `setup.sh` 仍建议使用 FTP 的 overlay IP。
更多快速步骤请参考:`QUICK_NODE_DEPLOY_zh.md`。
## 安装后自检(setup 自动执行)
- setup 会等待最多 5 分钟,确认以下条件后才报告完成:
- `/private/argus/agent/<hostname>/node.json` 已生成;
- `last_report` 在持续更新;
- `health.metric-argus-agent|metric-node-exporter|metric-fluent-bit|metric-dcgm-exporter` 均为 `healthy` 且 `error` 为空。
## 手工验证(可选)
- `cat /private/argus/agent/$(hostname)/node.json | jq '.'`
- `curl -s -o /dev/null -w "%{http_code}\n" http://localhost:9100/metrics` 应为 200
- 查看日志:`/var/log/argus-agent.log`、`/opt/argus-metric/versions/*/.install.log`

View File

@@ -1,57 +0,0 @@
# Argus Metric 客户端发布说明(FTP)
本说明面向“发布人员”,讲清楚如何把客户端离线包发布到 FTP,供各节点通过 `curl` 自动安装。
## 目录结构(构建后)
- `client-YYYYMMDD/`
- `argus-metric_YYYYMMDD.tar.gz` 客户端离线包
- `setup.sh` 客户端安装入口脚本(提供给节点用 curl 下载)
- `publish.sh` 发布脚本(将上述两项与 `LATEST_VERSION` 上传到 FTP)
- `LATEST_VERSION` 文本(内容为 `YYYYMMDD`,或 `YYYYMMDD-rN`)
- `INSTALL_CLIENT_zh.md` 本地安装指南(给使用者看,不会上载到 FTP)
- `PUBLISH_CLIENT_zh.md` 本说明
> 注意:`publish.sh`/`setup.sh` 为可执行脚本;构建脚本已保证二者具有执行权限。
## 前置条件
- FTP 服务已运行(默认容器:`argus-ftp`),并打开端口:21、20、21100–21110(被动模式)。
- FTP 账号:默认 `ftpuser / ZGClab1234!`(如有更改,以实际为准)。
## 发布步骤(在 server 机器或能直连 FTP 的任意机器上)
1) 进入发布目录:
- `cd client-YYYYMMDD`
2) 执行发布:
- `./publish.sh --server <FTP_HOST> --user <USER> --password '<PASS>' [--port 21]`
- 例如在服务端本机:`./publish.sh --server localhost --user ftpuser --password 'ZGClab1234!' --port 21`
脚本会上传三类文件到 FTP 根:
- `setup.sh`
- `argus-metric_YYYYMMDD[ -rN ].tar.gz`
- `LATEST_VERSION`(内容为当前版本号)
3) 发布后验证:
- `curl -u ftpuser:****** -I ftp://<FTP_HOST>:21/LATEST_VERSION` 应返回 200
- `curl -u ftpuser:****** -fsSL ftp://<FTP_HOST>:21/LATEST_VERSION` 内容为版本号(如 `20251104`)
- `curl -u ftpuser:****** -I ftp://<FTP_HOST>:21/argus-metric_YYYYMMDD.tar.gz` 返回 200
## 节点侧使用方式(摘要)
- 首次下载用 FTP 的“IP 地址”:
- `curl -u ftpuser:****** -fsSL ftp://<FTP_IP>:21/setup.sh -o /tmp/setup.sh && chmod +x /tmp/setup.sh`
- 执行安装:
- 必需元数据:`AGENT_ENV/AGENT_USER/AGENT_INSTANCE`,以及 `MASTER_ENDPOINT=http://master.argus.com:3000`
- `sudo /tmp/setup.sh --server <FTP_IP> --user ftpuser --password '******' --port 21`
- overlay 容器场景:
- 先将容器内 DNS 指向 Bind9 的 overlay IP:`echo "nameserver <BIND_OVERLAY_IP>" > /etc/resolv.conf`
- 然后再执行上述安装;安装后约 1–2 分钟内 DNS 即可解析 `*.argus.com` 域名。
## 常见问题
- `530 Access denied`:用户名/密码错误或 FTP 目录无权限;请核对账号与 FTP 容器状态。
- `Permission denied` 执行 `publish.sh`:为脚本权限问题;`chmod +x publish.sh`。构建脚本已修复默认权限。
- 被动端口不通导致失败:请开放 21100–21110。
- 客户端安装后短时 `curl http://master.argus.com:3000` 为 000:服务冷启动或 DNS 同步延迟,等待 1–2 分钟再试。
## 版本与回滚
- `LATEST_VERSION` 决定客户端默认安装的版本号。
- 如需回滚:将旧版本号写回 `LATEST_VERSION` 并重新发布(或手动指定 `--version` 安装)。

View File

@@ -1,58 +0,0 @@
# Argus Metric 节点快速部署(Overlay 网络容器)
本文档给出在 Docker Swarm external overlay 网络中,快速拉起一个测试节点并完成注册的最小可行步骤。
## 前提
- 服务端已在 Manager 机安装完成并运行良好(`server-selfcheck` 通过)。
- Overlay 网络名称:`argus-sys-net`(默认)。
- 已通过 FTP 发布 `setup.sh` 与客户端包,且能从 FTP 获取 `LATEST_VERSION`。
- 用于测试的镜像:`argus-sys-metric-test-node:latest` 已存在于目标机器。
## 步骤
- 获取 FTP 和 Bind 的 overlay IP(在 Manager 上执行)
- `FTPIP=$(docker inspect -f '{{ (index .NetworkSettings.Networks "argus-sys-net").IPAddress }}' argus-ftp)`
- `BINDIP=$(docker inspect -f '{{ (index .NetworkSettings.Networks "argus-sys-net").IPAddress }}' argus-bind-sys)`
- `echo "FTP=$FTPIP BIND=$BINDIP"`
- 准备宿主挂载目录(以 s4 为例)
- `mkdir -p /home2/yuyr/deploy/test-metric-node/s4`
- 启动测试节点容器(接入 overlay)
- `docker run -d --name argus-metric-test-node-s4 \
--hostname dev2-yuyr-node002s4 \
--network argus-sys-net \
-v /home2/yuyr/deploy/test-metric-node/s4:/private/argus/agent \
argus-sys-metric-test-node:latest sleep infinity`
- 在容器内执行安装(先用 FTP IP 引导,DNS 指向 Bind)
- `docker exec -it argus-metric-test-node-s4 bash`
- `echo "nameserver $BINDIP" > /etc/resolv.conf`
- `curl --ftp-method nocwd -u ftpuser:ZGClab1234! -fsSL "ftp://$FTPIP:21/setup.sh" -o /tmp/setup.sh`
- `chmod +x /tmp/setup.sh`
- `export AGENT_ENV=dev2 AGENT_USER=yuyr AGENT_INSTANCE=node002s4`
- `export MASTER_ENDPOINT=http://master.argus.com:3000`
- `/tmp/setup.sh --server "$FTPIP" --user ftpuser --password 'ZGClab1234!' --port 21`
- 说明:setup 会自动执行安装后自检(最多 5 分钟),无需手动轮询。
## 验证(推荐在容器内执行,避免宿主权限问题)
- 查看 node.json 关键字段
- `cat /private/argus/agent/dev2-yuyr-node002s4/node.json | jq '{last_report, health}'`
- 期望:四个 health 全部 healthy;等待 ≥70s 再查看,`last_report` 持续更新。
- 指标端口
- `curl -s -o /dev/null -w '%{http_code}\n' http://localhost:9100/metrics`(期望 200)
- (如测试 GPU)`curl -s -o /dev/null -w '%{http_code}\n' http://localhost:9400/metrics`(有 GPU 时 200)
- 与服务端连通(域名经 Bind 解析)
- `curl -s -o /dev/null -w '%{http_code}\n' http://master.argus.com:3000/readyz`(期望 200)
- `curl -s -o /dev/null -w '%{http_code}\n' http://es.log.argus.com:9200/_cluster/health`(期望 200)
## (可选)在服务器主机侧观察 Prometheus 目标更新
- `cat /home2/yuyr/deploy/versions/<VERSION>/private/argus/metric/prometheus/nodes.json | jq '.'`
## 常见提示
- 初次安装后短时 `curl` 域名返回 000/超时属正常,多等待 1–2 分钟 DNS 同步/组件冷启动完成。
- 如在宿主直接读取挂载的 node.json 报 Permission denied,请使用 `docker exec` 在容器内查看。
- MASTER_ENDPOINT 固定使用域名 `http://master.argus.com:3000`,客户端无需固定 IP。

View File

@@ -1,54 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
usage() { cat <<'EOF'
Publish Argus client package to FTP
Usage:
./publish.sh --server HOST --user USER --password PASS [--port 21]
Notes:
- This script expects to run inside the built client artifact directory.
- It reads LATEST_VERSION and uploads setup.sh, argus-metric_<ver>.tar.gz, and LATEST_VERSION.
EOF
}
HOST=""; USERNAME=""; PASSWORD=""; PORT=21
while [[ $# -gt 0 ]]; do
case "$1" in
--server) HOST="$2"; shift 2;;
--user) USERNAME="$2"; shift 2;;
--password) PASSWORD="$2"; shift 2;;
--port) PORT="$2"; shift 2;;
-h|--help) usage; exit 0;;
*) echo "unknown arg: $1" >&2; usage; exit 1;;
esac
done
[[ -n "$HOST" && -n "$USERNAME" && -n "$PASSWORD" ]] || { usage; exit 1; }
here="$(pwd)"
if [[ ! -f "$here/LATEST_VERSION" ]]; then
echo "LATEST_VERSION not found in $(pwd)" >&2; exit 1;
fi
VER=$(cat "$here/LATEST_VERSION" | tr -d '\n')
PKG="argus-metric_${VER}.tar.gz"
if [[ ! -f "$here/$PKG" ]]; then
echo "client tar not found: $PKG" >&2; exit 1
fi
# locate setup.sh (prefer colocated, fallback to bundled path if provided)
SETUP="${here}/setup.sh"
if [[ ! -f "$SETUP" ]]; then
echo "setup.sh not found in $(pwd)" >&2; exit 1
fi
echo "[PUBLISH] server=$HOST port=$PORT version=$VER"
curl -u "$USERNAME:$PASSWORD" -sfT "$SETUP" "ftp://$HOST:$PORT/setup.sh"
curl -u "$USERNAME:$PASSWORD" -sfT "$PKG" "ftp://$HOST:$PORT/$PKG"
printf "%s" "$VER" | curl -u "$USERNAME:$PASSWORD" -sfT - "ftp://$HOST:$PORT/LATEST_VERSION"
echo "[OK] publish completed"

View File

@@ -1,41 +0,0 @@
#!/usr/bin/awk -f
# Remove specific service blocks from a docker-compose.yml by service name.
# Usage: awk -f docker-compose.filter.awk -v remove="node-a,node-b,test-node,test-gpu-node" input.yml > output.yml
BEGIN{
split(remove, rm, ",");
for(i in rm){
gsub(/^\s+|\s+$/,"",rm[i]);
if (rm[i] != "") skipname[rm[i]] = 1;
}
in_services=0; skipping=0;
}
function service_header(line, m) {
# match exactly two leading spaces followed by name:
if (match(line, /^ ([A-Za-z0-9_-]+):[ ]*$/, m)) return m[1];
return "";
}
{
# Track top-level sections (no indentation)
if ($0 ~ /^[A-Za-z0-9_-]+:[ ]*$/ && $0 !~ /^\s/) {
in_services = ($0 ~ /^services:[ ]*$/) ? 1 : 0;
}
if (skipping) {
# Stop skipping at next service header or another top-level section
if (service_header($0) != "" || ($0 ~ /^(networks|volumes):[ ]*$/ && $0 !~ /^\s/)) {
skipping=0;
} else {
next;
}
}
if (in_services) {
name = service_header($0);
if (name != "" && (name in skipname)) { skipping=1; next; }
}
print;
}

View File

@@ -1,74 +0,0 @@
#!/usr/bin/awk -f
# Transform docker-compose.yml to use an external overlay network for all services
# - Remove top-level networks definition
# - Remove per-service networks block (including ipv4_address and sysnet refs)
# - Insert per-service networks: [argus-sys-net]
# - Append external networks mapping at the end
BEGIN{
in_top_networks=0; in_services=0; in_service=0; svc_indent=0; curr_name="";
}
function is_service_header(line){ return svc_name(line)!=""; }
function svc_name(line, m){ if (match(line, /^ ([A-Za-z0-9_-]+):[ ]*$/, m)) return m[1]; return ""; }
function indent_len(s, n){ n=match(s,/[^ ]/)-1; if(n<0) n=0; return n; }
{
# Detect entry into top-level sections
if ($0 ~ /^[A-Za-z0-9_-]+:[ ]*$/ && $0 !~ /^\s/) {
in_services = ($0 ~ /^services:[ ]*$/);
# If a new top-level section starts, stop skipping top networks
in_top_networks = 0;
}
# Handle removal of initial top-level 'networks:' block
if ($0 ~ /^networks:[ ]*$/ && $0 !~ /^\s/) {
in_top_networks = 1; next;
}
if (in_top_networks) {
# skip until next top-level section (non-indented key)
next;
}
if (in_services) {
# Track service boundaries
if (is_service_header($0)) {
in_service=1; svc_indent=2; networks_inserted=0; curr_name=svc_name($0); print; next;
}
if (in_service) {
# If line is indented <= service indent, we've left this service
if (indent_len($0) <= svc_indent && $0 !~ /^\s*$/) {
in_service=0;
}
}
if (in_service) {
# Skip any existing networks block under the service
if ($0 ~ /^\s{4}networks:[ ]*$/) { skipping_nets=1; next; }
if (skipping_nets) {
if (indent_len($0) <= 4) { skipping_nets=0; }
else next;
}
# After container_name or image, inject networks once
if (!networks_inserted && ($0 ~ /^\s{4}container_name:/ || $0 ~ /^\s{4}image:/)) {
print;
print " networks:";
print " - argus-sys-net";
networks_inserted=1; next;
}
# no host port injection; bind serves DNS inside overlay only
}
}
print;
}
END{
print "";
print "networks:";
print " argus-sys-net:";
print " external: true";
print " name: ${OVERLAY_NET_NAME:-argus-sys-net}";
}

View File

@@ -1,50 +0,0 @@
# Argus Server Offline Installation
## Prerequisites
- Linux x86_64 (Ubuntu 22.04 recommended; see OS compatibility for NixOS)
- Docker & Docker Compose installed
- Open ports: 32300, 9200, 5601, 9090, 9093, 8080..8085, 21, 20, 21100–21110 (or auto-fallback to high ports)
## Quick Start
1. Extract to a target dir, e.g. `/opt/argus-deploy/versions/<YYYYMMDD>`
2. `./server-install.sh` (non‑root is supported: it will precreate minimal dirs and auto-fix Kibana/ES/Bind in containers)
3. `./server-status.sh`
4. `./server-selfcheck.sh` (on failure it auto-runs diagnose and writes logs under `logs/`)
5. `./server-uninstall.sh` to tear down
## What the Installer Does
- Loads local images (`images/all-images.tar.gz`)
- Generates OS-compat override (`security_opt: ["label=disable"]`, `userns_mode: host`, bind `tmpfs:/run/named`)
- Starts server-only services: bind/master/es/kibana/ftp/prometheus/grafana/alertmanager/web-frontend/web-proxy
- DNS Bootstrap:
- Ensure `/private/argus/etc/dns.conf` exists (write `172.31.0.2` if missing);
- Run `/private/argus/etc/update-dns.sh` in dependent containers so `/etc/resolv.conf` points to bind;
- Wait for `*.argus.com` hint files, then reload bind;
- Restart web‑proxy to re-render nginx resolver from `dns.conf`;
- Writes `logs/selfcheck.json` as final summary
## OS Compatibility
- NixOS / non-xattr FS: containers run with `security_opt: ["label=disable"]` and `userns_mode: host`.
- If you cannot use sudo, the installer will:
- create minimal data dirs (incl. `private/argus/log/{elasticsearch,kibana}`) with permissive perms when possible;
- ensure inside containers: Kibana `data` → `/private/argus/log/kibana`, Elasticsearch `data` → `/private/argus/log/elasticsearch`, and Bind `rndc.key` is generated.
(Manual pre-creation scripts are no longer required.)
## Files & Layout
- `compose/` (docker-compose.yml, .env)
- `private/` (data mounts)
- `scripts/` (install/uninstall/status/selfcheck/diagnose)
- `logs/` (selfcheck + diagnose outputs)
## Troubleshooting (Quick)
- Run `./server-selfcheck.sh` → see `logs/selfcheck.json`
- Run `./server-diagnose.sh` → produces timestamped logs:
- `logs/diagnose_details_YYYYMMDD-HHMMSSZ.log`
- `logs/diagnose_error_YYYYMMDD-HHMMSSZ.log`
And updates `diagnose_details.log`/`diagnose_error.log` to the latest
- Error lines are tagged `[service][source]`, e.g. `[kibana][http] /api/status=503`
Common issues:
- Kibana 503: wait cold start or fix DNS so `es.log.argus.com` resolves
- web‑proxy 504: check nginx `resolver` includes `172.31.0.2 127.0.0.11`
- EACCES/locks: ensure `sudo ./server-prepare-dirs.sh` ran and ownership matches UID:GID

View File

@@ -1,29 +0,0 @@
# Argus 服务端离线安装指南
## 先决条件
- Linux x86_64(推荐 Ubuntu 22.04;NixOS 见“兼容说明”)
- 已安装 Docker 与 Docker Compose
- 端口:32300, 9200, 5601, 9090, 9093, 8080..8085, 21, 20, 21100–21110
## 快速开始
1. 解压到目标目录(例如 `/opt/argus-deploy/versions/<YYYYMMDD>`)
2. 安装:`./server-install.sh`(支持普通用户:会自动创建最小目录并在容器内修复 Kibana/ES/Bind)
3. 状态:`./server-status.sh`
4. 自检:`./server-selfcheck.sh`(失败会自动采集诊断)
5. 卸载:`./server-uninstall.sh`
## 安装流程要点
- 仅启动 10 个服务端组件(不包含测试节点);
- DNS Bootstrap:补齐首次部署 DNS 依赖(生成/确认 `dns.conf`、统一容器 resolv.conf、写入 `*.argus.com`、reload bind、重启 web‑proxy);
- 输出自检结果到 `logs/selfcheck.json`。
## 兼容说明(NixOS 等)
- 使用 `security_opt: ["label=disable"]` 与 `userns_mode: host`。
- 非 root 场景:安装器会创建最小目录(含 `private/argus/log/{elasticsearch,kibana}`),并在容器内完成:
- Kibana 的 `data` 软链到 `/private/argus/log/kibana`
- Elasticsearch 的 `data` 软链到 `/private/argus/log/elasticsearch`
- Bind 生成 `/etc/bind/rndc.key`
## 故障排查(见下文 Troubleshooting_zh)
- `./server-selfcheck.sh` → `logs/selfcheck.json`
- `./server-diagnose.sh` → `logs/diagnose_error_*.log` / `logs/diagnose_details_*.log`

View File

@@ -1,50 +0,0 @@
# Argus 多机部署(Docker Swarm + External Overlay)
- 前提:Docker ≥ 20.10;Manager/Worker 节点开放 2377/tcp、7946/tcp+udp、4789/udp。
- DNS:Bind9 为统一权威,解析 *.argus.com 至部署机固定对外主机 IP。
## 在部署机(Manager)
- 初始化 Swarm:`docker swarm init --advertise-addr <manager_ip>`
- 创建 overlay:`docker network create --driver overlay --attachable argus-sys-net`
- 解压离线包后执行:
- `./server-install.sh`(会验证 Swarm 状态、确保 overlay 存在、写入 dns.conf)
- `./server-selfcheck.sh`(失败会自动触发诊断)
## 在节点机(Worker 或非 Docker 主机)
- Swarm Worker:执行 Manager 的 `docker swarm join ...`;
- 运行客户端容器:
- `docker run -d --name argus-metric-node-001 --network argus-sys-net -v /data/argus/agent:/private/argus/agent argus-sys-metric-test-node:latest sleep infinity`
- 进入容器安装(先 IP 引导,后域名):
- `curl -u ftpuser:*** -fsSL ftp://<FTP_IP>:21/setup.sh -o /tmp/setup.sh && chmod +x /tmp/setup.sh`
- `AGENT_ENV=dev AGENT_USER=alice AGENT_INSTANCE=node001 MASTER_ENDPOINT=http://master.argus.com:3000 /tmp/setup.sh --server ftp.argus.com --user ftpuser --password '***' --port 21`
## 关键点
- 首次必须使用 FTP 的 IP 引导(下载 setup.sh 与 dns.conf)
- MASTER_ENDPOINT 永远使用域名:`http://master.argus.com:3000`
- docker compose 改为 external overlay;容器内不使用 Docker 服务名;web-proxy 与组件上游统一用域名
## 找回/轮换 Swarm 加入令牌与解锁密钥
在任意一个 Manager 节点上执行以下命令即可查看或轮换加入令牌(join token):
- 查看加入 Worker 的命令:
- `docker swarm join-token worker`
- 只打印 Worker 的 token:
- `docker swarm join-token -q worker`
- 查看加入 Manager 的命令:
- `docker swarm join-token manager`
- 只打印 Manager 的 token:
- `docker swarm join-token -q manager`
在待加入节点执行(示例,替换 Manager_IP):
- `docker swarm join --token <上面查到的token> <Manager_IP>:2377`
轮换 token(怀疑泄露或需要更新时):
- 轮换 Worker:`docker swarm join-token --rotate worker`
- 轮换 Manager:`docker swarm join-token --rotate manager`
如果你指的是“解锁密钥”(autolock 的 unlock key),在 Manager 上:
- 查看:`docker swarm unlock-key`
- 轮换:`docker swarm unlock-key --rotate`
提示:当看到 “This node is not a swarm manager.” 时,说明当前节点不是 Manager,需要到 Manager 节点执行,或在现有 Manager 上 `docker node promote <NODE-ID>` 将其提升为 Manager。

View File

@@ -1,20 +0,0 @@
# Troubleshooting
- Status: `scripts/server-status.sh`
- Selfcheck: `scripts/server-selfcheck.sh`
- Diagnose: `scripts/server-diagnose.sh`
Outputs:
- `logs/selfcheck.json`
- `logs/diagnose_details_*.log` (full details)
- `logs/diagnose_error_*.log` (tagged errors)
Web‑Proxy:
- 8083 expects 200/302/403; 8084/8085 must include CORS header
- nginx resolver should be `172.31.0.2 127.0.0.11`
Kibana/ES:
- Verify `es.log.argus.com` resolves inside Kibana
Permissions:
- The installer auto-creates minimal dirs and applies container-side fixes (Kibana/ES/Bind). If you still see EACCES/lock errors, rerun `./server-install.sh` and review diagnose logs.

View File

@@ -1,16 +0,0 @@
# 故障排查
- 状态:`scripts/server-status.sh`
- 自检:`scripts/server-selfcheck.sh`
- 诊断:`scripts/server-diagnose.sh`
输出:
- `logs/selfcheck.json`
- `logs/diagnose_error_*.log`(错误摘要)
- `logs/diagnose_details_*.log`(详细信息)
Web‑Proxy:8083=200/302/403;8084/8085 需包含 CORS
Kibana:确认可解析 `es.log.argus.com`
权限:
- 非 root 安装时,安装器会创建最小目录并在容器内修复 Kibana/ES/Bind;
- 如仍有 `EACCES`/锁文件报错,先重跑 `./server-install.sh`(会重复容器内修复),并查看诊断日志。

View File

@@ -1,281 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PKG_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)" # version root
PROJECT_NAME="argus-sys"
log() { echo -e "\033[0;34m[INSTALL]\033[0m $*"; }
err() { echo -e "\033[0;31m[ERROR ]\033[0m $*" >&2; }
require() { command -v "$1" >/dev/null 2>&1 || { err "missing command: $1"; exit 1; }; }
require docker
if docker compose version >/dev/null 2>&1; then COMPOSE=(docker compose); else require docker-compose; COMPOSE=(docker-compose); fi
ENV_FILE="$PKG_ROOT/compose/.env"
ENV_TEMPLATE="$PKG_ROOT/compose/.env.example"
find_free_port() {
local prefer="$1"; local start=${2:-20000}; local max=${3:-65000};
if ! ss -ltnH 2>/dev/null | awk -v pat=":"$prefer"$" '$4 ~ pat{f=1} END{exit f?0:1}'; then echo "$prefer"; return; fi
for ((p=start; p<=max; p++)); do
if ! ss -ltnH 2>/dev/null | awk -v pat=":"$p"$" '$4 ~ pat{f=1} END{exit f?0:1}'; then echo "$p"; return; fi
done
return 1
}
prepare_env() {
if [[ -f "$ENV_FILE" ]]; then log ".env exists, keep as-is"; return; fi
[[ -f "$ENV_TEMPLATE" ]] || { err "missing $ENV_TEMPLATE"; exit 1; }
cp "$ENV_TEMPLATE" "$ENV_FILE"
# overlay 模式下,避免为不同服务分配到同一新端口;保持模板端口不做自动改写
}
prepare_data_dirs() {
if [[ $EUID -ne 0 ]]; then
echo -e "\033[1;33m[WARN]\033[0m running as non-root: will not chown data dirs."
echo -e "\033[1;33m[WARN]\033[0m If you hit Permission denied, run: sudo $SCRIPT_DIR/server-prepare-dirs.sh"
# still ensure basic directories exist (no chown)
mkdir -p \
"$PKG_ROOT/private/argus/etc" \
"$PKG_ROOT/private/argus/log/elasticsearch" \
"$PKG_ROOT/private/argus/log/kibana" \
"$PKG_ROOT/private/argus/metric/prometheus" \
"$PKG_ROOT/private/argus/metric/prometheus/data" \
"$PKG_ROOT/private/argus/metric/prometheus/rules" \
"$PKG_ROOT/private/argus/metric/grafana" \
"$PKG_ROOT/private/argus/metric/grafana/data" \
"$PKG_ROOT/private/argus/metric/grafana/logs" \
"$PKG_ROOT/private/argus/metric/grafana/plugins" \
"$PKG_ROOT/private/argus/metric/grafana/provisioning/datasources" \
"$PKG_ROOT/private/argus/metric/grafana/provisioning/dashboards" \
"$PKG_ROOT/private/argus/metric/grafana/data/sessions" \
"$PKG_ROOT/private/argus/metric/grafana/data/dashboards" \
"$PKG_ROOT/private/argus/alert/alertmanager" \
"$PKG_ROOT/private/argus/metric/ftp/share"
# non-root: relax permissions to avoid container UID mismatch blocking writes
chmod -R a+rwx "$PKG_ROOT/private/argus" 2>/dev/null || true
fi
}
ensure_swarm_and_overlay() {
local net_name="${OVERLAY_NET_NAME:-argus-sys-net}"
# Require swarm active
local state
state=$(docker info --format '{{.Swarm.LocalNodeState}}' 2>/dev/null || echo "")
if [[ "$state" != "active" ]]; then
err "Docker Swarm is not active. On this host run:"
err " docker swarm init --advertise-addr <this_host_ip>"
exit 1
fi
# Create attachable overlay if missing
if ! docker network inspect "$net_name" >/dev/null 2>&1; then
log "creating attachable overlay network: $net_name"
docker network create --driver overlay --attachable "$net_name" >/dev/null
fi
}
bootstrap_dns_conf() {
local etc_dir="$PKG_ROOT/private/argus/etc"
mkdir -p "$etc_dir"
local dns_file="$etc_dir/dns.conf"
if [[ ! -s "$dns_file" ]]; then
# detect host primary IP
local host_ip
host_ip=$(ip route get 1.1.1.1 2>/dev/null | awk '/src/ {print $7; exit}')
[[ -z "$host_ip" ]] && host_ip=$(hostname -I 2>/dev/null | awk '{print $1}')
if [[ -n "$host_ip" ]]; then
echo "$host_ip" > "$dns_file"
log "wrote initial dns.conf with host IP: $host_ip"
else
err "failed to determine host IP for dns.conf; please edit $dns_file manually"
fi
fi
}
load_images() {
local tar="$PKG_ROOT/images/all-images.tar.gz"
[[ -f "$tar" ]] || { err "missing images tar: $tar"; exit 1; }
log "loading images from $(basename "$tar") (may take minutes)"
gunzip -c "$tar" | docker load >/dev/null
}
bring_up() {
log "starting services via compose"
ensure_swarm_and_overlay
bootstrap_dns_conf
local ov="$PKG_ROOT/compose/docker-compose.os-compat.override.yml"
if [[ ! -f "$ov" ]]; then
cat > "$ov" <<'YAML'
services:
bind:
security_opt: ["label=disable"]
userns_mode: "host"
tmpfs:
- /run/named
master:
security_opt: ["label=disable"]
userns_mode: "host"
es:
security_opt: ["label=disable"]
userns_mode: "host"
kibana:
security_opt: ["label=disable"]
userns_mode: "host"
ftp:
security_opt: ["label=disable"]
userns_mode: "host"
prometheus:
security_opt: ["label=disable"]
userns_mode: "host"
grafana:
security_opt: ["label=disable"]
userns_mode: "host"
alertmanager:
security_opt: ["label=disable"]
userns_mode: "host"
# ensure runtime path matches container expectation
volumes:
- ../private/argus/etc:/private/argus/etc
- ../private/argus/alert/alertmanager:/alertmanager
web-frontend:
security_opt: ["label=disable"]
userns_mode: "host"
web-proxy:
security_opt: ["label=disable"]
userns_mode: "host"
YAML
log "generated OS-compat override: $(basename "$ov")"
fi
# 仅启动服务端组件,避免误起测试节点(node-a/node-b/test-node/test-gpu-node)
local services=(bind master es kibana ftp prometheus grafana alertmanager web-frontend web-proxy)
log "services: ${services[*]}"
(cd "$PKG_ROOT/compose" && "${COMPOSE[@]}" -p "$PROJECT_NAME" -f docker-compose.yml -f $(basename "$ov") up -d "${services[@]}")
}
# Post bootstrap container-side fixes that do not require sudo on host.
post_bootstrap_fixes() {
# Kibana: ensure /usr/share/kibana/data is a symlink into mounted path to avoid EACCES
if docker ps --format '{{.Names}}' | grep -q '^argus-kibana-sys$'; then
docker exec argus-kibana-sys bash -lc '
set -e
mkdir -p /private/argus/log/kibana && chmod 777 /private/argus/log/kibana || true
if [ -d /usr/share/kibana/data ] && [ ! -L /usr/share/kibana/data ]; then rm -rf /usr/share/kibana/data; fi
if [ ! -e /usr/share/kibana/data ]; then ln -s /private/argus/log/kibana /usr/share/kibana/data; fi
' >/dev/null 2>&1 || true
fi
# Elasticsearch: ensure data path points to mounted path and is writable
if docker ps --format '{{.Names}}' | grep -q '^argus-es-sys$'; then
docker exec argus-es-sys bash -lc '
set -e
mkdir -p /private/argus/log/elasticsearch && chmod 777 /private/argus/log/elasticsearch || true
if [ -d /usr/share/elasticsearch/data ] && [ ! -L /usr/share/elasticsearch/data ]; then rm -rf /usr/share/elasticsearch/data; fi
if [ ! -e /usr/share/elasticsearch/data ]; then ln -s /private/argus/log/elasticsearch /usr/share/elasticsearch/data; fi
' >/dev/null 2>&1 || true
fi
# Bind9: ensure rndc.key exists
if docker ps --format '{{.Names}}' | grep -q '^argus-bind-sys$'; then
docker exec argus-bind-sys bash -lc '
set -e
mkdir -p /etc/bind
if [ ! -f /etc/bind/rndc.key ]; then rndc-confgen -a -c /etc/bind/rndc.key; fi
chmod 644 /etc/bind/rndc.key || true
' >/dev/null 2>&1 || true
fi
}
dns_bootstrap() {
log "DNS bootstrap: initializing shared dns.conf and container resolv.conf"
local etc_dir="$PKG_ROOT/private/argus/etc"
mkdir -p "$etc_dir"
# 1) ensure dns.conf exists (fallback to bind IP 172.31.0.2)
if [[ ! -s "$etc_dir/dns.conf" ]]; then
if echo "172.31.0.2" > "$etc_dir/dns.conf" 2>/dev/null; then
log "wrote fallback dns.conf with 172.31.0.2"
else
# host-side write denied (ownership 1000:1000); write via bind container instead
if docker ps --format '{{.Names}}' | grep -q '^argus-bind-sys$'; then
docker exec argus-bind-sys sh -lc 'echo 172.31.0.2 > /private/argus/etc/dns.conf && chmod 644 /private/argus/etc/dns.conf' || true
log "fallback dns.conf written via bind container"
else
log "bind not ready; skip writing fallback dns.conf"
fi
fi
fi
# 2) wait briefly for bind to copy update-dns.sh into shared etc (bind startup.sh does this)
local i=0
while [[ ! -x "$etc_dir/update-dns.sh" && $i -lt 20 ]]; do
sleep 0.5; ((i++));
done
if [[ ! -x "$etc_dir/update-dns.sh" ]]; then
log "update-dns.sh not present yet; continuing with existing resolv.conf"
fi
# 3) run update-dns.sh inside key containers so /etc/resolv.conf points to bind
local c
for c in argus-master-sys argus-es-sys argus-kibana-sys argus-grafana argus-prometheus argus-ftp argus-web-frontend argus-web-proxy argus-alertmanager; do
if docker ps --format '{{.Names}}' | grep -q "^${c}$"; then
docker exec "$c" sh -lc 'test -x /private/argus/etc/update-dns.sh && /private/argus/etc/update-dns.sh || true' >/dev/null 2>&1 || true
fi
done
# 4) wait for service A-record hint files generated by services (best-effort)
local need=( es.log.argus.com kibana.log.argus.com master.argus.com grafana.metric.argus.com prom.metric.argus.com alertmanager.alert.argus.com )
local waited=0; local missing=1
while (( waited < 15 )); do
missing=0
for f in "${need[@]}"; do [[ -s "$etc_dir/$f" ]] || { missing=1; break; }; done
[[ $missing -eq 0 ]] && break
sleep 1; ((waited++))
done
# 5) reload bind zone (script uses supervisor to restart bind9)
if docker ps --format '{{.Names}}' | grep -q '^argus-bind-sys$'; then
docker exec argus-bind-sys sh -lc '/usr/local/bin/reload-bind9.sh' >/dev/null 2>&1 || true
fi
# 6) restart web-proxy once to re-render nginx resolver with latest dns.conf
if docker ps --format '{{.Names}}' | grep -q '^argus-web-proxy$'; then
docker restart argus-web-proxy >/dev/null 2>&1 || true
fi
}
selfcheck() {
# Initial selfcheck with retries to absorb cold starts
local max_retries="${SELF_CHECK_RETRIES:-5}" # 重试次数(不含首次),默认 5
local wait_seconds="${SELF_CHECK_WAIT_SECONDS:-30}" # 每次重试前等待秒数,默认 30s
local attempt=0
while :; do
attempt=$((attempt+1))
if (( attempt == 1 )); then
log "running selfcheck (attempt ${attempt})"
else
log "running selfcheck (attempt ${attempt}/${max_retries}+1)"
fi
if bash "$PKG_ROOT/scripts/server-selfcheck.sh"; then
return 0
fi
# failed
if (( attempt > max_retries )); then
err "selfcheck failed after ${attempt} attempt(s)"
exit 1
fi
log "selfcheck not ready yet; retrying in ${wait_seconds}s..."
sleep "$wait_seconds"
done
}
main() {
mkdir -p "$PKG_ROOT/logs"
prepare_env
prepare_data_dirs
load_images
bring_up
post_bootstrap_fixes
dns_bootstrap
selfcheck
log "install completed. See logs in $PKG_ROOT/logs/"
}
main "$@"

View File

@@ -1,73 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PKG_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
if [[ $EUID -ne 0 ]]; then
echo "[PREPARE] This script requires root (sudo)." >&2
echo " Try: sudo $0" >&2
exit 1
fi
ENV_FILE="$PKG_ROOT/compose/.env"
[[ -f "$ENV_FILE" ]] && set -a && source "$ENV_FILE" && set +a
UIDV="${ARGUS_BUILD_UID:-1000}"; GIDV="${ARGUS_BUILD_GID:-1000}"
echo "[PREPARE] Using owner ${UIDV}:${GIDV}"
# Core etc and service data dirs (aligned with src/sys/tests/scripts/01_bootstrap.sh)
mkdir -p \
"$PKG_ROOT/private/argus/etc" \
"$PKG_ROOT/private/argus/bind" \
"$PKG_ROOT/private/argus/master" \
"$PKG_ROOT/private/argus/agent" \
"$PKG_ROOT/private/argus/log/elasticsearch" \
"$PKG_ROOT/private/argus/log/kibana"
# Prometheus
mkdir -p \
"$PKG_ROOT/private/argus/metric/prometheus" \
"$PKG_ROOT/private/argus/metric/prometheus/data" \
"$PKG_ROOT/private/argus/metric/prometheus/rules" \
"$PKG_ROOT/private/argus/metric/prometheus/targets"
# Grafana
mkdir -p \
"$PKG_ROOT/private/argus/metric/grafana" \
"$PKG_ROOT/private/argus/metric/grafana/data" \
"$PKG_ROOT/private/argus/metric/grafana/logs" \
"$PKG_ROOT/private/argus/metric/grafana/plugins" \
"$PKG_ROOT/private/argus/metric/grafana/provisioning/datasources" \
"$PKG_ROOT/private/argus/metric/grafana/provisioning/dashboards" \
"$PKG_ROOT/private/argus/metric/grafana/data/sessions" \
"$PKG_ROOT/private/argus/metric/grafana/data/dashboards" \
"$PKG_ROOT/private/argus/metric/grafana/config"
# FTP
mkdir -p "$PKG_ROOT/private/argus/metric/ftp/share"
# Alertmanager
mkdir -p "$PKG_ROOT/private/argus/alert/alertmanager"
chown -R "$UIDV":"$GIDV" \
"$PKG_ROOT/private/argus/etc" \
"$PKG_ROOT/private/argus/bind" \
"$PKG_ROOT/private/argus/master" \
"$PKG_ROOT/private/argus/agent" \
"$PKG_ROOT/private/argus/log/elasticsearch" \
"$PKG_ROOT/private/argus/log/kibana" \
"$PKG_ROOT/private/argus/metric/prometheus" \
"$PKG_ROOT/private/argus/metric/grafana" \
"$PKG_ROOT/private/argus/metric/ftp" \
"$PKG_ROOT/private/argus/alert"
chmod -R g+w "$PKG_ROOT/private/argus/alert" "$PKG_ROOT/private/argus/etc" || true
# Ensure parent directories also owned by runtime user for consistency
chown "$UIDV":"$GIDV" \
"$PKG_ROOT/private/argus" \
"$PKG_ROOT/private/argus/log" \
"$PKG_ROOT/private/argus/metric" || true
echo "[PREPARE] Done. You can now run server-install.sh"

View File

@@ -1,104 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
log() { echo -e "\033[0;34m[CHECK]\033[0m $*"; }
err() { echo -e "\033[0;31m[ERROR]\033[0m $*" >&2; }
ENV_FILE="$ROOT/compose/.env"; [[ -f "$ENV_FILE" ]] && set -a && source "$ENV_FILE" && set +a
wait_http() { local url="$1"; local attempts=${2:-120}; local i=1; while ((i<=attempts)); do curl -fsS "$url" >/dev/null 2>&1 && return 0; echo "[..] waiting $url ($i/$attempts)"; sleep 5; ((i++)); done; return 1; }
code_for() { curl -s -o /dev/null -w "%{http_code}" "$1" || echo 000; }
header_val() { curl -s -D - -o /dev/null "$@" | awk -F': ' 'BEGIN{IGNORECASE=1}$1=="Access-Control-Allow-Origin"{gsub("\r","",$2);print $2}'; }
LOG_DIR="$ROOT/logs"
mkdir -p "$LOG_DIR" || true
OUT_JSON="$LOG_DIR/selfcheck.json"
tmp=$(mktemp)
ok=1
log "checking overlay network"
net_ok=false
if docker network inspect "${OVERLAY_NET_NAME:-argus-sys-net}" >/dev/null 2>&1; then
if docker network inspect "${OVERLAY_NET_NAME:-argus-sys-net}" | grep -q '"Driver": "overlay"'; then net_ok=true; fi
fi
[[ "$net_ok" == true ]] || ok=0
log "checking Elasticsearch (via domain inside web-proxy)"
if docker exec argus-web-proxy sh -lc "curl -fsS http://es.log.argus.com:9200/_cluster/health?wait_for_status=yellow&timeout=1s" >/dev/null 2>&1; then es_ok=true; else es_ok=false; ok=0; fi
log "checking Kibana (via domain inside web-proxy)"
kb_code=$(docker exec argus-web-proxy sh -lc "curl -s -o /dev/null -w '%{http_code}' http://kibana.log.argus.com:5601/api/status" || echo 000)
kb_ok=false
if [[ "$kb_code" == "200" ]]; then body=$(curl -sS "http://localhost:${KIBANA_PORT:-5601}/api/status"); echo "$body" | grep -q '"level":"available"' && kb_ok=true; fi
[[ "$kb_ok" == true ]] || ok=0
log "checking Master (via domain inside web-proxy)"
if docker exec argus-web-proxy sh -lc "curl -fsS http://master.argus.com:3000/readyz" >/dev/null 2>&1; then true; else ok=0; fi
log "checking FTP"
if docker ps --format '{{.Names}}' | grep -q '^argus-ftp$'; then
if docker exec argus-ftp sh -lc 'test -w /private/argus/ftp/share'; then ftp_ok=true; else ftp_ok=false; ok=0; fi
else
ftp_ok=false; ok=0;
fi
log "checking Prometheus"
wait_http "http://localhost:${PROMETHEUS_PORT:-9090}/-/ready" 60 || ok=0
log "checking Grafana"
gf_code=$(curl -s -o /dev/null -w "%{http_code}" "http://localhost:${GRAFANA_PORT:-3000}/api/health" || echo 000)
gf_ok=false; if [[ "$gf_code" == "200" ]]; then body=$(curl -sS "http://localhost:${GRAFANA_PORT:-3000}/api/health"); echo "$body" | grep -q '"database"\s*:\s*"ok"' && gf_ok=true; fi
[[ "$gf_ok" == true ]] || ok=0
log "checking Alertmanager"
wait_http "http://localhost:${ALERTMANAGER_PORT:-9093}/api/v2/status" 60 || ok=0
log "checking Web-Proxy"
p8080=$(code_for "http://localhost:${WEB_PROXY_PORT_8080:-8080}/")
p8083=$(code_for "http://localhost:${WEB_PROXY_PORT_8083:-8083}/")
cors8084=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8084:-8084}/api/v2/status" || true)
cors8085=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8085:-8085}/api/v1/master/nodes" || true)
wp_ok=true
# 有些环境首页可能 403,此处接受 200/403
([[ "$p8080" == 200 || "$p8080" == 403 ]]) || wp_ok=false
([[ "$p8083" == 200 || "$p8083" == 302 || "$p8083" == 403 ]]) || wp_ok=false
[[ -n "$cors8084" && -n "$cors8085" ]] || wp_ok=false
[[ "$wp_ok" == true ]] || ok=0
cat > "$tmp" <<JSON
{
"es": $es_ok,
"kibana": $kb_ok,
"master_readyz": true,
"ftp_share_writable": $ftp_ok,
"prometheus": true,
"grafana": $gf_ok,
"alertmanager": true,
"web_proxy": $wp_ok,
"overlay_net": $net_ok,
"timestamp": "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
}
JSON
if ! mv "$tmp" "$OUT_JSON" 2>/dev/null; then
# fallback when logs dir not writable (no sudo allowed)
OUT_JSON="/tmp/selfcheck_$(date -u +%Y%m%d-%H%M%SZ).json"
cp "$tmp" "$OUT_JSON"
log "selfcheck.json written to $OUT_JSON (logs dir not writable)"
fi
if [[ "$ok" == 1 ]]; then
log "selfcheck OK"
exit 0
else
err "selfcheck FAILED (see $OUT_JSON)"
# If diagnose script exists, run it to collect more details
if [[ -x "$SCRIPT_DIR/server-diagnose.sh" ]]; then
# run diagnose; it will print the actual timestamped file paths and update 'latest' symlinks
"$SCRIPT_DIR/server-diagnose.sh" || true
fi
exit 1
fi

View File

@@ -1,28 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
PROJECT_NAME="argus-sys"
if docker compose version >/dev/null 2>&1; then COMPOSE=(docker compose); else COMPOSE=(docker-compose); fi
echo "== Containers =="
(cd "$ROOT/compose" && "${COMPOSE[@]}" -p "$PROJECT_NAME" ps)
echo
echo "== Key Endpoints =="
ENV_FILE="$ROOT/compose/.env"; [[ -f "$ENV_FILE" ]] && set -a && source "$ENV_FILE" && set +a
printf "master http://localhost:%s/readyz\n" "${MASTER_PORT:-32300}"
printf "es http://localhost:%s/_cluster/health\n" "${ES_HTTP_PORT:-9200}"
printf "kibana http://localhost:%s/api/status\n" "${KIBANA_PORT:-5601}"
printf "prom http://localhost:%s/-/ready\n" "${PROMETHEUS_PORT:-9090}"
printf "grafana http://localhost:%s/api/health\n" "${GRAFANA_PORT:-3000}"
printf "alert http://localhost:%s/api/v2/status\n" "${ALERTMANAGER_PORT:-9093}"
printf "web http://localhost:%s/ (8080)\n" "${WEB_PROXY_PORT_8080:-8080}"
echo
echo "== Selfcheck result =="
cat "$ROOT/logs/selfcheck.json" 2>/dev/null || echo "(no selfcheck yet)"

View File

@@ -1,16 +0,0 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
PKG_ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
PROJECT_NAME="argus-sys"
log() { echo -e "\033[0;34m[UNINSTALL]\033[0m $*"; }
if docker compose version >/dev/null 2>&1; then COMPOSE=(docker compose); else COMPOSE=(docker-compose); fi
(cd "$PKG_ROOT/compose" && "${COMPOSE[@]}" -p "$PROJECT_NAME" down -v || true)
log "compose stack removed"
log "you may remove data under $PKG_ROOT/private if you want a clean slate"

14
deployment_new/README.md Normal file
View File

@@ -0,0 +1,14 @@
# deployment_new
本目录用于新的部署打包与交付实现(不影响既有 `deployment/`)。
里程碑 M1(当前实现)
- `build/make_server_package.sh`:生成 Server 包(逐服务镜像 tar.gz、compose、.env.example、docs、private 骨架、manifest/checksums、打包 tar.gz)。
- `build/make_client_gpu_package.sh`:生成 Client‑GPU 包(GPU bundle 镜像 tar.gz、busybox.tar、compose、.env.example、docs、private 骨架、manifest/checksums、打包 tar.gz)。
模板
- `templates/server/compose/docker-compose.yml`:部署专用,镜像默认使用 `:${PKG_VERSION}` 版本 tag,可通过 `.env` 覆盖。
- `templates/client_gpu/compose/docker-compose.yml`:GPU 节点专用,使用 `:${PKG_VERSION}` 版本 tag。
注意:M1 仅产出安装包,不包含安装脚本落地;安装/运维脚本将在 M2 落地并纳入包内。

View File

@@ -6,14 +6,14 @@ warn() { echo -e "\033[1;33m[WARN]\033[0m $*"; }
err() { echo -e "\033[0;31m[ERR ]\033[0m $*" >&2; } err() { echo -e "\033[0;31m[ERR ]\033[0m $*" >&2; }
require_cmd() { require_cmd() {
local miss=0
for c in "$@"; do for c in "$@"; do
command -v "$c" >/dev/null 2>&1 || { err "missing command: $c"; exit 1; } if ! command -v "$c" >/dev/null 2>&1; then err "missing command: $c"; miss=1; fi
done done
[[ $miss -eq 0 ]]
} }
today_version() { today_version() { date +%Y%m%d; }
date +%Y%m%d
}
checksum_dir() { checksum_dir() {
local dir="$1"; local out="$2"; : > "$out"; local dir="$1"; local out="$2"; : > "$out";
@@ -28,6 +28,6 @@ copy_tree() {
gen_manifest() { gen_manifest() {
local root="$1"; local out="$2"; : > "$out"; local root="$1"; local out="$2"; : > "$out";
(cd "$root" && find . -maxdepth 3 -type f -printf "%p\n" | sort) >> "$out" (cd "$root" && find . -maxdepth 4 -type f -printf "%p\n" | sort) >> "$out"
} }

View File

@@ -0,0 +1,112 @@
#!/usr/bin/env bash
set -euo pipefail
# Make ARM Client-CPU package (versioned CPU node image, compose, env, docs)
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
TEMPL_DIR="$ROOT_DIR/deployment_new/templates/client_arm"
ART_ROOT="$ROOT_DIR/deployment_new/artifact/client_arm"
COMMON_SH="$ROOT_DIR/deployment_new/build/common.sh"
. "$COMMON_SH"
usage(){ cat <<EOF
Build ARM Client-CPU Package (deployment_new)
Usage: $(basename "$0") --version YYYYMMDD [--image IMAGE[:TAG]]
Defaults:
image = argus-sys-metric-test-node-arm64:latest
Outputs: deployment_new/artifact/client_arm/<YYYYMMDD>/ and client_arm_YYYYMMDD.tar.gz
EOF
}
VERSION=""
IMAGE="argus-sys-metric-test-node-arm64:latest"
while [[ $# -gt 0 ]]; do
case "$1" in
--version) VERSION="$2"; shift 2;;
--image) IMAGE="$2"; shift 2;;
-h|--help) usage; exit 0;;
*) err "unknown arg: $1"; usage; exit 1;;
esac
done
if [[ -z "$VERSION" ]]; then VERSION="$(today_version)"; fi
require_cmd docker tar gzip
STAGE="$(mktemp -d)"; trap 'rm -rf "$STAGE"' EXIT
PKG_DIR="$ART_ROOT/$VERSION"
mkdir -p "$PKG_DIR" "$STAGE/images" "$STAGE/compose" "$STAGE/docs" "$STAGE/scripts" "$STAGE/private/argus"
# 1) Save CPU node image with version tag
if ! docker image inspect "$IMAGE" >/dev/null 2>&1; then
err "missing image: $IMAGE"; exit 1; fi
REPO="${IMAGE%%:*}"; TAG_VER="$REPO:$VERSION"
docker tag "$IMAGE" "$TAG_VER"
out_tar="$STAGE/images/${REPO//\//-}-$VERSION.tar"
docker save -o "$out_tar" "$TAG_VER"
gzip -f "$out_tar"
# 2) Compose + env template
cp "$TEMPL_DIR/compose/docker-compose.yml" "$STAGE/compose/docker-compose.yml"
ENV_EX="$STAGE/compose/.env.example"
cat >"$ENV_EX" <<EOF
# Generated by make_arm_client_package.sh
PKG_VERSION=$VERSION
NODE_CPU_IMAGE_TAG=${REPO}:${VERSION}
# Compose project name (isolation from server stack)
COMPOSE_PROJECT_NAME=argus-client-arm
# Required fields(ARM Client 默认通过 overlay alias 访问 master,可按需覆盖 MASTER_ENDPOINT)
MASTER_ENDPOINT=http://master.argus.com:3000
AGENT_ENV=
AGENT_USER=
AGENT_INSTANCE=
CPU_NODE_HOSTNAME=
# UID/GID for volume ownership
ARGUS_BUILD_UID=2133
ARGUS_BUILD_GID=2015
EOF
# 3) Docs
CLIENT_DOC_SRC="$TEMPL_DIR/docs"
if [[ -d "$CLIENT_DOC_SRC" ]]; then
copy_tree "$CLIENT_DOC_SRC" "$STAGE/docs"
fi
# 4) Scripts & private skeleton
SCRIPTS_SRC="$TEMPL_DIR/scripts"
if [[ -d "$SCRIPTS_SRC" ]]; then
copy_tree "$SCRIPTS_SRC" "$STAGE/scripts"
find "$STAGE/scripts" -type f -name '*.sh' -exec chmod +x {} + 2>/dev/null || true
fi
# x86 专用脚本:用于在 x86_64 主机上运行 ARM Client 包
SCRIPTS_X86_SRC="$TEMPL_DIR/scripts_for_x86"
if [[ -d "$SCRIPTS_X86_SRC" ]]; then
copy_tree "$SCRIPTS_X86_SRC" "$STAGE/scripts_for_x86"
find "$STAGE/scripts_for_x86" -type f -name '*.sh' -exec chmod +x {} + 2>/dev/null || true
fi
mkdir -p "$STAGE/private/argus/agent"
# 5) Manifest & checksums
gen_manifest "$STAGE" "$STAGE/manifest.txt"
checksum_dir "$STAGE" "$STAGE/checksums.txt"
# 6) Move to artifact dir and pack
mkdir -p "$PKG_DIR"
copy_tree "$STAGE" "$PKG_DIR"
OUT_TAR_DIR="$(dirname "$PKG_DIR")"
OUT_TAR="$OUT_TAR_DIR/client_arm_${VERSION}.tar.gz"
log "Creating tarball: $OUT_TAR"
(cd "$PKG_DIR/.." && tar -czf "$OUT_TAR" "$(basename "$PKG_DIR")")
log "Client-ARM package ready: $PKG_DIR"
echo "$OUT_TAR"

View File

@@ -0,0 +1,175 @@
#!/usr/bin/env bash
set -euo pipefail
# Make ARM server deployment package (master + prometheus + grafana + alertmanager + web)
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
TEMPL_DIR="$ROOT_DIR/deployment_new/templates/server_arm"
ART_ROOT="$ROOT_DIR/deployment_new/artifact/server_arm"
COMMON_SH="$ROOT_DIR/deployment_new/build/common.sh"
. "$COMMON_SH"
usage(){ cat <<EOF
Build ARM Server Deployment Package (deployment_new)
Usage: $(basename "$0") --version YYYYMMDD
Outputs: deployment_new/artifact/server_arm/<YYYYMMDD>/ and server_arm_YYYYMMDD.tar.gz
EOF
}
VERSION=""
while [[ $# -gt 0 ]]; do
case "$1" in
--version) VERSION="$2"; shift 2;;
-h|--help) usage; exit 0;;
*) err "unknown arg: $1"; usage; exit 1;;
esac
done
if [[ -z "$VERSION" ]]; then VERSION="$(today_version)"; fi
require_cmd docker tar gzip awk sed
IMAGES=(
argus-master-arm64
argus-metric-prometheus-arm64
argus-metric-prometheus-targets-updater-arm64
argus-metric-grafana-arm64
argus-alertmanager-arm64
argus-web-frontend-arm64
argus-web-proxy-arm64
)
STAGE="$(mktemp -d)"; trap 'rm -rf "$STAGE"' EXIT
PKG_DIR="$ART_ROOT/$VERSION"
mkdir -p "$PKG_DIR" "$STAGE/images" "$STAGE/compose" "$STAGE/docs" "$STAGE/scripts" "$STAGE/private/argus"
# 1) Save per-image tars with version tag
log "Tagging and saving ARM server images (version=$VERSION)"
for repo in "${IMAGES[@]}"; do
if ! docker image inspect "$repo:latest" >/dev/null 2>&1 && ! docker image inspect "$repo:$VERSION" >/dev/null 2>&1; then
err "missing image: $repo (need :latest or :$VERSION)"; exit 1; fi
if docker image inspect "$repo:$VERSION" >/dev/null 2>&1; then
tag="$repo:$VERSION"
else
docker tag "$repo:latest" "$repo:$VERSION"
tag="$repo:$VERSION"
fi
out_tar="$STAGE/images/${repo//\//-}-$VERSION.tar"
docker save -o "$out_tar" "$tag"
gzip -f "$out_tar"
done
# 1.1) (可选)打包 x86 ARM 测试用基础镜像 ubuntu:22.04(供 scripts_for_x86/install.sh 使用)
TEST_BASE="ubuntu:22.04"
if docker image inspect "$TEST_BASE" >/dev/null 2>&1 || docker pull "$TEST_BASE" >/dev/null 2>&1; then
log "Saving ARM test base image for x86 host: $TEST_BASE"
test_tar="$STAGE/images/ubuntu-22.04-arm-test.tar"
docker save -o "$test_tar" "$TEST_BASE"
gzip -f "$test_tar"
else
warn "无法获取 $TEST_BASE,x86 上的 ARM 运行能力将无法使用该镜像做快速自检(可忽略,或在目标机自行拉取)。"
fi
# 1.2) 随包提供 tonistiigi/binfmt 镜像,便于 x86 离线机器安装 binfmt/QEMU
BINFMT_IMAGE="tonistiigi/binfmt:latest"
if docker image inspect "$BINFMT_IMAGE" >/dev/null 2>&1 || docker pull "$BINFMT_IMAGE" >/dev/null 2>&1; then
log "Saving binfmt helper image for x86 host: $BINFMT_IMAGE"
binfmt_tar="$STAGE/images/tonistiigi-binfmt.tar"
docker save -o "$binfmt_tar" "$BINFMT_IMAGE"
gzip -f "$binfmt_tar"
else
warn "无法获取 $BINFMT_IMAGE,x86 上将无法通过离线方式安装 binfmt(可在目标机上自行 docker pull)。"
fi
# 2) Compose + env template(ARM 专用)
cp "$TEMPL_DIR/compose/docker-compose.yml" "$STAGE/compose/docker-compose.yml"
ENV_EX="$STAGE/compose/.env.example"
cat >"$ENV_EX" <<EOF
# Generated by make_arm_server_package.sh
PKG_VERSION=$VERSION
# ARM server image tags
MASTER_IMAGE_TAG=argus-master-arm64:${VERSION}
PROM_IMAGE_TAG=argus-metric-prometheus-arm64:${VERSION}
PROM_UPDATER_IMAGE_TAG=argus-metric-prometheus-targets-updater-arm64:${VERSION}
GRAFANA_IMAGE_TAG=argus-metric-grafana-arm64:${VERSION}
ALERT_IMAGE_TAG=argus-alertmanager-arm64:${VERSION}
FRONT_IMAGE_TAG=argus-web-frontend-arm64:${VERSION}
WEB_PROXY_IMAGE_TAG=argus-web-proxy-arm64:${VERSION}
# Host ports for server compose
MASTER_PORT=32300
PROMETHEUS_PORT=9090
GRAFANA_PORT=3000
ALERTMANAGER_PORT=9093
WEB_PROXY_PORT_8080=8080
WEB_PROXY_PORT_8081=8081
WEB_PROXY_PORT_8082=8082
WEB_PROXY_PORT_8083=8083
WEB_PROXY_PORT_8084=8084
WEB_PROXY_PORT_8085=8085
# Overlay network name
ARGUS_OVERLAY_NET=argus-sys-net
# UID/GID for volume ownership
ARGUS_BUILD_UID=2133
ARGUS_BUILD_GID=2015
# Compose project name (isolation from other stacks on same host)
COMPOSE_PROJECT_NAME=argus-server-arm
EOF
# 3) Docs & scripts from templates/server_arm
DOCS_SRC="$TEMPL_DIR/docs"
if [[ -d "$DOCS_SRC" ]]; then
copy_tree "$DOCS_SRC" "$STAGE/docs"
fi
SCRIPTS_SRC="$TEMPL_DIR/scripts"
if [[ -d "$SCRIPTS_SRC" ]]; then
copy_tree "$SCRIPTS_SRC" "$STAGE/scripts"
find "$STAGE/scripts" -type f -name '*.sh' -exec chmod +x {} + 2>/dev/null || true
fi
# x86 专用脚本:用于在 x86_64 主机上运行 ARM server 包
SCRIPTS_X86_SRC="$TEMPL_DIR/scripts_for_x86"
if [[ -d "$SCRIPTS_X86_SRC" ]]; then
copy_tree "$SCRIPTS_X86_SRC" "$STAGE/scripts_for_x86"
find "$STAGE/scripts_for_x86" -type f -name '*.sh' -exec chmod +x {} + 2>/dev/null || true
fi
# 4) Private skeleton (最小 ARM server 所需目录)
mkdir -p \
"$STAGE/private/argus/etc" \
"$STAGE/private/argus/master" \
"$STAGE/private/argus/metric/prometheus" \
"$STAGE/private/argus/metric/prometheus/data" \
"$STAGE/private/argus/metric/prometheus/rules" \
"$STAGE/private/argus/metric/prometheus/targets" \
"$STAGE/private/argus/metric/grafana" \
"$STAGE/private/argus/metric/grafana/data" \
"$STAGE/private/argus/metric/grafana/logs" \
"$STAGE/private/argus/metric/grafana/plugins" \
"$STAGE/private/argus/metric/grafana/provisioning/datasources" \
"$STAGE/private/argus/metric/grafana/provisioning/dashboards" \
"$STAGE/private/argus/metric/grafana/data/sessions" \
"$STAGE/private/argus/metric/grafana/data/dashboards" \
"$STAGE/private/argus/metric/grafana/config"
# 5) Manifest & checksums
gen_manifest "$STAGE" "$STAGE/manifest.txt"
checksum_dir "$STAGE" "$STAGE/checksums.txt"
# 6) Move to artifact dir and pack
mkdir -p "$PKG_DIR"
copy_tree "$STAGE" "$PKG_DIR"
OUT_TAR_DIR="$(dirname "$PKG_DIR")"
OUT_TAR="$OUT_TAR_DIR/server_arm_${VERSION}.tar.gz"
log "Creating tarball: $OUT_TAR"
(cd "$PKG_DIR/.." && tar -czf "$OUT_TAR" "$(basename "$PKG_DIR")")
log "ARM Server package ready: $PKG_DIR"
echo "$OUT_TAR"

View File

@@ -0,0 +1,134 @@
#!/usr/bin/env bash
set -euo pipefail
# Make client GPU package (versioned gpu bundle image, compose, env, docs, busybox)
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
TEMPL_DIR="$ROOT_DIR/deployment_new/templates/client_gpu"
ART_ROOT="$ROOT_DIR/deployment_new/artifact/client_gpu"
# Use deployment_new local common helpers
COMMON_SH="$ROOT_DIR/deployment_new/build/common.sh"
. "$COMMON_SH"
usage(){ cat <<EOF
Build Client-GPU Package (deployment_new)
Usage: $(basename "$0") --version YYYYMMDD [--image IMAGE[:TAG]]
Defaults:
image = argus-sys-metric-test-node-bundle-gpu:latest
Outputs: deployment_new/artifact/client_gpu/<YYYYMMDD>/ and client_gpu_YYYYMMDD.tar.gz
EOF
}
VERSION=""
IMAGE="argus-sys-metric-test-node-bundle-gpu:latest"
while [[ $# -gt 0 ]]; do
case "$1" in
--version) VERSION="$2"; shift 2;;
--image) IMAGE="$2"; shift 2;;
-h|--help) usage; exit 0;;
*) err "unknown arg: $1"; usage; exit 1;;
esac
done
if [[ -z "$VERSION" ]]; then VERSION="$(today_version)"; fi
require_cmd docker tar gzip
STAGE="$(mktemp -d)"; trap 'rm -rf "$STAGE"' EXIT
PKG_DIR="$ART_ROOT/$VERSION"
mkdir -p "$PKG_DIR" "$STAGE/images" "$STAGE/compose" "$STAGE/docs" "$STAGE/scripts" "$STAGE/private/argus"
# 1) Save GPU bundle image with version tag
if ! docker image inspect "$IMAGE" >/dev/null 2>&1; then
err "missing image: $IMAGE"; exit 1; fi
REPO="${IMAGE%%:*}"; TAG_VER="$REPO:$VERSION"
docker tag "$IMAGE" "$TAG_VER"
out_tar="$STAGE/images/${REPO//\//-}-$VERSION.tar"
docker save -o "$out_tar" "$TAG_VER"
gzip -f "$out_tar"
# 2) Busybox tar for connectivity/overlay warmup (prefer local template; fallback to docker save)
BB_SRC="$TEMPL_DIR/images/busybox.tar"
if [[ -f "$BB_SRC" ]]; then
cp "$BB_SRC" "$STAGE/images/busybox.tar"
else
if docker image inspect busybox:latest >/dev/null 2>&1 || docker pull busybox:latest >/dev/null 2>&1; then
docker save -o "$STAGE/images/busybox.tar" busybox:latest
log "Included busybox from local docker daemon"
else
warn "busybox image not found and cannot pull; skipping busybox.tar"
fi
fi
# 3) Compose + env template and docs/scripts from templates
cp "$TEMPL_DIR/compose/docker-compose.yml" "$STAGE/compose/docker-compose.yml"
ENV_EX="$STAGE/compose/.env.example"
cat >"$ENV_EX" <<EOF
# Generated by make_client_gpu_package.sh
PKG_VERSION=$VERSION
NODE_GPU_BUNDLE_IMAGE_TAG=${REPO}:${VERSION}
# Compose project name (isolation from server stack)
COMPOSE_PROJECT_NAME=argus-client
# Required (no defaults). Must be filled before install.
AGENT_ENV=
AGENT_USER=
AGENT_INSTANCE=
GPU_NODE_HOSTNAME=
# From cluster-info.env (server package output)
BINDIP=
FTPIP=
SWARM_MANAGER_ADDR=
SWARM_JOIN_TOKEN_WORKER=
SWARM_JOIN_TOKEN_MANAGER=
# FTP defaults
FTP_USER=ftpuser
FTP_PASSWORD=NASPlab1234!
EOF
# 4) Docs from deployment_new templates
CLIENT_DOC_SRC="$TEMPL_DIR/docs"
if [[ -d "$CLIENT_DOC_SRC" ]]; then
rsync -a "$CLIENT_DOC_SRC/" "$STAGE/docs/" >/dev/null 2>&1 || cp -r "$CLIENT_DOC_SRC/." "$STAGE/docs/"
fi
# Placeholder scripts (will be implemented in M2)
cat >"$STAGE/scripts/README.md" <<'EOF'
# Client-GPU Scripts (Placeholder)
本目录将在 M2 引入:
- config.sh / install.sh
当前为占位,便于包结构审阅。
EOF
# 5) Scripts (from deployment_new templates) and Private skeleton
SCRIPTS_SRC="$TEMPL_DIR/scripts"
if [[ -d "$SCRIPTS_SRC" ]]; then
rsync -a "$SCRIPTS_SRC/" "$STAGE/scripts/" >/dev/null 2>&1 || cp -r "$SCRIPTS_SRC/." "$STAGE/scripts/"
find "$STAGE/scripts" -type f -name '*.sh' -exec chmod +x {} + 2>/dev/null || true
fi
mkdir -p "$STAGE/private/argus/agent"
# 6) Manifest & checksums
gen_manifest "$STAGE" "$STAGE/manifest.txt"
checksum_dir "$STAGE" "$STAGE/checksums.txt"
# 7) Move to artifact dir and pack
mkdir -p "$PKG_DIR"
rsync -a "$STAGE/" "$PKG_DIR/" >/dev/null 2>&1 || cp -r "$STAGE/." "$PKG_DIR/"
OUT_TAR_DIR="$(dirname "$PKG_DIR")"
OUT_TAR="$OUT_TAR_DIR/client_gpu_${VERSION}.tar.gz"
log "Creating tarball: $OUT_TAR"
(cd "$PKG_DIR/.." && tar -czf "$OUT_TAR" "$(basename "$PKG_DIR")")
log "Client-GPU package ready: $PKG_DIR"
echo "$OUT_TAR"

View File

@@ -0,0 +1,169 @@
#!/usr/bin/env bash
set -euo pipefail
# Make server deployment package (versioned, per-image tars, full compose, docs, skeleton)
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
TEMPL_DIR="$ROOT_DIR/deployment_new/templates/server"
ART_ROOT="$ROOT_DIR/deployment_new/artifact/server"
# Use deployment_new local common helpers
COMMON_SH="$ROOT_DIR/deployment_new/build/common.sh"
. "$COMMON_SH"
usage(){ cat <<EOF
Build Server Deployment Package (deployment_new)
Usage: $(basename "$0") --version YYYYMMDD
Outputs: deployment_new/artifact/server/<YYYYMMDD>/ and server_YYYYMMDD.tar.gz
EOF
}
VERSION=""
while [[ $# -gt 0 ]]; do
case "$1" in
--version) VERSION="$2"; shift 2;;
-h|--help) usage; exit 0;;
*) err "unknown arg: $1"; usage; exit 1;;
esac
done
if [[ -z "$VERSION" ]]; then VERSION="$(today_version)"; fi
require_cmd docker tar gzip awk sed
IMAGES=(
argus-bind9
argus-master
argus-elasticsearch
argus-kibana
argus-metric-ftp
argus-metric-prometheus
argus-metric-grafana
argus-alertmanager
argus-web-frontend
argus-web-proxy
)
STAGE="$(mktemp -d)"; trap 'rm -rf "$STAGE"' EXIT
PKG_DIR="$ART_ROOT/$VERSION"
mkdir -p "$PKG_DIR" "$STAGE/images" "$STAGE/compose" "$STAGE/docs" "$STAGE/scripts" "$STAGE/private/argus"
# 1) Save per-image tars with version tag
log "Tagging and saving images (version=$VERSION)"
for repo in "${IMAGES[@]}"; do
if ! docker image inspect "$repo:latest" >/dev/null 2>&1 && ! docker image inspect "$repo:$VERSION" >/dev/null 2>&1; then
err "missing image: $repo (need :latest or :$VERSION)"; exit 1; fi
if docker image inspect "$repo:$VERSION" >/dev/null 2>&1; then
tag="$repo:$VERSION"
else
docker tag "$repo:latest" "$repo:$VERSION"
tag="$repo:$VERSION"
fi
out_tar="$STAGE/images/${repo//\//-}-$VERSION.tar"
docker save -o "$out_tar" "$tag"
gzip -f "$out_tar"
done
# 2) Compose + env template
cp "$TEMPL_DIR/compose/docker-compose.yml" "$STAGE/compose/docker-compose.yml"
ENV_EX="$STAGE/compose/.env.example"
cat >"$ENV_EX" <<EOF
# Generated by make_server_package.sh
PKG_VERSION=$VERSION
# Image tags (can be overridden). Default to versioned tags
BIND_IMAGE_TAG=argus-bind9:
MASTER_IMAGE_TAG=argus-master:
ES_IMAGE_TAG=argus-elasticsearch:
KIBANA_IMAGE_TAG=argus-kibana:
FTP_IMAGE_TAG=argus-metric-ftp:
PROM_IMAGE_TAG=argus-metric-prometheus:
GRAFANA_IMAGE_TAG=argus-metric-grafana:
ALERT_IMAGE_TAG=argus-alertmanager:
FRONT_IMAGE_TAG=argus-web-frontend:
WEB_PROXY_IMAGE_TAG=argus-web-proxy:
EOF
sed -i "s#:\$#:${VERSION}#g" "$ENV_EX"
# Ports and defaults (based on swarm_tests .env.example)
cat >>"$ENV_EX" <<'EOF'
# Host ports for server compose
MASTER_PORT=32300
ES_HTTP_PORT=9200
KIBANA_PORT=5601
PROMETHEUS_PORT=9090
GRAFANA_PORT=3000
ALERTMANAGER_PORT=9093
WEB_PROXY_PORT_8080=8080
WEB_PROXY_PORT_8081=8081
WEB_PROXY_PORT_8082=8082
WEB_PROXY_PORT_8083=8083
WEB_PROXY_PORT_8084=8084
WEB_PROXY_PORT_8085=8085
# Overlay network name
ARGUS_OVERLAY_NET=argus-sys-net
# FTP defaults
FTP_USER=ftpuser
FTP_PASSWORD=NASPlab1234!
# UID/GID for volume ownership
ARGUS_BUILD_UID=2133
ARGUS_BUILD_GID=2015
# Compose project name (isolation from other stacks on same host)
COMPOSE_PROJECT_NAME=argus-server
EOF
# 3) Docs (from deployment_new templates)
DOCS_SRC="$TEMPL_DIR/docs"
if [[ -d "$DOCS_SRC" ]]; then
rsync -a "$DOCS_SRC/" "$STAGE/docs/" >/dev/null 2>&1 || cp -r "$DOCS_SRC/." "$STAGE/docs/"
fi
# 6) Scripts (from deployment_new templates)
SCRIPTS_SRC="$TEMPL_DIR/scripts"
if [[ -d "$SCRIPTS_SRC" ]]; then
rsync -a "$SCRIPTS_SRC/" "$STAGE/scripts/" >/dev/null 2>&1 || cp -r "$SCRIPTS_SRC/." "$STAGE/scripts/"
find "$STAGE/scripts" -type f -name '*.sh' -exec chmod +x {} + 2>/dev/null || true
fi
# 4) Private skeleton (minimum)
mkdir -p \
"$STAGE/private/argus/etc" \
"$STAGE/private/argus/master" \
"$STAGE/private/argus/metric/prometheus" \
"$STAGE/private/argus/metric/prometheus/data" \
"$STAGE/private/argus/metric/prometheus/rules" \
"$STAGE/private/argus/metric/prometheus/targets" \
"$STAGE/private/argus/metric/grafana" \
"$STAGE/private/argus/metric/grafana/data" \
"$STAGE/private/argus/metric/grafana/logs" \
"$STAGE/private/argus/metric/grafana/plugins" \
"$STAGE/private/argus/metric/grafana/provisioning/datasources" \
"$STAGE/private/argus/metric/grafana/provisioning/dashboards" \
"$STAGE/private/argus/metric/grafana/data/sessions" \
"$STAGE/private/argus/metric/grafana/data/dashboards" \
"$STAGE/private/argus/metric/grafana/config" \
"$STAGE/private/argus/metric/ftp" \
"$STAGE/private/argus/alert/alertmanager" \
"$STAGE/private/argus/log/elasticsearch" \
"$STAGE/private/argus/log/kibana"
# 7) Manifest & checksums
gen_manifest "$STAGE" "$STAGE/manifest.txt"
checksum_dir "$STAGE" "$STAGE/checksums.txt"
# 8) Move to artifact dir and pack
mkdir -p "$PKG_DIR"
rsync -a "$STAGE/" "$PKG_DIR/" >/dev/null 2>&1 || cp -r "$STAGE/." "$PKG_DIR/"
OUT_TAR_DIR="$(dirname "$PKG_DIR")"
OUT_TAR="$OUT_TAR_DIR/server_${VERSION}.tar.gz"
log "Creating tarball: $OUT_TAR"
(cd "$PKG_DIR/.." && tar -czf "$OUT_TAR" "$(basename "$PKG_DIR")")
log "Server package ready: $PKG_DIR"
echo "$OUT_TAR"

View File

@@ -0,0 +1,29 @@
version: "3.8"
networks:
argus-sys-net:
external: true
services:
metric-cpu-node:
image: ${NODE_CPU_IMAGE_TAG:-argus-sys-metric-test-node-arm64:${PKG_VERSION}}
platform: linux/arm64
container_name: argus-metric-cpu-node
hostname: ${CPU_NODE_HOSTNAME}
restart: unless-stopped
environment:
- TZ=Asia/Shanghai
- DEBIAN_FRONTEND=noninteractive
- MASTER_ENDPOINT=${MASTER_ENDPOINT}
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
- AGENT_ENV=${AGENT_ENV}
- AGENT_USER=${AGENT_USER}
- AGENT_INSTANCE=${AGENT_INSTANCE}
networks:
argus-sys-net:
aliases:
- ${AGENT_INSTANCE}.node.argus.com
volumes:
- ../private/argus/agent:/private/argus/agent

View File

@@ -0,0 +1,154 @@
# ARM Client-CPU 安装指南(client_arm)
本包用于在 ARM64 CPU 节点上部署 Argus 的最小客户端:
- `argus-agent`(负责注册/上报状态到 master)
- `node-exporter`(暴露该节点的系统指标)
> 注意:本 ARM 客户端包 **不包含 GPU/DCGM exporter 和 Fluent Bit**,仅用于 CPU 节点 metrics。
---
## 1. 前置条件
- 目标机器为 ARM64(例如:aarch64 服务器 / 节点);
- 已安装 Docker 和 docker compose(v2,支持 `docker compose` 命令);
- 能访问 master 所在网络,或已经通过 overlay 网络加入到与 server 相同的 `argus-sys-net` 中。
如采用 overlay 网络(Swarm 模式),需提前创建 `argus-sys-net`:
```bash
docker network create --driver overlay argus-sys-net || true
```
如仅使用本地 bridge 网络,可将 compose 中 `networks: argus-sys-net` 调整为本地自定义 network。
---
## 2. 解压和加载镜像
假设你已经将 `client_arm_YYYYMMDD.tar.gz` 拷贝到 ARM 节点:
```bash
tar -xzf client_arm_YYYYMMDD.tar.gz
cd client_arm/YYYYMMDD
```
加载 CPU node 镜像:
```bash
for f in images/*.tar.gz; do
gunzip -c "$f" | docker load
done
```
可以通过:
```bash
docker images | grep 'argus-sys-metric-test-node-arm64'
```
确认镜像已加载。
---
## 3. 配置 .env
进入 compose 目录:
```bash
cd compose
cp .env.example .env
```
需要填写的关键字段:
- `MASTER_ENDPOINT`
- 例如:`http://master.argus.com:3000` 或 `http://<Server-IP>:32300`
- `AGENT_ENV` / `AGENT_USER` / `AGENT_INSTANCE`
- 用于识别环境、用户和实例,在 UI 中区分节点:
- 示例:`AGENT_ENV=prod`, `AGENT_USER=ops`, `AGENT_INSTANCE=node001`
- `CPU_NODE_HOSTNAME`
- 节点在配置中的 hostname,一般与宿主 `hostname` 一致,例如 `swarm-metric-node-001`。
- `ARGUS_BUILD_UID` / `ARGUS_BUILD_GID`
- 用于挂载目录 `/private/argus/agent` 的权限映射。
例如:
```env
PKG_VERSION=20251124
NODE_CPU_IMAGE_TAG=argus-sys-metric-test-node-arm64:20251124
COMPOSE_PROJECT_NAME=argus-client-arm
MASTER_ENDPOINT=http://master.argus.com:3000
AGENT_ENV=dev2
AGENT_USER=yuyr
AGENT_INSTANCE=node001sX
CPU_NODE_HOSTNAME=swarm-metric-node-001
ARGUS_BUILD_UID=1000
ARGUS_BUILD_GID=1000
```
---
## 4. 准备挂载目录
包中已经预创建 `private/argus/agent` 目录:
- `../private/argus/agent`
可以根据需要设置所有者和权限:
```bash
cd ..
sudo chown -R $UID:$GID private/argus
```
确保与 `.env` 中的 `ARGUS_BUILD_UID/GID` 一致。
---
## 5. 启动客户端容器
在 `compose/` 目录执行:
```bash
docker compose --env-file .env up -d
```
查看状态:
```bash
docker compose ps
```
预期看到:
- `argus-metric-cpu-node` 处于 `Up` 状态。
---
## 6. 在 server 端验证
在 server 侧(master + Prometheus + Grafana 已经通过 ARM server 包部署):
1. 在 master 的 UI 或 API 中:
- `GET /api/v1/master/nodes` 应能看到新节点,`name=CPU_NODE_HOSTNAME`;
2. 在 Prometheus 中:
- `up{job="node"}` 中应有当前节点记录,且 `value=1`;
- 可以通过 `node_load1{hostname="swarm-metric-node-001"}` 等表达式查看指标;
3. 在 Web 门户 / Grafana:
- 节点列表页面中能看到该节点;
- 通过“按 hostname 的节点面板”查看对应指标。
---
## 7. 注意事项
- ARM 客户端包只负责“CPU 节点 metrics”,不包含 GPU/DCGM 或 Fluent Bit;
- 如需要 GPU 支持,请继续使用 `client_gpu` 包,并部署在 x86_64 + NVIDIA GPU 节点上;
- ARM 节点的网络连通性(到 master / Prometheus)需要运维侧提前规划;
本包不做网络路由/防火墙配置,仅假定可以直接访问 `MASTER_ENDPOINT`。

View File

@@ -0,0 +1,140 @@
#!/usr/bin/env bash
set -euo pipefail
# ARM Client-CPU 配置脚本:
# - 读取 server 侧生成的 cluster-info.env(包含 SWARM_MANAGER_ADDR / SWARM_JOIN_TOKEN_WORKER)
# - 加入 Swarm(worker)
# - 预热 overlay 网络,确保能通过 master.argus.com 访问 server
# - 生成/更新 compose/.env(填充 Swarm 相关字段,保留人工填写的 AGENT_*、CPU_NODE_HOSTNAME 等)
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_EX="$PKG_ROOT/compose/.env.example"
ENV_OUT="$PKG_ROOT/compose/.env"
info(){ echo -e "\033[34m[CONFIG-ARM-CLIENT]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
# Compose 检测:优先 docker compose(v2),回退 docker-compose(v1)
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker curl jq awk sed tar gzip
require_compose
[[ -f "$ENV_EX" ]] || { err "缺少模板文件: $ENV_EX"; exit 1; }
# 磁盘空间检查(MB)
check_disk(){ local p="$1"; local need=5120; local free
free=$(df -Pm "$p" | awk 'NR==2{print $4+0}')
if [[ -z "$free" || "$free" -lt "$need" ]]; then err "磁盘空间不足: $p 剩余 ${free:-0}MB (<${need}MB)"; fi
}
check_disk "$PKG_ROOT" || true
check_disk "/var/lib/docker" || true
# 导入 cluster-info.env(默认取当前包根,也可用 CLUSTER_INFO 指定路径)
CI_IN="${CLUSTER_INFO:-$PKG_ROOT/cluster-info.env}"
NET_NAME="${ARGUS_OVERLAY_NET:-argus-sys-net}"
SWARM_AVAILABLE=true
if [[ -f "$CI_IN" ]]; then
info "读取 cluster-info.env: $CI_IN"
set -a; source "$CI_IN"; set +a
# 尝试加入 Swarm(幂等),允许 join-token 为空
if [[ -n "${SWARM_MANAGER_ADDR:-}" && -n "${SWARM_JOIN_TOKEN_WORKER:-}" ]]; then
info "尝试加入 Swarm(幂等):$SWARM_MANAGER_ADDR"
docker swarm join --token "$SWARM_JOIN_TOKEN_WORKER" "$SWARM_MANAGER_ADDR":2377 >/dev/null 2>&1 || true
fi
# 检查当前节点 Swarm 状态
if ! docker info 2>/dev/null | grep -q "Swarm: active"; then
info "检测到当前节点 Swarm 未启用,client_arm 将退回本地 network 模式"
SWARM_AVAILABLE=false
fi
else
info "未找到 cluster-info.env($CI_IN),client_arm 将以本地 network 模式运行"
SWARM_AVAILABLE=false
fi
# 生成/更新 .env(保留人工填写项,不覆盖已有键)
if [[ ! -f "$ENV_OUT" ]]; then
cp "$ENV_EX" "$ENV_OUT"
fi
set_kv(){ local k="$1" v="$2"; if grep -q "^${k}=" "$ENV_OUT"; then sed -i -E "s#^${k}=.*#${k}=${v}#" "$ENV_OUT"; else echo "${k}=${v}" >> "$ENV_OUT"; fi }
if [[ "$SWARM_AVAILABLE" == true ]]; then
# ===== Swarm 模式:强依赖 overlay 网络和 DNS alias =====
# 导入 busybox 并做 overlay 预热与连通性
if ! docker image inspect busybox:latest >/dev/null 2>&1; then
if [[ -f "$PKG_ROOT/images/busybox.tar" ]]; then
info "加载 busybox.tar 以预热 overlay"
docker load -i "$PKG_ROOT/images/busybox.tar" >/dev/null
else
err "缺少 busybox 镜像(包内 images/busybox.tar 或本地 busybox:latest),无法预热 overlay $NET_NAME"; exit 1
fi
fi
docker rm -f argus-net-warmup >/dev/null 2>&1 || true
info "启动 warmup 容器加入 overlay: $NET_NAME"
docker run -d --rm --name argus-net-warmup --network "$NET_NAME" busybox:latest sleep 600 >/dev/null 2>&1 || true
for i in {1..60}; do docker network inspect "$NET_NAME" >/dev/null 2>&1 && { info "overlay 可见 (t=${i}s)"; break; }; sleep 1; done
docker network inspect "$NET_NAME" >/dev/null 2>&1 || { err "预热后仍未看到 overlay: $NET_NAME;请确认 server 侧 overlay 已创建且可达"; exit 1; }
if ! docker exec argus-net-warmup sh -lc "ping -c 1 -W 2 master.argus.com >/dev/null 2>&1"; then
err "warmup 容器内无法通过别名访问 master.argus.com;请确认 server compose 已启动并加入 overlay $NET_NAME"
exit 1
fi
info "warmup 容器内可达 master.argus.com(Docker DNS + alias 正常)"
# 写入与 Swarm 相关的字段,便于后续诊断
set_kv SWARM_MANAGER_ADDR "${SWARM_MANAGER_ADDR:-}"
set_kv SWARM_JOIN_TOKEN_WORKER "${SWARM_JOIN_TOKEN_WORKER:-}"
set_kv SWARM_JOIN_TOKEN_MANAGER "${SWARM_JOIN_TOKEN_MANAGER:-}"
# 若未显式设置 MASTER_ENDPOINT,则默认走 overlay 别名
if ! grep -q '^MASTER_ENDPOINT=' "$ENV_OUT"; then
echo "MASTER_ENDPOINT=http://master.argus.com:3000" >> "$ENV_OUT"
fi
else
# ===== 本地 network 退化模式:不依赖 Swarm 和 overlay =====
if ! docker network inspect "$NET_NAME" >/dev/null 2>&1; then
info "创建本地 bridge 网络: $NET_NAME(client_arm 退化模式)"
docker network create "$NET_NAME" >/dev/null
else
info "本地网络已存在: $NET_NAME"
fi
# 若未显式设置 MASTER_ENDPOINT,则提示用户后续手动配置
if ! grep -q '^MASTER_ENDPOINT=' "$ENV_OUT"; then
echo "MASTER_ENDPOINT=" >> "$ENV_OUT"
fi
ME=$(grep -E '^MASTER_ENDPOINT=' "$ENV_OUT" | head -1 | cut -d= -f2-)
if [[ -z "$ME" ]]; then
err "本地 network 模式下必须配置 MASTER_ENDPOINT(示例:http://<Server-IP>:<Port>),请编辑 compose/.env 后重试"
else
info "当前为本地 network 模式,metric-cpu-node 将通过 MASTER_ENDPOINT=${ME} 访问 master"
fi
fi
# 检查用户必须填写的字段
REQ_VARS=(AGENT_ENV AGENT_USER AGENT_INSTANCE CPU_NODE_HOSTNAME)
missing=()
for v in "${REQ_VARS[@]}"; do
val=$(grep -E "^$v=" "$ENV_OUT" | head -1 | cut -d= -f2-)
if [[ -z "$val" ]]; then missing+=("$v"); fi
done
if [[ ${#missing[@]} -gt 0 ]]; then
err "以下变量必须在 compose/.env 中填写:${missing[*]}(已保留你现有的内容,不会被覆盖)"
exit 1
fi
info "已生成 compose/.env;可执行 scripts/install.sh 启动 ARM Client-CPU"

View File

@@ -0,0 +1,48 @@
#!/usr/bin/env bash
set -euo pipefail
# ARM Client-CPU 安装脚本:
# - 导入 ARM CPU node 镜像
# - 启动 compose 中的 metric-cpu-node 服务
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
info(){ echo -e "\033[34m[INSTALL-ARM-CLIENT]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker
require_compose
[[ -f "$ENV_FILE" ]] || { err "缺少 compose/.env,请先运行 scripts/config.sh"; exit 1; }
info "使用环境文件: $ENV_FILE"
set -a; source "$ENV_FILE"; set +a
# 导入 ARM CPU node 镜像
IMG_TGZ=$(ls -1 "$PKG_ROOT"/images/argus-sys-metric-test-node-arm64-*.tar.gz 2>/dev/null | head -1 || true)
[[ -n "$IMG_TGZ" ]] || { err "找不到 ARM CPU node 镜像 tar.gz(argus-sys-metric-test-node-arm64-*.tar.gz)"; exit 1; }
info "导入 ARM CPU node 镜像: $(basename "$IMG_TGZ")"
tmp=$(mktemp); gunzip -c "$IMG_TGZ" > "$tmp"; docker load -i "$tmp" >/dev/null; rm -f "$tmp"
# 确保 agent 挂载目录存在
mkdir -p "$PKG_ROOT/private/argus/agent"
# 启动 compose
PROJECT="${COMPOSE_PROJECT_NAME:-argus-client-arm}"
info "启动 ARM CPU 节点 (docker compose -p $PROJECT up -d)"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" up -d
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" ps
info "ARM Client-CPU 安装完成。可通过 docker logs -f argus-metric-cpu-node 查看节点容器日志。"

View File

@@ -0,0 +1,28 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
if [[ -f "$ENV_FILE" ]]; then
set -a; source "$ENV_FILE"; set +a
fi
PROJECT="${COMPOSE_PROJECT_NAME:-argus-client-arm}"
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require_compose
echo "[UNINSTALL-ARM-CLIENT] stopping compose (project=$PROJECT)"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" down --remove-orphans || true
echo "[UNINSTALL-ARM-CLIENT] done"

View File

@@ -0,0 +1,125 @@
#!/usr/bin/env bash
set -euo pipefail
# x86 上安装 ARM Client-CPU 的配置脚本:
# 逻辑与 ARM 版基本一致(读取 cluster-info、加入 Swarm、预热 overlay、生成 .env),只是日志前缀不同。
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_EX="$PKG_ROOT/compose/.env.example"
ENV_OUT="$PKG_ROOT/compose/.env"
info(){ echo -e "\033[34m[CONFIG-ARM-CLIENT-X86]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker curl jq awk sed tar gzip
require_compose
[[ -f "$ENV_EX" ]] || { err "缺少模板文件: $ENV_EX"; exit 1; }
# 磁盘空间检查
check_disk(){ local p="$1"; local need=5120; local free
free=$(df -Pm "$p" | awk 'NR==2{print $4+0}')
if [[ -z "$free" || "$free" -lt "$need" ]]; then err "磁盘空间不足: $p 剩余 ${free:-0}MB (<${need}MB)"; fi
}
check_disk "$PKG_ROOT" || true
check_disk "/var/lib/docker" || true
CI_IN="${CLUSTER_INFO:-$PKG_ROOT/cluster-info.env}"
NET_NAME="${ARGUS_OVERLAY_NET:-argus-sys-net}"
SWARM_AVAILABLE=true
if [[ -f "$CI_IN" ]]; then
info "读取 cluster-info.env: $CI_IN"
set -a; source "$CI_IN"; set +a
if [[ -n "${SWARM_MANAGER_ADDR:-}" && -n "${SWARM_JOIN_TOKEN_WORKER:-}" ]]; then
info "加入 Swarm(幂等):$SWARM_MANAGER_ADDR"
docker swarm join --token "$SWARM_JOIN_TOKEN_WORKER" "$SWARM_MANAGER_ADDR":2377 >/dev/null 2>&1 || true
fi
if ! docker info 2>/dev/null | grep -q "Swarm: active"; then
info "检测到当前节点 Swarm 未启用,client_arm 将退回本地 network 模式"
SWARM_AVAILABLE=false
fi
else
info "未找到 cluster-info.env($CI_IN),client_arm 将以本地 network 模式运行"
SWARM_AVAILABLE=false
fi
if [[ ! -f "$ENV_OUT" ]]; then
cp "$ENV_EX" "$ENV_OUT"
fi
set_kv(){ local k="$1" v="$2"; if grep -q "^${k}=" "$ENV_OUT"; then sed -i -E "s#^${k}=.*#${k}=${v}#" "$ENV_OUT"; else echo "${k}=${v}" >> "$ENV_OUT"; fi }
if [[ "$SWARM_AVAILABLE" == true ]]; then
if ! docker image inspect busybox:latest >/dev/null 2>&1; then
if [[ -f "$PKG_ROOT/images/busybox.tar" ]]; then
info "加载 busybox.tar 以预热 overlay"
docker load -i "$PKG_ROOT/images/busybox.tar" >/dev/null
else
err "缺少 busybox 镜像(包内 images/busybox.tar 或本地 busybox:latest),无法预热 overlay $NET_NAME"; exit 1
fi
fi
docker rm -f argus-net-warmup >/dev/null 2>&1 || true
info "启动 warmup 容器加入 overlay: $NET_NAME"
docker run -d --rm --name argus-net-warmup --network "$NET_NAME" busybox:latest sleep 600 >/dev/null 2>&1 || true
for i in {1..60}; do docker network inspect "$NET_NAME" >/dev/null 2>&1 && { info "overlay 可见 (t=${i}s)"; break; }; sleep 1; done
docker network inspect "$NET_NAME" >/dev/null 2>&1 || { err "预热后仍未看到 overlay: $NET_NAME;请确认 server 侧 overlay 已创建且可达"; exit 1; }
if ! docker exec argus-net-warmup sh -lc "ping -c 1 -W 2 master.argus.com >/dev/null 2>&1"; then
err "warmup 容器内无法通过别名访问 master.argus.com;请确认 server compose 已启动并加入 overlay $NET_NAME"
exit 1
fi
info "warmup 容器内可达 master.argus.com(Docker DNS + alias 正常)"
set_kv SWARM_MANAGER_ADDR "${SWARM_MANAGER_ADDR:-}"
set_kv SWARM_JOIN_TOKEN_WORKER "${SWARM_JOIN_TOKEN_WORKER:-}"
set_kv SWARM_JOIN_TOKEN_MANAGER "${SWARM_JOIN_TOKEN_MANAGER:-}"
if ! grep -q '^MASTER_ENDPOINT=' "$ENV_OUT"; then
echo "MASTER_ENDPOINT=http://master.argus.com:3000" >> "$ENV_OUT"
fi
else
if ! docker network inspect "$NET_NAME" >/dev/null 2>&1; then
info "创建本地 bridge 网络: $NET_NAME(client_arm 退化模式)"
docker network create "$NET_NAME" >/dev/null
else
info "本地网络已存在: $NET_NAME"
fi
if ! grep -q '^MASTER_ENDPOINT=' "$ENV_OUT"; then
echo "MASTER_ENDPOINT=" >> "$ENV_OUT"
fi
ME=$(grep -E '^MASTER_ENDPOINT=' "$ENV_OUT" | head -1 | cut -d= -f2-)
if [[ -z "$ME" ]]; then
err "本地 network 模式下必须配置 MASTER_ENDPOINT(示例:http://<Server-IP>:<Port>),请编辑 compose/.env 后重试"
else
info "当前为本地 network 模式,metric-cpu-node 将通过 MASTER_ENDPOINT=${ME} 访问 master"
fi
fi
REQ_VARS=(AGENT_ENV AGENT_USER AGENT_INSTANCE CPU_NODE_HOSTNAME)
missing=()
for v in "${REQ_VARS[@]}"; do
val=$(grep -E "^$v=" "$ENV_OUT" | head -1 | cut -d= -f2-)
if [[ -z "$val" ]]; then missing+=("$v"); fi
done
if [[ ${#missing[@]} -gt 0 ]]; then
err "以下变量必须在 compose/.env 中填写:${missing[*]}(已保留你现有的内容,不会被覆盖)"
exit 1
fi
info "已生成 compose/.env;可执行 scripts_for_x86/install.sh 启动 ARM Client-CPU"

View File

@@ -0,0 +1,76 @@
#!/usr/bin/env bash
set -euo pipefail
# 在 x86_64 主机上安装 ARM Client-CPU:
# 1) 确认/安装 binfmt + QEMU(通过 tonistiigi/binfmt);
# 2) 调用通用的 scripts/install.sh 启动 ARM CPU node 容器。
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
info(){ echo -e "\033[34m[INSTALL-ARM-CLIENT-X86]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR-ARM-CLIENT-X86]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
ensure_binfmt_arm64() {
info "检查当前主机架构: $(uname -m)"
if [[ "$(uname -m)" != "x86_64" && "$(uname -m)" != "amd64" ]]; then
info "当前并非 x86_64($(uname -m)),通常应直接在 ARM 服务器上使用 scripts/install.sh。继续执行但不强制安装 binfmt。"
return 0
fi
require docker
# 优先使用随安装包一起提供的 ubuntu-22.04-arm-test 镜像做自检
local TEST_IMG="ubuntu:22.04"
local TEST_TAR=""
if [[ -d "$ROOT_DIR/images" ]]; then
TEST_TAR=$(ls "$ROOT_DIR"/images/ubuntu-22.04-arm-test*.tar.gz 2>/dev/null | head -n1 || true)
fi
if [[ -n "$TEST_TAR" && ! $(docker image inspect "$TEST_IMG" >/dev/null 2>&1; echo $?) -eq 0 ]]; then
info "从安装包加载 ARM 测试基础镜像: $(basename "$TEST_TAR")"
tmp=$(mktemp)
gunzip -c "$TEST_TAR" > "$tmp"
docker load -i "$tmp" >/dev/null
rm -f "$tmp"
fi
info "验证是否已能运行 linux/arm64 容器..."
if docker run --rm --platform=linux/arm64 "$TEST_IMG" uname -m 2>/dev/null | grep -q 'aarch64'; then
info "检测到本机已支持 ARM64 容器运行 (uname -m = aarch64)"
return 0
fi
info "未检测到 ARM64 运行能力,尝试通过 tonistiigi/binfmt 安装 binfmt/QEMU ..."
local BINFMT_IMG="tonistiigi/binfmt:latest"
local BINFMT_TAR=""
if [[ -d "$ROOT_DIR/images" ]]; then
BINFMT_TAR=$(ls "$ROOT_DIR"/images/tonistiigi-binfmt*.tar.gz 2>/dev/null | head -n1 || true)
fi
if [[ -n "$BINFMT_TAR" && ! $(docker image inspect "$BINFMT_IMG" >/dev/null 2>&1; echo $?) -eq 0 ]]; then
info "从安装包加载 tonistiigi/binfmt 离线镜像: $(basename "$BINFMT_TAR")"
tmp_b=$(mktemp)
gunzip -c "$BINFMT_TAR" > "$tmp_b"
docker load -i "$tmp_b" >/dev/null
rm -f "$tmp_b"
fi
if ! docker run --privileged --rm "$BINFMT_IMG" --install all; then
err "tonistiigi/binfmt 安装失败,请检查 Docker 权限或确认离线镜像是否完整。"
exit 1
fi
info "再次验证 ARM64 容器可运行状态..."
if ! docker run --rm --platform=linux/arm64 "$TEST_IMG" uname -m 2>/dev/null | grep -q 'aarch64'; then
err "安装 binfmt/QEMU 后仍无法运行 linux/arm64 容器,请手工排查 binfmt_misc 配置。"
exit 1
fi
info "ARM64 容器运行能力检查通过。"
}
ensure_binfmt_arm64
exec "$SCRIPT_DIR/../scripts/install.sh" "$@"

View File

@@ -0,0 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
exec "$SCRIPT_DIR/../scripts/uninstall.sh" "$@"

View File

@@ -0,0 +1,38 @@
version: "3.8"
networks:
argus-sys-net:
external: true
services:
metric-gpu-node:
image: ${NODE_GPU_BUNDLE_IMAGE_TAG:-argus-sys-metric-test-node-bundle-gpu:${PKG_VERSION}}
container_name: argus-metric-gpu-node-swarm
hostname: ${GPU_NODE_HOSTNAME}
restart: unless-stopped
privileged: true
runtime: nvidia
environment:
- TZ=Asia/Shanghai
- DEBIAN_FRONTEND=noninteractive
- MASTER_ENDPOINT=${MASTER_ENDPOINT:-http://master.argus.com:3000}
# Fluent Bit / 日志上报目标(固定域名)
- ES_HOST=es.log.argus.com
- ES_PORT=9200
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
- AGENT_ENV=${AGENT_ENV}
- AGENT_USER=${AGENT_USER}
- AGENT_INSTANCE=${AGENT_INSTANCE}
- NVIDIA_VISIBLE_DEVICES=all
- NVIDIA_DRIVER_CAPABILITIES=compute,utility
- GPU_MODE=gpu
networks:
argus-sys-net:
aliases:
- ${AGENT_INSTANCE}.node.argus.com
volumes:
- ../private/argus/agent:/private/argus/agent
- ../logs/infer:/logs/infer
- ../logs/train:/logs/train
command: ["sleep", "infinity"]

View File

@@ -0,0 +1,73 @@
# Argus Client‑GPU 安装指南(deployment_new)
## 一、准备条件(开始前确认)
- GPU 节点安装了 NVIDIA 驱动,`nvidia-smi` 正常;
- Docker & Docker Compose v2 已安装;
- 使用统一账户 `argus`(UID=2133,GID=2015)执行安装,并加入 `docker` 组(如已创建可跳过):
```bash
sudo groupadd --gid 2015 argus || true
sudo useradd --uid 2133 --gid 2015 --create-home --shell /bin/bash argus || true
sudo passwd argus
sudo usermod -aG docker argus
su - argus -c 'id; docker ps >/dev/null && echo OK || echo NO_DOCKER_PERMISSION'
```
后续解压与执行(config/install/uninstall)均使用 `argus` 账户进行。
- 从 Server 安装方拿到 `cluster-info.env`(包含 `SWARM_MANAGER_ADDR/SWARM_JOIN_TOKEN_*`;compose 架构下 BINDIP/FTPIP 不再使用)。
## 二、解包
- `tar -xzf client_gpu_YYYYMMDD.tar.gz`
- 进入目录:`cd client_gpu_YYYYMMDD/`
- 你应当看到:`images/`(GPU bundle、busybox)、`compose/`、`scripts/`、`docs/`。
## 三、配置 config(预热 overlay + 生成 .env)
命令:
```
cp /path/to/cluster-info.env ./ # 或 export CLUSTER_INFO=/abs/path/cluster-info.env
./scripts/config.sh
```
脚本做了什么:
- 读取 `cluster-info.env` 并 `docker swarm join`(幂等);
- 自动用 busybox 预热 external overlay `argus-sys-net`,等待最多 60s 直到本机可见;
- 生成/更新 `compose/.env`:填入 `SWARM_*`,并“保留你已填写的 AGENT_* 与 GPU_NODE_HOSTNAME”(不会覆盖)。
看到什么才算成功:
- 终端输出类似:`已预热 overlay=argus-sys-net 并生成 compose/.env;可执行 scripts/install.sh`;
- `compose/.env` 至少包含:
- `AGENT_ENV/AGENT_USER/AGENT_INSTANCE/GPU_NODE_HOSTNAME`(需要你提前填写);
- `SWARM_MANAGER_ADDR/SWARM_JOIN_TOKEN_*`;
- `NODE_GPU_BUNDLE_IMAGE_TAG=...:YYYYMMDD`。
### 日志映射(重要)
- 容器内 `/logs/infer` 与 `/logs/train` 已映射到包根 `./logs/infer` 与 `./logs/train`:
- 你可以直接在宿主机查看推理/训练日志:`tail -f logs/infer/*.log`、`tail -f logs/train/*.log`;
- install 脚本会自动创建这两个目录。
若提示缺少必填项:
- 打开 `compose/.env` 按提示补齐 `AGENT_*` 与 `GPU_NODE_HOSTNAME`,再次执行 `./scripts/config.sh`(脚本不会覆盖你已填的值)。
## 四、安装 install(加载镜像 + 起容器 + 跟日志)
命令:
```
./scripts/install.sh
```
脚本做了什么:
- 如有必要,先自动预热 overlay;
- 从 `images/` 导入 `argus-sys-metric-test-node-bundle-gpu-*.tar.gz` 到本地 Docker;
- `docker compose up -d` 启动 GPU 节点容器,并自动执行 `docker logs -f argus-metric-gpu-node-swarm` 跟踪安装过程。
看到什么才算成功:
- 日志中出现:`[BOOT] local bundle install OK: version=...` / `dcgm-exporter ... listening` / `node state present: /private/argus/agent/<hostname>/node.json`;
- `docker exec argus-metric-gpu-node-swarm nvidia-smi -L` 能列出 GPU;
- 在 Server 侧 Prometheus `/api/v1/targets` 中,GPU 节点 9100(node-exporter)与 9400(dcgm-exporter)至少其一 up。
## 五、卸载 uninstall
命令:
```
./scripts/uninstall.sh
```
行为:Compose down(如有 .env),并删除 warmup 容器与节点容器。
## 六、常见问题
- `本机未看到 overlay`:config/install 已自动预热;若仍失败,请检查与 manager 的网络连通性以及 manager 上是否已创建 `argus-sys-net`。
- `busybox 缺失`:确保包根 `images/busybox.tar` 在,或主机已有 `busybox:latest`。
- `加入 Swarm 失败`:确认 `cluster-info.env` 的 `SWARM_MANAGER_ADDR` 与 `SWARM_JOIN_TOKEN_WORKER` 正确,或在 manager 上重新 `docker swarm join-token -q worker` 后更新该文件。

Binary file not shown.

View File

@@ -0,0 +1,90 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_EX="$PKG_ROOT/compose/.env.example"
ENV_OUT="$PKG_ROOT/compose/.env"
info(){ echo -e "\033[34m[CONFIG-GPU]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
# Compose 检测:优先 docker compose(v2),回退 docker-compose(v1)
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker curl jq awk sed tar gzip
require_compose
# 磁盘空间检查(MB)
check_disk(){ local p="$1"; local need=10240; local free
free=$(df -Pm "$p" | awk 'NR==2{print $4+0}')
if [[ -z "$free" || "$free" -lt "$need" ]]; then err "磁盘空间不足: $p 剩余 ${free:-0}MB (<${need}MB)"; return 1; fi
}
check_disk "$PKG_ROOT"; check_disk "/var/lib/docker" || true
# 导入 cluster-info.env(默认取当前包根,也可用 CLUSTER_INFO 指定路径)
CI_IN="${CLUSTER_INFO:-$PKG_ROOT/cluster-info.env}"
info "读取 cluster-info.env: $CI_IN"
[[ -f "$CI_IN" ]] || { err "找不到 cluster-info.env(默认当前包根,或设置环境变量 CLUSTER_INFO 指定绝对路径)"; exit 1; }
set -a; source "$CI_IN"; set +a
[[ -n "${SWARM_MANAGER_ADDR:-}" && -n "${SWARM_JOIN_TOKEN_WORKER:-}" ]] || { err "cluster-info.env 缺少 SWARM 信息(SWARM_MANAGER_ADDR/SWARM_JOIN_TOKEN_WORKER)"; exit 1; }
# 加入 Swarm(幂等)
info "加入 Swarm(幂等):$SWARM_MANAGER_ADDR"
docker swarm join --token "$SWARM_JOIN_TOKEN_WORKER" "$SWARM_MANAGER_ADDR":2377 >/dev/null 2>&1 || true
# 导入 busybox 并做 overlay 预热与连通性(总是执行)
NET_NAME="${ARGUS_OVERLAY_NET:-argus-sys-net}"
# 准备 busybox
if ! docker image inspect busybox:latest >/dev/null 2>&1; then
if [[ -f "$PKG_ROOT/images/busybox.tar" ]]; then
info "加载 busybox.tar 以预热 overlay"
docker load -i "$PKG_ROOT/images/busybox.tar" >/dev/null
else
err "缺少 busybox 镜像(包内 images/busybox.tar 或本地 busybox:latest),无法预热 overlay $NET_NAME"; exit 1
fi
fi
# 预热容器(worker 侧加入 overlay 以便本地可见)
docker rm -f argus-net-warmup >/dev/null 2>&1 || true
info "启动 warmup 容器加入 overlay: $NET_NAME"
docker run -d --rm --name argus-net-warmup --network "$NET_NAME" busybox:latest sleep 600 >/dev/null 2>&1 || true
for i in {1..60}; do docker network inspect "$NET_NAME" >/dev/null 2>&1 && { info "overlay 可见 (t=${i}s)"; break; }; sleep 1; done
docker network inspect "$NET_NAME" >/dev/null 2>&1 || { err "预热后仍未看到 overlay: $NET_NAME;请确认 manager 已创建并网络可达"; exit 1; }
# 通过 warmup 容器测试实际数据通路(alias → master)
if ! docker exec argus-net-warmup sh -lc "ping -c 1 -W 2 master.argus.com >/dev/null 2>&1"; then
err "warmup 容器内无法通过别名访问 master.argus.com;请确认 server compose 已启动并加入 overlay $NET_NAME"
exit 1
fi
info "warmup 容器内可达 master.argus.com(Docker DNS + alias 正常)"
# 生成/更新 .env(保留人工填写项,不覆盖已有键)
if [[ ! -f "$ENV_OUT" ]]; then
cp "$ENV_EX" "$ENV_OUT"
fi
set_kv(){ local k="$1" v="$2"; if grep -q "^${k}=" "$ENV_OUT"; then sed -i -E "s#^${k}=.*#${k}=${v}#" "$ENV_OUT"; else echo "${k}=${v}" >> "$ENV_OUT"; fi }
set_kv SWARM_MANAGER_ADDR "${SWARM_MANAGER_ADDR:-}"
set_kv SWARM_JOIN_TOKEN_WORKER "${SWARM_JOIN_TOKEN_WORKER:-}"
set_kv SWARM_JOIN_TOKEN_MANAGER "${SWARM_JOIN_TOKEN_MANAGER:-}"
REQ_VARS=(AGENT_ENV AGENT_USER AGENT_INSTANCE GPU_NODE_HOSTNAME)
missing=()
for v in "${REQ_VARS[@]}"; do
val=$(grep -E "^$v=" "$ENV_OUT" | head -1 | cut -d= -f2-)
if [[ -z "$val" ]]; then missing+=("$v"); fi
done
if [[ ${#missing[@]} -gt 0 ]]; then
err "以下变量必须在 compose/.env 中填写:${missing[*]}(已保留你现有的内容,不会被覆盖)"; exit 1; fi
info "已生成 compose/.env;可执行 scripts/install.sh"
# 准备并赋权宿主日志目录(幂等,便于安装前人工检查/预创建)
mkdir -p "$PKG_ROOT/logs/train" "$PKG_ROOT/logs/infer"
chmod 1777 "$PKG_ROOT/logs/train" "$PKG_ROOT/logs/infer" || true
info "日志目录权限(期待 1777,含粘滞位):"
stat -c '%a %U:%G %n' "$PKG_ROOT/logs/train" "$PKG_ROOT/logs/infer" 2>/dev/null || true

View File

@@ -0,0 +1,72 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
info(){ echo -e "\033[34m[INSTALL-GPU]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
# Compose 检测:优先 docker compose(v2),回退 docker-compose(v1)
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker nvidia-smi
require_compose
[[ -f "$ENV_FILE" ]] || { err "缺少 compose/.env,请先运行 scripts/config.sh"; exit 1; }
info "使用环境文件: $ENV_FILE"
# 预热 overlay(当 config 执行很久之前或容器已被清理时,warmup 可能不存在)
set -a; source "$ENV_FILE"; set +a
NET_NAME="${ARGUS_OVERLAY_NET:-argus-sys-net}"
info "检查 overlay 网络可见性: $NET_NAME"
if ! docker network inspect "$NET_NAME" >/dev/null 2>&1; then
# 如 Overlay 不可见,尝试用 busybox 预热(仅为确保 worker 节点已加入 overlay)
if ! docker image inspect busybox:latest >/dev/null 2>&1; then
if [[ -f "$PKG_ROOT/images/busybox.tar" ]]; then docker load -i "$PKG_ROOT/images/busybox.tar"; else err "缺少 busybox 镜像(images/busybox.tar 或本地 busybox:latest)"; exit 1; fi
fi
docker rm -f argus-net-warmup >/dev/null 2>&1 || true
docker run -d --rm --name argus-net-warmup --network "$NET_NAME" busybox:latest sleep 600 >/dev/null 2>&1 || true
for i in {1..60}; do docker network inspect "$NET_NAME" >/dev/null 2>&1 && break; sleep 1; done
docker network inspect "$NET_NAME" >/dev/null 2>&1 || { err "预热后仍未看到 overlay: $NET_NAME;请确认 manager 已创建并网络可达"; exit 1; }
info "overlay 已可见(warmup=argus-net-warmup)"
fi
# 若本函数内重新创建了 warmup 容器,同样测试一次 alias 数据通路
if docker ps --format '{{.Names}}' | grep -q '^argus-net-warmup$'; then
if ! docker exec argus-net-warmup sh -lc "ping -c 1 -W 2 master.argus.com >/dev/null 2>&1"; then
err "GPU install 阶段:warmup 容器内无法通过别名访问 master.argus.com;请检查 overlay $NET_NAME 与 server 状态"
exit 1
fi
info "GPU install 阶段:warmup 容器内可达 master.argus.com"
fi
# 导入 GPU bundle 镜像
IMG_TGZ=$(ls -1 "$PKG_ROOT"/images/argus-sys-metric-test-node-bundle-gpu-*.tar.gz 2>/dev/null | head -1 || true)
[[ -n "$IMG_TGZ" ]] || { err "找不到 GPU bundle 镜像 tar.gz"; exit 1; }
info "导入 GPU bundle 镜像: $(basename "$IMG_TGZ")"
tmp=$(mktemp); gunzip -c "$IMG_TGZ" > "$tmp"; docker load -i "$tmp" >/dev/null; rm -f "$tmp"
# 确保日志目录存在(宿主侧,用于映射 /logs/infer 与 /logs/train),并赋权 1777(粘滞位)
mkdir -p "$PKG_ROOT/logs/infer" "$PKG_ROOT/logs/train"
chmod 1777 "$PKG_ROOT/logs/infer" "$PKG_ROOT/logs/train" || true
info "日志目录已准备并赋权 1777: logs/infer logs/train"
stat -c '%a %U:%G %n' "$PKG_ROOT/logs/infer" "$PKG_ROOT/logs/train" 2>/dev/null || true
# 启动 compose 并跟踪日志
PROJECT="${COMPOSE_PROJECT_NAME:-argus-client}"
info "启动 GPU 节点 (docker compose -p $PROJECT up -d)"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" up -d
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" ps
# 再次校准宿主日志目录权限,避免容器内脚本对 bind mount 权限回退
chmod 1777 "$PKG_ROOT/logs/infer" "$PKG_ROOT/logs/train" || true
stat -c '%a %U:%G %n' "$PKG_ROOT/logs/infer" "$PKG_ROOT/logs/train" 2>/dev/null || true
info "跟踪节点容器日志(按 Ctrl+C 退出)"
docker logs -f argus-metric-gpu-node-swarm || true

View File

@@ -0,0 +1,36 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
# load COMPOSE_PROJECT_NAME if provided in compose/.env
if [[ -f "$ENV_FILE" ]]; then set -a; source "$ENV_FILE"; set +a; fi
PROJECT="${COMPOSE_PROJECT_NAME:-argus-client}"
info(){ echo -e "\033[34m[UNINSTALL-GPU]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
# Compose 检测:优先 docker compose(v2),回退 docker-compose(v1)
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require_compose
if [[ -f "$ENV_FILE" ]]; then
info "stopping compose project (project=$PROJECT)"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" down --remove-orphans || true
else
info "compose/.env not found; attempting to remove container by name"
fi
# remove warmup container if still running
docker rm -f argus-net-warmup >/dev/null 2>&1 || true
# remove node container if present
docker rm -f argus-metric-gpu-node-swarm >/dev/null 2>&1 || true
info "uninstall completed"

View File

@@ -0,0 +1,169 @@
version: "3.8"
networks:
argus-sys-net:
external: true
services:
master:
image: ${MASTER_IMAGE_TAG:-argus-master:${PKG_VERSION}}
container_name: argus-master-sys
environment:
- OFFLINE_THRESHOLD_SECONDS=180
- ONLINE_THRESHOLD_SECONDS=120
- SCHEDULER_INTERVAL_SECONDS=30
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
ports:
- "${MASTER_PORT:-32300}:3000"
volumes:
- ../private/argus/master:/private/argus/master
- ../private/argus/metric/prometheus:/private/argus/metric/prometheus
- ../private/argus/etc:/private/argus/etc
networks:
argus-sys-net:
aliases:
- master.argus.com
restart: unless-stopped
es:
image: ${ES_IMAGE_TAG:-argus-elasticsearch:${PKG_VERSION}}
container_name: argus-es-sys
environment:
- discovery.type=single-node
- xpack.security.enabled=false
- ES_JAVA_OPTS=-Xms512m -Xmx512m
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
volumes:
- ../private/argus/log/elasticsearch:/private/argus/log/elasticsearch
- ../private/argus/etc:/private/argus/etc
ports:
- "${ES_HTTP_PORT:-9200}:9200"
restart: unless-stopped
networks:
argus-sys-net:
aliases:
- es.log.argus.com
kibana:
image: ${KIBANA_IMAGE_TAG:-argus-kibana:${PKG_VERSION}}
container_name: argus-kibana-sys
environment:
- ELASTICSEARCH_HOSTS=http://es.log.argus.com:9200
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
volumes:
- ../private/argus/log/kibana:/private/argus/log/kibana
- ../private/argus/etc:/private/argus/etc
depends_on: [es]
ports:
- "${KIBANA_PORT:-5601}:5601"
restart: unless-stopped
networks:
argus-sys-net:
aliases:
- kibana.log.argus.com
prometheus:
image: ${PROM_IMAGE_TAG:-argus-metric-prometheus:${PKG_VERSION}}
container_name: argus-prometheus
restart: unless-stopped
environment:
- TZ=Asia/Shanghai
- PROMETHEUS_BASE_PATH=/private/argus/metric/prometheus
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
ports:
- "${PROMETHEUS_PORT:-9090}:9090"
volumes:
- ../private/argus/metric/prometheus:/private/argus/metric/prometheus
- ../private/argus/etc:/private/argus/etc
networks:
argus-sys-net:
aliases:
- prom.metric.argus.com
grafana:
image: ${GRAFANA_IMAGE_TAG:-argus-metric-grafana:${PKG_VERSION}}
container_name: argus-grafana
restart: unless-stopped
environment:
- TZ=Asia/Shanghai
- GRAFANA_BASE_PATH=/private/argus/metric/grafana
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
- GF_SERVER_HTTP_PORT=3000
- GF_LOG_LEVEL=warn
- GF_LOG_MODE=console
- GF_PATHS_PROVISIONING=/private/argus/metric/grafana/provisioning
- GF_AUTH_ANONYMOUS_ENABLED=true
- GF_AUTH_ANONYMOUS_ORG_ROLE=Viewer
ports:
- "${GRAFANA_PORT:-3000}:3000"
volumes:
- ../private/argus/metric/grafana:/private/argus/metric/grafana
- ../private/argus/etc:/private/argus/etc
depends_on: [prometheus]
networks:
argus-sys-net:
aliases:
- grafana.metric.argus.com
alertmanager:
image: ${ALERT_IMAGE_TAG:-argus-alertmanager:${PKG_VERSION}}
container_name: argus-alertmanager
environment:
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
volumes:
- ../private/argus/etc:/private/argus/etc
- ../private/argus/alert/alertmanager:/private/argus/alert/alertmanager
networks:
argus-sys-net:
aliases:
- alertmanager.alert.argus.com
ports:
- "${ALERTMANAGER_PORT:-9093}:9093"
restart: unless-stopped
web-frontend:
image: ${FRONT_IMAGE_TAG:-argus-web-frontend:${PKG_VERSION}}
container_name: argus-web-frontend
environment:
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
- EXTERNAL_MASTER_PORT=${WEB_PROXY_PORT_8085:-8085}
- EXTERNAL_ALERTMANAGER_PORT=${WEB_PROXY_PORT_8084:-8084}
- EXTERNAL_GRAFANA_PORT=${WEB_PROXY_PORT_8081:-8081}
- EXTERNAL_PROMETHEUS_PORT=${WEB_PROXY_PORT_8082:-8082}
- EXTERNAL_KIBANA_PORT=${WEB_PROXY_PORT_8083:-8083}
volumes:
- ../private/argus/etc:/private/argus/etc
networks:
argus-sys-net:
aliases:
- web.argus.com
restart: unless-stopped
web-proxy:
image: ${WEB_PROXY_IMAGE_TAG:-argus-web-proxy:${PKG_VERSION}}
container_name: argus-web-proxy
depends_on: [master, grafana, prometheus, kibana, alertmanager]
environment:
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
volumes:
- ../private/argus/etc:/private/argus/etc
networks:
argus-sys-net:
aliases:
- proxy.argus.com
ports:
- "${WEB_PROXY_PORT_8080:-8080}:8080"
- "${WEB_PROXY_PORT_8081:-8081}:8081"
- "${WEB_PROXY_PORT_8082:-8082}:8082"
- "${WEB_PROXY_PORT_8083:-8083}:8083"
- "${WEB_PROXY_PORT_8084:-8084}:8084"
- "${WEB_PROXY_PORT_8085:-8085}:8085"
restart: unless-stopped

View File

@@ -0,0 +1,102 @@
# Argus Server 安装指南(deployment_new)
适用:通过 Server 安装包在 Docker Swarm + external overlay 网络一体化部署 Argus 服务端组件。
—— 本文强调“怎么做、看什么、符合了才继续”。
## 一、准备条件(开始前确认)
- Docker 与 Docker Compose v2 已安装;`docker info` 正常;`docker compose version` 可执行。
- 具备 root/sudo 权限;磁盘可用空间 ≥ 10GB(包根与 `/var/lib/docker`)。
- 你知道本机管理地址(SWARM_MANAGER_ADDR),该 IP 属于本机某网卡,可被其他节点访问。
- 很重要:以统一账户 `argus`(UID=2133,GID=2015)执行后续安装与运维,并将其加入 `docker` 组;示例命令如下(如需不同 UID/GID,请替换为贵方标准):
```bash
# 1) 创建主组(GID=2015,组名 argus;若已存在可跳过)
sudo groupadd --gid 2015 argus || true
# 2) 创建用户 argus(UID=2133、主组 GID=2015,创建家目录并用 bash 作为默认 shell;若已存在可用 usermod 调整)
sudo useradd --uid 2133 --gid 2015 --create-home --shell /bin/bash argus || true
sudo passwd argus
# 3) 将 argus 加入 docker 组,使其能调用 Docker Daemon(新登录后生效)
sudo usermod -aG docker argus
# 4) 验证(重新登录或执行 newgrp docker 使组生效)
su - argus -c 'id; docker ps >/dev/null && echo OK || echo NO_DOCKER_PERMISSION'
```
后续的解压与执行(config/install/selfcheck 等)均使用该 `argus` 账户进行。
## 二、解包与目录结构
- 解压:`tar -xzf server_YYYYMMDD.tar.gz`。
- 进入:`cd server_YYYYMMDD/`
- 你应当能看到:
- `images/`(逐服务镜像 tar.gz,如 `argus-master-YYYYMMDD.tar.gz`)
- `compose/`(`docker-compose.yml` 与 `.env.example`)
- `scripts/`(安装/运维脚本)
- `private/argus/`(数据与配置骨架)
- `docs/`(中文文档)
## 三、配置 config(生成 .env 与 SWARM_MANAGER_ADDR)
命令:
```
export SWARM_MANAGER_ADDR=<本机管理IP>
./scripts/config.sh
```
脚本做了什么:
- 检查依赖与磁盘空间;
- 自动从“端口 20000 起”分配所有服务端口,确保“系统未占用”且“彼此不冲突”;
- 写入 `compose/.env`(包含端口、镜像 tag、FTP 账号、overlay 名称等);
- 将当前执行账户的 UID/GID 写入 `ARGUS_BUILD_UID/GID`(若主组名是 docker,会改用“与用户名同名的组”的 GID,避免拿到 docker 组 999);
- 更新/追加 `cluster-info.env` 中的 `SWARM_MANAGER_ADDR`(不会覆盖其他键)。
看到什么才算成功:
- 终端输出:`已生成 compose/.env 并更新 cluster-info.env 的 SWARM_MANAGER_ADDR。`
- `compose/.env` 打开应当看到:
- 端口均 ≥20000 且没有重复;
- `ARGUS_BUILD_UID/GID` 与 `id -u/-g` 一致;
- `SWARM_MANAGER_ADDR=<你的IP>`。
遇到问题:
- 端口被异常占用:可删去 `.env` 后再次执行 `config.sh`,或手工编辑端口再执行 `install.sh`。
## 四、安装 install(一次到位)
命令:
```
./scripts/install.sh
```
脚本做了什么:
- 若 Swarm 未激活:执行 `docker swarm init --advertise-addr $SWARM_MANAGER_ADDR`;
- 确保 external overlay `argus-sys-net` 存在;
- 导入 `images/*.tar.gz` 到本机 Docker;
- `docker compose up -d` 启动服务;
- 等待“六项就绪”:
- Master `/readyz`=200、ES `/_cluster/health`=200、Prometheus TCP 可达、Grafana `/api/health`=200、Alertmanager `/api/v2/status`=200、Kibana `/api/status` level=available;
- 将各服务 overlay IP 写入 `private/argus/etc/<域名>`,Reload Bind9 与 Nginx;
- 写出 `cluster-info.env`(含 `SWARM_JOIN_TOKEN_{WORKER,MANAGER}/SWARM_MANAGER_ADDR`;compose 架构下不再依赖 BINDIP/FTPIP);
- 生成 `安装报告_YYYYMMDD-HHMMSS.md`(端口、健康检查摘要与提示)。
看到什么才算成功:
- `docker compose ps` 全部是 Up;
- `安装报告_…md` 中各项 HTTP 检查为 200/available;
- `cluster-info.env` 包含五个关键键:
- `SWARM_MANAGER_ADDR=...`
- `SWARM_MANAGER_ADDR=...` `SWARM_JOIN_TOKEN_*=...`
- `SWARM_JOIN_TOKEN_WORKER=SWMTKN-...`
- `SWARM_JOIN_TOKEN_MANAGER=SWMTKN-...`
## 五、健康自检与常用操作
- 健康自检:`./scripts/selfcheck.sh`
- 期望输出:`selfcheck OK -> logs/selfcheck.json`
- 文件 `logs/selfcheck.json` 中 `overlay_net/es/kibana/master_readyz/ftp_share_writable/prometheus/grafana/alertmanager/web_proxy_cors` 为 true。
- 状态:`./scripts/status.sh`(相当于 `docker compose ps`)。
- 诊断:`./scripts/diagnose.sh`(收集容器/HTTP/CORS/ES 细节,输出到 `logs/diagnose_*.log`)。
- 卸载:`./scripts/uninstall.sh`(Compose down)。
- ES 磁盘水位临时放宽/还原:`./scripts/es-watermark-relax.sh` / `./scripts/es-watermark-restore.sh`。
## 六、下一步:分发 cluster-info.env 给 Client
- 将 `cluster-info.env` 拷贝给安装 Client 的同事;
- 对方在 Client 机器的包根放置该文件(或设置 `CLUSTER_INFO=/绝对路径`)即可。
## 七、故障排查快览
- Proxy 502 或 8080 连接复位:多因 Bind 域名未更新到 overlay IP;重跑 `install.sh`(会写入私有域名文件并 reload)或查看 `logs/diagnose_error.log`。
- Kibana 不 available:等待 1–2 分钟、查看 `argus-kibana-sys` 日志;
- cluster-info.env 的 SWARM_MANAGER_ADDR 为空:重新 `export SWARM_MANAGER_ADDR=<IP>; ./scripts/config.sh` 或 `./scripts/install.sh`(会回读 `.env` 补写)。

View File

@@ -0,0 +1,7 @@
# Docker Swarm 部署要点
- 初始化 Swarm:`docker swarm init --advertise-addr <SWARM_MANAGER_ADDR>`
- 创建 overlay:`docker network create --driver overlay --attachable argus-sys-net`
- Server 包 `install.sh` 自动完成上述操作;如需手动执行,确保 `argus-sys-net` 存在且 attachable。
- Worker 节点加入:`docker swarm join --token <worker_token> <SWARM_MANAGER_ADDR>:2377`。

View File

@@ -0,0 +1,11 @@
# 故障排查(Server)
- 端口占用:查看 `安装报告_*.md` 中端口表;如需修改,编辑 `compose/.env` 后执行 `docker compose ... up -d`。
- 组件未就绪:
- Master: `curl http://127.0.0.1:${MASTER_PORT}/readyz -I`
- ES: `curl http://127.0.0.1:${ES_HTTP_PORT}/_cluster/health`
- Grafana: `curl http://127.0.0.1:${GRAFANA_PORT}/api/health`
- Prometheus TCP: `exec 3<>/dev/tcp/127.0.0.1/${PROMETHEUS_PORT}`
- 域名解析:进入 `argus-web-proxy` 或 `argus-master-sys` 容器:`getent hosts master.argus.com`。
- Swarm/Overlay:检查 `docker network ls | grep argus-sys-net`,或 `docker node ls`。

View File

@@ -0,0 +1,111 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_EX="$PKG_ROOT/compose/.env.example"
ENV_OUT="$PKG_ROOT/compose/.env"
info(){ echo -e "\033[34m[CONFIG]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
# Compose 检测:优先 docker compose(v2),回退 docker-compose(v1)
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker curl jq awk sed tar gzip
require_compose
# 磁盘空间检查(MB)
check_disk(){ local p="$1"; local need=10240; local free
free=$(df -Pm "$p" | awk 'NR==2{print $4+0}')
if [[ -z "$free" || "$free" -lt "$need" ]]; then err "磁盘空间不足: $p 剩余 ${free:-0}MB (<${need}MB)"; return 1; fi
}
check_disk "$PKG_ROOT"; check_disk "/var/lib/docker" || true
# 读取/生成 SWARM_MANAGER_ADDR
SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR:-}
if [[ -z "${SWARM_MANAGER_ADDR}" ]]; then
read -rp "请输入本机管理地址 SWARM_MANAGER_ADDR: " SWARM_MANAGER_ADDR
fi
info "SWARM_MANAGER_ADDR=$SWARM_MANAGER_ADDR"
# 校验 IP 属于本机网卡
if ! ip -o addr | awk '{print $4}' | cut -d'/' -f1 | grep -qx "$SWARM_MANAGER_ADDR"; then
err "SWARM_MANAGER_ADDR 非本机地址: $SWARM_MANAGER_ADDR"; exit 1; fi
info "开始分配服务端口(起始=20000,避免系统占用与相互冲突)"
is_port_used(){ local p="$1"; ss -tulnH 2>/dev/null | awk '{print $5}' | sed 's/.*://g' | grep -qx "$p"; }
declare -A PRESENT=() CHOSEN=() USED=()
START_PORT="${START_PORT:-20000}"; cur=$START_PORT
ORDER=(MASTER_PORT ES_HTTP_PORT KIBANA_PORT PROMETHEUS_PORT GRAFANA_PORT ALERTMANAGER_PORT \
WEB_PROXY_PORT_8080 WEB_PROXY_PORT_8081 WEB_PROXY_PORT_8082 WEB_PROXY_PORT_8083 WEB_PROXY_PORT_8084 WEB_PROXY_PORT_8085 \
FTP_PORT FTP_DATA_PORT)
# 标记 .env.example 中实际存在的键
for key in "${ORDER[@]}"; do
if grep -q "^${key}=" "$ENV_EX"; then PRESENT[$key]=1; fi
done
next_free(){ local p="$1"; while :; do if [[ -n "${USED[$p]:-}" ]] || is_port_used "$p"; then p=$((p+1)); else echo "$p"; return; fi; done; }
for key in "${ORDER[@]}"; do
[[ -z "${PRESENT[$key]:-}" ]] && continue
p=$(next_free "$cur"); CHOSEN[$key]="$p"; USED[$p]=1; cur=$((p+1))
done
info "端口分配结果:MASTER=${CHOSEN[MASTER_PORT]:-} ES=${CHOSEN[ES_HTTP_PORT]:-} KIBANA=${CHOSEN[KIBANA_PORT]:-} PROM=${CHOSEN[PROMETHEUS_PORT]:-} GRAFANA=${CHOSEN[GRAFANA_PORT]:-} ALERT=${CHOSEN[ALERTMANAGER_PORT]:-} WEB_PROXY(8080..8085)=${CHOSEN[WEB_PROXY_PORT_8080]:-}/${CHOSEN[WEB_PROXY_PORT_8081]:-}/${CHOSEN[WEB_PROXY_PORT_8082]:-}/${CHOSEN[WEB_PROXY_PORT_8083]:-}/${CHOSEN[WEB_PROXY_PORT_8084]:-}/${CHOSEN[WEB_PROXY_PORT_8085]:-}"
cp "$ENV_EX" "$ENV_OUT"
# 覆盖端口(按唯一化结果写回)
for key in "${ORDER[@]}"; do
val="${CHOSEN[$key]:-}"
[[ -z "$val" ]] && continue
sed -i -E "s#^$key=.*#$key=${val}#" "$ENV_OUT"
done
info "已写入 compose/.env 的端口配置"
# 覆盖/补充 Overlay 名称
grep -q '^ARGUS_OVERLAY_NET=' "$ENV_OUT" || echo 'ARGUS_OVERLAY_NET=argus-sys-net' >> "$ENV_OUT"
# FTP 默认
grep -q '^FTP_USER=' "$ENV_OUT" || echo 'FTP_USER=ftpuser' >> "$ENV_OUT"
grep -q '^FTP_PASSWORD=' "$ENV_OUT" || echo 'FTP_PASSWORD=NASPlab1234!' >> "$ENV_OUT"
# 以当前执行账户 UID/GID 写入(避免误选 docker 组)
RUID=$(id -u)
PRIMARY_GID=$(id -g)
PRIMARY_GRP=$(id -gn)
USER_NAME=$(id -un)
# 若主组名被解析为 docker,尝试用与用户名同名的组的 GID;否则回退主 GID
if [[ "$PRIMARY_GRP" == "docker" ]]; then
RGID=$(getent group "$USER_NAME" | awk -F: '{print $3}' 2>/dev/null || true)
[[ -z "$RGID" ]] && RGID="$PRIMARY_GID"
else
RGID="$PRIMARY_GID"
fi
info "使用构建账户 UID:GID=${RUID}:${RGID} (user=$USER_NAME primary_group=$PRIMARY_GRP)"
if grep -q '^ARGUS_BUILD_UID=' "$ENV_OUT"; then
sed -i -E "s#^ARGUS_BUILD_UID=.*#ARGUS_BUILD_UID=${RUID}#" "$ENV_OUT"
else
echo "ARGUS_BUILD_UID=${RUID}" >> "$ENV_OUT"
fi
if grep -q '^ARGUS_BUILD_GID=' "$ENV_OUT"; then
sed -i -E "s#^ARGUS_BUILD_GID=.*#ARGUS_BUILD_GID=${RGID}#" "$ENV_OUT"
else
echo "ARGUS_BUILD_GID=${RGID}" >> "$ENV_OUT"
fi
CI="$PKG_ROOT/cluster-info.env"
if [[ -f "$CI" ]]; then
if grep -q '^SWARM_MANAGER_ADDR=' "$CI"; then
sed -i -E "s#^SWARM_MANAGER_ADDR=.*#SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}#" "$CI"
else
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}" >> "$CI"
fi
else
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}" > "$CI"
fi
info "已生成 compose/.env 并更新 cluster-info.env 的 SWARM_MANAGER_ADDR。"
info "下一步可执行: scripts/install.sh"

View File

@@ -7,55 +7,39 @@ ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
ENV_FILE="$ROOT/compose/.env"; [[ -f "$ENV_FILE" ]] && set -a && source "$ENV_FILE" && set +a ENV_FILE="$ROOT/compose/.env"; [[ -f "$ENV_FILE" ]] && set -a && source "$ENV_FILE" && set +a
ts="$(date -u +%Y%m%d-%H%M%SZ)" ts="$(date -u +%Y%m%d-%H%M%SZ)"
LOG_DIR="$ROOT/logs" LOG_DIR="$ROOT/logs"; mkdir -p "$LOG_DIR" || true
mkdir -p "$LOG_DIR" || true if ! ( : > "$LOG_DIR/.w" 2>/dev/null ); then LOG_DIR="/tmp/argus-logs"; mkdir -p "$LOG_DIR" || true; fi
# Fallback to /tmp when logs dir is not writable
if ! ( : > "$LOG_DIR/.w" 2>/dev/null ); then # load compose project for accurate ps output
LOG_DIR="/tmp/argus-logs" ENV_FILE="$ROOT/compose/.env"
mkdir -p "$LOG_DIR" || true if [[ -f "$ENV_FILE" ]]; then set -a; source "$ENV_FILE"; set +a; fi
fi PROJECT="${COMPOSE_PROJECT_NAME:-argus-server}"
DETAILS="$LOG_DIR/diagnose_details_${ts}.log" DETAILS="$LOG_DIR/diagnose_details_${ts}.log"; ERRORS="$LOG_DIR/diagnose_error_${ts}.log"; : > "$DETAILS"; : > "$ERRORS"
ERRORS="$LOG_DIR/diagnose_error_${ts}.log"
: > "$DETAILS"; : > "$ERRORS"
logd() { echo "$(date '+%F %T') $*" >> "$DETAILS"; } logd() { echo "$(date '+%F %T') $*" >> "$DETAILS"; }
append_err() { echo "$*" >> "$ERRORS"; } append_err() { echo "$*" >> "$ERRORS"; }
http_code() { curl -s -o /dev/null -w "%{http_code}" "$1" || echo 000; } http_code() { curl -s -o /dev/null -w "%{http_code}" "$1" || echo 000; }
http_body_head() { curl -s --max-time 3 "$1" 2>/dev/null | sed -n '1,5p' || true; } http_body_head() { curl -s --max-time 3 "$1" 2>/dev/null | sed -n '1,5p' || true; }
header_val() { curl -s -D - -o /dev/null "$@" | awk -F': ' 'BEGIN{IGNORECASE=1}$1=="Access-Control-Allow-Origin"{gsub("\r","",$2);print $2}'; } header_val() { curl -s -D - -o /dev/null "$@" | awk -F': ' 'BEGIN{IGNORECASE=1}$1=="Access-Control-Allow-Origin"{gsub("\r","",$2);print $2}'; }
section() { section() { local name="$1"; logd "===== [$name] ====="; }
local name="$1"; logd "===== [$name] ====="; }
svc() { svc() {
local svc_name="$1"; local cname="$2"; shift 2 local svc_name="$1"; local cname="$2"; shift 2
section "$svc_name ($cname)" section "$svc_name ($cname)"
logd "docker ps:"; docker ps -a --format '{{.Names}}\t{{.Status}}\t{{.Image}}' | awk -v n="$cname" '$1==n' >> "$DETAILS" || true logd "docker ps:"; docker ps -a --format '{{.Names}}\t{{.Status}}\t{{.Image}}' | awk -v n="$cname" '$1==n' >> "$DETAILS" || true
logd "docker inspect (state/restartcount):"; docker inspect -f '{{.State.Status}} rc={{.RestartCount}} started={{.State.StartedAt}}' "$cname" >> "$DETAILS" 2>&1 || true logd "docker inspect:"; docker inspect -f '{{.State.Status}} rc={{.RestartCount}} started={{.State.StartedAt}}' "$cname" >> "$DETAILS" 2>&1 || true
logd "last 200 container logs:"; docker logs --tail 200 "$cname" >> "$DETAILS" 2>&1 || true logd "last 200 container logs:"; docker logs --tail 200 "$cname" >> "$DETAILS" 2>&1 || true
docker logs --tail 200 "$cname" 2>&1 | grep -Ei '\\b(error|failed|fail|exception|panic|fatal|critical|unhealthy|permission denied|forbidden|refused|traceback|错误|失败)\\b' | sed "s/^/[${svc_name}][container] /" >> "$ERRORS" || true
# extract error lines from container logs
docker logs --tail 200 "$cname" 2>&1 | \
grep -Ei '\b(error|failed|fail|exception|panic|fatal|critical|unhealthy|permission denied|forbidden|refused|traceback|错误|失败)\b' | \
sed "s/^/[${svc_name}][container] /" >> "$ERRORS" || true
# supervisor status and logs
if docker exec "$cname" sh -lc 'command -v supervisorctl >/dev/null 2>&1' >/dev/null 2>&1; then if docker exec "$cname" sh -lc 'command -v supervisorctl >/dev/null 2>&1' >/dev/null 2>&1; then
logd "supervisorctl status:"; docker exec "$cname" sh -lc 'supervisorctl status' >> "$DETAILS" 2>&1 || true logd "supervisorctl status:"; docker exec "$cname" sh -lc 'supervisorctl status' >> "$DETAILS" 2>&1 || true
# iterate supervisor logs and collect tails + errors per file local files; files=$(docker exec "$cname" sh -lc 'ls /var/log/supervisor/*.log 2>/dev/null' || true)
local files
files=$(docker exec "$cname" sh -lc 'ls /var/log/supervisor/*.log 2>/dev/null' || true)
for f in $files; do for f in $files; do
logd "tail -n 80 $f:"; docker exec "$cname" sh -lc "tail -n 80 $f 2>/dev/null || true" >> "$DETAILS" 2>&1 || true logd "tail -n 80 $f:"; docker exec "$cname" sh -lc "tail -n 80 $f 2>/dev/null || true" >> "$DETAILS" 2>&1 || true
docker exec "$cname" sh -lc "tail -n 200 $f 2>/dev/null" 2>/dev/null | \ docker exec "$cname" sh -lc "tail -n 200 $f 2>/dev/null" 2>/dev/null | grep -Ei '\\b(error|failed|fail|exception|panic|fatal|critical|unhealthy|permission denied|forbidden|refused|traceback|错误|失败)\\b' | sed "s/^/[${svc_name}][supervisor:$(basename $f)] /" >> "$ERRORS" || true
grep -Ei '\b(error|failed|fail|exception|panic|fatal|critical|unhealthy|permission denied|forbidden|refused|traceback|错误|失败)\b' | \
sed "s/^/[${svc_name}][supervisor:$(basename $f)] /" >> "$ERRORS" || true
done done
fi fi
} }
# Core services
svc bind argus-bind-sys svc bind argus-bind-sys
svc master argus-master-sys svc master argus-master-sys
svc es argus-es-sys svc es argus-es-sys
@@ -67,59 +51,48 @@ svc alertmanager argus-alertmanager
svc web-frontend argus-web-frontend svc web-frontend argus-web-frontend
svc web-proxy argus-web-proxy svc web-proxy argus-web-proxy
# HTTP checks (host side)
section HTTP section HTTP
logd "ES: $(http_code "http://localhost:${ES_HTTP_PORT:-9200}/_cluster/health")" logd "ES: $(http_code \"http://localhost:${ES_HTTP_PORT:-9200}/_cluster/health\")"; http_body_head "http://localhost:${ES_HTTP_PORT:-9200}/_cluster/health" >> "$DETAILS" 2>&1 || true
http_body_head "http://localhost:${ES_HTTP_PORT:-9200}/_cluster/health" >> "$DETAILS" 2>&1 || true logd "Kibana: $(http_code \"http://localhost:${KIBANA_PORT:-5601}/api/status\")"; http_body_head "http://localhost:${KIBANA_PORT:-5601}/api/status" >> "$DETAILS" 2>&1 || true
logd "Master readyz: $(http_code \"http://localhost:${MASTER_PORT:-32300}/readyz\")"
logd "Kibana: $(http_code "http://localhost:${KIBANA_PORT:-5601}/api/status")" logd "Prometheus: $(http_code \"http://localhost:${PROMETHEUS_PORT:-9090}/-/ready\")"
http_body_head "http://localhost:${KIBANA_PORT:-5601}/api/status" >> "$DETAILS" 2>&1 || true logd "Grafana: $(http_code \"http://localhost:${GRAFANA_PORT:-3000}/api/health\")"; http_body_head "http://localhost:${GRAFANA_PORT:-3000}/api/health" >> "$DETAILS" 2>&1 || true
logd "Alertmanager: $(http_code \"http://localhost:${ALERTMANAGER_PORT:-9093}/api/v2/status\")"
logd "Master readyz: $(http_code "http://localhost:${MASTER_PORT:-32300}/readyz")"
logd "Prometheus: $(http_code "http://localhost:${PROMETHEUS_PORT:-9090}/-/ready")"
logd "Grafana: $(http_code "http://localhost:${GRAFANA_PORT:-3000}/api/health")"
http_body_head "http://localhost:${GRAFANA_PORT:-3000}/api/health" >> "$DETAILS" 2>&1 || true
logd "Alertmanager: $(http_code "http://localhost:${ALERTMANAGER_PORT:-9093}/api/v2/status")"
cors8084=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8084:-8084}/api/v2/status" || true) cors8084=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8084:-8084}/api/v2/status" || true)
cors8085=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8085:-8085}/api/v1/master/nodes" || true) cors8085=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8085:-8085}/api/v1/master/nodes" || true)
logd "Web-Proxy 8080: $(http_code "http://localhost:${WEB_PROXY_PORT_8080:-8080}/")" logd "Web-Proxy 8080: $(http_code \"http://localhost:${WEB_PROXY_PORT_8080:-8080}/\")"
logd "Web-Proxy 8083: $(http_code "http://localhost:${WEB_PROXY_PORT_8083:-8083}/")" logd "Web-Proxy 8083: $(http_code \"http://localhost:${WEB_PROXY_PORT_8083:-8083}/\")"
logd "Web-Proxy 8084 CORS: ${cors8084}" logd "Web-Proxy 8084 CORS: ${cors8084}"
logd "Web-Proxy 8085 CORS: ${cors8085}" logd "Web-Proxy 8085 CORS: ${cors8085}"
# Overlay network diagnostics section ES-CHECKS
section OVERLAY-NET ch=$(curl -s --max-time 3 "http://localhost:${ES_HTTP_PORT:-9200}/_cluster/health" || true)
if docker network inspect "${OVERLAY_NET_NAME:-argus-sys-net}" >/dev/null 2>&1; then status=$(printf '%s' "$ch" | awk -F'\"' '/"status"/{print $4; exit}')
logd "overlay present: ${OVERLAY_NET_NAME:-argus-sys-net}" if [[ -n "$status" ]]; then logd "cluster.status=$status"; fi
docker network inspect "${OVERLAY_NET_NAME:-argus-sys-net}" | sed -n '1,60p' >> "$DETAILS" 2>/dev/null || true if [[ "$status" != "green" ]]; then append_err "[es][cluster] status=$status"; fi
else if docker ps --format '{{.Names}}' | grep -q '^argus-es-sys$'; then
append_err "[overlay][network] missing ${OVERLAY_NET_NAME:-argus-sys-net}" duse=$(docker exec argus-es-sys sh -lc 'df -P /usr/share/elasticsearch/data | awk "NR==2{print \$5}"' 2>/dev/null || true)
logd "es.data.df_use=$duse"; usep=${duse%%%}
if [[ -n "$usep" ]] && (( usep >= 90 )); then append_err "[es][disk] data path usage=${usep}%"; fi
fi fi
# Domain resolution & reachability from inside web-proxy (bind-backed) section DNS-IN-PROXY
section DOMAIN for d in master.argus.com es.log.argus.com kibana.log.argus.com grafana.metric.argus.com prom.metric.argus.com alertmanager.alert.argus.com; do
for d in master.argus.com ftp.argus.com kibana.argus.com es.log.argus.com prom.argus.com grafana.argus.com; do docker exec argus-web-proxy sh -lc "getent hosts $d || nslookup $d 2>/dev/null | tail -n+1" >> "$DETAILS" 2>&1 || true
logd "getent $d (web-proxy):"
docker exec argus-web-proxy sh -lc "getent hosts $d || true" >> "$DETAILS" 2>&1 || true
done done
logd "HTTP (web-proxy): master.readyz=$(docker exec argus-web-proxy sh -lc \"curl -s -o /dev/null -w '%{http_code}' http://master.argus.com:3000/readyz\" 2>/dev/null || echo 000)" logd "HTTP (web-proxy): master.readyz=$(docker exec argus-web-proxy sh -lc \"curl -s -o /dev/null -w '%{http_code}' http://master.argus.com:3000/readyz\" 2>/dev/null || echo 000)"
logd "HTTP (web-proxy): es.health=$(docker exec argus-web-proxy sh -lc \"curl -s -o /dev/null -w '%{http_code}' http://es.log.argus.com:9200/_cluster/health\" 2>/dev/null || echo 000)" logd "HTTP (web-proxy): es.health=$(docker exec argus-web-proxy sh -lc \"curl -s -o /dev/null -w '%{http_code}' http://es.log.argus.com:9200/_cluster/health\" 2>/dev/null || echo 000)"
logd "HTTP (web-proxy): kibana.status=$(docker exec argus-web-proxy sh -lc \"curl -s -o /dev/null -w '%{http_code}' http://kibana.argus.com:5601/api/status\" 2>/dev/null || echo 000)" logd "HTTP (web-proxy): kibana.status=$(docker exec argus-web-proxy sh -lc \"curl -s -o /dev/null -w '%{http_code}' http://kibana.log.argus.com:5601/api/status\" 2>/dev/null || echo 000)"
# FTP share writability (container perspective)
section FTP-SHARE section FTP-SHARE
docker exec argus-ftp sh -lc 'ls -ld /private/argus/ftp /private/argus/ftp/share; test -w /private/argus/ftp/share && echo "write:OK" || echo "write:FAIL"' >> "$DETAILS" 2>&1 || true docker exec argus-ftp sh -lc 'ls -ld /private/argus/ftp /private/argus/ftp/share; test -w /private/argus/ftp/share && echo "write:OK" || echo "write:FAIL"' >> "$DETAILS" 2>&1 || true
# Collect system info for context
section SYSTEM section SYSTEM
logd "uname -a:"; uname -a >> "$DETAILS" logd "uname -a:"; uname -a >> "$DETAILS"
logd "docker version:"; docker version --format '{{.Server.Version}}' >> "$DETAILS" 2>&1 || true logd "docker version:"; docker version --format '{{.Server.Version}}' >> "$DETAILS" 2>&1 || true
logd "compose ps:"; (cd "$ROOT/compose" && if docker compose version >/dev/null 2>&1; then docker compose -p argus-sys ps; else docker-compose -p argus-sys ps; fi) >> "$DETAILS" 2>&1 || true logd "compose ps (project=$PROJECT):"; (cd "$ROOT/compose" && docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f docker-compose.yml ps) >> "$DETAILS" 2>&1 || true
section SUMMARY section SUMMARY
# Add HTTP failures and CORS problems to error log with tags
[[ $(http_code "http://localhost:${ES_HTTP_PORT:-9200}/_cluster/health") != 200 ]] && echo "[es][http] health not 200" >> "$ERRORS" [[ $(http_code "http://localhost:${ES_HTTP_PORT:-9200}/_cluster/health") != 200 ]] && echo "[es][http] health not 200" >> "$ERRORS"
kbcode=$(http_code "http://localhost:${KIBANA_PORT:-5601}/api/status"); [[ "$kbcode" != 200 ]] && echo "[kibana][http] /api/status=$kbcode" >> "$ERRORS" kbcode=$(http_code "http://localhost:${KIBANA_PORT:-5601}/api/status"); [[ "$kbcode" != 200 ]] && echo "[kibana][http] /api/status=$kbcode" >> "$ERRORS"
[[ $(http_code "http://localhost:${MASTER_PORT:-32300}/readyz") != 200 ]] && echo "[master][http] /readyz not 200" >> "$ERRORS" [[ $(http_code "http://localhost:${MASTER_PORT:-32300}/readyz") != 200 ]] && echo "[master][http] /readyz not 200" >> "$ERRORS"
@@ -128,20 +101,14 @@ gfcode=$(http_code "http://localhost:${GRAFANA_PORT:-3000}/api/health"); [[ "$gf
[[ $(http_code "http://localhost:${ALERTMANAGER_PORT:-9093}/api/v2/status") != 200 ]] && echo "[alertmanager][http] /api/v2/status not 200" >> "$ERRORS" [[ $(http_code "http://localhost:${ALERTMANAGER_PORT:-9093}/api/v2/status") != 200 ]] && echo "[alertmanager][http] /api/v2/status not 200" >> "$ERRORS"
[[ -z "$cors8084" ]] && echo "[web-proxy][cors] 8084 missing Access-Control-Allow-Origin" >> "$ERRORS" [[ -z "$cors8084" ]] && echo "[web-proxy][cors] 8084 missing Access-Control-Allow-Origin" >> "$ERRORS"
[[ -z "$cors8085" ]] && echo "[web-proxy][cors] 8085 missing Access-Control-Allow-Origin" >> "$ERRORS" [[ -z "$cors8085" ]] && echo "[web-proxy][cors] 8085 missing Access-Control-Allow-Origin" >> "$ERRORS"
# Deduplicate errors
sort -u -o "$ERRORS" "$ERRORS" sort -u -o "$ERRORS" "$ERRORS"
echo "Diagnostic details -> $DETAILS" echo "Diagnostic details -> $DETAILS"
echo "Detected errors -> $ERRORS" echo "Detected errors -> $ERRORS"
if [[ "$LOG_DIR" == "$ROOT/logs" ]]; then if [[ "$LOG_DIR" == "$ROOT/logs" ]]; then
# maintain latest symlinks when writing under package logs
ln -sfn "$(basename "$DETAILS")" "$ROOT/logs/diagnose_details.log" 2>/dev/null || cp "$DETAILS" "$ROOT/logs/diagnose_details.log" 2>/dev/null || true ln -sfn "$(basename "$DETAILS")" "$ROOT/logs/diagnose_details.log" 2>/dev/null || cp "$DETAILS" "$ROOT/logs/diagnose_details.log" 2>/dev/null || true
ln -sfn "$(basename "$ERRORS")" "$ROOT/logs/diagnose_error.log" 2>/dev/null || cp "$ERRORS" "$ROOT/logs/diagnose_error.log" 2>/dev/null || true ln -sfn "$(basename "$ERRORS")" "$ROOT/logs/diagnose_error.log" 2>/dev/null || cp "$ERRORS" "$ROOT/logs/diagnose_error.log" 2>/dev/null || true
else
echo "Diagnostic details -> $DETAILS"
echo "Detected errors -> $ERRORS"
fi fi
exit 0 exit 0

View File

@@ -0,0 +1,11 @@
#!/usr/bin/env bash
set -euo pipefail
HOST="${1:-http://127.0.0.1:9200}"
echo "设置 ES watermark 为 95%/96%/97%: $HOST"
curl -fsS -XPUT "$HOST/_cluster/settings" -H 'Content-Type: application/json' -d '{
"transient": {
"cluster.routing.allocation.disk.watermark.low": "95%",
"cluster.routing.allocation.disk.watermark.high": "96%",
"cluster.routing.allocation.disk.watermark.flood_stage": "97%"
}
}' && echo "\nOK"

View File

@@ -0,0 +1,11 @@
#!/usr/bin/env bash
set -euo pipefail
HOST="${1:-http://127.0.0.1:9200}"
echo "恢复 ES watermark 为默认值: $HOST"
curl -fsS -XPUT "$HOST/_cluster/settings" -H 'Content-Type: application/json' -d '{
"transient": {
"cluster.routing.allocation.disk.watermark.low": null,
"cluster.routing.allocation.disk.watermark.high": null,
"cluster.routing.allocation.disk.watermark.flood_stage": null
}
}' && echo "\nOK"

View File

@@ -0,0 +1,137 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
info(){ echo -e "\033[34m[INSTALL]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
# Compose 检测:优先 docker compose(v2),回退 docker-compose(v1)
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/devnull 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker curl jq awk sed tar gzip
require_compose
[[ -f "$ENV_FILE" ]] || { err "缺少 compose/.env,请先运行 scripts/config.sh"; exit 1; }
info "使用环境文件: $ENV_FILE"
set -a; source "$ENV_FILE"; set +a
# 兼容:若 .env 未包含 SWARM_MANAGER_ADDR,则从已存在的 cluster-info.env 读取以避免写空
SMADDR="${SWARM_MANAGER_ADDR:-}"
CI_FILE="$PKG_ROOT/cluster-info.env"
if [[ -z "$SMADDR" && -f "$CI_FILE" ]]; then
SMADDR=$(sed -n 's/^SWARM_MANAGER_ADDR=\(.*\)$/\1/p' "$CI_FILE" | head -n1)
fi
SWARM_MANAGER_ADDR="$SMADDR"
# Swarm init & overlay
if ! docker info 2>/dev/null | grep -q "Swarm: active"; then
[[ -n "${SWARM_MANAGER_ADDR:-}" ]] || { err "SWARM_MANAGER_ADDR 未设置,请在 scripts/config.sh 中配置"; exit 1; }
info "初始化 Swarm (--advertise-addr $SWARM_MANAGER_ADDR)"
docker swarm init --advertise-addr "$SWARM_MANAGER_ADDR" >/dev/null 2>&1 || true
else
info "Swarm 已激活"
fi
NET_NAME="${ARGUS_OVERLAY_NET:-argus-sys-net}"
if ! docker network inspect "$NET_NAME" >/dev/null 2>&1; then
info "创建 overlay 网络: $NET_NAME"
docker network create -d overlay --attachable "$NET_NAME" >/dev/null
else
info "overlay 网络已存在: $NET_NAME"
fi
# Load images
IMAGES_DIR="$PKG_ROOT/images"
shopt -s nullglob
tars=("$IMAGES_DIR"/*.tar.gz)
if [[ ${#tars[@]} -eq 0 ]]; then err "images 目录为空,缺少镜像 tar.gz"; exit 1; fi
total=${#tars[@]}; idx=0
for tgz in "${tars[@]}"; do
idx=$((idx+1))
info "导入镜像 ($idx/$total): $(basename "$tgz")"
tmp=$(mktemp); gunzip -c "$tgz" > "$tmp"; docker load -i "$tmp" >/dev/null; rm -f "$tmp"
done
shopt -u nullglob
# Compose up
PROJECT="${COMPOSE_PROJECT_NAME:-argus-server}"
info "启动服务栈 (docker compose -p $PROJECT up -d)"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" up -d
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" ps
# Wait readiness (best-effort)
code(){ curl -4 -s -o /dev/null -w "%{http_code}" "$1" || echo 000; }
prom_ok(){ (exec 3<>/dev/tcp/127.0.0.1/${PROMETHEUS_PORT:-9090}) >/dev/null 2>&1 && return 0 || return 1; }
kb_ok(){ local body; body=$(curl -s "http://127.0.0.1:${KIBANA_PORT:-5601}/api/status" || true); echo "$body" | grep -q '"level"\s*:\s*"available"'; }
RETRIES=${RETRIES:-60}; SLEEP=${SLEEP:-5}; ok=0
info "等待基础服务就绪 (<= $((RETRIES*SLEEP))s)"
for i in $(seq 1 "$RETRIES"); do
e1=$(code "http://127.0.0.1:${MASTER_PORT:-32300}/readyz")
e2=$(code "http://127.0.0.1:${ES_HTTP_PORT:-9200}/_cluster/health")
e3=000; prom_ok && e3=200
e4=$(code "http://127.0.0.1:${GRAFANA_PORT:-3000}/api/health")
e5=$(code "http://127.0.0.1:${ALERTMANAGER_PORT:-9093}/api/v2/status")
e6=$(kb_ok && echo 200 || echo 000)
info "[ready] t=$((i*SLEEP))s master=$e1 es=$e2 prom=$e3 graf=$e4 alert=$e5 kibana=$e6"
[[ "$e1" == 200 ]] && ok=$((ok+1))
[[ "$e2" == 200 ]] && ok=$((ok+1))
[[ "$e3" == 200 ]] && ok=$((ok+1))
[[ "$e4" == 200 ]] && ok=$((ok+1))
[[ "$e5" == 200 ]] && ok=$((ok+1))
[[ "$e6" == 200 ]] && ok=$((ok+1))
if [[ $ok -ge 6 ]]; then break; fi; ok=0; sleep "$SLEEP"
done
[[ $ok -ge 6 ]] || err "部分服务未就绪(可稍后重试 selfcheck)"
# Swarm join tokens
TOKEN_WORKER=$(docker swarm join-token -q worker 2>/dev/null || echo "")
TOKEN_MANAGER=$(docker swarm join-token -q manager 2>/dev/null || echo "")
# cluster-info.env(compose 场景下不再依赖 BINDIP/FTPIP)
CI="$PKG_ROOT/cluster-info.env"
info "写入 cluster-info.env (manager/token)"
{
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR:-}"
echo "SWARM_JOIN_TOKEN_WORKER=${TOKEN_WORKER:-}"
echo "SWARM_JOIN_TOKEN_MANAGER=${TOKEN_MANAGER:-}"
} > "$CI"
info "已输出 $CI"
# 安装报告
ts=$(date +%Y%m%d-%H%M%S)
RPT="$PKG_ROOT/安装报告_${ts}.md"
{
echo "# Argus Server 安装报告 (${ts})"
echo
echo "## 端口映射"
echo "- MASTER_PORT=${MASTER_PORT}"
echo "- ES_HTTP_PORT=${ES_HTTP_PORT}"
echo "- KIBANA_PORT=${KIBANA_PORT}"
echo "- PROMETHEUS_PORT=${PROMETHEUS_PORT}"
echo "- GRAFANA_PORT=${GRAFANA_PORT}"
echo "- ALERTMANAGER_PORT=${ALERTMANAGER_PORT}"
echo "- WEB_PROXY_PORT_8080=${WEB_PROXY_PORT_8080} ... 8085=${WEB_PROXY_PORT_8085}"
echo
echo "## Swarm/Overlay"
echo "- SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR:-}"
echo "- NET=${NET_NAME}"
echo "- JOIN_TOKEN_WORKER=${TOKEN_WORKER:-}"
echo "- JOIN_TOKEN_MANAGER=${TOKEN_MANAGER:-}"
echo
echo "## 健康检查(简要)"
echo "- master/readyz=$(code http://127.0.0.1:${MASTER_PORT:-32300}/readyz)"
echo "- es/_cluster/health=$(code http://127.0.0.1:${ES_HTTP_PORT:-9200}/_cluster/health)"
echo "- grafana/api/health=$(code http://127.0.0.1:${GRAFANA_PORT:-3000}/api/health)"
echo "- prometheus/tcp=$([[ $(prom_ok; echo $?) == 0 ]] && echo 200 || echo 000)"
echo "- alertmanager/api/v2/status=$(code http://127.0.0.1:${ALERTMANAGER_PORT:-9093}/api/v2/status)"
echo "- kibana/api/status=$([[ $(kb_ok; echo $?) == 0 ]] && echo available || echo not-ready)"
} > "$RPT"
info "已生成报告: $RPT"
info "安装完成。可将 cluster-info.env 分发给 Client-GPU 安装方。"
docker exec argus-web-proxy nginx -t >/dev/null 2>&1 && docker exec argus-web-proxy nginx -s reload >/dev/null 2>&1 || true

View File

@@ -0,0 +1,89 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
log() { echo -e "\033[0;34m[CHECK]\033[0m $*"; }
err() { echo -e "\033[0;31m[ERROR]\033[0m $*" >&2; }
ENV_FILE="$ROOT/compose/.env"; [[ -f "$ENV_FILE" ]] && set -a && source "$ENV_FILE" && set +a
wait_http() { local url="$1"; local attempts=${2:-120}; local i=1; while ((i<=attempts)); do curl -fsS "$url" >/dev/null 2>&1 && return 0; echo "[..] waiting $url ($i/$attempts)"; sleep 5; ((i++)); done; return 1; }
code_for() { curl -s -o /dev/null -w "%{http_code}" "$1" || echo 000; }
header_val() { curl -s -D - -o /dev/null "$@" | awk -F': ' 'BEGIN{IGNORECASE=1}$1=="Access-Control-Allow-Origin"{gsub("\r","",$2);print $2}'; }
LOG_DIR="$ROOT/logs"; mkdir -p "$LOG_DIR" || true
OUT_JSON="$LOG_DIR/selfcheck.json"; tmp=$(mktemp)
ok=1
log "checking overlay network"
net_ok=false
if docker network inspect "${ARGUS_OVERLAY_NET:-argus-sys-net}" >/dev/null 2>&1; then
if docker network inspect "${ARGUS_OVERLAY_NET:-argus-sys-net}" | grep -q '"Driver": "overlay"'; then net_ok=true; fi
fi
[[ "$net_ok" == true ]] || ok=0
log "checking Elasticsearch"
wait_http "http://localhost:${ES_HTTP_PORT:-9200}/_cluster/health" 60 || ok=0
log "checking Kibana"
kb_code=$(code_for "http://localhost:${KIBANA_PORT:-5601}/api/status")
kb_ok=false
if [[ "$kb_code" == 200 ]]; then
body=$(curl -sS "http://localhost:${KIBANA_PORT:-5601}/api/status" || true)
echo "$body" | grep -q '"level"\s*:\s*"available"' && kb_ok=true
fi
[[ "$kb_ok" == true ]] || ok=0
log "checking Master"
[[ $(code_for "http://localhost:${MASTER_PORT:-32300}/readyz") == 200 ]] || ok=0
log "checking FTP"
if docker ps --format '{{.Names}}' | grep -q '^argus-ftp$'; then
docker exec argus-ftp sh -lc 'test -w /private/argus/ftp/share' >/dev/null 2>&1 || ok=0
else ok=0; fi
log "checking Prometheus"
wait_http "http://localhost:${PROMETHEUS_PORT:-9090}/-/ready" 60 || ok=0
log "checking Grafana"
gf_code=$(code_for "http://localhost:${GRAFANA_PORT:-3000}/api/health")
gf_ok=false; if [[ "$gf_code" == 200 ]]; then body=$(curl -sS "http://localhost:${GRAFANA_PORT:-3000}/api/health" || true); echo "$body" | grep -q '"database"\s*:\s*"ok"' && gf_ok=true; fi
[[ "$gf_ok" == true ]] || ok=0
log "checking Alertmanager"
wait_http "http://localhost:${ALERTMANAGER_PORT:-9093}/api/v2/status" 60 || ok=0
log "checking Web-Proxy (CORS)"
cors8084=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8084:-8084}/api/v2/status" || true)
cors8085=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8085:-8085}/api/v1/master/nodes" || true)
wp_ok=true
[[ -n "$cors8084" && -n "$cors8085" ]] || wp_ok=false
[[ "$wp_ok" == true ]] || ok=0
cat > "$tmp" <<JSON
{
"overlay_net": $net_ok,
"es": true,
"kibana": $kb_ok,
"master_readyz": true,
"ftp_share_writable": true,
"prometheus": true,
"grafana": $gf_ok,
"alertmanager": true,
"web_proxy_cors": $wp_ok,
"timestamp": "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
}
JSON
mv "$tmp" "$OUT_JSON" 2>/dev/null || cp "$tmp" "$OUT_JSON"
if [[ "$ok" == 1 ]]; then
log "selfcheck OK -> $OUT_JSON"
exit 0
else
err "selfcheck FAILED -> $OUT_JSON"
exit 1
fi

View File

@@ -0,0 +1,9 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
if [[ -f "$ENV_FILE" ]]; then set -a; source "$ENV_FILE"; set +a; fi
PROJECT="${COMPOSE_PROJECT_NAME:-argus-server}"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" ps

View File

@@ -0,0 +1,23 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
# load COMPOSE_PROJECT_NAME from env file if present
if [[ -f "$ENV_FILE" ]]; then set -a; source "$ENV_FILE"; set +a; fi
PROJECT="${COMPOSE_PROJECT_NAME:-argus-server}"
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
# Compose 检测:优先 docker compose(v2),回退 docker-compose(v1)
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require_compose
echo "[UNINSTALL] stopping compose (project=$PROJECT)"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" down --remove-orphans || true
echo "[UNINSTALL] done"

View File

@@ -0,0 +1,154 @@
version: "3.8"
networks:
argus-sys-net:
external: true
services:
master:
image: ${MASTER_IMAGE_TAG:-argus-master-arm64:${PKG_VERSION}}
platform: linux/arm64
container_name: argus-master-sys
security_opt:
- seccomp:unconfined
environment:
- OFFLINE_THRESHOLD_SECONDS=180
- ONLINE_THRESHOLD_SECONDS=120
- SCHEDULER_INTERVAL_SECONDS=30
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
ports:
- "${MASTER_PORT:-32300}:3000"
volumes:
- ../private/argus/master:/private/argus/master
- ../private/argus/metric/prometheus:/private/argus/metric/prometheus
- ../private/argus/etc:/private/argus/etc
networks:
argus-sys-net:
aliases:
- master.argus.com
restart: unless-stopped
prometheus:
image: ${PROM_IMAGE_TAG:-argus-metric-prometheus-arm64:${PKG_VERSION}}
platform: linux/arm64
container_name: argus-prometheus
restart: unless-stopped
environment:
- TZ=Asia/Shanghai
- PROMETHEUS_BASE_PATH=/private/argus/metric/prometheus
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
ports:
- "${PROMETHEUS_PORT:-9090}:9090"
volumes:
- ../private/argus/metric/prometheus:/private/argus/metric/prometheus
- ../private/argus/etc:/private/argus/etc
networks:
argus-sys-net:
aliases:
- prom.metric.argus.com
prometheus-targets-updater:
image: ${PROM_UPDATER_IMAGE_TAG:-argus-metric-prometheus-targets-updater-arm64:${PKG_VERSION}}
platform: linux/arm64
container_name: argus-prometheus-targets-updater
restart: unless-stopped
environment:
- TZ=Asia/Shanghai
volumes:
- ../private/argus/metric/prometheus:/private/argus/metric/prometheus
networks:
argus-sys-net:
aliases:
- prom-updater.metric.argus.com
depends_on:
- master
- prometheus
grafana:
image: ${GRAFANA_IMAGE_TAG:-argus-metric-grafana-arm64:${PKG_VERSION}}
platform: linux/arm64
container_name: argus-grafana
restart: unless-stopped
environment:
- TZ=Asia/Shanghai
- GRAFANA_BASE_PATH=/private/argus/metric/grafana
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
- GF_SERVER_HTTP_PORT=3000
- GF_LOG_LEVEL=warn
- GF_LOG_MODE=console
- GF_PATHS_PROVISIONING=/private/argus/metric/grafana/provisioning
- GF_AUTH_ANONYMOUS_ENABLED=true
- GF_AUTH_ANONYMOUS_ORG_ROLE=Viewer
ports:
- "${GRAFANA_PORT:-3000}:3000"
volumes:
- ../private/argus/metric/grafana:/private/argus/metric/grafana
- ../private/argus/etc:/private/argus/etc
depends_on: [prometheus]
networks:
argus-sys-net:
aliases:
- grafana.metric.argus.com
alertmanager:
image: ${ALERT_IMAGE_TAG:-argus-alertmanager-arm64:${PKG_VERSION}}
platform: linux/arm64
container_name: argus-alertmanager
environment:
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
volumes:
- ../private/argus/etc:/private/argus/etc
- ../private/argus/alert/alertmanager:/private/argus/alert/alertmanager
networks:
argus-sys-net:
aliases:
- alertmanager.alert.argus.com
ports:
- "${ALERTMANAGER_PORT:-9093}:9093"
restart: unless-stopped
web-frontend:
image: ${FRONT_IMAGE_TAG:-argus-web-frontend-arm64:${PKG_VERSION}}
platform: linux/arm64
container_name: argus-web-frontend
environment:
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
- EXTERNAL_MASTER_PORT=${WEB_PROXY_PORT_8085:-8085}
- EXTERNAL_ALERTMANAGER_PORT=${WEB_PROXY_PORT_8084:-8084}
- EXTERNAL_GRAFANA_PORT=${WEB_PROXY_PORT_8081:-8081}
- EXTERNAL_PROMETHEUS_PORT=${WEB_PROXY_PORT_8082:-8082}
volumes:
- ../private/argus/etc:/private/argus/etc
networks:
argus-sys-net:
aliases:
- web.argus.com
restart: unless-stopped
web-proxy:
image: ${WEB_PROXY_IMAGE_TAG:-argus-web-proxy-arm64:${PKG_VERSION}}
platform: linux/arm64
container_name: argus-web-proxy
depends_on: [master, grafana, prometheus, alertmanager]
environment:
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
volumes:
- ../private/argus/etc:/private/argus/etc
networks:
argus-sys-net:
aliases:
- proxy.argus.com
ports:
- "${WEB_PROXY_PORT_8080:-8080}:8080"
- "${WEB_PROXY_PORT_8081:-8081}:8081"
- "${WEB_PROXY_PORT_8082:-8082}:8082"
- "${WEB_PROXY_PORT_8083:-8083}:8083"
- "${WEB_PROXY_PORT_8084:-8084}:8084"
- "${WEB_PROXY_PORT_8085:-8085}:8085"
restart: unless-stopped

View File

@@ -0,0 +1,158 @@
# ARM Server 安装指南(server_arm)
本包用于在 ARM64 服务器上部署 Argus 的最小 server 端组件集:
- master
- Prometheus
- Prometheus targets-updater sidecar
- Grafana
- Alertmanager
- Web 前端 + Web Proxy
> 注意:本 ARM 版本 **不包含 Elasticsearch / Kibana / FTP / BIND9**,仅提供指标与告警能力。
---
## 1. 前置条件
- 目标机器为 ARM64(例如:aarch64 服务器);
- 已安装 Docker 和 docker compose(v2,支持 `docker compose` 命令);
- 已准备好 overlay 网络 `argus-sys-net`(如不需要也可将 compose 改为 `bridge` 网络)。
创建 overlay 网络示例:
```bash
docker network create --driver overlay argus-sys-net || true
```
如未使用 Swarm,也可以在 compose 中将 `networks: argus-sys-net` 改成本地 `bridge` 网络。
---
## 2. 解压和加载镜像
假设你已经将 `server_arm_YYYYMMDD.tar.gz` 拷贝到目标 ARM 服务器:
```bash
tar -xzf server_arm_YYYYMMDD.tar.gz
cd server_arm/YYYYMMDD
```
加载镜像:
```bash
for f in images/*.tar.gz; do
gunzip -c "$f" | docker load
done
```
加载后可以用:
```bash
docker images | grep 'argus-.*arm64'
```
确认 master / prometheus / grafana / alertmanager / web 镜像已就绪。
---
## 3. 配置 .env
在 `compose/` 目录下有自动生成的 `.env.example`:
```bash
cd compose
cp .env.example .env
```
`.env` 中主要字段:
- `PKG_VERSION`:包版本(不建议修改)。
- 镜像 tag(默认使用构建时的 `*-arm64:${PKG_VERSION}`):
- `MASTER_IMAGE_TAG`
- `PROM_IMAGE_TAG`
- `PROM_UPDATER_IMAGE_TAG`
- `GRAFANA_IMAGE_TAG`
- `ALERT_IMAGE_TAG`
- `FRONT_IMAGE_TAG`
- `WEB_PROXY_IMAGE_TAG`
- 端口:
- `MASTER_PORT`(默认 32300)
- `PROMETHEUS_PORT`(默认 9090)
- `GRAFANA_PORT`(默认 3000)
- `ALERTMANAGER_PORT`(默认 9093)
- `WEB_PROXY_PORT_8080` ~ `WEB_PROXY_PORT_8085`
- 其他:
- `ARGUS_BUILD_UID` / `ARGUS_BUILD_GID`(用于挂载目录的权限对齐)。
根据实际环境按需调整端口和 UID/GID。
---
## 4. 准备挂载目录
在 package 根目录(与 `compose/` 同级)下,已预创建所需的 `private/argus` 目录骨架:
- `private/argus/etc`
- `private/argus/master`
- `private/argus/metric/prometheus`(含 `data` / `rules` / `targets`)
- `private/argus/metric/grafana`(含 `data` / `logs` / `plugins` / `provisioning` 等)
可根据需要调整访问权限,例如:
```bash
cd ..
sudo chown -R $UID:$GID private/argus
```
确保与 `.env` 中的 `ARGUS_BUILD_UID/GID` 一致。
---
## 5. 启动服务
在 `compose/` 目录执行:
```bash
docker compose --env-file .env up -d
```
检查服务状态:
```bash
docker compose ps
```
预期看到:
- `argus-master-sys`
- `argus-prometheus`
- `argus-prometheus-targets-updater`
- `argus-grafana`
- `argus-alertmanager`
- `argus-web-frontend`
- `argus-web-proxy`
---
## 6. 验证
1. Master:
- `curl http://localhost:${MASTER_PORT}/readyz`
2. Prometheus:
- 浏览器访问:`http://<host>:${PROMETHEUS_PORT}`
3. Grafana:
- 浏览器访问:`http://<host>:${GRAFANA_PORT}`
4. Web 门户:
- 浏览器访问:`http://<host>:${WEB_PROXY_PORT_8080}`
确认这些页面能够正常打开,并且在没有任何节点注册时,Prometheus 中 `job="node"` 可为空,但基础 UI 应可访问。
---
## 7. 注意事项
- 本 ARM server 包仅包含 metrics + alert 相关组件,不提供日志链路(ES/Kibana),也不包含 DNS/FTP 相关服务;
- 若后续需要扩展 ES/Kibana,可在新的 spec 中单独设计 ARM 日志栈;
- 对于多节点/Swarm 部署,请结合 `src/sys/arm_swarm_tests` 中的 compose 与脚本进行扩展。

View File

@@ -0,0 +1,107 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_EX="$PKG_ROOT/compose/.env.example"
ENV_OUT="$PKG_ROOT/compose/.env"
info(){ echo -e "\033[34m[CONFIG]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
# Compose 检测:优先 docker compose(v2),回退 docker-compose(v1)
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker awk sed
require_compose
[[ -f "$ENV_EX" ]] || { err "缺少模板文件: $ENV_EX"; exit 1; }
# 磁盘空间检查(简化版,默认 >= 5GB)
check_disk(){ local p="$1"; local need="${2:-5120}"; local free;
free=$(df -Pm "$p" 2>/dev/null | awk 'NR==2{print $4+0}')
if [[ -z "$free" || "$free" -lt "$need" ]]; then
err "磁盘空间不足: $p 剩余 ${free:-0}MB (<${need}MB),请清理后再继续"
fi
}
check_disk "$PKG_ROOT" 5120 || true
check_disk "/var/lib/docker" 5120 || true
cp "$ENV_EX" "$ENV_OUT"
# 读取/生成 SWARM_MANAGER_ADDR(ARM 下同样引导 Swarm + cluster-info)
SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR:-}
if [[ -z "${SWARM_MANAGER_ADDR}" ]]; then
read -rp "请输入本机管理地址 SWARM_MANAGER_ADDR(可填当前主机会被其他节点访问到的 IP 或主机名): " SWARM_MANAGER_ADDR
fi
info "SWARM_MANAGER_ADDR=$SWARM_MANAGER_ADDR"
info "开始分配服务端口(起始=20000,避免系统占用与相互冲突)"
is_port_used(){ local p="$1"; ss -tulnH 2>/dev/null | awk '{print $5}' | sed 's/.*://g' | grep -qx "$p"; }
declare -A PRESENT=() CHOSEN=() USED=()
START_PORT="${START_PORT:-20000}"; cur=$START_PORT
ORDER=(MASTER_PORT PROMETHEUS_PORT GRAFANA_PORT ALERTMANAGER_PORT \
WEB_PROXY_PORT_8080 WEB_PROXY_PORT_8081 WEB_PROXY_PORT_8082 \
WEB_PROXY_PORT_8083 WEB_PROXY_PORT_8084 WEB_PROXY_PORT_8085)
# 标记 .env.example 中实际存在的键
for key in "${ORDER[@]}"; do
if grep -q "^${key}=" "$ENV_EX"; then PRESENT[$key]=1; fi
done
next_free(){ local p="$1"; while :; do if [[ -n "${USED[$p]:-}" ]] || is_port_used "$p"; then p=$((p+1)); else echo "$p"; return; fi; done; }
for key in "${ORDER[@]}"; do
[[ -z "${PRESENT[$key]:-}" ]] && continue
p=$(next_free "$cur"); CHOSEN[$key]="$p"; USED[$p]=1; cur=$((p+1))
done
info "端口分配结果:MASTER=${CHOSEN[MASTER_PORT]:-} PROM=${CHOSEN[PROMETHEUS_PORT]:-} GRAFANA=${CHOSEN[GRAFANA_PORT]:-} ALERT=${CHOSEN[ALERTMANAGER_PORT]:-} WEB_PROXY(8080..8085)=${CHOSEN[WEB_PROXY_PORT_8080]:-}/${CHOSEN[WEB_PROXY_PORT_8081]:-}/${CHOSEN[WEB_PROXY_PORT_8082]:-}/${CHOSEN[WEB_PROXY_PORT_8083]:-}/${CHOSEN[WEB_PROXY_PORT_8084]:-}/${CHOSEN[WEB_PROXY_PORT_8085]:-}"
# 覆盖端口(按唯一化结果写回)
for key in "${ORDER[@]}"; do
val="${CHOSEN[$key]:-}"
[[ -z "$val" ]] && continue
sed -i -E "s#^$key=.*#$key=${val}#" "$ENV_OUT"
done
# 覆盖/补充 Overlay 名称
grep -q '^ARGUS_OVERLAY_NET=' "$ENV_OUT" || echo 'ARGUS_OVERLAY_NET=argus-sys-net' >> "$ENV_OUT"
# 覆盖/补充构建账户 UID/GID,避免权限问题
RUID=$(id -u)
RGID=$(id -g)
if grep -q '^ARGUS_BUILD_UID=' "$ENV_OUT"; then
sed -i -E "s#^ARGUS_BUILD_UID=.*#ARGUS_BUILD_UID=${RUID}#" "$ENV_OUT"
else
echo "ARGUS_BUILD_UID=${RUID}" >> "$ENV_OUT"
fi
if grep -q '^ARGUS_BUILD_GID=' "$ENV_OUT"; then
sed -i -E "s#^ARGUS_BUILD_GID=.*#ARGUS_BUILD_GID=${RGID}#" "$ENV_OUT"
else
echo "ARGUS_BUILD_GID=${RGID}" >> "$ENV_OUT"
fi
# 更新 cluster-info.env 的 SWARM_MANAGER_ADDR(ARM 版本也保留该机制)
CI="$PKG_ROOT/cluster-info.env"
if [[ -f "$CI" ]]; then
if grep -q '^SWARM_MANAGER_ADDR=' "$CI"; then
sed -i -E "s#^SWARM_MANAGER_ADDR=.*#SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}#" "$CI"
else
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}" >> "$CI"
fi
else
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}" > "$CI"
fi
info "已生成 compose/.env 并更新 cluster-info.env 的 SWARM_MANAGER_ADDR。"
info "下一步可执行: scripts/install.sh"

View File

@@ -0,0 +1,91 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
ENV_FILE="$ROOT/compose/.env"; [[ -f "$ENV_FILE" ]] && set -a && source "$ENV_FILE" && set +a
ts="$(date -u +%Y%m%d-%H%M%SZ)"
LOG_DIR="$ROOT/logs"; mkdir -p "$LOG_DIR" || true
if ! ( : > "$LOG_DIR/.w" 2>/dev/null ); then LOG_DIR="/tmp/argus-logs"; mkdir -p "$LOG_DIR" || true; fi
PROJECT="${COMPOSE_PROJECT_NAME:-argus-server-arm}"
DETAILS="$LOG_DIR/diagnose_arm_server_details_${ts}.log"
ERRORS="$LOG_DIR/diagnose_arm_server_error_${ts}.log"
: > "$DETAILS"; : > "$ERRORS"
logd() { echo "$(date '+%F %T') $*" >> "$DETAILS"; }
append_err() { echo "$*" >> "$ERRORS"; }
http_code() { curl -s -o /dev/null -w "%{http_code}" "$1" || echo 000; }
http_body_head() { curl -s --max-time 3 "$1" 2>/dev/null | sed -n '1,5p' || true; }
header_val() { curl -s -D - -o /dev/null "$@" | awk -F': ' 'BEGIN{IGNORECASE=1}$1=="Access-Control-Allow-Origin"{gsub("\r","",$2);print $2}'; }
section() { local name="$1"; logd "===== [$name] ====="; }
svc() {
local svc_name="$1"; local cname="$2"
section "$svc_name ($cname)"
logd "docker ps:"; docker ps -a --format '{{.Names}}\t{{.Status}}\t{{.Image}}' | awk -v n="$cname" '$1==n' >> "$DETAILS" || true
logd "docker inspect:"; docker inspect -f '{{.State.Status}} rc={{.RestartCount}} started={{.State.StartedAt}}' "$cname" >> "$DETAILS" 2>&1 || true
logd "last 200 container logs:"; docker logs --tail 200 "$cname" >> "$DETAILS" 2>&1 || true
docker logs --tail 200 "$cname" 2>&1 | \
grep -Ei '\b(error|failed|fail|exception|panic|fatal|critical|unhealthy|permission denied|forbidden|refused|traceback|错误|失败)\b' | \
sed "s/^/[${svc_name}][container] /" >> "$ERRORS" || true
if docker exec "$cname" sh -lc 'command -v supervisorctl >/dev/null 2>&1' >/dev/null 2>&1; then
logd "supervisorctl status:"; docker exec "$cname" sh -lc 'supervisorctl status' >> "$DETAILS" 2>&1 || true
local files; files=$(docker exec "$cname" sh -lc 'ls /var/log/supervisor/*.log 2>/dev/null' || true)
for f in $files; do
logd "tail -n 80 $f:"; docker exec "$cname" sh -lc "tail -n 80 $f 2>/dev/null || true" >> "$DETAILS" 2>&1 || true
docker exec "$cname" sh -lc "tail -n 200 $f 2>/dev/null" 2>/dev/null | \
grep -Ei '\b(error|failed|fail|exception|panic|fatal|critical|unhealthy|permission denied|forbidden|refused|traceback|错误|失败)\b' | \
sed "s/^/[${svc_name}][supervisor:$(basename "$f")] /" >> "$ERRORS" || true
done
fi
}
# 关键服务
svc master argus-master-sys
svc prometheus argus-prometheus
svc prom-updater argus-prometheus-targets-updater
svc grafana argus-grafana
svc alertmanager argus-alertmanager
svc web-frontend argus-web-frontend
svc web-proxy argus-web-proxy
section HTTP
logd "Master: $(http_code \"http://localhost:${MASTER_PORT:-32300}/readyz\")"
logd "Prometheus: $(http_code \"http://localhost:${PROMETHEUS_PORT:-9090}/-/ready\")"
logd "Grafana: $(http_code \"http://localhost:${GRAFANA_PORT:-3000}/api/health\")"; http_body_head "http://localhost:${GRAFANA_PORT:-3000}/api/health" >> "$DETAILS" 2>&1 || true
logd "Alertmanager: $(http_code \"http://localhost:${ALERTMANAGER_PORT:-9093}/api/v2/status\")"
cors8084=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8084:-8084}/api/v2/status" || true)
cors8085=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8085:-8085}/api/v1/master/nodes" || true)
logd "Web-Proxy 8080: $(http_code \"http://localhost:${WEB_PROXY_PORT_8080:-8080}/\")"
logd "Web-Proxy 8083: $(http_code \"http://localhost:${WEB_PROXY_PORT_8083:-8083}/\")"
logd "Web-Proxy 8084 CORS: ${cors8084}"
logd "Web-Proxy 8085 CORS: ${cors8085}"
section SYSTEM
logd "uname -a:"; uname -a >> "$DETAILS"
logd "docker version:"; docker version --format '{{.Server.Version}}' >> "$DETAILS" 2>&1 || true
logd "compose ps (project=$PROJECT):"; (cd "$ROOT/compose" && docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f docker-compose.yml ps) >> "$DETAILS" 2>&1 || true
section SUMMARY
[[ $(http_code "http://localhost:${MASTER_PORT:-32300}/readyz") != 200 ]] && echo "[master][http] /readyz not 200" >> "$ERRORS"
[[ $(http_code "http://localhost:${PROMETHEUS_PORT:-9090}/-/ready") != 200 ]] && echo "[prometheus][http] /-/ready not 200" >> "$ERRORS"
gfcode=$(http_code "http://localhost:${GRAFANA_PORT:-3000}/api/health"); [[ "$gfcode" != 200 ]] && echo "[grafana][http] /api/health=$gfcode" >> "$ERRORS"
[[ $(http_code "http://localhost:${ALERTMANAGER_PORT:-9093}/api/v2/status") != 200 ]] && echo "[alertmanager][http] /api/v2/status not 200" >> "$ERRORS"
[[ -z "$cors8084" ]] && echo "[web-proxy][cors] 8084 missing Access-Control-Allow-Origin" >> "$ERRORS"
[[ -z "$cors8085" ]] && echo "[web-proxy][cors] 8085 missing Access-Control-Allow-Origin" >> "$ERRORS"
sort -u -o "$ERRORS" "$ERRORS"
echo "Diagnostic details -> $DETAILS"
echo "Detected errors -> $ERRORS"
if [[ "$LOG_DIR" == "$ROOT/logs" ]]; then
ln -sfn "$(basename "$DETAILS")" "$ROOT/logs/diagnose_arm_server_details.log" 2>/dev/null || cp "$DETAILS" "$ROOT/logs/diagnose_arm_server_details.log" 2>/dev/null || true
ln -sfn "$(basename "$ERRORS")" "$ROOT/logs/diagnose_arm_server_error.log" 2>/dev/null || cp "$ERRORS" "$ROOT/logs/diagnose_arm_server_error.log" 2>/dev/null || true
fi
exit 0

View File

@@ -0,0 +1,159 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
info(){ echo -e "\033[34m[INSTALL]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker curl jq awk sed tar gzip
require_compose
[[ -f "$ENV_FILE" ]] || { err "缺少 compose/.env,请先运行 scripts/config.sh"; exit 1; }
info "使用环境文件: $ENV_FILE"
set -a; source "$ENV_FILE"; set +a
# 兼容:若 .env 未包含 SWARM_MANAGER_ADDR,则从已存在的 cluster-info.env 读取以避免写空
SMADDR="${SWARM_MANAGER_ADDR:-}"
CI_FILE="$PKG_ROOT/cluster-info.env"
if [[ -z "$SMADDR" && -f "$CI_FILE" ]]; then
SMADDR=$(sed -n 's/^SWARM_MANAGER_ADDR=\(.*\)$/\1/p' "$CI_FILE" | head -n1)
fi
SWARM_MANAGER_ADDR="$SMADDR"
NET_NAME="${ARGUS_OVERLAY_NET:-argus-sys-net}"
SWARM_AVAILABLE=true
# Swarm init & overlay(ARM 版优先按生产方式启用 Swarm,但在无法启用时退回本地 bridge)
if ! docker info 2>/dev/null | grep -q "Swarm: active"; then
if [[ -n "${SWARM_MANAGER_ADDR:-}" ]]; then
info "Swarm 未激活,尝试初始化 Swarm (--advertise-addr $SWARM_MANAGER_ADDR)"
if ! docker swarm init --advertise-addr "$SWARM_MANAGER_ADDR" >/dev/null 2>&1; then
err "Swarm 初始化失败,本次部署将退回本地 bridge 网络: $NET_NAME(仅支持单机测试,不支持 swarm client)"
SWARM_AVAILABLE=false
else
info "Swarm 初始化成功"
fi
else
info "SWARM_MANAGER_ADDR 未设置且 Swarm 未激活,将使用本地 bridge 网络: $NET_NAME(仅支持单机测试)"
SWARM_AVAILABLE=false
fi
else
info "Swarm 已激活"
fi
if [[ "$SWARM_AVAILABLE" == true ]]; then
if ! docker network inspect "$NET_NAME" >/dev/null 2>&1; then
info "创建 overlay 网络: $NET_NAME"
docker network create -d overlay --attachable "$NET_NAME" >/dev/null
else
info "overlay 网络已存在: $NET_NAME"
fi
# 将 Swarm join token 写入 cluster-info.env,供 ARM client 使用
TOKEN_WORKER=$(docker swarm join-token -q worker 2>/dev/null || echo "")
TOKEN_MANAGER=$(docker swarm join-token -q manager 2>/dev/null || echo "")
CI_OUT="$PKG_ROOT/cluster-info.env"
info "写入 cluster-info.env (manager/token)"
{
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR:-}"
echo "SWARM_JOIN_TOKEN_WORKER=${TOKEN_WORKER:-}"
echo "SWARM_JOIN_TOKEN_MANAGER=${TOKEN_MANAGER:-}"
} > "$CI_OUT"
else
# 本机无法启用 Swarm,只维护 SWARM_MANAGER_ADDR,join token 留空
CI_OUT="$PKG_ROOT/cluster-info.env"
info "写入 cluster-info.env(Swarm 未启用,仅保存 SWARM_MANAGER_ADDR)"
{
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR:-}"
echo "SWARM_JOIN_TOKEN_WORKER="
echo "SWARM_JOIN_TOKEN_MANAGER="
} > "$CI_OUT"
if ! docker network inspect "$NET_NAME" >/dev/null 2>&1; then
info "创建本地 bridge 网络: $NET_NAME"
docker network create "$NET_NAME" >/dev/null
else
info "本地网络已存在: $NET_NAME"
fi
fi
# 导入镜像
IMAGES_DIR="$PKG_ROOT/images"
shopt -s nullglob
tars=("$IMAGES_DIR"/*.tar.gz)
if [[ ${#tars[@]} -eq 0 ]]; then err "images 目录为空,缺少镜像 tar.gz"; exit 1; fi
total=${#tars[@]}; idx=0
for tgz in "${tars[@]}"; do
idx=$((idx+1))
info "导入镜像 ($idx/$total): $(basename "$tgz")"
tmp=$(mktemp); gunzip -c "$tgz" > "$tmp"; docker load -i "$tmp" >/dev/null; rm -f "$tmp"
done
shopt -u nullglob
# 启动服务
PROJECT="${COMPOSE_PROJECT_NAME:-argus-server-arm}"
info "启动服务栈 (docker compose -p $PROJECT up -d)"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" up -d
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" ps
# 简单就绪检查(best-effort,不阻塞安装)
code(){ curl -4 -s -o /dev/null -w "%{http_code}" "$1" || echo 000; }
gf_ok(){ local body; body=$(curl -s "http://127.0.0.1:${GRAFANA_PORT:-3000}/api/health" || true); echo "$body" | grep -q '"database"\s*:\s*"ok"'; }
RETRIES=${RETRIES:-60}; SLEEP=${SLEEP:-5}; ok=0
info "等待基础服务就绪 (<= $((RETRIES*SLEEP))s)"
for i in $(seq 1 "$RETRIES"); do
e1=$(code "http://127.0.0.1:${MASTER_PORT:-32300}/readyz")
e2=$(code "http://127.0.0.1:${PROMETHEUS_PORT:-9090}/-/ready")
e3=$(code "http://127.0.0.1:${ALERTMANAGER_PORT:-9093}/api/v2/status")
e4=000; gf_ok && e4=200
info "[ready] t=$((i*SLEEP))s master=$e1 prom=$e2 graf=$e4 alert=$e3"
[[ "$e1" == 200 ]] && ok=$((ok+1))
[[ "$e2" == 200 ]] && ok=$((ok+1))
[[ "$e3" == 200 ]] && ok=$((ok+1))
[[ "$e4" == 200 ]] && ok=$((ok+1))
if [[ $ok -ge 4 ]]; then break; fi; ok=0; sleep "$SLEEP"
done
[[ $ok -ge 4 ]] || err "部分服务未就绪(可稍后执行 scripts/selfcheck.sh 进行复查)"
# 安装报告(ARM 精简版)
ts=$(date +%Y%m%d-%H%M%S)
RPT="$PKG_ROOT/安装报告_ARM_${ts}.md"
{
echo "# Argus ARM Server 安装报告 (${ts})"
echo
echo "## 端口映射"
echo "- MASTER_PORT=${MASTER_PORT}"
echo "- PROMETHEUS_PORT=${PROMETHEUS_PORT}"
echo "- GRAFANA_PORT=${GRAFANA_PORT}"
echo "- ALERTMANAGER_PORT=${ALERTMANAGER_PORT}"
echo "- WEB_PROXY_PORT_8080=${WEB_PROXY_PORT_8080} ... 8085=${WEB_PROXY_PORT_8085}"
echo
echo "## 网络"
echo "- NET=${NET_NAME}"
echo "- ARGUS_OVERLAY_NET=${ARGUS_OVERLAY_NET:-argus-sys-net}"
echo
echo "## 健康检查(简要)"
echo "- master/readyz=$(code http://127.0.0.1:${MASTER_PORT:-32300}/readyz)"
echo "- prometheus/ready=$(code http://127.0.0.1:${PROMETHEUS_PORT:-9090}/-/ready)"
echo "- grafana/api/health=$(code http://127.0.0.1:${GRAFANA_PORT:-3000}/api/health)"
echo "- alertmanager/api/v2/status=$(code http://127.0.0.1:${ALERTMANAGER_PORT:-9093}/api/v2/status)"
} > "$RPT"
info "已生成报告: $RPT"
# 刷新 web-proxy 上的 nginx 配置(best-effort)
docker exec argus-web-proxy nginx -t >/dev/null 2>&1 && docker exec argus-web-proxy nginx -s reload >/dev/null 2>&1 || true
info "安装完成。可通过 scripts/status.sh 查看当前服务状态。"

View File

@@ -0,0 +1,74 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
log() { echo -e "\033[0;34m[CHECK]\033[0m $*"; }
err() { echo -e "\033[0;31m[ERROR]\033[0m $*" >&2; }
ENV_FILE="$ROOT/compose/.env"; [[ -f "$ENV_FILE" ]] && set -a && source "$ENV_FILE" && set +a
wait_http() { local url="$1"; local attempts=${2:-120}; local i=1; while ((i<=attempts)); do curl -fsS "$url" >/dev/null 2>&1 && return 0; echo "[..] waiting $url ($i/$attempts)"; sleep 5; ((i++)); done; return 1; }
code_for() { curl -s -o /dev/null -w "%{http_code}" "$1" || echo 000; }
header_val() { curl -s -D - -o /dev/null "$@" | awk -F': ' 'BEGIN{IGNORECASE=1}$1=="Access-Control-Allow-Origin"{gsub("\r","",$2);print $2}'; }
LOG_DIR="$ROOT/logs"; mkdir -p "$LOG_DIR" || true
OUT_JSON="$LOG_DIR/selfcheck_arm_server.json"; tmp=$(mktemp)
ok=1
log "checking network (${ARGUS_OVERLAY_NET:-argus-sys-net})"
net_ok=false
if docker network inspect "${ARGUS_OVERLAY_NET:-argus-sys-net}" >/dev/null 2>&1; then
net_ok=true
fi
[[ "$net_ok" == true ]] || ok=0
log "checking Master"
[[ $(code_for "http://localhost:${MASTER_PORT:-32300}/readyz") == 200 ]] || ok=0
log "checking Prometheus"
wait_http "http://localhost:${PROMETHEUS_PORT:-9090}/-/ready" 60 || ok=0
log "checking Grafana"
gf_code=$(code_for "http://localhost:${GRAFANA_PORT:-3000}/api/health")
gf_ok=false
if [[ "$gf_code" == 200 ]]; then
body=$(curl -sS "http://localhost:${GRAFANA_PORT:-3000}/api/health" || true)
echo "$body" | grep -q '"database"\s*:\s*"ok"' && gf_ok=true
fi
[[ "$gf_ok" == true ]] || ok=0
log "checking Alertmanager"
wait_http "http://localhost:${ALERTMANAGER_PORT:-9093}/api/v2/status" 60 || ok=0
log "checking Web-Proxy (CORS)"
cors8084=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8084:-8084}/api/v2/status" || true)
cors8085=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8085:-8085}/api/v1/master/nodes" || true)
wp_ok=true
[[ -n "$cors8084" && -n "$cors8085" ]] || wp_ok=false
[[ "$wp_ok" == true ]] || ok=0
cat > "$tmp" <<JSON
{
"overlay_net": $net_ok,
"master_readyz": true,
"prometheus": true,
"grafana": $gf_ok,
"alertmanager": true,
"web_proxy_cors": $wp_ok,
"timestamp": "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
}
JSON
mv "$tmp" "$OUT_JSON" 2>/dev/null || cp "$tmp" "$OUT_JSON"
if [[ "$ok" == 1 ]]; then
log "selfcheck OK -> $OUT_JSON"
exit 0
else
err "selfcheck FAILED -> $OUT_JSON"
exit 1
fi

View File

@@ -0,0 +1,15 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
if [[ -f "$ENV_FILE" ]]; then
set -a; source "$ENV_FILE"; set +a
fi
PROJECT="${COMPOSE_PROJECT_NAME:-argus-server-arm}"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" ps

View File

@@ -0,0 +1,28 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
if [[ -f "$ENV_FILE" ]]; then
set -a; source "$ENV_FILE"; set +a
fi
PROJECT="${COMPOSE_PROJECT_NAME:-argus-server-arm}"
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require_compose
echo "[UNINSTALL] stopping compose (project=$PROJECT)"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" down --remove-orphans || true
echo "[UNINSTALL] done"

View File

@@ -0,0 +1,116 @@
#!/usr/bin/env bash
set -euo pipefail
# x86 专用:为 ARM Server 包生成 compose/.env
# - 带磁盘空间检查
# - 自动分配端口,避免与本机已有服务冲突
# - 写入 ARGUS_OVERLAY_NET 和 ARGUS_BUILD_UID/GID
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_EX="$PKG_ROOT/compose/.env.example"
ENV_OUT="$PKG_ROOT/compose/.env"
info(){ echo -e "\033[34m[CONFIG_X86]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR_X86]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
# Compose 检测:优先 docker compose(v2),回退 docker-compose(v1)
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker awk sed
require_compose
[[ -f "$ENV_EX" ]] || { err "缺少模板文件: $ENV_EX"; exit 1; }
# 磁盘空间检查(简化版,默认 >= 5GB)
check_disk(){ local p="$1"; local need="${2:-5120}"; local free;
free=$(df -Pm "$p" 2>/dev/null | awk 'NR==2{print $4+0}')
if [[ -z "$free" || "$free" -lt "$need" ]]; then
err "磁盘空间不足: $p 剩余 ${free:-0}MB (<${need}MB),请清理后再继续"
fi
}
check_disk "$PKG_ROOT" 5120 || true
check_disk "/var/lib/docker" 5120 || true
cp "$ENV_EX" "$ENV_OUT"
# 读取/生成 SWARM_MANAGER_ADDR(与 ARM server 配置保持一致)
SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR:-}
if [[ -z "${SWARM_MANAGER_ADDR}" ]]; then
read -rp "请输入本机管理地址 SWARM_MANAGER_ADDR: " SWARM_MANAGER_ADDR
fi
info "SWARM_MANAGER_ADDR=$SWARM_MANAGER_ADDR"
# 校验 IP 属于本机网卡
if ! ip -o addr | awk '{print $4}' | cut -d'/' -f1 | grep -qx "$SWARM_MANAGER_ADDR"; then
err "SWARM_MANAGER_ADDR 非本机地址: $SWARM_MANAGER_ADDR"; exit 1; fi
info "开始分配服务端口(起始=20000,避免系统占用与相互冲突)"
is_port_used(){ local p="$1"; ss -tulnH 2>/dev/null | awk '{print $5}' | sed 's/.*://g' | grep -qx "$p"; }
declare -A PRESENT=() CHOSEN=() USED=()
START_PORT="${START_PORT:-20000}"; cur=$START_PORT
ORDER=(MASTER_PORT PROMETHEUS_PORT GRAFANA_PORT ALERTMANAGER_PORT \
WEB_PROXY_PORT_8080 WEB_PROXY_PORT_8081 WEB_PROXY_PORT_8082 \
WEB_PROXY_PORT_8083 WEB_PROXY_PORT_8084 WEB_PROXY_PORT_8085)
# 标记 .env.example 中实际存在的键
for key in "${ORDER[@]}"; do
if grep -q "^${key}=" "$ENV_EX"; then PRESENT[$key]=1; fi
done
next_free(){ local p="$1"; while :; do if [[ -n "${USED[$p]:-}" ]] || is_port_used "$p"; then p=$((p+1)); else echo "$p"; return; fi; done; }
for key in "${ORDER[@]}"; do
[[ -z "${PRESENT[$key]:-}" ]] && continue
p=$(next_free "$cur"); CHOSEN[$key]="$p"; USED[$p]=1; cur=$((p+1))
done
info "端口分配结果:MASTER=${CHOSEN[MASTER_PORT]:-} PROM=${CHOSEN[PROMETHEUS_PORT]:-} GRAFANA=${CHOSEN[GRAFANA_PORT]:-} ALERT=${CHOSEN[ALERTMANAGER_PORT]:-} WEB_PROXY(8080..8085)=${CHOSEN[WEB_PROXY_PORT_8080]:-}/${CHOSEN[WEB_PROXY_PORT_8081]:-}/${CHOSEN[WEB_PROXY_PORT_8082]:-}/${CHOSEN[WEB_PROXY_PORT_8083]:-}/${CHOSEN[WEB_PROXY_PORT_8084]:-}/${CHOSEN[WEB_PROXY_PORT_8085]:-}"
# 覆盖端口(按唯一化结果写回)
for key in "${ORDER[@]}"; do
val="${CHOSEN[$key]:-}"
[[ -z "$val" ]] && continue
sed -i -E "s#^$key=.*#$key=${val}#" "$ENV_OUT"
done
# 覆盖/补充 Overlay 名称
grep -q '^ARGUS_OVERLAY_NET=' "$ENV_OUT" || echo 'ARGUS_OVERLAY_NET=argus-sys-net' >> "$ENV_OUT"
# 覆盖/补充构建账户 UID/GID,避免权限问题
RUID=$(id -u)
RGID=$(id -g)
if grep -q '^ARGUS_BUILD_UID=' "$ENV_OUT"; then
sed -i -E "s#^ARGUS_BUILD_UID=.*#ARGUS_BUILD_UID=${RUID}#" "$ENV_OUT"
else
echo "ARGUS_BUILD_UID=${RUID}" >> "$ENV_OUT"
fi
if grep -q '^ARGUS_BUILD_GID=' "$ENV_OUT"; then
sed -i -E "s#^ARGUS_BUILD_GID=.*#ARGUS_BUILD_GID=${RGID}#" "$ENV_OUT"
else
echo "ARGUS_BUILD_GID=${RGID}" >> "$ENV_OUT"
fi
# 更新 cluster-info.env 的 SWARM_MANAGER_ADDR(ARM + x86 一致)
CI="$PKG_ROOT/cluster-info.env"
if [[ -f "$CI" ]]; then
if grep -q '^SWARM_MANAGER_ADDR=' "$CI"; then
sed -i -E "s#^SWARM_MANAGER_ADDR=.*#SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}#" "$CI"
else
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}" >> "$CI"
fi
else
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}" > "$CI"
fi
info "已生成 compose/.env 并更新 cluster-info.env 的 SWARM_MANAGER_ADDR。"
info "下一步可执行: scripts_for_x86/install.sh"

View File

@@ -0,0 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
exec "$SCRIPT_DIR/../scripts/diagnose.sh" "$@"

View File

@@ -0,0 +1,77 @@
#!/usr/bin/env bash
set -euo pipefail
# 在 x86_64 主机上安装 ARM server 包:
# 1) 确认/安装 binfmt + QEMU(通过 tonistiigi/binfmt);
# 2) 调用通用的 scripts/install.sh 启动服务。
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
info(){ echo -e "\033[34m[INSTALL_X86]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR_X86]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
ensure_binfmt_arm64() {
info "检查当前主机架构: $(uname -m)"
if [[ "$(uname -m)" != "x86_64" && "$(uname -m)" != "amd64" ]]; then
info "当前并非 x86_64($(uname -m)),通常应直接在 ARM 服务器上使用 scripts/install.sh。继续执行但不强制安装 binfmt。"
return 0
fi
require docker
# 优先使用随安装包一起提供的 ubuntu-22.04-arm-test 镜像做自检
local TEST_IMG="ubuntu:22.04"
local TEST_TAR=""
if [[ -d "$ROOT_DIR/images" ]]; then
TEST_TAR=$(ls "$ROOT_DIR"/images/ubuntu-22.04-arm-test*.tar.gz 2>/dev/null | head -n1 || true)
fi
if [[ -n "$TEST_TAR" && ! $(docker image inspect "$TEST_IMG" >/dev/null 2>&1; echo $?) -eq 0 ]]; then
info "从安装包加载 ARM 测试基础镜像: $(basename "$TEST_TAR")"
tmp=$(mktemp)
gunzip -c "$TEST_TAR" > "$tmp"
docker load -i "$tmp" >/dev/null
rm -f "$tmp"
fi
info "验证是否已能运行 linux/arm64 容器..."
if docker run --rm --platform=linux/arm64 "$TEST_IMG" uname -m 2>/dev/null | grep -q 'aarch64'; then
info "检测到本机已支持 ARM64 容器运行 (uname -m = aarch64)"
return 0
fi
info "未检测到 ARM64 运行能力,尝试通过 tonistiigi/binfmt 安装 binfmt/QEMU ..."
# 优先从离线包加载 tonistiigi/binfmt 镜像
local BINFMT_IMG="tonistiigi/binfmt:latest"
local BINFMT_TAR=""
if [[ -d "$ROOT_DIR/images" ]]; then
BINFMT_TAR=$(ls "$ROOT_DIR"/images/tonistiigi-binfmt*.tar.gz 2>/dev/null | head -n1 || true)
fi
if [[ -n "$BINFMT_TAR" && ! $(docker image inspect "$BINFMT_IMG" >/dev/null 2>&1; echo $?) -eq 0 ]]; then
info "从安装包加载 tonistiigi/binfmt 离线镜像: $(basename "$BINFMT_TAR")"
tmp_b=$(mktemp)
gunzip -c "$BINFMT_TAR" > "$tmp_b"
docker load -i "$tmp_b" >/dev/null
rm -f "$tmp_b"
fi
if ! docker run --privileged --rm "$BINFMT_IMG" --install all; then
err "tonistiigi/binfmt 安装失败,请检查 Docker 权限或确认离线镜像是否完整。"
exit 1
fi
info "再次验证 ARM64 容器可运行状态..."
if ! docker run --rm --platform=linux/arm64 "$TEST_IMG" uname -m 2>/dev/null | grep -q 'aarch64'; then
err "安装 binfmt/QEMU 后仍无法运行 linux/arm64 容器,请手工排查 binfmt_misc 配置。"
exit 1
fi
info "ARM64 容器运行能力检查通过。"
}
ensure_binfmt_arm64
# 准备完 x86 -> ARM 运行环境后,交由通用 install.sh 负责后续部署
exec "$SCRIPT_DIR/../scripts/install.sh" "$@"

View File

@@ -0,0 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
exec "$SCRIPT_DIR/../scripts/selfcheck.sh" "$@"

View File

@@ -0,0 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
exec "$SCRIPT_DIR/../scripts/status.sh" "$@"

View File

@@ -0,0 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
exec "$SCRIPT_DIR/../scripts/uninstall.sh" "$@"

View File

@@ -37,22 +37,11 @@ _argus_is_number() {
[[ "$1" =~ ^[0-9]+$ ]] [[ "$1" =~ ^[0-9]+$ ]]
} }
load_build_user() { _argus_read_user_from_files() {
if [[ "$_ARGUS_BUILD_USER_LOADED" == "1" ]]; then local uid_out_var="$1" gid_out_var="$2"; shift 2
return 0 local uid_val="$ARGUS_BUILD_UID_DEFAULT" gid_val="$ARGUS_BUILD_GID_DEFAULT"
fi local config
for config in "$@"; do
local project_root config_files config uid gid
project_root="$(argus_project_root)"
config_files=(
"$project_root/configs/build_user.local.conf"
"$project_root/configs/build_user.conf"
)
uid="$ARGUS_BUILD_UID_DEFAULT"
gid="$ARGUS_BUILD_GID_DEFAULT"
for config in "${config_files[@]}"; do
if [[ -f "$config" ]]; then if [[ -f "$config" ]]; then
while IFS= read -r raw_line || [[ -n "$raw_line" ]]; do while IFS= read -r raw_line || [[ -n "$raw_line" ]]; do
local line key value local line key value
@@ -68,42 +57,58 @@ load_build_user() {
key="$(_argus_trim "$key")" key="$(_argus_trim "$key")"
value="$(_argus_trim "$value")" value="$(_argus_trim "$value")"
case "$key" in case "$key" in
UID) UID) uid_val="$value" ;;
uid="$value" GID) gid_val="$value" ;;
;; *) echo "[ARGUS build_user] Unknown key '$key' in $config" >&2 ;;
GID)
gid="$value"
;;
*)
echo "[ARGUS build_user] Unknown key '$key' in $config" >&2
;;
esac esac
done < "$config" done < "$config"
break break
fi fi
done done
printf -v "$uid_out_var" '%s' "$uid_val"
printf -v "$gid_out_var" '%s' "$gid_val"
}
if [[ -n "${ARGUS_BUILD_UID:-}" ]]; then load_build_user_profile() {
uid="$ARGUS_BUILD_UID" local profile="${1:-default}"
fi if [[ "$_ARGUS_BUILD_USER_LOADED" == "1" ]]; then
if [[ -n "${ARGUS_BUILD_GID:-}" ]]; then return 0
gid="$ARGUS_BUILD_GID"
fi fi
local project_root uid gid
project_root="$(argus_project_root)"
case "$profile" in
pkg)
_argus_read_user_from_files uid gid \
"$project_root/configs/build_user.pkg.conf" \
"$project_root/configs/build_user.local.conf" \
"$project_root/configs/build_user.conf"
;;
default|*)
_argus_read_user_from_files uid gid \
"$project_root/configs/build_user.local.conf" \
"$project_root/configs/build_user.conf"
;;
esac
if [[ -n "${ARGUS_BUILD_UID:-}" ]]; then uid="$ARGUS_BUILD_UID"; fi
if [[ -n "${ARGUS_BUILD_GID:-}" ]]; then gid="$ARGUS_BUILD_GID"; fi
if ! _argus_is_number "$uid"; then if ! _argus_is_number "$uid"; then
echo "[ARGUS build_user] Invalid UID '$uid'" >&2 echo "[ARGUS build_user] Invalid UID '$uid'" >&2; return 1
return 1
fi fi
if ! _argus_is_number "$gid"; then if ! _argus_is_number "$gid"; then
echo "[ARGUS build_user] Invalid GID '$gid'" >&2 echo "[ARGUS build_user] Invalid GID '$gid'" >&2; return 1
return 1
fi fi
export ARGUS_BUILD_UID="$uid" export ARGUS_BUILD_UID="$uid"
export ARGUS_BUILD_GID="$gid" export ARGUS_BUILD_GID="$gid"
_ARGUS_BUILD_USER_LOADED=1 _ARGUS_BUILD_USER_LOADED=1
} }
load_build_user() {
local profile="${ARGUS_BUILD_PROFILE:-default}"
load_build_user_profile "$profile"
}
argus_build_user_args() { argus_build_user_args() {
load_build_user load_build_user
printf '%s' "--build-arg ARGUS_BUILD_UID=${ARGUS_BUILD_UID} --build-arg ARGUS_BUILD_GID=${ARGUS_BUILD_GID}" printf '%s' "--build-arg ARGUS_BUILD_UID=${ARGUS_BUILD_UID} --build-arg ARGUS_BUILD_GID=${ARGUS_BUILD_GID}"

View File

@@ -3,3 +3,4 @@ build/
__pycache__/ __pycache__/
.env .env
dist/

View File

@@ -4,6 +4,7 @@ import os
import re import re
import socket import socket
import subprocess import subprocess
import ipaddress
from pathlib import Path from pathlib import Path
from typing import Any, Dict from typing import Any, Dict
@@ -16,11 +17,47 @@ _HOSTNAME_PATTERN = re.compile(r"^([^-]+)-([^-]+)-([^-]+)-.*$")
def collect_metadata(config: AgentConfig) -> Dict[str, Any]: def collect_metadata(config: AgentConfig) -> Dict[str, Any]:
"""汇总节点注册需要的静态信息。""" """汇总节点注册需要的静态信息,带有更智能的 IP 选择。
规则(从高到低):
1) AGENT_PUBLISH_IP 指定;
2) Hostname A 记录(若命中优先网段);
3) 网卡扫描:排除 AGENT_EXCLUDE_IFACES,优先 AGENT_PREFER_NET_CIDRS;
4) 默认路由回退(UDP socket 技巧)。
额外发布:overlay_ip / gwbridge_ip / interfaces,便于 Master 与诊断使用。
"""
hostname = config.hostname hostname = config.hostname
meta = {
prefer_cidrs = _read_cidrs_env(
os.environ.get("AGENT_PREFER_NET_CIDRS", "10.0.0.0/8,172.31.0.0/16")
)
exclude_ifaces = _read_csv_env(
os.environ.get("AGENT_EXCLUDE_IFACES", "docker_gwbridge,lo")
)
# interface inventory
interfaces = _list_global_ipv4_addrs()
if exclude_ifaces:
interfaces = [it for it in interfaces if it[0] not in set(exclude_ifaces)]
# resolve hostname candidates
host_ips = _resolve_hostname_ips(hostname)
selected_ip, overlay_ip, gwbridge_ip = _select_publish_ips(
interfaces=interfaces,
host_ips=host_ips,
prefer_cidrs=prefer_cidrs,
)
meta: Dict[str, Any] = {
"hostname": hostname, "hostname": hostname,
"ip": _detect_ip_address(), "ip": os.environ.get("AGENT_PUBLISH_IP", selected_ip), # keep required field
"overlay_ip": overlay_ip or selected_ip,
"gwbridge_ip": gwbridge_ip,
"interfaces": [
{"iface": name, "ip": ip} for name, ip in interfaces
],
"env": config.environment, "env": config.environment,
"user": config.user, "user": config.user,
"instance": config.instance, "instance": config.instance,
@@ -96,7 +133,7 @@ def _detect_gpu_count() -> int:
def _detect_ip_address() -> str: def _detect_ip_address() -> str:
"""尝试通过 UDP socket 获得容器出口 IP,失败则回退解析主机名。""" """保留旧接口,作为最终回退:默认路由源地址 → 主机名解析 → 127.0.0.1。"""
try: try:
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock: with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock:
sock.connect(("8.8.8.8", 80)) sock.connect(("8.8.8.8", 80))
@@ -108,3 +145,118 @@ def _detect_ip_address() -> str:
except OSError: except OSError:
LOGGER.warning("Unable to resolve hostname to IP; defaulting to 127.0.0.1") LOGGER.warning("Unable to resolve hostname to IP; defaulting to 127.0.0.1")
return "127.0.0.1" return "127.0.0.1"
def _read_csv_env(raw: str | None) -> list[str]:
if not raw:
return []
return [x.strip() for x in raw.split(",") if x.strip()]
def _read_cidrs_env(raw: str | None) -> list[ipaddress.IPv4Network]:
cidrs: list[ipaddress.IPv4Network] = []
for item in _read_csv_env(raw):
try:
net = ipaddress.ip_network(item, strict=False)
if isinstance(net, (ipaddress.IPv4Network,)):
cidrs.append(net)
except ValueError:
LOGGER.warning("Ignoring invalid CIDR in AGENT_PREFER_NET_CIDRS", extra={"cidr": item})
return cidrs
def _list_global_ipv4_addrs() -> list[tuple[str, str]]:
"""列出 (iface, ip) 形式的全局 IPv4 地址。
依赖 iproute2:ip -4 -o addr show scope global
"""
results: list[tuple[str, str]] = []
try:
proc = subprocess.run(
["sh", "-lc", "ip -4 -o addr show scope global | awk '{print $2, $4}'"],
check=False,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
timeout=3,
)
if proc.returncode == 0:
for line in proc.stdout.splitlines():
line = line.strip()
if not line:
continue
parts = line.split()
if len(parts) != 2:
continue
iface, cidr = parts
ip = cidr.split("/")[0]
try:
ipaddress.IPv4Address(ip)
except ValueError:
continue
results.append((iface, ip))
except Exception as exc: # pragma: no cover - defensive
LOGGER.debug("Failed to list interfaces", extra={"error": str(exc)})
return results
def _resolve_hostname_ips(name: str) -> list[str]:
ips: list[str] = []
try:
infos = socket.getaddrinfo(name, None, family=socket.AF_INET)
for info in infos:
ip = info[4][0]
if ip not in ips:
ips.append(ip)
except OSError:
pass
return ips
def _pick_by_cidrs(candidates: list[str], prefer_cidrs: list[ipaddress.IPv4Network]) -> str | None:
for net in prefer_cidrs:
for ip in candidates:
try:
if ipaddress.ip_address(ip) in net:
return ip
except ValueError:
continue
return None
def _select_publish_ips(
*,
interfaces: list[tuple[str, str]],
host_ips: list[str],
prefer_cidrs: list[ipaddress.IPv4Network],
) -> tuple[str, str | None, str | None]:
"""返回 (selected_ip, overlay_ip, gwbridge_ip)。
- overlay_ip:优先命中 prefer_cidrs(10.0/8 先于 172.31/16)。
- gwbridge_ip:若存在 172.22/16 则记录。
- selected_ip:优先 AGENT_PUBLISH_IP;否则 overlay_ip;否则 hostname A 记录中的 prefer;否则默认路由回退。
"""
# detect gwbridge (172.22/16)
gwbridge_net = ipaddress.ip_network("172.22.0.0/16")
gwbridge_ip = None
for _, ip in interfaces:
try:
if ipaddress.ip_address(ip) in gwbridge_net:
gwbridge_ip = ip
break
except ValueError:
continue
# overlay candidate from interfaces by prefer cidrs
iface_ips = [ip for _, ip in interfaces]
overlay_ip = _pick_by_cidrs(iface_ips, prefer_cidrs)
# hostname A records filtered by prefer cidrs
host_pref = _pick_by_cidrs(host_ips, prefer_cidrs)
env_ip = os.environ.get("AGENT_PUBLISH_IP")
if env_ip:
selected = env_ip
else:
selected = overlay_ip or host_pref or _detect_ip_address()
return selected, overlay_ip, gwbridge_ip

Binary file not shown.

View File

@@ -11,12 +11,13 @@ RUN apt-get update && \
# 设置 Alertmanager 版本(与本地离线包保持一致) # 设置 Alertmanager 版本(与本地离线包保持一致)
ARG ALERTMANAGER_VERSION=0.28.1 ARG ALERTMANAGER_VERSION=0.28.1
ARG ALERTMANAGER_ARCH=amd64
# 使用仓库内预置的离线包构建(无需联网) # 使用仓库内预置的离线包构建(无需联网)
COPY src/alert/alertmanager/build/alertmanager-${ALERTMANAGER_VERSION}.linux-amd64.tar.gz /tmp/ COPY src/alert/alertmanager/build/alertmanager-${ALERTMANAGER_VERSION}.linux-${ALERTMANAGER_ARCH}.tar.gz /tmp/
RUN tar xvf /tmp/alertmanager-${ALERTMANAGER_VERSION}.linux-amd64.tar.gz -C /tmp && \ RUN tar xvf /tmp/alertmanager-${ALERTMANAGER_VERSION}.linux-${ALERTMANAGER_ARCH}.tar.gz -C /tmp && \
mv /tmp/alertmanager-${ALERTMANAGER_VERSION}.linux-amd64 /usr/local/alertmanager && \ mv /tmp/alertmanager-${ALERTMANAGER_VERSION}.linux-${ALERTMANAGER_ARCH} /usr/local/alertmanager && \
rm -f /tmp/alertmanager-${ALERTMANAGER_VERSION}.linux-amd64.tar.gz rm -f /tmp/alertmanager-${ALERTMANAGER_VERSION}.linux-${ALERTMANAGER_ARCH}.tar.gz
ENV ALERTMANAGER_BASE_PATH=/private/argus/alert/alertmanager ENV ALERTMANAGER_BASE_PATH=/private/argus/alert/alertmanager
@@ -31,26 +32,31 @@ RUN mkdir -p /usr/share/alertmanager && \
rm -rf /alertmanager && \ rm -rf /alertmanager && \
ln -s ${ALERTMANAGER_BASE_PATH} /alertmanager ln -s ${ALERTMANAGER_BASE_PATH} /alertmanager
# 创建 alertmanager 用户(可自定义 UID/GID) # 确保 ubuntu 账户存在并使用 ARGUS_BUILD_UID/GID
# 创建 alertmanager 用户组
RUN set -eux; \ RUN set -eux; \
# 确保目标 GID 存在;若已被占用,直接使用该 GID(组名不限)\ # 确保存在目标 GID 的组;若不存在则优先尝试将 ubuntu 组改为该 GID,否则创建新组
if ! getent group "${ARGUS_BUILD_GID}" >/dev/null; then \ if getent group "${ARGUS_BUILD_GID}" >/dev/null; then \
groupadd -g "${ARGUS_BUILD_GID}" alertmanager || true; \ :; \
fi; \
# 确保存在 alertmanager 用户;若 UID 已被占用,跳过并继续使用现有 UID 的用户
if ! id alertmanager >/dev/null 2>&1; then \
if getent passwd "${ARGUS_BUILD_UID}" >/dev/null; then \
# UID 已占用,则创建同名用户但不指定 UID(避免冲突),仅保证 user 存在
useradd -M -s /usr/sbin/nologin -g "${ARGUS_BUILD_GID}" alertmanager || true; \
else \ else \
useradd -M -s /usr/sbin/nologin -u "${ARGUS_BUILD_UID}" -g "${ARGUS_BUILD_GID}" alertmanager || true; \ if getent group ubuntu >/dev/null; then \
groupmod -g "${ARGUS_BUILD_GID}" ubuntu || true; \
else \
groupadd -g "${ARGUS_BUILD_GID}" ubuntu || groupadd -g "${ARGUS_BUILD_GID}" argus || true; \
fi; \
fi; \
# 创建或调整 ubuntu 用户
if id ubuntu >/dev/null 2>&1; then \
# 设置主组为目标 GID(可用 GID 数字指定)
usermod -g "${ARGUS_BUILD_GID}" ubuntu || true; \
# 若目标 UID 未被占用,则更新 ubuntu 的 UID
if [ "$(id -u ubuntu)" != "${ARGUS_BUILD_UID}" ] && ! getent passwd "${ARGUS_BUILD_UID}" >/dev/null; then \
usermod -u "${ARGUS_BUILD_UID}" ubuntu || true; \
fi; \ fi; \
else \ else \
usermod -g "${ARGUS_BUILD_GID}" alertmanager || true; \ useradd -m -s /bin/bash -u "${ARGUS_BUILD_UID}" -g "${ARGUS_BUILD_GID}" ubuntu || true; \
fi fi; \
# 调整关键目录属主为 ubuntu UID/GID
RUN chown -R "${ARGUS_BUILD_UID}:${ARGUS_BUILD_GID}" /usr/share/alertmanager /alertmanager ${ALERTMANAGER_BASE_PATH} /private/argus/etc /usr/local/bin || true chown -R "${ARGUS_BUILD_UID}:${ARGUS_BUILD_GID}" /usr/share/alertmanager /alertmanager ${ALERTMANAGER_BASE_PATH} /private/argus/etc /usr/local/bin || true
# 配置内网 apt 源 (如果指定了内网选项) # 配置内网 apt 源 (如果指定了内网选项)
RUN if [ "$USE_INTRANET" = "true" ]; then \ RUN if [ "$USE_INTRANET" = "true" ]; then \

5
src/alert/alertmanager/build/fetch-dist.sh Normal file → Executable file
View File

@@ -6,9 +6,11 @@ set -euo pipefail
# ./fetch-dist.sh [version] # ./fetch-dist.sh [version]
# 示例: # 示例:
# ./fetch-dist.sh 0.28.1 # ./fetch-dist.sh 0.28.1
# ARCH=arm64 ./fetch-dist.sh 0.28.1
VER="${1:-0.28.1}" VER="${1:-0.28.1}"
OUT="alertmanager-${VER}.linux-amd64.tar.gz" ARCH="${ARCH:-amd64}" # amd64 或 arm64
OUT="alertmanager-${VER}.linux-${ARCH}.tar.gz"
URL="https://github.com/prometheus/alertmanager/releases/download/v${VER}/${OUT}" URL="https://github.com/prometheus/alertmanager/releases/download/v${VER}/${OUT}"
if [[ -f "$OUT" ]]; then if [[ -f "$OUT" ]]; then
@@ -19,4 +21,3 @@ fi
echo "[INFO] Downloading $URL" echo "[INFO] Downloading $URL"
curl -fL --retry 3 --connect-timeout 10 -o "$OUT" "$URL" curl -fL --retry 3 --connect-timeout 10 -o "$OUT" "$URL"
echo "[OK] Saved to $(pwd)/$OUT" echo "[OK] Saved to $(pwd)/$OUT"

1
src/bundle/cpu-node-bundle/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
.build*/

View File

@@ -0,0 +1,33 @@
FROM ubuntu:22.04
ARG ARGUS_BUILD_UID=2133
ARG ARGUS_BUILD_GID=2015
ENV DEBIAN_FRONTEND=noninteractive \
TZ=Asia/Shanghai \
ARGUS_LOGS_WORLD_WRITABLE=1
RUN set -eux; \
apt-get update; \
apt-get install -y --no-install-recommends \
ca-certificates curl wget iproute2 iputils-ping net-tools jq tzdata \
cron procps supervisor vim less tar gzip python3; \
rm -rf /var/lib/apt/lists/*; \
ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
WORKDIR /
# Offline fluent-bit assets and bundle tarball are staged by the build script
COPY node-bootstrap.sh /usr/local/bin/node-bootstrap.sh
COPY health-watcher.sh /usr/local/bin/health-watcher.sh
COPY private/start-fluent-bit.sh /private/start-fluent-bit.sh
COPY private/etc /private/etc
COPY private/packages /private/packages
COPY bundle/ /bundle/
RUN chmod +x /usr/local/bin/node-bootstrap.sh /usr/local/bin/health-watcher.sh /private/start-fluent-bit.sh || true; \
mkdir -p /logs/train /logs/infer /buffers /opt/argus-metric; \
if [ "${ARGUS_LOGS_WORLD_WRITABLE}" = "1" ]; then chmod 1777 /logs/train /logs/infer || true; else chmod 755 /logs/train /logs/infer || true; fi; \
chmod 770 /buffers || true
ENTRYPOINT ["/usr/local/bin/node-bootstrap.sh"]

View File

@@ -0,0 +1,59 @@
#!/usr/bin/env bash
set -euo pipefail
# health-watcher.sh (CPU node bundle)
# 周期执行 check_health.sh 与 restart_unhealthy.sh,用于节点容器内自愈。
INSTALL_ROOT="/opt/argus-metric"
INTERVAL="${HEALTH_WATCH_INTERVAL:-60}"
VER_DIR="${1:-}"
log(){ echo "[HEALTH-WATCHER] $*"; }
resolve_ver_dir() {
local dir=""
if [[ -n "${VER_DIR:-}" && -d "$VER_DIR" ]]; then
dir="$VER_DIR"
elif [[ -L "$INSTALL_ROOT/current" ]]; then
dir="$(readlink -f "$INSTALL_ROOT/current" 2>/dev/null || true)"
fi
if [[ -z "$dir" ]]; then
dir="$(ls -d "$INSTALL_ROOT"/versions/* 2>/dev/null | sort -V | tail -n1 || true)"
fi
echo "$dir"
}
main() {
log "starting with interval=${INTERVAL}s"
local dir
dir="$(resolve_ver_dir)"
if [[ -z "$dir" || ! -d "$dir" ]]; then
log "no valid install dir found under $INSTALL_ROOT; exiting"
exit 0
fi
local chk="$dir/check_health.sh"
local rst="$dir/restart_unhealthy.sh"
if [[ ! -x "$chk" && ! -x "$rst" ]]; then
log "neither check_health.sh nor restart_unhealthy.sh is executable under $dir; exiting"
exit 0
fi
log "watching install dir: $dir"
while :; do
if [[ -x "$chk" ]]; then
log "running check_health.sh"
"$chk" >> "$dir/.health_check.watch.log" 2>&1 || log "check_health.sh reported issues (see .health_check.watch.log)"
fi
if [[ -x "$rst" ]]; then
log "running restart_unhealthy.sh"
"$rst" >> "$dir/.restart.watch.log" 2>&1 || log "restart_unhealthy.sh reported issues (see .restart.watch.log)"
fi
sleep "$INTERVAL"
done
}
main "$@"

View File

@@ -0,0 +1,131 @@
#!/usr/bin/env bash
set -euo pipefail
echo "[BOOT] CPU node bundle starting"
INSTALL_ROOT="/opt/argus-metric"
BUNDLE_DIR="/bundle"
STATE_DIR_BASE="/private/argus/agent"
mkdir -p "$INSTALL_ROOT" "$STATE_DIR_BASE" /logs/train /logs/infer /buffers || true
# Ensure world-writable logs dir with sticky bit (align with deployment_new policy)
if [[ "${ARGUS_LOGS_WORLD_WRITABLE:-1}" == "1" ]]; then
chmod 1777 /logs/train /logs/infer || true
else
chmod 755 /logs/train /logs/infer || true
fi
chmod 770 /buffers || true
installed_ok=0
# 1) already installed?
if [[ -L "$INSTALL_ROOT/current" && -d "$INSTALL_ROOT/current" ]]; then
echo "[BOOT] client already installed at $INSTALL_ROOT/current"
else
# 2) try local bundle first (argus-metric_*.tar.gz)
tarball=$(ls -1 "$BUNDLE_DIR"/argus-metric_*.tar.gz 2>/dev/null | head -1 || true)
if [[ -n "${tarball:-}" ]]; then
echo "[BOOT] installing from local bundle: $(basename "$tarball")"
tmp=$(mktemp -d)
tar -xzf "$tarball" -C "$tmp"
# locate root containing version.json
root="$tmp"
if [[ ! -f "$root/version.json" ]]; then
sub=$(find "$tmp" -mindepth 1 -maxdepth 1 -type d | head -n1 || true)
[[ -n "$sub" && -f "$sub/version.json" ]] && root="$sub"
fi
if [[ ! -f "$root/version.json" ]]; then
echo "[BOOT][WARN] version.json not found in bundle; fallback to FTP"
else
ver=$(sed -n 's/.*"version"\s*:\s*"\([^"]\+\)".*/\1/p' "$root/version.json" | head -n1)
if [[ -z "$ver" ]]; then
echo "[BOOT][WARN] failed to parse version from version.json; fallback to FTP"
else
target_root="$INSTALL_ROOT"
version_dir="$target_root/versions/$ver"
mkdir -p "$version_dir"
shopt -s dotglob
mv "$root"/* "$version_dir/" 2>/dev/null || true
shopt -u dotglob
if [[ -f "$version_dir/install.sh" ]]; then
chmod +x "$version_dir/install.sh" 2>/dev/null || true
(
export AUTO_START_DCGM="0" # N/A on CPU
cd "$version_dir" && ./install.sh "$version_dir"
)
echo "$ver" > "$target_root/LATEST_VERSION" 2>/dev/null || true
ln -sfn "$version_dir" "$target_root/current" 2>/dev/null || true
if [[ -L "$target_root/current" && -d "$target_root/current" ]]; then
installed_ok=1
echo "[BOOT] local bundle install OK: version=$ver"
else
echo "[BOOT][WARN] current symlink not present after install; will rely on healthcheck to confirm"
fi
else
echo "[BOOT][WARN] install.sh missing under $version_dir; fallback to FTP"
fi
fi
fi
fi
# 3) fallback: use FTP setup if not installed
if [[ ! -L "$INSTALL_ROOT/current" && "$installed_ok" -eq 0 ]]; then
echo "[BOOT] fallback to FTP setup"
if [[ -z "${FTPIP:-}" || -z "${FTP_USER:-}" || -z "${FTP_PASSWORD:-}" ]]; then
echo "[BOOT][ERROR] FTP variables not set (FTPIP/FTP_USER/FTP_PASSWORD)" >&2
exit 1
fi
curl -u "$FTP_USER:$FTP_PASSWORD" -fsSL "ftp://$FTPIP:21/setup.sh" -o /tmp/setup.sh
chmod +x /tmp/setup.sh
/tmp/setup.sh --server "$FTPIP" --user "$FTP_USER" --password "$FTP_PASSWORD" --port 21
fi
fi
# 4) ensure argus-agent is running (best-effort)
if ! pgrep -x argus-agent >/dev/null 2>&1; then
echo "[BOOT] starting argus-agent (not detected)"
setsid /usr/local/bin/argus-agent >/var/log/argus-agent.log 2>&1 < /dev/null &
fi
# 5) post-install selfcheck and state
ver_dir=""
if [[ -L "$INSTALL_ROOT/current" ]]; then
ver_dir="$(readlink -f "$INSTALL_ROOT/current" 2>/dev/null || true)"
fi
if [[ -z "$ver_dir" ]]; then
ver_dir="$(ls -d "$INSTALL_ROOT"/versions/* 2>/dev/null | sort -V | tail -n1 || true)"
fi
if [[ -n "$ver_dir" && -x "$ver_dir/check_health.sh" ]]; then
echo "[BOOT] running initial health check: $ver_dir/check_health.sh"
if "$ver_dir/check_health.sh" >> "$ver_dir/.health_check.init.log" 2>&1; then
echo "[BOOT] initial health check completed (see $ver_dir/.health_check.init.log)"
else
echo "[BOOT][WARN] initial health check reported issues (see $ver_dir/.health_check.init.log)"
fi
else
echo "[BOOT][WARN] initial health check skipped (script missing: $ver_dir/check_health.sh)"
fi
host="$(hostname)"
state_dir="$STATE_DIR_BASE/${host}"
mkdir -p "$state_dir" 2>/dev/null || true
for i in {1..60}; do
if [[ -s "$state_dir/node.json" ]]; then
echo "[BOOT] node state present: $state_dir/node.json"
break
fi
sleep 2
done
# 6) spawn health watcher (best-effort, non-blocking)
if command -v /usr/local/bin/health-watcher.sh >/dev/null 2>&1; then
echo "[BOOT] starting health watcher for $ver_dir"
setsid /usr/local/bin/health-watcher.sh "${ver_dir:-}" >/var/log/health-watcher.log 2>&1 < /dev/null || true &
else
echo "[BOOT][WARN] health-watcher.sh not found; skip health watcher"
fi
echo "[BOOT] ready; entering sleep"
exec sleep infinity

1
src/bundle/gpu-node-bundle/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
.build*/

View File

@@ -0,0 +1,44 @@
ARG CUDA_VER=12.2.2
FROM nvidia/cuda:${CUDA_VER}-runtime-ubuntu22.04
ARG CLIENT_VER=0.0.0
ARG BUNDLE_DATE=00000000
LABEL org.opencontainers.image.title="argus-sys-metric-test-node-bundle-gpu" \
org.opencontainers.image.description="GPU node bundle with embedded Argus client artifact" \
org.opencontainers.image.version="${CLIENT_VER}" \
org.opencontainers.image.revision_date="${BUNDLE_DATE}" \
maintainer="Argus"
ENV DEBIAN_FRONTEND=noninteractive \
TZ=Asia/Shanghai \
ARGUS_LOGS_WORLD_WRITABLE=1 \
ES_HOST=es.log.argus.com \
ES_PORT=9200 \
CLUSTER=local \
RACK=dev
RUN set -eux; \
apt-get update; \
apt-get install -y --no-install-recommends \
ca-certificates curl wget iproute2 iputils-ping net-tools jq tzdata cron procps vim less \
tar gzip; \
rm -rf /var/lib/apt/lists/*; \
ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
WORKDIR /
# Expect staged build context to provide these directories/files
COPY bundle/ /bundle/
COPY node-bootstrap.sh /usr/local/bin/node-bootstrap.sh
COPY health-watcher.sh /usr/local/bin/health-watcher.sh
COPY private/start-fluent-bit.sh /private/start-fluent-bit.sh
COPY private/etc /private/etc
COPY private/packages /private/packages
RUN chmod +x /usr/local/bin/node-bootstrap.sh /usr/local/bin/health-watcher.sh /private/start-fluent-bit.sh || true; \
mkdir -p /logs/train /logs/infer /buffers /opt/argus-metric; \
chmod 1777 /logs/train /logs/infer || true; \
chmod 770 /buffers || true
ENTRYPOINT ["/usr/local/bin/node-bootstrap.sh"]

View File

@@ -0,0 +1,59 @@
#!/usr/bin/env bash
set -euo pipefail
# health-watcher.sh (GPU bundle)
# 周期执行 check_health.sh 与 restart_unhealthy.sh,用于 GPU 节点容器内自愈。
INSTALL_ROOT="/opt/argus-metric"
INTERVAL="${HEALTH_WATCH_INTERVAL:-60}"
VER_DIR="${1:-}"
log(){ echo "[HEALTH-WATCHER] $*"; }
resolve_ver_dir() {
local dir=""
if [[ -n "${VER_DIR:-}" && -d "$VER_DIR" ]]; then
dir="$VER_DIR"
elif [[ -L "$INSTALL_ROOT/current" ]]; then
dir="$(readlink -f "$INSTALL_ROOT/current" 2>/dev/null || true)"
fi
if [[ -z "$dir" ]]; then
dir="$(ls -d "$INSTALL_ROOT"/versions/* 2>/dev/null | sort -V | tail -n1 || true)"
fi
echo "$dir"
}
main() {
log "starting with interval=${INTERVAL}s"
local dir
dir="$(resolve_ver_dir)"
if [[ -z "$dir" || ! -d "$dir" ]]; then
log "no valid install dir found under $INSTALL_ROOT; exiting"
exit 0
fi
local chk="$dir/check_health.sh"
local rst="$dir/restart_unhealthy.sh"
if [[ ! -x "$chk" && ! -x "$rst" ]]; then
log "neither check_health.sh nor restart_unhealthy.sh is executable under $dir; exiting"
exit 0
fi
log "watching install dir: $dir"
while :; do
if [[ -x "$chk" ]]; then
log "running check_health.sh"
"$chk" >> "$dir/.health_check.watch.log" 2>&1 || log "check_health.sh reported issues (see .health_check.watch.log)"
fi
if [[ -x "$rst" ]]; then
log "running restart_unhealthy.sh"
"$rst" >> "$dir/.restart.watch.log" 2>&1 || log "restart_unhealthy.sh reported issues (see .restart.watch.log)"
fi
sleep "$INTERVAL"
done
}
main "$@"

View File

@@ -0,0 +1,135 @@
#!/usr/bin/env bash
set -euo pipefail
echo "[BOOT] GPU node bundle starting"
INSTALL_ROOT="/opt/argus-metric"
BUNDLE_DIR="/bundle"
STATE_DIR_BASE="/private/argus/agent"
mkdir -p "$INSTALL_ROOT" "$STATE_DIR_BASE" /logs/train /logs/infer /buffers || true
# Ensure world-writable logs dir with sticky bit (align with deployment_new policy)
if [[ "${ARGUS_LOGS_WORLD_WRITABLE:-1}" == "1" ]]; then
chmod 1777 /logs/train /logs/infer || true
else
chmod 755 /logs/train /logs/infer || true
fi
chmod 770 /buffers || true
installed_ok=0
# 1) already installed?
if [[ -L "$INSTALL_ROOT/current" && -d "$INSTALL_ROOT/current" ]]; then
echo "[BOOT] client already installed at $INSTALL_ROOT/current"
else
# 2) try local bundle first (argus-metric_*.tar.gz)
tarball=$(ls -1 "$BUNDLE_DIR"/argus-metric_*.tar.gz 2>/dev/null | head -1 || true)
if [[ -n "${tarball:-}" ]]; then
echo "[BOOT] installing from local bundle: $(basename "$tarball")"
tmp=$(mktemp -d)
tar -xzf "$tarball" -C "$tmp"
# locate root containing version.json
root="$tmp"
if [[ ! -f "$root/version.json" ]]; then
sub=$(find "$tmp" -mindepth 1 -maxdepth 1 -type d | head -n1 || true)
[[ -n "$sub" && -f "$sub/version.json" ]] && root="$sub"
fi
if [[ ! -f "$root/version.json" ]]; then
echo "[BOOT][WARN] version.json not found in bundle; fallback to FTP"
else
ver=$(sed -n 's/.*"version"\s*:\s*"\([^"]\+\)".*/\1/p' "$root/version.json" | head -n1)
if [[ -z "$ver" ]]; then
echo "[BOOT][WARN] failed to parse version from version.json; fallback to FTP"
else
target_root="$INSTALL_ROOT"
version_dir="$target_root/versions/$ver"
mkdir -p "$version_dir"
shopt -s dotglob
mv "$root"/* "$version_dir/" 2>/dev/null || true
shopt -u dotglob
if [[ -f "$version_dir/install.sh" ]]; then
chmod +x "$version_dir/install.sh" 2>/dev/null || true
(
export AUTO_START_DCGM="${AUTO_START_DCGM:-1}"
export DCGM_EXPORTER_DISABLE_PROFILING="${DCGM_EXPORTER_DISABLE_PROFILING:-1}"
export DCGM_EXPORTER_LISTEN="${DCGM_EXPORTER_LISTEN:-:9400}"
cd "$version_dir" && ./install.sh "$version_dir"
)
echo "$ver" > "$target_root/LATEST_VERSION" 2>/dev/null || true
ln -sfn "$version_dir" "$target_root/current" 2>/dev/null || true
if [[ -L "$target_root/current" && -d "$target_root/current" ]]; then
installed_ok=1
echo "[BOOT] local bundle install OK: version=$ver"
else
echo "[BOOT][WARN] current symlink not present after install; will rely on healthcheck to confirm"
fi
else
echo "[BOOT][WARN] install.sh missing under $version_dir; fallback to FTP"
fi
fi
fi
fi
# 3) fallback: use FTP setup if not installed
if [[ ! -L "$INSTALL_ROOT/current" && "$installed_ok" -eq 0 ]]; then
echo "[BOOT] fallback to FTP setup"
if [[ -z "${FTPIP:-}" || -z "${FTP_USER:-}" || -z "${FTP_PASSWORD:-}" ]]; then
echo "[BOOT][ERROR] FTP variables not set (FTPIP/FTP_USER/FTP_PASSWORD)" >&2
exit 1
fi
curl -u "$FTP_USER:$FTP_PASSWORD" -fsSL "ftp://$FTPIP:21/setup.sh" -o /tmp/setup.sh
chmod +x /tmp/setup.sh
/tmp/setup.sh --server "$FTPIP" --user "$FTP_USER" --password "$FTP_PASSWORD" --port 21
fi
fi
# 4) ensure argus-agent is running (best-effort)
if ! pgrep -x argus-agent >/dev/null 2>&1; then
echo "[BOOT] starting argus-agent (not detected)"
setsid /usr/local/bin/argus-agent >/var/log/argus-agent.log 2>&1 < /dev/null &
fi
# 5) post-install selfcheck (run once) and state
# prefer current version dir; fallback to first version under /opt/argus-metric/versions
ver_dir=""
if [[ -L "$INSTALL_ROOT/current" ]]; then
ver_dir="$(readlink -f "$INSTALL_ROOT/current" 2>/dev/null || true)"
fi
if [[ -z "$ver_dir" ]]; then
# pick the latest by name (semver-like); best-effort
ver_dir="$(ls -d "$INSTALL_ROOT"/versions/* 2>/dev/null | sort -V | tail -n1 || true)"
fi
if [[ -n "$ver_dir" && -x "$ver_dir/check_health.sh" ]]; then
echo "[BOOT] running initial health check: $ver_dir/check_health.sh"
if "$ver_dir/check_health.sh" >> "$ver_dir/.health_check.init.log" 2>&1; then
echo "[BOOT] initial health check completed (see $ver_dir/.health_check.init.log)"
else
echo "[BOOT][WARN] initial health check reported issues (see $ver_dir/.health_check.init.log)"
fi
else
echo "[BOOT][WARN] initial health check skipped (script missing: $ver_dir/check_health.sh)"
fi
host="$(hostname)"
state_dir="$STATE_DIR_BASE/${host}"
mkdir -p "$state_dir" 2>/dev/null || true
for i in {1..60}; do
if [[ -s "$state_dir/node.json" ]]; then
echo "[BOOT] node state present: $state_dir/node.json"
break
fi
sleep 2
done
# 6) spawn health watcher (best-effort, non-blocking)
if command -v /usr/local/bin/health-watcher.sh >/dev/null 2>&1; then
echo "[BOOT] starting health watcher for $ver_dir"
setsid /usr/local/bin/health-watcher.sh "${ver_dir:-}" >/var/log/health-watcher.log 2>&1 < /dev/null || true &
else
echo "[BOOT][WARN] health-watcher.sh not found; skip health watcher"
fi
echo "[BOOT] ready; entering sleep"
exec sleep infinity

View File

@@ -0,0 +1,76 @@
FROM docker.elastic.co/elasticsearch/elasticsearch:8.17.10
# 切换到 root 用户进行系统级安装
USER root
ARG ARGUS_BUILD_UID=2133
ARG ARGUS_BUILD_GID=2015
ENV ARGUS_BUILD_UID=${ARGUS_BUILD_UID} \
ARGUS_BUILD_GID=${ARGUS_BUILD_GID}
# 调整 elasticsearch 用户与用户组 ID 以匹配宿主机配置
RUN set -eux; \
current_gid="$(getent group elasticsearch | awk -F: '{print $3}')"; \
if [ -z "$current_gid" ]; then \
groupadd -g "${ARGUS_BUILD_GID}" elasticsearch; \
elif [ "$current_gid" != "${ARGUS_BUILD_GID}" ]; then \
groupmod -g "${ARGUS_BUILD_GID}" elasticsearch; \
fi; \
if id elasticsearch >/dev/null 2>&1; then \
current_uid="$(id -u elasticsearch)"; \
if [ "$current_uid" != "${ARGUS_BUILD_UID}" ]; then \
usermod -u "${ARGUS_BUILD_UID}" elasticsearch; \
fi; \
else \
useradd -m -u "${ARGUS_BUILD_UID}" -g "${ARGUS_BUILD_GID}" elasticsearch; \
fi; \
chown -R "${ARGUS_BUILD_UID}:${ARGUS_BUILD_GID}" /usr/share/elasticsearch
# 设置构建参数
ARG USE_INTRANET=false
# 配置内网 apt 源 (如果指定了内网选项)
RUN if [ "$USE_INTRANET" = "true" ]; then \
echo "Configuring intranet apt sources..." && \
cp /etc/apt/sources.list /etc/apt/sources.list.bak && \
echo "deb [trusted=yes] http://10.68.64.1/ubuntu2204/ jammy main" > /etc/apt/sources.list && \
echo 'Acquire::https::Verify-Peer "false";' > /etc/apt/apt.conf.d/99disable-ssl-check && \
echo 'Acquire::https::Verify-Host "false";' >> /etc/apt/apt.conf.d/99disable-ssl-check; \
fi
# 安装 supervisor, net-tools, vim
RUN apt-get update && \
apt-get install -y supervisor net-tools inetutils-ping vim && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# 配置部署时使用的apt源
RUN if [ "$USE_INTRANET" = "true" ]; then \
echo "deb [trusted=yes] https://10.92.132.52/mirrors/ubuntu2204/ jammy main" > /etc/apt/sources.list; \
fi
# 创建 supervisor 日志目录
RUN mkdir -p /var/log/supervisor
# 复制 supervisor 配置文件
COPY src/log/elasticsearch/build/supervisord.conf /etc/supervisor/conf.d/supervisord.conf
# 复制启动脚本
COPY src/log/elasticsearch/build/start-es-supervised.sh /usr/local/bin/start-es-supervised.sh
RUN chmod +x /usr/local/bin/start-es-supervised.sh
# 复制DNS监控脚本
COPY src/log/elasticsearch/build/dns-monitor.sh /usr/local/bin/dns-monitor.sh
RUN chmod +x /usr/local/bin/dns-monitor.sh
# 保持 root 用户,由 supervisor 管理用户切换
USER root
# 暴露端口
EXPOSE 9200 9300
# 使用 supervisor 作为入口点
CMD ["/usr/bin/supervisord", "-c", "/etc/supervisor/conf.d/supervisord.conf"]

View File

@@ -22,8 +22,7 @@
[PARSER] [PARSER]
Name timestamp_parser Name timestamp_parser
Format regex Format regex
Regex ^(?<timestamp>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\s+(?<level>\w+)\s+(?<message>.*)$ Regex ^(?<timestamp>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:Z|[+-]\d{2}:?\d{2}))\s+(?<level>\w+)\s+(?<message>.*)$
Time_Key timestamp Time_Key timestamp
Time_Format %Y-%m-%d %H:%M:%S Time_Format %Y-%m-%dT%H:%M:%S%z
Time_Offset +0800
Time_Keep On Time_Keep On

View File

@@ -38,14 +38,16 @@ ensure_lib() {
ldconfig 2>/dev/null || true ldconfig 2>/dev/null || true
} }
DEB_ARCH="${DEB_ARCH:-$(dpkg --print-architecture 2>/dev/null || echo amd64)}"
# Offline-first: satisfy runtime deps from local debs, fallback to apt only if necessary # Offline-first: satisfy runtime deps from local debs, fallback to apt only if necessary
ensure_lib "libpq.so.5" "libpq5_*_amd64.deb" ensure_lib "libpq.so.5" "libpq5_*_${DEB_ARCH}.deb"
ensure_lib "libyaml-0.so.2" "libyaml-0-2_*_amd64.deb" ensure_lib "libyaml-0.so.2" "libyaml-0-2_*_${DEB_ARCH}.deb"
ensure_lib "libsasl2.so.2" "libsasl2-2_*_amd64.deb" ensure_lib "libsasl2.so.2" "libsasl2-2_*_${DEB_ARCH}.deb"
ensure_lib "libldap-2.5.so.0" "libldap-2.5-0_*_amd64.deb" ensure_lib "libldap-2.5.so.0" "libldap-2.5-0_*_${DEB_ARCH}.deb"
# Install fluent-bit main package from local bundle # Install fluent-bit main package from local bundle
FLB_DEB="$(ls /tmp/flb/packages/fluent-bit_*_amd64.deb 2>/dev/null | head -n1 || true)" FLB_DEB="$(ls /tmp/flb/packages/fluent-bit_*_${DEB_ARCH}.deb 2>/dev/null | head -n1 || true)"
if [[ -z "$FLB_DEB" ]]; then if [[ -z "$FLB_DEB" ]]; then
echo "[ERROR] fluent-bit deb not found under /private/packages" >&2 echo "[ERROR] fluent-bit deb not found under /private/packages" >&2
exit 1 exit 1
@@ -77,7 +79,20 @@ cp -r /tmp/flb/etc/* /etc/fluent-bit/
# Create logs/buffers dirs # Create logs/buffers dirs
mkdir -p /logs/train /logs/infer /buffers mkdir -p /logs/train /logs/infer /buffers
chmod 755 /logs/train /logs/infer /buffers
# 控制日志目录权限:默认对宿主 bind mount 目录采用 1777(可由环境变量关闭)
: "${ARGUS_LOGS_WORLD_WRITABLE:=1}"
if [[ "${ARGUS_LOGS_WORLD_WRITABLE}" == "1" ]]; then
chmod 1777 /logs/train /logs/infer || true
else
chmod 755 /logs/train /logs/infer || true
fi
# 缓冲目录仅供进程使用,不对外开放写入
chmod 770 /buffers || true
# 目录属主设置为 fluent-bit(不影响 1777 粘滞位)
chown -R fluent-bit:fluent-bit /logs /buffers 2>/dev/null || true
# Wait for Elasticsearch via bash /dev/tcp to avoid curl dependency # Wait for Elasticsearch via bash /dev/tcp to avoid curl dependency
echo "[INFO] Waiting for Elasticsearch to be ready (tcp ${ES_HOST}:${ES_PORT})..." echo "[INFO] Waiting for Elasticsearch to be ready (tcp ${ES_HOST}:${ES_PORT})..."

View File

@@ -0,0 +1,80 @@
FROM docker.elastic.co/kibana/kibana:8.17.10
# 切换到 root 用户进行系统级安装
USER root
ARG ARGUS_BUILD_UID=2133
ARG ARGUS_BUILD_GID=2015
ENV ARGUS_BUILD_UID=${ARGUS_BUILD_UID} \
ARGUS_BUILD_GID=${ARGUS_BUILD_GID}
# 调整 kibana 用户与用户组 ID 以匹配宿主机配置
RUN set -eux; \
current_gid="$(getent group kibana | awk -F: '{print $3}')"; \
if [ -z "$current_gid" ]; then \
groupadd -g "${ARGUS_BUILD_GID}" kibana; \
elif [ "$current_gid" != "${ARGUS_BUILD_GID}" ]; then \
groupmod -g "${ARGUS_BUILD_GID}" kibana; \
fi; \
if id kibana >/dev/null 2>&1; then \
current_uid="$(id -u kibana)"; \
if [ "$current_uid" != "${ARGUS_BUILD_UID}" ]; then \
usermod -u "${ARGUS_BUILD_UID}" kibana; \
fi; \
else \
useradd -m -u "${ARGUS_BUILD_UID}" -g "${ARGUS_BUILD_GID}" kibana; \
fi; \
chown -R "${ARGUS_BUILD_UID}:${ARGUS_BUILD_GID}" /usr/share/kibana
# 设置构建参数
ARG USE_INTRANET=false
# 配置内网 apt 源 (如果指定了内网选项)
RUN if [ "$USE_INTRANET" = "true" ]; then \
echo "Configuring intranet apt sources..." && \
cp /etc/apt/sources.list /etc/apt/sources.list.bak && \
echo "deb [trusted=yes] http://10.68.64.1/ubuntu2204/ jammy main" > /etc/apt/sources.list && \
echo 'Acquire::https::Verify-Peer "false";' > /etc/apt/apt.conf.d/99disable-ssl-check && \
echo 'Acquire::https::Verify-Host "false";' >> /etc/apt/apt.conf.d/99disable-ssl-check; \
fi
# 安装 supervisor, net-tools, vim
RUN apt-get update && \
apt-get install -y supervisor net-tools inetutils-ping vim && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# 配置部署时使用的apt源
RUN if [ "$USE_INTRANET" = "true" ]; then \
echo "deb [trusted=yes] https://10.92.132.52/mirrors/ubuntu2204/ jammy main" > /etc/apt/sources.list; \
fi
# 创建 supervisor 日志目录
RUN mkdir -p /var/log/supervisor
# 复制 supervisor 配置文件
COPY src/log/kibana/build/supervisord.conf /etc/supervisor/conf.d/supervisord.conf
# 复制启动脚本
COPY src/log/kibana/build/start-kibana-supervised.sh /usr/local/bin/start-kibana-supervised.sh
COPY src/log/kibana/build/kibana-post-start.sh /usr/local/bin/kibana-post-start.sh
RUN chmod +x /usr/local/bin/start-kibana-supervised.sh /usr/local/bin/kibana-post-start.sh
# 复制DNS监控脚本
COPY src/log/kibana/build/dns-monitor.sh /usr/local/bin/dns-monitor.sh
RUN chmod +x /usr/local/bin/dns-monitor.sh
# kibana需要用到 /root/.config/puppeteer 路径
RUN chmod 777 /root
# 保持 root 用户,由 supervisor 管理用户切换
USER root
# 暴露端口
EXPOSE 5601
# 使用 supervisor 作为入口点
CMD ["/usr/bin/supervisord", "-c", "/etc/supervisor/conf.d/supervisord.conf"]

View File

@@ -36,9 +36,28 @@ echo "[INFO] Fluent-bit files should be in fluent-bit/ directory"
# 准备 Fluent Bit 离线依赖(从 metric all-in-one-full 复制 deb 到 ../fluent-bit/build/packages) # 准备 Fluent Bit 离线依赖(从 metric all-in-one-full 复制 deb 到 ../fluent-bit/build/packages)
FLB_BUILD_PACKAGES_DIR="$root/../fluent-bit/build/packages" FLB_BUILD_PACKAGES_DIR="$root/../fluent-bit/build/packages"
mkdir -p "$FLB_BUILD_PACKAGES_DIR" mkdir -p "$FLB_BUILD_PACKAGES_DIR"
detect_deb_arch() {
local deb_arch="${1:-}"
if [[ -n "$deb_arch" ]]; then
echo "$deb_arch"; return
fi
if command -v dpkg >/dev/null 2>&1; then
dpkg --print-architecture # amd64 / arm64
else
case "$(uname -m)" in
x86_64) echo amd64 ;;
aarch64) echo arm64 ;;
*) echo amd64 ;;
esac
fi
}
DEB_ARCH="$(detect_deb_arch)"
FLB_BIN_DIR="$project_root/src/metric/client-plugins/all-in-one-full/plugins/fluent-bit/bin/${DEB_ARCH}"
for deb in \ for deb in \
"$project_root/src/metric/client-plugins/all-in-one-full/plugins/fluent-bit/bin/libyaml-0-2_"*_amd64.deb \ "$FLB_BIN_DIR/libyaml-0-2_"*_"${DEB_ARCH}.deb" \
"$project_root/src/metric/client-plugins/all-in-one-full/plugins/fluent-bit/bin/libpq5_"*_amd64.deb ; do "$FLB_BIN_DIR/libpq5_"*_"${DEB_ARCH}.deb" ; do
if ls $deb >/dev/null 2>&1; then if ls $deb >/dev/null 2>&1; then
for f in $deb; do for f in $deb; do
base="$(basename "$f")" base="$(basename "$f")"
@@ -56,12 +75,12 @@ if [[ -f "$CURLOPT_TAR" ]]; then
tmpdir=$(mktemp -d) tmpdir=$(mktemp -d)
if tar -xzf "$CURLOPT_TAR" -C "$tmpdir" 2>/dev/null; then if tar -xzf "$CURLOPT_TAR" -C "$tmpdir" 2>/dev/null; then
for p in \ for p in \
libsasl2-2_*_amd64.deb \ "libsasl2-2_*_${DEB_ARCH}.deb" \
libsasl2-modules-db_*_amd64.deb \ "libsasl2-modules-db_*_${DEB_ARCH}.deb" \
libldap-2.5-0_*_amd64.deb \ "libldap-2.5-0_*_${DEB_ARCH}.deb" \
libidn2-0_*_amd64.deb \ "libidn2-0_*_${DEB_ARCH}.deb" \
libbrotli1_*_amd64.deb \ "libbrotli1_*_${DEB_ARCH}.deb" \
libssl3_*_amd64.deb ; do "libssl3_*_${DEB_ARCH}.deb" ; do
src=$(ls "$tmpdir"/curl/$p 2>/dev/null | head -n1 || true) src=$(ls "$tmpdir"/curl/$p 2>/dev/null | head -n1 || true)
if [[ -n "$src" ]]; then if [[ -n "$src" ]]; then
base="$(basename "$src")" base="$(basename "$src")"

View File

@@ -28,11 +28,11 @@ fi
docker exec "$container_name" mkdir -p /logs/train /logs/infer docker exec "$container_name" mkdir -p /logs/train /logs/infer
# 写入训练日志 (host01) # 写入训练日志 (host01)
docker exec "$container_name" sh -c "printf '%s INFO [host01] training step=1 loss=1.23 model=bert\n' \"\$(date '+%F %T')\" >> /logs/train/train-demo.log" docker exec "$container_name" sh -c "printf '%s INFO [host01] training step=1 loss=1.23 model=bert\n' \"\$(date -u +%Y-%m-%dT%H:%M:%SZ)\" >> /logs/train/train-demo.log"
docker exec "$container_name" sh -c "printf '%s INFO [host01] training step=2 loss=1.15 model=bert\n' \"\$(date '+%F %T')\" >> /logs/train/train-demo.log" docker exec "$container_name" sh -c "printf '%s INFO [host01] training step=2 loss=1.15 model=bert\n' \"\$(date -u +%Y-%m-%dT%H:%M:%SZ)\" >> /logs/train/train-demo.log"
# 写入推理日志 (host01) # 写入推理日志 (host01)
docker exec "$container_name" sh -c "printf '%s ERROR [host01] inference failed on batch=1\n' \"\$(date '+%F %T')\" >> /logs/infer/infer-demo.log" docker exec "$container_name" sh -c "printf '%s ERROR [host01] inference failed on batch=1\n' \"\$(date -u +%Y-%m-%dT%H:%M:%SZ)\" >> /logs/infer/infer-demo.log"
docker exec "$container_name" sh -c "cat <<'STACK' >> /logs/infer/infer-demo.log docker exec "$container_name" sh -c "cat <<'STACK' >> /logs/infer/infer-demo.log
Traceback (most recent call last): Traceback (most recent call last):
File \"inference.py\", line 15, in <module> File \"inference.py\", line 15, in <module>

View File

@@ -28,13 +28,13 @@ fi
docker exec "$container_name" mkdir -p /logs/train /logs/infer docker exec "$container_name" mkdir -p /logs/train /logs/infer
# 写入训练日志 (host02) # 写入训练日志 (host02)
docker exec "$container_name" sh -c "printf '%s INFO [host02] training step=1 loss=1.45 model=gpt\n' \"\$(date '+%F %T')\" >> /logs/train/train-demo.log" docker exec "$container_name" sh -c "printf '%s INFO [host02] training step=1 loss=1.45 model=gpt\n' \"\$(date -u +%Y-%m-%dT%H:%M:%SZ)\" >> /logs/train/train-demo.log"
docker exec "$container_name" sh -c "printf '%s INFO [host02] training step=2 loss=1.38 model=gpt\n' \"\$(date '+%F %T')\" >> /logs/train/train-demo.log" docker exec "$container_name" sh -c "printf '%s INFO [host02] training step=2 loss=1.38 model=gpt\n' \"\$(date -u +%Y-%m-%dT%H:%M:%SZ)\" >> /logs/train/train-demo.log"
docker exec "$container_name" sh -c "printf '%s INFO [host02] training step=3 loss=1.32 model=gpt\n' \"\$(date '+%F %T')\" >> /logs/train/train-demo.log" docker exec "$container_name" sh -c "printf '%s INFO [host02] training step=3 loss=1.32 model=gpt\n' \"\$(date -u +%Y-%m-%dT%H:%M:%SZ)\" >> /logs/train/train-demo.log"
# 写入推理日志 (host02) # 写入推理日志 (host02)
docker exec "$container_name" sh -c "printf '%s WARN [host02] inference slow on batch=5 latency=2.3s\n' \"\$(date '+%F %T')\" >> /logs/infer/infer-demo.log" docker exec "$container_name" sh -c "printf '%s WARN [host02] inference slow on batch=5 latency=2.3s\n' \"\$(date -u +%Y-%m-%dT%H:%M:%SZ)\" >> /logs/infer/infer-demo.log"
docker exec "$container_name" sh -c "printf '%s INFO [host02] inference completed batch=6 latency=0.8s\n' \"\$(date '+%F %T')\" >> /logs/infer/infer-demo.log" docker exec "$container_name" sh -c "printf '%s INFO [host02] inference completed batch=6 latency=0.8s\n' \"\$(date -u +%Y-%m-%dT%H:%M:%SZ)\" >> /logs/infer/infer-demo.log"
echo "[OK] 已通过docker exec写入测试日志到 host02 容器内:" echo "[OK] 已通过docker exec写入测试日志到 host02 容器内:"
echo " - /logs/train/train-demo.log" echo " - /logs/train/train-demo.log"

View File

@@ -13,6 +13,8 @@ class AppConfig:
scheduler_interval_seconds: int scheduler_interval_seconds: int
node_id_prefix: str node_id_prefix: str
auth_mode: str auth_mode: str
target_prefer_net_cidrs: str
target_reachability_check: bool
def _get_int_env(name: str, default: int) -> int: def _get_int_env(name: str, default: int) -> int:
@@ -27,6 +29,12 @@ def _get_int_env(name: str, default: int) -> int:
def load_config() -> AppConfig: def load_config() -> AppConfig:
"""读取环境变量生成配置对象,方便统一管理运行参数。""" """读取环境变量生成配置对象,方便统一管理运行参数。"""
def _bool_env(name: str, default: bool) -> bool:
raw = os.environ.get(name)
if raw is None or raw.strip() == "":
return default
return raw.strip().lower() in ("1", "true", "yes", "on")
return AppConfig( return AppConfig(
db_path=os.environ.get("DB_PATH", "/private/argus/master/db.sqlite3"), db_path=os.environ.get("DB_PATH", "/private/argus/master/db.sqlite3"),
metric_nodes_json_path=os.environ.get( metric_nodes_json_path=os.environ.get(
@@ -37,4 +45,6 @@ def load_config() -> AppConfig:
scheduler_interval_seconds=_get_int_env("SCHEDULER_INTERVAL_SECONDS", 30), scheduler_interval_seconds=_get_int_env("SCHEDULER_INTERVAL_SECONDS", 30),
node_id_prefix=os.environ.get("NODE_ID_PREFIX", "A"), node_id_prefix=os.environ.get("NODE_ID_PREFIX", "A"),
auth_mode=os.environ.get("AUTH_MODE", "disabled"), auth_mode=os.environ.get("AUTH_MODE", "disabled"),
target_prefer_net_cidrs=os.environ.get("TARGET_PREFER_NET_CIDRS", "10.0.0.0/8,172.31.0.0/16"),
target_reachability_check=_bool_env("TARGET_REACHABILITY_CHECK", False),
) )

View File

@@ -1,8 +1,10 @@
from __future__ import annotations from __future__ import annotations
import ipaddress
import logging import logging
import socket
import threading import threading
from typing import Optional from typing import Optional, Iterable, Dict, Any, List
from .config import AppConfig from .config import AppConfig
from .storage import Storage from .storage import Storage
@@ -34,10 +36,117 @@ class StatusScheduler:
self._pending_nodes_json.set() self._pending_nodes_json.set()
def generate_nodes_json(self) -> None: def generate_nodes_json(self) -> None:
"""根据在线节点生成 Prometheus 抓取目标,优先 overlay IP。
候选顺序:meta.overlay_ip > hostname A 记录(命中偏好网段)> meta.ip。
可选 reachability 检查:TARGET_REACHABILITY_CHECK=true 时,对 9100/9400 做一次 1s TCP 连接测试,
选择首个可达的候选;全部失败则按顺序取第一个并记录日志。
"""
with self._nodes_json_lock: with self._nodes_json_lock:
online_nodes = self._storage.get_online_nodes() rows = self._storage.get_online_nodes_meta()
atomic_write_json(self._config.metric_nodes_json_path, online_nodes) prefer_cidrs = self._parse_cidrs(self._config.target_prefer_net_cidrs)
self._logger.info("nodes.json updated", extra={"count": len(online_nodes)}) reachability = self._config.target_reachability_check
result: List[Dict[str, Any]] = []
for row in rows:
meta = row.get("meta", {})
hostname = meta.get("hostname") or row.get("name")
labels = row.get("labels") or []
overlay_ip = meta.get("overlay_ip")
legacy_ip = meta.get("ip")
host_candidates = self._resolve_host_ips(hostname)
host_pref = self._pick_by_cidrs(host_candidates, prefer_cidrs)
candidates: List[str] = []
for ip in [overlay_ip, host_pref, legacy_ip]:
if ip and ip not in candidates:
candidates.append(ip)
chosen = None
if reachability:
ports = [9100]
try:
if int(meta.get("gpu_number", 0)) > 0:
ports.append(9400)
except Exception:
pass
for ip in candidates:
if any(self._reachable(ip, p, 1.0) for p in ports):
chosen = ip
break
if not chosen:
chosen = candidates[0] if candidates else legacy_ip
if not chosen:
# ultimate fallback: 127.0.0.1 (should not happen)
chosen = "127.0.0.1"
self._logger.warning("No candidate IPs for node; falling back", extra={"node": row.get("node_id")})
if chosen and ipaddress.ip_address(chosen) in ipaddress.ip_network("172.22.0.0/16"):
self._logger.warning(
"Prometheus target uses docker_gwbridge address; prefer overlay",
extra={"node": row.get("node_id"), "ip": chosen},
)
result.append(
{
"node_id": row.get("node_id"),
"user_id": meta.get("user"),
"ip": chosen,
"hostname": hostname,
"labels": labels if isinstance(labels, list) else [],
}
)
atomic_write_json(self._config.metric_nodes_json_path, result)
self._logger.info("nodes.json updated", extra={"count": len(result)})
# ---------------------------- helpers ----------------------------
@staticmethod
def _parse_cidrs(raw: str) -> List[ipaddress.IPv4Network]:
nets: List[ipaddress.IPv4Network] = []
for item in (x.strip() for x in (raw or "").split(",")):
if not item:
continue
try:
net = ipaddress.ip_network(item, strict=False)
if isinstance(net, ipaddress.IPv4Network):
nets.append(net)
except ValueError:
continue
return nets
@staticmethod
def _resolve_host_ips(hostname: str) -> List[str]:
ips: List[str] = []
try:
infos = socket.getaddrinfo(hostname, None, family=socket.AF_INET)
for info in infos:
ip = info[4][0]
if ip not in ips:
ips.append(ip)
except OSError:
pass
return ips
@staticmethod
def _pick_by_cidrs(candidates: Iterable[str], prefer: List[ipaddress.IPv4Network]) -> str | None:
for net in prefer:
for ip in candidates:
try:
if ipaddress.ip_address(ip) in net:
return ip
except ValueError:
continue
return None
@staticmethod
def _reachable(ip: str, port: int, timeout: float) -> bool:
try:
with socket.create_connection((ip, port), timeout=timeout):
return True
except OSError:
return False
# ------------------------------------------------------------------ # ------------------------------------------------------------------
# internal loop # internal loop

View File

@@ -324,9 +324,35 @@ class Storage:
{ {
"node_id": row["id"], "node_id": row["id"],
"user_id": meta.get("user"), "user_id": meta.get("user"),
"ip": meta.get("ip"), "ip": meta.get("ip"), # kept for backward-compat; preferred IP selection handled in scheduler
"hostname": meta.get("hostname", row["name"]), "hostname": meta.get("hostname", row["name"]),
"labels": labels if isinstance(labels, list) else [], "labels": labels if isinstance(labels, list) else [],
} }
) )
return result return result
def get_online_nodes_meta(self) -> List[Dict[str, Any]]:
"""返回在线节点的原始 meta 与名称、标签,交由上层选择目标 IP。
每项包含:{ node_id, name, meta, labels }
"""
with self._lock:
cur = self._conn.execute(
"SELECT id, name, meta_json, labels_json FROM nodes WHERE status = ? ORDER BY id ASC",
("online",),
)
rows = cur.fetchall()
result: List[Dict[str, Any]] = []
for row in rows:
meta = json.loads(row["meta_json"]) if row["meta_json"] else {}
labels = json.loads(row["labels_json"]) if row["labels_json"] else []
result.append(
{
"node_id": row["id"],
"name": row["name"],
"meta": meta if isinstance(meta, dict) else {},
"labels": labels if isinstance(labels, list) else [],
}
)
return result

View File

@@ -1 +1 @@
1.40.0 1.44.0

View File

@@ -0,0 +1 @@
bin/

Some files were not shown because too many files have changed in this diff Show More