29 Commits

Author SHA1 Message Date
debe91fc8b [#50] 完成arm服务器单台机器上部署 server & client,没有使用docker swarm 2025-12-05 11:51:56 +08:00
a6e1bf7fe7 [#50] x86机器使用qemu模拟方式安装部署arm版本安装包,当前未精简版,只有简单的node exporter功能,删减日志,GPU指标功能,仍使用Swarm组网 2025-12-01 15:27:12 +08:00
4c45166b44 [#50] 完成在x86_64平台上使用buildx来构建arm镜像;重新构建了core/web/prom/grafana以及cpu node镜像;跑通swarm test for arm; 暂时移除dcgm和log路径功能 2025-11-24 16:31:39 +08:00
2caf0fa214 [#49] 优化swarm test支持自动reboot和verify 2025-11-20 15:21:18 +08:00
d4e0dc1511 [#49] swarm test重启测试通过 2025-11-19 17:26:26 +08:00
1d38304936 [#49] 移除bind/ftp,改成使用docker 自带域名解析,实现重启后IP变化仍正常访问,完成swarm test通过 2025-11-18 15:10:38 +08:00
5b617f62a8 [#43] 区分server&client 不同compose project;修改require compose监测 2025-11-17 12:17:11 +08:00
69e7a3e2b8 [#47] 增加cpu bundle镜像构建 2025-11-14 16:43:34 +08:00
b402fdf960 [#37] 删除旧部署构建目录;跑通src/sys/tests; 增加一键构建 server pkg和client gpu pkg 2025-11-14 15:07:24 +08:00
fff90826a4 [#41] 增加一键从源码构建 server_pkg和client_pkg 2025-11-13 15:02:33 +08:00
d0411e6b97 [#41] 优化gpu bundle构建优化 2025-11-13 10:28:37 +08:00
06131a268a [#40] log目录供宿主机其他程序可写 2025-11-12 15:06:37 +08:00
df1f519355 [#39] 增加新docker compose部署方式,验证通过gpu部署;剩余fluent-bit logs目录权限问题 以及 gpu bundle打包优化问题 2025-11-12 12:07:04 +08:00
6837d96035 [#39] 增加使用busybox warmup接入overlay网络 2025-11-10 12:17:10 +08:00
dac180f12b [#37] 修复dcgm exporter启动 2025-11-07 17:29:06 +08:00
1819fb9c46 [#37] 修复alert镜像用户 2025-11-07 12:21:41 +08:00
7548e46d1f [#37] 增加gpu bundle node镜像构建 2025-11-07 10:23:59 +08:00
0b9268332f [#37] 修复log时间戳测试问题 2025-11-06 17:20:40 +08:00
d1fad4a05a [#37] 增加sys/swarm_tests(cpu) ;单独构建的node bundle镜像 2025-11-06 16:43:14 +08:00
94b3e910b3 [#37] 增加部署时自动检测空闲端口;增加es 水位检测和临时应急处理 2025-11-05 16:21:34 +08:00
2ff7c55f3b [#37] 测试通过swarm跨机部署节点;更新文档 2025-11-05 09:57:08 +08:00
9858f4471e [#37] server install 增加重试自检 2025-11-05 09:57:08 +08:00
c8279997a4 [#37] swarm 部署优化 2025-11-05 09:57:08 +08:00
4ed5c64804 [#37] 优化client构建 2025-11-05 09:57:08 +08:00
3551360687 [#37] 优化client安装包 2025-11-05 09:57:08 +08:00
3202e02b42 [#37] 测试NixOS部署server通过 2025-11-05 09:57:07 +08:00
29eb75a374 [#37] 构建安装包 2025-11-05 09:57:07 +08:00
ccc141f557 [#30] ftp容器增加动态检测并更新dns.conf到share目录 2025-11-05 09:57:07 +08:00
ed0d1ca904 [#30] wsl部署测试;更新README 2025-11-05 09:57:07 +08:00
171 changed files with 12202 additions and 231 deletions

150
build/README.md Normal file
View File

@@ -0,0 +1,150 @@
# ARGUS 统一构建脚本使用说明(build/build_images.sh)
本目录提供单一入口脚本 `build/build_images.sh`,覆盖常见三类场景:
- 系统集成测试(src/sys/tests)
- Swarm 系统集成测试(src/sys/swarm_tests)
- 构建离线安装包(deployment_new:Server/Client‑GPU)
文档还说明 UID/GID 取值规则、镜像 tag 策略、常用参数与重试机制。
## 环境前置
- Docker Engine ≥ 20.10(建议 ≥ 23.x/24.x)
- Docker Compose v2(`docker compose` 子命令)
- 可选:内网构建镜像源(`--intranet`)
## UID/GID 规则(用于容器内用户/卷属主)
- 非 pkg 构建(core/master/metric/web/alert/sys/gpu_bundle/cpu_bundle):
- 读取 `configs/build_user.local.conf` → `configs/build_user.conf`;
- 可被环境变量覆盖:`ARGUS_BUILD_UID`、`ARGUS_BUILD_GID`;
- pkg 构建(`--only server_pkg`、`--only client_pkg`):
- 读取 `configs/build_user.pkg.conf`(优先)→ `build_user.local.conf` → `build_user.conf`;
- 可被环境变量覆盖;
- CPU bundle 明确走“非 pkg”链(不读取 `build_user.pkg.conf`)。
- 说明:仅依赖 UID/GID 的 Docker 层会因参数变动而自动重建,不同构建剖面不会“打错包”。
## 镜像 tag 策略
- 非 pkg 构建:默认输出 `:latest`。
- `--only server_pkg`:所有镜像直接输出为 `:<VERSION>`(不覆盖 `:latest`)。
- `--only client_pkg`:GPU bundle 仅输出 `:<VERSION>`(不覆盖 `:latest`)。
- `--only cpu_bundle`:默认仅输出 `:<VERSION>`;可加 `--tag-latest` 同时打 `:latest` 以兼容 swarm_tests 默认 compose。
## 不加 --only 的默认构建目标
不指定 `--only` 时,脚本会构建“基础镜像集合”(不含 bundle 与安装包):
- core:`argus-elasticsearch:latest`、`argus-kibana:latest`、`argus-bind9:latest`
- master:`argus-master:latest`(非 offline)
- metric:`argus-metric-ftp:latest`、`argus-metric-prometheus:latest`、`argus-metric-grafana:latest`
- web:`argus-web-frontend:latest`、`argus-web-proxy:latest`
- alert:`argus-alertmanager:latest`
- sys:`argus-sys-node:latest`、`argus-sys-metric-test-node:latest`、`argus-sys-metric-test-gpu-node:latest`
说明:默认 tag 为 `:latest`;UID/GID 走“非 pkg”链(`build_user.local.conf → build_user.conf`,可被环境变量覆盖)。
## 通用参数
- `--intranet`:使用内网构建参数(各 Dockerfile 中按需启用)。
- `--no-cache`:禁用 Docker 层缓存。
- `--only <list>`:逗号分隔目标,例:`--only core,master,metric,web,alert`。
- `--version YYMMDD`:bundle/pkg 的日期标签(必填于 cpu_bundle/gpu_bundle/server_pkg/client_pkg)。
- `--client-semver X.Y.Z`:all‑in‑one‑full 客户端语义化版本(可选)。
- `--cuda VER`:GPU bundle CUDA 基镜版本(默认 12.2.2)。
- `--tag-latest`:CPU bundle 构建时同时打 `:latest`。
## 自动重试
- 构建单镜像失败会自动重试(默认 3 次,间隔 5s)。
- 最后一次自动使用 `DOCKER_BUILDKIT=0` 再试,缓解 “failed to receive status: context canceled”。
- 可调:`ARGUS_BUILD_RETRIES`、`ARGUS_BUILD_RETRY_DELAY` 环境变量。
---
## 场景一:系统集成测试(src/sys/tests)
构建用于系统级端到端测试的镜像(默认 `:latest`)。
示例:
```
# 构建核心与周边
./build/build_images.sh --only core,master,metric,web,alert,sys
```
产出:
- 本地镜像:`argus-elasticsearch:latest`、`argus-kibana:latest`、`argus-master:latest`、`argus-metric-ftp:latest`、`argus-metric-prometheus:latest`、`argus-metric-grafana:latest`、`argus-alertmanager:latest`、`argus-web-frontend:latest`、`argus-web-proxy:latest`、`argus-sys-node:latest` 等。
说明:
- UID/GID 读取 `build_user.local.conf → build_user.conf`(或环境变量覆盖)。
- sys/tests 的执行见 `src/sys/tests/README.md`。
---
## 场景二:Swarm 系统集成测试(src/sys/swarm_tests)
需要服务端镜像 + CPU 节点 bundle 镜像。
步骤:
1) 构建服务端镜像(默认 `:latest`)
```
./build/build_images.sh --only core,master,metric,web,alert
```
2) 构建 CPU bundle(直接 FROM ubuntu:22.04)
```
# 仅版本 tag 输出
./build/build_images.sh --only cpu_bundle --version 20251114
# 若要兼容 swarm_tests 默认 latest:
./build/build_images.sh --only cpu_bundle --version 20251114 --tag-latest
```
3) 运行 Swarm 测试
```
cd src/sys/swarm_tests
# 如未打 latest,可先指定:
export NODE_BUNDLE_IMAGE_TAG=argus-sys-metric-test-node-bundle:20251114
./scripts/01_server_up.sh
./scripts/02_wait_ready.sh
./scripts/03_nodes_up.sh
./scripts/04_metric_verify.sh # 验证 Prometheus/Grafana/nodes.json 与日志通路
./scripts/99_down.sh # 结束
```
产出:
- 本地镜像:`argus-*:latest` 与 `argus-sys-metric-test-node-bundle:20251114`(或 latest)。
- `swarm_tests/private-*`:运行态持久化文件。
说明:
- CPU bundle 构建用户走“非 pkg”链(local.conf → conf)。
- `04_metric_verify.sh` 已内置 Fluent Bit 启动与配置修正逻辑,偶发未就绪可重跑一次即通过。
---
## 场景三:构建离线安装包(deployment_new)
Server 与 Client‑GPU 安装包均采用“版本直出”,只输出 `:<VERSION>` 标签,不改动 `:latest`。
1) Server 包
```
./build/build_images.sh --only server_pkg --version 20251114
```
产出:
- 本地镜像:`argus-<模块>:20251114`(不触碰 latest)。
- 安装包:`deployment_new/artifact/server/20251114/` 与 `server_20251114.tar.gz`
- 包内包含:逐镜像 tar.gz、compose/.env.example、scripts(config/install/selfcheck/diagnose 等)、docs、manifest/checksums。
2) Client‑GPU 包
```
# 同步构建 GPU bundle(仅 :<VERSION>,不触碰 latest),并生成客户端包
./build/build_images.sh --only client_pkg --version 20251114 \\
--client-semver 1.44.0 --cuda 12.2.2
```
产出:
- 本地镜像:`argus-sys-metric-test-node-bundle-gpu:20251114`
- 安装包:`deployment_new/artifact/client_gpu/20251114/` 与 `client_gpu_20251114.tar.gz`
- 包内包含:GPU bundle 镜像 tar.gz、busybox.tar、compose/.env.example、scripts(config/install/uninstall)、docs、manifest/checksums。
说明:
- pkg 构建使用 `configs/build_user.pkg.conf` 的 UID/GID(可被环境覆盖)。
- 包内 `.env.example` 的 `PKG_VERSION=<VERSION>` 与镜像 tag 严格一致。
---
## 常见问题(FAQ)
- 构建报 `failed to receive status: context canceled`?
- 已内置单镜像多次重试,最后一次禁用 BuildKit;建议加 `--intranet` 与 `--no-cache` 重试,或 `docker builder prune -f` 后再试。
- 先跑非 pkg(latest),再跑 pkg(version)会不会“打错包”?
- 不会。涉及 UID/GID 的层因参数变化会重建,其它层按缓存命中复用,最终 pkg 产物的属主与运行账户按 `build_user.pkg.conf` 生效。
- swarm_tests 默认拉取 `:latest`,我只构建了 `:<VERSION>` 的 CPU bundle 怎么办?
- 在运行前 `export NODE_BUNDLE_IMAGE_TAG=argus-sys-metric-test-node-bundle:<VERSION>`,或在构建时加 `--tag-latest`。
---
如需进一步自动化(例如生成 BUILD_SUMMARY.txt 汇总镜像 digest 与构建参数),可在 pkg 产出阶段追加,我可以按需补齐。

View File

@@ -12,7 +12,11 @@ Options:
--master-offline Build master offline image (requires src/master/offline_wheels.tar.gz)
--metric Build metric module images (ftp, prometheus, grafana, test nodes)
--no-cache Build all images without using Docker layer cache
--only LIST Comma-separated targets to build: core,master,metric,web,alert,sys,all
--only LIST Comma-separated targets to build: core,master,metric,web,alert,sys,gpu_bundle,cpu_bundle,server_pkg,client_pkg,all
--version DATE Date tag used by gpu_bundle/server_pkg/client_pkg (e.g. 20251112)
--client-semver X.Y.Z Override client semver used in all-in-one-full artifact (optional)
--cuda VER CUDA runtime version for NVIDIA base (default: 12.2.2)
--tag-latest Also tag bundle image as :latest (for cpu_bundle only; default off)
-h, --help Show this help message
Examples:
@@ -32,8 +36,18 @@ build_metric=true
build_web=true
build_alert=true
build_sys=true
build_gpu_bundle=false
build_cpu_bundle=false
build_server_pkg=false
build_client_pkg=false
no_cache=false
bundle_date=""
client_semver=""
cuda_ver="12.2.2"
DEFAULT_IMAGE_TAG="latest"
tag_latest=false
while [[ $# -gt 0 ]]; do
case $1 in
--intranet)
@@ -63,7 +77,7 @@ while [[ $# -gt 0 ]]; do
fi
sel="$2"; shift 2
# reset all, then enable selected
build_core=false; build_master=false; build_metric=false; build_web=false; build_alert=false; build_sys=false
build_core=false; build_master=false; build_metric=false; build_web=false; build_alert=false; build_sys=false; build_gpu_bundle=false; build_cpu_bundle=false; build_server_pkg=false; build_client_pkg=false
IFS=',' read -ra parts <<< "$sel"
for p in "${parts[@]}"; do
case "$p" in
@@ -73,11 +87,31 @@ while [[ $# -gt 0 ]]; do
web) build_web=true ;;
alert) build_alert=true ;;
sys) build_sys=true ;;
gpu_bundle) build_gpu_bundle=true ;;
cpu_bundle) build_cpu_bundle=true ;;
server_pkg) build_server_pkg=true; build_core=true; build_master=true; build_metric=true; build_web=true; build_alert=true ;;
client_pkg) build_client_pkg=true ;;
all) build_core=true; build_master=true; build_metric=true; build_web=true; build_alert=true; build_sys=true ;;
*) echo "Unknown --only target: $p" >&2; exit 1 ;;
esac
done
;;
--version)
if [[ -z ${2:-} ]]; then echo "--version requires a value like 20251112" >&2; exit 1; fi
bundle_date="$2"; shift 2
;;
--client-semver)
if [[ -z ${2:-} ]]; then echo "--client-semver requires a value like 1.43.0" >&2; exit 1; fi
client_semver="$2"; shift 2
;;
--cuda)
if [[ -z ${2:-} ]]; then echo "--cuda requires a value like 12.2.2" >&2; exit 1; fi
cuda_ver="$2"; shift 2
;;
--tag-latest)
tag_latest=true
shift
;;
-h|--help)
show_help
exit 0
@@ -101,6 +135,16 @@ fi
cd "$root"
# Set default image tag policy before building
if [[ "$build_server_pkg" == true ]]; then
DEFAULT_IMAGE_TAG="${bundle_date:-latest}"
fi
# Select build user profile for pkg vs default
if [[ "$build_server_pkg" == true || "$build_client_pkg" == true ]]; then
export ARGUS_BUILD_PROFILE=pkg
fi
load_build_user
build_args+=("--build-arg" "ARGUS_BUILD_UID=${ARGUS_BUILD_UID}" "--build-arg" "ARGUS_BUILD_GID=${ARGUS_BUILD_GID}")
@@ -163,13 +207,31 @@ build_image() {
echo " Tag: $tag"
echo " Context: $context"
if docker build "${build_args[@]}" "${extra_args[@]}" -f "$dockerfile_path" -t "$tag" "$context"; then
echo "✅ $image_name image built successfully"
return 0
else
echo "❌ Failed to build $image_name image"
return 1
fi
local tries=${ARGUS_BUILD_RETRIES:-3}
local delay=${ARGUS_BUILD_RETRY_DELAY:-5}
local attempt=1
while (( attempt <= tries )); do
local prefix=""
if (( attempt == tries )); then
# final attempt: disable BuildKit to avoid docker/dockerfile front-end pulls
prefix="DOCKER_BUILDKIT=0"
echo " Attempt ${attempt}/${tries} (fallback: DOCKER_BUILDKIT=0)"
else
echo " Attempt ${attempt}/${tries}"
fi
if eval $prefix docker build "${build_args[@]}" "${extra_args[@]}" -f "$dockerfile_path" -t "$tag" "$context"; then
echo "✅ $image_name image built successfully"
return 0
fi
echo "⚠️ Build failed for $image_name (attempt ${attempt}/${tries})."
if (( attempt < tries )); then
echo " Retrying in ${delay}s..."
sleep "$delay"
fi
attempt=$((attempt+1))
done
echo "❌ Failed to build $image_name image after ${tries} attempts"
return 1
}
pull_base_image() {
@@ -203,25 +265,381 @@ pull_base_image() {
images_built=()
build_failed=false
build_gpu_bundle_image() {
local date_tag="$1" # e.g. 20251112
local cuda_ver_local="$2" # e.g. 12.2.2
local client_ver="$3" # semver like 1.43.0
if [[ -z "$date_tag" ]]; then
echo "❌ gpu_bundle requires --version YYMMDD (e.g. 20251112)" >&2
return 1
fi
# sanitize cuda version (trim trailing dots like '12.2.')
while [[ "$cuda_ver_local" == *"." ]]; do cuda_ver_local="${cuda_ver_local%.}"; done
# Resolve effective CUDA base tag
local resolve_cuda_base_tag
resolve_cuda_base_tag() {
local want="$1" # can be 12, 12.2 or 12.2.2
local major minor patch
if [[ "$want" =~ ^([0-9]+)\.([0-9]+)\.([0-9]+)$ ]]; then
major="${BASH_REMATCH[1]}"; minor="${BASH_REMATCH[2]}"; patch="${BASH_REMATCH[3]}"
echo "nvidia/cuda:${major}.${minor}.${patch}-runtime-ubuntu22.04"; return 0
elif [[ "$want" =~ ^([0-9]+)\.([0-9]+)$ ]]; then
major="${BASH_REMATCH[1]}"; minor="${BASH_REMATCH[2]}"
# try to find best local patch for major.minor
local best
best=$(docker images --format '{{.Repository}}:{{.Tag}}' nvidia/cuda 2>/dev/null | \
grep -E "^nvidia/cuda:${major}\.${minor}\\.[0-9]+-runtime-ubuntu22\.04$" | \
sed -E 's#^nvidia/cuda:([0-9]+\.[0-9]+\.)([0-9]+)-runtime-ubuntu22\.04$#\1\2#g' | \
sort -V | tail -n1 || true)
if [[ -n "$best" ]]; then
echo "nvidia/cuda:${best}-runtime-ubuntu22.04"; return 0
fi
# fallback patch if none local
echo "nvidia/cuda:${major}.${minor}.2-runtime-ubuntu22.04"; return 0
elif [[ "$want" =~ ^([0-9]+)$ ]]; then
major="${BASH_REMATCH[1]}"
# try to find best local for this major
local best
best=$(docker images --format '{{.Repository}}:{{.Tag}}' nvidia/cuda 2>/dev/null | \
grep -E "^nvidia/cuda:${major}\\.[0-9]+\\.[0-9]+-runtime-ubuntu22\.04$" | \
sed -E 's#^nvidia/cuda:([0-9]+\.[0-9]+\.[0-9]+)-runtime-ubuntu22\.04$#\1#g' | \
sort -V | tail -n1 || true)
if [[ -n "$best" ]]; then
echo "nvidia/cuda:${best}-runtime-ubuntu22.04"; return 0
fi
echo "nvidia/cuda:${major}.2.2-runtime-ubuntu22.04"; return 0
else
# invalid format, fallback to default
echo "nvidia/cuda:12.2.2-runtime-ubuntu22.04"; return 0
fi
}
local base_image
base_image=$(resolve_cuda_base_tag "$cuda_ver_local")
echo
echo "🔧 Preparing one-click GPU bundle build"
echo " CUDA runtime base: ${base_image}"
echo " Bundle tag : ${date_tag}"
# 1) Ensure NVIDIA base image (skip pull if local)
if ! pull_base_image "$base_image"; then
# try once more with default if resolution failed
if ! pull_base_image "nvidia/cuda:12.2.2-runtime-ubuntu22.04"; then
return 1
else
base_image="nvidia/cuda:12.2.2-runtime-ubuntu22.04"
fi
fi
# 2) Build latest argus-agent from source
echo "\n🛠 Building argus-agent from src/agent"
pushd "$root/src/agent" >/dev/null
if ! bash scripts/build_binary.sh; then
echo "❌ argus-agent build failed" >&2
popd >/dev/null
return 1
fi
if [[ ! -f "dist/argus-agent" ]]; then
echo "❌ argus-agent binary missing after build" >&2
popd >/dev/null
return 1
fi
popd >/dev/null
# 3) Inject agent into all-in-one-full plugin and package artifact
local aio_root="$root/src/metric/client-plugins/all-in-one-full"
local agent_bin_src="$root/src/agent/dist/argus-agent"
local agent_bin_dst="$aio_root/plugins/argus-agent/bin/argus-agent"
echo "\n📦 Updating all-in-one-full agent binary → $agent_bin_dst"
cp -f "$agent_bin_src" "$agent_bin_dst"
chmod +x "$agent_bin_dst" || true
pushd "$aio_root" >/dev/null
local prev_version
prev_version="$(cat config/VERSION 2>/dev/null || echo "1.0.0")"
local use_version="$prev_version"
if [[ -n "$client_semver" ]]; then
echo "${client_semver}" > config/VERSION
use_version="$client_semver"
fi
echo " Packaging all-in-one-full artifact version: $use_version"
if ! bash scripts/package_artifact.sh --force; then
echo "❌ package_artifact.sh failed" >&2
# restore VERSION if changed
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
return 1
fi
local artifact_dir="$aio_root/artifact/$use_version"
local artifact_tar
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
if [[ -z "$artifact_tar" ]]; then
echo " No argus-metric_*.tar.gz found; invoking publish_artifact.sh to assemble..."
local owner="$(id -u):$(id -g)"
if ! bash scripts/publish_artifact.sh "$use_version" --output-dir "$artifact_dir" --owner "$owner"; then
echo "❌ publish_artifact.sh failed" >&2
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
return 1
fi
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
fi
if [[ -z "$artifact_tar" ]]; then
echo "❌ artifact tar not found under $artifact_dir" >&2
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
return 1
fi
# restore VERSION if changed (keep filesystem clean)
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
# 4) Stage docker build context
local bundle_ctx="$root/src/bundle/gpu-node-bundle/.build-$date_tag"
echo "\n🧰 Staging docker build context: $bundle_ctx"
rm -rf "$bundle_ctx"
mkdir -p "$bundle_ctx/bundle" "$bundle_ctx/private"
cp "$root/src/bundle/gpu-node-bundle/Dockerfile" "$bundle_ctx/"
cp "$root/src/bundle/gpu-node-bundle/node-bootstrap.sh" "$bundle_ctx/"
cp "$root/src/bundle/gpu-node-bundle/health-watcher.sh" "$bundle_ctx/"
# bundle tar
cp "$artifact_tar" "$bundle_ctx/bundle/"
# offline fluent-bit assets (optional but useful)
if [[ -d "$root/src/log/fluent-bit/build/etc" ]]; then
cp -r "$root/src/log/fluent-bit/build/etc" "$bundle_ctx/private/"
fi
if [[ -d "$root/src/log/fluent-bit/build/packages" ]]; then
cp -r "$root/src/log/fluent-bit/build/packages" "$bundle_ctx/private/"
fi
if [[ -f "$root/src/log/fluent-bit/build/start-fluent-bit.sh" ]]; then
cp "$root/src/log/fluent-bit/build/start-fluent-bit.sh" "$bundle_ctx/private/"
fi
# 5) Build the final bundle image (directly from NVIDIA base)
local image_tag="argus-sys-metric-test-node-bundle-gpu:${date_tag}"
echo "\n🔄 Building GPU Bundle image"
if build_image "GPU Bundle" "$bundle_ctx/Dockerfile" "$image_tag" "$bundle_ctx" \
--build-arg CUDA_VER="$(echo "$base_image" | sed -E 's#^nvidia/cuda:([0-9]+\.[0-9]+\.[0-9]+)-runtime-ubuntu22\.04$#\1#')" \
--build-arg CLIENT_VER="$use_version" \
--build-arg BUNDLE_DATE="$date_tag"; then
images_built+=("$image_tag")
# In non-pkg mode, also tag latest for convenience
if [[ "${ARGUS_PKG_BUILD:-0}" != "1" ]]; then
docker tag "$image_tag" argus-sys-metric-test-node-bundle-gpu:latest >/dev/null 2>&1 || true
fi
return 0
else
return 1
fi
}
# Tag helper: ensure :<date_tag> exists for a list of repos
ensure_version_tags() {
local date_tag="$1"; shift
local repos=("$@")
for repo in "${repos[@]}"; do
if docker image inspect "$repo:$date_tag" >/dev/null 2>&1; then
:
elif docker image inspect "$repo:latest" >/dev/null 2>&1; then
docker tag "$repo:latest" "$repo:$date_tag" || true
else
echo "❌ missing image for tagging: $repo (need :latest or :$date_tag)" >&2
return 1
fi
done
return 0
}
# Build server package after images are built
build_server_pkg_bundle() {
local date_tag="$1"
if [[ -z "$date_tag" ]]; then
echo "❌ server_pkg requires --version YYMMDD" >&2
return 1
fi
local repos=(
argus-bind9 argus-master argus-elasticsearch argus-kibana \
argus-metric-ftp argus-metric-prometheus argus-metric-grafana \
argus-alertmanager argus-web-frontend argus-web-proxy
)
echo "\n🔖 Verifying server images with :$date_tag and collecting digests"
for repo in "${repos[@]}"; do
if ! docker image inspect "$repo:$date_tag" >/dev/null 2>&1; then
echo "❌ required image missing: $repo:$date_tag (build phase should have produced it)" >&2
return 1
fi
done
# Optional: show digests
for repo in "${repos[@]}"; do
local digest
digest=$(docker images --digests --format '{{.Repository}}:{{.Tag}} {{.Digest}}' | awk -v r="$repo:$date_tag" '$1==r{print $2}' | head -n1)
printf ' • %s@%s\n' "$repo:$date_tag" "${digest:-<none>}"
done
echo "\n📦 Building server package via deployment_new/build/make_server_package.sh --version $date_tag"
if ! "$root/deployment_new/build/make_server_package.sh" --version "$date_tag"; then
echo "❌ make_server_package.sh failed" >&2
return 1
fi
return 0
}
# Build client package: ensure gpu bundle image exists, then package client_gpu
build_client_pkg_bundle() {
local date_tag="$1"
local semver="$2"
local cuda="$3"
if [[ -z "$date_tag" ]]; then
echo "❌ client_pkg requires --version YYMMDD" >&2
return 1
fi
local bundle_tag="argus-sys-metric-test-node-bundle-gpu:${date_tag}"
if ! docker image inspect "$bundle_tag" >/dev/null 2>&1; then
echo "\n🧩 GPU bundle image $bundle_tag missing; building it first..."
ARGUS_PKG_BUILD=1
export ARGUS_PKG_BUILD
if ! build_gpu_bundle_image "$date_tag" "$cuda" "$semver"; then
return 1
fi
else
echo "\n✅ Using existing GPU bundle image: $bundle_tag"
fi
echo "\n📦 Building client GPU package via deployment_new/build/make_client_gpu_package.sh --version $date_tag --image $bundle_tag"
if ! "$root/deployment_new/build/make_client_gpu_package.sh" --version "$date_tag" --image "$bundle_tag"; then
echo "❌ make_client_gpu_package.sh failed" >&2
return 1
fi
return 0
}
# Build CPU bundle image directly FROM ubuntu:22.04 (no intermediate base)
build_cpu_bundle_image() {
local date_tag="$1" # e.g. 20251113
local client_ver_in="$2" # semver like 1.43.0 (optional)
local want_tag_latest="$3" # true/false
if [[ -z "$date_tag" ]]; then
echo "❌ cpu_bundle requires --version YYMMDD" >&2
return 1
fi
echo "\n🔧 Preparing one-click CPU bundle build"
echo " Base: ubuntu:22.04"
echo " Bundle tag: ${date_tag}"
# 1) Build latest argus-agent from source
echo "\n🛠 Building argus-agent from src/agent"
pushd "$root/src/agent" >/dev/null
if ! bash scripts/build_binary.sh; then
echo "❌ argus-agent build failed" >&2
popd >/dev/null
return 1
fi
if [[ ! -f "dist/argus-agent" ]]; then
echo "❌ argus-agent binary missing after build" >&2
popd >/dev/null
return 1
fi
popd >/dev/null
# 2) Inject agent into all-in-one-full plugin and package artifact
local aio_root="$root/src/metric/client-plugins/all-in-one-full"
local agent_bin_src="$root/src/agent/dist/argus-agent"
local agent_bin_dst="$aio_root/plugins/argus-agent/bin/argus-agent"
echo "\n📦 Updating all-in-one-full agent binary → $agent_bin_dst"
cp -f "$agent_bin_src" "$agent_bin_dst"
chmod +x "$agent_bin_dst" || true
pushd "$aio_root" >/dev/null
local prev_version use_version
prev_version="$(cat config/VERSION 2>/dev/null || echo "1.0.0")"
use_version="$prev_version"
if [[ -n "$client_ver_in" ]]; then
echo "$client_ver_in" > config/VERSION
use_version="$client_ver_in"
fi
echo " Packaging all-in-one-full artifact: version=$use_version"
if ! bash scripts/package_artifact.sh --force; then
echo "❌ package_artifact.sh failed" >&2
[[ -n "$client_ver_in" ]] && echo "$prev_version" > config/VERSION
popd >/dev/null
return 1
fi
local artifact_dir="$aio_root/artifact/$use_version"
local artifact_tar
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
if [[ -z "$artifact_tar" ]]; then
echo " No argus-metric_*.tar.gz found; invoking publish_artifact.sh ..."
local owner="$(id -u):$(id -g)"
if ! bash scripts/publish_artifact.sh "$use_version" --output-dir "$artifact_dir" --owner "$owner"; then
echo "❌ publish_artifact.sh failed" >&2
[[ -n "$client_ver_in" ]] && echo "$prev_version" > config/VERSION
popd >/dev/null
return 1
fi
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
fi
[[ -n "$client_ver_in" ]] && echo "$prev_version" > config/VERSION
popd >/dev/null
# 3) Stage docker build context
local bundle_ctx="$root/src/bundle/cpu-node-bundle/.build-$date_tag"
echo "\n🧰 Staging docker build context: $bundle_ctx"
rm -rf "$bundle_ctx"
mkdir -p "$bundle_ctx/bundle" "$bundle_ctx/private"
cp "$root/src/bundle/cpu-node-bundle/Dockerfile" "$bundle_ctx/"
cp "$root/src/bundle/cpu-node-bundle/node-bootstrap.sh" "$bundle_ctx/"
cp "$root/src/bundle/cpu-node-bundle/health-watcher.sh" "$bundle_ctx/"
# bundle tar
cp "$artifact_tar" "$bundle_ctx/bundle/"
# offline fluent-bit assets
if [[ -d "$root/src/log/fluent-bit/build/etc" ]]; then
cp -r "$root/src/log/fluent-bit/build/etc" "$bundle_ctx/private/"
fi
if [[ -d "$root/src/log/fluent-bit/build/packages" ]]; then
cp -r "$root/src/log/fluent-bit/build/packages" "$bundle_ctx/private/"
fi
if [[ -f "$root/src/log/fluent-bit/build/start-fluent-bit.sh" ]]; then
cp "$root/src/log/fluent-bit/build/start-fluent-bit.sh" "$bundle_ctx/private/"
fi
# 4) Build final bundle image
local image_tag="argus-sys-metric-test-node-bundle:${date_tag}"
echo "\n🔄 Building CPU Bundle image"
if build_image "CPU Bundle" "$bundle_ctx/Dockerfile" "$image_tag" "$bundle_ctx"; then
images_built+=("$image_tag")
if [[ "$want_tag_latest" == "true" ]]; then
docker tag "$image_tag" argus-sys-metric-test-node-bundle:latest >/dev/null 2>&1 || true
fi
return 0
else
return 1
fi
}
if [[ "$build_core" == true ]]; then
if build_image "Elasticsearch" "src/log/elasticsearch/build/Dockerfile" "argus-elasticsearch:latest"; then
images_built+=("argus-elasticsearch:latest")
if build_image "Elasticsearch" "src/log/elasticsearch/build/Dockerfile" "argus-elasticsearch:${DEFAULT_IMAGE_TAG}"; then
images_built+=("argus-elasticsearch:${DEFAULT_IMAGE_TAG}")
else
build_failed=true
fi
echo ""
if build_image "Kibana" "src/log/kibana/build/Dockerfile" "argus-kibana:latest"; then
images_built+=("argus-kibana:latest")
if build_image "Kibana" "src/log/kibana/build/Dockerfile" "argus-kibana:${DEFAULT_IMAGE_TAG}"; then
images_built+=("argus-kibana:${DEFAULT_IMAGE_TAG}")
else
build_failed=true
fi
echo ""
if build_image "BIND9" "src/bind/build/Dockerfile" "argus-bind9:latest"; then
images_built+=("argus-bind9:latest")
if build_image "BIND9" "src/bind/build/Dockerfile" "argus-bind9:${DEFAULT_IMAGE_TAG}"; then
images_built+=("argus-bind9:${DEFAULT_IMAGE_TAG}")
else
build_failed=true
fi
@@ -233,7 +651,7 @@ if [[ "$build_master" == true ]]; then
echo ""
echo "🔄 Building Master image..."
pushd "$master_root" >/dev/null
master_args=("--tag" "argus-master:latest")
master_args=("--tag" "argus-master:${DEFAULT_IMAGE_TAG}")
if [[ "$use_intranet" == true ]]; then
master_args+=("--intranet")
fi
@@ -247,7 +665,7 @@ if [[ "$build_master" == true ]]; then
if [[ "$build_master_offline" == true ]]; then
images_built+=("argus-master:offline")
else
images_built+=("argus-master:latest")
images_built+=("argus-master:${DEFAULT_IMAGE_TAG}")
fi
else
build_failed=true
@@ -272,9 +690,9 @@ if [[ "$build_metric" == true ]]; then
done
metric_builds=(
"Metric FTP|src/metric/ftp/build/Dockerfile|argus-metric-ftp:latest|src/metric/ftp/build"
"Metric Prometheus|src/metric/prometheus/build/Dockerfile|argus-metric-prometheus:latest|src/metric/prometheus/build"
"Metric Grafana|src/metric/grafana/build/Dockerfile|argus-metric-grafana:latest|src/metric/grafana/build"
"Metric FTP|src/metric/ftp/build/Dockerfile|argus-metric-ftp:${DEFAULT_IMAGE_TAG}|src/metric/ftp/build"
"Metric Prometheus|src/metric/prometheus/build/Dockerfile|argus-metric-prometheus:${DEFAULT_IMAGE_TAG}|src/metric/prometheus/build"
"Metric Grafana|src/metric/grafana/build/Dockerfile|argus-metric-grafana:${DEFAULT_IMAGE_TAG}|src/metric/grafana/build"
)
for build_spec in "${metric_builds[@]}"; do
@@ -346,8 +764,8 @@ if [[ "$build_web" == true || "$build_alert" == true ]]; then
if [[ "$build_web" == true ]]; then
web_builds=(
"Web Frontend|src/web/build_tools/frontend/Dockerfile|argus-web-frontend:latest|."
"Web Proxy|src/web/build_tools/proxy/Dockerfile|argus-web-proxy:latest|."
"Web Frontend|src/web/build_tools/frontend/Dockerfile|argus-web-frontend:${DEFAULT_IMAGE_TAG}|."
"Web Proxy|src/web/build_tools/proxy/Dockerfile|argus-web-proxy:${DEFAULT_IMAGE_TAG}|."
)
for build_spec in "${web_builds[@]}"; do
IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec"
@@ -362,7 +780,7 @@ if [[ "$build_web" == true || "$build_alert" == true ]]; then
if [[ "$build_alert" == true ]]; then
alert_builds=(
"Alertmanager|src/alert/alertmanager/build/Dockerfile|argus-alertmanager:latest|."
"Alertmanager|src/alert/alertmanager/build/Dockerfile|argus-alertmanager:${DEFAULT_IMAGE_TAG}|."
)
for build_spec in "${alert_builds[@]}"; do
IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec"
@@ -376,6 +794,49 @@ if [[ "$build_web" == true || "$build_alert" == true ]]; then
fi
fi
# =======================================
# One-click GPU bundle (direct NVIDIA base)
# =======================================
if [[ "$build_gpu_bundle" == true ]]; then
echo ""
echo "Building one-click GPU bundle image..."
if ! build_gpu_bundle_image "$bundle_date" "$cuda_ver" "$client_semver"; then
build_failed=true
fi
fi
# =======================================
# One-click CPU bundle (from ubuntu:22.04)
# =======================================
if [[ "$build_cpu_bundle" == true ]]; then
echo ""
echo "Building one-click CPU bundle image..."
if ! build_cpu_bundle_image "${bundle_date}" "${client_semver}" "${tag_latest}"; then
build_failed=true
fi
fi
# =======================================
# One-click Server/Client packaging
# =======================================
if [[ "$build_server_pkg" == true ]]; then
echo ""
echo "🧳 Building one-click Server package..."
if ! build_server_pkg_bundle "${bundle_date}"; then
build_failed=true
fi
fi
if [[ "$build_client_pkg" == true ]]; then
echo ""
echo "🧳 Building one-click Client-GPU package..."
if ! build_client_pkg_bundle "${bundle_date}" "${client_semver}" "${cuda_ver}"; then
build_failed=true
fi
fi
echo "======================================="
echo "📦 Build Summary"
echo "======================================="

979
build/build_images_for_arm.sh Executable file
View File

@@ -0,0 +1,979 @@
#!/usr/bin/env bash
set -euo pipefail
export ARGUS_TARGET_ARCH="arm64"
ARGUS_BUILDX_BUILDER="${ARGUS_BUILDX_BUILDER:-mybuilder}"
# 自动加载 HTTP/HTTPS 代理配置(仅在变量未预先设置时)
if [[ -z "${HTTP_PROXY:-}" && -z "${http_proxy:-}" ]]; then
if [[ -f /home/yuyr/.source_http_proxy.sh ]]; then
# shellcheck disable=SC1090
source /home/yuyr/.source_http_proxy.sh || true
fi
fi
# 自动准备并切换到指定的 buildx builder(用于 x86_64 上构建 ARM 镜像)
if command -v docker >/dev/null 2>&1; then
if docker buildx ls >/dev/null 2>&1; then
# 若指定的 builder 不存在,则自动创建(带代理环境变量)
if ! docker buildx ls | awk '{print $1}' | grep -qx "${ARGUS_BUILDX_BUILDER}"; then
echo "🔧 Creating buildx builder '${ARGUS_BUILDX_BUILDER}' for ARM builds..."
create_args=(create --name "${ARGUS_BUILDX_BUILDER}" --driver docker-container)
if [[ -n "${HTTP_PROXY:-}" ]]; then
create_args+=(--driver-opt "env.HTTP_PROXY=${HTTP_PROXY}" --driver-opt "env.http_proxy=${HTTP_PROXY}")
fi
if [[ -n "${HTTPS_PROXY:-}" ]]; then
create_args+=(--driver-opt "env.HTTPS_PROXY=${HTTPS_PROXY}" --driver-opt "env.https_proxy=${HTTPS_PROXY}")
fi
if [[ -n "${NO_PROXY:-}" ]]; then
create_args+=(--driver-opt "env.NO_PROXY=${NO_PROXY}" --driver-opt "env.no_proxy=${NO_PROXY}")
fi
docker buildx "${create_args[@]}" --bootstrap >/dev/null 2>&1 || true
fi
docker buildx use "${ARGUS_BUILDX_BUILDER}" >/dev/null 2>&1 || true
fi
fi
show_help() {
cat <<'EOF'
ARGUS Unified Build System - Image Build Tool
Usage: $0 [OPTIONS]
Options:
--intranet Use intranet mirror for log/bind builds
--master-offline Build master offline image (requires src/master/offline_wheels.tar.gz)
--metric Build metric module images (ftp, prometheus, grafana, test nodes)
--no-cache Build all images without using Docker layer cache
--only LIST Comma-separated targets to build: core,master,metric,web,alert,sys,gpu_bundle,cpu_bundle,server_pkg,client_pkg,arm_server_pkg,arm_client_pkg,all
--version DATE Date tag used by gpu_bundle/server_pkg/client_pkg/arm_server_pkg/arm_client_pkg (e.g. 20251112)
--client-semver X.Y.Z Override client semver used in all-in-one-full artifact (optional)
--cuda VER CUDA runtime version for NVIDIA base (default: 12.2.2)
--tag-latest Also tag bundle image as :latest (for cpu_bundle only; default off)
-h, --help Show this help message
Examples:
$0 # Build with default sources
$0 --intranet # Build with intranet mirror
$0 --master-offline # Additionally build argus-master:offline
$0 --metric # Additionally build metric module images
$0 --intranet --master-offline --metric
EOF
}
use_intranet=false
build_core=true
build_master=true
build_master_offline=false
build_metric=true
build_web=true
build_alert=true
build_sys=true
build_gpu_bundle=false
build_cpu_bundle=false
build_server_pkg=false
build_client_pkg=false
build_server_pkg_arm=false
build_client_pkg_arm=false
no_cache=false
bundle_date=""
client_semver=""
cuda_ver="12.2.2"
DEFAULT_IMAGE_TAG="latest"
tag_latest=false
while [[ $# -gt 0 ]]; do
case $1 in
--intranet)
use_intranet=true
shift
;;
--master)
build_master=true
shift
;;
--master-offline)
build_master=true
build_master_offline=true
shift
;;
--metric)
build_metric=true
shift
;;
--no-cache)
no_cache=true
shift
;;
--only)
if [[ -z ${2:-} ]]; then
echo "--only requires a target list" >&2; exit 1
fi
sel="$2"; shift 2
# reset all, then enable selected
build_core=false; build_master=false; build_metric=false; build_web=false; build_alert=false; build_sys=false; build_gpu_bundle=false; build_cpu_bundle=false; build_server_pkg=false; build_client_pkg=false; build_server_pkg_arm=false; build_client_pkg_arm=false
IFS=',' read -ra parts <<< "$sel"
for p in "${parts[@]}"; do
case "$p" in
core) build_core=true ;;
master) build_master=true ;;
metric) build_metric=true ;;
web) build_web=true ;;
alert) build_alert=true ;;
sys) build_sys=true ;;
gpu_bundle) build_gpu_bundle=true ;;
cpu_bundle) build_cpu_bundle=true ;;
# 在 ARM 构建脚本中,server_pkg 视作 arm_server_pkg 的别名,避免误用 x86 打包逻辑
server_pkg) build_server_pkg_arm=true; build_core=true; build_master=true; build_metric=true; build_web=true; build_alert=true ;;
# 在 ARM 构建脚本中,client_pkg 视作 arm_client_pkg 的别名,避免误用 x86 打包逻辑
client_pkg) build_client_pkg_arm=true; build_sys=true ;;
arm_server_pkg) build_server_pkg_arm=true; build_core=true; build_master=true; build_metric=true; build_web=true; build_alert=true ;;
arm_client_pkg) build_client_pkg_arm=true; build_sys=true ;;
all) build_core=true; build_master=true; build_metric=true; build_web=true; build_alert=true; build_sys=true ;;
*) echo "Unknown --only target: $p" >&2; exit 1 ;;
esac
done
;;
--version)
if [[ -z ${2:-} ]]; then echo "--version requires a value like 20251112" >&2; exit 1; fi
bundle_date="$2"; shift 2
;;
--client-semver)
if [[ -z ${2:-} ]]; then echo "--client-semver requires a value like 1.43.0" >&2; exit 1; fi
client_semver="$2"; shift 2
;;
--cuda)
if [[ -z ${2:-} ]]; then echo "--cuda requires a value like 12.2.2" >&2; exit 1; fi
cuda_ver="$2"; shift 2
;;
--tag-latest)
tag_latest=true
shift
;;
-h|--help)
show_help
exit 0
;;
*)
echo "Unknown option: $1" >&2
show_help
exit 1
;;
esac
done
root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
. "$root/scripts/common/build_user.sh"
declare -a build_args=()
if [[ "$use_intranet" == true ]]; then
build_args+=("--build-arg" "USE_INTRANET=true")
fi
cd "$root"
# Set default image tag policy before building
if [[ "$build_server_pkg" == true || "$build_client_pkg" == true || "$build_server_pkg_arm" == true || "$build_client_pkg_arm" == true ]]; then
DEFAULT_IMAGE_TAG="${bundle_date:-latest}"
fi
# Select build user profile for pkg vs default
if [[ "$build_server_pkg" == true || "$build_client_pkg" == true || "$build_server_pkg_arm" == true || "$build_client_pkg_arm" == true ]]; then
export ARGUS_BUILD_PROFILE=pkg
fi
load_build_user
build_args+=("--build-arg" "ARGUS_BUILD_UID=${ARGUS_BUILD_UID}" "--build-arg" "ARGUS_BUILD_GID=${ARGUS_BUILD_GID}")
if [[ "$no_cache" == true ]]; then
build_args+=("--no-cache")
fi
master_root="$root/src/master"
master_offline_tar="$master_root/offline_wheels.tar.gz"
master_offline_dir="$master_root/offline_wheels"
if [[ "$build_master_offline" == true ]]; then
if [[ ! -f "$master_offline_tar" ]]; then
echo "❌ offline wheels tar not found: $master_offline_tar" >&2
echo " 请提前准备好 offline_wheels.tar.gz 后再执行 --master-offline" >&2
exit 1
fi
echo "📦 Preparing offline wheels for master (extracting $master_offline_tar)"
rm -rf "$master_offline_dir"
mkdir -p "$master_offline_dir"
tar -xzf "$master_offline_tar" -C "$master_root"
has_wheel=$(find "$master_offline_dir" -maxdepth 1 -type f -name '*.whl' -print -quit)
if [[ -z "$has_wheel" ]]; then
echo "❌ offline_wheels extraction failed或无 wheel: $master_offline_dir" >&2
exit 1
fi
# ARM 构建下,offline 模式仍通过 Dockerfile 中的 USE_OFFLINE/USE_INTRANET 参数控制
build_args+=("--build-arg" "USE_OFFLINE=1" "--build-arg" "USE_INTRANET=true")
fi
echo "======================================="
echo "ARGUS Unified Build System"
echo "======================================="
if [[ "$use_intranet" == true ]]; then
echo "🌐 Mode: Intranet (Using internal mirror: 10.68.64.1)"
else
echo "🌐 Mode: Public (Using default package sources)"
fi
echo "👤 Build user UID:GID -> ${ARGUS_BUILD_UID}:${ARGUS_BUILD_GID}"
echo "📁 Build context: $root"
echo ""
build_image() {
local image_name=$1
local dockerfile_path=$2
local tag=$3
local context="."
shift 3
if [[ $# -gt 0 ]]; then
context=$1
shift
fi
local extra_args=("$@")
# ARM 专用:如果存在带 .arm64 后缀的 Dockerfile,则优先使用
local dockerfile_for_arch="$dockerfile_path"
if [[ -n "${ARGUS_TARGET_ARCH:-}" && "$ARGUS_TARGET_ARCH" == "arm64" ]]; then
if [[ -f "${dockerfile_path}.arm64" ]]; then
dockerfile_for_arch="${dockerfile_path}.arm64"
fi
fi
echo "🔄 Building $image_name image..."
echo " Dockerfile: $dockerfile_for_arch"
echo " Tag: $tag"
echo " Context: $context"
local tries=${ARGUS_BUILD_RETRIES:-3}
local delay=${ARGUS_BUILD_RETRY_DELAY:-5}
local attempt=1
# 在非 ARM 主机上构建 ARM 镜像时,使用 buildx+--platform=linux/arm64
local use_buildx=false
if [[ "${ARGUS_TARGET_ARCH:-}" == "arm64" && "$(uname -m)" != "aarch64" ]]; then
use_buildx=true
fi
while (( attempt <= tries )); do
echo " Attempt ${attempt}/${tries}"
if [[ "$use_buildx" == true ]]; then
# 通过 buildx 在 x86_64 等非 ARM 主机上构建 ARM64 镜像
if docker buildx build \
--builder "${ARGUS_BUILDX_BUILDER}" \
--platform=linux/arm64 \
"${build_args[@]}" "${extra_args[@]}" \
-f "$dockerfile_for_arch" \
-t "$tag" \
"$context" \
--load; then
echo "✅ $image_name image built successfully (via buildx, platform=linux/arm64)"
return 0
fi
else
# 在 ARM 主机上直接使用 docker build(保留原有 DOCKER_BUILDKIT 回退行为)
local prefix=""
if (( attempt == tries )); then
prefix="DOCKER_BUILDKIT=0"
echo " (final attempt with DOCKER_BUILDKIT=0)"
fi
if eval $prefix docker build "${build_args[@]}" "${extra_args[@]}" -f "$dockerfile_for_arch" -t "$tag" "$context"; then
echo "✅ $image_name image built successfully"
return 0
fi
fi
echo "⚠️ Build failed for $image_name (attempt ${attempt}/${tries})."
if (( attempt < tries )); then
echo " Retrying in ${delay}s..."
sleep "$delay"
fi
attempt=$((attempt+1))
done
echo "❌ Failed to build $image_name image after ${tries} attempts"
return 1
}
pull_base_image() {
local image_ref=$1
local attempts=${2:-3}
local delay=${3:-5}
# If the image already exists locally, skip pulling.
if docker image inspect "$image_ref" >/dev/null 2>&1; then
echo " Local image present; skip pull: $image_ref"
return 0
fi
for ((i=1; i<=attempts; i++)); do
echo " Pulling base image ($i/$attempts): $image_ref"
if docker pull "$image_ref" >/dev/null; then
echo " Base image ready: $image_ref"
return 0
fi
echo " Pull failed: $image_ref"
if (( i < attempts )); then
echo " Retrying in ${delay}s..."
sleep "$delay"
fi
done
echo "❌ Unable to pull base image after ${attempts} attempts: $image_ref"
return 1
}
images_built=()
build_failed=false
build_gpu_bundle_image() {
local date_tag="$1" # e.g. 20251112
local cuda_ver_local="$2" # e.g. 12.2.2
local client_ver="$3" # semver like 1.43.0
if [[ -z "$date_tag" ]]; then
echo "❌ gpu_bundle requires --version YYMMDD (e.g. 20251112)" >&2
return 1
fi
# sanitize cuda version (trim trailing dots like '12.2.')
while [[ "$cuda_ver_local" == *"." ]]; do cuda_ver_local="${cuda_ver_local%.}"; done
# Resolve effective CUDA base tag
local resolve_cuda_base_tag
resolve_cuda_base_tag() {
local want="$1" # can be 12, 12.2 or 12.2.2
local major minor patch
if [[ "$want" =~ ^([0-9]+)\.([0-9]+)\.([0-9]+)$ ]]; then
major="${BASH_REMATCH[1]}"; minor="${BASH_REMATCH[2]}"; patch="${BASH_REMATCH[3]}"
echo "nvidia/cuda:${major}.${minor}.${patch}-runtime-ubuntu22.04"; return 0
elif [[ "$want" =~ ^([0-9]+)\.([0-9]+)$ ]]; then
major="${BASH_REMATCH[1]}"; minor="${BASH_REMATCH[2]}"
# try to find best local patch for major.minor
local best
best=$(docker images --format '{{.Repository}}:{{.Tag}}' nvidia/cuda 2>/dev/null | \
grep -E "^nvidia/cuda:${major}\.${minor}\\.[0-9]+-runtime-ubuntu22\.04$" | \
sed -E 's#^nvidia/cuda:([0-9]+\.[0-9]+\.)([0-9]+)-runtime-ubuntu22\.04$#\1\2#g' | \
sort -V | tail -n1 || true)
if [[ -n "$best" ]]; then
echo "nvidia/cuda:${best}-runtime-ubuntu22.04"; return 0
fi
# fallback patch if none local
echo "nvidia/cuda:${major}.${minor}.2-runtime-ubuntu22.04"; return 0
elif [[ "$want" =~ ^([0-9]+)$ ]]; then
major="${BASH_REMATCH[1]}"
# try to find best local for this major
local best
best=$(docker images --format '{{.Repository}}:{{.Tag}}' nvidia/cuda 2>/dev/null | \
grep -E "^nvidia/cuda:${major}\\.[0-9]+\\.[0-9]+-runtime-ubuntu22\.04$" | \
sed -E 's#^nvidia/cuda:([0-9]+\.[0-9]+\.[0-9]+)-runtime-ubuntu22\.04$#\1#g' | \
sort -V | tail -n1 || true)
if [[ -n "$best" ]]; then
echo "nvidia/cuda:${best}-runtime-ubuntu22.04"; return 0
fi
echo "nvidia/cuda:${major}.2.2-runtime-ubuntu22.04"; return 0
else
# invalid format, fallback to default
echo "nvidia/cuda:12.2.2-runtime-ubuntu22.04"; return 0
fi
}
local base_image
base_image=$(resolve_cuda_base_tag "$cuda_ver_local")
echo
echo "🔧 Preparing one-click GPU bundle build"
echo " CUDA runtime base: ${base_image}"
echo " Bundle tag : ${date_tag}"
# 1) Ensure NVIDIA base image (skip pull if local)
if ! pull_base_image "$base_image"; then
# try once more with default if resolution failed
if ! pull_base_image "nvidia/cuda:12.2.2-runtime-ubuntu22.04"; then
return 1
else
base_image="nvidia/cuda:12.2.2-runtime-ubuntu22.04"
fi
fi
# 2) Build latest argus-agent from source
echo "\n🛠 Building argus-agent from src/agent"
pushd "$root/src/agent" >/dev/null
if ! bash scripts/build_binary.sh; then
echo "❌ argus-agent build failed" >&2
popd >/dev/null
return 1
fi
if [[ ! -f "dist/argus-agent" ]]; then
echo "❌ argus-agent binary missing after build" >&2
popd >/dev/null
return 1
fi
popd >/dev/null
# 3) Inject agent into all-in-one-full plugin and package artifact
local aio_root="$root/src/metric/client-plugins/all-in-one-full"
local agent_bin_src="$root/src/agent/dist/argus-agent"
local agent_bin_dst="$aio_root/plugins/argus-agent/bin/argus-agent"
echo "\n📦 Updating all-in-one-full agent binary → $agent_bin_dst"
cp -f "$agent_bin_src" "$agent_bin_dst"
chmod +x "$agent_bin_dst" || true
pushd "$aio_root" >/dev/null
local prev_version
prev_version="$(cat config/VERSION 2>/dev/null || echo "1.0.0")"
local use_version="$prev_version"
if [[ -n "$client_semver" ]]; then
echo "${client_semver}" > config/VERSION
use_version="$client_semver"
fi
echo " Packaging all-in-one-full artifact version: $use_version"
if ! bash scripts/package_artifact.sh --force; then
echo "❌ package_artifact.sh failed" >&2
# restore VERSION if changed
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
return 1
fi
local artifact_dir="$aio_root/artifact/$use_version"
local artifact_tar
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
if [[ -z "$artifact_tar" ]]; then
echo " No argus-metric_*.tar.gz found; invoking publish_artifact.sh to assemble..."
local owner="$(id -u):$(id -g)"
if ! bash scripts/publish_artifact.sh "$use_version" --output-dir "$artifact_dir" --owner "$owner"; then
echo "❌ publish_artifact.sh failed" >&2
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
return 1
fi
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
fi
if [[ -z "$artifact_tar" ]]; then
echo "❌ artifact tar not found under $artifact_dir" >&2
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
return 1
fi
# restore VERSION if changed (keep filesystem clean)
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
# 4) Stage docker build context
local bundle_ctx="$root/src/bundle/gpu-node-bundle/.build-$date_tag"
echo "\n🧰 Staging docker build context: $bundle_ctx"
rm -rf "$bundle_ctx"
mkdir -p "$bundle_ctx/bundle" "$bundle_ctx/private"
cp "$root/src/bundle/gpu-node-bundle/Dockerfile" "$bundle_ctx/"
cp "$root/src/bundle/gpu-node-bundle/node-bootstrap.sh" "$bundle_ctx/"
cp "$root/src/bundle/gpu-node-bundle/health-watcher.sh" "$bundle_ctx/"
# bundle tar
cp "$artifact_tar" "$bundle_ctx/bundle/"
# offline fluent-bit assets (optional but useful)
if [[ -d "$root/src/log/fluent-bit/build/etc" ]]; then
cp -r "$root/src/log/fluent-bit/build/etc" "$bundle_ctx/private/"
fi
if [[ -d "$root/src/log/fluent-bit/build/packages" ]]; then
cp -r "$root/src/log/fluent-bit/build/packages" "$bundle_ctx/private/"
fi
if [[ -f "$root/src/log/fluent-bit/build/start-fluent-bit.sh" ]]; then
cp "$root/src/log/fluent-bit/build/start-fluent-bit.sh" "$bundle_ctx/private/"
fi
# 5) Build the final bundle image (directly from NVIDIA base)
local image_tag="argus-sys-metric-test-node-bundle-gpu-arm64:${date_tag}"
echo "\n🔄 Building GPU Bundle image"
if build_image "GPU Bundle" "$bundle_ctx/Dockerfile" "$image_tag" "$bundle_ctx" \
--build-arg CUDA_VER="$(echo "$base_image" | sed -E 's#^nvidia/cuda:([0-9]+\.[0-9]+\.[0-9]+)-runtime-ubuntu22\.04$#\1#')" \
--build-arg CLIENT_VER="$use_version" \
--build-arg BUNDLE_DATE="$date_tag"; then
images_built+=("$image_tag")
# In non-pkg mode, also tag latest for convenience
if [[ "${ARGUS_PKG_BUILD:-0}" != "1" ]]; then
docker tag "$image_tag" argus-sys-metric-test-node-bundle-gpu-arm64:latest >/dev/null 2>&1 || true
fi
return 0
else
return 1
fi
}
# Tag helper: ensure :<date_tag> exists for a list of repos
ensure_version_tags() {
local date_tag="$1"; shift
local repos=("$@")
for repo in "${repos[@]}"; do
if docker image inspect "$repo:$date_tag" >/dev/null 2>&1; then
:
elif docker image inspect "$repo:latest" >/dev/null 2>&1; then
docker tag "$repo:latest" "$repo:$date_tag" || true
else
echo "❌ missing image for tagging: $repo (need :latest or :$date_tag)" >&2
return 1
fi
done
return 0
}
# Build server package after images are built
build_server_pkg_bundle() {
local date_tag="$1"
if [[ -z "$date_tag" ]]; then
echo "❌ server_pkg requires --version YYMMDD" >&2
return 1
fi
local repos=(
argus-bind9-arm64 argus-master-arm64 argus-elasticsearch-arm64 argus-kibana-arm64 \
argus-metric-ftp-arm64 argus-metric-prometheus-arm64 argus-metric-grafana-arm64 \
argus-alertmanager-arm64 argus-web-frontend-arm64 argus-web-proxy-arm64
)
echo "\n🔖 Verifying server images with :$date_tag and collecting digests"
for repo in "${repos[@]}"; do
if ! docker image inspect "$repo:$date_tag" >/dev/null 2>&1; then
echo "❌ required image missing: $repo:$date_tag (build phase should have produced it)" >&2
return 1
fi
done
# Optional: show digests
for repo in "${repos[@]}"; do
local digest
digest=$(docker images --digests --format '{{.Repository}}:{{.Tag}} {{.Digest}}' | awk -v r="$repo:$date_tag" '$1==r{print $2}' | head -n1)
printf ' • %s@%s\n' "$repo:$date_tag" "${digest:-<none>}"
done
echo "\n📦 Building server package via deployment_new/build/make_server_package.sh --version $date_tag"
if ! "$root/deployment_new/build/make_server_package.sh" --version "$date_tag"; then
echo "❌ make_server_package.sh failed" >&2
return 1
fi
return 0
}
# Build client package: ensure gpu bundle image exists, then package client_gpu
build_client_pkg_bundle() {
local date_tag="$1"
local semver="$2"
local cuda="$3"
if [[ -z "$date_tag" ]]; then
echo "❌ client_pkg requires --version YYMMDD" >&2
return 1
fi
local bundle_tag="argus-sys-metric-test-node-bundle-gpu-arm64:${date_tag}"
if ! docker image inspect "$bundle_tag" >/dev/null 2>&1; then
echo "\n🧩 GPU bundle image $bundle_tag missing; building it first..."
ARGUS_PKG_BUILD=1
export ARGUS_PKG_BUILD
if ! build_gpu_bundle_image "$date_tag" "$cuda" "$semver"; then
return 1
fi
else
echo "\n✅ Using existing GPU bundle image: $bundle_tag"
fi
echo "\n📦 Building client GPU package via deployment_new/build/make_client_gpu_package.sh --version $date_tag --image $bundle_tag"
if ! "$root/deployment_new/build/make_client_gpu_package.sh" --version "$date_tag" --image "$bundle_tag"; then
echo "❌ make_client_gpu_package.sh failed" >&2
return 1
fi
return 0
}
# Build CPU bundle image directly FROM ubuntu:22.04 (no intermediate base)
build_cpu_bundle_image() {
local date_tag="$1" # e.g. 20251113
local client_ver_in="$2" # semver like 1.43.0 (optional)
local want_tag_latest="$3" # true/false
if [[ -z "$date_tag" ]]; then
echo "❌ cpu_bundle requires --version YYMMDD" >&2
return 1
fi
echo "\n🔧 Preparing one-click CPU bundle build"
echo " Base: ubuntu:22.04"
echo " Bundle tag: ${date_tag}"
# 1) Build latest argus-agent from source
echo "\n🛠 Building argus-agent from src/agent"
pushd "$root/src/agent" >/dev/null
if ! bash scripts/build_binary.sh; then
echo "❌ argus-agent build failed" >&2
popd >/dev/null
return 1
fi
if [[ ! -f "dist/argus-agent" ]]; then
echo "❌ argus-agent binary missing after build" >&2
popd >/dev/null
return 1
fi
popd >/dev/null
# 2) Inject agent into all-in-one-full plugin and package artifact
local aio_root="$root/src/metric/client-plugins/all-in-one-full"
local agent_bin_src="$root/src/agent/dist/argus-agent"
local agent_bin_dst="$aio_root/plugins/argus-agent/bin/argus-agent"
echo "\n📦 Updating all-in-one-full agent binary → $agent_bin_dst"
cp -f "$agent_bin_src" "$agent_bin_dst"
chmod +x "$agent_bin_dst" || true
pushd "$aio_root" >/dev/null
local prev_version use_version
prev_version="$(cat config/VERSION 2>/dev/null || echo "1.0.0")"
use_version="$prev_version"
if [[ -n "$client_ver_in" ]]; then
echo "$client_ver_in" > config/VERSION
use_version="$client_ver_in"
fi
echo " Packaging all-in-one-full artifact: version=$use_version"
if ! bash scripts/package_artifact.sh --force; then
echo "❌ package_artifact.sh failed" >&2
[[ -n "$client_ver_in" ]] && echo "$prev_version" > config/VERSION
popd >/dev/null
return 1
fi
local artifact_dir="$aio_root/artifact/$use_version"
local artifact_tar
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
if [[ -z "$artifact_tar" ]]; then
echo " No argus-metric_*.tar.gz found; invoking publish_artifact.sh ..."
local owner="$(id -u):$(id -g)"
if ! bash scripts/publish_artifact.sh "$use_version" --output-dir "$artifact_dir" --owner "$owner"; then
echo "❌ publish_artifact.sh failed" >&2
[[ -n "$client_ver_in" ]] && echo "$prev_version" > config/VERSION
popd >/dev/null
return 1
fi
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
fi
[[ -n "$client_ver_in" ]] && echo "$prev_version" > config/VERSION
popd >/dev/null
# 3) Stage docker build context
local bundle_ctx="$root/src/bundle/cpu-node-bundle/.build-$date_tag"
echo "\n🧰 Staging docker build context: $bundle_ctx"
rm -rf "$bundle_ctx"
mkdir -p "$bundle_ctx/bundle" "$bundle_ctx/private"
cp "$root/src/bundle/cpu-node-bundle/Dockerfile" "$bundle_ctx/"
cp "$root/src/bundle/cpu-node-bundle/node-bootstrap.sh" "$bundle_ctx/"
cp "$root/src/bundle/cpu-node-bundle/health-watcher.sh" "$bundle_ctx/"
# bundle tar
cp "$artifact_tar" "$bundle_ctx/bundle/"
# offline fluent-bit assets
if [[ -d "$root/src/log/fluent-bit/build/etc" ]]; then
cp -r "$root/src/log/fluent-bit/build/etc" "$bundle_ctx/private/"
fi
if [[ -d "$root/src/log/fluent-bit/build/packages" ]]; then
cp -r "$root/src/log/fluent-bit/build/packages" "$bundle_ctx/private/"
fi
if [[ -f "$root/src/log/fluent-bit/build/start-fluent-bit.sh" ]]; then
cp "$root/src/log/fluent-bit/build/start-fluent-bit.sh" "$bundle_ctx/private/"
fi
# 4) Build final bundle image
local image_tag="argus-sys-metric-test-node-bundle-arm64:${date_tag}"
echo "\n🔄 Building CPU Bundle image"
if build_image "CPU Bundle" "$bundle_ctx/Dockerfile" "$image_tag" "$bundle_ctx"; then
images_built+=("$image_tag")
# 为兼容现有 compose/部署,额外打无后缀别名
docker tag "$image_tag" "argus-sys-metric-test-node-bundle:${date_tag}" >/dev/null 2>&1 || true
if [[ "$want_tag_latest" == "true" ]]; then
docker tag "$image_tag" argus-sys-metric-test-node-bundle-arm64:latest >/dev/null 2>&1 || true
docker tag "$image_tag" argus-sys-metric-test-node-bundle:latest >/dev/null 2>&1 || true
fi
return 0
else
return 1
fi
}
if [[ "$build_core" == true ]]; then
if build_image "Elasticsearch" "src/log/elasticsearch/build/Dockerfile" "argus-elasticsearch-arm64:${DEFAULT_IMAGE_TAG}"; then
images_built+=("argus-elasticsearch-arm64:${DEFAULT_IMAGE_TAG}")
else
build_failed=true
fi
echo ""
if build_image "Kibana" "src/log/kibana/build/Dockerfile" "argus-kibana-arm64:${DEFAULT_IMAGE_TAG}"; then
images_built+=("argus-kibana-arm64:${DEFAULT_IMAGE_TAG}")
else
build_failed=true
fi
echo ""
if build_image "BIND9" "src/bind/build/Dockerfile" "argus-bind9-arm64:${DEFAULT_IMAGE_TAG}"; then
images_built+=("argus-bind9-arm64:${DEFAULT_IMAGE_TAG}")
else
build_failed=true
fi
fi
echo ""
if [[ "$build_master" == true ]]; then
echo ""
echo "🔄 Building Master image..."
# 复用通用 build_image 函数,通过 buildx 构建 ARM64 master 镜像
if build_image "Master" "src/master/Dockerfile" "argus-master-arm64:${DEFAULT_IMAGE_TAG}" "."; then
images_built+=("argus-master-arm64:${DEFAULT_IMAGE_TAG}")
else
build_failed=true
fi
fi
if [[ "$build_metric" == true ]]; then
echo ""
echo "Building Metric module images..."
metric_base_images=(
"ubuntu:22.04"
"prom/prometheus:v3.5.0"
"grafana/grafana:11.1.0"
)
for base_image in "${metric_base_images[@]}"; do
if ! pull_base_image "$base_image"; then
build_failed=true
fi
done
metric_builds=(
"Metric FTP|src/metric/ftp/build/Dockerfile|argus-metric-ftp-arm64:${DEFAULT_IMAGE_TAG}|src/metric/ftp/build"
"Metric Prometheus|src/metric/prometheus/build/Dockerfile|argus-metric-prometheus-arm64:${DEFAULT_IMAGE_TAG}|src/metric/prometheus/build"
"Metric Grafana|src/metric/grafana/build/Dockerfile|argus-metric-grafana-arm64:${DEFAULT_IMAGE_TAG}|src/metric/grafana/build"
"Metric Prometheus Targets Updater|src/metric/prometheus/build/Dockerfile.targets-updater|argus-metric-prometheus-targets-updater-arm64:${DEFAULT_IMAGE_TAG}|src/metric/prometheus/build"
)
for build_spec in "${metric_builds[@]}"; do
IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec"
if build_image "$image_label" "$dockerfile_path" "$image_tag" "$build_context"; then
images_built+=("$image_tag")
else
build_failed=true
fi
echo ""
done
fi
# =======================================
# Sys (system tests) node images
# =======================================
if [[ "$build_sys" == true ]]; then
echo ""
echo "Building Sys node images..."
sys_base_images=(
"ubuntu:22.04"
)
# GPU 相关镜像目前仅在 x86_64 上支持;ARM 上不拉取 nvidia/cuda 基础镜像
if [[ "${ARGUS_TARGET_ARCH:-}" != "arm64" ]]; then
sys_base_images+=("nvidia/cuda:12.2.2-runtime-ubuntu22.04")
fi
for base_image in "${sys_base_images[@]}"; do
if ! pull_base_image "$base_image"; then
build_failed=true
fi
done
sys_builds=(
"Sys Node|src/sys/build/node/Dockerfile|argus-sys-node-arm64:latest|."
"Sys Metric Test Node|src/sys/build/arm-cpu-node/Dockerfile|argus-sys-metric-test-node-arm64:latest|."
)
# GPU 测试节点镜像仅在 x86_64 路径构建,ARM 版本暂不支持 DCGM/GPU
if [[ "${ARGUS_TARGET_ARCH:-}" != "arm64" ]]; then
sys_builds+=("Sys Metric Test GPU Node|src/sys/build/test-gpu-node/Dockerfile|argus-sys-metric-test-gpu-node:latest|.")
fi
for build_spec in "${sys_builds[@]}"; do
IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec"
if build_image "$image_label" "$dockerfile_path" "$image_tag" "$build_context"; then
images_built+=("$image_tag")
# 与历史 NODE_BUNDLE_IMAGE_TAG 保持兼容:为 ARM CPU 节点镜像打 bundle 别名
if [[ "$image_tag" == "argus-sys-metric-test-node-arm64:latest" ]]; then
docker tag "$image_tag" argus-sys-metric-test-node-bundle-arm64:latest >/dev/null 2>&1 || true
docker tag "$image_tag" argus-sys-metric-test-node-bundle:latest >/dev/null 2>&1 || true
fi
else
build_failed=true
fi
echo ""
done
fi
# =======================================
# Web & Alert module images
# =======================================
if [[ "$build_web" == true || "$build_alert" == true ]]; then
echo ""
echo "Building Web and Alert module images..."
# Pre-pull commonly used base images for stability
web_alert_base_images=(
"node:20"
"ubuntu:24.04"
)
for base_image in "${web_alert_base_images[@]}"; do
if ! pull_base_image "$base_image"; then
build_failed=true
fi
done
if [[ "$build_web" == true ]]; then
web_builds=(
"Web Frontend|src/web/build_tools/frontend/Dockerfile|argus-web-frontend-arm64:${DEFAULT_IMAGE_TAG}|."
"Web Proxy|src/web/build_tools/proxy/Dockerfile|argus-web-proxy-arm64:${DEFAULT_IMAGE_TAG}|."
)
for build_spec in "${web_builds[@]}"; do
IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec"
if build_image "$image_label" "$dockerfile_path" "$image_tag" "$build_context"; then
images_built+=("$image_tag")
else
build_failed=true
fi
echo ""
done
fi
if [[ "$build_alert" == true ]]; then
alert_builds=(
"Alertmanager|src/alert/alertmanager/build/Dockerfile|argus-alertmanager-arm64:${DEFAULT_IMAGE_TAG}|."
)
for build_spec in "${alert_builds[@]}"; do
IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec"
if build_image "$image_label" "$dockerfile_path" "$image_tag" "$build_context" --build-arg ALERTMANAGER_ARCH=arm64; then
images_built+=("$image_tag")
else
build_failed=true
fi
echo ""
done
fi
fi
# =======================================
# One-click GPU bundle (direct NVIDIA base)
# =======================================
if [[ "$build_gpu_bundle" == true ]]; then
echo ""
echo "Building one-click GPU bundle image..."
if ! build_gpu_bundle_image "$bundle_date" "$cuda_ver" "$client_semver"; then
build_failed=true
fi
fi
# =======================================
# One-click CPU bundle (from ubuntu:22.04)
# =======================================
if [[ "$build_cpu_bundle" == true ]]; then
echo ""
echo "Building one-click CPU bundle image..."
if ! build_cpu_bundle_image "${bundle_date}" "${client_semver}" "${tag_latest}"; then
build_failed=true
fi
fi
# =======================================
# One-click Server/Client packaging
# =======================================
if [[ "$build_server_pkg" == true ]]; then
echo ""
echo "🧳 Building one-click Server package..."
if ! build_server_pkg_bundle "${bundle_date}"; then
build_failed=true
fi
fi
if [[ "$build_client_pkg" == true ]]; then
echo ""
echo "🧳 Building one-click Client-GPU package..."
if ! build_client_pkg_bundle "${bundle_date}" "${client_semver}" "${cuda_ver}"; then
build_failed=true
fi
fi
# ARM 专用:Server/Client 部署包
if [[ "$build_server_pkg_arm" == true ]]; then
if [[ -z "$bundle_date" ]]; then
echo "❌ arm_server_pkg requires --version YYYYMMDD" >&2
build_failed=true
else
echo ""
echo "🧳 Building ARM Server package..."
if ! "$root/deployment_new/build/make_arm_server_package.sh" --version "$bundle_date"; then
build_failed=true
fi
fi
fi
if [[ "$build_client_pkg_arm" == true ]]; then
if [[ -z "$bundle_date" ]]; then
echo "❌ arm_client_pkg requires --version YYYYMMDD" >&2
build_failed=true
else
echo ""
echo "🧳 Building ARM Client-CPU package..."
cpu_repo="argus-sys-metric-test-node-arm64"
cpu_image_arg="$cpu_repo:latest"
# 若已存在 :<version>,优先使用;否则退回 latest 交由打包脚本打 tag
if docker image inspect "${cpu_repo}:${bundle_date}" >/dev/null 2>&1; then
cpu_image_arg="${cpu_repo}:${bundle_date}"
elif ! docker image inspect "${cpu_repo}:latest" >/dev/null 2>&1; then
echo "❌ required CPU node image missing: ${cpu_repo}:{${bundle_date},latest}" >&2
build_failed=true
fi
if [[ "$build_failed" != "true" ]]; then
if ! "$root/deployment_new/build/make_arm_client_package.sh" --version "$bundle_date" --image "$cpu_image_arg"; then
build_failed=true
fi
fi
fi
fi
echo "======================================="
echo "📦 Build Summary"
echo "======================================="
if [[ ${#images_built[@]} -gt 0 ]]; then
echo "✅ Successfully built images:"
for image in "${images_built[@]}"; do
echo " • $image"
done
fi
if [[ "$build_failed" == true ]]; then
echo ""
echo "❌ Some images failed to build. Please check the errors above."
exit 1
fi
if [[ "$use_intranet" == true ]]; then
echo ""
echo "🌐 Built with intranet mirror configuration"
fi
if [[ "$build_master_offline" == true ]]; then
echo ""
echo "🧳 Master offline wheels 已解压到 $master_offline_dir"
fi
echo ""
echo "🚀 Next steps:"
echo " ./build/save_images.sh --compress # 导出镜像"
echo " cd src/master/tests && MASTER_IMAGE_TAG=argus-master:offline ./scripts/00_e2e_test.sh"
echo ""

875
build/build_images_for_x64.sh Executable file
View File

@@ -0,0 +1,875 @@
#!/usr/bin/env bash
set -euo pipefail
# ARGUS x86_64 Image Build Entry
# 本脚本用于在 x86_64 平台上构建 Argus 镜像,
# 逻辑与历史版本的 build/build_images.sh 保持一致。
show_help() {
cat <<'EOF'
ARGUS Unified Build System - Image Build Tool
Usage: $0 [OPTIONS]
Options:
--intranet Use intranet mirror for log/bind builds
--master-offline Build master offline image (requires src/master/offline_wheels.tar.gz)
--metric Build metric module images (ftp, prometheus, grafana, test nodes)
--no-cache Build all images without using Docker layer cache
--only LIST Comma-separated targets to build: core,master,metric,web,alert,sys,gpu_bundle,cpu_bundle,server_pkg,client_pkg,all
--version DATE Date tag used by gpu_bundle/server_pkg/client_pkg (e.g. 20251112)
--client-semver X.Y.Z Override client semver used in all-in-one-full artifact (optional)
--cuda VER CUDA runtime version for NVIDIA base (default: 12.2.2)
--tag-latest Also tag bundle image as :latest (for cpu_bundle only; default off)
-h, --help Show this help message
Examples:
$0 # Build with default sources
$0 --intranet # Build with intranet mirror
$0 --master-offline # Additionally build argus-master:offline
$0 --metric # Additionally build metric module images
$0 --intranet --master-offline --metric
EOF
}
use_intranet=false
build_core=true
build_master=true
build_master_offline=false
build_metric=true
build_web=true
build_alert=true
build_sys=true
build_gpu_bundle=false
build_cpu_bundle=false
build_server_pkg=false
build_client_pkg=false
no_cache=false
bundle_date=""
client_semver=""
cuda_ver="12.2.2"
DEFAULT_IMAGE_TAG="latest"
tag_latest=false
while [[ $# -gt 0 ]]; do
case $1 in
--intranet)
use_intranet=true
shift
;;
--master)
build_master=true
shift
;;
--master-offline)
build_master=true
build_master_offline=true
shift
;;
--metric)
build_metric=true
shift
;;
--no-cache)
no_cache=true
shift
;;
--only)
if [[ -z ${2:-} ]]; then
echo "--only requires a target list" >&2; exit 1
fi
sel="$2"; shift 2
# reset all, then enable selected
build_core=false; build_master=false; build_metric=false; build_web=false; build_alert=false; build_sys=false; build_gpu_bundle=false; build_cpu_bundle=false; build_server_pkg=false; build_client_pkg=false
IFS=',' read -ra parts <<< "$sel"
for p in "${parts[@]}"; do
case "$p" in
core) build_core=true ;;
master) build_master=true ;;
metric) build_metric=true ;;
web) build_web=true ;;
alert) build_alert=true ;;
sys) build_sys=true ;;
gpu_bundle) build_gpu_bundle=true ;;
cpu_bundle) build_cpu_bundle=true ;;
server_pkg) build_server_pkg=true; build_core=true; build_master=true; build_metric=true; build_web=true; build_alert=true ;;
client_pkg) build_client_pkg=true ;;
all) build_core=true; build_master=true; build_metric=true; build_web=true; build_alert=true; build_sys=true ;;
*) echo "Unknown --only target: $p" >&2; exit 1 ;;
esac
done
;;
--version)
if [[ -z ${2:-} ]]; then echo "--version requires a value like 20251112" >&2; exit 1; fi
bundle_date="$2"; shift 2
;;
--client-semver)
if [[ -z ${2:-} ]]; then echo "--client-semver requires a value like 1.43.0" >&2; exit 1; fi
client_semver="$2"; shift 2
;;
--cuda)
if [[ -z ${2:-} ]]; then echo "--cuda requires a value like 12.2.2" >&2; exit 1; fi
cuda_ver="$2"; shift 2
;;
--tag-latest)
tag_latest=true
shift
;;
-h|--help)
show_help
exit 0
;;
*)
echo "Unknown option: $1" >&2
show_help
exit 1
;;
esac
done
root="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
. "$root/scripts/common/build_user.sh"
declare -a build_args=()
if [[ "$use_intranet" == true ]]; then
build_args+=("--build-arg" "USE_INTRANET=true")
fi
cd "$root"
# Set default image tag policy before building
if [[ "$build_server_pkg" == true ]]; then
DEFAULT_IMAGE_TAG="${bundle_date:-latest}"
fi
# Select build user profile for pkg vs default
if [[ "$build_server_pkg" == true || "$build_client_pkg" == true ]]; then
export ARGUS_BUILD_PROFILE=pkg
fi
load_build_user
build_args+=("--build-arg" "ARGUS_BUILD_UID=${ARGUS_BUILD_UID}" "--build-arg" "ARGUS_BUILD_GID=${ARGUS_BUILD_GID}")
if [[ "$no_cache" == true ]]; then
build_args+=("--no-cache")
fi
master_root="$root/src/master"
master_offline_tar="$master_root/offline_wheels.tar.gz"
master_offline_dir="$master_root/offline_wheels"
if [[ "$build_master_offline" == true ]]; then
if [[ ! -f "$master_offline_tar" ]]; then
echo "❌ offline wheels tar not found: $master_offline_tar" >&2
echo " 请提前准备好 offline_wheels.tar.gz 后再执行 --master-offline" >&2
exit 1
fi
echo "📦 Preparing offline wheels for master (extracting $master_offline_tar)"
rm -rf "$master_offline_dir"
mkdir -p "$master_offline_dir"
tar -xzf "$master_offline_tar" -C "$master_root"
has_wheel=$(find "$master_offline_dir" -maxdepth 1 -type f -name '*.whl' -print -quit)
if [[ -z "$has_wheel" ]]; then
echo "❌ offline_wheels extraction failed或无 wheel: $master_offline_dir" >&2
exit 1
fi
fi
echo "======================================="
echo "ARGUS Unified Build System"
echo "======================================="
if [[ "$use_intranet" == true ]]; then
echo "🌐 Mode: Intranet (Using internal mirror: 10.68.64.1)"
else
echo "🌐 Mode: Public (Using default package sources)"
fi
echo "👤 Build user UID:GID -> ${ARGUS_BUILD_UID}:${ARGUS_BUILD_GID}"
echo "📁 Build context: $root"
echo ""
build_image() {
local image_name=$1
local dockerfile_path=$2
local tag=$3
local context="."
shift 3
if [[ $# -gt 0 ]]; then
context=$1
shift
fi
local extra_args=("$@")
echo "🔄 Building $image_name image..."
echo " Dockerfile: $dockerfile_path"
echo " Tag: $tag"
echo " Context: $context"
local tries=${ARGUS_BUILD_RETRIES:-3}
local delay=${ARGUS_BUILD_RETRY_DELAY:-5}
local attempt=1
while (( attempt <= tries )); do
local prefix=""
if (( attempt == tries )); then
# final attempt: disable BuildKit to avoid docker/dockerfile front-end pulls
prefix="DOCKER_BUILDKIT=0"
echo " Attempt ${attempt}/${tries} (fallback: DOCKER_BUILDKIT=0)"
else
echo " Attempt ${attempt}/${tries}"
fi
if eval $prefix docker build "${build_args[@]}" "${extra_args[@]}" -f "$dockerfile_path" -t "$tag" "$context"; then
echo "✅ $image_name image built successfully"
return 0
fi
echo "⚠️ Build failed for $image_name (attempt ${attempt}/${tries})."
if (( attempt < tries )); then
echo " Retrying in ${delay}s..."
sleep "$delay"
fi
attempt=$((attempt+1))
done
echo "❌ Failed to build $image_name image after ${tries} attempts"
return 1
}
pull_base_image() {
local image_ref=$1
local attempts=${2:-3}
local delay=${3:-5}
# If the image already exists locally, skip pulling.
if docker image inspect "$image_ref" >/dev/null 2>&1; then
echo " Local image present; skip pull: $image_ref"
return 0
fi
for ((i=1; i<=attempts; i++)); do
echo " Pulling base image ($i/$attempts): $image_ref"
if docker pull "$image_ref" >/dev/null; then
echo " Base image ready: $image_ref"
return 0
fi
echo " Pull failed: $image_ref"
if (( i < attempts )); then
echo " Retrying in ${delay}s..."
sleep "$delay"
fi
done
echo "❌ Unable to pull base image after ${attempts} attempts: $image_ref"
return 1
}
images_built=()
build_failed=false
build_gpu_bundle_image() {
local date_tag="$1" # e.g. 20251112
local cuda_ver_local="$2" # e.g. 12.2.2
local client_ver="$3" # semver like 1.43.0
if [[ -z "$date_tag" ]]; then
echo "❌ gpu_bundle requires --version YYMMDD (e.g. 20251112)" >&2
return 1
fi
# sanitize cuda version (trim trailing dots like '12.2.')
while [[ "$cuda_ver_local" == *"." ]]; do cuda_ver_local="${cuda_ver_local%.}"; done
# Resolve effective CUDA base tag
local resolve_cuda_base_tag
resolve_cuda_base_tag() {
local want="$1" # can be 12, 12.2 or 12.2.2
local major minor patch
if [[ "$want" =~ ^([0-9]+)\.([0-9]+)\.([0-9]+)$ ]]; then
major="${BASH_REMATCH[1]}"; minor="${BASH_REMATCH[2]}"; patch="${BASH_REMATCH[3]}"
echo "nvidia/cuda:${major}.${minor}.${patch}-runtime-ubuntu22.04"; return 0
elif [[ "$want" =~ ^([0-9]+)\.([0-9]+)$ ]]; then
major="${BASH_REMATCH[1]}"; minor="${BASH_REMATCH[2]}"
# try to find best local patch for major.minor
local best
best=$(docker images --format '{{.Repository}}:{{.Tag}}' nvidia/cuda 2>/dev/null | \
grep -E "^nvidia/cuda:${major}\.${minor}\\.[0-9]+-runtime-ubuntu22\.04$" | \
sed -E 's#^nvidia/cuda:([0-9]+\.[0-9]+\.)([0-9]+)-runtime-ubuntu22\.04$#\1\2#g' | \
sort -V | tail -n1 || true)
if [[ -n "$best" ]]; then
echo "nvidia/cuda:${best}-runtime-ubuntu22.04"; return 0
fi
# fallback patch if none local
echo "nvidia/cuda:${major}.${minor}.2-runtime-ubuntu22.04"; return 0
elif [[ "$want" =~ ^([0-9]+)$ ]]; then
major="${BASH_REMATCH[1]}"
# try to find best local for this major
local best
best=$(docker images --format '{{.Repository}}:{{.Tag}}' nvidia/cuda 2>/dev/null | \
grep -E "^nvidia/cuda:${major}\\.[0-9]+\\.[0-9]+-runtime-ubuntu22\.04$" | \
sed -E 's#^nvidia/cuda:([0-9]+\.[0-9]+\.[0-9]+)-runtime-ubuntu22\.04$#\1#g' | \
sort -V | tail -n1 || true)
if [[ -n "$best" ]]; then
echo "nvidia/cuda:${best}-runtime-ubuntu22.04"; return 0
fi
echo "nvidia/cuda:${major}.2.2-runtime-ubuntu22.04"; return 0
else
# invalid format, fallback to default
echo "nvidia/cuda:12.2.2-runtime-ubuntu22.04"; return 0
fi
}
local base_image
base_image=$(resolve_cuda_base_tag "$cuda_ver_local")
echo
echo "🔧 Preparing one-click GPU bundle build"
echo " CUDA runtime base: ${base_image}"
echo " Bundle tag : ${date_tag}"
# 1) Ensure NVIDIA base image (skip pull if local)
if ! pull_base_image "$base_image"; then
# try once more with default if resolution failed
if ! pull_base_image "nvidia/cuda:12.2.2-runtime-ubuntu22.04"; then
return 1
else
base_image="nvidia/cuda:12.2.2-runtime-ubuntu22.04"
fi
fi
# 2) Build latest argus-agent from source
echo "\n🛠 Building argus-agent from src/agent"
pushd "$root/src/agent" >/dev/null
if ! bash scripts/build_binary.sh; then
echo "❌ argus-agent build failed" >&2
popd >/dev/null
return 1
fi
if [[ ! -f "dist/argus-agent" ]]; then
echo "❌ argus-agent binary missing after build" >&2
popd >/dev/null
return 1
fi
popd >/dev/null
# 3) Inject agent into all-in-one-full plugin and package artifact
local aio_root="$root/src/metric/client-plugins/all-in-one-full"
local agent_bin_src="$root/src/agent/dist/argus-agent"
local agent_bin_dst="$aio_root/plugins/argus-agent/bin/argus-agent"
echo "\n📦 Updating all-in-one-full agent binary → $agent_bin_dst"
cp -f "$agent_bin_src" "$agent_bin_dst"
chmod +x "$agent_bin_dst" || true
pushd "$aio_root" >/dev/null
local prev_version
prev_version="$(cat config/VERSION 2>/dev/null || echo "1.0.0")"
local use_version="$prev_version"
if [[ -n "$client_semver" ]]; then
echo "${client_semver}" > config/VERSION
use_version="$client_semver"
fi
echo " Packaging all-in-one-full artifact version: $use_version"
if ! bash scripts/package_artifact.sh --force; then
echo "❌ package_artifact.sh failed" >&2
# restore VERSION if changed
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
return 1
fi
local artifact_dir="$aio_root/artifact/$use_version"
local artifact_tar
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
if [[ -z "$artifact_tar" ]]; then
echo " No argus-metric_*.tar.gz found; invoking publish_artifact.sh to assemble..."
local owner="$(id -u):$(id -g)"
if ! bash scripts/publish_artifact.sh "$use_version" --output-dir "$artifact_dir" --owner "$owner"; then
echo "❌ publish_artifact.sh failed" >&2
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
return 1
fi
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
fi
if [[ -z "$artifact_tar" ]]; then
echo "❌ artifact tar not found under $artifact_dir" >&2
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
return 1
fi
# restore VERSION if changed (keep filesystem clean)
if [[ -n "$client_semver" ]]; then echo "$prev_version" > config/VERSION; fi
popd >/dev/null
# 4) Stage docker build context
local bundle_ctx="$root/src/bundle/gpu-node-bundle/.build-$date_tag"
echo "\n🧰 Staging docker build context: $bundle_ctx"
rm -rf "$bundle_ctx"
mkdir -p "$bundle_ctx/bundle" "$bundle_ctx/private"
cp "$root/src/bundle/gpu-node-bundle/Dockerfile" "$bundle_ctx/"
cp "$root/src/bundle/gpu-node-bundle/node-bootstrap.sh" "$bundle_ctx/"
cp "$root/src/bundle/gpu-node-bundle/health-watcher.sh" "$bundle_ctx/"
# bundle tar
cp "$artifact_tar" "$bundle_ctx/bundle/"
# offline fluent-bit assets (optional but useful)
if [[ -d "$root/src/log/fluent-bit/build/etc" ]]; then
cp -r "$root/src/log/fluent-bit/build/etc" "$bundle_ctx/private/"
fi
if [[ -d "$root/src/log/fluent-bit/build/packages" ]]; then
cp -r "$root/src/log/fluent-bit/build/packages" "$bundle_ctx/private/"
fi
if [[ -f "$root/src/log/fluent-bit/build/start-fluent-bit.sh" ]]; then
cp "$root/src/log/fluent-bit/build/start-fluent-bit.sh" "$bundle_ctx/private/"
fi
# 5) Build the final bundle image (directly from NVIDIA base)
local image_tag="argus-sys-metric-test-node-bundle-gpu:${date_tag}"
echo "\n🔄 Building GPU Bundle image"
if build_image "GPU Bundle" "$bundle_ctx/Dockerfile" "$image_tag" "$bundle_ctx" \
--build-arg CUDA_VER="$(echo "$base_image" | sed -E 's#^nvidia/cuda:([0-9]+\.[0-9]+\.[0-9]+)-runtime-ubuntu22\.04$#\1#')" \
--build-arg CLIENT_VER="$use_version" \
--build-arg BUNDLE_DATE="$date_tag"; then
images_built+=("$image_tag")
# In non-pkg mode, also tag latest for convenience
if [[ "${ARGUS_PKG_BUILD:-0}" != "1" ]]; then
docker tag "$image_tag" argus-sys-metric-test-node-bundle-gpu:latest >/dev/null 2>&1 || true
fi
return 0
else
return 1
fi
}
# Tag helper: ensure :<date_tag> exists for a list of repos
ensure_version_tags() {
local date_tag="$1"; shift
local repos=("$@")
for repo in "${repos[@]}"; do
if docker image inspect "$repo:$date_tag" >/dev/null 2>&1; then
:
elif docker image inspect "$repo:latest" >/dev/null 2>&1; then
docker tag "$repo:latest" "$repo:$date_tag" || true
else
echo "❌ missing image for tagging: $repo (need :latest or :$date_tag)" >&2
return 1
fi
done
return 0
}
# Build server package after images are built
build_server_pkg_bundle() {
local date_tag="$1"
if [[ -z "$date_tag" ]]; then
echo "❌ server_pkg requires --version YYMMDD" >&2
return 1
fi
local repos=(
argus-bind9 argus-master argus-elasticsearch argus-kibana \
argus-metric-ftp argus-metric-prometheus argus-metric-grafana \
argus-alertmanager argus-web-frontend argus-web-proxy
)
echo "\n🔖 Verifying server images with :$date_tag and collecting digests"
for repo in "${repos[@]}"; do
if ! docker image inspect "$repo:$date_tag" >/dev/null 2>&1; then
echo "❌ required image missing: $repo:$date_tag (build phase should have produced it)" >&2
return 1
fi
done
# Optional: show digests
for repo in "${repos[@]}"; do
local digest
digest=$(docker images --digests --format '{{.Repository}}:{{.Tag}} {{.Digest}}' | awk -v r="$repo:$date_tag" '$1==r{print $2}' | head -n1)
printf ' • %s@%s\n' "$repo:$date_tag" "${digest:-<none>}"
done
echo "\n📦 Building server package via deployment_new/build/make_server_package.sh --version $date_tag"
if ! "$root/deployment_new/build/make_server_package.sh" --version "$date_tag"; then
echo "❌ make_server_package.sh failed" >&2
return 1
fi
return 0
}
# Build client package: ensure gpu bundle image exists, then package client_gpu
build_client_pkg_bundle() {
local date_tag="$1"
local semver="$2"
local cuda="$3"
if [[ -z "$date_tag" ]]; then
echo "❌ client_pkg requires --version YYMMDD" >&2
return 1
fi
local bundle_tag="argus-sys-metric-test-node-bundle-gpu:${date_tag}"
if ! docker image inspect "$bundle_tag" >/dev/null 2>&1; then
echo "\n🧩 GPU bundle image $bundle_tag missing; building it first..."
ARGUS_PKG_BUILD=1
export ARGUS_PKG_BUILD
if ! build_gpu_bundle_image "$date_tag" "$cuda" "$semver"; then
return 1
fi
else
echo "\n✅ Using existing GPU bundle image: $bundle_tag"
fi
echo "\n📦 Building client GPU package via deployment_new/build/make_client_gpu_package.sh --version $date_tag --image $bundle_tag"
if ! "$root/deployment_new/build/make_client_gpu_package.sh" --version "$date_tag" --image "$bundle_tag"; then
echo "❌ make_client_gpu_package.sh failed" >&2
return 1
fi
return 0
}
# Build CPU bundle image directly FROM ubuntu:22.04 (no intermediate base)
build_cpu_bundle_image() {
local date_tag="$1" # e.g. 20251113
local client_ver_in="$2" # semver like 1.43.0 (optional)
local want_tag_latest="$3" # true/false
if [[ -z "$date_tag" ]]; then
echo "❌ cpu_bundle requires --version YYMMDD" >&2
return 1
fi
echo "\n🔧 Preparing one-click CPU bundle build"
echo " Base: ubuntu:22.04"
echo " Bundle tag: ${date_tag}"
# 1) Build latest argus-agent from source
echo "\n🛠 Building argus-agent from src/agent"
pushd "$root/src/agent" >/dev/null
if ! bash scripts/build_binary.sh; then
echo "❌ argus-agent build failed" >&2
popd >/dev/null
return 1
fi
if [[ ! -f "dist/argus-agent" ]]; then
echo "❌ argus-agent binary missing after build" >&2
popd >/dev/null
return 1
fi
popd >/dev/null
# 2) Inject agent into all-in-one-full plugin and package artifact
local aio_root="$root/src/metric/client-plugins/all-in-one-full"
local agent_bin_src="$root/src/agent/dist/argus-agent"
local agent_bin_dst="$aio_root/plugins/argus-agent/bin/argus-agent"
echo "\n📦 Updating all-in-one-full agent binary → $agent_bin_dst"
cp -f "$agent_bin_src" "$agent_bin_dst"
chmod +x "$agent_bin_dst" || true
pushd "$aio_root" >/dev/null
local prev_version use_version
prev_version="$(cat config/VERSION 2>/dev/null || echo "1.0.0")"
use_version="$prev_version"
if [[ -n "$client_ver_in" ]]; then
echo "$client_ver_in" > config/VERSION
use_version="$client_ver_in"
fi
echo " Packaging all-in-one-full artifact: version=$use_version"
if ! bash scripts/package_artifact.sh --force; then
echo "❌ package_artifact.sh failed" >&2
[[ -n "$client_ver_in" ]] && echo "$prev_version" > config/VERSION
popd >/dev/null
return 1
fi
local artifact_dir="$aio_root/artifact/$use_version"
local artifact_tar
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
if [[ -z "$artifact_tar" ]]; then
echo " No argus-metric_*.tar.gz found; invoking publish_artifact.sh ..."
local owner="$(id -u):$(id -g)"
if ! bash scripts/publish_artifact.sh "$use_version" --output-dir "$artifact_dir" --owner "$owner"; then
echo "❌ publish_artifact.sh failed" >&2
[[ -n "$client_ver_in" ]] && echo "$prev_version" > config/VERSION
popd >/dev/null
return 1
fi
artifact_tar="$(ls -1 "$artifact_dir"/argus-metric_*.tar.gz 2>/dev/null | head -n1 || true)"
fi
[[ -n "$client_ver_in" ]] && echo "$prev_version" > config/VERSION
popd >/dev/null
# 3) Stage docker build context
local bundle_ctx="$root/src/bundle/cpu-node-bundle/.build-$date_tag"
echo "\n🧰 Staging docker build context: $bundle_ctx"
rm -rf "$bundle_ctx"
mkdir -p "$bundle_ctx/bundle" "$bundle_ctx/private"
cp "$root/src/bundle/cpu-node-bundle/Dockerfile" "$bundle_ctx/"
cp "$root/src/bundle/cpu-node-bundle/node-bootstrap.sh" "$bundle_ctx/"
cp "$root/src/bundle/cpu-node-bundle/health-watcher.sh" "$bundle_ctx/"
# bundle tar
cp "$artifact_tar" "$bundle_ctx/bundle/"
# offline fluent-bit assets
if [[ -d "$root/src/log/fluent-bit/build/etc" ]]; then
cp -r "$root/src/log/fluent-bit/build/etc" "$bundle_ctx/private/"
fi
if [[ -d "$root/src/log/fluent-bit/build/packages" ]]; then
cp -r "$root/src/log/fluent-bit/build/packages" "$bundle_ctx/private/"
fi
if [[ -f "$root/src/log/fluent-bit/build/start-fluent-bit.sh" ]]; then
cp "$root/src/log/fluent-bit/build/start-fluent-bit.sh" "$bundle_ctx/private/"
fi
# 4) Build final bundle image
local image_tag="argus-sys-metric-test-node-bundle:${date_tag}"
echo "\n🔄 Building CPU Bundle image"
if build_image "CPU Bundle" "$bundle_ctx/Dockerfile" "$image_tag" "$bundle_ctx"; then
images_built+=("$image_tag")
if [[ "$want_tag_latest" == "true" ]]; then
docker tag "$image_tag" argus-sys-metric-test-node-bundle:latest >/dev/null 2>&1 || true
fi
return 0
else
return 1
fi
}
if [[ "$build_core" == true ]]; then
if build_image "Elasticsearch" "src/log/elasticsearch/build/Dockerfile" "argus-elasticsearch:${DEFAULT_IMAGE_TAG}"; then
images_built+=("argus-elasticsearch:${DEFAULT_IMAGE_TAG}")
else
build_failed=true
fi
echo ""
if build_image "Kibana" "src/log/kibana/build/Dockerfile" "argus-kibana:${DEFAULT_IMAGE_TAG}"; then
images_built+=("argus-kibana:${DEFAULT_IMAGE_TAG}")
else
build_failed=true
fi
echo ""
if build_image "BIND9" "src/bind/build/Dockerfile" "argus-bind9:${DEFAULT_IMAGE_TAG}"; then
images_built+=("argus-bind9:${DEFAULT_IMAGE_TAG}")
else
build_failed=true
fi
fi
echo ""
if [[ "$build_master" == true ]]; then
echo ""
echo "🔄 Building Master image..."
pushd "$master_root" >/dev/null
master_args=("--tag" "argus-master:${DEFAULT_IMAGE_TAG}")
if [[ "$use_intranet" == true ]]; then
master_args+=("--intranet")
fi
if [[ "$build_master_offline" == true ]]; then
master_args+=("--offline")
fi
if [[ "$no_cache" == true ]]; then
master_args+=("--no-cache")
fi
if ./scripts/build_images.sh "${master_args[@]}"; then
if [[ "$build_master_offline" == true ]]; then
images_built+=("argus-master:offline")
else
images_built+=("argus-master:${DEFAULT_IMAGE_TAG}")
fi
else
build_failed=true
fi
popd >/dev/null
fi
if [[ "$build_metric" == true ]]; then
echo ""
echo "Building Metric module images..."
metric_base_images=(
"ubuntu:22.04"
"ubuntu/prometheus:3-24.04_stable"
"grafana/grafana:11.1.0"
)
for base_image in "${metric_base_images[@]}"; do
if ! pull_base_image "$base_image"; then
build_failed=true
fi
done
metric_builds=(
"Metric FTP|src/metric/ftp/build/Dockerfile|argus-metric-ftp:${DEFAULT_IMAGE_TAG}|src/metric/ftp/build"
"Metric Prometheus|src/metric/prometheus/build/Dockerfile|argus-metric-prometheus:${DEFAULT_IMAGE_TAG}|src/metric/prometheus/build"
"Metric Grafana|src/metric/grafana/build/Dockerfile|argus-metric-grafana:${DEFAULT_IMAGE_TAG}|src/metric/grafana/build"
"Metric Prometheus Targets Updater|src/metric/prometheus/build/Dockerfile.targets-updater|argus-metric-prometheus-targets-updater:${DEFAULT_IMAGE_TAG}|src/metric/prometheus/build"
)
for build_spec in "${metric_builds[@]}"; do
IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec"
if build_image "$image_label" "$dockerfile_path" "$image_tag" "$build_context"; then
images_built+=("$image_tag")
else
build_failed=true
fi
echo ""
done
fi
# =======================================
# Sys (system tests) node images
# =======================================
if [[ "$build_sys" == true ]]; then
echo ""
echo "Building Sys node images..."
sys_base_images=(
"ubuntu:22.04"
"nvidia/cuda:12.2.2-runtime-ubuntu22.04"
)
for base_image in "${sys_base_images[@]}"; do
if ! pull_base_image "$base_image"; then
build_failed=true
fi
done
sys_builds=(
"Sys Node|src/sys/build/node/Dockerfile|argus-sys-node:latest|."
"Sys Metric Test Node|src/sys/build/test-node/Dockerfile|argus-sys-metric-test-node:latest|."
"Sys Metric Test GPU Node|src/sys/build/test-gpu-node/Dockerfile|argus-sys-metric-test-gpu-node:latest|."
)
for build_spec in "${sys_builds[@]}"; do
IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec"
if build_image "$image_label" "$dockerfile_path" "$image_tag" "$build_context"; then
images_built+=("$image_tag")
else
build_failed=true
fi
echo ""
done
fi
# =======================================
# Web & Alert module images
# =======================================
if [[ "$build_web" == true || "$build_alert" == true ]]; then
echo ""
echo "Building Web and Alert module images..."
# Pre-pull commonly used base images for stability
web_alert_base_images=(
"node:20"
"ubuntu:24.04"
)
for base_image in "${web_alert_base_images[@]}"; do
if ! pull_base_image "$base_image"; then
build_failed=true
fi
done
if [[ "$build_web" == true ]]; then
web_builds=(
"Web Frontend|src/web/build_tools/frontend/Dockerfile|argus-web-frontend:${DEFAULT_IMAGE_TAG}|."
"Web Proxy|src/web/build_tools/proxy/Dockerfile|argus-web-proxy:${DEFAULT_IMAGE_TAG}|."
)
for build_spec in "${web_builds[@]}"; do
IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec"
if build_image "$image_label" "$dockerfile_path" "$image_tag" "$build_context"; then
images_built+=("$image_tag")
else
build_failed=true
fi
echo ""
done
fi
if [[ "$build_alert" == true ]]; then
alert_builds=(
"Alertmanager|src/alert/alertmanager/build/Dockerfile|argus-alertmanager:${DEFAULT_IMAGE_TAG}|."
)
for build_spec in "${alert_builds[@]}"; do
IFS='|' read -r image_label dockerfile_path image_tag build_context <<< "$build_spec"
if build_image "$image_label" "$dockerfile_path" "$image_tag" "$build_context"; then
images_built+=("$image_tag")
else
build_failed=true
fi
echo ""
done
fi
fi
# =======================================
# One-click GPU bundle (direct NVIDIA base)
# =======================================
if [[ "$build_gpu_bundle" == true ]]; then
echo ""
echo "Building one-click GPU bundle image..."
if ! build_gpu_bundle_image "$bundle_date" "$cuda_ver" "$client_semver"; then
build_failed=true
fi
fi
# =======================================
# One-click CPU bundle (from ubuntu:22.04)
# =======================================
if [[ "$build_cpu_bundle" == true ]]; then
echo ""
echo "Building one-click CPU bundle image..."
if ! build_cpu_bundle_image "${bundle_date}" "${client_semver}" "${tag_latest}"; then
build_failed=true
fi
fi
# =======================================
# One-click Server/Client packaging
# =======================================
if [[ "$build_server_pkg" == true ]]; then
echo ""
echo "🧳 Building one-click Server package..."
if ! build_server_pkg_bundle "${bundle_date}"; then
build_failed=true
fi
fi
if [[ "$build_client_pkg" == true ]]; then
echo ""
echo "🧳 Building one-click Client-GPU package..."
if ! build_client_pkg_bundle "${bundle_date}" "${client_semver}" "${cuda_ver}"; then
build_failed=true
fi
fi
echo "======================================="
echo "📦 Build Summary"
echo "======================================="
if [[ ${#images_built[@]} -gt 0 ]]; then
echo "✅ Successfully built images:"
for image in "${images_built[@]}"; do
echo " • $image"
done
fi
if [[ "$build_failed" == true ]]; then
echo ""
echo "❌ Some images failed to build. Please check the errors above."
exit 1
fi
if [[ "$use_intranet" == true ]]; then
echo ""
echo "🌐 Built with intranet mirror configuration"
fi
if [[ "$build_master_offline" == true ]]; then
echo ""
echo "🧳 Master offline wheels 已解压到 $master_offline_dir"
fi
echo ""
echo "🚀 Next steps:"
echo " ./build/save_images.sh --compress # 导出镜像"
echo " cd src/master/tests && MASTER_IMAGE_TAG=argus-master:offline ./scripts/00_e2e_test.sh"
echo ""

View File

@@ -0,0 +1,6 @@
# Default build-time UID/GID for Argus images
# Override by creating configs/build_user.local.conf with the same format.
# Syntax: KEY=VALUE, supports UID/GID only. Whitespace and lines starting with # are ignored.
UID=2133
GID=2015

1
deployment_new/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
artifact/

14
deployment_new/README.md Normal file
View File

@@ -0,0 +1,14 @@
# deployment_new
本目录用于新的部署打包与交付实现(不影响既有 `deployment/`)。
里程碑 M1(当前实现)
- `build/make_server_package.sh`:生成 Server 包(逐服务镜像 tar.gz、compose、.env.example、docs、private 骨架、manifest/checksums、打包 tar.gz)。
- `build/make_client_gpu_package.sh`:生成 Client‑GPU 包(GPU bundle 镜像 tar.gz、busybox.tar、compose、.env.example、docs、private 骨架、manifest/checksums、打包 tar.gz)。
模板
- `templates/server/compose/docker-compose.yml`:部署专用,镜像默认使用 `:${PKG_VERSION}` 版本 tag,可通过 `.env` 覆盖。
- `templates/client_gpu/compose/docker-compose.yml`:GPU 节点专用,使用 `:${PKG_VERSION}` 版本 tag。
注意:M1 仅产出安装包,不包含安装脚本落地;安装/运维脚本将在 M2 落地并纳入包内。

View File

@@ -0,0 +1,33 @@
#!/usr/bin/env bash
set -euo pipefail
log() { echo -e "\033[0;34m[INFO]\033[0m $*"; }
warn() { echo -e "\033[1;33m[WARN]\033[0m $*"; }
err() { echo -e "\033[0;31m[ERR ]\033[0m $*" >&2; }
require_cmd() {
local miss=0
for c in "$@"; do
if ! command -v "$c" >/dev/null 2>&1; then err "missing command: $c"; miss=1; fi
done
[[ $miss -eq 0 ]]
}
today_version() { date +%Y%m%d; }
checksum_dir() {
local dir="$1"; local out="$2"; : > "$out";
(cd "$dir" && find . -type f -print0 | sort -z | xargs -0 sha256sum) >> "$out"
}
make_dir() { mkdir -p "$1"; }
copy_tree() {
local src="$1" dst="$2"; rsync -a --delete "$src/" "$dst/" 2>/dev/null || cp -r "$src/." "$dst/";
}
gen_manifest() {
local root="$1"; local out="$2"; : > "$out";
(cd "$root" && find . -maxdepth 4 -type f -printf "%p\n" | sort) >> "$out"
}

View File

@@ -0,0 +1,112 @@
#!/usr/bin/env bash
set -euo pipefail
# Make ARM Client-CPU package (versioned CPU node image, compose, env, docs)
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
TEMPL_DIR="$ROOT_DIR/deployment_new/templates/client_arm"
ART_ROOT="$ROOT_DIR/deployment_new/artifact/client_arm"
COMMON_SH="$ROOT_DIR/deployment_new/build/common.sh"
. "$COMMON_SH"
usage(){ cat <<EOF
Build ARM Client-CPU Package (deployment_new)
Usage: $(basename "$0") --version YYYYMMDD [--image IMAGE[:TAG]]
Defaults:
image = argus-sys-metric-test-node-arm64:latest
Outputs: deployment_new/artifact/client_arm/<YYYYMMDD>/ and client_arm_YYYYMMDD.tar.gz
EOF
}
VERSION=""
IMAGE="argus-sys-metric-test-node-arm64:latest"
while [[ $# -gt 0 ]]; do
case "$1" in
--version) VERSION="$2"; shift 2;;
--image) IMAGE="$2"; shift 2;;
-h|--help) usage; exit 0;;
*) err "unknown arg: $1"; usage; exit 1;;
esac
done
if [[ -z "$VERSION" ]]; then VERSION="$(today_version)"; fi
require_cmd docker tar gzip
STAGE="$(mktemp -d)"; trap 'rm -rf "$STAGE"' EXIT
PKG_DIR="$ART_ROOT/$VERSION"
mkdir -p "$PKG_DIR" "$STAGE/images" "$STAGE/compose" "$STAGE/docs" "$STAGE/scripts" "$STAGE/private/argus"
# 1) Save CPU node image with version tag
if ! docker image inspect "$IMAGE" >/dev/null 2>&1; then
err "missing image: $IMAGE"; exit 1; fi
REPO="${IMAGE%%:*}"; TAG_VER="$REPO:$VERSION"
docker tag "$IMAGE" "$TAG_VER"
out_tar="$STAGE/images/${REPO//\//-}-$VERSION.tar"
docker save -o "$out_tar" "$TAG_VER"
gzip -f "$out_tar"
# 2) Compose + env template
cp "$TEMPL_DIR/compose/docker-compose.yml" "$STAGE/compose/docker-compose.yml"
ENV_EX="$STAGE/compose/.env.example"
cat >"$ENV_EX" <<EOF
# Generated by make_arm_client_package.sh
PKG_VERSION=$VERSION
NODE_CPU_IMAGE_TAG=${REPO}:${VERSION}
# Compose project name (isolation from server stack)
COMPOSE_PROJECT_NAME=argus-client-arm
# Required fields(ARM Client 默认通过 overlay alias 访问 master,可按需覆盖 MASTER_ENDPOINT)
MASTER_ENDPOINT=http://master.argus.com:3000
AGENT_ENV=
AGENT_USER=
AGENT_INSTANCE=
CPU_NODE_HOSTNAME=
# UID/GID for volume ownership
ARGUS_BUILD_UID=2133
ARGUS_BUILD_GID=2015
EOF
# 3) Docs
CLIENT_DOC_SRC="$TEMPL_DIR/docs"
if [[ -d "$CLIENT_DOC_SRC" ]]; then
copy_tree "$CLIENT_DOC_SRC" "$STAGE/docs"
fi
# 4) Scripts & private skeleton
SCRIPTS_SRC="$TEMPL_DIR/scripts"
if [[ -d "$SCRIPTS_SRC" ]]; then
copy_tree "$SCRIPTS_SRC" "$STAGE/scripts"
find "$STAGE/scripts" -type f -name '*.sh' -exec chmod +x {} + 2>/dev/null || true
fi
# x86 专用脚本:用于在 x86_64 主机上运行 ARM Client 包
SCRIPTS_X86_SRC="$TEMPL_DIR/scripts_for_x86"
if [[ -d "$SCRIPTS_X86_SRC" ]]; then
copy_tree "$SCRIPTS_X86_SRC" "$STAGE/scripts_for_x86"
find "$STAGE/scripts_for_x86" -type f -name '*.sh' -exec chmod +x {} + 2>/dev/null || true
fi
mkdir -p "$STAGE/private/argus/agent"
# 5) Manifest & checksums
gen_manifest "$STAGE" "$STAGE/manifest.txt"
checksum_dir "$STAGE" "$STAGE/checksums.txt"
# 6) Move to artifact dir and pack
mkdir -p "$PKG_DIR"
copy_tree "$STAGE" "$PKG_DIR"
OUT_TAR_DIR="$(dirname "$PKG_DIR")"
OUT_TAR="$OUT_TAR_DIR/client_arm_${VERSION}.tar.gz"
log "Creating tarball: $OUT_TAR"
(cd "$PKG_DIR/.." && tar -czf "$OUT_TAR" "$(basename "$PKG_DIR")")
log "Client-ARM package ready: $PKG_DIR"
echo "$OUT_TAR"

View File

@@ -0,0 +1,175 @@
#!/usr/bin/env bash
set -euo pipefail
# Make ARM server deployment package (master + prometheus + grafana + alertmanager + web)
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
TEMPL_DIR="$ROOT_DIR/deployment_new/templates/server_arm"
ART_ROOT="$ROOT_DIR/deployment_new/artifact/server_arm"
COMMON_SH="$ROOT_DIR/deployment_new/build/common.sh"
. "$COMMON_SH"
usage(){ cat <<EOF
Build ARM Server Deployment Package (deployment_new)
Usage: $(basename "$0") --version YYYYMMDD
Outputs: deployment_new/artifact/server_arm/<YYYYMMDD>/ and server_arm_YYYYMMDD.tar.gz
EOF
}
VERSION=""
while [[ $# -gt 0 ]]; do
case "$1" in
--version) VERSION="$2"; shift 2;;
-h|--help) usage; exit 0;;
*) err "unknown arg: $1"; usage; exit 1;;
esac
done
if [[ -z "$VERSION" ]]; then VERSION="$(today_version)"; fi
require_cmd docker tar gzip awk sed
IMAGES=(
argus-master-arm64
argus-metric-prometheus-arm64
argus-metric-prometheus-targets-updater-arm64
argus-metric-grafana-arm64
argus-alertmanager-arm64
argus-web-frontend-arm64
argus-web-proxy-arm64
)
STAGE="$(mktemp -d)"; trap 'rm -rf "$STAGE"' EXIT
PKG_DIR="$ART_ROOT/$VERSION"
mkdir -p "$PKG_DIR" "$STAGE/images" "$STAGE/compose" "$STAGE/docs" "$STAGE/scripts" "$STAGE/private/argus"
# 1) Save per-image tars with version tag
log "Tagging and saving ARM server images (version=$VERSION)"
for repo in "${IMAGES[@]}"; do
if ! docker image inspect "$repo:latest" >/dev/null 2>&1 && ! docker image inspect "$repo:$VERSION" >/dev/null 2>&1; then
err "missing image: $repo (need :latest or :$VERSION)"; exit 1; fi
if docker image inspect "$repo:$VERSION" >/dev/null 2>&1; then
tag="$repo:$VERSION"
else
docker tag "$repo:latest" "$repo:$VERSION"
tag="$repo:$VERSION"
fi
out_tar="$STAGE/images/${repo//\//-}-$VERSION.tar"
docker save -o "$out_tar" "$tag"
gzip -f "$out_tar"
done
# 1.1) (可选)打包 x86 ARM 测试用基础镜像 ubuntu:22.04(供 scripts_for_x86/install.sh 使用)
TEST_BASE="ubuntu:22.04"
if docker image inspect "$TEST_BASE" >/dev/null 2>&1 || docker pull "$TEST_BASE" >/dev/null 2>&1; then
log "Saving ARM test base image for x86 host: $TEST_BASE"
test_tar="$STAGE/images/ubuntu-22.04-arm-test.tar"
docker save -o "$test_tar" "$TEST_BASE"
gzip -f "$test_tar"
else
warn "无法获取 $TEST_BASE,x86 上的 ARM 运行能力将无法使用该镜像做快速自检(可忽略,或在目标机自行拉取)。"
fi
# 1.2) 随包提供 tonistiigi/binfmt 镜像,便于 x86 离线机器安装 binfmt/QEMU
BINFMT_IMAGE="tonistiigi/binfmt:latest"
if docker image inspect "$BINFMT_IMAGE" >/dev/null 2>&1 || docker pull "$BINFMT_IMAGE" >/dev/null 2>&1; then
log "Saving binfmt helper image for x86 host: $BINFMT_IMAGE"
binfmt_tar="$STAGE/images/tonistiigi-binfmt.tar"
docker save -o "$binfmt_tar" "$BINFMT_IMAGE"
gzip -f "$binfmt_tar"
else
warn "无法获取 $BINFMT_IMAGE,x86 上将无法通过离线方式安装 binfmt(可在目标机上自行 docker pull)。"
fi
# 2) Compose + env template(ARM 专用)
cp "$TEMPL_DIR/compose/docker-compose.yml" "$STAGE/compose/docker-compose.yml"
ENV_EX="$STAGE/compose/.env.example"
cat >"$ENV_EX" <<EOF
# Generated by make_arm_server_package.sh
PKG_VERSION=$VERSION
# ARM server image tags
MASTER_IMAGE_TAG=argus-master-arm64:${VERSION}
PROM_IMAGE_TAG=argus-metric-prometheus-arm64:${VERSION}
PROM_UPDATER_IMAGE_TAG=argus-metric-prometheus-targets-updater-arm64:${VERSION}
GRAFANA_IMAGE_TAG=argus-metric-grafana-arm64:${VERSION}
ALERT_IMAGE_TAG=argus-alertmanager-arm64:${VERSION}
FRONT_IMAGE_TAG=argus-web-frontend-arm64:${VERSION}
WEB_PROXY_IMAGE_TAG=argus-web-proxy-arm64:${VERSION}
# Host ports for server compose
MASTER_PORT=32300
PROMETHEUS_PORT=9090
GRAFANA_PORT=3000
ALERTMANAGER_PORT=9093
WEB_PROXY_PORT_8080=8080
WEB_PROXY_PORT_8081=8081
WEB_PROXY_PORT_8082=8082
WEB_PROXY_PORT_8083=8083
WEB_PROXY_PORT_8084=8084
WEB_PROXY_PORT_8085=8085
# Overlay network name
ARGUS_OVERLAY_NET=argus-sys-net
# UID/GID for volume ownership
ARGUS_BUILD_UID=2133
ARGUS_BUILD_GID=2015
# Compose project name (isolation from other stacks on same host)
COMPOSE_PROJECT_NAME=argus-server-arm
EOF
# 3) Docs & scripts from templates/server_arm
DOCS_SRC="$TEMPL_DIR/docs"
if [[ -d "$DOCS_SRC" ]]; then
copy_tree "$DOCS_SRC" "$STAGE/docs"
fi
SCRIPTS_SRC="$TEMPL_DIR/scripts"
if [[ -d "$SCRIPTS_SRC" ]]; then
copy_tree "$SCRIPTS_SRC" "$STAGE/scripts"
find "$STAGE/scripts" -type f -name '*.sh' -exec chmod +x {} + 2>/dev/null || true
fi
# x86 专用脚本:用于在 x86_64 主机上运行 ARM server 包
SCRIPTS_X86_SRC="$TEMPL_DIR/scripts_for_x86"
if [[ -d "$SCRIPTS_X86_SRC" ]]; then
copy_tree "$SCRIPTS_X86_SRC" "$STAGE/scripts_for_x86"
find "$STAGE/scripts_for_x86" -type f -name '*.sh' -exec chmod +x {} + 2>/dev/null || true
fi
# 4) Private skeleton (最小 ARM server 所需目录)
mkdir -p \
"$STAGE/private/argus/etc" \
"$STAGE/private/argus/master" \
"$STAGE/private/argus/metric/prometheus" \
"$STAGE/private/argus/metric/prometheus/data" \
"$STAGE/private/argus/metric/prometheus/rules" \
"$STAGE/private/argus/metric/prometheus/targets" \
"$STAGE/private/argus/metric/grafana" \
"$STAGE/private/argus/metric/grafana/data" \
"$STAGE/private/argus/metric/grafana/logs" \
"$STAGE/private/argus/metric/grafana/plugins" \
"$STAGE/private/argus/metric/grafana/provisioning/datasources" \
"$STAGE/private/argus/metric/grafana/provisioning/dashboards" \
"$STAGE/private/argus/metric/grafana/data/sessions" \
"$STAGE/private/argus/metric/grafana/data/dashboards" \
"$STAGE/private/argus/metric/grafana/config"
# 5) Manifest & checksums
gen_manifest "$STAGE" "$STAGE/manifest.txt"
checksum_dir "$STAGE" "$STAGE/checksums.txt"
# 6) Move to artifact dir and pack
mkdir -p "$PKG_DIR"
copy_tree "$STAGE" "$PKG_DIR"
OUT_TAR_DIR="$(dirname "$PKG_DIR")"
OUT_TAR="$OUT_TAR_DIR/server_arm_${VERSION}.tar.gz"
log "Creating tarball: $OUT_TAR"
(cd "$PKG_DIR/.." && tar -czf "$OUT_TAR" "$(basename "$PKG_DIR")")
log "ARM Server package ready: $PKG_DIR"
echo "$OUT_TAR"

View File

@@ -0,0 +1,134 @@
#!/usr/bin/env bash
set -euo pipefail
# Make client GPU package (versioned gpu bundle image, compose, env, docs, busybox)
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
TEMPL_DIR="$ROOT_DIR/deployment_new/templates/client_gpu"
ART_ROOT="$ROOT_DIR/deployment_new/artifact/client_gpu"
# Use deployment_new local common helpers
COMMON_SH="$ROOT_DIR/deployment_new/build/common.sh"
. "$COMMON_SH"
usage(){ cat <<EOF
Build Client-GPU Package (deployment_new)
Usage: $(basename "$0") --version YYYYMMDD [--image IMAGE[:TAG]]
Defaults:
image = argus-sys-metric-test-node-bundle-gpu:latest
Outputs: deployment_new/artifact/client_gpu/<YYYYMMDD>/ and client_gpu_YYYYMMDD.tar.gz
EOF
}
VERSION=""
IMAGE="argus-sys-metric-test-node-bundle-gpu:latest"
while [[ $# -gt 0 ]]; do
case "$1" in
--version) VERSION="$2"; shift 2;;
--image) IMAGE="$2"; shift 2;;
-h|--help) usage; exit 0;;
*) err "unknown arg: $1"; usage; exit 1;;
esac
done
if [[ -z "$VERSION" ]]; then VERSION="$(today_version)"; fi
require_cmd docker tar gzip
STAGE="$(mktemp -d)"; trap 'rm -rf "$STAGE"' EXIT
PKG_DIR="$ART_ROOT/$VERSION"
mkdir -p "$PKG_DIR" "$STAGE/images" "$STAGE/compose" "$STAGE/docs" "$STAGE/scripts" "$STAGE/private/argus"
# 1) Save GPU bundle image with version tag
if ! docker image inspect "$IMAGE" >/dev/null 2>&1; then
err "missing image: $IMAGE"; exit 1; fi
REPO="${IMAGE%%:*}"; TAG_VER="$REPO:$VERSION"
docker tag "$IMAGE" "$TAG_VER"
out_tar="$STAGE/images/${REPO//\//-}-$VERSION.tar"
docker save -o "$out_tar" "$TAG_VER"
gzip -f "$out_tar"
# 2) Busybox tar for connectivity/overlay warmup (prefer local template; fallback to docker save)
BB_SRC="$TEMPL_DIR/images/busybox.tar"
if [[ -f "$BB_SRC" ]]; then
cp "$BB_SRC" "$STAGE/images/busybox.tar"
else
if docker image inspect busybox:latest >/dev/null 2>&1 || docker pull busybox:latest >/dev/null 2>&1; then
docker save -o "$STAGE/images/busybox.tar" busybox:latest
log "Included busybox from local docker daemon"
else
warn "busybox image not found and cannot pull; skipping busybox.tar"
fi
fi
# 3) Compose + env template and docs/scripts from templates
cp "$TEMPL_DIR/compose/docker-compose.yml" "$STAGE/compose/docker-compose.yml"
ENV_EX="$STAGE/compose/.env.example"
cat >"$ENV_EX" <<EOF
# Generated by make_client_gpu_package.sh
PKG_VERSION=$VERSION
NODE_GPU_BUNDLE_IMAGE_TAG=${REPO}:${VERSION}
# Compose project name (isolation from server stack)
COMPOSE_PROJECT_NAME=argus-client
# Required (no defaults). Must be filled before install.
AGENT_ENV=
AGENT_USER=
AGENT_INSTANCE=
GPU_NODE_HOSTNAME=
# From cluster-info.env (server package output)
BINDIP=
FTPIP=
SWARM_MANAGER_ADDR=
SWARM_JOIN_TOKEN_WORKER=
SWARM_JOIN_TOKEN_MANAGER=
# FTP defaults
FTP_USER=ftpuser
FTP_PASSWORD=NASPlab1234!
EOF
# 4) Docs from deployment_new templates
CLIENT_DOC_SRC="$TEMPL_DIR/docs"
if [[ -d "$CLIENT_DOC_SRC" ]]; then
rsync -a "$CLIENT_DOC_SRC/" "$STAGE/docs/" >/dev/null 2>&1 || cp -r "$CLIENT_DOC_SRC/." "$STAGE/docs/"
fi
# Placeholder scripts (will be implemented in M2)
cat >"$STAGE/scripts/README.md" <<'EOF'
# Client-GPU Scripts (Placeholder)
本目录将在 M2 引入:
- config.sh / install.sh
当前为占位,便于包结构审阅。
EOF
# 5) Scripts (from deployment_new templates) and Private skeleton
SCRIPTS_SRC="$TEMPL_DIR/scripts"
if [[ -d "$SCRIPTS_SRC" ]]; then
rsync -a "$SCRIPTS_SRC/" "$STAGE/scripts/" >/dev/null 2>&1 || cp -r "$SCRIPTS_SRC/." "$STAGE/scripts/"
find "$STAGE/scripts" -type f -name '*.sh' -exec chmod +x {} + 2>/dev/null || true
fi
mkdir -p "$STAGE/private/argus/agent"
# 6) Manifest & checksums
gen_manifest "$STAGE" "$STAGE/manifest.txt"
checksum_dir "$STAGE" "$STAGE/checksums.txt"
# 7) Move to artifact dir and pack
mkdir -p "$PKG_DIR"
rsync -a "$STAGE/" "$PKG_DIR/" >/dev/null 2>&1 || cp -r "$STAGE/." "$PKG_DIR/"
OUT_TAR_DIR="$(dirname "$PKG_DIR")"
OUT_TAR="$OUT_TAR_DIR/client_gpu_${VERSION}.tar.gz"
log "Creating tarball: $OUT_TAR"
(cd "$PKG_DIR/.." && tar -czf "$OUT_TAR" "$(basename "$PKG_DIR")")
log "Client-GPU package ready: $PKG_DIR"
echo "$OUT_TAR"

View File

@@ -0,0 +1,169 @@
#!/usr/bin/env bash
set -euo pipefail
# Make server deployment package (versioned, per-image tars, full compose, docs, skeleton)
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/../.." && pwd)"
TEMPL_DIR="$ROOT_DIR/deployment_new/templates/server"
ART_ROOT="$ROOT_DIR/deployment_new/artifact/server"
# Use deployment_new local common helpers
COMMON_SH="$ROOT_DIR/deployment_new/build/common.sh"
. "$COMMON_SH"
usage(){ cat <<EOF
Build Server Deployment Package (deployment_new)
Usage: $(basename "$0") --version YYYYMMDD
Outputs: deployment_new/artifact/server/<YYYYMMDD>/ and server_YYYYMMDD.tar.gz
EOF
}
VERSION=""
while [[ $# -gt 0 ]]; do
case "$1" in
--version) VERSION="$2"; shift 2;;
-h|--help) usage; exit 0;;
*) err "unknown arg: $1"; usage; exit 1;;
esac
done
if [[ -z "$VERSION" ]]; then VERSION="$(today_version)"; fi
require_cmd docker tar gzip awk sed
IMAGES=(
argus-bind9
argus-master
argus-elasticsearch
argus-kibana
argus-metric-ftp
argus-metric-prometheus
argus-metric-grafana
argus-alertmanager
argus-web-frontend
argus-web-proxy
)
STAGE="$(mktemp -d)"; trap 'rm -rf "$STAGE"' EXIT
PKG_DIR="$ART_ROOT/$VERSION"
mkdir -p "$PKG_DIR" "$STAGE/images" "$STAGE/compose" "$STAGE/docs" "$STAGE/scripts" "$STAGE/private/argus"
# 1) Save per-image tars with version tag
log "Tagging and saving images (version=$VERSION)"
for repo in "${IMAGES[@]}"; do
if ! docker image inspect "$repo:latest" >/dev/null 2>&1 && ! docker image inspect "$repo:$VERSION" >/dev/null 2>&1; then
err "missing image: $repo (need :latest or :$VERSION)"; exit 1; fi
if docker image inspect "$repo:$VERSION" >/dev/null 2>&1; then
tag="$repo:$VERSION"
else
docker tag "$repo:latest" "$repo:$VERSION"
tag="$repo:$VERSION"
fi
out_tar="$STAGE/images/${repo//\//-}-$VERSION.tar"
docker save -o "$out_tar" "$tag"
gzip -f "$out_tar"
done
# 2) Compose + env template
cp "$TEMPL_DIR/compose/docker-compose.yml" "$STAGE/compose/docker-compose.yml"
ENV_EX="$STAGE/compose/.env.example"
cat >"$ENV_EX" <<EOF
# Generated by make_server_package.sh
PKG_VERSION=$VERSION
# Image tags (can be overridden). Default to versioned tags
BIND_IMAGE_TAG=argus-bind9:
MASTER_IMAGE_TAG=argus-master:
ES_IMAGE_TAG=argus-elasticsearch:
KIBANA_IMAGE_TAG=argus-kibana:
FTP_IMAGE_TAG=argus-metric-ftp:
PROM_IMAGE_TAG=argus-metric-prometheus:
GRAFANA_IMAGE_TAG=argus-metric-grafana:
ALERT_IMAGE_TAG=argus-alertmanager:
FRONT_IMAGE_TAG=argus-web-frontend:
WEB_PROXY_IMAGE_TAG=argus-web-proxy:
EOF
sed -i "s#:\$#:${VERSION}#g" "$ENV_EX"
# Ports and defaults (based on swarm_tests .env.example)
cat >>"$ENV_EX" <<'EOF'
# Host ports for server compose
MASTER_PORT=32300
ES_HTTP_PORT=9200
KIBANA_PORT=5601
PROMETHEUS_PORT=9090
GRAFANA_PORT=3000
ALERTMANAGER_PORT=9093
WEB_PROXY_PORT_8080=8080
WEB_PROXY_PORT_8081=8081
WEB_PROXY_PORT_8082=8082
WEB_PROXY_PORT_8083=8083
WEB_PROXY_PORT_8084=8084
WEB_PROXY_PORT_8085=8085
# Overlay network name
ARGUS_OVERLAY_NET=argus-sys-net
# FTP defaults
FTP_USER=ftpuser
FTP_PASSWORD=NASPlab1234!
# UID/GID for volume ownership
ARGUS_BUILD_UID=2133
ARGUS_BUILD_GID=2015
# Compose project name (isolation from other stacks on same host)
COMPOSE_PROJECT_NAME=argus-server
EOF
# 3) Docs (from deployment_new templates)
DOCS_SRC="$TEMPL_DIR/docs"
if [[ -d "$DOCS_SRC" ]]; then
rsync -a "$DOCS_SRC/" "$STAGE/docs/" >/dev/null 2>&1 || cp -r "$DOCS_SRC/." "$STAGE/docs/"
fi
# 6) Scripts (from deployment_new templates)
SCRIPTS_SRC="$TEMPL_DIR/scripts"
if [[ -d "$SCRIPTS_SRC" ]]; then
rsync -a "$SCRIPTS_SRC/" "$STAGE/scripts/" >/dev/null 2>&1 || cp -r "$SCRIPTS_SRC/." "$STAGE/scripts/"
find "$STAGE/scripts" -type f -name '*.sh' -exec chmod +x {} + 2>/dev/null || true
fi
# 4) Private skeleton (minimum)
mkdir -p \
"$STAGE/private/argus/etc" \
"$STAGE/private/argus/master" \
"$STAGE/private/argus/metric/prometheus" \
"$STAGE/private/argus/metric/prometheus/data" \
"$STAGE/private/argus/metric/prometheus/rules" \
"$STAGE/private/argus/metric/prometheus/targets" \
"$STAGE/private/argus/metric/grafana" \
"$STAGE/private/argus/metric/grafana/data" \
"$STAGE/private/argus/metric/grafana/logs" \
"$STAGE/private/argus/metric/grafana/plugins" \
"$STAGE/private/argus/metric/grafana/provisioning/datasources" \
"$STAGE/private/argus/metric/grafana/provisioning/dashboards" \
"$STAGE/private/argus/metric/grafana/data/sessions" \
"$STAGE/private/argus/metric/grafana/data/dashboards" \
"$STAGE/private/argus/metric/grafana/config" \
"$STAGE/private/argus/metric/ftp" \
"$STAGE/private/argus/alert/alertmanager" \
"$STAGE/private/argus/log/elasticsearch" \
"$STAGE/private/argus/log/kibana"
# 7) Manifest & checksums
gen_manifest "$STAGE" "$STAGE/manifest.txt"
checksum_dir "$STAGE" "$STAGE/checksums.txt"
# 8) Move to artifact dir and pack
mkdir -p "$PKG_DIR"
rsync -a "$STAGE/" "$PKG_DIR/" >/dev/null 2>&1 || cp -r "$STAGE/." "$PKG_DIR/"
OUT_TAR_DIR="$(dirname "$PKG_DIR")"
OUT_TAR="$OUT_TAR_DIR/server_${VERSION}.tar.gz"
log "Creating tarball: $OUT_TAR"
(cd "$PKG_DIR/.." && tar -czf "$OUT_TAR" "$(basename "$PKG_DIR")")
log "Server package ready: $PKG_DIR"
echo "$OUT_TAR"

View File

@@ -0,0 +1,29 @@
version: "3.8"
networks:
argus-sys-net:
external: true
services:
metric-cpu-node:
image: ${NODE_CPU_IMAGE_TAG:-argus-sys-metric-test-node-arm64:${PKG_VERSION}}
platform: linux/arm64
container_name: argus-metric-cpu-node
hostname: ${CPU_NODE_HOSTNAME}
restart: unless-stopped
environment:
- TZ=Asia/Shanghai
- DEBIAN_FRONTEND=noninteractive
- MASTER_ENDPOINT=${MASTER_ENDPOINT}
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
- AGENT_ENV=${AGENT_ENV}
- AGENT_USER=${AGENT_USER}
- AGENT_INSTANCE=${AGENT_INSTANCE}
networks:
argus-sys-net:
aliases:
- ${AGENT_INSTANCE}.node.argus.com
volumes:
- ../private/argus/agent:/private/argus/agent

View File

@@ -0,0 +1,154 @@
# ARM Client-CPU 安装指南(client_arm)
本包用于在 ARM64 CPU 节点上部署 Argus 的最小客户端:
- `argus-agent`(负责注册/上报状态到 master)
- `node-exporter`(暴露该节点的系统指标)
> 注意:本 ARM 客户端包 **不包含 GPU/DCGM exporter 和 Fluent Bit**,仅用于 CPU 节点 metrics。
---
## 1. 前置条件
- 目标机器为 ARM64(例如:aarch64 服务器 / 节点);
- 已安装 Docker 和 docker compose(v2,支持 `docker compose` 命令);
- 能访问 master 所在网络,或已经通过 overlay 网络加入到与 server 相同的 `argus-sys-net` 中。
如采用 overlay 网络(Swarm 模式),需提前创建 `argus-sys-net`:
```bash
docker network create --driver overlay argus-sys-net || true
```
如仅使用本地 bridge 网络,可将 compose 中 `networks: argus-sys-net` 调整为本地自定义 network。
---
## 2. 解压和加载镜像
假设你已经将 `client_arm_YYYYMMDD.tar.gz` 拷贝到 ARM 节点:
```bash
tar -xzf client_arm_YYYYMMDD.tar.gz
cd client_arm/YYYYMMDD
```
加载 CPU node 镜像:
```bash
for f in images/*.tar.gz; do
gunzip -c "$f" | docker load
done
```
可以通过:
```bash
docker images | grep 'argus-sys-metric-test-node-arm64'
```
确认镜像已加载。
---
## 3. 配置 .env
进入 compose 目录:
```bash
cd compose
cp .env.example .env
```
需要填写的关键字段:
- `MASTER_ENDPOINT`
- 例如:`http://master.argus.com:3000` 或 `http://<Server-IP>:32300`
- `AGENT_ENV` / `AGENT_USER` / `AGENT_INSTANCE`
- 用于识别环境、用户和实例,在 UI 中区分节点:
- 示例:`AGENT_ENV=prod`, `AGENT_USER=ops`, `AGENT_INSTANCE=node001`
- `CPU_NODE_HOSTNAME`
- 节点在配置中的 hostname,一般与宿主 `hostname` 一致,例如 `swarm-metric-node-001`。
- `ARGUS_BUILD_UID` / `ARGUS_BUILD_GID`
- 用于挂载目录 `/private/argus/agent` 的权限映射。
例如:
```env
PKG_VERSION=20251124
NODE_CPU_IMAGE_TAG=argus-sys-metric-test-node-arm64:20251124
COMPOSE_PROJECT_NAME=argus-client-arm
MASTER_ENDPOINT=http://master.argus.com:3000
AGENT_ENV=dev2
AGENT_USER=yuyr
AGENT_INSTANCE=node001sX
CPU_NODE_HOSTNAME=swarm-metric-node-001
ARGUS_BUILD_UID=1000
ARGUS_BUILD_GID=1000
```
---
## 4. 准备挂载目录
包中已经预创建 `private/argus/agent` 目录:
- `../private/argus/agent`
可以根据需要设置所有者和权限:
```bash
cd ..
sudo chown -R $UID:$GID private/argus
```
确保与 `.env` 中的 `ARGUS_BUILD_UID/GID` 一致。
---
## 5. 启动客户端容器
在 `compose/` 目录执行:
```bash
docker compose --env-file .env up -d
```
查看状态:
```bash
docker compose ps
```
预期看到:
- `argus-metric-cpu-node` 处于 `Up` 状态。
---
## 6. 在 server 端验证
在 server 侧(master + Prometheus + Grafana 已经通过 ARM server 包部署):
1. 在 master 的 UI 或 API 中:
- `GET /api/v1/master/nodes` 应能看到新节点,`name=CPU_NODE_HOSTNAME`;
2. 在 Prometheus 中:
- `up{job="node"}` 中应有当前节点记录,且 `value=1`;
- 可以通过 `node_load1{hostname="swarm-metric-node-001"}` 等表达式查看指标;
3. 在 Web 门户 / Grafana:
- 节点列表页面中能看到该节点;
- 通过“按 hostname 的节点面板”查看对应指标。
---
## 7. 注意事项
- ARM 客户端包只负责“CPU 节点 metrics”,不包含 GPU/DCGM 或 Fluent Bit;
- 如需要 GPU 支持,请继续使用 `client_gpu` 包,并部署在 x86_64 + NVIDIA GPU 节点上;
- ARM 节点的网络连通性(到 master / Prometheus)需要运维侧提前规划;
本包不做网络路由/防火墙配置,仅假定可以直接访问 `MASTER_ENDPOINT`。

View File

@@ -0,0 +1,140 @@
#!/usr/bin/env bash
set -euo pipefail
# ARM Client-CPU 配置脚本:
# - 读取 server 侧生成的 cluster-info.env(包含 SWARM_MANAGER_ADDR / SWARM_JOIN_TOKEN_WORKER)
# - 加入 Swarm(worker)
# - 预热 overlay 网络,确保能通过 master.argus.com 访问 server
# - 生成/更新 compose/.env(填充 Swarm 相关字段,保留人工填写的 AGENT_*、CPU_NODE_HOSTNAME 等)
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_EX="$PKG_ROOT/compose/.env.example"
ENV_OUT="$PKG_ROOT/compose/.env"
info(){ echo -e "\033[34m[CONFIG-ARM-CLIENT]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
# Compose 检测:优先 docker compose(v2),回退 docker-compose(v1)
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker curl jq awk sed tar gzip
require_compose
[[ -f "$ENV_EX" ]] || { err "缺少模板文件: $ENV_EX"; exit 1; }
# 磁盘空间检查(MB)
check_disk(){ local p="$1"; local need=5120; local free
free=$(df -Pm "$p" | awk 'NR==2{print $4+0}')
if [[ -z "$free" || "$free" -lt "$need" ]]; then err "磁盘空间不足: $p 剩余 ${free:-0}MB (<${need}MB)"; fi
}
check_disk "$PKG_ROOT" || true
check_disk "/var/lib/docker" || true
# 导入 cluster-info.env(默认取当前包根,也可用 CLUSTER_INFO 指定路径)
CI_IN="${CLUSTER_INFO:-$PKG_ROOT/cluster-info.env}"
NET_NAME="${ARGUS_OVERLAY_NET:-argus-sys-net}"
SWARM_AVAILABLE=true
if [[ -f "$CI_IN" ]]; then
info "读取 cluster-info.env: $CI_IN"
set -a; source "$CI_IN"; set +a
# 尝试加入 Swarm(幂等),允许 join-token 为空
if [[ -n "${SWARM_MANAGER_ADDR:-}" && -n "${SWARM_JOIN_TOKEN_WORKER:-}" ]]; then
info "尝试加入 Swarm(幂等):$SWARM_MANAGER_ADDR"
docker swarm join --token "$SWARM_JOIN_TOKEN_WORKER" "$SWARM_MANAGER_ADDR":2377 >/dev/null 2>&1 || true
fi
# 检查当前节点 Swarm 状态
if ! docker info 2>/dev/null | grep -q "Swarm: active"; then
info "检测到当前节点 Swarm 未启用,client_arm 将退回本地 network 模式"
SWARM_AVAILABLE=false
fi
else
info "未找到 cluster-info.env($CI_IN),client_arm 将以本地 network 模式运行"
SWARM_AVAILABLE=false
fi
# 生成/更新 .env(保留人工填写项,不覆盖已有键)
if [[ ! -f "$ENV_OUT" ]]; then
cp "$ENV_EX" "$ENV_OUT"
fi
set_kv(){ local k="$1" v="$2"; if grep -q "^${k}=" "$ENV_OUT"; then sed -i -E "s#^${k}=.*#${k}=${v}#" "$ENV_OUT"; else echo "${k}=${v}" >> "$ENV_OUT"; fi }
if [[ "$SWARM_AVAILABLE" == true ]]; then
# ===== Swarm 模式:强依赖 overlay 网络和 DNS alias =====
# 导入 busybox 并做 overlay 预热与连通性
if ! docker image inspect busybox:latest >/dev/null 2>&1; then
if [[ -f "$PKG_ROOT/images/busybox.tar" ]]; then
info "加载 busybox.tar 以预热 overlay"
docker load -i "$PKG_ROOT/images/busybox.tar" >/dev/null
else
err "缺少 busybox 镜像(包内 images/busybox.tar 或本地 busybox:latest),无法预热 overlay $NET_NAME"; exit 1
fi
fi
docker rm -f argus-net-warmup >/dev/null 2>&1 || true
info "启动 warmup 容器加入 overlay: $NET_NAME"
docker run -d --rm --name argus-net-warmup --network "$NET_NAME" busybox:latest sleep 600 >/dev/null 2>&1 || true
for i in {1..60}; do docker network inspect "$NET_NAME" >/dev/null 2>&1 && { info "overlay 可见 (t=${i}s)"; break; }; sleep 1; done
docker network inspect "$NET_NAME" >/dev/null 2>&1 || { err "预热后仍未看到 overlay: $NET_NAME;请确认 server 侧 overlay 已创建且可达"; exit 1; }
if ! docker exec argus-net-warmup sh -lc "ping -c 1 -W 2 master.argus.com >/dev/null 2>&1"; then
err "warmup 容器内无法通过别名访问 master.argus.com;请确认 server compose 已启动并加入 overlay $NET_NAME"
exit 1
fi
info "warmup 容器内可达 master.argus.com(Docker DNS + alias 正常)"
# 写入与 Swarm 相关的字段,便于后续诊断
set_kv SWARM_MANAGER_ADDR "${SWARM_MANAGER_ADDR:-}"
set_kv SWARM_JOIN_TOKEN_WORKER "${SWARM_JOIN_TOKEN_WORKER:-}"
set_kv SWARM_JOIN_TOKEN_MANAGER "${SWARM_JOIN_TOKEN_MANAGER:-}"
# 若未显式设置 MASTER_ENDPOINT,则默认走 overlay 别名
if ! grep -q '^MASTER_ENDPOINT=' "$ENV_OUT"; then
echo "MASTER_ENDPOINT=http://master.argus.com:3000" >> "$ENV_OUT"
fi
else
# ===== 本地 network 退化模式:不依赖 Swarm 和 overlay =====
if ! docker network inspect "$NET_NAME" >/dev/null 2>&1; then
info "创建本地 bridge 网络: $NET_NAME(client_arm 退化模式)"
docker network create "$NET_NAME" >/dev/null
else
info "本地网络已存在: $NET_NAME"
fi
# 若未显式设置 MASTER_ENDPOINT,则提示用户后续手动配置
if ! grep -q '^MASTER_ENDPOINT=' "$ENV_OUT"; then
echo "MASTER_ENDPOINT=" >> "$ENV_OUT"
fi
ME=$(grep -E '^MASTER_ENDPOINT=' "$ENV_OUT" | head -1 | cut -d= -f2-)
if [[ -z "$ME" ]]; then
err "本地 network 模式下必须配置 MASTER_ENDPOINT(示例:http://<Server-IP>:<Port>),请编辑 compose/.env 后重试"
else
info "当前为本地 network 模式,metric-cpu-node 将通过 MASTER_ENDPOINT=${ME} 访问 master"
fi
fi
# 检查用户必须填写的字段
REQ_VARS=(AGENT_ENV AGENT_USER AGENT_INSTANCE CPU_NODE_HOSTNAME)
missing=()
for v in "${REQ_VARS[@]}"; do
val=$(grep -E "^$v=" "$ENV_OUT" | head -1 | cut -d= -f2-)
if [[ -z "$val" ]]; then missing+=("$v"); fi
done
if [[ ${#missing[@]} -gt 0 ]]; then
err "以下变量必须在 compose/.env 中填写:${missing[*]}(已保留你现有的内容,不会被覆盖)"
exit 1
fi
info "已生成 compose/.env;可执行 scripts/install.sh 启动 ARM Client-CPU"

View File

@@ -0,0 +1,48 @@
#!/usr/bin/env bash
set -euo pipefail
# ARM Client-CPU 安装脚本:
# - 导入 ARM CPU node 镜像
# - 启动 compose 中的 metric-cpu-node 服务
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
info(){ echo -e "\033[34m[INSTALL-ARM-CLIENT]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker
require_compose
[[ -f "$ENV_FILE" ]] || { err "缺少 compose/.env,请先运行 scripts/config.sh"; exit 1; }
info "使用环境文件: $ENV_FILE"
set -a; source "$ENV_FILE"; set +a
# 导入 ARM CPU node 镜像
IMG_TGZ=$(ls -1 "$PKG_ROOT"/images/argus-sys-metric-test-node-arm64-*.tar.gz 2>/dev/null | head -1 || true)
[[ -n "$IMG_TGZ" ]] || { err "找不到 ARM CPU node 镜像 tar.gz(argus-sys-metric-test-node-arm64-*.tar.gz)"; exit 1; }
info "导入 ARM CPU node 镜像: $(basename "$IMG_TGZ")"
tmp=$(mktemp); gunzip -c "$IMG_TGZ" > "$tmp"; docker load -i "$tmp" >/dev/null; rm -f "$tmp"
# 确保 agent 挂载目录存在
mkdir -p "$PKG_ROOT/private/argus/agent"
# 启动 compose
PROJECT="${COMPOSE_PROJECT_NAME:-argus-client-arm}"
info "启动 ARM CPU 节点 (docker compose -p $PROJECT up -d)"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" up -d
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" ps
info "ARM Client-CPU 安装完成。可通过 docker logs -f argus-metric-cpu-node 查看节点容器日志。"

View File

@@ -0,0 +1,28 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
if [[ -f "$ENV_FILE" ]]; then
set -a; source "$ENV_FILE"; set +a
fi
PROJECT="${COMPOSE_PROJECT_NAME:-argus-client-arm}"
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require_compose
echo "[UNINSTALL-ARM-CLIENT] stopping compose (project=$PROJECT)"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" down --remove-orphans || true
echo "[UNINSTALL-ARM-CLIENT] done"

View File

@@ -0,0 +1,125 @@
#!/usr/bin/env bash
set -euo pipefail
# x86 上安装 ARM Client-CPU 的配置脚本:
# 逻辑与 ARM 版基本一致(读取 cluster-info、加入 Swarm、预热 overlay、生成 .env),只是日志前缀不同。
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_EX="$PKG_ROOT/compose/.env.example"
ENV_OUT="$PKG_ROOT/compose/.env"
info(){ echo -e "\033[34m[CONFIG-ARM-CLIENT-X86]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker curl jq awk sed tar gzip
require_compose
[[ -f "$ENV_EX" ]] || { err "缺少模板文件: $ENV_EX"; exit 1; }
# 磁盘空间检查
check_disk(){ local p="$1"; local need=5120; local free
free=$(df -Pm "$p" | awk 'NR==2{print $4+0}')
if [[ -z "$free" || "$free" -lt "$need" ]]; then err "磁盘空间不足: $p 剩余 ${free:-0}MB (<${need}MB)"; fi
}
check_disk "$PKG_ROOT" || true
check_disk "/var/lib/docker" || true
CI_IN="${CLUSTER_INFO:-$PKG_ROOT/cluster-info.env}"
NET_NAME="${ARGUS_OVERLAY_NET:-argus-sys-net}"
SWARM_AVAILABLE=true
if [[ -f "$CI_IN" ]]; then
info "读取 cluster-info.env: $CI_IN"
set -a; source "$CI_IN"; set +a
if [[ -n "${SWARM_MANAGER_ADDR:-}" && -n "${SWARM_JOIN_TOKEN_WORKER:-}" ]]; then
info "加入 Swarm(幂等):$SWARM_MANAGER_ADDR"
docker swarm join --token "$SWARM_JOIN_TOKEN_WORKER" "$SWARM_MANAGER_ADDR":2377 >/dev/null 2>&1 || true
fi
if ! docker info 2>/dev/null | grep -q "Swarm: active"; then
info "检测到当前节点 Swarm 未启用,client_arm 将退回本地 network 模式"
SWARM_AVAILABLE=false
fi
else
info "未找到 cluster-info.env($CI_IN),client_arm 将以本地 network 模式运行"
SWARM_AVAILABLE=false
fi
if [[ ! -f "$ENV_OUT" ]]; then
cp "$ENV_EX" "$ENV_OUT"
fi
set_kv(){ local k="$1" v="$2"; if grep -q "^${k}=" "$ENV_OUT"; then sed -i -E "s#^${k}=.*#${k}=${v}#" "$ENV_OUT"; else echo "${k}=${v}" >> "$ENV_OUT"; fi }
if [[ "$SWARM_AVAILABLE" == true ]]; then
if ! docker image inspect busybox:latest >/dev/null 2>&1; then
if [[ -f "$PKG_ROOT/images/busybox.tar" ]]; then
info "加载 busybox.tar 以预热 overlay"
docker load -i "$PKG_ROOT/images/busybox.tar" >/dev/null
else
err "缺少 busybox 镜像(包内 images/busybox.tar 或本地 busybox:latest),无法预热 overlay $NET_NAME"; exit 1
fi
fi
docker rm -f argus-net-warmup >/dev/null 2>&1 || true
info "启动 warmup 容器加入 overlay: $NET_NAME"
docker run -d --rm --name argus-net-warmup --network "$NET_NAME" busybox:latest sleep 600 >/dev/null 2>&1 || true
for i in {1..60}; do docker network inspect "$NET_NAME" >/dev/null 2>&1 && { info "overlay 可见 (t=${i}s)"; break; }; sleep 1; done
docker network inspect "$NET_NAME" >/dev/null 2>&1 || { err "预热后仍未看到 overlay: $NET_NAME;请确认 server 侧 overlay 已创建且可达"; exit 1; }
if ! docker exec argus-net-warmup sh -lc "ping -c 1 -W 2 master.argus.com >/dev/null 2>&1"; then
err "warmup 容器内无法通过别名访问 master.argus.com;请确认 server compose 已启动并加入 overlay $NET_NAME"
exit 1
fi
info "warmup 容器内可达 master.argus.com(Docker DNS + alias 正常)"
set_kv SWARM_MANAGER_ADDR "${SWARM_MANAGER_ADDR:-}"
set_kv SWARM_JOIN_TOKEN_WORKER "${SWARM_JOIN_TOKEN_WORKER:-}"
set_kv SWARM_JOIN_TOKEN_MANAGER "${SWARM_JOIN_TOKEN_MANAGER:-}"
if ! grep -q '^MASTER_ENDPOINT=' "$ENV_OUT"; then
echo "MASTER_ENDPOINT=http://master.argus.com:3000" >> "$ENV_OUT"
fi
else
if ! docker network inspect "$NET_NAME" >/dev/null 2>&1; then
info "创建本地 bridge 网络: $NET_NAME(client_arm 退化模式)"
docker network create "$NET_NAME" >/dev/null
else
info "本地网络已存在: $NET_NAME"
fi
if ! grep -q '^MASTER_ENDPOINT=' "$ENV_OUT"; then
echo "MASTER_ENDPOINT=" >> "$ENV_OUT"
fi
ME=$(grep -E '^MASTER_ENDPOINT=' "$ENV_OUT" | head -1 | cut -d= -f2-)
if [[ -z "$ME" ]]; then
err "本地 network 模式下必须配置 MASTER_ENDPOINT(示例:http://<Server-IP>:<Port>),请编辑 compose/.env 后重试"
else
info "当前为本地 network 模式,metric-cpu-node 将通过 MASTER_ENDPOINT=${ME} 访问 master"
fi
fi
REQ_VARS=(AGENT_ENV AGENT_USER AGENT_INSTANCE CPU_NODE_HOSTNAME)
missing=()
for v in "${REQ_VARS[@]}"; do
val=$(grep -E "^$v=" "$ENV_OUT" | head -1 | cut -d= -f2-)
if [[ -z "$val" ]]; then missing+=("$v"); fi
done
if [[ ${#missing[@]} -gt 0 ]]; then
err "以下变量必须在 compose/.env 中填写:${missing[*]}(已保留你现有的内容,不会被覆盖)"
exit 1
fi
info "已生成 compose/.env;可执行 scripts_for_x86/install.sh 启动 ARM Client-CPU"

View File

@@ -0,0 +1,76 @@
#!/usr/bin/env bash
set -euo pipefail
# 在 x86_64 主机上安装 ARM Client-CPU:
# 1) 确认/安装 binfmt + QEMU(通过 tonistiigi/binfmt);
# 2) 调用通用的 scripts/install.sh 启动 ARM CPU node 容器。
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
info(){ echo -e "\033[34m[INSTALL-ARM-CLIENT-X86]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR-ARM-CLIENT-X86]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
ensure_binfmt_arm64() {
info "检查当前主机架构: $(uname -m)"
if [[ "$(uname -m)" != "x86_64" && "$(uname -m)" != "amd64" ]]; then
info "当前并非 x86_64($(uname -m)),通常应直接在 ARM 服务器上使用 scripts/install.sh。继续执行但不强制安装 binfmt。"
return 0
fi
require docker
# 优先使用随安装包一起提供的 ubuntu-22.04-arm-test 镜像做自检
local TEST_IMG="ubuntu:22.04"
local TEST_TAR=""
if [[ -d "$ROOT_DIR/images" ]]; then
TEST_TAR=$(ls "$ROOT_DIR"/images/ubuntu-22.04-arm-test*.tar.gz 2>/dev/null | head -n1 || true)
fi
if [[ -n "$TEST_TAR" && ! $(docker image inspect "$TEST_IMG" >/dev/null 2>&1; echo $?) -eq 0 ]]; then
info "从安装包加载 ARM 测试基础镜像: $(basename "$TEST_TAR")"
tmp=$(mktemp)
gunzip -c "$TEST_TAR" > "$tmp"
docker load -i "$tmp" >/dev/null
rm -f "$tmp"
fi
info "验证是否已能运行 linux/arm64 容器..."
if docker run --rm --platform=linux/arm64 "$TEST_IMG" uname -m 2>/dev/null | grep -q 'aarch64'; then
info "检测到本机已支持 ARM64 容器运行 (uname -m = aarch64)"
return 0
fi
info "未检测到 ARM64 运行能力,尝试通过 tonistiigi/binfmt 安装 binfmt/QEMU ..."
local BINFMT_IMG="tonistiigi/binfmt:latest"
local BINFMT_TAR=""
if [[ -d "$ROOT_DIR/images" ]]; then
BINFMT_TAR=$(ls "$ROOT_DIR"/images/tonistiigi-binfmt*.tar.gz 2>/dev/null | head -n1 || true)
fi
if [[ -n "$BINFMT_TAR" && ! $(docker image inspect "$BINFMT_IMG" >/dev/null 2>&1; echo $?) -eq 0 ]]; then
info "从安装包加载 tonistiigi/binfmt 离线镜像: $(basename "$BINFMT_TAR")"
tmp_b=$(mktemp)
gunzip -c "$BINFMT_TAR" > "$tmp_b"
docker load -i "$tmp_b" >/dev/null
rm -f "$tmp_b"
fi
if ! docker run --privileged --rm "$BINFMT_IMG" --install all; then
err "tonistiigi/binfmt 安装失败,请检查 Docker 权限或确认离线镜像是否完整。"
exit 1
fi
info "再次验证 ARM64 容器可运行状态..."
if ! docker run --rm --platform=linux/arm64 "$TEST_IMG" uname -m 2>/dev/null | grep -q 'aarch64'; then
err "安装 binfmt/QEMU 后仍无法运行 linux/arm64 容器,请手工排查 binfmt_misc 配置。"
exit 1
fi
info "ARM64 容器运行能力检查通过。"
}
ensure_binfmt_arm64
exec "$SCRIPT_DIR/../scripts/install.sh" "$@"

View File

@@ -0,0 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
exec "$SCRIPT_DIR/../scripts/uninstall.sh" "$@"

View File

@@ -0,0 +1,38 @@
version: "3.8"
networks:
argus-sys-net:
external: true
services:
metric-gpu-node:
image: ${NODE_GPU_BUNDLE_IMAGE_TAG:-argus-sys-metric-test-node-bundle-gpu:${PKG_VERSION}}
container_name: argus-metric-gpu-node-swarm
hostname: ${GPU_NODE_HOSTNAME}
restart: unless-stopped
privileged: true
runtime: nvidia
environment:
- TZ=Asia/Shanghai
- DEBIAN_FRONTEND=noninteractive
- MASTER_ENDPOINT=${MASTER_ENDPOINT:-http://master.argus.com:3000}
# Fluent Bit / 日志上报目标(固定域名)
- ES_HOST=es.log.argus.com
- ES_PORT=9200
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
- AGENT_ENV=${AGENT_ENV}
- AGENT_USER=${AGENT_USER}
- AGENT_INSTANCE=${AGENT_INSTANCE}
- NVIDIA_VISIBLE_DEVICES=all
- NVIDIA_DRIVER_CAPABILITIES=compute,utility
- GPU_MODE=gpu
networks:
argus-sys-net:
aliases:
- ${AGENT_INSTANCE}.node.argus.com
volumes:
- ../private/argus/agent:/private/argus/agent
- ../logs/infer:/logs/infer
- ../logs/train:/logs/train
command: ["sleep", "infinity"]

View File

@@ -0,0 +1,73 @@
# Argus Client‑GPU 安装指南(deployment_new)
## 一、准备条件(开始前确认)
- GPU 节点安装了 NVIDIA 驱动,`nvidia-smi` 正常;
- Docker & Docker Compose v2 已安装;
- 使用统一账户 `argus`(UID=2133,GID=2015)执行安装,并加入 `docker` 组(如已创建可跳过):
```bash
sudo groupadd --gid 2015 argus || true
sudo useradd --uid 2133 --gid 2015 --create-home --shell /bin/bash argus || true
sudo passwd argus
sudo usermod -aG docker argus
su - argus -c 'id; docker ps >/dev/null && echo OK || echo NO_DOCKER_PERMISSION'
```
后续解压与执行(config/install/uninstall)均使用 `argus` 账户进行。
- 从 Server 安装方拿到 `cluster-info.env`(包含 `SWARM_MANAGER_ADDR/SWARM_JOIN_TOKEN_*`;compose 架构下 BINDIP/FTPIP 不再使用)。
## 二、解包
- `tar -xzf client_gpu_YYYYMMDD.tar.gz`
- 进入目录:`cd client_gpu_YYYYMMDD/`
- 你应当看到:`images/`(GPU bundle、busybox)、`compose/`、`scripts/`、`docs/`。
## 三、配置 config(预热 overlay + 生成 .env)
命令:
```
cp /path/to/cluster-info.env ./ # 或 export CLUSTER_INFO=/abs/path/cluster-info.env
./scripts/config.sh
```
脚本做了什么:
- 读取 `cluster-info.env` 并 `docker swarm join`(幂等);
- 自动用 busybox 预热 external overlay `argus-sys-net`,等待最多 60s 直到本机可见;
- 生成/更新 `compose/.env`:填入 `SWARM_*`,并“保留你已填写的 AGENT_* 与 GPU_NODE_HOSTNAME”(不会覆盖)。
看到什么才算成功:
- 终端输出类似:`已预热 overlay=argus-sys-net 并生成 compose/.env;可执行 scripts/install.sh`;
- `compose/.env` 至少包含:
- `AGENT_ENV/AGENT_USER/AGENT_INSTANCE/GPU_NODE_HOSTNAME`(需要你提前填写);
- `SWARM_MANAGER_ADDR/SWARM_JOIN_TOKEN_*`;
- `NODE_GPU_BUNDLE_IMAGE_TAG=...:YYYYMMDD`。
### 日志映射(重要)
- 容器内 `/logs/infer` 与 `/logs/train` 已映射到包根 `./logs/infer` 与 `./logs/train`:
- 你可以直接在宿主机查看推理/训练日志:`tail -f logs/infer/*.log`、`tail -f logs/train/*.log`;
- install 脚本会自动创建这两个目录。
若提示缺少必填项:
- 打开 `compose/.env` 按提示补齐 `AGENT_*` 与 `GPU_NODE_HOSTNAME`,再次执行 `./scripts/config.sh`(脚本不会覆盖你已填的值)。
## 四、安装 install(加载镜像 + 起容器 + 跟日志)
命令:
```
./scripts/install.sh
```
脚本做了什么:
- 如有必要,先自动预热 overlay;
- 从 `images/` 导入 `argus-sys-metric-test-node-bundle-gpu-*.tar.gz` 到本地 Docker;
- `docker compose up -d` 启动 GPU 节点容器,并自动执行 `docker logs -f argus-metric-gpu-node-swarm` 跟踪安装过程。
看到什么才算成功:
- 日志中出现:`[BOOT] local bundle install OK: version=...` / `dcgm-exporter ... listening` / `node state present: /private/argus/agent/<hostname>/node.json`;
- `docker exec argus-metric-gpu-node-swarm nvidia-smi -L` 能列出 GPU;
- 在 Server 侧 Prometheus `/api/v1/targets` 中,GPU 节点 9100(node-exporter)与 9400(dcgm-exporter)至少其一 up。
## 五、卸载 uninstall
命令:
```
./scripts/uninstall.sh
```
行为:Compose down(如有 .env),并删除 warmup 容器与节点容器。
## 六、常见问题
- `本机未看到 overlay`:config/install 已自动预热;若仍失败,请检查与 manager 的网络连通性以及 manager 上是否已创建 `argus-sys-net`。
- `busybox 缺失`:确保包根 `images/busybox.tar` 在,或主机已有 `busybox:latest`。
- `加入 Swarm 失败`:确认 `cluster-info.env` 的 `SWARM_MANAGER_ADDR` 与 `SWARM_JOIN_TOKEN_WORKER` 正确,或在 manager 上重新 `docker swarm join-token -q worker` 后更新该文件。

Binary file not shown.

View File

@@ -0,0 +1,90 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_EX="$PKG_ROOT/compose/.env.example"
ENV_OUT="$PKG_ROOT/compose/.env"
info(){ echo -e "\033[34m[CONFIG-GPU]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
# Compose 检测:优先 docker compose(v2),回退 docker-compose(v1)
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker curl jq awk sed tar gzip
require_compose
# 磁盘空间检查(MB)
check_disk(){ local p="$1"; local need=10240; local free
free=$(df -Pm "$p" | awk 'NR==2{print $4+0}')
if [[ -z "$free" || "$free" -lt "$need" ]]; then err "磁盘空间不足: $p 剩余 ${free:-0}MB (<${need}MB)"; return 1; fi
}
check_disk "$PKG_ROOT"; check_disk "/var/lib/docker" || true
# 导入 cluster-info.env(默认取当前包根,也可用 CLUSTER_INFO 指定路径)
CI_IN="${CLUSTER_INFO:-$PKG_ROOT/cluster-info.env}"
info "读取 cluster-info.env: $CI_IN"
[[ -f "$CI_IN" ]] || { err "找不到 cluster-info.env(默认当前包根,或设置环境变量 CLUSTER_INFO 指定绝对路径)"; exit 1; }
set -a; source "$CI_IN"; set +a
[[ -n "${SWARM_MANAGER_ADDR:-}" && -n "${SWARM_JOIN_TOKEN_WORKER:-}" ]] || { err "cluster-info.env 缺少 SWARM 信息(SWARM_MANAGER_ADDR/SWARM_JOIN_TOKEN_WORKER)"; exit 1; }
# 加入 Swarm(幂等)
info "加入 Swarm(幂等):$SWARM_MANAGER_ADDR"
docker swarm join --token "$SWARM_JOIN_TOKEN_WORKER" "$SWARM_MANAGER_ADDR":2377 >/dev/null 2>&1 || true
# 导入 busybox 并做 overlay 预热与连通性(总是执行)
NET_NAME="${ARGUS_OVERLAY_NET:-argus-sys-net}"
# 准备 busybox
if ! docker image inspect busybox:latest >/dev/null 2>&1; then
if [[ -f "$PKG_ROOT/images/busybox.tar" ]]; then
info "加载 busybox.tar 以预热 overlay"
docker load -i "$PKG_ROOT/images/busybox.tar" >/dev/null
else
err "缺少 busybox 镜像(包内 images/busybox.tar 或本地 busybox:latest),无法预热 overlay $NET_NAME"; exit 1
fi
fi
# 预热容器(worker 侧加入 overlay 以便本地可见)
docker rm -f argus-net-warmup >/dev/null 2>&1 || true
info "启动 warmup 容器加入 overlay: $NET_NAME"
docker run -d --rm --name argus-net-warmup --network "$NET_NAME" busybox:latest sleep 600 >/dev/null 2>&1 || true
for i in {1..60}; do docker network inspect "$NET_NAME" >/dev/null 2>&1 && { info "overlay 可见 (t=${i}s)"; break; }; sleep 1; done
docker network inspect "$NET_NAME" >/dev/null 2>&1 || { err "预热后仍未看到 overlay: $NET_NAME;请确认 manager 已创建并网络可达"; exit 1; }
# 通过 warmup 容器测试实际数据通路(alias → master)
if ! docker exec argus-net-warmup sh -lc "ping -c 1 -W 2 master.argus.com >/dev/null 2>&1"; then
err "warmup 容器内无法通过别名访问 master.argus.com;请确认 server compose 已启动并加入 overlay $NET_NAME"
exit 1
fi
info "warmup 容器内可达 master.argus.com(Docker DNS + alias 正常)"
# 生成/更新 .env(保留人工填写项,不覆盖已有键)
if [[ ! -f "$ENV_OUT" ]]; then
cp "$ENV_EX" "$ENV_OUT"
fi
set_kv(){ local k="$1" v="$2"; if grep -q "^${k}=" "$ENV_OUT"; then sed -i -E "s#^${k}=.*#${k}=${v}#" "$ENV_OUT"; else echo "${k}=${v}" >> "$ENV_OUT"; fi }
set_kv SWARM_MANAGER_ADDR "${SWARM_MANAGER_ADDR:-}"
set_kv SWARM_JOIN_TOKEN_WORKER "${SWARM_JOIN_TOKEN_WORKER:-}"
set_kv SWARM_JOIN_TOKEN_MANAGER "${SWARM_JOIN_TOKEN_MANAGER:-}"
REQ_VARS=(AGENT_ENV AGENT_USER AGENT_INSTANCE GPU_NODE_HOSTNAME)
missing=()
for v in "${REQ_VARS[@]}"; do
val=$(grep -E "^$v=" "$ENV_OUT" | head -1 | cut -d= -f2-)
if [[ -z "$val" ]]; then missing+=("$v"); fi
done
if [[ ${#missing[@]} -gt 0 ]]; then
err "以下变量必须在 compose/.env 中填写:${missing[*]}(已保留你现有的内容,不会被覆盖)"; exit 1; fi
info "已生成 compose/.env;可执行 scripts/install.sh"
# 准备并赋权宿主日志目录(幂等,便于安装前人工检查/预创建)
mkdir -p "$PKG_ROOT/logs/train" "$PKG_ROOT/logs/infer"
chmod 1777 "$PKG_ROOT/logs/train" "$PKG_ROOT/logs/infer" || true
info "日志目录权限(期待 1777,含粘滞位):"
stat -c '%a %U:%G %n' "$PKG_ROOT/logs/train" "$PKG_ROOT/logs/infer" 2>/dev/null || true

View File

@@ -0,0 +1,72 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
info(){ echo -e "\033[34m[INSTALL-GPU]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
# Compose 检测:优先 docker compose(v2),回退 docker-compose(v1)
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker nvidia-smi
require_compose
[[ -f "$ENV_FILE" ]] || { err "缺少 compose/.env,请先运行 scripts/config.sh"; exit 1; }
info "使用环境文件: $ENV_FILE"
# 预热 overlay(当 config 执行很久之前或容器已被清理时,warmup 可能不存在)
set -a; source "$ENV_FILE"; set +a
NET_NAME="${ARGUS_OVERLAY_NET:-argus-sys-net}"
info "检查 overlay 网络可见性: $NET_NAME"
if ! docker network inspect "$NET_NAME" >/dev/null 2>&1; then
# 如 Overlay 不可见,尝试用 busybox 预热(仅为确保 worker 节点已加入 overlay)
if ! docker image inspect busybox:latest >/dev/null 2>&1; then
if [[ -f "$PKG_ROOT/images/busybox.tar" ]]; then docker load -i "$PKG_ROOT/images/busybox.tar"; else err "缺少 busybox 镜像(images/busybox.tar 或本地 busybox:latest)"; exit 1; fi
fi
docker rm -f argus-net-warmup >/dev/null 2>&1 || true
docker run -d --rm --name argus-net-warmup --network "$NET_NAME" busybox:latest sleep 600 >/dev/null 2>&1 || true
for i in {1..60}; do docker network inspect "$NET_NAME" >/dev/null 2>&1 && break; sleep 1; done
docker network inspect "$NET_NAME" >/dev/null 2>&1 || { err "预热后仍未看到 overlay: $NET_NAME;请确认 manager 已创建并网络可达"; exit 1; }
info "overlay 已可见(warmup=argus-net-warmup)"
fi
# 若本函数内重新创建了 warmup 容器,同样测试一次 alias 数据通路
if docker ps --format '{{.Names}}' | grep -q '^argus-net-warmup$'; then
if ! docker exec argus-net-warmup sh -lc "ping -c 1 -W 2 master.argus.com >/dev/null 2>&1"; then
err "GPU install 阶段:warmup 容器内无法通过别名访问 master.argus.com;请检查 overlay $NET_NAME 与 server 状态"
exit 1
fi
info "GPU install 阶段:warmup 容器内可达 master.argus.com"
fi
# 导入 GPU bundle 镜像
IMG_TGZ=$(ls -1 "$PKG_ROOT"/images/argus-sys-metric-test-node-bundle-gpu-*.tar.gz 2>/dev/null | head -1 || true)
[[ -n "$IMG_TGZ" ]] || { err "找不到 GPU bundle 镜像 tar.gz"; exit 1; }
info "导入 GPU bundle 镜像: $(basename "$IMG_TGZ")"
tmp=$(mktemp); gunzip -c "$IMG_TGZ" > "$tmp"; docker load -i "$tmp" >/dev/null; rm -f "$tmp"
# 确保日志目录存在(宿主侧,用于映射 /logs/infer 与 /logs/train),并赋权 1777(粘滞位)
mkdir -p "$PKG_ROOT/logs/infer" "$PKG_ROOT/logs/train"
chmod 1777 "$PKG_ROOT/logs/infer" "$PKG_ROOT/logs/train" || true
info "日志目录已准备并赋权 1777: logs/infer logs/train"
stat -c '%a %U:%G %n' "$PKG_ROOT/logs/infer" "$PKG_ROOT/logs/train" 2>/dev/null || true
# 启动 compose 并跟踪日志
PROJECT="${COMPOSE_PROJECT_NAME:-argus-client}"
info "启动 GPU 节点 (docker compose -p $PROJECT up -d)"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" up -d
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" ps
# 再次校准宿主日志目录权限,避免容器内脚本对 bind mount 权限回退
chmod 1777 "$PKG_ROOT/logs/infer" "$PKG_ROOT/logs/train" || true
stat -c '%a %U:%G %n' "$PKG_ROOT/logs/infer" "$PKG_ROOT/logs/train" 2>/dev/null || true
info "跟踪节点容器日志(按 Ctrl+C 退出)"
docker logs -f argus-metric-gpu-node-swarm || true

View File

@@ -0,0 +1,36 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
# load COMPOSE_PROJECT_NAME if provided in compose/.env
if [[ -f "$ENV_FILE" ]]; then set -a; source "$ENV_FILE"; set +a; fi
PROJECT="${COMPOSE_PROJECT_NAME:-argus-client}"
info(){ echo -e "\033[34m[UNINSTALL-GPU]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
# Compose 检测:优先 docker compose(v2),回退 docker-compose(v1)
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require_compose
if [[ -f "$ENV_FILE" ]]; then
info "stopping compose project (project=$PROJECT)"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" down --remove-orphans || true
else
info "compose/.env not found; attempting to remove container by name"
fi
# remove warmup container if still running
docker rm -f argus-net-warmup >/dev/null 2>&1 || true
# remove node container if present
docker rm -f argus-metric-gpu-node-swarm >/dev/null 2>&1 || true
info "uninstall completed"

View File

@@ -0,0 +1,169 @@
version: "3.8"
networks:
argus-sys-net:
external: true
services:
master:
image: ${MASTER_IMAGE_TAG:-argus-master:${PKG_VERSION}}
container_name: argus-master-sys
environment:
- OFFLINE_THRESHOLD_SECONDS=180
- ONLINE_THRESHOLD_SECONDS=120
- SCHEDULER_INTERVAL_SECONDS=30
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
ports:
- "${MASTER_PORT:-32300}:3000"
volumes:
- ../private/argus/master:/private/argus/master
- ../private/argus/metric/prometheus:/private/argus/metric/prometheus
- ../private/argus/etc:/private/argus/etc
networks:
argus-sys-net:
aliases:
- master.argus.com
restart: unless-stopped
es:
image: ${ES_IMAGE_TAG:-argus-elasticsearch:${PKG_VERSION}}
container_name: argus-es-sys
environment:
- discovery.type=single-node
- xpack.security.enabled=false
- ES_JAVA_OPTS=-Xms512m -Xmx512m
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
volumes:
- ../private/argus/log/elasticsearch:/private/argus/log/elasticsearch
- ../private/argus/etc:/private/argus/etc
ports:
- "${ES_HTTP_PORT:-9200}:9200"
restart: unless-stopped
networks:
argus-sys-net:
aliases:
- es.log.argus.com
kibana:
image: ${KIBANA_IMAGE_TAG:-argus-kibana:${PKG_VERSION}}
container_name: argus-kibana-sys
environment:
- ELASTICSEARCH_HOSTS=http://es.log.argus.com:9200
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
volumes:
- ../private/argus/log/kibana:/private/argus/log/kibana
- ../private/argus/etc:/private/argus/etc
depends_on: [es]
ports:
- "${KIBANA_PORT:-5601}:5601"
restart: unless-stopped
networks:
argus-sys-net:
aliases:
- kibana.log.argus.com
prometheus:
image: ${PROM_IMAGE_TAG:-argus-metric-prometheus:${PKG_VERSION}}
container_name: argus-prometheus
restart: unless-stopped
environment:
- TZ=Asia/Shanghai
- PROMETHEUS_BASE_PATH=/private/argus/metric/prometheus
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
ports:
- "${PROMETHEUS_PORT:-9090}:9090"
volumes:
- ../private/argus/metric/prometheus:/private/argus/metric/prometheus
- ../private/argus/etc:/private/argus/etc
networks:
argus-sys-net:
aliases:
- prom.metric.argus.com
grafana:
image: ${GRAFANA_IMAGE_TAG:-argus-metric-grafana:${PKG_VERSION}}
container_name: argus-grafana
restart: unless-stopped
environment:
- TZ=Asia/Shanghai
- GRAFANA_BASE_PATH=/private/argus/metric/grafana
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
- GF_SERVER_HTTP_PORT=3000
- GF_LOG_LEVEL=warn
- GF_LOG_MODE=console
- GF_PATHS_PROVISIONING=/private/argus/metric/grafana/provisioning
- GF_AUTH_ANONYMOUS_ENABLED=true
- GF_AUTH_ANONYMOUS_ORG_ROLE=Viewer
ports:
- "${GRAFANA_PORT:-3000}:3000"
volumes:
- ../private/argus/metric/grafana:/private/argus/metric/grafana
- ../private/argus/etc:/private/argus/etc
depends_on: [prometheus]
networks:
argus-sys-net:
aliases:
- grafana.metric.argus.com
alertmanager:
image: ${ALERT_IMAGE_TAG:-argus-alertmanager:${PKG_VERSION}}
container_name: argus-alertmanager
environment:
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
volumes:
- ../private/argus/etc:/private/argus/etc
- ../private/argus/alert/alertmanager:/private/argus/alert/alertmanager
networks:
argus-sys-net:
aliases:
- alertmanager.alert.argus.com
ports:
- "${ALERTMANAGER_PORT:-9093}:9093"
restart: unless-stopped
web-frontend:
image: ${FRONT_IMAGE_TAG:-argus-web-frontend:${PKG_VERSION}}
container_name: argus-web-frontend
environment:
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
- EXTERNAL_MASTER_PORT=${WEB_PROXY_PORT_8085:-8085}
- EXTERNAL_ALERTMANAGER_PORT=${WEB_PROXY_PORT_8084:-8084}
- EXTERNAL_GRAFANA_PORT=${WEB_PROXY_PORT_8081:-8081}
- EXTERNAL_PROMETHEUS_PORT=${WEB_PROXY_PORT_8082:-8082}
- EXTERNAL_KIBANA_PORT=${WEB_PROXY_PORT_8083:-8083}
volumes:
- ../private/argus/etc:/private/argus/etc
networks:
argus-sys-net:
aliases:
- web.argus.com
restart: unless-stopped
web-proxy:
image: ${WEB_PROXY_IMAGE_TAG:-argus-web-proxy:${PKG_VERSION}}
container_name: argus-web-proxy
depends_on: [master, grafana, prometheus, kibana, alertmanager]
environment:
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
volumes:
- ../private/argus/etc:/private/argus/etc
networks:
argus-sys-net:
aliases:
- proxy.argus.com
ports:
- "${WEB_PROXY_PORT_8080:-8080}:8080"
- "${WEB_PROXY_PORT_8081:-8081}:8081"
- "${WEB_PROXY_PORT_8082:-8082}:8082"
- "${WEB_PROXY_PORT_8083:-8083}:8083"
- "${WEB_PROXY_PORT_8084:-8084}:8084"
- "${WEB_PROXY_PORT_8085:-8085}:8085"
restart: unless-stopped

View File

@@ -0,0 +1,102 @@
# Argus Server 安装指南(deployment_new)
适用:通过 Server 安装包在 Docker Swarm + external overlay 网络一体化部署 Argus 服务端组件。
—— 本文强调“怎么做、看什么、符合了才继续”。
## 一、准备条件(开始前确认)
- Docker 与 Docker Compose v2 已安装;`docker info` 正常;`docker compose version` 可执行。
- 具备 root/sudo 权限;磁盘可用空间 ≥ 10GB(包根与 `/var/lib/docker`)。
- 你知道本机管理地址(SWARM_MANAGER_ADDR),该 IP 属于本机某网卡,可被其他节点访问。
- 很重要:以统一账户 `argus`(UID=2133,GID=2015)执行后续安装与运维,并将其加入 `docker` 组;示例命令如下(如需不同 UID/GID,请替换为贵方标准):
```bash
# 1) 创建主组(GID=2015,组名 argus;若已存在可跳过)
sudo groupadd --gid 2015 argus || true
# 2) 创建用户 argus(UID=2133、主组 GID=2015,创建家目录并用 bash 作为默认 shell;若已存在可用 usermod 调整)
sudo useradd --uid 2133 --gid 2015 --create-home --shell /bin/bash argus || true
sudo passwd argus
# 3) 将 argus 加入 docker 组,使其能调用 Docker Daemon(新登录后生效)
sudo usermod -aG docker argus
# 4) 验证(重新登录或执行 newgrp docker 使组生效)
su - argus -c 'id; docker ps >/dev/null && echo OK || echo NO_DOCKER_PERMISSION'
```
后续的解压与执行(config/install/selfcheck 等)均使用该 `argus` 账户进行。
## 二、解包与目录结构
- 解压:`tar -xzf server_YYYYMMDD.tar.gz`。
- 进入:`cd server_YYYYMMDD/`
- 你应当能看到:
- `images/`(逐服务镜像 tar.gz,如 `argus-master-YYYYMMDD.tar.gz`)
- `compose/`(`docker-compose.yml` 与 `.env.example`)
- `scripts/`(安装/运维脚本)
- `private/argus/`(数据与配置骨架)
- `docs/`(中文文档)
## 三、配置 config(生成 .env 与 SWARM_MANAGER_ADDR)
命令:
```
export SWARM_MANAGER_ADDR=<本机管理IP>
./scripts/config.sh
```
脚本做了什么:
- 检查依赖与磁盘空间;
- 自动从“端口 20000 起”分配所有服务端口,确保“系统未占用”且“彼此不冲突”;
- 写入 `compose/.env`(包含端口、镜像 tag、FTP 账号、overlay 名称等);
- 将当前执行账户的 UID/GID 写入 `ARGUS_BUILD_UID/GID`(若主组名是 docker,会改用“与用户名同名的组”的 GID,避免拿到 docker 组 999);
- 更新/追加 `cluster-info.env` 中的 `SWARM_MANAGER_ADDR`(不会覆盖其他键)。
看到什么才算成功:
- 终端输出:`已生成 compose/.env 并更新 cluster-info.env 的 SWARM_MANAGER_ADDR。`
- `compose/.env` 打开应当看到:
- 端口均 ≥20000 且没有重复;
- `ARGUS_BUILD_UID/GID` 与 `id -u/-g` 一致;
- `SWARM_MANAGER_ADDR=<你的IP>`。
遇到问题:
- 端口被异常占用:可删去 `.env` 后再次执行 `config.sh`,或手工编辑端口再执行 `install.sh`。
## 四、安装 install(一次到位)
命令:
```
./scripts/install.sh
```
脚本做了什么:
- 若 Swarm 未激活:执行 `docker swarm init --advertise-addr $SWARM_MANAGER_ADDR`;
- 确保 external overlay `argus-sys-net` 存在;
- 导入 `images/*.tar.gz` 到本机 Docker;
- `docker compose up -d` 启动服务;
- 等待“六项就绪”:
- Master `/readyz`=200、ES `/_cluster/health`=200、Prometheus TCP 可达、Grafana `/api/health`=200、Alertmanager `/api/v2/status`=200、Kibana `/api/status` level=available;
- 将各服务 overlay IP 写入 `private/argus/etc/<域名>`,Reload Bind9 与 Nginx;
- 写出 `cluster-info.env`(含 `SWARM_JOIN_TOKEN_{WORKER,MANAGER}/SWARM_MANAGER_ADDR`;compose 架构下不再依赖 BINDIP/FTPIP);
- 生成 `安装报告_YYYYMMDD-HHMMSS.md`(端口、健康检查摘要与提示)。
看到什么才算成功:
- `docker compose ps` 全部是 Up;
- `安装报告_…md` 中各项 HTTP 检查为 200/available;
- `cluster-info.env` 包含五个关键键:
- `SWARM_MANAGER_ADDR=...`
- `SWARM_MANAGER_ADDR=...` `SWARM_JOIN_TOKEN_*=...`
- `SWARM_JOIN_TOKEN_WORKER=SWMTKN-...`
- `SWARM_JOIN_TOKEN_MANAGER=SWMTKN-...`
## 五、健康自检与常用操作
- 健康自检:`./scripts/selfcheck.sh`
- 期望输出:`selfcheck OK -> logs/selfcheck.json`
- 文件 `logs/selfcheck.json` 中 `overlay_net/es/kibana/master_readyz/ftp_share_writable/prometheus/grafana/alertmanager/web_proxy_cors` 为 true。
- 状态:`./scripts/status.sh`(相当于 `docker compose ps`)。
- 诊断:`./scripts/diagnose.sh`(收集容器/HTTP/CORS/ES 细节,输出到 `logs/diagnose_*.log`)。
- 卸载:`./scripts/uninstall.sh`(Compose down)。
- ES 磁盘水位临时放宽/还原:`./scripts/es-watermark-relax.sh` / `./scripts/es-watermark-restore.sh`。
## 六、下一步:分发 cluster-info.env 给 Client
- 将 `cluster-info.env` 拷贝给安装 Client 的同事;
- 对方在 Client 机器的包根放置该文件(或设置 `CLUSTER_INFO=/绝对路径`)即可。
## 七、故障排查快览
- Proxy 502 或 8080 连接复位:多因 Bind 域名未更新到 overlay IP;重跑 `install.sh`(会写入私有域名文件并 reload)或查看 `logs/diagnose_error.log`。
- Kibana 不 available:等待 1–2 分钟、查看 `argus-kibana-sys` 日志;
- cluster-info.env 的 SWARM_MANAGER_ADDR 为空:重新 `export SWARM_MANAGER_ADDR=<IP>; ./scripts/config.sh` 或 `./scripts/install.sh`(会回读 `.env` 补写)。

View File

@@ -0,0 +1,7 @@
# Docker Swarm 部署要点
- 初始化 Swarm:`docker swarm init --advertise-addr <SWARM_MANAGER_ADDR>`
- 创建 overlay:`docker network create --driver overlay --attachable argus-sys-net`
- Server 包 `install.sh` 自动完成上述操作;如需手动执行,确保 `argus-sys-net` 存在且 attachable。
- Worker 节点加入:`docker swarm join --token <worker_token> <SWARM_MANAGER_ADDR>:2377`。

View File

@@ -0,0 +1,11 @@
# 故障排查(Server)
- 端口占用:查看 `安装报告_*.md` 中端口表;如需修改,编辑 `compose/.env` 后执行 `docker compose ... up -d`。
- 组件未就绪:
- Master: `curl http://127.0.0.1:${MASTER_PORT}/readyz -I`
- ES: `curl http://127.0.0.1:${ES_HTTP_PORT}/_cluster/health`
- Grafana: `curl http://127.0.0.1:${GRAFANA_PORT}/api/health`
- Prometheus TCP: `exec 3<>/dev/tcp/127.0.0.1/${PROMETHEUS_PORT}`
- 域名解析:进入 `argus-web-proxy` 或 `argus-master-sys` 容器:`getent hosts master.argus.com`。
- Swarm/Overlay:检查 `docker network ls | grep argus-sys-net`,或 `docker node ls`。

View File

@@ -0,0 +1,111 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_EX="$PKG_ROOT/compose/.env.example"
ENV_OUT="$PKG_ROOT/compose/.env"
info(){ echo -e "\033[34m[CONFIG]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
# Compose 检测:优先 docker compose(v2),回退 docker-compose(v1)
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker curl jq awk sed tar gzip
require_compose
# 磁盘空间检查(MB)
check_disk(){ local p="$1"; local need=10240; local free
free=$(df -Pm "$p" | awk 'NR==2{print $4+0}')
if [[ -z "$free" || "$free" -lt "$need" ]]; then err "磁盘空间不足: $p 剩余 ${free:-0}MB (<${need}MB)"; return 1; fi
}
check_disk "$PKG_ROOT"; check_disk "/var/lib/docker" || true
# 读取/生成 SWARM_MANAGER_ADDR
SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR:-}
if [[ -z "${SWARM_MANAGER_ADDR}" ]]; then
read -rp "请输入本机管理地址 SWARM_MANAGER_ADDR: " SWARM_MANAGER_ADDR
fi
info "SWARM_MANAGER_ADDR=$SWARM_MANAGER_ADDR"
# 校验 IP 属于本机网卡
if ! ip -o addr | awk '{print $4}' | cut -d'/' -f1 | grep -qx "$SWARM_MANAGER_ADDR"; then
err "SWARM_MANAGER_ADDR 非本机地址: $SWARM_MANAGER_ADDR"; exit 1; fi
info "开始分配服务端口(起始=20000,避免系统占用与相互冲突)"
is_port_used(){ local p="$1"; ss -tulnH 2>/dev/null | awk '{print $5}' | sed 's/.*://g' | grep -qx "$p"; }
declare -A PRESENT=() CHOSEN=() USED=()
START_PORT="${START_PORT:-20000}"; cur=$START_PORT
ORDER=(MASTER_PORT ES_HTTP_PORT KIBANA_PORT PROMETHEUS_PORT GRAFANA_PORT ALERTMANAGER_PORT \
WEB_PROXY_PORT_8080 WEB_PROXY_PORT_8081 WEB_PROXY_PORT_8082 WEB_PROXY_PORT_8083 WEB_PROXY_PORT_8084 WEB_PROXY_PORT_8085 \
FTP_PORT FTP_DATA_PORT)
# 标记 .env.example 中实际存在的键
for key in "${ORDER[@]}"; do
if grep -q "^${key}=" "$ENV_EX"; then PRESENT[$key]=1; fi
done
next_free(){ local p="$1"; while :; do if [[ -n "${USED[$p]:-}" ]] || is_port_used "$p"; then p=$((p+1)); else echo "$p"; return; fi; done; }
for key in "${ORDER[@]}"; do
[[ -z "${PRESENT[$key]:-}" ]] && continue
p=$(next_free "$cur"); CHOSEN[$key]="$p"; USED[$p]=1; cur=$((p+1))
done
info "端口分配结果:MASTER=${CHOSEN[MASTER_PORT]:-} ES=${CHOSEN[ES_HTTP_PORT]:-} KIBANA=${CHOSEN[KIBANA_PORT]:-} PROM=${CHOSEN[PROMETHEUS_PORT]:-} GRAFANA=${CHOSEN[GRAFANA_PORT]:-} ALERT=${CHOSEN[ALERTMANAGER_PORT]:-} WEB_PROXY(8080..8085)=${CHOSEN[WEB_PROXY_PORT_8080]:-}/${CHOSEN[WEB_PROXY_PORT_8081]:-}/${CHOSEN[WEB_PROXY_PORT_8082]:-}/${CHOSEN[WEB_PROXY_PORT_8083]:-}/${CHOSEN[WEB_PROXY_PORT_8084]:-}/${CHOSEN[WEB_PROXY_PORT_8085]:-}"
cp "$ENV_EX" "$ENV_OUT"
# 覆盖端口(按唯一化结果写回)
for key in "${ORDER[@]}"; do
val="${CHOSEN[$key]:-}"
[[ -z "$val" ]] && continue
sed -i -E "s#^$key=.*#$key=${val}#" "$ENV_OUT"
done
info "已写入 compose/.env 的端口配置"
# 覆盖/补充 Overlay 名称
grep -q '^ARGUS_OVERLAY_NET=' "$ENV_OUT" || echo 'ARGUS_OVERLAY_NET=argus-sys-net' >> "$ENV_OUT"
# FTP 默认
grep -q '^FTP_USER=' "$ENV_OUT" || echo 'FTP_USER=ftpuser' >> "$ENV_OUT"
grep -q '^FTP_PASSWORD=' "$ENV_OUT" || echo 'FTP_PASSWORD=NASPlab1234!' >> "$ENV_OUT"
# 以当前执行账户 UID/GID 写入(避免误选 docker 组)
RUID=$(id -u)
PRIMARY_GID=$(id -g)
PRIMARY_GRP=$(id -gn)
USER_NAME=$(id -un)
# 若主组名被解析为 docker,尝试用与用户名同名的组的 GID;否则回退主 GID
if [[ "$PRIMARY_GRP" == "docker" ]]; then
RGID=$(getent group "$USER_NAME" | awk -F: '{print $3}' 2>/dev/null || true)
[[ -z "$RGID" ]] && RGID="$PRIMARY_GID"
else
RGID="$PRIMARY_GID"
fi
info "使用构建账户 UID:GID=${RUID}:${RGID} (user=$USER_NAME primary_group=$PRIMARY_GRP)"
if grep -q '^ARGUS_BUILD_UID=' "$ENV_OUT"; then
sed -i -E "s#^ARGUS_BUILD_UID=.*#ARGUS_BUILD_UID=${RUID}#" "$ENV_OUT"
else
echo "ARGUS_BUILD_UID=${RUID}" >> "$ENV_OUT"
fi
if grep -q '^ARGUS_BUILD_GID=' "$ENV_OUT"; then
sed -i -E "s#^ARGUS_BUILD_GID=.*#ARGUS_BUILD_GID=${RGID}#" "$ENV_OUT"
else
echo "ARGUS_BUILD_GID=${RGID}" >> "$ENV_OUT"
fi
CI="$PKG_ROOT/cluster-info.env"
if [[ -f "$CI" ]]; then
if grep -q '^SWARM_MANAGER_ADDR=' "$CI"; then
sed -i -E "s#^SWARM_MANAGER_ADDR=.*#SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}#" "$CI"
else
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}" >> "$CI"
fi
else
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}" > "$CI"
fi
info "已生成 compose/.env 并更新 cluster-info.env 的 SWARM_MANAGER_ADDR。"
info "下一步可执行: scripts/install.sh"

View File

@@ -0,0 +1,114 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
ENV_FILE="$ROOT/compose/.env"; [[ -f "$ENV_FILE" ]] && set -a && source "$ENV_FILE" && set +a
ts="$(date -u +%Y%m%d-%H%M%SZ)"
LOG_DIR="$ROOT/logs"; mkdir -p "$LOG_DIR" || true
if ! ( : > "$LOG_DIR/.w" 2>/dev/null ); then LOG_DIR="/tmp/argus-logs"; mkdir -p "$LOG_DIR" || true; fi
# load compose project for accurate ps output
ENV_FILE="$ROOT/compose/.env"
if [[ -f "$ENV_FILE" ]]; then set -a; source "$ENV_FILE"; set +a; fi
PROJECT="${COMPOSE_PROJECT_NAME:-argus-server}"
DETAILS="$LOG_DIR/diagnose_details_${ts}.log"; ERRORS="$LOG_DIR/diagnose_error_${ts}.log"; : > "$DETAILS"; : > "$ERRORS"
logd() { echo "$(date '+%F %T') $*" >> "$DETAILS"; }
append_err() { echo "$*" >> "$ERRORS"; }
http_code() { curl -s -o /dev/null -w "%{http_code}" "$1" || echo 000; }
http_body_head() { curl -s --max-time 3 "$1" 2>/dev/null | sed -n '1,5p' || true; }
header_val() { curl -s -D - -o /dev/null "$@" | awk -F': ' 'BEGIN{IGNORECASE=1}$1=="Access-Control-Allow-Origin"{gsub("\r","",$2);print $2}'; }
section() { local name="$1"; logd "===== [$name] ====="; }
svc() {
local svc_name="$1"; local cname="$2"; shift 2
section "$svc_name ($cname)"
logd "docker ps:"; docker ps -a --format '{{.Names}}\t{{.Status}}\t{{.Image}}' | awk -v n="$cname" '$1==n' >> "$DETAILS" || true
logd "docker inspect:"; docker inspect -f '{{.State.Status}} rc={{.RestartCount}} started={{.State.StartedAt}}' "$cname" >> "$DETAILS" 2>&1 || true
logd "last 200 container logs:"; docker logs --tail 200 "$cname" >> "$DETAILS" 2>&1 || true
docker logs --tail 200 "$cname" 2>&1 | grep -Ei '\\b(error|failed|fail|exception|panic|fatal|critical|unhealthy|permission denied|forbidden|refused|traceback|错误|失败)\\b' | sed "s/^/[${svc_name}][container] /" >> "$ERRORS" || true
if docker exec "$cname" sh -lc 'command -v supervisorctl >/dev/null 2>&1' >/dev/null 2>&1; then
logd "supervisorctl status:"; docker exec "$cname" sh -lc 'supervisorctl status' >> "$DETAILS" 2>&1 || true
local files; files=$(docker exec "$cname" sh -lc 'ls /var/log/supervisor/*.log 2>/dev/null' || true)
for f in $files; do
logd "tail -n 80 $f:"; docker exec "$cname" sh -lc "tail -n 80 $f 2>/dev/null || true" >> "$DETAILS" 2>&1 || true
docker exec "$cname" sh -lc "tail -n 200 $f 2>/dev/null" 2>/dev/null | grep -Ei '\\b(error|failed|fail|exception|panic|fatal|critical|unhealthy|permission denied|forbidden|refused|traceback|错误|失败)\\b' | sed "s/^/[${svc_name}][supervisor:$(basename $f)] /" >> "$ERRORS" || true
done
fi
}
svc bind argus-bind-sys
svc master argus-master-sys
svc es argus-es-sys
svc kibana argus-kibana-sys
svc ftp argus-ftp
svc prometheus argus-prometheus
svc grafana argus-grafana
svc alertmanager argus-alertmanager
svc web-frontend argus-web-frontend
svc web-proxy argus-web-proxy
section HTTP
logd "ES: $(http_code \"http://localhost:${ES_HTTP_PORT:-9200}/_cluster/health\")"; http_body_head "http://localhost:${ES_HTTP_PORT:-9200}/_cluster/health" >> "$DETAILS" 2>&1 || true
logd "Kibana: $(http_code \"http://localhost:${KIBANA_PORT:-5601}/api/status\")"; http_body_head "http://localhost:${KIBANA_PORT:-5601}/api/status" >> "$DETAILS" 2>&1 || true
logd "Master readyz: $(http_code \"http://localhost:${MASTER_PORT:-32300}/readyz\")"
logd "Prometheus: $(http_code \"http://localhost:${PROMETHEUS_PORT:-9090}/-/ready\")"
logd "Grafana: $(http_code \"http://localhost:${GRAFANA_PORT:-3000}/api/health\")"; http_body_head "http://localhost:${GRAFANA_PORT:-3000}/api/health" >> "$DETAILS" 2>&1 || true
logd "Alertmanager: $(http_code \"http://localhost:${ALERTMANAGER_PORT:-9093}/api/v2/status\")"
cors8084=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8084:-8084}/api/v2/status" || true)
cors8085=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8085:-8085}/api/v1/master/nodes" || true)
logd "Web-Proxy 8080: $(http_code \"http://localhost:${WEB_PROXY_PORT_8080:-8080}/\")"
logd "Web-Proxy 8083: $(http_code \"http://localhost:${WEB_PROXY_PORT_8083:-8083}/\")"
logd "Web-Proxy 8084 CORS: ${cors8084}"
logd "Web-Proxy 8085 CORS: ${cors8085}"
section ES-CHECKS
ch=$(curl -s --max-time 3 "http://localhost:${ES_HTTP_PORT:-9200}/_cluster/health" || true)
status=$(printf '%s' "$ch" | awk -F'\"' '/"status"/{print $4; exit}')
if [[ -n "$status" ]]; then logd "cluster.status=$status"; fi
if [[ "$status" != "green" ]]; then append_err "[es][cluster] status=$status"; fi
if docker ps --format '{{.Names}}' | grep -q '^argus-es-sys$'; then
duse=$(docker exec argus-es-sys sh -lc 'df -P /usr/share/elasticsearch/data | awk "NR==2{print \$5}"' 2>/dev/null || true)
logd "es.data.df_use=$duse"; usep=${duse%%%}
if [[ -n "$usep" ]] && (( usep >= 90 )); then append_err "[es][disk] data path usage=${usep}%"; fi
fi
section DNS-IN-PROXY
for d in master.argus.com es.log.argus.com kibana.log.argus.com grafana.metric.argus.com prom.metric.argus.com alertmanager.alert.argus.com; do
docker exec argus-web-proxy sh -lc "getent hosts $d || nslookup $d 2>/dev/null | tail -n+1" >> "$DETAILS" 2>&1 || true
done
logd "HTTP (web-proxy): master.readyz=$(docker exec argus-web-proxy sh -lc \"curl -s -o /dev/null -w '%{http_code}' http://master.argus.com:3000/readyz\" 2>/dev/null || echo 000)"
logd "HTTP (web-proxy): es.health=$(docker exec argus-web-proxy sh -lc \"curl -s -o /dev/null -w '%{http_code}' http://es.log.argus.com:9200/_cluster/health\" 2>/dev/null || echo 000)"
logd "HTTP (web-proxy): kibana.status=$(docker exec argus-web-proxy sh -lc \"curl -s -o /dev/null -w '%{http_code}' http://kibana.log.argus.com:5601/api/status\" 2>/dev/null || echo 000)"
section FTP-SHARE
docker exec argus-ftp sh -lc 'ls -ld /private/argus/ftp /private/argus/ftp/share; test -w /private/argus/ftp/share && echo "write:OK" || echo "write:FAIL"' >> "$DETAILS" 2>&1 || true
section SYSTEM
logd "uname -a:"; uname -a >> "$DETAILS"
logd "docker version:"; docker version --format '{{.Server.Version}}' >> "$DETAILS" 2>&1 || true
logd "compose ps (project=$PROJECT):"; (cd "$ROOT/compose" && docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f docker-compose.yml ps) >> "$DETAILS" 2>&1 || true
section SUMMARY
[[ $(http_code "http://localhost:${ES_HTTP_PORT:-9200}/_cluster/health") != 200 ]] && echo "[es][http] health not 200" >> "$ERRORS"
kbcode=$(http_code "http://localhost:${KIBANA_PORT:-5601}/api/status"); [[ "$kbcode" != 200 ]] && echo "[kibana][http] /api/status=$kbcode" >> "$ERRORS"
[[ $(http_code "http://localhost:${MASTER_PORT:-32300}/readyz") != 200 ]] && echo "[master][http] /readyz not 200" >> "$ERRORS"
[[ $(http_code "http://localhost:${PROMETHEUS_PORT:-9090}/-/ready") != 200 ]] && echo "[prometheus][http] /-/ready not 200" >> "$ERRORS"
gfcode=$(http_code "http://localhost:${GRAFANA_PORT:-3000}/api/health"); [[ "$gfcode" != 200 ]] && echo "[grafana][http] /api/health=$gfcode" >> "$ERRORS"
[[ $(http_code "http://localhost:${ALERTMANAGER_PORT:-9093}/api/v2/status") != 200 ]] && echo "[alertmanager][http] /api/v2/status not 200" >> "$ERRORS"
[[ -z "$cors8084" ]] && echo "[web-proxy][cors] 8084 missing Access-Control-Allow-Origin" >> "$ERRORS"
[[ -z "$cors8085" ]] && echo "[web-proxy][cors] 8085 missing Access-Control-Allow-Origin" >> "$ERRORS"
sort -u -o "$ERRORS" "$ERRORS"
echo "Diagnostic details -> $DETAILS"
echo "Detected errors -> $ERRORS"
if [[ "$LOG_DIR" == "$ROOT/logs" ]]; then
ln -sfn "$(basename "$DETAILS")" "$ROOT/logs/diagnose_details.log" 2>/dev/null || cp "$DETAILS" "$ROOT/logs/diagnose_details.log" 2>/dev/null || true
ln -sfn "$(basename "$ERRORS")" "$ROOT/logs/diagnose_error.log" 2>/dev/null || cp "$ERRORS" "$ROOT/logs/diagnose_error.log" 2>/dev/null || true
fi
exit 0

View File

@@ -0,0 +1,11 @@
#!/usr/bin/env bash
set -euo pipefail
HOST="${1:-http://127.0.0.1:9200}"
echo "设置 ES watermark 为 95%/96%/97%: $HOST"
curl -fsS -XPUT "$HOST/_cluster/settings" -H 'Content-Type: application/json' -d '{
"transient": {
"cluster.routing.allocation.disk.watermark.low": "95%",
"cluster.routing.allocation.disk.watermark.high": "96%",
"cluster.routing.allocation.disk.watermark.flood_stage": "97%"
}
}' && echo "\nOK"

View File

@@ -0,0 +1,11 @@
#!/usr/bin/env bash
set -euo pipefail
HOST="${1:-http://127.0.0.1:9200}"
echo "恢复 ES watermark 为默认值: $HOST"
curl -fsS -XPUT "$HOST/_cluster/settings" -H 'Content-Type: application/json' -d '{
"transient": {
"cluster.routing.allocation.disk.watermark.low": null,
"cluster.routing.allocation.disk.watermark.high": null,
"cluster.routing.allocation.disk.watermark.flood_stage": null
}
}' && echo "\nOK"

View File

@@ -0,0 +1,137 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
info(){ echo -e "\033[34m[INSTALL]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
# Compose 检测:优先 docker compose(v2),回退 docker-compose(v1)
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/devnull 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker curl jq awk sed tar gzip
require_compose
[[ -f "$ENV_FILE" ]] || { err "缺少 compose/.env,请先运行 scripts/config.sh"; exit 1; }
info "使用环境文件: $ENV_FILE"
set -a; source "$ENV_FILE"; set +a
# 兼容:若 .env 未包含 SWARM_MANAGER_ADDR,则从已存在的 cluster-info.env 读取以避免写空
SMADDR="${SWARM_MANAGER_ADDR:-}"
CI_FILE="$PKG_ROOT/cluster-info.env"
if [[ -z "$SMADDR" && -f "$CI_FILE" ]]; then
SMADDR=$(sed -n 's/^SWARM_MANAGER_ADDR=\(.*\)$/\1/p' "$CI_FILE" | head -n1)
fi
SWARM_MANAGER_ADDR="$SMADDR"
# Swarm init & overlay
if ! docker info 2>/dev/null | grep -q "Swarm: active"; then
[[ -n "${SWARM_MANAGER_ADDR:-}" ]] || { err "SWARM_MANAGER_ADDR 未设置,请在 scripts/config.sh 中配置"; exit 1; }
info "初始化 Swarm (--advertise-addr $SWARM_MANAGER_ADDR)"
docker swarm init --advertise-addr "$SWARM_MANAGER_ADDR" >/dev/null 2>&1 || true
else
info "Swarm 已激活"
fi
NET_NAME="${ARGUS_OVERLAY_NET:-argus-sys-net}"
if ! docker network inspect "$NET_NAME" >/dev/null 2>&1; then
info "创建 overlay 网络: $NET_NAME"
docker network create -d overlay --attachable "$NET_NAME" >/dev/null
else
info "overlay 网络已存在: $NET_NAME"
fi
# Load images
IMAGES_DIR="$PKG_ROOT/images"
shopt -s nullglob
tars=("$IMAGES_DIR"/*.tar.gz)
if [[ ${#tars[@]} -eq 0 ]]; then err "images 目录为空,缺少镜像 tar.gz"; exit 1; fi
total=${#tars[@]}; idx=0
for tgz in "${tars[@]}"; do
idx=$((idx+1))
info "导入镜像 ($idx/$total): $(basename "$tgz")"
tmp=$(mktemp); gunzip -c "$tgz" > "$tmp"; docker load -i "$tmp" >/dev/null; rm -f "$tmp"
done
shopt -u nullglob
# Compose up
PROJECT="${COMPOSE_PROJECT_NAME:-argus-server}"
info "启动服务栈 (docker compose -p $PROJECT up -d)"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" up -d
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" ps
# Wait readiness (best-effort)
code(){ curl -4 -s -o /dev/null -w "%{http_code}" "$1" || echo 000; }
prom_ok(){ (exec 3<>/dev/tcp/127.0.0.1/${PROMETHEUS_PORT:-9090}) >/dev/null 2>&1 && return 0 || return 1; }
kb_ok(){ local body; body=$(curl -s "http://127.0.0.1:${KIBANA_PORT:-5601}/api/status" || true); echo "$body" | grep -q '"level"\s*:\s*"available"'; }
RETRIES=${RETRIES:-60}; SLEEP=${SLEEP:-5}; ok=0
info "等待基础服务就绪 (<= $((RETRIES*SLEEP))s)"
for i in $(seq 1 "$RETRIES"); do
e1=$(code "http://127.0.0.1:${MASTER_PORT:-32300}/readyz")
e2=$(code "http://127.0.0.1:${ES_HTTP_PORT:-9200}/_cluster/health")
e3=000; prom_ok && e3=200
e4=$(code "http://127.0.0.1:${GRAFANA_PORT:-3000}/api/health")
e5=$(code "http://127.0.0.1:${ALERTMANAGER_PORT:-9093}/api/v2/status")
e6=$(kb_ok && echo 200 || echo 000)
info "[ready] t=$((i*SLEEP))s master=$e1 es=$e2 prom=$e3 graf=$e4 alert=$e5 kibana=$e6"
[[ "$e1" == 200 ]] && ok=$((ok+1))
[[ "$e2" == 200 ]] && ok=$((ok+1))
[[ "$e3" == 200 ]] && ok=$((ok+1))
[[ "$e4" == 200 ]] && ok=$((ok+1))
[[ "$e5" == 200 ]] && ok=$((ok+1))
[[ "$e6" == 200 ]] && ok=$((ok+1))
if [[ $ok -ge 6 ]]; then break; fi; ok=0; sleep "$SLEEP"
done
[[ $ok -ge 6 ]] || err "部分服务未就绪(可稍后重试 selfcheck)"
# Swarm join tokens
TOKEN_WORKER=$(docker swarm join-token -q worker 2>/dev/null || echo "")
TOKEN_MANAGER=$(docker swarm join-token -q manager 2>/dev/null || echo "")
# cluster-info.env(compose 场景下不再依赖 BINDIP/FTPIP)
CI="$PKG_ROOT/cluster-info.env"
info "写入 cluster-info.env (manager/token)"
{
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR:-}"
echo "SWARM_JOIN_TOKEN_WORKER=${TOKEN_WORKER:-}"
echo "SWARM_JOIN_TOKEN_MANAGER=${TOKEN_MANAGER:-}"
} > "$CI"
info "已输出 $CI"
# 安装报告
ts=$(date +%Y%m%d-%H%M%S)
RPT="$PKG_ROOT/安装报告_${ts}.md"
{
echo "# Argus Server 安装报告 (${ts})"
echo
echo "## 端口映射"
echo "- MASTER_PORT=${MASTER_PORT}"
echo "- ES_HTTP_PORT=${ES_HTTP_PORT}"
echo "- KIBANA_PORT=${KIBANA_PORT}"
echo "- PROMETHEUS_PORT=${PROMETHEUS_PORT}"
echo "- GRAFANA_PORT=${GRAFANA_PORT}"
echo "- ALERTMANAGER_PORT=${ALERTMANAGER_PORT}"
echo "- WEB_PROXY_PORT_8080=${WEB_PROXY_PORT_8080} ... 8085=${WEB_PROXY_PORT_8085}"
echo
echo "## Swarm/Overlay"
echo "- SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR:-}"
echo "- NET=${NET_NAME}"
echo "- JOIN_TOKEN_WORKER=${TOKEN_WORKER:-}"
echo "- JOIN_TOKEN_MANAGER=${TOKEN_MANAGER:-}"
echo
echo "## 健康检查(简要)"
echo "- master/readyz=$(code http://127.0.0.1:${MASTER_PORT:-32300}/readyz)"
echo "- es/_cluster/health=$(code http://127.0.0.1:${ES_HTTP_PORT:-9200}/_cluster/health)"
echo "- grafana/api/health=$(code http://127.0.0.1:${GRAFANA_PORT:-3000}/api/health)"
echo "- prometheus/tcp=$([[ $(prom_ok; echo $?) == 0 ]] && echo 200 || echo 000)"
echo "- alertmanager/api/v2/status=$(code http://127.0.0.1:${ALERTMANAGER_PORT:-9093}/api/v2/status)"
echo "- kibana/api/status=$([[ $(kb_ok; echo $?) == 0 ]] && echo available || echo not-ready)"
} > "$RPT"
info "已生成报告: $RPT"
info "安装完成。可将 cluster-info.env 分发给 Client-GPU 安装方。"
docker exec argus-web-proxy nginx -t >/dev/null 2>&1 && docker exec argus-web-proxy nginx -s reload >/dev/null 2>&1 || true

View File

@@ -0,0 +1,89 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
log() { echo -e "\033[0;34m[CHECK]\033[0m $*"; }
err() { echo -e "\033[0;31m[ERROR]\033[0m $*" >&2; }
ENV_FILE="$ROOT/compose/.env"; [[ -f "$ENV_FILE" ]] && set -a && source "$ENV_FILE" && set +a
wait_http() { local url="$1"; local attempts=${2:-120}; local i=1; while ((i<=attempts)); do curl -fsS "$url" >/dev/null 2>&1 && return 0; echo "[..] waiting $url ($i/$attempts)"; sleep 5; ((i++)); done; return 1; }
code_for() { curl -s -o /dev/null -w "%{http_code}" "$1" || echo 000; }
header_val() { curl -s -D - -o /dev/null "$@" | awk -F': ' 'BEGIN{IGNORECASE=1}$1=="Access-Control-Allow-Origin"{gsub("\r","",$2);print $2}'; }
LOG_DIR="$ROOT/logs"; mkdir -p "$LOG_DIR" || true
OUT_JSON="$LOG_DIR/selfcheck.json"; tmp=$(mktemp)
ok=1
log "checking overlay network"
net_ok=false
if docker network inspect "${ARGUS_OVERLAY_NET:-argus-sys-net}" >/dev/null 2>&1; then
if docker network inspect "${ARGUS_OVERLAY_NET:-argus-sys-net}" | grep -q '"Driver": "overlay"'; then net_ok=true; fi
fi
[[ "$net_ok" == true ]] || ok=0
log "checking Elasticsearch"
wait_http "http://localhost:${ES_HTTP_PORT:-9200}/_cluster/health" 60 || ok=0
log "checking Kibana"
kb_code=$(code_for "http://localhost:${KIBANA_PORT:-5601}/api/status")
kb_ok=false
if [[ "$kb_code" == 200 ]]; then
body=$(curl -sS "http://localhost:${KIBANA_PORT:-5601}/api/status" || true)
echo "$body" | grep -q '"level"\s*:\s*"available"' && kb_ok=true
fi
[[ "$kb_ok" == true ]] || ok=0
log "checking Master"
[[ $(code_for "http://localhost:${MASTER_PORT:-32300}/readyz") == 200 ]] || ok=0
log "checking FTP"
if docker ps --format '{{.Names}}' | grep -q '^argus-ftp$'; then
docker exec argus-ftp sh -lc 'test -w /private/argus/ftp/share' >/dev/null 2>&1 || ok=0
else ok=0; fi
log "checking Prometheus"
wait_http "http://localhost:${PROMETHEUS_PORT:-9090}/-/ready" 60 || ok=0
log "checking Grafana"
gf_code=$(code_for "http://localhost:${GRAFANA_PORT:-3000}/api/health")
gf_ok=false; if [[ "$gf_code" == 200 ]]; then body=$(curl -sS "http://localhost:${GRAFANA_PORT:-3000}/api/health" || true); echo "$body" | grep -q '"database"\s*:\s*"ok"' && gf_ok=true; fi
[[ "$gf_ok" == true ]] || ok=0
log "checking Alertmanager"
wait_http "http://localhost:${ALERTMANAGER_PORT:-9093}/api/v2/status" 60 || ok=0
log "checking Web-Proxy (CORS)"
cors8084=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8084:-8084}/api/v2/status" || true)
cors8085=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8085:-8085}/api/v1/master/nodes" || true)
wp_ok=true
[[ -n "$cors8084" && -n "$cors8085" ]] || wp_ok=false
[[ "$wp_ok" == true ]] || ok=0
cat > "$tmp" <<JSON
{
"overlay_net": $net_ok,
"es": true,
"kibana": $kb_ok,
"master_readyz": true,
"ftp_share_writable": true,
"prometheus": true,
"grafana": $gf_ok,
"alertmanager": true,
"web_proxy_cors": $wp_ok,
"timestamp": "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
}
JSON
mv "$tmp" "$OUT_JSON" 2>/dev/null || cp "$tmp" "$OUT_JSON"
if [[ "$ok" == 1 ]]; then
log "selfcheck OK -> $OUT_JSON"
exit 0
else
err "selfcheck FAILED -> $OUT_JSON"
exit 1
fi

View File

@@ -0,0 +1,9 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
if [[ -f "$ENV_FILE" ]]; then set -a; source "$ENV_FILE"; set +a; fi
PROJECT="${COMPOSE_PROJECT_NAME:-argus-server}"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" ps

View File

@@ -0,0 +1,23 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
# load COMPOSE_PROJECT_NAME from env file if present
if [[ -f "$ENV_FILE" ]]; then set -a; source "$ENV_FILE"; set +a; fi
PROJECT="${COMPOSE_PROJECT_NAME:-argus-server}"
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
# Compose 检测:优先 docker compose(v2),回退 docker-compose(v1)
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require_compose
echo "[UNINSTALL] stopping compose (project=$PROJECT)"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" down --remove-orphans || true
echo "[UNINSTALL] done"

View File

@@ -0,0 +1,154 @@
version: "3.8"
networks:
argus-sys-net:
external: true
services:
master:
image: ${MASTER_IMAGE_TAG:-argus-master-arm64:${PKG_VERSION}}
platform: linux/arm64
container_name: argus-master-sys
security_opt:
- seccomp:unconfined
environment:
- OFFLINE_THRESHOLD_SECONDS=180
- ONLINE_THRESHOLD_SECONDS=120
- SCHEDULER_INTERVAL_SECONDS=30
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
ports:
- "${MASTER_PORT:-32300}:3000"
volumes:
- ../private/argus/master:/private/argus/master
- ../private/argus/metric/prometheus:/private/argus/metric/prometheus
- ../private/argus/etc:/private/argus/etc
networks:
argus-sys-net:
aliases:
- master.argus.com
restart: unless-stopped
prometheus:
image: ${PROM_IMAGE_TAG:-argus-metric-prometheus-arm64:${PKG_VERSION}}
platform: linux/arm64
container_name: argus-prometheus
restart: unless-stopped
environment:
- TZ=Asia/Shanghai
- PROMETHEUS_BASE_PATH=/private/argus/metric/prometheus
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
ports:
- "${PROMETHEUS_PORT:-9090}:9090"
volumes:
- ../private/argus/metric/prometheus:/private/argus/metric/prometheus
- ../private/argus/etc:/private/argus/etc
networks:
argus-sys-net:
aliases:
- prom.metric.argus.com
prometheus-targets-updater:
image: ${PROM_UPDATER_IMAGE_TAG:-argus-metric-prometheus-targets-updater-arm64:${PKG_VERSION}}
platform: linux/arm64
container_name: argus-prometheus-targets-updater
restart: unless-stopped
environment:
- TZ=Asia/Shanghai
volumes:
- ../private/argus/metric/prometheus:/private/argus/metric/prometheus
networks:
argus-sys-net:
aliases:
- prom-updater.metric.argus.com
depends_on:
- master
- prometheus
grafana:
image: ${GRAFANA_IMAGE_TAG:-argus-metric-grafana-arm64:${PKG_VERSION}}
platform: linux/arm64
container_name: argus-grafana
restart: unless-stopped
environment:
- TZ=Asia/Shanghai
- GRAFANA_BASE_PATH=/private/argus/metric/grafana
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
- GF_SERVER_HTTP_PORT=3000
- GF_LOG_LEVEL=warn
- GF_LOG_MODE=console
- GF_PATHS_PROVISIONING=/private/argus/metric/grafana/provisioning
- GF_AUTH_ANONYMOUS_ENABLED=true
- GF_AUTH_ANONYMOUS_ORG_ROLE=Viewer
ports:
- "${GRAFANA_PORT:-3000}:3000"
volumes:
- ../private/argus/metric/grafana:/private/argus/metric/grafana
- ../private/argus/etc:/private/argus/etc
depends_on: [prometheus]
networks:
argus-sys-net:
aliases:
- grafana.metric.argus.com
alertmanager:
image: ${ALERT_IMAGE_TAG:-argus-alertmanager-arm64:${PKG_VERSION}}
platform: linux/arm64
container_name: argus-alertmanager
environment:
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
volumes:
- ../private/argus/etc:/private/argus/etc
- ../private/argus/alert/alertmanager:/private/argus/alert/alertmanager
networks:
argus-sys-net:
aliases:
- alertmanager.alert.argus.com
ports:
- "${ALERTMANAGER_PORT:-9093}:9093"
restart: unless-stopped
web-frontend:
image: ${FRONT_IMAGE_TAG:-argus-web-frontend-arm64:${PKG_VERSION}}
platform: linux/arm64
container_name: argus-web-frontend
environment:
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
- EXTERNAL_MASTER_PORT=${WEB_PROXY_PORT_8085:-8085}
- EXTERNAL_ALERTMANAGER_PORT=${WEB_PROXY_PORT_8084:-8084}
- EXTERNAL_GRAFANA_PORT=${WEB_PROXY_PORT_8081:-8081}
- EXTERNAL_PROMETHEUS_PORT=${WEB_PROXY_PORT_8082:-8082}
volumes:
- ../private/argus/etc:/private/argus/etc
networks:
argus-sys-net:
aliases:
- web.argus.com
restart: unless-stopped
web-proxy:
image: ${WEB_PROXY_IMAGE_TAG:-argus-web-proxy-arm64:${PKG_VERSION}}
platform: linux/arm64
container_name: argus-web-proxy
depends_on: [master, grafana, prometheus, alertmanager]
environment:
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
volumes:
- ../private/argus/etc:/private/argus/etc
networks:
argus-sys-net:
aliases:
- proxy.argus.com
ports:
- "${WEB_PROXY_PORT_8080:-8080}:8080"
- "${WEB_PROXY_PORT_8081:-8081}:8081"
- "${WEB_PROXY_PORT_8082:-8082}:8082"
- "${WEB_PROXY_PORT_8083:-8083}:8083"
- "${WEB_PROXY_PORT_8084:-8084}:8084"
- "${WEB_PROXY_PORT_8085:-8085}:8085"
restart: unless-stopped

View File

@@ -0,0 +1,158 @@
# ARM Server 安装指南(server_arm)
本包用于在 ARM64 服务器上部署 Argus 的最小 server 端组件集:
- master
- Prometheus
- Prometheus targets-updater sidecar
- Grafana
- Alertmanager
- Web 前端 + Web Proxy
> 注意:本 ARM 版本 **不包含 Elasticsearch / Kibana / FTP / BIND9**,仅提供指标与告警能力。
---
## 1. 前置条件
- 目标机器为 ARM64(例如:aarch64 服务器);
- 已安装 Docker 和 docker compose(v2,支持 `docker compose` 命令);
- 已准备好 overlay 网络 `argus-sys-net`(如不需要也可将 compose 改为 `bridge` 网络)。
创建 overlay 网络示例:
```bash
docker network create --driver overlay argus-sys-net || true
```
如未使用 Swarm,也可以在 compose 中将 `networks: argus-sys-net` 改成本地 `bridge` 网络。
---
## 2. 解压和加载镜像
假设你已经将 `server_arm_YYYYMMDD.tar.gz` 拷贝到目标 ARM 服务器:
```bash
tar -xzf server_arm_YYYYMMDD.tar.gz
cd server_arm/YYYYMMDD
```
加载镜像:
```bash
for f in images/*.tar.gz; do
gunzip -c "$f" | docker load
done
```
加载后可以用:
```bash
docker images | grep 'argus-.*arm64'
```
确认 master / prometheus / grafana / alertmanager / web 镜像已就绪。
---
## 3. 配置 .env
在 `compose/` 目录下有自动生成的 `.env.example`:
```bash
cd compose
cp .env.example .env
```
`.env` 中主要字段:
- `PKG_VERSION`:包版本(不建议修改)。
- 镜像 tag(默认使用构建时的 `*-arm64:${PKG_VERSION}`):
- `MASTER_IMAGE_TAG`
- `PROM_IMAGE_TAG`
- `PROM_UPDATER_IMAGE_TAG`
- `GRAFANA_IMAGE_TAG`
- `ALERT_IMAGE_TAG`
- `FRONT_IMAGE_TAG`
- `WEB_PROXY_IMAGE_TAG`
- 端口:
- `MASTER_PORT`(默认 32300)
- `PROMETHEUS_PORT`(默认 9090)
- `GRAFANA_PORT`(默认 3000)
- `ALERTMANAGER_PORT`(默认 9093)
- `WEB_PROXY_PORT_8080` ~ `WEB_PROXY_PORT_8085`
- 其他:
- `ARGUS_BUILD_UID` / `ARGUS_BUILD_GID`(用于挂载目录的权限对齐)。
根据实际环境按需调整端口和 UID/GID。
---
## 4. 准备挂载目录
在 package 根目录(与 `compose/` 同级)下,已预创建所需的 `private/argus` 目录骨架:
- `private/argus/etc`
- `private/argus/master`
- `private/argus/metric/prometheus`(含 `data` / `rules` / `targets`)
- `private/argus/metric/grafana`(含 `data` / `logs` / `plugins` / `provisioning` 等)
可根据需要调整访问权限,例如:
```bash
cd ..
sudo chown -R $UID:$GID private/argus
```
确保与 `.env` 中的 `ARGUS_BUILD_UID/GID` 一致。
---
## 5. 启动服务
在 `compose/` 目录执行:
```bash
docker compose --env-file .env up -d
```
检查服务状态:
```bash
docker compose ps
```
预期看到:
- `argus-master-sys`
- `argus-prometheus`
- `argus-prometheus-targets-updater`
- `argus-grafana`
- `argus-alertmanager`
- `argus-web-frontend`
- `argus-web-proxy`
---
## 6. 验证
1. Master:
- `curl http://localhost:${MASTER_PORT}/readyz`
2. Prometheus:
- 浏览器访问:`http://<host>:${PROMETHEUS_PORT}`
3. Grafana:
- 浏览器访问:`http://<host>:${GRAFANA_PORT}`
4. Web 门户:
- 浏览器访问:`http://<host>:${WEB_PROXY_PORT_8080}`
确认这些页面能够正常打开,并且在没有任何节点注册时,Prometheus 中 `job="node"` 可为空,但基础 UI 应可访问。
---
## 7. 注意事项
- 本 ARM server 包仅包含 metrics + alert 相关组件,不提供日志链路(ES/Kibana),也不包含 DNS/FTP 相关服务;
- 若后续需要扩展 ES/Kibana,可在新的 spec 中单独设计 ARM 日志栈;
- 对于多节点/Swarm 部署,请结合 `src/sys/arm_swarm_tests` 中的 compose 与脚本进行扩展。

View File

@@ -0,0 +1,107 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_EX="$PKG_ROOT/compose/.env.example"
ENV_OUT="$PKG_ROOT/compose/.env"
info(){ echo -e "\033[34m[CONFIG]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
# Compose 检测:优先 docker compose(v2),回退 docker-compose(v1)
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker awk sed
require_compose
[[ -f "$ENV_EX" ]] || { err "缺少模板文件: $ENV_EX"; exit 1; }
# 磁盘空间检查(简化版,默认 >= 5GB)
check_disk(){ local p="$1"; local need="${2:-5120}"; local free;
free=$(df -Pm "$p" 2>/dev/null | awk 'NR==2{print $4+0}')
if [[ -z "$free" || "$free" -lt "$need" ]]; then
err "磁盘空间不足: $p 剩余 ${free:-0}MB (<${need}MB),请清理后再继续"
fi
}
check_disk "$PKG_ROOT" 5120 || true
check_disk "/var/lib/docker" 5120 || true
cp "$ENV_EX" "$ENV_OUT"
# 读取/生成 SWARM_MANAGER_ADDR(ARM 下同样引导 Swarm + cluster-info)
SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR:-}
if [[ -z "${SWARM_MANAGER_ADDR}" ]]; then
read -rp "请输入本机管理地址 SWARM_MANAGER_ADDR(可填当前主机会被其他节点访问到的 IP 或主机名): " SWARM_MANAGER_ADDR
fi
info "SWARM_MANAGER_ADDR=$SWARM_MANAGER_ADDR"
info "开始分配服务端口(起始=20000,避免系统占用与相互冲突)"
is_port_used(){ local p="$1"; ss -tulnH 2>/dev/null | awk '{print $5}' | sed 's/.*://g' | grep -qx "$p"; }
declare -A PRESENT=() CHOSEN=() USED=()
START_PORT="${START_PORT:-20000}"; cur=$START_PORT
ORDER=(MASTER_PORT PROMETHEUS_PORT GRAFANA_PORT ALERTMANAGER_PORT \
WEB_PROXY_PORT_8080 WEB_PROXY_PORT_8081 WEB_PROXY_PORT_8082 \
WEB_PROXY_PORT_8083 WEB_PROXY_PORT_8084 WEB_PROXY_PORT_8085)
# 标记 .env.example 中实际存在的键
for key in "${ORDER[@]}"; do
if grep -q "^${key}=" "$ENV_EX"; then PRESENT[$key]=1; fi
done
next_free(){ local p="$1"; while :; do if [[ -n "${USED[$p]:-}" ]] || is_port_used "$p"; then p=$((p+1)); else echo "$p"; return; fi; done; }
for key in "${ORDER[@]}"; do
[[ -z "${PRESENT[$key]:-}" ]] && continue
p=$(next_free "$cur"); CHOSEN[$key]="$p"; USED[$p]=1; cur=$((p+1))
done
info "端口分配结果:MASTER=${CHOSEN[MASTER_PORT]:-} PROM=${CHOSEN[PROMETHEUS_PORT]:-} GRAFANA=${CHOSEN[GRAFANA_PORT]:-} ALERT=${CHOSEN[ALERTMANAGER_PORT]:-} WEB_PROXY(8080..8085)=${CHOSEN[WEB_PROXY_PORT_8080]:-}/${CHOSEN[WEB_PROXY_PORT_8081]:-}/${CHOSEN[WEB_PROXY_PORT_8082]:-}/${CHOSEN[WEB_PROXY_PORT_8083]:-}/${CHOSEN[WEB_PROXY_PORT_8084]:-}/${CHOSEN[WEB_PROXY_PORT_8085]:-}"
# 覆盖端口(按唯一化结果写回)
for key in "${ORDER[@]}"; do
val="${CHOSEN[$key]:-}"
[[ -z "$val" ]] && continue
sed -i -E "s#^$key=.*#$key=${val}#" "$ENV_OUT"
done
# 覆盖/补充 Overlay 名称
grep -q '^ARGUS_OVERLAY_NET=' "$ENV_OUT" || echo 'ARGUS_OVERLAY_NET=argus-sys-net' >> "$ENV_OUT"
# 覆盖/补充构建账户 UID/GID,避免权限问题
RUID=$(id -u)
RGID=$(id -g)
if grep -q '^ARGUS_BUILD_UID=' "$ENV_OUT"; then
sed -i -E "s#^ARGUS_BUILD_UID=.*#ARGUS_BUILD_UID=${RUID}#" "$ENV_OUT"
else
echo "ARGUS_BUILD_UID=${RUID}" >> "$ENV_OUT"
fi
if grep -q '^ARGUS_BUILD_GID=' "$ENV_OUT"; then
sed -i -E "s#^ARGUS_BUILD_GID=.*#ARGUS_BUILD_GID=${RGID}#" "$ENV_OUT"
else
echo "ARGUS_BUILD_GID=${RGID}" >> "$ENV_OUT"
fi
# 更新 cluster-info.env 的 SWARM_MANAGER_ADDR(ARM 版本也保留该机制)
CI="$PKG_ROOT/cluster-info.env"
if [[ -f "$CI" ]]; then
if grep -q '^SWARM_MANAGER_ADDR=' "$CI"; then
sed -i -E "s#^SWARM_MANAGER_ADDR=.*#SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}#" "$CI"
else
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}" >> "$CI"
fi
else
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}" > "$CI"
fi
info "已生成 compose/.env 并更新 cluster-info.env 的 SWARM_MANAGER_ADDR。"
info "下一步可执行: scripts/install.sh"

View File

@@ -0,0 +1,91 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
ENV_FILE="$ROOT/compose/.env"; [[ -f "$ENV_FILE" ]] && set -a && source "$ENV_FILE" && set +a
ts="$(date -u +%Y%m%d-%H%M%SZ)"
LOG_DIR="$ROOT/logs"; mkdir -p "$LOG_DIR" || true
if ! ( : > "$LOG_DIR/.w" 2>/dev/null ); then LOG_DIR="/tmp/argus-logs"; mkdir -p "$LOG_DIR" || true; fi
PROJECT="${COMPOSE_PROJECT_NAME:-argus-server-arm}"
DETAILS="$LOG_DIR/diagnose_arm_server_details_${ts}.log"
ERRORS="$LOG_DIR/diagnose_arm_server_error_${ts}.log"
: > "$DETAILS"; : > "$ERRORS"
logd() { echo "$(date '+%F %T') $*" >> "$DETAILS"; }
append_err() { echo "$*" >> "$ERRORS"; }
http_code() { curl -s -o /dev/null -w "%{http_code}" "$1" || echo 000; }
http_body_head() { curl -s --max-time 3 "$1" 2>/dev/null | sed -n '1,5p' || true; }
header_val() { curl -s -D - -o /dev/null "$@" | awk -F': ' 'BEGIN{IGNORECASE=1}$1=="Access-Control-Allow-Origin"{gsub("\r","",$2);print $2}'; }
section() { local name="$1"; logd "===== [$name] ====="; }
svc() {
local svc_name="$1"; local cname="$2"
section "$svc_name ($cname)"
logd "docker ps:"; docker ps -a --format '{{.Names}}\t{{.Status}}\t{{.Image}}' | awk -v n="$cname" '$1==n' >> "$DETAILS" || true
logd "docker inspect:"; docker inspect -f '{{.State.Status}} rc={{.RestartCount}} started={{.State.StartedAt}}' "$cname" >> "$DETAILS" 2>&1 || true
logd "last 200 container logs:"; docker logs --tail 200 "$cname" >> "$DETAILS" 2>&1 || true
docker logs --tail 200 "$cname" 2>&1 | \
grep -Ei '\b(error|failed|fail|exception|panic|fatal|critical|unhealthy|permission denied|forbidden|refused|traceback|错误|失败)\b' | \
sed "s/^/[${svc_name}][container] /" >> "$ERRORS" || true
if docker exec "$cname" sh -lc 'command -v supervisorctl >/dev/null 2>&1' >/dev/null 2>&1; then
logd "supervisorctl status:"; docker exec "$cname" sh -lc 'supervisorctl status' >> "$DETAILS" 2>&1 || true
local files; files=$(docker exec "$cname" sh -lc 'ls /var/log/supervisor/*.log 2>/dev/null' || true)
for f in $files; do
logd "tail -n 80 $f:"; docker exec "$cname" sh -lc "tail -n 80 $f 2>/dev/null || true" >> "$DETAILS" 2>&1 || true
docker exec "$cname" sh -lc "tail -n 200 $f 2>/dev/null" 2>/dev/null | \
grep -Ei '\b(error|failed|fail|exception|panic|fatal|critical|unhealthy|permission denied|forbidden|refused|traceback|错误|失败)\b' | \
sed "s/^/[${svc_name}][supervisor:$(basename "$f")] /" >> "$ERRORS" || true
done
fi
}
# 关键服务
svc master argus-master-sys
svc prometheus argus-prometheus
svc prom-updater argus-prometheus-targets-updater
svc grafana argus-grafana
svc alertmanager argus-alertmanager
svc web-frontend argus-web-frontend
svc web-proxy argus-web-proxy
section HTTP
logd "Master: $(http_code \"http://localhost:${MASTER_PORT:-32300}/readyz\")"
logd "Prometheus: $(http_code \"http://localhost:${PROMETHEUS_PORT:-9090}/-/ready\")"
logd "Grafana: $(http_code \"http://localhost:${GRAFANA_PORT:-3000}/api/health\")"; http_body_head "http://localhost:${GRAFANA_PORT:-3000}/api/health" >> "$DETAILS" 2>&1 || true
logd "Alertmanager: $(http_code \"http://localhost:${ALERTMANAGER_PORT:-9093}/api/v2/status\")"
cors8084=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8084:-8084}/api/v2/status" || true)
cors8085=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8085:-8085}/api/v1/master/nodes" || true)
logd "Web-Proxy 8080: $(http_code \"http://localhost:${WEB_PROXY_PORT_8080:-8080}/\")"
logd "Web-Proxy 8083: $(http_code \"http://localhost:${WEB_PROXY_PORT_8083:-8083}/\")"
logd "Web-Proxy 8084 CORS: ${cors8084}"
logd "Web-Proxy 8085 CORS: ${cors8085}"
section SYSTEM
logd "uname -a:"; uname -a >> "$DETAILS"
logd "docker version:"; docker version --format '{{.Server.Version}}' >> "$DETAILS" 2>&1 || true
logd "compose ps (project=$PROJECT):"; (cd "$ROOT/compose" && docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f docker-compose.yml ps) >> "$DETAILS" 2>&1 || true
section SUMMARY
[[ $(http_code "http://localhost:${MASTER_PORT:-32300}/readyz") != 200 ]] && echo "[master][http] /readyz not 200" >> "$ERRORS"
[[ $(http_code "http://localhost:${PROMETHEUS_PORT:-9090}/-/ready") != 200 ]] && echo "[prometheus][http] /-/ready not 200" >> "$ERRORS"
gfcode=$(http_code "http://localhost:${GRAFANA_PORT:-3000}/api/health"); [[ "$gfcode" != 200 ]] && echo "[grafana][http] /api/health=$gfcode" >> "$ERRORS"
[[ $(http_code "http://localhost:${ALERTMANAGER_PORT:-9093}/api/v2/status") != 200 ]] && echo "[alertmanager][http] /api/v2/status not 200" >> "$ERRORS"
[[ -z "$cors8084" ]] && echo "[web-proxy][cors] 8084 missing Access-Control-Allow-Origin" >> "$ERRORS"
[[ -z "$cors8085" ]] && echo "[web-proxy][cors] 8085 missing Access-Control-Allow-Origin" >> "$ERRORS"
sort -u -o "$ERRORS" "$ERRORS"
echo "Diagnostic details -> $DETAILS"
echo "Detected errors -> $ERRORS"
if [[ "$LOG_DIR" == "$ROOT/logs" ]]; then
ln -sfn "$(basename "$DETAILS")" "$ROOT/logs/diagnose_arm_server_details.log" 2>/dev/null || cp "$DETAILS" "$ROOT/logs/diagnose_arm_server_details.log" 2>/dev/null || true
ln -sfn "$(basename "$ERRORS")" "$ROOT/logs/diagnose_arm_server_error.log" 2>/dev/null || cp "$ERRORS" "$ROOT/logs/diagnose_arm_server_error.log" 2>/dev/null || true
fi
exit 0

View File

@@ -0,0 +1,159 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
info(){ echo -e "\033[34m[INSTALL]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker curl jq awk sed tar gzip
require_compose
[[ -f "$ENV_FILE" ]] || { err "缺少 compose/.env,请先运行 scripts/config.sh"; exit 1; }
info "使用环境文件: $ENV_FILE"
set -a; source "$ENV_FILE"; set +a
# 兼容:若 .env 未包含 SWARM_MANAGER_ADDR,则从已存在的 cluster-info.env 读取以避免写空
SMADDR="${SWARM_MANAGER_ADDR:-}"
CI_FILE="$PKG_ROOT/cluster-info.env"
if [[ -z "$SMADDR" && -f "$CI_FILE" ]]; then
SMADDR=$(sed -n 's/^SWARM_MANAGER_ADDR=\(.*\)$/\1/p' "$CI_FILE" | head -n1)
fi
SWARM_MANAGER_ADDR="$SMADDR"
NET_NAME="${ARGUS_OVERLAY_NET:-argus-sys-net}"
SWARM_AVAILABLE=true
# Swarm init & overlay(ARM 版优先按生产方式启用 Swarm,但在无法启用时退回本地 bridge)
if ! docker info 2>/dev/null | grep -q "Swarm: active"; then
if [[ -n "${SWARM_MANAGER_ADDR:-}" ]]; then
info "Swarm 未激活,尝试初始化 Swarm (--advertise-addr $SWARM_MANAGER_ADDR)"
if ! docker swarm init --advertise-addr "$SWARM_MANAGER_ADDR" >/dev/null 2>&1; then
err "Swarm 初始化失败,本次部署将退回本地 bridge 网络: $NET_NAME(仅支持单机测试,不支持 swarm client)"
SWARM_AVAILABLE=false
else
info "Swarm 初始化成功"
fi
else
info "SWARM_MANAGER_ADDR 未设置且 Swarm 未激活,将使用本地 bridge 网络: $NET_NAME(仅支持单机测试)"
SWARM_AVAILABLE=false
fi
else
info "Swarm 已激活"
fi
if [[ "$SWARM_AVAILABLE" == true ]]; then
if ! docker network inspect "$NET_NAME" >/dev/null 2>&1; then
info "创建 overlay 网络: $NET_NAME"
docker network create -d overlay --attachable "$NET_NAME" >/dev/null
else
info "overlay 网络已存在: $NET_NAME"
fi
# 将 Swarm join token 写入 cluster-info.env,供 ARM client 使用
TOKEN_WORKER=$(docker swarm join-token -q worker 2>/dev/null || echo "")
TOKEN_MANAGER=$(docker swarm join-token -q manager 2>/dev/null || echo "")
CI_OUT="$PKG_ROOT/cluster-info.env"
info "写入 cluster-info.env (manager/token)"
{
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR:-}"
echo "SWARM_JOIN_TOKEN_WORKER=${TOKEN_WORKER:-}"
echo "SWARM_JOIN_TOKEN_MANAGER=${TOKEN_MANAGER:-}"
} > "$CI_OUT"
else
# 本机无法启用 Swarm,只维护 SWARM_MANAGER_ADDR,join token 留空
CI_OUT="$PKG_ROOT/cluster-info.env"
info "写入 cluster-info.env(Swarm 未启用,仅保存 SWARM_MANAGER_ADDR)"
{
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR:-}"
echo "SWARM_JOIN_TOKEN_WORKER="
echo "SWARM_JOIN_TOKEN_MANAGER="
} > "$CI_OUT"
if ! docker network inspect "$NET_NAME" >/dev/null 2>&1; then
info "创建本地 bridge 网络: $NET_NAME"
docker network create "$NET_NAME" >/dev/null
else
info "本地网络已存在: $NET_NAME"
fi
fi
# 导入镜像
IMAGES_DIR="$PKG_ROOT/images"
shopt -s nullglob
tars=("$IMAGES_DIR"/*.tar.gz)
if [[ ${#tars[@]} -eq 0 ]]; then err "images 目录为空,缺少镜像 tar.gz"; exit 1; fi
total=${#tars[@]}; idx=0
for tgz in "${tars[@]}"; do
idx=$((idx+1))
info "导入镜像 ($idx/$total): $(basename "$tgz")"
tmp=$(mktemp); gunzip -c "$tgz" > "$tmp"; docker load -i "$tmp" >/dev/null; rm -f "$tmp"
done
shopt -u nullglob
# 启动服务
PROJECT="${COMPOSE_PROJECT_NAME:-argus-server-arm}"
info "启动服务栈 (docker compose -p $PROJECT up -d)"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" up -d
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" ps
# 简单就绪检查(best-effort,不阻塞安装)
code(){ curl -4 -s -o /dev/null -w "%{http_code}" "$1" || echo 000; }
gf_ok(){ local body; body=$(curl -s "http://127.0.0.1:${GRAFANA_PORT:-3000}/api/health" || true); echo "$body" | grep -q '"database"\s*:\s*"ok"'; }
RETRIES=${RETRIES:-60}; SLEEP=${SLEEP:-5}; ok=0
info "等待基础服务就绪 (<= $((RETRIES*SLEEP))s)"
for i in $(seq 1 "$RETRIES"); do
e1=$(code "http://127.0.0.1:${MASTER_PORT:-32300}/readyz")
e2=$(code "http://127.0.0.1:${PROMETHEUS_PORT:-9090}/-/ready")
e3=$(code "http://127.0.0.1:${ALERTMANAGER_PORT:-9093}/api/v2/status")
e4=000; gf_ok && e4=200
info "[ready] t=$((i*SLEEP))s master=$e1 prom=$e2 graf=$e4 alert=$e3"
[[ "$e1" == 200 ]] && ok=$((ok+1))
[[ "$e2" == 200 ]] && ok=$((ok+1))
[[ "$e3" == 200 ]] && ok=$((ok+1))
[[ "$e4" == 200 ]] && ok=$((ok+1))
if [[ $ok -ge 4 ]]; then break; fi; ok=0; sleep "$SLEEP"
done
[[ $ok -ge 4 ]] || err "部分服务未就绪(可稍后执行 scripts/selfcheck.sh 进行复查)"
# 安装报告(ARM 精简版)
ts=$(date +%Y%m%d-%H%M%S)
RPT="$PKG_ROOT/安装报告_ARM_${ts}.md"
{
echo "# Argus ARM Server 安装报告 (${ts})"
echo
echo "## 端口映射"
echo "- MASTER_PORT=${MASTER_PORT}"
echo "- PROMETHEUS_PORT=${PROMETHEUS_PORT}"
echo "- GRAFANA_PORT=${GRAFANA_PORT}"
echo "- ALERTMANAGER_PORT=${ALERTMANAGER_PORT}"
echo "- WEB_PROXY_PORT_8080=${WEB_PROXY_PORT_8080} ... 8085=${WEB_PROXY_PORT_8085}"
echo
echo "## 网络"
echo "- NET=${NET_NAME}"
echo "- ARGUS_OVERLAY_NET=${ARGUS_OVERLAY_NET:-argus-sys-net}"
echo
echo "## 健康检查(简要)"
echo "- master/readyz=$(code http://127.0.0.1:${MASTER_PORT:-32300}/readyz)"
echo "- prometheus/ready=$(code http://127.0.0.1:${PROMETHEUS_PORT:-9090}/-/ready)"
echo "- grafana/api/health=$(code http://127.0.0.1:${GRAFANA_PORT:-3000}/api/health)"
echo "- alertmanager/api/v2/status=$(code http://127.0.0.1:${ALERTMANAGER_PORT:-9093}/api/v2/status)"
} > "$RPT"
info "已生成报告: $RPT"
# 刷新 web-proxy 上的 nginx 配置(best-effort)
docker exec argus-web-proxy nginx -t >/dev/null 2>&1 && docker exec argus-web-proxy nginx -s reload >/dev/null 2>&1 || true
info "安装完成。可通过 scripts/status.sh 查看当前服务状态。"

View File

@@ -0,0 +1,74 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
log() { echo -e "\033[0;34m[CHECK]\033[0m $*"; }
err() { echo -e "\033[0;31m[ERROR]\033[0m $*" >&2; }
ENV_FILE="$ROOT/compose/.env"; [[ -f "$ENV_FILE" ]] && set -a && source "$ENV_FILE" && set +a
wait_http() { local url="$1"; local attempts=${2:-120}; local i=1; while ((i<=attempts)); do curl -fsS "$url" >/dev/null 2>&1 && return 0; echo "[..] waiting $url ($i/$attempts)"; sleep 5; ((i++)); done; return 1; }
code_for() { curl -s -o /dev/null -w "%{http_code}" "$1" || echo 000; }
header_val() { curl -s -D - -o /dev/null "$@" | awk -F': ' 'BEGIN{IGNORECASE=1}$1=="Access-Control-Allow-Origin"{gsub("\r","",$2);print $2}'; }
LOG_DIR="$ROOT/logs"; mkdir -p "$LOG_DIR" || true
OUT_JSON="$LOG_DIR/selfcheck_arm_server.json"; tmp=$(mktemp)
ok=1
log "checking network (${ARGUS_OVERLAY_NET:-argus-sys-net})"
net_ok=false
if docker network inspect "${ARGUS_OVERLAY_NET:-argus-sys-net}" >/dev/null 2>&1; then
net_ok=true
fi
[[ "$net_ok" == true ]] || ok=0
log "checking Master"
[[ $(code_for "http://localhost:${MASTER_PORT:-32300}/readyz") == 200 ]] || ok=0
log "checking Prometheus"
wait_http "http://localhost:${PROMETHEUS_PORT:-9090}/-/ready" 60 || ok=0
log "checking Grafana"
gf_code=$(code_for "http://localhost:${GRAFANA_PORT:-3000}/api/health")
gf_ok=false
if [[ "$gf_code" == 200 ]]; then
body=$(curl -sS "http://localhost:${GRAFANA_PORT:-3000}/api/health" || true)
echo "$body" | grep -q '"database"\s*:\s*"ok"' && gf_ok=true
fi
[[ "$gf_ok" == true ]] || ok=0
log "checking Alertmanager"
wait_http "http://localhost:${ALERTMANAGER_PORT:-9093}/api/v2/status" 60 || ok=0
log "checking Web-Proxy (CORS)"
cors8084=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8084:-8084}/api/v2/status" || true)
cors8085=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8085:-8085}/api/v1/master/nodes" || true)
wp_ok=true
[[ -n "$cors8084" && -n "$cors8085" ]] || wp_ok=false
[[ "$wp_ok" == true ]] || ok=0
cat > "$tmp" <<JSON
{
"overlay_net": $net_ok,
"master_readyz": true,
"prometheus": true,
"grafana": $gf_ok,
"alertmanager": true,
"web_proxy_cors": $wp_ok,
"timestamp": "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
}
JSON
mv "$tmp" "$OUT_JSON" 2>/dev/null || cp "$tmp" "$OUT_JSON"
if [[ "$ok" == 1 ]]; then
log "selfcheck OK -> $OUT_JSON"
exit 0
else
err "selfcheck FAILED -> $OUT_JSON"
exit 1
fi

View File

@@ -0,0 +1,15 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
if [[ -f "$ENV_FILE" ]]; then
set -a; source "$ENV_FILE"; set +a
fi
PROJECT="${COMPOSE_PROJECT_NAME:-argus-server-arm}"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" ps

View File

@@ -0,0 +1,28 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
if [[ -f "$ENV_FILE" ]]; then
set -a; source "$ENV_FILE"; set +a
fi
PROJECT="${COMPOSE_PROJECT_NAME:-argus-server-arm}"
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require_compose
echo "[UNINSTALL] stopping compose (project=$PROJECT)"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" down --remove-orphans || true
echo "[UNINSTALL] done"

View File

@@ -0,0 +1,116 @@
#!/usr/bin/env bash
set -euo pipefail
# x86 专用:为 ARM Server 包生成 compose/.env
# - 带磁盘空间检查
# - 自动分配端口,避免与本机已有服务冲突
# - 写入 ARGUS_OVERLAY_NET 和 ARGUS_BUILD_UID/GID
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_EX="$PKG_ROOT/compose/.env.example"
ENV_OUT="$PKG_ROOT/compose/.env"
info(){ echo -e "\033[34m[CONFIG_X86]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR_X86]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
# Compose 检测:优先 docker compose(v2),回退 docker-compose(v1)
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose,请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker awk sed
require_compose
[[ -f "$ENV_EX" ]] || { err "缺少模板文件: $ENV_EX"; exit 1; }
# 磁盘空间检查(简化版,默认 >= 5GB)
check_disk(){ local p="$1"; local need="${2:-5120}"; local free;
free=$(df -Pm "$p" 2>/dev/null | awk 'NR==2{print $4+0}')
if [[ -z "$free" || "$free" -lt "$need" ]]; then
err "磁盘空间不足: $p 剩余 ${free:-0}MB (<${need}MB),请清理后再继续"
fi
}
check_disk "$PKG_ROOT" 5120 || true
check_disk "/var/lib/docker" 5120 || true
cp "$ENV_EX" "$ENV_OUT"
# 读取/生成 SWARM_MANAGER_ADDR(与 ARM server 配置保持一致)
SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR:-}
if [[ -z "${SWARM_MANAGER_ADDR}" ]]; then
read -rp "请输入本机管理地址 SWARM_MANAGER_ADDR: " SWARM_MANAGER_ADDR
fi
info "SWARM_MANAGER_ADDR=$SWARM_MANAGER_ADDR"
# 校验 IP 属于本机网卡
if ! ip -o addr | awk '{print $4}' | cut -d'/' -f1 | grep -qx "$SWARM_MANAGER_ADDR"; then
err "SWARM_MANAGER_ADDR 非本机地址: $SWARM_MANAGER_ADDR"; exit 1; fi
info "开始分配服务端口(起始=20000,避免系统占用与相互冲突)"
is_port_used(){ local p="$1"; ss -tulnH 2>/dev/null | awk '{print $5}' | sed 's/.*://g' | grep -qx "$p"; }
declare -A PRESENT=() CHOSEN=() USED=()
START_PORT="${START_PORT:-20000}"; cur=$START_PORT
ORDER=(MASTER_PORT PROMETHEUS_PORT GRAFANA_PORT ALERTMANAGER_PORT \
WEB_PROXY_PORT_8080 WEB_PROXY_PORT_8081 WEB_PROXY_PORT_8082 \
WEB_PROXY_PORT_8083 WEB_PROXY_PORT_8084 WEB_PROXY_PORT_8085)
# 标记 .env.example 中实际存在的键
for key in "${ORDER[@]}"; do
if grep -q "^${key}=" "$ENV_EX"; then PRESENT[$key]=1; fi
done
next_free(){ local p="$1"; while :; do if [[ -n "${USED[$p]:-}" ]] || is_port_used "$p"; then p=$((p+1)); else echo "$p"; return; fi; done; }
for key in "${ORDER[@]}"; do
[[ -z "${PRESENT[$key]:-}" ]] && continue
p=$(next_free "$cur"); CHOSEN[$key]="$p"; USED[$p]=1; cur=$((p+1))
done
info "端口分配结果:MASTER=${CHOSEN[MASTER_PORT]:-} PROM=${CHOSEN[PROMETHEUS_PORT]:-} GRAFANA=${CHOSEN[GRAFANA_PORT]:-} ALERT=${CHOSEN[ALERTMANAGER_PORT]:-} WEB_PROXY(8080..8085)=${CHOSEN[WEB_PROXY_PORT_8080]:-}/${CHOSEN[WEB_PROXY_PORT_8081]:-}/${CHOSEN[WEB_PROXY_PORT_8082]:-}/${CHOSEN[WEB_PROXY_PORT_8083]:-}/${CHOSEN[WEB_PROXY_PORT_8084]:-}/${CHOSEN[WEB_PROXY_PORT_8085]:-}"
# 覆盖端口(按唯一化结果写回)
for key in "${ORDER[@]}"; do
val="${CHOSEN[$key]:-}"
[[ -z "$val" ]] && continue
sed -i -E "s#^$key=.*#$key=${val}#" "$ENV_OUT"
done
# 覆盖/补充 Overlay 名称
grep -q '^ARGUS_OVERLAY_NET=' "$ENV_OUT" || echo 'ARGUS_OVERLAY_NET=argus-sys-net' >> "$ENV_OUT"
# 覆盖/补充构建账户 UID/GID,避免权限问题
RUID=$(id -u)
RGID=$(id -g)
if grep -q '^ARGUS_BUILD_UID=' "$ENV_OUT"; then
sed -i -E "s#^ARGUS_BUILD_UID=.*#ARGUS_BUILD_UID=${RUID}#" "$ENV_OUT"
else
echo "ARGUS_BUILD_UID=${RUID}" >> "$ENV_OUT"
fi
if grep -q '^ARGUS_BUILD_GID=' "$ENV_OUT"; then
sed -i -E "s#^ARGUS_BUILD_GID=.*#ARGUS_BUILD_GID=${RGID}#" "$ENV_OUT"
else
echo "ARGUS_BUILD_GID=${RGID}" >> "$ENV_OUT"
fi
# 更新 cluster-info.env 的 SWARM_MANAGER_ADDR(ARM + x86 一致)
CI="$PKG_ROOT/cluster-info.env"
if [[ -f "$CI" ]]; then
if grep -q '^SWARM_MANAGER_ADDR=' "$CI"; then
sed -i -E "s#^SWARM_MANAGER_ADDR=.*#SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}#" "$CI"
else
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}" >> "$CI"
fi
else
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}" > "$CI"
fi
info "已生成 compose/.env 并更新 cluster-info.env 的 SWARM_MANAGER_ADDR。"
info "下一步可执行: scripts_for_x86/install.sh"

View File

@@ -0,0 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
exec "$SCRIPT_DIR/../scripts/diagnose.sh" "$@"

View File

@@ -0,0 +1,77 @@
#!/usr/bin/env bash
set -euo pipefail
# 在 x86_64 主机上安装 ARM server 包:
# 1) 确认/安装 binfmt + QEMU(通过 tonistiigi/binfmt);
# 2) 调用通用的 scripts/install.sh 启动服务。
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT_DIR="$(cd "$SCRIPT_DIR/.." && pwd)"
info(){ echo -e "\033[34m[INSTALL_X86]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR_X86]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
ensure_binfmt_arm64() {
info "检查当前主机架构: $(uname -m)"
if [[ "$(uname -m)" != "x86_64" && "$(uname -m)" != "amd64" ]]; then
info "当前并非 x86_64($(uname -m)),通常应直接在 ARM 服务器上使用 scripts/install.sh。继续执行但不强制安装 binfmt。"
return 0
fi
require docker
# 优先使用随安装包一起提供的 ubuntu-22.04-arm-test 镜像做自检
local TEST_IMG="ubuntu:22.04"
local TEST_TAR=""
if [[ -d "$ROOT_DIR/images" ]]; then
TEST_TAR=$(ls "$ROOT_DIR"/images/ubuntu-22.04-arm-test*.tar.gz 2>/dev/null | head -n1 || true)
fi
if [[ -n "$TEST_TAR" && ! $(docker image inspect "$TEST_IMG" >/dev/null 2>&1; echo $?) -eq 0 ]]; then
info "从安装包加载 ARM 测试基础镜像: $(basename "$TEST_TAR")"
tmp=$(mktemp)
gunzip -c "$TEST_TAR" > "$tmp"
docker load -i "$tmp" >/dev/null
rm -f "$tmp"
fi
info "验证是否已能运行 linux/arm64 容器..."
if docker run --rm --platform=linux/arm64 "$TEST_IMG" uname -m 2>/dev/null | grep -q 'aarch64'; then
info "检测到本机已支持 ARM64 容器运行 (uname -m = aarch64)"
return 0
fi
info "未检测到 ARM64 运行能力,尝试通过 tonistiigi/binfmt 安装 binfmt/QEMU ..."
# 优先从离线包加载 tonistiigi/binfmt 镜像
local BINFMT_IMG="tonistiigi/binfmt:latest"
local BINFMT_TAR=""
if [[ -d "$ROOT_DIR/images" ]]; then
BINFMT_TAR=$(ls "$ROOT_DIR"/images/tonistiigi-binfmt*.tar.gz 2>/dev/null | head -n1 || true)
fi
if [[ -n "$BINFMT_TAR" && ! $(docker image inspect "$BINFMT_IMG" >/dev/null 2>&1; echo $?) -eq 0 ]]; then
info "从安装包加载 tonistiigi/binfmt 离线镜像: $(basename "$BINFMT_TAR")"
tmp_b=$(mktemp)
gunzip -c "$BINFMT_TAR" > "$tmp_b"
docker load -i "$tmp_b" >/dev/null
rm -f "$tmp_b"
fi
if ! docker run --privileged --rm "$BINFMT_IMG" --install all; then
err "tonistiigi/binfmt 安装失败,请检查 Docker 权限或确认离线镜像是否完整。"
exit 1
fi
info "再次验证 ARM64 容器可运行状态..."
if ! docker run --rm --platform=linux/arm64 "$TEST_IMG" uname -m 2>/dev/null | grep -q 'aarch64'; then
err "安装 binfmt/QEMU 后仍无法运行 linux/arm64 容器,请手工排查 binfmt_misc 配置。"
exit 1
fi
info "ARM64 容器运行能力检查通过。"
}
ensure_binfmt_arm64
# 准备完 x86 -> ARM 运行环境后,交由通用 install.sh 负责后续部署
exec "$SCRIPT_DIR/../scripts/install.sh" "$@"

View File

@@ -0,0 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
exec "$SCRIPT_DIR/../scripts/selfcheck.sh" "$@"

View File

@@ -0,0 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
exec "$SCRIPT_DIR/../scripts/status.sh" "$@"

View File

@@ -0,0 +1,6 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
exec "$SCRIPT_DIR/../scripts/uninstall.sh" "$@"

BIN
doc/metric_lists.xlsx Normal file

Binary file not shown.

View File

@@ -37,22 +37,11 @@ _argus_is_number() {
[[ "$1" =~ ^[0-9]+$ ]]
}
load_build_user() {
if [[ "$_ARGUS_BUILD_USER_LOADED" == "1" ]]; then
return 0
fi
local project_root config_files config uid gid
project_root="$(argus_project_root)"
config_files=(
"$project_root/configs/build_user.local.conf"
"$project_root/configs/build_user.conf"
)
uid="$ARGUS_BUILD_UID_DEFAULT"
gid="$ARGUS_BUILD_GID_DEFAULT"
for config in "${config_files[@]}"; do
_argus_read_user_from_files() {
local uid_out_var="$1" gid_out_var="$2"; shift 2
local uid_val="$ARGUS_BUILD_UID_DEFAULT" gid_val="$ARGUS_BUILD_GID_DEFAULT"
local config
for config in "$@"; do
if [[ -f "$config" ]]; then
while IFS= read -r raw_line || [[ -n "$raw_line" ]]; do
local line key value
@@ -68,42 +57,58 @@ load_build_user() {
key="$(_argus_trim "$key")"
value="$(_argus_trim "$value")"
case "$key" in
UID)
uid="$value"
;;
GID)
gid="$value"
;;
*)
echo "[ARGUS build_user] Unknown key '$key' in $config" >&2
;;
UID) uid_val="$value" ;;
GID) gid_val="$value" ;;
*) echo "[ARGUS build_user] Unknown key '$key' in $config" >&2 ;;
esac
done < "$config"
break
fi
done
printf -v "$uid_out_var" '%s' "$uid_val"
printf -v "$gid_out_var" '%s' "$gid_val"
}
if [[ -n "${ARGUS_BUILD_UID:-}" ]]; then
uid="$ARGUS_BUILD_UID"
fi
if [[ -n "${ARGUS_BUILD_GID:-}" ]]; then
gid="$ARGUS_BUILD_GID"
load_build_user_profile() {
local profile="${1:-default}"
if [[ "$_ARGUS_BUILD_USER_LOADED" == "1" ]]; then
return 0
fi
local project_root uid gid
project_root="$(argus_project_root)"
case "$profile" in
pkg)
_argus_read_user_from_files uid gid \
"$project_root/configs/build_user.pkg.conf" \
"$project_root/configs/build_user.local.conf" \
"$project_root/configs/build_user.conf"
;;
default|*)
_argus_read_user_from_files uid gid \
"$project_root/configs/build_user.local.conf" \
"$project_root/configs/build_user.conf"
;;
esac
if [[ -n "${ARGUS_BUILD_UID:-}" ]]; then uid="$ARGUS_BUILD_UID"; fi
if [[ -n "${ARGUS_BUILD_GID:-}" ]]; then gid="$ARGUS_BUILD_GID"; fi
if ! _argus_is_number "$uid"; then
echo "[ARGUS build_user] Invalid UID '$uid'" >&2
return 1
echo "[ARGUS build_user] Invalid UID '$uid'" >&2; return 1
fi
if ! _argus_is_number "$gid"; then
echo "[ARGUS build_user] Invalid GID '$gid'" >&2
return 1
echo "[ARGUS build_user] Invalid GID '$gid'" >&2; return 1
fi
export ARGUS_BUILD_UID="$uid"
export ARGUS_BUILD_GID="$gid"
_ARGUS_BUILD_USER_LOADED=1
}
load_build_user() {
local profile="${ARGUS_BUILD_PROFILE:-default}"
load_build_user_profile "$profile"
}
argus_build_user_args() {
load_build_user
printf '%s' "--build-arg ARGUS_BUILD_UID=${ARGUS_BUILD_UID} --build-arg ARGUS_BUILD_GID=${ARGUS_BUILD_GID}"

View File

@@ -3,3 +3,4 @@ build/
__pycache__/
.env
dist/

View File

@@ -4,6 +4,7 @@ import os
import re
import socket
import subprocess
import ipaddress
from pathlib import Path
from typing import Any, Dict
@@ -16,11 +17,47 @@ _HOSTNAME_PATTERN = re.compile(r"^([^-]+)-([^-]+)-([^-]+)-.*$")
def collect_metadata(config: AgentConfig) -> Dict[str, Any]:
"""汇总节点注册需要的静态信息。"""
"""汇总节点注册需要的静态信息,带有更智能的 IP 选择。
规则(从高到低):
1) AGENT_PUBLISH_IP 指定;
2) Hostname A 记录(若命中优先网段);
3) 网卡扫描:排除 AGENT_EXCLUDE_IFACES,优先 AGENT_PREFER_NET_CIDRS;
4) 默认路由回退(UDP socket 技巧)。
额外发布:overlay_ip / gwbridge_ip / interfaces,便于 Master 与诊断使用。
"""
hostname = config.hostname
meta = {
prefer_cidrs = _read_cidrs_env(
os.environ.get("AGENT_PREFER_NET_CIDRS", "10.0.0.0/8,172.31.0.0/16")
)
exclude_ifaces = _read_csv_env(
os.environ.get("AGENT_EXCLUDE_IFACES", "docker_gwbridge,lo")
)
# interface inventory
interfaces = _list_global_ipv4_addrs()
if exclude_ifaces:
interfaces = [it for it in interfaces if it[0] not in set(exclude_ifaces)]
# resolve hostname candidates
host_ips = _resolve_hostname_ips(hostname)
selected_ip, overlay_ip, gwbridge_ip = _select_publish_ips(
interfaces=interfaces,
host_ips=host_ips,
prefer_cidrs=prefer_cidrs,
)
meta: Dict[str, Any] = {
"hostname": hostname,
"ip": _detect_ip_address(),
"ip": os.environ.get("AGENT_PUBLISH_IP", selected_ip), # keep required field
"overlay_ip": overlay_ip or selected_ip,
"gwbridge_ip": gwbridge_ip,
"interfaces": [
{"iface": name, "ip": ip} for name, ip in interfaces
],
"env": config.environment,
"user": config.user,
"instance": config.instance,
@@ -96,7 +133,7 @@ def _detect_gpu_count() -> int:
def _detect_ip_address() -> str:
"""尝试通过 UDP socket 获得容器出口 IP,失败则回退解析主机名。"""
"""保留旧接口,作为最终回退:默认路由源地址 → 主机名解析 → 127.0.0.1。"""
try:
with socket.socket(socket.AF_INET, socket.SOCK_DGRAM) as sock:
sock.connect(("8.8.8.8", 80))
@@ -108,3 +145,118 @@ def _detect_ip_address() -> str:
except OSError:
LOGGER.warning("Unable to resolve hostname to IP; defaulting to 127.0.0.1")
return "127.0.0.1"
def _read_csv_env(raw: str | None) -> list[str]:
if not raw:
return []
return [x.strip() for x in raw.split(",") if x.strip()]
def _read_cidrs_env(raw: str | None) -> list[ipaddress.IPv4Network]:
cidrs: list[ipaddress.IPv4Network] = []
for item in _read_csv_env(raw):
try:
net = ipaddress.ip_network(item, strict=False)
if isinstance(net, (ipaddress.IPv4Network,)):
cidrs.append(net)
except ValueError:
LOGGER.warning("Ignoring invalid CIDR in AGENT_PREFER_NET_CIDRS", extra={"cidr": item})
return cidrs
def _list_global_ipv4_addrs() -> list[tuple[str, str]]:
"""列出 (iface, ip) 形式的全局 IPv4 地址。
依赖 iproute2:ip -4 -o addr show scope global
"""
results: list[tuple[str, str]] = []
try:
proc = subprocess.run(
["sh", "-lc", "ip -4 -o addr show scope global | awk '{print $2, $4}'"],
check=False,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
text=True,
timeout=3,
)
if proc.returncode == 0:
for line in proc.stdout.splitlines():
line = line.strip()
if not line:
continue
parts = line.split()
if len(parts) != 2:
continue
iface, cidr = parts
ip = cidr.split("/")[0]
try:
ipaddress.IPv4Address(ip)
except ValueError:
continue
results.append((iface, ip))
except Exception as exc: # pragma: no cover - defensive
LOGGER.debug("Failed to list interfaces", extra={"error": str(exc)})
return results
def _resolve_hostname_ips(name: str) -> list[str]:
ips: list[str] = []
try:
infos = socket.getaddrinfo(name, None, family=socket.AF_INET)
for info in infos:
ip = info[4][0]
if ip not in ips:
ips.append(ip)
except OSError:
pass
return ips
def _pick_by_cidrs(candidates: list[str], prefer_cidrs: list[ipaddress.IPv4Network]) -> str | None:
for net in prefer_cidrs:
for ip in candidates:
try:
if ipaddress.ip_address(ip) in net:
return ip
except ValueError:
continue
return None
def _select_publish_ips(
*,
interfaces: list[tuple[str, str]],
host_ips: list[str],
prefer_cidrs: list[ipaddress.IPv4Network],
) -> tuple[str, str | None, str | None]:
"""返回 (selected_ip, overlay_ip, gwbridge_ip)。
- overlay_ip:优先命中 prefer_cidrs(10.0/8 先于 172.31/16)。
- gwbridge_ip:若存在 172.22/16 则记录。
- selected_ip:优先 AGENT_PUBLISH_IP;否则 overlay_ip;否则 hostname A 记录中的 prefer;否则默认路由回退。
"""
# detect gwbridge (172.22/16)
gwbridge_net = ipaddress.ip_network("172.22.0.0/16")
gwbridge_ip = None
for _, ip in interfaces:
try:
if ipaddress.ip_address(ip) in gwbridge_net:
gwbridge_ip = ip
break
except ValueError:
continue
# overlay candidate from interfaces by prefer cidrs
iface_ips = [ip for _, ip in interfaces]
overlay_ip = _pick_by_cidrs(iface_ips, prefer_cidrs)
# hostname A records filtered by prefer cidrs
host_pref = _pick_by_cidrs(host_ips, prefer_cidrs)
env_ip = os.environ.get("AGENT_PUBLISH_IP")
if env_ip:
selected = env_ip
else:
selected = overlay_ip or host_pref or _detect_ip_address()
return selected, overlay_ip, gwbridge_ip

Binary file not shown.

View File

@@ -11,12 +11,13 @@ RUN apt-get update && \
# 设置 Alertmanager 版本(与本地离线包保持一致)
ARG ALERTMANAGER_VERSION=0.28.1
ARG ALERTMANAGER_ARCH=amd64
# 使用仓库内预置的离线包构建(无需联网)
COPY src/alert/alertmanager/build/alertmanager-${ALERTMANAGER_VERSION}.linux-amd64.tar.gz /tmp/
RUN tar xvf /tmp/alertmanager-${ALERTMANAGER_VERSION}.linux-amd64.tar.gz -C /tmp && \
mv /tmp/alertmanager-${ALERTMANAGER_VERSION}.linux-amd64 /usr/local/alertmanager && \
rm -f /tmp/alertmanager-${ALERTMANAGER_VERSION}.linux-amd64.tar.gz
COPY src/alert/alertmanager/build/alertmanager-${ALERTMANAGER_VERSION}.linux-${ALERTMANAGER_ARCH}.tar.gz /tmp/
RUN tar xvf /tmp/alertmanager-${ALERTMANAGER_VERSION}.linux-${ALERTMANAGER_ARCH}.tar.gz -C /tmp && \
mv /tmp/alertmanager-${ALERTMANAGER_VERSION}.linux-${ALERTMANAGER_ARCH} /usr/local/alertmanager && \
rm -f /tmp/alertmanager-${ALERTMANAGER_VERSION}.linux-${ALERTMANAGER_ARCH}.tar.gz
ENV ALERTMANAGER_BASE_PATH=/private/argus/alert/alertmanager
@@ -31,26 +32,31 @@ RUN mkdir -p /usr/share/alertmanager && \
rm -rf /alertmanager && \
ln -s ${ALERTMANAGER_BASE_PATH} /alertmanager
# 创建 alertmanager 用户(可自定义 UID/GID)
# 创建 alertmanager 用户组
# 确保 ubuntu 账户存在并使用 ARGUS_BUILD_UID/GID
RUN set -eux; \
# 确保目标 GID 存在;若已被占用,直接使用该 GID(组名不限)\
if ! getent group "${ARGUS_BUILD_GID}" >/dev/null; then \
groupadd -g "${ARGUS_BUILD_GID}" alertmanager || true; \
fi; \
# 确保存在 alertmanager 用户;若 UID 已被占用,跳过并继续使用现有 UID 的用户
if ! id alertmanager >/dev/null 2>&1; then \
if getent passwd "${ARGUS_BUILD_UID}" >/dev/null; then \
# UID 已占用,则创建同名用户但不指定 UID(避免冲突),仅保证 user 存在
useradd -M -s /usr/sbin/nologin -g "${ARGUS_BUILD_GID}" alertmanager || true; \
else \
useradd -M -s /usr/sbin/nologin -u "${ARGUS_BUILD_UID}" -g "${ARGUS_BUILD_GID}" alertmanager || true; \
fi; \
# 确保存在目标 GID 的组;若不存在则优先尝试将 ubuntu 组改为该 GID,否则创建新组
if getent group "${ARGUS_BUILD_GID}" >/dev/null; then \
:; \
else \
usermod -g "${ARGUS_BUILD_GID}" alertmanager || true; \
fi
RUN chown -R "${ARGUS_BUILD_UID}:${ARGUS_BUILD_GID}" /usr/share/alertmanager /alertmanager ${ALERTMANAGER_BASE_PATH} /private/argus/etc /usr/local/bin || true
if getent group ubuntu >/dev/null; then \
groupmod -g "${ARGUS_BUILD_GID}" ubuntu || true; \
else \
groupadd -g "${ARGUS_BUILD_GID}" ubuntu || groupadd -g "${ARGUS_BUILD_GID}" argus || true; \
fi; \
fi; \
# 创建或调整 ubuntu 用户
if id ubuntu >/dev/null 2>&1; then \
# 设置主组为目标 GID(可用 GID 数字指定)
usermod -g "${ARGUS_BUILD_GID}" ubuntu || true; \
# 若目标 UID 未被占用,则更新 ubuntu 的 UID
if [ "$(id -u ubuntu)" != "${ARGUS_BUILD_UID}" ] && ! getent passwd "${ARGUS_BUILD_UID}" >/dev/null; then \
usermod -u "${ARGUS_BUILD_UID}" ubuntu || true; \
fi; \
else \
useradd -m -s /bin/bash -u "${ARGUS_BUILD_UID}" -g "${ARGUS_BUILD_GID}" ubuntu || true; \
fi; \
# 调整关键目录属主为 ubuntu UID/GID
chown -R "${ARGUS_BUILD_UID}:${ARGUS_BUILD_GID}" /usr/share/alertmanager /alertmanager ${ALERTMANAGER_BASE_PATH} /private/argus/etc /usr/local/bin || true
# 配置内网 apt 源 (如果指定了内网选项)
RUN if [ "$USE_INTRANET" = "true" ]; then \

5
src/alert/alertmanager/build/fetch-dist.sh Normal file → Executable file
View File

@@ -6,9 +6,11 @@ set -euo pipefail
# ./fetch-dist.sh [version]
# 示例:
# ./fetch-dist.sh 0.28.1
# ARCH=arm64 ./fetch-dist.sh 0.28.1
VER="${1:-0.28.1}"
OUT="alertmanager-${VER}.linux-amd64.tar.gz"
ARCH="${ARCH:-amd64}" # amd64 或 arm64
OUT="alertmanager-${VER}.linux-${ARCH}.tar.gz"
URL="https://github.com/prometheus/alertmanager/releases/download/v${VER}/${OUT}"
if [[ -f "$OUT" ]]; then
@@ -19,4 +21,3 @@ fi
echo "[INFO] Downloading $URL"
curl -fL --retry 3 --connect-timeout 10 -o "$OUT" "$URL"
echo "[OK] Saved to $(pwd)/$OUT"

1
src/bundle/cpu-node-bundle/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
.build*/

View File

@@ -0,0 +1,33 @@
FROM ubuntu:22.04
ARG ARGUS_BUILD_UID=2133
ARG ARGUS_BUILD_GID=2015
ENV DEBIAN_FRONTEND=noninteractive \
TZ=Asia/Shanghai \
ARGUS_LOGS_WORLD_WRITABLE=1
RUN set -eux; \
apt-get update; \
apt-get install -y --no-install-recommends \
ca-certificates curl wget iproute2 iputils-ping net-tools jq tzdata \
cron procps supervisor vim less tar gzip python3; \
rm -rf /var/lib/apt/lists/*; \
ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
WORKDIR /
# Offline fluent-bit assets and bundle tarball are staged by the build script
COPY node-bootstrap.sh /usr/local/bin/node-bootstrap.sh
COPY health-watcher.sh /usr/local/bin/health-watcher.sh
COPY private/start-fluent-bit.sh /private/start-fluent-bit.sh
COPY private/etc /private/etc
COPY private/packages /private/packages
COPY bundle/ /bundle/
RUN chmod +x /usr/local/bin/node-bootstrap.sh /usr/local/bin/health-watcher.sh /private/start-fluent-bit.sh || true; \
mkdir -p /logs/train /logs/infer /buffers /opt/argus-metric; \
if [ "${ARGUS_LOGS_WORLD_WRITABLE}" = "1" ]; then chmod 1777 /logs/train /logs/infer || true; else chmod 755 /logs/train /logs/infer || true; fi; \
chmod 770 /buffers || true
ENTRYPOINT ["/usr/local/bin/node-bootstrap.sh"]

View File

@@ -0,0 +1,59 @@
#!/usr/bin/env bash
set -euo pipefail
# health-watcher.sh (CPU node bundle)
# 周期执行 check_health.sh 与 restart_unhealthy.sh,用于节点容器内自愈。
INSTALL_ROOT="/opt/argus-metric"
INTERVAL="${HEALTH_WATCH_INTERVAL:-60}"
VER_DIR="${1:-}"
log(){ echo "[HEALTH-WATCHER] $*"; }
resolve_ver_dir() {
local dir=""
if [[ -n "${VER_DIR:-}" && -d "$VER_DIR" ]]; then
dir="$VER_DIR"
elif [[ -L "$INSTALL_ROOT/current" ]]; then
dir="$(readlink -f "$INSTALL_ROOT/current" 2>/dev/null || true)"
fi
if [[ -z "$dir" ]]; then
dir="$(ls -d "$INSTALL_ROOT"/versions/* 2>/dev/null | sort -V | tail -n1 || true)"
fi
echo "$dir"
}
main() {
log "starting with interval=${INTERVAL}s"
local dir
dir="$(resolve_ver_dir)"
if [[ -z "$dir" || ! -d "$dir" ]]; then
log "no valid install dir found under $INSTALL_ROOT; exiting"
exit 0
fi
local chk="$dir/check_health.sh"
local rst="$dir/restart_unhealthy.sh"
if [[ ! -x "$chk" && ! -x "$rst" ]]; then
log "neither check_health.sh nor restart_unhealthy.sh is executable under $dir; exiting"
exit 0
fi
log "watching install dir: $dir"
while :; do
if [[ -x "$chk" ]]; then
log "running check_health.sh"
"$chk" >> "$dir/.health_check.watch.log" 2>&1 || log "check_health.sh reported issues (see .health_check.watch.log)"
fi
if [[ -x "$rst" ]]; then
log "running restart_unhealthy.sh"
"$rst" >> "$dir/.restart.watch.log" 2>&1 || log "restart_unhealthy.sh reported issues (see .restart.watch.log)"
fi
sleep "$INTERVAL"
done
}
main "$@"

View File

@@ -0,0 +1,131 @@
#!/usr/bin/env bash
set -euo pipefail
echo "[BOOT] CPU node bundle starting"
INSTALL_ROOT="/opt/argus-metric"
BUNDLE_DIR="/bundle"
STATE_DIR_BASE="/private/argus/agent"
mkdir -p "$INSTALL_ROOT" "$STATE_DIR_BASE" /logs/train /logs/infer /buffers || true
# Ensure world-writable logs dir with sticky bit (align with deployment_new policy)
if [[ "${ARGUS_LOGS_WORLD_WRITABLE:-1}" == "1" ]]; then
chmod 1777 /logs/train /logs/infer || true
else
chmod 755 /logs/train /logs/infer || true
fi
chmod 770 /buffers || true
installed_ok=0
# 1) already installed?
if [[ -L "$INSTALL_ROOT/current" && -d "$INSTALL_ROOT/current" ]]; then
echo "[BOOT] client already installed at $INSTALL_ROOT/current"
else
# 2) try local bundle first (argus-metric_*.tar.gz)
tarball=$(ls -1 "$BUNDLE_DIR"/argus-metric_*.tar.gz 2>/dev/null | head -1 || true)
if [[ -n "${tarball:-}" ]]; then
echo "[BOOT] installing from local bundle: $(basename "$tarball")"
tmp=$(mktemp -d)
tar -xzf "$tarball" -C "$tmp"
# locate root containing version.json
root="$tmp"
if [[ ! -f "$root/version.json" ]]; then
sub=$(find "$tmp" -mindepth 1 -maxdepth 1 -type d | head -n1 || true)
[[ -n "$sub" && -f "$sub/version.json" ]] && root="$sub"
fi
if [[ ! -f "$root/version.json" ]]; then
echo "[BOOT][WARN] version.json not found in bundle; fallback to FTP"
else
ver=$(sed -n 's/.*"version"\s*:\s*"\([^"]\+\)".*/\1/p' "$root/version.json" | head -n1)
if [[ -z "$ver" ]]; then
echo "[BOOT][WARN] failed to parse version from version.json; fallback to FTP"
else
target_root="$INSTALL_ROOT"
version_dir="$target_root/versions/$ver"
mkdir -p "$version_dir"
shopt -s dotglob
mv "$root"/* "$version_dir/" 2>/dev/null || true
shopt -u dotglob
if [[ -f "$version_dir/install.sh" ]]; then
chmod +x "$version_dir/install.sh" 2>/dev/null || true
(
export AUTO_START_DCGM="0" # N/A on CPU
cd "$version_dir" && ./install.sh "$version_dir"
)
echo "$ver" > "$target_root/LATEST_VERSION" 2>/dev/null || true
ln -sfn "$version_dir" "$target_root/current" 2>/dev/null || true
if [[ -L "$target_root/current" && -d "$target_root/current" ]]; then
installed_ok=1
echo "[BOOT] local bundle install OK: version=$ver"
else
echo "[BOOT][WARN] current symlink not present after install; will rely on healthcheck to confirm"
fi
else
echo "[BOOT][WARN] install.sh missing under $version_dir; fallback to FTP"
fi
fi
fi
fi
# 3) fallback: use FTP setup if not installed
if [[ ! -L "$INSTALL_ROOT/current" && "$installed_ok" -eq 0 ]]; then
echo "[BOOT] fallback to FTP setup"
if [[ -z "${FTPIP:-}" || -z "${FTP_USER:-}" || -z "${FTP_PASSWORD:-}" ]]; then
echo "[BOOT][ERROR] FTP variables not set (FTPIP/FTP_USER/FTP_PASSWORD)" >&2
exit 1
fi
curl -u "$FTP_USER:$FTP_PASSWORD" -fsSL "ftp://$FTPIP:21/setup.sh" -o /tmp/setup.sh
chmod +x /tmp/setup.sh
/tmp/setup.sh --server "$FTPIP" --user "$FTP_USER" --password "$FTP_PASSWORD" --port 21
fi
fi
# 4) ensure argus-agent is running (best-effort)
if ! pgrep -x argus-agent >/dev/null 2>&1; then
echo "[BOOT] starting argus-agent (not detected)"
setsid /usr/local/bin/argus-agent >/var/log/argus-agent.log 2>&1 < /dev/null &
fi
# 5) post-install selfcheck and state
ver_dir=""
if [[ -L "$INSTALL_ROOT/current" ]]; then
ver_dir="$(readlink -f "$INSTALL_ROOT/current" 2>/dev/null || true)"
fi
if [[ -z "$ver_dir" ]]; then
ver_dir="$(ls -d "$INSTALL_ROOT"/versions/* 2>/dev/null | sort -V | tail -n1 || true)"
fi
if [[ -n "$ver_dir" && -x "$ver_dir/check_health.sh" ]]; then
echo "[BOOT] running initial health check: $ver_dir/check_health.sh"
if "$ver_dir/check_health.sh" >> "$ver_dir/.health_check.init.log" 2>&1; then
echo "[BOOT] initial health check completed (see $ver_dir/.health_check.init.log)"
else
echo "[BOOT][WARN] initial health check reported issues (see $ver_dir/.health_check.init.log)"
fi
else
echo "[BOOT][WARN] initial health check skipped (script missing: $ver_dir/check_health.sh)"
fi
host="$(hostname)"
state_dir="$STATE_DIR_BASE/${host}"
mkdir -p "$state_dir" 2>/dev/null || true
for i in {1..60}; do
if [[ -s "$state_dir/node.json" ]]; then
echo "[BOOT] node state present: $state_dir/node.json"
break
fi
sleep 2
done
# 6) spawn health watcher (best-effort, non-blocking)
if command -v /usr/local/bin/health-watcher.sh >/dev/null 2>&1; then
echo "[BOOT] starting health watcher for $ver_dir"
setsid /usr/local/bin/health-watcher.sh "${ver_dir:-}" >/var/log/health-watcher.log 2>&1 < /dev/null || true &
else
echo "[BOOT][WARN] health-watcher.sh not found; skip health watcher"
fi
echo "[BOOT] ready; entering sleep"
exec sleep infinity

1
src/bundle/gpu-node-bundle/.gitignore vendored Normal file
View File

@@ -0,0 +1 @@
.build*/

View File

@@ -0,0 +1,44 @@
ARG CUDA_VER=12.2.2
FROM nvidia/cuda:${CUDA_VER}-runtime-ubuntu22.04
ARG CLIENT_VER=0.0.0
ARG BUNDLE_DATE=00000000
LABEL org.opencontainers.image.title="argus-sys-metric-test-node-bundle-gpu" \
org.opencontainers.image.description="GPU node bundle with embedded Argus client artifact" \
org.opencontainers.image.version="${CLIENT_VER}" \
org.opencontainers.image.revision_date="${BUNDLE_DATE}" \
maintainer="Argus"
ENV DEBIAN_FRONTEND=noninteractive \
TZ=Asia/Shanghai \
ARGUS_LOGS_WORLD_WRITABLE=1 \
ES_HOST=es.log.argus.com \
ES_PORT=9200 \
CLUSTER=local \
RACK=dev
RUN set -eux; \
apt-get update; \
apt-get install -y --no-install-recommends \
ca-certificates curl wget iproute2 iputils-ping net-tools jq tzdata cron procps vim less \
tar gzip; \
rm -rf /var/lib/apt/lists/*; \
ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ > /etc/timezone
WORKDIR /
# Expect staged build context to provide these directories/files
COPY bundle/ /bundle/
COPY node-bootstrap.sh /usr/local/bin/node-bootstrap.sh
COPY health-watcher.sh /usr/local/bin/health-watcher.sh
COPY private/start-fluent-bit.sh /private/start-fluent-bit.sh
COPY private/etc /private/etc
COPY private/packages /private/packages
RUN chmod +x /usr/local/bin/node-bootstrap.sh /usr/local/bin/health-watcher.sh /private/start-fluent-bit.sh || true; \
mkdir -p /logs/train /logs/infer /buffers /opt/argus-metric; \
chmod 1777 /logs/train /logs/infer || true; \
chmod 770 /buffers || true
ENTRYPOINT ["/usr/local/bin/node-bootstrap.sh"]

View File

@@ -0,0 +1,59 @@
#!/usr/bin/env bash
set -euo pipefail
# health-watcher.sh (GPU bundle)
# 周期执行 check_health.sh 与 restart_unhealthy.sh,用于 GPU 节点容器内自愈。
INSTALL_ROOT="/opt/argus-metric"
INTERVAL="${HEALTH_WATCH_INTERVAL:-60}"
VER_DIR="${1:-}"
log(){ echo "[HEALTH-WATCHER] $*"; }
resolve_ver_dir() {
local dir=""
if [[ -n "${VER_DIR:-}" && -d "$VER_DIR" ]]; then
dir="$VER_DIR"
elif [[ -L "$INSTALL_ROOT/current" ]]; then
dir="$(readlink -f "$INSTALL_ROOT/current" 2>/dev/null || true)"
fi
if [[ -z "$dir" ]]; then
dir="$(ls -d "$INSTALL_ROOT"/versions/* 2>/dev/null | sort -V | tail -n1 || true)"
fi
echo "$dir"
}
main() {
log "starting with interval=${INTERVAL}s"
local dir
dir="$(resolve_ver_dir)"
if [[ -z "$dir" || ! -d "$dir" ]]; then
log "no valid install dir found under $INSTALL_ROOT; exiting"
exit 0
fi
local chk="$dir/check_health.sh"
local rst="$dir/restart_unhealthy.sh"
if [[ ! -x "$chk" && ! -x "$rst" ]]; then
log "neither check_health.sh nor restart_unhealthy.sh is executable under $dir; exiting"
exit 0
fi
log "watching install dir: $dir"
while :; do
if [[ -x "$chk" ]]; then
log "running check_health.sh"
"$chk" >> "$dir/.health_check.watch.log" 2>&1 || log "check_health.sh reported issues (see .health_check.watch.log)"
fi
if [[ -x "$rst" ]]; then
log "running restart_unhealthy.sh"
"$rst" >> "$dir/.restart.watch.log" 2>&1 || log "restart_unhealthy.sh reported issues (see .restart.watch.log)"
fi
sleep "$INTERVAL"
done
}
main "$@"

View File

@@ -0,0 +1,135 @@
#!/usr/bin/env bash
set -euo pipefail
echo "[BOOT] GPU node bundle starting"
INSTALL_ROOT="/opt/argus-metric"
BUNDLE_DIR="/bundle"
STATE_DIR_BASE="/private/argus/agent"
mkdir -p "$INSTALL_ROOT" "$STATE_DIR_BASE" /logs/train /logs/infer /buffers || true
# Ensure world-writable logs dir with sticky bit (align with deployment_new policy)
if [[ "${ARGUS_LOGS_WORLD_WRITABLE:-1}" == "1" ]]; then
chmod 1777 /logs/train /logs/infer || true
else
chmod 755 /logs/train /logs/infer || true
fi
chmod 770 /buffers || true
installed_ok=0
# 1) already installed?
if [[ -L "$INSTALL_ROOT/current" && -d "$INSTALL_ROOT/current" ]]; then
echo "[BOOT] client already installed at $INSTALL_ROOT/current"
else
# 2) try local bundle first (argus-metric_*.tar.gz)
tarball=$(ls -1 "$BUNDLE_DIR"/argus-metric_*.tar.gz 2>/dev/null | head -1 || true)
if [[ -n "${tarball:-}" ]]; then
echo "[BOOT] installing from local bundle: $(basename "$tarball")"
tmp=$(mktemp -d)
tar -xzf "$tarball" -C "$tmp"
# locate root containing version.json
root="$tmp"
if [[ ! -f "$root/version.json" ]]; then
sub=$(find "$tmp" -mindepth 1 -maxdepth 1 -type d | head -n1 || true)
[[ -n "$sub" && -f "$sub/version.json" ]] && root="$sub"
fi
if [[ ! -f "$root/version.json" ]]; then
echo "[BOOT][WARN] version.json not found in bundle; fallback to FTP"
else
ver=$(sed -n 's/.*"version"\s*:\s*"\([^"]\+\)".*/\1/p' "$root/version.json" | head -n1)
if [[ -z "$ver" ]]; then
echo "[BOOT][WARN] failed to parse version from version.json; fallback to FTP"
else
target_root="$INSTALL_ROOT"
version_dir="$target_root/versions/$ver"
mkdir -p "$version_dir"
shopt -s dotglob
mv "$root"/* "$version_dir/" 2>/dev/null || true
shopt -u dotglob
if [[ -f "$version_dir/install.sh" ]]; then
chmod +x "$version_dir/install.sh" 2>/dev/null || true
(
export AUTO_START_DCGM="${AUTO_START_DCGM:-1}"
export DCGM_EXPORTER_DISABLE_PROFILING="${DCGM_EXPORTER_DISABLE_PROFILING:-1}"
export DCGM_EXPORTER_LISTEN="${DCGM_EXPORTER_LISTEN:-:9400}"
cd "$version_dir" && ./install.sh "$version_dir"
)
echo "$ver" > "$target_root/LATEST_VERSION" 2>/dev/null || true
ln -sfn "$version_dir" "$target_root/current" 2>/dev/null || true
if [[ -L "$target_root/current" && -d "$target_root/current" ]]; then
installed_ok=1
echo "[BOOT] local bundle install OK: version=$ver"
else
echo "[BOOT][WARN] current symlink not present after install; will rely on healthcheck to confirm"
fi
else
echo "[BOOT][WARN] install.sh missing under $version_dir; fallback to FTP"
fi
fi
fi
fi
# 3) fallback: use FTP setup if not installed
if [[ ! -L "$INSTALL_ROOT/current" && "$installed_ok" -eq 0 ]]; then
echo "[BOOT] fallback to FTP setup"
if [[ -z "${FTPIP:-}" || -z "${FTP_USER:-}" || -z "${FTP_PASSWORD:-}" ]]; then
echo "[BOOT][ERROR] FTP variables not set (FTPIP/FTP_USER/FTP_PASSWORD)" >&2
exit 1
fi
curl -u "$FTP_USER:$FTP_PASSWORD" -fsSL "ftp://$FTPIP:21/setup.sh" -o /tmp/setup.sh
chmod +x /tmp/setup.sh
/tmp/setup.sh --server "$FTPIP" --user "$FTP_USER" --password "$FTP_PASSWORD" --port 21
fi
fi
# 4) ensure argus-agent is running (best-effort)
if ! pgrep -x argus-agent >/dev/null 2>&1; then
echo "[BOOT] starting argus-agent (not detected)"
setsid /usr/local/bin/argus-agent >/var/log/argus-agent.log 2>&1 < /dev/null &
fi
# 5) post-install selfcheck (run once) and state
# prefer current version dir; fallback to first version under /opt/argus-metric/versions
ver_dir=""
if [[ -L "$INSTALL_ROOT/current" ]]; then
ver_dir="$(readlink -f "$INSTALL_ROOT/current" 2>/dev/null || true)"
fi
if [[ -z "$ver_dir" ]]; then
# pick the latest by name (semver-like); best-effort
ver_dir="$(ls -d "$INSTALL_ROOT"/versions/* 2>/dev/null | sort -V | tail -n1 || true)"
fi
if [[ -n "$ver_dir" && -x "$ver_dir/check_health.sh" ]]; then
echo "[BOOT] running initial health check: $ver_dir/check_health.sh"
if "$ver_dir/check_health.sh" >> "$ver_dir/.health_check.init.log" 2>&1; then
echo "[BOOT] initial health check completed (see $ver_dir/.health_check.init.log)"
else
echo "[BOOT][WARN] initial health check reported issues (see $ver_dir/.health_check.init.log)"
fi
else
echo "[BOOT][WARN] initial health check skipped (script missing: $ver_dir/check_health.sh)"
fi
host="$(hostname)"
state_dir="$STATE_DIR_BASE/${host}"
mkdir -p "$state_dir" 2>/dev/null || true
for i in {1..60}; do
if [[ -s "$state_dir/node.json" ]]; then
echo "[BOOT] node state present: $state_dir/node.json"
break
fi
sleep 2
done
# 6) spawn health watcher (best-effort, non-blocking)
if command -v /usr/local/bin/health-watcher.sh >/dev/null 2>&1; then
echo "[BOOT] starting health watcher for $ver_dir"
setsid /usr/local/bin/health-watcher.sh "${ver_dir:-}" >/var/log/health-watcher.log 2>&1 < /dev/null || true &
else
echo "[BOOT][WARN] health-watcher.sh not found; skip health watcher"
fi
echo "[BOOT] ready; entering sleep"
exec sleep infinity

View File

@@ -0,0 +1,76 @@
FROM docker.elastic.co/elasticsearch/elasticsearch:8.17.10
# 切换到 root 用户进行系统级安装
USER root
ARG ARGUS_BUILD_UID=2133
ARG ARGUS_BUILD_GID=2015
ENV ARGUS_BUILD_UID=${ARGUS_BUILD_UID} \
ARGUS_BUILD_GID=${ARGUS_BUILD_GID}
# 调整 elasticsearch 用户与用户组 ID 以匹配宿主机配置
RUN set -eux; \
current_gid="$(getent group elasticsearch | awk -F: '{print $3}')"; \
if [ -z "$current_gid" ]; then \
groupadd -g "${ARGUS_BUILD_GID}" elasticsearch; \
elif [ "$current_gid" != "${ARGUS_BUILD_GID}" ]; then \
groupmod -g "${ARGUS_BUILD_GID}" elasticsearch; \
fi; \
if id elasticsearch >/dev/null 2>&1; then \
current_uid="$(id -u elasticsearch)"; \
if [ "$current_uid" != "${ARGUS_BUILD_UID}" ]; then \
usermod -u "${ARGUS_BUILD_UID}" elasticsearch; \
fi; \
else \
useradd -m -u "${ARGUS_BUILD_UID}" -g "${ARGUS_BUILD_GID}" elasticsearch; \
fi; \
chown -R "${ARGUS_BUILD_UID}:${ARGUS_BUILD_GID}" /usr/share/elasticsearch
# 设置构建参数
ARG USE_INTRANET=false
# 配置内网 apt 源 (如果指定了内网选项)
RUN if [ "$USE_INTRANET" = "true" ]; then \
echo "Configuring intranet apt sources..." && \
cp /etc/apt/sources.list /etc/apt/sources.list.bak && \
echo "deb [trusted=yes] http://10.68.64.1/ubuntu2204/ jammy main" > /etc/apt/sources.list && \
echo 'Acquire::https::Verify-Peer "false";' > /etc/apt/apt.conf.d/99disable-ssl-check && \
echo 'Acquire::https::Verify-Host "false";' >> /etc/apt/apt.conf.d/99disable-ssl-check; \
fi
# 安装 supervisor, net-tools, vim
RUN apt-get update && \
apt-get install -y supervisor net-tools inetutils-ping vim && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# 配置部署时使用的apt源
RUN if [ "$USE_INTRANET" = "true" ]; then \
echo "deb [trusted=yes] https://10.92.132.52/mirrors/ubuntu2204/ jammy main" > /etc/apt/sources.list; \
fi
# 创建 supervisor 日志目录
RUN mkdir -p /var/log/supervisor
# 复制 supervisor 配置文件
COPY src/log/elasticsearch/build/supervisord.conf /etc/supervisor/conf.d/supervisord.conf
# 复制启动脚本
COPY src/log/elasticsearch/build/start-es-supervised.sh /usr/local/bin/start-es-supervised.sh
RUN chmod +x /usr/local/bin/start-es-supervised.sh
# 复制DNS监控脚本
COPY src/log/elasticsearch/build/dns-monitor.sh /usr/local/bin/dns-monitor.sh
RUN chmod +x /usr/local/bin/dns-monitor.sh
# 保持 root 用户,由 supervisor 管理用户切换
USER root
# 暴露端口
EXPOSE 9200 9300
# 使用 supervisor 作为入口点
CMD ["/usr/bin/supervisord", "-c", "/etc/supervisor/conf.d/supervisord.conf"]

View File

@@ -22,8 +22,7 @@
[PARSER]
Name timestamp_parser
Format regex
Regex ^(?<timestamp>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\s+(?<level>\w+)\s+(?<message>.*)$
Regex ^(?<timestamp>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:Z|[+-]\d{2}:?\d{2}))\s+(?<level>\w+)\s+(?<message>.*)$
Time_Key timestamp
Time_Format %Y-%m-%d %H:%M:%S
Time_Offset +0800
Time_Format %Y-%m-%dT%H:%M:%S%z
Time_Keep On

View File

@@ -38,14 +38,16 @@ ensure_lib() {
ldconfig 2>/dev/null || true
}
DEB_ARCH="${DEB_ARCH:-$(dpkg --print-architecture 2>/dev/null || echo amd64)}"
# Offline-first: satisfy runtime deps from local debs, fallback to apt only if necessary
ensure_lib "libpq.so.5" "libpq5_*_amd64.deb"
ensure_lib "libyaml-0.so.2" "libyaml-0-2_*_amd64.deb"
ensure_lib "libsasl2.so.2" "libsasl2-2_*_amd64.deb"
ensure_lib "libldap-2.5.so.0" "libldap-2.5-0_*_amd64.deb"
ensure_lib "libpq.so.5" "libpq5_*_${DEB_ARCH}.deb"
ensure_lib "libyaml-0.so.2" "libyaml-0-2_*_${DEB_ARCH}.deb"
ensure_lib "libsasl2.so.2" "libsasl2-2_*_${DEB_ARCH}.deb"
ensure_lib "libldap-2.5.so.0" "libldap-2.5-0_*_${DEB_ARCH}.deb"
# Install fluent-bit main package from local bundle
FLB_DEB="$(ls /tmp/flb/packages/fluent-bit_*_amd64.deb 2>/dev/null | head -n1 || true)"
FLB_DEB="$(ls /tmp/flb/packages/fluent-bit_*_${DEB_ARCH}.deb 2>/dev/null | head -n1 || true)"
if [[ -z "$FLB_DEB" ]]; then
echo "[ERROR] fluent-bit deb not found under /private/packages" >&2
exit 1
@@ -77,7 +79,20 @@ cp -r /tmp/flb/etc/* /etc/fluent-bit/
# Create logs/buffers dirs
mkdir -p /logs/train /logs/infer /buffers
chmod 755 /logs/train /logs/infer /buffers
# 控制日志目录权限:默认对宿主 bind mount 目录采用 1777(可由环境变量关闭)
: "${ARGUS_LOGS_WORLD_WRITABLE:=1}"
if [[ "${ARGUS_LOGS_WORLD_WRITABLE}" == "1" ]]; then
chmod 1777 /logs/train /logs/infer || true
else
chmod 755 /logs/train /logs/infer || true
fi
# 缓冲目录仅供进程使用,不对外开放写入
chmod 770 /buffers || true
# 目录属主设置为 fluent-bit(不影响 1777 粘滞位)
chown -R fluent-bit:fluent-bit /logs /buffers 2>/dev/null || true
# Wait for Elasticsearch via bash /dev/tcp to avoid curl dependency
echo "[INFO] Waiting for Elasticsearch to be ready (tcp ${ES_HOST}:${ES_PORT})..."

View File

@@ -0,0 +1,80 @@
FROM docker.elastic.co/kibana/kibana:8.17.10
# 切换到 root 用户进行系统级安装
USER root
ARG ARGUS_BUILD_UID=2133
ARG ARGUS_BUILD_GID=2015
ENV ARGUS_BUILD_UID=${ARGUS_BUILD_UID} \
ARGUS_BUILD_GID=${ARGUS_BUILD_GID}
# 调整 kibana 用户与用户组 ID 以匹配宿主机配置
RUN set -eux; \
current_gid="$(getent group kibana | awk -F: '{print $3}')"; \
if [ -z "$current_gid" ]; then \
groupadd -g "${ARGUS_BUILD_GID}" kibana; \
elif [ "$current_gid" != "${ARGUS_BUILD_GID}" ]; then \
groupmod -g "${ARGUS_BUILD_GID}" kibana; \
fi; \
if id kibana >/dev/null 2>&1; then \
current_uid="$(id -u kibana)"; \
if [ "$current_uid" != "${ARGUS_BUILD_UID}" ]; then \
usermod -u "${ARGUS_BUILD_UID}" kibana; \
fi; \
else \
useradd -m -u "${ARGUS_BUILD_UID}" -g "${ARGUS_BUILD_GID}" kibana; \
fi; \
chown -R "${ARGUS_BUILD_UID}:${ARGUS_BUILD_GID}" /usr/share/kibana
# 设置构建参数
ARG USE_INTRANET=false
# 配置内网 apt 源 (如果指定了内网选项)
RUN if [ "$USE_INTRANET" = "true" ]; then \
echo "Configuring intranet apt sources..." && \
cp /etc/apt/sources.list /etc/apt/sources.list.bak && \
echo "deb [trusted=yes] http://10.68.64.1/ubuntu2204/ jammy main" > /etc/apt/sources.list && \
echo 'Acquire::https::Verify-Peer "false";' > /etc/apt/apt.conf.d/99disable-ssl-check && \
echo 'Acquire::https::Verify-Host "false";' >> /etc/apt/apt.conf.d/99disable-ssl-check; \
fi
# 安装 supervisor, net-tools, vim
RUN apt-get update && \
apt-get install -y supervisor net-tools inetutils-ping vim && \
apt-get clean && \
rm -rf /var/lib/apt/lists/*
# 配置部署时使用的apt源
RUN if [ "$USE_INTRANET" = "true" ]; then \
echo "deb [trusted=yes] https://10.92.132.52/mirrors/ubuntu2204/ jammy main" > /etc/apt/sources.list; \
fi
# 创建 supervisor 日志目录
RUN mkdir -p /var/log/supervisor
# 复制 supervisor 配置文件
COPY src/log/kibana/build/supervisord.conf /etc/supervisor/conf.d/supervisord.conf
# 复制启动脚本
COPY src/log/kibana/build/start-kibana-supervised.sh /usr/local/bin/start-kibana-supervised.sh
COPY src/log/kibana/build/kibana-post-start.sh /usr/local/bin/kibana-post-start.sh
RUN chmod +x /usr/local/bin/start-kibana-supervised.sh /usr/local/bin/kibana-post-start.sh
# 复制DNS监控脚本
COPY src/log/kibana/build/dns-monitor.sh /usr/local/bin/dns-monitor.sh
RUN chmod +x /usr/local/bin/dns-monitor.sh
# kibana需要用到 /root/.config/puppeteer 路径
RUN chmod 777 /root
# 保持 root 用户,由 supervisor 管理用户切换
USER root
# 暴露端口
EXPOSE 5601
# 使用 supervisor 作为入口点
CMD ["/usr/bin/supervisord", "-c", "/etc/supervisor/conf.d/supervisord.conf"]

View File

@@ -36,9 +36,28 @@ echo "[INFO] Fluent-bit files should be in fluent-bit/ directory"
# 准备 Fluent Bit 离线依赖(从 metric all-in-one-full 复制 deb 到 ../fluent-bit/build/packages)
FLB_BUILD_PACKAGES_DIR="$root/../fluent-bit/build/packages"
mkdir -p "$FLB_BUILD_PACKAGES_DIR"
detect_deb_arch() {
local deb_arch="${1:-}"
if [[ -n "$deb_arch" ]]; then
echo "$deb_arch"; return
fi
if command -v dpkg >/dev/null 2>&1; then
dpkg --print-architecture # amd64 / arm64
else
case "$(uname -m)" in
x86_64) echo amd64 ;;
aarch64) echo arm64 ;;
*) echo amd64 ;;
esac
fi
}
DEB_ARCH="$(detect_deb_arch)"
FLB_BIN_DIR="$project_root/src/metric/client-plugins/all-in-one-full/plugins/fluent-bit/bin/${DEB_ARCH}"
for deb in \
"$project_root/src/metric/client-plugins/all-in-one-full/plugins/fluent-bit/bin/libyaml-0-2_"*_amd64.deb \
"$project_root/src/metric/client-plugins/all-in-one-full/plugins/fluent-bit/bin/libpq5_"*_amd64.deb ; do
"$FLB_BIN_DIR/libyaml-0-2_"*_"${DEB_ARCH}.deb" \
"$FLB_BIN_DIR/libpq5_"*_"${DEB_ARCH}.deb" ; do
if ls $deb >/dev/null 2>&1; then
for f in $deb; do
base="$(basename "$f")"
@@ -56,12 +75,12 @@ if [[ -f "$CURLOPT_TAR" ]]; then
tmpdir=$(mktemp -d)
if tar -xzf "$CURLOPT_TAR" -C "$tmpdir" 2>/dev/null; then
for p in \
libsasl2-2_*_amd64.deb \
libsasl2-modules-db_*_amd64.deb \
libldap-2.5-0_*_amd64.deb \
libidn2-0_*_amd64.deb \
libbrotli1_*_amd64.deb \
libssl3_*_amd64.deb ; do
"libsasl2-2_*_${DEB_ARCH}.deb" \
"libsasl2-modules-db_*_${DEB_ARCH}.deb" \
"libldap-2.5-0_*_${DEB_ARCH}.deb" \
"libidn2-0_*_${DEB_ARCH}.deb" \
"libbrotli1_*_${DEB_ARCH}.deb" \
"libssl3_*_${DEB_ARCH}.deb" ; do
src=$(ls "$tmpdir"/curl/$p 2>/dev/null | head -n1 || true)
if [[ -n "$src" ]]; then
base="$(basename "$src")"

View File

@@ -28,11 +28,11 @@ fi
docker exec "$container_name" mkdir -p /logs/train /logs/infer
# 写入训练日志 (host01)
docker exec "$container_name" sh -c "printf '%s INFO [host01] training step=1 loss=1.23 model=bert\n' \"\$(date '+%F %T')\" >> /logs/train/train-demo.log"
docker exec "$container_name" sh -c "printf '%s INFO [host01] training step=2 loss=1.15 model=bert\n' \"\$(date '+%F %T')\" >> /logs/train/train-demo.log"
docker exec "$container_name" sh -c "printf '%s INFO [host01] training step=1 loss=1.23 model=bert\n' \"\$(date -u +%Y-%m-%dT%H:%M:%SZ)\" >> /logs/train/train-demo.log"
docker exec "$container_name" sh -c "printf '%s INFO [host01] training step=2 loss=1.15 model=bert\n' \"\$(date -u +%Y-%m-%dT%H:%M:%SZ)\" >> /logs/train/train-demo.log"
# 写入推理日志 (host01)
docker exec "$container_name" sh -c "printf '%s ERROR [host01] inference failed on batch=1\n' \"\$(date '+%F %T')\" >> /logs/infer/infer-demo.log"
docker exec "$container_name" sh -c "printf '%s ERROR [host01] inference failed on batch=1\n' \"\$(date -u +%Y-%m-%dT%H:%M:%SZ)\" >> /logs/infer/infer-demo.log"
docker exec "$container_name" sh -c "cat <<'STACK' >> /logs/infer/infer-demo.log
Traceback (most recent call last):
File \"inference.py\", line 15, in <module>

View File

@@ -28,13 +28,13 @@ fi
docker exec "$container_name" mkdir -p /logs/train /logs/infer
# 写入训练日志 (host02)
docker exec "$container_name" sh -c "printf '%s INFO [host02] training step=1 loss=1.45 model=gpt\n' \"\$(date '+%F %T')\" >> /logs/train/train-demo.log"
docker exec "$container_name" sh -c "printf '%s INFO [host02] training step=2 loss=1.38 model=gpt\n' \"\$(date '+%F %T')\" >> /logs/train/train-demo.log"
docker exec "$container_name" sh -c "printf '%s INFO [host02] training step=3 loss=1.32 model=gpt\n' \"\$(date '+%F %T')\" >> /logs/train/train-demo.log"
docker exec "$container_name" sh -c "printf '%s INFO [host02] training step=1 loss=1.45 model=gpt\n' \"\$(date -u +%Y-%m-%dT%H:%M:%SZ)\" >> /logs/train/train-demo.log"
docker exec "$container_name" sh -c "printf '%s INFO [host02] training step=2 loss=1.38 model=gpt\n' \"\$(date -u +%Y-%m-%dT%H:%M:%SZ)\" >> /logs/train/train-demo.log"
docker exec "$container_name" sh -c "printf '%s INFO [host02] training step=3 loss=1.32 model=gpt\n' \"\$(date -u +%Y-%m-%dT%H:%M:%SZ)\" >> /logs/train/train-demo.log"
# 写入推理日志 (host02)
docker exec "$container_name" sh -c "printf '%s WARN [host02] inference slow on batch=5 latency=2.3s\n' \"\$(date '+%F %T')\" >> /logs/infer/infer-demo.log"
docker exec "$container_name" sh -c "printf '%s INFO [host02] inference completed batch=6 latency=0.8s\n' \"\$(date '+%F %T')\" >> /logs/infer/infer-demo.log"
docker exec "$container_name" sh -c "printf '%s WARN [host02] inference slow on batch=5 latency=2.3s\n' \"\$(date -u +%Y-%m-%dT%H:%M:%SZ)\" >> /logs/infer/infer-demo.log"
docker exec "$container_name" sh -c "printf '%s INFO [host02] inference completed batch=6 latency=0.8s\n' \"\$(date -u +%Y-%m-%dT%H:%M:%SZ)\" >> /logs/infer/infer-demo.log"
echo "[OK] 已通过docker exec写入测试日志到 host02 容器内:"
echo " - /logs/train/train-demo.log"

View File

@@ -13,6 +13,8 @@ class AppConfig:
scheduler_interval_seconds: int
node_id_prefix: str
auth_mode: str
target_prefer_net_cidrs: str
target_reachability_check: bool
def _get_int_env(name: str, default: int) -> int:
@@ -27,6 +29,12 @@ def _get_int_env(name: str, default: int) -> int:
def load_config() -> AppConfig:
"""读取环境变量生成配置对象,方便统一管理运行参数。"""
def _bool_env(name: str, default: bool) -> bool:
raw = os.environ.get(name)
if raw is None or raw.strip() == "":
return default
return raw.strip().lower() in ("1", "true", "yes", "on")
return AppConfig(
db_path=os.environ.get("DB_PATH", "/private/argus/master/db.sqlite3"),
metric_nodes_json_path=os.environ.get(
@@ -37,4 +45,6 @@ def load_config() -> AppConfig:
scheduler_interval_seconds=_get_int_env("SCHEDULER_INTERVAL_SECONDS", 30),
node_id_prefix=os.environ.get("NODE_ID_PREFIX", "A"),
auth_mode=os.environ.get("AUTH_MODE", "disabled"),
target_prefer_net_cidrs=os.environ.get("TARGET_PREFER_NET_CIDRS", "10.0.0.0/8,172.31.0.0/16"),
target_reachability_check=_bool_env("TARGET_REACHABILITY_CHECK", False),
)

View File

@@ -1,8 +1,10 @@
from __future__ import annotations
import ipaddress
import logging
import socket
import threading
from typing import Optional
from typing import Optional, Iterable, Dict, Any, List
from .config import AppConfig
from .storage import Storage
@@ -34,10 +36,117 @@ class StatusScheduler:
self._pending_nodes_json.set()
def generate_nodes_json(self) -> None:
"""根据在线节点生成 Prometheus 抓取目标,优先 overlay IP。
候选顺序:meta.overlay_ip > hostname A 记录(命中偏好网段)> meta.ip。
可选 reachability 检查:TARGET_REACHABILITY_CHECK=true 时,对 9100/9400 做一次 1s TCP 连接测试,
选择首个可达的候选;全部失败则按顺序取第一个并记录日志。
"""
with self._nodes_json_lock:
online_nodes = self._storage.get_online_nodes()
atomic_write_json(self._config.metric_nodes_json_path, online_nodes)
self._logger.info("nodes.json updated", extra={"count": len(online_nodes)})
rows = self._storage.get_online_nodes_meta()
prefer_cidrs = self._parse_cidrs(self._config.target_prefer_net_cidrs)
reachability = self._config.target_reachability_check
result: List[Dict[str, Any]] = []
for row in rows:
meta = row.get("meta", {})
hostname = meta.get("hostname") or row.get("name")
labels = row.get("labels") or []
overlay_ip = meta.get("overlay_ip")
legacy_ip = meta.get("ip")
host_candidates = self._resolve_host_ips(hostname)
host_pref = self._pick_by_cidrs(host_candidates, prefer_cidrs)
candidates: List[str] = []
for ip in [overlay_ip, host_pref, legacy_ip]:
if ip and ip not in candidates:
candidates.append(ip)
chosen = None
if reachability:
ports = [9100]
try:
if int(meta.get("gpu_number", 0)) > 0:
ports.append(9400)
except Exception:
pass
for ip in candidates:
if any(self._reachable(ip, p, 1.0) for p in ports):
chosen = ip
break
if not chosen:
chosen = candidates[0] if candidates else legacy_ip
if not chosen:
# ultimate fallback: 127.0.0.1 (should not happen)
chosen = "127.0.0.1"
self._logger.warning("No candidate IPs for node; falling back", extra={"node": row.get("node_id")})
if chosen and ipaddress.ip_address(chosen) in ipaddress.ip_network("172.22.0.0/16"):
self._logger.warning(
"Prometheus target uses docker_gwbridge address; prefer overlay",
extra={"node": row.get("node_id"), "ip": chosen},
)
result.append(
{
"node_id": row.get("node_id"),
"user_id": meta.get("user"),
"ip": chosen,
"hostname": hostname,
"labels": labels if isinstance(labels, list) else [],
}
)
atomic_write_json(self._config.metric_nodes_json_path, result)
self._logger.info("nodes.json updated", extra={"count": len(result)})
# ---------------------------- helpers ----------------------------
@staticmethod
def _parse_cidrs(raw: str) -> List[ipaddress.IPv4Network]:
nets: List[ipaddress.IPv4Network] = []
for item in (x.strip() for x in (raw or "").split(",")):
if not item:
continue
try:
net = ipaddress.ip_network(item, strict=False)
if isinstance(net, ipaddress.IPv4Network):
nets.append(net)
except ValueError:
continue
return nets
@staticmethod
def _resolve_host_ips(hostname: str) -> List[str]:
ips: List[str] = []
try:
infos = socket.getaddrinfo(hostname, None, family=socket.AF_INET)
for info in infos:
ip = info[4][0]
if ip not in ips:
ips.append(ip)
except OSError:
pass
return ips
@staticmethod
def _pick_by_cidrs(candidates: Iterable[str], prefer: List[ipaddress.IPv4Network]) -> str | None:
for net in prefer:
for ip in candidates:
try:
if ipaddress.ip_address(ip) in net:
return ip
except ValueError:
continue
return None
@staticmethod
def _reachable(ip: str, port: int, timeout: float) -> bool:
try:
with socket.create_connection((ip, port), timeout=timeout):
return True
except OSError:
return False
# ------------------------------------------------------------------
# internal loop

View File

@@ -324,9 +324,35 @@ class Storage:
{
"node_id": row["id"],
"user_id": meta.get("user"),
"ip": meta.get("ip"),
"ip": meta.get("ip"), # kept for backward-compat; preferred IP selection handled in scheduler
"hostname": meta.get("hostname", row["name"]),
"labels": labels if isinstance(labels, list) else [],
}
)
return result
def get_online_nodes_meta(self) -> List[Dict[str, Any]]:
"""返回在线节点的原始 meta 与名称、标签,交由上层选择目标 IP。
每项包含:{ node_id, name, meta, labels }
"""
with self._lock:
cur = self._conn.execute(
"SELECT id, name, meta_json, labels_json FROM nodes WHERE status = ? ORDER BY id ASC",
("online",),
)
rows = cur.fetchall()
result: List[Dict[str, Any]] = []
for row in rows:
meta = json.loads(row["meta_json"]) if row["meta_json"] else {}
labels = json.loads(row["labels_json"]) if row["labels_json"] else []
result.append(
{
"node_id": row["id"],
"name": row["name"],
"meta": meta if isinstance(meta, dict) else {},
"labels": labels if isinstance(labels, list) else [],
}
)
return result

View File

@@ -1 +1 @@
1.35.0
1.44.0

View File

@@ -0,0 +1 @@
bin/

View File

@@ -14,6 +14,16 @@ log_info() {
echo -e "${BLUE}[INFO]${NC} $1"
}
# 运行时开关(可通过环境变量覆盖)
# 1) 是否自动启动 nv-hostengine(容器内通常没有 systemd)
AUTO_START_DCGM="${AUTO_START_DCGM:-1}"
# 2) 是否默认禁用 Profiling 指标(避免在部分环境触发 DCGM Profiling 崩溃)
DCGM_EXPORTER_DISABLE_PROFILING="${DCGM_EXPORTER_DISABLE_PROFILING:-1}"
# 3) 自定义 collectors 文件;若为空且禁用 Profiling,则自动生成 no-prof 清单
DCGM_EXPORTER_COLLECTORS="${DCGM_EXPORTER_COLLECTORS:-}"
# 4) 监听地址
DCGM_EXPORTER_LISTEN="${DCGM_EXPORTER_LISTEN:-:9400}"
log_success() {
echo -e "${GREEN}[SUCCESS]${NC} $1"
}
@@ -160,10 +170,21 @@ check_dcgm_service() {
elif pgrep -f nv-hostengine > /dev/null; then
log_success "nv-hostengine 进程已在运行"
else
log_warning "DCGM 服务未运行,需要手动启动"
log_info "启动 DCGM 服务的方法:"
log_info " 1. 使用 systemd: sudo systemctl start dcgm"
log_info " 2. 手动启动: nohup nv-hostengine > /var/log/nv-hostengine.log 2>&1 &"
log_warning "DCGM 服务未运行"
if [[ "${AUTO_START_DCGM}" == "1" ]]; then
log_info "尝试自动启动 nv-hostengine(容器内无 systemd 场景)..."
nohup nv-hostengine > /var/log/nv-hostengine.log 2>&1 &
sleep 2
if pgrep -f nv-hostengine >/dev/null; then
log_success "nv-hostengine 已启动"
else
log_error "nv-hostengine 启动失败,请手动检查 /var/log/nv-hostengine.log"
fi
else
log_info "启动 DCGM 服务的方法:"
log_info " 1. 使用 systemd: sudo systemctl start dcgm"
log_info " 2. 手动启动: nohup nv-hostengine > /var/log/nv-hostengine.log 2>&1 &"
fi
fi
# 测试 DCGM 连接
@@ -172,7 +193,7 @@ check_dcgm_service() {
if dcgmi discovery -l > /dev/null 2>&1; then
log_success "DCGM 连接测试成功"
else
log_warning "DCGM 连接测试失败,请检查服务状态"
log_warning "DCGM 连接测试失败,请检查服务状态(驱动/权限/设备可见性)"
fi
fi
}
@@ -269,6 +290,7 @@ start_dcgm_exporter() {
local binary_path="/usr/local/bin/dcgm-exporter"
local log_file="/var/log/dcgm-exporter.log"
local pid_file="/var/run/dcgm-exporter.pid"
local collectors_arg=""
# 检查服务是否已经在运行
if [[ -f "$pid_file" ]]; then
@@ -282,15 +304,48 @@ start_dcgm_exporter() {
fi
fi
# 计算 collectors 参数
if [[ -n "${DCGM_EXPORTER_COLLECTORS}" ]]; then
if [[ -f "${DCGM_EXPORTER_COLLECTORS}" ]]; then
collectors_arg=(--collectors "${DCGM_EXPORTER_COLLECTORS}")
log_info "使用自定义 collectors: ${DCGM_EXPORTER_COLLECTORS}"
else
log_warning "指定的 DCGM_EXPORTER_COLLECTORS 文件不存在: ${DCGM_EXPORTER_COLLECTORS}(将忽略)"
fi
elif [[ "${DCGM_EXPORTER_DISABLE_PROFILING}" == "1" ]]; then
local cfg_dir="/etc/dcgm-exporter"
local default_cfg="${cfg_dir}/default-counters.csv"
local no_prof_cfg="${cfg_dir}/no-prof.csv"
mkdir -p "${cfg_dir}"
if [[ -f "${default_cfg}" ]]; then
grep -v 'DCGM_FI_PROF_' "${default_cfg}" > "${no_prof_cfg}" || true
collectors_arg=(--collectors "${no_prof_cfg}")
log_info "已生成无 Profiling 的 collectors: ${no_prof_cfg}"
else
log_warning "未找到默认 collectors 文件: ${default_cfg}"
fi
fi
# 检查端口是否被占用
if netstat -tuln 2>/dev/null | grep -q ":9400 "; then
if netstat -tuln 2>/dev/null | grep -q ":${DCGM_EXPORTER_LISTEN#:} "; then
log_warning "端口 9400 已被占用,请检查是否有其他服务在运行"
return 1
fi
# 启动前再校验一次 DCGM 主机引擎
if ! (systemctl is-active --quiet dcgm 2>/dev/null || pgrep -f nv-hostengine >/dev/null); then
log_warning "nv-hostengine 未运行,尝试自动启动"
nohup nv-hostengine > /var/log/nv-hostengine.log 2>&1 &
sleep 2
fi
# 启动服务
log_info "正在启动 DCGM Exporter..."
nohup "$binary_path" --address=:9400 > "$log_file" 2>&1 &
if [[ ${#collectors_arg[@]} -gt 0 ]]; then
nohup "$binary_path" --address="${DCGM_EXPORTER_LISTEN}" "${collectors_arg[@]}" > "$log_file" 2>&1 &
else
nohup "$binary_path" --address="${DCGM_EXPORTER_LISTEN}" > "$log_file" 2>&1 &
fi
local pid=$!
# 保存 PID
@@ -310,6 +365,20 @@ start_dcgm_exporter() {
else
log_error "DCGM Exporter 服务启动失败"
rm -f "$pid_file"
# 失败回退:若未禁用 Profiling,也未指定 collectors,则尝试自动回退到 no-prof 再起一次
if [[ -z "${DCGM_EXPORTER_COLLECTORS}" && "${DCGM_EXPORTER_DISABLE_PROFILING}" != "1" ]]; then
log_warning "尝试以无 Profiling 清单回退启动"
local cfg_dir="/etc/dcgm-exporter"; local default_cfg="${cfg_dir}/default-counters.csv"; local no_prof_cfg="${cfg_dir}/no-prof.csv"
if [[ -f "${default_cfg}" ]]; then
grep -v 'DCGM_FI_PROF_' "${default_cfg}" > "${no_prof_cfg}" || true
nohup "$binary_path" --address="${DCGM_EXPORTER_LISTEN}" --collectors "${no_prof_cfg}" > "$log_file" 2>&1 &
sleep 2
if pgrep -f dcgm-exporter >/dev/null; then
log_success "DCGM Exporter 已用无 Profiling 清单启动"
return 0
fi
fi
fi
return 1
fi
}

View File

@@ -48,6 +48,15 @@ if [[ ${#missing_files[@]} -gt 0 ]]; then
exit 1
fi
# 防御:阻止将 Git LFS 指针文件打包
for f in bin/dcgm-exporter bin/datacenter-gpu-manager_3.3.9_amd64.deb; do
if head -n1 "$f" 2>/dev/null | grep -q '^version https://git-lfs.github.com/spec/v1$'; then
echo "[ERROR] $f 是 Git LFS 指针文件,未还原为真实制品"
echo " 请在仓库根目录执行: git lfs fetch --all && git lfs checkout"
exit 1
fi
done
log_success "所有必要文件检查完成"
# 创建临时目录

View File

@@ -1,9 +1,11 @@
# 重要:使用 Logstash_Format + Logstash_Prefix,生成 train-*/infer-* 索引
# 说明:Fluent Bit 配置仅支持 ${VAR} 占位符,不支持 Bash 的 ${VAR:-default}
# 固定域名要求:使用 es.log.argus.com 与端口 9200
[OUTPUT]
Name es
Match app.train
Host ${ES_HOST:-localhost}
Port ${ES_PORT:-9200}
Host es.log.argus.com
Port 9200
Logstash_Format On
Logstash_Prefix train
Replace_Dots On
@@ -14,8 +16,8 @@
[OUTPUT]
Name es
Match app.infer
Host ${ES_HOST:-localhost}
Port ${ES_PORT:-9200}
Host es.log.argus.com
Port 9200
Logstash_Format On
Logstash_Prefix infer
Replace_Dots On

View File

@@ -22,6 +22,6 @@
[PARSER]
Name timestamp_parser
Format regex
Regex ^(?<timestamp>\d{4}-\d{2}-\d{2} \d{2}:\d{2}:\d{2})\s+(?<level>\w+)\s+(?<message>.*)$
Regex ^(?<timestamp>\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}(?:Z|[+-]\d{2}:?\d{2}))\s+(?<level>\w+)\s+(?<message>.*)$
Time_Key timestamp
Time_Format %Y-%m-%d %H:%M:%S
Time_Format %Y-%m-%dT%H:%M:%S%z

View File

@@ -140,11 +140,13 @@ if [[ -d "/etc/fluent-bit" ]]; then
rm -rf /etc/fluent-bit
fi
# 安装 Fluent Bit 主包
# 安装 Fluent Bit 主包(按架构选择 deb)
log_info "Installing Fluent Bit from deb package..."
deb_file="bin/fluent-bit_3.1.9_amd64.deb"
if [[ ! -f "$deb_file" ]]; then
log_error "Fluent Bit package not found: $deb_file"
deb_arch="$(dpkg --print-architecture 2>/dev/null || echo amd64)"
deb_pattern="bin/${deb_arch}/fluent-bit_*_${deb_arch}.deb"
deb_file=$(ls $deb_pattern 2>/dev/null | head -n1 || true)
if [[ -z "$deb_file" || ! -f "$deb_file" ]]; then
log_error "Fluent Bit package not found matching: $deb_pattern"
exit 1
fi
@@ -171,9 +173,16 @@ fi
# 创建日志和缓冲区目录
log_info "Creating log and buffer directories..."
mkdir -p /logs/train /logs/infer /buffers
chmod 755 /logs/train /logs/infer
chmod 770 /buffers
chown -R fluent-bit:fluent-bit /logs /buffers
# 对共享日志目录采用 1777(含粘滞位),便于宿主任意账号创建文件/目录
if [[ "${ARGUS_LOGS_WORLD_WRITABLE:-1}" == "1" ]]; then
chmod 1777 /logs/train /logs/infer || true
else
chmod 755 /logs/train /logs/infer || true
fi
# 缓冲目录限进程使用
chmod 770 /buffers || true
# 目录属主设置,不影响 1777 粘滞位
chown -R fluent-bit:fluent-bit /logs /buffers 2>/dev/null || true
# 启动 Fluent Bit
log_info "Starting Fluent Bit with configuration from /etc/fluent-bit/"
@@ -206,7 +215,8 @@ export HOSTNAME
export CLUSTER="${CLUSTER:-local}"
export RACK="${RACK:-dev}"
export ES_HOST="${ES_HOST:-localhost}"
# 默认使用固定域名(满足“固定域名”需求);若外部传入覆盖,则使用外部值
export ES_HOST="${ES_HOST:-es.log.argus.com}"
export ES_PORT="${ES_PORT:-9200}"
log_info "Environment variables:"

View File

@@ -28,10 +28,18 @@ log_info "检查必要文件..."
required_files=(
"install.sh"
"uninstall.sh"
"bin/fluent-bit_3.1.9_amd64.deb"
"check_health.sh"
)
# 架构特定的 deb 至少各有一个(版本可不同)
for arch in amd64 arm64; do
if ! ls "bin/${arch}/fluent-bit_"*"_${arch}.deb" >/dev/null 2>&1; then
echo "缺少以下文件:"
echo " - bin/${arch}/fluent-bit_*_${arch}.deb"
exit 1
fi
done
missing_files=()
for file in "${required_files[@]}"; do
if [[ ! -f "$file" ]]; then

View File

@@ -47,6 +47,13 @@ if [[ ${#missing_files[@]} -gt 0 ]]; then
exit 1
fi
# 防御:阻止将 Git LFS 指针文件打包
if head -n1 bin/node_exporter 2>/dev/null | grep -q '^version https://git-lfs.github.com/spec/v1$'; then
echo "[ERROR] bin/node_exporter 是 Git LFS 指针文件,未还原为真实二进制"
echo " 请在仓库根目录执行: git lfs fetch --all && git lfs checkout"
exit 1
fi
log_success "所有必要文件检查完成"
# 创建临时目录

View File

@@ -274,19 +274,33 @@ verify_checksums() {
log_info "Artifact 目录: $artifact_dir"
failed_verification=0
# 尝试解析 version.json 中的 install_order,用于锁定精确文件名,避免同一目录下多份历史 tar 产生歧义
local order_file="$TEMP_DIR/install_order.txt"
if [[ -f "$TEMP_DIR/checksums.txt" ]]; then
while IFS= read -r line; do
component=$(echo "$line" | cut -d':' -f1)
expected_checksum=$(echo "$line" | cut -d':' -f2-)
# 查找匹配的 tar 文件
# 优先从 install_order 中推导精确文件名
actual_file=""
for file in "$artifact_dir/${component}-"*.tar.gz; do
if [[ -f "$file" ]]; then
actual_file="$file"
break
fi
done
if [[ -f "$order_file" ]]; then
while IFS= read -r fname; do
if [[ "$fname" == ${component}-*.tar.gz && -f "$artifact_dir/$fname" ]]; then
actual_file="$artifact_dir/$fname"
break
fi
done < "$order_file"
fi
# 回退:按前缀匹配首个(不推荐,但保持兼容)
if [[ -z "$actual_file" ]]; then
for file in "$artifact_dir/${component}-"*.tar.gz; do
if [[ -f "$file" ]]; then
actual_file="$file"
break
fi
done
fi
if [[ -z "$actual_file" ]]; then
log_error "找不到组件文件: $component"

View File

@@ -59,6 +59,12 @@ ARTIFACT_DIR="artifact/$VERSION"
log_info "开始打包 AIOps All-in-One 安装包 v$VERSION"
# 若强制打包且目录已存在,先清理旧产物以避免同一版本下残留多个 tar.gz 导致校验混乱
if [[ "$FORCE_PACKAGE" == "true" && -d "$ARTIFACT_DIR" ]]; then
log_info "--force: 清理旧的 $ARTIFACT_DIR 下的 tar 与元数据"
rm -rf "$ARTIFACT_DIR"
fi
# 检查必要文件
log_info "检查必要文件..."
if [[ ! -f "config/VERSION" ]]; then
@@ -130,7 +136,7 @@ if [[ -d "$ARTIFACT_DIR" && "$FORCE_PACKAGE" == "false" ]]; then
fi
fi
# 创建 artifact 目录
# 创建 artifact 目录(清理后重建)
mkdir -p "$ARTIFACT_DIR"
log_info "创建输出目录: $ARTIFACT_DIR"
@@ -216,6 +222,36 @@ if [[ ${#missing_components[@]} -gt 0 ]]; then
exit 1
fi
# 额外校验:阻止将 Git LFS 指针文件打进安装包
# 仅检查各组件目录下的 bin/ 内文件(常见为二进制或 .deb/.tar.gz 制品)
is_lfs_pointer() {
local f="$1"
# 读取首行判断是否为 LFS pointer(无需依赖 file 命令)
head -n1 "$f" 2>/dev/null | grep -q '^version https://git-lfs.github.com/spec/v1$'
}
log_info "检查组件二进制是否已从 LFS 拉取..."
while IFS= read -r component; do
component_path=$(grep "^$component:" "$TEMP_DIR/component_paths.txt" | cut -d':' -f2-)
bin_dir="$component_path/bin"
[[ -d "$bin_dir" ]] || continue
while IFS= read -r f; do
# 只检查常见可执行/包后缀;无后缀的也检查
case "$f" in
*.sh) continue;;
*) :;;
esac
if is_lfs_pointer "$f"; then
log_error "检测到 Git LFS 指针文件: $f"
log_error "请在仓库根目录执行: git lfs fetch --all && git lfs checkout"
log_error "或确保 CI 在打包前已还原 LFS 大文件。"
rm -rf "$TEMP_DIR"
exit 1
fi
done < <(find "$bin_dir" -maxdepth 1 -type f 2>/dev/null | sort)
done < "$COMPONENTS_FILE"
log_success "LFS 校验通过:未发现指针文件"
# 打包各个组件
log_info "开始打包组件..."
@@ -235,6 +271,18 @@ while IFS= read -r component; do
# 进入组件目录
cd "$component_path"
# 组件内二次防御:若包脚本缺失 LFS 校验,这里再次阻断
if [[ -d bin ]]; then
for f in bin/*; do
[[ -f "$f" ]] || continue
if head -n1 "$f" 2>/dev/null | grep -q '^version https://git-lfs.github.com/spec/v1$'; then
log_error "组件 $component 含 LFS 指针文件: $f"
log_error "请执行: git lfs fetch --all && git lfs checkout"
cd "$CURRENT_DIR"; rm -rf "$TEMP_DIR"; exit 1
fi
done
fi
# 检查组件是否有 package.sh
if [[ ! -f "package.sh" ]]; then
log_error "$component 缺少 package.sh 文件"
@@ -243,10 +291,13 @@ while IFS= read -r component; do
exit 1
fi
# 清理组件目录内历史 tar 包,避免 find 误选旧文件
rm -f ./*.tar.gz 2>/dev/null || true
# 执行组件的打包脚本
if ./package.sh; then
# 查找生成的 tar 包
tar_file=$(find . -name "*.tar.gz" -type f | head -1)
tar_file=$(ls -1t ./*.tar.gz 2>/dev/null | head -1)
if [[ -n "$tar_file" ]]; then
# 移动到 artifact 目录
mv "$tar_file" "$CURRENT_DIR/$ARTIFACT_DIR/"

View File

@@ -130,20 +130,40 @@ fi
TEMP_PACKAGE_DIR="/tmp/argus-metric-package-$$"
mkdir -p "$TEMP_PACKAGE_DIR"
# 复制所有 tar.gz 文件到临时目录
log_info "准备 artifact 文件..."
tar_files=$(find "$ARTIFACT_DIR" -name "*.tar.gz" -type f)
# 仅复制 version.json 中 install_order 列出的 tar.gz,防止同一版本目录下历史残留文件导致校验不一致
log_info "准备 artifact 文件(按 install_order)..."
if [[ -z "$tar_files" ]]; then
log_error "在 $ARTIFACT_DIR 中未找到 tar.gz 文件"
exit 1
install_list_file="$TEMP_DIR/install_list.txt"
if command -v jq >/dev/null 2>&1; then
jq -r '.install_order[]' "$ARTIFACT_DIR/version.json" > "$install_list_file" 2>/dev/null || true
else
# 简易解析
grep -A 200 '"install_order"' "$ARTIFACT_DIR/version.json" | grep -E '".*"' | sed 's/.*"\([^"]*\)".*/\1/' > "$install_list_file" 2>/dev/null || true
fi
for file in $tar_files; do
filename=$(basename "$file")
log_info " 准备: $filename"
cp "$file" "$TEMP_PACKAGE_DIR/"
done
if [[ -s "$install_list_file" ]]; then
while IFS= read -r filename; do
src="$ARTIFACT_DIR/$filename"
if [[ -f "$src" ]]; then
log_info " 拷贝: $filename"
cp "$src" "$TEMP_PACKAGE_DIR/"
else
log_warning " 未找到: $filename(跳过)"
fi
done < "$install_list_file"
else
log_warning "未能解析 install_order,将回退复制全部 tar.gz(可能包含历史残留,建议安装端使用严格校验)"
tar_files=$(find "$ARTIFACT_DIR" -name "*.tar.gz" -type f)
if [[ -z "$tar_files" ]]; then
log_error "在 $ARTIFACT_DIR 中未找到 tar.gz 文件"
exit 1
fi
for file in $tar_files; do
filename=$(basename "$file")
log_info " 准备: $filename"
cp "$file" "$TEMP_PACKAGE_DIR/"
done
fi
# 复制版本信息文件
if [[ -f "$ARTIFACT_DIR/version.json" ]]; then

View File

@@ -48,6 +48,31 @@ BACKUPS_DIR="$INSTALL_DIR/backups" # 备份目录
CURRENT_LINK="$INSTALL_DIR/current" # 当前版本软链接
LATEST_VERSION_FILE="$INSTALL_DIR/LATEST_VERSION" # 当前版本记录文件
# 预检查:Agent 元数据与 hostname 约束
require_agent_metadata() {
local hn
hn="$(hostname)"
local ok=false
# 三元环境变量
if [[ -n "${AGENT_ENV:-}" && -n "${AGENT_USER:-}" && -n "${AGENT_INSTANCE:-}" ]]; then
ok=true
fi
# host 形如 env-user-instance-xxx
if [[ "$hn" =~ ^[^-]+-[^-]+-[^-]+-.*$ ]]; then
ok=true
fi
if [[ "$ok" == false ]]; then
log_error "检测到 hostname 与 Agent 元数据不完整:"
log_error " 当前 hostname: $hn"
log_error " AGENT_ENV='${AGENT_ENV:-}' AGENT_USER='${AGENT_USER:-}' AGENT_INSTANCE='${AGENT_INSTANCE:-}'"
echo
log_info "请满足以下其一后重试:"
log_info " 方式A:设置 hostname 为 env-user-instance-任意,例如 dev-alice-node001-pod-0"
log_info " 方式B:导出环境变量:export AGENT_ENV=dev AGENT_USER=alice AGENT_INSTANCE=node001"
exit 1
fi
}
# 检查必需的FTP参数
check_ftp_params() {
local missing_params=()
@@ -873,6 +898,47 @@ rollback_version() {
fi
}
# 自检实现:等待 node.json 就绪且健康,并验证 last_report 持续更新
selfcheck_post_install() {
local hn="$(hostname)"
local node_file="/private/argus/agent/${AGENT_HOSTNAME:-$hn}/node.json"
local deadline=$(( $(date +%s) + 300 ))
local t1="" t2=""
while :; do
if [[ -f "$node_file" ]]; then
if command -v jq >/dev/null 2>&1; then
local ok_health lr
ok_health=$(jq -er '(.health["metric-argus-agent"].status=="healthy") and (.health["metric-node-exporter"].status=="healthy") and (.health["metric-fluent-bit"].status=="healthy") and (.health["metric-dcgm-exporter"].status=="healthy")' "$node_file" 2>/dev/null || echo false)
lr=$(jq -r '.last_report // ""' "$node_file" 2>/dev/null)
if [[ "$ok_health" == true && -n "$lr" ]]; then
if [[ -z "$t1" ]]; then
t1="$lr"
# agent 默认 60s 上报,等待 70s 再校验一次
sleep 70
continue
fi
t2="$lr"
if [[ "$t2" != "$t1" ]]; then
return 0
fi
# 若未变化,再等待一会儿直到超时
sleep 10
fi
else
# 无 jq 时的宽松校验
if grep -q '"status"\s*:\s*"healthy"' "$node_file"; then
return 0
fi
fi
fi
if (( $(date +%s) >= deadline )); then
log_error "自检超时:未在 5 分钟内确认 last_report 持续更新 或 健康状态不满足(路径:$node_file)"
return 1
fi
sleep 5
done
}
# 主函数
main() {
echo "=========================================="
@@ -912,8 +978,9 @@ main() {
# return 0
# fi
check_ftp_params
check_system
check_ftp_params
check_system
require_agent_metadata
if [[ "$ACTION" == "uninstall" ]]; then
uninstall_argus_metric
@@ -921,8 +988,16 @@ main() {
install_argus_metric
fi
# 安装后自检:最多等待 5 分钟,确认 node.json 存在且健康
echo
log_info "操作完成!"
log_info "开始安装后自检(最多等待 5 分钟)..."
selfcheck_post_install || {
log_error "安装后自检未通过,请查看 /var/log/argus-agent.log 以及 /opt/argus-metric/versions/*/.install.log"
exit 1
}
echo
log_success "全部自检通过,安装完成!"
}
# 脚本入口

View File

@@ -67,7 +67,8 @@ RUN chmod +x /usr/local/bin/start-ftp-supervised.sh
COPY vsftpd.conf /etc/vsftpd/vsftpd.conf
COPY dns-monitor.sh /usr/local/bin/dns-monitor.sh
RUN chmod +x /usr/local/bin/dns-monitor.sh
COPY dns-publish.sh /usr/local/bin/dns-publish.sh
RUN chmod +x /usr/local/bin/dns-monitor.sh /usr/local/bin/dns-publish.sh
USER root

Some files were not shown because too many files have changed in this diff Show More