NASP/argus
83
0
Fork 0
Code Issues 8 Pull Requests 1 Actions Packages Projects 1 Releases 1 Wiki Activity

v1.0.0 发布合并回master (#52)

Browse Source 
Co-authored-by: sundapeng <sundp@mail.zgclab.edu.cn>
Co-authored-by: xuxt <xuxt@zgclab.edu.cn>
Reviewed-on: #52
This commit was merged in pull request #52.
This commit is contained in:
yuyr
2025-11-25 16:00:50 +08:00
parent 9e1336d13e
commit 7430896fab
464 changed files with 45636 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files

38
deployment_new/templates/client_gpu/compose/docker-compose.yml Normal file
View File

@@ -0,0 +1,38 @@
version: "3.8"
networks:
argus-sys-net:
external: true
services:
metric-gpu-node:
image: ${NODE_GPU_BUNDLE_IMAGE_TAG:-argus-sys-metric-test-node-bundle-gpu:${PKG_VERSION}}
container_name: argus-metric-gpu-node-swarm
hostname: ${GPU_NODE_HOSTNAME}
restart: unless-stopped
privileged: true
runtime: nvidia
environment:
- TZ=Asia/Shanghai
- DEBIAN_FRONTEND=noninteractive
- MASTER_ENDPOINT=${MASTER_ENDPOINT:-http://master.argus.com:3000}
# Fluent Bit / 日志上报目标（固定域名）
- ES_HOST=es.log.argus.com
- ES_PORT=9200
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
- AGENT_ENV=${AGENT_ENV}
- AGENT_USER=${AGENT_USER}
- AGENT_INSTANCE=${AGENT_INSTANCE}
- NVIDIA_VISIBLE_DEVICES=all
- NVIDIA_DRIVER_CAPABILITIES=compute,utility
- GPU_MODE=gpu
networks:
argus-sys-net:
aliases:
- ${AGENT_INSTANCE}.node.argus.com
volumes:
- ../private/argus/agent:/private/argus/agent
- ../logs/infer:/logs/infer
- ../logs/train:/logs/train
command: ["sleep", "infinity"]

73
deployment_new/templates/client_gpu/docs/INSTALL_CLIENT_zh.md Normal file
View File

@@ -0,0 +1,73 @@
# Argus Client‑GPU 安装指南（deployment_new）
## 一、准备条件（开始前确认）
- GPU 节点安装了 NVIDIA 驱动，`nvidia-smi` 正常；
- Docker & Docker Compose v2 已安装；
- 使用统一账户 `argus`（UID=2133，GID=2015）执行安装，并加入 `docker` 组（如已创建可跳过）：
```bash
sudo groupadd --gid 2015 argus || true
sudo useradd --uid 2133 --gid 2015 --create-home --shell /bin/bash argus || true
sudo passwd argus
sudo usermod -aG docker argus
su - argus -c 'id; docker ps >/dev/null && echo OK || echo NO_DOCKER_PERMISSION'
```
后续解压与执行（config/install/uninstall）均使用 `argus` 账户进行。
- 从 Server 安装方拿到 `cluster-info.env`（包含 `SWARM_MANAGER_ADDR/SWARM_JOIN_TOKEN_*`；compose 架构下 BINDIP/FTPIP 不再使用）。
## 二、解包
- `tar -xzf client_gpu_YYYYMMDD.tar.gz`
- 进入目录：`cd client_gpu_YYYYMMDD/`
- 你应当看到：`images/`（GPU bundle、busybox）、`compose/`、`scripts/`、`docs/`。
## 三、配置 config（预热 overlay + 生成 .env）
命令：
```
cp /path/to/cluster-info.env ./ # 或 export CLUSTER_INFO=/abs/path/cluster-info.env
./scripts/config.sh
```
脚本做了什么：
- 读取 `cluster-info.env` 并 `docker swarm join`（幂等）；
- 自动用 busybox 预热 external overlay `argus-sys-net`，等待最多 60s 直到本机可见；
- 生成/更新 `compose/.env`：填入 `SWARM_*`，并“保留你已填写的 AGENT_* 与 GPU_NODE_HOSTNAME”（不会覆盖）。
看到什么才算成功：
- 终端输出类似：`已预热 overlay=argus-sys-net 并生成 compose/.env；可执行 scripts/install.sh`；
- `compose/.env` 至少包含：
- `AGENT_ENV/AGENT_USER/AGENT_INSTANCE/GPU_NODE_HOSTNAME`（需要你提前填写）；
- `SWARM_MANAGER_ADDR/SWARM_JOIN_TOKEN_*`；
- `NODE_GPU_BUNDLE_IMAGE_TAG=...:YYYYMMDD`。
### 日志映射（重要）
- 容器内 `/logs/infer` 与 `/logs/train` 已映射到包根 `./logs/infer` 与 `./logs/train`：
- 你可以直接在宿主机查看推理/训练日志：`tail -f logs/infer/*.log`、`tail -f logs/train/*.log`；
- install 脚本会自动创建这两个目录。
若提示缺少必填项：
- 打开 `compose/.env` 按提示补齐 `AGENT_*` 与 `GPU_NODE_HOSTNAME`，再次执行 `./scripts/config.sh`（脚本不会覆盖你已填的值）。
## 四、安装 install（加载镜像 + 起容器 + 跟日志）
命令：
```
./scripts/install.sh
```
脚本做了什么：
- 如有必要，先自动预热 overlay；
- 从 `images/` 导入 `argus-sys-metric-test-node-bundle-gpu-*.tar.gz` 到本地 Docker；
- `docker compose up -d` 启动 GPU 节点容器，并自动执行 `docker logs -f argus-metric-gpu-node-swarm` 跟踪安装过程。
看到什么才算成功：
- 日志中出现：`[BOOT] local bundle install OK: version=...` / `dcgm-exporter ... listening` / `node state present: /private/argus/agent/<hostname>/node.json`；
- `docker exec argus-metric-gpu-node-swarm nvidia-smi -L` 能列出 GPU；
- 在 Server 侧 Prometheus `/api/v1/targets` 中，GPU 节点 9100（node-exporter）与 9400（dcgm-exporter）至少其一 up。
## 五、卸载 uninstall
命令：
```
./scripts/uninstall.sh
```
行为：Compose down（如有 .env），并删除 warmup 容器与节点容器。
## 六、常见问题
- `本机未看到 overlay`：config/install 已自动预热；若仍失败，请检查与 manager 的网络连通性以及 manager 上是否已创建 `argus-sys-net`。
- `busybox 缺失`：确保包根 `images/busybox.tar` 在，或主机已有 `busybox:latest`。
- `加入 Swarm 失败`：确认 `cluster-info.env` 的 `SWARM_MANAGER_ADDR` 与 `SWARM_JOIN_TOKEN_WORKER` 正确，或在 manager 上重新 `docker swarm join-token -q worker` 后更新该文件。

BIN
deployment_new/templates/client_gpu/images/busybox.tar Normal file
View File

Binary file not shown.

90
deployment_new/templates/client_gpu/scripts/config.sh Normal file
View File

@@ -0,0 +1,90 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_EX="$PKG_ROOT/compose/.env.example"
ENV_OUT="$PKG_ROOT/compose/.env"
info(){ echo -e "\033[34m[CONFIG-GPU]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
# Compose 检测：优先 docker compose（v2），回退 docker-compose（v1）
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose，请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker curl jq awk sed tar gzip
require_compose
# 磁盘空间检查（MB）
check_disk(){ local p="$1"; local need=10240; local free
free=$(df -Pm "$p" | awk 'NR==2{print $4+0}')
if [[ -z "$free" || "$free" -lt "$need" ]]; then err "磁盘空间不足: $p 剩余 ${free:-0}MB (<${need}MB)"; return 1; fi
}
check_disk "$PKG_ROOT"; check_disk "/var/lib/docker" || true
# 导入 cluster-info.env（默认取当前包根，也可用 CLUSTER_INFO 指定路径）
CI_IN="${CLUSTER_INFO:-$PKG_ROOT/cluster-info.env}"
info "读取 cluster-info.env: $CI_IN"
[[ -f "$CI_IN" ]] || { err "找不到 cluster-info.env（默认当前包根，或设置环境变量 CLUSTER_INFO 指定绝对路径）"; exit 1; }
set -a; source "$CI_IN"; set +a
[[ -n "${SWARM_MANAGER_ADDR:-}" && -n "${SWARM_JOIN_TOKEN_WORKER:-}" ]] || { err "cluster-info.env 缺少 SWARM 信息（SWARM_MANAGER_ADDR/SWARM_JOIN_TOKEN_WORKER）"; exit 1; }
# 加入 Swarm（幂等）
info "加入 Swarm（幂等）：$SWARM_MANAGER_ADDR"
docker swarm join --token "$SWARM_JOIN_TOKEN_WORKER" "$SWARM_MANAGER_ADDR":2377 >/dev/null 2>&1 || true
# 导入 busybox 并做 overlay 预热与连通性（总是执行）
NET_NAME="${ARGUS_OVERLAY_NET:-argus-sys-net}"
# 准备 busybox
if ! docker image inspect busybox:latest >/dev/null 2>&1; then
if [[ -f "$PKG_ROOT/images/busybox.tar" ]]; then
info "加载 busybox.tar 以预热 overlay"
docker load -i "$PKG_ROOT/images/busybox.tar" >/dev/null
else
err "缺少 busybox 镜像（包内 images/busybox.tar 或本地 busybox:latest），无法预热 overlay $NET_NAME"; exit 1
fi
fi
# 预热容器（worker 侧加入 overlay 以便本地可见）
docker rm -f argus-net-warmup >/dev/null 2>&1 || true
info "启动 warmup 容器加入 overlay: $NET_NAME"
docker run -d --rm --name argus-net-warmup --network "$NET_NAME" busybox:latest sleep 600 >/dev/null 2>&1 || true
for i in {1..60}; do docker network inspect "$NET_NAME" >/dev/null 2>&1 && { info "overlay 可见 (t=${i}s)"; break; }; sleep 1; done
docker network inspect "$NET_NAME" >/dev/null 2>&1 || { err "预热后仍未看到 overlay: $NET_NAME；请确认 manager 已创建并网络可达"; exit 1; }
# 通过 warmup 容器测试实际数据通路（alias → master）
if ! docker exec argus-net-warmup sh -lc "ping -c 1 -W 2 master.argus.com >/dev/null 2>&1"; then
err "warmup 容器内无法通过别名访问 master.argus.com；请确认 server compose 已启动并加入 overlay $NET_NAME"
exit 1
fi
info "warmup 容器内可达 master.argus.com（Docker DNS + alias 正常）"
# 生成/更新 .env（保留人工填写项，不覆盖已有键）
if [[ ! -f "$ENV_OUT" ]]; then
cp "$ENV_EX" "$ENV_OUT"
fi
set_kv(){ local k="$1" v="$2"; if grep -q "^${k}=" "$ENV_OUT"; then sed -i -E "s#^${k}=.*#${k}=${v}#" "$ENV_OUT"; else echo "${k}=${v}" >> "$ENV_OUT"; fi }
set_kv SWARM_MANAGER_ADDR "${SWARM_MANAGER_ADDR:-}"
set_kv SWARM_JOIN_TOKEN_WORKER "${SWARM_JOIN_TOKEN_WORKER:-}"
set_kv SWARM_JOIN_TOKEN_MANAGER "${SWARM_JOIN_TOKEN_MANAGER:-}"
REQ_VARS=(AGENT_ENV AGENT_USER AGENT_INSTANCE GPU_NODE_HOSTNAME)
missing=()
for v in "${REQ_VARS[@]}"; do
val=$(grep -E "^$v=" "$ENV_OUT" | head -1 | cut -d= -f2-)
if [[ -z "$val" ]]; then missing+=("$v"); fi
done
if [[ ${#missing[@]} -gt 0 ]]; then
err "以下变量必须在 compose/.env 中填写：${missing[*]}（已保留你现有的内容，不会被覆盖）"; exit 1; fi
info "已生成 compose/.env；可执行 scripts/install.sh"
# 准备并赋权宿主日志目录（幂等，便于安装前人工检查/预创建）
mkdir -p "$PKG_ROOT/logs/train" "$PKG_ROOT/logs/infer"
chmod 1777 "$PKG_ROOT/logs/train" "$PKG_ROOT/logs/infer" || true
info "日志目录权限（期待 1777，含粘滞位）:"
stat -c '%a %U:%G %n' "$PKG_ROOT/logs/train" "$PKG_ROOT/logs/infer" 2>/dev/null || true

72
deployment_new/templates/client_gpu/scripts/install.sh Normal file
View File

@@ -0,0 +1,72 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
info(){ echo -e "\033[34m[INSTALL-GPU]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
# Compose 检测：优先 docker compose（v2），回退 docker-compose（v1）
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose，请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker nvidia-smi
require_compose
[[ -f "$ENV_FILE" ]] || { err "缺少 compose/.env，请先运行 scripts/config.sh"; exit 1; }
info "使用环境文件: $ENV_FILE"
# 预热 overlay（当 config 执行很久之前或容器已被清理时，warmup 可能不存在）
set -a; source "$ENV_FILE"; set +a
NET_NAME="${ARGUS_OVERLAY_NET:-argus-sys-net}"
info "检查 overlay 网络可见性: $NET_NAME"
if ! docker network inspect "$NET_NAME" >/dev/null 2>&1; then
# 如 Overlay 不可见，尝试用 busybox 预热（仅为确保 worker 节点已加入 overlay）
if ! docker image inspect busybox:latest >/dev/null 2>&1; then
if [[ -f "$PKG_ROOT/images/busybox.tar" ]]; then docker load -i "$PKG_ROOT/images/busybox.tar"; else err "缺少 busybox 镜像（images/busybox.tar 或本地 busybox:latest）"; exit 1; fi
fi
docker rm -f argus-net-warmup >/dev/null 2>&1 || true
docker run -d --rm --name argus-net-warmup --network "$NET_NAME" busybox:latest sleep 600 >/dev/null 2>&1 || true
for i in {1..60}; do docker network inspect "$NET_NAME" >/dev/null 2>&1 && break; sleep 1; done
docker network inspect "$NET_NAME" >/dev/null 2>&1 || { err "预热后仍未看到 overlay: $NET_NAME；请确认 manager 已创建并网络可达"; exit 1; }
info "overlay 已可见（warmup=argus-net-warmup）"
fi
# 若本函数内重新创建了 warmup 容器，同样测试一次 alias 数据通路
if docker ps --format '{{.Names}}' | grep -q '^argus-net-warmup$'; then
if ! docker exec argus-net-warmup sh -lc "ping -c 1 -W 2 master.argus.com >/dev/null 2>&1"; then
err "GPU install 阶段：warmup 容器内无法通过别名访问 master.argus.com；请检查 overlay $NET_NAME 与 server 状态"
exit 1
fi
info "GPU install 阶段：warmup 容器内可达 master.argus.com"
fi
# 导入 GPU bundle 镜像
IMG_TGZ=$(ls -1 "$PKG_ROOT"/images/argus-sys-metric-test-node-bundle-gpu-*.tar.gz 2>/dev/null | head -1 || true)
[[ -n "$IMG_TGZ" ]] || { err "找不到 GPU bundle 镜像 tar.gz"; exit 1; }
info "导入 GPU bundle 镜像: $(basename "$IMG_TGZ")"
tmp=$(mktemp); gunzip -c "$IMG_TGZ" > "$tmp"; docker load -i "$tmp" >/dev/null; rm -f "$tmp"
# 确保日志目录存在（宿主侧，用于映射 /logs/infer 与 /logs/train），并赋权 1777（粘滞位）
mkdir -p "$PKG_ROOT/logs/infer" "$PKG_ROOT/logs/train"
chmod 1777 "$PKG_ROOT/logs/infer" "$PKG_ROOT/logs/train" || true
info "日志目录已准备并赋权 1777: logs/infer logs/train"
stat -c '%a %U:%G %n' "$PKG_ROOT/logs/infer" "$PKG_ROOT/logs/train" 2>/dev/null || true
# 启动 compose 并跟踪日志
PROJECT="${COMPOSE_PROJECT_NAME:-argus-client}"
info "启动 GPU 节点 (docker compose -p $PROJECT up -d)"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" up -d
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" ps
# 再次校准宿主日志目录权限，避免容器内脚本对 bind mount 权限回退
chmod 1777 "$PKG_ROOT/logs/infer" "$PKG_ROOT/logs/train" || true
stat -c '%a %U:%G %n' "$PKG_ROOT/logs/infer" "$PKG_ROOT/logs/train" 2>/dev/null || true
info "跟踪节点容器日志（按 Ctrl+C 退出）"
docker logs -f argus-metric-gpu-node-swarm || true

36
deployment_new/templates/client_gpu/scripts/uninstall.sh Normal file
View File

@@ -0,0 +1,36 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
# load COMPOSE_PROJECT_NAME if provided in compose/.env
if [[ -f "$ENV_FILE" ]]; then set -a; source "$ENV_FILE"; set +a; fi
PROJECT="${COMPOSE_PROJECT_NAME:-argus-client}"
info(){ echo -e "\033[34m[UNINSTALL-GPU]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
# Compose 检测：优先 docker compose（v2），回退 docker-compose（v1）
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose，请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require_compose
if [[ -f "$ENV_FILE" ]]; then
info "stopping compose project (project=$PROJECT)"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" down --remove-orphans || true
else
info "compose/.env not found; attempting to remove container by name"
fi
# remove warmup container if still running
docker rm -f argus-net-warmup >/dev/null 2>&1 || true
# remove node container if present
docker rm -f argus-metric-gpu-node-swarm >/dev/null 2>&1 || true
info "uninstall completed"

169
deployment_new/templates/server/compose/docker-compose.yml Normal file
View File

@@ -0,0 +1,169 @@
version: "3.8"
networks:
argus-sys-net:
external: true
services:
master:
image: ${MASTER_IMAGE_TAG:-argus-master:${PKG_VERSION}}
container_name: argus-master-sys
environment:
- OFFLINE_THRESHOLD_SECONDS=6
- ONLINE_THRESHOLD_SECONDS=2
- SCHEDULER_INTERVAL_SECONDS=1
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
ports:
- "${MASTER_PORT:-32300}:3000"
volumes:
- ../private/argus/master:/private/argus/master
- ../private/argus/metric/prometheus:/private/argus/metric/prometheus
- ../private/argus/etc:/private/argus/etc
networks:
argus-sys-net:
aliases:
- master.argus.com
restart: unless-stopped
es:
image: ${ES_IMAGE_TAG:-argus-elasticsearch:${PKG_VERSION}}
container_name: argus-es-sys
environment:
- discovery.type=single-node
- xpack.security.enabled=false
- ES_JAVA_OPTS=-Xms512m -Xmx512m
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
volumes:
- ../private/argus/log/elasticsearch:/private/argus/log/elasticsearch
- ../private/argus/etc:/private/argus/etc
ports:
- "${ES_HTTP_PORT:-9200}:9200"
restart: unless-stopped
networks:
argus-sys-net:
aliases:
- es.log.argus.com
kibana:
image: ${KIBANA_IMAGE_TAG:-argus-kibana:${PKG_VERSION}}
container_name: argus-kibana-sys
environment:
- ELASTICSEARCH_HOSTS=http://es.log.argus.com:9200
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
volumes:
- ../private/argus/log/kibana:/private/argus/log/kibana
- ../private/argus/etc:/private/argus/etc
depends_on: [es]
ports:
- "${KIBANA_PORT:-5601}:5601"
restart: unless-stopped
networks:
argus-sys-net:
aliases:
- kibana.log.argus.com
prometheus:
image: ${PROM_IMAGE_TAG:-argus-metric-prometheus:${PKG_VERSION}}
container_name: argus-prometheus
restart: unless-stopped
environment:
- TZ=Asia/Shanghai
- PROMETHEUS_BASE_PATH=/private/argus/metric/prometheus
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
ports:
- "${PROMETHEUS_PORT:-9090}:9090"
volumes:
- ../private/argus/metric/prometheus:/private/argus/metric/prometheus
- ../private/argus/etc:/private/argus/etc
networks:
argus-sys-net:
aliases:
- prom.metric.argus.com
grafana:
image: ${GRAFANA_IMAGE_TAG:-argus-metric-grafana:${PKG_VERSION}}
container_name: argus-grafana
restart: unless-stopped
environment:
- TZ=Asia/Shanghai
- GRAFANA_BASE_PATH=/private/argus/metric/grafana
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
- GF_SERVER_HTTP_PORT=3000
- GF_LOG_LEVEL=warn
- GF_LOG_MODE=console
- GF_PATHS_PROVISIONING=/private/argus/metric/grafana/provisioning
- GF_AUTH_ANONYMOUS_ENABLED=true
- GF_AUTH_ANONYMOUS_ORG_ROLE=Viewer
ports:
- "${GRAFANA_PORT:-3000}:3000"
volumes:
- ../private/argus/metric/grafana:/private/argus/metric/grafana
- ../private/argus/etc:/private/argus/etc
depends_on: [prometheus]
networks:
argus-sys-net:
aliases:
- grafana.metric.argus.com
alertmanager:
image: ${ALERT_IMAGE_TAG:-argus-alertmanager:${PKG_VERSION}}
container_name: argus-alertmanager
environment:
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
volumes:
- ../private/argus/etc:/private/argus/etc
- ../private/argus/alert/alertmanager:/private/argus/alert/alertmanager
networks:
argus-sys-net:
aliases:
- alertmanager.alert.argus.com
ports:
- "${ALERTMANAGER_PORT:-9093}:9093"
restart: unless-stopped
web-frontend:
image: ${FRONT_IMAGE_TAG:-argus-web-frontend:${PKG_VERSION}}
container_name: argus-web-frontend
environment:
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
- EXTERNAL_MASTER_PORT=${WEB_PROXY_PORT_8085:-8085}
- EXTERNAL_ALERTMANAGER_PORT=${WEB_PROXY_PORT_8084:-8084}
- EXTERNAL_GRAFANA_PORT=${WEB_PROXY_PORT_8081:-8081}
- EXTERNAL_PROMETHEUS_PORT=${WEB_PROXY_PORT_8082:-8082}
- EXTERNAL_KIBANA_PORT=${WEB_PROXY_PORT_8083:-8083}
volumes:
- ../private/argus/etc:/private/argus/etc
networks:
argus-sys-net:
aliases:
- web.argus.com
restart: unless-stopped
web-proxy:
image: ${WEB_PROXY_IMAGE_TAG:-argus-web-proxy:${PKG_VERSION}}
container_name: argus-web-proxy
depends_on: [master, grafana, prometheus, kibana, alertmanager]
environment:
- ARGUS_BUILD_UID=${ARGUS_BUILD_UID:-2133}
- ARGUS_BUILD_GID=${ARGUS_BUILD_GID:-2015}
volumes:
- ../private/argus/etc:/private/argus/etc
networks:
argus-sys-net:
aliases:
- proxy.argus.com
ports:
- "${WEB_PROXY_PORT_8080:-8080}:8080"
- "${WEB_PROXY_PORT_8081:-8081}:8081"
- "${WEB_PROXY_PORT_8082:-8082}:8082"
- "${WEB_PROXY_PORT_8083:-8083}:8083"
- "${WEB_PROXY_PORT_8084:-8084}:8084"
- "${WEB_PROXY_PORT_8085:-8085}:8085"
restart: unless-stopped

102
deployment_new/templates/server/docs/INSTALL_SERVER_zh.md Normal file
View File

@@ -0,0 +1,102 @@
# Argus Server 安装指南（deployment_new）
适用：通过 Server 安装包在 Docker Swarm + external overlay 网络一体化部署 Argus 服务端组件。
—— 本文强调“怎么做、看什么、符合了才继续”。
## 一、准备条件（开始前确认）
- Docker 与 Docker Compose v2 已安装；`docker info` 正常；`docker compose version` 可执行。
- 具备 root/sudo 权限；磁盘可用空间 ≥ 10GB（包根与 `/var/lib/docker`）。
- 你知道本机管理地址（SWARM_MANAGER_ADDR），该 IP 属于本机某网卡，可被其他节点访问。
- 很重要：以统一账户 `argus`（UID=2133，GID=2015）执行后续安装与运维，并将其加入 `docker` 组；示例命令如下（如需不同 UID/GID，请替换为贵方标准）：
```bash
# 1) 创建主组（GID=2015，组名 argus；若已存在可跳过）
sudo groupadd --gid 2015 argus || true
# 2) 创建用户 argus（UID=2133、主组 GID=2015，创建家目录并用 bash 作为默认 shell；若已存在可用 usermod 调整）
sudo useradd --uid 2133 --gid 2015 --create-home --shell /bin/bash argus || true
sudo passwd argus
# 3) 将 argus 加入 docker 组，使其能调用 Docker Daemon（新登录后生效）
sudo usermod -aG docker argus
# 4) 验证（重新登录或执行 newgrp docker 使组生效）
su - argus -c 'id; docker ps >/dev/null && echo OK || echo NO_DOCKER_PERMISSION'
```
后续的解压与执行（config/install/selfcheck 等）均使用该 `argus` 账户进行。
## 二、解包与目录结构
- 解压：`tar -xzf server_YYYYMMDD.tar.gz`。
- 进入：`cd server_YYYYMMDD/`
- 你应当能看到：
- `images/`（逐服务镜像 tar.gz，如 `argus-master-YYYYMMDD.tar.gz`）
- `compose/`（`docker-compose.yml` 与 `.env.example`）
- `scripts/`（安装/运维脚本）
- `private/argus/`（数据与配置骨架）
- `docs/`（中文文档）
## 三、配置 config（生成 .env 与 SWARM_MANAGER_ADDR）
命令：
```
export SWARM_MANAGER_ADDR=<本机管理IP>
./scripts/config.sh
```
脚本做了什么：
- 检查依赖与磁盘空间；
- 自动从“端口 20000 起”分配所有服务端口，确保“系统未占用”且“彼此不冲突”；
- 写入 `compose/.env`（包含端口、镜像 tag、overlay 名称与 UID/GID 等）；
- 将当前执行账户的 UID/GID 写入 `ARGUS_BUILD_UID/GID`（若主组名是 docker，会改用“与用户名同名的组”的 GID，避免拿到 docker 组 999）；
- 更新/追加 `cluster-info.env` 中的 `SWARM_MANAGER_ADDR`（不会覆盖其他键）。
看到什么才算成功：
- 终端输出：`已生成 compose/.env 并更新 cluster-info.env 的 SWARM_MANAGER_ADDR。`
- `compose/.env` 打开应当看到：
- 端口均 ≥20000 且没有重复；
- `ARGUS_BUILD_UID/GID` 与 `id -u/-g` 一致；
- `SWARM_MANAGER_ADDR=<你的IP>`。
遇到问题：
- 端口被异常占用：可删去 `.env` 后再次执行 `config.sh`，或手工编辑端口再执行 `install.sh`。
## 四、安装 install（一次到位）
命令：
```
./scripts/install.sh
```
脚本做了什么：
- 若 Swarm 未激活：执行 `docker swarm init --advertise-addr $SWARM_MANAGER_ADDR`；
- 确保 external overlay `argus-sys-net` 存在；
- 导入 `images/*.tar.gz` 到本机 Docker；
- `docker compose up -d` 启动服务；
- 等待“六项就绪”：
- Master `/readyz`=200、ES `/_cluster/health`=200、Prometheus TCP 可达、Grafana `/api/health`=200、Alertmanager `/api/v2/status`=200、Kibana `/api/status` level=available；
- 校验 Docker DNS + overlay alias：在 `argus-web-proxy` 内通过 `getent hosts` 与 `curl` 检查 `master.argus.com`、`grafana.metric.argus.com` 等域名连通性；
- 写出 `cluster-info.env`（含 `SWARM_JOIN_TOKEN_{WORKER,MANAGER}/SWARM_MANAGER_ADDR`；compose 架构下不再依赖 BINDIP/FTPIP）；
- 生成 `安装报告_YYYYMMDD-HHMMSS.md`（端口、健康检查摘要与提示）。
看到什么才算成功：
- `docker compose ps` 全部是 Up；
- `安装报告_…md` 中各项 HTTP 检查为 200/available；
- `cluster-info.env` 包含五个关键键：
- `SWARM_MANAGER_ADDR=...`
- `SWARM_MANAGER_ADDR=...` `SWARM_JOIN_TOKEN_*=...`
- `SWARM_JOIN_TOKEN_WORKER=SWMTKN-...`
- `SWARM_JOIN_TOKEN_MANAGER=SWMTKN-...`
## 五、健康自检与常用操作
- 健康自检：`./scripts/selfcheck.sh`
- 期望输出：`selfcheck OK -> logs/selfcheck.json`
- 文件 `logs/selfcheck.json` 中 `overlay_net/es/kibana/master_readyz/prometheus/grafana/alertmanager/web_proxy_cors` 为 true。
- 状态：`./scripts/status.sh`（相当于 `docker compose ps`）。
- 诊断：`./scripts/diagnose.sh`（收集容器/HTTP/CORS/ES 细节，输出到 `logs/diagnose_*.log`）。
- 卸载：`./scripts/uninstall.sh`（Compose down）。
- ES 磁盘水位临时放宽/还原：`./scripts/es-watermark-relax.sh` / `./scripts/es-watermark-restore.sh`。
## 六、下一步：分发 cluster-info.env 给 Client
- 将 `cluster-info.env` 拷贝给安装 Client 的同事；
- 对方在 Client 机器的包根放置该文件（或设置 `CLUSTER_INFO=/绝对路径`）即可。
## 七、故障排查快览
- Proxy 502 或 8080 连接复位：通常是 overlay alias 未生效或 web-proxy 尚未解析到其它服务；重跑 `install.sh`（会重启栈并在容器内校验 DNS），或查看 `logs/diagnose_error.log`。
- Kibana 不 available：等待 1–2 分钟、查看 `argus-kibana-sys` 日志；
- cluster-info.env 的 SWARM_MANAGER_ADDR 为空：重新 `export SWARM_MANAGER_ADDR=<IP>; ./scripts/config.sh` 或 `./scripts/install.sh`（会回读 `.env` 补写）。

7
deployment_new/templates/server/docs/SWARM_DEPLOY_zh.md Normal file
View File

@@ -0,0 +1,7 @@
# Docker Swarm 部署要点
- 初始化 Swarm：`docker swarm init --advertise-addr <SWARM_MANAGER_ADDR>`
- 创建 overlay：`docker network create --driver overlay --attachable argus-sys-net`
- Server 包 `install.sh` 自动完成上述操作；如需手动执行，确保 `argus-sys-net` 存在且 attachable。
- Worker 节点加入：`docker swarm join --token <worker_token> <SWARM_MANAGER_ADDR>:2377`。

11
deployment_new/templates/server/docs/TROUBLESHOOTING_zh.md Normal file
View File

@@ -0,0 +1,11 @@
# 故障排查（Server）
- 端口占用：查看 `安装报告_*.md` 中端口表；如需修改，编辑 `compose/.env` 后执行 `docker compose ... up -d`。
- 组件未就绪：
- Master: `curl http://127.0.0.1:${MASTER_PORT}/readyz -I`
- ES: `curl http://127.0.0.1:${ES_HTTP_PORT}/_cluster/health`
- Grafana: `curl http://127.0.0.1:${GRAFANA_PORT}/api/health`
- Prometheus TCP: `exec 3<>/dev/tcp/127.0.0.1/${PROMETHEUS_PORT}`
- 域名解析：进入 `argus-web-proxy` 或 `argus-master-sys` 容器：`getent hosts master.argus.com`。
- Swarm/Overlay：检查 `docker network ls | grep argus-sys-net`，或 `docker node ls`。

108
deployment_new/templates/server/scripts/config.sh Normal file
View File

@@ -0,0 +1,108 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_EX="$PKG_ROOT/compose/.env.example"
ENV_OUT="$PKG_ROOT/compose/.env"
info(){ echo -e "\033[34m[CONFIG]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
# Compose 检测：优先 docker compose（v2），回退 docker-compose（v1）
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose，请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker curl jq awk sed tar gzip
require_compose
# 磁盘空间检查（MB）
check_disk(){ local p="$1"; local need=10240; local free
free=$(df -Pm "$p" | awk 'NR==2{print $4+0}')
if [[ -z "$free" || "$free" -lt "$need" ]]; then err "磁盘空间不足: $p 剩余 ${free:-0}MB (<${need}MB)"; return 1; fi
}
check_disk "$PKG_ROOT"; check_disk "/var/lib/docker" || true
# 读取/生成 SWARM_MANAGER_ADDR
SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR:-}
if [[ -z "${SWARM_MANAGER_ADDR}" ]]; then
read -rp "请输入本机管理地址 SWARM_MANAGER_ADDR: " SWARM_MANAGER_ADDR
fi
info "SWARM_MANAGER_ADDR=$SWARM_MANAGER_ADDR"
# 校验 IP 属于本机网卡
if ! ip -o addr | awk '{print $4}' | cut -d'/' -f1 | grep -qx "$SWARM_MANAGER_ADDR"; then
err "SWARM_MANAGER_ADDR 非本机地址: $SWARM_MANAGER_ADDR"; exit 1; fi
info "开始分配服务端口（起始=20000，避免系统占用与相互冲突）"
is_port_used(){ local p="$1"; ss -tulnH 2>/dev/null | awk '{print $5}' | sed 's/.*://g' | grep -qx "$p"; }
declare -A PRESENT=() CHOSEN=() USED=()
START_PORT="${START_PORT:-20000}"; cur=$START_PORT
ORDER=(MASTER_PORT ES_HTTP_PORT KIBANA_PORT PROMETHEUS_PORT GRAFANA_PORT ALERTMANAGER_PORT \
WEB_PROXY_PORT_8080 WEB_PROXY_PORT_8081 WEB_PROXY_PORT_8082 WEB_PROXY_PORT_8083 WEB_PROXY_PORT_8084 WEB_PROXY_PORT_8085 \
FTP_PORT FTP_DATA_PORT)
# 标记 .env.example 中实际存在的键
for key in "${ORDER[@]}"; do
if grep -q "^${key}=" "$ENV_EX"; then PRESENT[$key]=1; fi
done
next_free(){ local p="$1"; while :; do if [[ -n "${USED[$p]:-}" ]] || is_port_used "$p"; then p=$((p+1)); else echo "$p"; return; fi; done; }
for key in "${ORDER[@]}"; do
[[ -z "${PRESENT[$key]:-}" ]] && continue
p=$(next_free "$cur"); CHOSEN[$key]="$p"; USED[$p]=1; cur=$((p+1))
done
info "端口分配结果：MASTER=${CHOSEN[MASTER_PORT]:-} ES=${CHOSEN[ES_HTTP_PORT]:-} KIBANA=${CHOSEN[KIBANA_PORT]:-} PROM=${CHOSEN[PROMETHEUS_PORT]:-} GRAFANA=${CHOSEN[GRAFANA_PORT]:-} ALERT=${CHOSEN[ALERTMANAGER_PORT]:-} WEB_PROXY(8080..8085)=${CHOSEN[WEB_PROXY_PORT_8080]:-}/${CHOSEN[WEB_PROXY_PORT_8081]:-}/${CHOSEN[WEB_PROXY_PORT_8082]:-}/${CHOSEN[WEB_PROXY_PORT_8083]:-}/${CHOSEN[WEB_PROXY_PORT_8084]:-}/${CHOSEN[WEB_PROXY_PORT_8085]:-}"
cp "$ENV_EX" "$ENV_OUT"
# 覆盖端口（按唯一化结果写回）
for key in "${ORDER[@]}"; do
val="${CHOSEN[$key]:-}"
[[ -z "$val" ]] && continue
sed -i -E "s#^$key=.*#$key=${val}#" "$ENV_OUT"
done
info "已写入 compose/.env 的端口配置"
# 覆盖/补充 Overlay 名称
grep -q '^ARGUS_OVERLAY_NET=' "$ENV_OUT" || echo 'ARGUS_OVERLAY_NET=argus-sys-net' >> "$ENV_OUT"
# 以当前执行账户 UID/GID 写入（避免误选 docker 组）
RUID=$(id -u)
PRIMARY_GID=$(id -g)
PRIMARY_GRP=$(id -gn)
USER_NAME=$(id -un)
# 若主组名被解析为 docker，尝试用与用户名同名的组的 GID；否则回退主 GID
if [[ "$PRIMARY_GRP" == "docker" ]]; then
RGID=$(getent group "$USER_NAME" | awk -F: '{print $3}' 2>/dev/null || true)
[[ -z "$RGID" ]] && RGID="$PRIMARY_GID"
else
RGID="$PRIMARY_GID"
fi
info "使用构建账户 UID:GID=${RUID}:${RGID} (user=$USER_NAME primary_group=$PRIMARY_GRP)"
if grep -q '^ARGUS_BUILD_UID=' "$ENV_OUT"; then
sed -i -E "s#^ARGUS_BUILD_UID=.*#ARGUS_BUILD_UID=${RUID}#" "$ENV_OUT"
else
echo "ARGUS_BUILD_UID=${RUID}" >> "$ENV_OUT"
fi
if grep -q '^ARGUS_BUILD_GID=' "$ENV_OUT"; then
sed -i -E "s#^ARGUS_BUILD_GID=.*#ARGUS_BUILD_GID=${RGID}#" "$ENV_OUT"
else
echo "ARGUS_BUILD_GID=${RGID}" >> "$ENV_OUT"
fi
CI="$PKG_ROOT/cluster-info.env"
if [[ -f "$CI" ]]; then
if grep -q '^SWARM_MANAGER_ADDR=' "$CI"; then
sed -i -E "s#^SWARM_MANAGER_ADDR=.*#SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}#" "$CI"
else
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}" >> "$CI"
fi
else
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR}" > "$CI"
fi
info "已生成 compose/.env 并更新 cluster-info.env 的 SWARM_MANAGER_ADDR。"
info "下一步可执行: scripts/install.sh"

109
deployment_new/templates/server/scripts/diagnose.sh Normal file
View File

@@ -0,0 +1,109 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
ENV_FILE="$ROOT/compose/.env"; [[ -f "$ENV_FILE" ]] && set -a && source "$ENV_FILE" && set +a
ts="$(date -u +%Y%m%d-%H%M%SZ)"
LOG_DIR="$ROOT/logs"; mkdir -p "$LOG_DIR" || true
if ! ( : > "$LOG_DIR/.w" 2>/dev/null ); then LOG_DIR="/tmp/argus-logs"; mkdir -p "$LOG_DIR" || true; fi
# load compose project for accurate ps output
ENV_FILE="$ROOT/compose/.env"
if [[ -f "$ENV_FILE" ]]; then set -a; source "$ENV_FILE"; set +a; fi
PROJECT="${COMPOSE_PROJECT_NAME:-argus-server}"
DETAILS="$LOG_DIR/diagnose_details_${ts}.log"; ERRORS="$LOG_DIR/diagnose_error_${ts}.log"; : > "$DETAILS"; : > "$ERRORS"
logd() { echo "$(date '+%F %T') $*" >> "$DETAILS"; }
append_err() { echo "$*" >> "$ERRORS"; }
http_code() { curl -s -o /dev/null -w "%{http_code}" "$1" || echo 000; }
http_body_head() { curl -s --max-time 3 "$1" 2>/dev/null | sed -n '1,5p' || true; }
header_val() { curl -s -D - -o /dev/null "$@" | awk -F': ' 'BEGIN{IGNORECASE=1}$1=="Access-Control-Allow-Origin"{gsub("\r","",$2);print $2}'; }
section() { local name="$1"; logd "===== [$name] ====="; }
svc() {
local svc_name="$1"; local cname="$2"; shift 2
section "$svc_name ($cname)"
logd "docker ps:"; docker ps -a --format '{{.Names}}\t{{.Status}}\t{{.Image}}' | awk -v n="$cname" '$1==n' >> "$DETAILS" || true
logd "docker inspect:"; docker inspect -f '{{.State.Status}} rc={{.RestartCount}} started={{.State.StartedAt}}' "$cname" >> "$DETAILS" 2>&1 || true
logd "last 200 container logs:"; docker logs --tail 200 "$cname" >> "$DETAILS" 2>&1 || true
docker logs --tail 200 "$cname" 2>&1 | grep -Ei '\\b(error|failed|fail|exception|panic|fatal|critical|unhealthy|permission denied|forbidden|refused|traceback|错误|失败)\\b' | sed "s/^/[${svc_name}][container] /" >> "$ERRORS" || true
if docker exec "$cname" sh -lc 'command -v supervisorctl >/dev/null 2>&1' >/dev/null 2>&1; then
logd "supervisorctl status:"; docker exec "$cname" sh -lc 'supervisorctl status' >> "$DETAILS" 2>&1 || true
local files; files=$(docker exec "$cname" sh -lc 'ls /var/log/supervisor/*.log 2>/dev/null' || true)
for f in $files; do
logd "tail -n 80 $f:"; docker exec "$cname" sh -lc "tail -n 80 $f 2>/dev/null || true" >> "$DETAILS" 2>&1 || true
docker exec "$cname" sh -lc "tail -n 200 $f 2>/dev/null" 2>/dev/null | grep -Ei '\\b(error|failed|fail|exception|panic|fatal|critical|unhealthy|permission denied|forbidden|refused|traceback|错误|失败)\\b' | sed "s/^/[${svc_name}][supervisor:$(basename $f)] /" >> "$ERRORS" || true
done
fi
}
svc master argus-master-sys
svc es argus-es-sys
svc kibana argus-kibana-sys
svc prometheus argus-prometheus
svc grafana argus-grafana
svc alertmanager argus-alertmanager
svc web-frontend argus-web-frontend
svc web-proxy argus-web-proxy
section HTTP
logd "ES: $(http_code \"http://localhost:${ES_HTTP_PORT:-9200}/_cluster/health\")"; http_body_head "http://localhost:${ES_HTTP_PORT:-9200}/_cluster/health" >> "$DETAILS" 2>&1 || true
logd "Kibana: $(http_code \"http://localhost:${KIBANA_PORT:-5601}/api/status\")"; http_body_head "http://localhost:${KIBANA_PORT:-5601}/api/status" >> "$DETAILS" 2>&1 || true
logd "Master readyz: $(http_code \"http://localhost:${MASTER_PORT:-32300}/readyz\")"
logd "Prometheus: $(http_code \"http://localhost:${PROMETHEUS_PORT:-9090}/-/ready\")"
logd "Grafana: $(http_code \"http://localhost:${GRAFANA_PORT:-3000}/api/health\")"; http_body_head "http://localhost:${GRAFANA_PORT:-3000}/api/health" >> "$DETAILS" 2>&1 || true
logd "Alertmanager: $(http_code \"http://localhost:${ALERTMANAGER_PORT:-9093}/api/v2/status\")"
cors8084=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8084:-8084}/api/v2/status" || true)
cors8085=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8085:-8085}/api/v1/master/nodes" || true)
logd "Web-Proxy 8080: $(http_code \"http://localhost:${WEB_PROXY_PORT_8080:-8080}/\")"
logd "Web-Proxy 8083: $(http_code \"http://localhost:${WEB_PROXY_PORT_8083:-8083}/\")"
logd "Web-Proxy 8084 CORS: ${cors8084}"
logd "Web-Proxy 8085 CORS: ${cors8085}"
section ES-CHECKS
ch=$(curl -s --max-time 3 "http://localhost:${ES_HTTP_PORT:-9200}/_cluster/health" || true)
status=$(printf '%s' "$ch" | awk -F'\"' '/"status"/{print $4; exit}')
if [[ -n "$status" ]]; then logd "cluster.status=$status"; fi
if [[ "$status" != "green" ]]; then append_err "[es][cluster] status=$status"; fi
if docker ps --format '{{.Names}}' | grep -q '^argus-es-sys$'; then
duse=$(docker exec argus-es-sys sh -lc 'df -P /usr/share/elasticsearch/data | awk "NR==2{print \$5}"' 2>/dev/null || true)
logd "es.data.df_use=$duse"; usep=${duse%%%}
if [[ -n "$usep" ]] && (( usep >= 90 )); then append_err "[es][disk] data path usage=${usep}%"; fi
fi
section DNS-IN-PROXY
for d in master.argus.com es.log.argus.com kibana.log.argus.com grafana.metric.argus.com prom.metric.argus.com alertmanager.alert.argus.com; do
docker exec argus-web-proxy sh -lc "getent hosts $d || nslookup $d 2>/dev/null | tail -n+1" >> "$DETAILS" 2>&1 || true
done
logd "HTTP (web-proxy): master.readyz=$(docker exec argus-web-proxy sh -lc \"curl -s -o /dev/null -w '%{http_code}' http://master.argus.com:3000/readyz\" 2>/dev/null || echo 000)"
logd "HTTP (web-proxy): es.health=$(docker exec argus-web-proxy sh -lc \"curl -s -o /dev/null -w '%{http_code}' http://es.log.argus.com:9200/_cluster/health\" 2>/dev/null || echo 000)"
logd "HTTP (web-proxy): kibana.status=$(docker exec argus-web-proxy sh -lc \"curl -s -o /dev/null -w '%{http_code}' http://kibana.log.argus.com:5601/api/status\" 2>/dev/null || echo 000)"
section SYSTEM
logd "uname -a:"; uname -a >> "$DETAILS"
logd "docker version:"; docker version --format '{{.Server.Version}}' >> "$DETAILS" 2>&1 || true
logd "compose ps (project=$PROJECT):"; (cd "$ROOT/compose" && docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f docker-compose.yml ps) >> "$DETAILS" 2>&1 || true
section SUMMARY
[[ $(http_code "http://localhost:${ES_HTTP_PORT:-9200}/_cluster/health") != 200 ]] && echo "[es][http] health not 200" >> "$ERRORS"
kbcode=$(http_code "http://localhost:${KIBANA_PORT:-5601}/api/status"); [[ "$kbcode" != 200 ]] && echo "[kibana][http] /api/status=$kbcode" >> "$ERRORS"
[[ $(http_code "http://localhost:${MASTER_PORT:-32300}/readyz") != 200 ]] && echo "[master][http] /readyz not 200" >> "$ERRORS"
[[ $(http_code "http://localhost:${PROMETHEUS_PORT:-9090}/-/ready") != 200 ]] && echo "[prometheus][http] /-/ready not 200" >> "$ERRORS"
gfcode=$(http_code "http://localhost:${GRAFANA_PORT:-3000}/api/health"); [[ "$gfcode" != 200 ]] && echo "[grafana][http] /api/health=$gfcode" >> "$ERRORS"
[[ $(http_code "http://localhost:${ALERTMANAGER_PORT:-9093}/api/v2/status") != 200 ]] && echo "[alertmanager][http] /api/v2/status not 200" >> "$ERRORS"
[[ -z "$cors8084" ]] && echo "[web-proxy][cors] 8084 missing Access-Control-Allow-Origin" >> "$ERRORS"
[[ -z "$cors8085" ]] && echo "[web-proxy][cors] 8085 missing Access-Control-Allow-Origin" >> "$ERRORS"
sort -u -o "$ERRORS" "$ERRORS"
echo "Diagnostic details -> $DETAILS"
echo "Detected errors -> $ERRORS"
if [[ "$LOG_DIR" == "$ROOT/logs" ]]; then
ln -sfn "$(basename "$DETAILS")" "$ROOT/logs/diagnose_details.log" 2>/dev/null || cp "$DETAILS" "$ROOT/logs/diagnose_details.log" 2>/dev/null || true
ln -sfn "$(basename "$ERRORS")" "$ROOT/logs/diagnose_error.log" 2>/dev/null || cp "$ERRORS" "$ROOT/logs/diagnose_error.log" 2>/dev/null || true
fi
exit 0

11
deployment_new/templates/server/scripts/es-watermark-relax.sh Normal file
View File

@@ -0,0 +1,11 @@
#!/usr/bin/env bash
set -euo pipefail
HOST="${1:-http://127.0.0.1:9200}"
echo "设置 ES watermark 为 95%/96%/97%: $HOST"
curl -fsS -XPUT "$HOST/_cluster/settings" -H 'Content-Type: application/json' -d '{
"transient": {
"cluster.routing.allocation.disk.watermark.low": "95%",
"cluster.routing.allocation.disk.watermark.high": "96%",
"cluster.routing.allocation.disk.watermark.flood_stage": "97%"
}
}' && echo "\nOK"

11
deployment_new/templates/server/scripts/es-watermark-restore.sh Normal file
View File

@@ -0,0 +1,11 @@
#!/usr/bin/env bash
set -euo pipefail
HOST="${1:-http://127.0.0.1:9200}"
echo "恢复 ES watermark 为默认值: $HOST"
curl -fsS -XPUT "$HOST/_cluster/settings" -H 'Content-Type: application/json' -d '{
"transient": {
"cluster.routing.allocation.disk.watermark.low": null,
"cluster.routing.allocation.disk.watermark.high": null,
"cluster.routing.allocation.disk.watermark.flood_stage": null
}
}' && echo "\nOK"

137
deployment_new/templates/server/scripts/install.sh Normal file
View File

@@ -0,0 +1,137 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
info(){ echo -e "\033[34m[INSTALL]\033[0m $*"; }
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
require(){ local ok=1; for c in "$@"; do command -v "$c" >/dev/null 2>&1 || { err "缺少依赖: $c"; ok=0; }; done; [[ $ok -eq 1 ]]; }
# Compose 检测：优先 docker compose（v2），回退 docker-compose（v1）
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/devnull 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose，请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require docker curl jq awk sed tar gzip
require_compose
[[ -f "$ENV_FILE" ]] || { err "缺少 compose/.env，请先运行 scripts/config.sh"; exit 1; }
info "使用环境文件: $ENV_FILE"
set -a; source "$ENV_FILE"; set +a
# 兼容：若 .env 未包含 SWARM_MANAGER_ADDR，则从已存在的 cluster-info.env 读取以避免写空
SMADDR="${SWARM_MANAGER_ADDR:-}"
CI_FILE="$PKG_ROOT/cluster-info.env"
if [[ -z "$SMADDR" && -f "$CI_FILE" ]]; then
SMADDR=$(sed -n 's/^SWARM_MANAGER_ADDR=\(.*\)$/\1/p' "$CI_FILE" | head -n1)
fi
SWARM_MANAGER_ADDR="$SMADDR"
# Swarm init & overlay
if ! docker info 2>/dev/null | grep -q "Swarm: active"; then
[[ -n "${SWARM_MANAGER_ADDR:-}" ]] || { err "SWARM_MANAGER_ADDR 未设置，请在 scripts/config.sh 中配置"; exit 1; }
info "初始化 Swarm (--advertise-addr $SWARM_MANAGER_ADDR)"
docker swarm init --advertise-addr "$SWARM_MANAGER_ADDR" >/dev/null 2>&1 || true
else
info "Swarm 已激活"
fi
NET_NAME="${ARGUS_OVERLAY_NET:-argus-sys-net}"
if ! docker network inspect "$NET_NAME" >/dev/null 2>&1; then
info "创建 overlay 网络: $NET_NAME"
docker network create -d overlay --attachable "$NET_NAME" >/dev/null
else
info "overlay 网络已存在: $NET_NAME"
fi
# Load images
IMAGES_DIR="$PKG_ROOT/images"
shopt -s nullglob
tars=("$IMAGES_DIR"/*.tar.gz)
if [[ ${#tars[@]} -eq 0 ]]; then err "images 目录为空，缺少镜像 tar.gz"; exit 1; fi
total=${#tars[@]}; idx=0
for tgz in "${tars[@]}"; do
idx=$((idx+1))
info "导入镜像 ($idx/$total): $(basename "$tgz")"
tmp=$(mktemp); gunzip -c "$tgz" > "$tmp"; docker load -i "$tmp" >/dev/null; rm -f "$tmp"
done
shopt -u nullglob
# Compose up
PROJECT="${COMPOSE_PROJECT_NAME:-argus-server}"
info "启动服务栈 (docker compose -p $PROJECT up -d)"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" up -d
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" ps
# Wait readiness (best-effort)
code(){ curl -4 -s -o /dev/null -w "%{http_code}" "$1" || echo 000; }
prom_ok(){ (exec 3<>/dev/tcp/127.0.0.1/${PROMETHEUS_PORT:-9090}) >/dev/null 2>&1 && return 0 || return 1; }
kb_ok(){ local body; body=$(curl -s "http://127.0.0.1:${KIBANA_PORT:-5601}/api/status" || true); echo "$body" | grep -q '"level"\s*:\s*"available"'; }
RETRIES=${RETRIES:-60}; SLEEP=${SLEEP:-5}; ok=0
info "等待基础服务就绪 (<= $((RETRIES*SLEEP))s)"
for i in $(seq 1 "$RETRIES"); do
e1=$(code "http://127.0.0.1:${MASTER_PORT:-32300}/readyz")
e2=$(code "http://127.0.0.1:${ES_HTTP_PORT:-9200}/_cluster/health")
e3=000; prom_ok && e3=200
e4=$(code "http://127.0.0.1:${GRAFANA_PORT:-3000}/api/health")
e5=$(code "http://127.0.0.1:${ALERTMANAGER_PORT:-9093}/api/v2/status")
e6=$(kb_ok && echo 200 || echo 000)
info "[ready] t=$((i*SLEEP))s master=$e1 es=$e2 prom=$e3 graf=$e4 alert=$e5 kibana=$e6"
[[ "$e1" == 200 ]] && ok=$((ok+1))
[[ "$e2" == 200 ]] && ok=$((ok+1))
[[ "$e3" == 200 ]] && ok=$((ok+1))
[[ "$e4" == 200 ]] && ok=$((ok+1))
[[ "$e5" == 200 ]] && ok=$((ok+1))
[[ "$e6" == 200 ]] && ok=$((ok+1))
if [[ $ok -ge 6 ]]; then break; fi; ok=0; sleep "$SLEEP"
done
[[ $ok -ge 6 ]] || err "部分服务未就绪（可稍后重试 selfcheck）"
# Swarm join tokens
TOKEN_WORKER=$(docker swarm join-token -q worker 2>/dev/null || echo "")
TOKEN_MANAGER=$(docker swarm join-token -q manager 2>/dev/null || echo "")
# cluster-info.env（compose 场景下不再依赖 BINDIP/FTPIP）
CI="$PKG_ROOT/cluster-info.env"
info "写入 cluster-info.env (manager/token)"
{
echo "SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR:-}"
echo "SWARM_JOIN_TOKEN_WORKER=${TOKEN_WORKER:-}"
echo "SWARM_JOIN_TOKEN_MANAGER=${TOKEN_MANAGER:-}"
} > "$CI"
info "已输出 $CI"
# 安装报告
ts=$(date +%Y%m%d-%H%M%S)
RPT="$PKG_ROOT/安装报告_${ts}.md"
{
echo "# Argus Server 安装报告 (${ts})"
echo
echo "## 端口映射"
echo "- MASTER_PORT=${MASTER_PORT}"
echo "- ES_HTTP_PORT=${ES_HTTP_PORT}"
echo "- KIBANA_PORT=${KIBANA_PORT}"
echo "- PROMETHEUS_PORT=${PROMETHEUS_PORT}"
echo "- GRAFANA_PORT=${GRAFANA_PORT}"
echo "- ALERTMANAGER_PORT=${ALERTMANAGER_PORT}"
echo "- WEB_PROXY_PORT_8080=${WEB_PROXY_PORT_8080} ... 8085=${WEB_PROXY_PORT_8085}"
echo
echo "## Swarm/Overlay"
echo "- SWARM_MANAGER_ADDR=${SWARM_MANAGER_ADDR:-}"
echo "- NET=${NET_NAME}"
echo "- JOIN_TOKEN_WORKER=${TOKEN_WORKER:-}"
echo "- JOIN_TOKEN_MANAGER=${TOKEN_MANAGER:-}"
echo
echo "## 健康检查（简要）"
echo "- master/readyz=$(code http://127.0.0.1:${MASTER_PORT:-32300}/readyz)"
echo "- es/_cluster/health=$(code http://127.0.0.1:${ES_HTTP_PORT:-9200}/_cluster/health)"
echo "- grafana/api/health=$(code http://127.0.0.1:${GRAFANA_PORT:-3000}/api/health)"
echo "- prometheus/tcp=$([[ $(prom_ok; echo $?) == 0 ]] && echo 200 || echo 000)"
echo "- alertmanager/api/v2/status=$(code http://127.0.0.1:${ALERTMANAGER_PORT:-9093}/api/v2/status)"
echo "- kibana/api/status=$([[ $(kb_ok; echo $?) == 0 ]] && echo available || echo not-ready)"
} > "$RPT"
info "已生成报告: $RPT"
info "安装完成。可将 cluster-info.env 分发给 Client-GPU 安装方。"
docker exec argus-web-proxy nginx -t >/dev/null 2>&1 && docker exec argus-web-proxy nginx -s reload >/dev/null 2>&1 || true

83
deployment_new/templates/server/scripts/selfcheck.sh Normal file
View File

@@ -0,0 +1,83 @@
#!/usr/bin/env bash
set -euo pipefail
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd)"
log() { echo -e "\033[0;34m[CHECK]\033[0m $*"; }
err() { echo -e "\033[0;31m[ERROR]\033[0m $*" >&2; }
ENV_FILE="$ROOT/compose/.env"; [[ -f "$ENV_FILE" ]] && set -a && source "$ENV_FILE" && set +a
wait_http() { local url="$1"; local attempts=${2:-120}; local i=1; while ((i<=attempts)); do curl -fsS "$url" >/dev/null 2>&1 && return 0; echo "[..] waiting $url ($i/$attempts)"; sleep 5; ((i++)); done; return 1; }
code_for() { curl -s -o /dev/null -w "%{http_code}" "$1" || echo 000; }
header_val() { curl -s -D - -o /dev/null "$@" | awk -F': ' 'BEGIN{IGNORECASE=1}$1=="Access-Control-Allow-Origin"{gsub("\r","",$2);print $2}'; }
LOG_DIR="$ROOT/logs"; mkdir -p "$LOG_DIR" || true
OUT_JSON="$LOG_DIR/selfcheck.json"; tmp=$(mktemp)
ok=1
log "checking overlay network"
net_ok=false
if docker network inspect "${ARGUS_OVERLAY_NET:-argus-sys-net}" >/dev/null 2>&1; then
if docker network inspect "${ARGUS_OVERLAY_NET:-argus-sys-net}" | grep -q '"Driver": "overlay"'; then net_ok=true; fi
fi
[[ "$net_ok" == true ]] || ok=0
log "checking Elasticsearch"
wait_http "http://localhost:${ES_HTTP_PORT:-9200}/_cluster/health" 60 || ok=0
log "checking Kibana"
kb_code=$(code_for "http://localhost:${KIBANA_PORT:-5601}/api/status")
kb_ok=false
if [[ "$kb_code" == 200 ]]; then
body=$(curl -sS "http://localhost:${KIBANA_PORT:-5601}/api/status" || true)
echo "$body" | grep -q '"level"\s*:\s*"available"' && kb_ok=true
fi
[[ "$kb_ok" == true ]] || ok=0
log "checking Master"
[[ $(code_for "http://localhost:${MASTER_PORT:-32300}/readyz") == 200 ]] || ok=0
log "checking Prometheus"
wait_http "http://localhost:${PROMETHEUS_PORT:-9090}/-/ready" 60 || ok=0
log "checking Grafana"
gf_code=$(code_for "http://localhost:${GRAFANA_PORT:-3000}/api/health")
gf_ok=false; if [[ "$gf_code" == 200 ]]; then body=$(curl -sS "http://localhost:${GRAFANA_PORT:-3000}/api/health" || true); echo "$body" | grep -q '"database"\s*:\s*"ok"' && gf_ok=true; fi
[[ "$gf_ok" == true ]] || ok=0
log "checking Alertmanager"
wait_http "http://localhost:${ALERTMANAGER_PORT:-9093}/api/v2/status" 60 || ok=0
log "checking Web-Proxy (CORS)"
cors8084=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8084:-8084}/api/v2/status" || true)
cors8085=$(header_val -H "Origin: http://localhost:${WEB_PROXY_PORT_8080:-8080}" "http://localhost:${WEB_PROXY_PORT_8085:-8085}/api/v1/master/nodes" || true)
wp_ok=true
[[ -n "$cors8084" && -n "$cors8085" ]] || wp_ok=false
[[ "$wp_ok" == true ]] || ok=0
cat > "$tmp" <<JSON
{
"overlay_net": $net_ok,
"es": true,
"kibana": $kb_ok,
"master_readyz": true,
"prometheus": true,
"grafana": $gf_ok,
"alertmanager": true,
"web_proxy_cors": $wp_ok,
"timestamp": "$(date -u +%Y-%m-%dT%H:%M:%SZ)"
}
JSON
mv "$tmp" "$OUT_JSON" 2>/dev/null || cp "$tmp" "$OUT_JSON"
if [[ "$ok" == 1 ]]; then
log "selfcheck OK -> $OUT_JSON"
exit 0
else
err "selfcheck FAILED -> $OUT_JSON"
exit 1
fi

9
deployment_new/templates/server/scripts/status.sh Normal file
View File

@@ -0,0 +1,9 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
if [[ -f "$ENV_FILE" ]]; then set -a; source "$ENV_FILE"; set +a; fi
PROJECT="${COMPOSE_PROJECT_NAME:-argus-server}"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" ps

23
deployment_new/templates/server/scripts/uninstall.sh Normal file
View File

@@ -0,0 +1,23 @@
#!/usr/bin/env bash
set -euo pipefail
ROOT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
PKG_ROOT="$ROOT_DIR"
ENV_FILE="$PKG_ROOT/compose/.env"
COMPOSE_FILE="$PKG_ROOT/compose/docker-compose.yml"
# load COMPOSE_PROJECT_NAME from env file if present
if [[ -f "$ENV_FILE" ]]; then set -a; source "$ENV_FILE"; set +a; fi
PROJECT="${COMPOSE_PROJECT_NAME:-argus-server}"
err(){ echo -e "\033[31m[ERROR]\033[0m $*" >&2; }
# Compose 检测：优先 docker compose（v2），回退 docker-compose（v1）
require_compose(){
if docker compose version >/dev/null 2>&1; then return 0; fi
if command -v docker-compose >/dev/null 2>&1 && docker-compose version >/dev/null 2>&1; then return 0; fi
err "未检测到 Docker Compose，请安装 docker compose v2 或 docker-compose v1"; exit 1
}
require_compose
echo "[UNINSTALL] stopping compose (project=$PROJECT)"
docker compose -p "$PROJECT" --env-file "$ENV_FILE" -f "$COMPOSE_FILE" down --remove-orphans || true
echo "[UNINSTALL] done"