mvp v1.1 save

src/mvp/v1.1/README.md

@@ -0,0 +1,61 @@
+# MVP v1.1（GRPO + SFT on Ray）运行说明
+
+本目录是一套**独立可运行**的 v1.1 交付：使用 1 个 Ray head（不跑训练）+ 2 个 Ray worker（各 4 GPU）在同一宿主机通过 `docker exec` 协调容器，并通过 **head 上的 `ray job submit`** 提交作业，同时强制 driver 落到 worker。
+
+> 远程 dev 环境推荐目录布局：
+>
+> - `/home2/argus/infra/mvp/`
+>   - `shared/`（持久化：datasets/hf/jobs/...）
+>   - `verl/`（代码仓库，用于 prepare / snapshot）
+>   - `v1.1/`（本目录内容：compose + scripts）
+
+---
+
+## 快速开始（远程机 argus@h1）
+
+在 `/home2/argus/infra/mvp/v1.1/` 下执行：
+
+```bash
+./scripts/00_prereq_check.sh
+./scripts/01_up.sh
+./scripts/20_start_head.sh
+./scripts/21_start_workers.sh
+./scripts/30_prepare_data_and_model.sh
+./scripts/12_install_py_deps.sh
+./scripts/44_submit_sdk.sh /workspace/mvp/v1.1/py/configs/dev.yaml /workspace/mvp/v1.1/py/jobspecs/ppo.yaml
+./scripts/44_submit_sdk.sh /workspace/mvp/v1.1/py/configs/dev.yaml /workspace/mvp/v1.1/py/jobspecs/grpo.yaml
+./scripts/44_submit_sdk.sh /workspace/mvp/v1.1/py/configs/dev.yaml /workspace/mvp/v1.1/py/jobspecs/sft.yaml
+./scripts/40_submit_ppo_epoch1.sh
+./scripts/41_submit_grpo_epoch1.sh
+./scripts/42_submit_sft_minimal.sh
+./scripts/50_status.sh
+```
+
+说明：
+
+- `scripts/40/41/42` 是历史的 “CLI 提交脚本”（仍可用），但 v1.1 的工程化目标是把提交机制迁移到 `scripts/44_submit_sdk.sh`（Ray Python SDK + YAML 配置）。
+
+停止并清理：
+
+```bash
+./scripts/02_down.sh
+```
+
+---
+
+## 关键约束（必须满足）
+
+- **必须通过 head 执行 `ray job submit`** 提交任务（满足“从 head 提交”要求）。
+- **head 不跑训练**：head 以 `--num-cpus=0 --num-gpus=0` 启动；worker 具备自定义资源 `worker_node`；提交时 `--entrypoint-resources='{"worker_node": 1}'` 强制 driver 落 worker。
+- **共享路径统一为 `/private`（容器内）**：compose 将宿主机 `../shared` 挂载到容器内 `/private`，对齐生产环境。
+- **多版本 verl**：通过 Ray Job `runtime_env.env_vars.PYTHONPATH` 注入 `${SHARED_ROOT}/common/code/verl/...`，job 粒度选择代码快照。
+
+---
+
+## 共享目录（容器内 /private）
+
+- `/private/datasets/`：数据（PPO 的 gsm8k RL parquet、SFT parquet）
+- `/private/hf/`：HF 缓存（模型持久化，避免重复下载）
+- `/private/jobs/<submission_id>/`：每个 Ray Job 的输出目录（logs/config/debug/checkpoints）
+- `/private/common/`：共享区（模型/数据/代码快照）
+- `/private/user/`：用户自定义代码（例如 reward_fn）

src/mvp/v1.1/docker-compose.yaml

@@ -0,0 +1,89 @@
+version: "3.8"
+
+services:
+  ray_head:
+    image: verlai/verl:sgl055.latest
+    container_name: mvp11-ray-head
+    command: sleep infinity
+    ports:
+      - "8265:8265"
+    volumes:
+      - ../verl:/workspace/verl
+      - ../shared:/private
+      - .:/workspace/mvp/v1.1
+    shm_size: "10g"
+    ulimits:
+      nofile:
+        soft: 65536
+        hard: 65536
+    cap_add:
+      - SYS_ADMIN
+      - SYS_PTRACE
+    networks:
+      - mvp11-ray-net
+    environment:
+      HF_HOME: "/private/hf"
+      HUGGINGFACE_HUB_CACHE: "/private/hf/hub"
+      TRANSFORMERS_CACHE: "/private/hf/transformers"
+      HF_ENDPOINT: "https://hf-mirror.com"
+      PYTHONUNBUFFERED: "1"
+
+  ray_worker_0:
+    image: verlai/verl:sgl055.latest
+    container_name: mvp11-ray-worker-0
+    command: sleep infinity
+    volumes:
+      - ../verl:/workspace/verl
+      - ../shared:/private
+      - .:/workspace/mvp/v1.1
+    shm_size: "10g"
+    ulimits:
+      nofile:
+        soft: 65536
+        hard: 65536
+    cap_add:
+      - SYS_ADMIN
+      - SYS_PTRACE
+    networks:
+      - mvp11-ray-net
+    runtime: nvidia
+    environment:
+      NVIDIA_VISIBLE_DEVICES: "0,1,2,3"
+      NVIDIA_DRIVER_CAPABILITIES: "all"
+      HF_HOME: "/private/hf"
+      HUGGINGFACE_HUB_CACHE: "/private/hf/hub"
+      TRANSFORMERS_CACHE: "/private/hf/transformers"
+      HF_ENDPOINT: "https://hf-mirror.com"
+      PYTHONUNBUFFERED: "1"
+
+  ray_worker_1:
+    image: verlai/verl:sgl055.latest
+    container_name: mvp11-ray-worker-1
+    command: sleep infinity
+    volumes:
+      - ../verl:/workspace/verl
+      - ../shared:/private
+      - .:/workspace/mvp/v1.1
+    shm_size: "10g"
+    ulimits:
+      nofile:
+        soft: 65536
+        hard: 65536
+    cap_add:
+      - SYS_ADMIN
+      - SYS_PTRACE
+    networks:
+      - mvp11-ray-net
+    runtime: nvidia
+    environment:
+      NVIDIA_VISIBLE_DEVICES: "4,5,6,7"
+      NVIDIA_DRIVER_CAPABILITIES: "all"
+      HF_HOME: "/private/hf"
+      HUGGINGFACE_HUB_CACHE: "/private/hf/hub"
+      TRANSFORMERS_CACHE: "/private/hf/transformers"
+      HF_ENDPOINT: "https://hf-mirror.com"
+      PYTHONUNBUFFERED: "1"
+
+networks:
+  mvp11-ray-net:
+    driver: bridge

src/mvp/v1.1/job_spec.schema.json

@@ -0,0 +1,33 @@
+{
+  "$schema": "https://json-schema.org/draft/2020-12/schema",
+  "title": "mvp-v1.1-job-spec",
+  "type": "object",
+  "required": ["workload", "shared_root", "code_path", "model_id", "ray", "runtime_env"],
+  "properties": {
+    "submission_id": { "type": "string" },
+    "workload": { "type": "string", "enum": ["ppo", "grpo", "sft"] },
+    "shared_root": { "type": "string" },
+    "code_path": { "type": "string" },
+    "model_id": { "type": "string" },
+    "ppo": { "type": "object" },
+    "grpo": { "type": "object" },
+    "sft": { "type": "object" },
+    "ray": {
+      "type": "object",
+      "required": ["address", "entrypoint_num_cpus", "entrypoint_resources"],
+      "properties": {
+        "address": { "type": "string" },
+        "entrypoint_num_cpus": { "type": "number" },
+        "entrypoint_resources": { "type": "object" }
+      }
+    },
+    "runtime_env": {
+      "type": "object",
+      "required": ["env_vars"],
+      "properties": {
+        "env_vars": { "type": "object" }
+      }
+    }
+  }
+}
+

src/mvp/v1.1/py/configs/dev.yaml

@@ -0,0 +1,20 @@
+# Ray 基础配置（dev 环境 / head 容器内视角）
+address: "http://127.0.0.1:8265"
+
+# 容器内共享根路径（对齐生产 /private）
+shared_root: "/private"
+
+# 强制 driver 落 worker（head 不跑训练）
+entrypoint_num_cpus: 1
+entrypoint_resources:
+  worker_node: 1
+
+# 运行时环境变量（所有 job 通用）
+runtime_env:
+  env_vars:
+    HF_ENDPOINT: "https://hf-mirror.com"
+    PYTHONUNBUFFERED: "1"
+
+# 用户自定义代码目录（可被 PYTHONPATH 注入）
+user_code_path: "/private/user/code"
+

src/mvp/v1.1/py/jobspecs/grpo.yaml

@@ -0,0 +1,20 @@
+workload: "grpo"
+
+submission_id: ""
+
+code_path: "/private/common/code/verl/verl_repo"
+
+model_id: "Qwen/Qwen2.5-0.5B-Instruct"
+
+train_file: "/private/datasets/gsm8k/train.parquet"
+val_file: "/private/datasets/gsm8k/test.parquet"
+
+nnodes: 2
+n_gpus_per_node: 4
+
+total_epochs: 1
+total_training_steps: 10
+
+save_freq: 10
+test_freq: -1
+

src/mvp/v1.1/py/jobspecs/ppo.yaml

@@ -0,0 +1,22 @@
+workload: "ppo"
+
+# 可选：不填则 submitter 自动生成
+submission_id: ""
+
+# 多版本：指向 code snapshot（由 scripts/31_snapshot_verl_code.sh 生成）
+code_path: "/private/common/code/verl/verl_repo"
+
+model_id: "Qwen/Qwen2.5-0.5B-Instruct"
+
+train_file: "/private/datasets/gsm8k/train.parquet"
+val_file: "/private/datasets/gsm8k/test.parquet"
+
+nnodes: 2
+n_gpus_per_node: 4
+
+total_epochs: 1
+total_training_steps: 10
+
+save_freq: 10
+test_freq: -1
+

src/mvp/v1.1/py/jobspecs/sft.yaml

@@ -0,0 +1,22 @@
+workload: "sft"
+
+submission_id: ""
+
+code_path: "/private/common/code/verl/verl_repo"
+
+model_id: "Qwen/Qwen2.5-0.5B-Instruct"
+
+train_file: "/private/datasets/gsm8k_sft/train.parquet"
+val_file: null
+
+nnodes: 2
+n_gpus_per_node: 4
+
+total_epochs: 1
+total_training_steps: 10
+
+save_freq: 10
+
+# SFT driver 默认不分配 GPU（ray job entrypoint 不指定 entrypoint_num_gpus），因此 driver 侧不要依赖 CUDA
+trainer_device: "cpu"
+

src/mvp/v1.1/py/mvp_v11/__init__.py

@@ -0,0 +1 @@
+

src/mvp/v1.1/py/mvp_v11/builders.py

@@ -0,0 +1,96 @@
+from __future__ import annotations
+
+from dataclasses import dataclass
+
+from .models import JobSpec
+
+
+@dataclass(frozen=True)
+class BuiltCommand:
+    argv: list[str]
+
+
+def build_training_argv(spec: JobSpec, submission_id: str, job_dir: str) -> BuiltCommand:
+    """
+    Returns argv for the actual training process (Hydra overrides preserved).
+    This argv is executed by a lightweight Python driver entrypoint.
+    """
+    if spec.workload in ("ppo", "grpo"):
+        algo_overrides: list[str] = []
+        if spec.workload == "grpo":
+            algo_overrides.append("algorithm.adv_estimator=grpo")
+
+        test_freq = spec.test_freq if spec.test_freq is not None else -1
+        val_file = spec.val_file if spec.val_file is not None else "null"
+
+        argv = [
+            "python3",
+            "-m",
+            "verl.trainer.main_ppo",
+            f"data.train_files={spec.train_file}",
+            f"data.val_files={val_file}",
+            "data.train_batch_size=256",
+            "data.max_prompt_length=512",
+            "data.max_response_length=512",
+            f"actor_rollout_ref.model.path={spec.model_id}",
+            "actor_rollout_ref.actor.optim.lr=1e-6",
+            "actor_rollout_ref.actor.ppo_mini_batch_size=64",
+            "actor_rollout_ref.actor.ppo_micro_batch_size_per_gpu=4",
+            "actor_rollout_ref.rollout.name=sglang",
+            "actor_rollout_ref.rollout.log_prob_micro_batch_size_per_gpu=8",
+            "actor_rollout_ref.rollout.tensor_model_parallel_size=1",
+            "actor_rollout_ref.rollout.gpu_memory_utilization=0.4",
+            "actor_rollout_ref.ref.log_prob_micro_batch_size_per_gpu=4",
+            "critic.optim.lr=1e-5",
+            f"critic.model.path={spec.model_id}",
+            "critic.ppo_micro_batch_size_per_gpu=4",
+            "algorithm.kl_ctrl.kl_coef=0.001",
+            *algo_overrides,
+            "trainer.logger=console",
+            "trainer.val_before_train=False",
+            f"trainer.n_gpus_per_node={spec.n_gpus_per_node}",
+            f"trainer.nnodes={spec.nnodes}",
+            f"trainer.save_freq={spec.save_freq}",
+            f"trainer.test_freq={test_freq}",
+            f"trainer.total_epochs={spec.total_epochs}",
+            f"trainer.total_training_steps={spec.total_training_steps}",
+            "trainer.resume_mode=disable",
+            f"trainer.default_local_dir={job_dir}/checkpoints",
+            "+ray_kwargs.ray_init.address=auto",
+            f"hydra.run.dir={job_dir}/logs/hydra",
+        ]
+        return BuiltCommand(argv=argv)
+
+    if spec.workload == "sft":
+        val_override = "null" if spec.val_file is None else spec.val_file
+        trainer_device = spec.trainer_device or "cpu"
+
+        argv = [
+            "python3",
+            "-m",
+            "verl.trainer.sft_trainer_ray",
+            f"model.path={spec.model_id}",
+            f"data.train_files={spec.train_file}",
+            f"data.val_files={val_override}",
+            "data.train_batch_size=64",
+            "data.micro_batch_size_per_gpu=1",
+            "data.max_token_len_per_gpu=2048",
+            "data.max_length=1024",
+            "trainer.logger=console",
+            "trainer.project_name=mvp11-sft",
+            f"trainer.experiment_name={submission_id}",
+            f"trainer.total_epochs={spec.total_epochs}",
+            f"trainer.total_training_steps={spec.total_training_steps}",
+            f"trainer.save_freq={spec.save_freq}",
+            "trainer.test_freq=-1",
+            "trainer.resume_mode=disable",
+            f"trainer.device={trainer_device}",
+            f"trainer.default_local_dir={job_dir}/checkpoints",
+            f"trainer.nnodes={spec.nnodes}",
+            f"trainer.n_gpus_per_node={spec.n_gpus_per_node}",
+            f"hydra.run.dir={job_dir}/logs/hydra",
+        ]
+        return BuiltCommand(argv=argv)
+
+    raise ValueError(f"unsupported workload: {spec.workload}")
+

src/mvp/v1.1/py/mvp_v11/driver_entrypoint.py

@@ -0,0 +1,63 @@
+from __future__ import annotations
+
+import argparse
+import os
+import shlex
+import subprocess
+import sys
+from pathlib import Path
+
+
+def _preflight() -> None:
+    print("MVP_PRECHECK_PYTHON:", sys.executable, flush=True)
+    print("MVP_PRECHECK_PYTHONPATH:", os.environ.get("PYTHONPATH"), flush=True)
+    print("MVP_PRECHECK_MVP_CODE_PATH:", os.environ.get("MVP_CODE_PATH"), flush=True)
+    try:
+        import verl  # type: ignore
+
+        print("MVP_PRECHECK_VERL_FILE:", getattr(verl, "__file__", None), flush=True)
+    except Exception as e:
+        print("MVP_PRECHECK_VERL_IMPORT_ERROR:", repr(e), flush=True)
+
+    try:
+        import mvp_marker  # type: ignore
+
+        print("MVP_PRECHECK_MARKER:", getattr(mvp_marker, "MARKER", None), flush=True)
+    except Exception as e:
+        print("MVP_PRECHECK_MARKER_MISSING:", repr(e), flush=True)
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--job-dir", required=True)
+    parser.add_argument("cmd", nargs=argparse.REMAINDER)
+    args = parser.parse_args()
+
+    job_dir = Path(args.job_dir)
+    job_dir.mkdir(parents=True, exist_ok=True)
+
+    _preflight()
+
+    if not args.cmd:
+        print("no command provided", file=sys.stderr)
+        return 2
+
+    # argparse includes the leading "--" if the caller uses it; strip it.
+    cmd = list(args.cmd)
+    if cmd and cmd[0] == "--":
+        cmd = cmd[1:]
+    if not cmd:
+        print("no command provided", file=sys.stderr)
+        return 2
+
+    # Execute training command as a subprocess so that logs are captured by Ray job logs.
+    cmd_str = " ".join(shlex.quote(x) for x in cmd)
+    print("MVP_DRIVER_EXEC:", cmd_str, flush=True)
+
+    proc = subprocess.run(cmd, check=False)
+    print("MVP_DRIVER_EXIT_CODE:", proc.returncode, flush=True)
+    return proc.returncode
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

src/mvp/v1.1/py/mvp_v11/models.py

@@ -0,0 +1,121 @@
+from __future__ import annotations
+
+from dataclasses import dataclass
+from typing import Any
+
+
+def _require(d: dict[str, Any], key: str) -> Any:
+    if key not in d or d[key] in (None, ""):
+        raise ValueError(f"missing required field: {key}")
+    return d[key]
+
+
+@dataclass(frozen=True)
+class RayConfig:
+    address: str
+    shared_root: str
+    entrypoint_num_cpus: float
+    entrypoint_resources: dict[str, float]
+    runtime_env_env_vars: dict[str, str]
+    user_code_path: str
+
+    @staticmethod
+    def from_dict(d: dict[str, Any]) -> "RayConfig":
+        runtime_env = d.get("runtime_env") or {}
+        env_vars = (runtime_env.get("env_vars") or {}) if isinstance(runtime_env, dict) else {}
+        if not isinstance(env_vars, dict):
+            raise ValueError("runtime_env.env_vars must be a mapping")
+
+        entrypoint_resources = d.get("entrypoint_resources") or {}
+        if not isinstance(entrypoint_resources, dict):
+            raise ValueError("entrypoint_resources must be a mapping")
+
+        return RayConfig(
+            address=str(_require(d, "address")),
+            shared_root=str(_require(d, "shared_root")),
+            entrypoint_num_cpus=float(d.get("entrypoint_num_cpus", 1)),
+            entrypoint_resources={str(k): float(v) for k, v in entrypoint_resources.items()},
+            runtime_env_env_vars={str(k): str(v) for k, v in env_vars.items()},
+            user_code_path=str(d.get("user_code_path", f"{_require(d, 'shared_root')}/user/code")),
+        )
+
+    def to_public_dict(self) -> dict[str, Any]:
+        return {
+            "address": self.address,
+            "shared_root": self.shared_root,
+            "entrypoint_num_cpus": self.entrypoint_num_cpus,
+            "entrypoint_resources": self.entrypoint_resources,
+            "runtime_env": {"env_vars": self.runtime_env_env_vars},
+            "user_code_path": self.user_code_path,
+        }
+
+
+@dataclass(frozen=True)
+class JobSpec:
+    workload: str  # ppo|grpo|sft
+    submission_id: str | None
+    code_path: str
+    model_id: str
+
+    train_file: str
+    val_file: str | None
+
+    nnodes: int
+    n_gpus_per_node: int
+
+    total_epochs: int
+    total_training_steps: int
+
+    save_freq: int
+    test_freq: int | None
+
+    trainer_device: str | None  # only for sft (driver-side device)
+
+    @staticmethod
+    def from_dict(d: dict[str, Any]) -> "JobSpec":
+        workload = str(_require(d, "workload"))
+        if workload not in ("ppo", "grpo", "sft"):
+            raise ValueError(f"unsupported workload: {workload}")
+
+        val_file = d.get("val_file", None)
+        if val_file in ("", "null"):
+            val_file = None
+
+        test_freq = d.get("test_freq", None)
+        if test_freq in ("", "null"):
+            test_freq = None
+
+        return JobSpec(
+            workload=workload,
+            submission_id=(str(d["submission_id"]) if d.get("submission_id") else None),
+            code_path=str(_require(d, "code_path")),
+            model_id=str(_require(d, "model_id")),
+            train_file=str(_require(d, "train_file")),
+            val_file=(str(val_file) if val_file is not None else None),
+            nnodes=int(d.get("nnodes", 2)),
+            n_gpus_per_node=int(d.get("n_gpus_per_node", 4)),
+            total_epochs=int(d.get("total_epochs", 1)),
+            total_training_steps=int(d.get("total_training_steps", 10)),
+            save_freq=int(d.get("save_freq", 10)),
+            test_freq=(int(test_freq) if test_freq is not None else None),
+            trainer_device=(str(d.get("trainer_device")) if d.get("trainer_device") else None),
+        )
+
+    def to_public_dict(self) -> dict[str, Any]:
+        out: dict[str, Any] = {
+            "workload": self.workload,
+            "submission_id": self.submission_id or "",
+            "code_path": self.code_path,
+            "model_id": self.model_id,
+            "train_file": self.train_file,
+            "val_file": self.val_file,
+            "nnodes": self.nnodes,
+            "n_gpus_per_node": self.n_gpus_per_node,
+            "total_epochs": self.total_epochs,
+            "total_training_steps": self.total_training_steps,
+            "save_freq": self.save_freq,
+            "test_freq": self.test_freq,
+        }
+        if self.workload == "sft":
+            out["trainer_device"] = self.trainer_device or "cpu"
+        return out

src/mvp/v1.1/py/mvp_v11/ray_job_tool.py

@@ -0,0 +1,171 @@
+from __future__ import annotations
+
+import json
+import os
+import shlex
+from datetime import datetime
+from pathlib import Path
+from typing import Any
+
+import ray
+from ray.job_submission import JobSubmissionClient
+
+from .builders import build_training_argv
+from .models import JobSpec, RayConfig
+from .yaml_io import dump_yaml
+
+
+def _ts() -> str:
+    return datetime.now().strftime("%Y%m%d_%H%M%S")
+
+
+def _mkdir(p: Path) -> None:
+    p.mkdir(parents=True, exist_ok=True)
+
+
+def _write_text(p: Path, content: str) -> None:
+    _mkdir(p.parent)
+    p.write_text(content, encoding="utf-8")
+
+
+def _write_json(p: Path, obj: Any) -> None:
+    _write_text(p, json.dumps(obj, indent=2, ensure_ascii=False) + "\n")
+
+
+def _safe_basename(path: str) -> str:
+    return path.rstrip("/").split("/")[-1]
+
+
+class RayJobTool:
+    def __init__(self, cfg: RayConfig):
+        self.cfg = cfg
+        self.client = JobSubmissionClient(cfg.address)
+
+    def _job_dir(self, submission_id: str) -> str:
+        return f"{self.cfg.shared_root}/jobs/{submission_id}"
+
+    def _runtime_env(self, spec: JobSpec) -> dict[str, Any]:
+        env_vars = dict(self.cfg.runtime_env_env_vars)
+
+        # Default HF cache
+        env_vars.setdefault("HF_HOME", f"{self.cfg.shared_root}/hf")
+        env_vars.setdefault("HUGGINGFACE_HUB_CACHE", f"{self.cfg.shared_root}/hf/hub")
+        env_vars.setdefault("TRANSFORMERS_CACHE", f"{self.cfg.shared_root}/hf/transformers")
+        env_vars.setdefault("PYTHONUNBUFFERED", "1")
+
+        # Tool code path must be importable on workers (compose mounts v1.1 into all containers).
+        # Place it before verl code to avoid interfering with verl import priority.
+        tool_code_path = os.environ.get("MVP_TOOL_CODE_PATH", "/workspace/mvp/v1.1/py")
+
+        user_code_path = self.cfg.user_code_path
+        code_path = spec.code_path
+
+        existing = env_vars.get("PYTHONPATH", "")
+        prefix = f"{tool_code_path}:{code_path}:{user_code_path}"
+        env_vars["PYTHONPATH"] = f"{prefix}:{existing}" if existing else prefix
+
+        # For debugging / log visibility
+        env_vars["MVP_CODE_PATH"] = code_path
+
+        # SFT: ensure ray.init() connects to the cluster
+        if spec.workload == "sft":
+            env_vars.setdefault("RAY_ADDRESS", "auto")
+
+        return {"env_vars": env_vars}
+
+    def submit(self, spec: JobSpec, no_wait: bool) -> str:
+        submission_id = spec.submission_id or f"mvp11_{spec.workload}_{_ts()}_{os.getpid()}"
+        job_dir = self._job_dir(submission_id)
+
+        built = build_training_argv(spec, submission_id=submission_id, job_dir=job_dir)
+        entrypoint_argv = [
+            "python3",
+            "-m",
+            "mvp_v11.driver_entrypoint",
+            "--job-dir",
+            job_dir,
+            *built.argv,
+        ]
+        entrypoint = " ".join(shlex.quote(x) for x in entrypoint_argv)
+
+        runtime_env = self._runtime_env(spec)
+
+        # Prepare job artifacts directory
+        job_root = Path(job_dir)
+        _mkdir(job_root / "config")
+        _mkdir(job_root / "logs")
+        _mkdir(job_root / "debug")
+        _mkdir(job_root / "checkpoints")
+
+        _write_text(job_root / "config" / "ray_config.yaml", dump_yaml(self.cfg.to_public_dict()))
+        _write_text(job_root / "config" / "jobspec.yaml", dump_yaml(spec.to_public_dict()))
+        _write_json(job_root / "config" / "submit_payload.json", {
+            "submission_id": submission_id,
+            "address": self.cfg.address,
+            "entrypoint": entrypoint,
+            "entrypoint_num_cpus": self.cfg.entrypoint_num_cpus,
+            "entrypoint_resources": self.cfg.entrypoint_resources,
+            "runtime_env": runtime_env,
+        })
+
+        # Pre-submit debug snapshot (ray cluster resources via ray.init)
+        try:
+            ray.init(address="auto", ignore_reinit_error=True, log_to_driver=False)
+            _write_json(job_root / "debug" / "ray_cluster_resources_pre.json", ray.cluster_resources())
+            _write_json(job_root / "debug" / "ray_available_resources_pre.json", ray.available_resources())
+        except Exception as e:
+            _write_text(job_root / "debug" / "ray_resources_pre.error.txt", repr(e) + "\n")
+
+        try:
+            submitted = self.client.submit_job(
+                entrypoint=entrypoint,
+                submission_id=submission_id,
+                runtime_env=runtime_env,
+                entrypoint_num_cpus=self.cfg.entrypoint_num_cpus,
+                entrypoint_resources=self.cfg.entrypoint_resources,
+            )
+        except Exception as e:
+            _write_text(job_root / "logs" / "submit.error.txt", repr(e) + "\n")
+            raise
+
+        _write_text(job_root / "config" / "ray_submission_id.txt", submitted + "\n")
+
+        # Post-submit debug snapshot via SDK
+        try:
+            jobs = self.client.list_jobs()
+            _write_text(
+                job_root / "debug" / "ray_job_list_post.json",
+                json.dumps([_job_details_to_dict(j) for j in jobs], indent=2) + "\n",
+            )
+        except Exception as e:
+            _write_text(job_root / "debug" / "ray_job_list_post.error.txt", repr(e) + "\n")
+
+        if not no_wait:
+            # caller can separately wait; keep submit non-blocking by default in scripts
+            pass
+
+        return submitted
+
+    def status(self, submission_id: str) -> str:
+        return str(self.client.get_job_status(submission_id))
+
+    def stop(self, submission_id: str) -> bool:
+        return bool(self.client.stop_job(submission_id))
+
+    def logs(self, submission_id: str) -> str:
+        return self.client.get_job_logs(submission_id)
+
+    def list(self) -> list[dict[str, Any]]:
+        return [_job_details_to_dict(j) for j in self.client.list_jobs()]
+
+
+def _job_details_to_dict(obj: Any) -> dict[str, Any]:
+    # Ray uses pydantic models internally, but depending on bundled pydantic version
+    # we might get `.model_dump()` (v2) or `.dict()` (v1).
+    if hasattr(obj, "model_dump"):
+        return obj.model_dump()  # type: ignore[no-any-return]
+    if hasattr(obj, "dict"):
+        return obj.dict()  # type: ignore[no-any-return]
+    if hasattr(obj, "__dict__"):
+        return dict(obj.__dict__)
+    return {"repr": repr(obj)}

src/mvp/v1.1/py/mvp_v11/yaml_io.py

@@ -0,0 +1,21 @@
+from __future__ import annotations
+
+from pathlib import Path
+from typing import Any
+
+import yaml
+
+
+def load_yaml(path: str) -> dict[str, Any]:
+    p = Path(path)
+    data = yaml.safe_load(p.read_text(encoding="utf-8"))
+    if data is None:
+        return {}
+    if not isinstance(data, dict):
+        raise ValueError(f"yaml root must be a mapping: {path}")
+    return data
+
+
+def dump_yaml(data: dict[str, Any]) -> str:
+    return yaml.safe_dump(data, sort_keys=False, allow_unicode=True)
+

src/mvp/v1.1/py/requirements.txt

@@ -0,0 +1,2 @@
+PyYAML>=6.0.1
+

src/mvp/v1.1/py/run.py

@@ -0,0 +1,69 @@
+#!/usr/bin/env python3
+from __future__ import annotations
+
+import argparse
+import json
+import os
+import sys
+
+
+def _ensure_import_path() -> None:
+    # Allow `python3 /workspace/.../py/run.py` to import `mvp_v11.*`
+    here = os.path.dirname(os.path.abspath(__file__))
+    if here not in sys.path:
+        sys.path.insert(0, here)
+
+
+def main() -> int:
+    _ensure_import_path()
+
+    from mvp_v11.models import JobSpec, RayConfig
+    from mvp_v11.ray_job_tool import RayJobTool
+    from mvp_v11.yaml_io import load_yaml
+
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--config", required=True, help="Ray base config yaml")
+    parser.add_argument("--jobspec", help="Training jobspec yaml (required for submit)")
+    parser.add_argument("--action", required=True, choices=["submit", "status", "stop", "logs", "list"])
+    parser.add_argument("--submission-id", help="For status/stop/logs")
+    parser.add_argument("--no-wait", action="store_true", help="Submit and return immediately")
+    parser.add_argument("--tail", type=int, default=0, help="Tail N lines for logs")
+    args = parser.parse_args()
+
+    cfg = RayConfig.from_dict(load_yaml(args.config))
+    tool = RayJobTool(cfg)
+
+    if args.action == "submit":
+        if not args.jobspec:
+            raise SystemExit("--jobspec is required for submit")
+        spec = JobSpec.from_dict(load_yaml(args.jobspec))
+        submitted = tool.submit(spec, no_wait=args.no_wait)
+        print(submitted)
+        return 0
+
+    if args.action in ("status", "stop", "logs"):
+        sid = args.submission_id or ""
+        if not sid:
+            raise SystemExit("--submission-id is required for status/stop/logs")
+        if args.action == "status":
+            print(tool.status(sid))
+            return 0
+        if args.action == "stop":
+            print(tool.stop(sid))
+            return 0
+        logs = tool.logs(sid)
+        if args.tail and args.tail > 0:
+            lines = logs.splitlines()
+            logs = "\n".join(lines[-args.tail :]) + ("\n" if lines else "")
+        print(logs, end="")
+        return 0
+
+    if args.action == "list":
+        print(json.dumps(tool.list(), indent=2))
+        return 0
+
+    raise SystemExit(f"unknown action: {args.action}")
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())

src/mvp/v1.1/py/sitecustomize.py

@@ -0,0 +1,57 @@
+"""
+Job-scoped compatibility shims loaded automatically by Python at startup.
+
+This is intentionally lightweight and safe-by-default:
+- Only patches missing symbols.
+- Never raises (best-effort).
+
+Primary use case in MVP v1.1:
+- Allow multiple `verl` versions (e.g. v0.6.0 vs v0.6.1) to run on the same
+  base image where `sglang` APIs may differ slightly.
+"""
+
+from __future__ import annotations
+
+
+def _patch_sglang_get_ip() -> None:
+    try:
+        import sglang.srt.utils as srt_utils  # type: ignore
+    except Exception:
+        return
+
+    if hasattr(srt_utils, "get_ip"):
+        return
+
+    def get_ip() -> str:
+        # Best-effort local IP without external dependency.
+        try:
+            import socket
+
+            s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
+            try:
+                # Doesn't send packets; used to pick the default route/interface.
+                s.connect(("8.8.8.8", 80))
+                return str(s.getsockname()[0])
+            finally:
+                s.close()
+        except Exception:
+            # Fallback: hostname resolution
+            try:
+                import socket
+
+                return str(socket.gethostbyname(socket.gethostname()))
+            except Exception:
+                return "127.0.0.1"
+
+    try:
+        setattr(srt_utils, "get_ip", get_ip)
+    except Exception:
+        return
+
+
+try:
+    _patch_sglang_get_ip()
+except Exception:
+    # Never block interpreter startup.
+    pass
+

src/mvp/v1.1/scripts/00_prereq_check.sh

@@ -0,0 +1,42 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=lib.sh
+source "${SCRIPT_DIR}/lib.sh"
+
+echo "[host] prereq check"
+
+require_cmd docker
+require_cmd bash
+
+if ! docker info >/dev/null 2>&1; then
+  echo "docker is not available (docker info failed)" >&2
+  exit 1
+fi
+
+if ! docker compose version >/dev/null 2>&1; then
+  echo "docker compose is not available" >&2
+  exit 1
+fi
+
+if ! command -v nvidia-smi >/dev/null 2>&1; then
+  echo "WARN: nvidia-smi not found on host; GPU validation skipped"
+else
+  echo "[host] GPU summary"
+  nvidia-smi -L || true
+fi
+
+echo "[host] ensure shared dirs exist under ../shared"
+mkdir -p "${ROOT_DIR}/../shared"/{datasets,hf,jobs,outputs,ray,common,user}
+mkdir -p "${ROOT_DIR}/../shared/common"/{code,datasets,models}
+mkdir -p "${ROOT_DIR}/../shared/user"/{code}
+
+echo "[host] ensure verl repo exists under ../verl (required by prepare scripts)"
+if [[ ! -d "${ROOT_DIR}/../verl" ]]; then
+  echo "missing ../verl. On remote, ensure /home2/argus/infra/mvp/verl exists (git clone volcengine/verl)." >&2
+  exit 1
+fi
+
+echo "ok"
+

src/mvp/v1.1/scripts/01_up.sh

@@ -0,0 +1,16 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=lib.sh
+source "${SCRIPT_DIR}/lib.sh"
+
+if [[ "${SKIP_CLEANUP_V1:-0}" != "1" ]]; then
+  "${SCRIPT_DIR}/03_cleanup_v1_legacy.sh" || true
+fi
+
+echo "[host] docker compose up -d (v1.1)"
+dc up -d
+
+echo "[host] containers:"
+docker ps --format 'table {{.Names}}\t{{.Status}}\t{{.Ports}}' | (head -n 1 && grep -E "mvp11-ray-") || true

src/mvp/v1.1/scripts/02_down.sh

@@ -0,0 +1,12 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=lib.sh
+source "${SCRIPT_DIR}/lib.sh"
+
+echo "[host] docker compose down (v1.1)"
+dc down -v || true
+
+echo "[host] done"
+

src/mvp/v1.1/scripts/03_cleanup_v1_legacy.sh

@@ -0,0 +1,16 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+echo "[host] cleanup v1 legacy containers (best-effort)"
+
+LEGACY=(mvp-ray-head mvp-ray-worker-0 mvp-ray-worker-1)
+
+for c in "${LEGACY[@]}"; do
+  if docker ps -a --format '{{.Names}}' | grep -qx "${c}"; then
+    echo "[host] removing legacy container: ${c}"
+    docker rm -f "${c}" >/dev/null 2>&1 || true
+  fi
+done
+
+echo "[host] legacy cleanup done"
+

src/mvp/v1.1/scripts/05_ensure_verl_repo.sh

@@ -0,0 +1,23 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=lib.sh
+source "${SCRIPT_DIR}/lib.sh"
+
+VERL_DIR="${ROOT_DIR}/../verl"
+
+echo "[host] ensure verl repo exists at: ${VERL_DIR}"
+if [[ -d "${VERL_DIR}/.git" ]]; then
+  echo "verl_repo_exists: skip"
+  exit 0
+fi
+
+if [[ -d "${VERL_DIR}" && ! -d "${VERL_DIR}/.git" ]]; then
+  echo "ERROR: ${VERL_DIR} exists but is not a git repo; please fix manually." >&2
+  exit 1
+fi
+
+echo "cloning volcengine/verl -> ${VERL_DIR}"
+git clone https://github.com/volcengine/verl.git "${VERL_DIR}"
+

src/mvp/v1.1/scripts/12_install_py_deps.sh

@@ -0,0 +1,10 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=lib.sh
+source "${SCRIPT_DIR}/lib.sh"
+
+echo "[head] install python deps for v1.1 SDK submitter (PyYAML)"
+dexec "${HEAD_CONTAINER}" bash -lc "pip install -r /workspace/mvp/v1.1/py/requirements.txt"
+

src/mvp/v1.1/scripts/20_start_head.sh

@@ -0,0 +1,18 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=lib.sh
+source "${SCRIPT_DIR}/lib.sh"
+
+HEAD_IP="$(container_ip "${HEAD_CONTAINER}")"
+
+echo "[head] ray stop (best-effort)"
+dexec "${HEAD_CONTAINER}" bash -lc "ray stop --force || true"
+
+echo "[head] start ray head (CPU=0 GPU=0): ${HEAD_IP}"
+dexec "${HEAD_CONTAINER}" bash -lc "ray start --head --node-ip-address='${HEAD_IP}' --port=6379 --dashboard-host=0.0.0.0 --dashboard-port=8265 --num-cpus=0 --num-gpus=0 --disable-usage-stats"
+
+echo "[head] ray status"
+dexec "${HEAD_CONTAINER}" bash -lc "ray status || true"
+

src/mvp/v1.1/scripts/21_start_workers.sh

@@ -0,0 +1,26 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=lib.sh
+source "${SCRIPT_DIR}/lib.sh"
+
+HEAD_IP="$(container_ip "${HEAD_CONTAINER}")"
+HEAD_ADDR="${HEAD_IP}:6379"
+
+start_one() {
+  local worker="$1"
+  local ip
+  ip="$(container_ip "${worker}")"
+  echo "[${worker}] ray stop (best-effort)"
+  dexec "${worker}" bash -lc "ray stop --force || true"
+  echo "[${worker}] start ray worker -> head ${HEAD_ADDR}"
+  dexec "${worker}" bash -lc "ray start --address='${HEAD_ADDR}' --node-ip-address='${ip}' --resources='{\"worker_node\": 100}' --disable-usage-stats"
+}
+
+start_one "${WORKER0_CONTAINER}"
+start_one "${WORKER1_CONTAINER}"
+
+echo "[head] ray status"
+dexec "${HEAD_CONTAINER}" bash -lc "ray status || true"
+

src/mvp/v1.1/scripts/30_prepare_data_and_model.sh

@@ -0,0 +1,86 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=lib.sh
+source "${SCRIPT_DIR}/lib.sh"
+
+MODEL_ID="${MODEL_ID:-Qwen/Qwen2.5-0.5B-Instruct}"
+
+PPO_DATA_DIR="${SHARED_ROOT}/datasets/gsm8k"
+SFT_DATA_DIR="${SHARED_ROOT}/datasets/gsm8k_sft"
+
+CODE_SNAPSHOT_DIR="${SHARED_ROOT}/common/code/verl/verl_repo"
+
+echo "[head] ensure dataset dirs exist"
+dexec "${HEAD_CONTAINER}" bash -lc "mkdir -p '${PPO_DATA_DIR}' '${SFT_DATA_DIR}'"
+
+echo "[head] prepare PPO dataset (gsm8k RL parquet) -> ${PPO_DATA_DIR}"
+dexec "${HEAD_CONTAINER}" bash -lc "if [[ -f '${PPO_DATA_DIR}/train.parquet' && -f '${PPO_DATA_DIR}/test.parquet' ]]; then echo 'ppo_dataset_exists: skip'; else python3 /workspace/verl/examples/data_preprocess/gsm8k.py --local_save_dir '${PPO_DATA_DIR}'; fi"
+
+echo "[head] prepare SFT dataset (gsm8k messages parquet) -> ${SFT_DATA_DIR}"
+if dexec "${HEAD_CONTAINER}" bash -lc "test -f '${SFT_DATA_DIR}/train.parquet'"; then
+  echo "[head] sft_dataset_exists: skip"
+else
+  SFT_PY_CODE="$(cat <<'PY'
+import os
+
+import pandas as pd
+from datasets import load_dataset
+
+out_dir = os.environ["SFT_DATA_DIR"]
+os.makedirs(out_dir, exist_ok=True)
+
+ds = load_dataset("openai/gsm8k", "main")
+
+instruction = "Let's think step by step and output the final answer after \"####\"."
+
+def to_messages(example):
+    q = example["question"].strip() + " " + instruction
+    a = example["answer"]
+    return {
+        "messages": [
+            {"role": "user", "content": q},
+            {"role": "assistant", "content": a},
+        ]
+    }
+
+train = ds["train"].map(to_messages, remove_columns=ds["train"].column_names)
+test = ds["test"].map(to_messages, remove_columns=ds["test"].column_names)
+
+pd.DataFrame(train).to_parquet(os.path.join(out_dir, "train.parquet"), index=False)
+pd.DataFrame(test).to_parquet(os.path.join(out_dir, "test.parquet"), index=False)
+
+print("sft_dataset_written_ok:", out_dir)
+PY
+)"
+  printf "%s\n" "${SFT_PY_CODE}" | dexec "${HEAD_CONTAINER}" bash -lc "SFT_DATA_DIR='${SFT_DATA_DIR}' python3 -"
+fi
+
+echo "[head] ensure model cached to persistent HF_HOME (idempotent) -> ${MODEL_ID}"
+PY_CODE="$(cat <<'PY'
+import os
+
+model_id = os.environ["MODEL_ID"]
+
+hf_home = os.environ.get("HF_HOME", "/private/hf")
+os.environ.setdefault("HF_HOME", hf_home)
+os.environ.setdefault("HUGGINGFACE_HUB_CACHE", os.path.join(hf_home, "hub"))
+os.environ.setdefault("TRANSFORMERS_CACHE", os.path.join(hf_home, "transformers"))
+
+from huggingface_hub import snapshot_download
+
+try:
+    snapshot_download(repo_id=model_id, local_files_only=True)
+    print("model_cache_exists: skip", model_id)
+except Exception:
+    print("model_cache_missing: downloading", model_id)
+    snapshot_download(repo_id=model_id)
+    print("model_cached_ok:", model_id)
+PY
+)"
+
+printf "%s\n" "${PY_CODE}" | dexec "${HEAD_CONTAINER}" bash -lc "MODEL_ID='${MODEL_ID}' python3 -"
+
+echo "[head] snapshot verl repo into shared common code path (idempotent best-effort) -> ${CODE_SNAPSHOT_DIR}"
+dexec "${HEAD_CONTAINER}" bash -lc "mkdir -p '${CODE_SNAPSHOT_DIR}' && if command -v rsync >/dev/null 2>&1; then rsync -a --delete /workspace/verl/ '${CODE_SNAPSHOT_DIR}/'; else rm -rf '${CODE_SNAPSHOT_DIR:?}/'* && cp -a /workspace/verl/. '${CODE_SNAPSHOT_DIR}/'; fi && echo 'code_snapshot_ok'"

src/mvp/v1.1/scripts/31_snapshot_verl_code.sh

@@ -0,0 +1,42 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=lib.sh
+source "${SCRIPT_DIR}/lib.sh"
+
+# Create an immutable-ish code snapshot under:
+#   ${SHARED_ROOT}/common/code/verl/<code_id>
+#
+# By default, code_id is the git commit hash of /workspace/verl (mounted from ../verl).
+#
+# This enables job-level multi-version coexistence via runtime_env PYTHONPATH injection.
+
+CODE_ID="${CODE_ID:-}"
+
+if [[ -z "${CODE_ID}" ]]; then
+  CODE_ID="$(dexec "${HEAD_CONTAINER}" bash -lc "git config --global --add safe.directory /workspace/verl >/dev/null 2>&1 || true; git -C /workspace/verl rev-parse HEAD")"
+fi
+
+DEST_DIR="${SHARED_ROOT}/common/code/verl/${CODE_ID}"
+
+echo "[head] snapshot verl repo -> ${DEST_DIR}"
+
+if dexec "${HEAD_CONTAINER}" bash -lc "test -d '${DEST_DIR}'"; then
+  echo "[head] code_snapshot_exists: skip"
+  exit 0
+fi
+
+dexec "${HEAD_CONTAINER}" bash -lc "mkdir -p '${DEST_DIR}'"
+
+# Copy code (no .git needed for runtime)
+if dexec "${HEAD_CONTAINER}" bash -lc "command -v rsync >/dev/null 2>&1"; then
+  dexec "${HEAD_CONTAINER}" bash -lc "rsync -a --delete --exclude='.git' /workspace/verl/ '${DEST_DIR}/'"
+else
+  dexec "${HEAD_CONTAINER}" bash -lc "tar -C /workspace/verl -cf - --exclude='.git' . | tar -C '${DEST_DIR}' -xf -"
+fi
+
+# Add a tiny marker module for multi-version validation in Ray job logs.
+dexec "${HEAD_CONTAINER}" bash -lc "printf \"%s\\n\" \"MARKER = '${CODE_ID}'\" > '${DEST_DIR}/mvp_marker.py'"
+
+echo "[head] code_snapshot_ok: ${CODE_ID}"

src/mvp/v1.1/scripts/32_clone_verl_tags.sh

@@ -0,0 +1,39 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=lib.sh
+source "${SCRIPT_DIR}/lib.sh"
+
+VERL_REPO_URL="${VERL_REPO_URL:-https://github.com/volcengine/verl.git}"
+DEST_BASE="${SHARED_ROOT}/common/code/verl"
+
+TAGS=("v0.6.0" "v0.6.1")
+
+echo "[head] ensure base dir: ${DEST_BASE}"
+dexec "${HEAD_CONTAINER}" bash -lc "mkdir -p '${DEST_BASE}'"
+
+for tag in "${TAGS[@]}"; do
+  dest="${DEST_BASE}/verl_${tag}"
+  echo "[head] prepare verl tag ${tag} -> ${dest}"
+
+  verify_repo_cmd="test -d '${dest}/.git' && git -C '${dest}' rev-parse --is-inside-work-tree >/dev/null 2>&1"
+  if dexec "${HEAD_CONTAINER}" bash -lc "${verify_repo_cmd}"; then
+    echo "[head] exists: verified git repo: ${dest}"
+  else
+    echo "[head] cloning ${tag} (retry with HTTP/1.1 if needed)"
+    dexec "${HEAD_CONTAINER}" bash -lc "rm -rf '${dest}'"
+    # Retry a few times because GitHub over HTTP/2 can occasionally fail with curl framing errors/timeouts.
+    dexec "${HEAD_CONTAINER}" bash -lc "set -euo pipefail; for i in 1 2 3; do echo \"clone_attempt=\\$i\"; if git -c http.version=HTTP/1.1 clone --filter=blob:none --single-branch --branch '${tag}' --depth 1 '${VERL_REPO_URL}' '${dest}'; then exit 0; fi; rm -rf '${dest}'; sleep 3; done; exit 1"
+    dexec "${HEAD_CONTAINER}" bash -lc "${verify_repo_cmd}" || { echo \"[head] clone failed or repo invalid: ${dest}\" >&2; exit 1; }
+  fi
+
+  # Avoid git safe.directory issues when reading repo state
+  dexec "${HEAD_CONTAINER}" bash -lc "git config --global --add safe.directory '${dest}' >/dev/null 2>&1 || true"
+  dexec "${HEAD_CONTAINER}" bash -lc "printf 'tag='; git -C '${dest}' describe --tags --exact-match 2>/dev/null || true; printf '\\nhead='; git -C '${dest}' rev-parse HEAD; printf '\\n'"
+
+  # Add marker for multi-version verification in Ray job logs
+  dexec "${HEAD_CONTAINER}" bash -lc "printf \"%s\\n\" \"MARKER = '${tag}'\" > '${dest}/mvp_marker.py'"
+done
+
+echo "[head] done"

src/mvp/v1.1/scripts/40_submit_ppo_epoch1.sh

@@ -0,0 +1,72 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=lib.sh
+source "${SCRIPT_DIR}/lib.sh"
+
+SUBMISSION_ID="${SUBMISSION_ID:-mvp11_ppo_$(timestamp)_$RANDOM}"
+JOB_DIR="${SHARED_ROOT}/jobs/${SUBMISSION_ID}"
+
+MODEL_ID="${MODEL_ID:-Qwen/Qwen2.5-0.5B-Instruct}"
+TRAIN_FILE="${SHARED_ROOT}/datasets/gsm8k/train.parquet"
+VAL_FILE="${SHARED_ROOT}/datasets/gsm8k/test.parquet"
+
+CODE_PATH="${CODE_PATH:-${SHARED_ROOT}/common/code/verl/verl_repo}"
+TOTAL_TRAINING_STEPS="${TOTAL_TRAINING_STEPS:-10}"
+SAVE_FREQ="${SAVE_FREQ:-10}"
+TEST_FREQ="${TEST_FREQ:--1}"
+
+echo "[head] create job dir: ${JOB_DIR}"
+dexec "${HEAD_CONTAINER}" bash -lc "mkdir -p '${JOB_DIR}'/{logs,checkpoints,config,debug}"
+
+SUBMIT_CMD="python3 -m verl.trainer.main_ppo \
+data.train_files=${TRAIN_FILE} \
+data.val_files=${VAL_FILE} \
+data.train_batch_size=256 \
+data.max_prompt_length=512 \
+data.max_response_length=512 \
+actor_rollout_ref.model.path=${MODEL_ID} \
+actor_rollout_ref.actor.optim.lr=1e-6 \
+actor_rollout_ref.actor.ppo_mini_batch_size=64 \
+actor_rollout_ref.actor.ppo_micro_batch_size_per_gpu=4 \
+actor_rollout_ref.rollout.name=sglang \
+actor_rollout_ref.rollout.log_prob_micro_batch_size_per_gpu=8 \
+actor_rollout_ref.rollout.tensor_model_parallel_size=1 \
+actor_rollout_ref.rollout.gpu_memory_utilization=0.4 \
+actor_rollout_ref.ref.log_prob_micro_batch_size_per_gpu=4 \
+critic.optim.lr=1e-5 \
+critic.model.path=${MODEL_ID} \
+critic.ppo_micro_batch_size_per_gpu=4 \
+algorithm.kl_ctrl.kl_coef=0.001 \
+trainer.logger=console \
+trainer.val_before_train=False \
+trainer.n_gpus_per_node=4 \
+trainer.nnodes=2 \
+trainer.save_freq=${SAVE_FREQ} \
+trainer.test_freq=${TEST_FREQ} \
+trainer.total_epochs=1 \
+trainer.total_training_steps=${TOTAL_TRAINING_STEPS} \
+trainer.resume_mode=disable \
+trainer.default_local_dir=${JOB_DIR}/checkpoints \
++ray_kwargs.ray_init.address=auto \
+hydra.run.dir=${JOB_DIR}/logs/hydra"
+
+printf "%s\n" "${SUBMIT_CMD}" | dexec "${HEAD_CONTAINER}" bash -lc "cat > '${JOB_DIR}/config/submit_cmd.txt'"
+
+echo "[head] debug snapshot (pre-submit)"
+dexec "${HEAD_CONTAINER}" bash -lc "ray status >'${JOB_DIR}/debug/ray_status_pre.txt' 2>&1 || true"
+
+echo "[head] submit PPO via ray job submit (driver forced on worker)"
+SUBMIT_OUT="$(dexec "${HEAD_CONTAINER}" bash -lc "ray job submit --address='${RAY_DASHBOARD_ADDR}' --submission-id='${SUBMISSION_ID}' --entrypoint-num-cpus=1 --entrypoint-resources='{\"worker_node\": 1}' --runtime-env-json='{\"env_vars\":{\"HF_HOME\":\"${SHARED_ROOT}/hf\",\"HUGGINGFACE_HUB_CACHE\":\"${SHARED_ROOT}/hf/hub\",\"TRANSFORMERS_CACHE\":\"${SHARED_ROOT}/hf/transformers\",\"HF_ENDPOINT\":\"https://hf-mirror.com\",\"PYTHONUNBUFFERED\":\"1\",\"PYTHONPATH\":\"${CODE_PATH}:${SHARED_ROOT}/user/code\"}}' --no-wait -- ${SUBMIT_CMD}")"
+
+printf "%s\n" "${SUBMIT_OUT}"
+printf "%s\n" "${SUBMIT_OUT}" | dexec "${HEAD_CONTAINER}" bash -lc "cat > '${JOB_DIR}/logs/ray_job_submit.out'"
+echo "${SUBMISSION_ID}" | dexec "${HEAD_CONTAINER}" bash -lc "cat > '${JOB_DIR}/config/ray_submission_id.txt'"
+
+echo "[head] debug snapshot (post-submit)"
+dexec "${HEAD_CONTAINER}" bash -lc "ray job list >'${JOB_DIR}/debug/ray_job_list_post.txt' 2>&1 || true"
+dexec "${HEAD_CONTAINER}" bash -lc "ray status >'${JOB_DIR}/debug/ray_status_post.txt' 2>&1 || true"
+
+echo "submitted: ${SUBMISSION_ID}"
+echo "job dir: ${JOB_DIR}"

src/mvp/v1.1/scripts/41_submit_grpo_epoch1.sh

@@ -0,0 +1,73 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=lib.sh
+source "${SCRIPT_DIR}/lib.sh"
+
+SUBMISSION_ID="${SUBMISSION_ID:-mvp11_grpo_$(timestamp)_$RANDOM}"
+JOB_DIR="${SHARED_ROOT}/jobs/${SUBMISSION_ID}"
+
+MODEL_ID="${MODEL_ID:-Qwen/Qwen2.5-0.5B-Instruct}"
+TRAIN_FILE="${SHARED_ROOT}/datasets/gsm8k/train.parquet"
+VAL_FILE="${SHARED_ROOT}/datasets/gsm8k/test.parquet"
+
+CODE_PATH="${CODE_PATH:-${SHARED_ROOT}/common/code/verl/verl_repo}"
+TOTAL_TRAINING_STEPS="${TOTAL_TRAINING_STEPS:-10}"
+SAVE_FREQ="${SAVE_FREQ:-10}"
+TEST_FREQ="${TEST_FREQ:--1}"
+
+echo "[head] create job dir: ${JOB_DIR}"
+dexec "${HEAD_CONTAINER}" bash -lc "mkdir -p '${JOB_DIR}'/{logs,checkpoints,config,debug}"
+
+SUBMIT_CMD="python3 -m verl.trainer.main_ppo \
+data.train_files=${TRAIN_FILE} \
+data.val_files=${VAL_FILE} \
+data.train_batch_size=256 \
+data.max_prompt_length=512 \
+data.max_response_length=512 \
+actor_rollout_ref.model.path=${MODEL_ID} \
+actor_rollout_ref.actor.optim.lr=1e-6 \
+actor_rollout_ref.actor.ppo_mini_batch_size=64 \
+actor_rollout_ref.actor.ppo_micro_batch_size_per_gpu=4 \
+actor_rollout_ref.rollout.name=sglang \
+actor_rollout_ref.rollout.log_prob_micro_batch_size_per_gpu=8 \
+actor_rollout_ref.rollout.tensor_model_parallel_size=1 \
+actor_rollout_ref.rollout.gpu_memory_utilization=0.4 \
+actor_rollout_ref.ref.log_prob_micro_batch_size_per_gpu=4 \
+critic.optim.lr=1e-5 \
+critic.model.path=${MODEL_ID} \
+critic.ppo_micro_batch_size_per_gpu=4 \
+algorithm.adv_estimator=grpo \
+algorithm.kl_ctrl.kl_coef=0.001 \
+trainer.logger=console \
+trainer.val_before_train=False \
+trainer.n_gpus_per_node=4 \
+trainer.nnodes=2 \
+trainer.save_freq=${SAVE_FREQ} \
+trainer.test_freq=${TEST_FREQ} \
+trainer.total_epochs=1 \
+trainer.total_training_steps=${TOTAL_TRAINING_STEPS} \
+trainer.resume_mode=disable \
+trainer.default_local_dir=${JOB_DIR}/checkpoints \
++ray_kwargs.ray_init.address=auto \
+hydra.run.dir=${JOB_DIR}/logs/hydra"
+
+printf "%s\n" "${SUBMIT_CMD}" | dexec "${HEAD_CONTAINER}" bash -lc "cat > '${JOB_DIR}/config/submit_cmd.txt'"
+
+echo "[head] debug snapshot (pre-submit)"
+dexec "${HEAD_CONTAINER}" bash -lc "ray status >'${JOB_DIR}/debug/ray_status_pre.txt' 2>&1 || true"
+
+echo "[head] submit GRPO via ray job submit (driver forced on worker)"
+SUBMIT_OUT="$(dexec "${HEAD_CONTAINER}" bash -lc "ray job submit --address='${RAY_DASHBOARD_ADDR}' --submission-id='${SUBMISSION_ID}' --entrypoint-num-cpus=1 --entrypoint-resources='{\"worker_node\": 1}' --runtime-env-json='{\"env_vars\":{\"HF_HOME\":\"${SHARED_ROOT}/hf\",\"HUGGINGFACE_HUB_CACHE\":\"${SHARED_ROOT}/hf/hub\",\"TRANSFORMERS_CACHE\":\"${SHARED_ROOT}/hf/transformers\",\"HF_ENDPOINT\":\"https://hf-mirror.com\",\"PYTHONUNBUFFERED\":\"1\",\"PYTHONPATH\":\"${CODE_PATH}:${SHARED_ROOT}/user/code\"}}' --no-wait -- ${SUBMIT_CMD}")"
+
+printf "%s\n" "${SUBMIT_OUT}"
+printf "%s\n" "${SUBMIT_OUT}" | dexec "${HEAD_CONTAINER}" bash -lc "cat > '${JOB_DIR}/logs/ray_job_submit.out'"
+echo "${SUBMISSION_ID}" | dexec "${HEAD_CONTAINER}" bash -lc "cat > '${JOB_DIR}/config/ray_submission_id.txt'"
+
+echo "[head] debug snapshot (post-submit)"
+dexec "${HEAD_CONTAINER}" bash -lc "ray job list >'${JOB_DIR}/debug/ray_job_list_post.txt' 2>&1 || true"
+dexec "${HEAD_CONTAINER}" bash -lc "ray status >'${JOB_DIR}/debug/ray_status_post.txt' 2>&1 || true"
+
+echo "submitted: ${SUBMISSION_ID}"
+echo "job dir: ${JOB_DIR}"

src/mvp/v1.1/scripts/42_submit_sft_minimal.sh

@@ -0,0 +1,62 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=lib.sh
+source "${SCRIPT_DIR}/lib.sh"
+
+SUBMISSION_ID="${SUBMISSION_ID:-mvp11_sft_$(timestamp)_$RANDOM}"
+JOB_DIR="${SHARED_ROOT}/jobs/${SUBMISSION_ID}"
+
+MODEL_ID="${MODEL_ID:-Qwen/Qwen2.5-0.5B-Instruct}"
+TRAIN_FILE="${SHARED_ROOT}/datasets/gsm8k_sft/train.parquet"
+VAL_FILE="${SHARED_ROOT}/datasets/gsm8k_sft/test.parquet"
+
+CODE_PATH="${CODE_PATH:-${SHARED_ROOT}/common/code/verl/verl_repo}"
+TOTAL_TRAINING_STEPS="${TOTAL_TRAINING_STEPS:-10}"
+SAVE_FREQ="${SAVE_FREQ:-10}"
+SFT_DRIVER_DEVICE="${SFT_DRIVER_DEVICE:-cpu}"
+
+echo "[head] create job dir: ${JOB_DIR}"
+dexec "${HEAD_CONTAINER}" bash -lc "mkdir -p '${JOB_DIR}'/{logs,checkpoints,config,debug}"
+
+SUBMIT_CMD="python3 -m verl.trainer.sft_trainer_ray \
+model.path=${MODEL_ID} \
+data.train_files=${TRAIN_FILE} \
+data.val_files=null \
+data.train_batch_size=64 \
+data.micro_batch_size_per_gpu=1 \
+data.max_token_len_per_gpu=2048 \
+data.max_length=1024 \
+trainer.logger=console \
+trainer.project_name=mvp11-sft \
+trainer.experiment_name=${SUBMISSION_ID} \
+trainer.total_epochs=1 \
+trainer.total_training_steps=${TOTAL_TRAINING_STEPS} \
+trainer.save_freq=${SAVE_FREQ} \
+trainer.test_freq=-1 \
+trainer.resume_mode=disable \
+trainer.device=${SFT_DRIVER_DEVICE} \
+trainer.default_local_dir=${JOB_DIR}/checkpoints \
+trainer.nnodes=2 \
+trainer.n_gpus_per_node=4 \
+hydra.run.dir=${JOB_DIR}/logs/hydra"
+
+printf "%s\n" "${SUBMIT_CMD}" | dexec "${HEAD_CONTAINER}" bash -lc "cat > '${JOB_DIR}/config/submit_cmd.txt'"
+
+echo "[head] debug snapshot (pre-submit)"
+dexec "${HEAD_CONTAINER}" bash -lc "ray status >'${JOB_DIR}/debug/ray_status_pre.txt' 2>&1 || true"
+
+echo "[head] submit SFT via ray job submit (driver forced on worker)"
+SUBMIT_OUT="$(dexec "${HEAD_CONTAINER}" bash -lc "ray job submit --address='${RAY_DASHBOARD_ADDR}' --submission-id='${SUBMISSION_ID}' --entrypoint-num-cpus=1 --entrypoint-resources='{\"worker_node\": 1}' --runtime-env-json='{\"env_vars\":{\"HF_HOME\":\"${SHARED_ROOT}/hf\",\"HUGGINGFACE_HUB_CACHE\":\"${SHARED_ROOT}/hf/hub\",\"TRANSFORMERS_CACHE\":\"${SHARED_ROOT}/hf/transformers\",\"HF_ENDPOINT\":\"https://hf-mirror.com\",\"PYTHONUNBUFFERED\":\"1\",\"RAY_ADDRESS\":\"auto\",\"PYTHONPATH\":\"${CODE_PATH}:${SHARED_ROOT}/user/code\"}}' --no-wait -- ${SUBMIT_CMD}")"
+
+printf "%s\n" "${SUBMIT_OUT}"
+printf "%s\n" "${SUBMIT_OUT}" | dexec "${HEAD_CONTAINER}" bash -lc "cat > '${JOB_DIR}/logs/ray_job_submit.out'"
+echo "${SUBMISSION_ID}" | dexec "${HEAD_CONTAINER}" bash -lc "cat > '${JOB_DIR}/config/ray_submission_id.txt'"
+
+echo "[head] debug snapshot (post-submit)"
+dexec "${HEAD_CONTAINER}" bash -lc "ray job list >'${JOB_DIR}/debug/ray_job_list_post.txt' 2>&1 || true"
+dexec "${HEAD_CONTAINER}" bash -lc "ray status >'${JOB_DIR}/debug/ray_status_post.txt' 2>&1 || true"
+
+echo "submitted: ${SUBMISSION_ID}"
+echo "job dir: ${JOB_DIR}"

src/mvp/v1.1/scripts/43_submit_jobspec.sh

@@ -0,0 +1,17 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=lib.sh
+source "${SCRIPT_DIR}/lib.sh"
+
+SPEC_PATH="${1:-}"
+if [[ -z "${SPEC_PATH}" ]]; then
+  echo "usage: $0 <spec_path_inside_container>" >&2
+  echo "example: $0 /workspace/mvp/v1.1/templates/ppo.json" >&2
+  exit 1
+fi
+
+# Submit from head container (required), but with driver forced onto worker via entrypoint resources in spec.
+dexec "${HEAD_CONTAINER}" bash -lc "SHARED_ROOT='${SHARED_ROOT}' python3 /workspace/mvp/v1.1/submit_job.py --spec '${SPEC_PATH}' --no-wait"
+

src/mvp/v1.1/scripts/44_submit_sdk.sh

@@ -0,0 +1,19 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=lib.sh
+source "${SCRIPT_DIR}/lib.sh"
+
+CONFIG_PATH="${1:-/workspace/mvp/v1.1/py/configs/dev.yaml}"
+JOBSPEC_PATH="${2:-}"
+
+if [[ -z "${JOBSPEC_PATH}" ]]; then
+  echo "usage: $0 <ray_config_yaml_in_container> <jobspec_yaml_in_container>" >&2
+  echo "example: $0 /workspace/mvp/v1.1/py/configs/dev.yaml /workspace/mvp/v1.1/py/jobspecs/ppo.yaml" >&2
+  exit 1
+fi
+
+echo "[head] submit via Ray Python SDK"
+dexec "${HEAD_CONTAINER}" bash -lc "python3 /workspace/mvp/v1.1/py/run.py --config '${CONFIG_PATH}' --jobspec '${JOBSPEC_PATH}' --action submit --no-wait"
+

src/mvp/v1.1/scripts/46_submit_ppo_two_verl_tags.sh

@@ -0,0 +1,86 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=lib.sh
+source "${SCRIPT_DIR}/lib.sh"
+
+CONFIG_PATH="${1:-/workspace/mvp/v1.1/py/configs/dev.yaml}"
+
+TS="$(timestamp)"
+BASE="/workspace/mvp/v1.1/py/jobspecs"
+
+NNODES="${NNODES:-1}"
+N_GPUS_PER_NODE="${N_GPUS_PER_NODE:-1}"
+
+wait_job() {
+  local sid="$1"
+  echo "[head] wait: ${sid}"
+  while true; do
+    # Ray returns one of: PENDING/RUNNING/SUCCEEDED/FAILED/STOPPED
+    st="$(dexec "${HEAD_CONTAINER}" bash -lc "python3 /workspace/mvp/v1.1/py/run.py --config '${CONFIG_PATH}' --action status --submission-id '${sid}'" | tr -d '\r' | tail -n 1)"
+    echo "[head] status: ${sid} -> ${st}"
+    case "${st}" in
+      *SUCCEEDED*)
+        return 0
+        ;;
+      *FAILED*|*STOPPED*)
+        echo "[head] job not successful: ${sid} (${st})" >&2
+        echo "[head] last logs:"
+        dexec "${HEAD_CONTAINER}" bash -lc "python3 /workspace/mvp/v1.1/py/run.py --config '${CONFIG_PATH}' --action logs --submission-id '${sid}' --tail 200" || true
+        return 1
+        ;;
+      *)
+        sleep 10
+        ;;
+    esac
+  done
+}
+
+show_precheck() {
+  local sid="$1"
+  echo "[head] verify precheck: ${sid}"
+  dexec "${HEAD_CONTAINER}" bash -lc "python3 /workspace/mvp/v1.1/py/run.py --config '${CONFIG_PATH}' --action logs --submission-id '${sid}' --tail 2000 | egrep 'MVP_PRECHECK_VERL_FILE|MVP_PRECHECK_MARKER' || true"
+}
+
+make_spec() {
+  local tag="$1"
+  local code_path="$2"
+  local out_path="$3"
+  local sid="mvp11_ppo_${tag//./_}_${TS}"
+  dexec "${HEAD_CONTAINER}" bash -lc "cat > '${out_path}' <<'YAML'
+workload: \"ppo\"
+submission_id: \"${sid}\"
+code_path: \"${code_path}\"
+model_id: \"Qwen/Qwen2.5-0.5B-Instruct\"
+train_file: \"/private/datasets/gsm8k/train.parquet\"
+val_file: \"/private/datasets/gsm8k/test.parquet\"
+nnodes: ${NNODES}
+n_gpus_per_node: ${N_GPUS_PER_NODE}
+total_epochs: 1
+total_training_steps: 10
+save_freq: 10
+test_freq: -1
+YAML"
+  echo "${sid}"
+}
+
+echo "[head] submit PPO sequentially with verl v0.6.0 then v0.6.1"
+echo "[head] resources: nnodes=${NNODES} n_gpus_per_node=${N_GPUS_PER_NODE}"
+
+sid0="$(make_spec "v0.6.0" "/private/common/code/verl/verl_v0.6.0" "${BASE}/tmp_ppo_verl_v0.6.0_${TS}.yaml")"
+echo "[head] submit: ${sid0}"
+dexec "${HEAD_CONTAINER}" bash -lc "python3 /workspace/mvp/v1.1/py/run.py --config '${CONFIG_PATH}' --jobspec '${BASE}/tmp_ppo_verl_v0.6.0_${TS}.yaml' --action submit --no-wait"
+wait_job "${sid0}"
+show_precheck "${sid0}"
+
+sid1="$(make_spec "v0.6.1" "/private/common/code/verl/verl_v0.6.1" "${BASE}/tmp_ppo_verl_v0.6.1_${TS}.yaml")"
+echo "[head] submit: ${sid1}"
+dexec "${HEAD_CONTAINER}" bash -lc "python3 /workspace/mvp/v1.1/py/run.py --config '${CONFIG_PATH}' --jobspec '${BASE}/tmp_ppo_verl_v0.6.1_${TS}.yaml' --action submit --no-wait"
+wait_job "${sid1}"
+show_precheck "${sid1}"
+
+echo "[head] done"
+echo "submitted:"
+echo "  ${sid0} (verl v0.6.0)"
+echo "  ${sid1} (verl v0.6.1)"

src/mvp/v1.1/scripts/50_status.sh

@@ -0,0 +1,13 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+# shellcheck source=lib.sh
+source "${SCRIPT_DIR}/lib.sh"
+
+echo "[head] ray status"
+dexec "${HEAD_CONTAINER}" bash -lc "ray status || true"
+
+echo "[head] ray job list"
+dexec "${HEAD_CONTAINER}" bash -lc "ray job list || true"
+

src/mvp/v1.1/scripts/lib.sh

@@ -0,0 +1,52 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+ROOT_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
+
+COMPOSE_FILE="${ROOT_DIR}/docker-compose.yaml"
+
+HEAD_CONTAINER="mvp11-ray-head"
+WORKER0_CONTAINER="mvp11-ray-worker-0"
+WORKER1_CONTAINER="mvp11-ray-worker-1"
+
+SHARED_ROOT="${SHARED_ROOT:-/private}"
+RAY_DASHBOARD_ADDR="${RAY_DASHBOARD_ADDR:-http://127.0.0.1:8265}"
+
+dc() {
+  docker compose --project-directory "${ROOT_DIR}" -f "${COMPOSE_FILE}" "$@"
+}
+
+require_cmd() {
+  local cmd="$1"
+  command -v "${cmd}" >/dev/null 2>&1 || {
+    echo "missing required command: ${cmd}" >&2
+    exit 1
+  }
+}
+
+ensure_container_running() {
+  local name="$1"
+  if ! docker ps --format '{{.Names}}' | grep -qx "${name}"; then
+    echo "container not running: ${name}" >&2
+    exit 1
+  fi
+}
+
+dexec() {
+  local name="$1"
+  shift
+  ensure_container_running "${name}"
+  docker exec -i "${name}" "$@"
+}
+
+container_ip() {
+  local name="$1"
+  ensure_container_running "${name}"
+  docker inspect -f '{{range .NetworkSettings.Networks}}{{.IPAddress}}{{end}}' "${name}"
+}
+
+timestamp() {
+  date +"%Y%m%d_%H%M%S"
+}
+

src/mvp/v1.1/scripts/run_all.sh

@@ -0,0 +1,15 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+
+"${SCRIPT_DIR}/00_prereq_check.sh"
+"${SCRIPT_DIR}/01_up.sh"
+"${SCRIPT_DIR}/20_start_head.sh"
+"${SCRIPT_DIR}/21_start_workers.sh"
+"${SCRIPT_DIR}/30_prepare_data_and_model.sh"
+"${SCRIPT_DIR}/40_submit_ppo_epoch1.sh"
+"${SCRIPT_DIR}/41_submit_grpo_epoch1.sh"
+"${SCRIPT_DIR}/42_submit_sft_minimal.sh"
+"${SCRIPT_DIR}/50_status.sh"
+

src/mvp/v1.1/submit_job.py

@@ -0,0 +1,282 @@
+#!/usr/bin/env python3
+import argparse
+import json
+import os
+import shlex
+import subprocess
+from datetime import datetime
+from pathlib import Path
+
+
+def _ts():
+    return datetime.now().strftime("%Y%m%d_%H%M%S")
+
+
+def _expand(value: str) -> str:
+    return os.path.expandvars(value)
+
+
+def _mkdir(path: Path) -> None:
+    path.mkdir(parents=True, exist_ok=True)
+
+
+def _write_text(path: Path, content: str) -> None:
+    _mkdir(path.parent)
+    path.write_text(content, encoding="utf-8")
+
+
+def _write_json(path: Path, obj) -> None:
+    _mkdir(path.parent)
+    path.write_text(json.dumps(obj, indent=2, ensure_ascii=False) + "\n", encoding="utf-8")
+
+
+def _require(spec: dict, key: str):
+    if key not in spec:
+        raise SystemExit(f"missing required key in spec: {key}")
+    return spec[key]
+
+
+def _default_submission_id(workload: str) -> str:
+    return f"mvp11_{workload}_{_ts()}_{os.getpid()}"
+
+
+def _runtime_env(spec: dict) -> dict:
+    shared_root = _expand(_require(spec, "shared_root"))
+    code_path = _expand(_require(spec, "code_path"))
+    env_vars = dict(spec.get("runtime_env", {}).get("env_vars", {}))
+
+    env_vars.setdefault("HF_HOME", f"{shared_root}/hf")
+    env_vars.setdefault("HUGGINGFACE_HUB_CACHE", f"{shared_root}/hf/hub")
+    env_vars.setdefault("TRANSFORMERS_CACHE", f"{shared_root}/hf/transformers")
+    env_vars.setdefault("PYTHONUNBUFFERED", "1")
+
+    user_code = f"{shared_root}/user/code"
+    existing = env_vars.get("PYTHONPATH", "")
+    prefix = f"{code_path}:{user_code}"
+    env_vars["PYTHONPATH"] = f"{prefix}:{existing}" if existing else prefix
+
+    # Helpful marker for logs/debugging
+    env_vars.setdefault("MVP_CODE_PATH", code_path)
+
+    return {"env_vars": env_vars}
+
+
+def _preflight_shell() -> str:
+    # Make multi-version/debugging observable in Ray job logs.
+    # `mvp_marker.py` is written by our snapshot script (optional); if missing, ignore.
+    py = r"""
+import os
+import sys
+print("MVP_PRECHECK_PYTHON:", sys.executable)
+print("MVP_PRECHECK_PYTHONPATH:", os.environ.get("PYTHONPATH"))
+try:
+    import verl
+    print("MVP_PRECHECK_VERL_FILE:", getattr(verl, "__file__", None))
+except Exception as e:
+    print("MVP_PRECHECK_VERL_IMPORT_ERROR:", repr(e))
+try:
+    import mvp_marker
+    print("MVP_PRECHECK_MARKER:", getattr(mvp_marker, "MARKER", None))
+except Exception as e:
+    print("MVP_PRECHECK_MARKER_MISSING:", repr(e))
+"""
+    return f"python3 - <<'PY'\n{py.strip()}\nPY"
+
+
+def _build_entrypoint(spec: dict, submission_id: str, job_dir: str) -> str:
+    workload = _require(spec, "workload")
+    model_id = _expand(_require(spec, "model_id"))
+    shared_root = _expand(_require(spec, "shared_root"))
+
+    if workload in ("ppo", "grpo"):
+        cfg = spec.get(workload, {})
+        train_file = _expand(cfg.get("train_file", f"{shared_root}/datasets/gsm8k/train.parquet"))
+        val_file = _expand(cfg.get("val_file", f"{shared_root}/datasets/gsm8k/test.parquet"))
+        nnodes = int(cfg.get("nnodes", 2))
+        gpus_per_node = int(cfg.get("n_gpus_per_node", 4))
+        total_epochs = int(cfg.get("total_epochs", 1))
+        total_steps = int(cfg.get("total_training_steps", 10))
+        save_freq = int(cfg.get("save_freq", 10))
+        test_freq = int(cfg.get("test_freq", -1))
+
+        algo_overrides = ""
+        if workload == "grpo":
+            algo_overrides = "algorithm.adv_estimator=grpo"
+
+        cmd = f"""python3 -m verl.trainer.main_ppo \
+data.train_files={train_file} \
+data.val_files={val_file} \
+data.train_batch_size=256 \
+data.max_prompt_length=512 \
+data.max_response_length=512 \
+actor_rollout_ref.model.path={model_id} \
+actor_rollout_ref.actor.optim.lr=1e-6 \
+actor_rollout_ref.actor.ppo_mini_batch_size=64 \
+actor_rollout_ref.actor.ppo_micro_batch_size_per_gpu=4 \
+actor_rollout_ref.rollout.name=sglang \
+actor_rollout_ref.rollout.log_prob_micro_batch_size_per_gpu=8 \
+actor_rollout_ref.rollout.tensor_model_parallel_size=1 \
+actor_rollout_ref.rollout.gpu_memory_utilization=0.4 \
+actor_rollout_ref.ref.log_prob_micro_batch_size_per_gpu=4 \
+critic.optim.lr=1e-5 \
+critic.model.path={model_id} \
+critic.ppo_micro_batch_size_per_gpu=4 \
+algorithm.kl_ctrl.kl_coef=0.001 \
+{algo_overrides} \
+trainer.logger=console \
+trainer.val_before_train=False \
+trainer.n_gpus_per_node={gpus_per_node} \
+trainer.nnodes={nnodes} \
+trainer.save_freq={save_freq} \
+trainer.test_freq={test_freq} \
+trainer.total_epochs={total_epochs} \
+trainer.total_training_steps={total_steps} \
+trainer.resume_mode=disable \
+trainer.default_local_dir={job_dir}/checkpoints \
++ray_kwargs.ray_init.address=auto \
+hydra.run.dir={job_dir}/logs/hydra"""
+        return "\n".join([_preflight_shell(), "exec " + cmd])
+
+    if workload == "sft":
+        cfg = spec.get("sft", {})
+        train_file = _expand(cfg.get("train_file", f"{shared_root}/datasets/gsm8k_sft/train.parquet"))
+        val_file = cfg.get("val_file", None)
+        nnodes = int(cfg.get("nnodes", 2))
+        gpus_per_node = int(cfg.get("n_gpus_per_node", 4))
+        total_epochs = int(cfg.get("total_epochs", 1))
+        total_steps = int(cfg.get("total_training_steps", 10))
+        save_freq = int(cfg.get("save_freq", 10))
+        device = cfg.get("device", "cpu")
+
+        val_override = "data.val_files=null" if val_file is None else f"data.val_files={_expand(val_file)}"
+
+        # Note: driver should not require CUDA under ray job submit (no entrypoint GPUs by default).
+        cmd = f"""python3 -m verl.trainer.sft_trainer_ray \
+model.path={model_id} \
+data.train_files={train_file} \
+{val_override} \
+data.train_batch_size=64 \
+data.micro_batch_size_per_gpu=1 \
+data.max_token_len_per_gpu=2048 \
+data.max_length=1024 \
+trainer.logger=console \
+trainer.project_name=mvp11-sft \
+trainer.experiment_name={submission_id} \
+trainer.total_epochs={total_epochs} \
+trainer.total_training_steps={total_steps} \
+trainer.save_freq={save_freq} \
+trainer.test_freq=-1 \
+trainer.resume_mode=disable \
+trainer.device={device} \
+trainer.default_local_dir={job_dir}/checkpoints \
+trainer.nnodes={nnodes} \
+trainer.n_gpus_per_node={gpus_per_node} \
+hydra.run.dir={job_dir}/logs/hydra"""
+        return "\n".join([_preflight_shell(), "exec " + cmd])
+
+    raise SystemExit(f"unsupported workload: {workload}")
+
+
+def main() -> int:
+    parser = argparse.ArgumentParser()
+    parser.add_argument("--spec", required=True, help="Path to JobSpec json (inside this container)")
+    parser.add_argument("--no-wait", action="store_true", help="Submit and return immediately")
+    parser.add_argument("--dry-run", action="store_true", help="Only print the submit command")
+    args = parser.parse_args()
+
+    spec_path = Path(args.spec)
+    spec = json.loads(spec_path.read_text(encoding="utf-8"))
+
+    shared_root = _expand(_require(spec, "shared_root"))
+    workload = _require(spec, "workload")
+
+    submission_id = spec.get("submission_id") or _default_submission_id(workload)
+    job_dir = f"{shared_root}/jobs/{submission_id}"
+
+    ray_cfg = _require(spec, "ray")
+    ray_addr = ray_cfg.get("address", "http://127.0.0.1:8265")
+    entrypoint_num_cpus = ray_cfg.get("entrypoint_num_cpus", 1)
+    entrypoint_resources = ray_cfg.get("entrypoint_resources", {"worker_node": 1})
+
+    runtime_env = _runtime_env(spec)
+    entrypoint = _build_entrypoint(spec, submission_id=submission_id, job_dir=job_dir)
+
+    # Prepare job dir
+    job_root = Path(job_dir)
+    _mkdir(job_root / "config")
+    _mkdir(job_root / "logs")
+    _mkdir(job_root / "checkpoints")
+    _mkdir(job_root / "debug")
+
+    # Snapshot config for audit/debug
+    _write_json(job_root / "config" / "job_spec.json", spec)
+    _write_json(job_root / "config" / "runtime_env.json", runtime_env)
+    _write_text(job_root / "config" / "ray_submission_id.txt", submission_id + "\n")
+
+    submit_cmd_txt = "\n".join(
+        [
+            "ray job submit",
+            f"  --address={ray_addr}",
+            f"  --submission-id={submission_id}",
+            f"  --entrypoint-num-cpus={entrypoint_num_cpus}",
+            f"  --entrypoint-resources={json.dumps(entrypoint_resources)}",
+            f"  --runtime-env-json=<see runtime_env.json>",
+            f"  {'--no-wait' if args.no_wait else ''}",
+            "  -- bash -lc '<entrypoint>'",
+        ]
+    )
+    _write_text(job_root / "config" / "submit_cmd.txt", submit_cmd_txt + "\n")
+
+    # Debug snapshot (pre-submit)
+    try:
+        pre = subprocess.run(["ray", "status"], capture_output=True, text=True, check=False)
+        _write_text(job_root / "debug" / "ray_status_pre.txt", (pre.stdout or "") + (pre.stderr or ""))
+    except FileNotFoundError:
+        _write_text(job_root / "debug" / "ray_status_pre.txt", "ray cli not found\n")
+
+    submit_args = [
+        "ray",
+        "job",
+        "submit",
+        "--address",
+        ray_addr,
+        "--submission-id",
+        submission_id,
+        "--entrypoint-num-cpus",
+        str(entrypoint_num_cpus),
+        "--entrypoint-resources",
+        json.dumps(entrypoint_resources),
+        "--runtime-env-json",
+        json.dumps(runtime_env),
+    ]
+    if args.no_wait:
+        submit_args.append("--no-wait")
+    submit_args += ["--", "bash", "-lc", entrypoint]
+
+    if args.dry_run:
+        print(" ".join(shlex.quote(x) for x in submit_args))
+        return 0
+
+    proc = subprocess.run(submit_args, capture_output=True, text=True, check=False)
+    _write_text(job_root / "logs" / "ray_job_submit.out", (proc.stdout or "") + (proc.stderr or ""))
+    print(proc.stdout, end="")
+    if proc.returncode != 0:
+        print(proc.stderr, end="", file=os.sys.stderr)
+        return proc.returncode
+
+    # Debug snapshot (post-submit)
+    try:
+        post = subprocess.run(["ray", "job", "list", "--log-style=record", "-v"], capture_output=True, text=True, check=False)
+        _write_text(job_root / "debug" / "ray_job_list_post.txt", (post.stdout or "") + (post.stderr or ""))
+        post2 = subprocess.run(["ray", "status"], capture_output=True, text=True, check=False)
+        _write_text(job_root / "debug" / "ray_status_post.txt", (post2.stdout or "") + (post2.stderr or ""))
+    except FileNotFoundError:
+        pass
+
+    print(f"job_dir: {job_dir}")
+    return 0
+
+
+if __name__ == "__main__":
+    raise SystemExit(main())
+

src/mvp/v1.1/templates/grpo.json

@@ -0,0 +1,31 @@
+{
+  "submission_id": "",
+  "workload": "grpo",
+  "shared_root": "${SHARED_ROOT}",
+  "code_path": "${SHARED_ROOT}/common/code/verl/verl_repo",
+  "model_id": "Qwen/Qwen2.5-0.5B-Instruct",
+  "grpo": {
+    "train_file": "${SHARED_ROOT}/datasets/gsm8k/train.parquet",
+    "val_file": "${SHARED_ROOT}/datasets/gsm8k/test.parquet",
+    "nnodes": 2,
+    "n_gpus_per_node": 4,
+    "total_epochs": 1,
+    "total_training_steps": 10,
+    "save_freq": 10,
+    "test_freq": -1
+  },
+  "ray": {
+    "address": "http://127.0.0.1:8265",
+    "entrypoint_num_cpus": 1,
+    "entrypoint_resources": {
+      "worker_node": 1
+    }
+  },
+  "runtime_env": {
+    "env_vars": {
+      "HF_ENDPOINT": "https://hf-mirror.com",
+      "PYTHONUNBUFFERED": "1"
+    }
+  }
+}
+

src/mvp/v1.1/templates/ppo.json

@@ -0,0 +1,31 @@
+{
+  "submission_id": "",
+  "workload": "ppo",
+  "shared_root": "${SHARED_ROOT}",
+  "code_path": "${SHARED_ROOT}/common/code/verl/verl_repo",
+  "model_id": "Qwen/Qwen2.5-0.5B-Instruct",
+  "ppo": {
+    "train_file": "${SHARED_ROOT}/datasets/gsm8k/train.parquet",
+    "val_file": "${SHARED_ROOT}/datasets/gsm8k/test.parquet",
+    "nnodes": 2,
+    "n_gpus_per_node": 4,
+    "total_epochs": 1,
+    "total_training_steps": 10,
+    "save_freq": 10,
+    "test_freq": -1
+  },
+  "ray": {
+    "address": "http://127.0.0.1:8265",
+    "entrypoint_num_cpus": 1,
+    "entrypoint_resources": {
+      "worker_node": 1
+    }
+  },
+  "runtime_env": {
+    "env_vars": {
+      "HF_ENDPOINT": "https://hf-mirror.com",
+      "PYTHONUNBUFFERED": "1"
+    }
+  }
+}
+

src/mvp/v1.1/templates/sft.json

@@ -0,0 +1,32 @@
+{
+  "submission_id": "",
+  "workload": "sft",
+  "shared_root": "${SHARED_ROOT}",
+  "code_path": "${SHARED_ROOT}/common/code/verl/verl_repo",
+  "model_id": "Qwen/Qwen2.5-0.5B-Instruct",
+  "sft": {
+    "train_file": "${SHARED_ROOT}/datasets/gsm8k_sft/train.parquet",
+    "val_file": null,
+    "nnodes": 2,
+    "n_gpus_per_node": 4,
+    "total_epochs": 1,
+    "total_training_steps": 10,
+    "save_freq": 10,
+    "device": "cpu"
+  },
+  "ray": {
+    "address": "http://127.0.0.1:8265",
+    "entrypoint_num_cpus": 1,
+    "entrypoint_resources": {
+      "worker_node": 1
+    }
+  },
+  "runtime_env": {
+    "env_vars": {
+      "HF_ENDPOINT": "https://hf-mirror.com",
+      "PYTHONUNBUFFERED": "1",
+      "RAY_ADDRESS": "auto"
+    }
+  }
+}
+