mirror of
https://github.com/NVIDIA/cuda-samples.git
synced 2026-09-12 01:06:53 +08:00
5443602d89
Release 13.4 of the CUDA Samples supported by CUDA Toolkit 13.4. See Changelog for more information.
131 lines
4.9 KiB
Markdown
131 lines
4.9 KiB
Markdown
# Sample: simpleMultiGPU
|
|
|
|
## Description
|
|
|
|
A multi-GPU reduction demo: the input vector is split across every GPU in the system, each
|
|
GPU sums its slice concurrently, and the host combines the per-GPU results. Each GPU gets its
|
|
own CUDA stream, so the H2D copy → reduction kernel → D2H copy pipelines run in parallel across
|
|
devices. The device-side reduction uses `cub::BlockReduce`; the host finishes the sum across
|
|
blocks and across GPUs and checks it against a CPU reference. The whole GPU phase is timed with
|
|
CUDA events.
|
|
|
|
## What You'll Learn
|
|
|
|
- Enumerating and driving multiple GPUs with `cudaGetDeviceCount` and `cudaSetDevice`
|
|
- Partitioning a workload across GPUs and running them concurrently, one CUDA stream per GPU
|
|
- Overlapping H2D copy, kernel, and D2H copy with `cudaMemcpyAsync` on independent streams
|
|
- Reducing within a block using `cub::BlockReduce`, then finishing the reduction on the host
|
|
- Timing GPU work with CUDA events (`cudaEventRecord` / `cudaEventElapsedTime`)
|
|
- Why pinned host memory (`cudaMallocHost`) is required for `cudaMemcpyAsync` to be asynchronous
|
|
|
|
## Key Concepts
|
|
|
|
### One Stream per GPU
|
|
|
|
The work is held in a `std::vector<TGPUplan>` with one entry per device, each carrying its own stream,
|
|
buffers, and data slice. Selecting a device with `cudaSetDevice` and issuing async work on that
|
|
device's stream lets all GPUs run at the same time:
|
|
|
|
```text
|
|
GPU 0 stream: [H2D slice 0] → [reduceKernel] → [D2H partials 0]
|
|
GPU 1 stream: [H2D slice 1] → [reduceKernel] → [D2H partials 1]
|
|
...
|
|
```
|
|
|
|
The host launches every GPU's pipeline before synchronizing any of them, so the copies and kernels
|
|
overlap across devices.
|
|
|
|
### Multi-Level Reduction
|
|
|
|
The reduction happens in three stages. On each GPU, `reduceKernel` launches `BLOCK_N` blocks of `THREAD_N`
|
|
threads: every thread accumulates a grid-strided partial sum, then `cub::BlockReduce` combines the
|
|
per-thread partials within its block and thread 0 writes one value per block. The host then adds the
|
|
`BLOCK_N` per-block partials into that GPU's sum, and finally adds the per-GPU sums together.
|
|
|
|
### Pinned Host Memory
|
|
|
|
`cudaMemcpyAsync` only runs asynchronously when the host buffer is page-locked (pinned), so the OS
|
|
cannot move the pages mid-transfer; with ordinary pageable memory the copy falls back to synchronous
|
|
behavior. Both the input slice and the partial-sum buffer are allocated with `cudaMallocHost`.
|
|
|
|
### Timing with CUDA Events
|
|
|
|
CUDA events are recorded on device 0's stream to bracket the GPU phase. `cudaEventElapsedTime` returns
|
|
the milliseconds between the recorded events.
|
|
|
|
## Key APIs
|
|
|
|
### CUDA Runtime
|
|
|
|
- `cudaGetDeviceCount` — count the CUDA-capable GPUs in the system
|
|
- `cudaSetDevice` — select the active GPU for subsequent CUDA calls
|
|
- `cudaStreamCreate` / `cudaStreamDestroy` — per-GPU stream that orders each pipeline
|
|
- `cudaMalloc` / `cudaFree` — allocate and free device memory
|
|
- `cudaMallocHost` / `cudaFreeHost` — allocate and free pinned host memory
|
|
- `cudaMemcpyAsync` — non-blocking H2D and D2H transfers on a stream
|
|
- `cudaStreamSynchronize` — block the host until a GPU's stream is done
|
|
- `cudaEventCreate` / `cudaEventRecord` / `cudaEventSynchronize` / `cudaEventElapsedTime` / `cudaEventDestroy` — event-based timing
|
|
|
|
### CCCL / CUB
|
|
|
|
- `cub::BlockReduce` — block-wide reduction inside the kernel
|
|
|
|
## Requirements
|
|
|
|
### Hardware
|
|
|
|
- Two or more NVIDIA GPUs with Compute Capability 7.5 or higher
|
|
|
|
### Software
|
|
|
|
- CMake 3.20 or newer
|
|
- A C++17-capable host compiler
|
|
|
|
## How to Build
|
|
|
|
See the [top-level README](../../../README.md#building-cuda-samples) for full build instructions, including how to build all samples or a single sample standalone.
|
|
|
|
## How to Run
|
|
|
|
```bash
|
|
./simpleMultiGPU
|
|
```
|
|
|
|
No command-line arguments are required. The sample uses every GPU it detects, and exits without
|
|
running if it finds fewer than two.
|
|
|
|
## Expected Output
|
|
|
|
```text
|
|
Starting simpleMultiGPU
|
|
CUDA-capable device count: 2
|
|
Generating input data...
|
|
|
|
Computing with 2 GPUs...
|
|
GPU Processing time: 8.138752 (ms)
|
|
|
|
Computing with Host CPU...
|
|
|
|
Comparing GPU and Host CPU results...
|
|
GPU sum: 16777294.000000
|
|
CPU sum: 16777294.395033
|
|
Relative difference: 2.354566E-08
|
|
```
|
|
|
|
**Reading the output:**
|
|
- The device count and processing time depend on your system
|
|
- The sums are the same every run because the input is filled from an unseeded `rand()`
|
|
- The sample passes when the GPU and CPU results agree to within a relative difference of `1e-5`
|
|
|
|
## Files
|
|
|
|
- `simpleMultiGPU.cu` — reduction kernel and the multi-GPU driver
|
|
- `README.md` — this file
|
|
- `CMakeLists.txt` — build configuration
|
|
|
|
## See Also
|
|
|
|
- [CUDA Programming Guide — CUDA Streams](https://docs.nvidia.com/cuda/cuda-programming-guide/02-basics/asynchronous-execution.html#cuda-streams)
|
|
- [CUDA Programming Guide — CUDA Events](https://docs.nvidia.com/cuda/cuda-programming-guide/02-basics/asynchronous-execution.html#cuda-events)
|
|
- [CUDA Core Compute Libraries — What is CUB?](https://nvidia.github.io/cccl/unstable/cub/index.html#what-is-cub)
|