mirror of
https://github.com/NVIDIA/cuda-samples.git
synced 2026-11-17 17:38:18 +08:00
CUDA 13.4 samples update - v13.4-public
Release 13.4 of the CUDA Samples supported by CUDA Toolkit 13.4. See Changelog for more information.
This commit is contained in:
@@ -1,32 +1,130 @@
|
||||
# simpleMultiGPU - Simple Multi-GPU
|
||||
# Sample: simpleMultiGPU
|
||||
|
||||
## Description
|
||||
|
||||
This application demonstrates how to use the new CUDA 4.0 API for CUDA context management and multi-threaded access to run CUDA kernels on multiple-GPUs.
|
||||
A multi-GPU reduction demo: the input vector is split across every GPU in the system, each
|
||||
GPU sums its slice concurrently, and the host combines the per-GPU results. Each GPU gets its
|
||||
own CUDA stream, so the H2D copy → reduction kernel → D2H copy pipelines run in parallel across
|
||||
devices. The device-side reduction uses `cub::BlockReduce`; the host finishes the sum across
|
||||
blocks and across GPUs and checks it against a CPU reference. The whole GPU phase is timed with
|
||||
CUDA events.
|
||||
|
||||
## What You'll Learn
|
||||
|
||||
- Enumerating and driving multiple GPUs with `cudaGetDeviceCount` and `cudaSetDevice`
|
||||
- Partitioning a workload across GPUs and running them concurrently, one CUDA stream per GPU
|
||||
- Overlapping H2D copy, kernel, and D2H copy with `cudaMemcpyAsync` on independent streams
|
||||
- Reducing within a block using `cub::BlockReduce`, then finishing the reduction on the host
|
||||
- Timing GPU work with CUDA events (`cudaEventRecord` / `cudaEventElapsedTime`)
|
||||
- Why pinned host memory (`cudaMallocHost`) is required for `cudaMemcpyAsync` to be asynchronous
|
||||
|
||||
## Key Concepts
|
||||
|
||||
Asynchronous Data Transfers, CUDA Streams and Events, Multithreading, Multi-GPU
|
||||
### One Stream per GPU
|
||||
|
||||
## Supported SM Architectures
|
||||
The work is held in a `std::vector<TGPUplan>` with one entry per device, each carrying its own stream,
|
||||
buffers, and data slice. Selecting a device with `cudaSetDevice` and issuing async work on that
|
||||
device's stream lets all GPUs run at the same time:
|
||||
|
||||
[SM 5.0 ](https://developer.nvidia.com/cuda-gpus) [SM 5.2 ](https://developer.nvidia.com/cuda-gpus) [SM 5.3 ](https://developer.nvidia.com/cuda-gpus) [SM 6.0 ](https://developer.nvidia.com/cuda-gpus) [SM 6.1 ](https://developer.nvidia.com/cuda-gpus) [SM 7.0 ](https://developer.nvidia.com/cuda-gpus) [SM 7.2 ](https://developer.nvidia.com/cuda-gpus) [SM 7.5 ](https://developer.nvidia.com/cuda-gpus) [SM 8.0 ](https://developer.nvidia.com/cuda-gpus) [SM 8.6 ](https://developer.nvidia.com/cuda-gpus) [SM 8.7 ](https://developer.nvidia.com/cuda-gpus) [SM 8.9 ](https://developer.nvidia.com/cuda-gpus) [SM 9.0 ](https://developer.nvidia.com/cuda-gpus)
|
||||
```text
|
||||
GPU 0 stream: [H2D slice 0] → [reduceKernel] → [D2H partials 0]
|
||||
GPU 1 stream: [H2D slice 1] → [reduceKernel] → [D2H partials 1]
|
||||
...
|
||||
```
|
||||
|
||||
## Supported OSes
|
||||
The host launches every GPU's pipeline before synchronizing any of them, so the copies and kernels
|
||||
overlap across devices.
|
||||
|
||||
Linux, Windows
|
||||
### Multi-Level Reduction
|
||||
|
||||
## Supported CPU Architecture
|
||||
The reduction happens in three stages. On each GPU, `reduceKernel` launches `BLOCK_N` blocks of `THREAD_N`
|
||||
threads: every thread accumulates a grid-strided partial sum, then `cub::BlockReduce` combines the
|
||||
per-thread partials within its block and thread 0 writes one value per block. The host then adds the
|
||||
`BLOCK_N` per-block partials into that GPU's sum, and finally adds the per-GPU sums together.
|
||||
|
||||
x86_64, armv7l
|
||||
### Pinned Host Memory
|
||||
|
||||
## CUDA APIs involved
|
||||
`cudaMemcpyAsync` only runs asynchronously when the host buffer is page-locked (pinned), so the OS
|
||||
cannot move the pages mid-transfer; with ordinary pageable memory the copy falls back to synchronous
|
||||
behavior. Both the input slice and the partial-sum buffer are allocated with `cudaMallocHost`.
|
||||
|
||||
### [CUDA Runtime API](http://docs.nvidia.com/cuda/cuda-runtime-api/index.html)
|
||||
cudaStreamDestroy, cudaFree, cudaMallocHost, cudaSetDevice, cudaFreeHost, cudaStreamSynchronize, cudaMalloc, cudaMemcpyAsync, cudaStreamCreate, cudaGetDeviceCount
|
||||
### Timing with CUDA Events
|
||||
|
||||
## Prerequisites
|
||||
CUDA events are recorded on device 0's stream to bracket the GPU phase. `cudaEventElapsedTime` returns
|
||||
the milliseconds between the recorded events.
|
||||
|
||||
Download and install the [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) for your corresponding platform.
|
||||
## Key APIs
|
||||
|
||||
## References (for more details)
|
||||
### CUDA Runtime
|
||||
|
||||
- `cudaGetDeviceCount` — count the CUDA-capable GPUs in the system
|
||||
- `cudaSetDevice` — select the active GPU for subsequent CUDA calls
|
||||
- `cudaStreamCreate` / `cudaStreamDestroy` — per-GPU stream that orders each pipeline
|
||||
- `cudaMalloc` / `cudaFree` — allocate and free device memory
|
||||
- `cudaMallocHost` / `cudaFreeHost` — allocate and free pinned host memory
|
||||
- `cudaMemcpyAsync` — non-blocking H2D and D2H transfers on a stream
|
||||
- `cudaStreamSynchronize` — block the host until a GPU's stream is done
|
||||
- `cudaEventCreate` / `cudaEventRecord` / `cudaEventSynchronize` / `cudaEventElapsedTime` / `cudaEventDestroy` — event-based timing
|
||||
|
||||
### CCCL / CUB
|
||||
|
||||
- `cub::BlockReduce` — block-wide reduction inside the kernel
|
||||
|
||||
## Requirements
|
||||
|
||||
### Hardware
|
||||
|
||||
- Two or more NVIDIA GPUs with Compute Capability 7.5 or higher
|
||||
|
||||
### Software
|
||||
|
||||
- CMake 3.20 or newer
|
||||
- A C++17-capable host compiler
|
||||
|
||||
## How to Build
|
||||
|
||||
See the [top-level README](../../../README.md#building-cuda-samples) for full build instructions, including how to build all samples or a single sample standalone.
|
||||
|
||||
## How to Run
|
||||
|
||||
```bash
|
||||
./simpleMultiGPU
|
||||
```
|
||||
|
||||
No command-line arguments are required. The sample uses every GPU it detects, and exits without
|
||||
running if it finds fewer than two.
|
||||
|
||||
## Expected Output
|
||||
|
||||
```text
|
||||
Starting simpleMultiGPU
|
||||
CUDA-capable device count: 2
|
||||
Generating input data...
|
||||
|
||||
Computing with 2 GPUs...
|
||||
GPU Processing time: 8.138752 (ms)
|
||||
|
||||
Computing with Host CPU...
|
||||
|
||||
Comparing GPU and Host CPU results...
|
||||
GPU sum: 16777294.000000
|
||||
CPU sum: 16777294.395033
|
||||
Relative difference: 2.354566E-08
|
||||
```
|
||||
|
||||
**Reading the output:**
|
||||
- The device count and processing time depend on your system
|
||||
- The sums are the same every run because the input is filled from an unseeded `rand()`
|
||||
- The sample passes when the GPU and CPU results agree to within a relative difference of `1e-5`
|
||||
|
||||
## Files
|
||||
|
||||
- `simpleMultiGPU.cu` — reduction kernel and the multi-GPU driver
|
||||
- `README.md` — this file
|
||||
- `CMakeLists.txt` — build configuration
|
||||
|
||||
## See Also
|
||||
|
||||
- [CUDA Programming Guide — CUDA Streams](https://docs.nvidia.com/cuda/cuda-programming-guide/02-basics/asynchronous-execution.html#cuda-streams)
|
||||
- [CUDA Programming Guide — CUDA Events](https://docs.nvidia.com/cuda/cuda-programming-guide/02-basics/asynchronous-execution.html#cuda-events)
|
||||
- [CUDA Core Compute Libraries — What is CUB?](https://nvidia.github.io/cccl/unstable/cub/index.html#what-is-cub)
|
||||
|
||||
Reference in New Issue
Block a user