mirror of
https://github.com/NVIDIA/cuda-samples.git
synced 2026-11-17 17:38:18 +08:00
CUDA 13.4 samples update - v13.4-public
Release 13.4 of the CUDA Samples supported by CUDA Toolkit 13.4. See Changelog for more information.
This commit is contained in:
@@ -1,32 +1,133 @@
|
||||
# simpleCallback - Simple CUDA Callbacks
|
||||
# Sample: simpleCallback
|
||||
|
||||
## Description
|
||||
|
||||
This sample implements multi-threaded heterogeneous computing workloads with the new CPU callbacks for CUDA streams and events introduced with CUDA 5.0.
|
||||
A heterogeneous pipeline demo: **CPU pre-process → GPU kernel → CPU post-process**, all
|
||||
coordinated by a single CUDA stream. A worker thread (`std::thread`) fills the input buffer,
|
||||
enqueues the GPU work (H2D copy, kernel, D2H copy), and registers a host function with
|
||||
`cudaLaunchHostFunc` — the modern replacement for the deprecated `cudaStreamAddCallback`.
|
||||
CUDA invokes that host function automatically once all preceding stream work completes,
|
||||
so the CPU post-processing (result verification) runs without the main thread ever polling.
|
||||
|
||||
## What You'll Learn
|
||||
|
||||
- Building a CPU → GPU → CPU pipeline ordered by a single CUDA stream
|
||||
- Scheduling CPU post-processing on a stream with `cudaLaunchHostFunc`
|
||||
- Enqueuing GPU work from a `std::thread` worker — all host threads share the device's primary context
|
||||
- The rule that a host function must never call CUDA APIs
|
||||
- Why pinned host memory (`cudaMallocHost`) is required for `cudaMemcpyAsync` to be truly asynchronous
|
||||
- Querying GPU properties (compute capability, SM count) with `cudaDeviceGetAttribute`
|
||||
|
||||
## Key Concepts
|
||||
|
||||
CUDA Streams, Callback Functions, Multithreading
|
||||
### Stream-Ordered Pipeline
|
||||
|
||||
## Supported SM Architectures
|
||||
A stream executes its operations strictly in order. The worker thread pushes the whole
|
||||
pipeline onto one stream and CUDA handles every dependency:
|
||||
|
||||
[SM 5.0 ](https://developer.nvidia.com/cuda-gpus) [SM 5.2 ](https://developer.nvidia.com/cuda-gpus) [SM 5.3 ](https://developer.nvidia.com/cuda-gpus) [SM 6.0 ](https://developer.nvidia.com/cuda-gpus) [SM 6.1 ](https://developer.nvidia.com/cuda-gpus) [SM 7.0 ](https://developer.nvidia.com/cuda-gpus) [SM 7.2 ](https://developer.nvidia.com/cuda-gpus) [SM 7.5 ](https://developer.nvidia.com/cuda-gpus) [SM 8.0 ](https://developer.nvidia.com/cuda-gpus) [SM 8.6 ](https://developer.nvidia.com/cuda-gpus) [SM 8.7 ](https://developer.nvidia.com/cuda-gpus) [SM 8.9 ](https://developer.nvidia.com/cuda-gpus) [SM 9.0 ](https://developer.nvidia.com/cuda-gpus)
|
||||
```text
|
||||
stream: [H2D copy] → [incrementKernel] → [D2H copy] → [postprocess host func]
|
||||
```
|
||||
|
||||
## Supported OSes
|
||||
The main thread only has to `cudaStreamSynchronize` once at the end — when the stream
|
||||
drains, the results are already verified.
|
||||
|
||||
Linux, Windows
|
||||
### Host Functions (`cudaLaunchHostFunc`)
|
||||
|
||||
## Supported CPU Architecture
|
||||
A host function is a CPU callback enqueued on a stream like any other operation. CUDA runs
|
||||
it on an internal thread once all prior work in the stream has finished. Two rules apply:
|
||||
|
||||
x86_64, armv7l
|
||||
- It must **not** call any CUDA runtime or driver API (no `cudaFree`, no kernel launches)
|
||||
- It should be short — the stream cannot proceed until it returns
|
||||
|
||||
## CUDA APIs involved
|
||||
### Threads Share the Primary Context
|
||||
|
||||
### [CUDA Runtime API](http://docs.nvidia.com/cuda/cuda-runtime-api/index.html)
|
||||
cudaHostAlloc, cudaStreamDestroy, cudaFree, cudaSetDevice, cudaGetDeviceCount, cudaFreeHost, cudaStreamAddCallback, cudaMalloc, cudaMemcpyAsync, cudaStreamCreate, cudaGetDeviceProperties
|
||||
With the CUDA runtime API, every host thread in the process shares the device's primary
|
||||
context. That is why the worker thread can enqueue copies and kernels onto a stream that
|
||||
`main` created.
|
||||
|
||||
## Prerequisites
|
||||
### Pinned Host Memory
|
||||
|
||||
Download and install the [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) for your corresponding platform.
|
||||
`cudaMemcpyAsync` only runs *truly asynchronously* when the host buffer is page-locked (pinned),
|
||||
so the OS cannot move the pages mid-transfer; with ordinary pageable memory the copy falls back to
|
||||
synchronous behavior. The sample allocates the host buffer with `cudaMallocHost`.
|
||||
|
||||
## References (for more details)
|
||||
## Key APIs
|
||||
|
||||
### CUDA Runtime
|
||||
|
||||
- `cudaSetDevice` — select the active GPU for all subsequent CUDA calls
|
||||
- `cudaDeviceGetAttribute` — query compute capability and SM count
|
||||
- `cudaMallocHost` / `cudaFreeHost` — allocate and free pinned host memory
|
||||
- `cudaMalloc` / `cudaFree` — allocate and free device memory
|
||||
- `cudaStreamCreate` / `cudaStreamDestroy` — create and destroy the stream that orders the pipeline
|
||||
- `cudaMemcpyAsync` — non-blocking H2D and D2H transfers on the stream
|
||||
- `cudaLaunchHostFunc` — enqueue a CPU callback on the stream
|
||||
- `cudaStreamSynchronize` — block the host until the stream (including the host function) is done
|
||||
|
||||
### C++ Standard Library
|
||||
|
||||
- `std::thread` / `join()` — worker-thread creation
|
||||
|
||||
## Requirements
|
||||
|
||||
### Hardware
|
||||
|
||||
- NVIDIA GPU with Compute Capability 7.5 or higher
|
||||
|
||||
### Software
|
||||
|
||||
- CMake 3.20 or newer
|
||||
- A C++17-capable host compiler
|
||||
|
||||
## How to Build
|
||||
|
||||
See the [top-level README](../../../README.md#building-cuda-samples) for full build instructions, including how to build all samples or a single sample standalone.
|
||||
|
||||
## How to Run
|
||||
|
||||
```bash
|
||||
./simpleCallback
|
||||
```
|
||||
|
||||
No command-line arguments are required. The sample always runs on device 0.
|
||||
|
||||
## Expected Output
|
||||
|
||||
```text
|
||||
=====================================================
|
||||
simpleCallback: CPU -> GPU -> CPU pipeline demo
|
||||
=====================================================
|
||||
Using GPU 0: compute capability X.Y, <smCount> SMs
|
||||
[thread] Stage 1: CPU pre-processing on a worker thread...
|
||||
[thread] Filled 100000 elements. First 3 inputs: 42, 43, 44
|
||||
|
||||
[thread] Stage 2: enqueuing H2D copy, kernel, D2H copy on the stream
|
||||
[thread] Registering post-processing callback with cudaLaunchHostFunc
|
||||
|
||||
[main] Worker thread joined; GPU work has been enqueued.
|
||||
[main] Waiting for the stream (and callback) to finish...
|
||||
|
||||
[host func] Stage 3: callback fired automatically - the GPU work is done!
|
||||
[host func] First 3 results: 43, 44, 45 (each input +1)
|
||||
[host func] Verified all 100000 results: PASS
|
||||
|
||||
[main] Pipeline complete. Result: SUCCESS
|
||||
```
|
||||
|
||||
**Reading the output:**
|
||||
- Inputs start at 42 because each element is `workload.id + i` with `id = 42`
|
||||
- Each result is its input plus one — the kernel's only job
|
||||
- The `[host func]` lines print from a CUDA internal thread, not from `main`
|
||||
|
||||
## Files
|
||||
|
||||
- `simpleCallback.cu` — kernel, worker-thread function, host-function callback, and main driver
|
||||
- `README.md` — this file
|
||||
- `CMakeLists.txt` — build configuration
|
||||
|
||||
## See Also
|
||||
|
||||
- [CUDA Programming Guide — Callback Functions from Streams](https://docs.nvidia.com/cuda/cuda-programming-guide/02-basics/asynchronous-execution.html#callback-functions-from-streams)
|
||||
- [CUDA Programming Guide — CUDA Streams](https://docs.nvidia.com/cuda/cuda-programming-guide/02-basics/asynchronous-execution.html#cuda-streams)
|
||||
- [CUDA Runtime API — Execution Control (cudaLaunchHostFunc)](https://docs.nvidia.com/cuda/cuda-runtime-api/group__CUDART__EXECUTION.html#group__CUDART__EXECUTION_1g05841eaa5f90f27124241baafb3e856f)
|
||||
|
||||
Reference in New Issue
Block a user