NASP/cuda-samples
84
0
Fork 0
mirror of https://github.com/NVIDIA/cuda-samples.git synced 2026-10-13 00:08:26 +08:00
Code Issues Packages Projects Releases Wiki Activity

CUDA 13.2 samples update (#432)

Browse Source 
- Added Python samples for CUDA Python 1.0 release
- Renamed top-level `Samples` directory to `cpp` to accommodate Python samples.
This commit is contained in:
Dheemanth
2026-05-13 15:13:18 -07:00
committed by GitHub
parent a4526d5229
commit aeab82ff30
2072 changed files with 15077 additions and 1970 deletions
Download Patch File Download Diff File Expand all files Collapse all files

190
python/4_DistributedComputing/simpleP2P/README.md Normal file
View File

@@ -0,0 +1,190 @@
# Sample: simpleP2P (Python)
## Description
This sample demonstrates peer-to-peer (P2P) memory access between multiple GPUs in CUDA using the cuda.core Python library. P2P allows GPUs to directly access each other's memory without routing data through the host (CPU), enabling efficient multi-GPU applications. This sample detects P2P-capable GPUs, enables peer access, measures bandwidth using CUDA events for accurate GPU-side timing, and launches kernels (using grid-stride loops) that read from one GPU's memory and write to another GPU's memory.
## What you will learn
- How to detect multiple CUDA-capable GPUs using `system.get_num_devices()` and `Device(id)`
- How to check P2P capability between GPU pairs using `device.can_access_peer()`
- How to enable and disable peer access using `DeviceMemoryResource.peer_accessible_by`
- How to allocate device memory on specific GPUs using `DeviceMemoryResource`
- How to perform direct GPU-to-GPU memory transfers with explicit event-based synchronization
- How to measure P2P bandwidth using CUDA events for accurate GPU-side timing
- How to use event-based synchronization between streams for sequential bandwidth measurement
- How to launch kernels on one GPU that access memory from another GPU
- How to compile and launch CUDA kernels using cuda.core's `Program` and `launch` APIs with grid-stride loops
- How to validate multi-GPU computation results
- How to properly clean up resources using try/finally blocks
## Key libraries
- `numpy` - CPU array operations and data initialization
- `cuda-core` - Modern Python interface to CUDA runtime with full P2P support
## Key APIs
**From cuda.core:**
- `system` – Pre-instantiated singleton for system-level CUDA information
- `system.get_num_devices()` – Get number of CUDA-capable devices
- `Device(id)` – Get specific CUDA device handle
- `device.can_access_peer(peer)` – Check if this device can access peer device memory
- `device.set_current()` – Set active device for subsequent operations
- `device.create_stream()` – Create CUDA stream for kernel execution
- `DeviceMemoryResource(device)` – Create memory resource for specific GPU
- `memory_resource.peer_accessible_by` – Get/set which devices can access this memory pool's allocations
- Example: `mr.peer_accessible_by = [1]` grants device 1 access
- Example: `mr.peer_accessible_by = []` revokes all access
- `PinnedMemoryResource()` – Allocate pinned (page-locked) host memory
- `EventOptions(enable_timing=True)` – Create options for CUDA events with timing enabled
- `stream.record(options=event_options)` – Record a CUDA event on a stream
- `event.elapsed_time(start_event)` – Get elapsed time in milliseconds between two events
- `stream.wait_event(event)` – Make a stream wait for an event to complete
- `stream.close()` – Clean up stream resources
- `Program()` – Compile CUDA C++ kernel code
- `LaunchConfig()` – Configure kernel launch parameters (grid, block)
- `launch()` – Launch compiled kernel with arguments
- `buffer.copy_from(src, stream=stream)` – Copy data from source buffer asynchronously
- `buffer.copy_to(dst, stream=stream)` – Copy data to destination buffer asynchronously
**From DLPack:**
- `numpy.from_dlpack()` – Create NumPy array view of memory buffer
**Memory Management:**
- Resources (streams, buffers) should be cleaned up using try/finally blocks to ensure proper cleanup even if errors occur
- Streams should be explicitly closed with `stream.close()` in finally blocks
## Peer-to-Peer (P2P): When to Use
### Benefits
- **Direct GPU-to-GPU transfers**: Bypass host memory for faster communication
- **Higher bandwidth**: PCIe or NVLink bandwidth between GPUs (up to 600 GB/s with NVLink)
- **Lower latency**: No CPU involvement in data transfers
- **Efficient multi-GPU**: Essential for scaling deep learning, HPC, and simulation workloads
- **Simplified programming**: Kernels can directly access remote GPU memory
### Requirements
- **Two or more GPUs**: System must have multiple CUDA-capable GPUs
- **P2P support**: GPUs must be P2P-capable (check with `can_access_peer()`)
- **PCIe topology**: Usually requires GPUs on same PCIe root complex
- **Platform support**: Not available on Mac OSX, limited on ARM platforms
### Best Use Cases
1. Multi-GPU deep learning training (model parallelism, data parallelism)
2. Large-scale scientific simulations across multiple GPUs
3. Real-time rendering with multiple GPUs
4. GPU clusters with direct GPU communication
5. Reducing CPU-GPU traffic in multi-GPU systems
## Requirements
1. **Two or more NVIDIA Graphics Cards** with CUDA support and P2P capability
2. **CUDA Drivers** installed on your system
3. **CUDA Toolkit 13.0+** installed on your system
4. **Python 3.10 or newer**
5. **Proper PCIe topology** (GPUs should be on same PCIe root complex for best performance)
**Note**: This sample will gracefully exit if fewer than 2 GPUs are detected or if P2P is not supported between any GPU pair.
**Install packages:**
```bash
pip install -r requirements.txt
```
Or manually:
```bash
pip install numpy>=2.3.2 cuda-core>=0.6.0 cuda-python>=13.0.0
```
## How to run
Basic usage:
```bash
# Run with default parameters (16M elements = 64MB)
python simpleP2P.py
```
With custom parameters:
```bash
# Use 32M elements (128MB)
python simpleP2P.py --num_elements 33554432
# Show help
python simpleP2P.py --help
```
### Command line arguments
- `--num_elements`: Number of elements in arrays (default: 16777216)
- Each array uses `num_elements * 4 bytes` (float32)
- Default: 64 MB per array
- Sample allocates 2 device buffers + 1 host buffer
## Expected Output
```
======================================================================
simpleP2P - CUDA Python Sample
======================================================================
Starting...
Checking for multiple GPUs...
CUDA-capable device count: 2
Checking GPU(s) for support of peer to peer memory access...
> Peer access from Tesla T10 (GPU0) -> Tesla T10 (GPU1): Yes
> Peer access from Tesla T10 (GPU1) -> Tesla T10 (GPU0): Yes
Using GPU0 (Tesla T10) and GPU1 (Tesla T10)
Allocating buffers (64MB on GPU0, GPU1 and CPU Host)...
Peer access enabled: GPU0 <-> GPU1
Peer access status: MR0 accessible by (1,), MR1 accessible by (0,)
Memory allocated successfully
Measuring P2P bandwidth...
Performing 100 ping-pong copies between GPUs...
P2P bandwidth: 12.37 GB/s
Preparing host buffer and memcpy to GPU0...
Data initialized and copied to GPU
Compiling CUDA kernel...
Kernels compiled successfully
Run kernel on GPU1, taking source data from GPU0 and writing to GPU1...
Kernel execution complete
Run kernel on GPU0, taking source data from GPU1 and writing to GPU0...
Kernel execution complete
Copy data back to host from GPU0 and verify results...
Checking results...
Comparing 16,777,216 elements...
Test PASSED
[PASS] Validation PASSED
Disabling peer access...
Peer access revoked: MR0 accessible by (), MR1 accessible by ()
======================================================================
simpleP2P completed successfully!
======================================================================
Shutting down...
```
**Note**: P2P bandwidth varies based on:
- PCIe generation
- NVLink
- System topology and configuration
## Files
- `simpleP2P.py` – Main Python implementation
- `README.md` – This file
- `requirements.txt` – Python package dependencies

6
python/4_DistributedComputing/simpleP2P/requirements.txt Normal file
View File

@@ -0,0 +1,6 @@
# simpleP2P - Requirements
# Install with: pip install -r requirements.txt
numpy>=2.3.2
cuda-python>=13.0.0
cuda-core>=0.6.0

377
python/4_DistributedComputing/simpleP2P/simpleP2P.py Normal file
View File

@@ -0,0 +1,377 @@
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# distribution and/or other materials provided with the distribution.
# * Neither the name of NVIDIA CORPORATION nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
import argparse
import sys
from pathlib import Path
try:
import numpy as np
from cuda.core import (
Device,
DeviceMemoryResource,
EventOptions,
LaunchConfig,
PinnedMemoryResource,
Program,
ProgramOptions,
launch,
system,
)
except ImportError as e:
print(f"Error: Required package not found: {e}")
print("Please install from requirements.txt:")
print(" pip install -r requirements.txt")
sys.exit(1)
# Add parent directory to path to import utilities
sys.path.insert(0, str(Path(__file__).parent.parent.parent / "Utilities"))
from cuda_samples_utils import verify_array_result # noqa: E402
# CUDA kernel for simple P2P operation
SIMPLE_P2P_KERNEL = """
extern "C" __global__
void SimpleKernel(float *src, float *dst, int N) {
// Grid-stride loop pattern for canonical CUDA kernel
size_t tid = blockIdx.x * blockDim.x + threadIdx.x;
size_t stride = gridDim.x * blockDim.x;
for (size_t i = tid; i < N; i += stride) {
dst[i] = src[i] * 2.0f;
}
}
"""
def run(num_elements=1024 * 1024 * 16):
"""
Demonstrates peer-to-peer (P2P) memory access between multiple GPUs using cuda.core.
This function shows how to:
1. Detect and verify multiple GPUs with P2P capability
2. Enable peer access between GPUs
3. Perform direct GPU-to-GPU memory transfers
4. Launch kernels that access memory from other GPUs
5. Measure P2P bandwidth
6. Validate results
Parameters
----------
num_elements : int
Number of elements in arrays (default: 16M elements = 64MB)
"""
print("\n" + "=" * 70)
print("simpleP2P - CUDA Python Sample")
print("=" * 70)
print("\nStarting...")
# Check for multiple GPUs
print("\nChecking for multiple GPUs...")
num_devices = system.get_num_devices()
print(f"CUDA-capable device count: {num_devices}")
if num_devices < 2:
print(
"Two or more GPUs with Peer-to-Peer access capability are "
"required, waiving this sample."
)
return 2
# Get device properties
devices = [Device(i) for i in range(num_devices)]
# Check for P2P capability
print("\nChecking GPU(s) for support of peer to peer memory access...")
p2p_capable_gpus = [-1, -1]
for i in range(num_devices):
p2p_capable_gpus[0] = i
for j in range(num_devices):
if i == j:
continue
# Check peer access capability using cuda.core
i_access_j = devices[i].can_access_peer(devices[j])
j_access_i = devices[j].can_access_peer(devices[i])
print(
f"> Peer access from {devices[i].name} (GPU{i}) -> "
f"{devices[j].name} (GPU{j}): {'Yes' if i_access_j else 'No'}"
)
print(
f"> Peer access from {devices[j].name} (GPU{j}) -> "
f"{devices[i].name} (GPU{i}): {'Yes' if j_access_i else 'No'}"
)
if i_access_j and j_access_i:
p2p_capable_gpus[1] = j
break
if p2p_capable_gpus[1] != -1:
break
if p2p_capable_gpus[0] == -1 or p2p_capable_gpus[1] == -1:
print("\nTwo or more GPUs with Peer-to-Peer access capability are required.")
print(
"Peer to Peer access is not available amongst GPUs in the system, "
"waiving test."
)
return 2
# Use first pair of P2P capable GPUs detected
gpuid = [p2p_capable_gpus[0], p2p_capable_gpus[1]]
dev0 = devices[gpuid[0]]
dev1 = devices[gpuid[1]]
print(f"\nUsing GPU{gpuid[0]} ({dev0.name}) and GPU{gpuid[1]} ({dev1.name})")
# Allocate buffers with P2P access
buf_size = num_elements * np.dtype(np.float32).itemsize
print(
f"\nAllocating buffers ({int(buf_size / 1024 / 1024)}MB on "
f"GPU{gpuid[0]}, GPU{gpuid[1]} and CPU Host)..."
)
# Allocate on GPU 0 and grant access to GPU 1
dev0.set_current()
mr0 = DeviceMemoryResource(dev0)
mr0.peer_accessible_by = [gpuid[1]] # Grant GPU 1 access to GPU 0's memory
g0 = mr0.allocate(buf_size)
# Allocate on GPU 1 and grant access to GPU 0
dev1.set_current()
mr1 = DeviceMemoryResource(dev1)
mr1.peer_accessible_by = [gpuid[0]] # Grant GPU 0 access to GPU 1's memory
g1 = mr1.allocate(buf_size)
print(f" Peer access enabled: GPU{gpuid[0]} <-> GPU{gpuid[1]}")
print(
f" Peer access status: MR0 accessible by {mr0.peer_accessible_by}, "
f"MR1 accessible by {mr1.peer_accessible_by}"
)
# Allocate pinned host memory
pinned_mr = PinnedMemoryResource()
h0 = pinned_mr.allocate(buf_size)
print(" Memory allocated successfully")
# Create streams
stream0 = dev0.create_stream()
stream1 = dev1.create_stream()
try:
# P2P bandwidth test using CUDA events for accurate GPU-side timing
print("\nMeasuring P2P bandwidth...")
print(" Performing 100 ping-pong copies between GPUs...")
event_options = EventOptions(enable_timing=True)
sync_event0 = None
sync_event1 = None
# Record start event on stream0
start_event = stream0.record(options=event_options)
for i in range(100):
# Ping-pong copy between GPUs with explicit event-based synchronization
if i % 2 == 0:
# Wait for previous stream1 copy to complete (if any)
if sync_event1 is not None:
stream0.wait(sync_event1)
# Copy g0 -> g1 on stream0
g1.copy_from(g0, stream=stream0)
# Record event on stream0 to signal completion of this copy
sync_event0 = stream0.record(options=EventOptions(enable_timing=False))
else:
# Wait for previous stream0 copy to complete
if sync_event0 is not None:
stream1.wait(sync_event0)
# Copy g1 -> g0 on stream1
g0.copy_from(g1, stream=stream1)
# Record event on stream1 to signal completion of this copy
sync_event1 = stream1.record(options=EventOptions(enable_timing=False))
# Wait for last stream1 copy to complete
if sync_event1 is not None:
stream0.wait(sync_event1)
# Record end event on stream0 after all copies have been enqueued
end_event = stream0.record(options=event_options)
end_event.sync()
# Elapsed time in milliseconds (using subtraction operator)
time_memcpy = end_event - start_event
bandwidth = (1.0 / (time_memcpy / 1000.0)) * (100.0 * buf_size) / (1024.0**3)
print(f" P2P bandwidth: {bandwidth:.2f} GB/s")
# Prepare host buffer and initialize data
print(f"\nPreparing host buffer and memcpy to GPU{gpuid[0]}...")
# Create numpy view and initialize
h0_array = np.from_dlpack(h0).view(dtype=np.float32)
h0_array[:] = (np.arange(num_elements, dtype=np.float32) % 4096).astype(
np.float32
)
# Copy to GPU 0
dev0.set_current()
g0.copy_from(h0, stream=stream0)
stream0.sync()
print(" Data initialized and copied to GPU")
# Compile kernel for both GPUs
print("\nCompiling CUDA kernel...")
dev0.set_current()
program_options = ProgramOptions(std="c++17", arch=f"sm_{dev0.arch}")
prog = Program(SIMPLE_P2P_KERNEL, code_type="c++", options=program_options)
mod0 = prog.compile("cubin")
kernel0 = mod0.get_kernel("SimpleKernel")
dev1.set_current()
program_options = ProgramOptions(std="c++17", arch=f"sm_{dev1.arch}")
prog = Program(SIMPLE_P2P_KERNEL, code_type="c++", options=program_options)
mod1 = prog.compile("cubin")
kernel1 = mod1.get_kernel("SimpleKernel")
print(" Kernels compiled successfully")
# Launch configuration
threads = 512
blocks = (num_elements + threads - 1) // threads
config = LaunchConfig(grid=blocks, block=threads)
# Run kernel on GPU 1, reading from GPU 0, writing to GPU 1
print(
f"\nRun kernel on GPU{gpuid[1]}, taking source data from "
f"GPU{gpuid[0]} and writing to GPU{gpuid[1]}..."
)
dev1.set_current()
launch(stream1, config, kernel1, g0, g1, np.int32(num_elements))
stream1.sync()
print(" Kernel execution complete")
# Run kernel on GPU 0, reading from GPU 1, writing to GPU 0
print(
f"\nRun kernel on GPU{gpuid[0]}, taking source data from "
f"GPU{gpuid[1]} and writing to GPU{gpuid[0]}..."
)
dev0.set_current()
launch(stream0, config, kernel0, g1, g0, np.int32(num_elements))
stream0.sync()
print(" Kernel execution complete")
# Copy data back to host and verify
print(f"\nCopy data back to host from GPU{gpuid[0]} and verify results...")
g0.copy_to(h0, stream=stream0)
stream0.sync()
# Verify results
print("\nChecking results...")
print(f" Comparing {num_elements:,} elements...")
# Input data goes through two kernels, each multiplying by 2.0.
expected = (np.arange(num_elements, dtype=np.float32) % 4096) * 4.0
# Use utility function for verification (handles both numpy and cupy arrays)
if verify_array_result(h0_array, expected, rtol=1e-5, atol=1e-6, verbose=True):
print(" [PASS] Validation PASSED")
success = True
else:
print(" [FAIL] Validation FAILED")
# Show first few errors for debugging
errors = np.where(~np.isclose(h0_array, expected, rtol=1e-5, atol=1e-6))[0]
print(f" Number of mismatches: {len(errors)}")
for idx in errors[:10]:
print(
f" Error @ element {idx}: got {h0_array[idx]}, "
f"expected {expected[idx]}"
)
success = False
# Disable peer access
print("\nDisabling peer access...")
mr0.peer_accessible_by = [] # Revoke GPU 1's access to GPU 0's memory
mr1.peer_accessible_by = [] # Revoke GPU 0's access to GPU 1's memory
print(
f" Peer access revoked: MR0 accessible by {mr0.peer_accessible_by}, "
f"MR1 accessible by {mr1.peer_accessible_by}"
)
print("\n" + "=" * 70)
if success:
print("simpleP2P completed successfully!")
else:
print("simpleP2P FAILED!")
print("=" * 70 + "\n")
return 0 if success else 1
finally:
# Cleanup streams and buffers
print("Shutting down...")
stream0.close()
stream1.close()
def main():
"""Main entry point with argument parsing."""
parser = argparse.ArgumentParser(
description=(
"Demonstrate peer-to-peer (P2P) memory access between "
"multiple GPUs with CUDA"
)
)
parser.add_argument(
"--num_elements",
type=int,
default=1024 * 1024 * 16, # 16M elements = 64MB
help="Number of elements in arrays (default: 16777216 = 64MB)",
)
args = parser.parse_args()
# Validate arguments
if args.num_elements <= 0:
print("Error: num_elements must be positive")
return 1
try:
exit_code = run(num_elements=args.num_elements)
sys.exit(exit_code)
except Exception as e:
print(f"\nError: {e}")
import traceback
traceback.print_exc()
sys.exit(1)
if __name__ == "__main__":
main()