CUDA 13.2 samples update (#432)

- Added Python samples for CUDA Python 1.0 release
- Renamed top-level `Samples` directory to `cpp` to accommodate Python samples.
This commit is contained in:
Dheemanth
2026-05-13 15:13:18 -07:00
committed by GitHub
parent a4526d5229
commit aeab82ff30
2072 changed files with 15077 additions and 1970 deletions

View File

@@ -0,0 +1,168 @@
# Sample: Image Blur with Unified Memory (Python)
## Description
Blur images on GPU using modern `cuda.core` APIs for kernel compilation, execution, and memory management. This sample demonstrates **zero-copy data sharing** between CPU and GPU using unified (managed) memory.
## What You'll Learn
- Compiling CUDA kernels at runtime with `cuda.core.Program`
- Launching kernels with `cuda.core.launch` and `LaunchConfig`
- Using unified memory with `cuda.core.ManagedMemoryResource`
- **Zero-copy CPU access** to unified memory via `np.from_dlpack()`
- Seamless CPU/GPU memory access without explicit transfers
## Key Concepts
### Kernel Compilation with cuda.core.Program
```python
# Compile CUDA C++ kernel at runtime
program = Program(KERNEL_CODE, code_type="c++", options=options)
compiled = program.compile(target_type="cubin")
kernel = compiled.get_kernel("box_blur_3x3")
```
### Kernel Launch with cuda.core.launch
```python
# Configure and launch kernel
config = LaunchConfig(grid=grid_size, block=block_size)
# Buffers can be passed directly as kernel arguments
launch(stream, config, kernel, src_buf, dst_buf, H, W)
```
### Unified Memory (Managed Memory)
This sample uses `ManagedMemoryResource` for simplicity: a single allocation is accessible from both CPU and GPU without explicit transfers. For performance-critical workloads, consider `LegacyPinnedMemoryResource` + `DeviceMemoryResource` instead, which gives explicit control over host/device placement and transfer costs.
Unified memory is accessible from both CPU and GPU without explicit data transfers:
```python
# Allocate unified memory
options = ManagedMemoryResourceOptions(preferred_location=device.device_id)
mr = ManagedMemoryResource(options)
src_buf = mr.allocate(n_bytes, stream)
dst_buf = mr.allocate(n_bytes, stream)
try:
# Synchronize to ensure allocations are complete before CPU access
stream.sync()
# Create numpy views of unified memory using DLPack protocol (zero-copy)
src_np = np.from_dlpack(src_buf).view(np.float32).reshape(H, W)
dst_np = np.from_dlpack(dst_buf).view(np.float32).reshape(H, W)
# CPU writes directly to unified memory
src_np[:] = input_data
# Launch kernel - buffers can be passed directly as arguments
launch(stream, config, kernel, src_buf, dst_buf, H, W)
stream.sync()
# Return zero-copy view; caller must close buffers when done
return dst_np, src_buf, dst_buf
except Exception:
src_buf.close()
dst_buf.close()
raise
```
When returning a zero-copy view, the caller must close the buffers after use (e.g., in a `try/finally` block) to avoid leaking managed memory.
## Key APIs
### From `cuda.core`:
- `Device` - CUDA device management
- `Program` - Runtime kernel compilation (NVRTC)
- `ProgramOptions` - Compilation options (architecture target)
- `LaunchConfig` - Kernel launch configuration (grid/block dimensions)
- `launch` - Execute compiled kernel
- `ManagedMemoryResource` - Unified memory allocation
### Zero-Copy Techniques:
- `np.from_dlpack(buffer)` - Create numpy view of unified memory using DLPack protocol
- Pass `buffer` directly to `launch()` as kernel arguments
- When returning a zero-copy view, return `(view, src_buf, dst_buf)` and have the caller close buffers in `try/finally` after use
## Kernel Techniques
- **2D Thread Mapping** - Each thread computes one output pixel
- **Stencil Pattern** - Read neighboring pixels (3x3 neighborhood)
- **Boundary Handling** - Clamp to edge for border pixels
- **Box Filter** - 3x3 averaging for blur effect
## Requirements
### Hardware:
- NVIDIA GPU with CUDA support
- Minimum GPU memory: 256 MB
### Software:
- CUDA Toolkit 13.0 or newer
- Python 3.10 or newer
- `cuda-python` package (13.0.0+)
- `cuda-core` package (>=0.6.0)
- `numpy` package (>=2.3.2)
- `pillow` package (10.0.0+)
## Installation
```bash
cd /path/to/cuda-samples/python/1_GettingStarted/blurImageUnifiedMemory
pip install -r requirements.txt
```
## How to Run
```bash
python blurImageUnifiedMemory.py
```
## Expected Output
```
============================================================
Image Blur with Unified Memory (cuda.core)
============================================================
Device: <Your GPU Name>
Compute Capability: sm_<XY>
Compiling CUDA kernel with cuda.core.Program...
Compiled for architecture: sm_<XY>
Image size: 256x256 grayscale
Creating sample image...
Blurring image on GPU...
Saving results...
Saved: original_image.png
Saved: blurred_image.png
Verifying result...
Test PASSED
Max difference from original: <value>
```
## Output Files
- `original_image.png` - Test pattern image before blur
- `blurred_image.png` - Image after 3x3 box blur
## Files
- `blurImageUnifiedMemory.py` - Python implementation using cuda.core
- `README.md` - This file
- `requirements.txt` - Sample dependencies
## See Also
- [cuda.core Documentation](https://nvidia.github.io/cuda-python/cuda-core/latest/)
- [cuda.core.Program](https://nvidia.github.io/cuda-python/cuda-core/latest/generated/cuda.core.Program.html)
- [cuda.core.ManagedMemoryResource](https://nvidia.github.io/cuda-python/cuda-core/latest/generated/cuda.core.ManagedMemoryResource.html)
- [CUDA Managed Memory](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#um-unified-memory-programming-hd)

View File

@@ -0,0 +1,269 @@
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of NVIDIA CORPORATION nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""
Image Blur with Unified Memory using cuda.core
Demonstrates GPU image blurring using cuda.core APIs for kernel compilation,
launch, and unified memory allocation.
"""
import sys
try:
import numpy as np
from cuda.core import (
Device,
LaunchConfig,
ManagedMemoryResource,
ManagedMemoryResourceOptions,
Program,
ProgramOptions,
launch,
)
from PIL import Image
except ImportError as e:
print(f"Error: Required package not found: {e}")
print("Please install from requirements.txt:")
print(" pip install -r requirements.txt")
sys.exit(1)
# CUDA kernel source code - compiled at runtime by cuda.core.Program
BOX_BLUR_KERNEL_CODE = r"""
extern "C" __global__
void box_blur_3x3(const float* __restrict__ src,
float* __restrict__ dst, int H, int W) {
/*
* Simple 3x3 box blur CUDA kernel.
*
* Each thread computes one output pixel by averaging
* the 3x3 neighborhood of input pixels (stencil pattern).
*/
int x = blockIdx.x * blockDim.x + threadIdx.x;
int y = blockIdx.y * blockDim.y + threadIdx.y;
if (x >= W || y >= H) return;
float sum = 0.0f;
int count = 0;
// 3x3 stencil: iterate over neighborhood
for (int dy = -1; dy <= 1; dy++) {
for (int dx = -1; dx <= 1; dx++) {
int nx = x + dx;
int ny = y + dy;
// Boundary check (clamp to edge)
if (nx >= 0 && nx < W && ny >= 0 && ny < H) {
sum += src[ny * W + nx];
count++;
}
}
}
dst[y * W + x] = sum / count;
}
"""
def make_test_image(h: int, w: int, dtype=np.uint8) -> np.ndarray:
"""Create a test grayscale image for demonstration."""
img = np.zeros((h, w), dtype=dtype)
# Create horizontal stripes
for i in range(0, h, 50):
img[i : i + 25, :] = 255
# Create vertical stripes with different intensity
for j in range(0, w, 50):
img[:, j : j + 25] = 128
# Add circular pattern for interesting blur effects
center_y, center_x = h // 2, w // 2
y, x = np.ogrid[:h, :w]
circle_mask = (x - center_x) ** 2 + (y - center_y) ** 2 <= (min(h, w) // 6) ** 2
img[circle_mask] = 200
return np.ascontiguousarray(img)
def blur_image_unified_memory(
host_np: np.ndarray, device: Device, stream, kernel
) -> tuple[np.ndarray, object, object]:
"""
Blur image on GPU using unified memory with cuda.core.
This function demonstrates:
1. Allocate managed memory using ManagedMemoryResource
2. Create zero-copy numpy views using np.from_dlpack()
3. Launch kernel via cuda.core.launch
Args:
host_np: NumPy array containing image data on CPU
device: CUDA device to use
stream: cuda.core Stream for async operations
kernel: Compiled cuda.core Kernel object
Returns:
Tuple of (dst_np, src_buf, dst_buf). dst_np is a zero-copy view into
unified memory. Caller must close src_buf and dst_buf when done with
dst_np to avoid leaking managed memory.
"""
H, W = host_np.shape
n_bytes = H * W * np.dtype(np.float32).itemsize
# Create managed memory resource for unified memory allocation
options = ManagedMemoryResourceOptions(preferred_location=device.device_id)
mr = ManagedMemoryResource(options)
# Allocate unified memory buffers for source and destination images
src_buf = mr.allocate(n_bytes, stream)
dst_buf = mr.allocate(n_bytes, stream)
try:
# Synchronize to ensure allocations are complete before CPU access
stream.sync()
# Create numpy views of unified memory using DLPack protocol (zero-copy)
src_np = np.from_dlpack(src_buf).view(np.float32).reshape(H, W)
dst_np = np.from_dlpack(dst_buf).view(np.float32).reshape(H, W)
# Write input data to unified memory (CPU can access directly)
src_np[:] = host_np.astype(np.float32) / 255.0
# Configure kernel launch parameters
block_size = (16, 16)
grid_size = (
(W + block_size[0] - 1) // block_size[0],
(H + block_size[1] - 1) // block_size[1],
)
# Create LaunchConfig for kernel execution
config = LaunchConfig(grid=grid_size, block=block_size)
# Launch kernel - buffers can be passed directly as kernel arguments
launch(
stream,
config,
kernel,
src_buf,
dst_buf,
np.int32(H),
np.int32(W),
)
# Synchronize to ensure kernel completion before reading results
stream.sync()
# Return zero-copy view; caller closes buffers when done
return (dst_np, src_buf, dst_buf)
except Exception:
src_buf.close()
dst_buf.close()
raise
def main():
"""
Complete demonstration of GPU image blurring with cuda.core.
This example shows:
1. Device initialization with cuda.core.Device
2. Kernel compilation with cuda.core.Program
3. Unified memory with cuda.core.ManagedMemoryResource
4. Kernel launch with cuda.core.launch and LaunchConfig
"""
print("=" * 60)
print("Image Blur with Unified Memory (cuda.core)")
print("=" * 60)
# Initialize CUDA device
device = Device(0)
device.set_current()
print(f"\nDevice: {device.name}")
print(f"Compute Capability: sm_{device.arch}")
# Create stream for async operations
stream = device.create_stream()
try:
# Compile kernel using cuda.core.Program
print("\nCompiling CUDA kernel with cuda.core.Program...")
arch = f"sm_{device.arch}"
options = ProgramOptions(arch=arch)
program = Program(BOX_BLUR_KERNEL_CODE, code_type="c++", options=options)
compiled = program.compile(target_type="cubin")
kernel = compiled.get_kernel("box_blur_3x3")
print(f" Compiled for architecture: {arch}")
# Image parameters
H, W = 256, 256
print(f"\nImage size: {H}x{W} grayscale")
# Create test image
print("Creating sample image...")
host_np = make_test_image(H, W, dtype=np.uint8)
# Blur image on GPU using cuda.core (returns zero-copy view + buffers)
print("Blurring image on GPU...")
blurred_result, src_buf, dst_buf = blur_image_unified_memory(
host_np, device, stream, kernel
)
try:
# Save images (use zero-copy view before releasing buffers)
print("\nSaving results...")
original_pil = Image.fromarray(host_np, mode="L")
original_pil.save("original_image.png")
print(" Saved: original_image.png")
blurred_uint8 = (np.clip(blurred_result, 0, 1) * 255).astype(np.uint8)
blurred_pil = Image.fromarray(blurred_uint8, mode="L")
blurred_pil.save("blurred_image.png")
print(" Saved: blurred_image.png")
# Verify blur was applied
print("\nVerifying result...")
original_float = host_np.astype(np.float32) / 255.0
max_diff = np.max(np.abs(blurred_result - original_float))
blur_applied = max_diff > 0.01
if blur_applied:
print(" Test PASSED")
else:
print(" Test FAILED - blur not applied")
sys.exit(1)
print(f" Max difference from original: {max_diff:.4f}")
finally:
src_buf.close()
dst_buf.close()
finally:
stream.close()
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,6 @@
# Image Blur with Unified Memory Sample Requirements
cuda-python>=13.0.0
cuda-core>=0.6.0
numpy>=2.3.2
pillow>=10.0.0

View File

@@ -0,0 +1,119 @@
# Sample: Image Array Copy to GPU (Python)
## Description
Copy image arrays between CPU and GPU memory using the modern `cuda.core` API with optimal performance through pinned memory and asynchronous transfers.
## What You'll Learn
- How to use pinned memory for faster CPU↔GPU transfers
- Using the `cuda.core` API for memory management
- Working with DLPack for zero-copy array views
- Performing asynchronous memory transfers with CUDA streams
- Interoperability between CUDA Core API and CuPy
- Proper CUDA resource management and cleanup
## Key Libraries
- `cuda.core` - Modern CUDA Python API
- `numpy` - Array operations and DLPack support
- `cupy` - GPU array operations and CUDA interoperability
## Key APIs
### From `cuda.core`:
- `Device()` - Initialize and access CUDA device
- `Device.set_current()` - Set the current device for API calls
- `Device.create_stream()` - Create CUDA stream for async operations
- `Device.memory_resource` - Access device memory allocator
- `PinnedMemoryResource()` - Allocate pinned host memory
- `buffer.copy_to()` - Copy data between memory spaces
- `buffer.close()` - Release allocated memory
### From `numpy`:
- `np.from_dlpack()` - Create array view from DLPack capsule
- `np.copyto()` - Copy data between arrays
### From `cupy`:
- `cp.from_dlpack()` - Create GPU array view from DLPack capsule
- `cp.cuda.ExternalStream()` - Use external CUDA stream
### From `cuda_samples_utils`:
- `verify_array_result()` - Verify computation results
## Requirements
### Hardware:
- NVIDIA GPU with CUDA support
- Sufficient GPU memory for image data (sample uses ~200KB for 256×256×3 image)
### Software:
- CUDA Toolkit 13.0 or newer
- Python 3.10 or newer
- NumPy 2.3.2 or newer (required for DLPack support)
- `cuda-python` package (>=13.0.0+)
- `cuda-core` package (>=0.6.0)
- `cupy-cuda13x` package (13.0.0+)
## Installation
Install the required packages from requirements.txt:
```bash
cd /path/to/cuda-samples/python/1_GettingStarted/copyImageArraytoGPU
pip install -r requirements.txt
```
The requirements.txt installs:
- `numpy` (2.3.2+, required for DLPack)
- `cuda-python` (>=13.0.0+)
- `cuda-core` (>=0.6.0)
- `cupy-cuda13x` (13.0.0+)
## How to Run
### Basic usage:
```bash
cd samples/python/1_GettingStarted/copyImageArraytoGPU
python copyImageArraytoGPU.py
```
## Expected Output
```
[Image Array Copy to GPU using CUDA Core API]
Device: NVIDIA GeForce RTX 4090
[Image array copy of 256x256x3 image]
Creating sample image...
Copying image to GPU...
Creating CuPy view of GPU data...
Mean pixel value (computed on GPU): 127.50
Copying image back from GPU...
Verifying result...
Test PASSED
Done
```
**Note:** Device name will vary based on your GPU.
## Files
- `copyImageArraytoGPU.py` - Python implementation using cuda.core API
- `README.md` - This file
- `requirements.txt` - Sample dependencies
- `../../Utilities/cuda_samples_utils.py` - Common utilities (imported by this sample)
## See Also
- [CUDA Python Documentation](https://nvidia.github.io/cuda-python/)
- [cuda.core API Guide](https://nvidia.github.io/cuda-python/cuda-core/latest/)
- [DLPack Specification](https://dmlc.github.io/dlpack/latest/)
- [CuPy Documentation](https://docs.cupy.dev/)

View File

@@ -0,0 +1,239 @@
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of NVIDIA CORPORATION nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""
Image Array Copy to GPU using CUDA Core API
This sample demonstrates how to copy image arrays between CPU and GPU memory
using NVIDIA's CUDA Core Python API with optimal performance.
"""
import sys
from pathlib import Path
# Add parent directory to path to import utilities
sys.path.insert(0, str(Path(__file__).parent.parent.parent / "Utilities"))
from cuda_samples_utils import verify_array_result
try:
import cupy as cp
import numpy as np
from cuda.core import Buffer, Device, PinnedMemoryResource, Stream
except ImportError as e:
print(f"Error: Required package not found: {e}")
print("Please install from requirements.txt:")
print(" pip install -r requirements.txt")
sys.exit(1)
# ----------------------------- Helper Functions ------------------------------
def make_random_image(h: int, w: int, c: int, dtype=np.uint8) -> np.ndarray:
"""
Create a random test image for demonstration.
Args:
h: Image height in pixels
w: Image width in pixels
c: Number of channels (e.g., 3 for RGB)
dtype: NumPy data type (e.g., np.uint8 for 0-255 pixel values)
Returns:
A contiguous NumPy array representing the image
"""
img = np.random.randint(0, 256, size=(h, w, c), dtype=dtype)
return np.ascontiguousarray(img) # Ensure memory is contiguous for GPU transfer
# ----------------------------- Core GPU Functions ---------------------------
def copy_image_to_gpu_cuda_core(
host_np: np.ndarray, dev: Device, stream: Stream
) -> tuple[Buffer, Buffer]:
"""
Copy image from CPU memory to GPU memory using optimal transfer method.
This function demonstrates the recommended approach:
1. Use pinned memory for faster transfers
2. Use DLPack for zero-copy array views
3. Perform async transfers on a CUDA stream
Args:
host_np: NumPy array containing image data on CPU
dev: CUDA device object
stream: CUDA stream for async operations
Returns:
Tuple of (device_buffer, pinned_buffer) - both need to be cleaned up later
"""
nbytes = host_np.nbytes # Calculate total bytes needed
# Step 1: Set up memory resources
# Device memory resource - allocates on GPU
device_mr = dev.memory_resource
# Pinned memory resource - allocates CPU memory that GPU can access faster
pinned_mr = PinnedMemoryResource()
# Step 2: Allocate memory buffers
pinned_buffer = pinned_mr.allocate(nbytes, stream=stream) # Fast CPU memory
device_buffer = device_mr.allocate(nbytes, stream=stream) # GPU memory
# Step 3: Create a NumPy view of pinned memory using DLPack
# This allows us to work with pinned memory as if it's a regular NumPy array
pinned_view = (
np.from_dlpack(pinned_buffer).view(dtype=host_np.dtype).reshape(host_np.shape)
)
# Step 4: Copy image data from regular CPU memory to pinned CPU memory
# This is a CPU-to-CPU copy, so it's very fast
np.copyto(pinned_view, host_np)
# Step 5: Copy from pinned CPU memory to GPU memory
# This is the actual CPU-to-GPU transfer, done asynchronously
pinned_buffer.copy_to(device_buffer, stream=stream)
return device_buffer, pinned_buffer
def copy_image_from_gpu_cuda_core(
device_buffer: Buffer, shape: tuple, dtype: type, dev: Device, stream: Stream
) -> np.ndarray:
"""
Copy image from GPU memory back to CPU memory.
This function reverses the GPU-to-CPU transfer process:
1. Allocate pinned CPU memory for fast transfer
2. Copy from GPU to pinned CPU memory
3. Create NumPy view and copy to regular CPU memory
Args:
device_buffer: GPU buffer containing image data
shape: Original image shape tuple (height, width, channels)
dtype: Original image data type
dev: CUDA device object
stream: CUDA stream for async operations
Returns:
NumPy array with image data copied from GPU
"""
nbytes = np.prod(shape) * np.dtype(dtype).itemsize # Calculate total bytes
# Step 1: Create pinned memory for fast GPU-to-CPU transfer
pinned_mr = PinnedMemoryResource()
pinned_buffer = pinned_mr.allocate(nbytes, stream=stream)
# Step 2: Copy from GPU memory to pinned CPU memory
device_buffer.copy_to(pinned_buffer, stream=stream)
stream.sync() # Wait for the GPU transfer to complete
# Step 3: Create NumPy view of pinned memory using DLPack
pinned_view = np.from_dlpack(pinned_buffer).view(dtype=dtype).reshape(shape)
# Step 4: Copy from pinned CPU memory to regular CPU memory
# This creates the final result that can be used normally
host_result = pinned_view.copy()
# Step 5: Clean up the temporary pinned buffer
pinned_buffer.close(stream)
return host_result
# ------------------------------ Main Demo ------------------------------------
def main():
"""
Complete demonstration of GPU image copying workflow.
This example shows:
1. Setting up CUDA device and stream
2. Creating a sample image
3. Copying image to GPU
4. Accessing GPU data with CuPy (optional)
5. Copying image back from GPU
6. Verifying data integrity
7. Proper cleanup of resources
"""
print("[Image Array Copy to GPU using CUDA Core API]")
# Image parameters - modify these to test different sizes
H, W, C = 256, 256, 3 # Height=256, Width=256, Channels=3 (RGB)
dtype = np.uint8 # Standard image pixel type (0-255 values)
# Step 1: Set up CUDA device and stream
dev = Device() # Get default CUDA device (GPU 0)
dev.set_current() # Make this device the active one
stream = dev.create_stream() # Create stream for async operations
print(f"Device: {dev.name}")
print(f"[Image array copy of {H}x{W}x{C} image]")
# Step 2: Configure CuPy to use our CUDA stream (for interoperability)
cp.cuda.ExternalStream(int(stream.handle)).use()
# Step 3: Create a test image on CPU
print("Creating sample image...")
host_np = make_random_image(H, W, C, dtype=dtype)
# Step 4: Copy image from CPU to GPU
print("Copying image to GPU...")
device_buffer, pinned_buffer = copy_image_to_gpu_cuda_core(host_np, dev, stream)
# Step 5: (Optional) Get a CuPy view of GPU data for processing
# This shows how you can work with the GPU data without copying it back
print("Creating CuPy view of GPU data...")
device_cp = cp.from_dlpack(device_buffer).view(dtype=dtype).reshape(H, W, C)
# Example: compute mean pixel value on GPU
mean_value = float(cp.mean(device_cp))
print(f"Mean pixel value (computed on GPU): {mean_value:.2f}")
# Step 6: Copy image back from GPU to CPU
print("Copying image back from GPU...")
host_back = copy_image_from_gpu_cuda_core(
device_buffer, host_np.shape, host_np.dtype, dev, stream
)
# Step 7: Verify that the data survived the round trip
print("Verifying result...")
host_back_cp = cp.asarray(host_back)
host_np_cp = cp.asarray(host_np)
verify_array_result(host_back_cp, host_np_cp, rtol=0, atol=0)
# Step 8: Clean up all allocated resources
device_buffer.close(stream) # Free GPU memory
pinned_buffer.close(stream) # Free pinned CPU memory
stream.close() # Close CUDA stream
cp.cuda.Stream.null.use() # Reset CuPy's stream to default
print("\nDone")
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,6 @@
# Image Array Copy to GPU Sample Requirements
numpy>=2.3.2
cuda-python>=13.0.0
cuda-core>=0.6.0
cupy-cuda13x>=13.0.0

View File

@@ -0,0 +1,189 @@
# Sample: Device Query (Python)
## Description
Query and display detailed properties of all CUDA-capable devices in your system using the modern `cuda.core` API.
## What You'll Learn
- How to enumerate CUDA devices in the system
- Using the `cuda.core` API for device management
- Querying comprehensive device properties (compute capability, memory, limits)
- Accessing low-level device attributes via `cuda.bindings`
- Checking peer-to-peer (P2P) access capabilities between GPUs
## Key Libraries
- `cuda.core` - Modern CUDA Python API
- `cuda.bindings` - Low-level CUDA bindings for runtime and driver APIs
## Key APIs
### From `cuda.core`:
- `Device.get_all_devices()` - Get tuple of all available Device instances
- `Device(device_id)` - Get Device object for specific device ID
- `system.get_driver_version()` - Query CUDA driver version
- `Device.set_current()` - Set the current device for API calls
- `Device.properties` - Access comprehensive device properties
- `Device.name` - Get device name string
- `Device.can_access_peer()` - Check P2P access to peer device
### From `cuda.bindings.runtime`:
- `cudart.cudaRuntimeGetVersion()` - Get CUDA runtime version
- `cudart.cudaDeviceGetAttribute()` - Query specific device attributes
### From `cuda.bindings.driver`:
- `cuda.cuMemGetInfo()` - Get memory information for current device
## Device Properties Queried
### Compute Capabilities:
- Compute capability version (major.minor)
- Driver and runtime versions
- Number of multiprocessors and CUDA cores
### Memory Information:
- Total global memory
- Memory clock rate and bus width
- L2 cache size
- Constant and shared memory sizes
- Maximum memory pitch
### Execution Configuration Limits:
- Maximum threads per block and per multiprocessor
- Maximum block dimensions (x, y, z)
- Maximum grid dimensions (x, y, z)
- Warp size
- Registers per block
### Texture Capabilities:
- Maximum texture dimensions (1D, 2D, 3D)
- Maximum layered texture sizes
### Feature Support:
- Unified Addressing (UVA)
- Managed Memory
- Compute Preemption
- Cooperative Kernel Launch
- ECC support
- Host page-locked memory mapping
- Concurrent copy and kernel execution
### System Information:
- PCI bus information
- Compute mode
- Driver mode (Windows only)
- P2P access matrix (multi-GPU systems)
## Requirements
### Hardware:
- NVIDIA GPU with CUDA support (any compute capability)
- No specific GPU memory requirement (query only)
### Software:
- CUDA Toolkit 13.0 or newer (recommended; matches `cuda-python` 13.x)
- Python 3.10 or newer
- `cuda-python` package (>=13.0.0)
- `cuda-core` package (>=0.6.0)
## Installation
Install the required packages from requirements.txt:
```bash
cd cuda-samples/python/1_GettingStarted/deviceQuery
pip install -r requirements.txt
```
The requirements.txt installs:
- `cuda-python` (>=13.0.0)
- `cuda-core` (>=0.6.0)
## How to Run
### Basic usage:
```bash
cd cuda-samples/python/1_GettingStarted/deviceQuery
python deviceQuery.py
```
### Skip P2P information:
```bash
python deviceQuery.py --no-p2p
```
## Expected Output
```
[CUDA Device Query using CUDA Core API]
Detected 1 CUDA Capable device(s)
Device 0: <Your GPU Name>
CUDA Driver Version / Runtime Version 12.4 / 12.6
CUDA Capability Major/Minor version number: 8.9
Total amount of global memory: 24217 MBytes (25393954816 bytes)
(132) Multiprocessors, (128) CUDA Cores/MP: 16896 CUDA Cores
GPU Max Clock rate: 1980 MHz (1.98 GHz)
Memory Clock rate: 10501 Mhz
Memory Bus Width: 384-bit
L2 Cache Size: 67108864 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
Maximum Layered 1D Texture Size, (num) layers 1D=(32768), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(32768, 32768), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total shared memory per multiprocessor: 102400 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 1536
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 2 copy engine(s)
Run time limit on kernels: Yes
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Device has ECC support: Enabled
Device supports Unified Addressing (UVA): Yes
Device supports Managed Memory: Yes
Device supports Compute Preemption: Yes
Supports Cooperative Kernel Launch: Yes
Device PCI Domain ID / Bus ID / location ID: 0 / 1 / 0
Compute Mode:
< Default (multiple host threads can use cudaSetDevice() with device simultaneously) >
Done
```
**Note:** Output will vary based on your specific GPU model and system configuration.
For multi-GPU systems, the output will include information for all detected devices and a P2P access matrix showing which GPUs can directly access each other's memory.
## Files
- `deviceQuery.py` - Python implementation using cuda.core API
- `requirements.txt` - Sample dependencies
## Use Cases
- **System Diagnostics** - Verify CUDA installation and GPU detection
- **Hardware Profiling** - Understand GPU capabilities before optimization
- **Multi-GPU Systems** - Identify P2P topology for optimal data placement
- **Kernel Development** - Determine execution configuration limits
- **Compatibility Checks** - Verify compute capability requirements
## See Also
- [CUDA Python Documentation](https://nvidia.github.io/cuda-python/)
- [cuda.core API Guide](https://nvidia.github.io/cuda-python/cuda-core/latest/)
- [CUDA Programming Guide - Device Information](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#device-enumeration)

View File

@@ -0,0 +1,389 @@
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of NVIDIA CORPORATION nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""
Device Query using CUDA Core API
This sample enumerates the properties of the CUDA devices present in the system.
"""
import platform
import sys
# cuda.bindings used for properties not yet exposed in cuda.core (see comments below)
try:
from cuda.bindings import driver as cuda, runtime as cudart
from cuda.core import Device, system
except ImportError as e:
print(f"Error: Required package not found: {e}")
print("Please install from requirements.txt:")
print(" pip install -r requirements.txt")
sys.exit(1)
def print_property(label, value, indent=2):
"""
Helper function to print device properties with aligned formatting.
Parameters
----------
label : str
Property label
value : any
Property value
indent : int
Number of spaces for indentation (default: 2)
"""
field_width = 47
spaces = " " * indent
print(f"{spaces}{label:<{field_width}}{value}")
def fmt_bytes(size_in_bytes):
"""Format bytes to human-readable string with MBytes."""
return f"{size_in_bytes / (1024 * 1024):.0f} MBytes ({size_in_bytes} bytes)"
def fmt_hz(rate_in_khz):
"""Format frequency in kHz to MHz and GHz."""
return f"{rate_in_khz * 1e-3:.0f} MHz ({rate_in_khz * 1e-6:.2f} GHz)"
def fmt_yes_no(val):
"""Format boolean value to Yes/No string."""
return "Yes" if val else "No"
def convert_sm_ver_to_cores(major, minor):
"""
Maps SM version to the number of CUDA cores per SM.
Information taken from:
https://github.com/NVIDIA/cuda-samples/blob/master/Common/helper_cuda.h
Parameters
----------
major : int
Major compute capability version
minor : int
Minor compute capability version
Returns
-------
int
Number of CUDA cores per SM, or 0 if unknown
"""
sm_to_cores = {
(3, 0): 192,
(3, 2): 192,
(3, 5): 192,
(3, 7): 192,
(5, 0): 128,
(5, 2): 128,
(5, 3): 128,
(6, 0): 64,
(6, 1): 128,
(6, 2): 128,
(7, 0): 64,
(7, 2): 64,
(7, 5): 64,
(8, 0): 64,
(8, 6): 128,
(8, 7): 128,
(8, 9): 128,
(9, 0): 128,
(10, 0): 128,
(10, 1): 128,
(10, 3): 128,
(11, 0): 128,
(12, 0): 128,
(12, 1): 128,
}
return sm_to_cores.get((major, minor), 0)
def print_device_info(dev_id, device):
"""
Print detailed information for a single CUDA device.
Uses device.properties (cuda.core) for most fields; cuda.bindings for
runtime version and global memory (not yet in high-level API).
"""
device.set_current()
props = device.properties
print()
print(f"Device {dev_id}: {device.name}")
# cuda.bindings workaround: runtime version not in cuda.core
driver_major, driver_minor = system.get_driver_version()
err, runtime_version = cudart.cudaRuntimeGetVersion()
if err != cudart.cudaError_t.cudaSuccess:
raise RuntimeError(f"Failed to get CUDA runtime version: {err}")
runtime_major = runtime_version // 1000
runtime_minor = (runtime_version % 1000) // 10
print_property(
"CUDA Driver Version / Runtime Version",
f"{driver_major}.{driver_minor} / {runtime_major}.{runtime_minor}",
)
print_property(
"CUDA Capability Major/Minor version number:",
f"{props.compute_capability_major}.{props.compute_capability_minor}",
)
# cuda.bindings workaround: global memory (free/total) not in device.properties
err, free_mem, total_mem_bytes = cuda.cuMemGetInfo()
if err != cuda.CUresult.CUDA_SUCCESS:
raise RuntimeError(f"Failed to get memory info: {err}")
print_property("Total amount of global memory:", fmt_bytes(total_mem_bytes))
sm_cores = convert_sm_ver_to_cores(
props.compute_capability_major, props.compute_capability_minor
)
total_cores = sm_cores * props.multiprocessor_count
print_property(
f"({props.multiprocessor_count:3d}) Multiprocessors, "
f"({sm_cores:3d}) CUDA Cores/MP:",
f"{total_cores} CUDA Cores",
)
print_property("GPU Max Clock rate:", fmt_hz(props.clock_rate))
print_property("Memory Clock rate:", f"{props.memory_clock_rate * 1e-3:.0f} Mhz")
print_property("Memory Bus Width:", f"{props.global_memory_bus_width}-bit")
if props.l2_cache_size > 0:
print_property("L2 Cache Size:", f"{props.l2_cache_size} bytes")
print_property(
"Maximum Texture Dimension Size (x,y,z)",
f"1D=({props.maximum_texture1d_width}), "
f"2D=({props.maximum_texture2d_width}, {props.maximum_texture2d_height}), "
f"3D=({props.maximum_texture3d_width}, {props.maximum_texture3d_height}, "
f"{props.maximum_texture3d_depth})",
)
print_property(
"Maximum Layered 1D Texture Size, (num) layers",
f"1D=({props.maximum_texture1d_layered_width}), "
f"{props.maximum_texture1d_layered_layers} layers",
)
print_property(
"Maximum Layered 2D Texture Size, (num) layers",
f"2D=({props.maximum_texture2d_layered_width}, "
f"{props.maximum_texture2d_layered_height}), "
f"{props.maximum_texture2d_layered_layers} layers",
)
print_property(
"Total amount of constant memory:", f"{props.total_constant_memory} bytes"
)
print_property(
"Total amount of shared memory per block:",
f"{props.max_shared_memory_per_block} bytes",
)
print_property(
"Total shared memory per multiprocessor:",
f"{props.max_shared_memory_per_multiprocessor} bytes",
)
print_property(
"Total number of registers available per block:", props.max_registers_per_block
)
print_property("Warp size:", props.warp_size)
print_property(
"Maximum number of threads per multiprocessor:",
props.max_threads_per_multiprocessor,
)
print_property("Maximum number of threads per block:", props.max_threads_per_block)
print_property(
"Max dimension size of a thread block (x,y,z):",
f"({props.max_block_dim_x}, {props.max_block_dim_y}, {props.max_block_dim_z})",
)
print_property(
"Max dimension size of a grid size (x,y,z):",
f"({props.max_grid_dim_x}, {props.max_grid_dim_y}, {props.max_grid_dim_z})",
)
print_property("Maximum memory pitch:", f"{props.max_pitch} bytes")
print_property("Texture alignment:", f"{props.texture_alignment} bytes")
print_property(
"Concurrent copy and kernel execution:",
f"{fmt_yes_no(props.gpu_overlap)} with "
f"{props.async_engine_count} copy engine(s)",
)
print_property("Run time limit on kernels:", fmt_yes_no(props.kernel_exec_timeout))
print_property("Integrated GPU sharing Host Memory:", fmt_yes_no(props.integrated))
print_property(
"Support host page-locked memory mapping:",
fmt_yes_no(props.can_map_host_memory),
)
print_property(
"Device has ECC support:", "Enabled" if props.ecc_enabled else "Disabled"
)
if platform.system() == "Windows":
mode = (
"TCC (Tesla Compute Cluster Driver)"
if props.tcc_driver
else "WDDM (Windows Display Driver Model)"
)
print_property("CUDA Device Driver Mode (TCC or WDDM):", mode)
print_property(
"Device supports Unified Addressing (UVA):",
fmt_yes_no(props.unified_addressing),
)
print_property("Device supports Managed Memory:", fmt_yes_no(props.managed_memory))
print_property(
"Device supports Compute Preemption:",
fmt_yes_no(props.compute_preemption_supported),
)
print_property(
"Supports Cooperative Kernel Launch:", fmt_yes_no(props.cooperative_launch)
)
print_property(
"Device PCI Domain ID / Bus ID / location ID:",
f"{props.pci_domain_id} / {props.pci_bus_id} / {props.pci_device_id}",
)
compute_modes = {
0: (
"Default (multiple host threads can use cudaSetDevice() "
"with device simultaneously)"
),
1: (
"Exclusive (only one host thread in one process is able to "
"use cudaSetDevice() with this device)"
),
2: "Prohibited (no host thread can use cudaSetDevice() with this device)",
3: (
"Exclusive Process (many threads in one process is able to "
"use cudaSetDevice() with this device)"
),
}
print_property("Compute Mode:", "")
print(f" < {compute_modes.get(props.compute_mode, 'Unknown')} >")
def print_p2p_access_info(devices):
"""
Print peer-to-peer access information for multi-GPU systems.
Parameters
----------
devices : tuple of Device
Tuple of CUDA device objects
"""
print()
print("Peer-to-Peer (P2P) access support:")
for i, dev_i in enumerate(devices):
for j, dev_j in enumerate(devices):
if i == j:
continue
try:
can_access = dev_i.can_access_peer(dev_j)
print(
f"> Peer access from {dev_i.name} (GPU{i}) -> "
f"{dev_j.name} (GPU{j}) : {fmt_yes_no(can_access)}"
)
except Exception as e:
print(
"Warning: Could not check peer access between "
f"device {i} and {j}: {e}"
)
def query_devices(show_p2p=True):
"""
Query and display information about all CUDA devices.
Parameters
----------
show_p2p : bool
Whether to show peer-to-peer access information (default: True)
Returns
-------
bool
True if successful, False otherwise
"""
try:
print("[CUDA Device Query using CUDA Core API]")
devices = Device.get_all_devices()
except Exception as e:
print(f"Error: Failed to get devices: {e}")
import traceback
traceback.print_exc()
return False
if len(devices) == 0:
print("There are no available device(s) that support CUDA")
return True
print(f"Detected {len(devices)} CUDA Capable device(s)")
for dev_id, device in enumerate(devices):
try:
print_device_info(dev_id, device)
except Exception as e:
print(f"Error: Failed to get information for device {dev_id}: {e}")
import traceback
traceback.print_exc()
return False
if show_p2p and len(devices) >= 2:
print_p2p_access_info(devices)
return True
def main():
"""
Main entry point for the device query sample.
"""
import argparse
parser = argparse.ArgumentParser(
description="Query CUDA Device Properties using CUDA Core API",
formatter_class=argparse.RawDescriptionHelpFormatter,
)
parser.add_argument(
"--no-p2p", action="store_true", help="Skip peer-to-peer access information"
)
args = parser.parse_args()
success = query_devices(show_p2p=not args.no_p2p)
if success:
print("\nDone")
return 0
else:
return 1
if __name__ == "__main__":
sys.exit(main())

View File

@@ -0,0 +1,4 @@
# Device Query Sample Requirements
cuda-python>=13.0.0
cuda-core>=0.6.0

View File

@@ -0,0 +1,72 @@
# Sample: Kernel Nsys Profiling - CUDA C++ Kernel Profiling with cuda.core (Python)
## Description
This sample demonstrates how to profile custom CUDA C++ kernels compiled and launched with `cuda.core` using NVIDIA Nsight Systems. It implements three GPU operations (vector addition, SAXPY, vector transform) as custom kernels and shows how to instrument code with NVTX markers for profiling analysis.
## What you will learn
- How to write and compile CUDA C++ kernels with `cuda.core.Program`
- How to launch kernels with `LaunchConfig` and manage CUDA streams
- How to use NVTX markers (`nvtx.annotate()`) to annotate code sections
- How to profile kernels with Nsight Systems and analyze performance
- Modern CUDA Python workflow with `cuda.core.Device` and proper resource cleanup
## Requirements
- NVIDIA GPU with Compute Capability 7.0+
- CUDA Toolkit 13.0+
- Python 3.10+
- Packages: `numpy`, `cuda-python`, `cuda-core`, `cupy-cuda13x`, `nvtx` (see `requirements.txt`; NumPy >=2.3.2)
**Install:**
```bash
pip install -r requirements.txt
```
## How to run
```bash
python kernelNsysProfile.py
python kernelNsysProfile.py --array-size 10000000 # Custom size
```
## Nsys Profiling
**Basic profile:**
```bash
nsys profile -o gpu_profile python kernelNsysProfile.py
nsys-ui gpu_profile.nsys-rep # View results
```
The program uses color-coded NVTX markers:
- **Purple**: Phase 2 (cuda.core Custom Kernels - main focus)
- **Yellow/Blue/Green**: Other phases
- **Cyan**: Nested operations
Focus on Phase 2 to analyze kernel execution times, launch overhead, and GPU utilization.
**For detailed Nsys usage and analysis techniques, see the [NVIDIA Nsight Systems documentation](https://docs.nvidia.com/nsight-systems/).**
## Troubleshooting
**Missing packages:**
```bash
pip install -r requirements.txt
```
**Out of memory:**
```bash
python kernelNsysProfile.py -n 10000000 # Reduce array size
```
**Nsys not found:**
```bash
export PATH=/usr/local/cuda/bin:$PATH
```
## See Also
- [CUDA Python Documentation](https://nvidia.github.io/cuda-python/)
- [NVIDIA Nsight Systems Documentation](https://docs.nvidia.com/nsight-systems/)
- [CuPy Documentation](https://docs.cupy.dev/)

View File

@@ -0,0 +1,327 @@
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# distribution and/or other materials provided with the distribution.
# * Neither the name of NVIDIA CORPORATION nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""
Kernel Nsys Profiling Sample - CUDA C++ Kernel Profiling with cuda.core
This sample demonstrates how to profile custom CUDA C++ kernels compiled and
launched with cuda.core using NVIDIA Nsight Systems.
The sample implements three common GPU operations as custom CUDA C++ kernels:
- Vector addition: c = a + b
- SAXPY: y = alpha * x + y
- Vector transform: sqrt(x*x + 1) + sin(x)
Use Nsight Systems to analyze:
- Custom kernel execution times
- Kernel launch patterns and overhead
- GPU utilization and memory access patterns
- NVTX markers for structured profiling
Workflow:
- Phase 1: Create GPU arrays
- Phase 2: Compile and execute cuda.core custom kernels (profiling focus)
- Phase 3: Verify correctness with CuPy reference implementation
- Phase 4: Validate results
"""
import argparse
import sys
from pathlib import Path
try:
import cupy as cp
import numpy as np
import nvtx
from cuda.core import Device, LaunchConfig, launch
except ImportError as e:
print(f"Error: Required package not found: {e}")
print("Please install from requirements.txt:")
print(" pip install -r requirements.txt")
sys.exit(1)
# Add parent directory to path to import utilities
sys.path.insert(0, str(Path(__file__).parent.parent.parent / "Utilities"))
from cuda_samples_utils import verify_array_result # noqa: E402
# CUDA C++ kernel definitions
# For larger projects, separating kernels into a separate file is also valid.
KERNELS_CODE = """
template<typename T>
__global__ void vector_add(const T* a, const T* b, T* c, size_t N) {
const unsigned int tid = threadIdx.x + blockIdx.x * blockDim.x;
for (size_t i = tid; i < N; i += gridDim.x * blockDim.x) {
c[i] = a[i] + b[i];
}
}
template<typename T>
__global__ void saxpy(const T alpha, const T* x, T* y, size_t N) {
const unsigned int tid = threadIdx.x + blockIdx.x * blockDim.x;
for (size_t i = tid; i < N; i += gridDim.x * blockDim.x) {
y[i] = alpha * x[i] + y[i];
}
}
template<typename T>
__global__ void vector_transform(const T* a, T* b, size_t N) {
const unsigned int tid = threadIdx.x + blockIdx.x * blockDim.x;
for (size_t i = tid; i < N; i += gridDim.x * blockDim.x) {
T val = a[i];
b[i] = sqrt(val * val + T(1.0)) + sin(val);
}
}
"""
def get_cuda_core_kernels(device):
"""
Compile cuda.core kernels and return them.
Args:
device: cuda.core.Device object
Returns:
dict: Dictionary of compiled kernels
"""
from cuda.core import Program, ProgramOptions
# Compile all kernels at once
program_options = ProgramOptions(std="c++17", arch=f"sm_{device.arch}")
prog = Program(KERNELS_CODE, code_type="c++", options=program_options)
mod = prog.compile(
"cubin",
name_expressions=(
"vector_add<float>",
"saxpy<float>",
"vector_transform<float>",
),
)
# Extract individual kernels
return {
"vector_add": mod.get_kernel("vector_add<float>"),
"saxpy": mod.get_kernel("saxpy<float>"),
"vector_transform": mod.get_kernel("vector_transform<float>"),
}
def run(size):
"""Main execution function"""
# =================================================================
# Device Initialization using cuda.core
# =================================================================
with nvtx.annotate("Device Initialization", color="green"):
try:
# Create device object (defaults to device 0)
dev = Device()
dev.set_current()
print()
print(f"Device: {dev.name}")
print(f"Compute Capability: sm_{dev.arch}")
print()
# Synchronize device
dev.sync()
except Exception as e:
print("ERROR: CUDA initialization failed!")
print(f"Error: {e}")
sys.exit(1)
print("Profiling cuda.core Custom Kernels")
print(f"Array size: {size:,}\n")
# Constant for SAXPY operation
alpha = 2.5
# Initialize random seed
rng = cp.random.default_rng(42)
# =================================================================
# Phase 1: Create GPU Arrays with CuPy
# =================================================================
with nvtx.annotate("Create GPU Arrays", color="yellow"):
a_gpu = rng.standard_normal(size, dtype=cp.float32)
b_gpu = rng.standard_normal(size, dtype=cp.float32)
dev.sync()
print("Phase 1: Created arrays on GPU with CuPy")
print(f" Array shape: {a_gpu.shape}")
print(f" Array dtype: {a_gpu.dtype}")
print(
f" Array a - Mean: {float(cp.mean(a_gpu)):.4f}, "
f"Std: {float(cp.std(a_gpu)):.4f}"
)
print(
f" Array b - Mean: {float(cp.mean(b_gpu)):.4f}, "
f"Std: {float(cp.std(b_gpu)):.4f}\n"
)
# =================================================================
# Phase 2: cuda.core Custom Kernels on GPU
# =================================================================
with nvtx.annotate("cuda.core Custom Kernels", color="purple"):
print("Phase 2: cuda.core custom CUDA C++ kernels on GPU")
# Create a stream for cuda.core operations
stream = dev.create_stream()
try:
with nvtx.annotate("Compile Kernels", color="cyan"):
kernels_dict = get_cuda_core_kernels(dev)
stream.sync()
print("Compiled custom CUDA C++ kernels")
# Prepare launch configuration
# Grid-stride loops in kernels handle any grid size robustly
block = 256
grid = (size + block - 1) // block
config = LaunchConfig(grid=grid, block=block)
# Execute cuda.core vector_add kernel
with nvtx.annotate("Vector Add (cuda.core)", color="cyan"):
c_cuda = cp.empty_like(a_gpu)
launch(
stream,
config,
kernels_dict["vector_add"],
a_gpu.data.ptr,
b_gpu.data.ptr,
c_cuda.data.ptr,
cp.uint64(size),
)
stream.sync()
# Execute cuda.core SAXPY kernel
with nvtx.annotate("SAXPY (cuda.core)", color="cyan"):
y_cuda = b_gpu.copy()
launch(
stream,
config,
kernels_dict["saxpy"],
np.float32(alpha),
a_gpu.data.ptr,
y_cuda.data.ptr,
cp.uint64(size),
)
stream.sync()
# Execute cuda.core vector_transform kernel
with nvtx.annotate("Vector Transform (cuda.core)", color="cyan"):
transform_cuda = cp.empty_like(a_gpu)
launch(
stream,
config,
kernels_dict["vector_transform"],
a_gpu.data.ptr,
transform_cuda.data.ptr,
cp.uint64(size),
)
stream.sync()
print("Vector Addition (custom kernel)")
print("SAXPY (custom kernel)")
print("Vector Transform (custom kernel)\n")
finally:
stream.close()
# =================================================================
# Phase 3: Generate Reference Results with CuPy (for verification)
# =================================================================
with nvtx.annotate("Generate Reference Results", color="blue"):
print("Phase 3: Generate reference results for verification")
with nvtx.annotate("Vector Add (Reference)", color="cyan"):
c_cupy = a_gpu + b_gpu
dev.sync()
with nvtx.annotate("SAXPY (Reference)", color="cyan"):
y_cupy = alpha * a_gpu + b_gpu
dev.sync()
with nvtx.annotate("Vector Transform (Reference)", color="cyan"):
transform_cupy = cp.sqrt(a_gpu * a_gpu + 1.0) + cp.sin(a_gpu)
dev.sync()
print("Reference results generated\n")
# =================================================================
# Phase 4: Verify Kernel Correctness
# =================================================================
with nvtx.annotate("Verification", color="green"):
print("Phase 4: Verify kernel correctness")
# Verify custom kernels against reference results
# Use relaxed tolerances for single-precision float comparisons
# Small differences can occur due to instruction ordering and
# compiler optimizations
print(" Validating cuda.core kernels:")
print(" Vector Add: ", end="")
vec_add_match = verify_array_result(c_cuda, c_cupy, rtol=1e-5, atol=1e-6)
print(" SAXPY: ", end="")
saxpy_match = verify_array_result(y_cuda, y_cupy, rtol=1e-5, atol=1e-6)
print(" Transform: ", end="")
transform_match = verify_array_result(
transform_cuda, transform_cupy, rtol=1e-5, atol=1e-6
)
all_pass = vec_add_match and saxpy_match and transform_match
if not all_pass:
print("\n ERROR: Kernel verification failed!")
return 1
print()
# Final synchronization
dev.sync()
print("The sample is complete PASSED!")
def main():
parser = argparse.ArgumentParser(
description="Kernel Nsys Profiling - Profile custom CUDA C++ "
"kernels with cuda.core"
)
parser.add_argument(
"-n",
"--array-size",
type=int,
default=50000,
metavar="N",
help="Array size (default: 50,000)",
)
args = parser.parse_args()
run(size=args.array_size)
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,7 @@
# Nsight System Kernels Profiling Sample - Requirements
numpy>=2.3.2
cuda-python>=13.0.0
cuda-core>=0.6.0
cupy-cuda13x>=13.0.0
nvtx

View File

@@ -0,0 +1,73 @@
# Sample: Numpy vs. Cupy (Python)
## Description
This sample demonstrates performance comparison between NumPy (CPU) and CuPy (GPU) for matrix multiplication operations. It benchmarks the execution time of matrix dot products on both CPU and GPU, showing the performance benefits of GPU acceleration for numerical computations.
## What you will learn
- How to set up and use CuPy for GPU-accelerated numerical computations.
- How to benchmark NumPy vs CuPy performance for matrix operations.
- How to transfer data between CPU (NumPy) and GPU (CuPy) memory using `cp.asarray()`.
- How to use CUDA device management with the cuda-core library.
- How to validate computational results between CPU and GPU implementations using `np.testing.assert_allclose()`.
- How to handle GPU warmup to avoid first-run overhead in benchmarking.
- How to create and manage explicit CUDA streams with `device.create_stream()`.
- How to properly cleanup streams with `stream.close()` in try/finally blocks.
- How to access GPU device information (name, compute capability).
- How to create timing context managers for performance measurement using CUDA events.
## Key libraries
- `numpy`
- `cupy`
- `cuda-core`
## Key APIs
**From cuda.core:**
- `Device()` – Get CUDA device object for specific GPU
- `device.create_stream()` – Create explicit CUDA stream
- `stream.close()` – Close and cleanup stream resources
## Requirements
1. **NVIDIA Graphics Card** with CUDA support
2. **CUDA Drivers** installed on your system
3. **CUDA Toolkit** installed on your system
4. **Python 3.12 or newer**
**Install packages:**
```bash
pip install -r requirements.txt
```
## How to run
Basic usage:
```bash
# Pre-steps:
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
# Run from the Python directory:
cd /path/to/numpyVsCupy/Python
python -m 1_GettingStarted.numpyVsCupy.numpyVsCupy
```
With custom parameters:
```bash
python -m 1_GettingStarted.numpyVsCupy.numpyVsCupy --n_size 5000
```
### Command line arguments
- `--n_size`, `-n`: Size of the matrix (n * n) for benchmarking (default: 4096)
## Expected Output
```
Validation PASSED: NumPy and CuPy results match within tolerance
Demo completed successfully!
```
## Files
- `numpyVsCupy.py` – Python implementation
- `README.md` – This file
- `requirements.txt` – Required packages

View File

@@ -0,0 +1,141 @@
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# distribution and/or other materials provided with the distribution.
# * Neither the name of NVIDIA CORPORATION nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
import argparse
import contextlib
import sys
import time
from pathlib import Path
try:
import cupy as cp
import numpy as np
from cuda.core import Device, EventOptions
except ImportError as e:
print(f"Error: Required package not found: {e}")
print("Please install from requirements.txt:")
print(" pip install -r requirements.txt")
sys.exit(1)
# Add parent directory to path to import utilities
sys.path.insert(0, str(Path(__file__).parent.parent.parent / "Utilities"))
from cuda_samples_utils import verify_array_result
@contextlib.contextmanager
def timer(message):
"""CPU timing context manager."""
start = time.time()
yield
end = time.time()
print(f"{message}: {(end - start):.6f} seconds")
@contextlib.contextmanager
def gpu_timer(message, stream):
"""GPU timing context manager using cuda.core CUDA events."""
event_options = EventOptions(enable_timing=True)
start_event = stream.record(options=event_options)
yield
end_event = stream.record(options=event_options)
end_event.sync()
elapsed_time_ms = end_event - start_event # Returns milliseconds
elapsed_time_s = elapsed_time_ms / 1000.0 # Convert to seconds
print(f"{message}: {elapsed_time_s:.6f} seconds")
def warmup():
# Pre-runs a simple GPU operation to avoid first-run overhead in benchmarking.
print("Warmup...")
a_cp = cp.ones((16, 16))
b_cp = cp.ones((16, 16))
result_cp = cp.dot(a_cp, b_cp)
return result_cp
def run(n):
# Benchmarks NumPy vs. CuPy matrix multiplication for n x n random arrays.
# Prints timing results.
device = Device() # Use device 0 explicitly
device.set_current()
major, minor = device.compute_capability
print()
print(f"Device Name: {device.name}, SM: {major}.{minor}")
print()
# Create explicit stream
stream = device.create_stream()
try:
# Warm up GPU before measuring
warmup()
stream.sync()
# Generate random matrices on CPU
a_np = np.random.rand(n, n)
b_np = np.random.rand(n, n)
# NumPy dot product (CPU)
with timer(f"NumPy dot of {n}*{n} arrays"):
result_np = np.dot(a_np, b_np)
# Transfer NumPy arrays to GPU (using events for timing)
with gpu_timer("Transfer arrays to GPU", stream):
a_cp = cp.asarray(a_np)
b_cp = cp.asarray(b_np)
# CuPy dot product (GPU) - using events for accurate GPU timing
with gpu_timer(f"CuPy dot of {n}*{n} arrays", stream):
result_cp = cp.dot(a_cp, b_cp)
print()
# Result validation
if not verify_array_result(result_np, result_cp.get()):
print(
"Validation FAILED: NumPy and CuPy results do not match "
"within tolerance"
)
sys.exit(1)
print("Validation PASSED: NumPy and CuPy results match within tolerance")
finally:
stream.close()
def main():
parser = argparse.ArgumentParser()
parser.add_argument(
"--n_size", "-n", default=4096, type=int, help="Size of the matrix(n * n)."
)
args = parser.parse_args()
run(args.n_size)
print("Demo completed successfully!")
if __name__ == "__main__":
main()

View File

@@ -0,0 +1,7 @@
# Numpy vs. Cupy - Requirements
# Install with: pip install -r requirements.txt
numpy>=2.3.2
cuda-python>=13.0.0
cuda-core>=0.6.0
cupy-cuda13x>=13.0.0

View File

@@ -0,0 +1,263 @@
# simplePrint - Printing from CUDA Kernels
## Description
This sample demonstrates how to use `printf()` inside CUDA kernels using **two different approaches**:
1. **CUDA C++ kernels** compiled with `cuda.core.Program` - Full C++ features and control
2. **Numba CUDA kernels** - Pythonic kernel authoring using `numba.cuda.grid()` for modern indexing
The sample shows basic device management, kernel compilation with inline CUDA C++ code, and multi-dimensional kernel launches (2D grid × 3D blocks) using modern CUDA Python. The Numba example demonstrates the recommended `numba.cuda.grid()` indexing style while also showing how it relates to classic CUDA C++ block/thread IDs. Both approaches use `cuda.core` APIs for stream management and synchronization, demonstrating interoperability.
This is the Python equivalent of the C++ `simplePrintf` sample, enhanced with Numba CUDA examples.
## Key Concepts
CUDA Python (cuda.core), Numba CUDA, Kernel Compilation, Printf in Kernels, Multi-dimensional Launch, Pythonic GPU Programming, Modern Thread Indexing (grid()), Stream-based Execution, cuda.core/Numba Interoperability
## CUDA APIs involved
### [cuda.core (cuda-python)](https://nvidia.github.io/cuda-python/)
- `Device()` - Device management
- `Device.create_stream()` - Create CUDA streams
- `Stream.sync()` - Synchronize stream execution
- `Program()` - Compile CUDA C++ kernels
- `LaunchConfig()` - Configure kernel launch
- `launch()` - Execute kernels on streams
### [Numba CUDA](https://nvidia.github.io/numba-cuda/)
- `@cuda.jit` - JIT compile Python functions to CUDA kernels
- `cuda.grid()` - Get global thread position (recommended modern approach)
- `cuda.blockIdx`, `cuda.threadIdx` - Thread/block indices (classic style)
- `cuda.gridDim`, `cuda.blockDim` - Grid/block dimensions
- **Note:** Uses `cuda.core` APIs for stream management (interoperability)
### CUDA Kernel Functions
- `printf()` - Print from device code (C++)
- `print()` - Print from device code (Numba, limited formatting)
- `blockIdx`, `threadIdx` - Thread/block indices
- `gridDim`, `blockDim` - Grid/block dimensions
### What You Learn
- Device initialization with `cuda.core.Device`
- Compiling CUDA C++ kernels with `Program` and `ProgramOptions`
- Writing Pythonic CUDA kernels with Numba's `@cuda.jit` decorator
- Using `numba.cuda.grid()` for modern thread indexing (recommended approach)
- Understanding the relationship between global coordinates and classic block/thread IDs
- **Interoperability**: Using `cuda.core` streams with Numba CUDA kernels
- Comparing CUDA C++ vs Pythonic kernel authoring approaches
- Multi-dimensional kernel launches (2D grid, 3D blocks)
- Using streams for kernel execution and synchronization
- Using `printf()` and `print()` in GPU kernels for debugging
- Understanding print limitations in Numba CUDA (no f-strings)
- Proper error handling and resource management
## Requirements
### Hardware:
- NVIDIA GPU with Compute Capability 7.0 or higher
- Minimum GPU memory: 512 MB
### Software:
- CUDA Toolkit 13.0 or newer
- Python 3.10 or newer
- `cuda-python` package (13.0+)
- `cuda-core` package (>=0.6.0)
- `numba-cuda` package (0.24.0+, for Pythonic kernel authoring)
Download and install:
- [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads)
- [cuda-python package](https://nvidia.github.io/cuda-python/): `pip install cuda-python`
- [numba-cuda](https://nvidia.github.io/numba-cuda/): `pip install numba-cuda`
## Build and Run
```bash
# Install dependencies
pip install -r requirements.txt
# Run the sample
python simplePrint.py
```
## Expected Output
```
Simple Print - Printing from CUDA Kernels
Demonstrating both CUDA C++ and Numba CUDA approaches
Device: <Your GPU Name>
Compute Capability: sm_<XX>
======================================================================
METHOD 1: CUDA C++ Kernel (via cuda.core.Program)
======================================================================
Advantage: Full C++ features, better for complex kernels
Compiling CUDA C++ kernel...
Kernel compiled successfully.
Kernel configuration:
Grid: (2, 2)
Block: (2, 2, 2)
Total threads: 32
Launching kernel with value=10. Output:
[0, 0]: Value is: 10
[0, 1]: Value is: 10
[0, 2]: Value is: 10
[0, 3]: Value is: 10
[0, 4]: Value is: 10
[0, 5]: Value is: 10
[0, 6]: Value is: 10
[0, 7]: Value is: 10
[1, 0]: Value is: 10
...
[3, 7]: Value is: 10
CUDA C++ kernel execution complete.
======================================================================
METHOD 2: Numba CUDA Kernel (Pythonic / modern indexing)
======================================================================
Advantage: Uses numba.cuda.grid(3) for global indexing,
while still showing classic CUDA C++ IDs for reference.
Uses cuda.core for stream management (interoperability).
Kernel configuration:
Grid: (2, 2)
Block: (2, 2, 2)
Total threads: 32
Launching Numba CUDA kernel (grid(3) + classic IDs) with value=10:
Uses numba.cuda.grid(3) to get global (x, y, z),
and prints the corresponding blockId/threadId like the C++ sample.
Stream managed by cuda.core for consistency with C++ example.
global[ 0 , 0 , 0 ] -> [ 0 , 0 ]: Value is: 10
global[ 1 , 0 , 0 ] -> [ 0 , 1 ]: Value is: 10
global[ 0 , 1 , 0 ] -> [ 0 , 2 ]: Value is: 10
...
global[ 3 , 3 , 1 ] -> [ 3 , 7 ]: Value is: 10
Numba CUDA kernel execution complete.
======================================================================
Done! Both kernel approaches demonstrated successfully.
======================================================================
```
## Understanding the Output
- **Grid**: 2×2 = 4 blocks (labeled 0-3)
- **Block**: 2×2×2 = 8 threads per block (labeled 0-7)
- **Total**: 32 threads, each printing its position and value
### CUDA C++ Kernel:
Each thread calculates:
- Block ID (linear): `blockIdx.y * gridDim.x + blockIdx.x`
- Thread ID (linear): `threadIdx.z * blockDim.x * blockDim.y + threadIdx.y * blockDim.x + threadIdx.x`
### Numba CUDA Kernel:
Each thread shows:
- **Global position** using `numba.cuda.grid(3)` → `(x, y, z)` coordinates across entire grid
- **Classic IDs** (block ID, thread ID) calculated the same way as C++ for comparison
- This demonstrates how modern indexing relates to traditional CUDA C++ style
### Comparing the Two Approaches
**CUDA C++ Kernel (Method 1):**
- Uses C++ syntax and `printf()` with full formatting control
- Requires compilation via `cuda.core.Program`
- Best for complex kernels needing C++ features (templates, libraries, etc.)
- Uses classic block/thread ID indexing
- Output: `[0, 0]: Value is: 10` (clean formatting)
**Numba CUDA Kernel (Method 2):**
- Uses Python syntax with `@cuda.jit` decorator
- JIT compiled automatically when called
- Best for prototyping and simpler kernels
- **Modern indexing**: Uses `numba.cuda.grid(3)` to get global thread coordinates (recommended)
- Also shows classic block/thread IDs to help relate the two indexing models
- **Interoperability**: Uses `cuda.core` streams via `stream` for consistency
- Demonstrates that numba-cuda kernels can work seamlessly with cuda.core infrastructure
- Limited print formatting (no f-strings, basic `print()` only; adds spaces between arguments)
- Output: `global[ 0 , 0 , 0 ] -> [ 0 , 0 ]: Value is: 10` (shows both indexing styles; note extra spaces due to `print()` behavior)
## Experiments
Try modifying:
### For Both Approaches:
- **Grid size**: Change `grid=(4, 4)` for 16 blocks
- **Block size**: Change `block=(4, 4, 4)` for 64 threads per block
- **Conditional printing**: Print only from specific threads (e.g., `if threadId == 0:`)
### CUDA C++ Specific:
- **Format strings**: Experiment with different `printf()` formats
- **Kernel code**: Add complex C++ computations before printing
- **External libraries**: Include CUDA math libraries or device functions (e.g., `<cuda/std/cmath>`, `<cub/cub.cuh>`)
### Numba CUDA Specific:
- **Grid indexing**: Try `numba.cuda.grid(1)` or `numba.cuda.grid(2)` for different dimensions
- **Conditional printing**: Print only from threads where `x == 0` or `y == z`
- **Python operations**: Use NumPy-like operations in the kernel
- **Device math libraries**: Use [nvmath-python device APIs](https://docs.nvidia.com/cuda/nvmath-python/latest/device-apis/index.html) for optimized math operations (similar to CUDA math libraries in C++)
- **Shared memory**: Add `numba.cuda.shared.array()` for fast inter-thread communication
- **Atomic operations**: Try `numba.cuda.atomic.add()` for thread-safe updates
- **Print variations**: Experiment with what numba-cuda's `print()` can and cannot handle
- **Streams**: Create multiple `cuda.core` streams and launch numba-cuda kernels on them concurrently
- **Interoperability**: Mix numba-cuda kernels and CUDA C++ kernels on the same stream
## Notes
### General:
- Printing from GPU is relatively slow - use sparingly in production code
- Printf output is buffered and limited (~1MB buffer on most GPUs)
### CUDA C++ Kernels:
- Always call `stream.sync()` after kernel launch to flush printf output
- Full `printf()` format string support (%, flags, width, precision)
### Numba CUDA Kernels:
- **Recommended**: Use `numba.cuda.grid(ndim)` for thread indexing (modern, Pythonic)
- `grid(1)` for 1D indexing, `grid(2)` for 2D, `grid(3)` for 3D
- Returns global thread position across the entire grid
- **Interoperability**: Use `cuda.core` streams with Numba kernels via `stream`
- Create streams: `stream = device.create_stream()`
- Launch kernels: `kernel[grid, block, stream](args)`
- Synchronize: `stream.sync()`
- Numba's `print()` has limited capabilities compared to Python's `print()`
- F-strings are NOT supported in Numba CUDA kernels
- Use comma-separated arguments: `print("Value:", x)` instead of f-strings
- **Note**: `print()` automatically adds spaces between comma-separated arguments (e.g., `print("[", x, "]")` outputs `[ 0 ]` not `[0]`)
- Always synchronize the stream to flush output
## Files
- `simplePrint.py` - Python implementation using cuda.core API
- `README.md` - This file
- `requirements.txt` - Sample dependencies
## See Also
### CUDA Python (cuda.core):
- [cuda.core Documentation](https://nvidia.github.io/cuda-python/)
- [CUDA Python Examples](https://github.com/NVIDIA/cuda-python/tree/main/cuda_core/examples)
### Numba CUDA:
- [Numba CUDA Documentation](https://nvidia.github.io/numba-cuda/)
- [numba.cuda.grid() Reference](https://nvidia.github.io/numba-cuda/reference/kernel.html#numba.cuda.grid)
- [nvmath-python Device APIs](https://docs.nvidia.com/cuda/nvmath-python/latest/device-apis/index.html) - Optimized math operations for Numba CUDA kernels
### CUDA References:
- [CUDA C Programming Guide - Printf](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#formatted-output)
- [C++ simplePrintf Sample](https://github.com/NVIDIA/cuda-samples/tree/master/Samples/0_Introduction/simplePrintf)

View File

@@ -0,0 +1,7 @@
# Simple Printf Sample - Requirements
cuda-python>=13.0.0
cuda-core>=0.6.0
# Numba JIT uses nvJitLink from pip; keep in step with cuda-bindings (e.g. 13.2.x).
nvidia-nvjitlink>=13.2.0
numba-cuda>=0.24.0

View File

@@ -0,0 +1,287 @@
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of NVIDIA CORPORATION nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""
Simple Print - Printing from CUDA Kernels
This sample demonstrates how to print output from CUDA kernels using printf().
It shows:
1. Device management with cuda.core.Device
2. Compiling CUDA C++ code that uses printf()
3. Launching kernels with 2D grids and 3D blocks
4. Seeing kernel output printed to stdout
5. Using Numba CUDA for Pythonic kernel authoring
This sample demonstrates both approaches:
- CUDA C++ kernels compiled via cuda.core.Program (more control, C++ features)
- Numba CUDA kernels (more Pythonic, easier to write)
This is the Python equivalent of the C++ simplePrintf sample.
"""
import sys
import traceback
try:
from cuda.core import Device, LaunchConfig, Program, ProgramOptions, launch
except ImportError as e:
print(f"Error: Required package not found: {e}")
print("Please install from requirements.txt:")
print(" pip install -r requirements.txt")
sys.exit(1)
try:
from numba import cuda as numba_cuda
NUMBA_AVAILABLE = True
except ImportError:
NUMBA_AVAILABLE = False
print("Warning: numba not found. Numba CUDA example will be skipped.")
print("To install: pip install numba")
# CUDA C++ kernel with printf
# This kernel prints the block index, thread index, and a value from each thread
PRINTF_KERNEL = """
extern "C"
__global__ void printKernel(int val) {
// Calculate linear block index from 2D grid
int blockId = blockIdx.y * gridDim.x + blockIdx.x;
// Calculate linear thread index from 3D block
int threadId = threadIdx.z * blockDim.x * blockDim.y +
threadIdx.y * blockDim.x +
threadIdx.x;
// Print from each thread
printf("[%d, %d]:\\t\\tValue is: %d\\n", blockId, threadId, val);
}
"""
# Numba CUDA kernel - Pythonic equivalent using numba.cuda.grid()
# This demonstrates the same functionality using Numba's Python-based kernel syntax
if NUMBA_AVAILABLE:
@numba_cuda.jit
def numba_print_kernel(val):
"""
Numba CUDA kernel showing the *recommended* grid() indexing style,
while also relating it to the classic CUDA C++ blockId/threadId.
- Primary view: global 3D coordinates from numba.cuda.grid(3)
(modern, Pythonic way to index work for a 3D thread layout).
- Secondary view: linear blockId / threadId matching the CUDA C++
printf sample, to help CUDA C++ users connect the two models.
"""
# Modern / recommended view: global 3D thread coordinates
x, y, z = numba_cuda.grid(3)
# Classic CUDA-style indices, same formulas as the C++ sample
block_id = numba_cuda.blockIdx.y * numba_cuda.gridDim.x + numba_cuda.blockIdx.x
thread_id = (
numba_cuda.threadIdx.z * numba_cuda.blockDim.x * numba_cuda.blockDim.y
+ numba_cuda.threadIdx.y * numba_cuda.blockDim.x
+ numba_cuda.threadIdx.x
)
# Print both views side-by-side
# Note: Numba print() adds spaces between comma-separated args
print(
"global[",
x,
",",
y,
",",
z,
"] -> [",
block_id,
",",
thread_id,
"]:\t\tValue is:",
val,
)
def run_cuda_cpp_kernel(device, test_value=10):
"""
Demonstrate printing from CUDA C++ kernel compiled with cuda.core.
This approach gives you full access to CUDA C++ features and allows
for more complex kernel implementations.
"""
print("=" * 70)
print("METHOD 1: CUDA C++ Kernel (via cuda.core.Program)")
print("=" * 70)
print("Advantage: Full C++ features, better for complex kernels")
print()
# Compile the kernel
print("Compiling CUDA C++ kernel...")
program_options = ProgramOptions(std="c++17", arch=f"sm_{device.arch}")
prog = Program(PRINTF_KERNEL, code_type="c++", options=program_options)
mod = prog.compile("cubin", name_expressions=("printKernel",))
kernel = mod.get_kernel("printKernel")
print("Kernel compiled successfully.\n")
# Create stream for kernel execution
stream = device.create_stream()
# Configure kernel launch
# Using 2D grid (2x2) and 3D blocks (2x2x2)
grid_x, grid_y = 2, 2
block_x, block_y, block_z = 2, 2, 2
print("Kernel configuration:")
print(f" Grid: ({grid_x}, {grid_y})")
print(f" Block: ({block_x}, {block_y}, {block_z})")
print(f" Total threads: {grid_x * grid_y * block_x * block_y * block_z}")
print()
# Launch configuration with 2D grid and 3D block
config = LaunchConfig(grid=(grid_x, grid_y), block=(block_x, block_y, block_z))
print(f"Launching kernel with value={test_value}. Output:\n")
try:
# Launch kernel
launch(stream, config, kernel, test_value)
# Synchronize to ensure printf output is flushed
stream.sync()
print("\nCUDA C++ kernel execution complete.")
except Exception as e:
print(f"\nError during kernel execution: {e}")
traceback.print_exc()
return 1
finally:
# Cleanup
stream.close()
return 0
def run_numba_kernel(device, test_value=10):
"""
Demonstrate printing from a Numba CUDA kernel.
This example uses numba.cuda.grid(3) as the primary indexing mechanism
(recommended modern style), and also prints the equivalent blockId /
threadId used in the CUDA C++ printf sample for side-by-side comparison.
Uses cuda.core APIs for stream management, demonstrating interoperability
between Numba CUDA kernels and cuda.core infrastructure.
"""
print("\n")
print("=" * 70)
print("METHOD 2: Numba CUDA Kernel (Pythonic / modern indexing)")
print("=" * 70)
print("Advantage: Uses numba.cuda.grid(3) for global indexing,")
print(" while still showing classic CUDA C++ IDs for reference.")
print(" Uses cuda.core for stream management (interoperability).")
print()
# Same launch configuration as the C++ version
grid_x, grid_y = 2, 2
block_x, block_y, block_z = 2, 2, 2
print("Kernel configuration:")
print(f" Grid: ({grid_x}, {grid_y})")
print(f" Block: ({block_x}, {block_y}, {block_z})")
print(f" Total threads: {grid_x * grid_y * block_x * block_y * block_z}")
print()
# Use cuda.core stream (same as C++ example) instead of numba.cuda.stream()
stream = device.create_stream()
print(f"Launching Numba kernel (grid(3) + classic IDs) with value={test_value}:")
print("Uses numba.cuda.grid(3) to get global (x, y, z),")
print("and prints the corresponding blockId/threadId like the C++ sample.")
print("Stream managed by cuda.core for consistency with C++ example.\n")
try:
# Launch Numba kernel on cuda.core stream
numba_print_kernel[(grid_x, grid_y), (block_x, block_y, block_z), stream](
test_value
)
# Synchronize cuda.core stream (same as C++ example)
stream.sync()
print("\nNumba CUDA kernel execution complete.")
except Exception as e:
print(f"\nError during Numba kernel execution: {e}")
traceback.print_exc()
return 1
finally:
# Cleanup
stream.close()
return 0
def main():
"""Main function demonstrating printing from CUDA kernels using both approaches"""
print("Simple Print - Printing from CUDA Kernels")
print("Demonstrating both CUDA C++ and Numba CUDA approaches")
print()
# Initialize device
device = Device()
device.set_current()
# Get device properties
print(f"Device: {device.name}")
print(f"Compute Capability: sm_{device.arch}")
print()
# Value to pass to both kernels
test_value = 10
# Run CUDA C++ kernel
result = run_cuda_cpp_kernel(device, test_value)
if result != 0:
return result
# Run Numba kernel if available
if NUMBA_AVAILABLE:
result = run_numba_kernel(device, test_value)
if result != 0:
return result
else:
print("\n" + "=" * 70)
print("Numba CUDA example skipped (numba not installed)")
print("To run the Numba example: pip install numba")
print("=" * 70)
print("\n" + "=" * 70)
print("Done! Both kernel approaches demonstrated successfully.")
print("=" * 70)
return 0
if __name__ == "__main__":
sys.exit(main())

View File

@@ -0,0 +1,152 @@
# Sample: System Information Query (Python)
## Description
This sample demonstrates how to inspect the CUDA driver, NVML, and every
installed GPU through the
[`cuda.core.system`](https://nvidia.github.io/cuda-python/cuda-core/latest/)
module.
`cuda.core.system` wraps the NVIDIA Management Library (NVML) and can be
imported without CUDA being installed or initialized, so it is useful as a
lightweight pre-flight check before any CUDA context is created. The script
prints driver and NVML versions, the current process name, per-device
metadata (name, compute capability, architecture, memory, PCI info,
temperature, performance state), and, on multi-GPU systems, the topology
and peer-to-peer capabilities between each pair of devices.
## What You'll Learn
- Querying CUDA driver and NVML versions with `cuda.core.system`
- Enumerating GPUs without creating a CUDA context
- Reading per-device metadata exposed by NVML (name, UUID, memory usage,
temperature, performance state)
- Inspecting GPU-to-GPU topology and peer-to-peer (P2P) capabilities
## Key Libraries
- [`cuda.core.system`](https://nvidia.github.io/cuda-python/cuda-core/latest/) - Python wrapper around NVML
## Key APIs
From `cuda.core.system`:
- `get_driver_version()`, `get_driver_version_full()`, `get_driver_branch()` - CUDA driver version tuple and branch string
- `get_nvml_version()` - NVML library version
- `get_num_devices()` - number of GPUs visible to NVML
- `get_process_name(pid)` - process name for a given PID
- `Device(index=...)` - NVML-backed device handle (no CUDA context required)
- `name`, `uuid`, `cuda_compute_capability`, `arch`, `brand`
- `memory_info` (`total`, `used`, `free`)
- `pci_info` (`domain`, `bus`, `device`, `bus_id`)
- `temperature.sensor(TemperatureSensors.TEMPERATURE_GPU)`
- `performance_state`
- `get_topology_common_ancestor(dev0, dev1)` - `GpuTopologyLevel` between two devices
- `get_p2p_status(dev0, dev1, GpuP2PCapsIndex.P2P_CAPS_INDEX_READ)` - peer-access capability between two devices
Import stable symbols from the top-level `cuda.core` package (not `cuda.core.experimental`).
## Requirements
1. **NVIDIA Graphics Card** with CUDA support
2. **CUDA Drivers** installed on your system
3. **CUDA Toolkit** installed on your system
4. **Python 3.12 or newer**
### Hardware
- One or more NVIDIA GPUs
- Driver compatible with `cuda-python` 13.x
### Software
- CUDA Toolkit 13.0 or newer (matches `cuda-python` 13.x)
- Python 3.10 or newer
- `cuda-python` (>=13.0.0)
- `cuda-core` (>=0.6.0)
## Installation
Install the required packages from `requirements.txt`:
```bash
cd /path/to/cuda-samples/python/1_GettingStarted/systemInfo
pip install -r requirements.txt
```
The `requirements.txt` installs:
- `cuda-python` (>=13.0.0)
- `cuda-core` (>=0.6.0)
## How to Run
### Basic usage
```bash
cd cuda-samples/python/1_GettingStarted/systemInfo
python systemInfo.py
```
### Skip topology queries
Useful on machines with only one GPU or to shorten the output:
```bash
python systemInfo.py --no-topology
```
## Expected Output
Output varies with your hardware. On a machine with two GPUs you should see
something like:
```
======================================================================
Driver / NVML
======================================================================
CUDA driver version: 13.2
CUDA driver version (full): (13, 2, 0)
NVML version: (13, 595, 58, 3)
Driver branch: r595_88
Current process: /usr/bin/python
======================================================================
Devices detected: 2
======================================================================
-- Device 0 --
Name: <Your GPU Name>
UUID: ...
Compute capability: 8.9
Architecture: ADA
Brand: BRAND_GEFORCE
Memory: total=23.99 GiB, used=960.00 KiB, free=23.52 GiB
PCI: domain=0000 bus=41 device=00 id=00000000:41:00.0
Temperature (GPU sensor): 47 C
Performance state: <Pstates.PSTATE_8: 8>
...
======================================================================
GPU topology and peer-to-peer
======================================================================
Device 0 <-> Device 1: topology=TOPOLOGY_HOSTBRIDGE, p2p_read=..., p2p_write=...
Done
```
**Note:** Device names, compute capability, temperatures, and topology
details will vary based on your GPUs and system.
## Files
- `systemInfo.py` - Python implementation using `cuda.core.system`
- `README.md` - This file
- `requirements.txt` - Sample dependencies
## See Also
- [CUDA Python Documentation](https://nvidia.github.io/cuda-python/)
- [`cuda.core.system` API reference](https://nvidia.github.io/cuda-python/cuda-core/latest/api.html)
- [NVML reference](https://docs.nvidia.com/deploy/nvml-api/)

View File

@@ -0,0 +1,4 @@
# System Information Sample Requirements
cuda-python>=13.0.0
cuda-core>=0.6.0

View File

@@ -0,0 +1,215 @@
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of NVIDIA CORPORATION nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""
System Information via cuda.core.system (NVML)
Demonstrates the ``cuda.core.system`` module, which wraps NVIDIA Management
Library (NVML) functionality.
This sample prints:
* Driver and NVML versions
* Current process name
* Per-device: name, UUID, compute capability / arch, PCI info, memory usage,
temperature, performance state
* GPU-to-GPU topology and peer-to-peer status (when more than one GPU)
"""
import os
import sys
try:
from cuda.core import system
from cuda.core.system import (
CUDA_BINDINGS_NVML_IS_COMPATIBLE,
GpuP2PCapsIndex,
TemperatureSensors,
)
except ImportError as e:
print(f"Error: Required package not found: {e}")
print("Please install from requirements.txt:")
print(" pip install -r requirements.txt")
sys.exit(1)
def print_header(title: str) -> None:
print()
print("=" * 70)
print(title)
print("=" * 70)
def format_bytes(nbytes: int) -> str:
"""Format a byte count as a human-readable string."""
units = ["B", "KiB", "MiB", "GiB", "TiB"]
size = float(nbytes)
for unit in units:
if size < 1024.0:
return f"{size:.2f} {unit}"
size /= 1024.0
return f"{size:.2f} PiB"
def print_driver_info() -> None:
print_header("Driver / NVML")
major, minor = system.get_driver_version()
print(f"CUDA driver version: {major}.{minor}")
print(f"CUDA driver version (full): {system.get_driver_version_full()}")
if CUDA_BINDINGS_NVML_IS_COMPATIBLE:
print(f"NVML version: {system.get_nvml_version()}")
try:
print(f"Driver branch: {system.get_driver_branch()}")
except Exception as e: # noqa: BLE001 - driver branch is informational
print(f"Driver branch: unavailable ({e})")
else:
print(
"NVML bindings are not compatible with this driver; "
"device info will be limited."
)
print(f"Current process: {system.get_process_name(os.getpid())}")
def print_device_info(device: "system.Device") -> None:
print(f"\n-- Device {device.index} --")
print(f"Name: {device.name}")
print(f"UUID: {device.uuid}")
try:
cc_major, cc_minor = device.cuda_compute_capability
print(f"Compute capability: {cc_major}.{cc_minor}")
except Exception as e: # noqa: BLE001
print(f"Compute capability: unavailable ({e})")
try:
print(f"Architecture: {device.arch.name}")
except Exception as e: # noqa: BLE001
print(f"Architecture: unavailable ({e})")
try:
print(f"Brand: {device.brand.name}")
except Exception as e: # noqa: BLE001
print(f"Brand: unavailable ({e})")
# Memory
try:
mem = device.memory_info
print(
f"Memory: total={format_bytes(mem.total)}, "
f"used={format_bytes(mem.used)}, "
f"free={format_bytes(mem.free)}"
)
except Exception as e: # noqa: BLE001
print(f"Memory: unavailable ({e})")
# PCI
try:
pci = device.pci_info
print(
f"PCI: domain={pci.domain:04x} bus={pci.bus:02x} "
f"device={pci.device:02x} id={pci.bus_id}"
)
except Exception as e: # noqa: BLE001
print(f"PCI: unavailable ({e})")
# Temperature (GPU sensor)
try:
temp_c = device.temperature.sensor(TemperatureSensors.TEMPERATURE_GPU)
print(f"Temperature (GPU sensor): {temp_c} C")
except Exception as e: # noqa: BLE001
print(f"Temperature: unavailable ({e})")
# Performance state
try:
pstate = device.performance_state
print(f"Performance state: {pstate}")
except Exception as e: # noqa: BLE001
print(f"Performance state: unavailable ({e})")
def print_topology(devices: list) -> None:
if len(devices) < 2:
return
print_header("GPU topology and peer-to-peer")
for i, d0 in enumerate(devices):
for d1 in devices[i + 1 :]:
try:
level = system.get_topology_common_ancestor(d0, d1)
level_name = level.name
except Exception as e: # noqa: BLE001
level_name = f"unavailable ({e})"
try:
read = system.get_p2p_status(
d0, d1, GpuP2PCapsIndex.P2P_CAPS_INDEX_READ
)
write = system.get_p2p_status(
d0, d1, GpuP2PCapsIndex.P2P_CAPS_INDEX_WRITE
)
read_name = read.name
write_name = write.name
except Exception as e: # noqa: BLE001
read_name = write_name = f"unavailable ({e})"
print(
f"Device {d0.index} <-> Device {d1.index}: "
f"topology={level_name}, p2p_read={read_name}, p2p_write={write_name}"
)
def main() -> int:
import argparse
parser = argparse.ArgumentParser(
description="Print CUDA system / NVML information via cuda.core.system"
)
parser.add_argument(
"--no-topology",
action="store_true",
help="Skip cross-device topology/P2P queries",
)
args = parser.parse_args()
print_driver_info()
num_devices = system.get_num_devices()
print_header(f"Devices detected: {num_devices}")
if num_devices == 0:
print("No CUDA-capable devices found.")
return 0
if not CUDA_BINDINGS_NVML_IS_COMPATIBLE:
print(
"NVML is not compatible with the installed driver; skipping device detail."
)
return 0
devices = [system.Device(index=i) for i in range(num_devices)]
for device in devices:
print_device_info(device)
if not args.no_topology:
print_topology(devices)
print("\nDone")
return 0
if __name__ == "__main__":
sys.exit(main())

View File

@@ -0,0 +1,130 @@
# Sample: Vector Addition (Python)
## Description
Run your first GPU kernel: add two vectors element-wise on the GPU using the [`cuda.core`](https://nvidia.github.io/cuda-python/cuda-core/latest/) API with runtime compilation.
## What You'll Learn
- Writing CUDA kernels in C++ with template support
- Runtime compilation of CUDA kernels from Python
- Using `cuda.core` for device management, programs, and launches
- Configuring and launching kernels with grid and block dimensions
- Using CuPy for GPU memory management
- Verifying GPU results against CPU computation
## Key Libraries
- [`cuda.core`](https://nvidia.github.io/cuda-python/cuda-core/latest/) — Pythonic access to CUDA runtime and compilation
- `cupy` — GPU array library for Python
## Key APIs
### From `cuda.core`
- `Device` — Initialize and manage CUDA device
- `Program` — Create program from kernel source code
- `ProgramOptions` — Set compilation options (C++ standard, architecture)
- `LaunchConfig` — Configure kernel launch parameters
- `launch` — Execute kernel on specified stream
Import stable symbols from the top-level package (not `cuda.core.experimental`). See the [cuda.core documentation](https://nvidia.github.io/cuda-python/cuda-core/latest/).
### From CuPy
- `cp.random.rand()` — Generate random arrays on GPU
- `cp.empty()` — Allocate uninitialized GPU arrays
- `cp.allclose()` — Verify results with tolerance
### From `cuda_samples_utils`
- `verify_array_result()` — Verify computation results
## Kernel Techniques
- **1D Grid-Stride Loop** — Handle arbitrary array sizes with fixed grid
- **Template Programming** — Generic kernel for different data types
- **Bounds Checking** — Prevent out-of-bounds memory access
## Requirements
### Hardware
- NVIDIA GPU with Compute Capability 7.0 or higher
- Minimum GPU memory: 512 MB
### Software
- CUDA Toolkit 13.0 or newer (matches `cuda-python` 13.x)
- Python 3.10 or newer
- `cuda-python` (>=13.0.0)
- `cuda-core` (>=0.6.0)
- `cupy-cuda13x` (>=13.0.0)
## Installation
Install the required packages from requirements.txt:
```bash
cd /path/to/cuda-samples/python/1_GettingStarted/vectorAdd
pip install -r requirements.txt
```
The requirements.txt installs:
- `cuda-python` (>=13.0.0)
- `cuda-core` (>=0.6.0)
- `cupy-cuda13x` (>=13.0.0)
## How to Run
### Basic usage
```bash
cd samples/python/1_GettingStarted/vectorAdd
python vectorAdd.py
```
### With custom parameters
```bash
# Custom vector size
python vectorAdd.py --elements 1000000
# Use specific GPU
python vectorAdd.py --device 1
# Skip verification for benchmarking
python vectorAdd.py --no-verify
```
## Expected Output
```
[Vector addition using CUDA Core API]
Device: <Your GPU Name>
Compute Capability: sm_<XX>
Compiling kernel 'vectorAdd<float>'...
Kernel compiled successfully
[Vector addition of 50000 elements]
CUDA kernel launch with 196 blocks of 256 threads
Verifying result...
Test PASSED
Done
```
**Note:** Device name and compute capability will vary based on your GPU.
## Files
- `vectorAdd.py` — Python implementation using cuda.core API
- `README.md` — This file
- `requirements.txt` — Sample dependencies
- `../../Utilities/cuda_samples_utils.py` — Common utilities (imported by this sample)
## See Also
- [CUDA Python Documentation](https://nvidia.github.io/cuda-python/)
- [cuda.core API](https://nvidia.github.io/cuda-python/cuda-core/latest/)
- [CuPy Documentation](https://docs.cupy.dev/)

View File

@@ -0,0 +1,5 @@
# Vector Addition Sample Requirements
cuda-python>=13.0.0
cuda-core>=0.6.0
cupy-cuda13x>=13.0.0

View File

@@ -0,0 +1,196 @@
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of NVIDIA CORPORATION nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""
Vector Addition using CUDA Core API
This sample demonstrates element-wise vector addition: C = A + B
using cuda.core for runtime compilation and kernel launch.
"""
import sys
from pathlib import Path
# Add parent directory to path to import utilities
sys.path.insert(0, str(Path(__file__).parent.parent.parent / "Utilities"))
from cuda_samples_utils import verify_array_result # noqa: E402
try:
import cupy as cp
from cuda.core import Device, LaunchConfig, Program, ProgramOptions, launch
except ImportError as e:
print(f"Error: Required package not found: {e}")
print("Please install from requirements.txt:")
print(" pip install -r requirements.txt")
sys.exit(1)
# CUDA kernel source code
VECTOR_ADD_KERNEL = """
/**
* CUDA Kernel for vector addition
* Computes the vector addition of A and B into C.
*/
template<typename T>
__global__ void vectorAdd(const T *A, const T *B, T *C, int numElements)
{
int i = blockDim.x * blockIdx.x + threadIdx.x;
if (i < numElements)
{
C[i] = A[i] + B[i];
}
}
"""
def vector_add_cuda_core(num_elements=50000, device_id=0, verify=True):
"""
Perform vector addition using cuda.core API.
Parameters
----------
num_elements : int
Number of elements in each vector
device_id : int
CUDA device ID to use
verify : bool
Whether to verify the result
Returns
-------
bool
True if successful, False otherwise
"""
try:
# Initialize device
print("[Vector addition using CUDA Core API]")
device = Device(device_id)
device.set_current()
print(f"Device: {device.name}")
print(f"Compute Capability: sm_{device.arch}")
stream = device.create_stream()
# Compile kernel
print("Compiling kernel 'vectorAdd<float>'...")
program_options = ProgramOptions(std="c++17", arch=f"sm_{device.arch}")
program = Program(VECTOR_ADD_KERNEL, code_type="c++", options=program_options)
module = program.compile("cubin", name_expressions=("vectorAdd<float>",))
kernel = module.get_kernel("vectorAdd<float>")
print("Kernel compiled successfully")
# Allocate and initialize vectors
print(f"[Vector addition of {num_elements} elements]")
dtype = cp.float32
a = cp.random.rand(num_elements).astype(dtype)
b = cp.random.rand(num_elements).astype(dtype)
c = cp.empty(num_elements, dtype=dtype)
# Synchronize before kernel launch
device.sync()
# Configure and launch kernel
threads_per_block = 256
blocks_per_grid = (num_elements + threads_per_block - 1) // threads_per_block
print(
f"CUDA kernel launch with {blocks_per_grid} blocks "
f"of {threads_per_block} threads"
)
config = LaunchConfig(grid=blocks_per_grid, block=threads_per_block)
# Launch kernel
launch(
stream,
config,
kernel,
a.data.ptr,
b.data.ptr,
c.data.ptr,
cp.int32(num_elements),
)
stream.sync()
# Verify result
if verify:
print("Verifying result...")
expected = a + b
if not verify_array_result(c, expected):
return False
return True
except Exception as e:
print(f"Error: {e}")
import traceback
traceback.print_exc()
return False
def main():
"""
Main entry point for the vector addition sample.
"""
import argparse
parser = argparse.ArgumentParser(description="Vector Addition using CUDA Core API")
parser.add_argument(
"--elements",
type=int,
default=50000,
help="Number of elements in vectors (default: 50000)",
)
parser.add_argument(
"--device", type=int, default=0, help="CUDA device ID (default: 0)"
)
parser.add_argument(
"--no-verify", action="store_true", help="Skip result verification"
)
args = parser.parse_args()
if args.elements <= 0:
print("Error: Number of elements must be positive")
return 1
success = vector_add_cuda_core(
num_elements=args.elements, device_id=args.device, verify=not args.no_verify
)
if success:
print("\nDone")
return 0
else:
return 1
if __name__ == "__main__":
sys.exit(main())