mirror of
https://github.com/NVIDIA/cuda-samples.git
synced 2026-10-13 00:08:26 +08:00
CUDA 13.2 samples update (#432)
- Added Python samples for CUDA Python 1.0 release - Renamed top-level `Samples` directory to `cpp` to accommodate Python samples.
This commit is contained in:
168
python/1_GettingStarted/blurImageUnifiedMemory/README.md
Normal file
168
python/1_GettingStarted/blurImageUnifiedMemory/README.md
Normal file
@@ -0,0 +1,168 @@
|
||||
# Sample: Image Blur with Unified Memory (Python)
|
||||
|
||||
## Description
|
||||
|
||||
Blur images on GPU using modern `cuda.core` APIs for kernel compilation, execution, and memory management. This sample demonstrates **zero-copy data sharing** between CPU and GPU using unified (managed) memory.
|
||||
|
||||
## What You'll Learn
|
||||
|
||||
- Compiling CUDA kernels at runtime with `cuda.core.Program`
|
||||
- Launching kernels with `cuda.core.launch` and `LaunchConfig`
|
||||
- Using unified memory with `cuda.core.ManagedMemoryResource`
|
||||
- **Zero-copy CPU access** to unified memory via `np.from_dlpack()`
|
||||
- Seamless CPU/GPU memory access without explicit transfers
|
||||
|
||||
## Key Concepts
|
||||
|
||||
### Kernel Compilation with cuda.core.Program
|
||||
|
||||
```python
|
||||
# Compile CUDA C++ kernel at runtime
|
||||
program = Program(KERNEL_CODE, code_type="c++", options=options)
|
||||
compiled = program.compile(target_type="cubin")
|
||||
kernel = compiled.get_kernel("box_blur_3x3")
|
||||
```
|
||||
|
||||
### Kernel Launch with cuda.core.launch
|
||||
|
||||
```python
|
||||
# Configure and launch kernel
|
||||
config = LaunchConfig(grid=grid_size, block=block_size)
|
||||
|
||||
# Buffers can be passed directly as kernel arguments
|
||||
launch(stream, config, kernel, src_buf, dst_buf, H, W)
|
||||
```
|
||||
|
||||
### Unified Memory (Managed Memory)
|
||||
|
||||
This sample uses `ManagedMemoryResource` for simplicity: a single allocation is accessible from both CPU and GPU without explicit transfers. For performance-critical workloads, consider `LegacyPinnedMemoryResource` + `DeviceMemoryResource` instead, which gives explicit control over host/device placement and transfer costs.
|
||||
|
||||
Unified memory is accessible from both CPU and GPU without explicit data transfers:
|
||||
|
||||
```python
|
||||
# Allocate unified memory
|
||||
options = ManagedMemoryResourceOptions(preferred_location=device.device_id)
|
||||
mr = ManagedMemoryResource(options)
|
||||
src_buf = mr.allocate(n_bytes, stream)
|
||||
dst_buf = mr.allocate(n_bytes, stream)
|
||||
try:
|
||||
# Synchronize to ensure allocations are complete before CPU access
|
||||
stream.sync()
|
||||
|
||||
# Create numpy views of unified memory using DLPack protocol (zero-copy)
|
||||
src_np = np.from_dlpack(src_buf).view(np.float32).reshape(H, W)
|
||||
dst_np = np.from_dlpack(dst_buf).view(np.float32).reshape(H, W)
|
||||
|
||||
# CPU writes directly to unified memory
|
||||
src_np[:] = input_data
|
||||
|
||||
# Launch kernel - buffers can be passed directly as arguments
|
||||
launch(stream, config, kernel, src_buf, dst_buf, H, W)
|
||||
stream.sync()
|
||||
|
||||
# Return zero-copy view; caller must close buffers when done
|
||||
return dst_np, src_buf, dst_buf
|
||||
except Exception:
|
||||
src_buf.close()
|
||||
dst_buf.close()
|
||||
raise
|
||||
```
|
||||
|
||||
When returning a zero-copy view, the caller must close the buffers after use (e.g., in a `try/finally` block) to avoid leaking managed memory.
|
||||
|
||||
## Key APIs
|
||||
|
||||
### From `cuda.core`:
|
||||
|
||||
- `Device` - CUDA device management
|
||||
- `Program` - Runtime kernel compilation (NVRTC)
|
||||
- `ProgramOptions` - Compilation options (architecture target)
|
||||
- `LaunchConfig` - Kernel launch configuration (grid/block dimensions)
|
||||
- `launch` - Execute compiled kernel
|
||||
- `ManagedMemoryResource` - Unified memory allocation
|
||||
|
||||
### Zero-Copy Techniques:
|
||||
|
||||
- `np.from_dlpack(buffer)` - Create numpy view of unified memory using DLPack protocol
|
||||
- Pass `buffer` directly to `launch()` as kernel arguments
|
||||
- When returning a zero-copy view, return `(view, src_buf, dst_buf)` and have the caller close buffers in `try/finally` after use
|
||||
|
||||
## Kernel Techniques
|
||||
|
||||
- **2D Thread Mapping** - Each thread computes one output pixel
|
||||
- **Stencil Pattern** - Read neighboring pixels (3x3 neighborhood)
|
||||
- **Boundary Handling** - Clamp to edge for border pixels
|
||||
- **Box Filter** - 3x3 averaging for blur effect
|
||||
|
||||
## Requirements
|
||||
|
||||
### Hardware:
|
||||
|
||||
- NVIDIA GPU with CUDA support
|
||||
- Minimum GPU memory: 256 MB
|
||||
|
||||
### Software:
|
||||
|
||||
- CUDA Toolkit 13.0 or newer
|
||||
- Python 3.10 or newer
|
||||
- `cuda-python` package (13.0.0+)
|
||||
- `cuda-core` package (>=0.6.0)
|
||||
- `numpy` package (>=2.3.2)
|
||||
- `pillow` package (10.0.0+)
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
cd /path/to/cuda-samples/python/1_GettingStarted/blurImageUnifiedMemory
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
## How to Run
|
||||
|
||||
```bash
|
||||
python blurImageUnifiedMemory.py
|
||||
```
|
||||
|
||||
## Expected Output
|
||||
|
||||
```
|
||||
============================================================
|
||||
Image Blur with Unified Memory (cuda.core)
|
||||
============================================================
|
||||
|
||||
Device: <Your GPU Name>
|
||||
Compute Capability: sm_<XY>
|
||||
|
||||
Compiling CUDA kernel with cuda.core.Program...
|
||||
Compiled for architecture: sm_<XY>
|
||||
|
||||
Image size: 256x256 grayscale
|
||||
Creating sample image...
|
||||
Blurring image on GPU...
|
||||
|
||||
Saving results...
|
||||
Saved: original_image.png
|
||||
Saved: blurred_image.png
|
||||
|
||||
Verifying result...
|
||||
Test PASSED
|
||||
Max difference from original: <value>
|
||||
```
|
||||
|
||||
## Output Files
|
||||
|
||||
- `original_image.png` - Test pattern image before blur
|
||||
- `blurred_image.png` - Image after 3x3 box blur
|
||||
|
||||
## Files
|
||||
|
||||
- `blurImageUnifiedMemory.py` - Python implementation using cuda.core
|
||||
- `README.md` - This file
|
||||
- `requirements.txt` - Sample dependencies
|
||||
|
||||
## See Also
|
||||
|
||||
- [cuda.core Documentation](https://nvidia.github.io/cuda-python/cuda-core/latest/)
|
||||
- [cuda.core.Program](https://nvidia.github.io/cuda-python/cuda-core/latest/generated/cuda.core.Program.html)
|
||||
- [cuda.core.ManagedMemoryResource](https://nvidia.github.io/cuda-python/cuda-core/latest/generated/cuda.core.ManagedMemoryResource.html)
|
||||
- [CUDA Managed Memory](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#um-unified-memory-programming-hd)
|
||||
@@ -0,0 +1,269 @@
|
||||
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
|
||||
#
|
||||
# Redistribution and use in source and binary forms, with or without
|
||||
# modification, are permitted provided that the following conditions
|
||||
# are met:
|
||||
# * Redistributions of source code must retain the above copyright
|
||||
# notice, this list of conditions and the following disclaimer.
|
||||
# * Redistributions in binary form must reproduce the above copyright
|
||||
# notice, this list of conditions and the following disclaimer in the
|
||||
# documentation and/or other materials provided with the distribution.
|
||||
# * Neither the name of NVIDIA CORPORATION nor the names of its
|
||||
# contributors may be used to endorse or promote products derived
|
||||
# from this software without specific prior written permission.
|
||||
#
|
||||
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
|
||||
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
||||
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
||||
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
|
||||
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
|
||||
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
|
||||
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
|
||||
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
|
||||
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
"""
|
||||
Image Blur with Unified Memory using cuda.core
|
||||
|
||||
Demonstrates GPU image blurring using cuda.core APIs for kernel compilation,
|
||||
launch, and unified memory allocation.
|
||||
"""
|
||||
|
||||
import sys
|
||||
|
||||
try:
|
||||
import numpy as np
|
||||
from cuda.core import (
|
||||
Device,
|
||||
LaunchConfig,
|
||||
ManagedMemoryResource,
|
||||
ManagedMemoryResourceOptions,
|
||||
Program,
|
||||
ProgramOptions,
|
||||
launch,
|
||||
)
|
||||
from PIL import Image
|
||||
except ImportError as e:
|
||||
print(f"Error: Required package not found: {e}")
|
||||
print("Please install from requirements.txt:")
|
||||
print(" pip install -r requirements.txt")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
# CUDA kernel source code - compiled at runtime by cuda.core.Program
|
||||
BOX_BLUR_KERNEL_CODE = r"""
|
||||
extern "C" __global__
|
||||
void box_blur_3x3(const float* __restrict__ src,
|
||||
float* __restrict__ dst, int H, int W) {
|
||||
/*
|
||||
* Simple 3x3 box blur CUDA kernel.
|
||||
*
|
||||
* Each thread computes one output pixel by averaging
|
||||
* the 3x3 neighborhood of input pixels (stencil pattern).
|
||||
*/
|
||||
|
||||
int x = blockIdx.x * blockDim.x + threadIdx.x;
|
||||
int y = blockIdx.y * blockDim.y + threadIdx.y;
|
||||
|
||||
if (x >= W || y >= H) return;
|
||||
|
||||
float sum = 0.0f;
|
||||
int count = 0;
|
||||
|
||||
// 3x3 stencil: iterate over neighborhood
|
||||
for (int dy = -1; dy <= 1; dy++) {
|
||||
for (int dx = -1; dx <= 1; dx++) {
|
||||
int nx = x + dx;
|
||||
int ny = y + dy;
|
||||
|
||||
// Boundary check (clamp to edge)
|
||||
if (nx >= 0 && nx < W && ny >= 0 && ny < H) {
|
||||
sum += src[ny * W + nx];
|
||||
count++;
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
dst[y * W + x] = sum / count;
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
def make_test_image(h: int, w: int, dtype=np.uint8) -> np.ndarray:
|
||||
"""Create a test grayscale image for demonstration."""
|
||||
img = np.zeros((h, w), dtype=dtype)
|
||||
|
||||
# Create horizontal stripes
|
||||
for i in range(0, h, 50):
|
||||
img[i : i + 25, :] = 255
|
||||
|
||||
# Create vertical stripes with different intensity
|
||||
for j in range(0, w, 50):
|
||||
img[:, j : j + 25] = 128
|
||||
|
||||
# Add circular pattern for interesting blur effects
|
||||
center_y, center_x = h // 2, w // 2
|
||||
y, x = np.ogrid[:h, :w]
|
||||
circle_mask = (x - center_x) ** 2 + (y - center_y) ** 2 <= (min(h, w) // 6) ** 2
|
||||
img[circle_mask] = 200
|
||||
|
||||
return np.ascontiguousarray(img)
|
||||
|
||||
|
||||
def blur_image_unified_memory(
|
||||
host_np: np.ndarray, device: Device, stream, kernel
|
||||
) -> tuple[np.ndarray, object, object]:
|
||||
"""
|
||||
Blur image on GPU using unified memory with cuda.core.
|
||||
|
||||
This function demonstrates:
|
||||
1. Allocate managed memory using ManagedMemoryResource
|
||||
2. Create zero-copy numpy views using np.from_dlpack()
|
||||
3. Launch kernel via cuda.core.launch
|
||||
|
||||
Args:
|
||||
host_np: NumPy array containing image data on CPU
|
||||
device: CUDA device to use
|
||||
stream: cuda.core Stream for async operations
|
||||
kernel: Compiled cuda.core Kernel object
|
||||
|
||||
Returns:
|
||||
Tuple of (dst_np, src_buf, dst_buf). dst_np is a zero-copy view into
|
||||
unified memory. Caller must close src_buf and dst_buf when done with
|
||||
dst_np to avoid leaking managed memory.
|
||||
"""
|
||||
H, W = host_np.shape
|
||||
n_bytes = H * W * np.dtype(np.float32).itemsize
|
||||
|
||||
# Create managed memory resource for unified memory allocation
|
||||
options = ManagedMemoryResourceOptions(preferred_location=device.device_id)
|
||||
mr = ManagedMemoryResource(options)
|
||||
|
||||
# Allocate unified memory buffers for source and destination images
|
||||
src_buf = mr.allocate(n_bytes, stream)
|
||||
dst_buf = mr.allocate(n_bytes, stream)
|
||||
try:
|
||||
# Synchronize to ensure allocations are complete before CPU access
|
||||
stream.sync()
|
||||
|
||||
# Create numpy views of unified memory using DLPack protocol (zero-copy)
|
||||
src_np = np.from_dlpack(src_buf).view(np.float32).reshape(H, W)
|
||||
dst_np = np.from_dlpack(dst_buf).view(np.float32).reshape(H, W)
|
||||
|
||||
# Write input data to unified memory (CPU can access directly)
|
||||
src_np[:] = host_np.astype(np.float32) / 255.0
|
||||
|
||||
# Configure kernel launch parameters
|
||||
block_size = (16, 16)
|
||||
grid_size = (
|
||||
(W + block_size[0] - 1) // block_size[0],
|
||||
(H + block_size[1] - 1) // block_size[1],
|
||||
)
|
||||
|
||||
# Create LaunchConfig for kernel execution
|
||||
config = LaunchConfig(grid=grid_size, block=block_size)
|
||||
|
||||
# Launch kernel - buffers can be passed directly as kernel arguments
|
||||
launch(
|
||||
stream,
|
||||
config,
|
||||
kernel,
|
||||
src_buf,
|
||||
dst_buf,
|
||||
np.int32(H),
|
||||
np.int32(W),
|
||||
)
|
||||
|
||||
# Synchronize to ensure kernel completion before reading results
|
||||
stream.sync()
|
||||
|
||||
# Return zero-copy view; caller closes buffers when done
|
||||
return (dst_np, src_buf, dst_buf)
|
||||
except Exception:
|
||||
src_buf.close()
|
||||
dst_buf.close()
|
||||
raise
|
||||
|
||||
|
||||
def main():
|
||||
"""
|
||||
Complete demonstration of GPU image blurring with cuda.core.
|
||||
|
||||
This example shows:
|
||||
1. Device initialization with cuda.core.Device
|
||||
2. Kernel compilation with cuda.core.Program
|
||||
3. Unified memory with cuda.core.ManagedMemoryResource
|
||||
4. Kernel launch with cuda.core.launch and LaunchConfig
|
||||
"""
|
||||
print("=" * 60)
|
||||
print("Image Blur with Unified Memory (cuda.core)")
|
||||
print("=" * 60)
|
||||
|
||||
# Initialize CUDA device
|
||||
device = Device(0)
|
||||
device.set_current()
|
||||
|
||||
print(f"\nDevice: {device.name}")
|
||||
print(f"Compute Capability: sm_{device.arch}")
|
||||
|
||||
# Create stream for async operations
|
||||
stream = device.create_stream()
|
||||
try:
|
||||
# Compile kernel using cuda.core.Program
|
||||
print("\nCompiling CUDA kernel with cuda.core.Program...")
|
||||
arch = f"sm_{device.arch}"
|
||||
options = ProgramOptions(arch=arch)
|
||||
program = Program(BOX_BLUR_KERNEL_CODE, code_type="c++", options=options)
|
||||
compiled = program.compile(target_type="cubin")
|
||||
kernel = compiled.get_kernel("box_blur_3x3")
|
||||
print(f" Compiled for architecture: {arch}")
|
||||
|
||||
# Image parameters
|
||||
H, W = 256, 256
|
||||
print(f"\nImage size: {H}x{W} grayscale")
|
||||
|
||||
# Create test image
|
||||
print("Creating sample image...")
|
||||
host_np = make_test_image(H, W, dtype=np.uint8)
|
||||
|
||||
# Blur image on GPU using cuda.core (returns zero-copy view + buffers)
|
||||
print("Blurring image on GPU...")
|
||||
blurred_result, src_buf, dst_buf = blur_image_unified_memory(
|
||||
host_np, device, stream, kernel
|
||||
)
|
||||
try:
|
||||
# Save images (use zero-copy view before releasing buffers)
|
||||
print("\nSaving results...")
|
||||
original_pil = Image.fromarray(host_np, mode="L")
|
||||
original_pil.save("original_image.png")
|
||||
print(" Saved: original_image.png")
|
||||
|
||||
blurred_uint8 = (np.clip(blurred_result, 0, 1) * 255).astype(np.uint8)
|
||||
blurred_pil = Image.fromarray(blurred_uint8, mode="L")
|
||||
blurred_pil.save("blurred_image.png")
|
||||
print(" Saved: blurred_image.png")
|
||||
|
||||
# Verify blur was applied
|
||||
print("\nVerifying result...")
|
||||
original_float = host_np.astype(np.float32) / 255.0
|
||||
max_diff = np.max(np.abs(blurred_result - original_float))
|
||||
blur_applied = max_diff > 0.01
|
||||
|
||||
if blur_applied:
|
||||
print(" Test PASSED")
|
||||
else:
|
||||
print(" Test FAILED - blur not applied")
|
||||
sys.exit(1)
|
||||
|
||||
print(f" Max difference from original: {max_diff:.4f}")
|
||||
finally:
|
||||
src_buf.close()
|
||||
dst_buf.close()
|
||||
finally:
|
||||
stream.close()
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,6 @@
|
||||
# Image Blur with Unified Memory Sample Requirements
|
||||
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
numpy>=2.3.2
|
||||
pillow>=10.0.0
|
||||
119
python/1_GettingStarted/copyImageArraytoGPU/README.md
Normal file
119
python/1_GettingStarted/copyImageArraytoGPU/README.md
Normal file
@@ -0,0 +1,119 @@
|
||||
# Sample: Image Array Copy to GPU (Python)
|
||||
|
||||
## Description
|
||||
|
||||
Copy image arrays between CPU and GPU memory using the modern `cuda.core` API with optimal performance through pinned memory and asynchronous transfers.
|
||||
|
||||
## What You'll Learn
|
||||
|
||||
- How to use pinned memory for faster CPU↔GPU transfers
|
||||
- Using the `cuda.core` API for memory management
|
||||
- Working with DLPack for zero-copy array views
|
||||
- Performing asynchronous memory transfers with CUDA streams
|
||||
- Interoperability between CUDA Core API and CuPy
|
||||
- Proper CUDA resource management and cleanup
|
||||
|
||||
## Key Libraries
|
||||
|
||||
- `cuda.core` - Modern CUDA Python API
|
||||
- `numpy` - Array operations and DLPack support
|
||||
- `cupy` - GPU array operations and CUDA interoperability
|
||||
|
||||
## Key APIs
|
||||
|
||||
### From `cuda.core`:
|
||||
|
||||
- `Device()` - Initialize and access CUDA device
|
||||
- `Device.set_current()` - Set the current device for API calls
|
||||
- `Device.create_stream()` - Create CUDA stream for async operations
|
||||
- `Device.memory_resource` - Access device memory allocator
|
||||
- `PinnedMemoryResource()` - Allocate pinned host memory
|
||||
- `buffer.copy_to()` - Copy data between memory spaces
|
||||
- `buffer.close()` - Release allocated memory
|
||||
|
||||
### From `numpy`:
|
||||
|
||||
- `np.from_dlpack()` - Create array view from DLPack capsule
|
||||
- `np.copyto()` - Copy data between arrays
|
||||
|
||||
### From `cupy`:
|
||||
|
||||
- `cp.from_dlpack()` - Create GPU array view from DLPack capsule
|
||||
- `cp.cuda.ExternalStream()` - Use external CUDA stream
|
||||
|
||||
### From `cuda_samples_utils`:
|
||||
|
||||
- `verify_array_result()` - Verify computation results
|
||||
|
||||
## Requirements
|
||||
|
||||
### Hardware:
|
||||
|
||||
- NVIDIA GPU with CUDA support
|
||||
- Sufficient GPU memory for image data (sample uses ~200KB for 256×256×3 image)
|
||||
|
||||
### Software:
|
||||
|
||||
- CUDA Toolkit 13.0 or newer
|
||||
- Python 3.10 or newer
|
||||
- NumPy 2.3.2 or newer (required for DLPack support)
|
||||
- `cuda-python` package (>=13.0.0+)
|
||||
- `cuda-core` package (>=0.6.0)
|
||||
- `cupy-cuda13x` package (13.0.0+)
|
||||
|
||||
## Installation
|
||||
|
||||
Install the required packages from requirements.txt:
|
||||
|
||||
```bash
|
||||
cd /path/to/cuda-samples/python/1_GettingStarted/copyImageArraytoGPU
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
The requirements.txt installs:
|
||||
- `numpy` (2.3.2+, required for DLPack)
|
||||
- `cuda-python` (>=13.0.0+)
|
||||
- `cuda-core` (>=0.6.0)
|
||||
- `cupy-cuda13x` (13.0.0+)
|
||||
|
||||
## How to Run
|
||||
|
||||
### Basic usage:
|
||||
|
||||
```bash
|
||||
cd samples/python/1_GettingStarted/copyImageArraytoGPU
|
||||
python copyImageArraytoGPU.py
|
||||
```
|
||||
|
||||
## Expected Output
|
||||
|
||||
```
|
||||
[Image Array Copy to GPU using CUDA Core API]
|
||||
Device: NVIDIA GeForce RTX 4090
|
||||
[Image array copy of 256x256x3 image]
|
||||
Creating sample image...
|
||||
Copying image to GPU...
|
||||
Creating CuPy view of GPU data...
|
||||
Mean pixel value (computed on GPU): 127.50
|
||||
Copying image back from GPU...
|
||||
Verifying result...
|
||||
Test PASSED
|
||||
|
||||
Done
|
||||
```
|
||||
|
||||
**Note:** Device name will vary based on your GPU.
|
||||
|
||||
## Files
|
||||
|
||||
- `copyImageArraytoGPU.py` - Python implementation using cuda.core API
|
||||
- `README.md` - This file
|
||||
- `requirements.txt` - Sample dependencies
|
||||
- `../../Utilities/cuda_samples_utils.py` - Common utilities (imported by this sample)
|
||||
|
||||
## See Also
|
||||
|
||||
- [CUDA Python Documentation](https://nvidia.github.io/cuda-python/)
|
||||
- [cuda.core API Guide](https://nvidia.github.io/cuda-python/cuda-core/latest/)
|
||||
- [DLPack Specification](https://dmlc.github.io/dlpack/latest/)
|
||||
- [CuPy Documentation](https://docs.cupy.dev/)
|
||||
@@ -0,0 +1,239 @@
|
||||
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
|
||||
#
|
||||
# Redistribution and use in source and binary forms, with or without
|
||||
# modification, are permitted provided that the following conditions
|
||||
# are met:
|
||||
# * Redistributions of source code must retain the above copyright
|
||||
# notice, this list of conditions and the following disclaimer.
|
||||
# * Redistributions in binary form must reproduce the above copyright
|
||||
# notice, this list of conditions and the following disclaimer in the
|
||||
# documentation and/or other materials provided with the distribution.
|
||||
# * Neither the name of NVIDIA CORPORATION nor the names of its
|
||||
# contributors may be used to endorse or promote products derived
|
||||
# from this software without specific prior written permission.
|
||||
#
|
||||
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
|
||||
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
||||
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
||||
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
|
||||
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
|
||||
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
|
||||
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
|
||||
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
|
||||
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
"""
|
||||
Image Array Copy to GPU using CUDA Core API
|
||||
|
||||
This sample demonstrates how to copy image arrays between CPU and GPU memory
|
||||
using NVIDIA's CUDA Core Python API with optimal performance.
|
||||
"""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Add parent directory to path to import utilities
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent.parent / "Utilities"))
|
||||
from cuda_samples_utils import verify_array_result
|
||||
|
||||
try:
|
||||
import cupy as cp
|
||||
import numpy as np
|
||||
from cuda.core import Buffer, Device, PinnedMemoryResource, Stream
|
||||
except ImportError as e:
|
||||
print(f"Error: Required package not found: {e}")
|
||||
print("Please install from requirements.txt:")
|
||||
print(" pip install -r requirements.txt")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
# ----------------------------- Helper Functions ------------------------------
|
||||
|
||||
|
||||
def make_random_image(h: int, w: int, c: int, dtype=np.uint8) -> np.ndarray:
|
||||
"""
|
||||
Create a random test image for demonstration.
|
||||
|
||||
Args:
|
||||
h: Image height in pixels
|
||||
w: Image width in pixels
|
||||
c: Number of channels (e.g., 3 for RGB)
|
||||
dtype: NumPy data type (e.g., np.uint8 for 0-255 pixel values)
|
||||
|
||||
Returns:
|
||||
A contiguous NumPy array representing the image
|
||||
"""
|
||||
img = np.random.randint(0, 256, size=(h, w, c), dtype=dtype)
|
||||
return np.ascontiguousarray(img) # Ensure memory is contiguous for GPU transfer
|
||||
|
||||
|
||||
# ----------------------------- Core GPU Functions ---------------------------
|
||||
|
||||
|
||||
def copy_image_to_gpu_cuda_core(
|
||||
host_np: np.ndarray, dev: Device, stream: Stream
|
||||
) -> tuple[Buffer, Buffer]:
|
||||
"""
|
||||
Copy image from CPU memory to GPU memory using optimal transfer method.
|
||||
|
||||
This function demonstrates the recommended approach:
|
||||
1. Use pinned memory for faster transfers
|
||||
2. Use DLPack for zero-copy array views
|
||||
3. Perform async transfers on a CUDA stream
|
||||
|
||||
Args:
|
||||
host_np: NumPy array containing image data on CPU
|
||||
dev: CUDA device object
|
||||
stream: CUDA stream for async operations
|
||||
|
||||
Returns:
|
||||
Tuple of (device_buffer, pinned_buffer) - both need to be cleaned up later
|
||||
"""
|
||||
nbytes = host_np.nbytes # Calculate total bytes needed
|
||||
|
||||
# Step 1: Set up memory resources
|
||||
# Device memory resource - allocates on GPU
|
||||
device_mr = dev.memory_resource
|
||||
# Pinned memory resource - allocates CPU memory that GPU can access faster
|
||||
pinned_mr = PinnedMemoryResource()
|
||||
|
||||
# Step 2: Allocate memory buffers
|
||||
pinned_buffer = pinned_mr.allocate(nbytes, stream=stream) # Fast CPU memory
|
||||
device_buffer = device_mr.allocate(nbytes, stream=stream) # GPU memory
|
||||
|
||||
# Step 3: Create a NumPy view of pinned memory using DLPack
|
||||
# This allows us to work with pinned memory as if it's a regular NumPy array
|
||||
pinned_view = (
|
||||
np.from_dlpack(pinned_buffer).view(dtype=host_np.dtype).reshape(host_np.shape)
|
||||
)
|
||||
|
||||
# Step 4: Copy image data from regular CPU memory to pinned CPU memory
|
||||
# This is a CPU-to-CPU copy, so it's very fast
|
||||
np.copyto(pinned_view, host_np)
|
||||
|
||||
# Step 5: Copy from pinned CPU memory to GPU memory
|
||||
# This is the actual CPU-to-GPU transfer, done asynchronously
|
||||
pinned_buffer.copy_to(device_buffer, stream=stream)
|
||||
|
||||
return device_buffer, pinned_buffer
|
||||
|
||||
|
||||
def copy_image_from_gpu_cuda_core(
|
||||
device_buffer: Buffer, shape: tuple, dtype: type, dev: Device, stream: Stream
|
||||
) -> np.ndarray:
|
||||
"""
|
||||
Copy image from GPU memory back to CPU memory.
|
||||
|
||||
This function reverses the GPU-to-CPU transfer process:
|
||||
1. Allocate pinned CPU memory for fast transfer
|
||||
2. Copy from GPU to pinned CPU memory
|
||||
3. Create NumPy view and copy to regular CPU memory
|
||||
|
||||
Args:
|
||||
device_buffer: GPU buffer containing image data
|
||||
shape: Original image shape tuple (height, width, channels)
|
||||
dtype: Original image data type
|
||||
dev: CUDA device object
|
||||
stream: CUDA stream for async operations
|
||||
|
||||
Returns:
|
||||
NumPy array with image data copied from GPU
|
||||
"""
|
||||
nbytes = np.prod(shape) * np.dtype(dtype).itemsize # Calculate total bytes
|
||||
|
||||
# Step 1: Create pinned memory for fast GPU-to-CPU transfer
|
||||
pinned_mr = PinnedMemoryResource()
|
||||
pinned_buffer = pinned_mr.allocate(nbytes, stream=stream)
|
||||
|
||||
# Step 2: Copy from GPU memory to pinned CPU memory
|
||||
device_buffer.copy_to(pinned_buffer, stream=stream)
|
||||
stream.sync() # Wait for the GPU transfer to complete
|
||||
|
||||
# Step 3: Create NumPy view of pinned memory using DLPack
|
||||
pinned_view = np.from_dlpack(pinned_buffer).view(dtype=dtype).reshape(shape)
|
||||
|
||||
# Step 4: Copy from pinned CPU memory to regular CPU memory
|
||||
# This creates the final result that can be used normally
|
||||
host_result = pinned_view.copy()
|
||||
|
||||
# Step 5: Clean up the temporary pinned buffer
|
||||
pinned_buffer.close(stream)
|
||||
|
||||
return host_result
|
||||
|
||||
|
||||
# ------------------------------ Main Demo ------------------------------------
|
||||
|
||||
|
||||
def main():
|
||||
"""
|
||||
Complete demonstration of GPU image copying workflow.
|
||||
|
||||
This example shows:
|
||||
1. Setting up CUDA device and stream
|
||||
2. Creating a sample image
|
||||
3. Copying image to GPU
|
||||
4. Accessing GPU data with CuPy (optional)
|
||||
5. Copying image back from GPU
|
||||
6. Verifying data integrity
|
||||
7. Proper cleanup of resources
|
||||
"""
|
||||
print("[Image Array Copy to GPU using CUDA Core API]")
|
||||
|
||||
# Image parameters - modify these to test different sizes
|
||||
H, W, C = 256, 256, 3 # Height=256, Width=256, Channels=3 (RGB)
|
||||
dtype = np.uint8 # Standard image pixel type (0-255 values)
|
||||
|
||||
# Step 1: Set up CUDA device and stream
|
||||
dev = Device() # Get default CUDA device (GPU 0)
|
||||
dev.set_current() # Make this device the active one
|
||||
stream = dev.create_stream() # Create stream for async operations
|
||||
|
||||
print(f"Device: {dev.name}")
|
||||
print(f"[Image array copy of {H}x{W}x{C} image]")
|
||||
|
||||
# Step 2: Configure CuPy to use our CUDA stream (for interoperability)
|
||||
cp.cuda.ExternalStream(int(stream.handle)).use()
|
||||
|
||||
# Step 3: Create a test image on CPU
|
||||
print("Creating sample image...")
|
||||
host_np = make_random_image(H, W, C, dtype=dtype)
|
||||
|
||||
# Step 4: Copy image from CPU to GPU
|
||||
print("Copying image to GPU...")
|
||||
device_buffer, pinned_buffer = copy_image_to_gpu_cuda_core(host_np, dev, stream)
|
||||
|
||||
# Step 5: (Optional) Get a CuPy view of GPU data for processing
|
||||
# This shows how you can work with the GPU data without copying it back
|
||||
print("Creating CuPy view of GPU data...")
|
||||
device_cp = cp.from_dlpack(device_buffer).view(dtype=dtype).reshape(H, W, C)
|
||||
|
||||
# Example: compute mean pixel value on GPU
|
||||
mean_value = float(cp.mean(device_cp))
|
||||
print(f"Mean pixel value (computed on GPU): {mean_value:.2f}")
|
||||
|
||||
# Step 6: Copy image back from GPU to CPU
|
||||
print("Copying image back from GPU...")
|
||||
host_back = copy_image_from_gpu_cuda_core(
|
||||
device_buffer, host_np.shape, host_np.dtype, dev, stream
|
||||
)
|
||||
|
||||
# Step 7: Verify that the data survived the round trip
|
||||
print("Verifying result...")
|
||||
host_back_cp = cp.asarray(host_back)
|
||||
host_np_cp = cp.asarray(host_np)
|
||||
verify_array_result(host_back_cp, host_np_cp, rtol=0, atol=0)
|
||||
|
||||
# Step 8: Clean up all allocated resources
|
||||
device_buffer.close(stream) # Free GPU memory
|
||||
pinned_buffer.close(stream) # Free pinned CPU memory
|
||||
stream.close() # Close CUDA stream
|
||||
cp.cuda.Stream.null.use() # Reset CuPy's stream to default
|
||||
|
||||
print("\nDone")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,6 @@
|
||||
# Image Array Copy to GPU Sample Requirements
|
||||
|
||||
numpy>=2.3.2
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
cupy-cuda13x>=13.0.0
|
||||
189
python/1_GettingStarted/deviceQuery/README.md
Normal file
189
python/1_GettingStarted/deviceQuery/README.md
Normal file
@@ -0,0 +1,189 @@
|
||||
# Sample: Device Query (Python)
|
||||
|
||||
## Description
|
||||
|
||||
Query and display detailed properties of all CUDA-capable devices in your system using the modern `cuda.core` API.
|
||||
|
||||
## What You'll Learn
|
||||
|
||||
- How to enumerate CUDA devices in the system
|
||||
- Using the `cuda.core` API for device management
|
||||
- Querying comprehensive device properties (compute capability, memory, limits)
|
||||
- Accessing low-level device attributes via `cuda.bindings`
|
||||
- Checking peer-to-peer (P2P) access capabilities between GPUs
|
||||
|
||||
## Key Libraries
|
||||
|
||||
- `cuda.core` - Modern CUDA Python API
|
||||
- `cuda.bindings` - Low-level CUDA bindings for runtime and driver APIs
|
||||
|
||||
## Key APIs
|
||||
|
||||
### From `cuda.core`:
|
||||
|
||||
- `Device.get_all_devices()` - Get tuple of all available Device instances
|
||||
- `Device(device_id)` - Get Device object for specific device ID
|
||||
- `system.get_driver_version()` - Query CUDA driver version
|
||||
- `Device.set_current()` - Set the current device for API calls
|
||||
- `Device.properties` - Access comprehensive device properties
|
||||
- `Device.name` - Get device name string
|
||||
- `Device.can_access_peer()` - Check P2P access to peer device
|
||||
|
||||
### From `cuda.bindings.runtime`:
|
||||
|
||||
- `cudart.cudaRuntimeGetVersion()` - Get CUDA runtime version
|
||||
- `cudart.cudaDeviceGetAttribute()` - Query specific device attributes
|
||||
|
||||
### From `cuda.bindings.driver`:
|
||||
|
||||
- `cuda.cuMemGetInfo()` - Get memory information for current device
|
||||
|
||||
## Device Properties Queried
|
||||
|
||||
### Compute Capabilities:
|
||||
- Compute capability version (major.minor)
|
||||
- Driver and runtime versions
|
||||
- Number of multiprocessors and CUDA cores
|
||||
|
||||
### Memory Information:
|
||||
- Total global memory
|
||||
- Memory clock rate and bus width
|
||||
- L2 cache size
|
||||
- Constant and shared memory sizes
|
||||
- Maximum memory pitch
|
||||
|
||||
### Execution Configuration Limits:
|
||||
- Maximum threads per block and per multiprocessor
|
||||
- Maximum block dimensions (x, y, z)
|
||||
- Maximum grid dimensions (x, y, z)
|
||||
- Warp size
|
||||
- Registers per block
|
||||
|
||||
### Texture Capabilities:
|
||||
- Maximum texture dimensions (1D, 2D, 3D)
|
||||
- Maximum layered texture sizes
|
||||
|
||||
### Feature Support:
|
||||
- Unified Addressing (UVA)
|
||||
- Managed Memory
|
||||
- Compute Preemption
|
||||
- Cooperative Kernel Launch
|
||||
- ECC support
|
||||
- Host page-locked memory mapping
|
||||
- Concurrent copy and kernel execution
|
||||
|
||||
### System Information:
|
||||
- PCI bus information
|
||||
- Compute mode
|
||||
- Driver mode (Windows only)
|
||||
- P2P access matrix (multi-GPU systems)
|
||||
|
||||
## Requirements
|
||||
|
||||
### Hardware:
|
||||
|
||||
- NVIDIA GPU with CUDA support (any compute capability)
|
||||
- No specific GPU memory requirement (query only)
|
||||
|
||||
### Software:
|
||||
|
||||
- CUDA Toolkit 13.0 or newer (recommended; matches `cuda-python` 13.x)
|
||||
- Python 3.10 or newer
|
||||
- `cuda-python` package (>=13.0.0)
|
||||
- `cuda-core` package (>=0.6.0)
|
||||
|
||||
## Installation
|
||||
|
||||
Install the required packages from requirements.txt:
|
||||
|
||||
```bash
|
||||
cd cuda-samples/python/1_GettingStarted/deviceQuery
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
The requirements.txt installs:
|
||||
- `cuda-python` (>=13.0.0)
|
||||
- `cuda-core` (>=0.6.0)
|
||||
|
||||
## How to Run
|
||||
|
||||
### Basic usage:
|
||||
|
||||
```bash
|
||||
cd cuda-samples/python/1_GettingStarted/deviceQuery
|
||||
python deviceQuery.py
|
||||
```
|
||||
|
||||
### Skip P2P information:
|
||||
|
||||
```bash
|
||||
python deviceQuery.py --no-p2p
|
||||
```
|
||||
|
||||
## Expected Output
|
||||
|
||||
```
|
||||
[CUDA Device Query using CUDA Core API]
|
||||
Detected 1 CUDA Capable device(s)
|
||||
|
||||
Device 0: <Your GPU Name>
|
||||
CUDA Driver Version / Runtime Version 12.4 / 12.6
|
||||
CUDA Capability Major/Minor version number: 8.9
|
||||
Total amount of global memory: 24217 MBytes (25393954816 bytes)
|
||||
(132) Multiprocessors, (128) CUDA Cores/MP: 16896 CUDA Cores
|
||||
GPU Max Clock rate: 1980 MHz (1.98 GHz)
|
||||
Memory Clock rate: 10501 Mhz
|
||||
Memory Bus Width: 384-bit
|
||||
L2 Cache Size: 67108864 bytes
|
||||
Maximum Texture Dimension Size (x,y,z) 1D=(131072), 2D=(131072, 65536), 3D=(16384, 16384, 16384)
|
||||
Maximum Layered 1D Texture Size, (num) layers 1D=(32768), 2048 layers
|
||||
Maximum Layered 2D Texture Size, (num) layers 2D=(32768, 32768), 2048 layers
|
||||
Total amount of constant memory: 65536 bytes
|
||||
Total amount of shared memory per block: 49152 bytes
|
||||
Total shared memory per multiprocessor: 102400 bytes
|
||||
Total number of registers available per block: 65536
|
||||
Warp size: 32
|
||||
Maximum number of threads per multiprocessor: 1536
|
||||
Maximum number of threads per block: 1024
|
||||
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
|
||||
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
|
||||
Maximum memory pitch: 2147483647 bytes
|
||||
Texture alignment: 512 bytes
|
||||
Concurrent copy and kernel execution: Yes with 2 copy engine(s)
|
||||
Run time limit on kernels: Yes
|
||||
Integrated GPU sharing Host Memory: No
|
||||
Support host page-locked memory mapping: Yes
|
||||
Device has ECC support: Enabled
|
||||
Device supports Unified Addressing (UVA): Yes
|
||||
Device supports Managed Memory: Yes
|
||||
Device supports Compute Preemption: Yes
|
||||
Supports Cooperative Kernel Launch: Yes
|
||||
Device PCI Domain ID / Bus ID / location ID: 0 / 1 / 0
|
||||
Compute Mode:
|
||||
< Default (multiple host threads can use cudaSetDevice() with device simultaneously) >
|
||||
|
||||
Done
|
||||
```
|
||||
|
||||
**Note:** Output will vary based on your specific GPU model and system configuration.
|
||||
|
||||
For multi-GPU systems, the output will include information for all detected devices and a P2P access matrix showing which GPUs can directly access each other's memory.
|
||||
|
||||
## Files
|
||||
|
||||
- `deviceQuery.py` - Python implementation using cuda.core API
|
||||
- `requirements.txt` - Sample dependencies
|
||||
|
||||
## Use Cases
|
||||
|
||||
- **System Diagnostics** - Verify CUDA installation and GPU detection
|
||||
- **Hardware Profiling** - Understand GPU capabilities before optimization
|
||||
- **Multi-GPU Systems** - Identify P2P topology for optimal data placement
|
||||
- **Kernel Development** - Determine execution configuration limits
|
||||
- **Compatibility Checks** - Verify compute capability requirements
|
||||
|
||||
## See Also
|
||||
|
||||
- [CUDA Python Documentation](https://nvidia.github.io/cuda-python/)
|
||||
- [cuda.core API Guide](https://nvidia.github.io/cuda-python/cuda-core/latest/)
|
||||
- [CUDA Programming Guide - Device Information](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#device-enumeration)
|
||||
389
python/1_GettingStarted/deviceQuery/deviceQuery.py
Executable file
389
python/1_GettingStarted/deviceQuery/deviceQuery.py
Executable file
@@ -0,0 +1,389 @@
|
||||
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
|
||||
#
|
||||
# Redistribution and use in source and binary forms, with or without
|
||||
# modification, are permitted provided that the following conditions
|
||||
# are met:
|
||||
# * Redistributions of source code must retain the above copyright
|
||||
# notice, this list of conditions and the following disclaimer.
|
||||
# * Redistributions in binary form must reproduce the above copyright
|
||||
# notice, this list of conditions and the following disclaimer in the
|
||||
# documentation and/or other materials provided with the distribution.
|
||||
# * Neither the name of NVIDIA CORPORATION nor the names of its
|
||||
# contributors may be used to endorse or promote products derived
|
||||
# from this software without specific prior written permission.
|
||||
#
|
||||
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
|
||||
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
||||
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
||||
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
|
||||
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
|
||||
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
|
||||
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
|
||||
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
|
||||
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
"""
|
||||
Device Query using CUDA Core API
|
||||
|
||||
This sample enumerates the properties of the CUDA devices present in the system.
|
||||
"""
|
||||
|
||||
import platform
|
||||
import sys
|
||||
|
||||
# cuda.bindings used for properties not yet exposed in cuda.core (see comments below)
|
||||
try:
|
||||
from cuda.bindings import driver as cuda, runtime as cudart
|
||||
from cuda.core import Device, system
|
||||
except ImportError as e:
|
||||
print(f"Error: Required package not found: {e}")
|
||||
print("Please install from requirements.txt:")
|
||||
print(" pip install -r requirements.txt")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def print_property(label, value, indent=2):
|
||||
"""
|
||||
Helper function to print device properties with aligned formatting.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
label : str
|
||||
Property label
|
||||
value : any
|
||||
Property value
|
||||
indent : int
|
||||
Number of spaces for indentation (default: 2)
|
||||
"""
|
||||
field_width = 47
|
||||
spaces = " " * indent
|
||||
print(f"{spaces}{label:<{field_width}}{value}")
|
||||
|
||||
|
||||
def fmt_bytes(size_in_bytes):
|
||||
"""Format bytes to human-readable string with MBytes."""
|
||||
return f"{size_in_bytes / (1024 * 1024):.0f} MBytes ({size_in_bytes} bytes)"
|
||||
|
||||
|
||||
def fmt_hz(rate_in_khz):
|
||||
"""Format frequency in kHz to MHz and GHz."""
|
||||
return f"{rate_in_khz * 1e-3:.0f} MHz ({rate_in_khz * 1e-6:.2f} GHz)"
|
||||
|
||||
|
||||
def fmt_yes_no(val):
|
||||
"""Format boolean value to Yes/No string."""
|
||||
return "Yes" if val else "No"
|
||||
|
||||
|
||||
def convert_sm_ver_to_cores(major, minor):
|
||||
"""
|
||||
Maps SM version to the number of CUDA cores per SM.
|
||||
|
||||
Information taken from:
|
||||
https://github.com/NVIDIA/cuda-samples/blob/master/Common/helper_cuda.h
|
||||
|
||||
Parameters
|
||||
----------
|
||||
major : int
|
||||
Major compute capability version
|
||||
minor : int
|
||||
Minor compute capability version
|
||||
|
||||
Returns
|
||||
-------
|
||||
int
|
||||
Number of CUDA cores per SM, or 0 if unknown
|
||||
"""
|
||||
sm_to_cores = {
|
||||
(3, 0): 192,
|
||||
(3, 2): 192,
|
||||
(3, 5): 192,
|
||||
(3, 7): 192,
|
||||
(5, 0): 128,
|
||||
(5, 2): 128,
|
||||
(5, 3): 128,
|
||||
(6, 0): 64,
|
||||
(6, 1): 128,
|
||||
(6, 2): 128,
|
||||
(7, 0): 64,
|
||||
(7, 2): 64,
|
||||
(7, 5): 64,
|
||||
(8, 0): 64,
|
||||
(8, 6): 128,
|
||||
(8, 7): 128,
|
||||
(8, 9): 128,
|
||||
(9, 0): 128,
|
||||
(10, 0): 128,
|
||||
(10, 1): 128,
|
||||
(10, 3): 128,
|
||||
(11, 0): 128,
|
||||
(12, 0): 128,
|
||||
(12, 1): 128,
|
||||
}
|
||||
return sm_to_cores.get((major, minor), 0)
|
||||
|
||||
|
||||
def print_device_info(dev_id, device):
|
||||
"""
|
||||
Print detailed information for a single CUDA device.
|
||||
Uses device.properties (cuda.core) for most fields; cuda.bindings for
|
||||
runtime version and global memory (not yet in high-level API).
|
||||
"""
|
||||
device.set_current()
|
||||
props = device.properties
|
||||
|
||||
print()
|
||||
print(f"Device {dev_id}: {device.name}")
|
||||
|
||||
# cuda.bindings workaround: runtime version not in cuda.core
|
||||
driver_major, driver_minor = system.get_driver_version()
|
||||
err, runtime_version = cudart.cudaRuntimeGetVersion()
|
||||
if err != cudart.cudaError_t.cudaSuccess:
|
||||
raise RuntimeError(f"Failed to get CUDA runtime version: {err}")
|
||||
runtime_major = runtime_version // 1000
|
||||
runtime_minor = (runtime_version % 1000) // 10
|
||||
|
||||
print_property(
|
||||
"CUDA Driver Version / Runtime Version",
|
||||
f"{driver_major}.{driver_minor} / {runtime_major}.{runtime_minor}",
|
||||
)
|
||||
print_property(
|
||||
"CUDA Capability Major/Minor version number:",
|
||||
f"{props.compute_capability_major}.{props.compute_capability_minor}",
|
||||
)
|
||||
|
||||
# cuda.bindings workaround: global memory (free/total) not in device.properties
|
||||
err, free_mem, total_mem_bytes = cuda.cuMemGetInfo()
|
||||
if err != cuda.CUresult.CUDA_SUCCESS:
|
||||
raise RuntimeError(f"Failed to get memory info: {err}")
|
||||
print_property("Total amount of global memory:", fmt_bytes(total_mem_bytes))
|
||||
|
||||
sm_cores = convert_sm_ver_to_cores(
|
||||
props.compute_capability_major, props.compute_capability_minor
|
||||
)
|
||||
total_cores = sm_cores * props.multiprocessor_count
|
||||
print_property(
|
||||
f"({props.multiprocessor_count:3d}) Multiprocessors, "
|
||||
f"({sm_cores:3d}) CUDA Cores/MP:",
|
||||
f"{total_cores} CUDA Cores",
|
||||
)
|
||||
|
||||
print_property("GPU Max Clock rate:", fmt_hz(props.clock_rate))
|
||||
print_property("Memory Clock rate:", f"{props.memory_clock_rate * 1e-3:.0f} Mhz")
|
||||
print_property("Memory Bus Width:", f"{props.global_memory_bus_width}-bit")
|
||||
if props.l2_cache_size > 0:
|
||||
print_property("L2 Cache Size:", f"{props.l2_cache_size} bytes")
|
||||
|
||||
print_property(
|
||||
"Maximum Texture Dimension Size (x,y,z)",
|
||||
f"1D=({props.maximum_texture1d_width}), "
|
||||
f"2D=({props.maximum_texture2d_width}, {props.maximum_texture2d_height}), "
|
||||
f"3D=({props.maximum_texture3d_width}, {props.maximum_texture3d_height}, "
|
||||
f"{props.maximum_texture3d_depth})",
|
||||
)
|
||||
print_property(
|
||||
"Maximum Layered 1D Texture Size, (num) layers",
|
||||
f"1D=({props.maximum_texture1d_layered_width}), "
|
||||
f"{props.maximum_texture1d_layered_layers} layers",
|
||||
)
|
||||
print_property(
|
||||
"Maximum Layered 2D Texture Size, (num) layers",
|
||||
f"2D=({props.maximum_texture2d_layered_width}, "
|
||||
f"{props.maximum_texture2d_layered_height}), "
|
||||
f"{props.maximum_texture2d_layered_layers} layers",
|
||||
)
|
||||
|
||||
print_property(
|
||||
"Total amount of constant memory:", f"{props.total_constant_memory} bytes"
|
||||
)
|
||||
print_property(
|
||||
"Total amount of shared memory per block:",
|
||||
f"{props.max_shared_memory_per_block} bytes",
|
||||
)
|
||||
print_property(
|
||||
"Total shared memory per multiprocessor:",
|
||||
f"{props.max_shared_memory_per_multiprocessor} bytes",
|
||||
)
|
||||
print_property(
|
||||
"Total number of registers available per block:", props.max_registers_per_block
|
||||
)
|
||||
|
||||
print_property("Warp size:", props.warp_size)
|
||||
print_property(
|
||||
"Maximum number of threads per multiprocessor:",
|
||||
props.max_threads_per_multiprocessor,
|
||||
)
|
||||
print_property("Maximum number of threads per block:", props.max_threads_per_block)
|
||||
print_property(
|
||||
"Max dimension size of a thread block (x,y,z):",
|
||||
f"({props.max_block_dim_x}, {props.max_block_dim_y}, {props.max_block_dim_z})",
|
||||
)
|
||||
print_property(
|
||||
"Max dimension size of a grid size (x,y,z):",
|
||||
f"({props.max_grid_dim_x}, {props.max_grid_dim_y}, {props.max_grid_dim_z})",
|
||||
)
|
||||
print_property("Maximum memory pitch:", f"{props.max_pitch} bytes")
|
||||
print_property("Texture alignment:", f"{props.texture_alignment} bytes")
|
||||
|
||||
print_property(
|
||||
"Concurrent copy and kernel execution:",
|
||||
f"{fmt_yes_no(props.gpu_overlap)} with "
|
||||
f"{props.async_engine_count} copy engine(s)",
|
||||
)
|
||||
print_property("Run time limit on kernels:", fmt_yes_no(props.kernel_exec_timeout))
|
||||
|
||||
print_property("Integrated GPU sharing Host Memory:", fmt_yes_no(props.integrated))
|
||||
print_property(
|
||||
"Support host page-locked memory mapping:",
|
||||
fmt_yes_no(props.can_map_host_memory),
|
||||
)
|
||||
print_property(
|
||||
"Device has ECC support:", "Enabled" if props.ecc_enabled else "Disabled"
|
||||
)
|
||||
if platform.system() == "Windows":
|
||||
mode = (
|
||||
"TCC (Tesla Compute Cluster Driver)"
|
||||
if props.tcc_driver
|
||||
else "WDDM (Windows Display Driver Model)"
|
||||
)
|
||||
print_property("CUDA Device Driver Mode (TCC or WDDM):", mode)
|
||||
|
||||
print_property(
|
||||
"Device supports Unified Addressing (UVA):",
|
||||
fmt_yes_no(props.unified_addressing),
|
||||
)
|
||||
print_property("Device supports Managed Memory:", fmt_yes_no(props.managed_memory))
|
||||
print_property(
|
||||
"Device supports Compute Preemption:",
|
||||
fmt_yes_no(props.compute_preemption_supported),
|
||||
)
|
||||
print_property(
|
||||
"Supports Cooperative Kernel Launch:", fmt_yes_no(props.cooperative_launch)
|
||||
)
|
||||
|
||||
print_property(
|
||||
"Device PCI Domain ID / Bus ID / location ID:",
|
||||
f"{props.pci_domain_id} / {props.pci_bus_id} / {props.pci_device_id}",
|
||||
)
|
||||
compute_modes = {
|
||||
0: (
|
||||
"Default (multiple host threads can use cudaSetDevice() "
|
||||
"with device simultaneously)"
|
||||
),
|
||||
1: (
|
||||
"Exclusive (only one host thread in one process is able to "
|
||||
"use cudaSetDevice() with this device)"
|
||||
),
|
||||
2: "Prohibited (no host thread can use cudaSetDevice() with this device)",
|
||||
3: (
|
||||
"Exclusive Process (many threads in one process is able to "
|
||||
"use cudaSetDevice() with this device)"
|
||||
),
|
||||
}
|
||||
print_property("Compute Mode:", "")
|
||||
print(f" < {compute_modes.get(props.compute_mode, 'Unknown')} >")
|
||||
|
||||
|
||||
def print_p2p_access_info(devices):
|
||||
"""
|
||||
Print peer-to-peer access information for multi-GPU systems.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
devices : tuple of Device
|
||||
Tuple of CUDA device objects
|
||||
"""
|
||||
print()
|
||||
print("Peer-to-Peer (P2P) access support:")
|
||||
for i, dev_i in enumerate(devices):
|
||||
for j, dev_j in enumerate(devices):
|
||||
if i == j:
|
||||
continue
|
||||
try:
|
||||
can_access = dev_i.can_access_peer(dev_j)
|
||||
print(
|
||||
f"> Peer access from {dev_i.name} (GPU{i}) -> "
|
||||
f"{dev_j.name} (GPU{j}) : {fmt_yes_no(can_access)}"
|
||||
)
|
||||
except Exception as e:
|
||||
print(
|
||||
"Warning: Could not check peer access between "
|
||||
f"device {i} and {j}: {e}"
|
||||
)
|
||||
|
||||
|
||||
def query_devices(show_p2p=True):
|
||||
"""
|
||||
Query and display information about all CUDA devices.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
show_p2p : bool
|
||||
Whether to show peer-to-peer access information (default: True)
|
||||
|
||||
Returns
|
||||
-------
|
||||
bool
|
||||
True if successful, False otherwise
|
||||
"""
|
||||
try:
|
||||
print("[CUDA Device Query using CUDA Core API]")
|
||||
devices = Device.get_all_devices()
|
||||
except Exception as e:
|
||||
print(f"Error: Failed to get devices: {e}")
|
||||
import traceback
|
||||
|
||||
traceback.print_exc()
|
||||
return False
|
||||
|
||||
if len(devices) == 0:
|
||||
print("There are no available device(s) that support CUDA")
|
||||
return True
|
||||
|
||||
print(f"Detected {len(devices)} CUDA Capable device(s)")
|
||||
|
||||
for dev_id, device in enumerate(devices):
|
||||
try:
|
||||
print_device_info(dev_id, device)
|
||||
except Exception as e:
|
||||
print(f"Error: Failed to get information for device {dev_id}: {e}")
|
||||
import traceback
|
||||
|
||||
traceback.print_exc()
|
||||
return False
|
||||
|
||||
if show_p2p and len(devices) >= 2:
|
||||
print_p2p_access_info(devices)
|
||||
|
||||
return True
|
||||
|
||||
|
||||
def main():
|
||||
"""
|
||||
Main entry point for the device query sample.
|
||||
"""
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Query CUDA Device Properties using CUDA Core API",
|
||||
formatter_class=argparse.RawDescriptionHelpFormatter,
|
||||
)
|
||||
parser.add_argument(
|
||||
"--no-p2p", action="store_true", help="Skip peer-to-peer access information"
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
success = query_devices(show_p2p=not args.no_p2p)
|
||||
|
||||
if success:
|
||||
print("\nDone")
|
||||
return 0
|
||||
else:
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
4
python/1_GettingStarted/deviceQuery/requirements.txt
Normal file
4
python/1_GettingStarted/deviceQuery/requirements.txt
Normal file
@@ -0,0 +1,4 @@
|
||||
# Device Query Sample Requirements
|
||||
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
72
python/1_GettingStarted/kernelNsysProfile/README.md
Normal file
72
python/1_GettingStarted/kernelNsysProfile/README.md
Normal file
@@ -0,0 +1,72 @@
|
||||
# Sample: Kernel Nsys Profiling - CUDA C++ Kernel Profiling with cuda.core (Python)
|
||||
|
||||
## Description
|
||||
|
||||
This sample demonstrates how to profile custom CUDA C++ kernels compiled and launched with `cuda.core` using NVIDIA Nsight Systems. It implements three GPU operations (vector addition, SAXPY, vector transform) as custom kernels and shows how to instrument code with NVTX markers for profiling analysis.
|
||||
|
||||
## What you will learn
|
||||
|
||||
- How to write and compile CUDA C++ kernels with `cuda.core.Program`
|
||||
- How to launch kernels with `LaunchConfig` and manage CUDA streams
|
||||
- How to use NVTX markers (`nvtx.annotate()`) to annotate code sections
|
||||
- How to profile kernels with Nsight Systems and analyze performance
|
||||
- Modern CUDA Python workflow with `cuda.core.Device` and proper resource cleanup
|
||||
|
||||
## Requirements
|
||||
|
||||
- NVIDIA GPU with Compute Capability 7.0+
|
||||
- CUDA Toolkit 13.0+
|
||||
- Python 3.10+
|
||||
- Packages: `numpy`, `cuda-python`, `cuda-core`, `cupy-cuda13x`, `nvtx` (see `requirements.txt`; NumPy >=2.3.2)
|
||||
|
||||
**Install:**
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
## How to run
|
||||
|
||||
```bash
|
||||
python kernelNsysProfile.py
|
||||
python kernelNsysProfile.py --array-size 10000000 # Custom size
|
||||
```
|
||||
|
||||
## Nsys Profiling
|
||||
|
||||
**Basic profile:**
|
||||
```bash
|
||||
nsys profile -o gpu_profile python kernelNsysProfile.py
|
||||
nsys-ui gpu_profile.nsys-rep # View results
|
||||
```
|
||||
|
||||
The program uses color-coded NVTX markers:
|
||||
- **Purple**: Phase 2 (cuda.core Custom Kernels - main focus)
|
||||
- **Yellow/Blue/Green**: Other phases
|
||||
- **Cyan**: Nested operations
|
||||
|
||||
Focus on Phase 2 to analyze kernel execution times, launch overhead, and GPU utilization.
|
||||
|
||||
**For detailed Nsys usage and analysis techniques, see the [NVIDIA Nsight Systems documentation](https://docs.nvidia.com/nsight-systems/).**
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
**Missing packages:**
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
**Out of memory:**
|
||||
```bash
|
||||
python kernelNsysProfile.py -n 10000000 # Reduce array size
|
||||
```
|
||||
|
||||
**Nsys not found:**
|
||||
```bash
|
||||
export PATH=/usr/local/cuda/bin:$PATH
|
||||
```
|
||||
|
||||
## See Also
|
||||
|
||||
- [CUDA Python Documentation](https://nvidia.github.io/cuda-python/)
|
||||
- [NVIDIA Nsight Systems Documentation](https://docs.nvidia.com/nsight-systems/)
|
||||
- [CuPy Documentation](https://docs.cupy.dev/)
|
||||
327
python/1_GettingStarted/kernelNsysProfile/kernelNsysProfile.py
Normal file
327
python/1_GettingStarted/kernelNsysProfile/kernelNsysProfile.py
Normal file
@@ -0,0 +1,327 @@
|
||||
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
|
||||
#
|
||||
# Redistribution and use in source and binary forms, with or without
|
||||
# modification, are permitted provided that the following conditions
|
||||
# are met:
|
||||
# * Redistributions of source code must retain the above copyright
|
||||
# notice, this list of conditions and the following disclaimer.
|
||||
# * Redistributions in binary form must reproduce the above copyright
|
||||
# notice, this list of conditions and the following disclaimer in the
|
||||
# distribution and/or other materials provided with the distribution.
|
||||
# * Neither the name of NVIDIA CORPORATION nor the names of its
|
||||
# contributors may be used to endorse or promote products derived
|
||||
# from this software without specific prior written permission.
|
||||
#
|
||||
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
|
||||
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
||||
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
||||
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
|
||||
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
|
||||
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
|
||||
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
|
||||
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
|
||||
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
"""
|
||||
Kernel Nsys Profiling Sample - CUDA C++ Kernel Profiling with cuda.core
|
||||
|
||||
This sample demonstrates how to profile custom CUDA C++ kernels compiled and
|
||||
launched with cuda.core using NVIDIA Nsight Systems.
|
||||
|
||||
The sample implements three common GPU operations as custom CUDA C++ kernels:
|
||||
- Vector addition: c = a + b
|
||||
- SAXPY: y = alpha * x + y
|
||||
- Vector transform: sqrt(x*x + 1) + sin(x)
|
||||
|
||||
Use Nsight Systems to analyze:
|
||||
- Custom kernel execution times
|
||||
- Kernel launch patterns and overhead
|
||||
- GPU utilization and memory access patterns
|
||||
- NVTX markers for structured profiling
|
||||
|
||||
Workflow:
|
||||
- Phase 1: Create GPU arrays
|
||||
- Phase 2: Compile and execute cuda.core custom kernels (profiling focus)
|
||||
- Phase 3: Verify correctness with CuPy reference implementation
|
||||
- Phase 4: Validate results
|
||||
"""
|
||||
|
||||
import argparse
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
import cupy as cp
|
||||
import numpy as np
|
||||
import nvtx
|
||||
from cuda.core import Device, LaunchConfig, launch
|
||||
except ImportError as e:
|
||||
print(f"Error: Required package not found: {e}")
|
||||
print("Please install from requirements.txt:")
|
||||
print(" pip install -r requirements.txt")
|
||||
sys.exit(1)
|
||||
|
||||
# Add parent directory to path to import utilities
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent.parent / "Utilities"))
|
||||
from cuda_samples_utils import verify_array_result # noqa: E402
|
||||
|
||||
# CUDA C++ kernel definitions
|
||||
# For larger projects, separating kernels into a separate file is also valid.
|
||||
KERNELS_CODE = """
|
||||
template<typename T>
|
||||
__global__ void vector_add(const T* a, const T* b, T* c, size_t N) {
|
||||
const unsigned int tid = threadIdx.x + blockIdx.x * blockDim.x;
|
||||
for (size_t i = tid; i < N; i += gridDim.x * blockDim.x) {
|
||||
c[i] = a[i] + b[i];
|
||||
}
|
||||
}
|
||||
|
||||
template<typename T>
|
||||
__global__ void saxpy(const T alpha, const T* x, T* y, size_t N) {
|
||||
const unsigned int tid = threadIdx.x + blockIdx.x * blockDim.x;
|
||||
for (size_t i = tid; i < N; i += gridDim.x * blockDim.x) {
|
||||
y[i] = alpha * x[i] + y[i];
|
||||
}
|
||||
}
|
||||
|
||||
template<typename T>
|
||||
__global__ void vector_transform(const T* a, T* b, size_t N) {
|
||||
const unsigned int tid = threadIdx.x + blockIdx.x * blockDim.x;
|
||||
for (size_t i = tid; i < N; i += gridDim.x * blockDim.x) {
|
||||
T val = a[i];
|
||||
b[i] = sqrt(val * val + T(1.0)) + sin(val);
|
||||
}
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
def get_cuda_core_kernels(device):
|
||||
"""
|
||||
Compile cuda.core kernels and return them.
|
||||
|
||||
Args:
|
||||
device: cuda.core.Device object
|
||||
|
||||
Returns:
|
||||
dict: Dictionary of compiled kernels
|
||||
"""
|
||||
from cuda.core import Program, ProgramOptions
|
||||
|
||||
# Compile all kernels at once
|
||||
program_options = ProgramOptions(std="c++17", arch=f"sm_{device.arch}")
|
||||
prog = Program(KERNELS_CODE, code_type="c++", options=program_options)
|
||||
mod = prog.compile(
|
||||
"cubin",
|
||||
name_expressions=(
|
||||
"vector_add<float>",
|
||||
"saxpy<float>",
|
||||
"vector_transform<float>",
|
||||
),
|
||||
)
|
||||
|
||||
# Extract individual kernels
|
||||
return {
|
||||
"vector_add": mod.get_kernel("vector_add<float>"),
|
||||
"saxpy": mod.get_kernel("saxpy<float>"),
|
||||
"vector_transform": mod.get_kernel("vector_transform<float>"),
|
||||
}
|
||||
|
||||
|
||||
def run(size):
|
||||
"""Main execution function"""
|
||||
|
||||
# =================================================================
|
||||
# Device Initialization using cuda.core
|
||||
# =================================================================
|
||||
with nvtx.annotate("Device Initialization", color="green"):
|
||||
try:
|
||||
# Create device object (defaults to device 0)
|
||||
dev = Device()
|
||||
dev.set_current()
|
||||
|
||||
print()
|
||||
print(f"Device: {dev.name}")
|
||||
print(f"Compute Capability: sm_{dev.arch}")
|
||||
print()
|
||||
|
||||
# Synchronize device
|
||||
dev.sync()
|
||||
|
||||
except Exception as e:
|
||||
print("ERROR: CUDA initialization failed!")
|
||||
print(f"Error: {e}")
|
||||
sys.exit(1)
|
||||
|
||||
print("Profiling cuda.core Custom Kernels")
|
||||
print(f"Array size: {size:,}\n")
|
||||
|
||||
# Constant for SAXPY operation
|
||||
alpha = 2.5
|
||||
|
||||
# Initialize random seed
|
||||
rng = cp.random.default_rng(42)
|
||||
|
||||
# =================================================================
|
||||
# Phase 1: Create GPU Arrays with CuPy
|
||||
# =================================================================
|
||||
with nvtx.annotate("Create GPU Arrays", color="yellow"):
|
||||
a_gpu = rng.standard_normal(size, dtype=cp.float32)
|
||||
b_gpu = rng.standard_normal(size, dtype=cp.float32)
|
||||
dev.sync()
|
||||
|
||||
print("Phase 1: Created arrays on GPU with CuPy")
|
||||
print(f" Array shape: {a_gpu.shape}")
|
||||
print(f" Array dtype: {a_gpu.dtype}")
|
||||
print(
|
||||
f" Array a - Mean: {float(cp.mean(a_gpu)):.4f}, "
|
||||
f"Std: {float(cp.std(a_gpu)):.4f}"
|
||||
)
|
||||
print(
|
||||
f" Array b - Mean: {float(cp.mean(b_gpu)):.4f}, "
|
||||
f"Std: {float(cp.std(b_gpu)):.4f}\n"
|
||||
)
|
||||
|
||||
# =================================================================
|
||||
# Phase 2: cuda.core Custom Kernels on GPU
|
||||
# =================================================================
|
||||
with nvtx.annotate("cuda.core Custom Kernels", color="purple"):
|
||||
print("Phase 2: cuda.core custom CUDA C++ kernels on GPU")
|
||||
|
||||
# Create a stream for cuda.core operations
|
||||
stream = dev.create_stream()
|
||||
try:
|
||||
with nvtx.annotate("Compile Kernels", color="cyan"):
|
||||
kernels_dict = get_cuda_core_kernels(dev)
|
||||
stream.sync()
|
||||
print("Compiled custom CUDA C++ kernels")
|
||||
|
||||
# Prepare launch configuration
|
||||
# Grid-stride loops in kernels handle any grid size robustly
|
||||
block = 256
|
||||
grid = (size + block - 1) // block
|
||||
config = LaunchConfig(grid=grid, block=block)
|
||||
|
||||
# Execute cuda.core vector_add kernel
|
||||
with nvtx.annotate("Vector Add (cuda.core)", color="cyan"):
|
||||
c_cuda = cp.empty_like(a_gpu)
|
||||
launch(
|
||||
stream,
|
||||
config,
|
||||
kernels_dict["vector_add"],
|
||||
a_gpu.data.ptr,
|
||||
b_gpu.data.ptr,
|
||||
c_cuda.data.ptr,
|
||||
cp.uint64(size),
|
||||
)
|
||||
stream.sync()
|
||||
|
||||
# Execute cuda.core SAXPY kernel
|
||||
with nvtx.annotate("SAXPY (cuda.core)", color="cyan"):
|
||||
y_cuda = b_gpu.copy()
|
||||
launch(
|
||||
stream,
|
||||
config,
|
||||
kernels_dict["saxpy"],
|
||||
np.float32(alpha),
|
||||
a_gpu.data.ptr,
|
||||
y_cuda.data.ptr,
|
||||
cp.uint64(size),
|
||||
)
|
||||
stream.sync()
|
||||
|
||||
# Execute cuda.core vector_transform kernel
|
||||
with nvtx.annotate("Vector Transform (cuda.core)", color="cyan"):
|
||||
transform_cuda = cp.empty_like(a_gpu)
|
||||
launch(
|
||||
stream,
|
||||
config,
|
||||
kernels_dict["vector_transform"],
|
||||
a_gpu.data.ptr,
|
||||
transform_cuda.data.ptr,
|
||||
cp.uint64(size),
|
||||
)
|
||||
stream.sync()
|
||||
|
||||
print("Vector Addition (custom kernel)")
|
||||
print("SAXPY (custom kernel)")
|
||||
print("Vector Transform (custom kernel)\n")
|
||||
finally:
|
||||
stream.close()
|
||||
|
||||
# =================================================================
|
||||
# Phase 3: Generate Reference Results with CuPy (for verification)
|
||||
# =================================================================
|
||||
with nvtx.annotate("Generate Reference Results", color="blue"):
|
||||
print("Phase 3: Generate reference results for verification")
|
||||
|
||||
with nvtx.annotate("Vector Add (Reference)", color="cyan"):
|
||||
c_cupy = a_gpu + b_gpu
|
||||
dev.sync()
|
||||
|
||||
with nvtx.annotate("SAXPY (Reference)", color="cyan"):
|
||||
y_cupy = alpha * a_gpu + b_gpu
|
||||
dev.sync()
|
||||
|
||||
with nvtx.annotate("Vector Transform (Reference)", color="cyan"):
|
||||
transform_cupy = cp.sqrt(a_gpu * a_gpu + 1.0) + cp.sin(a_gpu)
|
||||
dev.sync()
|
||||
|
||||
print("Reference results generated\n")
|
||||
|
||||
# =================================================================
|
||||
# Phase 4: Verify Kernel Correctness
|
||||
# =================================================================
|
||||
with nvtx.annotate("Verification", color="green"):
|
||||
print("Phase 4: Verify kernel correctness")
|
||||
|
||||
# Verify custom kernels against reference results
|
||||
# Use relaxed tolerances for single-precision float comparisons
|
||||
# Small differences can occur due to instruction ordering and
|
||||
# compiler optimizations
|
||||
print(" Validating cuda.core kernels:")
|
||||
|
||||
print(" Vector Add: ", end="")
|
||||
vec_add_match = verify_array_result(c_cuda, c_cupy, rtol=1e-5, atol=1e-6)
|
||||
|
||||
print(" SAXPY: ", end="")
|
||||
saxpy_match = verify_array_result(y_cuda, y_cupy, rtol=1e-5, atol=1e-6)
|
||||
|
||||
print(" Transform: ", end="")
|
||||
transform_match = verify_array_result(
|
||||
transform_cuda, transform_cupy, rtol=1e-5, atol=1e-6
|
||||
)
|
||||
|
||||
all_pass = vec_add_match and saxpy_match and transform_match
|
||||
|
||||
if not all_pass:
|
||||
print("\n ERROR: Kernel verification failed!")
|
||||
return 1
|
||||
print()
|
||||
|
||||
# Final synchronization
|
||||
dev.sync()
|
||||
print("The sample is complete PASSED!")
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Kernel Nsys Profiling - Profile custom CUDA C++ "
|
||||
"kernels with cuda.core"
|
||||
)
|
||||
parser.add_argument(
|
||||
"-n",
|
||||
"--array-size",
|
||||
type=int,
|
||||
default=50000,
|
||||
metavar="N",
|
||||
help="Array size (default: 50,000)",
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
run(size=args.array_size)
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
@@ -0,0 +1,7 @@
|
||||
# Nsight System Kernels Profiling Sample - Requirements
|
||||
|
||||
numpy>=2.3.2
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
cupy-cuda13x>=13.0.0
|
||||
nvtx
|
||||
73
python/1_GettingStarted/numpyVsCupy/README.md
Normal file
73
python/1_GettingStarted/numpyVsCupy/README.md
Normal file
@@ -0,0 +1,73 @@
|
||||
# Sample: Numpy vs. Cupy (Python)
|
||||
|
||||
## Description
|
||||
|
||||
This sample demonstrates performance comparison between NumPy (CPU) and CuPy (GPU) for matrix multiplication operations. It benchmarks the execution time of matrix dot products on both CPU and GPU, showing the performance benefits of GPU acceleration for numerical computations.
|
||||
|
||||
## What you will learn
|
||||
|
||||
- How to set up and use CuPy for GPU-accelerated numerical computations.
|
||||
- How to benchmark NumPy vs CuPy performance for matrix operations.
|
||||
- How to transfer data between CPU (NumPy) and GPU (CuPy) memory using `cp.asarray()`.
|
||||
- How to use CUDA device management with the cuda-core library.
|
||||
- How to validate computational results between CPU and GPU implementations using `np.testing.assert_allclose()`.
|
||||
- How to handle GPU warmup to avoid first-run overhead in benchmarking.
|
||||
- How to create and manage explicit CUDA streams with `device.create_stream()`.
|
||||
- How to properly cleanup streams with `stream.close()` in try/finally blocks.
|
||||
- How to access GPU device information (name, compute capability).
|
||||
- How to create timing context managers for performance measurement using CUDA events.
|
||||
|
||||
## Key libraries
|
||||
|
||||
- `numpy`
|
||||
- `cupy`
|
||||
- `cuda-core`
|
||||
|
||||
## Key APIs
|
||||
|
||||
**From cuda.core:**
|
||||
- `Device()` – Get CUDA device object for specific GPU
|
||||
- `device.create_stream()` – Create explicit CUDA stream
|
||||
- `stream.close()` – Close and cleanup stream resources
|
||||
|
||||
## Requirements
|
||||
1. **NVIDIA Graphics Card** with CUDA support
|
||||
2. **CUDA Drivers** installed on your system
|
||||
3. **CUDA Toolkit** installed on your system
|
||||
4. **Python 3.12 or newer**
|
||||
|
||||
**Install packages:**
|
||||
```bash
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
## How to run
|
||||
|
||||
Basic usage:
|
||||
```bash
|
||||
# Pre-steps:
|
||||
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
|
||||
# Run from the Python directory:
|
||||
cd /path/to/numpyVsCupy/Python
|
||||
python -m 1_GettingStarted.numpyVsCupy.numpyVsCupy
|
||||
```
|
||||
|
||||
With custom parameters:
|
||||
```bash
|
||||
python -m 1_GettingStarted.numpyVsCupy.numpyVsCupy --n_size 5000
|
||||
```
|
||||
|
||||
### Command line arguments
|
||||
|
||||
- `--n_size`, `-n`: Size of the matrix (n * n) for benchmarking (default: 4096)
|
||||
|
||||
## Expected Output
|
||||
```
|
||||
Validation PASSED: NumPy and CuPy results match within tolerance
|
||||
Demo completed successfully!
|
||||
```
|
||||
|
||||
## Files
|
||||
- `numpyVsCupy.py` – Python implementation
|
||||
- `README.md` – This file
|
||||
- `requirements.txt` – Required packages
|
||||
141
python/1_GettingStarted/numpyVsCupy/numpyVsCupy.py
Normal file
141
python/1_GettingStarted/numpyVsCupy/numpyVsCupy.py
Normal file
@@ -0,0 +1,141 @@
|
||||
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
|
||||
#
|
||||
# Redistribution and use in source and binary forms, with or without
|
||||
# modification, are permitted provided that the following conditions
|
||||
# are met:
|
||||
# * Redistributions of source code must retain the above copyright
|
||||
# notice, this list of conditions and the following disclaimer.
|
||||
# * Redistributions in binary form must reproduce the above copyright
|
||||
# notice, this list of conditions and the following disclaimer in the
|
||||
# distribution and/or other materials provided with the distribution.
|
||||
# * Neither the name of NVIDIA CORPORATION nor the names of its
|
||||
# contributors may be used to endorse or promote products derived
|
||||
# from this software without specific prior written permission.
|
||||
#
|
||||
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
|
||||
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
||||
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
||||
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
|
||||
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
|
||||
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
|
||||
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
|
||||
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
|
||||
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
import argparse
|
||||
import contextlib
|
||||
import sys
|
||||
import time
|
||||
from pathlib import Path
|
||||
|
||||
try:
|
||||
import cupy as cp
|
||||
import numpy as np
|
||||
from cuda.core import Device, EventOptions
|
||||
except ImportError as e:
|
||||
print(f"Error: Required package not found: {e}")
|
||||
print("Please install from requirements.txt:")
|
||||
print(" pip install -r requirements.txt")
|
||||
sys.exit(1)
|
||||
|
||||
# Add parent directory to path to import utilities
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent.parent / "Utilities"))
|
||||
from cuda_samples_utils import verify_array_result
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def timer(message):
|
||||
"""CPU timing context manager."""
|
||||
start = time.time()
|
||||
yield
|
||||
end = time.time()
|
||||
print(f"{message}: {(end - start):.6f} seconds")
|
||||
|
||||
|
||||
@contextlib.contextmanager
|
||||
def gpu_timer(message, stream):
|
||||
"""GPU timing context manager using cuda.core CUDA events."""
|
||||
event_options = EventOptions(enable_timing=True)
|
||||
start_event = stream.record(options=event_options)
|
||||
yield
|
||||
end_event = stream.record(options=event_options)
|
||||
end_event.sync()
|
||||
|
||||
elapsed_time_ms = end_event - start_event # Returns milliseconds
|
||||
elapsed_time_s = elapsed_time_ms / 1000.0 # Convert to seconds
|
||||
print(f"{message}: {elapsed_time_s:.6f} seconds")
|
||||
|
||||
|
||||
def warmup():
|
||||
# Pre-runs a simple GPU operation to avoid first-run overhead in benchmarking.
|
||||
print("Warmup...")
|
||||
a_cp = cp.ones((16, 16))
|
||||
b_cp = cp.ones((16, 16))
|
||||
result_cp = cp.dot(a_cp, b_cp)
|
||||
return result_cp
|
||||
|
||||
|
||||
def run(n):
|
||||
# Benchmarks NumPy vs. CuPy matrix multiplication for n x n random arrays.
|
||||
# Prints timing results.
|
||||
|
||||
device = Device() # Use device 0 explicitly
|
||||
device.set_current()
|
||||
major, minor = device.compute_capability
|
||||
print()
|
||||
print(f"Device Name: {device.name}, SM: {major}.{minor}")
|
||||
print()
|
||||
|
||||
# Create explicit stream
|
||||
stream = device.create_stream()
|
||||
|
||||
try:
|
||||
# Warm up GPU before measuring
|
||||
warmup()
|
||||
stream.sync()
|
||||
|
||||
# Generate random matrices on CPU
|
||||
a_np = np.random.rand(n, n)
|
||||
b_np = np.random.rand(n, n)
|
||||
|
||||
# NumPy dot product (CPU)
|
||||
with timer(f"NumPy dot of {n}*{n} arrays"):
|
||||
result_np = np.dot(a_np, b_np)
|
||||
|
||||
# Transfer NumPy arrays to GPU (using events for timing)
|
||||
with gpu_timer("Transfer arrays to GPU", stream):
|
||||
a_cp = cp.asarray(a_np)
|
||||
b_cp = cp.asarray(b_np)
|
||||
|
||||
# CuPy dot product (GPU) - using events for accurate GPU timing
|
||||
with gpu_timer(f"CuPy dot of {n}*{n} arrays", stream):
|
||||
result_cp = cp.dot(a_cp, b_cp)
|
||||
|
||||
print()
|
||||
# Result validation
|
||||
if not verify_array_result(result_np, result_cp.get()):
|
||||
print(
|
||||
"Validation FAILED: NumPy and CuPy results do not match "
|
||||
"within tolerance"
|
||||
)
|
||||
sys.exit(1)
|
||||
|
||||
print("Validation PASSED: NumPy and CuPy results match within tolerance")
|
||||
finally:
|
||||
stream.close()
|
||||
|
||||
|
||||
def main():
|
||||
parser = argparse.ArgumentParser()
|
||||
parser.add_argument(
|
||||
"--n_size", "-n", default=4096, type=int, help="Size of the matrix(n * n)."
|
||||
)
|
||||
args = parser.parse_args()
|
||||
run(args.n_size)
|
||||
print("Demo completed successfully!")
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
main()
|
||||
7
python/1_GettingStarted/numpyVsCupy/requirements.txt
Normal file
7
python/1_GettingStarted/numpyVsCupy/requirements.txt
Normal file
@@ -0,0 +1,7 @@
|
||||
# Numpy vs. Cupy - Requirements
|
||||
# Install with: pip install -r requirements.txt
|
||||
|
||||
numpy>=2.3.2
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
cupy-cuda13x>=13.0.0
|
||||
263
python/1_GettingStarted/simplePrint/README.md
Normal file
263
python/1_GettingStarted/simplePrint/README.md
Normal file
@@ -0,0 +1,263 @@
|
||||
# simplePrint - Printing from CUDA Kernels
|
||||
|
||||
## Description
|
||||
|
||||
This sample demonstrates how to use `printf()` inside CUDA kernels using **two different approaches**:
|
||||
|
||||
1. **CUDA C++ kernels** compiled with `cuda.core.Program` - Full C++ features and control
|
||||
2. **Numba CUDA kernels** - Pythonic kernel authoring using `numba.cuda.grid()` for modern indexing
|
||||
|
||||
The sample shows basic device management, kernel compilation with inline CUDA C++ code, and multi-dimensional kernel launches (2D grid × 3D blocks) using modern CUDA Python. The Numba example demonstrates the recommended `numba.cuda.grid()` indexing style while also showing how it relates to classic CUDA C++ block/thread IDs. Both approaches use `cuda.core` APIs for stream management and synchronization, demonstrating interoperability.
|
||||
|
||||
This is the Python equivalent of the C++ `simplePrintf` sample, enhanced with Numba CUDA examples.
|
||||
|
||||
## Key Concepts
|
||||
|
||||
CUDA Python (cuda.core), Numba CUDA, Kernel Compilation, Printf in Kernels, Multi-dimensional Launch, Pythonic GPU Programming, Modern Thread Indexing (grid()), Stream-based Execution, cuda.core/Numba Interoperability
|
||||
|
||||
## CUDA APIs involved
|
||||
|
||||
### [cuda.core (cuda-python)](https://nvidia.github.io/cuda-python/)
|
||||
|
||||
- `Device()` - Device management
|
||||
- `Device.create_stream()` - Create CUDA streams
|
||||
- `Stream.sync()` - Synchronize stream execution
|
||||
- `Program()` - Compile CUDA C++ kernels
|
||||
- `LaunchConfig()` - Configure kernel launch
|
||||
- `launch()` - Execute kernels on streams
|
||||
|
||||
### [Numba CUDA](https://nvidia.github.io/numba-cuda/)
|
||||
|
||||
- `@cuda.jit` - JIT compile Python functions to CUDA kernels
|
||||
- `cuda.grid()` - Get global thread position (recommended modern approach)
|
||||
- `cuda.blockIdx`, `cuda.threadIdx` - Thread/block indices (classic style)
|
||||
- `cuda.gridDim`, `cuda.blockDim` - Grid/block dimensions
|
||||
- **Note:** Uses `cuda.core` APIs for stream management (interoperability)
|
||||
|
||||
### CUDA Kernel Functions
|
||||
|
||||
- `printf()` - Print from device code (C++)
|
||||
- `print()` - Print from device code (Numba, limited formatting)
|
||||
- `blockIdx`, `threadIdx` - Thread/block indices
|
||||
- `gridDim`, `blockDim` - Grid/block dimensions
|
||||
|
||||
### What You Learn
|
||||
|
||||
- Device initialization with `cuda.core.Device`
|
||||
- Compiling CUDA C++ kernels with `Program` and `ProgramOptions`
|
||||
- Writing Pythonic CUDA kernels with Numba's `@cuda.jit` decorator
|
||||
- Using `numba.cuda.grid()` for modern thread indexing (recommended approach)
|
||||
- Understanding the relationship between global coordinates and classic block/thread IDs
|
||||
- **Interoperability**: Using `cuda.core` streams with Numba CUDA kernels
|
||||
- Comparing CUDA C++ vs Pythonic kernel authoring approaches
|
||||
- Multi-dimensional kernel launches (2D grid, 3D blocks)
|
||||
- Using streams for kernel execution and synchronization
|
||||
- Using `printf()` and `print()` in GPU kernels for debugging
|
||||
- Understanding print limitations in Numba CUDA (no f-strings)
|
||||
- Proper error handling and resource management
|
||||
|
||||
## Requirements
|
||||
|
||||
### Hardware:
|
||||
|
||||
- NVIDIA GPU with Compute Capability 7.0 or higher
|
||||
- Minimum GPU memory: 512 MB
|
||||
|
||||
### Software:
|
||||
|
||||
- CUDA Toolkit 13.0 or newer
|
||||
- Python 3.10 or newer
|
||||
- `cuda-python` package (13.0+)
|
||||
- `cuda-core` package (>=0.6.0)
|
||||
- `numba-cuda` package (0.24.0+, for Pythonic kernel authoring)
|
||||
|
||||
Download and install:
|
||||
- [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads)
|
||||
- [cuda-python package](https://nvidia.github.io/cuda-python/): `pip install cuda-python`
|
||||
- [numba-cuda](https://nvidia.github.io/numba-cuda/): `pip install numba-cuda`
|
||||
|
||||
## Build and Run
|
||||
|
||||
```bash
|
||||
# Install dependencies
|
||||
pip install -r requirements.txt
|
||||
|
||||
# Run the sample
|
||||
python simplePrint.py
|
||||
```
|
||||
|
||||
## Expected Output
|
||||
|
||||
```
|
||||
Simple Print - Printing from CUDA Kernels
|
||||
Demonstrating both CUDA C++ and Numba CUDA approaches
|
||||
|
||||
Device: <Your GPU Name>
|
||||
Compute Capability: sm_<XX>
|
||||
|
||||
======================================================================
|
||||
METHOD 1: CUDA C++ Kernel (via cuda.core.Program)
|
||||
======================================================================
|
||||
Advantage: Full C++ features, better for complex kernels
|
||||
|
||||
Compiling CUDA C++ kernel...
|
||||
Kernel compiled successfully.
|
||||
|
||||
Kernel configuration:
|
||||
Grid: (2, 2)
|
||||
Block: (2, 2, 2)
|
||||
Total threads: 32
|
||||
|
||||
Launching kernel with value=10. Output:
|
||||
|
||||
[0, 0]: Value is: 10
|
||||
[0, 1]: Value is: 10
|
||||
[0, 2]: Value is: 10
|
||||
[0, 3]: Value is: 10
|
||||
[0, 4]: Value is: 10
|
||||
[0, 5]: Value is: 10
|
||||
[0, 6]: Value is: 10
|
||||
[0, 7]: Value is: 10
|
||||
[1, 0]: Value is: 10
|
||||
...
|
||||
[3, 7]: Value is: 10
|
||||
|
||||
CUDA C++ kernel execution complete.
|
||||
|
||||
|
||||
======================================================================
|
||||
METHOD 2: Numba CUDA Kernel (Pythonic / modern indexing)
|
||||
======================================================================
|
||||
Advantage: Uses numba.cuda.grid(3) for global indexing,
|
||||
while still showing classic CUDA C++ IDs for reference.
|
||||
Uses cuda.core for stream management (interoperability).
|
||||
|
||||
Kernel configuration:
|
||||
Grid: (2, 2)
|
||||
Block: (2, 2, 2)
|
||||
Total threads: 32
|
||||
|
||||
Launching Numba CUDA kernel (grid(3) + classic IDs) with value=10:
|
||||
Uses numba.cuda.grid(3) to get global (x, y, z),
|
||||
and prints the corresponding blockId/threadId like the C++ sample.
|
||||
Stream managed by cuda.core for consistency with C++ example.
|
||||
|
||||
global[ 0 , 0 , 0 ] -> [ 0 , 0 ]: Value is: 10
|
||||
global[ 1 , 0 , 0 ] -> [ 0 , 1 ]: Value is: 10
|
||||
global[ 0 , 1 , 0 ] -> [ 0 , 2 ]: Value is: 10
|
||||
...
|
||||
global[ 3 , 3 , 1 ] -> [ 3 , 7 ]: Value is: 10
|
||||
|
||||
Numba CUDA kernel execution complete.
|
||||
|
||||
======================================================================
|
||||
Done! Both kernel approaches demonstrated successfully.
|
||||
======================================================================
|
||||
```
|
||||
|
||||
## Understanding the Output
|
||||
|
||||
- **Grid**: 2×2 = 4 blocks (labeled 0-3)
|
||||
- **Block**: 2×2×2 = 8 threads per block (labeled 0-7)
|
||||
- **Total**: 32 threads, each printing its position and value
|
||||
|
||||
### CUDA C++ Kernel:
|
||||
Each thread calculates:
|
||||
- Block ID (linear): `blockIdx.y * gridDim.x + blockIdx.x`
|
||||
- Thread ID (linear): `threadIdx.z * blockDim.x * blockDim.y + threadIdx.y * blockDim.x + threadIdx.x`
|
||||
|
||||
### Numba CUDA Kernel:
|
||||
Each thread shows:
|
||||
- **Global position** using `numba.cuda.grid(3)` → `(x, y, z)` coordinates across entire grid
|
||||
- **Classic IDs** (block ID, thread ID) calculated the same way as C++ for comparison
|
||||
- This demonstrates how modern indexing relates to traditional CUDA C++ style
|
||||
|
||||
### Comparing the Two Approaches
|
||||
|
||||
**CUDA C++ Kernel (Method 1):**
|
||||
- Uses C++ syntax and `printf()` with full formatting control
|
||||
- Requires compilation via `cuda.core.Program`
|
||||
- Best for complex kernels needing C++ features (templates, libraries, etc.)
|
||||
- Uses classic block/thread ID indexing
|
||||
- Output: `[0, 0]: Value is: 10` (clean formatting)
|
||||
|
||||
**Numba CUDA Kernel (Method 2):**
|
||||
- Uses Python syntax with `@cuda.jit` decorator
|
||||
- JIT compiled automatically when called
|
||||
- Best for prototyping and simpler kernels
|
||||
- **Modern indexing**: Uses `numba.cuda.grid(3)` to get global thread coordinates (recommended)
|
||||
- Also shows classic block/thread IDs to help relate the two indexing models
|
||||
- **Interoperability**: Uses `cuda.core` streams via `stream` for consistency
|
||||
- Demonstrates that numba-cuda kernels can work seamlessly with cuda.core infrastructure
|
||||
- Limited print formatting (no f-strings, basic `print()` only; adds spaces between arguments)
|
||||
- Output: `global[ 0 , 0 , 0 ] -> [ 0 , 0 ]: Value is: 10` (shows both indexing styles; note extra spaces due to `print()` behavior)
|
||||
|
||||
## Experiments
|
||||
|
||||
Try modifying:
|
||||
|
||||
### For Both Approaches:
|
||||
- **Grid size**: Change `grid=(4, 4)` for 16 blocks
|
||||
- **Block size**: Change `block=(4, 4, 4)` for 64 threads per block
|
||||
- **Conditional printing**: Print only from specific threads (e.g., `if threadId == 0:`)
|
||||
|
||||
### CUDA C++ Specific:
|
||||
- **Format strings**: Experiment with different `printf()` formats
|
||||
- **Kernel code**: Add complex C++ computations before printing
|
||||
- **External libraries**: Include CUDA math libraries or device functions (e.g., `<cuda/std/cmath>`, `<cub/cub.cuh>`)
|
||||
|
||||
### Numba CUDA Specific:
|
||||
- **Grid indexing**: Try `numba.cuda.grid(1)` or `numba.cuda.grid(2)` for different dimensions
|
||||
- **Conditional printing**: Print only from threads where `x == 0` or `y == z`
|
||||
- **Python operations**: Use NumPy-like operations in the kernel
|
||||
- **Device math libraries**: Use [nvmath-python device APIs](https://docs.nvidia.com/cuda/nvmath-python/latest/device-apis/index.html) for optimized math operations (similar to CUDA math libraries in C++)
|
||||
- **Shared memory**: Add `numba.cuda.shared.array()` for fast inter-thread communication
|
||||
- **Atomic operations**: Try `numba.cuda.atomic.add()` for thread-safe updates
|
||||
- **Print variations**: Experiment with what numba-cuda's `print()` can and cannot handle
|
||||
- **Streams**: Create multiple `cuda.core` streams and launch numba-cuda kernels on them concurrently
|
||||
- **Interoperability**: Mix numba-cuda kernels and CUDA C++ kernels on the same stream
|
||||
|
||||
## Notes
|
||||
|
||||
### General:
|
||||
- Printing from GPU is relatively slow - use sparingly in production code
|
||||
- Printf output is buffered and limited (~1MB buffer on most GPUs)
|
||||
|
||||
### CUDA C++ Kernels:
|
||||
- Always call `stream.sync()` after kernel launch to flush printf output
|
||||
- Full `printf()` format string support (%, flags, width, precision)
|
||||
|
||||
### Numba CUDA Kernels:
|
||||
- **Recommended**: Use `numba.cuda.grid(ndim)` for thread indexing (modern, Pythonic)
|
||||
- `grid(1)` for 1D indexing, `grid(2)` for 2D, `grid(3)` for 3D
|
||||
- Returns global thread position across the entire grid
|
||||
- **Interoperability**: Use `cuda.core` streams with Numba kernels via `stream`
|
||||
- Create streams: `stream = device.create_stream()`
|
||||
- Launch kernels: `kernel[grid, block, stream](args)`
|
||||
- Synchronize: `stream.sync()`
|
||||
- Numba's `print()` has limited capabilities compared to Python's `print()`
|
||||
- F-strings are NOT supported in Numba CUDA kernels
|
||||
- Use comma-separated arguments: `print("Value:", x)` instead of f-strings
|
||||
- **Note**: `print()` automatically adds spaces between comma-separated arguments (e.g., `print("[", x, "]")` outputs `[ 0 ]` not `[0]`)
|
||||
- Always synchronize the stream to flush output
|
||||
|
||||
## Files
|
||||
|
||||
- `simplePrint.py` - Python implementation using cuda.core API
|
||||
- `README.md` - This file
|
||||
- `requirements.txt` - Sample dependencies
|
||||
|
||||
## See Also
|
||||
|
||||
### CUDA Python (cuda.core):
|
||||
- [cuda.core Documentation](https://nvidia.github.io/cuda-python/)
|
||||
- [CUDA Python Examples](https://github.com/NVIDIA/cuda-python/tree/main/cuda_core/examples)
|
||||
|
||||
### Numba CUDA:
|
||||
- [Numba CUDA Documentation](https://nvidia.github.io/numba-cuda/)
|
||||
- [numba.cuda.grid() Reference](https://nvidia.github.io/numba-cuda/reference/kernel.html#numba.cuda.grid)
|
||||
- [nvmath-python Device APIs](https://docs.nvidia.com/cuda/nvmath-python/latest/device-apis/index.html) - Optimized math operations for Numba CUDA kernels
|
||||
|
||||
### CUDA References:
|
||||
- [CUDA C Programming Guide - Printf](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#formatted-output)
|
||||
- [C++ simplePrintf Sample](https://github.com/NVIDIA/cuda-samples/tree/master/Samples/0_Introduction/simplePrintf)
|
||||
7
python/1_GettingStarted/simplePrint/requirements.txt
Normal file
7
python/1_GettingStarted/simplePrint/requirements.txt
Normal file
@@ -0,0 +1,7 @@
|
||||
# Simple Printf Sample - Requirements
|
||||
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
# Numba JIT uses nvJitLink from pip; keep in step with cuda-bindings (e.g. 13.2.x).
|
||||
nvidia-nvjitlink>=13.2.0
|
||||
numba-cuda>=0.24.0
|
||||
287
python/1_GettingStarted/simplePrint/simplePrint.py
Normal file
287
python/1_GettingStarted/simplePrint/simplePrint.py
Normal file
@@ -0,0 +1,287 @@
|
||||
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
|
||||
#
|
||||
# Redistribution and use in source and binary forms, with or without
|
||||
# modification, are permitted provided that the following conditions
|
||||
# are met:
|
||||
# * Redistributions of source code must retain the above copyright
|
||||
# notice, this list of conditions and the following disclaimer.
|
||||
# * Redistributions in binary form must reproduce the above copyright
|
||||
# notice, this list of conditions and the following disclaimer in the
|
||||
# documentation and/or other materials provided with the distribution.
|
||||
# * Neither the name of NVIDIA CORPORATION nor the names of its
|
||||
# contributors may be used to endorse or promote products derived
|
||||
# from this software without specific prior written permission.
|
||||
#
|
||||
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
|
||||
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
||||
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
||||
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
|
||||
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
|
||||
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
|
||||
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
|
||||
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
|
||||
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
"""
|
||||
Simple Print - Printing from CUDA Kernels
|
||||
|
||||
This sample demonstrates how to print output from CUDA kernels using printf().
|
||||
It shows:
|
||||
1. Device management with cuda.core.Device
|
||||
2. Compiling CUDA C++ code that uses printf()
|
||||
3. Launching kernels with 2D grids and 3D blocks
|
||||
4. Seeing kernel output printed to stdout
|
||||
5. Using Numba CUDA for Pythonic kernel authoring
|
||||
|
||||
This sample demonstrates both approaches:
|
||||
- CUDA C++ kernels compiled via cuda.core.Program (more control, C++ features)
|
||||
- Numba CUDA kernels (more Pythonic, easier to write)
|
||||
|
||||
This is the Python equivalent of the C++ simplePrintf sample.
|
||||
"""
|
||||
|
||||
import sys
|
||||
import traceback
|
||||
|
||||
try:
|
||||
from cuda.core import Device, LaunchConfig, Program, ProgramOptions, launch
|
||||
except ImportError as e:
|
||||
print(f"Error: Required package not found: {e}")
|
||||
print("Please install from requirements.txt:")
|
||||
print(" pip install -r requirements.txt")
|
||||
sys.exit(1)
|
||||
|
||||
try:
|
||||
from numba import cuda as numba_cuda
|
||||
|
||||
NUMBA_AVAILABLE = True
|
||||
except ImportError:
|
||||
NUMBA_AVAILABLE = False
|
||||
print("Warning: numba not found. Numba CUDA example will be skipped.")
|
||||
print("To install: pip install numba")
|
||||
|
||||
|
||||
# CUDA C++ kernel with printf
|
||||
# This kernel prints the block index, thread index, and a value from each thread
|
||||
PRINTF_KERNEL = """
|
||||
extern "C"
|
||||
__global__ void printKernel(int val) {
|
||||
// Calculate linear block index from 2D grid
|
||||
int blockId = blockIdx.y * gridDim.x + blockIdx.x;
|
||||
|
||||
// Calculate linear thread index from 3D block
|
||||
int threadId = threadIdx.z * blockDim.x * blockDim.y +
|
||||
threadIdx.y * blockDim.x +
|
||||
threadIdx.x;
|
||||
|
||||
// Print from each thread
|
||||
printf("[%d, %d]:\\t\\tValue is: %d\\n", blockId, threadId, val);
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
# Numba CUDA kernel - Pythonic equivalent using numba.cuda.grid()
|
||||
# This demonstrates the same functionality using Numba's Python-based kernel syntax
|
||||
if NUMBA_AVAILABLE:
|
||||
|
||||
@numba_cuda.jit
|
||||
def numba_print_kernel(val):
|
||||
"""
|
||||
Numba CUDA kernel showing the *recommended* grid() indexing style,
|
||||
while also relating it to the classic CUDA C++ blockId/threadId.
|
||||
|
||||
- Primary view: global 3D coordinates from numba.cuda.grid(3)
|
||||
(modern, Pythonic way to index work for a 3D thread layout).
|
||||
- Secondary view: linear blockId / threadId matching the CUDA C++
|
||||
printf sample, to help CUDA C++ users connect the two models.
|
||||
"""
|
||||
# Modern / recommended view: global 3D thread coordinates
|
||||
x, y, z = numba_cuda.grid(3)
|
||||
|
||||
# Classic CUDA-style indices, same formulas as the C++ sample
|
||||
block_id = numba_cuda.blockIdx.y * numba_cuda.gridDim.x + numba_cuda.blockIdx.x
|
||||
|
||||
thread_id = (
|
||||
numba_cuda.threadIdx.z * numba_cuda.blockDim.x * numba_cuda.blockDim.y
|
||||
+ numba_cuda.threadIdx.y * numba_cuda.blockDim.x
|
||||
+ numba_cuda.threadIdx.x
|
||||
)
|
||||
|
||||
# Print both views side-by-side
|
||||
# Note: Numba print() adds spaces between comma-separated args
|
||||
print(
|
||||
"global[",
|
||||
x,
|
||||
",",
|
||||
y,
|
||||
",",
|
||||
z,
|
||||
"] -> [",
|
||||
block_id,
|
||||
",",
|
||||
thread_id,
|
||||
"]:\t\tValue is:",
|
||||
val,
|
||||
)
|
||||
|
||||
|
||||
def run_cuda_cpp_kernel(device, test_value=10):
|
||||
"""
|
||||
Demonstrate printing from CUDA C++ kernel compiled with cuda.core.
|
||||
|
||||
This approach gives you full access to CUDA C++ features and allows
|
||||
for more complex kernel implementations.
|
||||
"""
|
||||
print("=" * 70)
|
||||
print("METHOD 1: CUDA C++ Kernel (via cuda.core.Program)")
|
||||
print("=" * 70)
|
||||
print("Advantage: Full C++ features, better for complex kernels")
|
||||
print()
|
||||
|
||||
# Compile the kernel
|
||||
print("Compiling CUDA C++ kernel...")
|
||||
program_options = ProgramOptions(std="c++17", arch=f"sm_{device.arch}")
|
||||
prog = Program(PRINTF_KERNEL, code_type="c++", options=program_options)
|
||||
mod = prog.compile("cubin", name_expressions=("printKernel",))
|
||||
kernel = mod.get_kernel("printKernel")
|
||||
print("Kernel compiled successfully.\n")
|
||||
|
||||
# Create stream for kernel execution
|
||||
stream = device.create_stream()
|
||||
|
||||
# Configure kernel launch
|
||||
# Using 2D grid (2x2) and 3D blocks (2x2x2)
|
||||
grid_x, grid_y = 2, 2
|
||||
block_x, block_y, block_z = 2, 2, 2
|
||||
|
||||
print("Kernel configuration:")
|
||||
print(f" Grid: ({grid_x}, {grid_y})")
|
||||
print(f" Block: ({block_x}, {block_y}, {block_z})")
|
||||
print(f" Total threads: {grid_x * grid_y * block_x * block_y * block_z}")
|
||||
print()
|
||||
|
||||
# Launch configuration with 2D grid and 3D block
|
||||
config = LaunchConfig(grid=(grid_x, grid_y), block=(block_x, block_y, block_z))
|
||||
|
||||
print(f"Launching kernel with value={test_value}. Output:\n")
|
||||
try:
|
||||
# Launch kernel
|
||||
launch(stream, config, kernel, test_value)
|
||||
|
||||
# Synchronize to ensure printf output is flushed
|
||||
stream.sync()
|
||||
|
||||
print("\nCUDA C++ kernel execution complete.")
|
||||
except Exception as e:
|
||||
print(f"\nError during kernel execution: {e}")
|
||||
traceback.print_exc()
|
||||
return 1
|
||||
finally:
|
||||
# Cleanup
|
||||
stream.close()
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
def run_numba_kernel(device, test_value=10):
|
||||
"""
|
||||
Demonstrate printing from a Numba CUDA kernel.
|
||||
|
||||
This example uses numba.cuda.grid(3) as the primary indexing mechanism
|
||||
(recommended modern style), and also prints the equivalent blockId /
|
||||
threadId used in the CUDA C++ printf sample for side-by-side comparison.
|
||||
|
||||
Uses cuda.core APIs for stream management, demonstrating interoperability
|
||||
between Numba CUDA kernels and cuda.core infrastructure.
|
||||
"""
|
||||
print("\n")
|
||||
print("=" * 70)
|
||||
print("METHOD 2: Numba CUDA Kernel (Pythonic / modern indexing)")
|
||||
print("=" * 70)
|
||||
print("Advantage: Uses numba.cuda.grid(3) for global indexing,")
|
||||
print(" while still showing classic CUDA C++ IDs for reference.")
|
||||
print(" Uses cuda.core for stream management (interoperability).")
|
||||
print()
|
||||
|
||||
# Same launch configuration as the C++ version
|
||||
grid_x, grid_y = 2, 2
|
||||
block_x, block_y, block_z = 2, 2, 2
|
||||
|
||||
print("Kernel configuration:")
|
||||
print(f" Grid: ({grid_x}, {grid_y})")
|
||||
print(f" Block: ({block_x}, {block_y}, {block_z})")
|
||||
print(f" Total threads: {grid_x * grid_y * block_x * block_y * block_z}")
|
||||
print()
|
||||
|
||||
# Use cuda.core stream (same as C++ example) instead of numba.cuda.stream()
|
||||
stream = device.create_stream()
|
||||
|
||||
print(f"Launching Numba kernel (grid(3) + classic IDs) with value={test_value}:")
|
||||
print("Uses numba.cuda.grid(3) to get global (x, y, z),")
|
||||
print("and prints the corresponding blockId/threadId like the C++ sample.")
|
||||
print("Stream managed by cuda.core for consistency with C++ example.\n")
|
||||
|
||||
try:
|
||||
# Launch Numba kernel on cuda.core stream
|
||||
numba_print_kernel[(grid_x, grid_y), (block_x, block_y, block_z), stream](
|
||||
test_value
|
||||
)
|
||||
|
||||
# Synchronize cuda.core stream (same as C++ example)
|
||||
stream.sync()
|
||||
print("\nNumba CUDA kernel execution complete.")
|
||||
except Exception as e:
|
||||
print(f"\nError during Numba kernel execution: {e}")
|
||||
traceback.print_exc()
|
||||
return 1
|
||||
finally:
|
||||
# Cleanup
|
||||
stream.close()
|
||||
|
||||
return 0
|
||||
|
||||
|
||||
def main():
|
||||
"""Main function demonstrating printing from CUDA kernels using both approaches"""
|
||||
|
||||
print("Simple Print - Printing from CUDA Kernels")
|
||||
print("Demonstrating both CUDA C++ and Numba CUDA approaches")
|
||||
print()
|
||||
# Initialize device
|
||||
device = Device()
|
||||
device.set_current()
|
||||
|
||||
# Get device properties
|
||||
print(f"Device: {device.name}")
|
||||
print(f"Compute Capability: sm_{device.arch}")
|
||||
print()
|
||||
|
||||
# Value to pass to both kernels
|
||||
test_value = 10
|
||||
|
||||
# Run CUDA C++ kernel
|
||||
result = run_cuda_cpp_kernel(device, test_value)
|
||||
if result != 0:
|
||||
return result
|
||||
|
||||
# Run Numba kernel if available
|
||||
if NUMBA_AVAILABLE:
|
||||
result = run_numba_kernel(device, test_value)
|
||||
if result != 0:
|
||||
return result
|
||||
else:
|
||||
print("\n" + "=" * 70)
|
||||
print("Numba CUDA example skipped (numba not installed)")
|
||||
print("To run the Numba example: pip install numba")
|
||||
print("=" * 70)
|
||||
|
||||
print("\n" + "=" * 70)
|
||||
print("Done! Both kernel approaches demonstrated successfully.")
|
||||
print("=" * 70)
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
152
python/1_GettingStarted/systemInfo/README.md
Normal file
152
python/1_GettingStarted/systemInfo/README.md
Normal file
@@ -0,0 +1,152 @@
|
||||
# Sample: System Information Query (Python)
|
||||
|
||||
## Description
|
||||
|
||||
This sample demonstrates how to inspect the CUDA driver, NVML, and every
|
||||
installed GPU through the
|
||||
[`cuda.core.system`](https://nvidia.github.io/cuda-python/cuda-core/latest/)
|
||||
module.
|
||||
|
||||
`cuda.core.system` wraps the NVIDIA Management Library (NVML) and can be
|
||||
imported without CUDA being installed or initialized, so it is useful as a
|
||||
lightweight pre-flight check before any CUDA context is created. The script
|
||||
prints driver and NVML versions, the current process name, per-device
|
||||
metadata (name, compute capability, architecture, memory, PCI info,
|
||||
temperature, performance state), and, on multi-GPU systems, the topology
|
||||
and peer-to-peer capabilities between each pair of devices.
|
||||
|
||||
## What You'll Learn
|
||||
|
||||
- Querying CUDA driver and NVML versions with `cuda.core.system`
|
||||
- Enumerating GPUs without creating a CUDA context
|
||||
- Reading per-device metadata exposed by NVML (name, UUID, memory usage,
|
||||
temperature, performance state)
|
||||
- Inspecting GPU-to-GPU topology and peer-to-peer (P2P) capabilities
|
||||
|
||||
## Key Libraries
|
||||
|
||||
- [`cuda.core.system`](https://nvidia.github.io/cuda-python/cuda-core/latest/) - Python wrapper around NVML
|
||||
|
||||
## Key APIs
|
||||
|
||||
From `cuda.core.system`:
|
||||
|
||||
- `get_driver_version()`, `get_driver_version_full()`, `get_driver_branch()` - CUDA driver version tuple and branch string
|
||||
- `get_nvml_version()` - NVML library version
|
||||
- `get_num_devices()` - number of GPUs visible to NVML
|
||||
- `get_process_name(pid)` - process name for a given PID
|
||||
- `Device(index=...)` - NVML-backed device handle (no CUDA context required)
|
||||
- `name`, `uuid`, `cuda_compute_capability`, `arch`, `brand`
|
||||
- `memory_info` (`total`, `used`, `free`)
|
||||
- `pci_info` (`domain`, `bus`, `device`, `bus_id`)
|
||||
- `temperature.sensor(TemperatureSensors.TEMPERATURE_GPU)`
|
||||
- `performance_state`
|
||||
- `get_topology_common_ancestor(dev0, dev1)` - `GpuTopologyLevel` between two devices
|
||||
- `get_p2p_status(dev0, dev1, GpuP2PCapsIndex.P2P_CAPS_INDEX_READ)` - peer-access capability between two devices
|
||||
|
||||
Import stable symbols from the top-level `cuda.core` package (not `cuda.core.experimental`).
|
||||
|
||||
## Requirements
|
||||
|
||||
1. **NVIDIA Graphics Card** with CUDA support
|
||||
2. **CUDA Drivers** installed on your system
|
||||
3. **CUDA Toolkit** installed on your system
|
||||
4. **Python 3.12 or newer**
|
||||
|
||||
### Hardware
|
||||
|
||||
- One or more NVIDIA GPUs
|
||||
- Driver compatible with `cuda-python` 13.x
|
||||
|
||||
### Software
|
||||
|
||||
- CUDA Toolkit 13.0 or newer (matches `cuda-python` 13.x)
|
||||
- Python 3.10 or newer
|
||||
- `cuda-python` (>=13.0.0)
|
||||
- `cuda-core` (>=0.6.0)
|
||||
|
||||
## Installation
|
||||
|
||||
Install the required packages from `requirements.txt`:
|
||||
|
||||
```bash
|
||||
cd /path/to/cuda-samples/python/1_GettingStarted/systemInfo
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
The `requirements.txt` installs:
|
||||
|
||||
- `cuda-python` (>=13.0.0)
|
||||
- `cuda-core` (>=0.6.0)
|
||||
|
||||
## How to Run
|
||||
|
||||
### Basic usage
|
||||
|
||||
```bash
|
||||
cd cuda-samples/python/1_GettingStarted/systemInfo
|
||||
python systemInfo.py
|
||||
```
|
||||
|
||||
### Skip topology queries
|
||||
|
||||
Useful on machines with only one GPU or to shorten the output:
|
||||
|
||||
```bash
|
||||
python systemInfo.py --no-topology
|
||||
```
|
||||
|
||||
## Expected Output
|
||||
|
||||
Output varies with your hardware. On a machine with two GPUs you should see
|
||||
something like:
|
||||
|
||||
```
|
||||
======================================================================
|
||||
Driver / NVML
|
||||
======================================================================
|
||||
CUDA driver version: 13.2
|
||||
CUDA driver version (full): (13, 2, 0)
|
||||
NVML version: (13, 595, 58, 3)
|
||||
Driver branch: r595_88
|
||||
Current process: /usr/bin/python
|
||||
|
||||
======================================================================
|
||||
Devices detected: 2
|
||||
======================================================================
|
||||
|
||||
-- Device 0 --
|
||||
Name: <Your GPU Name>
|
||||
UUID: ...
|
||||
Compute capability: 8.9
|
||||
Architecture: ADA
|
||||
Brand: BRAND_GEFORCE
|
||||
Memory: total=23.99 GiB, used=960.00 KiB, free=23.52 GiB
|
||||
PCI: domain=0000 bus=41 device=00 id=00000000:41:00.0
|
||||
Temperature (GPU sensor): 47 C
|
||||
Performance state: <Pstates.PSTATE_8: 8>
|
||||
|
||||
...
|
||||
|
||||
======================================================================
|
||||
GPU topology and peer-to-peer
|
||||
======================================================================
|
||||
Device 0 <-> Device 1: topology=TOPOLOGY_HOSTBRIDGE, p2p_read=..., p2p_write=...
|
||||
|
||||
Done
|
||||
```
|
||||
|
||||
**Note:** Device names, compute capability, temperatures, and topology
|
||||
details will vary based on your GPUs and system.
|
||||
|
||||
## Files
|
||||
|
||||
- `systemInfo.py` - Python implementation using `cuda.core.system`
|
||||
- `README.md` - This file
|
||||
- `requirements.txt` - Sample dependencies
|
||||
|
||||
## See Also
|
||||
|
||||
- [CUDA Python Documentation](https://nvidia.github.io/cuda-python/)
|
||||
- [`cuda.core.system` API reference](https://nvidia.github.io/cuda-python/cuda-core/latest/api.html)
|
||||
- [NVML reference](https://docs.nvidia.com/deploy/nvml-api/)
|
||||
4
python/1_GettingStarted/systemInfo/requirements.txt
Normal file
4
python/1_GettingStarted/systemInfo/requirements.txt
Normal file
@@ -0,0 +1,4 @@
|
||||
# System Information Sample Requirements
|
||||
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
215
python/1_GettingStarted/systemInfo/systemInfo.py
Normal file
215
python/1_GettingStarted/systemInfo/systemInfo.py
Normal file
@@ -0,0 +1,215 @@
|
||||
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
|
||||
#
|
||||
# Redistribution and use in source and binary forms, with or without
|
||||
# modification, are permitted provided that the following conditions
|
||||
# are met:
|
||||
# * Redistributions of source code must retain the above copyright
|
||||
# notice, this list of conditions and the following disclaimer.
|
||||
# * Redistributions in binary form must reproduce the above copyright
|
||||
# notice, this list of conditions and the following disclaimer in the
|
||||
# documentation and/or other materials provided with the distribution.
|
||||
# * Neither the name of NVIDIA CORPORATION nor the names of its
|
||||
# contributors may be used to endorse or promote products derived
|
||||
# from this software without specific prior written permission.
|
||||
#
|
||||
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
|
||||
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
||||
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
||||
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
|
||||
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
|
||||
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
|
||||
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
|
||||
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
|
||||
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
"""
|
||||
System Information via cuda.core.system (NVML)
|
||||
|
||||
Demonstrates the ``cuda.core.system`` module, which wraps NVIDIA Management
|
||||
Library (NVML) functionality.
|
||||
|
||||
This sample prints:
|
||||
* Driver and NVML versions
|
||||
* Current process name
|
||||
* Per-device: name, UUID, compute capability / arch, PCI info, memory usage,
|
||||
temperature, performance state
|
||||
* GPU-to-GPU topology and peer-to-peer status (when more than one GPU)
|
||||
"""
|
||||
|
||||
import os
|
||||
import sys
|
||||
|
||||
try:
|
||||
from cuda.core import system
|
||||
from cuda.core.system import (
|
||||
CUDA_BINDINGS_NVML_IS_COMPATIBLE,
|
||||
GpuP2PCapsIndex,
|
||||
TemperatureSensors,
|
||||
)
|
||||
except ImportError as e:
|
||||
print(f"Error: Required package not found: {e}")
|
||||
print("Please install from requirements.txt:")
|
||||
print(" pip install -r requirements.txt")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def print_header(title: str) -> None:
|
||||
print()
|
||||
print("=" * 70)
|
||||
print(title)
|
||||
print("=" * 70)
|
||||
|
||||
|
||||
def format_bytes(nbytes: int) -> str:
|
||||
"""Format a byte count as a human-readable string."""
|
||||
units = ["B", "KiB", "MiB", "GiB", "TiB"]
|
||||
size = float(nbytes)
|
||||
for unit in units:
|
||||
if size < 1024.0:
|
||||
return f"{size:.2f} {unit}"
|
||||
size /= 1024.0
|
||||
return f"{size:.2f} PiB"
|
||||
|
||||
|
||||
def print_driver_info() -> None:
|
||||
print_header("Driver / NVML")
|
||||
major, minor = system.get_driver_version()
|
||||
print(f"CUDA driver version: {major}.{minor}")
|
||||
print(f"CUDA driver version (full): {system.get_driver_version_full()}")
|
||||
if CUDA_BINDINGS_NVML_IS_COMPATIBLE:
|
||||
print(f"NVML version: {system.get_nvml_version()}")
|
||||
try:
|
||||
print(f"Driver branch: {system.get_driver_branch()}")
|
||||
except Exception as e: # noqa: BLE001 - driver branch is informational
|
||||
print(f"Driver branch: unavailable ({e})")
|
||||
else:
|
||||
print(
|
||||
"NVML bindings are not compatible with this driver; "
|
||||
"device info will be limited."
|
||||
)
|
||||
print(f"Current process: {system.get_process_name(os.getpid())}")
|
||||
|
||||
|
||||
def print_device_info(device: "system.Device") -> None:
|
||||
print(f"\n-- Device {device.index} --")
|
||||
print(f"Name: {device.name}")
|
||||
print(f"UUID: {device.uuid}")
|
||||
try:
|
||||
cc_major, cc_minor = device.cuda_compute_capability
|
||||
print(f"Compute capability: {cc_major}.{cc_minor}")
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"Compute capability: unavailable ({e})")
|
||||
try:
|
||||
print(f"Architecture: {device.arch.name}")
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"Architecture: unavailable ({e})")
|
||||
try:
|
||||
print(f"Brand: {device.brand.name}")
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"Brand: unavailable ({e})")
|
||||
|
||||
# Memory
|
||||
try:
|
||||
mem = device.memory_info
|
||||
print(
|
||||
f"Memory: total={format_bytes(mem.total)}, "
|
||||
f"used={format_bytes(mem.used)}, "
|
||||
f"free={format_bytes(mem.free)}"
|
||||
)
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"Memory: unavailable ({e})")
|
||||
|
||||
# PCI
|
||||
try:
|
||||
pci = device.pci_info
|
||||
print(
|
||||
f"PCI: domain={pci.domain:04x} bus={pci.bus:02x} "
|
||||
f"device={pci.device:02x} id={pci.bus_id}"
|
||||
)
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"PCI: unavailable ({e})")
|
||||
|
||||
# Temperature (GPU sensor)
|
||||
try:
|
||||
temp_c = device.temperature.sensor(TemperatureSensors.TEMPERATURE_GPU)
|
||||
print(f"Temperature (GPU sensor): {temp_c} C")
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"Temperature: unavailable ({e})")
|
||||
|
||||
# Performance state
|
||||
try:
|
||||
pstate = device.performance_state
|
||||
print(f"Performance state: {pstate}")
|
||||
except Exception as e: # noqa: BLE001
|
||||
print(f"Performance state: unavailable ({e})")
|
||||
|
||||
|
||||
def print_topology(devices: list) -> None:
|
||||
if len(devices) < 2:
|
||||
return
|
||||
print_header("GPU topology and peer-to-peer")
|
||||
for i, d0 in enumerate(devices):
|
||||
for d1 in devices[i + 1 :]:
|
||||
try:
|
||||
level = system.get_topology_common_ancestor(d0, d1)
|
||||
level_name = level.name
|
||||
except Exception as e: # noqa: BLE001
|
||||
level_name = f"unavailable ({e})"
|
||||
try:
|
||||
read = system.get_p2p_status(
|
||||
d0, d1, GpuP2PCapsIndex.P2P_CAPS_INDEX_READ
|
||||
)
|
||||
write = system.get_p2p_status(
|
||||
d0, d1, GpuP2PCapsIndex.P2P_CAPS_INDEX_WRITE
|
||||
)
|
||||
read_name = read.name
|
||||
write_name = write.name
|
||||
except Exception as e: # noqa: BLE001
|
||||
read_name = write_name = f"unavailable ({e})"
|
||||
print(
|
||||
f"Device {d0.index} <-> Device {d1.index}: "
|
||||
f"topology={level_name}, p2p_read={read_name}, p2p_write={write_name}"
|
||||
)
|
||||
|
||||
|
||||
def main() -> int:
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Print CUDA system / NVML information via cuda.core.system"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--no-topology",
|
||||
action="store_true",
|
||||
help="Skip cross-device topology/P2P queries",
|
||||
)
|
||||
args = parser.parse_args()
|
||||
|
||||
print_driver_info()
|
||||
|
||||
num_devices = system.get_num_devices()
|
||||
print_header(f"Devices detected: {num_devices}")
|
||||
if num_devices == 0:
|
||||
print("No CUDA-capable devices found.")
|
||||
return 0
|
||||
if not CUDA_BINDINGS_NVML_IS_COMPATIBLE:
|
||||
print(
|
||||
"NVML is not compatible with the installed driver; skipping device detail."
|
||||
)
|
||||
return 0
|
||||
|
||||
devices = [system.Device(index=i) for i in range(num_devices)]
|
||||
for device in devices:
|
||||
print_device_info(device)
|
||||
|
||||
if not args.no_topology:
|
||||
print_topology(devices)
|
||||
|
||||
print("\nDone")
|
||||
return 0
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
130
python/1_GettingStarted/vectorAdd/README.md
Normal file
130
python/1_GettingStarted/vectorAdd/README.md
Normal file
@@ -0,0 +1,130 @@
|
||||
# Sample: Vector Addition (Python)
|
||||
|
||||
## Description
|
||||
|
||||
Run your first GPU kernel: add two vectors element-wise on the GPU using the [`cuda.core`](https://nvidia.github.io/cuda-python/cuda-core/latest/) API with runtime compilation.
|
||||
|
||||
## What You'll Learn
|
||||
|
||||
- Writing CUDA kernels in C++ with template support
|
||||
- Runtime compilation of CUDA kernels from Python
|
||||
- Using `cuda.core` for device management, programs, and launches
|
||||
- Configuring and launching kernels with grid and block dimensions
|
||||
- Using CuPy for GPU memory management
|
||||
- Verifying GPU results against CPU computation
|
||||
|
||||
## Key Libraries
|
||||
|
||||
- [`cuda.core`](https://nvidia.github.io/cuda-python/cuda-core/latest/) — Pythonic access to CUDA runtime and compilation
|
||||
- `cupy` — GPU array library for Python
|
||||
|
||||
## Key APIs
|
||||
|
||||
### From `cuda.core`
|
||||
|
||||
- `Device` — Initialize and manage CUDA device
|
||||
- `Program` — Create program from kernel source code
|
||||
- `ProgramOptions` — Set compilation options (C++ standard, architecture)
|
||||
- `LaunchConfig` — Configure kernel launch parameters
|
||||
- `launch` — Execute kernel on specified stream
|
||||
|
||||
Import stable symbols from the top-level package (not `cuda.core.experimental`). See the [cuda.core documentation](https://nvidia.github.io/cuda-python/cuda-core/latest/).
|
||||
|
||||
### From CuPy
|
||||
|
||||
- `cp.random.rand()` — Generate random arrays on GPU
|
||||
- `cp.empty()` — Allocate uninitialized GPU arrays
|
||||
- `cp.allclose()` — Verify results with tolerance
|
||||
|
||||
### From `cuda_samples_utils`
|
||||
|
||||
- `verify_array_result()` — Verify computation results
|
||||
|
||||
## Kernel Techniques
|
||||
|
||||
- **1D Grid-Stride Loop** — Handle arbitrary array sizes with fixed grid
|
||||
- **Template Programming** — Generic kernel for different data types
|
||||
- **Bounds Checking** — Prevent out-of-bounds memory access
|
||||
|
||||
## Requirements
|
||||
|
||||
### Hardware
|
||||
|
||||
- NVIDIA GPU with Compute Capability 7.0 or higher
|
||||
- Minimum GPU memory: 512 MB
|
||||
|
||||
### Software
|
||||
|
||||
- CUDA Toolkit 13.0 or newer (matches `cuda-python` 13.x)
|
||||
- Python 3.10 or newer
|
||||
- `cuda-python` (>=13.0.0)
|
||||
- `cuda-core` (>=0.6.0)
|
||||
- `cupy-cuda13x` (>=13.0.0)
|
||||
|
||||
## Installation
|
||||
|
||||
Install the required packages from requirements.txt:
|
||||
|
||||
```bash
|
||||
cd /path/to/cuda-samples/python/1_GettingStarted/vectorAdd
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
The requirements.txt installs:
|
||||
|
||||
- `cuda-python` (>=13.0.0)
|
||||
- `cuda-core` (>=0.6.0)
|
||||
- `cupy-cuda13x` (>=13.0.0)
|
||||
|
||||
## How to Run
|
||||
|
||||
### Basic usage
|
||||
|
||||
```bash
|
||||
cd samples/python/1_GettingStarted/vectorAdd
|
||||
python vectorAdd.py
|
||||
```
|
||||
|
||||
### With custom parameters
|
||||
|
||||
```bash
|
||||
# Custom vector size
|
||||
python vectorAdd.py --elements 1000000
|
||||
|
||||
# Use specific GPU
|
||||
python vectorAdd.py --device 1
|
||||
|
||||
# Skip verification for benchmarking
|
||||
python vectorAdd.py --no-verify
|
||||
```
|
||||
|
||||
## Expected Output
|
||||
|
||||
```
|
||||
[Vector addition using CUDA Core API]
|
||||
Device: <Your GPU Name>
|
||||
Compute Capability: sm_<XX>
|
||||
Compiling kernel 'vectorAdd<float>'...
|
||||
Kernel compiled successfully
|
||||
[Vector addition of 50000 elements]
|
||||
CUDA kernel launch with 196 blocks of 256 threads
|
||||
Verifying result...
|
||||
Test PASSED
|
||||
|
||||
Done
|
||||
```
|
||||
|
||||
**Note:** Device name and compute capability will vary based on your GPU.
|
||||
|
||||
## Files
|
||||
|
||||
- `vectorAdd.py` — Python implementation using cuda.core API
|
||||
- `README.md` — This file
|
||||
- `requirements.txt` — Sample dependencies
|
||||
- `../../Utilities/cuda_samples_utils.py` — Common utilities (imported by this sample)
|
||||
|
||||
## See Also
|
||||
|
||||
- [CUDA Python Documentation](https://nvidia.github.io/cuda-python/)
|
||||
- [cuda.core API](https://nvidia.github.io/cuda-python/cuda-core/latest/)
|
||||
- [CuPy Documentation](https://docs.cupy.dev/)
|
||||
5
python/1_GettingStarted/vectorAdd/requirements.txt
Normal file
5
python/1_GettingStarted/vectorAdd/requirements.txt
Normal file
@@ -0,0 +1,5 @@
|
||||
# Vector Addition Sample Requirements
|
||||
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
cupy-cuda13x>=13.0.0
|
||||
196
python/1_GettingStarted/vectorAdd/vectorAdd.py
Executable file
196
python/1_GettingStarted/vectorAdd/vectorAdd.py
Executable file
@@ -0,0 +1,196 @@
|
||||
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
|
||||
#
|
||||
# Redistribution and use in source and binary forms, with or without
|
||||
# modification, are permitted provided that the following conditions
|
||||
# are met:
|
||||
# * Redistributions of source code must retain the above copyright
|
||||
# notice, this list of conditions and the following disclaimer.
|
||||
# * Redistributions in binary form must reproduce the above copyright
|
||||
# notice, this list of conditions and the following disclaimer in the
|
||||
# documentation and/or other materials provided with the distribution.
|
||||
# * Neither the name of NVIDIA CORPORATION nor the names of its
|
||||
# contributors may be used to endorse or promote products derived
|
||||
# from this software without specific prior written permission.
|
||||
#
|
||||
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
|
||||
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
||||
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
||||
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
|
||||
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
|
||||
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
|
||||
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
|
||||
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
|
||||
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
"""
|
||||
Vector Addition using CUDA Core API
|
||||
|
||||
This sample demonstrates element-wise vector addition: C = A + B
|
||||
using cuda.core for runtime compilation and kernel launch.
|
||||
"""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
# Add parent directory to path to import utilities
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent.parent / "Utilities"))
|
||||
from cuda_samples_utils import verify_array_result # noqa: E402
|
||||
|
||||
try:
|
||||
import cupy as cp
|
||||
from cuda.core import Device, LaunchConfig, Program, ProgramOptions, launch
|
||||
except ImportError as e:
|
||||
print(f"Error: Required package not found: {e}")
|
||||
print("Please install from requirements.txt:")
|
||||
print(" pip install -r requirements.txt")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
# CUDA kernel source code
|
||||
VECTOR_ADD_KERNEL = """
|
||||
/**
|
||||
* CUDA Kernel for vector addition
|
||||
* Computes the vector addition of A and B into C.
|
||||
*/
|
||||
template<typename T>
|
||||
__global__ void vectorAdd(const T *A, const T *B, T *C, int numElements)
|
||||
{
|
||||
int i = blockDim.x * blockIdx.x + threadIdx.x;
|
||||
|
||||
if (i < numElements)
|
||||
{
|
||||
C[i] = A[i] + B[i];
|
||||
}
|
||||
}
|
||||
"""
|
||||
|
||||
|
||||
def vector_add_cuda_core(num_elements=50000, device_id=0, verify=True):
|
||||
"""
|
||||
Perform vector addition using cuda.core API.
|
||||
|
||||
Parameters
|
||||
----------
|
||||
num_elements : int
|
||||
Number of elements in each vector
|
||||
device_id : int
|
||||
CUDA device ID to use
|
||||
verify : bool
|
||||
Whether to verify the result
|
||||
|
||||
Returns
|
||||
-------
|
||||
bool
|
||||
True if successful, False otherwise
|
||||
"""
|
||||
try:
|
||||
# Initialize device
|
||||
print("[Vector addition using CUDA Core API]")
|
||||
device = Device(device_id)
|
||||
device.set_current()
|
||||
|
||||
print(f"Device: {device.name}")
|
||||
print(f"Compute Capability: sm_{device.arch}")
|
||||
|
||||
stream = device.create_stream()
|
||||
|
||||
# Compile kernel
|
||||
print("Compiling kernel 'vectorAdd<float>'...")
|
||||
program_options = ProgramOptions(std="c++17", arch=f"sm_{device.arch}")
|
||||
program = Program(VECTOR_ADD_KERNEL, code_type="c++", options=program_options)
|
||||
module = program.compile("cubin", name_expressions=("vectorAdd<float>",))
|
||||
kernel = module.get_kernel("vectorAdd<float>")
|
||||
print("Kernel compiled successfully")
|
||||
|
||||
# Allocate and initialize vectors
|
||||
print(f"[Vector addition of {num_elements} elements]")
|
||||
dtype = cp.float32
|
||||
|
||||
a = cp.random.rand(num_elements).astype(dtype)
|
||||
b = cp.random.rand(num_elements).astype(dtype)
|
||||
c = cp.empty(num_elements, dtype=dtype)
|
||||
|
||||
# Synchronize before kernel launch
|
||||
device.sync()
|
||||
|
||||
# Configure and launch kernel
|
||||
threads_per_block = 256
|
||||
blocks_per_grid = (num_elements + threads_per_block - 1) // threads_per_block
|
||||
|
||||
print(
|
||||
f"CUDA kernel launch with {blocks_per_grid} blocks "
|
||||
f"of {threads_per_block} threads"
|
||||
)
|
||||
|
||||
config = LaunchConfig(grid=blocks_per_grid, block=threads_per_block)
|
||||
|
||||
# Launch kernel
|
||||
launch(
|
||||
stream,
|
||||
config,
|
||||
kernel,
|
||||
a.data.ptr,
|
||||
b.data.ptr,
|
||||
c.data.ptr,
|
||||
cp.int32(num_elements),
|
||||
)
|
||||
stream.sync()
|
||||
|
||||
# Verify result
|
||||
if verify:
|
||||
print("Verifying result...")
|
||||
expected = a + b
|
||||
if not verify_array_result(c, expected):
|
||||
return False
|
||||
|
||||
return True
|
||||
|
||||
except Exception as e:
|
||||
print(f"Error: {e}")
|
||||
import traceback
|
||||
|
||||
traceback.print_exc()
|
||||
return False
|
||||
|
||||
|
||||
def main():
|
||||
"""
|
||||
Main entry point for the vector addition sample.
|
||||
"""
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(description="Vector Addition using CUDA Core API")
|
||||
parser.add_argument(
|
||||
"--elements",
|
||||
type=int,
|
||||
default=50000,
|
||||
help="Number of elements in vectors (default: 50000)",
|
||||
)
|
||||
parser.add_argument(
|
||||
"--device", type=int, default=0, help="CUDA device ID (default: 0)"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--no-verify", action="store_true", help="Skip result verification"
|
||||
)
|
||||
|
||||
args = parser.parse_args()
|
||||
|
||||
if args.elements <= 0:
|
||||
print("Error: Number of elements must be positive")
|
||||
return 1
|
||||
|
||||
success = vector_add_cuda_core(
|
||||
num_elements=args.elements, device_id=args.device, verify=not args.no_verify
|
||||
)
|
||||
|
||||
if success:
|
||||
print("\nDone")
|
||||
return 0
|
||||
else:
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
Reference in New Issue
Block a user