NASP/cuda-samples
84
0
Fork 0
mirror of https://github.com/NVIDIA/cuda-samples.git synced 2026-10-13 00:08:26 +08:00
Code Issues Packages Projects Releases Wiki Activity

CUDA 13.2 samples update (#432)

Browse Source 
- Added Python samples for CUDA Python 1.0 release
- Renamed top-level `Samples` directory to `cpp` to accommodate Python samples.
This commit is contained in:
Dheemanth
2026-05-13 15:13:18 -07:00
committed by GitHub
parent a4526d5229
commit aeab82ff30
2072 changed files with 15077 additions and 1970 deletions
Download Patch File Download Diff File Expand all files Collapse all files

263
python/1_GettingStarted/simplePrint/README.md Normal file
View File

@@ -0,0 +1,263 @@
# simplePrint - Printing from CUDA Kernels
## Description
This sample demonstrates how to use `printf()` inside CUDA kernels using **two different approaches**:
1. **CUDA C++ kernels** compiled with `cuda.core.Program` - Full C++ features and control
2. **Numba CUDA kernels** - Pythonic kernel authoring using `numba.cuda.grid()` for modern indexing
The sample shows basic device management, kernel compilation with inline CUDA C++ code, and multi-dimensional kernel launches (2D grid × 3D blocks) using modern CUDA Python. The Numba example demonstrates the recommended `numba.cuda.grid()` indexing style while also showing how it relates to classic CUDA C++ block/thread IDs. Both approaches use `cuda.core` APIs for stream management and synchronization, demonstrating interoperability.
This is the Python equivalent of the C++ `simplePrintf` sample, enhanced with Numba CUDA examples.
## Key Concepts
CUDA Python (cuda.core), Numba CUDA, Kernel Compilation, Printf in Kernels, Multi-dimensional Launch, Pythonic GPU Programming, Modern Thread Indexing (grid()), Stream-based Execution, cuda.core/Numba Interoperability
## CUDA APIs involved
### [cuda.core (cuda-python)](https://nvidia.github.io/cuda-python/)
- `Device()` - Device management
- `Device.create_stream()` - Create CUDA streams
- `Stream.sync()` - Synchronize stream execution
- `Program()` - Compile CUDA C++ kernels
- `LaunchConfig()` - Configure kernel launch
- `launch()` - Execute kernels on streams
### [Numba CUDA](https://nvidia.github.io/numba-cuda/)
- `@cuda.jit` - JIT compile Python functions to CUDA kernels
- `cuda.grid()` - Get global thread position (recommended modern approach)
- `cuda.blockIdx`, `cuda.threadIdx` - Thread/block indices (classic style)
- `cuda.gridDim`, `cuda.blockDim` - Grid/block dimensions
- **Note:** Uses `cuda.core` APIs for stream management (interoperability)
### CUDA Kernel Functions
- `printf()` - Print from device code (C++)
- `print()` - Print from device code (Numba, limited formatting)
- `blockIdx`, `threadIdx` - Thread/block indices
- `gridDim`, `blockDim` - Grid/block dimensions
### What You Learn
- Device initialization with `cuda.core.Device`
- Compiling CUDA C++ kernels with `Program` and `ProgramOptions`
- Writing Pythonic CUDA kernels with Numba's `@cuda.jit` decorator
- Using `numba.cuda.grid()` for modern thread indexing (recommended approach)
- Understanding the relationship between global coordinates and classic block/thread IDs
- **Interoperability**: Using `cuda.core` streams with Numba CUDA kernels
- Comparing CUDA C++ vs Pythonic kernel authoring approaches
- Multi-dimensional kernel launches (2D grid, 3D blocks)
- Using streams for kernel execution and synchronization
- Using `printf()` and `print()` in GPU kernels for debugging
- Understanding print limitations in Numba CUDA (no f-strings)
- Proper error handling and resource management
## Requirements
### Hardware:
- NVIDIA GPU with Compute Capability 7.0 or higher
- Minimum GPU memory: 512 MB
### Software:
- CUDA Toolkit 13.0 or newer
- Python 3.10 or newer
- `cuda-python` package (13.0+)
- `cuda-core` package (>=0.6.0)
- `numba-cuda` package (0.24.0+, for Pythonic kernel authoring)
Download and install:
- [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads)
- [cuda-python package](https://nvidia.github.io/cuda-python/): `pip install cuda-python`
- [numba-cuda](https://nvidia.github.io/numba-cuda/): `pip install numba-cuda`
## Build and Run
```bash
# Install dependencies
pip install -r requirements.txt
# Run the sample
python simplePrint.py
```
## Expected Output
```
Simple Print - Printing from CUDA Kernels
Demonstrating both CUDA C++ and Numba CUDA approaches
Device: <Your GPU Name>
Compute Capability: sm_<XX>
======================================================================
METHOD 1: CUDA C++ Kernel (via cuda.core.Program)
======================================================================
Advantage: Full C++ features, better for complex kernels
Compiling CUDA C++ kernel...
Kernel compiled successfully.
Kernel configuration:
Grid: (2, 2)
Block: (2, 2, 2)
Total threads: 32
Launching kernel with value=10. Output:
[0, 0]: Value is: 10
[0, 1]: Value is: 10
[0, 2]: Value is: 10
[0, 3]: Value is: 10
[0, 4]: Value is: 10
[0, 5]: Value is: 10
[0, 6]: Value is: 10
[0, 7]: Value is: 10
[1, 0]: Value is: 10
...
[3, 7]: Value is: 10
CUDA C++ kernel execution complete.
======================================================================
METHOD 2: Numba CUDA Kernel (Pythonic / modern indexing)
======================================================================
Advantage: Uses numba.cuda.grid(3) for global indexing,
while still showing classic CUDA C++ IDs for reference.
Uses cuda.core for stream management (interoperability).
Kernel configuration:
Grid: (2, 2)
Block: (2, 2, 2)
Total threads: 32
Launching Numba CUDA kernel (grid(3) + classic IDs) with value=10:
Uses numba.cuda.grid(3) to get global (x, y, z),
and prints the corresponding blockId/threadId like the C++ sample.
Stream managed by cuda.core for consistency with C++ example.
global[ 0 , 0 , 0 ] -> [ 0 , 0 ]: Value is: 10
global[ 1 , 0 , 0 ] -> [ 0 , 1 ]: Value is: 10
global[ 0 , 1 , 0 ] -> [ 0 , 2 ]: Value is: 10
...
global[ 3 , 3 , 1 ] -> [ 3 , 7 ]: Value is: 10
Numba CUDA kernel execution complete.
======================================================================
Done! Both kernel approaches demonstrated successfully.
======================================================================
```
## Understanding the Output
- **Grid**: 2×2 = 4 blocks (labeled 0-3)
- **Block**: 2×2×2 = 8 threads per block (labeled 0-7)
- **Total**: 32 threads, each printing its position and value
### CUDA C++ Kernel:
Each thread calculates:
- Block ID (linear): `blockIdx.y * gridDim.x + blockIdx.x`
- Thread ID (linear): `threadIdx.z * blockDim.x * blockDim.y + threadIdx.y * blockDim.x + threadIdx.x`
### Numba CUDA Kernel:
Each thread shows:
- **Global position** using `numba.cuda.grid(3)` → `(x, y, z)` coordinates across entire grid
- **Classic IDs** (block ID, thread ID) calculated the same way as C++ for comparison
- This demonstrates how modern indexing relates to traditional CUDA C++ style
### Comparing the Two Approaches
**CUDA C++ Kernel (Method 1):**
- Uses C++ syntax and `printf()` with full formatting control
- Requires compilation via `cuda.core.Program`
- Best for complex kernels needing C++ features (templates, libraries, etc.)
- Uses classic block/thread ID indexing
- Output: `[0, 0]: Value is: 10` (clean formatting)
**Numba CUDA Kernel (Method 2):**
- Uses Python syntax with `@cuda.jit` decorator
- JIT compiled automatically when called
- Best for prototyping and simpler kernels
- **Modern indexing**: Uses `numba.cuda.grid(3)` to get global thread coordinates (recommended)
- Also shows classic block/thread IDs to help relate the two indexing models
- **Interoperability**: Uses `cuda.core` streams via `stream` for consistency
- Demonstrates that numba-cuda kernels can work seamlessly with cuda.core infrastructure
- Limited print formatting (no f-strings, basic `print()` only; adds spaces between arguments)
- Output: `global[ 0 , 0 , 0 ] -> [ 0 , 0 ]: Value is: 10` (shows both indexing styles; note extra spaces due to `print()` behavior)
## Experiments
Try modifying:
### For Both Approaches:
- **Grid size**: Change `grid=(4, 4)` for 16 blocks
- **Block size**: Change `block=(4, 4, 4)` for 64 threads per block
- **Conditional printing**: Print only from specific threads (e.g., `if threadId == 0:`)
### CUDA C++ Specific:
- **Format strings**: Experiment with different `printf()` formats
- **Kernel code**: Add complex C++ computations before printing
- **External libraries**: Include CUDA math libraries or device functions (e.g., `<cuda/std/cmath>`, `<cub/cub.cuh>`)
### Numba CUDA Specific:
- **Grid indexing**: Try `numba.cuda.grid(1)` or `numba.cuda.grid(2)` for different dimensions
- **Conditional printing**: Print only from threads where `x == 0` or `y == z`
- **Python operations**: Use NumPy-like operations in the kernel
- **Device math libraries**: Use [nvmath-python device APIs](https://docs.nvidia.com/cuda/nvmath-python/latest/device-apis/index.html) for optimized math operations (similar to CUDA math libraries in C++)
- **Shared memory**: Add `numba.cuda.shared.array()` for fast inter-thread communication
- **Atomic operations**: Try `numba.cuda.atomic.add()` for thread-safe updates
- **Print variations**: Experiment with what numba-cuda's `print()` can and cannot handle
- **Streams**: Create multiple `cuda.core` streams and launch numba-cuda kernels on them concurrently
- **Interoperability**: Mix numba-cuda kernels and CUDA C++ kernels on the same stream
## Notes
### General:
- Printing from GPU is relatively slow - use sparingly in production code
- Printf output is buffered and limited (~1MB buffer on most GPUs)
### CUDA C++ Kernels:
- Always call `stream.sync()` after kernel launch to flush printf output
- Full `printf()` format string support (%, flags, width, precision)
### Numba CUDA Kernels:
- **Recommended**: Use `numba.cuda.grid(ndim)` for thread indexing (modern, Pythonic)
- `grid(1)` for 1D indexing, `grid(2)` for 2D, `grid(3)` for 3D
- Returns global thread position across the entire grid
- **Interoperability**: Use `cuda.core` streams with Numba kernels via `stream`
- Create streams: `stream = device.create_stream()`
- Launch kernels: `kernel[grid, block, stream](args)`
- Synchronize: `stream.sync()`
- Numba's `print()` has limited capabilities compared to Python's `print()`
- F-strings are NOT supported in Numba CUDA kernels
- Use comma-separated arguments: `print("Value:", x)` instead of f-strings
- **Note**: `print()` automatically adds spaces between comma-separated arguments (e.g., `print("[", x, "]")` outputs `[ 0 ]` not `[0]`)
- Always synchronize the stream to flush output
## Files
- `simplePrint.py` - Python implementation using cuda.core API
- `README.md` - This file
- `requirements.txt` - Sample dependencies
## See Also
### CUDA Python (cuda.core):
- [cuda.core Documentation](https://nvidia.github.io/cuda-python/)
- [CUDA Python Examples](https://github.com/NVIDIA/cuda-python/tree/main/cuda_core/examples)
### Numba CUDA:
- [Numba CUDA Documentation](https://nvidia.github.io/numba-cuda/)
- [numba.cuda.grid() Reference](https://nvidia.github.io/numba-cuda/reference/kernel.html#numba.cuda.grid)
- [nvmath-python Device APIs](https://docs.nvidia.com/cuda/nvmath-python/latest/device-apis/index.html) - Optimized math operations for Numba CUDA kernels
### CUDA References:
- [CUDA C Programming Guide - Printf](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#formatted-output)
- [C++ simplePrintf Sample](https://github.com/NVIDIA/cuda-samples/tree/master/Samples/0_Introduction/simplePrintf)

7
python/1_GettingStarted/simplePrint/requirements.txt Normal file
View File

@@ -0,0 +1,7 @@
# Simple Printf Sample - Requirements
cuda-python>=13.0.0
cuda-core>=0.6.0
# Numba JIT uses nvJitLink from pip; keep in step with cuda-bindings (e.g. 13.2.x).
nvidia-nvjitlink>=13.2.0
numba-cuda>=0.24.0

287
python/1_GettingStarted/simplePrint/simplePrint.py Normal file
View File

@@ -0,0 +1,287 @@
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions
# are met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of NVIDIA CORPORATION nor the names of its
# contributors may be used to endorse or promote products derived
# from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
"""
Simple Print - Printing from CUDA Kernels
This sample demonstrates how to print output from CUDA kernels using printf().
It shows:
1. Device management with cuda.core.Device
2. Compiling CUDA C++ code that uses printf()
3. Launching kernels with 2D grids and 3D blocks
4. Seeing kernel output printed to stdout
5. Using Numba CUDA for Pythonic kernel authoring
This sample demonstrates both approaches:
- CUDA C++ kernels compiled via cuda.core.Program (more control, C++ features)
- Numba CUDA kernels (more Pythonic, easier to write)
This is the Python equivalent of the C++ simplePrintf sample.
"""
import sys
import traceback
try:
from cuda.core import Device, LaunchConfig, Program, ProgramOptions, launch
except ImportError as e:
print(f"Error: Required package not found: {e}")
print("Please install from requirements.txt:")
print(" pip install -r requirements.txt")
sys.exit(1)
try:
from numba import cuda as numba_cuda
NUMBA_AVAILABLE = True
except ImportError:
NUMBA_AVAILABLE = False
print("Warning: numba not found. Numba CUDA example will be skipped.")
print("To install: pip install numba")
# CUDA C++ kernel with printf
# This kernel prints the block index, thread index, and a value from each thread
PRINTF_KERNEL = """
extern "C"
__global__ void printKernel(int val) {
// Calculate linear block index from 2D grid
int blockId = blockIdx.y * gridDim.x + blockIdx.x;
// Calculate linear thread index from 3D block
int threadId = threadIdx.z * blockDim.x * blockDim.y +
threadIdx.y * blockDim.x +
threadIdx.x;
// Print from each thread
printf("[%d, %d]:\\t\\tValue is: %d\\n", blockId, threadId, val);
}
"""
# Numba CUDA kernel - Pythonic equivalent using numba.cuda.grid()
# This demonstrates the same functionality using Numba's Python-based kernel syntax
if NUMBA_AVAILABLE:
@numba_cuda.jit
def numba_print_kernel(val):
"""
Numba CUDA kernel showing the *recommended* grid() indexing style,
while also relating it to the classic CUDA C++ blockId/threadId.
- Primary view: global 3D coordinates from numba.cuda.grid(3)
(modern, Pythonic way to index work for a 3D thread layout).
- Secondary view: linear blockId / threadId matching the CUDA C++
printf sample, to help CUDA C++ users connect the two models.
"""
# Modern / recommended view: global 3D thread coordinates
x, y, z = numba_cuda.grid(3)
# Classic CUDA-style indices, same formulas as the C++ sample
block_id = numba_cuda.blockIdx.y * numba_cuda.gridDim.x + numba_cuda.blockIdx.x
thread_id = (
numba_cuda.threadIdx.z * numba_cuda.blockDim.x * numba_cuda.blockDim.y
+ numba_cuda.threadIdx.y * numba_cuda.blockDim.x
+ numba_cuda.threadIdx.x
)
# Print both views side-by-side
# Note: Numba print() adds spaces between comma-separated args
print(
"global[",
x,
",",
y,
",",
z,
"] -> [",
block_id,
",",
thread_id,
"]:\t\tValue is:",
val,
)
def run_cuda_cpp_kernel(device, test_value=10):
"""
Demonstrate printing from CUDA C++ kernel compiled with cuda.core.
This approach gives you full access to CUDA C++ features and allows
for more complex kernel implementations.
"""
print("=" * 70)
print("METHOD 1: CUDA C++ Kernel (via cuda.core.Program)")
print("=" * 70)
print("Advantage: Full C++ features, better for complex kernels")
print()
# Compile the kernel
print("Compiling CUDA C++ kernel...")
program_options = ProgramOptions(std="c++17", arch=f"sm_{device.arch}")
prog = Program(PRINTF_KERNEL, code_type="c++", options=program_options)
mod = prog.compile("cubin", name_expressions=("printKernel",))
kernel = mod.get_kernel("printKernel")
print("Kernel compiled successfully.\n")
# Create stream for kernel execution
stream = device.create_stream()
# Configure kernel launch
# Using 2D grid (2x2) and 3D blocks (2x2x2)
grid_x, grid_y = 2, 2
block_x, block_y, block_z = 2, 2, 2
print("Kernel configuration:")
print(f" Grid: ({grid_x}, {grid_y})")
print(f" Block: ({block_x}, {block_y}, {block_z})")
print(f" Total threads: {grid_x * grid_y * block_x * block_y * block_z}")
print()
# Launch configuration with 2D grid and 3D block
config = LaunchConfig(grid=(grid_x, grid_y), block=(block_x, block_y, block_z))
print(f"Launching kernel with value={test_value}. Output:\n")
try:
# Launch kernel
launch(stream, config, kernel, test_value)
# Synchronize to ensure printf output is flushed
stream.sync()
print("\nCUDA C++ kernel execution complete.")
except Exception as e:
print(f"\nError during kernel execution: {e}")
traceback.print_exc()
return 1
finally:
# Cleanup
stream.close()
return 0
def run_numba_kernel(device, test_value=10):
"""
Demonstrate printing from a Numba CUDA kernel.
This example uses numba.cuda.grid(3) as the primary indexing mechanism
(recommended modern style), and also prints the equivalent blockId /
threadId used in the CUDA C++ printf sample for side-by-side comparison.
Uses cuda.core APIs for stream management, demonstrating interoperability
between Numba CUDA kernels and cuda.core infrastructure.
"""
print("\n")
print("=" * 70)
print("METHOD 2: Numba CUDA Kernel (Pythonic / modern indexing)")
print("=" * 70)
print("Advantage: Uses numba.cuda.grid(3) for global indexing,")
print(" while still showing classic CUDA C++ IDs for reference.")
print(" Uses cuda.core for stream management (interoperability).")
print()
# Same launch configuration as the C++ version
grid_x, grid_y = 2, 2
block_x, block_y, block_z = 2, 2, 2
print("Kernel configuration:")
print(f" Grid: ({grid_x}, {grid_y})")
print(f" Block: ({block_x}, {block_y}, {block_z})")
print(f" Total threads: {grid_x * grid_y * block_x * block_y * block_z}")
print()
# Use cuda.core stream (same as C++ example) instead of numba.cuda.stream()
stream = device.create_stream()
print(f"Launching Numba kernel (grid(3) + classic IDs) with value={test_value}:")
print("Uses numba.cuda.grid(3) to get global (x, y, z),")
print("and prints the corresponding blockId/threadId like the C++ sample.")
print("Stream managed by cuda.core for consistency with C++ example.\n")
try:
# Launch Numba kernel on cuda.core stream
numba_print_kernel[(grid_x, grid_y), (block_x, block_y, block_z), stream](
test_value
)
# Synchronize cuda.core stream (same as C++ example)
stream.sync()
print("\nNumba CUDA kernel execution complete.")
except Exception as e:
print(f"\nError during Numba kernel execution: {e}")
traceback.print_exc()
return 1
finally:
# Cleanup
stream.close()
return 0
def main():
"""Main function demonstrating printing from CUDA kernels using both approaches"""
print("Simple Print - Printing from CUDA Kernels")
print("Demonstrating both CUDA C++ and Numba CUDA approaches")
print()
# Initialize device
device = Device()
device.set_current()
# Get device properties
print(f"Device: {device.name}")
print(f"Compute Capability: sm_{device.arch}")
print()
# Value to pass to both kernels
test_value = 10
# Run CUDA C++ kernel
result = run_cuda_cpp_kernel(device, test_value)
if result != 0:
return result
# Run Numba kernel if available
if NUMBA_AVAILABLE:
result = run_numba_kernel(device, test_value)
if result != 0:
return result
else:
print("\n" + "=" * 70)
print("Numba CUDA example skipped (numba not installed)")
print("To run the Numba example: pip install numba")
print("=" * 70)
print("\n" + "=" * 70)
print("Done! Both kernel approaches demonstrated successfully.")
print("=" * 70)
return 0
if __name__ == "__main__":
sys.exit(main())