mirror of
https://github.com/NVIDIA/cuda-samples.git
synced 2026-10-13 00:08:26 +08:00
Release v13.3 of the CUDA samples with CUDA 13.3 Toolkit (#435)
This is the release of the CUDA 13.3 samples, which include additions for CUDA Tile C++, and updated CCCL and Python samples.
This commit is contained in:
129
python/2_CoreConcepts/binarySearch/README.md
Normal file
129
python/2_CoreConcepts/binarySearch/README.md
Normal file
@@ -0,0 +1,129 @@
|
||||
# binarySearch (Python)
|
||||
|
||||
## Description
|
||||
|
||||
This sample demonstrates the parallel binary-search algorithms
|
||||
exposed by **cuda.compute** (from the `cuda-cccl` package). Given
|
||||
a sorted `d_data` array and a batch of `d_values` to locate, one
|
||||
device-wide call returns the insertion index for every value:
|
||||
|
||||
- `cuda.compute.lower_bound` writes, for each value, the lowest index
|
||||
where it could be inserted into `d_data` without breaking the sort
|
||||
order. Equivalent to `numpy.searchsorted(..., side="left")`.
|
||||
- `cuda.compute.upper_bound` is the analogous upper form, equivalent
|
||||
to `numpy.searchsorted(..., side="right")`.
|
||||
|
||||
The sample runs both algorithms on two curated inputs: one with
|
||||
distinct elements (where `lower_bound` and `upper_bound` agree on
|
||||
any value not in the data) and one with duplicates (where they
|
||||
diverge on present values). Results are verified against
|
||||
`numpy.searchsorted`.
|
||||
|
||||
## What You'll Learn
|
||||
|
||||
- How to call `cuda.compute.lower_bound` / `upper_bound` with CuPy
|
||||
arrays
|
||||
- The semantic difference between `lower_bound` and `upper_bound`,
|
||||
especially for inputs containing duplicates
|
||||
- How the output dtype (`np.uintp`) is used for indices
|
||||
|
||||
## Key Libraries
|
||||
|
||||
- [`cuda.compute`](https://nvidia.github.io/cccl/python.html) (from the `cuda-cccl` package) - device algorithms
|
||||
- [`cuda.core`](https://nvidia.github.io/cuda-python/cuda-core/latest/) - device setup
|
||||
- `cupy` - device buffers
|
||||
- `numpy` - host-side reference via `numpy.searchsorted`
|
||||
|
||||
## Key APIs
|
||||
|
||||
### From `cuda.compute`
|
||||
|
||||
- `cuda.compute.lower_bound(d_data, num_items, d_values, num_values, d_out)`
|
||||
- `cuda.compute.upper_bound(d_data, num_items, d_values, num_values, d_out)`
|
||||
|
||||
### From `cuda_samples_utils`
|
||||
|
||||
- `print_gpu_info()` - print device name and compute capability
|
||||
|
||||
## Requirements
|
||||
|
||||
### Hardware
|
||||
|
||||
- NVIDIA GPU with Compute Capability 7.0 or higher
|
||||
- Minimum GPU memory: 512 MB
|
||||
|
||||
### Software
|
||||
|
||||
- CUDA Toolkit 13.0 or newer
|
||||
- Python 3.10 or newer
|
||||
- `cuda-cccl` (>=1.0.0)
|
||||
- `cuda-core` (>=1.0.0)
|
||||
- `cupy-cuda13x` (>=14.0.0)
|
||||
|
||||
If the CUDA toolkit is not on your `PATH`, set `CUDA_HOME` so that
|
||||
cuda.compute's JIT path can locate its dependencies:
|
||||
|
||||
```bash
|
||||
export CUDA_HOME=/usr/local/cuda
|
||||
```
|
||||
|
||||
## Installation
|
||||
|
||||
Install the required packages from `requirements.txt`:
|
||||
|
||||
```bash
|
||||
cd /path/to/cuda-samples/python/2_CoreConcepts/binarySearch
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
The `requirements.txt` installs:
|
||||
|
||||
- `cuda-cccl` (>=1.0.0) - ships the `cuda.compute` module
|
||||
- `cuda-core` (>=1.0.0)
|
||||
- `cupy-cuda13x` (>=14.0.0)
|
||||
- `numpy` (>=1.24.0)
|
||||
|
||||
## How to Run
|
||||
|
||||
### Basic usage
|
||||
|
||||
```bash
|
||||
cd cuda-samples/python/2_CoreConcepts/binarySearch
|
||||
python binarySearch.py
|
||||
```
|
||||
|
||||
### With custom parameters
|
||||
|
||||
```bash
|
||||
python binarySearch.py --device 1
|
||||
```
|
||||
|
||||
## Expected Output
|
||||
|
||||
```
|
||||
Device: <Your GPU Name>
|
||||
Compute Capability: <X.Y>
|
||||
|
||||
Case 1: distinct data, mixed queries
|
||||
data = [1, 3, 5, 7, 9]
|
||||
values = [0, 3, 4, 10]
|
||||
lower_bound: got [0, 1, 2, 5] expected [0, 1, 2, 5] OK
|
||||
upper_bound: got [0, 2, 2, 5] expected [0, 2, 2, 5] OK
|
||||
|
||||
Case 2: duplicates in data
|
||||
data = [1, 3, 3, 5, 7, 9]
|
||||
values = [3, 3, 5, 8]
|
||||
lower_bound: got [1, 1, 3, 5] expected [1, 1, 3, 5] OK
|
||||
upper_bound: got [3, 3, 4, 5] expected [3, 3, 4, 5] OK
|
||||
|
||||
Done
|
||||
```
|
||||
|
||||
**Note:** Device name and compute capability will vary based on your GPU.
|
||||
|
||||
## Files
|
||||
|
||||
- `binarySearch.py` - Python implementation
|
||||
- `README.md` - This file
|
||||
- `requirements.txt` - Sample dependencies
|
||||
- `../../Utilities/cuda_samples_utils.py` - Common utilities (imported by this sample)
|
||||
147
python/2_CoreConcepts/binarySearch/binarySearch.py
Normal file
147
python/2_CoreConcepts/binarySearch/binarySearch.py
Normal file
@@ -0,0 +1,147 @@
|
||||
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
|
||||
#
|
||||
# Redistribution and use in source and binary forms, with or without
|
||||
# modification, are permitted provided that the following conditions
|
||||
# are met:
|
||||
# * Redistributions of source code must retain the above copyright
|
||||
# notice, this list of conditions and the following disclaimer.
|
||||
# * Redistributions in binary form must reproduce the above copyright
|
||||
# notice, this list of conditions and the following disclaimer in the
|
||||
# documentation and/or other materials provided with the distribution.
|
||||
# * Neither the name of NVIDIA CORPORATION nor the names of its
|
||||
# contributors may be used to endorse or promote products derived
|
||||
# from this software without specific prior written permission.
|
||||
#
|
||||
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
|
||||
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
||||
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
||||
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
|
||||
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
|
||||
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
|
||||
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
|
||||
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
|
||||
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
"""
|
||||
This sample demonstrates the parallel binary-search algorithms exposed
|
||||
by cuda.compute (from the cuda-cccl package). Given a sorted
|
||||
``d_data`` array and a batch of ``d_values`` to locate, cuda.compute:
|
||||
|
||||
- ``cuda.compute.lower_bound(d_data, num_items, d_values, num_values, d_out)``
|
||||
writes, for each value, the lowest index where it could be inserted
|
||||
into d_data without breaking the sort order. Matches
|
||||
``numpy.searchsorted(..., side="left")``.
|
||||
|
||||
- ``cuda.compute.upper_bound(d_data, num_items, d_values, num_values, d_out)``
|
||||
is the analogous upper form, matching ``side="right"``.
|
||||
|
||||
The sample runs both algorithms on a curated sorted input with
|
||||
duplicates so the lower/upper distinction is visible, verifies the
|
||||
results against ``numpy.searchsorted``, and prints both sets of
|
||||
indices side-by-side.
|
||||
"""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent.parent / "Utilities"))
|
||||
|
||||
try:
|
||||
import cuda.compute
|
||||
import cupy as cp
|
||||
import numpy as np
|
||||
from cuda.core import Device
|
||||
from cuda_samples_utils import print_gpu_info # noqa: E402
|
||||
except ImportError as e:
|
||||
print(f"Error: Required package not found: {e}")
|
||||
print("Please install from requirements.txt:")
|
||||
print(" pip install -r requirements.txt")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def run_binary_search(h_data: np.ndarray, h_values: np.ndarray) -> bool:
|
||||
d_data = cp.asarray(h_data)
|
||||
d_values = cp.asarray(h_values)
|
||||
|
||||
d_lb = cp.empty(len(h_values), dtype=np.uintp)
|
||||
d_ub = cp.empty(len(h_values), dtype=np.uintp)
|
||||
|
||||
cuda.compute.lower_bound(
|
||||
d_data=d_data,
|
||||
num_items=len(d_data),
|
||||
d_values=d_values,
|
||||
num_values=len(d_values),
|
||||
d_out=d_lb,
|
||||
)
|
||||
cuda.compute.upper_bound(
|
||||
d_data=d_data,
|
||||
num_items=len(d_data),
|
||||
d_values=d_values,
|
||||
num_values=len(d_values),
|
||||
d_out=d_ub,
|
||||
)
|
||||
|
||||
got_lb = cp.asnumpy(d_lb)
|
||||
got_ub = cp.asnumpy(d_ub)
|
||||
expected_lb = np.searchsorted(h_data, h_values, side="left").astype(np.uintp)
|
||||
expected_ub = np.searchsorted(h_data, h_values, side="right").astype(np.uintp)
|
||||
|
||||
ok_lb = np.array_equal(got_lb, expected_lb)
|
||||
ok_ub = np.array_equal(got_ub, expected_ub)
|
||||
|
||||
print(f" data = {h_data.tolist()}")
|
||||
print(f" values = {h_values.tolist()}")
|
||||
print(
|
||||
f" lower_bound: got {got_lb.tolist()} "
|
||||
f"expected {expected_lb.tolist()} {'OK' if ok_lb else 'FAIL'}"
|
||||
)
|
||||
print(
|
||||
f" upper_bound: got {got_ub.tolist()} "
|
||||
f"expected {expected_ub.tolist()} {'OK' if ok_ub else 'FAIL'}"
|
||||
)
|
||||
return ok_lb and ok_ub
|
||||
|
||||
|
||||
def main():
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Parallel upper_bound / lower_bound via cuda.compute"
|
||||
)
|
||||
parser.add_argument("--device", type=int, default=0, help="CUDA device id")
|
||||
args = parser.parse_args()
|
||||
|
||||
device = Device(args.device)
|
||||
device.set_current()
|
||||
print_gpu_info(device)
|
||||
print()
|
||||
|
||||
ok = True
|
||||
|
||||
# Case 1: values both inside and outside the data range; no duplicates
|
||||
# in the data. lower_bound and upper_bound agree on values not present.
|
||||
print("Case 1: distinct data, mixed queries")
|
||||
h_data1 = np.array([1, 3, 5, 7, 9], dtype=np.int32)
|
||||
h_values1 = np.array([0, 3, 4, 10], dtype=np.int32)
|
||||
ok &= run_binary_search(h_data1, h_values1)
|
||||
print()
|
||||
|
||||
# Case 2: duplicates in the data so lower_bound and upper_bound diverge
|
||||
# on present values.
|
||||
print("Case 2: duplicates in data")
|
||||
h_data2 = np.array([1, 3, 3, 5, 7, 9], dtype=np.int32)
|
||||
h_values2 = np.array([3, 3, 5, 8], dtype=np.int32)
|
||||
ok &= run_binary_search(h_data2, h_values2)
|
||||
|
||||
print()
|
||||
if ok:
|
||||
print("Done")
|
||||
return 0
|
||||
print("FAILED")
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
4
python/2_CoreConcepts/binarySearch/requirements.txt
Normal file
4
python/2_CoreConcepts/binarySearch/requirements.txt
Normal file
@@ -0,0 +1,4 @@
|
||||
cuda-cccl[cu13]>=1.0.0
|
||||
cuda-core>=1.0.0
|
||||
cupy-cuda13x>=14.0.0
|
||||
numpy>=1.24.0
|
||||
@@ -139,7 +139,7 @@ def run_sample(num_elements: int = 1024 * 1024, device_id: int = 0) -> bool:
|
||||
|
||||
try:
|
||||
# Make CuPy use our stream
|
||||
cp.cuda.ExternalStream(int(stream.handle)).use()
|
||||
cp.cuda.Stream.from_external(stream).use()
|
||||
|
||||
# Compile kernels
|
||||
program = Program(
|
||||
@@ -216,7 +216,7 @@ def run_sample(num_elements: int = 1024 * 1024, device_id: int = 0) -> bool:
|
||||
test3 = verify_array_result(d_partial, expected_partial)
|
||||
|
||||
# Performance timing
|
||||
event_opts = EventOptions(enable_timing=True)
|
||||
event_opts = EventOptions(timing_enabled=True)
|
||||
iterations = 100
|
||||
|
||||
stream.sync()
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# Block-wise Array Sum Requirements
|
||||
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
cupy-cuda13x>=13.0.0
|
||||
cuda-core>=1.0.0
|
||||
cupy-cuda13x>=14.0.0
|
||||
numpy>=2.3.2
|
||||
|
||||
130
python/2_CoreConcepts/cudaComputeLambdas/README.md
Normal file
130
python/2_CoreConcepts/cudaComputeLambdas/README.md
Normal file
@@ -0,0 +1,130 @@
|
||||
# cudaComputeLambdas (Python)
|
||||
|
||||
## Description
|
||||
|
||||
This sample demonstrates how **cuda.compute** (from the
|
||||
`cuda-cccl` package) accepts plain Python callables, including
|
||||
lambdas, as the operators that drive device-wide reductions,
|
||||
transforms, and scans. Internally `cuda.compute` JIT-compiles the
|
||||
callable through Numba for the GPU, so you can iterate on the
|
||||
operator in pure Python and still get a fused device-wide kernel.
|
||||
|
||||
The sample exercises three algorithm families:
|
||||
|
||||
1. `cuda.compute.reduce_into` - sum via `lambda a, b: a + b`.
|
||||
2. `cuda.compute.unary_transform` - elementwise `y = x*x + 1` via a
|
||||
lambda.
|
||||
3. `cuda.compute.inclusive_scan` - prefix sum over only the even
|
||||
values, driven by a regular Python function as the binary
|
||||
operator.
|
||||
|
||||
## What You'll Learn
|
||||
|
||||
- Passing a Python `lambda` directly as the operator to a cuda.compute
|
||||
device algorithm
|
||||
- Using a regular Python `def` function for the same purpose when the
|
||||
op is non-trivial
|
||||
- The three core algorithm families in cuda.compute: reductions,
|
||||
transforms, and scans
|
||||
- How cuda.compute auto-compiles the op to LTO-IR via Numba
|
||||
|
||||
## Key Libraries
|
||||
|
||||
- [`cuda.compute`](https://nvidia.github.io/cccl/python.html) (from the `cuda-cccl` package) - device algorithms and JIT-compiled Python ops
|
||||
- [`cuda.core`](https://nvidia.github.io/cuda-python/cuda-core/latest/) - device setup
|
||||
- `cupy` - device buffers
|
||||
- `numpy` - scalar init values and host-side verification
|
||||
|
||||
## Key APIs
|
||||
|
||||
### From `cuda.compute`
|
||||
|
||||
- `cuda.compute.reduce_into(d_in, d_out, num_items, op, h_init)` - device-wide reduction
|
||||
- `cuda.compute.unary_transform(d_in, d_out, num_items, op)` - elementwise unary transform
|
||||
- `cuda.compute.inclusive_scan(d_in, d_out, op, init_value, num_items)` - inclusive prefix scan
|
||||
|
||||
### From `cuda_samples_utils`
|
||||
|
||||
- `print_gpu_info()` - print device name and compute capability
|
||||
|
||||
## Requirements
|
||||
|
||||
### Hardware
|
||||
|
||||
- NVIDIA GPU with Compute Capability 7.0 or higher
|
||||
|
||||
### Software
|
||||
|
||||
- CUDA Toolkit 13.0 or newer (cuda.compute compiles ops to LTO-IR via
|
||||
Numba, which needs the toolkit's `nvvm` and `libdevice`).
|
||||
- Python 3.10 or newer
|
||||
- `cuda-cccl` (>=1.0.0)
|
||||
- `cuda-core` (>=1.0.0)
|
||||
- `cupy-cuda13x` (>=14.0.0)
|
||||
- `numba-cuda` (pulled in transitively by `cuda-cccl`)
|
||||
|
||||
If the CUDA toolkit is not on your `PATH`, set `CUDA_HOME` so Numba
|
||||
can locate `libdevice`:
|
||||
|
||||
```bash
|
||||
export CUDA_HOME=/usr/local/cuda
|
||||
```
|
||||
|
||||
## Installation
|
||||
|
||||
Install the required packages from `requirements.txt`:
|
||||
|
||||
```bash
|
||||
cd /path/to/cuda-samples/python/2_CoreConcepts/cudaComputeLambdas
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
The `requirements.txt` installs:
|
||||
|
||||
- `cuda-cccl` (>=1.0.0) - ships the `cuda.compute` module
|
||||
- `cuda-core` (>=1.0.0)
|
||||
- `cupy-cuda13x` (>=14.0.0)
|
||||
- `numpy` (>=1.24.0)
|
||||
|
||||
## How to Run
|
||||
|
||||
### Basic usage
|
||||
|
||||
```bash
|
||||
cd cuda-samples/python/2_CoreConcepts/cudaComputeLambdas
|
||||
python cudaComputeLambdas.py
|
||||
```
|
||||
|
||||
### With custom parameters
|
||||
|
||||
```bash
|
||||
python cudaComputeLambdas.py --device 1
|
||||
```
|
||||
|
||||
## Expected Output
|
||||
|
||||
```
|
||||
Device: <Your GPU Name>
|
||||
Compute Capability: <X.Y>
|
||||
|
||||
reduce_into(lambda a,b: a+b) over 1..10 -> 55 (expected 55) OK
|
||||
|
||||
unary_transform(lambda x: x*x + 1):
|
||||
got = [1, 2, 5, 10, 17, 26, 37, 50]
|
||||
expected = [1, 2, 5, 10, 17, 26, 37, 50] OK
|
||||
|
||||
inclusive_scan(add-evens-only) over [1,2,3,4,5,6]:
|
||||
got = [0, 2, 2, 6, 6, 12]
|
||||
expected = [0, 2, 2, 6, 6, 12] OK
|
||||
|
||||
Done
|
||||
```
|
||||
|
||||
**Note:** Device name and compute capability will vary based on your GPU.
|
||||
|
||||
## Files
|
||||
|
||||
- `cudaComputeLambdas.py` - Python implementation
|
||||
- `README.md` - This file
|
||||
- `requirements.txt` - Sample dependencies
|
||||
- `../../Utilities/cuda_samples_utils.py` - Common utilities (imported by this sample)
|
||||
179
python/2_CoreConcepts/cudaComputeLambdas/cudaComputeLambdas.py
Normal file
179
python/2_CoreConcepts/cudaComputeLambdas/cudaComputeLambdas.py
Normal file
@@ -0,0 +1,179 @@
|
||||
# Copyright (c) 2025, NVIDIA CORPORATION. All rights reserved.
|
||||
#
|
||||
# Redistribution and use in source and binary forms, with or without
|
||||
# modification, are permitted provided that the following conditions
|
||||
# are met:
|
||||
# * Redistributions of source code must retain the above copyright
|
||||
# notice, this list of conditions and the following disclaimer.
|
||||
# * Redistributions in binary form must reproduce the above copyright
|
||||
# notice, this list of conditions and the following disclaimer in the
|
||||
# documentation and/or other materials provided with the distribution.
|
||||
# * Neither the name of NVIDIA CORPORATION nor the names of its
|
||||
# contributors may be used to endorse or promote products derived
|
||||
# from this software without specific prior written permission.
|
||||
#
|
||||
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
|
||||
# EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
|
||||
# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
|
||||
# PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
|
||||
# CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
|
||||
# EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
|
||||
# PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
|
||||
# PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
|
||||
# OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
|
||||
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
|
||||
# OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
||||
|
||||
"""
|
||||
cuda.compute: Python lambdas as device-wide operators
|
||||
|
||||
This sample demonstrates how cuda.compute 1.0 (from the cuda-cccl
|
||||
package) accepts plain Python callables, including lambdas, as the
|
||||
operators that drive device-wide reductions, transforms, and scans.
|
||||
Internally cuda.compute JIT-compiles the callable with Numba for the
|
||||
device, so you can iterate on the operator in pure Python and still
|
||||
get a fused GPU kernel.
|
||||
|
||||
The sample exercises three algorithm families with Python lambdas /
|
||||
regular functions:
|
||||
|
||||
1. cuda.compute.reduce_into - sum via a lambda.
|
||||
2. cuda.compute.unary_transform - elementwise y = x*x + 1 via a lambda.
|
||||
3. cuda.compute.inclusive_scan - prefix sum over only the even values,
|
||||
using a regular Python function as the binary operator.
|
||||
"""
|
||||
|
||||
import sys
|
||||
from pathlib import Path
|
||||
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent.parent / "Utilities"))
|
||||
|
||||
try:
|
||||
import cuda.compute
|
||||
import cupy as cp
|
||||
import numpy as np
|
||||
from cuda.core import Device
|
||||
from cuda_samples_utils import print_gpu_info # noqa: E402
|
||||
except ImportError as e:
|
||||
print(f"Error: Required package not found: {e}")
|
||||
print("Please install from requirements.txt:")
|
||||
print(" pip install -r requirements.txt")
|
||||
sys.exit(1)
|
||||
|
||||
|
||||
def demo_reduce_lambda() -> bool:
|
||||
"""reduce_into driven by a lambda."""
|
||||
dtype = np.int32
|
||||
h_init = np.array([0], dtype=dtype)
|
||||
d_in = cp.arange(1, 11, dtype=dtype) # 1..10
|
||||
d_out = cp.empty(1, dtype=dtype)
|
||||
|
||||
cuda.compute.reduce_into(
|
||||
d_in=d_in,
|
||||
d_out=d_out,
|
||||
num_items=int(d_in.size),
|
||||
op=lambda a, b: a + b,
|
||||
h_init=h_init,
|
||||
)
|
||||
|
||||
got = int(d_out.get()[0])
|
||||
expected = int(d_in.get().sum())
|
||||
ok = got == expected
|
||||
print(
|
||||
f"reduce_into(lambda a,b: a+b) over 1..10 -> {got} "
|
||||
f"(expected {expected}) {'OK' if ok else 'FAIL'}"
|
||||
)
|
||||
return ok
|
||||
|
||||
|
||||
def demo_unary_transform_lambda() -> bool:
|
||||
"""unary_transform driven by a lambda: y = x*x + 1."""
|
||||
d_in = cp.arange(8, dtype=cp.int32)
|
||||
d_out = cp.empty_like(d_in)
|
||||
|
||||
cuda.compute.unary_transform(
|
||||
d_in=d_in,
|
||||
d_out=d_out,
|
||||
num_items=int(d_in.size),
|
||||
op=lambda x: x * x + 1,
|
||||
)
|
||||
|
||||
got = d_out.get()
|
||||
expected = (d_in.get().astype(np.int64) ** 2 + 1).astype(np.int32)
|
||||
ok = np.array_equal(got, expected)
|
||||
print(
|
||||
f"unary_transform(lambda x: x*x + 1):\n"
|
||||
f" got = {got.tolist()}\n"
|
||||
f" expected = {expected.tolist()} {'OK' if ok else 'FAIL'}"
|
||||
)
|
||||
return ok
|
||||
|
||||
|
||||
def demo_scan_custom_op() -> bool:
|
||||
"""inclusive_scan with a Python function that sums only even values.
|
||||
|
||||
This shows the same pattern that also works for reduce/transform:
|
||||
the Python callable is JIT-compiled for the device by cuda.compute.
|
||||
"""
|
||||
dtype = np.int32
|
||||
d_in = cp.array([1, 2, 3, 4, 5, 6], dtype=dtype)
|
||||
d_out = cp.empty_like(d_in)
|
||||
h_init = np.array([0], dtype=dtype)
|
||||
|
||||
def add_evens(a, b):
|
||||
# Treat odd operands as zero; scan accumulates only even values.
|
||||
return (a if a % 2 == 0 else 0) + (b if b % 2 == 0 else 0)
|
||||
|
||||
cuda.compute.inclusive_scan(
|
||||
d_in=d_in,
|
||||
d_out=d_out,
|
||||
op=add_evens,
|
||||
init_value=h_init,
|
||||
num_items=int(d_in.size),
|
||||
)
|
||||
|
||||
got = d_out.get()
|
||||
# Host reference: running sum of even-only projection of the input.
|
||||
h_in = d_in.get()
|
||||
proj = np.where(h_in % 2 == 0, h_in, 0)
|
||||
expected = np.cumsum(proj).astype(dtype)
|
||||
ok = np.array_equal(got, expected)
|
||||
print(
|
||||
f"inclusive_scan(add-evens-only) over [1,2,3,4,5,6]:\n"
|
||||
f" got = {got.tolist()}\n"
|
||||
f" expected = {expected.tolist()} {'OK' if ok else 'FAIL'}"
|
||||
)
|
||||
return ok
|
||||
|
||||
|
||||
def main():
|
||||
import argparse
|
||||
|
||||
parser = argparse.ArgumentParser(
|
||||
description="Drive cuda.compute device algorithms with Python lambdas / callables"
|
||||
)
|
||||
parser.add_argument("--device", type=int, default=0, help="CUDA device id")
|
||||
args = parser.parse_args()
|
||||
|
||||
device = Device(args.device)
|
||||
device.set_current()
|
||||
print_gpu_info(device)
|
||||
print()
|
||||
|
||||
ok = True
|
||||
ok &= demo_reduce_lambda()
|
||||
print()
|
||||
ok &= demo_unary_transform_lambda()
|
||||
print()
|
||||
ok &= demo_scan_custom_op()
|
||||
|
||||
print()
|
||||
if ok:
|
||||
print("Done")
|
||||
return 0
|
||||
print("FAILED")
|
||||
return 1
|
||||
|
||||
|
||||
if __name__ == "__main__":
|
||||
sys.exit(main())
|
||||
@@ -0,0 +1,4 @@
|
||||
cuda-cccl[cu13]>=1.0.0
|
||||
cuda-core>=1.0.0
|
||||
cupy-cuda13x>=14.0.0
|
||||
numpy>=1.24.0
|
||||
@@ -62,8 +62,8 @@ rebuilding.
|
||||
- CUDA Toolkit 13.0 or newer (matches `cuda-python` 13.x)
|
||||
- Python 3.10 or newer
|
||||
- `cuda-python` (>=13.0.0)
|
||||
- `cuda-core` (>=0.6.0)
|
||||
- `cupy-cuda13x` (>=13.0.0)
|
||||
- `cuda-core` (>=1.0.0)
|
||||
- `cupy-cuda13x` (>=14.0.0)
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -77,8 +77,8 @@ pip install -r requirements.txt
|
||||
The `requirements.txt` installs:
|
||||
|
||||
- `cuda-python` (>=13.0.0)
|
||||
- `cuda-core` (>=0.6.0)
|
||||
- `cupy-cuda13x` (>=13.0.0)
|
||||
- `cuda-core` (>=1.0.0)
|
||||
- `cupy-cuda13x` (>=14.0.0)
|
||||
|
||||
## How to Run
|
||||
|
||||
|
||||
@@ -182,7 +182,7 @@ def main() -> int:
|
||||
stream = device.create_stream()
|
||||
# Tell CuPy to order its allocations on our stream so buffer initialization
|
||||
# below is serialized with the kernels we launch.
|
||||
cp.cuda.ExternalStream(int(stream.handle)).use()
|
||||
cp.cuda.Stream.from_external(stream).use()
|
||||
|
||||
graph_builder = graph = None
|
||||
try:
|
||||
@@ -214,9 +214,9 @@ def main() -> int:
|
||||
t_individual = run_pipeline_individual(
|
||||
stream, kernels, config, buffers, N, n_iters=args.iters
|
||||
)
|
||||
assert cp.allclose(r3, expected, rtol=1e-5, atol=1e-5), (
|
||||
"Individual pipeline produced incorrect results"
|
||||
)
|
||||
assert cp.allclose(
|
||||
r3, expected, rtol=1e-5, atol=1e-5
|
||||
), "Individual pipeline produced incorrect results"
|
||||
print(
|
||||
f"\nIndividual launches: {args.iters} iters in {t_individual:.4f}s"
|
||||
f" ({t_individual * 1e6 / args.iters:.2f} us/iter)"
|
||||
@@ -228,9 +228,9 @@ def main() -> int:
|
||||
|
||||
run_pipeline_graph(stream, graph, n_iters=5) # warm up
|
||||
t_graph = run_pipeline_graph(stream, graph, n_iters=args.iters)
|
||||
assert cp.allclose(r3, expected, rtol=1e-5, atol=1e-5), (
|
||||
"Graph pipeline produced incorrect results"
|
||||
)
|
||||
assert cp.allclose(
|
||||
r3, expected, rtol=1e-5, atol=1e-5
|
||||
), "Graph pipeline produced incorrect results"
|
||||
print(
|
||||
f"Graph replay: {args.iters} iters in {t_graph:.4f}s"
|
||||
f" ({t_graph * 1e6 / args.iters:.2f} us/iter)"
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# CUDA Graphs Sample Requirements
|
||||
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
cupy-cuda13x>=13.0.0
|
||||
cuda-core>=1.0.0
|
||||
cupy-cuda13x>=14.0.0
|
||||
|
||||
@@ -28,7 +28,7 @@ This sample demonstrates CuPy integration with cuda.core streams:
|
||||
stream = device.create_stream()
|
||||
|
||||
# Use with CuPy operations
|
||||
cp.cuda.ExternalStream(int(stream.handle)).use()
|
||||
cp.cuda.Stream.from_external(stream).use()
|
||||
```
|
||||
|
||||
## Key APIs
|
||||
@@ -43,7 +43,7 @@ cp.cuda.ExternalStream(int(stream.handle)).use()
|
||||
|
||||
- `cp.fft.rfft()` - Real-to-complex FFT (GPU-accelerated via cuFFT)
|
||||
- `cp.fft.rfftfreq()` - Generate frequency bins for rfft
|
||||
- `cp.cuda.ExternalStream()` - Interop with cuda.core streams
|
||||
- `cp.cuda.Stream.from_external()` - Interop with cuda.core streams
|
||||
|
||||
### From NumPy:
|
||||
|
||||
@@ -115,7 +115,7 @@ VERIFICATION
|
||||
GPU vs CPU FFT magnitude: Test PASSED
|
||||
|
||||
Frequency Detection Accuracy:
|
||||
440 Hz: ✓
|
||||
440 Hz: [OK]
|
||||
...
|
||||
|
||||
Done
|
||||
|
||||
@@ -180,7 +180,7 @@ def run_fft_analysis(
|
||||
print(f"Compute Capability: sm_{device.arch}")
|
||||
|
||||
# Make CuPy use our cuda.core stream
|
||||
cp.cuda.ExternalStream(int(stream.handle)).use()
|
||||
cp.cuda.Stream.from_external(stream).use()
|
||||
|
||||
# Define test signal: composite of multiple frequencies
|
||||
test_frequencies = [440.0, 880.0, 1320.0, 2000.0, 5000.0] # Hz
|
||||
@@ -208,7 +208,7 @@ def run_fft_analysis(
|
||||
print("GPU FFT (cuFFT)")
|
||||
print("-" * 60)
|
||||
|
||||
event_opts = EventOptions(enable_timing=True)
|
||||
event_opts = EventOptions(timing_enabled=True)
|
||||
|
||||
# Warmup
|
||||
d_fft_result = cp.fft.rfft(d_signal)
|
||||
@@ -291,7 +291,7 @@ def run_fft_analysis(
|
||||
all_found = True
|
||||
for expected_freq in test_frequencies:
|
||||
found = any(abs(f - expected_freq) < 10 for f in detected_freqs)
|
||||
status = "✓" if found else "✗"
|
||||
status = "[OK]" if found else "[FAIL]"
|
||||
print(f" {expected_freq:6.0f} Hz: {status}")
|
||||
all_found = all_found and found
|
||||
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
# FFT Signal Analysis Requirements
|
||||
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
cupy-cuda13x>=13.0.0
|
||||
cuda-core>=1.0.0
|
||||
cupy-cuda13x>=14.0.0
|
||||
numpy>=2.3.2
|
||||
|
||||
@@ -417,7 +417,7 @@ def run_critical_alone(
|
||||
Establishes the pure compute time with every SM on the device available.
|
||||
"""
|
||||
stream = device.create_stream()
|
||||
out = device.allocate(critical_n * 4)
|
||||
out = device.allocate(critical_n * 4, stream=stream)
|
||||
total_sm = device.resources.sm.sm_count
|
||||
try:
|
||||
opts = EventOptions(timing_enabled=True)
|
||||
@@ -461,7 +461,7 @@ def run_baseline(
|
||||
"""Both kernels on the primary context, two non-blocking streams."""
|
||||
long_stream = device.create_stream()
|
||||
critical_stream = device.create_stream()
|
||||
out = device.allocate(critical_n * 4)
|
||||
out = device.allocate(critical_n * 4, stream=critical_stream)
|
||||
total_sm = device.resources.sm.sm_count
|
||||
try:
|
||||
return _run_one(
|
||||
@@ -513,7 +513,7 @@ def run_green_context(
|
||||
|
||||
long_stream = ctx_long.create_stream()
|
||||
critical_stream = ctx_crit.create_stream()
|
||||
out = device.allocate(critical_n * 4)
|
||||
out = device.allocate(critical_n * 4, stream=critical_stream)
|
||||
|
||||
return _run_one(
|
||||
device,
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.7.0
|
||||
cuda-core>=1.0.0
|
||||
numpy>=2.3.2
|
||||
|
||||
@@ -64,8 +64,8 @@ linking modes are verified against a NumPy reference.
|
||||
- CUDA Toolkit 13.0 or newer (matches `cuda-python` 13.x)
|
||||
- Python 3.10 or newer
|
||||
- `cuda-python` (>=13.0.0)
|
||||
- `cuda-core` (>=0.6.0)
|
||||
- `cupy-cuda13x` (>=13.0.0)
|
||||
- `cuda-core` (>=1.0.0)
|
||||
- `cupy-cuda13x` (>=14.0.0)
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -79,8 +79,8 @@ pip install -r requirements.txt
|
||||
The `requirements.txt` installs:
|
||||
|
||||
- `cuda-python` (>=13.0.0)
|
||||
- `cuda-core` (>=0.6.0)
|
||||
- `cupy-cuda13x` (>=13.0.0)
|
||||
- `cuda-core` (>=1.0.0)
|
||||
- `cupy-cuda13x` (>=14.0.0)
|
||||
|
||||
## How to Run
|
||||
|
||||
|
||||
@@ -140,9 +140,7 @@ def link_lto(device):
|
||||
main_obj = Program(MAIN_SRC, "c++", options=prog_opts).compile("ltoir")
|
||||
user_obj = Program(USER_SRC, "c++", options=prog_opts).compile("ltoir")
|
||||
|
||||
linker_opts = LinkerOptions(
|
||||
arch=f"sm_{device.arch}", link_time_optimization=True
|
||||
)
|
||||
linker_opts = LinkerOptions(arch=f"sm_{device.arch}", link_time_optimization=True)
|
||||
linker = Linker(main_obj, user_obj, options=linker_opts)
|
||||
return linker.link("cubin")
|
||||
|
||||
@@ -175,7 +173,9 @@ def main() -> int:
|
||||
description="JIT + LTO linking of two device modules with cuda.core"
|
||||
)
|
||||
parser.add_argument(
|
||||
"--elements", type=int, default=1 << 16,
|
||||
"--elements",
|
||||
type=int,
|
||||
default=1 << 16,
|
||||
help="Number of float32 elements (default: 65536)",
|
||||
)
|
||||
parser.add_argument("--device", type=int, default=0, help="CUDA device id")
|
||||
@@ -186,7 +186,7 @@ def main() -> int:
|
||||
print_gpu_info(device)
|
||||
|
||||
stream = device.create_stream()
|
||||
cp.cuda.ExternalStream(int(stream.handle)).use()
|
||||
cp.cuda.Stream.from_external(stream).use()
|
||||
|
||||
try:
|
||||
N = args.elements
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# JIT + LTO Linking Sample Requirements
|
||||
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
cupy-cuda13x>=13.0.0
|
||||
cuda-core>=1.0.0
|
||||
cupy-cuda13x>=14.0.0
|
||||
|
||||
@@ -90,6 +90,15 @@ elapsed_ms = (end_event - start_event) / n_iterations
|
||||
- Python 3.10 or newer
|
||||
- See `requirements.txt` for Python packages
|
||||
|
||||
### Platform Support:
|
||||
|
||||
The benchmark loops in this sample read kernel results back from
|
||||
`ManagedMemoryResource` allocations between launches, which requires the
|
||||
device property `concurrent_managed_access=True`. This is only supported on
|
||||
Linux with HMM (Pascal and newer). On Windows (WDDM/MCDM/TCC) the property
|
||||
is `False`, so the sample exits early with a waive message and exit code
|
||||
`2`.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
@@ -115,8 +124,8 @@ Compute Capability: X.X
|
||||
|
||||
Compiling CUDA kernels with cuda.core.Program...
|
||||
Target architecture: sm_XX
|
||||
✓ vector_add kernel compiled
|
||||
✓ reduce_sum kernel compiled
|
||||
[OK] vector_add kernel compiled
|
||||
[OK] reduce_sum kernel compiled
|
||||
|
||||
============================================================
|
||||
VECTOR ADDITION - Launch Configuration Tuning
|
||||
@@ -132,11 +141,11 @@ Block Size: 64 | Blocks: 156250 | Time: X.XXXX ± X.XXXX ms
|
||||
...
|
||||
------------------------------------------------------------
|
||||
|
||||
✓ OPTIMAL: block_size=XXX (X.XXXX ms)
|
||||
✗ WORST: block_size=XXX (X.XXXX ms)
|
||||
[OK] OPTIMAL: block_size=XXX (X.XXXX ms)
|
||||
[FAIL] WORST: block_size=XXX (X.XXXX ms)
|
||||
Speedup: X.XXx
|
||||
|
||||
✓ Results verified correct!
|
||||
[OK] Results verified correct!
|
||||
|
||||
...
|
||||
|
||||
|
||||
@@ -148,7 +148,7 @@ def benchmark_kernel_1d(
|
||||
stream.sync()
|
||||
|
||||
# Timed runs with CUDA events
|
||||
event_opts = EventOptions(enable_timing=True)
|
||||
event_opts = EventOptions(timing_enabled=True)
|
||||
start_event = device.create_event(options=event_opts)
|
||||
end_event = device.create_event(options=event_opts)
|
||||
|
||||
@@ -178,7 +178,7 @@ def print_gpu_info(device):
|
||||
def allocate_managed_array(mr, stream, n_elements, dtype=np.float32):
|
||||
"""Allocate device-preferred unified memory and return buffer with numpy view."""
|
||||
n_bytes = n_elements * np.dtype(dtype).itemsize
|
||||
buffer = mr.allocate(n_bytes, stream)
|
||||
buffer = mr.allocate(n_bytes, stream=stream)
|
||||
stream.sync()
|
||||
|
||||
# Zero-copy numpy view via DLPack (holds reference to buffer)
|
||||
@@ -240,11 +240,11 @@ def demo_vector_add_tuning(device, stream, mr, kernel):
|
||||
|
||||
print("-" * 60)
|
||||
print(
|
||||
f"\n✓ OPTIMAL: block_size={best['block_size']} "
|
||||
f"\n[OK] OPTIMAL: block_size={best['block_size']} "
|
||||
f"({best['mean_time_ms']:.4f} ms)"
|
||||
)
|
||||
print(
|
||||
f"✗ WORST: block_size={worst['block_size']} "
|
||||
f"[FAIL] WORST: block_size={worst['block_size']} "
|
||||
f"({worst['mean_time_ms']:.4f} ms)"
|
||||
)
|
||||
print(f" Speedup: {worst['mean_time_ms']/best['mean_time_ms']:.2f}x")
|
||||
@@ -253,7 +253,7 @@ def demo_vector_add_tuning(device, stream, mr, kernel):
|
||||
stream.sync()
|
||||
expected = np_a + np_b
|
||||
if np.allclose(np_c, expected):
|
||||
print("\n✓ Results verified correct!")
|
||||
print("\n[OK] Results verified correct!")
|
||||
|
||||
return results
|
||||
finally:
|
||||
@@ -316,7 +316,7 @@ def demo_reduction_tuning(device, stream, mr, kernel):
|
||||
worst = max(results, key=lambda x: x["mean_time_ms"])
|
||||
|
||||
print("-" * 60)
|
||||
print(f"\n✓ OPTIMAL: block_size={best['block_size']}")
|
||||
print(f"\n[OK] OPTIMAL: block_size={best['block_size']}")
|
||||
print(
|
||||
f" Speedup over worst: {worst['mean_time_ms']/best['mean_time_ms']:.2f}x"
|
||||
)
|
||||
@@ -341,6 +341,14 @@ def main():
|
||||
3. Benchmarking different thread block configurations
|
||||
4. Finding optimal threads-per-block for various kernel types
|
||||
"""
|
||||
if sys.platform == "win32":
|
||||
print(
|
||||
"This sample relies on ManagedMemoryResource with concurrent host "
|
||||
"access, which is not supported on Windows "
|
||||
"(concurrent_managed_access=False). Waiving this sample."
|
||||
)
|
||||
sys.exit(2)
|
||||
|
||||
print("=" * 60)
|
||||
print("Launch Configuration Tuning (cuda.core)")
|
||||
print("Finding the Best Block Size for Your Kernel")
|
||||
@@ -365,10 +373,10 @@ def main():
|
||||
print(f" Target architecture: {arch}")
|
||||
|
||||
vec_add_kernel = compile_kernel(device, VECTOR_ADD_KERNEL, "vector_add")
|
||||
print(" ✓ vector_add kernel compiled")
|
||||
print(" [OK] vector_add kernel compiled")
|
||||
|
||||
reduction_kernel = compile_kernel(device, REDUCTION_KERNEL, "reduce_sum")
|
||||
print(" ✓ reduce_sum kernel compiled")
|
||||
print(" [OK] reduce_sum kernel compiled")
|
||||
|
||||
# Run demonstrations
|
||||
demo_vector_add_tuning(device, stream, mr, vec_add_kernel)
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# Requires Python 3.10+, CUDA Toolkit 13.0+
|
||||
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
cuda-core>=1.0.0
|
||||
numpy>=2.3.2
|
||||
|
||||
@@ -1,5 +1,25 @@
|
||||
# Matrix Multiplication with Shared Memory (GEMM)
|
||||
|
||||
> **Known issue — version-pinned sample.** Unlike the other samples in this
|
||||
> repository, this sample is pinned to `cuda-core==0.7.0` and
|
||||
> `nvmath-python[cu13]==0.9.0`. The reason is that nvmath-python 0.9.0
|
||||
> still uses `cuda-core`'s pre-1.0 API name `EventOptions(enable_timing=...)`
|
||||
> in its own internals, which `cuda-core>=1.0` no longer accepts.
|
||||
>
|
||||
> If you install this sample's `requirements.txt` into the same environment
|
||||
> as the other samples, pip will downgrade `cuda-core` and the other
|
||||
> samples (which use the 1.0 API) will stop working. The recommended
|
||||
> workflow is one of:
|
||||
>
|
||||
> - Install this sample's requirements in a **dedicated virtual
|
||||
> environment**, or
|
||||
> - Re-run the other samples' `pip install -r requirements.txt` afterwards
|
||||
> to upgrade `cuda-core` back to 1.0.
|
||||
>
|
||||
> This sample will be re-aligned with the rest of the repository
|
||||
> (`cuda-core>=1.0.0`) once nvmath-python ships a release that targets
|
||||
> cuda-core's 1.0 naming audit.
|
||||
|
||||
Demonstrates efficient matrix multiplication using nvmath-python APIs and custom CUDA kernels with tiling, shared memory, and loop unrolling.
|
||||
|
||||
## Overview
|
||||
@@ -78,7 +98,7 @@ Using nvmath and cuda.core APIs
|
||||
Device: NVIDIA GeForce RTX 4090
|
||||
Compute Capability: sm_89
|
||||
|
||||
Custom kernel compiled ✓
|
||||
Custom kernel compiled [OK]
|
||||
|
||||
Matrix dimensions: A(1024x1024) × B(1024x1024) = C(1024x1024)
|
||||
Custom kernel tile size: 16x16
|
||||
|
||||
@@ -31,7 +31,7 @@ Demonstrates efficient matrix multiplication using:
|
||||
- nvmath.linalg.advanced.Matmul for high-performance GEMM via cuBLASLt
|
||||
- Custom CUDA kernel with tiling, shared memory, and loop unrolling
|
||||
|
||||
Uses cuda.core APIs with CuPy arrays via ExternalStream.
|
||||
Uses cuda.core APIs with CuPy arrays via Stream.from_external.
|
||||
"""
|
||||
|
||||
import sys
|
||||
@@ -121,17 +121,22 @@ def run_matmul_benchmark(
|
||||
print(f"Compute Capability: sm_{device.arch}")
|
||||
|
||||
# Make CuPy use our cuda.core stream
|
||||
cp.cuda.ExternalStream(int(stream.handle)).use()
|
||||
cp.cuda.Stream.from_external(stream).use()
|
||||
|
||||
# Compile custom kernel
|
||||
arch = f"sm_{device.arch}"
|
||||
program = Program(MATMUL_KERNEL, code_type="c++", options=ProgramOptions(arch=arch))
|
||||
kernel = program.compile(target_type="cubin").get_kernel("matmul_shared")
|
||||
print("Custom kernel compiled ✓")
|
||||
print("Custom kernel compiled [OK]")
|
||||
|
||||
# Setup
|
||||
print(f"\nMatrix: A({m}x{k}) × B({k}x{n}) = C({m}x{n})")
|
||||
total_ops = 2 * m * n * k
|
||||
# NOTE: this sample is pinned to cuda-core==0.7.0 (see requirements.txt)
|
||||
# because nvmath-python 0.9.0 still uses cuda-core's pre-1.0 API name
|
||||
# `enable_timing`. Once nvmath ships a release compatible with cuda-core
|
||||
# 1.0, bump the pins in requirements.txt and rename this kwarg to
|
||||
# `timing_enabled` to match the rest of the samples.
|
||||
event_opts = EventOptions(enable_timing=True)
|
||||
|
||||
# Allocate matrices
|
||||
|
||||
@@ -1,7 +1,20 @@
|
||||
# Matrix Multiplication with Shared Memory (GEMM) Requirements
|
||||
#
|
||||
# IMPORTANT: this sample pins older versions of cuda-core and nvmath-python
|
||||
# on purpose. nvmath-python 0.9.0 (the current CUDA-13 release at the time
|
||||
# of CTK 13.3) calls cuda-core's pre-1.0 API name `EventOptions(enable_timing=...)`
|
||||
# in its own internals. With cuda-core 1.0+ that kwarg was renamed to
|
||||
# `timing_enabled` and the old name is rejected, so any cuda-core>=1.0 +
|
||||
# nvmath-python 0.9.0 combination raises a TypeError at runtime.
|
||||
#
|
||||
# Until nvmath-python ships a release that targets the cuda-core 1.0 naming
|
||||
# audit, this sample requires the older cuda-core 0.7 line. Installing this
|
||||
# requirements.txt into the same environment as the other samples will
|
||||
# downgrade cuda-core; use a dedicated venv for this sample, or reinstall
|
||||
# the other samples' requirements afterwards to upgrade cuda-core back.
|
||||
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
cupy-cuda13x>=13.0.0
|
||||
cuda-core==0.7.0
|
||||
cupy-cuda13x>=14.0.0
|
||||
numpy>=2.3.2
|
||||
nvmath-python[cu13]>=0.3.0
|
||||
nvmath-python[cu13]==0.9.0
|
||||
|
||||
@@ -60,15 +60,27 @@ verified on the host.
|
||||
### Hardware
|
||||
|
||||
- NVIDIA GPU with Compute Capability 7.0 or higher
|
||||
- Managed memory support (most discrete GPUs on Linux and Windows)
|
||||
- Managed memory support (most discrete GPUs)
|
||||
|
||||
### Software
|
||||
|
||||
- CUDA Toolkit 13.0 or newer (matches `cuda-python` 13.x)
|
||||
- Python 3.10 or newer
|
||||
- `cuda-python` (>=13.0.0)
|
||||
- `cuda-core` (>=0.6.0)
|
||||
- `cupy-cuda13x` (>=13.0.0)
|
||||
- `cuda-core` (>=1.0.0)
|
||||
- `cupy-cuda13x` (>=14.0.0)
|
||||
|
||||
### Platform Support
|
||||
|
||||
The `ManagedMemoryResource` demo in this sample exercises **concurrent host
|
||||
access** to managed allocations while the GPU is active, which requires the
|
||||
device property `concurrent_managed_access=True`. This is only supported on
|
||||
Linux with HMM (Pascal and newer). On Windows (WDDM/MCDM/TCC) the property
|
||||
is `False`, so the sample exits early with a waive message and exit code
|
||||
`2`. The `DeviceMemoryResource` + `PinnedMemoryResource` demos in this
|
||||
sample would still work on Windows on their own, but to keep the sample
|
||||
self-contained the entire script waives when concurrent managed access is
|
||||
unavailable.
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -82,8 +94,8 @@ pip install -r requirements.txt
|
||||
The `requirements.txt` installs:
|
||||
|
||||
- `cuda-python` (>=13.0.0)
|
||||
- `cuda-core` (>=0.6.0)
|
||||
- `cupy-cuda13x` (>=13.0.0)
|
||||
- `cuda-core` (>=1.0.0)
|
||||
- `cupy-cuda13x` (>=14.0.0)
|
||||
|
||||
## How to Run
|
||||
|
||||
|
||||
@@ -165,9 +165,9 @@ def demo_managed(device, stream, kernel, size):
|
||||
stream.sync()
|
||||
|
||||
# No explicit copy: the same numpy view observes the GPU's writes.
|
||||
assert np.allclose(managed_view, original * 0.5 + 10.0), (
|
||||
"Managed memory result mismatch"
|
||||
)
|
||||
assert np.allclose(
|
||||
managed_view, original * 0.5 + 10.0
|
||||
), "Managed memory result mismatch"
|
||||
print(" GPU writes observed directly through the host-visible mapping")
|
||||
finally:
|
||||
managed_buffer.close(stream)
|
||||
@@ -222,6 +222,14 @@ def main():
|
||||
parser.add_argument("--device", type=int, default=0, help="CUDA device id")
|
||||
args = parser.parse_args()
|
||||
|
||||
if sys.platform == "win32":
|
||||
print(
|
||||
"This sample relies on ManagedMemoryResource with concurrent host "
|
||||
"access, which is not supported on Windows "
|
||||
"(concurrent_managed_access=False). Waiving this sample."
|
||||
)
|
||||
sys.exit(2)
|
||||
|
||||
device = Device(args.device)
|
||||
device.set_current()
|
||||
print_gpu_info(device)
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
# Memory Resources Sample Requirements
|
||||
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
cupy-cuda13x>=13.0.0
|
||||
cuda-core>=1.0.0
|
||||
cupy-cuda13x>=14.0.0
|
||||
|
||||
@@ -1,5 +1,23 @@
|
||||
# Sample: PageRank Algorithm (Python)
|
||||
|
||||
> **Known issue — version-pinned sample.** Unlike the other samples in this
|
||||
> repository, this sample is pinned to `cuda-core<1.0.0`. The reason is that
|
||||
> `cudf-cu13` transitively requires `numba-cuda<0.29.0`, and every
|
||||
> `numba-cuda` release in that range pins `cuda-core<1.0.0`. Installing this
|
||||
> sample's `requirements.txt` into a shared environment will downgrade
|
||||
> `cuda-core` and break the other samples (which use the 1.0 API).
|
||||
>
|
||||
> The recommended workflow is one of:
|
||||
>
|
||||
> - Install this sample's requirements in a **dedicated virtual
|
||||
> environment**, or
|
||||
> - Re-run the other samples' `pip install -r requirements.txt` afterwards
|
||||
> to upgrade `cuda-core` back to 1.0.
|
||||
>
|
||||
> This sample will be re-aligned with the rest of the repository
|
||||
> (`cuda-core>=1.0.0`) once `cudf-cu13` ships a release that lifts its
|
||||
> `numba-cuda` upper bound.
|
||||
|
||||
## Description
|
||||
|
||||
Demonstrates GPU-accelerated PageRank computation for graph analysis using RAPIDS cuGraph, with cuda.core for device, stream, and GPU timing. This sample focuses on cuda.core integration with high-level libraries (cuGraph/cuDF); for custom kernel programming (Program, LaunchConfig, launch), see the blockwiseSum sample.
|
||||
@@ -16,7 +34,7 @@ Demonstrates GPU-accelerated PageRank computation for graph analysis using RAPID
|
||||
- `cugraph` - RAPIDS GPU-accelerated graph analytics
|
||||
- `cudf` - RAPIDS GPU DataFrame library
|
||||
- `cuda.core` - Device, stream, and event APIs for GPU timing
|
||||
- `cupy` - GPU array library (ExternalStream for cuDF/cuGraph)
|
||||
- `cupy` - GPU array library (Stream.from_external for cuDF/cuGraph)
|
||||
- `numpy` - CPU reference implementation
|
||||
|
||||
## Key APIs
|
||||
@@ -25,7 +43,7 @@ Demonstrates GPU-accelerated PageRank computation for graph analysis using RAPID
|
||||
|
||||
- `Device(0)` - Create device, `device.set_current()`, `device.create_stream()`
|
||||
- `EventOptions(enable_timing=True)` - GPU timing via `stream.record()`
|
||||
- `cp.cuda.ExternalStream(stream.handle).use()` - Make cuDF/cuGraph use cuda.core stream
|
||||
- `cp.cuda.Stream.from_external(stream).use()` - Make cuDF/cuGraph use cuda.core stream
|
||||
|
||||
### From cuGraph:
|
||||
|
||||
@@ -50,6 +68,14 @@ Demonstrates GPU-accelerated PageRank computation for graph analysis using RAPID
|
||||
- Python 3.10 or newer
|
||||
- See requirements.txt for package dependencies
|
||||
|
||||
### Platform Support:
|
||||
|
||||
This sample depends on RAPIDS (`cugraph-cu13`, `cudf-cu13`, `dask-cuda`),
|
||||
which is currently published only as **Linux (manylinux) wheels** on
|
||||
`pypi.nvidia.com` — no Windows wheels exist. On Windows the sample exits
|
||||
early with a waive message and exit code `2` instead of attempting an
|
||||
install that cannot succeed.
|
||||
|
||||
## Installation
|
||||
|
||||
```bash
|
||||
|
||||
@@ -46,6 +46,14 @@ from pathlib import Path
|
||||
sys.path.insert(0, str(Path(__file__).parent.parent.parent / "Utilities"))
|
||||
from cuda_samples_utils import print_gpu_info, verify_array_result # noqa: E402
|
||||
|
||||
if sys.platform == "win32":
|
||||
print(
|
||||
"This sample depends on RAPIDS (cugraph-cu13 / cudf-cu13), which is "
|
||||
"currently published only as Linux (manylinux) wheels on "
|
||||
"pypi.nvidia.com. Waiving this sample on Windows."
|
||||
)
|
||||
sys.exit(2)
|
||||
|
||||
try:
|
||||
import cudf
|
||||
import cugraph
|
||||
@@ -210,8 +218,22 @@ def run_pagerank_benchmark(
|
||||
print()
|
||||
print_gpu_info(device)
|
||||
|
||||
# RAPIDS cuGraph wheels currently don't ship kernel binaries for
|
||||
# every CUDA architecture. Skip cleanly on architectures known to
|
||||
# be unsupported instead of failing deep inside cuGraph with a
|
||||
# cryptic cudaErrorNoKernelImageForDevice. Remove an arch from this
|
||||
# set once the matching cuGraph release ships kernels for it.
|
||||
_CUGRAPH_UNSUPPORTED_ARCHES = {"110"} # sm_110 = Thor / Tegra
|
||||
if device.arch in _CUGRAPH_UNSUPPORTED_ARCHES:
|
||||
print(
|
||||
f"RAPIDS cuGraph does not yet ship kernels for sm_{device.arch}, "
|
||||
"waiving this sample."
|
||||
)
|
||||
stream.close()
|
||||
sys.exit(2)
|
||||
|
||||
# Make CuPy/cuDF use our cuda.core stream
|
||||
cp.cuda.ExternalStream(int(stream.handle)).use()
|
||||
cp.cuda.Stream.from_external(stream).use()
|
||||
|
||||
# Generate random graph
|
||||
print("\nGraph Parameters:")
|
||||
|
||||
@@ -1,8 +1,11 @@
|
||||
# PageRank Requirements (RAPIDS cuGraph)
|
||||
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
cugraph-cu13>=25.0.0
|
||||
cudf-cu13>=25.0.0
|
||||
cupy-cuda13x>=13.0.0
|
||||
# cudf-cu13 transitively pins numba-cuda<0.29.0 which requires cuda-core<1.0.0
|
||||
cuda-core<1.0.0
|
||||
cugraph-cu13>=26.0.0
|
||||
cudf-cu13>=26.0.0
|
||||
# dask-cuda <26.4 incorrectly pins cuda-core==0.3.*; require the fixed release
|
||||
dask-cuda>=26.4.0
|
||||
cupy-cuda13x>=14.0.0
|
||||
numpy>=2.3.2
|
||||
|
||||
@@ -190,7 +190,7 @@ def _run_histogram(device, stream):
|
||||
# Benchmark using cuda.core Events (explicit Event objects recorded on stream)
|
||||
print("\nBenchmarking (100 iterations)...")
|
||||
num_iterations = 100
|
||||
event_opts = EventOptions(enable_timing=True)
|
||||
event_opts = EventOptions(timing_enabled=True)
|
||||
start_event = device.create_event(options=event_opts)
|
||||
end_event = device.create_event(options=event_opts)
|
||||
|
||||
|
||||
@@ -2,6 +2,6 @@
|
||||
# Requires Python 3.10+, CUDA Toolkit 13.0+
|
||||
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
cuda-core>=1.0.0
|
||||
numpy>=2.3.2
|
||||
cupy-cuda13x>=13.0.0
|
||||
cupy-cuda13x>=14.0.0
|
||||
|
||||
@@ -52,9 +52,9 @@ if (tid % (2 * s) == 0) { // Don't do this!
|
||||
- CUDA Toolkit 13.0+
|
||||
- Python 3.10+
|
||||
- `cuda-python` (13.0.0+)
|
||||
- `cuda-core` (>=0.6.0)
|
||||
- `cuda-core` (>=1.0.0)
|
||||
- `cuda-cccl` (1.0.0+)
|
||||
- `cupy` (13.0.0+)
|
||||
- `cupy-cuda13x` (>=14.0.0)
|
||||
- `numpy` (>=2.3.2)
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -35,7 +35,7 @@ Key Concepts:
|
||||
- Reduction tree pattern: Divide-and-conquer parallel algorithm
|
||||
- Thread synchronization: Using __syncthreads() for coordination
|
||||
- Sequential thread IDs: How to avoid warp divergence
|
||||
- cuda.core Stream integration with CuPy via ExternalStream
|
||||
- cuda.core Stream integration with CuPy via Stream.from_external
|
||||
"""
|
||||
|
||||
import math
|
||||
@@ -217,7 +217,7 @@ def benchmark_custom(
|
||||
stream, kernel, d_input, block_size=block_size, work_buffers=work_buffers
|
||||
)
|
||||
|
||||
event_opts = {"enable_timing": True}
|
||||
event_opts = {"timing_enabled": True}
|
||||
start_event = stream.device.create_event(options=event_opts)
|
||||
end_event = stream.device.create_event(options=event_opts)
|
||||
|
||||
@@ -264,7 +264,7 @@ def benchmark_cuda_compute(
|
||||
stream.sync()
|
||||
|
||||
d_output = cp.empty(1, dtype=cp.float32)
|
||||
event_opts = {"enable_timing": True}
|
||||
event_opts = {"timing_enabled": True}
|
||||
start_event = stream.device.create_event(options=event_opts)
|
||||
end_event = stream.device.create_event(options=event_opts)
|
||||
|
||||
@@ -299,7 +299,7 @@ def main() -> bool:
|
||||
device = Device(0)
|
||||
device.set_current()
|
||||
stream = device.create_stream()
|
||||
cp_stream = cp.cuda.ExternalStream(int(stream.handle))
|
||||
cp_stream = cp.cuda.Stream.from_external(stream)
|
||||
|
||||
print()
|
||||
print_gpu_info(device)
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
# Parallel Reduction Sample Requirements
|
||||
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
cuda-core>=1.0.0
|
||||
cuda-cccl>=1.0.0
|
||||
cupy-cuda13x>=13.0.0
|
||||
cupy-cuda13x>=14.0.0
|
||||
numpy>=2.3.2
|
||||
|
||||
@@ -8,7 +8,7 @@ Demonstrates parallel prefix sum (scan) algorithms using cuda.compute with cuda.
|
||||
- Exclusive scan: `output[i] = init_value + input[0] + input[1] + ... + input[i-1]`
|
||||
- Uses cuda.compute APIs for optimized CUB-based implementations
|
||||
- Uses cuda.core APIs for device and stream management
|
||||
- Demonstrates CuPy integration via `ExternalStream`
|
||||
- Demonstrates CuPy integration via `Stream.from_external`
|
||||
|
||||
## Requirements
|
||||
|
||||
@@ -21,9 +21,9 @@ Demonstrates parallel prefix sum (scan) algorithms using cuda.compute with cuda.
|
||||
- CUDA Toolkit 13.0+
|
||||
- Python 3.10+
|
||||
- `cuda-python` (13.0.0+)
|
||||
- `cuda-core` (>=0.6.0)
|
||||
- `cuda-core` (>=1.0.0)
|
||||
- `cuda-cccl` (1.0.0+)
|
||||
- `cupy-cuda13x` (13.0.0+)
|
||||
- `cupy-cuda13x` (14.0.0+)
|
||||
- `numpy` (>=2.3.2)
|
||||
|
||||
## Usage
|
||||
@@ -56,8 +56,8 @@ This sample demonstrates proper stream usage across libraries:
|
||||
# Create stream with cuda.core
|
||||
stream = device.create_stream()
|
||||
|
||||
# Wrap for CuPy compatibility (requires int handle)
|
||||
cp_stream = cp.cuda.ExternalStream(int(stream.handle))
|
||||
# Wrap for CuPy compatibility (cuda.core Stream implements the __cuda_stream__ protocol)
|
||||
cp_stream = cp.cuda.Stream.from_external(stream)
|
||||
|
||||
# Use with CuPy operations
|
||||
with cp_stream:
|
||||
|
||||
@@ -62,7 +62,7 @@ def main() -> bool:
|
||||
device = Device(0)
|
||||
device.set_current()
|
||||
stream = device.create_stream()
|
||||
cp_stream = cp.cuda.ExternalStream(int(stream.handle))
|
||||
cp_stream = cp.cuda.Stream.from_external(stream)
|
||||
|
||||
ok = True
|
||||
try:
|
||||
@@ -153,7 +153,7 @@ def main() -> bool:
|
||||
)
|
||||
stream.sync()
|
||||
|
||||
event_opts = EventOptions(enable_timing=True)
|
||||
event_opts = EventOptions(timing_enabled=True)
|
||||
start_event = device.create_event(options=event_opts)
|
||||
end_event = device.create_event(options=event_opts)
|
||||
|
||||
@@ -184,7 +184,7 @@ def main() -> bool:
|
||||
print("• Inclusive: output[i] includes input[i]")
|
||||
print("• Exclusive: output[i] excludes input[i], starts with init_value")
|
||||
print("• cuda.compute provides CUB-based optimized implementations")
|
||||
print("• cuda.core Stream integrates with CuPy via ExternalStream")
|
||||
print("• cuda.core Stream integrates with CuPy via Stream.from_external")
|
||||
print("• Applications: stream compaction, radix sort, histograms")
|
||||
print("=" * 60)
|
||||
return ok
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
# Requires Python 3.10+, CUDA Toolkit 13.0+
|
||||
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
cuda-core>=1.0.0
|
||||
cuda-cccl>=1.0.0
|
||||
cupy-cuda13x>=13.0.0
|
||||
cupy-cuda13x>=14.0.0
|
||||
numpy>=2.3.2
|
||||
|
||||
@@ -195,6 +195,19 @@ def main():
|
||||
print(f"Buffer size: {args.buffer_mib} MiB")
|
||||
print(f"Lock timeout: {args.lock_timeout_ms} ms")
|
||||
|
||||
# CUDA process checkpointing relies on kernel-mode driver features
|
||||
# that aren't shipped on integrated-GPU platforms (e.g. Tegra /
|
||||
# Jetson / Thor). On those, Process.lock() can hang indefinitely
|
||||
# instead of returning a clean "not supported" error. Skip cleanly
|
||||
# rather than hanging. Remove this guard once integrated platforms
|
||||
# gain checkpoint support.
|
||||
if device.properties.integrated:
|
||||
print(
|
||||
f"CUDA process checkpointing is not supported on integrated "
|
||||
f"GPUs (sm_{device.arch}), waiving this sample."
|
||||
)
|
||||
return 2
|
||||
|
||||
print()
|
||||
print("Compiling kernel ...")
|
||||
fill_kernel = compile_fill_kernel(device)
|
||||
|
||||
@@ -1,3 +1,4 @@
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.7.0
|
||||
# cuda-bindings 13.3.0 drops the CUcheckpointRestoreArgs alias that cuda-core requires
|
||||
cuda-python>=13.0.0,<13.3.0
|
||||
cuda-core>=1.0.0
|
||||
numpy>=2.3.2
|
||||
|
||||
@@ -82,7 +82,7 @@ Two-Stage Reduction Strategy:
|
||||
- Elements per block: 512
|
||||
- Output: 32768 partial sums
|
||||
Stage 2: CPU final reduction
|
||||
- Combine 32768 partial sums → 1 final result
|
||||
- Combine 32768 partial sums -> 1 final result
|
||||
|
||||
Compiling CUDA kernel...
|
||||
Kernel 'blockReduceKernel_float' compiled successfully
|
||||
|
||||
@@ -259,7 +259,7 @@ def run(
|
||||
print(f" - Elements per block: {threads_per_block * 2}")
|
||||
print(f" - Output: {num_blocks} partial sums")
|
||||
print(" Stage 2: CPU final reduction")
|
||||
print(f" - Combine {num_blocks} partial sums → 1 final result")
|
||||
print(f" - Combine {num_blocks} partial sums -> 1 final result")
|
||||
|
||||
# Compile kernel
|
||||
print("\nCompiling CUDA kernel...")
|
||||
@@ -320,7 +320,7 @@ def run(
|
||||
|
||||
# cuda.core event elapsed time (end - start) is in milliseconds (CUDA API).
|
||||
stage1_times_ms = []
|
||||
event_options = EventOptions(enable_timing=True)
|
||||
event_options = EventOptions(timing_enabled=True)
|
||||
start_event = stream.device.create_event(options=event_options)
|
||||
end_event = stream.device.create_event(options=event_options)
|
||||
for _ in range(test_iterations):
|
||||
@@ -342,7 +342,7 @@ def run(
|
||||
|
||||
# Stage 2 (CPU)
|
||||
print("\n> Running Stage 2 (CPU final reduction)...")
|
||||
# Device → Host: after stream sync, partial sums are visible on host.
|
||||
# Device -> Host: after stream sync, partial sums are visible on host.
|
||||
stream.sync()
|
||||
with cp_stream:
|
||||
h_blockSums = cp.asnumpy(d_blockSums)
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
# Install with: pip install -r requirements.txt
|
||||
|
||||
numpy>=2.3.2
|
||||
cuda-core>=0.6.0
|
||||
cuda-core>=1.0.0
|
||||
cuda-python>=13.0.0
|
||||
# Use cupy-cuda13x>=14.0.0 for cp.cuda.Stream.from_external(stream)
|
||||
cupy-cuda13x>=14.0.0
|
||||
|
||||
@@ -33,13 +33,24 @@ Pick a CuPy wheel that matches your CUDA major version (e.g. `cupy-cuda13x` in `
|
||||
|
||||
## How to run
|
||||
|
||||
**`--cuda-include-dir` is required** (colon-separated list). Typical desktop layout:
|
||||
**`--cuda-include-dir` is required.** Multiple paths can be combined using the
|
||||
OS path separator (`:` on Linux/macOS, `;` on Windows).
|
||||
|
||||
Linux / macOS:
|
||||
|
||||
```bash
|
||||
python reductionMultiBlockCG.py \
|
||||
--cuda-include-dir /usr/local/cuda/include/cccl:/usr/local/cuda/include
|
||||
```
|
||||
|
||||
Windows (PowerShell or cmd, note the `;` separator and quotes around the
|
||||
combined value):
|
||||
|
||||
```powershell
|
||||
python reductionMultiBlockCG.py `
|
||||
--cuda-include-dir "$env:CUDA_PATH\include;$env:CUDA_PATH\include\cccl"
|
||||
```
|
||||
|
||||
**Jetson / split include trees:** pass every directory NVRTC needs in one `--cuda-include-dir` argument, e.g.
|
||||
`/usr/local/cuda/include/cccl:/usr/local/cuda/targets/sbsa-linux/include` (adjust paths to your image). If headers are scattered, you can instead merge them into one tree with symlinks and point `--cuda-include-dir` at that folder.
|
||||
|
||||
@@ -94,15 +105,15 @@ Summary
|
||||
======================================================================
|
||||
|
||||
Single-kernel two-stage reduction:
|
||||
Stage 1: 20 blocks → 20 partial sums
|
||||
grid.sync() ← All blocks synchronize (KEY innovation)
|
||||
Stage 2: Block 0 → 1 final result
|
||||
Stage 1: 20 blocks -> 20 partial sums
|
||||
grid.sync() <- All blocks synchronize (KEY innovation)
|
||||
Stage 2: Block 0 -> 1 final result
|
||||
Total: 1 kernel launch, 137.35 GB/s
|
||||
|
||||
Comparison:
|
||||
• Traditional: 2 kernel launches or kernel + CPU
|
||||
• This sample: 1 kernel with grid.sync() between stages
|
||||
• Benefit: Eliminates ~5-20μs launch overhead per stage
|
||||
• Benefit: Eliminates ~5-20us launch overhead per stage
|
||||
|
||||
======================================================================
|
||||
Single-Pass Multi-Block Reduction completed successfully!
|
||||
|
||||
@@ -238,8 +238,10 @@ def run(
|
||||
|
||||
# Compile kernel
|
||||
print("\nCompiling CUDA kernel...")
|
||||
# Support colon-separated multiple include paths
|
||||
include_paths = cuda_include_dir.split(":")
|
||||
# Support multiple include paths separated by the OS path separator
|
||||
# (':' on POSIX, ';' on Windows). os.pathsep avoids splitting Windows
|
||||
# drive prefixes like "C:\..." by accident.
|
||||
include_paths = cuda_include_dir.split(os.pathsep)
|
||||
program_options = ProgramOptions(
|
||||
std="c++17", arch=f"sm_{device.arch}", include_path=include_paths
|
||||
)
|
||||
@@ -295,7 +297,7 @@ def run(
|
||||
grid=(num_blocks, 1, 1),
|
||||
block=(threads_per_block, 1, 1),
|
||||
shmem_size=shared_mem_bytes,
|
||||
cooperative_launch=True,
|
||||
is_cooperative=True,
|
||||
)
|
||||
|
||||
n_u32 = np.uint32(num_elements)
|
||||
@@ -313,7 +315,7 @@ def run(
|
||||
|
||||
# Benchmark (CUDA events — not host wall clock around the whole loop)
|
||||
print(f"\n> Running benchmark ({test_iterations} iterations)...")
|
||||
event_options = EventOptions(enable_timing=True)
|
||||
event_options = EventOptions(timing_enabled=True)
|
||||
start_event = stream.device.create_event(options=event_options)
|
||||
end_event = stream.device.create_event(options=event_options)
|
||||
# cuda.core event elapsed time (end - start) is in milliseconds (CUDA API).
|
||||
@@ -362,15 +364,15 @@ def run(
|
||||
print("=" * 70)
|
||||
print(f"""
|
||||
Single-kernel two-stage reduction:
|
||||
Stage 1: {num_blocks} blocks → {num_blocks} partial sums
|
||||
grid.sync() ← All blocks synchronize (KEY innovation)
|
||||
Stage 2: Block 0 → 1 final result
|
||||
Stage 1: {num_blocks} blocks -> {num_blocks} partial sums
|
||||
grid.sync() <- All blocks synchronize (KEY innovation)
|
||||
Stage 2: Block 0 -> 1 final result
|
||||
Total: 1 kernel launch, {throughput_gb_s:.2f} GB/s
|
||||
|
||||
Comparison:
|
||||
• Traditional: 2 kernel launches or kernel + CPU
|
||||
• This sample: 1 kernel with grid.sync() between stages
|
||||
• Benefit: Eliminates ~5-20μs launch overhead per stage
|
||||
• Benefit: Eliminates ~5-20us launch overhead per stage
|
||||
""")
|
||||
|
||||
print("=" * 70)
|
||||
@@ -430,8 +432,9 @@ def main():
|
||||
type=str,
|
||||
required=True,
|
||||
help=(
|
||||
"CUDA include directory for NVRTC "
|
||||
"(can use colon-separated paths, e.g., /path1:/path2)"
|
||||
"CUDA include directory for NVRTC. "
|
||||
"Use os.pathsep to separate multiple paths "
|
||||
"(':' on POSIX, ';' on Windows)."
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
# Install with: pip install -r requirements.txt
|
||||
|
||||
numpy>=2.3.2
|
||||
cuda-core>=0.6.0
|
||||
cuda-core>=1.0.0
|
||||
cuda-python>=13.0.0
|
||||
# Headers for NVRTC: cooperative_groups.h includes cuda/std/* (CCCL)
|
||||
cuda-cccl>=1.0.0
|
||||
|
||||
@@ -62,7 +62,7 @@ pip install -r requirements.txt
|
||||
|
||||
Or manually:
|
||||
```bash
|
||||
pip install numpy>=2.3.2 cuda-core>=0.6.0 cuda-python>=13.0.0
|
||||
pip install numpy>=2.3.2 cuda-core>=1.0.0 cuda-python>=13.0.0
|
||||
```
|
||||
|
||||
## How to run
|
||||
|
||||
@@ -3,4 +3,4 @@
|
||||
|
||||
numpy>=2.3.2
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
cuda-core>=1.0.0
|
||||
|
||||
@@ -67,7 +67,7 @@ Using pure cuda.core APIs
|
||||
============================================================
|
||||
|
||||
Device: NVIDIA GeForce RTX XXXX
|
||||
Kernel compiled ✓
|
||||
Kernel compiled [OK]
|
||||
|
||||
Problem size: 16,000,000 elements (61 MB)
|
||||
|
||||
|
||||
@@ -2,5 +2,5 @@
|
||||
# Requires Python 3.10 or newer
|
||||
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
cuda-core>=1.0.0
|
||||
numpy>=2.3.2
|
||||
|
||||
@@ -102,7 +102,7 @@ def main():
|
||||
VECTOR_SCALE_KERNEL, code_type="c++", options=ProgramOptions(arch=arch)
|
||||
)
|
||||
kernel = program.compile(target_type="cubin").get_kernel("vector_scale")
|
||||
print("Kernel compiled ✓")
|
||||
print("Kernel compiled [OK]")
|
||||
|
||||
# Parameters
|
||||
N = 16_000_000 # 16M elements
|
||||
@@ -126,10 +126,10 @@ def main():
|
||||
h_in = h_out = d_in = d_out = None
|
||||
try:
|
||||
# Pre-allocate buffers
|
||||
h_in = pinned_mr.allocate(n_bytes, default_stream)
|
||||
h_out = pinned_mr.allocate(n_bytes, default_stream)
|
||||
d_in = device_mr.allocate(n_bytes, default_stream)
|
||||
d_out = device_mr.allocate(n_bytes, default_stream)
|
||||
h_in = pinned_mr.allocate(n_bytes, stream=default_stream)
|
||||
h_out = pinned_mr.allocate(n_bytes, stream=default_stream)
|
||||
d_in = device_mr.allocate(n_bytes, stream=default_stream)
|
||||
d_out = device_mr.allocate(n_bytes, stream=default_stream)
|
||||
# Sync before numpy access (numpy operations aren't stream ordered)
|
||||
default_stream.sync()
|
||||
|
||||
@@ -138,7 +138,7 @@ def main():
|
||||
np_in[:] = np.random.rand(N).astype(np.float32) * 100
|
||||
|
||||
config = LaunchConfig(grid=((N + 255) // 256,), block=(256,))
|
||||
event_opts = EventOptions(enable_timing=True)
|
||||
event_opts = EventOptions(timing_enabled=True)
|
||||
|
||||
# Warm up
|
||||
h_in.copy_to(d_in, stream=default_stream)
|
||||
@@ -214,10 +214,10 @@ def main():
|
||||
h_ins, h_outs, d_ins, d_outs = [], [], [], []
|
||||
try:
|
||||
for i in range(n_streams):
|
||||
h_ins.append(pinned_mr.allocate(chunk_bytes, streams[i]))
|
||||
h_outs.append(pinned_mr.allocate(chunk_bytes, streams[i]))
|
||||
d_ins.append(device_mr.allocate(chunk_bytes, streams[i]))
|
||||
d_outs.append(device_mr.allocate(chunk_bytes, streams[i]))
|
||||
h_ins.append(pinned_mr.allocate(chunk_bytes, stream=streams[i]))
|
||||
h_outs.append(pinned_mr.allocate(chunk_bytes, stream=streams[i]))
|
||||
d_ins.append(device_mr.allocate(chunk_bytes, stream=streams[i]))
|
||||
d_outs.append(device_mr.allocate(chunk_bytes, stream=streams[i]))
|
||||
|
||||
# Initialize input data
|
||||
for i in range(n_streams):
|
||||
@@ -245,7 +245,7 @@ def main():
|
||||
|
||||
# Benchmark with CUDA events (use stream 0 for timing)
|
||||
times = []
|
||||
event_opts = EventOptions(enable_timing=True)
|
||||
event_opts = EventOptions(timing_enabled=True)
|
||||
for _ in range(n_runs):
|
||||
start_ev = device.create_event(options=event_opts)
|
||||
end_ev = device.create_event(options=event_opts)
|
||||
|
||||
@@ -73,8 +73,8 @@ The sample:
|
||||
- CUDA Toolkit 13.0 or newer with libcudacxx (cccl) headers
|
||||
- Python 3.10 or newer
|
||||
- `cuda-python` (>=13.0.0)
|
||||
- `cuda-core` (>=0.6.0)
|
||||
- `cupy-cuda13x` (>=13.0.0)
|
||||
- `cuda-core` (>=1.0.0)
|
||||
- `cupy-cuda13x` (>=14.0.0)
|
||||
|
||||
## Installation
|
||||
|
||||
@@ -88,8 +88,8 @@ pip install -r requirements.txt
|
||||
The `requirements.txt` installs:
|
||||
|
||||
- `cuda-python` (>=13.0.0)
|
||||
- `cuda-core` (>=0.6.0)
|
||||
- `cupy-cuda13x` (>=13.0.0)
|
||||
- `cuda-core` (>=1.0.0)
|
||||
- `cupy-cuda13x` (>=14.0.0)
|
||||
|
||||
## How to Run
|
||||
|
||||
|
||||
@@ -1,4 +1,4 @@
|
||||
cuda-python>=13.0.0
|
||||
cuda-core>=0.6.0
|
||||
cupy-cuda13x>=13.0.0
|
||||
cuda-core>=1.0.0
|
||||
cupy-cuda13x>=14.0.0
|
||||
numpy>=1.24.0
|
||||
|
||||
@@ -64,9 +64,9 @@ try:
|
||||
LaunchConfig,
|
||||
Program,
|
||||
ProgramOptions,
|
||||
StridedMemoryView,
|
||||
launch,
|
||||
)
|
||||
from cuda.core.utils import StridedMemoryView
|
||||
from cuda.pathfinder import find_nvidia_header_directory, get_cuda_path_or_home
|
||||
from cuda_samples_utils import print_gpu_info
|
||||
except ImportError as e:
|
||||
@@ -147,11 +147,7 @@ def _get_cccl_include_paths() -> list:
|
||||
# CUDA runtime headers - needed for the CUtensorMap driver type.
|
||||
try:
|
||||
cudart_dir = find_nvidia_header_directory("cudart")
|
||||
if (
|
||||
cudart_dir
|
||||
and os.path.isdir(cudart_dir)
|
||||
and cudart_dir not in include_path
|
||||
):
|
||||
if cudart_dir and os.path.isdir(cudart_dir) and cudart_dir not in include_path:
|
||||
include_path.append(cudart_dir)
|
||||
except Exception: # noqa: S110 - fallback probes continue below
|
||||
pass
|
||||
@@ -232,9 +228,9 @@ def main() -> int:
|
||||
output = cp.zeros(n, dtype=cp.float32)
|
||||
dev.sync() # CuPy uses its own stream
|
||||
|
||||
tensor_map = StridedMemoryView.from_any_interface(
|
||||
src, stream_ptr=-1
|
||||
).as_tensor_map(box_dim=(TILE_SIZE,))
|
||||
tensor_map = StridedMemoryView.from_any_interface(src, stream_ptr=-1).as_tensor_map(
|
||||
box_dim=(TILE_SIZE,)
|
||||
)
|
||||
|
||||
n_tiles = n // TILE_SIZE
|
||||
config = LaunchConfig(grid=n_tiles, block=TILE_SIZE)
|
||||
|
||||
Reference in New Issue
Block a user