CUDA 13.4 samples update - v13.4-public

Release 13.4 of the CUDA Samples supported by CUDA Toolkit 13.4.
See Changelog for more information.
This commit is contained in:
Dheemanth
2026-09-09 15:07:08 -07:00
committed by GitHub
parent b7c5481c55
commit 5443602d89
871 changed files with 6843 additions and 321849 deletions

View File

@@ -0,0 +1,175 @@
# Persistent Program Cache (Python)
## Description
This sample demonstrates how to persist and reuse compiled CUDA artifacts
with `cuda.core`. It generates a specialized matmul + epilogue kernel,
compiles it to CUBIN on a cache miss, stores the artifact in
`FileStreamProgramCache`, and reloads the CUBIN on later runs.
## Core Cache Flow
The central cache logic is in `compile_or_load_kernel`:
```python
options = ProgramOptions(std="c++17", arch=f"sm_{device.arch}")
program = Program(source, code_type="c++", options=options)
key = make_program_cache_key(...)
with FileStreamProgramCache(cache_dir) as cache:
cached = cache.get(key)
if cached is None:
module = program.compile("cubin")
cache[key] = module
status = "MISS"
else:
module = ObjectCode.from_cubin(cached, name="persistentProgramCache")
status = "HIT"
kernel = module.get_kernel(KERNEL_NAME)
```
## What You'll Learn
- Generating a CUDA source string from workload configuration
- Building a persistent cache key with `make_program_cache_key`
- Storing compiled CUBIN bytes with `FileStreamProgramCache`
- Reconstructing a loadable `ObjectCode` with `ObjectCode.from_cubin`
- Measuring cache lookup, compile, module load, and kernel execution time
- Validating that cached and freshly compiled artifacts produce the same result
## Key Libraries
- [`cuda.core`][cuda-core] - Pythonic access to CUDA programs, object code, launches, and events
- `cupy` - input and output buffers on the GPU
- `numpy` - deterministic input generation and host reference computation
## Key APIs
### From `cuda.core`
- `Program(...).compile("cubin")` - compile generated CUDA source to CUBIN
- `ObjectCode.from_cubin(...)` - reconstruct loadable object code from cached bytes
- `ObjectCode.get_kernel(name)` - fetch the kernel from the compiled artifact
- `LaunchConfig` and `launch(...)` - configure and launch the generated kernel
- `EventOptions(timing_enabled=True)` - time repeated kernel launches
### From `cuda.core.utils`
- `FileStreamProgramCache` - disk-backed, process-safe program cache
- `make_program_cache_key(...)` - derive a cache key from source, options, and target type
## Requirements
### Hardware
- NVIDIA GPU with Compute Capability 7.0 or higher
### Software
- CUDA Toolkit 13.0 or newer (matches `cuda-python` 13.x)
- Python 3.10 or newer
- `cuda-python` (>=13.0.0)
- `cuda-core` (>=1.0.0)
- `cupy-cuda13x` (>=14.0.0)
- `numpy` (>=2.3.2)
## Installation
Install the required packages from `requirements.txt`:
```bash
cd /path/to/cuda-samples/python/2_CoreConcepts/persistentProgramCache
pip install -r requirements.txt
```
The `requirements.txt` installs:
- `cuda-python` (>=13.0.0)
- `cuda-core` (>=1.0.0)
- `cupy-cuda13x` (>=14.0.0)
- `numpy` (>=2.3.2)
## How to Run
### Basic usage
Run once with a cleared cache to force compilation:
```bash
cd cuda-samples/python/2_CoreConcepts/persistentProgramCache
python persistentProgramCache.py --clear-cache
```
Run again with the same configuration to reuse the cached artifact:
```bash
python persistentProgramCache.py
```
### With custom parameters
```bash
# Compile a different generated kernel variant
python persistentProgramCache.py --tile-size 32 --epilogue identity
# Use a custom matrix size
python persistentProgramCache.py --m 1024 --n 1024 --k 1024
# Use a specific GPU
python persistentProgramCache.py --device 1
```
Changing the tile size, epilogue, source code, compile options, or target
GPU architecture changes the cache key and produces a cache miss.
## Expected Output
The output includes the run configuration, cache status, timings, and
validation result:
```text
Persistent Program Cache
Device: <Your GPU Name>
Compute Capability: <X.Y>
Cache directory: <cache path>
Workload: C = relu(A @ B + bias)
Matrix sizes: M=512, N=512, K=512
Tile size: 16
Timed launches: warmup=5, iterations=20
Cache status: MISS | HIT
Cache key: <key prefix>...
Artifact size: <size> KiB
Cache lookup: <time> ms
Compile time: <time> ms | skipped
Cache store: <time> ms (MISS only)
Module load: <time> ms
Host prep time: <time> ms
Kernel time: <time> ms
Max error: <error>
Validation: PASSED
```
Run with `--clear-cache` first to force a miss, then run again with the same
configuration to get a hit. Compare `Host prep time`: the miss path compiles
and stores the CUBIN, while the hit path reloads the cached CUBIN.
**Note:** Device name, timing, artifact size, and cache key will vary based
on GPU, driver, CUDA Toolkit, and host system.
## Files
- `persistentProgramCache.py` - Python implementation using `cuda.core` program cache utilities
- `README.md` - This file
- `requirements.txt` - Sample dependencies
## See Also
- [CUDA Python Documentation](https://nvidia.github.io/cuda-python/)
- [`cuda.core` program caches API](
https://nvidia.github.io/cuda-python/cuda-core/latest/api.html#program-caches)
- [`cuda.core` examples](https://github.com/NVIDIA/cuda-python/tree/main/cuda_core/examples)
[cuda-core]: https://nvidia.github.io/cuda-python/cuda-core/latest/