mirror of
https://github.com/NVIDIA/cuda-samples.git
synced 2026-11-17 17:38:18 +08:00
CUDA 13.4 samples update - v13.4-public
Release 13.4 of the CUDA Samples supported by CUDA Toolkit 13.4. See Changelog for more information.
This commit is contained in:
175
python/2_CoreConcepts/persistentProgramCache/README.md
Normal file
175
python/2_CoreConcepts/persistentProgramCache/README.md
Normal file
@@ -0,0 +1,175 @@
|
||||
# Persistent Program Cache (Python)
|
||||
|
||||
## Description
|
||||
|
||||
This sample demonstrates how to persist and reuse compiled CUDA artifacts
|
||||
with `cuda.core`. It generates a specialized matmul + epilogue kernel,
|
||||
compiles it to CUBIN on a cache miss, stores the artifact in
|
||||
`FileStreamProgramCache`, and reloads the CUBIN on later runs.
|
||||
|
||||
## Core Cache Flow
|
||||
|
||||
The central cache logic is in `compile_or_load_kernel`:
|
||||
|
||||
```python
|
||||
options = ProgramOptions(std="c++17", arch=f"sm_{device.arch}")
|
||||
program = Program(source, code_type="c++", options=options)
|
||||
key = make_program_cache_key(...)
|
||||
|
||||
with FileStreamProgramCache(cache_dir) as cache:
|
||||
cached = cache.get(key)
|
||||
|
||||
if cached is None:
|
||||
module = program.compile("cubin")
|
||||
cache[key] = module
|
||||
status = "MISS"
|
||||
else:
|
||||
module = ObjectCode.from_cubin(cached, name="persistentProgramCache")
|
||||
status = "HIT"
|
||||
|
||||
kernel = module.get_kernel(KERNEL_NAME)
|
||||
```
|
||||
|
||||
## What You'll Learn
|
||||
|
||||
- Generating a CUDA source string from workload configuration
|
||||
- Building a persistent cache key with `make_program_cache_key`
|
||||
- Storing compiled CUBIN bytes with `FileStreamProgramCache`
|
||||
- Reconstructing a loadable `ObjectCode` with `ObjectCode.from_cubin`
|
||||
- Measuring cache lookup, compile, module load, and kernel execution time
|
||||
- Validating that cached and freshly compiled artifacts produce the same result
|
||||
|
||||
## Key Libraries
|
||||
|
||||
- [`cuda.core`][cuda-core] - Pythonic access to CUDA programs, object code, launches, and events
|
||||
- `cupy` - input and output buffers on the GPU
|
||||
- `numpy` - deterministic input generation and host reference computation
|
||||
|
||||
## Key APIs
|
||||
|
||||
### From `cuda.core`
|
||||
|
||||
- `Program(...).compile("cubin")` - compile generated CUDA source to CUBIN
|
||||
- `ObjectCode.from_cubin(...)` - reconstruct loadable object code from cached bytes
|
||||
- `ObjectCode.get_kernel(name)` - fetch the kernel from the compiled artifact
|
||||
- `LaunchConfig` and `launch(...)` - configure and launch the generated kernel
|
||||
- `EventOptions(timing_enabled=True)` - time repeated kernel launches
|
||||
|
||||
### From `cuda.core.utils`
|
||||
|
||||
- `FileStreamProgramCache` - disk-backed, process-safe program cache
|
||||
- `make_program_cache_key(...)` - derive a cache key from source, options, and target type
|
||||
|
||||
## Requirements
|
||||
|
||||
### Hardware
|
||||
|
||||
- NVIDIA GPU with Compute Capability 7.0 or higher
|
||||
|
||||
### Software
|
||||
|
||||
- CUDA Toolkit 13.0 or newer (matches `cuda-python` 13.x)
|
||||
- Python 3.10 or newer
|
||||
- `cuda-python` (>=13.0.0)
|
||||
- `cuda-core` (>=1.0.0)
|
||||
- `cupy-cuda13x` (>=14.0.0)
|
||||
- `numpy` (>=2.3.2)
|
||||
|
||||
## Installation
|
||||
|
||||
Install the required packages from `requirements.txt`:
|
||||
|
||||
```bash
|
||||
cd /path/to/cuda-samples/python/2_CoreConcepts/persistentProgramCache
|
||||
pip install -r requirements.txt
|
||||
```
|
||||
|
||||
The `requirements.txt` installs:
|
||||
|
||||
- `cuda-python` (>=13.0.0)
|
||||
- `cuda-core` (>=1.0.0)
|
||||
- `cupy-cuda13x` (>=14.0.0)
|
||||
- `numpy` (>=2.3.2)
|
||||
|
||||
## How to Run
|
||||
|
||||
### Basic usage
|
||||
|
||||
Run once with a cleared cache to force compilation:
|
||||
|
||||
```bash
|
||||
cd cuda-samples/python/2_CoreConcepts/persistentProgramCache
|
||||
python persistentProgramCache.py --clear-cache
|
||||
```
|
||||
|
||||
Run again with the same configuration to reuse the cached artifact:
|
||||
|
||||
```bash
|
||||
python persistentProgramCache.py
|
||||
```
|
||||
|
||||
### With custom parameters
|
||||
|
||||
```bash
|
||||
# Compile a different generated kernel variant
|
||||
python persistentProgramCache.py --tile-size 32 --epilogue identity
|
||||
|
||||
# Use a custom matrix size
|
||||
python persistentProgramCache.py --m 1024 --n 1024 --k 1024
|
||||
|
||||
# Use a specific GPU
|
||||
python persistentProgramCache.py --device 1
|
||||
```
|
||||
|
||||
Changing the tile size, epilogue, source code, compile options, or target
|
||||
GPU architecture changes the cache key and produces a cache miss.
|
||||
|
||||
## Expected Output
|
||||
|
||||
The output includes the run configuration, cache status, timings, and
|
||||
validation result:
|
||||
|
||||
```text
|
||||
Persistent Program Cache
|
||||
Device: <Your GPU Name>
|
||||
Compute Capability: <X.Y>
|
||||
Cache directory: <cache path>
|
||||
Workload: C = relu(A @ B + bias)
|
||||
Matrix sizes: M=512, N=512, K=512
|
||||
Tile size: 16
|
||||
Timed launches: warmup=5, iterations=20
|
||||
|
||||
Cache status: MISS | HIT
|
||||
Cache key: <key prefix>...
|
||||
Artifact size: <size> KiB
|
||||
Cache lookup: <time> ms
|
||||
Compile time: <time> ms | skipped
|
||||
Cache store: <time> ms (MISS only)
|
||||
Module load: <time> ms
|
||||
Host prep time: <time> ms
|
||||
Kernel time: <time> ms
|
||||
Max error: <error>
|
||||
Validation: PASSED
|
||||
```
|
||||
|
||||
Run with `--clear-cache` first to force a miss, then run again with the same
|
||||
configuration to get a hit. Compare `Host prep time`: the miss path compiles
|
||||
and stores the CUBIN, while the hit path reloads the cached CUBIN.
|
||||
|
||||
**Note:** Device name, timing, artifact size, and cache key will vary based
|
||||
on GPU, driver, CUDA Toolkit, and host system.
|
||||
|
||||
## Files
|
||||
|
||||
- `persistentProgramCache.py` - Python implementation using `cuda.core` program cache utilities
|
||||
- `README.md` - This file
|
||||
- `requirements.txt` - Sample dependencies
|
||||
|
||||
## See Also
|
||||
|
||||
- [CUDA Python Documentation](https://nvidia.github.io/cuda-python/)
|
||||
- [`cuda.core` program caches API](
|
||||
https://nvidia.github.io/cuda-python/cuda-core/latest/api.html#program-caches)
|
||||
- [`cuda.core` examples](https://github.com/NVIDIA/cuda-python/tree/main/cuda_core/examples)
|
||||
|
||||
[cuda-core]: https://nvidia.github.io/cuda-python/cuda-core/latest/
|
||||
Reference in New Issue
Block a user