NASP/cuda-samples
84
0
Fork 0
mirror of https://github.com/NVIDIA/cuda-samples.git synced 2026-10-13 00:08:26 +08:00
Code Issues Packages Projects Releases Wiki Activity

CUDA 13.4 samples update - v13.4-public

Browse Source 
Release 13.4 of the CUDA Samples supported by CUDA Toolkit 13.4.
See Changelog for more information.
This commit is contained in:
Dheemanth
2026-09-09 15:07:08 -07:00
committed by GitHub
parent b7c5481c55
commit 5443602d89
871 changed files with 6843 additions and 321849 deletions
Download Patch File Download Diff File Expand all files Collapse all files

36
cpp/3_CUDA_Features/dmabufInterop/CMakeLists.txt Normal file
View File

@@ -0,0 +1,36 @@
cmake_minimum_required(VERSION 3.20)
include("${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/DetectCudaArch.cmake")
if(SAMPLE_SKIP_BUILD)
return()
endif()
project(dmabufInterop LANGUAGES C CXX CUDA)
# dma-buf is a Linux-only kernel facility. Skip the sample on all other platforms.
if(NOT (CMAKE_SYSTEM_NAME STREQUAL "Linux"))
message(STATUS "Skipping dmabufInterop: requires Linux (dma-buf is Linux-only). "
"Found ${CMAKE_SYSTEM_NAME}.")
return()
endif()
include("${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/CudaSampleCommon.cmake")
find_package(CUDAToolkit REQUIRED)
include_directories(../../../Common)
add_executable(dmabufInterop dmabufInterop.cu)
target_compile_options(dmabufInterop PRIVATE $<$<COMPILE_LANGUAGE:CUDA>:--extended-lambda>)
target_compile_features(dmabufInterop PRIVATE cxx_std_17 cuda_std_17)
target_link_libraries(dmabufInterop PRIVATE
CUDA::cuda_driver
CUDA::cudart
)
include(InstallSamples)
setup_samples_install()

124
cpp/3_CUDA_Features/dmabufInterop/README.md Normal file
View File

@@ -0,0 +1,124 @@
# Sample: CUDA dma-buf Interoperability
## Description
Linux **dma-buf** is the kernel's standard mechanism for sharing a buffer between subsystems by representing it as a file descriptor. This sample shows CUDA acting as both a dma-buf **producer** (taking one of its own allocations and obtaining a dma-buf fd for it with `cuMemGetHandleForAddressRange`) and a dma-buf **consumer** (taking an existing dma-buf fd and turning it into a CUDA-mappable buffer with `cuImportExternalMemory` + `cuExternalMemoryGetMappedBuffer`). A single binary runs three demos in sequence:
1. **Same-process round-trip** — one CUDA context exports a host allocation and imports it back as external memory.
2. **Cross-process IPC** — a `fork()`-ed child exports an allocation and passes the fd to the parent over a Unix domain socket using `SCM_RIGHTS`.
3. **Cross-GPU sharing across processes** — the same `fork()` + `SCM_RIGHTS` path as demo 2, but the producer child runs on one GPU and the consumer parent imports the fd into a different GPU's context. The sample auto-detects any pair of GPUs whose driver reports support for the dma-buf capability attributes. This demo self-skips when no qualifying pair is present.
Producer-side allocations always use `cuMemAllocHost`; using a single allocator keeps the producer code identical across all three demos.
## What You'll Learn
- Exporting a CUDA host allocation as a Linux dma-buf file descriptor
- Importing a dma-buf fd into CUDA as external memory and mapping it to a device pointer
- Passing a dma-buf fd between processes via a Unix-domain socket using `SCM_RIGHTS`
- Sharing a single buffer between two GPUs in two different processes, with the producer-completes-before-consumer-reads ordering enforced by the existing fork + socket handshake
- Detecting dma-buf platform support at runtime with CUDA device attributes
- Coordinating fork()ed children so output and CUDA work appear in source order rather than the order processes happen to be scheduled
## Key Concepts
- **dma-buf** — kernel-side shareable buffer represented as a file descriptor; usable across CUDA, V4L2, DRM/KMS, Vulkan, and any other subsystem that speaks dma-buf
- **Page-aligned export** — `cuMemGetHandleForAddressRange` requires a page-aligned base, so the producer over-allocates by one page and rounds up
- **fd as IPC primitive** — once CUDA has produced a dma-buf fd, the fd is just a POSIX file descriptor and can be sent across `fork()` boundaries via `SCM_RIGHTS`
- **Capability-gated execution** — `CU_DEVICE_ATTRIBUTE_HOST_ALLOC_DMA_BUF_SUPPORTED` (producer side) and `CU_DEVICE_ATTRIBUTE_DMA_BUF_SUPPORTED` (consumer side) determine which paths run on the current hardware
- **Parent-arbitrated synchronisation** — both fork()ed children block on a one-byte "go" from the parent before doing any CUDA work, so output across the three demos appears in source order even when stdout is fully buffered (e.g., over ssh)
## Key APIs
### CUDA Driver API
- `cuMemAllocHost` — allocate page-locked host memory that is also a valid CUDA device pointer
- `cuMemGetHandleForAddressRange` (with `CU_MEM_RANGE_HANDLE_TYPE_DMA_BUF_FD`) — export a CUDA allocation as a Linux dma-buf fd
- `cuImportExternalMemory` (with `CU_EXTERNAL_MEMORY_HANDLE_TYPE_DMABUF_FD`) — bring a dma-buf fd into CUDA as external memory
- `cuExternalMemoryGetMappedBuffer` — obtain a `CUdeviceptr` from an imported external memory object
- `cuDeviceGetAttribute` — runtime capability query for `CU_DEVICE_ATTRIBUTE_HOST_ALLOC_DMA_BUF_SUPPORTED` and `CU_DEVICE_ATTRIBUTE_DMA_BUF_SUPPORTED`
### CUDA Runtime API
- `cudaSetDevice` — pin the runtime to the producer GPU (child process) or the consumer GPU (parent process); both demos that use multiple GPUs do this on each side after the fork
- `cudaGetDeviceCount` — used inside the cross-GPU pair detection to enumerate the available GPUs
- `cudaDeviceSynchronize` — used on the producer side to make sure the kernel that wrote the buffer has retired before the fd is sent to the consumer
### Linux / POSIX
- `socketpair` + `fork` + `sendmsg`/`recvmsg` with `SCM_RIGHTS` — pass a file descriptor between processes
## Requirements
### Hardware
- An NVIDIA GPU whose `CU_DEVICE_ATTRIBUTE_HOST_ALLOC_DMA_BUF_SUPPORTED` is `1`. The sample queries this at startup and skips entirely on unsupported devices.
- The cross-GPU section additionally needs at least two GPUs in the system, with one reporting `CU_DEVICE_ATTRIBUTE_HOST_ALLOC_DMA_BUF_SUPPORTED = 1` (producer) and the other reporting `CU_DEVICE_ATTRIBUTE_DMA_BUF_SUPPORTED = 1` (consumer). On systems without a qualifying pair the cross-GPU section prints a skip message and the sample continues.
### Software
- CUDA Toolkit 13.4 or newer
- CMake 3.20 or newer
- Linux. dma-buf is a Linux kernel facility; this sample does not build on Windows or QNX.
## How to Build
See the [top-level README](../../../README.md#building-cuda-samples) for full build instructions, including how to build all samples or a single sample standalone.
## How to Run
The sample takes no arguments:
```bash
./dmabufInterop
```
All three sections run in sequence. The cross-GPU section self-skips when the running system has no qualifying GPU pair, so the same binary is appropriate for a single-GPU workstation, a multi-GPU server, or a heterogeneous-GPU platform.
## Expected Output
On a system with a single discrete GPU (the cross-GPU section self-skips):
```text
Device 0: <GPU name> (Compute Capability X.Y)
Same-process round-trip
producer: allocated 16384 bytes, exported as dma-buf fd 45
producer: wrote pattern
consumer: imported dma-buf as device pointer
consumer: verified (0 mismatches)
Cross-process IPC (fork + SCM_RIGHTS)
child: allocated 16384 bytes, exported as dma-buf fd 45
child: wrote pattern
child: sent fd to parent
parent: received dma-buf fd 45 from child
parent: imported dma-buf as device pointer
parent: verified (0 mismatches)
Cross-GPU sharing across processes
no GPU pair satisfies the dma-buf capability attributes. Skipping.
Done
```
On a system with two or more dma-buf-capable GPUs, the cross-GPU section runs to completion:
```text
Cross-GPU sharing across processes
child: on device 1, allocated 16384 bytes, exported as dma-buf fd 53
child: wrote pattern
child: sent fd to parent
parent: on device 0, received dma-buf fd 4 from child
parent: imported dma-buf as device pointer
parent: verified (0 mismatches)
```
If a verification kernel reports any mismatches, the corresponding line ends with `Error (N mismatches)` and the sample exits with a non-zero status.
## Files
- `dmabufInterop.cu` — kernels, CUDA export/import helpers, and the three demo functions. Reading this file end-to-end gives the full CUDA dma-buf flow without needing to open anything else.
- `dmabufInterop_helpers.h` — POSIX-only helpers used by the cross-process demos: `sendFd` / `recvFd` for `SCM_RIGHTS`-based file-descriptor passing, and `reapCrossProcessChild` (a `waitpid` wrapper). Split out so the main `.cu` stays focused on CUDA; open this file only if you want to see the SCM_RIGHTS mechanics.
- `README.md` — this file
- `CMakeLists.txt` — build configuration (Linux-only guard, CUDA driver + runtime link)
## See Also
- [Linux kernel dma-buf documentation](https://docs.kernel.org/driver-api/dma-buf.html)
- [CUDA External Resource Interoperability](https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__EXTRES__INTEROP.html)
- [`cuMemGetHandleForAddressRange`](https://docs.nvidia.com/cuda/cuda-driver-api/group__CUDA__MEM.html)

507
cpp/3_CUDA_Features/dmabufInterop/dmabufInterop.cu Normal file
View File

@@ -0,0 +1,507 @@
/* Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/**
* CUDA dma-buf interoperability.
*
* Three demos exercise CUDA's ability to export a host allocation as a Linux
* dma-buf file descriptor and to import a dma-buf fd back as a CUDA-mappable
* buffer: (1) same-process round-trip, (2) cross-process IPC via fork() and
* SCM_RIGHTS with both sides on one GPU, and (3) cross-GPU sharing across
* processes -- the same fork+SCM_RIGHTS path as demo 2, but the producer
* child runs on one GPU and the consumer parent imports the fd into a
* different GPU's context. Demo 3 self-skips when the system does not have
* two GPUs that satisfy the dma-buf capability attributes.
*
* Producer allocations are always pinned host memory; using a single
* allocator keeps the producer path identical across all three demos.
*
* See README.md for build instructions, expected output, and the full list
* of APIs the sample exercises.
*/
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <cuda.h>
#include <cuda_runtime.h>
#include "dmabufInterop_helpers.h"
#define NUM_ELEMENTS 4096
#define BLOCK_SIZE 256
#define PATTERN_BASE 0x1000
/* Kernel: writes ptr[i] = base + i. */
__global__ void writePatternKernel(int *ptr, int base, int count)
{
int idx = blockIdx.x * blockDim.x + threadIdx.x;
if (idx < count) {
ptr[idx] = base + idx;
}
}
/* Kernel: counts elements that do not match base + i. */
__global__ void verifyPatternKernel(int *ptr, int base, int count, int *errors)
{
int idx = blockIdx.x * blockDim.x + threadIdx.x;
if (idx < count) {
if (ptr[idx] != base + idx) {
atomicAdd(errors, 1);
}
}
}
/* Align v up to the next multiple of a. */
static size_t alignUp(size_t v, size_t a)
{
size_t r = v % a;
return (r == 0) ? v : v + (a - r);
}
/* Result of a producer-side export: the allocation base (for cuMemFreeHost),
* the page-aligned pointer usable as both host and device pointer, the
* dma-buf fd, and the aligned size. */
struct DmabufExport
{
void *allocBase;
void *alignedPtr;
int fd;
size_t size;
};
/* Allocate page-aligned pinned host memory and export it as a dma-buf fd.
* cuMemGetHandleForAddressRange requires a page-aligned base, so we allocate
* an extra page of headroom and round up. */
static DmabufExport exportHostAllocAsDmabuf(size_t size)
{
long pageSize = sysconf(_SC_PAGESIZE);
DmabufExport e = {};
e.size = alignUp(size, (size_t)pageSize);
cuMemAllocHost(&e.allocBase, e.size + (size_t)pageSize);
e.alignedPtr = (void *)alignUp((size_t)e.allocBase, (size_t)pageSize);
cuMemGetHandleForAddressRange(&e.fd,
(CUdeviceptr)e.alignedPtr,
e.size,
CU_MEM_RANGE_HANDLE_TYPE_DMA_BUF_FD,
0ULL);
return e;
}
/* Import a dma-buf fd into the current CUDA context and obtain a CUdeviceptr
* that kernels can read and write. */
static void importDmabuf(int fd, size_t size, CUexternalMemory *outExtMem, CUdeviceptr *outDevPtr)
{
CUDA_EXTERNAL_MEMORY_HANDLE_DESC memDesc = {};
memDesc.type = CU_EXTERNAL_MEMORY_HANDLE_TYPE_DMABUF_FD;
memDesc.handle.fd = fd;
memDesc.size = size;
cuImportExternalMemory(outExtMem, &memDesc);
CUDA_EXTERNAL_MEMORY_BUFFER_DESC bufDesc = {};
bufDesc.size = size;
cuExternalMemoryGetMappedBuffer(outDevPtr, *outExtMem, &bufDesc);
}
/* Same-process round-trip: one context exports a host allocation as a
* dma-buf fd and imports it back. Producer writes the pattern; consumer
* reads via the imported mapping and verifies. */
static int runSameProcess(int device)
{
cudaSetDevice(device);
DmabufExport e = exportHostAllocAsDmabuf(NUM_ELEMENTS * sizeof(int));
printf(" producer: allocated %zu bytes, exported as dma-buf fd %d\n", e.size, e.fd);
int blocks = (NUM_ELEMENTS + BLOCK_SIZE - 1) / BLOCK_SIZE;
writePatternKernel<<<blocks, BLOCK_SIZE>>>((int *)e.alignedPtr, PATTERN_BASE, NUM_ELEMENTS);
cudaDeviceSynchronize();
printf(" producer: wrote pattern\n");
CUexternalMemory extMem = NULL;
CUdeviceptr importedPtr = 0;
importDmabuf(e.fd, e.size, &extMem, &importedPtr);
printf(" consumer: imported dma-buf as device pointer\n");
int *errors = NULL;
cudaMallocHost(&errors, sizeof(int));
*errors = 0;
verifyPatternKernel<<<blocks, BLOCK_SIZE>>>(
(int *)importedPtr, PATTERN_BASE, NUM_ELEMENTS, errors);
cudaDeviceSynchronize();
int rc = (*errors == 0) ? 0 : -1;
printf(" consumer: %s (%d mismatches)\n", rc == 0 ? "verified" : "Error", *errors);
/* Free up resources. */
cudaFreeHost(errors);
cuDestroyExternalMemory(extMem);
close(e.fd);
cuMemFreeHost(e.allocBase);
return rc;
}
/* Cross-process child: producer. Allocates, exports, writes the pattern,
* sends the fd to the parent, waits for a one-byte ack so it does not free
* the underlying memory before the parent finishes reading.
*
* The child waits for a one-byte "go" from the parent before doing any
* CUDA work, so that the parent can serialise output across the same-process
* demo and the two cross-process demos. If the parent closes the socket
* without writing (because some upstream gate failed), the child exits
* cleanly. */
static int runCrossProcessChild(int device, int sock)
{
cuInit(0);
cudaSetDevice(device);
char go = 0;
if (read(sock, &go, 1) <= 0) return 0;
DmabufExport e = exportHostAllocAsDmabuf(NUM_ELEMENTS * sizeof(int));
printf(" child: allocated %zu bytes, exported as dma-buf fd %d\n", e.size, e.fd);
int blocks = (NUM_ELEMENTS + BLOCK_SIZE - 1) / BLOCK_SIZE;
writePatternKernel<<<blocks, BLOCK_SIZE>>>((int *)e.alignedPtr, PATTERN_BASE, NUM_ELEMENTS);
cudaDeviceSynchronize();
printf(" child: wrote pattern\n");
sendFd(sock, e.fd);
printf(" child: sent fd to parent\n");
char ack = 0;
[[maybe_unused]] ssize_t r = read(sock, &ack, 1);
/* Free up resources. */
close(e.fd);
cuMemFreeHost(e.allocBase);
return 0;
}
/* Cross-process parent: consumer. Signals the child to start producing,
* receives the fd, imports it, verifies the pattern, and acks the child so
* it can clean up. */
static int runCrossProcessParent(int device, int sock)
{
cudaSetDevice(device);
char go = 'G';
[[maybe_unused]] ssize_t r_go = write(sock, &go, 1);
/* fd is the dma-buf that the producer child just wrote to; it refers to
* the same underlying memory the child exported, just a different fd
* number in this process. */
int fd = recvFd(sock);
printf(" parent: received dma-buf fd %d from child\n", fd);
const size_t size = alignUp(NUM_ELEMENTS * sizeof(int), (size_t)sysconf(_SC_PAGESIZE));
CUexternalMemory extMem = NULL;
CUdeviceptr importedPtr = 0;
importDmabuf(fd, size, &extMem, &importedPtr);
printf(" parent: imported dma-buf as device pointer\n");
int *errors = NULL;
cudaMallocHost(&errors, sizeof(int));
*errors = 0;
int blocks = (NUM_ELEMENTS + BLOCK_SIZE - 1) / BLOCK_SIZE;
verifyPatternKernel<<<blocks, BLOCK_SIZE>>>(
(int *)importedPtr, PATTERN_BASE, NUM_ELEMENTS, errors);
cudaDeviceSynchronize();
int rc = (*errors == 0) ? 0 : -1;
printf(" parent: %s (%d mismatches)\n", rc == 0 ? "verified" : "Error", *errors);
char ack = 'A';
[[maybe_unused]] ssize_t r_ack = write(sock, &ack, 1);
/* Free up resources. */
cudaFreeHost(errors);
cuDestroyExternalMemory(extMem);
close(fd);
return rc;
}
/* Cross-process orchestration. The fork() must happen BEFORE either side
* calls cuInit: the CUDA driver does not survive fork-after-init, so child
* and parent each initialise CUDA independently after the fork. main()
* spawns the child here and drives the parent side later. */
static pid_t spawnCrossProcessChild(int device, int *outParentSocket)
{
int sockets[2];
if (socketpair(AF_UNIX, SOCK_STREAM, 0, sockets) != 0) {
perror("socketpair");
return -1;
}
/* Flush stdio before fork() so the child does not inherit a buffer that
* would then be flushed twice. */
fflush(stdout);
fflush(stderr);
pid_t pid = fork();
if (pid < 0) {
perror("fork");
close(sockets[0]);
close(sockets[1]);
return -1;
}
if (pid == 0) {
close(sockets[1]);
int rc = runCrossProcessChild(device, sockets[0]);
close(sockets[0]);
fflush(stdout);
fflush(stderr);
_exit(rc == 0 ? 0 : 1);
}
close(sockets[0]);
*outParentSocket = sockets[1];
return pid;
}
/* Find any pair of GPUs where one supports producing a dma-buf fd from a
* host allocation and the other supports importing a dma-buf fd. Iteration
* is device-order so the result is deterministic -- child and parent each
* call this independently after the fork and arrive at the same (producer,
* consumer) pair. Both outputs are -1 if no qualifying pair is present. */
static void findCrossGpuPair(int *outProducer, int *outConsumer)
{
*outProducer = -1;
*outConsumer = -1;
int deviceCount = 0;
cudaGetDeviceCount(&deviceCount);
if (deviceCount < 2) return;
for (int p = 0; p < deviceCount; ++p) {
int hostAllocOk = 0;
cuDeviceGetAttribute(&hostAllocOk,
CU_DEVICE_ATTRIBUTE_HOST_ALLOC_DMA_BUF_SUPPORTED, p);
if (!hostAllocOk) continue;
for (int c = 0; c < deviceCount; ++c) {
if (c == p) continue;
int importOk = 0;
cuDeviceGetAttribute(&importOk,
CU_DEVICE_ATTRIBUTE_DMA_BUF_SUPPORTED, c);
if (importOk) { *outProducer = p; *outConsumer = c; return; }
}
}
}
/* Cross-GPU cross-process child: producer on the discovered producer GPU.
* Mirrors runCrossProcessChild but runs on a specific device chosen by
* findCrossGpuPair. The parent is the arbiter for skip decisions: child
* waits for a "go" byte from the parent (or an EOF if the parent decided
* to skip the demo) before allocating anything. */
static int runCrossProcCrossGpuChild(int sock)
{
cuInit(0);
char go = 0;
if (read(sock, &go, 1) <= 0) return 0;
int producer = -1, consumer = -1;
findCrossGpuPair(&producer, &consumer);
if (producer < 0) return 0;
cudaSetDevice(producer);
DmabufExport e = exportHostAllocAsDmabuf(NUM_ELEMENTS * sizeof(int));
printf(" child: on device %d, allocated %zu bytes, exported as dma-buf fd %d\n",
producer, e.size, e.fd);
int blocks = (NUM_ELEMENTS + BLOCK_SIZE - 1) / BLOCK_SIZE;
writePatternKernel<<<blocks, BLOCK_SIZE>>>((int *)e.alignedPtr, PATTERN_BASE, NUM_ELEMENTS);
cudaDeviceSynchronize();
printf(" child: wrote pattern\n");
sendFd(sock, e.fd);
printf(" child: sent fd to parent\n");
char ack = 0;
[[maybe_unused]] ssize_t r = read(sock, &ack, 1);
/* Free up resources. */
close(e.fd);
cuMemFreeHost(e.allocBase);
return 0;
}
/* Cross-GPU cross-process parent: consumer on the discovered consumer GPU.
* Decides whether the demo proceeds: signals "go" to the child if a
* qualifying pair exists, otherwise prints a skip message and returns
* (closing the socket without writing, which the child sees as EOF). */
static int runCrossProcCrossGpuParent(int sock)
{
int producer = -1, consumer = -1;
findCrossGpuPair(&producer, &consumer);
if (producer < 0) {
printf(" no GPU pair satisfies the dma-buf capability attributes. Skipping.\n");
return 0;
}
cudaSetDevice(consumer);
char go = 'G';
[[maybe_unused]] ssize_t r_go = write(sock, &go, 1);
/* fd is the dma-buf that the producer child just wrote to on its own GPU;
* import it into this parent's context on the consumer GPU below to
* access the same underlying memory. */
int fd = recvFd(sock);
printf(" parent: on device %d, received dma-buf fd %d from child\n", consumer, fd);
const size_t size = alignUp(NUM_ELEMENTS * sizeof(int), (size_t)sysconf(_SC_PAGESIZE));
CUexternalMemory extMem = NULL;
CUdeviceptr importedPtr = 0;
importDmabuf(fd, size, &extMem, &importedPtr);
printf(" parent: imported dma-buf as device pointer\n");
int *errors = NULL;
cudaMallocHost(&errors, sizeof(int));
*errors = 0;
int blocks = (NUM_ELEMENTS + BLOCK_SIZE - 1) / BLOCK_SIZE;
verifyPatternKernel<<<blocks, BLOCK_SIZE>>>(
(int *)importedPtr, PATTERN_BASE, NUM_ELEMENTS, errors);
cudaDeviceSynchronize();
int rc = (*errors == 0) ? 0 : -1;
printf(" parent: %s (%d mismatches)\n", rc == 0 ? "verified" : "Error", *errors);
char ack = 'A';
[[maybe_unused]] ssize_t r_ack = write(sock, &ack, 1);
/* Free up resources. */
cudaFreeHost(errors);
cuDestroyExternalMemory(extMem);
close(fd);
return rc;
}
/* Cross-GPU cross-process orchestration: spawn the producer child the same
* way as spawnCrossProcessChild. Like the single-GPU IPC demo, this fork
* must happen before any cuInit in the parent process. */
static pid_t spawnCrossProcCrossGpuChild(int *outParentSocket)
{
int sockets[2];
if (socketpair(AF_UNIX, SOCK_STREAM, 0, sockets) != 0) {
perror("socketpair");
return -1;
}
fflush(stdout);
fflush(stderr);
pid_t pid = fork();
if (pid < 0) {
perror("fork");
close(sockets[0]);
close(sockets[1]);
return -1;
}
if (pid == 0) {
close(sockets[1]);
int rc = runCrossProcCrossGpuChild(sockets[0]);
close(sockets[0]);
fflush(stdout);
fflush(stderr);
_exit(rc == 0 ? 0 : 1);
}
close(sockets[0]);
*outParentSocket = sockets[1];
return pid;
}
int main()
{
/* Make stdout line-buffered so printf output from the parent and the
* fork()ed children appears in temporal order even when stdout is
* redirected to a pipe (e.g. via ssh) rather than a terminal. */
setvbuf(stdout, NULL, _IOLBF, 0);
const int device = 0;
/* Spawn both children before any cuInit in this process. Each child runs
* a producer-side role and blocks waiting for its parent to drive the
* import. fork() must precede cuInit -- the CUDA driver does not survive
* fork-after-init, so each child does its own cuInit independently. */
int sockIpc = -1, sockCrossGpu = -1;
pid_t childIpcPid = spawnCrossProcessChild(device, &sockIpc);
if (childIpcPid < 0) return 1;
pid_t childCrossGpuPid = spawnCrossProcCrossGpuChild(&sockCrossGpu);
if (childCrossGpuPid < 0) {
close(sockIpc);
reapCrossProcessChild(childIpcPid);
return 1;
}
cuInit(0);
int dmabufExportSupported = 0;
cuDeviceGetAttribute(&dmabufExportSupported,
CU_DEVICE_ATTRIBUTE_HOST_ALLOC_DMA_BUF_SUPPORTED, device);
if (!dmabufExportSupported) {
printf("Device %d does not report CU_DEVICE_ATTRIBUTE_HOST_ALLOC_DMA_BUF_SUPPORTED. "
"Skipping sample.\n", device);
close(sockIpc);
close(sockCrossGpu);
reapCrossProcessChild(childIpcPid);
reapCrossProcessChild(childCrossGpuPid);
return 0;
}
cudaDeviceProp props;
cudaGetDeviceProperties(&props, device);
printf("Device %d: %s (Compute Capability %d.%d)\n\n",
device, props.name, props.major, props.minor);
printf("Same-process round-trip\n");
int rc = runSameProcess(device);
printf("\nCross-process IPC (fork + SCM_RIGHTS)\n");
int ipcParentRc = (rc == 0) ? runCrossProcessParent(device, sockIpc) : -1;
close(sockIpc);
int ipcChildRc = reapCrossProcessChild(childIpcPid);
if (rc == 0 && (ipcParentRc != 0 || ipcChildRc != 0)) rc = -1;
printf("\nCross-GPU sharing across processes\n");
int crossGpuParentRc = (rc == 0) ? runCrossProcCrossGpuParent(sockCrossGpu) : -1;
close(sockCrossGpu);
int crossGpuChildRc = reapCrossProcessChild(childCrossGpuPid);
if (rc == 0 && (crossGpuParentRc != 0 || crossGpuChildRc != 0)) rc = -1;
if (rc == 0) printf("\nDone\n");
return rc == 0 ? 0 : 1;
}

110
cpp/3_CUDA_Features/dmabufInterop/dmabufInterop_helpers.h Normal file
View File

@@ -0,0 +1,110 @@
/* Copyright (c) 2026, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
* POSIX helpers used by the cross-process demos in dmabufInterop.cu:
* - sendFd / recvFd: transport a file descriptor between two processes
* over a UNIX domain socket via the SCM_RIGHTS control message.
* - reapCrossProcessChild: waitpid() wrapper that returns 0 iff the child
* exited cleanly with status 0.
*
* These are POSIX-only (no CUDA code) and are separated out so that
* dmabufInterop.cu stays focused on the CUDA dma-buf export/import flow.
*/
#ifndef DMABUF_INTEROP_HELPERS_H
#define DMABUF_INTEROP_HELPERS_H
#include <stdio.h>
#include <string.h>
#include <sys/socket.h>
#include <sys/types.h>
#include <sys/un.h>
#include <sys/wait.h>
#include <unistd.h>
/* Send a file descriptor over a UNIX domain socket using SCM_RIGHTS. */
static inline void sendFd(int sock, int fd)
{
struct msghdr msg = {};
char dummy = 'F';
struct iovec iov = { &dummy, 1 };
msg.msg_iov = &iov;
msg.msg_iovlen = 1;
char cbuf[CMSG_SPACE(sizeof(int))] = {};
msg.msg_control = cbuf;
msg.msg_controllen = sizeof(cbuf);
struct cmsghdr *cmsg = CMSG_FIRSTHDR(&msg);
cmsg->cmsg_level = SOL_SOCKET;
cmsg->cmsg_type = SCM_RIGHTS;
cmsg->cmsg_len = CMSG_LEN(sizeof(int));
memcpy(CMSG_DATA(cmsg), &fd, sizeof(int));
(void)sendmsg(sock, &msg, 0);
}
/* Receive a file descriptor from a UNIX domain socket. Returns -1 if no
* valid single-fd SCM_RIGHTS control message was received. */
static inline int recvFd(int sock)
{
struct msghdr msg = {};
char dummy = 0;
struct iovec iov = { &dummy, 1 };
msg.msg_iov = &iov;
msg.msg_iovlen = 1;
char cbuf[CMSG_SPACE(sizeof(int))] = {};
msg.msg_control = cbuf;
msg.msg_controllen = sizeof(cbuf);
(void)recvmsg(sock, &msg, 0);
struct cmsghdr *cmsg = CMSG_FIRSTHDR(&msg);
/* Validate that recvmsg actually delivered a single-fd SCM_RIGHTS control
* message before dereferencing -- a truncated or absent ancillary buffer
* would otherwise crash on the memcpy below. */
if (cmsg == NULL || cmsg->cmsg_len != CMSG_LEN(sizeof(int))) return -1;
int fd = -1;
memcpy(&fd, CMSG_DATA(cmsg), sizeof(int));
return fd;
}
/* Wait for a child process to exit and return 0 iff it exited cleanly with
* status 0. */
static inline int reapCrossProcessChild(pid_t pid)
{
int status = 0;
if (waitpid(pid, &status, 0) < 0) {
perror("waitpid");
return -1;
}
return (WIFEXITED(status) && WEXITSTATUS(status) == 0) ? 0 : -1;
}
#endif /* DMABUF_INTEROP_HELPERS_H */