7.0 KiB
cftUnicastPut - Unicast Point-to-Point Puts with Compute Fabric Transport
Description
This sample moves data directly between GPUs with Compute Fabric Transport (CFT) unicast logical endpoints. A logical endpoint (LE) is a fabric-addressable name for a range of device memory: a rank creates an LE, binds VMM fabric memory to it, shares it with peers through a fabric IPC handle, and peers then write into it from device code with fabric.try_put, addressing the destination by (endpoint id, offset) rather than by a mapped peer pointer.
The ranks form a ring: rank r puts its payload into rank (r + 1) % N and expects the payload of rank (r - 1 + N) % N to land in its own buffer.
Two receive-completion schemes are shown, selected on the command line:
- Default (plain puts) —
cftTryPutKernelonly moves payload. A second, one-threadcftSignalFlagKernelthen puts a 16-byte flag into the neighbor's sync region and spins on its own flag. Because the two kernels are ordered on the same stream, the flag cannot be observed before the payload puts of that rank have completed. --counted(counted puts) —cftTryPutCountedKernelissuesfabric.try_put_counted, which additionally increments a byte counter in the destination endpoint. The receiver simply waits until its own counter reaches the number of bytes it expects, so no separate signal kernel is needed.
After the transfer each rank copies its buffer back to the host and checks the rank-tagged pattern its receive neighbor sent, and rank 0 prints the combined result.
What You'll Learn
How to:
- discover the unicast logical-endpoint clique a GPU belongs to and confirm all ranks agree on it
- reserve logical-endpoint ids, create an endpoint, and share it across processes with a fabric IPC handle
- bind VMM fabric memory to an endpoint and tear the endpoint down safely
- issue
fabric.try_put/fabric.try_put_countedfrom device code and complete them with anmbarrier - decode fabric operation errors from the
mbarrier.try_waitstatus operand - pick a receive-completion scheme: an explicit inter-GPU signal flag put, or a counted put with a byte-arrival counter
Key Concepts
- Logical endpoint (LE) — a fabric-addressable handle for a bound range of device memory; remote writes target
(endpoint id, offset)instead of a mapped peer address - Clique — the set of devices that can address each other's logical endpoints; every participating rank must find the same
CU_CLIQUE_TYPE_UNICAST_LOGICAL_ENDPOINTclique id - Fabric put — an asynchronous device-side write to a peer endpoint issued by a single thread, completed through an
mbarrierwithcomplete_tx::16Baccounting - Counted put — a fabric put that also accumulates delivered bytes into a 256-byte-aligned counter in the destination endpoint, giving the receiver a completion signal without an extra message
- Fabric error reporting — when the
mbarrier.try_waitreport predicate is set,cudaFabricOpErrorStatus*decodes failures from thetry_waitstatus operand.
Key APIs
CUDA Driver API
cuDeviceGetCliqueCount/cuDeviceGetCliqueInfo— find the unicast logical-endpoint clique of a devicecuLogicalEndpointIdReserve/cuLogicalEndpointIdRelease— reserve a contiguous range of local endpoint idscuLogicalEndpointCreate/cuLogicalEndpointDestroy— create and destroy an endpointcuLogicalEndpointGetLimits— query the bind alignment and the maximum endpoint sizecuLogicalEndpointExport/cuLogicalEndpointImport— share an endpoint through a fabric IPC handlecuLogicalEndpointQuery— wait until a created or imported endpoint is ready to usecuLogicalEndpointBindMem/cuLogicalEndpointUnbind— attach and detach the backing allocationcuMemCreate,cuMemAddressReserve,cuMemMap,cuMemSetAccess— allocate the fabric-shareable backing memory
Device-side PTX wrappers (cuda::ptx)
fabric_try_put/fabric_try_put_counted— issue the remote write, optionally with byte countingfabric_submit— submit the batch of issued fabric operationsmbarrier_init(layout_v1),mbarrier_arrive_expect_tx,mbarrier_try_wait_parity— track completion of the puts
CUDA Runtime API
cudaFabricOpErrorStatusCount/cudaFabricOpErrorStatusGet— decode fabric errors from thembarrier.try_waitstatus field (cudaFabricOpStatusSourceMbarrierV1)
Requirements
Hardware
- Two or more NVIDIA GPUs of compute capability 10.0 or newer.
- Two or more NVLink-capable datacenter class NVIDIA GPUs of compute capability 10.0 or newer (
sm_100/sm_103/sm_107). - Each rank needs its own device, so a node must expose at least as many GPUs as it runs ranks.
Software
- CUDA Toolkit 13.4 or newer and a driver with
cuLogicalEndpoint*support - The NVIDIA IMEX daemon with IMEX channels configured, as required for any
CU_MEM_HANDLE_TYPE_FABRICsharing - MPI
- Linux on x86_64 or aarch64
How to Build
See the top-level README for full build instructions, including how to build all samples or a single sample standalone.
How to Run
The sample must be launched under MPI with at least two ranks:
mpirun -np 2 ./cftUnicastPut [chunkCnt] [--counted]
[chunkCnt] is optional (default: 1000) and is the number of 8192-byte chunks each rank sends; it also becomes the grid size (one block per chunk). --counted switches from plain puts plus a flag handshake to counted puts plus a byte-arrival counter.
mpirun -np 2 ./cftUnicastPut # defaults: 1000 chunks, plain puts
mpirun -np 2 ./cftUnicastPut --counted # defaults: 1000 chunks, counted puts
mpirun -np 2 ./cftUnicastPut 1000 # plain puts + inter-GPU flag
mpirun -np 2 ./cftUnicastPut 1000 --counted # counted puts + byte counter
Debug builds (-DENABLE_CUDA_DEBUG=ON, or any build without NDEBUG) cap every device-side spin-wait at 30 seconds and print which wait timed out, so a stalled fabric operation reports itself instead of hanging.
Expected Output
mode = put, chunkCnt = 1000, totalBytes = 8192000, ranks = 2
Result = PASS
If any rank's received payload does not match what its neighbor sent, that rank prints the first mismatching element to stderr, rank 0 prints Result = FAIL, and the sample exits with a non-zero status.
Files
cft_unicast_put.cu— the complete sample: endpoint setup and teardown, both put kernels, the flag-signal kernel, and host-side verificationREADME.md— this fileCMakeLists.txt— build configuration