NASP/cuda-samples
84
0
Fork 0
mirror of https://github.com/NVIDIA/cuda-samples.git synced 2026-10-13 00:08:26 +08:00
Code Issues Packages Projects Releases Wiki Activity

CUDA 13.2 samples update (#432)

Browse Source 
- Added Python samples for CUDA Python 1.0 release
- Renamed top-level `Samples` directory to `cpp` to accommodate Python samples.
This commit is contained in:
Dheemanth
2026-05-13 15:13:18 -07:00
committed by GitHub
parent a4526d5229
commit aeab82ff30
2072 changed files with 15077 additions and 1970 deletions
Download Patch File Download Diff File Expand all files Collapse all files

46
cpp/0_Introduction/CMakeLists.txt Normal file
View File

@@ -0,0 +1,46 @@
add_subdirectory(UnifiedMemoryStreams)
add_subdirectory(asyncAPI)
add_subdirectory(clock)
add_subdirectory(clock_nvrtc)
add_subdirectory(cudaOpenMP)
add_subdirectory(fp16ScalarProduct)
add_subdirectory(matrixMul)
add_subdirectory(matrixMulDrv)
add_subdirectory(matrixMulDynlinkJIT)
add_subdirectory(matrixMul_nvrtc)
add_subdirectory(mergeSort)
add_subdirectory(simpleAWBarrier)
add_subdirectory(simpleAssert)
add_subdirectory(simpleAssert_nvrtc)
add_subdirectory(simpleAtomicIntrinsics)
add_subdirectory(simpleAtomicIntrinsics_nvrtc)
add_subdirectory(simpleAttributes)
add_subdirectory(simpleCUDA2GL)
add_subdirectory(simpleCallback)
add_subdirectory(simpleCooperativeGroups)
add_subdirectory(simpleCubemapTexture)
add_subdirectory(simpleDrvRuntime)
add_subdirectory(simpleHyperQ)
add_subdirectory(simpleIPC)
add_subdirectory(simpleLayeredTexture)
add_subdirectory(simpleMPI)
add_subdirectory(simpleMultiCopy)
add_subdirectory(simpleMultiGPU)
add_subdirectory(simpleOccupancy)
add_subdirectory(simpleP2P)
add_subdirectory(simplePitchLinearTexture)
add_subdirectory(simplePrintf)
add_subdirectory(simpleStreams)
add_subdirectory(simpleSurfaceWrite)
add_subdirectory(simpleTemplates)
add_subdirectory(simpleTexture)
add_subdirectory(simpleTexture3D)
add_subdirectory(simpleTextureDrv)
add_subdirectory(simpleVoteIntrinsics)
add_subdirectory(simpleZeroCopy)
add_subdirectory(template)
add_subdirectory(systemWideAtomics)
add_subdirectory(vectorAdd)
add_subdirectory(vectorAddDrv)
add_subdirectory(vectorAddMMAP)
add_subdirectory(vectorAdd_nvrtc)

140
cpp/0_Introduction/README.md Normal file
View File

@@ -0,0 +1,140 @@
# 0. Introduction
### [asyncAPI](./asyncAPI)
This sample illustrates the usage of CUDA events for both GPU timing and overlapping CPU and GPU execution. Events are inserted into a stream of CUDA calls. Since CUDA stream calls are asynchronous, the CPU can perform computations while GPU is executing (including DMA memcopies between the host and device). CPU can query CUDA events to determine whether GPU has completed tasks.
### [clock](./clock)
This example shows how to use the clock function to measure the performance of block of threads of a kernel accurately.
### [clock_nvrtc](./clock_nvrtc)
This example shows how to use the clock function using libNVRTC to measure the performance of block of threads of a kernel accurately.
### [cudaOpenMP](./cudaOpenMP)
This sample demonstrates how to use OpenMP API to write an application for multiple GPUs.
### [fp16ScalarProduct](./fp16ScalarProduct)
Calculates scalar product of two vectors of FP16 numbers.
### [matrixMul](./matrixMul)
This sample implements matrix multiplication and is exactly the same as Chapter 6 of the programming guide. It has been written for clarity of exposition to illustrate various CUDA programming principles, not with the goal of providing the most performant generic kernel for matrix multiplication. To illustrate GPU performance for matrix multiply, this sample also shows how to use the new CUDA 4.0 interface for CUBLAS to demonstrate high-performance performance for matrix multiplication.
### [matrixMul_nvrtc](./matrixMul_nvrtc)
This sample implements matrix multiplication and is exactly the same as Chapter 6 of the programming guide. It has been written for clarity of exposition to illustrate various CUDA programming principles, not with the goal of providing the most performant generic kernel for matrix multiplication. To illustrate GPU performance for matrix multiply, this sample also shows how to use the new CUDA 4.0 interface for CUBLAS to demonstrate high-performance performance for matrix multiplication.
### [matrixMulDrv](./matrixMulDrv)
This sample implements matrix multiplication and uses the new CUDA 4.0 kernel launch Driver API. It has been written for clarity of exposition to illustrate various CUDA programming principles, not with the goal of providing the most performant generic kernel for matrix multiplication. CUBLAS provides high-performance matrix multiplication.
### [matrixMulDynlinkJIT](./matrixMulDynlinkJIT)
This sample revisits matrix multiplication using the CUDA driver API. It demonstrates how to link to CUDA driver at runtime and how to use JIT (just-in-time) compilation from PTX code. It has been written for clarity of exposition to illustrate various CUDA programming principles, not with the goal of providing the most performant generic kernel for matrix multiplication. CUBLAS provides high-performance matrix multiplication.
### [mergeSort](./mergeSort)
This sample implements a merge sort (also known as Batcher's sort), algorithms belonging to the class of sorting networks. While generally subefficient on large sequences compared to algorithms with better asymptotic algorithmic complexity (i.e. merge sort or radix sort), may be the algorithms of choice for sorting batches of short- to mid-sized (key, value) array pairs. Refer to the excellent tutorial by H. W. Lang http://www.iti.fh-flensburg.de/lang/algorithmen/sortieren/networks/indexen.htm
### [simpleAssert](./simpleAssert)
This CUDA Runtime API sample is a very basic sample that implements how to use the assert function in the device code. Requires Compute Capability 2.0 .
### [simpleAssert_nvrtc](./simpleAssert_nvrtc)
This CUDA Runtime API sample is a very basic sample that implements how to use the assert function in the device code. Requires Compute Capability 2.0 .
### [simpleAtomicIntrinsics](./simpleAtomicIntrinsics)
A simple demonstration of global memory atomic instructions.
### [simpleAtomicIntrinsics_nvrtc](./simpleAtomicIntrinsics_nvrtc)
A simple demonstration of global memory atomic instructions.This sample makes use of NVRTC for Runtime Compilation.
### [simpleAttributes](./simpleAttributes)
This CUDA Runtime API sample is a very basic example that implements how to use the stream attributes that affect L2 locality. Performance improvement due to use of L2 access policy window can only be noticed on Compute capability 8.0 or higher.
### [simpleAWBarrier](./simpleAWBarrier)
A simple demonstration of arrive wait barriers.
### [simpleCallback](./simpleCallback)
This sample implements multi-threaded heterogeneous computing workloads with the new CPU callbacks for CUDA streams and events introduced with CUDA 5.0.
### [simpleCooperativeGroups](./simpleCooperativeGroups)
This sample is a simple code that illustrates basic usage of cooperative groups within the thread block.
### [simpleCubemapTexture](./simpleCubemapTexture)
Simple example that demonstrates how to use a new CUDA 4.1 feature to support cubemap Textures in CUDA C.
### [simpleCUDA2GL](./simpleCUDA2GL)
This sample shows how to copy CUDA image back to OpenGL using the most efficient methods.
### [simpleDrvRuntime](./simpleDrvRuntime)
A simple example which demonstrates how CUDA Driver and Runtime APIs can work together to load cuda fatbinary of vector add kernel and performing vector addition.
### [simpleHyperQ](./simpleHyperQ)
This sample demonstrates the use of CUDA streams for concurrent execution of several kernels on devices which provide HyperQ (SM 3.5). Devices without HyperQ (SM 2.0 and SM 3.0) will run a maximum of two kernels concurrently.
### [simpleIPC](./simpleIPC)
This CUDA Runtime API sample is a very basic sample that demonstrates Inter Process Communication with one process per GPU for computation. Requires Compute Capability 3.0 or higher and a Linux Operating System, or a Windows Operating System with TCC enabled GPUs
### [simpleLayeredTexture](./simpleLayeredTexture)
Simple example that demonstrates how to use a new CUDA 4.0 feature to support layered Textures in CUDA C.
### [simpleMPI](./simpleMPI)
Simple example demonstrating how to use MPI in combination with CUDA.
### [simpleMultiCopy](./simpleMultiCopy)
Supported in GPUs with Compute Capability 1.1, overlapping compute with one memcopy is possible from the host system. For Quadro and Tesla GPUs with Compute Capability 2.0, a second overlapped copy operation in either direction at full speed is possible (PCI-e is symmetric). This sample illustrates the usage of CUDA streams to achieve overlapping of kernel execution with data copies to and from the device.
### [simpleMultiGPU](./simpleMultiGPU)
This application demonstrates how to use the new CUDA 4.0 API for CUDA context management and multi-threaded access to run CUDA kernels on multiple-GPUs.
### [simpleOccupancy](./simpleOccupancy)
This sample demonstrates the basic usage of the CUDA occupancy calculator and occupancy-based launch configurator APIs by launching a kernel with the launch configurator, and measures the utilization difference against a manually configured launch.
### [simpleP2P](./simpleP2P)
This application demonstrates CUDA APIs that support Peer-To-Peer (P2P) copies, Peer-To-Peer (P2P) addressing, and Unified Virtual Memory Addressing (UVA) between multiple GPUs. In general, P2P is supported between two same GPUs with some exceptions, such as some Tesla and Quadro GPUs.
### [simplePitchLinearTexture](./simplePitchLinearTexture)
Use of Pitch Linear Textures
### [simplePrintf](./simplePrintf)
This basic CUDA Runtime API sample demonstrates how to use the printf function in the device code.
### [simpleStreams](./simpleStreams)
This sample uses CUDA streams to overlap kernel executions with memory copies between the host and a GPU device. This sample uses a new CUDA 4.0 feature that supports pinning of generic host memory. Requires Compute Capability 2.0 or higher.
### [simpleSurfaceWrite](./simpleSurfaceWrite)
Simple example that demonstrates the use of 2D surface references (Write-to-Texture)
### [simpleTemplates](./simpleTemplates)
This sample is a templatized version of the template project. It also shows how to correctly templatize dynamically allocated shared memory arrays.
### [simpleTexture](./simpleTexture)
Simple example that demonstrates use of Textures in CUDA.
### [simpleTexture3D](./simpleTexture3D)
Simple example that demonstrates use of 3D Textures in CUDA.
### [simpleTextureDrv](./simpleTextureDrv)
Simple example that demonstrates use of Textures in CUDA. This sample uses the new CUDA 4.0 kernel launch Driver API.
### [simpleVoteIntrinsics](./simpleVoteIntrinsics)
Simple program which demonstrates how to use the Vote (__any_sync, __all_sync) intrinsic instruction in a CUDA kernel.
### [simpleZeroCopy](./simpleZeroCopy)
This sample illustrates how to use Zero MemCopy, kernels can read and write directly to pinned system memory.
### [systemWideAtomics](./systemWideAtomics)
A simple demonstration of system wide atomic instructions.
### [template](./template)
A trivial template project that can be used as a starting point to create new CUDA projects.
### [UnifiedMemoryStreams](./UnifiedMemoryStreams)
This sample demonstrates the use of OpenMP and streams with Unified Memory on a single GPU.
### [vectorAdd](./vectorAdd)
This CUDA Runtime API sample is a very basic sample that implements element by element vector addition. It is the same as the sample illustrating Chapter 3 of the programming guide with some additions like error checking.
### [vectorAdd_nvrtc](./vectorAdd_nvrtc)
This CUDA Driver API sample uses NVRTC for runtime compilation of vector addition kernel. Vector addition kernel demonstrated is the same as the sample illustrating Chapter 3 of the programming guide.
### [vectorAddDrv](./vectorAddDrv)
This Vector Addition sample is a basic sample that is implemented element by element. It is the same as the sample illustrating Chapter 3 of the programming guide with some additions like error checking. This sample also uses the new CUDA 4.0 kernel launch Driver API.
### [vectorAddMMAP](./vectorAddMMAP)
This sample replaces the device allocation in the vectorAddDrv sample with cuMemMap-ed allocations. This sample demonstrates that the cuMemMap api allows the user to specify the physical properties of their memory while retaining the contiguous nature of their access, thus not requiring a change in their program structure.

18
cpp/0_Introduction/UnifiedMemoryStreams/.vscode/c_cpp_properties.json vendored Normal file
View File

@@ -0,0 +1,18 @@
{
"configurations": [
{
"name": "Linux",
"includePath": [
"${workspaceFolder}/**",
"${workspaceFolder}/../../../Common"
],
"defines": [],
"compilerPath": "/usr/local/cuda/bin/nvcc",
"cStandard": "gnu17",
"cppStandard": "gnu++14",
"intelliSenseMode": "linux-gcc-x64",
"configurationProvider": "ms-vscode.makefile-tools"
}
],
"version": 4
}

7
cpp/0_Introduction/UnifiedMemoryStreams/.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,7 @@
{
"recommendations": [
"nvidia.nsight-vscode-edition",
"ms-vscode.cpptools",
"ms-vscode.makefile-tools"
]
}

53
cpp/0_Introduction/UnifiedMemoryStreams/CMakeLists.txt Normal file
View File

@@ -0,0 +1,53 @@
cmake_minimum_required(VERSION 3.20)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/Modules")
project(UnifiedMemoryStreams LANGUAGES C CXX CUDA)
find_package(CUDAToolkit REQUIRED)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(CMAKE_CUDA_ARCHITECTURES 75 80 86 87 89 90 100 110 120)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -Wno-deprecated-gpu-targets")
if(ENABLE_CUDA_DEBUG)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -G") # enable cuda-gdb (may significantly affect performance on some targets)
else()
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -lineinfo") # add line information to all builds for debug tools (exclusive to -G option)
endif()
# Include directories and libraries
include_directories(../../../Common)
# This sample is not supported on QNX
if(CMAKE_SYSTEM_NAME STREQUAL "QNX")
message(STATUS "Will not build sample ${PROJECT_NAME} - not supported on QNX")
return()
endif()
# FindOpenMP: request COMPONENTS CXX only. From CMake 3.31 on, a bare find_package(OpenMP) also
# probes OpenMP for CUDA when the project enables CUDA; that check can fail even when C++ OpenMP
# works, and this sample only needs the C++ OpenMP package. Link OpenMP::OpenMP_CXX for libs/headers.
find_package(OpenMP COMPONENTS CXX)
if(OpenMP_CXX_FOUND)
# Add target for UnifiedMemoryStreams
add_executable(UnifiedMemoryStreams UnifiedMemoryStreams.cu)
target_compile_options(UnifiedMemoryStreams PRIVATE $<$<COMPILE_LANGUAGE:CUDA>:--extended-lambda>)
target_compile_features(UnifiedMemoryStreams PRIVATE cxx_std_17 cuda_std_17)
set_target_properties(UnifiedMemoryStreams PROPERTIES CUDA_SEPARABLE_COMPILATION ON)
target_link_libraries(UnifiedMemoryStreams PUBLIC
CUDA::cublas
OpenMP::OpenMP_CXX
)
else()
message(STATUS "OpenMP not found - will not build sample 'UnifiedMemoryStreams'")
endif()
# Include installation configuration
include(${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/InstallSamples.cmake)
setup_samples_install()

34
cpp/0_Introduction/UnifiedMemoryStreams/README.md Normal file
View File

@@ -0,0 +1,34 @@
# UnifiedMemoryStreams - Unified Memory Streams
## Description
This sample demonstrates the use of OpenMP and streams with Unified Memory on a single GPU.
## Key Concepts
CUDA Systems Integration, OpenMP, CUBLAS, Multithreading, Unified Memory, CUDA Streams and Events
## Supported SM Architectures
## Supported OSes
Linux, Windows
## Supported CPU Architecture
x86_64, armv7l
## CUDA APIs involved
### [CUDA Runtime API](http://docs.nvidia.com/cuda/cuda-runtime-api/index.html)
cudaStreamDestroy, cudaFree, cudaMallocManaged, cudaStreamAttachMemAsync, cudaSetDevice, cudaDeviceSynchronize, cudaStreamSynchronize, cudaStreamCreate, cudaGetDeviceProperties
## Dependencies needed to build/run
[OpenMP](../../../README.md#openmp), [UVM](../../../README.md#unified-virtual-memory), [CUBLAS](../../../README.md#cublas)
## Prerequisites
Download and install the [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) for your corresponding platform.
Make sure the dependencies mentioned in [Dependencies]() section above are installed.
## References (for more details)

339
cpp/0_Introduction/UnifiedMemoryStreams/UnifiedMemoryStreams.cu Normal file
View File

@@ -0,0 +1,339 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
* This sample implements a simple task consumer using threads and streams
* with all data in Unified Memory, and tasks consumed by both host and device
*/
// system includes
#include <algorithm>
#include <cstdio>
#include <ctime>
#include <vector>
#ifdef USE_PTHREADS
#include <pthread.h>
#else
#include <omp.h>
#endif
#include <stdlib.h>
// cuBLAS
#include <cublas_v2.h>
// utilities
#include <helper_cuda.h>
#if defined(WIN32) || defined(_WIN32) || defined(WIN64) || defined(_WIN64)
// SRAND48 and DRAND48 don't exist on windows, but these are the equivalent
// functions
void srand48(long seed) { srand((unsigned int)seed); }
double drand48() { return double(rand()) / RAND_MAX; }
#endif
const char *sSDKname = "UnifiedMemoryStreams";
// simple task
template <typename T> struct Task
{
unsigned int size, id;
T *data;
T *result;
T *vector;
Task()
: size(0)
, id(0)
, data(NULL)
, result(NULL)
, vector(NULL) {};
Task(unsigned int s)
: size(s)
, id(0)
, data(NULL)
, result(NULL)
{
// allocate unified memory -- the operation performed in this example will
// be a DGEMV
checkCudaErrors(cudaMallocManaged(&data, sizeof(T) * size * size));
checkCudaErrors(cudaMallocManaged(&result, sizeof(T) * size));
checkCudaErrors(cudaMallocManaged(&vector, sizeof(T) * size));
checkCudaErrors(cudaDeviceSynchronize());
}
~Task()
{
// ensure all memory is deallocated
checkCudaErrors(cudaDeviceSynchronize());
checkCudaErrors(cudaFree(data));
checkCudaErrors(cudaFree(result));
checkCudaErrors(cudaFree(vector));
}
void allocate(const unsigned int s, const unsigned int unique_id)
{
// allocate unified memory outside of constructor
id = unique_id;
size = s;
checkCudaErrors(cudaMallocManaged(&data, sizeof(T) * size * size));
checkCudaErrors(cudaMallocManaged(&result, sizeof(T) * size));
checkCudaErrors(cudaMallocManaged(&vector, sizeof(T) * size));
checkCudaErrors(cudaDeviceSynchronize());
// populate data with random elements
for (unsigned int i = 0; i < size * size; i++) {
data[i] = drand48();
}
for (unsigned int i = 0; i < size; i++) {
result[i] = 0.;
vector[i] = drand48();
}
}
};
#ifdef USE_PTHREADS
struct threadData_t
{
int tid;
Task<double> *TaskListPtr;
cudaStream_t *streams;
cublasHandle_t *handles;
int taskSize;
};
typedef struct threadData_t threadData;
#endif
// simple host dgemv: assume data is in row-major format and square
template <typename T> void gemv(int m, int n, T alpha, T *A, T *x, T beta, T *result)
{
// rows
for (int i = 0; i < n; i++) {
result[i] *= beta;
for (int j = 0; j < n; j++) {
result[i] += A[i * n + j] * x[j];
}
}
}
// execute a single task on either host or device depending on size
#ifdef USE_PTHREADS
void *execute(void *inpArgs)
{
threadData *dataPtr = (threadData *)inpArgs;
cudaStream_t *stream = dataPtr->streams;
cublasHandle_t *handle = dataPtr->handles;
int tid = dataPtr->tid;
for (int i = 0; i < dataPtr->taskSize; i++) {
Task<double> &t = dataPtr->TaskListPtr[i];
if (t.size < 100) {
// perform on host
printf("Task [%d], thread [%d] executing on host (%d)\n", t.id, tid, t.size);
// attach managed memory to a (dummy) stream to allow host access while
// the device is running
checkCudaErrors(cudaStreamAttachMemAsync(stream[0], t.data, 0, cudaMemAttachHost));
checkCudaErrors(cudaStreamAttachMemAsync(stream[0], t.vector, 0, cudaMemAttachHost));
checkCudaErrors(cudaStreamAttachMemAsync(stream[0], t.result, 0, cudaMemAttachHost));
// necessary to ensure Async cudaStreamAttachMemAsync calls have finished
checkCudaErrors(cudaStreamSynchronize(stream[0]));
// call the host operation
gemv(t.size, t.size, 1.0, t.data, t.vector, 0.0, t.result);
}
else {
// perform on device
printf("Task [%d], thread [%d] executing on device (%d)\n", t.id, tid, t.size);
double one = 1.0;
double zero = 0.0;
// attach managed memory to my stream
checkCudaErrors(cublasSetStream(handle[tid + 1], stream[tid + 1]));
checkCudaErrors(cudaStreamAttachMemAsync(stream[tid + 1], t.data, 0, cudaMemAttachSingle));
checkCudaErrors(cudaStreamAttachMemAsync(stream[tid + 1], t.vector, 0, cudaMemAttachSingle));
checkCudaErrors(cudaStreamAttachMemAsync(stream[tid + 1], t.result, 0, cudaMemAttachSingle));
// call the device operation
checkCudaErrors(cublasDgemv(
handle[tid + 1], CUBLAS_OP_N, t.size, t.size, &one, t.data, t.size, t.vector, 1, &zero, t.result, 1));
}
}
pthread_exit(NULL);
}
#else
template <typename T> void execute(Task<T> &t, cublasHandle_t *handle, cudaStream_t *stream, int tid)
{
if (t.size < 100) {
// perform on host
printf("Task [%d], thread [%d] executing on host (%d)\n", t.id, tid, t.size);
// attach managed memory to a (dummy) stream to allow host access while the
// device is running
checkCudaErrors(cudaStreamAttachMemAsync(stream[0], t.data, 0, cudaMemAttachHost));
checkCudaErrors(cudaStreamAttachMemAsync(stream[0], t.vector, 0, cudaMemAttachHost));
checkCudaErrors(cudaStreamAttachMemAsync(stream[0], t.result, 0, cudaMemAttachHost));
// necessary to ensure Async cudaStreamAttachMemAsync calls have finished
checkCudaErrors(cudaStreamSynchronize(stream[0]));
// call the host operation
gemv(t.size, t.size, 1.0, t.data, t.vector, 0.0, t.result);
}
else {
// perform on device
printf("Task [%d], thread [%d] executing on device (%d)\n", t.id, tid, t.size);
double one = 1.0;
double zero = 0.0;
// attach managed memory to my stream
checkCudaErrors(cublasSetStream(handle[tid + 1], stream[tid + 1]));
checkCudaErrors(cudaStreamAttachMemAsync(stream[tid + 1], t.data, 0, cudaMemAttachSingle));
checkCudaErrors(cudaStreamAttachMemAsync(stream[tid + 1], t.vector, 0, cudaMemAttachSingle));
checkCudaErrors(cudaStreamAttachMemAsync(stream[tid + 1], t.result, 0, cudaMemAttachSingle));
// call the device operation
checkCudaErrors(cublasDgemv(
handle[tid + 1], CUBLAS_OP_N, t.size, t.size, &one, t.data, t.size, t.vector, 1, &zero, t.result, 1));
}
}
#endif
// populate a list of tasks with random sizes
template <typename T> void initialise_tasks(std::vector<Task<T>> &TaskList)
{
for (unsigned int i = 0; i < TaskList.size(); i++) {
// generate random size
int size;
size = std::max((int)(drand48() * 1000.0), 64);
TaskList[i].allocate(size, i);
}
}
int main(int argc, char **argv)
{
// set device
cudaDeviceProp device_prop;
int dev_id = findCudaDevice(argc, (const char **)argv);
checkCudaErrors(cudaGetDeviceProperties(&device_prop, dev_id));
if (!device_prop.managedMemory) {
// This samples requires being run on a device that supports Unified Memory
fprintf(stderr, "Unified Memory not supported on this device\n");
exit(EXIT_WAIVED);
}
int computeMode;
checkCudaErrors(cudaDeviceGetAttribute(&computeMode, cudaDevAttrComputeMode, dev_id));
if (computeMode == cudaComputeModeProhibited) {
// This sample requires being run with a default or process exclusive mode
fprintf(stderr,
"This sample requires a device in either default or process "
"exclusive mode\n");
exit(EXIT_WAIVED);
}
// randomise task sizes
int seed = (int)time(NULL);
srand48(seed);
// set number of threads
const int nthreads = 4;
// number of streams = number of threads
cudaStream_t *streams = new cudaStream_t[nthreads + 1];
cublasHandle_t *handles = new cublasHandle_t[nthreads + 1];
for (int i = 0; i < nthreads + 1; i++) {
checkCudaErrors(cudaStreamCreate(&streams[i]));
checkCudaErrors(cublasCreate(&handles[i]));
}
// create list of N tasks
unsigned int N = 40;
std::vector<Task<double>> TaskList(N);
initialise_tasks(TaskList);
printf("Executing tasks on host / device\n");
// run through all tasks using threads and streams
#ifdef USE_PTHREADS
pthread_t threads[nthreads];
threadData *InputToThreads = new threadData[nthreads];
for (int i = 0; i < nthreads; i++) {
checkCudaErrors(cudaSetDevice(dev_id));
InputToThreads[i].tid = i;
InputToThreads[i].streams = streams;
InputToThreads[i].handles = handles;
if ((TaskList.size() / nthreads) == 0) {
InputToThreads[i].taskSize = (TaskList.size() / nthreads);
InputToThreads[i].TaskListPtr = &TaskList[i * (TaskList.size() / nthreads)];
}
else {
if (i == nthreads - 1) {
InputToThreads[i].taskSize = (TaskList.size() / nthreads) + (TaskList.size() % nthreads);
InputToThreads[i].TaskListPtr =
&TaskList[i * (TaskList.size() / nthreads) + (TaskList.size() % nthreads)];
}
else {
InputToThreads[i].taskSize = (TaskList.size() / nthreads);
InputToThreads[i].TaskListPtr = &TaskList[i * (TaskList.size() / nthreads)];
}
}
pthread_create(&threads[i], NULL, &execute, &InputToThreads[i]);
}
for (int i = 0; i < nthreads; i++) {
pthread_join(threads[i], NULL);
}
#else
omp_set_num_threads(nthreads);
#pragma omp parallel for schedule(dynamic)
for (int i = 0; i < TaskList.size(); i++) {
checkCudaErrors(cudaSetDevice(dev_id));
int tid = omp_get_thread_num();
execute(TaskList[i], handles, streams, tid);
}
#endif
cudaDeviceSynchronize();
// Destroy CUDA Streams, cuBlas handles
for (int i = 0; i < nthreads + 1; i++) {
cudaStreamDestroy(streams[i]);
cublasDestroy(handles[i]);
}
// Free TaskList
std::vector<Task<double>>().swap(TaskList);
printf("All Done!\n");
exit(EXIT_SUCCESS);
}

18
cpp/0_Introduction/asyncAPI/.vscode/c_cpp_properties.json vendored Normal file
View File

@@ -0,0 +1,18 @@
{
"configurations": [
{
"name": "Linux",
"includePath": [
"${workspaceFolder}/**",
"${workspaceFolder}/../../../Common"
],
"defines": [],
"compilerPath": "/usr/local/cuda/bin/nvcc",
"cStandard": "gnu17",
"cppStandard": "gnu++14",
"intelliSenseMode": "linux-gcc-x64",
"configurationProvider": "ms-vscode.makefile-tools"
}
],
"version": 4
}

7
cpp/0_Introduction/asyncAPI/.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,7 @@
{
"recommendations": [
"nvidia.nsight-vscode-edition",
"ms-vscode.cpptools",
"ms-vscode.makefile-tools"
]
}

34
cpp/0_Introduction/asyncAPI/CMakeLists.txt Normal file
View File

@@ -0,0 +1,34 @@
cmake_minimum_required(VERSION 3.20)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/Modules")
project(asyncAPI LANGUAGES C CXX CUDA)
find_package(CUDAToolkit REQUIRED)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(CMAKE_CUDA_ARCHITECTURES 75 80 86 87 89 90 100 110 120)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -Wno-deprecated-gpu-targets")
if(ENABLE_CUDA_DEBUG)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -G") # enable cuda-gdb (may significantly affect performance on some targets)
else()
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -lineinfo") # add line information to all builds for debug tools (exclusive to -G option)
endif()
# Include directories and libraries
include_directories(../../../Common)
# Source file
# Add target for asyncAPI
add_executable(asyncAPI asyncAPI.cu)
target_compile_options(asyncAPI PRIVATE $<$<COMPILE_LANGUAGE:CUDA>:--extended-lambda>)
target_compile_features(asyncAPI PRIVATE cxx_std_17 cuda_std_17)
set_target_properties(asyncAPI PROPERTIES CUDA_SEPARABLE_COMPILATION ON)
# Include installation configuration
include(${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/InstallSamples.cmake)
setup_samples_install()

32
cpp/0_Introduction/asyncAPI/README.md Normal file
View File

@@ -0,0 +1,32 @@
# asyncAPI - asyncAPI
## Description
This sample illustrates the usage of CUDA events for both GPU timing and overlapping CPU and GPU execution. Events are inserted into a stream of CUDA calls. Since CUDA stream calls are asynchronous, the CPU can perform computations while GPU is executing (including DMA memcopies between the host and device). CPU can query CUDA events to determine whether GPU has completed tasks.
## Key Concepts
Asynchronous Data Transfers, CUDA Streams and Events
## Supported SM Architectures
[SM 5.0 ](https://developer.nvidia.com/cuda-gpus) [SM 5.2 ](https://developer.nvidia.com/cuda-gpus) [SM 5.3 ](https://developer.nvidia.com/cuda-gpus) [SM 6.0 ](https://developer.nvidia.com/cuda-gpus) [SM 6.1 ](https://developer.nvidia.com/cuda-gpus) [SM 7.0 ](https://developer.nvidia.com/cuda-gpus) [SM 7.2 ](https://developer.nvidia.com/cuda-gpus) [SM 7.5 ](https://developer.nvidia.com/cuda-gpus) [SM 8.0 ](https://developer.nvidia.com/cuda-gpus) [SM 8.6 ](https://developer.nvidia.com/cuda-gpus) [SM 8.7 ](https://developer.nvidia.com/cuda-gpus) [SM 8.9 ](https://developer.nvidia.com/cuda-gpus) [SM 9.0 ](https://developer.nvidia.com/cuda-gpus)
## Supported OSes
Linux, Windows
## Supported CPU Architecture
x86_64, armv7l
## CUDA APIs involved
### [CUDA Runtime API](http://docs.nvidia.com/cuda/cuda-runtime-api/index.html)
cudaProfilerStop, cudaMalloc, cudaMemcpyAsync, cudaFree, cudaMallocHost, cudaProfilerStart, cudaDeviceSynchronize, cudaEventRecord, cudaFreeHost, cudaMemset, cudaEventDestroy, cudaEventQuery, cudaEventElapsedTime, cudaGetDeviceProperties, cudaEventCreate
## Prerequisites
Download and install the [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) for your corresponding platform.
## References (for more details)

144
cpp/0_Introduction/asyncAPI/asyncAPI.cu Normal file
View File

@@ -0,0 +1,144 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
* This sample illustrates the usage of CUDA events for both GPU timing and
* overlapping CPU and GPU execution. Events are inserted into a stream
* of CUDA calls. Since CUDA stream calls are asynchronous, the CPU can
* perform computations while GPU is executing (including DMA memcopies
* between the host and device). CPU can query CUDA events to determine
* whether GPU has completed tasks.
*/
// includes, system
#include <stdio.h>
// includes CUDA Runtime
#include <cuda_profiler_api.h>
#include <cuda_runtime.h>
// includes, project
#include <helper_cuda.h>
#include <helper_functions.h> // helper utility functions
__global__ void increment_kernel(int *g_data, int inc_value)
{
int idx = blockIdx.x * blockDim.x + threadIdx.x;
g_data[idx] = g_data[idx] + inc_value;
}
bool correct_output(int *data, const int n, const int x)
{
for (int i = 0; i < n; i++)
if (data[i] != x) {
printf("Error! data[%d] = %d, ref = %d\n", i, data[i], x);
return false;
}
return true;
}
int main(int argc, char *argv[])
{
int devID;
cudaDeviceProp deviceProps;
printf("[%s] - Starting...\n", argv[0]);
// This will pick the best possible CUDA capable device
devID = findCudaDevice(argc, (const char **)argv);
// get device name
checkCudaErrors(cudaGetDeviceProperties(&deviceProps, devID));
printf("CUDA device [%s]\n", deviceProps.name);
int n = 16 * 1024 * 1024;
int nbytes = n * sizeof(int);
int value = 26;
// allocate host memory
int *a = 0;
checkCudaErrors(cudaMallocHost((void **)&a, nbytes));
memset(a, 0, nbytes);
// allocate device memory
int *d_a = 0;
checkCudaErrors(cudaMalloc((void **)&d_a, nbytes));
checkCudaErrors(cudaMemset(d_a, 255, nbytes));
// set kernel launch configuration
dim3 threads = dim3(512, 1);
dim3 blocks = dim3(n / threads.x, 1);
// create cuda event handles
cudaEvent_t start, stop;
checkCudaErrors(cudaEventCreate(&start));
checkCudaErrors(cudaEventCreate(&stop));
StopWatchInterface *timer = NULL;
sdkCreateTimer(&timer);
sdkResetTimer(&timer);
checkCudaErrors(cudaDeviceSynchronize());
float gpu_time = 0.0f;
// asynchronously issue work to the GPU (all to stream 0)
checkCudaErrors(cudaProfilerStart());
sdkStartTimer(&timer);
cudaEventRecord(start, 0);
cudaMemcpyAsync(d_a, a, nbytes, cudaMemcpyHostToDevice, 0);
increment_kernel<<<blocks, threads, 0, 0>>>(d_a, value);
cudaMemcpyAsync(a, d_a, nbytes, cudaMemcpyDeviceToHost, 0);
cudaEventRecord(stop, 0);
sdkStopTimer(&timer);
checkCudaErrors(cudaProfilerStop());
// have CPU do some work while waiting for stage 1 to finish
unsigned long int counter = 0;
while (cudaEventQuery(stop) == cudaErrorNotReady) {
counter++;
}
checkCudaErrors(cudaEventElapsedTime(&gpu_time, start, stop));
// print the cpu and gpu times
printf("time spent executing by the GPU: %.2f\n", gpu_time);
printf("time spent by CPU in CUDA calls: %.2f\n", sdkGetTimerValue(&timer));
printf("CPU executed %lu iterations while waiting for GPU to finish\n", counter);
// check the output for correctness
bool bFinalResults = correct_output(a, n, value);
// release resources
checkCudaErrors(cudaEventDestroy(start));
checkCudaErrors(cudaEventDestroy(stop));
checkCudaErrors(cudaFreeHost(a));
checkCudaErrors(cudaFree(d_a));
exit(bFinalResults ? EXIT_SUCCESS : EXIT_FAILURE);
}

18
cpp/0_Introduction/clock/.vscode/c_cpp_properties.json vendored Normal file
View File

@@ -0,0 +1,18 @@
{
"configurations": [
{
"name": "Linux",
"includePath": [
"${workspaceFolder}/**",
"${workspaceFolder}/../../../Common"
],
"defines": [],
"compilerPath": "/usr/local/cuda/bin/nvcc",
"cStandard": "gnu17",
"cppStandard": "gnu++14",
"intelliSenseMode": "linux-gcc-x64",
"configurationProvider": "ms-vscode.makefile-tools"
}
],
"version": 4
}

7
cpp/0_Introduction/clock/.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,7 @@
{
"recommendations": [
"nvidia.nsight-vscode-edition",
"ms-vscode.cpptools",
"ms-vscode.makefile-tools"
]
}

34
cpp/0_Introduction/clock/CMakeLists.txt Normal file
View File

@@ -0,0 +1,34 @@
cmake_minimum_required(VERSION 3.20)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/Modules")
project(clock LANGUAGES C CXX CUDA)
find_package(CUDAToolkit REQUIRED)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(CMAKE_CUDA_ARCHITECTURES 75 80 86 87 89 90 100 110 120)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -Wno-deprecated-gpu-targets")
if(ENABLE_CUDA_DEBUG)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -G") # enable cuda-gdb (may significantly affect performance on some targets)
else()
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -lineinfo") # add line information to all builds for debug tools (exclusive to -G option)
endif()
# Include directories and libraries
include_directories(../../../Common)
# Source file
# Add target for asyncAPI
add_executable(clock clock.cu)
target_compile_options(clock PRIVATE $<$<COMPILE_LANGUAGE:CUDA>:--extended-lambda>)
target_compile_features(clock PRIVATE cxx_std_17 cuda_std_17)
set_target_properties(clock PROPERTIES CUDA_SEPARABLE_COMPILATION ON)
# Include installation configuration
include(${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/InstallSamples.cmake)
setup_samples_install()

32
cpp/0_Introduction/clock/README.md Normal file
View File

@@ -0,0 +1,32 @@
# clock - Clock
## Description
This example shows how to use the clock function to measure the performance of block of threads of a kernel accurately.
## Key Concepts
Performance Strategies
## Supported SM Architectures
[SM 5.0 ](https://developer.nvidia.com/cuda-gpus) [SM 5.2 ](https://developer.nvidia.com/cuda-gpus) [SM 5.3 ](https://developer.nvidia.com/cuda-gpus) [SM 6.0 ](https://developer.nvidia.com/cuda-gpus) [SM 6.1 ](https://developer.nvidia.com/cuda-gpus) [SM 7.0 ](https://developer.nvidia.com/cuda-gpus) [SM 7.2 ](https://developer.nvidia.com/cuda-gpus) [SM 7.5 ](https://developer.nvidia.com/cuda-gpus) [SM 8.0 ](https://developer.nvidia.com/cuda-gpus) [SM 8.6 ](https://developer.nvidia.com/cuda-gpus) [SM 8.7 ](https://developer.nvidia.com/cuda-gpus) [SM 8.9 ](https://developer.nvidia.com/cuda-gpus) [SM 9.0 ](https://developer.nvidia.com/cuda-gpus)
## Supported OSes
Linux, Windows
## Supported CPU Architecture
x86_64, armv7l
## CUDA APIs involved
### [CUDA Runtime API](http://docs.nvidia.com/cuda/cuda-runtime-api/index.html)
cudaMalloc, cudaMemcpy, cudaFree
## Prerequisites
Download and install the [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) for your corresponding platform.
## References (for more details)

152
cpp/0_Introduction/clock/clock.cu Normal file
View File

@@ -0,0 +1,152 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
* This example shows how to use the clock function to measure the performance
* of block of threads of a kernel accurately. Blocks are executed in parallel
* and out of order. Since there's no synchronization mechanism between blocks,
* we measure the clock once for each block. The clock samples are written to
* device memory.
*/
// System includes
#include <assert.h>
#include <stdint.h>
#include <stdio.h>
// CUDA runtime
#include <cuda_runtime.h>
// helper functions and utilities to work with CUDA
#include <helper_cuda.h>
#include <helper_functions.h>
// This kernel computes a standard parallel reduction and evaluates the
// time it takes to do that for each block. The timing results are stored
// in device memory.
__global__ static void timedReduction(const float *input, float *output, clock_t *timer)
{
// __shared__ float shared[2 * blockDim.x];
extern __shared__ float shared[];
const int tid = threadIdx.x;
const int bid = blockIdx.x;
if (tid == 0)
timer[bid] = clock();
// Copy input.
shared[tid] = input[tid];
shared[tid + blockDim.x] = input[tid + blockDim.x];
// Perform reduction to find minimum.
for (int d = blockDim.x; d > 0; d /= 2) {
__syncthreads();
if (tid < d) {
float f0 = shared[tid];
float f1 = shared[tid + d];
if (f1 < f0) {
shared[tid] = f1;
}
}
}
// Write result.
if (tid == 0)
output[bid] = shared[0];
__syncthreads();
if (tid == 0)
timer[bid + gridDim.x] = clock();
}
#define NUM_BLOCKS 64
#define NUM_THREADS 256
// It's interesting to change the number of blocks and the number of threads to
// understand how to keep the hardware busy.
//
// Here are some numbers I get on my G80:
// blocks - clocks
// 1 - 3096
// 8 - 3232
// 16 - 3364
// 32 - 4615
// 64 - 9981
//
// With less than 16 blocks some of the multiprocessors of the device are idle.
// With more than 16 you are using all the multiprocessors, but there's only one
// block per multiprocessor and that doesn't allow you to hide the latency of
// the memory. With more than 32 the speed scales linearly.
// Start the main CUDA Sample here
int main(int argc, char **argv)
{
printf("CUDA Clock sample\n");
// This will pick the best possible CUDA capable device
int dev = findCudaDevice(argc, (const char **)argv);
float *dinput = NULL;
float *doutput = NULL;
clock_t *dtimer = NULL;
clock_t timer[NUM_BLOCKS * 2];
float input[NUM_THREADS * 2];
for (int i = 0; i < NUM_THREADS * 2; i++) {
input[i] = (float)i;
}
checkCudaErrors(cudaMalloc((void **)&dinput, sizeof(float) * NUM_THREADS * 2));
checkCudaErrors(cudaMalloc((void **)&doutput, sizeof(float) * NUM_BLOCKS));
checkCudaErrors(cudaMalloc((void **)&dtimer, sizeof(clock_t) * NUM_BLOCKS * 2));
checkCudaErrors(cudaMemcpy(dinput, input, sizeof(float) * NUM_THREADS * 2, cudaMemcpyHostToDevice));
timedReduction<<<NUM_BLOCKS, NUM_THREADS, sizeof(float) * 2 * NUM_THREADS>>>(dinput, doutput, dtimer);
checkCudaErrors(cudaMemcpy(timer, dtimer, sizeof(clock_t) * NUM_BLOCKS * 2, cudaMemcpyDeviceToHost));
checkCudaErrors(cudaFree(dinput));
checkCudaErrors(cudaFree(doutput));
checkCudaErrors(cudaFree(dtimer));
long double avgElapsedClocks = 0;
for (int i = 0; i < NUM_BLOCKS; i++) {
avgElapsedClocks += (long double)(timer[i + NUM_BLOCKS] - timer[i]);
}
avgElapsedClocks = avgElapsedClocks / NUM_BLOCKS;
printf("Average clocks/block = %Lf\n", avgElapsedClocks);
return EXIT_SUCCESS;
}

18
cpp/0_Introduction/clock_nvrtc/.vscode/c_cpp_properties.json vendored Normal file
View File

@@ -0,0 +1,18 @@
{
"configurations": [
{
"name": "Linux",
"includePath": [
"${workspaceFolder}/**",
"${workspaceFolder}/../../../Common"
],
"defines": [],
"compilerPath": "/usr/local/cuda/bin/nvcc",
"cStandard": "gnu17",
"cppStandard": "gnu++14",
"intelliSenseMode": "linux-gcc-x64",
"configurationProvider": "ms-vscode.makefile-tools"
}
],
"version": 4
}

7
cpp/0_Introduction/clock_nvrtc/.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,7 @@
{
"recommendations": [
"nvidia.nsight-vscode-edition",
"ms-vscode.cpptools",
"ms-vscode.makefile-tools"
]
}

43
cpp/0_Introduction/clock_nvrtc/CMakeLists.txt Normal file
View File

@@ -0,0 +1,43 @@
cmake_minimum_required(VERSION 3.20)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/Modules")
project(clock_nvrtc LANGUAGES C CXX CUDA)
find_package(CUDAToolkit REQUIRED)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(CMAKE_CUDA_ARCHITECTURES 75 80 86 87 89 90 100 110 120)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -Wno-deprecated-gpu-targets")
if(ENABLE_CUDA_DEBUG)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -G") # enable cuda-gdb (may significantly affect performance on some targets)
else()
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -lineinfo") # add line information to all builds for debug tools (exclusive to -G option)
endif()
# Include directories and libraries
include_directories(../../../Common)
# Source file
# Add sample target executable
add_executable(clock_nvrtc clock.cpp)
target_compile_options(clock_nvrtc PRIVATE $<$<COMPILE_LANGUAGE:CUDA>:--extended-lambda>)
target_compile_features(clock_nvrtc PRIVATE cxx_std_17 cuda_std_17)
target_link_libraries(clock_nvrtc PRIVATE
CUDA::nvrtc
CUDA::cuda_driver
)
# Copy clock_kernel.cu to the output directory
add_custom_command(TARGET clock_nvrtc POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
${CMAKE_CURRENT_SOURCE_DIR}/clock_kernel.cu ${CMAKE_CURRENT_BINARY_DIR}
)
# Include installation configuration
include(${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/InstallSamples.cmake)
setup_samples_install()

39
cpp/0_Introduction/clock_nvrtc/README.md Normal file
View File

@@ -0,0 +1,39 @@
# clock_nvrtc - Clock libNVRTC
## Description
This example shows how to use the clock function using libNVRTC to measure the performance of block of threads of a kernel accurately.
## Key Concepts
Performance Strategies, Runtime Compilation
## Supported SM Architectures
[SM 5.0 ](https://developer.nvidia.com/cuda-gpus) [SM 5.2 ](https://developer.nvidia.com/cuda-gpus) [SM 5.3 ](https://developer.nvidia.com/cuda-gpus) [SM 6.0 ](https://developer.nvidia.com/cuda-gpus) [SM 6.1 ](https://developer.nvidia.com/cuda-gpus) [SM 7.0 ](https://developer.nvidia.com/cuda-gpus) [SM 7.2 ](https://developer.nvidia.com/cuda-gpus) [SM 7.5 ](https://developer.nvidia.com/cuda-gpus) [SM 8.0 ](https://developer.nvidia.com/cuda-gpus) [SM 8.6 ](https://developer.nvidia.com/cuda-gpus) [SM 8.7 ](https://developer.nvidia.com/cuda-gpus) [SM 8.9 ](https://developer.nvidia.com/cuda-gpus) [SM 9.0 ](https://developer.nvidia.com/cuda-gpus)
## Supported OSes
Linux, Windows, QNX
## Supported CPU Architecture
x86_64, aarch64
## CUDA APIs involved
### [CUDA Driver API](http://docs.nvidia.com/cuda/cuda-driver-api/index.html)
cuMemcpyDtoH, cuLaunchKernel, cuMemcpyHtoD, cuCtxSynchronize, cuMemAlloc, cuMemFree, cuModuleGetFunction
### [CUDA Runtime API](http://docs.nvidia.com/cuda/cuda-runtime-api/index.html)
cudaBlockSize, cudaGridSize
## Dependencies needed to build/run
[NVRTC](../../../README.md#nvrtc)
## Prerequisites
Download and install the [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) for your corresponding platform.
Make sure the dependencies mentioned in [Dependencies]() section above are installed.
## References (for more details)

137
cpp/0_Introduction/clock_nvrtc/clock.cpp Normal file
View File

@@ -0,0 +1,137 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
* This example shows how to use the clock function to measure the performance
* of block of threads of a kernel accurately. Blocks are executed in parallel
* and out of order. Since there's no synchronization mechanism between blocks,
* we measure the clock once for each block. The clock samples are written to
* device memory.
*/
// System includes
#include <assert.h>
#include <cuda_runtime.h>
#include <nvrtc_helper.h>
#include <stdint.h>
#include <stdio.h>
// helper functions and utilities to work with CUDA
#include <helper_functions.h>
#define NUM_BLOCKS 64
#define NUM_THREADS 256
// It's interesting to change the number of blocks and the number of threads to
// understand how to keep the hardware busy.
//
// Here are some numbers I get on my G80:
// blocks - clocks
// 1 - 3096
// 8 - 3232
// 16 - 3364
// 32 - 4615
// 64 - 9981
//
// With less than 16 blocks some of the multiprocessors of the device are idle.
// With
// more than 16 you are using all the multiprocessors, but there's only one
// block per
// multiprocessor and that doesn't allow you to hide the latency of the memory.
// With
// more than 32 the speed scales linearly.
// Start the main CUDA Sample here
int main(int argc, char **argv)
{
printf("CUDA Clock sample\n");
typedef long clock_t;
clock_t timer[NUM_BLOCKS * 2];
float input[NUM_THREADS * 2];
for (int i = 0; i < NUM_THREADS * 2; i++) {
input[i] = (float)i;
}
char *cubin, *kernel_file;
size_t cubinSize;
kernel_file = sdkFindFilePath("clock_kernel.cu", argv[0]);
compileFileToCUBIN(kernel_file, argc, argv, &cubin, &cubinSize, 0);
CUmodule module = loadCUBIN(cubin, argc, argv);
CUfunction kernel_addr;
checkCudaErrors(cuModuleGetFunction(&kernel_addr, module, "timedReduction"));
dim3 cudaBlockSize(NUM_THREADS, 1, 1);
dim3 cudaGridSize(NUM_BLOCKS, 1, 1);
CUdeviceptr dinput, doutput, dtimer;
checkCudaErrors(cuMemAlloc(&dinput, sizeof(float) * NUM_THREADS * 2));
checkCudaErrors(cuMemAlloc(&doutput, sizeof(float) * NUM_BLOCKS));
checkCudaErrors(cuMemAlloc(&dtimer, sizeof(clock_t) * NUM_BLOCKS * 2));
checkCudaErrors(cuMemcpyHtoD(dinput, input, sizeof(float) * NUM_THREADS * 2));
void *arr[] = {(void *)&dinput, (void *)&doutput, (void *)&dtimer};
checkCudaErrors(cuLaunchKernel(kernel_addr,
cudaGridSize.x,
cudaGridSize.y,
cudaGridSize.z, /* grid dim */
cudaBlockSize.x,
cudaBlockSize.y,
cudaBlockSize.z, /* block dim */
sizeof(float) * 2 * NUM_THREADS,
0, /* shared mem, stream */
&arr[0], /* arguments */
0));
checkCudaErrors(cuCtxSynchronize());
checkCudaErrors(cuMemcpyDtoH(timer, dtimer, sizeof(clock_t) * NUM_BLOCKS * 2));
checkCudaErrors(cuMemFree(dinput));
checkCudaErrors(cuMemFree(doutput));
checkCudaErrors(cuMemFree(dtimer));
long double avgElapsedClocks = 0;
for (int i = 0; i < NUM_BLOCKS; i++) {
avgElapsedClocks += (long double)(timer[i + NUM_BLOCKS] - timer[i]);
}
avgElapsedClocks = avgElapsedClocks / NUM_BLOCKS;
printf("Average clocks/block = %Lf\n", avgElapsedClocks);
return EXIT_SUCCESS;
}

77
cpp/0_Introduction/clock_nvrtc/clock_kernel.cu Normal file
View File

@@ -0,0 +1,77 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
* This example shows how to use the clock function to measure the performance
* of block of threads of a kernel accurately. Blocks are executed in parallel
* and out of order. Since there's no synchronization mechanism between blocks,
* we measure the clock once for each block. The clock samples are written to
* device memory.
*/
// This kernel computes a standard parallel reduction and evaluates the
// time it takes to do that for each block. The timing results are stored
// in device memory.
extern "C" __global__ void timedReduction(const float *input, float *output, clock_t *timer)
{
// __shared__ float shared[2 * blockDim.x];
extern __shared__ float shared[];
const int tid = threadIdx.x;
const int bid = blockIdx.x;
if (tid == 0)
timer[bid] = clock();
// Copy input.
shared[tid] = input[tid];
shared[tid + blockDim.x] = input[tid + blockDim.x];
// Perform reduction to find minimum.
for (int d = blockDim.x; d > 0; d /= 2) {
__syncthreads();
if (tid < d) {
float f0 = shared[tid];
float f1 = shared[tid + d];
if (f1 < f0) {
shared[tid] = f1;
}
}
}
// Write result.
if (tid == 0)
output[bid] = shared[0];
__syncthreads();
if (tid == 0)
timer[bid + gridDim.x] = clock();
}

18
cpp/0_Introduction/cudaOpenMP/.vscode/c_cpp_properties.json vendored Normal file
View File

@@ -0,0 +1,18 @@
{
"configurations": [
{
"name": "Linux",
"includePath": [
"${workspaceFolder}/**",
"${workspaceFolder}/../../../Common"
],
"defines": [],
"compilerPath": "/usr/local/cuda/bin/nvcc",
"cStandard": "gnu17",
"cppStandard": "gnu++14",
"intelliSenseMode": "linux-gcc-x64",
"configurationProvider": "ms-vscode.makefile-tools"
}
],
"version": 4
}

7
cpp/0_Introduction/cudaOpenMP/.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,7 @@
{
"recommendations": [
"nvidia.nsight-vscode-edition",
"ms-vscode.cpptools",
"ms-vscode.makefile-tools"
]
}

43
cpp/0_Introduction/cudaOpenMP/CMakeLists.txt Normal file
View File

@@ -0,0 +1,43 @@
cmake_minimum_required(VERSION 3.20)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/Modules")
project(cudaOpenMP LANGUAGES C CXX CUDA)
find_package(CUDAToolkit REQUIRED)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(CMAKE_CUDA_ARCHITECTURES 75 80 86 87 89 90 100 110 120)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -Wno-deprecated-gpu-targets")
if(ENABLE_CUDA_DEBUG)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -G") # enable cuda-gdb (may significantly affect performance on some targets)
else()
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -lineinfo") # add line information to all builds for debug tools (exclusive to -G option)
endif()
# Include directories and libraries
include_directories(../../../Common)
find_package(OpenMP)
# Source file
if(OpenMP_CXX_FOUND)
# Add target for asyncAPI
add_executable(cudaOpenMP cudaOpenMP.cu)
target_compile_options(cudaOpenMP PRIVATE $<$<COMPILE_LANGUAGE:CUDA>:--extended-lambda>)
target_compile_features(cudaOpenMP PRIVATE cxx_std_17 cuda_std_17)
set_target_properties(cudaOpenMP PROPERTIES CUDA_SEPARABLE_COMPILATION ON)
target_link_libraries(cudaOpenMP PUBLIC
OpenMP::OpenMP_CXX
)
else()
message(STATUS "OpenMP not found - will not build sample 'cudaOpenMP'")
endif()
# Include installation configuration
include(${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/InstallSamples.cmake)
setup_samples_install()

36
cpp/0_Introduction/cudaOpenMP/README.md Normal file
View File

@@ -0,0 +1,36 @@
# cudaOpenMP - cudaOpenMP
## Description
This sample demonstrates how to use OpenMP API to write an application for multiple GPUs.
## Key Concepts
CUDA Systems Integration, OpenMP, Multithreading
## Supported SM Architectures
[SM 5.0 ](https://developer.nvidia.com/cuda-gpus) [SM 5.2 ](https://developer.nvidia.com/cuda-gpus) [SM 5.3 ](https://developer.nvidia.com/cuda-gpus) [SM 6.0 ](https://developer.nvidia.com/cuda-gpus) [SM 6.1 ](https://developer.nvidia.com/cuda-gpus) [SM 7.0 ](https://developer.nvidia.com/cuda-gpus) [SM 7.2 ](https://developer.nvidia.com/cuda-gpus) [SM 7.5 ](https://developer.nvidia.com/cuda-gpus) [SM 8.0 ](https://developer.nvidia.com/cuda-gpus) [SM 8.6 ](https://developer.nvidia.com/cuda-gpus) [SM 8.7 ](https://developer.nvidia.com/cuda-gpus) [SM 8.9 ](https://developer.nvidia.com/cuda-gpus) [SM 9.0 ](https://developer.nvidia.com/cuda-gpus)
## Supported OSes
Linux, Windows
## Supported CPU Architecture
x86_64, armv7l
## CUDA APIs involved
### [CUDA Runtime API](http://docs.nvidia.com/cuda/cuda-runtime-api/index.html)
cudaMemcpy, cudaGetErrorString, cudaFree, cudaGetLastError, cudaSetDevice, cudaGetDeviceCount, cudaGetDevice, cudaMemset, cudaMalloc, cudaGetDeviceProperties
## Dependencies needed to build/run
[OpenMP](../../../README.md#openmp)
## Prerequisites
Download and install the [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) for your corresponding platform.
Make sure the dependencies mentioned in [Dependencies]() section above are installed.
## References (for more details)

156
cpp/0_Introduction/cudaOpenMP/cudaOpenMP.cu Normal file
View File

@@ -0,0 +1,156 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
* Multi-GPU sample using OpenMP for threading on the CPU side
* needs a compiler that supports OpenMP 2.0
*/
#include <helper_cuda.h>
#include <omp.h>
#include <stdio.h> // stdio functions are used since C++ streams aren't necessarily thread safe
using namespace std;
// a simple kernel that simply increments each array element by b
__global__ void kernelAddConstant(int *g_a, const int b)
{
int idx = blockIdx.x * blockDim.x + threadIdx.x;
g_a[idx] += b;
}
// a predicate that checks whether each array element is set to its index plus b
int correctResult(int *data, const int n, const int b)
{
for (int i = 0; i < n; i++)
if (data[i] != i + b)
return 0;
return 1;
}
int main(int argc, char *argv[])
{
int num_gpus = 0; // number of CUDA GPUs
printf("%s Starting...\n\n", argv[0]);
/////////////////////////////////////////////////////////////////
// determine the number of CUDA capable GPUs
//
cudaGetDeviceCount(&num_gpus);
if (num_gpus < 1) {
printf("no CUDA capable devices were detected\n");
return 1;
}
/////////////////////////////////////////////////////////////////
// display CPU and GPU configuration
//
printf("number of host CPUs:\t%d\n", omp_get_num_procs());
printf("number of CUDA devices:\t%d\n", num_gpus);
for (int i = 0; i < num_gpus; i++) {
cudaDeviceProp dprop;
cudaGetDeviceProperties(&dprop, i);
printf(" %d: %s\n", i, dprop.name);
}
printf("---------------------------\n");
/////////////////////////////////////////////////////////////////
// initialize data
//
unsigned int n = num_gpus * 8192;
unsigned int nbytes = n * sizeof(int);
int *a = 0; // pointer to data on the CPU
int b = 3; // value by which the array is incremented
a = (int *)malloc(nbytes);
if (0 == a) {
printf("couldn't allocate CPU memory\n");
return 1;
}
for (unsigned int i = 0; i < n; i++)
a[i] = i;
////////////////////////////////////////////////////////////////
// run as many CPU threads as there are CUDA devices
// each CPU thread controls a different device, processing its
// portion of the data. It's possible to use more CPU threads
// than there are CUDA devices, in which case several CPU
// threads will be allocating resources and launching kernels
// on the same device. For example, try omp_set_num_threads(2*num_gpus);
// Recall that all variables declared inside an "omp parallel" scope are
// local to each CPU thread
//
omp_set_num_threads(num_gpus); // create as many CPU threads as there are CUDA devices
// omp_set_num_threads(2*num_gpus);// create twice as many CPU threads as there
// are CUDA devices
#pragma omp parallel
{
unsigned int cpu_thread_id = omp_get_thread_num();
unsigned int num_cpu_threads = omp_get_num_threads();
// set and check the CUDA device for this CPU thread
int gpu_id = -1;
checkCudaErrors(
cudaSetDevice(cpu_thread_id % num_gpus)); // "% num_gpus" allows more CPU threads than GPU devices
checkCudaErrors(cudaGetDevice(&gpu_id));
printf("CPU thread %d (of %d) uses CUDA device %d\n", cpu_thread_id, num_cpu_threads, gpu_id);
int *d_a = 0; // pointer to memory on the device associated with this CPU thread
int *sub_a = a + cpu_thread_id * n / num_cpu_threads; // pointer to this CPU thread's portion of data
unsigned int nbytes_per_kernel = nbytes / num_cpu_threads;
dim3 gpu_threads(128); // 128 threads per block
dim3 gpu_blocks(n / (gpu_threads.x * num_cpu_threads));
checkCudaErrors(cudaMalloc((void **)&d_a, nbytes_per_kernel));
checkCudaErrors(cudaMemset(d_a, 0, nbytes_per_kernel));
checkCudaErrors(cudaMemcpy(d_a, sub_a, nbytes_per_kernel, cudaMemcpyHostToDevice));
kernelAddConstant<<<gpu_blocks, gpu_threads>>>(d_a, b);
checkCudaErrors(cudaMemcpy(sub_a, d_a, nbytes_per_kernel, cudaMemcpyDeviceToHost));
checkCudaErrors(cudaFree(d_a));
}
printf("---------------------------\n");
if (cudaSuccess != cudaGetLastError())
printf("%s\n", cudaGetErrorString(cudaGetLastError()));
////////////////////////////////////////////////////////////////
// check the result
//
bool bResult = correctResult(a, n, b);
if (a)
free(a); // free CPU memory
exit(bResult ? EXIT_SUCCESS : EXIT_FAILURE);
}

18
cpp/0_Introduction/fp16ScalarProduct/.vscode/c_cpp_properties.json vendored Normal file
View File

@@ -0,0 +1,18 @@
{
"configurations": [
{
"name": "Linux",
"includePath": [
"${workspaceFolder}/**",
"${workspaceFolder}/../../../Common"
],
"defines": [],
"compilerPath": "/usr/local/cuda/bin/nvcc",
"cStandard": "gnu17",
"cppStandard": "gnu++14",
"intelliSenseMode": "linux-gcc-x64",
"configurationProvider": "ms-vscode.makefile-tools"
}
],
"version": 4
}

7
cpp/0_Introduction/fp16ScalarProduct/.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,7 @@
{
"recommendations": [
"nvidia.nsight-vscode-edition",
"ms-vscode.cpptools",
"ms-vscode.makefile-tools"
]
}

33
cpp/0_Introduction/fp16ScalarProduct/CMakeLists.txt Normal file
View File

@@ -0,0 +1,33 @@
cmake_minimum_required(VERSION 3.20)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/Modules")
project(fp16ScalarProduct LANGUAGES C CXX CUDA)
find_package(CUDAToolkit REQUIRED)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(CMAKE_CUDA_ARCHITECTURES 75 80 86 87 89 90 100 110 120)
if(ENABLE_CUDA_DEBUG)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -G") # enable cuda-gdb (may significantly affect performance on some targets)
else()
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -lineinfo") # add line information to all builds for debug tools (exclusive to -G option)
endif()
# Include directories and libraries
include_directories(../../../Common)
# Source file
# Add target for asyncAPI
add_executable(fp16ScalarProduct fp16ScalarProduct.cu)
target_compile_options(fp16ScalarProduct PRIVATE $<$<COMPILE_LANGUAGE:CUDA>:--extended-lambda>)
target_compile_features(fp16ScalarProduct PRIVATE cxx_std_17 cuda_std_17)
set_target_properties(fp16ScalarProduct PROPERTIES CUDA_SEPARABLE_COMPILATION ON)
# Include installation configuration
include(${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/InstallSamples.cmake)
setup_samples_install()

36
cpp/0_Introduction/fp16ScalarProduct/README.md Normal file
View File

@@ -0,0 +1,36 @@
# fp16ScalarProduct - FP16 Scalar Product
## Description
Calculates scalar product of two vectors of FP16 numbers.
## Key Concepts
CUDA Runtime API
## Supported SM Architectures
[SM 6.0 ](https://developer.nvidia.com/cuda-gpus) [SM 6.1 ](https://developer.nvidia.com/cuda-gpus) [SM 7.0 ](https://developer.nvidia.com/cuda-gpus) [SM 7.2 ](https://developer.nvidia.com/cuda-gpus) [SM 7.5 ](https://developer.nvidia.com/cuda-gpus) [SM 8.0 ](https://developer.nvidia.com/cuda-gpus) [SM 8.6 ](https://developer.nvidia.com/cuda-gpus) [SM 8.7 ](https://developer.nvidia.com/cuda-gpus) [SM 8.9 ](https://developer.nvidia.com/cuda-gpus) [SM 9.0 ](https://developer.nvidia.com/cuda-gpus)
## Supported OSes
Linux, Windows
## Supported CPU Architecture
x86_64, armv7l
## CUDA APIs involved
### [CUDA Runtime API](http://docs.nvidia.com/cuda/cuda-runtime-api/index.html)
cudaMemcpy, cudaFree, cudaMallocHost, cudaFreeHost, cudaMalloc, cudaGetDeviceProperties
## Dependencies needed to build/run
[FP16](../../../README.md#16-bit-floating-point)
## Prerequisites
Download and install the [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) for your corresponding platform.
Make sure the dependencies mentioned in [Dependencies]() section above are installed.
## References (for more details)

212
cpp/0_Introduction/fp16ScalarProduct/fp16ScalarProduct.cu Normal file
View File

@@ -0,0 +1,212 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <cstdio>
#include <cstdlib>
#include <ctime>
#include "cuda_fp16.h"
#include "helper_cuda.h"
#define NUM_OF_BLOCKS 128
#define NUM_OF_THREADS 128
__forceinline__ __device__ void reduceInShared_intrinsics(half2 *const v)
{
if (threadIdx.x < 64)
v[threadIdx.x] = __hadd2(v[threadIdx.x], v[threadIdx.x + 64]);
__syncthreads();
if (threadIdx.x < 32)
v[threadIdx.x] = __hadd2(v[threadIdx.x], v[threadIdx.x + 32]);
__syncthreads();
if (threadIdx.x < 16)
v[threadIdx.x] = __hadd2(v[threadIdx.x], v[threadIdx.x + 16]);
__syncthreads();
if (threadIdx.x < 8)
v[threadIdx.x] = __hadd2(v[threadIdx.x], v[threadIdx.x + 8]);
__syncthreads();
if (threadIdx.x < 4)
v[threadIdx.x] = __hadd2(v[threadIdx.x], v[threadIdx.x + 4]);
__syncthreads();
if (threadIdx.x < 2)
v[threadIdx.x] = __hadd2(v[threadIdx.x], v[threadIdx.x + 2]);
__syncthreads();
if (threadIdx.x < 1)
v[threadIdx.x] = __hadd2(v[threadIdx.x], v[threadIdx.x + 1]);
__syncthreads();
}
__forceinline__ __device__ void reduceInShared_native(half2 *const v)
{
if (threadIdx.x < 64)
v[threadIdx.x] = v[threadIdx.x] + v[threadIdx.x + 64];
__syncthreads();
if (threadIdx.x < 32)
v[threadIdx.x] = v[threadIdx.x] + v[threadIdx.x + 32];
__syncthreads();
if (threadIdx.x < 16)
v[threadIdx.x] = v[threadIdx.x] + v[threadIdx.x + 16];
__syncthreads();
if (threadIdx.x < 8)
v[threadIdx.x] = v[threadIdx.x] + v[threadIdx.x + 8];
__syncthreads();
if (threadIdx.x < 4)
v[threadIdx.x] = v[threadIdx.x] + v[threadIdx.x + 4];
__syncthreads();
if (threadIdx.x < 2)
v[threadIdx.x] = v[threadIdx.x] + v[threadIdx.x + 2];
__syncthreads();
if (threadIdx.x < 1)
v[threadIdx.x] = v[threadIdx.x] + v[threadIdx.x + 1];
__syncthreads();
}
__global__ void
scalarProductKernel_intrinsics(half2 const *const a, half2 const *const b, float *const results, size_t const size)
{
const int stride = gridDim.x * blockDim.x;
__shared__ half2 shArray[NUM_OF_THREADS];
shArray[threadIdx.x] = __float2half2_rn(0.f);
half2 value = __float2half2_rn(0.f);
for (int i = threadIdx.x + blockDim.x * blockIdx.x; i < size; i += stride) {
value = __hfma2(a[i], b[i], value);
}
shArray[threadIdx.x] = value;
__syncthreads();
reduceInShared_intrinsics(shArray);
if (threadIdx.x == 0) {
half2 result = shArray[0];
float f_result = __low2float(result) + __high2float(result);
results[blockIdx.x] = f_result;
}
}
__global__ void
scalarProductKernel_native(half2 const *const a, half2 const *const b, float *const results, size_t const size)
{
const int stride = gridDim.x * blockDim.x;
__shared__ half2 shArray[NUM_OF_THREADS];
half2 value(0.f, 0.f);
shArray[threadIdx.x] = value;
for (int i = threadIdx.x + blockDim.x * blockIdx.x; i < size; i += stride) {
value = a[i] * b[i] + value;
}
shArray[threadIdx.x] = value;
__syncthreads();
reduceInShared_native(shArray);
if (threadIdx.x == 0) {
half2 result = shArray[0];
float f_result = (float)result.y + (float)result.x;
results[blockIdx.x] = f_result;
}
}
void generateInput(half2 *a, size_t size)
{
for (size_t i = 0; i < size; ++i) {
half2 temp;
temp.x = static_cast<float>(rand() % 4);
temp.y = static_cast<float>(rand() % 2);
a[i] = temp;
}
}
int main(int argc, char *argv[])
{
srand((unsigned int)time(NULL));
size_t size = NUM_OF_BLOCKS * NUM_OF_THREADS * 16;
half2 *vec[2];
half2 *devVec[2];
float *results;
float *devResults;
int devID = findCudaDevice(argc, (const char **)argv);
cudaDeviceProp devProp;
checkCudaErrors(cudaGetDeviceProperties(&devProp, devID));
if (devProp.major < 5 || (devProp.major == 5 && devProp.minor < 3)) {
printf("ERROR: fp16ScalarProduct requires GPU devices with compute SM 5.3 or "
"higher.\n");
return EXIT_WAIVED;
}
for (int i = 0; i < 2; ++i) {
checkCudaErrors(cudaMallocHost((void **)&vec[i], size * sizeof *vec[i]));
checkCudaErrors(cudaMalloc((void **)&devVec[i], size * sizeof *devVec[i]));
}
checkCudaErrors(cudaMallocHost((void **)&results, NUM_OF_BLOCKS * sizeof *results));
checkCudaErrors(cudaMalloc((void **)&devResults, NUM_OF_BLOCKS * sizeof *devResults));
for (int i = 0; i < 2; ++i) {
generateInput(vec[i], size);
checkCudaErrors(cudaMemcpy(devVec[i], vec[i], size * sizeof *vec[i], cudaMemcpyHostToDevice));
}
scalarProductKernel_native<<<NUM_OF_BLOCKS, NUM_OF_THREADS>>>(devVec[0], devVec[1], devResults, size);
checkCudaErrors(cudaMemcpy(results, devResults, NUM_OF_BLOCKS * sizeof *results, cudaMemcpyDeviceToHost));
float result_native = 0;
for (int i = 0; i < NUM_OF_BLOCKS; ++i) {
result_native += results[i];
}
printf("Result native operators\t: %f \n", result_native);
scalarProductKernel_intrinsics<<<NUM_OF_BLOCKS, NUM_OF_THREADS>>>(devVec[0], devVec[1], devResults, size);
checkCudaErrors(cudaMemcpy(results, devResults, NUM_OF_BLOCKS * sizeof *results, cudaMemcpyDeviceToHost));
float result_intrinsics = 0;
for (int i = 0; i < NUM_OF_BLOCKS; ++i) {
result_intrinsics += results[i];
}
printf("Result intrinsics\t: %f \n", result_intrinsics);
printf("&&&& fp16ScalarProduct %s\n", (fabs(result_intrinsics - result_native) < 0.00001) ? "PASSED" : "FAILED");
for (int i = 0; i < 2; ++i) {
checkCudaErrors(cudaFree(devVec[i]));
checkCudaErrors(cudaFreeHost(vec[i]));
}
checkCudaErrors(cudaFree(devResults));
checkCudaErrors(cudaFreeHost(results));
return EXIT_SUCCESS;
}

18
cpp/0_Introduction/matrixMul/.vscode/c_cpp_properties.json vendored Normal file
View File

@@ -0,0 +1,18 @@
{
"configurations": [
{
"name": "Linux",
"includePath": [
"${workspaceFolder}/**",
"${workspaceFolder}/../../../Common"
],
"defines": [],
"compilerPath": "/usr/local/cuda/bin/nvcc",
"cStandard": "gnu17",
"cppStandard": "gnu++14",
"intelliSenseMode": "linux-gcc-x64",
"configurationProvider": "ms-vscode.makefile-tools"
}
],
"version": 4
}

7
cpp/0_Introduction/matrixMul/.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,7 @@
{
"recommendations": [
"nvidia.nsight-vscode-edition",
"ms-vscode.cpptools",
"ms-vscode.makefile-tools"
]
}

34
cpp/0_Introduction/matrixMul/CMakeLists.txt Normal file
View File

@@ -0,0 +1,34 @@
cmake_minimum_required(VERSION 3.20)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/Modules")
project(matrixMul LANGUAGES C CXX CUDA)
find_package(CUDAToolkit REQUIRED)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(CMAKE_CUDA_ARCHITECTURES 75 80 86 87 89 90 100 110 120)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -Wno-deprecated-gpu-targets")
if(ENABLE_CUDA_DEBUG)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -G") # enable cuda-gdb (may significantly affect performance on some targets)
else()
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -lineinfo") # add line information to all builds for debug tools (exclusive to -G option)
endif()
# Include directories and libraries
include_directories(../../../Common)
# Source file
# Add target for asyncAPI
add_executable(matrixMul matrixMul.cu)
target_compile_options(matrixMul PRIVATE $<$<COMPILE_LANGUAGE:CUDA>:--extended-lambda>)
target_compile_features(matrixMul PRIVATE cxx_std_17 cuda_std_17)
set_target_properties(matrixMul PROPERTIES CUDA_SEPARABLE_COMPILATION ON)
# Include installation configuration
include(${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/InstallSamples.cmake)
setup_samples_install()

32
cpp/0_Introduction/matrixMul/README.md Normal file
View File

@@ -0,0 +1,32 @@
# matrixMul - Matrix Multiplication (CUDA Runtime API Version)
## Description
This sample implements matrix multiplication and is exactly the same as the second example of the [Shared Memory](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#shared-memory) section of the programming guide. It has been written for clarity of exposition to illustrate various CUDA programming principles, not with the goal of providing the most performant generic kernel for matrix multiplication. To illustrate GPU performance for matrix multiply, this sample also shows how to use the CUDA 4.0+ interface for cuBLAS to demonstrate high-performance performance for matrix multiplication.
## Key Concepts
CUDA Runtime API, Linear Algebra
## Supported SM Architectures
[SM 5.0 ](https://developer.nvidia.com/cuda-gpus) [SM 5.2 ](https://developer.nvidia.com/cuda-gpus) [SM 5.3 ](https://developer.nvidia.com/cuda-gpus) [SM 6.0 ](https://developer.nvidia.com/cuda-gpus) [SM 6.1 ](https://developer.nvidia.com/cuda-gpus) [SM 7.0 ](https://developer.nvidia.com/cuda-gpus) [SM 7.2 ](https://developer.nvidia.com/cuda-gpus) [SM 7.5 ](https://developer.nvidia.com/cuda-gpus) [SM 8.0 ](https://developer.nvidia.com/cuda-gpus) [SM 8.6 ](https://developer.nvidia.com/cuda-gpus) [SM 8.7 ](https://developer.nvidia.com/cuda-gpus) [SM 8.9 ](https://developer.nvidia.com/cuda-gpus) [SM 9.0 ](https://developer.nvidia.com/cuda-gpus)
## Supported OSes
Linux, Windows
## Supported CPU Architecture
x86_64, armv7l, aarch64
## CUDA APIs involved
### [CUDA Runtime API](http://docs.nvidia.com/cuda/cuda-runtime-api/index.html)
cudaStreamCreateWithFlags, cudaProfilerStop, cudaMalloc, cudaFree, cudaMallocHost, cudaProfilerStart, cudaEventSynchronize, cudaEventRecord, cudaFreeHost, cudaStreamSynchronize, cudaEventDestroy, cudaEventElapsedTime, cudaMemcpyAsync, cudaEventCreate
## Prerequisites
Download and install the [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) for your corresponding platform.
## References (for more details)

342
cpp/0_Introduction/matrixMul/matrixMul.cu Normal file
View File

@@ -0,0 +1,342 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/**
* Matrix multiplication: C = A * B.
* Host code.
*
* This sample implements matrix multiplication which makes use of shared memory
* to ensure data reuse, the matrix multiplication is done using tiling approach.
* It has been written for clarity of exposition to illustrate various CUDA programming
* principles, not with the goal of providing the most performant generic kernel for matrix multiplication.
* See also:
* V. Volkov and J. Demmel, "Benchmarking GPUs to tune dense linear algebra,"
* in Proc. 2008 ACM/IEEE Conf. on Supercomputing (SC '08),
* Piscataway, NJ: IEEE Press, 2008, pp. Art. 31:1-11.
*/
// System includes
#include <assert.h>
#include <stdio.h>
// CUDA runtime
#include <cuda_profiler_api.h>
#include <cuda_runtime.h>
// Helper functions and utilities to work with CUDA
#include <helper_cuda.h>
#include <helper_functions.h>
/**
* Matrix multiplication (CUDA Kernel) on the device: C = A * B
* wA is A's width and wB is B's width
*/
template <int BLOCK_SIZE> __global__ void MatrixMulCUDA(float *C, float *A, float *B, int wA, int wB)
{
// Block index
int bx = blockIdx.x;
int by = blockIdx.y;
// Thread index
int tx = threadIdx.x;
int ty = threadIdx.y;
// Index of the first sub-matrix of A processed by the block
int aBegin = wA * BLOCK_SIZE * by;
// Index of the last sub-matrix of A processed by the block
int aEnd = aBegin + wA - 1;
// Step size used to iterate through the sub-matrices of A
int aStep = BLOCK_SIZE;
// Index of the first sub-matrix of B processed by the block
int bBegin = BLOCK_SIZE * bx;
// Step size used to iterate through the sub-matrices of B
int bStep = BLOCK_SIZE * wB;
// Csub is used to store the element of the block sub-matrix
// that is computed by the thread
float Csub = 0;
// Loop over all the sub-matrices of A and B
// required to compute the block sub-matrix
for (int a = aBegin, b = bBegin; a <= aEnd; a += aStep, b += bStep) {
// Declaration of the shared memory array As used to
// store the sub-matrix of A
__shared__ float As[BLOCK_SIZE][BLOCK_SIZE];
// Declaration of the shared memory array Bs used to
// store the sub-matrix of B
__shared__ float Bs[BLOCK_SIZE][BLOCK_SIZE];
// Load the matrices from device memory
// to shared memory; each thread loads
// one element of each matrix
As[ty][tx] = A[a + wA * ty + tx];
Bs[ty][tx] = B[b + wB * ty + tx];
// Synchronize to make sure the matrices are loaded
__syncthreads();
// Multiply the two matrices together;
// each thread computes one element
// of the block sub-matrix
#pragma unroll
for (int k = 0; k < BLOCK_SIZE; ++k) {
Csub += As[ty][k] * Bs[k][tx];
}
// Synchronize to make sure that the preceding
// computation is done before loading two new
// sub-matrices of A and B in the next iteration
__syncthreads();
}
// Write the block sub-matrix to device memory;
// each thread writes one element
int c = wB * BLOCK_SIZE * by + BLOCK_SIZE * bx;
C[c + wB * ty + tx] = Csub;
}
void ConstantInit(float *data, int size, float val)
{
for (int i = 0; i < size; ++i) {
data[i] = val;
}
}
/**
* Run a simple test of matrix multiplication using CUDA
*/
int MatrixMultiply(int argc, char **argv, int block_size, const dim3 &dimsA, const dim3 &dimsB)
{
// Allocate host memory for matrices A and B
unsigned int size_A = dimsA.x * dimsA.y;
unsigned int mem_size_A = sizeof(float) * size_A;
float *h_A;
checkCudaErrors(cudaMallocHost(&h_A, mem_size_A));
unsigned int size_B = dimsB.x * dimsB.y;
unsigned int mem_size_B = sizeof(float) * size_B;
float *h_B;
checkCudaErrors(cudaMallocHost(&h_B, mem_size_B));
cudaStream_t stream;
// Initialize host memory
const float valB = 0.01f;
ConstantInit(h_A, size_A, 1.0f);
ConstantInit(h_B, size_B, valB);
// Allocate device memory
float *d_A, *d_B, *d_C;
// Allocate host matrix C
dim3 dimsC(dimsB.x, dimsA.y, 1);
unsigned int mem_size_C = dimsC.x * dimsC.y * sizeof(float);
float *h_C;
checkCudaErrors(cudaMallocHost(&h_C, mem_size_C));
if (h_C == NULL) {
fprintf(stderr, "Failed to allocate host matrix C!\n");
exit(EXIT_FAILURE);
}
checkCudaErrors(cudaMalloc(reinterpret_cast<void **>(&d_A), mem_size_A));
checkCudaErrors(cudaMalloc(reinterpret_cast<void **>(&d_B), mem_size_B));
checkCudaErrors(cudaMalloc(reinterpret_cast<void **>(&d_C), mem_size_C));
// Allocate CUDA events that we'll use for timing
cudaEvent_t start, stop;
checkCudaErrors(cudaEventCreate(&start));
checkCudaErrors(cudaEventCreate(&stop));
checkCudaErrors(cudaStreamCreateWithFlags(&stream, cudaStreamNonBlocking));
// copy host memory to device
checkCudaErrors(cudaMemcpyAsync(d_A, h_A, mem_size_A, cudaMemcpyHostToDevice, stream));
checkCudaErrors(cudaMemcpyAsync(d_B, h_B, mem_size_B, cudaMemcpyHostToDevice, stream));
// Setup execution parameters
dim3 threads(block_size, block_size);
dim3 grid(dimsB.x / threads.x, dimsA.y / threads.y);
// Create and start timer
printf("Computing result using CUDA Kernel...\n");
// Performs warmup operation using matrixMul CUDA kernel
if (block_size == 16) {
MatrixMulCUDA<16><<<grid, threads, 0, stream>>>(d_C, d_A, d_B, dimsA.x, dimsB.x);
}
else {
MatrixMulCUDA<32><<<grid, threads, 0, stream>>>(d_C, d_A, d_B, dimsA.x, dimsB.x);
}
printf("done\n");
checkCudaErrors(cudaStreamSynchronize(stream));
// Record the start event
checkCudaErrors(cudaEventRecord(start, stream));
// Execute the kernel
int nIter = 300;
for (int j = 0; j < nIter; j++) {
if (block_size == 16) {
MatrixMulCUDA<16><<<grid, threads, 0, stream>>>(d_C, d_A, d_B, dimsA.x, dimsB.x);
}
else {
MatrixMulCUDA<32><<<grid, threads, 0, stream>>>(d_C, d_A, d_B, dimsA.x, dimsB.x);
}
}
// Record the stop event
checkCudaErrors(cudaEventRecord(stop, stream));
// Wait for the stop event to complete
checkCudaErrors(cudaEventSynchronize(stop));
float msecTotal = 0.0f;
checkCudaErrors(cudaEventElapsedTime(&msecTotal, start, stop));
// Compute and print the performance
float msecPerMatrixMul = msecTotal / nIter;
double flopsPerMatrixMul =
2.0 * static_cast<double>(dimsA.x) * static_cast<double>(dimsA.y) * static_cast<double>(dimsB.x);
double gigaFlops = (flopsPerMatrixMul * 1.0e-9f) / (msecPerMatrixMul / 1000.0f);
printf("Performance= %.2f GFlop/s, Time= %.3f msec, Size= %.0f Ops,"
" WorkgroupSize= %u threads/block\n",
gigaFlops,
msecPerMatrixMul,
flopsPerMatrixMul,
threads.x * threads.y);
// Copy result from device to host
checkCudaErrors(cudaMemcpyAsync(h_C, d_C, mem_size_C, cudaMemcpyDeviceToHost, stream));
checkCudaErrors(cudaStreamSynchronize(stream));
printf("Checking computed result for correctness: ");
bool correct = true;
// test relative error by the formula
// |<x, y>_cpu - <x,y>_gpu|/<|x|, |y|> < eps
double eps = 1.e-6; // machine zero
for (int i = 0; i < static_cast<int>(dimsC.x * dimsC.y); i++) {
double abs_err = fabs(h_C[i] - (dimsA.x * valB));
double dot_length = dimsA.x;
double abs_val = fabs(h_C[i]);
double rel_err = abs_err / abs_val / dot_length;
if (rel_err > eps) {
printf("Error! Matrix[%05d]=%.8f, ref=%.8f error term is > %E\n", i, h_C[i], dimsA.x * valB, eps);
correct = false;
}
}
printf("%s\n", correct ? "Result = PASS" : "Result = FAIL");
// Clean up memory
checkCudaErrors(cudaFreeHost(h_A));
checkCudaErrors(cudaFreeHost(h_B));
checkCudaErrors(cudaFreeHost(h_C));
checkCudaErrors(cudaFree(d_A));
checkCudaErrors(cudaFree(d_B));
checkCudaErrors(cudaFree(d_C));
checkCudaErrors(cudaEventDestroy(start));
checkCudaErrors(cudaEventDestroy(stop));
printf("\nNOTE: The CUDA Samples are not meant for performance "
"measurements. Results may vary when GPU Boost is enabled.\n");
if (correct) {
return EXIT_SUCCESS;
}
else {
return EXIT_FAILURE;
}
}
/**
* Program main
*/
int main(int argc, char **argv)
{
printf("[Matrix Multiply Using CUDA] - Starting...\n");
if (checkCmdLineFlag(argc, (const char **)argv, "help") || checkCmdLineFlag(argc, (const char **)argv, "?")) {
printf("Usage -device=n (n >= 0 for deviceID)\n");
printf(" -wA=WidthA -hA=HeightA (Width x Height of Matrix A)\n");
printf(" -wB=WidthB -hB=HeightB (Width x Height of Matrix B)\n");
printf(" Note: Outer matrix dimensions of A & B matrices"
" must be equal.\n");
exit(EXIT_SUCCESS);
}
// This will pick the best possible CUDA capable device, otherwise
// override the device ID based on input provided at the command line
int dev = findCudaDevice(argc, (const char **)argv);
int block_size = 32;
dim3 dimsA(5 * 2 * block_size, 5 * 2 * block_size, 1);
dim3 dimsB(5 * 4 * block_size, 5 * 2 * block_size, 1);
// width of Matrix A
if (checkCmdLineFlag(argc, (const char **)argv, "wA")) {
dimsA.x = getCmdLineArgumentInt(argc, (const char **)argv, "wA");
}
// height of Matrix A
if (checkCmdLineFlag(argc, (const char **)argv, "hA")) {
dimsA.y = getCmdLineArgumentInt(argc, (const char **)argv, "hA");
}
// width of Matrix B
if (checkCmdLineFlag(argc, (const char **)argv, "wB")) {
dimsB.x = getCmdLineArgumentInt(argc, (const char **)argv, "wB");
}
// height of Matrix B
if (checkCmdLineFlag(argc, (const char **)argv, "hB")) {
dimsB.y = getCmdLineArgumentInt(argc, (const char **)argv, "hB");
}
if (dimsA.x != dimsB.y) {
printf("Error: outer matrix dimensions must be equal. (%d != %d)\n", dimsA.x, dimsB.y);
exit(EXIT_FAILURE);
}
printf("MatrixA(%d,%d), MatrixB(%d,%d)\n", dimsA.x, dimsA.y, dimsB.x, dimsB.y);
checkCudaErrors(cudaProfilerStart());
int matrix_result = MatrixMultiply(argc, argv, block_size, dimsA, dimsB);
checkCudaErrors(cudaProfilerStop());
exit(matrix_result);
}

18
cpp/0_Introduction/matrixMulDrv/.vscode/c_cpp_properties.json vendored Normal file
View File

@@ -0,0 +1,18 @@
{
"configurations": [
{
"name": "Linux",
"includePath": [
"${workspaceFolder}/**",
"${workspaceFolder}/../../../Common"
],
"defines": [],
"compilerPath": "/usr/local/cuda/bin/nvcc",
"cStandard": "gnu17",
"cppStandard": "gnu++14",
"intelliSenseMode": "linux-gcc-x64",
"configurationProvider": "ms-vscode.makefile-tools"
}
],
"version": 4
}

7
cpp/0_Introduction/matrixMulDrv/.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,7 @@
{
"recommendations": [
"nvidia.nsight-vscode-edition",
"ms-vscode.cpptools",
"ms-vscode.makefile-tools"
]
}

72
cpp/0_Introduction/matrixMulDrv/CMakeLists.txt Normal file
View File

@@ -0,0 +1,72 @@
cmake_minimum_required(VERSION 3.20)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/Modules")
project(matrixMulDrv LANGUAGES C CXX CUDA)
find_package(CUDAToolkit REQUIRED)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(CMAKE_CUDA_ARCHITECTURES 75 80 86 87 89 90 100 110 120)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -Wno-deprecated-gpu-targets")
if(ENABLE_CUDA_DEBUG)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -G") # enable cuda-gdb (may significantly affect performance on some targets)
else()
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -lineinfo") # add line information to all builds for debug tools (exclusive to -G option)
endif()
# Include directories and libraries
include_directories(../../../Common)
# Source file
# Add target for matrixMulDrv
add_executable(matrixMulDrv matrixMulDrv.cpp)
target_compile_options(matrixMulDrv PRIVATE $<$<COMPILE_LANGUAGE:CUDA>:--extended-lambda>)
target_compile_features(matrixMulDrv PRIVATE cxx_std_17 cuda_std_17)
set_target_properties(matrixMulDrv PROPERTIES CUDA_SEPARABLE_COMPILATION ON)
target_include_directories(matrixMulDrv PRIVATE
${CUDAToolkit_INCLUDE_DIRS}
)
target_link_libraries(matrixMulDrv PUBLIC
CUDA::cuda_driver
)
set(CUDA_FATBIN_FILE "${CMAKE_CURRENT_BINARY_DIR}/matrixMul_kernel64.fatbin")
set(CUDA_KERNEL_SOURCE "${CMAKE_CURRENT_SOURCE_DIR}/matrixMul_kernel.cu")
# Construct GENCODE_FLAGS explicitly from CUDA architectures
set(GENCODE_FLAGS "")
foreach(arch ${CMAKE_CUDA_ARCHITECTURES})
list(APPEND GENCODE_FLAGS "-gencode=arch=compute_${arch},code=sm_${arch}")
endforeach()
if(CMAKE_SYSTEM_NAME STREQUAL "QNX")
set(INCLUDES_LIST)
foreach(dir ${CUDAToolkit_INCLUDE_DIRS})
list(APPEND INCLUDES_LIST "-I${dir}")
endforeach()
string(JOIN " " INCLUDES "${INCLUDES_LIST}")
endif()
add_custom_command(
OUTPUT ${CUDA_FATBIN_FILE}
COMMAND ${CMAKE_CUDA_COMPILER} ${INCLUDES} ${ALL_CCFLAGS} -Wno-deprecated-gpu-targets ${GENCODE_FLAGS} -o ${CUDA_FATBIN_FILE} -fatbin ${CUDA_KERNEL_SOURCE}
DEPENDS ${CUDA_KERNEL_SOURCE}
COMMENT "Building CUDA fatbin: ${CUDA_FATBIN_FILE}"
)
# Create a dummy target for fatbin generation
add_custom_target(generate_fatbin_matmulDrv ALL DEPENDS ${CUDA_FATBIN_FILE})
# Ensure matrixMulDrv depends on the fatbin
add_dependencies(matrixMulDrv generate_fatbin_matmulDrv)
# Include installation configuration
include(${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/InstallSamples.cmake)
setup_samples_install()

32
cpp/0_Introduction/matrixMulDrv/README.md Normal file
View File

@@ -0,0 +1,32 @@
# matrixMulDrv - Matrix Multiplication (CUDA Driver API Version)
## Description
This sample implements matrix multiplication and uses the new CUDA 4.0 kernel launch Driver API. It has been written for clarity of exposition to illustrate various CUDA programming principles, not with the goal of providing the most performant generic kernel for matrix multiplication. CUBLAS provides high-performance matrix multiplication.
## Key Concepts
CUDA Driver API, Matrix Multiply
## Supported SM Architectures
[SM 5.0 ](https://developer.nvidia.com/cuda-gpus) [SM 5.2 ](https://developer.nvidia.com/cuda-gpus) [SM 5.3 ](https://developer.nvidia.com/cuda-gpus) [SM 6.0 ](https://developer.nvidia.com/cuda-gpus) [SM 6.1 ](https://developer.nvidia.com/cuda-gpus) [SM 7.0 ](https://developer.nvidia.com/cuda-gpus) [SM 7.2 ](https://developer.nvidia.com/cuda-gpus) [SM 7.5 ](https://developer.nvidia.com/cuda-gpus) [SM 8.0 ](https://developer.nvidia.com/cuda-gpus) [SM 8.6 ](https://developer.nvidia.com/cuda-gpus) [SM 8.7 ](https://developer.nvidia.com/cuda-gpus) [SM 8.9 ](https://developer.nvidia.com/cuda-gpus) [SM 9.0 ](https://developer.nvidia.com/cuda-gpus)
## Supported OSes
Linux, Windows
## Supported CPU Architecture
x86_64, armv7l, aarch64
## CUDA APIs involved
### [CUDA Driver API](http://docs.nvidia.com/cuda/cuda-driver-api/index.html)
cuMemcpyDtoH, cuLaunchKernel, cuMemcpyHtoD, cuDeviceGetName, cuDeviceTotalMem, cuDeviceGetAttribute, cuModuleLoadData, cuOccupancyMaxPotentialBlockSize, cuMemAlloc, cuMemFree, cuCtxDestroy, cuModuleGetFunction, cuCtxCreate
## Prerequisites
Download and install the [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) for your corresponding platform.
## References (for more details)

40
cpp/0_Introduction/matrixMulDrv/matrixMul.h Normal file
View File

@@ -0,0 +1,40 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef _MATRIXMUL_H_
#define _MATRIXMUL_H_
// Matrix dimensions
// (chosen as multiples of the thread block size for simplicity)
#define WA (4 * block_size) // Matrix A width
#define HA (6 * block_size) // Matrix A height
#define WB (4 * block_size) // Matrix B width
#define HB WA // Matrix B height
#define WC WB // Matrix C width
#define HC HA // Matrix C height
#endif // _MATRIXMUL_H_

335
cpp/0_Introduction/matrixMulDrv/matrixMulDrv.cpp Normal file
View File

@@ -0,0 +1,335 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/* Matrix multiplication: C = A * B.
* Host code.
*
* This sample implements matrix multiplication using the CUDA driver API.
* It has been written for clarity of exposition to illustrate various CUDA
* programming principles, not with the goal of providing the most
* performant generic kernel for matrix multiplication.
*
* CUBLAS provides high-performance matrix multiplication.
* See also:
* V. Volkov and J. Demmel, "Benchmarking GPUs to tune dense linear algebra,"
* in Proc. 2008 ACM/IEEE Conf. on Supercomputing (SC '08),
* Piscataway, NJ: IEEE Press, 2008, pp. Art. 31:1-11.
*
* Volkov, V. 2010. Better performance at lower occupancy,
* GPU Technology Conference 2~010 (GTC 2010).
*
*/
// includes, system
#include <builtin_types.h>
#include <cstring>
#include <iostream>
#include <math.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
// includes, project, CUDA
#include <cstring>
#include <cuda.h>
#include <helper_cuda_drvapi.h>
#include <helper_image.h>
#include <helper_string.h>
#include <helper_timer.h>
#include <iostream>
#include <string>
#include "matrixMul.h"
////////////////////////////////////////////////////////////////////////////////
// declaration, forward
void runTest(int argc, char **argv);
void randomInit(float *, int);
extern "C" void computeGold(float *, const float *, const float *, unsigned int, unsigned int, unsigned int);
static int initCUDA(int argc, char **argv, CUfunction *pMatrixMul, int *blk_size);
#ifndef FATBIN_FILE
#define FATBIN_FILE "matrixMul_kernel64.fatbin"
#endif
////////////////////////////////////////////////////////////////////////////////
// Globals
////////////////////////////////////////////////////////////////////////////////
CUdevice cuDevice;
CUcontext cuContext;
CUmodule cuModule;
size_t totalGlobalMem;
const char *sSDKsample = "matrixMulDrv (Driver API)";
void constantInit(float *data, int size, float val)
{
for (int i = 0; i < size; ++i) {
data[i] = val;
}
}
////////////////////////////////////////////////////////////////////////////////
// Program main
////////////////////////////////////////////////////////////////////////////////
int main(int argc, char **argv)
{
printf("[ %s ]\n", sSDKsample);
runTest(argc, argv);
}
////////////////////////////////////////////////////////////////////////////////
//! Run a simple test for CUDA
////////////////////////////////////////////////////////////////////////////////
void runTest(int argc, char **argv)
{
// initialize CUDA
CUfunction matrixMul = NULL;
int block_size = 0;
initCUDA(argc, argv, &matrixMul, &block_size);
// set seed for rand()
srand(2006);
// allocate host memory for matrices A and B
unsigned int size_A = WA * HA;
unsigned int mem_size_A = sizeof(float) * size_A;
float *h_A = reinterpret_cast<float *>(malloc(mem_size_A));
unsigned int size_B = WB * HB;
unsigned int mem_size_B = sizeof(float) * size_B;
float *h_B = reinterpret_cast<float *>(malloc(mem_size_B));
// initialize host memory
const float valB = 0.01f;
constantInit(h_A, size_A, 1.0f);
constantInit(h_B, size_B, valB);
// allocate device memory
CUdeviceptr d_A;
checkCudaErrors(cuMemAlloc(&d_A, mem_size_A));
CUdeviceptr d_B;
checkCudaErrors(cuMemAlloc(&d_B, mem_size_B));
// copy host memory to device
checkCudaErrors(cuMemcpyHtoD(d_A, h_A, mem_size_A));
checkCudaErrors(cuMemcpyHtoD(d_B, h_B, mem_size_B));
// allocate device memory for result
size_t size_C = WC * HC;
size_t mem_size_C = sizeof(float) * size_C;
CUdeviceptr d_C;
checkCudaErrors(cuMemAlloc(&d_C, mem_size_C));
// allocate mem for the result on host side
float *h_C = reinterpret_cast<float *>(malloc(mem_size_C));
// create and start timer
StopWatchInterface *timer = NULL;
sdkCreateTimer(&timer);
// start the timer
sdkStartTimer(&timer);
// There are two ways to launch CUDA kernels via the Driver API.
// In this CUDA Sample, we illustrate both ways to pass parameters
// and specify parameters. By default we use the simpler method.
dim3 block(block_size, block_size, 1);
dim3 grid(WC / block_size, HC / block_size, 1);
if (1) {
// This is the new CUDA 4.0 API for Kernel Parameter passing and Kernel
// Launching (simplier method)
size_t Matrix_Width_A = (size_t)WA;
size_t Matrix_Width_B = (size_t)WB;
void *args[5] = {&d_C, &d_A, &d_B, &Matrix_Width_A, &Matrix_Width_B};
// new CUDA 4.0 Driver API Kernel launch call
checkCudaErrors(cuLaunchKernel(matrixMul,
grid.x,
grid.y,
grid.z,
block.x,
block.y,
block.z,
2 * block_size * block_size * sizeof(float),
NULL,
args,
NULL));
}
else {
// This is the new CUDA 4.0 API for Kernel Parameter passing and Kernel
// Launching (advanced method)
int offset = 0;
char argBuffer[256];
// pass in launch parameters (not actually de-referencing CUdeviceptr).
// CUdeviceptr is storing the value of the parameters
*(reinterpret_cast<CUdeviceptr *>(&argBuffer[offset])) = d_C;
offset += sizeof(d_C);
*(reinterpret_cast<CUdeviceptr *>(&argBuffer[offset])) = d_A;
offset += sizeof(d_A);
*(reinterpret_cast<CUdeviceptr *>(&argBuffer[offset])) = d_B;
offset += sizeof(d_B);
size_t Matrix_Width_A = (size_t)WA;
size_t Matrix_Width_B = (size_t)WB;
*(reinterpret_cast<CUdeviceptr *>(&argBuffer[offset])) = Matrix_Width_A;
offset += sizeof(Matrix_Width_A);
*(reinterpret_cast<CUdeviceptr *>(&argBuffer[offset])) = Matrix_Width_B;
offset += sizeof(Matrix_Width_B);
void *kernel_launch_config[5] = {
CU_LAUNCH_PARAM_BUFFER_POINTER, argBuffer, CU_LAUNCH_PARAM_BUFFER_SIZE, &offset, CU_LAUNCH_PARAM_END};
// new CUDA 4.0 Driver API Kernel launch call
checkCudaErrors(cuLaunchKernel(matrixMul,
grid.x,
grid.y,
grid.z,
block.x,
block.y,
block.z,
2 * block_size * block_size * sizeof(float),
NULL,
NULL,
reinterpret_cast<void **>(&kernel_launch_config)));
}
// copy result from device to host
checkCudaErrors(cuMemcpyDtoH(reinterpret_cast<void *>(h_C), d_C, mem_size_C));
// stop and destroy timer
sdkStopTimer(&timer);
printf("Processing time: %f (ms)\n", sdkGetTimerValue(&timer));
sdkDeleteTimer(&timer);
printf("Checking computed result for correctness: ");
bool correct = true;
for (int i = 0; i < static_cast<int>(WC * HC); i++) {
if (fabs(h_C[i] - (WA * valB)) > 1e-5) {
printf("Error! Matrix[%05d]=%.8f, ref=%.8f error term is > 1e-5\n", i, h_C[i], WA * valB);
correct = false;
}
}
printf("%s\n", correct ? "Result = PASS" : "Result = FAIL");
printf("\nNOTE: The CUDA Samples are not meant for performance measurements. "
"Results may vary when GPU Boost is enabled.\n");
// clean up memory
free(h_A);
free(h_B);
free(h_C);
checkCudaErrors(cuMemFree(d_A));
checkCudaErrors(cuMemFree(d_B));
checkCudaErrors(cuMemFree(d_C));
checkCudaErrors(cuModuleUnload(cuModule));
checkCudaErrors(cuCtxDestroy(cuContext));
}
// Allocates a matrix with random float entries.
void randomInit(float *data, int size)
{
for (int i = 0; i < size; ++i) {
data[i] = rand() / static_cast<float>(RAND_MAX);
}
}
static int initCUDA(int argc, char **argv, CUfunction *pMatrixMul, int *blk_size)
{
CUfunction cuFunction = 0;
int major = 0, minor = 0;
char deviceName[100];
CUctxCreateParams ctxCreateParams = {};
cuDevice = findCudaDeviceDRV(argc, (const char **)argv);
// get compute capabilities and the devicename
checkCudaErrors(cuDeviceGetAttribute(&major, CU_DEVICE_ATTRIBUTE_COMPUTE_CAPABILITY_MAJOR, cuDevice));
checkCudaErrors(cuDeviceGetAttribute(&minor, CU_DEVICE_ATTRIBUTE_COMPUTE_CAPABILITY_MINOR, cuDevice));
checkCudaErrors(cuDeviceGetName(deviceName, sizeof(deviceName), cuDevice));
printf("> GPU Device has SM %d.%d compute capability\n", major, minor);
checkCudaErrors(cuDeviceTotalMem(&totalGlobalMem, cuDevice));
printf(" Total amount of global memory: %llu bytes\n", (long long unsigned int)totalGlobalMem);
checkCudaErrors(cuCtxCreate(&cuContext, &ctxCreateParams, 0, cuDevice));
// first search for the module path before we load the results
std::string module_path;
std::ostringstream fatbin;
if (!findFatbinPath(FATBIN_FILE, module_path, argv, fatbin)) {
exit(EXIT_FAILURE);
}
else {
printf("> initCUDA loading module: <%s>\n", module_path.c_str());
}
if (!fatbin.str().size()) {
printf("fatbin file empty. exiting..\n");
exit(EXIT_FAILURE);
}
// Create module from binary file (FATBIN)
checkCudaErrors(cuModuleLoadData(&cuModule, fatbin.str().c_str()));
// select the suitable kernel function
const char *kernels[] = {"matrixMul_bs32_64bit", "matrixMul_bs16_64bit", "matrixMul_bs8_64bit"};
int idx = 0;
int block_size = 32;
while (idx < 3) {
int threadsPerBlock = 0;
int blocksPerGrid = 0;
checkCudaErrors(cuModuleGetFunction(&cuFunction, cuModule, kernels[idx]));
checkCudaErrors(cuOccupancyMaxPotentialBlockSize(
&blocksPerGrid, &threadsPerBlock, cuFunction, 0, 2 * block_size * block_size * sizeof(float), 0));
if (block_size * block_size <= threadsPerBlock) {
printf("> %d block size selected\n", block_size);
break;
}
else {
block_size /= 2;
}
idx++;
}
*pMatrixMul = cuFunction;
*blk_size = block_size;
return 0;
}

128
cpp/0_Introduction/matrixMulDrv/matrixMul_kernel.cu Normal file
View File

@@ -0,0 +1,128 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/* Matrix multiplication: C = A * B.
* Device code.
*/
#ifndef _MATRIXMUL_KERNEL_H_
#define _MATRIXMUL_KERNEL_H_
#include <stdio.h>
#define AS(i, j) As[i][j]
#define BS(i, j) Bs[i][j]
////////////////////////////////////////////////////////////////////////////////
//! Matrix multiplication on the device: C = A * B
//! wA is A's width and wB is B's width
////////////////////////////////////////////////////////////////////////////////
template <int block_size, typename size_type>
__device__ void matrixMul(float *C, float *A, float *B, size_type wA, size_type wB)
{
// Block index
size_type bx = blockIdx.x;
size_type by = blockIdx.y;
// Thread index
size_type tx = threadIdx.x;
size_type ty = threadIdx.y;
// Index of the first sub-matrix of A processed by the block
size_type aBegin = wA * block_size * by;
// Index of the last sub-matrix of A processed by the block
size_type aEnd = aBegin + wA - 1;
// Step size used to iterate through the sub-matrices of A
size_type aStep = block_size;
// Index of the first sub-matrix of B processed by the block
size_type bBegin = block_size * bx;
// Step size used to iterate through the sub-matrices of B
size_type bStep = block_size * wB;
// Csub is used to store the element of the block sub-matrix
// that is computed by the thread
float Csub = 0;
// Loop over all the sub-matrices of A and B
// required to compute the block sub-matrix
for (size_type a = aBegin, b = bBegin; a <= aEnd; a += aStep, b += bStep) {
// Declaration of the shared memory array As used to
// store the sub-matrix of A
__shared__ float As[block_size][block_size];
// Declaration of the shared memory array Bs used to
// store the sub-matrix of B
__shared__ float Bs[block_size][block_size];
// Load the matrices from device memory
// to shared memory; each thread loads
// one element of each matrix
AS(ty, tx) = A[a + wA * ty + tx];
BS(ty, tx) = B[b + wB * ty + tx];
// Synchronize to make sure the matrices are loaded
__syncthreads();
// Multiply the two matrices together;
// each thread computes one element
// of the block sub-matrix
#pragma unroll
for (size_type k = 0; k < block_size; ++k)
Csub += AS(ty, k) * BS(k, tx);
// Synchronize to make sure that the preceding
// computation is done before loading two new
// sub-matrices of A and B in the next iteration
__syncthreads();
}
// Write the block sub-matrix to device memory;
// each thread writes one element
size_type c = wB * block_size * by + block_size * bx;
C[c + wB * ty + tx] = Csub;
}
// C wrappers around our template kernel
extern "C" __global__ void matrixMul_bs8_64bit(float *C, float *A, float *B, size_t wA, size_t wB)
{
matrixMul<8, size_t>(C, A, B, wA, wB);
}
extern "C" __global__ void matrixMul_bs16_64bit(float *C, float *A, float *B, size_t wA, size_t wB)
{
matrixMul<16, size_t>(C, A, B, wA, wB);
}
extern "C" __global__ void matrixMul_bs32_64bit(float *C, float *A, float *B, size_t wA, size_t wB)
{
matrixMul<32, size_t>(C, A, B, wA, wB);
}
#endif // #ifndef _MATRIXMUL_KERNEL_H_

18
cpp/0_Introduction/matrixMulDynlinkJIT/.vscode/c_cpp_properties.json vendored Normal file
View File

@@ -0,0 +1,18 @@
{
"configurations": [
{
"name": "Linux",
"includePath": [
"${workspaceFolder}/**",
"${workspaceFolder}/../../../Common"
],
"defines": [],
"compilerPath": "/usr/local/cuda/bin/nvcc",
"cStandard": "gnu17",
"cppStandard": "gnu++14",
"intelliSenseMode": "linux-gcc-x64",
"configurationProvider": "ms-vscode.makefile-tools"
}
],
"version": 4
}

7
cpp/0_Introduction/matrixMulDynlinkJIT/.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,7 @@
{
"recommendations": [
"nvidia.nsight-vscode-edition",
"ms-vscode.cpptools",
"ms-vscode.makefile-tools"
]
}

51
cpp/0_Introduction/matrixMulDynlinkJIT/CMakeLists.txt Normal file
View File

@@ -0,0 +1,51 @@
cmake_minimum_required(VERSION 3.20)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/Modules")
project(matrixMulDynlinkJIT LANGUAGES C CXX CUDA)
find_package(CUDAToolkit REQUIRED)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(CMAKE_CUDA_ARCHITECTURES 75 80 86 87 89 90 100 110 120)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -Wno-deprecated-gpu-targets")
if(ENABLE_CUDA_DEBUG)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -G") # enable cuda-gdb (may significantly affect performance on some targets)
else()
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -lineinfo") # add line information to all builds for debug tools (exclusive to -G option)
endif()
# Include directories and libraries
include_directories(../../../Common)
# Source file
# Add target for matrixMulDynlinkJIT
add_executable(matrixMulDynlinkJIT cuda_drvapi_dynlink.c matrixMulDynlinkJIT.cpp matrixMul_gold.cpp matrixMul_kernel_32_ptxdump.c matrixMul_kernel_64_ptxdump.c)
target_compile_options(matrixMulDynlinkJIT PRIVATE $<$<COMPILE_LANGUAGE:CUDA>:--extended-lambda>)
target_compile_features(matrixMulDynlinkJIT PRIVATE cxx_std_17 cuda_std_17)
set_target_properties(matrixMulDynlinkJIT PROPERTIES
CUDA_SEPARABLE_COMPILATION ON
POSITION_INDEPENDENT_CODE OFF
)
# Only add -no-pie for GCC or Clang
if (CMAKE_CXX_COMPILER_ID STREQUAL "GNU" OR CMAKE_CXX_COMPILER_ID STREQUAL "Clang")
set(CMAKE_EXE_LINKER_FLAGS "${CMAKE_EXE_LINKER_FLAGS} -no-pie")
endif()
target_link_libraries(matrixMulDynlinkJIT PUBLIC
CUDA::cudart
CUDA::cuda_driver
)
if(${CMAKE_SYSTEM_NAME} STREQUAL "Linux")
target_link_libraries(matrixMulDynlinkJIT PUBLIC dl)
endif()
# Include installation configuration
include(${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/InstallSamples.cmake)
setup_samples_install()

32
cpp/0_Introduction/matrixMulDynlinkJIT/README.md Normal file
View File

@@ -0,0 +1,32 @@
# matrixMulDynlinkJIT - Matrix Multiplication (CUDA Driver API version with Dynamic Linking Version)
## Description
This sample revisits matrix multiplication using the CUDA driver API. It demonstrates how to link to CUDA driver at runtime and how to use JIT (just-in-time) compilation from PTX code. It has been written for clarity of exposition to illustrate various CUDA programming principles, not with the goal of providing the most performant generic kernel for matrix multiplication. CUBLAS provides high-performance matrix multiplication.
## Key Concepts
CUDA Driver API, CUDA Dynamically Linked Library
## Supported SM Architectures
[SM 5.0 ](https://developer.nvidia.com/cuda-gpus) [SM 5.2 ](https://developer.nvidia.com/cuda-gpus) [SM 5.3 ](https://developer.nvidia.com/cuda-gpus) [SM 6.0 ](https://developer.nvidia.com/cuda-gpus) [SM 6.1 ](https://developer.nvidia.com/cuda-gpus) [SM 7.0 ](https://developer.nvidia.com/cuda-gpus) [SM 7.2 ](https://developer.nvidia.com/cuda-gpus) [SM 7.5 ](https://developer.nvidia.com/cuda-gpus) [SM 8.0 ](https://developer.nvidia.com/cuda-gpus) [SM 8.6 ](https://developer.nvidia.com/cuda-gpus) [SM 8.7 ](https://developer.nvidia.com/cuda-gpus) [SM 8.9 ](https://developer.nvidia.com/cuda-gpus) [SM 9.0 ](https://developer.nvidia.com/cuda-gpus)
## Supported OSes
Linux, Windows
## Supported CPU Architecture
x86_64, armv7l
## CUDA APIs involved
### [CUDA Driver API](http://docs.nvidia.com/cuda/cuda-driver-api/index.html)
cuMemcpyDtoH, cuDeviceGetName, cuParamSeti, cuModuleLoadDataEx, cuModuleGetFunction, cuLaunchGrid, cuFuncSetSharedSize, cuMemFree, cuParamSetSize, cuParamSetv, cuInit, cuMemcpyHtoD, cuLaunchKernel, cuDeviceGet, cuFuncSetBlockShape, cuCtxDestroy, cuDeviceGetCount, cuDeviceComputeCapability, cuCtxSynchronize, cuMemAlloc, cuCtxCreate
## Prerequisites
Download and install the [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) for your corresponding platform.
## References (for more details)

668
cpp/0_Introduction/matrixMulDynlinkJIT/cuda_drvapi_dynlink.c Normal file
View File

@@ -0,0 +1,668 @@
/*
* Copyright 1993-2014 NVIDIA Corporation. All rights reserved.
*
* NVIDIA Corporation and its licensors retain all intellectual property and
* proprietary rights in and to this software and related documentation.
* Any use, reproduction, disclosure, or distribution of this software
* and related documentation without an express license agreement from
* NVIDIA Corporation is strictly prohibited.
*
* Please refer to the applicable NVIDIA end user license agreement (EULA)
* associated with this source code for terms and conditions that govern
* your use of this NVIDIA software.
*
*/
// With these flags defined, this source file will dynamically
// load the corresponding functions. Disabled by default.
// #define CUDA_INIT_D3D9
// #define CUDA_INIT_D3D10
// #define CUDA_INIT_D3D11
// #define CUDA_INIT_OPENGL
#include "cuda_drvapi_dynlink.h"
#include <stdio.h>
tcuInit *_cuInit;
tcuDriverGetVersion *cuDriverGetVersion;
tcuDeviceGet *cuDeviceGet;
tcuDeviceGetCount *cuDeviceGetCount;
tcuDeviceGetName *cuDeviceGetName;
tcuDeviceComputeCapability *cuDeviceComputeCapability;
tcuDeviceTotalMem *cuDeviceTotalMem;
tcuDeviceGetProperties *cuDeviceGetProperties;
tcuDeviceGetAttribute *cuDeviceGetAttribute;
tcuGetErrorString *cuGetErrorString;
tcuCtxCreate *cuCtxCreate;
tcuCtxDestroy *cuCtxDestroy;
tcuCtxAttach *cuCtxAttach;
tcuCtxDetach *cuCtxDetach;
tcuCtxPushCurrent *cuCtxPushCurrent;
tcuCtxPopCurrent *cuCtxPopCurrent;
tcuCtxGetCurrent *cuCtxGetCurrent;
tcuCtxSetCurrent *cuCtxSetCurrent;
tcuCtxGetDevice *cuCtxGetDevice;
tcuCtxSynchronize *cuCtxSynchronize;
tcuModuleLoad *cuModuleLoad;
tcuModuleLoadData *cuModuleLoadData;
tcuModuleLoadDataEx *cuModuleLoadDataEx;
tcuModuleLoadFatBinary *cuModuleLoadFatBinary;
tcuModuleUnload *cuModuleUnload;
tcuModuleGetFunction *cuModuleGetFunction;
tcuModuleGetGlobal *cuModuleGetGlobal;
tcuModuleGetTexRef *cuModuleGetTexRef;
tcuModuleGetSurfRef *cuModuleGetSurfRef;
tcuMemGetInfo *cuMemGetInfo;
tcuMemAlloc *cuMemAlloc;
tcuMemAllocPitch *cuMemAllocPitch;
tcuMemFree *cuMemFree;
tcuMemGetAddressRange *cuMemGetAddressRange;
tcuMemAllocHost *cuMemAllocHost;
tcuMemFreeHost *cuMemFreeHost;
tcuMemHostAlloc *cuMemHostAlloc;
tcuMemHostGetFlags *cuMemHostGetFlags;
tcuMemHostGetDevicePointer *cuMemHostGetDevicePointer;
tcuDeviceGetByPCIBusId *cuDeviceGetByPCIBusId;
tcuDeviceGetPCIBusId *cuDeviceGetPCIBusId;
tcuIpcGetEventHandle *cuIpcGetEventHandle;
tcuIpcOpenEventHandle *cuIpcOpenEventHandle;
tcuIpcGetMemHandle *cuIpcGetMemHandle;
tcuIpcOpenMemHandle *cuIpcOpenMemHandle;
tcuIpcCloseMemHandle *cuIpcCloseMemHandle;
tcuMemHostRegister *cuMemHostRegister;
tcuMemHostUnregister *cuMemHostUnregister;
tcuMemcpyHtoD *cuMemcpyHtoD;
tcuMemcpyDtoH *cuMemcpyDtoH;
tcuMemcpyDtoD *cuMemcpyDtoD;
tcuMemcpyDtoA *cuMemcpyDtoA;
tcuMemcpyAtoD *cuMemcpyAtoD;
tcuMemcpyHtoA *cuMemcpyHtoA;
tcuMemcpyAtoH *cuMemcpyAtoH;
tcuMemcpyAtoA *cuMemcpyAtoA;
tcuMemcpy2D *cuMemcpy2D;
tcuMemcpy2DUnaligned *cuMemcpy2DUnaligned;
tcuMemcpy3D *cuMemcpy3D;
tcuMemcpyHtoDAsync *cuMemcpyHtoDAsync;
tcuMemcpyDtoHAsync *cuMemcpyDtoHAsync;
tcuMemcpyDtoDAsync *cuMemcpyDtoDAsync;
tcuMemcpyHtoAAsync *cuMemcpyHtoAAsync;
tcuMemcpyAtoHAsync *cuMemcpyAtoHAsync;
tcuMemcpy2DAsync *cuMemcpy2DAsync;
tcuMemcpy3DAsync *cuMemcpy3DAsync;
tcuMemcpy *cuMemcpy;
tcuMemcpyPeer *cuMemcpyPeer;
tcuMemsetD8 *cuMemsetD8;
tcuMemsetD16 *cuMemsetD16;
tcuMemsetD32 *cuMemsetD32;
tcuMemsetD2D8 *cuMemsetD2D8;
tcuMemsetD2D16 *cuMemsetD2D16;
tcuMemsetD2D32 *cuMemsetD2D32;
tcuFuncSetBlockShape *cuFuncSetBlockShape;
tcuFuncSetSharedSize *cuFuncSetSharedSize;
tcuFuncGetAttribute *cuFuncGetAttribute;
tcuFuncSetCacheConfig *cuFuncSetCacheConfig;
tcuFuncSetSharedMemConfig *cuFuncSetSharedMemConfig;
tcuLaunchKernel *cuLaunchKernel;
tcuArrayCreate *cuArrayCreate;
tcuArrayGetDescriptor *cuArrayGetDescriptor;
tcuArrayDestroy *cuArrayDestroy;
tcuArray3DCreate *cuArray3DCreate;
tcuArray3DGetDescriptor *cuArray3DGetDescriptor;
tcuTexRefCreate *cuTexRefCreate;
tcuTexRefDestroy *cuTexRefDestroy;
tcuTexRefSetArray *cuTexRefSetArray;
tcuTexRefSetAddress *cuTexRefSetAddress;
tcuTexRefSetAddress2D *cuTexRefSetAddress2D;
tcuTexRefSetFormat *cuTexRefSetFormat;
tcuTexRefSetAddressMode *cuTexRefSetAddressMode;
tcuTexRefSetFilterMode *cuTexRefSetFilterMode;
tcuTexRefSetFlags *cuTexRefSetFlags;
tcuTexRefGetAddress *cuTexRefGetAddress;
tcuTexRefGetArray *cuTexRefGetArray;
tcuTexRefGetAddressMode *cuTexRefGetAddressMode;
tcuTexRefGetFilterMode *cuTexRefGetFilterMode;
tcuTexRefGetFormat *cuTexRefGetFormat;
tcuTexRefGetFlags *cuTexRefGetFlags;
tcuSurfRefSetArray *cuSurfRefSetArray;
tcuSurfRefGetArray *cuSurfRefGetArray;
tcuParamSetSize *cuParamSetSize;
tcuParamSeti *cuParamSeti;
tcuParamSetf *cuParamSetf;
tcuParamSetv *cuParamSetv;
tcuParamSetTexRef *cuParamSetTexRef;
tcuLaunch *cuLaunch;
tcuLaunchGrid *cuLaunchGrid;
tcuLaunchGridAsync *cuLaunchGridAsync;
tcuEventCreate *cuEventCreate;
tcuEventRecord *cuEventRecord;
tcuEventQuery *cuEventQuery;
tcuEventSynchronize *cuEventSynchronize;
tcuEventDestroy *cuEventDestroy;
tcuEventElapsedTime *cuEventElapsedTime;
tcuStreamCreate *cuStreamCreate;
tcuStreamWaitEvent *cuStreamWaitEvent;
tcuStreamAddCallback *cuStreamAddCallback;
tcuStreamQuery *cuStreamQuery;
tcuStreamSynchronize *cuStreamSynchronize;
tcuStreamDestroy *cuStreamDestroy;
tcuGraphicsUnregisterResource *cuGraphicsUnregisterResource;
tcuGraphicsSubResourceGetMappedArray *cuGraphicsSubResourceGetMappedArray;
tcuGraphicsResourceGetMappedPointer *cuGraphicsResourceGetMappedPointer;
tcuGraphicsResourceSetMapFlags *cuGraphicsResourceSetMapFlags;
tcuGraphicsMapResources *cuGraphicsMapResources;
tcuGraphicsUnmapResources *cuGraphicsUnmapResources;
tcuGetExportTable *cuGetExportTable;
tcuCtxSetLimit *cuCtxSetLimit;
tcuCtxGetLimit *cuCtxGetLimit;
tcuCtxGetCacheConfig *cuCtxGetCacheConfig;
tcuCtxSetCacheConfig *cuCtxSetCacheConfig;
tcuCtxGetSharedMemConfig *cuCtxGetSharedMemConfig;
tcuCtxSetSharedMemConfig *cuCtxSetSharedMemConfig;
tcuCtxGetApiVersion *cuCtxGetApiVersion;
tcuMipmappedArrayCreate *cuMipmappedArrayCreate;
tcuMipmappedArrayGetLevel *cuMipmappedArrayGetLevel;
tcuMipmappedArrayDestroy *cuMipmappedArrayDestroy;
tcuProfilerStop *cuProfilerStop;
#ifdef CUDA_INIT_D3D9
// D3D9/CUDA interop (CUDA 1.x compatible API). These functions
// are deprecated; please use the ones below
tcuD3D9Begin *cuD3D9Begin;
tcuD3D9End *cuD3DEnd;
tcuD3D9RegisterVertexBuffer *cuD3D9RegisterVertexBuffer;
tcuD3D9MapVertexBuffer *cuD3D9MapVertexBuffer;
tcuD3D9UnmapVertexBuffer *cuD3D9UnmapVertexBuffer;
tcuD3D9UnregisterVertexBuffer *cuD3D9UnregisterVertexBuffer;
// D3D9/CUDA interop (CUDA 2.x compatible)
tcuD3D9GetDirect3DDevice *cuD3D9GetDirect3DDevice;
tcuD3D9RegisterResource *cuD3D9RegisterResource;
tcuD3D9UnregisterResource *cuD3D9UnregisterResource;
tcuD3D9MapResources *cuD3D9MapResources;
tcuD3D9UnmapResources *cuD3D9UnmapResources;
tcuD3D9ResourceSetMapFlags *cuD3D9ResourceSetMapFlags;
tcuD3D9ResourceGetSurfaceDimensions *cuD3D9ResourceGetSurfaceDimensions;
tcuD3D9ResourceGetMappedArray *cuD3D9ResourceGetMappedArray;
tcuD3D9ResourceGetMappedPointer *cuD3D9ResourceGetMappedPointer;
tcuD3D9ResourceGetMappedSize *cuD3D9ResourceGetMappedSize;
tcuD3D9ResourceGetMappedPitch *cuD3D9ResourceGetMappedPitch;
// D3D9/CUDA interop (CUDA 2.0+)
tcuD3D9GetDevice *cuD3D9GetDevice;
tcuD3D9CtxCreate *cuD3D9CtxCreate;
tcuGraphicsD3D9RegisterResource *cuGraphicsD3D9RegisterResource;
#endif
#ifdef CUDA_INIT_D3D10
// D3D10/CUDA interop (CUDA 3.0+)
tcuD3D10GetDevice *cuD3D10GetDevice;
tcuD3D10CtxCreate *cuD3D10CtxCreate;
tcuGraphicsD3D10RegisterResource *cuGraphicsD3D10RegisterResource;
#endif
#ifdef CUDA_INIT_D3D11
// D3D11/CUDA interop (CUDA 3.0+)
tcuD3D11GetDevice *cuD3D11GetDevice;
tcuD3D11CtxCreate *cuD3D11CtxCreate;
tcuGraphicsD3D11RegisterResource *cuGraphicsD3D11RegisterResource;
#endif
// GL/CUDA interop
#ifdef CUDA_INIT_OPENGL
tcuGLCtxCreate *cuGLCtxCreate;
tcuGraphicsGLRegisterBuffer *cuGraphicsGLRegisterBuffer;
tcuGraphicsGLRegisterImage *cuGraphicsGLRegisterImage;
#if defined(WIN32) || defined(_WIN32) || defined(WIN64) || defined(_WIN64)
tcuWGLGetDevice *cuWGLGetDevice;
#endif
#endif
#define STRINGIFY(X) #X
#if defined(WIN32) || defined(_WIN32) || defined(WIN64) || defined(_WIN64)
#include <Windows.h>
#ifdef UNICODE
static LPCWSTR __CudaLibName = L"nvcuda.dll";
#else
static LPCSTR __CudaLibName = "nvcuda.dll";
#endif
typedef HMODULE CUDADRIVER;
static CUresult LOAD_LIBRARY(CUDADRIVER *pInstance)
{
*pInstance = LoadLibrary(__CudaLibName);
if (*pInstance == NULL) {
printf("LoadLibrary \"%s\" failed!\n", __CudaLibName);
exit(EXIT_FAILURE);
}
return CUDA_SUCCESS;
}
CUresult GET_DRIVER_HANDLE(CUDADRIVER *pInstance)
{
*pInstance = GetModuleHandle(__CudaLibName);
if (*pInstance) {
return CUDA_SUCCESS;
}
else {
return CUDA_ERROR_UNKNOWN;
}
}
#define GET_PROC_EX(name, alias, required) \
alias = (t##name *)GetProcAddress(CudaDrvLib, #name); \
if (alias == NULL && required) { \
printf("Failed to find required function \"%s\" in %s\n", #name, __CudaLibName); \
return CUDA_ERROR_UNKNOWN; \
}
#define GET_PROC_EX_V2(name, alias, required) \
alias = (t##name *)GetProcAddress(CudaDrvLib, STRINGIFY(name##_v2)); \
if (alias == NULL && required) { \
printf("Failed to find required function \"%s\" in %s\n", STRINGIFY(name##_v2), __CudaLibName); \
return CUDA_ERROR_UNKNOWN; \
}
#define GET_PROC_EX_V3(name, alias, required) \
alias = (t##name *)GetProcAddress(CudaDrvLib, STRINGIFY(name##_v3)); \
if (alias == NULL && required) { \
printf("Failed to find required function \"%s\" in %s\n", STRINGIFY(name##_v3), __CudaLibName); \
return CUDA_ERROR_UNKNOWN; \
}
#define GET_PROC_ERROR_FUNCTIONS(name, alias, required) \
alias = (t##name *)GetProcAddress(CudaDrvLib, #name); \
if (alias == NULL && required) { \
printf("Failed to find error function \"%s\" in %s\n", #name, __CudaLibName); \
exit(EXIT_FAILURE); \
}
#elif defined(__unix__) || defined(__QNX__) || defined(__APPLE__) || defined(__MACOSX)
#include <dlfcn.h>
#if defined(__APPLE__) || defined(__MACOSX)
static char __CudaLibName[] = "/usr/local/cuda/lib/libcuda.dylib";
#elif defined(__ANDROID__)
#if defined(__aarch64__)
static char __CudaLibName[] = "/system/vendor/lib64/libcuda.so";
#elif defined(__arm__)
static char __CudaLibName[] = "/system/vendor/lib/libcuda.so";
#endif
#else
static char __CudaLibName[] = "libcuda.so.1";
#endif
typedef void *CUDADRIVER;
static CUresult LOAD_LIBRARY(CUDADRIVER *pInstance)
{
*pInstance = dlopen(__CudaLibName, RTLD_NOW);
if (*pInstance == NULL) {
printf("dlopen \"%s\" failed!\n", __CudaLibName);
exit(EXIT_FAILURE);
}
return CUDA_SUCCESS;
}
CUresult GET_DRIVER_HANDLE(CUDADRIVER *pInstance)
{
*pInstance = dlopen(__CudaLibName, RTLD_LAZY);
if (*pInstance) {
return CUDA_SUCCESS;
}
else {
return CUDA_ERROR_UNKNOWN;
}
}
#define GET_PROC_EX(name, alias, required) \
alias = (t##name *)dlsym(CudaDrvLib, #name); \
if (alias == NULL && required) { \
printf("Failed to find required function \"%s\" in %s\n", #name, __CudaLibName); \
return CUDA_ERROR_UNKNOWN; \
}
#define GET_PROC_EX_V2(name, alias, required) \
alias = (t##name *)dlsym(CudaDrvLib, STRINGIFY(name##_v2)); \
if (alias == NULL && required) { \
printf("Failed to find required function \"%s\" in %s\n", STRINGIFY(name##_v2), __CudaLibName); \
return CUDA_ERROR_UNKNOWN; \
}
#define GET_PROC_EX_V3(name, alias, required) \
alias = (t##name *)dlsym(CudaDrvLib, STRINGIFY(name##_v3)); \
if (alias == NULL && required) { \
printf("Failed to find required function \"%s\" in %s\n", STRINGIFY(name##_v3), __CudaLibName); \
return CUDA_ERROR_UNKNOWN; \
}
#define GET_PROC_ERROR_FUNCTIONS(name, alias, required) \
alias = (t##name *)dlsym(CudaDrvLib, #name); \
if (alias == NULL && required) { \
printf("Failed to find error function \"%s\" in %s\n", #name, __CudaLibName); \
exit(EXIT_FAILURE); \
}
#else
#error unsupported platform
#endif
#define CHECKED_CALL(call) \
do { \
CUresult result = (call); \
if (CUDA_SUCCESS != result) { \
return result; \
} \
} while (0)
#define GET_PROC_REQUIRED(name) GET_PROC_EX(name, name, 1)
#define GET_PROC_OPTIONAL(name) GET_PROC_EX(name, name, 0)
#define GET_PROC(name) GET_PROC_REQUIRED(name)
#define GET_PROC_V2(name) GET_PROC_EX_V2(name, name, 1)
#define GET_PROC_V3(name) GET_PROC_EX_V3(name, name, 1)
CUresult INIT_ERROR_FUNCTIONS(void)
{
CUDADRIVER CudaDrvLib;
CUresult result = CUDA_SUCCESS;
result = GET_DRIVER_HANDLE(&CudaDrvLib);
GET_PROC_ERROR_FUNCTIONS(cuGetErrorString, cuGetErrorString, 1);
return result;
}
CUresult CUDAAPI cuInit(unsigned int Flags, int cudaVersion)
{
CUDADRIVER CudaDrvLib;
int driverVer = 1000;
CHECKED_CALL(LOAD_LIBRARY(&CudaDrvLib));
// cuInit is required; alias it to _cuInit
GET_PROC_EX(cuInit, _cuInit, 1);
CHECKED_CALL(_cuInit(Flags));
// available since 2.2. if not present, version 1.0 is assumed
GET_PROC_OPTIONAL(cuDriverGetVersion);
if (cuDriverGetVersion) {
CHECKED_CALL(cuDriverGetVersion(&driverVer));
}
// fetch all function pointers
GET_PROC(cuDeviceGet);
GET_PROC(cuDeviceGetCount);
GET_PROC(cuDeviceGetName);
GET_PROC(cuDeviceComputeCapability);
GET_PROC(cuDeviceGetProperties);
GET_PROC(cuDeviceGetAttribute);
GET_PROC(cuGetErrorString);
GET_PROC(cuCtxDestroy);
GET_PROC(cuCtxAttach);
GET_PROC(cuCtxDetach);
GET_PROC(cuCtxPushCurrent);
GET_PROC(cuCtxPopCurrent);
GET_PROC(cuCtxGetDevice);
GET_PROC(cuCtxSynchronize);
GET_PROC(cuCtxSetLimit);
GET_PROC(cuCtxGetCacheConfig);
GET_PROC(cuCtxSetCacheConfig);
GET_PROC(cuCtxGetApiVersion);
GET_PROC(cuModuleLoad);
GET_PROC(cuModuleLoadData);
GET_PROC(cuModuleUnload);
GET_PROC(cuModuleGetFunction);
GET_PROC(cuModuleGetTexRef);
GET_PROC(cuMemFreeHost);
GET_PROC(cuMemHostAlloc);
GET_PROC(cuFuncSetBlockShape);
GET_PROC(cuFuncSetSharedSize);
GET_PROC(cuFuncGetAttribute);
GET_PROC(cuArrayDestroy);
GET_PROC(cuTexRefCreate);
GET_PROC(cuTexRefDestroy);
GET_PROC(cuTexRefSetArray);
GET_PROC(cuTexRefSetFormat);
GET_PROC(cuTexRefSetAddressMode);
GET_PROC(cuTexRefSetFilterMode);
GET_PROC(cuTexRefSetFlags);
GET_PROC(cuTexRefGetArray);
GET_PROC(cuTexRefGetAddressMode);
GET_PROC(cuTexRefGetFilterMode);
GET_PROC(cuTexRefGetFormat);
GET_PROC(cuTexRefGetFlags);
GET_PROC(cuParamSetSize);
GET_PROC(cuParamSeti);
GET_PROC(cuParamSetf);
GET_PROC(cuParamSetv);
GET_PROC(cuParamSetTexRef);
GET_PROC(cuLaunch);
GET_PROC(cuLaunchGrid);
GET_PROC(cuLaunchGridAsync);
GET_PROC(cuEventCreate);
GET_PROC(cuEventRecord);
GET_PROC(cuEventQuery);
GET_PROC(cuEventSynchronize);
GET_PROC(cuEventDestroy);
GET_PROC(cuEventElapsedTime);
GET_PROC(cuStreamCreate);
GET_PROC(cuStreamWaitEvent);
GET_PROC(cuStreamAddCallback);
GET_PROC(cuStreamQuery);
GET_PROC(cuStreamSynchronize);
GET_PROC(cuStreamDestroy);
// These are CUDA 5.0 new functions
if (driverVer >= 5000) {
GET_PROC(cuMipmappedArrayCreate);
GET_PROC(cuMipmappedArrayDestroy);
GET_PROC(cuMipmappedArrayGetLevel);
}
// These are CUDA 4.2 new functions
if (driverVer >= 4020) {
GET_PROC(cuFuncSetSharedMemConfig);
GET_PROC(cuCtxGetSharedMemConfig);
GET_PROC(cuCtxSetSharedMemConfig);
}
// These are CUDA 4.1 new functions
if (cudaVersion >= 4010 && __CUDA_API_VERSION >= 4010) {
GET_PROC(cuDeviceGetByPCIBusId);
GET_PROC(cuDeviceGetPCIBusId);
GET_PROC(cuIpcGetEventHandle);
GET_PROC(cuIpcOpenEventHandle);
GET_PROC(cuIpcGetMemHandle);
GET_PROC(cuIpcOpenMemHandle);
GET_PROC(cuIpcCloseMemHandle);
}
// These could be _v2 interfaces
if (cudaVersion >= 4000 && __CUDA_API_VERSION >= 4000) {
GET_PROC_V2(cuCtxDestroy);
GET_PROC_V2(cuCtxPopCurrent);
GET_PROC_V2(cuCtxPushCurrent);
GET_PROC_V2(cuStreamDestroy);
GET_PROC_V2(cuEventDestroy);
}
if (cudaVersion >= 3020 && __CUDA_API_VERSION >= 3020) {
GET_PROC_V2(cuDeviceTotalMem);
GET_PROC_V2(cuCtxCreate);
GET_PROC_V2(cuModuleGetGlobal);
GET_PROC_V2(cuMemGetInfo);
GET_PROC_V2(cuMemAlloc);
GET_PROC_V2(cuMemAllocPitch);
GET_PROC_V2(cuMemFree);
GET_PROC_V2(cuMemGetAddressRange);
GET_PROC_V2(cuMemAllocHost);
GET_PROC_V2(cuMemHostGetDevicePointer);
GET_PROC_V2(cuMemcpyHtoD);
GET_PROC_V2(cuMemcpyDtoH);
GET_PROC_V2(cuMemcpyDtoD);
GET_PROC_V2(cuMemcpyDtoA);
GET_PROC_V2(cuMemcpyAtoD);
GET_PROC_V2(cuMemcpyHtoA);
GET_PROC_V2(cuMemcpyAtoH);
GET_PROC_V2(cuMemcpyAtoA);
GET_PROC_V2(cuMemcpy2D);
GET_PROC_V2(cuMemcpy2DUnaligned);
GET_PROC_V2(cuMemcpy3D);
GET_PROC_V2(cuMemcpyHtoDAsync);
GET_PROC_V2(cuMemcpyDtoHAsync);
GET_PROC_V2(cuMemcpyHtoAAsync);
GET_PROC_V2(cuMemcpyAtoHAsync);
GET_PROC_V2(cuMemcpy2DAsync);
GET_PROC_V2(cuMemcpy3DAsync);
GET_PROC_V2(cuMemsetD8);
GET_PROC_V2(cuMemsetD16);
GET_PROC_V2(cuMemsetD32);
GET_PROC_V2(cuMemsetD2D8);
GET_PROC_V2(cuMemsetD2D16);
GET_PROC_V2(cuMemsetD2D32);
GET_PROC_V2(cuArrayCreate);
GET_PROC_V2(cuArrayGetDescriptor);
GET_PROC_V2(cuArray3DCreate);
GET_PROC_V2(cuArray3DGetDescriptor);
GET_PROC_V2(cuTexRefSetAddress);
GET_PROC_V2(cuTexRefGetAddress);
if (cudaVersion >= 4010 && __CUDA_API_VERSION >= 4010) {
GET_PROC_V3(cuTexRefSetAddress2D);
}
else {
GET_PROC_V2(cuTexRefSetAddress2D);
}
}
else {
// versions earlier than 3020
GET_PROC(cuDeviceTotalMem);
GET_PROC(cuCtxCreate);
GET_PROC(cuModuleGetGlobal);
GET_PROC(cuMemGetInfo);
GET_PROC(cuMemAlloc);
GET_PROC(cuMemAllocPitch);
GET_PROC(cuMemFree);
GET_PROC(cuMemGetAddressRange);
GET_PROC(cuMemAllocHost);
GET_PROC(cuMemHostGetDevicePointer);
GET_PROC(cuMemcpyHtoD);
GET_PROC(cuMemcpyDtoH);
GET_PROC(cuMemcpyDtoD);
GET_PROC(cuMemcpyDtoA);
GET_PROC(cuMemcpyAtoD);
GET_PROC(cuMemcpyHtoA);
GET_PROC(cuMemcpyAtoH);
GET_PROC(cuMemcpyAtoA);
GET_PROC(cuMemcpy2D);
GET_PROC(cuMemcpy2DUnaligned);
GET_PROC(cuMemcpy3D);
GET_PROC(cuMemcpyHtoDAsync);
GET_PROC(cuMemcpyDtoHAsync);
GET_PROC(cuMemcpyHtoAAsync);
GET_PROC(cuMemcpyAtoHAsync);
GET_PROC(cuMemcpy2DAsync);
GET_PROC(cuMemcpy3DAsync);
GET_PROC(cuMemsetD8);
GET_PROC(cuMemsetD16);
GET_PROC(cuMemsetD32);
GET_PROC(cuMemsetD2D8);
GET_PROC(cuMemsetD2D16);
GET_PROC(cuMemsetD2D32);
GET_PROC(cuArrayCreate);
GET_PROC(cuArrayGetDescriptor);
GET_PROC(cuArray3DCreate);
GET_PROC(cuArray3DGetDescriptor);
GET_PROC(cuTexRefSetAddress);
GET_PROC(cuTexRefSetAddress2D);
GET_PROC(cuTexRefGetAddress);
}
// The following functions are specific to CUDA versions
if (driverVer >= 4000) {
GET_PROC(cuCtxSetCurrent);
GET_PROC(cuCtxGetCurrent);
GET_PROC(cuMemHostRegister);
GET_PROC(cuMemHostUnregister);
GET_PROC(cuMemcpy);
GET_PROC(cuMemcpyPeer);
GET_PROC(cuLaunchKernel);
GET_PROC(cuProfilerStop);
}
if (driverVer >= 3010) {
GET_PROC(cuModuleGetSurfRef);
GET_PROC(cuSurfRefSetArray);
GET_PROC(cuSurfRefGetArray);
GET_PROC(cuCtxSetLimit);
GET_PROC(cuCtxGetLimit);
}
if (driverVer >= 3000) {
GET_PROC(cuMemcpyDtoDAsync);
GET_PROC(cuFuncSetCacheConfig);
#ifdef CUDA_INIT_D3D11
GET_PROC(cuD3D11GetDevice);
GET_PROC(cuD3D11CtxCreate);
GET_PROC(cuGraphicsD3D11RegisterResource);
#endif
GET_PROC(cuGraphicsUnregisterResource);
GET_PROC(cuGraphicsSubResourceGetMappedArray);
if (cudaVersion >= 3020 && __CUDA_API_VERSION >= 3020) {
GET_PROC_V2(cuGraphicsResourceGetMappedPointer);
}
else {
GET_PROC(cuGraphicsResourceGetMappedPointer);
}
GET_PROC(cuGraphicsResourceSetMapFlags);
GET_PROC(cuGraphicsMapResources);
GET_PROC(cuGraphicsUnmapResources);
GET_PROC(cuGetExportTable);
}
if (driverVer >= 2030) {
GET_PROC(cuMemHostGetFlags);
#ifdef CUDA_INIT_D3D10
GET_PROC(cuD3D10GetDevice);
GET_PROC(cuD3D10CtxCreate);
GET_PROC(cuGraphicsD3D10RegisterResource);
#endif
#ifdef CUDA_INIT_OPENGL
GET_PROC(cuGraphicsGLRegisterBuffer);
GET_PROC(cuGraphicsGLRegisterImage);
#endif
}
if (driverVer >= 2010) {
GET_PROC(cuModuleLoadDataEx);
GET_PROC(cuModuleLoadFatBinary);
#ifdef CUDA_INIT_OPENGL
GET_PROC(cuGLCtxCreate);
GET_PROC(cuGraphicsGLRegisterBuffer);
GET_PROC(cuGraphicsGLRegisterImage);
#ifdef WIN32
GET_PROC(cuWGLGetDevice);
#endif
#endif
#ifdef CUDA_INIT_D3D9
GET_PROC(cuD3D9GetDevice);
GET_PROC(cuD3D9CtxCreate);
GET_PROC(cuGraphicsD3D9RegisterResource);
#endif
}
return CUDA_SUCCESS;
}

18
cpp/0_Introduction/matrixMulDynlinkJIT/cuda_drvapi_dynlink.h Normal file
View File

@@ -0,0 +1,18 @@
/*
* Copyright 1993-2015 NVIDIA Corporation. All rights reserved.
*
* Please refer to the NVIDIA end user license agreement (EULA) associated
* with this source code for terms and conditions that govern your use of
* this software. Any use, reproduction, disclosure, or distribution of
* this software and related documentation outside the terms of the EULA
* is strictly prohibited.
*
*/
#ifndef __cuda_drvapi_dynlink_h__
#define __cuda_drvapi_dynlink_h__
#include "cuda_drvapi_dynlink_cuda.h"
#endif //__cuda_drvapi_dynlink_h__

2030
cpp/0_Introduction/matrixMulDynlinkJIT/cuda_drvapi_dynlink_cuda.h Normal file
View File

File diff suppressed because it is too large Load Diff

12
cpp/0_Introduction/matrixMulDynlinkJIT/extras/README.TXT Normal file
View File

@@ -0,0 +1,12 @@
The auto-generated pair of files named matrixMul_ptxdump.c and
matrixMul_ptxdump.h can be acquired by treating matrixMul_kernel.ptx
as binary file and representing its contents as an array of chars.
An example of the script written in Python language that performs
such translation can be found in "extras" directory of the sample.
The matrixMul_kernel.ptx contains the same PTX code as the file in
"data" directory of matrixMulDrv sample after compilation by nvcc.
The command line for generation using Python script is as follows:
> python ptx2c.py matrixMul_kernel.ptx matrixMul

727
cpp/0_Introduction/matrixMulDynlinkJIT/extras/matrixMul_kernel_32.ptx Normal file
View File

@@ -0,0 +1,727 @@
.version 1.4
.target sm_20, map_f64_to_f32
// compiled with C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v4.0\\bin/../open64/lib//be.exe
// nvopencc 4.0 built on 2011-02-21
//-----------------------------------------------------------
// Compiling C:/Users/EYOUNG~1.COM/AppData/Local/Temp/tmpxft_000014c0_00000000-11_matrixMul_kernel.cpp3.i (C:/Users/EYOUNG~1.COM/AppData/Local/Temp/ccBI#.a04000)
//-----------------------------------------------------------
//-----------------------------------------------------------
// Options:
//-----------------------------------------------------------
// Target:ptx, ISA:sm_20, Endian:little, Pointer Size:32
// -O3 (Optimization level)
// -g0 (Debug level)
// -m2 (Report advisories)
//-----------------------------------------------------------
.shared .align 4 .b8 __cuda_local_var_87382_38_non_const_As__6[1024];
.shared .align 4 .b8 __cuda_local_var_87386_38_non_const_Bs__7[1024];
.entry matrixMul_bs16_32bit (
.param .u32 __cudaparm_matrixMul_bs16_32bit_C,
.param .u32 __cudaparm_matrixMul_bs16_32bit_A,
.param .u32 __cudaparm_matrixMul_bs16_32bit_B,
.param .s32 __cudaparm_matrixMul_bs16_32bit_wA,
.param .s32 __cudaparm_matrixMul_bs16_32bit_wB)
{
.reg .u32 %r<57>;
.reg .f32 %f<52>;
.reg .pred %p<4>;
.loc 16 107 0
$LDWbegin_matrixMul_bs16_32bit:
.loc 16 66 0
cvt.s32.u16 %r1, %ctaid.x;
mul24.lo.s32 %r2, %r1, 16;
cvt.s32.u16 %r3, %ctaid.y;
ld.param.s32 %r4, [__cudaparm_matrixMul_bs16_32bit_wA];
mul.lo.s32 %r5, %r3, %r4;
mul.lo.s32 %r6, %r5, 16;
add.s32 %r7, %r6, %r4;
sub.s32 %r8, %r7, 1;
cvt.s32.u16 %r9, %tid.x;
cvt.s32.u16 %r10, %tid.y;
ld.param.s32 %r11, [__cudaparm_matrixMul_bs16_32bit_wB];
setp.lt.s32 %p1, %r8, %r6;
@%p1 bra $Lt_0_3330;
mov.u32 %r12, __cuda_local_var_87382_38_non_const_As__6;
mov.u32 %r13, __cuda_local_var_87386_38_non_const_Bs__7;
ld.param.s32 %r4, [__cudaparm_matrixMul_bs16_32bit_wA];
add.s32 %r14, %r4, 15;
shr.s32 %r15, %r14, 31;
mov.s32 %r16, 15;
and.b32 %r17, %r15, %r16;
add.s32 %r18, %r17, %r14;
shr.s32 %r19, %r18, 4;
ld.param.s32 %r11, [__cudaparm_matrixMul_bs16_32bit_wB];
mul.lo.s32 %r20, %r10, %r11;
mul.lo.s32 %r21, %r10, %r4;
mul24.lo.u32 %r22, %r10, 16;
mul24.lo.u32 %r23, %r10, 64;
mul24.lo.u32 %r24, %r9, 4;
mul.lo.s32 %r25, %r11, 16;
add.s32 %r26, %r20, %r2;
add.s32 %r27, %r21, %r6;
add.u32 %r28, %r9, %r22;
add.u32 %r29, %r23, %r12;
add.u32 %r30, %r24, %r13;
add.s32 %r31, %r26, %r9;
add.s32 %r32, %r27, %r9;
mul.lo.u32 %r33, %r28, 4;
mul.lo.u32 %r34, %r31, 4;
mul.lo.u32 %r35, %r25, 4;
mul.lo.u32 %r36, %r32, 4;
add.u32 %r37, %r33, %r12;
add.u32 %r38, %r33, %r13;
add.s32 %r39, %r21, %r8;
ld.param.u32 %r40, [__cudaparm_matrixMul_bs16_32bit_B];
add.u32 %r41, %r40, %r34;
ld.param.u32 %r42, [__cudaparm_matrixMul_bs16_32bit_A];
add.u32 %r43, %r36, %r42;
add.s32 %r44, %r39, %r9;
mul.lo.u32 %r45, %r44, 4;
add.u32 %r46, %r45, %r42;
mov.f32 %f1, 0f00000000; // 0
mov.s32 %r47, %r19;
$Lt_0_2818:
//<loop> Loop body line 66, nesting depth: 1, estimated iterations: unknown
.loc 16 81 0
ld.global.f32 %f2, [%r43+0];
st.shared.f32 [%r37+0], %f2;
.loc 16 82 0
ld.global.f32 %f3, [%r41+0];
st.shared.f32 [%r38+0], %f3;
.loc 16 85 0
bar.sync 0;
.loc 16 92 0
ld.shared.f32 %f4, [%r29+0];
ld.shared.f32 %f5, [%r30+0];
mad.f32 %f6, %f4, %f5, %f1;
ld.shared.f32 %f7, [%r29+4];
ld.shared.f32 %f8, [%r30+64];
mad.f32 %f9, %f7, %f8, %f6;
ld.shared.f32 %f10, [%r29+8];
ld.shared.f32 %f11, [%r30+128];
mad.f32 %f12, %f10, %f11, %f9;
ld.shared.f32 %f13, [%r29+12];
ld.shared.f32 %f14, [%r30+192];
mad.f32 %f15, %f13, %f14, %f12;
ld.shared.f32 %f16, [%r29+16];
ld.shared.f32 %f17, [%r30+256];
mad.f32 %f18, %f16, %f17, %f15;
ld.shared.f32 %f19, [%r29+20];
ld.shared.f32 %f20, [%r30+320];
mad.f32 %f21, %f19, %f20, %f18;
ld.shared.f32 %f22, [%r29+24];
ld.shared.f32 %f23, [%r30+384];
mad.f32 %f24, %f22, %f23, %f21;
ld.shared.f32 %f25, [%r29+28];
ld.shared.f32 %f26, [%r30+448];
mad.f32 %f27, %f25, %f26, %f24;
ld.shared.f32 %f28, [%r29+32];
ld.shared.f32 %f29, [%r30+512];
mad.f32 %f30, %f28, %f29, %f27;
ld.shared.f32 %f31, [%r29+36];
ld.shared.f32 %f32, [%r30+576];
mad.f32 %f33, %f31, %f32, %f30;
ld.shared.f32 %f34, [%r29+40];
ld.shared.f32 %f35, [%r30+640];
mad.f32 %f36, %f34, %f35, %f33;
ld.shared.f32 %f37, [%r29+44];
ld.shared.f32 %f38, [%r30+704];
mad.f32 %f39, %f37, %f38, %f36;
ld.shared.f32 %f40, [%r29+48];
ld.shared.f32 %f41, [%r30+768];
mad.f32 %f42, %f40, %f41, %f39;
ld.shared.f32 %f43, [%r29+52];
ld.shared.f32 %f44, [%r30+832];
mad.f32 %f45, %f43, %f44, %f42;
ld.shared.f32 %f46, [%r29+56];
ld.shared.f32 %f47, [%r30+896];
mad.f32 %f48, %f46, %f47, %f45;
ld.shared.f32 %f49, [%r29+60];
ld.shared.f32 %f50, [%r30+960];
mad.f32 %f1, %f49, %f50, %f48;
.loc 16 97 0
bar.sync 0;
.loc 16 66 0
add.u32 %r41, %r35, %r41;
add.u32 %r43, %r43, 64;
setp.le.u32 %p2, %r43, %r46;
@%p2 bra $Lt_0_2818;
bra.uni $Lt_0_2306;
$Lt_0_3330:
ld.param.s32 %r11, [__cudaparm_matrixMul_bs16_32bit_wB];
mul.lo.s32 %r20, %r10, %r11;
mov.f32 %f1, 0f00000000; // 0
$Lt_0_2306:
.loc 16 103 0
ld.param.u32 %r48, [__cudaparm_matrixMul_bs16_32bit_C];
mul.lo.s32 %r49, %r11, %r3;
add.s32 %r50, %r1, %r49;
mul.lo.s32 %r51, %r50, 16;
add.s32 %r52, %r20, %r51;
add.s32 %r53, %r9, %r52;
mul.lo.u32 %r54, %r53, 4;
add.u32 %r55, %r48, %r54;
st.global.f32 [%r55+0], %f1;
.loc 16 110 0
exit;
$LDWend_matrixMul_bs16_32bit:
} // matrixMul_bs16_32bit
.shared .align 4 .b8 __cuda_local_var_87382_38_non_const_As__4[1024];
.shared .align 4 .b8 __cuda_local_var_87386_38_non_const_Bs__5[1024];
.entry matrixMul_bs16_64bit (
.param .u32 __cudaparm_matrixMul_bs16_64bit_C,
.param .u32 __cudaparm_matrixMul_bs16_64bit_A,
.param .u32 __cudaparm_matrixMul_bs16_64bit_B,
.param .u32 __cudaparm_matrixMul_bs16_64bit_wA,
.param .u32 __cudaparm_matrixMul_bs16_64bit_wB)
{
.reg .u32 %r<54>;
.reg .f32 %f<52>;
.reg .pred %p<4>;
.loc 16 111 0
$LDWbegin_matrixMul_bs16_64bit:
.loc 16 66 0
cvt.u32.u16 %r1, %ctaid.x;
mul24.lo.u32 %r2, %r1, 16;
cvt.u32.u16 %r3, %ctaid.y;
ld.param.u32 %r4, [__cudaparm_matrixMul_bs16_64bit_wA];
mul.lo.u32 %r5, %r3, %r4;
mul.lo.u32 %r6, %r5, 16;
mov.s32 %r7, %r6;
add.u32 %r8, %r6, %r4;
sub.u32 %r9, %r8, 1;
cvt.u32.u16 %r10, %tid.x;
cvt.u32.u16 %r11, %tid.y;
ld.param.u32 %r12, [__cudaparm_matrixMul_bs16_64bit_wB];
setp.lt.u32 %p1, %r9, %r6;
@%p1 bra $Lt_1_3330;
mov.u32 %r13, __cuda_local_var_87382_38_non_const_As__4;
mov.u32 %r14, __cuda_local_var_87386_38_non_const_Bs__5;
ld.param.u32 %r4, [__cudaparm_matrixMul_bs16_64bit_wA];
add.u32 %r15, %r4, 15;
shr.s32 %r16, %r15, 31;
mov.s32 %r17, 15;
and.b32 %r18, %r16, %r17;
add.s32 %r19, %r18, %r15;
shr.s32 %r20, %r19, 4;
ld.param.u32 %r12, [__cudaparm_matrixMul_bs16_64bit_wB];
mul.lo.u32 %r21, %r11, %r12;
mul.lo.u32 %r22, %r11, %r4;
mul24.lo.u32 %r23, %r11, 16;
mul24.lo.u32 %r24, %r11, 64;
mul24.lo.u32 %r25, %r10, 4;
mul.lo.u32 %r26, %r12, 16;
add.u32 %r27, %r21, %r2;
add.u32 %r28, %r22, %r6;
add.u32 %r29, %r23, %r10;
add.u32 %r30, %r24, %r13;
add.u32 %r31, %r25, %r14;
add.u32 %r32, %r27, %r10;
add.u32 %r33, %r28, %r10;
mul.lo.u32 %r34, %r29, 4;
mul.lo.u32 %r35, %r32, 4;
mul.lo.u32 %r36, %r26, 4;
mul.lo.u32 %r37, %r33, 4;
add.u32 %r38, %r34, %r13;
add.u32 %r39, %r34, %r14;
ld.param.u32 %r40, [__cudaparm_matrixMul_bs16_64bit_B];
add.u32 %r41, %r40, %r35;
ld.param.u32 %r42, [__cudaparm_matrixMul_bs16_64bit_A];
add.u32 %r43, %r42, %r37;
mov.f32 %f1, 0f00000000; // 0
mov.s32 %r44, %r20;
$Lt_1_2818:
//<loop> Loop body line 66, nesting depth: 1, estimated iterations: unknown
.loc 16 81 0
ld.global.f32 %f2, [%r43+0];
st.shared.f32 [%r38+0], %f2;
.loc 16 82 0
ld.global.f32 %f3, [%r41+0];
st.shared.f32 [%r39+0], %f3;
.loc 16 85 0
bar.sync 0;
.loc 16 92 0
ld.shared.f32 %f4, [%r30+0];
ld.shared.f32 %f5, [%r31+0];
mad.f32 %f6, %f4, %f5, %f1;
ld.shared.f32 %f7, [%r30+4];
ld.shared.f32 %f8, [%r31+64];
mad.f32 %f9, %f7, %f8, %f6;
ld.shared.f32 %f10, [%r30+8];
ld.shared.f32 %f11, [%r31+128];
mad.f32 %f12, %f10, %f11, %f9;
ld.shared.f32 %f13, [%r30+12];
ld.shared.f32 %f14, [%r31+192];
mad.f32 %f15, %f13, %f14, %f12;
ld.shared.f32 %f16, [%r30+16];
ld.shared.f32 %f17, [%r31+256];
mad.f32 %f18, %f16, %f17, %f15;
ld.shared.f32 %f19, [%r30+20];
ld.shared.f32 %f20, [%r31+320];
mad.f32 %f21, %f19, %f20, %f18;
ld.shared.f32 %f22, [%r30+24];
ld.shared.f32 %f23, [%r31+384];
mad.f32 %f24, %f22, %f23, %f21;
ld.shared.f32 %f25, [%r30+28];
ld.shared.f32 %f26, [%r31+448];
mad.f32 %f27, %f25, %f26, %f24;
ld.shared.f32 %f28, [%r30+32];
ld.shared.f32 %f29, [%r31+512];
mad.f32 %f30, %f28, %f29, %f27;
ld.shared.f32 %f31, [%r30+36];
ld.shared.f32 %f32, [%r31+576];
mad.f32 %f33, %f31, %f32, %f30;
ld.shared.f32 %f34, [%r30+40];
ld.shared.f32 %f35, [%r31+640];
mad.f32 %f36, %f34, %f35, %f33;
ld.shared.f32 %f37, [%r30+44];
ld.shared.f32 %f38, [%r31+704];
mad.f32 %f39, %f37, %f38, %f36;
ld.shared.f32 %f40, [%r30+48];
ld.shared.f32 %f41, [%r31+768];
mad.f32 %f42, %f40, %f41, %f39;
ld.shared.f32 %f43, [%r30+52];
ld.shared.f32 %f44, [%r31+832];
mad.f32 %f45, %f43, %f44, %f42;
ld.shared.f32 %f46, [%r30+56];
ld.shared.f32 %f47, [%r31+896];
mad.f32 %f48, %f46, %f47, %f45;
ld.shared.f32 %f49, [%r30+60];
ld.shared.f32 %f50, [%r31+960];
mad.f32 %f1, %f49, %f50, %f48;
.loc 16 97 0
bar.sync 0;
.loc 16 66 0
add.u32 %r41, %r36, %r41;
add.u32 %r7, %r7, 16;
add.u32 %r43, %r43, 64;
setp.ge.u32 %p2, %r9, %r7;
@%p2 bra $Lt_1_2818;
bra.uni $Lt_1_2306;
$Lt_1_3330:
ld.param.u32 %r12, [__cudaparm_matrixMul_bs16_64bit_wB];
mul.lo.u32 %r21, %r11, %r12;
mov.f32 %f1, 0f00000000; // 0
$Lt_1_2306:
.loc 16 103 0
ld.param.u32 %r45, [__cudaparm_matrixMul_bs16_64bit_C];
mul.lo.u32 %r46, %r12, %r3;
add.u32 %r47, %r1, %r46;
mul.lo.u32 %r48, %r47, 16;
add.u32 %r49, %r21, %r48;
add.u32 %r50, %r10, %r49;
mul.lo.u32 %r51, %r50, 4;
add.u32 %r52, %r45, %r51;
st.global.f32 [%r52+0], %f1;
.loc 16 114 0
exit;
$LDWend_matrixMul_bs16_64bit:
} // matrixMul_bs16_64bit
.shared .align 4 .b8 __cuda_local_var_87382_38_non_const_As__2[4096];
.shared .align 4 .b8 __cuda_local_var_87386_38_non_const_Bs__3[4096];
.entry matrixMul_bs32_32bit (
.param .u32 __cudaparm_matrixMul_bs32_32bit_C,
.param .u32 __cudaparm_matrixMul_bs32_32bit_A,
.param .u32 __cudaparm_matrixMul_bs32_32bit_B,
.param .s32 __cudaparm_matrixMul_bs32_32bit_wA,
.param .s32 __cudaparm_matrixMul_bs32_32bit_wB)
{
.reg .u32 %r<57>;
.reg .f32 %f<100>;
.reg .pred %p<4>;
.loc 16 115 0
$LDWbegin_matrixMul_bs32_32bit:
.loc 16 66 0
cvt.s32.u16 %r1, %ctaid.x;
mul24.lo.s32 %r2, %r1, 32;
cvt.s32.u16 %r3, %ctaid.y;
ld.param.s32 %r4, [__cudaparm_matrixMul_bs32_32bit_wA];
mul.lo.s32 %r5, %r3, %r4;
mul.lo.s32 %r6, %r5, 32;
add.s32 %r7, %r6, %r4;
sub.s32 %r8, %r7, 1;
cvt.s32.u16 %r9, %tid.x;
cvt.s32.u16 %r10, %tid.y;
ld.param.s32 %r11, [__cudaparm_matrixMul_bs32_32bit_wB];
setp.lt.s32 %p1, %r8, %r6;
@%p1 bra $Lt_2_3330;
mov.u32 %r12, __cuda_local_var_87382_38_non_const_As__2;
mov.u32 %r13, __cuda_local_var_87386_38_non_const_Bs__3;
ld.param.s32 %r4, [__cudaparm_matrixMul_bs32_32bit_wA];
add.s32 %r14, %r4, 31;
shr.s32 %r15, %r14, 31;
mov.s32 %r16, 31;
and.b32 %r17, %r15, %r16;
add.s32 %r18, %r17, %r14;
shr.s32 %r19, %r18, 5;
ld.param.s32 %r11, [__cudaparm_matrixMul_bs32_32bit_wB];
mul.lo.s32 %r20, %r10, %r11;
mul.lo.s32 %r21, %r10, %r4;
mul24.lo.u32 %r22, %r10, 32;
mul24.lo.u32 %r23, %r10, 128;
mul24.lo.u32 %r24, %r9, 4;
mul.lo.s32 %r25, %r11, 32;
add.s32 %r26, %r20, %r2;
add.s32 %r27, %r21, %r6;
add.u32 %r28, %r9, %r22;
add.u32 %r29, %r23, %r12;
add.u32 %r30, %r24, %r13;
add.s32 %r31, %r26, %r9;
add.s32 %r32, %r27, %r9;
mul.lo.u32 %r33, %r28, 4;
mul.lo.u32 %r34, %r31, 4;
mul.lo.u32 %r35, %r25, 4;
mul.lo.u32 %r36, %r32, 4;
add.u32 %r37, %r33, %r12;
add.u32 %r38, %r33, %r13;
add.s32 %r39, %r21, %r8;
ld.param.u32 %r40, [__cudaparm_matrixMul_bs32_32bit_B];
add.u32 %r41, %r40, %r34;
ld.param.u32 %r42, [__cudaparm_matrixMul_bs32_32bit_A];
add.u32 %r43, %r36, %r42;
add.s32 %r44, %r39, %r9;
mul.lo.u32 %r45, %r44, 4;
add.u32 %r46, %r45, %r42;
mov.f32 %f1, 0f00000000; // 0
mov.s32 %r47, %r19;
$Lt_2_2818:
//<loop> Loop body line 66, nesting depth: 1, estimated iterations: unknown
.loc 16 81 0
ld.global.f32 %f2, [%r43+0];
st.shared.f32 [%r37+0], %f2;
.loc 16 82 0
ld.global.f32 %f3, [%r41+0];
st.shared.f32 [%r38+0], %f3;
.loc 16 85 0
bar.sync 0;
.loc 16 92 0
ld.shared.f32 %f4, [%r29+0];
ld.shared.f32 %f5, [%r30+0];
mad.f32 %f6, %f4, %f5, %f1;
ld.shared.f32 %f7, [%r29+4];
ld.shared.f32 %f8, [%r30+128];
mad.f32 %f9, %f7, %f8, %f6;
ld.shared.f32 %f10, [%r29+8];
ld.shared.f32 %f11, [%r30+256];
mad.f32 %f12, %f10, %f11, %f9;
ld.shared.f32 %f13, [%r29+12];
ld.shared.f32 %f14, [%r30+384];
mad.f32 %f15, %f13, %f14, %f12;
ld.shared.f32 %f16, [%r29+16];
ld.shared.f32 %f17, [%r30+512];
mad.f32 %f18, %f16, %f17, %f15;
ld.shared.f32 %f19, [%r29+20];
ld.shared.f32 %f20, [%r30+640];
mad.f32 %f21, %f19, %f20, %f18;
ld.shared.f32 %f22, [%r29+24];
ld.shared.f32 %f23, [%r30+768];
mad.f32 %f24, %f22, %f23, %f21;
ld.shared.f32 %f25, [%r29+28];
ld.shared.f32 %f26, [%r30+896];
mad.f32 %f27, %f25, %f26, %f24;
ld.shared.f32 %f28, [%r29+32];
ld.shared.f32 %f29, [%r30+1024];
mad.f32 %f30, %f28, %f29, %f27;
ld.shared.f32 %f31, [%r29+36];
ld.shared.f32 %f32, [%r30+1152];
mad.f32 %f33, %f31, %f32, %f30;
ld.shared.f32 %f34, [%r29+40];
ld.shared.f32 %f35, [%r30+1280];
mad.f32 %f36, %f34, %f35, %f33;
ld.shared.f32 %f37, [%r29+44];
ld.shared.f32 %f38, [%r30+1408];
mad.f32 %f39, %f37, %f38, %f36;
ld.shared.f32 %f40, [%r29+48];
ld.shared.f32 %f41, [%r30+1536];
mad.f32 %f42, %f40, %f41, %f39;
ld.shared.f32 %f43, [%r29+52];
ld.shared.f32 %f44, [%r30+1664];
mad.f32 %f45, %f43, %f44, %f42;
ld.shared.f32 %f46, [%r29+56];
ld.shared.f32 %f47, [%r30+1792];
mad.f32 %f48, %f46, %f47, %f45;
ld.shared.f32 %f49, [%r29+60];
ld.shared.f32 %f50, [%r30+1920];
mad.f32 %f51, %f49, %f50, %f48;
ld.shared.f32 %f52, [%r29+64];
ld.shared.f32 %f53, [%r30+2048];
mad.f32 %f54, %f52, %f53, %f51;
ld.shared.f32 %f55, [%r29+68];
ld.shared.f32 %f56, [%r30+2176];
mad.f32 %f57, %f55, %f56, %f54;
ld.shared.f32 %f58, [%r29+72];
ld.shared.f32 %f59, [%r30+2304];
mad.f32 %f60, %f58, %f59, %f57;
ld.shared.f32 %f61, [%r29+76];
ld.shared.f32 %f62, [%r30+2432];
mad.f32 %f63, %f61, %f62, %f60;
ld.shared.f32 %f64, [%r29+80];
ld.shared.f32 %f65, [%r30+2560];
mad.f32 %f66, %f64, %f65, %f63;
ld.shared.f32 %f67, [%r29+84];
ld.shared.f32 %f68, [%r30+2688];
mad.f32 %f69, %f67, %f68, %f66;
ld.shared.f32 %f70, [%r29+88];
ld.shared.f32 %f71, [%r30+2816];
mad.f32 %f72, %f70, %f71, %f69;
ld.shared.f32 %f73, [%r29+92];
ld.shared.f32 %f74, [%r30+2944];
mad.f32 %f75, %f73, %f74, %f72;
ld.shared.f32 %f76, [%r29+96];
ld.shared.f32 %f77, [%r30+3072];
mad.f32 %f78, %f76, %f77, %f75;
ld.shared.f32 %f79, [%r29+100];
ld.shared.f32 %f80, [%r30+3200];
mad.f32 %f81, %f79, %f80, %f78;
ld.shared.f32 %f82, [%r29+104];
ld.shared.f32 %f83, [%r30+3328];
mad.f32 %f84, %f82, %f83, %f81;
ld.shared.f32 %f85, [%r29+108];
ld.shared.f32 %f86, [%r30+3456];
mad.f32 %f87, %f85, %f86, %f84;
ld.shared.f32 %f88, [%r29+112];
ld.shared.f32 %f89, [%r30+3584];
mad.f32 %f90, %f88, %f89, %f87;
ld.shared.f32 %f91, [%r29+116];
ld.shared.f32 %f92, [%r30+3712];
mad.f32 %f93, %f91, %f92, %f90;
ld.shared.f32 %f94, [%r29+120];
ld.shared.f32 %f95, [%r30+3840];
mad.f32 %f96, %f94, %f95, %f93;
ld.shared.f32 %f97, [%r29+124];
ld.shared.f32 %f98, [%r30+3968];
mad.f32 %f1, %f97, %f98, %f96;
.loc 16 97 0
bar.sync 0;
.loc 16 66 0
add.u32 %r41, %r35, %r41;
add.u32 %r43, %r43, 128;
setp.le.u32 %p2, %r43, %r46;
@%p2 bra $Lt_2_2818;
bra.uni $Lt_2_2306;
$Lt_2_3330:
ld.param.s32 %r11, [__cudaparm_matrixMul_bs32_32bit_wB];
mul.lo.s32 %r20, %r10, %r11;
mov.f32 %f1, 0f00000000; // 0
$Lt_2_2306:
.loc 16 103 0
ld.param.u32 %r48, [__cudaparm_matrixMul_bs32_32bit_C];
mul.lo.s32 %r49, %r11, %r3;
add.s32 %r50, %r1, %r49;
mul.lo.s32 %r51, %r50, 32;
add.s32 %r52, %r20, %r51;
add.s32 %r53, %r9, %r52;
mul.lo.u32 %r54, %r53, 4;
add.u32 %r55, %r48, %r54;
st.global.f32 [%r55+0], %f1;
.loc 16 118 0
exit;
$LDWend_matrixMul_bs32_32bit:
} // matrixMul_bs32_32bit
.shared .align 4 .b8 __cuda_local_var_87382_38_non_const_As__0[4096];
.shared .align 4 .b8 __cuda_local_var_87386_38_non_const_Bs__1[4096];
.entry matrixMul_bs32_64bit (
.param .u32 __cudaparm_matrixMul_bs32_64bit_C,
.param .u32 __cudaparm_matrixMul_bs32_64bit_A,
.param .u32 __cudaparm_matrixMul_bs32_64bit_B,
.param .u32 __cudaparm_matrixMul_bs32_64bit_wA,
.param .u32 __cudaparm_matrixMul_bs32_64bit_wB)
{
.reg .u32 %r<54>;
.reg .f32 %f<100>;
.reg .pred %p<4>;
.loc 16 119 0
$LDWbegin_matrixMul_bs32_64bit:
.loc 16 66 0
cvt.u32.u16 %r1, %ctaid.x;
mul24.lo.u32 %r2, %r1, 32;
cvt.u32.u16 %r3, %ctaid.y;
ld.param.u32 %r4, [__cudaparm_matrixMul_bs32_64bit_wA];
mul.lo.u32 %r5, %r3, %r4;
mul.lo.u32 %r6, %r5, 32;
mov.s32 %r7, %r6;
add.u32 %r8, %r6, %r4;
sub.u32 %r9, %r8, 1;
cvt.u32.u16 %r10, %tid.x;
cvt.u32.u16 %r11, %tid.y;
ld.param.u32 %r12, [__cudaparm_matrixMul_bs32_64bit_wB];
setp.lt.u32 %p1, %r9, %r6;
@%p1 bra $Lt_3_3330;
mov.u32 %r13, __cuda_local_var_87382_38_non_const_As__0;
mov.u32 %r14, __cuda_local_var_87386_38_non_const_Bs__1;
ld.param.u32 %r4, [__cudaparm_matrixMul_bs32_64bit_wA];
add.u32 %r15, %r4, 31;
shr.s32 %r16, %r15, 31;
mov.s32 %r17, 31;
and.b32 %r18, %r16, %r17;
add.s32 %r19, %r18, %r15;
shr.s32 %r20, %r19, 5;
ld.param.u32 %r12, [__cudaparm_matrixMul_bs32_64bit_wB];
mul.lo.u32 %r21, %r11, %r12;
mul.lo.u32 %r22, %r11, %r4;
mul24.lo.u32 %r23, %r11, 32;
mul24.lo.u32 %r24, %r11, 128;
mul24.lo.u32 %r25, %r10, 4;
mul.lo.u32 %r26, %r12, 32;
add.u32 %r27, %r21, %r2;
add.u32 %r28, %r22, %r6;
add.u32 %r29, %r23, %r10;
add.u32 %r30, %r24, %r13;
add.u32 %r31, %r25, %r14;
add.u32 %r32, %r27, %r10;
add.u32 %r33, %r28, %r10;
mul.lo.u32 %r34, %r29, 4;
mul.lo.u32 %r35, %r32, 4;
mul.lo.u32 %r36, %r26, 4;
mul.lo.u32 %r37, %r33, 4;
add.u32 %r38, %r34, %r13;
add.u32 %r39, %r34, %r14;
ld.param.u32 %r40, [__cudaparm_matrixMul_bs32_64bit_B];
add.u32 %r41, %r40, %r35;
ld.param.u32 %r42, [__cudaparm_matrixMul_bs32_64bit_A];
add.u32 %r43, %r42, %r37;
mov.f32 %f1, 0f00000000; // 0
mov.s32 %r44, %r20;
$Lt_3_2818:
//<loop> Loop body line 66, nesting depth: 1, estimated iterations: unknown
.loc 16 81 0
ld.global.f32 %f2, [%r43+0];
st.shared.f32 [%r38+0], %f2;
.loc 16 82 0
ld.global.f32 %f3, [%r41+0];
st.shared.f32 [%r39+0], %f3;
.loc 16 85 0
bar.sync 0;
.loc 16 92 0
ld.shared.f32 %f4, [%r30+0];
ld.shared.f32 %f5, [%r31+0];
mad.f32 %f6, %f4, %f5, %f1;
ld.shared.f32 %f7, [%r30+4];
ld.shared.f32 %f8, [%r31+128];
mad.f32 %f9, %f7, %f8, %f6;
ld.shared.f32 %f10, [%r30+8];
ld.shared.f32 %f11, [%r31+256];
mad.f32 %f12, %f10, %f11, %f9;
ld.shared.f32 %f13, [%r30+12];
ld.shared.f32 %f14, [%r31+384];
mad.f32 %f15, %f13, %f14, %f12;
ld.shared.f32 %f16, [%r30+16];
ld.shared.f32 %f17, [%r31+512];
mad.f32 %f18, %f16, %f17, %f15;
ld.shared.f32 %f19, [%r30+20];
ld.shared.f32 %f20, [%r31+640];
mad.f32 %f21, %f19, %f20, %f18;
ld.shared.f32 %f22, [%r30+24];
ld.shared.f32 %f23, [%r31+768];
mad.f32 %f24, %f22, %f23, %f21;
ld.shared.f32 %f25, [%r30+28];
ld.shared.f32 %f26, [%r31+896];
mad.f32 %f27, %f25, %f26, %f24;
ld.shared.f32 %f28, [%r30+32];
ld.shared.f32 %f29, [%r31+1024];
mad.f32 %f30, %f28, %f29, %f27;
ld.shared.f32 %f31, [%r30+36];
ld.shared.f32 %f32, [%r31+1152];
mad.f32 %f33, %f31, %f32, %f30;
ld.shared.f32 %f34, [%r30+40];
ld.shared.f32 %f35, [%r31+1280];
mad.f32 %f36, %f34, %f35, %f33;
ld.shared.f32 %f37, [%r30+44];
ld.shared.f32 %f38, [%r31+1408];
mad.f32 %f39, %f37, %f38, %f36;
ld.shared.f32 %f40, [%r30+48];
ld.shared.f32 %f41, [%r31+1536];
mad.f32 %f42, %f40, %f41, %f39;
ld.shared.f32 %f43, [%r30+52];
ld.shared.f32 %f44, [%r31+1664];
mad.f32 %f45, %f43, %f44, %f42;
ld.shared.f32 %f46, [%r30+56];
ld.shared.f32 %f47, [%r31+1792];
mad.f32 %f48, %f46, %f47, %f45;
ld.shared.f32 %f49, [%r30+60];
ld.shared.f32 %f50, [%r31+1920];
mad.f32 %f51, %f49, %f50, %f48;
ld.shared.f32 %f52, [%r30+64];
ld.shared.f32 %f53, [%r31+2048];
mad.f32 %f54, %f52, %f53, %f51;
ld.shared.f32 %f55, [%r30+68];
ld.shared.f32 %f56, [%r31+2176];
mad.f32 %f57, %f55, %f56, %f54;
ld.shared.f32 %f58, [%r30+72];
ld.shared.f32 %f59, [%r31+2304];
mad.f32 %f60, %f58, %f59, %f57;
ld.shared.f32 %f61, [%r30+76];
ld.shared.f32 %f62, [%r31+2432];
mad.f32 %f63, %f61, %f62, %f60;
ld.shared.f32 %f64, [%r30+80];
ld.shared.f32 %f65, [%r31+2560];
mad.f32 %f66, %f64, %f65, %f63;
ld.shared.f32 %f67, [%r30+84];
ld.shared.f32 %f68, [%r31+2688];
mad.f32 %f69, %f67, %f68, %f66;
ld.shared.f32 %f70, [%r30+88];
ld.shared.f32 %f71, [%r31+2816];
mad.f32 %f72, %f70, %f71, %f69;
ld.shared.f32 %f73, [%r30+92];
ld.shared.f32 %f74, [%r31+2944];
mad.f32 %f75, %f73, %f74, %f72;
ld.shared.f32 %f76, [%r30+96];
ld.shared.f32 %f77, [%r31+3072];
mad.f32 %f78, %f76, %f77, %f75;
ld.shared.f32 %f79, [%r30+100];
ld.shared.f32 %f80, [%r31+3200];
mad.f32 %f81, %f79, %f80, %f78;
ld.shared.f32 %f82, [%r30+104];
ld.shared.f32 %f83, [%r31+3328];
mad.f32 %f84, %f82, %f83, %f81;
ld.shared.f32 %f85, [%r30+108];
ld.shared.f32 %f86, [%r31+3456];
mad.f32 %f87, %f85, %f86, %f84;
ld.shared.f32 %f88, [%r30+112];
ld.shared.f32 %f89, [%r31+3584];
mad.f32 %f90, %f88, %f89, %f87;
ld.shared.f32 %f91, [%r30+116];
ld.shared.f32 %f92, [%r31+3712];
mad.f32 %f93, %f91, %f92, %f90;
ld.shared.f32 %f94, [%r30+120];
ld.shared.f32 %f95, [%r31+3840];
mad.f32 %f96, %f94, %f95, %f93;
ld.shared.f32 %f97, [%r30+124];
ld.shared.f32 %f98, [%r31+3968];
mad.f32 %f1, %f97, %f98, %f96;
.loc 16 97 0
bar.sync 0;
.loc 16 66 0
add.u32 %r41, %r36, %r41;
add.u32 %r7, %r7, 32;
add.u32 %r43, %r43, 128;
setp.ge.u32 %p2, %r9, %r7;
@%p2 bra $Lt_3_2818;
bra.uni $Lt_3_2306;
$Lt_3_3330:
ld.param.u32 %r12, [__cudaparm_matrixMul_bs32_64bit_wB];
mul.lo.u32 %r21, %r11, %r12;
mov.f32 %f1, 0f00000000; // 0
$Lt_3_2306:
.loc 16 103 0
ld.param.u32 %r45, [__cudaparm_matrixMul_bs32_64bit_C];
mul.lo.u32 %r46, %r12, %r3;
add.u32 %r47, %r1, %r46;
mul.lo.u32 %r48, %r47, 32;
add.u32 %r49, %r21, %r48;
add.u32 %r50, %r10, %r49;
mul.lo.u32 %r51, %r50, 4;
add.u32 %r52, %r45, %r51;
st.global.f32 [%r52+0], %f1;
.loc 16 122 0
exit;
$LDWend_matrixMul_bs32_64bit:
} // matrixMul_bs32_64bit

737
cpp/0_Introduction/matrixMulDynlinkJIT/extras/matrixMul_kernel_64.ptx Normal file
View File

@@ -0,0 +1,737 @@
.version 1.4
.target sm_20, map_f64_to_f32
// compiled with C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v4.0\\bin/../open64/lib//be.exe
// nvopencc 4.0 built on 2011-02-21
//-----------------------------------------------------------
// Compiling C:/Users/EYOUNG~1.COM/AppData/Local/Temp/tmpxft_00000c2c_00000000-11_matrixMul_kernel.cpp3.i (C:/Users/EYOUNG~1.COM/AppData/Local/Temp/ccBI#.a04524)
//-----------------------------------------------------------
//-----------------------------------------------------------
// Options:
//-----------------------------------------------------------
// Target:ptx, ISA:sm_20, Endian:little, Pointer Size:64
// -O3 (Optimization level)
// -g0 (Debug level)
// -m2 (Report advisories)
//-----------------------------------------------------------
.shared .align 4 .b8 __cuda_local_var_87382_38_non_const_As__6[1024];
.shared .align 4 .b8 __cuda_local_var_87386_38_non_const_Bs__7[1024];
.entry matrixMul_bs16_32bit (
.param .u64 __cudaparm_matrixMul_bs16_32bit_C,
.param .u64 __cudaparm_matrixMul_bs16_32bit_A,
.param .u64 __cudaparm_matrixMul_bs16_32bit_B,
.param .s32 __cudaparm_matrixMul_bs16_32bit_wA,
.param .s32 __cudaparm_matrixMul_bs16_32bit_wB)
{
.reg .u32 %r<34>;
.reg .u64 %rd<29>;
.reg .f32 %f<52>;
.reg .pred %p<4>;
.loc 16 107 0
$LDWbegin_matrixMul_bs16_32bit:
.loc 16 66 0
cvt.s32.u16 %r1, %ctaid.x;
mul24.lo.s32 %r2, %r1, 16;
cvt.s32.u16 %r3, %ctaid.y;
ld.param.s32 %r4, [__cudaparm_matrixMul_bs16_32bit_wA];
mul.lo.s32 %r5, %r3, %r4;
mul.lo.s32 %r6, %r5, 16;
add.s32 %r7, %r6, %r4;
sub.s32 %r8, %r7, 1;
cvt.s32.u16 %r9, %tid.x;
cvt.s32.u16 %r10, %tid.y;
ld.param.s32 %r11, [__cudaparm_matrixMul_bs16_32bit_wB];
setp.lt.s32 %p1, %r8, %r6;
@%p1 bra $Lt_0_3330;
mov.u64 %rd1, __cuda_local_var_87386_38_non_const_Bs__7;
mov.u64 %rd2, __cuda_local_var_87382_38_non_const_As__6;
ld.param.s32 %r4, [__cudaparm_matrixMul_bs16_32bit_wA];
add.s32 %r12, %r4, 15;
shr.s32 %r13, %r12, 31;
mov.s32 %r14, 15;
and.b32 %r15, %r13, %r14;
add.s32 %r16, %r15, %r12;
shr.s32 %r17, %r16, 4;
ld.param.s32 %r11, [__cudaparm_matrixMul_bs16_32bit_wB];
mul.lo.s32 %r18, %r10, %r11;
mul.lo.s32 %r19, %r10, %r4;
cvt.s64.s32 %rd3, %r9;
cvt.s64.s32 %rd4, %r10;
add.s32 %r20, %r19, %r6;
add.s32 %r21, %r9, %r20;
mul.wide.s32 %rd5, %r9, 4;
add.u64 %rd6, %rd1, %rd5;
mul.wide.s32 %rd7, %r10, 64;
add.u64 %rd8, %rd2, %rd7;
mul.wide.s32 %rd9, %r10, 16;
add.u64 %rd10, %rd3, %rd9;
mul.lo.u64 %rd11, %rd10, 4;
add.s32 %r22, %r19, %r8;
mul.lo.s32 %r23, %r11, 16;
cvt.s64.s32 %rd12, %r23;
mul.wide.s32 %rd13, %r23, 4;
add.u64 %rd14, %rd11, %rd2;
add.u64 %rd15, %rd11, %rd1;
add.s32 %r24, %r22, %r9;
ld.param.u64 %rd16, [__cudaparm_matrixMul_bs16_32bit_B];
add.s32 %r25, %r18, %r2;
add.s32 %r26, %r9, %r25;
cvt.s64.s32 %rd17, %r26;
mul.wide.s32 %rd18, %r26, 4;
add.u64 %rd19, %rd16, %rd18;
ld.param.u64 %rd20, [__cudaparm_matrixMul_bs16_32bit_A];
cvt.s64.s32 %rd21, %r21;
mul.wide.s32 %rd22, %r21, 4;
add.u64 %rd23, %rd20, %rd22;
mov.f32 %f1, 0f00000000; // 0
mov.s32 %r27, %r17;
$Lt_0_2818:
//<loop> Loop body line 66, nesting depth: 1, estimated iterations: unknown
.loc 16 81 0
ld.global.f32 %f2, [%rd23+0];
st.shared.f32 [%rd14+0], %f2;
.loc 16 82 0
ld.global.f32 %f3, [%rd19+0];
st.shared.f32 [%rd15+0], %f3;
.loc 16 85 0
bar.sync 0;
.loc 16 92 0
ld.shared.f32 %f4, [%rd8+0];
ld.shared.f32 %f5, [%rd6+0];
mad.f32 %f6, %f4, %f5, %f1;
ld.shared.f32 %f7, [%rd8+4];
ld.shared.f32 %f8, [%rd6+64];
mad.f32 %f9, %f7, %f8, %f6;
ld.shared.f32 %f10, [%rd8+8];
ld.shared.f32 %f11, [%rd6+128];
mad.f32 %f12, %f10, %f11, %f9;
ld.shared.f32 %f13, [%rd8+12];
ld.shared.f32 %f14, [%rd6+192];
mad.f32 %f15, %f13, %f14, %f12;
ld.shared.f32 %f16, [%rd8+16];
ld.shared.f32 %f17, [%rd6+256];
mad.f32 %f18, %f16, %f17, %f15;
ld.shared.f32 %f19, [%rd8+20];
ld.shared.f32 %f20, [%rd6+320];
mad.f32 %f21, %f19, %f20, %f18;
ld.shared.f32 %f22, [%rd8+24];
ld.shared.f32 %f23, [%rd6+384];
mad.f32 %f24, %f22, %f23, %f21;
ld.shared.f32 %f25, [%rd8+28];
ld.shared.f32 %f26, [%rd6+448];
mad.f32 %f27, %f25, %f26, %f24;
ld.shared.f32 %f28, [%rd8+32];
ld.shared.f32 %f29, [%rd6+512];
mad.f32 %f30, %f28, %f29, %f27;
ld.shared.f32 %f31, [%rd8+36];
ld.shared.f32 %f32, [%rd6+576];
mad.f32 %f33, %f31, %f32, %f30;
ld.shared.f32 %f34, [%rd8+40];
ld.shared.f32 %f35, [%rd6+640];
mad.f32 %f36, %f34, %f35, %f33;
ld.shared.f32 %f37, [%rd8+44];
ld.shared.f32 %f38, [%rd6+704];
mad.f32 %f39, %f37, %f38, %f36;
ld.shared.f32 %f40, [%rd8+48];
ld.shared.f32 %f41, [%rd6+768];
mad.f32 %f42, %f40, %f41, %f39;
ld.shared.f32 %f43, [%rd8+52];
ld.shared.f32 %f44, [%rd6+832];
mad.f32 %f45, %f43, %f44, %f42;
ld.shared.f32 %f46, [%rd8+56];
ld.shared.f32 %f47, [%rd6+896];
mad.f32 %f48, %f46, %f47, %f45;
ld.shared.f32 %f49, [%rd8+60];
ld.shared.f32 %f50, [%rd6+960];
mad.f32 %f1, %f49, %f50, %f48;
.loc 16 97 0
bar.sync 0;
.loc 16 66 0
add.u64 %rd19, %rd13, %rd19;
add.s32 %r21, %r21, 16;
add.u64 %rd23, %rd23, 64;
setp.le.s32 %p2, %r21, %r24;
@%p2 bra $Lt_0_2818;
bra.uni $Lt_0_2306;
$Lt_0_3330:
ld.param.s32 %r11, [__cudaparm_matrixMul_bs16_32bit_wB];
mul.lo.s32 %r18, %r10, %r11;
mov.f32 %f1, 0f00000000; // 0
$Lt_0_2306:
.loc 16 103 0
ld.param.u64 %rd24, [__cudaparm_matrixMul_bs16_32bit_C];
mul.lo.s32 %r28, %r11, %r3;
add.s32 %r29, %r1, %r28;
mul.lo.s32 %r30, %r29, 16;
add.s32 %r31, %r18, %r30;
add.s32 %r32, %r9, %r31;
cvt.s64.s32 %rd25, %r32;
mul.wide.s32 %rd26, %r32, 4;
add.u64 %rd27, %rd24, %rd26;
st.global.f32 [%rd27+0], %f1;
.loc 16 110 0
exit;
$LDWend_matrixMul_bs16_32bit:
} // matrixMul_bs16_32bit
.shared .align 4 .b8 __cuda_local_var_87382_38_non_const_As__4[1024];
.shared .align 4 .b8 __cuda_local_var_87386_38_non_const_Bs__5[1024];
.entry matrixMul_bs16_64bit (
.param .u64 __cudaparm_matrixMul_bs16_64bit_C,
.param .u64 __cudaparm_matrixMul_bs16_64bit_A,
.param .u64 __cudaparm_matrixMul_bs16_64bit_B,
.param .u64 __cudaparm_matrixMul_bs16_64bit_wA,
.param .u64 __cudaparm_matrixMul_bs16_64bit_wB)
{
.reg .u64 %rd<53>;
.reg .f32 %f<52>;
.reg .pred %p<4>;
.loc 16 111 0
$LDWbegin_matrixMul_bs16_64bit:
.loc 16 66 0
cvt.u64.u16 %rd1, %ctaid.x;
mul.lo.u64 %rd2, %rd1, 16;
cvt.u64.u16 %rd3, %ctaid.y;
ld.param.u64 %rd4, [__cudaparm_matrixMul_bs16_64bit_wA];
mul.lo.u64 %rd5, %rd3, %rd4;
mul.lo.u64 %rd6, %rd5, 16;
mov.s64 %rd7, %rd6;
add.u64 %rd8, %rd6, %rd4;
sub.u64 %rd9, %rd8, 1;
cvt.u64.u16 %rd10, %tid.x;
cvt.u64.u16 %rd11, %tid.y;
ld.param.u64 %rd12, [__cudaparm_matrixMul_bs16_64bit_wB];
setp.lt.u64 %p1, %rd9, %rd6;
@%p1 bra $Lt_1_3330;
mov.u64 %rd13, __cuda_local_var_87386_38_non_const_Bs__5;
mov.u64 %rd14, __cuda_local_var_87382_38_non_const_As__4;
ld.param.u64 %rd4, [__cudaparm_matrixMul_bs16_64bit_wA];
add.u64 %rd15, %rd4, 15;
shr.s64 %rd16, %rd15, 63;
mov.s64 %rd17, 15;
and.b64 %rd18, %rd16, %rd17;
add.s64 %rd19, %rd18, %rd15;
shr.s64 %rd20, %rd19, 4;
ld.param.u64 %rd12, [__cudaparm_matrixMul_bs16_64bit_wB];
mul.lo.u64 %rd21, %rd11, %rd12;
mul.lo.u64 %rd22, %rd11, 64;
add.u64 %rd23, %rd14, %rd22;
mul.lo.u64 %rd24, %rd10, 4;
add.u64 %rd25, %rd13, %rd24;
mul.lo.u64 %rd26, %rd11, 16;
add.u64 %rd27, %rd10, %rd26;
mul.lo.u64 %rd28, %rd27, 4;
mul.lo.u64 %rd29, %rd12, 64;
add.u64 %rd30, %rd28, %rd14;
add.u64 %rd31, %rd28, %rd13;
ld.param.u64 %rd32, [__cudaparm_matrixMul_bs16_64bit_B];
add.u64 %rd33, %rd21, %rd2;
add.u64 %rd34, %rd10, %rd33;
mul.lo.u64 %rd35, %rd34, 4;
add.u64 %rd36, %rd32, %rd35;
ld.param.u64 %rd37, [__cudaparm_matrixMul_bs16_64bit_A];
mul.lo.u64 %rd38, %rd11, %rd4;
add.u64 %rd39, %rd6, %rd38;
add.u64 %rd40, %rd10, %rd39;
mul.lo.u64 %rd41, %rd40, 4;
add.u64 %rd42, %rd37, %rd41;
mov.f32 %f1, 0f00000000; // 0
mov.s64 %rd43, %rd20;
$Lt_1_2818:
//<loop> Loop body line 66, nesting depth: 1, estimated iterations: unknown
.loc 16 81 0
ld.global.f32 %f2, [%rd42+0];
st.shared.f32 [%rd30+0], %f2;
.loc 16 82 0
ld.global.f32 %f3, [%rd36+0];
st.shared.f32 [%rd31+0], %f3;
.loc 16 85 0
bar.sync 0;
.loc 16 92 0
ld.shared.f32 %f4, [%rd23+0];
ld.shared.f32 %f5, [%rd25+0];
mad.f32 %f6, %f4, %f5, %f1;
ld.shared.f32 %f7, [%rd23+4];
ld.shared.f32 %f8, [%rd25+64];
mad.f32 %f9, %f7, %f8, %f6;
ld.shared.f32 %f10, [%rd23+8];
ld.shared.f32 %f11, [%rd25+128];
mad.f32 %f12, %f10, %f11, %f9;
ld.shared.f32 %f13, [%rd23+12];
ld.shared.f32 %f14, [%rd25+192];
mad.f32 %f15, %f13, %f14, %f12;
ld.shared.f32 %f16, [%rd23+16];
ld.shared.f32 %f17, [%rd25+256];
mad.f32 %f18, %f16, %f17, %f15;
ld.shared.f32 %f19, [%rd23+20];
ld.shared.f32 %f20, [%rd25+320];
mad.f32 %f21, %f19, %f20, %f18;
ld.shared.f32 %f22, [%rd23+24];
ld.shared.f32 %f23, [%rd25+384];
mad.f32 %f24, %f22, %f23, %f21;
ld.shared.f32 %f25, [%rd23+28];
ld.shared.f32 %f26, [%rd25+448];
mad.f32 %f27, %f25, %f26, %f24;
ld.shared.f32 %f28, [%rd23+32];
ld.shared.f32 %f29, [%rd25+512];
mad.f32 %f30, %f28, %f29, %f27;
ld.shared.f32 %f31, [%rd23+36];
ld.shared.f32 %f32, [%rd25+576];
mad.f32 %f33, %f31, %f32, %f30;
ld.shared.f32 %f34, [%rd23+40];
ld.shared.f32 %f35, [%rd25+640];
mad.f32 %f36, %f34, %f35, %f33;
ld.shared.f32 %f37, [%rd23+44];
ld.shared.f32 %f38, [%rd25+704];
mad.f32 %f39, %f37, %f38, %f36;
ld.shared.f32 %f40, [%rd23+48];
ld.shared.f32 %f41, [%rd25+768];
mad.f32 %f42, %f40, %f41, %f39;
ld.shared.f32 %f43, [%rd23+52];
ld.shared.f32 %f44, [%rd25+832];
mad.f32 %f45, %f43, %f44, %f42;
ld.shared.f32 %f46, [%rd23+56];
ld.shared.f32 %f47, [%rd25+896];
mad.f32 %f48, %f46, %f47, %f45;
ld.shared.f32 %f49, [%rd23+60];
ld.shared.f32 %f50, [%rd25+960];
mad.f32 %f1, %f49, %f50, %f48;
.loc 16 97 0
bar.sync 0;
.loc 16 66 0
add.u64 %rd36, %rd29, %rd36;
add.u64 %rd7, %rd7, 16;
add.u64 %rd42, %rd42, 64;
setp.ge.u64 %p2, %rd9, %rd7;
@%p2 bra $Lt_1_2818;
bra.uni $Lt_1_2306;
$Lt_1_3330:
ld.param.u64 %rd12, [__cudaparm_matrixMul_bs16_64bit_wB];
mul.lo.u64 %rd21, %rd11, %rd12;
mov.f32 %f1, 0f00000000; // 0
$Lt_1_2306:
.loc 16 103 0
ld.param.u64 %rd44, [__cudaparm_matrixMul_bs16_64bit_C];
mul.lo.u64 %rd45, %rd12, %rd3;
add.u64 %rd46, %rd1, %rd45;
mul.lo.u64 %rd47, %rd46, 16;
add.u64 %rd48, %rd21, %rd47;
add.u64 %rd49, %rd10, %rd48;
mul.lo.u64 %rd50, %rd49, 4;
add.u64 %rd51, %rd44, %rd50;
st.global.f32 [%rd51+0], %f1;
.loc 16 114 0
exit;
$LDWend_matrixMul_bs16_64bit:
} // matrixMul_bs16_64bit
.shared .align 4 .b8 __cuda_local_var_87382_38_non_const_As__2[4096];
.shared .align 4 .b8 __cuda_local_var_87386_38_non_const_Bs__3[4096];
.entry matrixMul_bs32_32bit (
.param .u64 __cudaparm_matrixMul_bs32_32bit_C,
.param .u64 __cudaparm_matrixMul_bs32_32bit_A,
.param .u64 __cudaparm_matrixMul_bs32_32bit_B,
.param .s32 __cudaparm_matrixMul_bs32_32bit_wA,
.param .s32 __cudaparm_matrixMul_bs32_32bit_wB)
{
.reg .u32 %r<34>;
.reg .u64 %rd<29>;
.reg .f32 %f<100>;
.reg .pred %p<4>;
.loc 16 115 0
$LDWbegin_matrixMul_bs32_32bit:
.loc 16 66 0
cvt.s32.u16 %r1, %ctaid.x;
mul24.lo.s32 %r2, %r1, 32;
cvt.s32.u16 %r3, %ctaid.y;
ld.param.s32 %r4, [__cudaparm_matrixMul_bs32_32bit_wA];
mul.lo.s32 %r5, %r3, %r4;
mul.lo.s32 %r6, %r5, 32;
add.s32 %r7, %r6, %r4;
sub.s32 %r8, %r7, 1;
cvt.s32.u16 %r9, %tid.x;
cvt.s32.u16 %r10, %tid.y;
ld.param.s32 %r11, [__cudaparm_matrixMul_bs32_32bit_wB];
setp.lt.s32 %p1, %r8, %r6;
@%p1 bra $Lt_2_3330;
mov.u64 %rd1, __cuda_local_var_87386_38_non_const_Bs__3;
mov.u64 %rd2, __cuda_local_var_87382_38_non_const_As__2;
ld.param.s32 %r4, [__cudaparm_matrixMul_bs32_32bit_wA];
add.s32 %r12, %r4, 31;
shr.s32 %r13, %r12, 31;
mov.s32 %r14, 31;
and.b32 %r15, %r13, %r14;
add.s32 %r16, %r15, %r12;
shr.s32 %r17, %r16, 5;
ld.param.s32 %r11, [__cudaparm_matrixMul_bs32_32bit_wB];
mul.lo.s32 %r18, %r10, %r11;
mul.lo.s32 %r19, %r10, %r4;
cvt.s64.s32 %rd3, %r9;
cvt.s64.s32 %rd4, %r10;
add.s32 %r20, %r19, %r6;
add.s32 %r21, %r9, %r20;
mul.wide.s32 %rd5, %r9, 4;
add.u64 %rd6, %rd1, %rd5;
mul.wide.s32 %rd7, %r10, 128;
add.u64 %rd8, %rd2, %rd7;
mul.wide.s32 %rd9, %r10, 32;
add.u64 %rd10, %rd3, %rd9;
mul.lo.u64 %rd11, %rd10, 4;
add.s32 %r22, %r19, %r8;
mul.lo.s32 %r23, %r11, 32;
cvt.s64.s32 %rd12, %r23;
mul.wide.s32 %rd13, %r23, 4;
add.u64 %rd14, %rd11, %rd2;
add.u64 %rd15, %rd11, %rd1;
add.s32 %r24, %r22, %r9;
ld.param.u64 %rd16, [__cudaparm_matrixMul_bs32_32bit_B];
add.s32 %r25, %r18, %r2;
add.s32 %r26, %r9, %r25;
cvt.s64.s32 %rd17, %r26;
mul.wide.s32 %rd18, %r26, 4;
add.u64 %rd19, %rd16, %rd18;
ld.param.u64 %rd20, [__cudaparm_matrixMul_bs32_32bit_A];
cvt.s64.s32 %rd21, %r21;
mul.wide.s32 %rd22, %r21, 4;
add.u64 %rd23, %rd20, %rd22;
mov.f32 %f1, 0f00000000; // 0
mov.s32 %r27, %r17;
$Lt_2_2818:
//<loop> Loop body line 66, nesting depth: 1, estimated iterations: unknown
.loc 16 81 0
ld.global.f32 %f2, [%rd23+0];
st.shared.f32 [%rd14+0], %f2;
.loc 16 82 0
ld.global.f32 %f3, [%rd19+0];
st.shared.f32 [%rd15+0], %f3;
.loc 16 85 0
bar.sync 0;
.loc 16 92 0
ld.shared.f32 %f4, [%rd8+0];
ld.shared.f32 %f5, [%rd6+0];
mad.f32 %f6, %f4, %f5, %f1;
ld.shared.f32 %f7, [%rd8+4];
ld.shared.f32 %f8, [%rd6+128];
mad.f32 %f9, %f7, %f8, %f6;
ld.shared.f32 %f10, [%rd8+8];
ld.shared.f32 %f11, [%rd6+256];
mad.f32 %f12, %f10, %f11, %f9;
ld.shared.f32 %f13, [%rd8+12];
ld.shared.f32 %f14, [%rd6+384];
mad.f32 %f15, %f13, %f14, %f12;
ld.shared.f32 %f16, [%rd8+16];
ld.shared.f32 %f17, [%rd6+512];
mad.f32 %f18, %f16, %f17, %f15;
ld.shared.f32 %f19, [%rd8+20];
ld.shared.f32 %f20, [%rd6+640];
mad.f32 %f21, %f19, %f20, %f18;
ld.shared.f32 %f22, [%rd8+24];
ld.shared.f32 %f23, [%rd6+768];
mad.f32 %f24, %f22, %f23, %f21;
ld.shared.f32 %f25, [%rd8+28];
ld.shared.f32 %f26, [%rd6+896];
mad.f32 %f27, %f25, %f26, %f24;
ld.shared.f32 %f28, [%rd8+32];
ld.shared.f32 %f29, [%rd6+1024];
mad.f32 %f30, %f28, %f29, %f27;
ld.shared.f32 %f31, [%rd8+36];
ld.shared.f32 %f32, [%rd6+1152];
mad.f32 %f33, %f31, %f32, %f30;
ld.shared.f32 %f34, [%rd8+40];
ld.shared.f32 %f35, [%rd6+1280];
mad.f32 %f36, %f34, %f35, %f33;
ld.shared.f32 %f37, [%rd8+44];
ld.shared.f32 %f38, [%rd6+1408];
mad.f32 %f39, %f37, %f38, %f36;
ld.shared.f32 %f40, [%rd8+48];
ld.shared.f32 %f41, [%rd6+1536];
mad.f32 %f42, %f40, %f41, %f39;
ld.shared.f32 %f43, [%rd8+52];
ld.shared.f32 %f44, [%rd6+1664];
mad.f32 %f45, %f43, %f44, %f42;
ld.shared.f32 %f46, [%rd8+56];
ld.shared.f32 %f47, [%rd6+1792];
mad.f32 %f48, %f46, %f47, %f45;
ld.shared.f32 %f49, [%rd8+60];
ld.shared.f32 %f50, [%rd6+1920];
mad.f32 %f51, %f49, %f50, %f48;
ld.shared.f32 %f52, [%rd8+64];
ld.shared.f32 %f53, [%rd6+2048];
mad.f32 %f54, %f52, %f53, %f51;
ld.shared.f32 %f55, [%rd8+68];
ld.shared.f32 %f56, [%rd6+2176];
mad.f32 %f57, %f55, %f56, %f54;
ld.shared.f32 %f58, [%rd8+72];
ld.shared.f32 %f59, [%rd6+2304];
mad.f32 %f60, %f58, %f59, %f57;
ld.shared.f32 %f61, [%rd8+76];
ld.shared.f32 %f62, [%rd6+2432];
mad.f32 %f63, %f61, %f62, %f60;
ld.shared.f32 %f64, [%rd8+80];
ld.shared.f32 %f65, [%rd6+2560];
mad.f32 %f66, %f64, %f65, %f63;
ld.shared.f32 %f67, [%rd8+84];
ld.shared.f32 %f68, [%rd6+2688];
mad.f32 %f69, %f67, %f68, %f66;
ld.shared.f32 %f70, [%rd8+88];
ld.shared.f32 %f71, [%rd6+2816];
mad.f32 %f72, %f70, %f71, %f69;
ld.shared.f32 %f73, [%rd8+92];
ld.shared.f32 %f74, [%rd6+2944];
mad.f32 %f75, %f73, %f74, %f72;
ld.shared.f32 %f76, [%rd8+96];
ld.shared.f32 %f77, [%rd6+3072];
mad.f32 %f78, %f76, %f77, %f75;
ld.shared.f32 %f79, [%rd8+100];
ld.shared.f32 %f80, [%rd6+3200];
mad.f32 %f81, %f79, %f80, %f78;
ld.shared.f32 %f82, [%rd8+104];
ld.shared.f32 %f83, [%rd6+3328];
mad.f32 %f84, %f82, %f83, %f81;
ld.shared.f32 %f85, [%rd8+108];
ld.shared.f32 %f86, [%rd6+3456];
mad.f32 %f87, %f85, %f86, %f84;
ld.shared.f32 %f88, [%rd8+112];
ld.shared.f32 %f89, [%rd6+3584];
mad.f32 %f90, %f88, %f89, %f87;
ld.shared.f32 %f91, [%rd8+116];
ld.shared.f32 %f92, [%rd6+3712];
mad.f32 %f93, %f91, %f92, %f90;
ld.shared.f32 %f94, [%rd8+120];
ld.shared.f32 %f95, [%rd6+3840];
mad.f32 %f96, %f94, %f95, %f93;
ld.shared.f32 %f97, [%rd8+124];
ld.shared.f32 %f98, [%rd6+3968];
mad.f32 %f1, %f97, %f98, %f96;
.loc 16 97 0
bar.sync 0;
.loc 16 66 0
add.u64 %rd19, %rd13, %rd19;
add.s32 %r21, %r21, 32;
add.u64 %rd23, %rd23, 128;
setp.le.s32 %p2, %r21, %r24;
@%p2 bra $Lt_2_2818;
bra.uni $Lt_2_2306;
$Lt_2_3330:
ld.param.s32 %r11, [__cudaparm_matrixMul_bs32_32bit_wB];
mul.lo.s32 %r18, %r10, %r11;
mov.f32 %f1, 0f00000000; // 0
$Lt_2_2306:
.loc 16 103 0
ld.param.u64 %rd24, [__cudaparm_matrixMul_bs32_32bit_C];
mul.lo.s32 %r28, %r11, %r3;
add.s32 %r29, %r1, %r28;
mul.lo.s32 %r30, %r29, 32;
add.s32 %r31, %r18, %r30;
add.s32 %r32, %r9, %r31;
cvt.s64.s32 %rd25, %r32;
mul.wide.s32 %rd26, %r32, 4;
add.u64 %rd27, %rd24, %rd26;
st.global.f32 [%rd27+0], %f1;
.loc 16 118 0
exit;
$LDWend_matrixMul_bs32_32bit:
} // matrixMul_bs32_32bit
.shared .align 4 .b8 __cuda_local_var_87382_38_non_const_As__0[4096];
.shared .align 4 .b8 __cuda_local_var_87386_38_non_const_Bs__1[4096];
.entry matrixMul_bs32_64bit (
.param .u64 __cudaparm_matrixMul_bs32_64bit_C,
.param .u64 __cudaparm_matrixMul_bs32_64bit_A,
.param .u64 __cudaparm_matrixMul_bs32_64bit_B,
.param .u64 __cudaparm_matrixMul_bs32_64bit_wA,
.param .u64 __cudaparm_matrixMul_bs32_64bit_wB)
{
.reg .u64 %rd<53>;
.reg .f32 %f<100>;
.reg .pred %p<4>;
.loc 16 119 0
$LDWbegin_matrixMul_bs32_64bit:
.loc 16 66 0
cvt.u64.u16 %rd1, %ctaid.x;
mul.lo.u64 %rd2, %rd1, 32;
cvt.u64.u16 %rd3, %ctaid.y;
ld.param.u64 %rd4, [__cudaparm_matrixMul_bs32_64bit_wA];
mul.lo.u64 %rd5, %rd3, %rd4;
mul.lo.u64 %rd6, %rd5, 32;
mov.s64 %rd7, %rd6;
add.u64 %rd8, %rd6, %rd4;
sub.u64 %rd9, %rd8, 1;
cvt.u64.u16 %rd10, %tid.x;
cvt.u64.u16 %rd11, %tid.y;
ld.param.u64 %rd12, [__cudaparm_matrixMul_bs32_64bit_wB];
setp.lt.u64 %p1, %rd9, %rd6;
@%p1 bra $Lt_3_3330;
mov.u64 %rd13, __cuda_local_var_87386_38_non_const_Bs__1;
mov.u64 %rd14, __cuda_local_var_87382_38_non_const_As__0;
ld.param.u64 %rd4, [__cudaparm_matrixMul_bs32_64bit_wA];
add.u64 %rd15, %rd4, 31;
shr.s64 %rd16, %rd15, 63;
mov.s64 %rd17, 31;
and.b64 %rd18, %rd16, %rd17;
add.s64 %rd19, %rd18, %rd15;
shr.s64 %rd20, %rd19, 5;
ld.param.u64 %rd12, [__cudaparm_matrixMul_bs32_64bit_wB];
mul.lo.u64 %rd21, %rd11, %rd12;
mul.lo.u64 %rd22, %rd11, 128;
add.u64 %rd23, %rd14, %rd22;
mul.lo.u64 %rd24, %rd10, 4;
add.u64 %rd25, %rd13, %rd24;
mul.lo.u64 %rd26, %rd11, 32;
add.u64 %rd27, %rd10, %rd26;
mul.lo.u64 %rd28, %rd27, 4;
mul.lo.u64 %rd29, %rd12, 128;
add.u64 %rd30, %rd28, %rd14;
add.u64 %rd31, %rd28, %rd13;
ld.param.u64 %rd32, [__cudaparm_matrixMul_bs32_64bit_B];
add.u64 %rd33, %rd21, %rd2;
add.u64 %rd34, %rd10, %rd33;
mul.lo.u64 %rd35, %rd34, 4;
add.u64 %rd36, %rd32, %rd35;
ld.param.u64 %rd37, [__cudaparm_matrixMul_bs32_64bit_A];
mul.lo.u64 %rd38, %rd11, %rd4;
add.u64 %rd39, %rd6, %rd38;
add.u64 %rd40, %rd10, %rd39;
mul.lo.u64 %rd41, %rd40, 4;
add.u64 %rd42, %rd37, %rd41;
mov.f32 %f1, 0f00000000; // 0
mov.s64 %rd43, %rd20;
$Lt_3_2818:
//<loop> Loop body line 66, nesting depth: 1, estimated iterations: unknown
.loc 16 81 0
ld.global.f32 %f2, [%rd42+0];
st.shared.f32 [%rd30+0], %f2;
.loc 16 82 0
ld.global.f32 %f3, [%rd36+0];
st.shared.f32 [%rd31+0], %f3;
.loc 16 85 0
bar.sync 0;
.loc 16 92 0
ld.shared.f32 %f4, [%rd23+0];
ld.shared.f32 %f5, [%rd25+0];
mad.f32 %f6, %f4, %f5, %f1;
ld.shared.f32 %f7, [%rd23+4];
ld.shared.f32 %f8, [%rd25+128];
mad.f32 %f9, %f7, %f8, %f6;
ld.shared.f32 %f10, [%rd23+8];
ld.shared.f32 %f11, [%rd25+256];
mad.f32 %f12, %f10, %f11, %f9;
ld.shared.f32 %f13, [%rd23+12];
ld.shared.f32 %f14, [%rd25+384];
mad.f32 %f15, %f13, %f14, %f12;
ld.shared.f32 %f16, [%rd23+16];
ld.shared.f32 %f17, [%rd25+512];
mad.f32 %f18, %f16, %f17, %f15;
ld.shared.f32 %f19, [%rd23+20];
ld.shared.f32 %f20, [%rd25+640];
mad.f32 %f21, %f19, %f20, %f18;
ld.shared.f32 %f22, [%rd23+24];
ld.shared.f32 %f23, [%rd25+768];
mad.f32 %f24, %f22, %f23, %f21;
ld.shared.f32 %f25, [%rd23+28];
ld.shared.f32 %f26, [%rd25+896];
mad.f32 %f27, %f25, %f26, %f24;
ld.shared.f32 %f28, [%rd23+32];
ld.shared.f32 %f29, [%rd25+1024];
mad.f32 %f30, %f28, %f29, %f27;
ld.shared.f32 %f31, [%rd23+36];
ld.shared.f32 %f32, [%rd25+1152];
mad.f32 %f33, %f31, %f32, %f30;
ld.shared.f32 %f34, [%rd23+40];
ld.shared.f32 %f35, [%rd25+1280];
mad.f32 %f36, %f34, %f35, %f33;
ld.shared.f32 %f37, [%rd23+44];
ld.shared.f32 %f38, [%rd25+1408];
mad.f32 %f39, %f37, %f38, %f36;
ld.shared.f32 %f40, [%rd23+48];
ld.shared.f32 %f41, [%rd25+1536];
mad.f32 %f42, %f40, %f41, %f39;
ld.shared.f32 %f43, [%rd23+52];
ld.shared.f32 %f44, [%rd25+1664];
mad.f32 %f45, %f43, %f44, %f42;
ld.shared.f32 %f46, [%rd23+56];
ld.shared.f32 %f47, [%rd25+1792];
mad.f32 %f48, %f46, %f47, %f45;
ld.shared.f32 %f49, [%rd23+60];
ld.shared.f32 %f50, [%rd25+1920];
mad.f32 %f51, %f49, %f50, %f48;
ld.shared.f32 %f52, [%rd23+64];
ld.shared.f32 %f53, [%rd25+2048];
mad.f32 %f54, %f52, %f53, %f51;
ld.shared.f32 %f55, [%rd23+68];
ld.shared.f32 %f56, [%rd25+2176];
mad.f32 %f57, %f55, %f56, %f54;
ld.shared.f32 %f58, [%rd23+72];
ld.shared.f32 %f59, [%rd25+2304];
mad.f32 %f60, %f58, %f59, %f57;
ld.shared.f32 %f61, [%rd23+76];
ld.shared.f32 %f62, [%rd25+2432];
mad.f32 %f63, %f61, %f62, %f60;
ld.shared.f32 %f64, [%rd23+80];
ld.shared.f32 %f65, [%rd25+2560];
mad.f32 %f66, %f64, %f65, %f63;
ld.shared.f32 %f67, [%rd23+84];
ld.shared.f32 %f68, [%rd25+2688];
mad.f32 %f69, %f67, %f68, %f66;
ld.shared.f32 %f70, [%rd23+88];
ld.shared.f32 %f71, [%rd25+2816];
mad.f32 %f72, %f70, %f71, %f69;
ld.shared.f32 %f73, [%rd23+92];
ld.shared.f32 %f74, [%rd25+2944];
mad.f32 %f75, %f73, %f74, %f72;
ld.shared.f32 %f76, [%rd23+96];
ld.shared.f32 %f77, [%rd25+3072];
mad.f32 %f78, %f76, %f77, %f75;
ld.shared.f32 %f79, [%rd23+100];
ld.shared.f32 %f80, [%rd25+3200];
mad.f32 %f81, %f79, %f80, %f78;
ld.shared.f32 %f82, [%rd23+104];
ld.shared.f32 %f83, [%rd25+3328];
mad.f32 %f84, %f82, %f83, %f81;
ld.shared.f32 %f85, [%rd23+108];
ld.shared.f32 %f86, [%rd25+3456];
mad.f32 %f87, %f85, %f86, %f84;
ld.shared.f32 %f88, [%rd23+112];
ld.shared.f32 %f89, [%rd25+3584];
mad.f32 %f90, %f88, %f89, %f87;
ld.shared.f32 %f91, [%rd23+116];
ld.shared.f32 %f92, [%rd25+3712];
mad.f32 %f93, %f91, %f92, %f90;
ld.shared.f32 %f94, [%rd23+120];
ld.shared.f32 %f95, [%rd25+3840];
mad.f32 %f96, %f94, %f95, %f93;
ld.shared.f32 %f97, [%rd23+124];
ld.shared.f32 %f98, [%rd25+3968];
mad.f32 %f1, %f97, %f98, %f96;
.loc 16 97 0
bar.sync 0;
.loc 16 66 0
add.u64 %rd36, %rd29, %rd36;
add.u64 %rd7, %rd7, 32;
add.u64 %rd42, %rd42, 128;
setp.ge.u64 %p2, %rd9, %rd7;
@%p2 bra $Lt_3_2818;
bra.uni $Lt_3_2306;
$Lt_3_3330:
ld.param.u64 %rd12, [__cudaparm_matrixMul_bs32_64bit_wB];
mul.lo.u64 %rd21, %rd11, %rd12;
mov.f32 %f1, 0f00000000; // 0
$Lt_3_2306:
.loc 16 103 0
ld.param.u64 %rd44, [__cudaparm_matrixMul_bs32_64bit_C];
mul.lo.u64 %rd45, %rd12, %rd3;
add.u64 %rd46, %rd1, %rd45;
mul.lo.u64 %rd47, %rd46, 32;
add.u64 %rd48, %rd21, %rd47;
add.u64 %rd49, %rd10, %rd48;
mul.lo.u64 %rd50, %rd49, 4;
add.u64 %rd51, %rd44, %rd50;
st.global.f32 [%rd51+0], %f1;
.loc 16 122 0
exit;
$LDWend_matrixMul_bs32_64bit:
} // matrixMul_bs32_64bit

91
cpp/0_Introduction/matrixMulDynlinkJIT/extras/ptx2c.py Normal file
View File

@@ -0,0 +1,91 @@
#!/usr/bin/env python
from string import *
import os, getopt, sys, platform
g_Header = '''/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
////////////////////////////////////////////////////////////////////////////////
// This file is auto-generated, do not edit
////////////////////////////////////////////////////////////////////////////////
'''
def Usage():
print("Usage: ptx2c.py in out")
print("Description: performs embedding in.cubin or in.ptx file into out.c and out.h files as character array")
sys.exit(0)
def FormatCharHex(d):
s = hex(ord(d))
if len(s) == 3:
s = "0x0" + s[2]
return s
args = sys.argv[1:]
if not(len(sys.argv[1:]) == 2):
Usage()
out_h = args[1] + "_ptxdump.h"
out_c = args[1] + "_ptxdump.c"
h_in = open(args[0], 'r')
source_bytes = h_in.read()
source_bytes_len = len(source_bytes)
h_out_c = open(out_c, 'w')
h_out_c.writelines(g_Header)
h_out_c.writelines("#include \"" + out_h + "\"\n\n")
h_out_c.writelines("unsigned char " + args[1] + "_ptxdump[" + str(source_bytes_len+1) + "] = {\n")
h_out_h = open(out_h, 'w')
macro_h = "__" + args[1] + "_ptxdump_h__"
h_out_h.writelines(g_Header)
h_out_h.writelines("#ifndef " + macro_h + "\n")
h_out_h.writelines("#define " + macro_h + "\n\n")
h_out_h.writelines('#if defined __cplusplus\nextern "C" {\n#endif\n\n')
h_out_h.writelines("extern unsigned char " + args[1] + "_ptxdump[" + str(source_bytes_len+1) + "];\n\n")
h_out_h.writelines("#if defined __cplusplus\n}\n#endif\n\n")
h_out_h.writelines("#endif //" + macro_h + "\n")
newlinecnt = 0
for i in range(0, source_bytes_len):
h_out_c.write(FormatCharHex(source_bytes[i]) + ", ")
newlinecnt += 1
if newlinecnt == 16:
newlinecnt = 0
h_out_c.write("\n")
h_out_c.write("0x00\n};\n")
h_in.close()
h_out_c.close()
h_out_h.close()
print("ptx2c: CUmodule " + args[0] + " packed successfully")

354
cpp/0_Introduction/matrixMulDynlinkJIT/helper_cuda_drvapi.h Normal file
View File

@@ -0,0 +1,354 @@
/**
* Copyright 1993-2017 NVIDIA Corporation. All rights reserved.
*
* Please refer to the NVIDIA end user license agreement (EULA) associated
* with this source code for terms and conditions that govern your use of
* this software. Any use, reproduction, disclosure, or distribution of
* this software and related documentation outside the terms of the EULA
* is strictly prohibited.
*
*/
// Helper functions for CUDA Driver API error handling (make sure that CUDA_H is
// included in your projects)
#ifndef HELPER_CUDA_DRVAPI_H
#define HELPER_CUDA_DRVAPI_H
#include <helper_string.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#ifndef MAX
#define MAX(a, b) (a > b ? a : b)
#endif
#ifndef HELPER_CUDA_DRVAPI_H
inline int ftoi(float value) { return (value >= 0 ? static_cast<int>(value + 0.5) : static_cast<int>(value - 0.5)); }
#endif
#ifndef EXIT_WAIVED
#define EXIT_WAIVED 2
#endif
////////////////////////////////////////////////////////////////////////////////
// These are CUDA Helper functions
// add a level of protection to the CUDA SDK samples, let's force samples to
// explicitly include CUDA.H
#ifdef __cuda_cuda_h__
// This will output the proper CUDA error strings in the event that a CUDA host
// call returns an error
#ifndef checkCudaErrors
#define checkCudaErrors(err) __checkCudaErrors(err, __FILE__, __LINE__)
extern "C" CUresult INIT_ERROR_FUNCTIONS(void);
// These are the inline versions for all of the SDK helper functions
inline void __checkCudaErrors(CUresult err, const char *file, const int line)
{
if (CUDA_SUCCESS != err) {
const char *errorStr = NULL;
if (!cuGetErrorString) {
CUresult result = INIT_ERROR_FUNCTIONS();
if (result != CUDA_SUCCESS) {
printf("CUDA driver API failed");
exit(EXIT_FAILURE);
}
}
cuGetErrorString(err, &errorStr);
fprintf(stderr,
"checkCudaErrors() Driver API error = %04d \"%s\" from file <%s>, "
"line %i.\n",
err,
errorStr,
file,
line);
exit(EXIT_FAILURE);
}
}
#endif
// This function wraps the CUDA Driver API into a template function
template <class T> inline void getCudaAttribute(T *attribute, CUdevice_attribute device_attribute, int device)
{
checkCudaErrors(cuDeviceGetAttribute(attribute, device_attribute, device));
}
#endif
// Beginning of GPU Architecture definitions
inline int _ConvertSMVer2CoresDRV(int major, int minor)
{
// Defines for GPU Architecture types (using the SM version to determine the #
// of cores per SM
typedef struct
{
int SM; // 0xMm (hexidecimal notation), M = SM Major version, and m = SM
// minor version
int Cores;
} sSMtoCores;
sSMtoCores nGpuArchCoresPerSM[] = {{0x30, 192},
{0x32, 192},
{0x35, 192},
{0x37, 192},
{0x50, 128},
{0x52, 128},
{0x53, 128},
{0x60, 64},
{0x61, 128},
{0x62, 128},
{0x70, 64},
{0x72, 64},
{0x75, 64},
{0x80, 64},
{0x86, 128},
{0x87, 128},
{0x90, 128},
{-1, -1}};
int index = 0;
while (nGpuArchCoresPerSM[index].SM != -1) {
if (nGpuArchCoresPerSM[index].SM == ((major << 4) + minor)) {
return nGpuArchCoresPerSM[index].Cores;
}
index++;
}
// If we don't find the values, we default use the previous one to run
// properly
printf("MapSMtoCores for SM %d.%d is undefined. Default to use %d Cores/SM\n",
major,
minor,
nGpuArchCoresPerSM[index - 1].Cores);
return nGpuArchCoresPerSM[index - 1].Cores;
}
// end of GPU Architecture definitions
#ifdef __cuda_cuda_h__
// General GPU Device CUDA Initialization
inline int gpuDeviceInitDRV(int ARGC, const char **ARGV)
{
int cuDevice = 0;
int deviceCount = 0;
checkCudaErrors(cuInit(0, __CUDA_API_VERSION));
checkCudaErrors(cuDeviceGetCount(&deviceCount));
if (deviceCount == 0) {
fprintf(stderr, "cudaDeviceInit error: no devices supporting CUDA\n");
exit(EXIT_FAILURE);
}
int dev = 0;
dev = getCmdLineArgumentInt(ARGC, (const char **)ARGV, "device=");
if (dev < 0) {
dev = 0;
}
if (dev > deviceCount - 1) {
fprintf(stderr, "\n");
fprintf(stderr, ">> %d CUDA capable GPU device(s) detected. <<\n", deviceCount);
fprintf(stderr, ">> cudaDeviceInit (-device=%d) is not a valid GPU device. <<\n", dev);
fprintf(stderr, "\n");
return -dev;
}
checkCudaErrors(cuDeviceGet(&cuDevice, dev));
char name[100];
checkCudaErrors(cuDeviceGetName(name, 100, cuDevice));
int computeMode;
getCudaAttribute<int>(&computeMode, CU_DEVICE_ATTRIBUTE_COMPUTE_MODE, dev);
if (computeMode == CU_COMPUTEMODE_PROHIBITED) {
fprintf(stderr,
"Error: device is running in <CU_COMPUTEMODE_PROHIBITED>, no "
"threads can use this CUDA Device.\n");
return -1;
}
if (checkCmdLineFlag(ARGC, (const char **)ARGV, "quiet") == false) {
printf("gpuDeviceInitDRV() Using CUDA Device [%d]: %s\n", dev, name);
}
return dev;
}
// This function returns the best GPU based on performance
inline int gpuGetMaxGflopsDeviceIdDRV()
{
CUdevice current_device = 0;
CUdevice max_perf_device = 0;
int device_count = 0;
int sm_per_multiproc = 0;
unsigned long long max_compute_perf = 0;
int major = 0;
int minor = 0;
int multiProcessorCount;
int clockRate;
int devices_prohibited = 0;
cuInit(0, __CUDA_API_VERSION);
checkCudaErrors(cuDeviceGetCount(&device_count));
if (device_count == 0) {
fprintf(stderr, "gpuGetMaxGflopsDeviceIdDRV error: no devices supporting CUDA\n");
exit(EXIT_FAILURE);
}
// Find the best CUDA capable GPU device
current_device = 0;
while (current_device < device_count) {
checkCudaErrors(
cuDeviceGetAttribute(&multiProcessorCount, CU_DEVICE_ATTRIBUTE_MULTIPROCESSOR_COUNT, current_device));
checkCudaErrors(cuDeviceGetAttribute(&clockRate, CU_DEVICE_ATTRIBUTE_CLOCK_RATE, current_device));
checkCudaErrors(cuDeviceGetAttribute(&major, CU_DEVICE_ATTRIBUTE_COMPUTE_CAPABILITY_MAJOR, current_device));
checkCudaErrors(cuDeviceGetAttribute(&minor, CU_DEVICE_ATTRIBUTE_COMPUTE_CAPABILITY_MINOR, current_device));
int computeMode;
getCudaAttribute<int>(&computeMode, CU_DEVICE_ATTRIBUTE_COMPUTE_MODE, current_device);
if (computeMode != CU_COMPUTEMODE_PROHIBITED) {
if (major == 9999 && minor == 9999) {
sm_per_multiproc = 1;
}
else {
sm_per_multiproc = _ConvertSMVer2CoresDRV(major, minor);
}
unsigned long long compute_perf = (unsigned long long)(multiProcessorCount * sm_per_multiproc * clockRate);
if (compute_perf > max_compute_perf) {
max_compute_perf = compute_perf;
max_perf_device = current_device;
}
}
else {
devices_prohibited++;
}
++current_device;
}
if (devices_prohibited == device_count) {
fprintf(stderr,
"gpuGetMaxGflopsDeviceIdDRV error: all devices have compute mode "
"prohibited.\n");
exit(EXIT_FAILURE);
}
return max_perf_device;
}
// General initialization call to pick the best CUDA Device
inline CUdevice findCudaDeviceDRV(int argc, const char **argv)
{
CUdevice cuDevice;
int devID = 0;
// If the command-line has a device number specified, use it
if (checkCmdLineFlag(argc, (const char **)argv, "device")) {
devID = gpuDeviceInitDRV(argc, argv);
if (devID < 0) {
printf("exiting...\n");
exit(EXIT_SUCCESS);
}
}
else {
// Otherwise pick the device with highest Gflops/s
char name[100];
devID = gpuGetMaxGflopsDeviceIdDRV();
checkCudaErrors(cuDeviceGet(&cuDevice, devID));
cuDeviceGetName(name, 100, cuDevice);
printf("> Using CUDA Device [%d]: %s\n", devID, name);
}
cuDeviceGet(&cuDevice, devID);
return cuDevice;
}
inline CUdevice findIntegratedGPUDrv()
{
CUdevice current_device = 0;
int device_count = 0;
int devices_prohibited = 0;
int isIntegrated;
cuInit(0, __CUDA_API_VERSION);
checkCudaErrors(cuDeviceGetCount(&device_count));
if (device_count == 0) {
fprintf(stderr, "CUDA error: no devices supporting CUDA.\n");
exit(EXIT_FAILURE);
}
// Find the integrated GPU which is compute capable
while (current_device < device_count) {
int computeMode = -1;
checkCudaErrors(cuDeviceGetAttribute(&isIntegrated, CU_DEVICE_ATTRIBUTE_INTEGRATED, current_device));
checkCudaErrors(cuDeviceGetAttribute(&computeMode, CU_DEVICE_ATTRIBUTE_COMPUTE_MODE, current_device));
// If GPU is integrated and is not running on Compute Mode prohibited use
// that
if (isIntegrated && (computeMode != CU_COMPUTEMODE_PROHIBITED)) {
int major = 0, minor = 0;
char deviceName[256];
checkCudaErrors(cuDeviceGetAttribute(&major, CU_DEVICE_ATTRIBUTE_COMPUTE_CAPABILITY_MAJOR, current_device));
checkCudaErrors(cuDeviceGetAttribute(&minor, CU_DEVICE_ATTRIBUTE_COMPUTE_CAPABILITY_MINOR, current_device));
checkCudaErrors(cuDeviceGetName(deviceName, 256, current_device));
printf("GPU Device %d: \"%s\" with compute capability %d.%d\n\n", current_device, deviceName, major, minor);
return current_device;
}
else {
devices_prohibited++;
}
current_device++;
}
if (devices_prohibited == device_count) {
fprintf(stderr, "CUDA error: No Integrated CUDA capable GPU found.\n");
exit(EXIT_FAILURE);
}
return -1;
}
// General check for CUDA GPU SM Capabilities
inline bool checkCudaCapabilitiesDRV(int major_version, int minor_version, int devID)
{
CUdevice cuDevice;
char name[256];
int major = 0, minor = 0;
checkCudaErrors(cuDeviceGet(&cuDevice, devID));
checkCudaErrors(cuDeviceGetName(name, 100, cuDevice));
checkCudaErrors(cuDeviceGetAttribute(&major, CU_DEVICE_ATTRIBUTE_COMPUTE_CAPABILITY_MAJOR, cuDevice));
checkCudaErrors(cuDeviceGetAttribute(&minor, CU_DEVICE_ATTRIBUTE_COMPUTE_CAPABILITY_MINOR, cuDevice));
if ((major > major_version) || (major == major_version && minor >= minor_version)) {
printf("> Device %d: <%16s >, Compute SM %d.%d detected\n", devID, name, major, minor);
return true;
}
else {
printf("No GPU device was found that can support CUDA compute capability "
"%d.%d.\n",
major_version,
minor_version);
return false;
}
}
#endif
// end of CUDA Helper Functions
#endif // HELPER_CUDA_DRVAPI_H

41
cpp/0_Introduction/matrixMulDynlinkJIT/matrixMul.h Normal file
View File

@@ -0,0 +1,41 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifndef _MATRIXMUL_H_
#define _MATRIXMUL_H_
// Matrix dimensions
// (chosen as multiples of the thread block size for simplicity)
#define WA (4 * block_size) // Matrix A width
#define HA (6 * block_size) // Matrix A height
#define WB (4 * block_size) // Matrix B width
#define HB WA // Matrix B height
#define WC WB // Matrix C width
#define HC HA // Matrix C height
#endif // _MATRIXMUL_H_

347
cpp/0_Introduction/matrixMulDynlinkJIT/matrixMulDynlinkJIT.cpp Normal file
View File

@@ -0,0 +1,347 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/* Matrix multiplication: C = A * B.
* Host code.
*
* This sample revisits matrix multiplication with CUDA task. The code of matrix
* multiplication is exactly the same as in matrixMulDrv sample of this SDK.
* This sample, however, demonstrates how to link CUDA driver at runtime and
* how to perform JIT (just-in-time) compilation of CUDA kernel from PTX image,
* stored in memory.
*
* For more details on acquiring auto-generated sources refer README.TXT file
* in "extras" directory.
*
* Unlike CUBLAS, the sample doesn't address high-performance matrix
* multiplication.
*/
// includes, system
#include <math.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
// includes, CUDA
#include "cuda_drvapi_dynlink.h"
#include "helper_cuda_drvapi.h"
// includes, project
#include "matrixMul.h"
#include "matrixMul_kernel_32_ptxdump.h"
#include "matrixMul_kernel_64_ptxdump.h"
extern "C" void computeGold(float *, const float *, const float *, unsigned int, unsigned int, unsigned int);
#if defined _MSC_VER
#pragma warning(disable : 4312)
#endif
////////////////////////////////////////////////////////////////////////////////
// Globals
////////////////////////////////////////////////////////////////////////////////
CUcontext g_cuContext;
CUmodule g_cuModule;
bool noprompt = false;
static const char *sSDKsample = "matrixMulDynlinkJIT (CUDA dynamic linking)";
////////////////////////////////////////////////////////////////////////////////
// Allocates a matrix with random float entries
////////////////////////////////////////////////////////////////////////////////
void randomInit(float *data, size_t size)
{
for (size_t i = 0; i < size; ++i) {
data[i] = rand() / (float)RAND_MAX;
}
}
////////////////////////////////////////////////////////////////////////////////
// CUDA driver runtime linking and initialization
////////////////////////////////////////////////////////////////////////////////
CUresult initCUDA(int argc, char **argv, CUfunction *pMatrixMul, int *block_size_out)
{
CUresult status;
CUdevice cuDevice;
CUfunction cuFunction;
int major, minor, block_size, devID = 0;
char deviceName[256];
// link to cuda driver dynamically
checkCudaErrors(cuInit(0, __CUDA_API_VERSION));
// This assumes that the user is attempting to specify a explicit device -device=n
if (argc > 1) {
bool bFound = false;
for (int param = 0; param < argc; param++) {
if (!strncmp(argv[param], "-device", 7)) {
int i = (int)strlen(argv[1]);
while (argv[1][i] != '=') {
i--;
}
devID = atoi(&argv[1][++i]);
bFound = true;
}
if (bFound)
break;
}
}
// get cuda-capable device count
int deviceCount = 0;
checkCudaErrors(cuDeviceGetCount(&deviceCount));
if (deviceCount == 0) {
fprintf(stderr, "No devices supporting CUDA detected, exiting...\n");
exit(EXIT_SUCCESS);
}
if (devID < 0)
devID = 0;
if (devID > deviceCount - 1) {
fprintf(stderr, "initCUDA (Device=%d) invalid GPU device. %d GPU device(s) detected.\n\n", devID, deviceCount);
status = CUDA_ERROR_NOT_FOUND;
cuCtxDestroy(g_cuContext);
exit(EXIT_FAILURE);
}
// pick up device with zero ordinal (default, or devID)
checkCudaErrors(cuDeviceGet(&cuDevice, devID));
// get compute capabilities and the devicename
checkCudaErrors(cuDeviceComputeCapability(&major, &minor, cuDevice));
checkCudaErrors(cuDeviceGetName(deviceName, 256, cuDevice));
printf("> Device %d: \"%s\" with Compute %d.%d capability\n", cuDevice, deviceName, major, minor);
block_size = 32;
*block_size_out = block_size;
// create context for picked device
status = cuCtxCreate(&g_cuContext, 0, cuDevice);
if (CUDA_SUCCESS != status) {
cuCtxDestroy(g_cuContext);
exit(EXIT_SUCCESS);
}
// setup JIT compilation options and perform compilation
{
// in this branch we use compilation with parameters
const unsigned int jitNumOptions = 3;
CUjit_option *jitOptions = new CUjit_option[jitNumOptions];
void **jitOptVals = new void *[jitNumOptions];
// set up size of compilation log buffer
jitOptions[0] = CU_JIT_INFO_LOG_BUFFER_SIZE_BYTES;
int jitLogBufferSize = 1024;
jitOptVals[0] = (void *)(size_t)jitLogBufferSize;
// set up pointer to the compilation log buffer
jitOptions[1] = CU_JIT_INFO_LOG_BUFFER;
char *jitLogBuffer = new char[jitLogBufferSize];
jitOptVals[1] = jitLogBuffer;
// set up pointer to set the Maximum # of registers for a particular kernel
jitOptions[2] = CU_JIT_MAX_REGISTERS;
int jitRegCount = 32;
jitOptVals[2] = (void *)(size_t)jitRegCount;
// compile with set parameters
printf("> Compiling CUDA module\n");
#if defined(_WIN64) || defined(__LP64__)
status = cuModuleLoadDataEx(
&g_cuModule, matrixMul_kernel_64_ptxdump, jitNumOptions, jitOptions, (void **)jitOptVals);
#else
status = cuModuleLoadDataEx(
&g_cuModule, matrixMul_kernel_32_ptxdump, jitNumOptions, jitOptions, (void **)jitOptVals);
#endif
printf("> PTX JIT log:\n%s\n", jitLogBuffer);
delete[] jitOptions;
delete[] jitOptVals;
delete[] jitLogBuffer;
}
if (CUDA_SUCCESS != status) {
printf("Error while compiling PTX\n");
cuCtxDestroy(g_cuContext);
exit(EXIT_FAILURE);
}
// retrieve CUDA function from the compiled module
status = cuModuleGetFunction(
&cuFunction, g_cuModule, (block_size == 16) ? "matrixMul_bs16_32bit" : "matrixMul_bs32_32bit");
if (CUDA_SUCCESS != status) {
cuModuleUnload(g_cuModule);
cuCtxDestroy(g_cuContext);
exit(EXIT_FAILURE);
}
*pMatrixMul = cuFunction;
return CUDA_SUCCESS;
}
////////////////////////////////////////////////////////////////////////////////
// Entry point
////////////////////////////////////////////////////////////////////////////////
int main(int argc, char **argv)
{
printf("[ %s ]\n", sSDKsample);
// initialize CUDA
CUfunction matrixMul = NULL;
int block_size = 0;
checkCudaErrors(initCUDA(argc, argv, &matrixMul, &block_size));
// set seed for rand()
srand(2006);
// allocate host memory for matrices A and B
size_t size_A = WA * HA;
size_t mem_size_A = sizeof(float) * size_A;
size_t size_B = WB * HB;
size_t mem_size_B = sizeof(float) * size_B;
float *h_A = (float *)malloc(mem_size_A);
float *h_B = (float *)malloc(mem_size_B);
// initialize host memory
randomInit(h_A, size_A);
randomInit(h_B, size_B);
// allocate device memory
CUdeviceptr d_A;
checkCudaErrors(cuMemAlloc(&d_A, mem_size_A));
CUdeviceptr d_B;
checkCudaErrors(cuMemAlloc(&d_B, mem_size_B));
// copy host memory to device
checkCudaErrors(cuMemcpyHtoD(d_A, h_A, mem_size_A));
checkCudaErrors(cuMemcpyHtoD(d_B, h_B, mem_size_B));
// allocate device memory for result
size_t size_C = WC * HC;
size_t mem_size_C = sizeof(float) * size_C;
CUdeviceptr d_C;
checkCudaErrors(cuMemAlloc(&d_C, mem_size_C));
// allocate mem for the result on host side
float *h_C = (float *)malloc(mem_size_C);
#if __CUDA_API_VERSION >= 4000
{
// This is the new CUDA 4.0 API for Kernel Parameter passing and Kernel Launching (simpler method)
int Matrix_Width_A = WA;
int Matrix_Width_B = WB;
void *args[5] = {&d_C, &d_A, &d_B, &Matrix_Width_A, &Matrix_Width_B};
checkCudaErrors(cuLaunchKernel(
matrixMul, (WC / block_size), (HC / block_size), 1, block_size, block_size, 1, 0, NULL, args, NULL));
}
#else // __CUDA_API_VERSION <= 3020
{
// This is the older CUDA Driver API for Kernel Parameter passing and Kernel Launching
int offset = 0;
{
// setup execution parameters
checkCudaErrors(cuParamSetv(matrixMul, offset, &d_C, sizeof(d_C)));
offset += sizeof(d_C);
checkCudaErrors(cuParamSetv(matrixMul, offset, &d_A, sizeof(d_A)));
offset += sizeof(d_A);
checkCudaErrors(cuParamSetv(matrixMul, offset, &d_B, sizeof(d_B)));
offset += sizeof(d_B);
}
int Matrix_Width_A = WA;
int Matrix_Width_B = WB;
checkCudaErrors(cuParamSeti(matrixMul, offset, Matrix_Width_A));
offset += sizeof(Matrix_Width_A);
checkCudaErrors(cuParamSeti(matrixMul, offset, Matrix_Width_B));
offset += sizeof(Matrix_Width_B);
checkCudaErrors(cuParamSetSize(matrixMul, offset));
checkCudaErrors(cuFuncSetBlockShape(matrixMul, block_size, block_size, 1));
checkCudaErrors(cuFuncSetSharedSize(matrixMul, 2 * block_size * block_size * sizeof(float)));
// set execution configuration for the CUDA kernel
checkCudaErrors(cuLaunchGrid(matrixMul, WC / block_size, HC / block_size));
}
#endif
checkCudaErrors(cuCtxSynchronize());
// copy result from device to host
checkCudaErrors(cuMemcpyDtoH((void *)h_C, d_C, mem_size_C));
// compute reference solution
float *reference = (float *)malloc(mem_size_C);
computeGold(reference, h_A, h_B, HA, WA, WB);
// check result
float diff = 0.0f;
for (unsigned int i = 0; i < size_C; i++) {
float tmp = reference[i] - h_C[i];
diff += tmp * tmp;
}
int res = (diff / (float)size_C < 1e-6f);
// clean up memory
free(h_A);
free(h_B);
free(h_C);
free(reference);
checkCudaErrors(cuMemFree(d_A));
checkCudaErrors(cuMemFree(d_B));
checkCudaErrors(cuMemFree(d_C));
checkCudaErrors(cuModuleUnload(g_cuModule));
checkCudaErrors(cuCtxDestroy(g_cuContext));
printf("Test run %s\n", (1 == res) ? "success!" : "failed!");
exit((1 == res) ? EXIT_SUCCESS : EXIT_FAILURE);
}

56
cpp/0_Introduction/matrixMulDynlinkJIT/matrixMul_gold.cpp Normal file
View File

@@ -0,0 +1,56 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
////////////////////////////////////////////////////////////////////////////////
// export C interface
extern "C" void computeGold(float *, const float *, const float *, unsigned int, unsigned int, unsigned int);
////////////////////////////////////////////////////////////////////////////////
//! Compute reference data set
//! C = A * B
//! @param C reference data, computed but preallocated
//! @param A matrix A as provided to device
//! @param B matrix B as provided to device
//! @param hA height of matrix A
//! @param wB width of matrix B
////////////////////////////////////////////////////////////////////////////////
void computeGold(float *C, const float *A, const float *B, unsigned int hA, unsigned int wA, unsigned int wB)
{
for (unsigned int i = 0; i < hA; ++i)
for (unsigned int j = 0; j < wB; ++j) {
double sum = 0;
for (unsigned int k = 0; k < wA; ++k) {
double a = A[i * wA + k];
double b = B[k * wB + j];
sum += a * b;
}
C[i * wB + j] = (float)sum;
}
}

1391
cpp/0_Introduction/matrixMulDynlinkJIT/matrixMul_kernel_32_ptxdump.c Normal file
View File

File diff suppressed because it is too large Load Diff

45
cpp/0_Introduction/matrixMulDynlinkJIT/matrixMul_kernel_32_ptxdump.h Normal file
View File

@@ -0,0 +1,45 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
////////////////////////////////////////////////////////////////////////////////
// This file is auto-generated, do not edit
////////////////////////////////////////////////////////////////////////////////
#ifndef __matrixMul_kernel_32_ptxdump_h__
#define __matrixMul_kernel_32_ptxdump_h__
#if defined __cplusplus
extern "C"
{
#endif
extern unsigned char matrixMul_kernel_32_ptxdump[25784];
#if defined __cplusplus
}
#endif
#endif //__matrixMul_kernel_32_ptxdump_h__

1428
cpp/0_Introduction/matrixMulDynlinkJIT/matrixMul_kernel_64_ptxdump.c Normal file
View File

File diff suppressed because it is too large Load Diff

45
cpp/0_Introduction/matrixMulDynlinkJIT/matrixMul_kernel_64_ptxdump.h Normal file
View File

@@ -0,0 +1,45 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
////////////////////////////////////////////////////////////////////////////////
// This file is auto-generated, do not edit
////////////////////////////////////////////////////////////////////////////////
#ifndef __matrixMul_kernel_64_ptxdump_h__
#define __matrixMul_kernel_64_ptxdump_h__
#if defined __cplusplus
extern "C"
{
#endif
extern unsigned char matrixMul_kernel_64_ptxdump[26489];
#if defined __cplusplus
}
#endif
#endif //__matrixMul_kernel_64_ptxdump_h__

18
cpp/0_Introduction/matrixMul_nvrtc/.vscode/c_cpp_properties.json vendored Normal file
View File

@@ -0,0 +1,18 @@
{
"configurations": [
{
"name": "Linux",
"includePath": [
"${workspaceFolder}/**",
"${workspaceFolder}/../../../Common"
],
"defines": [],
"compilerPath": "/usr/local/cuda/bin/nvcc",
"cStandard": "gnu17",
"cppStandard": "gnu++14",
"intelliSenseMode": "linux-gcc-x64",
"configurationProvider": "ms-vscode.makefile-tools"
}
],
"version": 4
}

7
cpp/0_Introduction/matrixMul_nvrtc/.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,7 @@
{
"recommendations": [
"nvidia.nsight-vscode-edition",
"ms-vscode.cpptools",
"ms-vscode.makefile-tools"
]
}

61
cpp/0_Introduction/matrixMul_nvrtc/CMakeLists.txt Normal file
View File

@@ -0,0 +1,61 @@
cmake_minimum_required(VERSION 3.20)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/Modules")
project(matrixMul_nvrtc LANGUAGES C CXX CUDA)
find_package(CUDAToolkit REQUIRED)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(CMAKE_CUDA_ARCHITECTURES 75 80 86 87 89 90 100 110 120)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -Wno-deprecated-gpu-targets")
if(ENABLE_CUDA_DEBUG)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -G") # enable cuda-gdb (may significantly affect performance on some targets)
else()
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -lineinfo") # add line information to all builds for debug tools (exclusive to -G option)
endif()
# Include directories and libraries
include_directories(../../../Common)
# Source file
# Add sample target executable
add_executable(matrixMul_nvrtc matrixMul.cpp)
target_compile_options(matrixMul_nvrtc PRIVATE $<$<COMPILE_LANGUAGE:CUDA>:--extended-lambda>)
target_compile_features(matrixMul_nvrtc PRIVATE cxx_std_17 cuda_std_17)
target_link_libraries(matrixMul_nvrtc PRIVATE
CUDA::nvrtc
CUDA::cuda_driver
)
# The primary directory of CUDAToolkit_INCLUDE_DIRS is the CUDA Toolkit's include directory for finding the header files.
list(GET CUDAToolkit_INCLUDE_DIRS 0 CUDA_INCLUDE_DIR)
# Copy clock_kernel.cu to the output directory
add_custom_command(TARGET matrixMul_nvrtc POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
${CMAKE_CURRENT_SOURCE_DIR}/matrixMul_kernel.cu ${CUDA_INCLUDE_DIR}/cooperative_groups.h ${CMAKE_CURRENT_BINARY_DIR}
)
add_custom_command(TARGET matrixMul_nvrtc POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
${CUDA_INCLUDE_DIR}/cooperative_groups ${CMAKE_CURRENT_BINARY_DIR}/cooperative_groups
)
add_custom_command(TARGET matrixMul_nvrtc POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
${CUDA_INCLUDE_DIR}/cccl/nv ${CMAKE_CURRENT_BINARY_DIR}/nv
)
add_custom_command(TARGET matrixMul_nvrtc POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_directory
${CUDA_INCLUDE_DIR}/cccl/cuda ${CMAKE_CURRENT_BINARY_DIR}/cuda
)
# Include installation configuration
include(${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/InstallSamples.cmake)
setup_samples_install()

36
cpp/0_Introduction/matrixMul_nvrtc/README.md Normal file
View File

@@ -0,0 +1,36 @@
# matrixMul_nvrtc - Matrix Multiplication with libNVRTC
## Description
This sample implements matrix multiplication and is exactly the same as the second example of the [Shared Memory](https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.html#shared-memory) section of the programming guide. It has been written for clarity of exposition to illustrate various CUDA programming principles, not with the goal of providing the most performant generic kernel for matrix multiplication. To illustrate GPU performance for matrix multiply, this sample also shows how to use the CUDA 4.0+ interface for cuBLAS to demonstrate high-performance performance for matrix multiplication.
## Key Concepts
CUDA Runtime API, Linear Algebra, Runtime Compilation
## Supported SM Architectures
[SM 5.0 ](https://developer.nvidia.com/cuda-gpus) [SM 5.2 ](https://developer.nvidia.com/cuda-gpus) [SM 5.3 ](https://developer.nvidia.com/cuda-gpus) [SM 6.0 ](https://developer.nvidia.com/cuda-gpus) [SM 6.1 ](https://developer.nvidia.com/cuda-gpus) [SM 7.0 ](https://developer.nvidia.com/cuda-gpus) [SM 7.2 ](https://developer.nvidia.com/cuda-gpus) [SM 7.5 ](https://developer.nvidia.com/cuda-gpus) [SM 8.0 ](https://developer.nvidia.com/cuda-gpus) [SM 8.6 ](https://developer.nvidia.com/cuda-gpus) [SM 8.7 ](https://developer.nvidia.com/cuda-gpus) [SM 8.9 ](https://developer.nvidia.com/cuda-gpus) [SM 9.0 ](https://developer.nvidia.com/cuda-gpus)
## Supported OSes
Linux, Windows, QNX
## Supported CPU Architecture
x86_64, aarch64
## CUDA APIs involved
### [CUDA Driver API](http://docs.nvidia.com/cuda/cuda-driver-api/index.html)
cuMemcpyDtoH, cuLaunchKernel, cuMemcpyHtoD, cuCtxSynchronize, cuMemAlloc, cuMemFree, cuModuleGetFunction
## Dependencies needed to build/run
[NVRTC](../../../README.md#nvrtc)
## Prerequisites
Download and install the [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) for your corresponding platform.
Make sure the dependencies mentioned in [Dependencies]() section above are installed.
## References (for more details)

249
cpp/0_Introduction/matrixMul_nvrtc/matrixMul.cpp Normal file
View File

@@ -0,0 +1,249 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/**
* Matrix multiplication: C = A * B.
* Host code.
*
* This sample implements matrix multiplication as described in Chapter 3
* of the programming guide.
* It has been written for clarity of exposition to illustrate various CUDA
* programming principles, not with the goal of providing the most
* performant generic kernel for matrix multiplication.
*
* See also:
* V. Volkov and J. Demmel, "Benchmarking GPUs to tune dense linear algebra,"
* in Proc. 2008 ACM/IEEE Conf. on Supercomputing (SC '08),
* Piscataway, NJ: IEEE Press, 2008, pp. Art. 31:1-11.
*/
// System includes
#include <assert.h>
#include <stdio.h>
// CUDA runtime
#include <cuda_runtime.h>
#include "nvrtc_helper.h"
// Helper functions and utilities to work with CUDA
#include <helper_functions.h>
void constantInit(float *data, int size, float val)
{
for (int i = 0; i < size; ++i) {
data[i] = val;
}
}
/**
* Run a simple test of matrix multiplication using CUDA
*/
int matrixMultiply(int argc, char **argv, int block_size, dim3 &dimsA, dim3 &dimsB)
{
// Allocate host memory for matrices A and B
unsigned int size_A = dimsA.x * dimsA.y;
unsigned int mem_size_A = sizeof(float) * size_A;
float *h_A = (float *)malloc(mem_size_A);
unsigned int size_B = dimsB.x * dimsB.y;
unsigned int mem_size_B = sizeof(float) * size_B;
float *h_B = (float *)malloc(mem_size_B);
// Initialize host memory
const float valB = 0.01f;
constantInit(h_A, size_A, 1.0f);
constantInit(h_B, size_B, valB);
// Allocate device memory
CUdeviceptr d_A, d_B, d_C;
char *cubin, *kernel_file;
size_t cubinSize;
kernel_file = sdkFindFilePath("matrixMul_kernel.cu", argv[0]);
compileFileToCUBIN(kernel_file, argc, argv, &cubin, &cubinSize, 1);
CUmodule module = loadCUBIN(cubin, argc, argv);
// Allocate host matrix C
dim3 dimsC(dimsB.x, dimsA.y, 1);
unsigned int mem_size_C = dimsC.x * dimsC.y * sizeof(float);
float *h_C = (float *)malloc(mem_size_C);
if (h_C == NULL) {
fprintf(stderr, "Failed to allocate host matrix C!\n");
exit(EXIT_FAILURE);
}
checkCudaErrors(cuMemAlloc(&d_A, mem_size_A));
checkCudaErrors(cuMemAlloc(&d_B, mem_size_B));
checkCudaErrors(cuMemAlloc(&d_C, mem_size_C));
// copy host memory to device
checkCudaErrors(cuMemcpyHtoD(d_A, h_A, mem_size_A));
checkCudaErrors(cuMemcpyHtoD(d_B, h_B, mem_size_B));
// Setup execution parameters
dim3 threads(block_size, block_size);
dim3 grid(dimsB.x / threads.x, dimsA.y / threads.y);
// Create and start timer
printf("Computing result using CUDA Kernel...\n");
CUfunction kernel_addr;
if (block_size == 16) {
checkCudaErrors(cuModuleGetFunction(&kernel_addr, module, "matrixMulCUDA_block16"));
}
else {
checkCudaErrors(cuModuleGetFunction(&kernel_addr, module, "matrixMulCUDA_block32"));
}
void *arr[] = {(void *)&d_C, (void *)&d_A, (void *)&d_B, (void *)&dimsA.x, (void *)&dimsB.x};
// Execute the kernel
int nIter = 300;
for (int j = 0; j < nIter; j++) {
checkCudaErrors(cuLaunchKernel(kernel_addr,
grid.x,
grid.y,
grid.z, /* grid dim */
threads.x,
threads.y,
threads.z, /* block dim */
0,
0, /* shared mem, stream */
&arr[0], /* arguments */
0));
checkCudaErrors(cuCtxSynchronize());
}
// Copy result from device to host
checkCudaErrors(cuMemcpyDtoH(h_C, d_C, mem_size_C));
printf("Checking computed result for correctness: ");
bool correct = true;
// test relative error by the formula
// |<x, y>_cpu - <x,y>_gpu|/<|x|, |y|> < eps
double eps = 1.e-6; // machine zero
for (int i = 0; i < (int)(dimsC.x * dimsC.y); i++) {
double abs_err = fabs(h_C[i] - (dimsA.x * valB));
double dot_length = dimsA.x;
double abs_val = fabs(h_C[i]);
double rel_err = abs_err / abs_val / dot_length;
if (rel_err > eps) {
printf("Error! Matrix[%05d]=%.8f, ref=%.8f error term is > %E\n", i, h_C[i], dimsA.x * valB, eps);
correct = false;
}
}
printf("%s\n", correct ? "Result = PASS" : "Result = FAIL");
printf("\nNOTE: The CUDA Samples are not meant for performance measurements. "
"Results may vary when GPU Boost is enabled.\n");
// Clean up memory
free(h_A);
free(h_B);
free(h_C);
checkCudaErrors(cuMemFree(d_A));
checkCudaErrors(cuMemFree(d_B));
checkCudaErrors(cuMemFree(d_C));
if (correct) {
return EXIT_SUCCESS;
}
else {
return EXIT_FAILURE;
}
}
/**
* Program main
*/
int main(int argc, char **argv)
{
printf("[Matrix Multiply Using CUDA] - Starting...\n");
if (checkCmdLineFlag(argc, (const char **)argv, "help") || checkCmdLineFlag(argc, (const char **)argv, "?")) {
printf("Usage -device=n (n >= 0 for deviceID)\n");
printf(" -wA=WidthA -hA=HeightA (Width x Height of Matrix A)\n");
printf(" -wB=WidthB -hB=HeightB (Width x Height of Matrix B)\n");
printf(" Note: Outer matrix dimensions of A & B matrices must be equal.\n");
exit(EXIT_SUCCESS);
}
int block_size = 32;
// original:
dim3 dimsA(5 * 2 * block_size, 5 * 2 * block_size, 1);
dim3 dimsB(5 * 4 * block_size, 5 * 2 * block_size, 1);
// reduce sizes to avoid running out of memory
// dim3 dimsA(32,32, 1);
// dim3 dimsB(32,32,1);
// width of Matrix A
if (checkCmdLineFlag(argc, (const char **)argv, "wA")) {
dimsA.x = getCmdLineArgumentInt(argc, (const char **)argv, "wA");
}
// height of Matrix A
if (checkCmdLineFlag(argc, (const char **)argv, "hA")) {
dimsA.y = getCmdLineArgumentInt(argc, (const char **)argv, "hA");
}
// width of Matrix B
if (checkCmdLineFlag(argc, (const char **)argv, "wB")) {
dimsB.x = getCmdLineArgumentInt(argc, (const char **)argv, "wB");
}
// height of Matrix B
if (checkCmdLineFlag(argc, (const char **)argv, "hB")) {
dimsB.y = getCmdLineArgumentInt(argc, (const char **)argv, "hB");
}
if (dimsA.x != dimsB.y) {
printf("Error: outer matrix dimensions must be equal. (%d != %d)\n", dimsA.x, dimsB.y);
exit(EXIT_FAILURE);
}
printf("MatrixA(%d,%d), MatrixB(%d,%d)\n", dimsA.x, dimsA.y, dimsB.x, dimsB.y);
int matrix_result = matrixMultiply(argc, argv, block_size, dimsA, dimsB);
exit(matrix_result);
}

130
cpp/0_Introduction/matrixMul_nvrtc/matrixMul_kernel.cu Normal file
View File

@@ -0,0 +1,130 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/**
* Matrix multiplication: C = A * B.
* Host code.
*
* This sample implements matrix multiplication as described in Chapter 3
* of the programming guide.
* It has been written for clarity of exposition to illustrate various CUDA
* programming principles, not with the goal of providing the most
* performant generic kernel for matrix multiplication.
*
* See also:
* V. Volkov and J. Demmel, "Benchmarking GPUs to tune dense linear algebra,"
* in Proc. 2008 ACM/IEEE Conf. on Supercomputing (SC '08),
* Piscataway, NJ: IEEE Press, 2008, pp. Art. 31:1-11.
*/
/**
* Matrix multiplication (CUDA Kernel) on the device: C = A * B
* wA is A's width and wB is B's width
*/
#include <cooperative_groups.h>
template <int BLOCK_SIZE> __device__ void matrixMulCUDA(float *C, float *A, float *B, int wA, int wB)
{
// Handle to thread block group
cooperative_groups::thread_block cta = cooperative_groups::this_thread_block();
// Block index
int bx = blockIdx.x;
int by = blockIdx.y;
// Thread index
int tx = threadIdx.x;
int ty = threadIdx.y;
// Index of the first sub-matrix of A processed by the block
int aBegin = wA * BLOCK_SIZE * by;
// Index of the last sub-matrix of A processed by the block
int aEnd = aBegin + wA - 1;
// Step size used to iterate through the sub-matrices of A
int aStep = BLOCK_SIZE;
// Index of the first sub-matrix of B processed by the block
int bBegin = BLOCK_SIZE * bx;
// Step size used to iterate through the sub-matrices of B
int bStep = BLOCK_SIZE * wB;
// Csub is used to store the element of the block sub-matrix
// that is computed by the thread
float Csub = 0;
// Loop over all the sub-matrices of A and B
// required to compute the block sub-matrix
for (int a = aBegin, b = bBegin; a <= aEnd; a += aStep, b += bStep) {
// Declaration of the shared memory array As used to
// store the sub-matrix of A
__shared__ float As[BLOCK_SIZE][BLOCK_SIZE];
// Declaration of the shared memory array Bs used to
// store the sub-matrix of B
__shared__ float Bs[BLOCK_SIZE][BLOCK_SIZE];
// Load the matrices from device memory
// to shared memory; each thread loads
// one element of each matrix
As[ty][tx] = A[a + wA * ty + tx];
Bs[ty][tx] = B[b + wB * ty + tx];
// Synchronize to make sure the matrices are loaded
cooperative_groups::sync(cta);
// Multiply the two matrices together;
// each thread computes one element
// of the block sub-matrix
#pragma unroll
for (int k = 0; k < BLOCK_SIZE; ++k) {
Csub += As[ty][k] * Bs[k][tx];
}
// Synchronize to make sure that the preceding
// computation is done before loading two new
// sub-matrices of A and B in the next iteration
cooperative_groups::sync(cta);
}
// Write the block sub-matrix to device memory;
// each thread writes one element
int c = wB * BLOCK_SIZE * by + BLOCK_SIZE * bx;
C[c + wB * ty + tx] = Csub;
}
extern "C" __global__ void matrixMulCUDA_block16(float *C, float *A, float *B, int wA, int wB)
{
matrixMulCUDA<16>(C, A, B, wA, wB);
}
extern "C" __global__ void matrixMulCUDA_block32(float *C, float *A, float *B, int wA, int wB)
{
matrixMulCUDA<32>(C, A, B, wA, wB);
}

18
cpp/0_Introduction/mergeSort/.vscode/c_cpp_properties.json vendored Normal file
View File

@@ -0,0 +1,18 @@
{
"configurations": [
{
"name": "Linux",
"includePath": [
"${workspaceFolder}/**",
"${workspaceFolder}/../../../Common"
],
"defines": [],
"compilerPath": "/usr/local/cuda/bin/nvcc",
"cStandard": "gnu17",
"cppStandard": "gnu++14",
"intelliSenseMode": "linux-gcc-x64",
"configurationProvider": "ms-vscode.makefile-tools"
}
],
"version": 4
}

7
cpp/0_Introduction/mergeSort/.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,7 @@
{
"recommendations": [
"nvidia.nsight-vscode-edition",
"ms-vscode.cpptools",
"ms-vscode.makefile-tools"
]
}

37
cpp/0_Introduction/mergeSort/CMakeLists.txt Normal file
View File

@@ -0,0 +1,37 @@
cmake_minimum_required(VERSION 3.20)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/Modules")
project(mergeSort LANGUAGES C CXX CUDA)
find_package(CUDAToolkit REQUIRED)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(CMAKE_CUDA_ARCHITECTURES 75 80 86 87 89 90 100 110 120)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -Wno-deprecated-gpu-targets")
if(ENABLE_CUDA_DEBUG)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -G") # enable cuda-gdb (may significantly affect performance on some targets)
else()
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -lineinfo") # add line information to all builds for debug tools (exclusive to -G option)
endif()
# Include directories and libraries
include_directories(../../../Common)
# Source file
# Add target for mergeSort
add_executable(mergeSort mergeSort.cu main.cpp mergeSort_host.cpp mergeSort_validate.cpp)
target_compile_options(mergeSort PRIVATE $<$<COMPILE_LANGUAGE:CUDA>:--extended-lambda>)
target_compile_features(mergeSort PRIVATE cxx_std_17 cuda_std_17)
set_target_properties(mergeSort PROPERTIES CUDA_SEPARABLE_COMPILATION ON)
target_include_directories(mergeSort PRIVATE
${CUDAToolkit_INCLUDE_DIRS}
)
# Include installation configuration
include(${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/InstallSamples.cmake)
setup_samples_install()

32
cpp/0_Introduction/mergeSort/README.md Normal file
View File

@@ -0,0 +1,32 @@
# mergeSort - Merge Sort
## Description
This sample implements a merge sort (also known as Batcher's sort), algorithms belonging to the class of sorting networks. While generally subefficient on large sequences compared to algorithms with better asymptotic algorithmic complexity (i.e. merge sort or radix sort), may be the algorithms of choice for sorting batches of short- to mid-sized (key, value) array pairs. Refer to the excellent tutorial by H. W. Lang http://www.iti.fh-flensburg.de/lang/algorithmen/sortieren/networks/indexen.htm
## Key Concepts
Data-Parallel Algorithms
## Supported SM Architectures
[SM 5.0 ](https://developer.nvidia.com/cuda-gpus) [SM 5.2 ](https://developer.nvidia.com/cuda-gpus) [SM 5.3 ](https://developer.nvidia.com/cuda-gpus) [SM 6.0 ](https://developer.nvidia.com/cuda-gpus) [SM 6.1 ](https://developer.nvidia.com/cuda-gpus) [SM 7.0 ](https://developer.nvidia.com/cuda-gpus) [SM 7.2 ](https://developer.nvidia.com/cuda-gpus) [SM 7.5 ](https://developer.nvidia.com/cuda-gpus) [SM 8.0 ](https://developer.nvidia.com/cuda-gpus) [SM 8.6 ](https://developer.nvidia.com/cuda-gpus) [SM 8.7 ](https://developer.nvidia.com/cuda-gpus) [SM 8.9 ](https://developer.nvidia.com/cuda-gpus) [SM 9.0 ](https://developer.nvidia.com/cuda-gpus)
## Supported OSes
Linux, Windows
## Supported CPU Architecture
x86_64, armv7l
## CUDA APIs involved
### [CUDA Runtime API](http://docs.nvidia.com/cuda/cuda-runtime-api/index.html)
cudaMalloc, cudaDeviceSynchronize, cudaMemcpy, cudaFree
## Prerequisites
Download and install the [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) for your corresponding platform.
## References (for more details)

281
cpp/0_Introduction/mergeSort/bitonic.cu Normal file
View File

@@ -0,0 +1,281 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <cooperative_groups.h>
namespace cg = cooperative_groups;
#include <assert.h>
#include <helper_cuda.h>
#include "mergeSort_common.h"
inline __device__ void Comparator(uint &keyA, uint &valA, uint &keyB, uint &valB, uint arrowDir)
{
uint t;
if ((keyA > keyB) == arrowDir) {
t = keyA;
keyA = keyB;
keyB = t;
t = valA;
valA = valB;
valB = t;
}
}
__global__ void
bitonicSortSharedKernel(uint *d_DstKey, uint *d_DstVal, uint *d_SrcKey, uint *d_SrcVal, uint arrayLength, uint sortDir)
{
// Handle to thread block group
cg::thread_block cta = cg::this_thread_block();
// Shared memory storage for one or more short vectors
__shared__ uint s_key[SHARED_SIZE_LIMIT];
__shared__ uint s_val[SHARED_SIZE_LIMIT];
// Offset to the beginning of subbatch and load data
d_SrcKey += blockIdx.x * SHARED_SIZE_LIMIT + threadIdx.x;
d_SrcVal += blockIdx.x * SHARED_SIZE_LIMIT + threadIdx.x;
d_DstKey += blockIdx.x * SHARED_SIZE_LIMIT + threadIdx.x;
d_DstVal += blockIdx.x * SHARED_SIZE_LIMIT + threadIdx.x;
s_key[threadIdx.x + 0] = d_SrcKey[0];
s_val[threadIdx.x + 0] = d_SrcVal[0];
s_key[threadIdx.x + (SHARED_SIZE_LIMIT / 2)] = d_SrcKey[(SHARED_SIZE_LIMIT / 2)];
s_val[threadIdx.x + (SHARED_SIZE_LIMIT / 2)] = d_SrcVal[(SHARED_SIZE_LIMIT / 2)];
for (uint size = 2; size < arrayLength; size <<= 1) {
// Bitonic merge
uint dir = (threadIdx.x & (size / 2)) != 0;
for (uint stride = size / 2; stride > 0; stride >>= 1) {
cg::sync(cta);
uint pos = 2 * threadIdx.x - (threadIdx.x & (stride - 1));
Comparator(s_key[pos + 0], s_val[pos + 0], s_key[pos + stride], s_val[pos + stride], dir);
}
}
// ddd == sortDir for the last bitonic merge step
{
for (uint stride = arrayLength / 2; stride > 0; stride >>= 1) {
cg::sync(cta);
uint pos = 2 * threadIdx.x - (threadIdx.x & (stride - 1));
Comparator(s_key[pos + 0], s_val[pos + 0], s_key[pos + stride], s_val[pos + stride], sortDir);
}
}
cg::sync(cta);
d_DstKey[0] = s_key[threadIdx.x + 0];
d_DstVal[0] = s_val[threadIdx.x + 0];
d_DstKey[(SHARED_SIZE_LIMIT / 2)] = s_key[threadIdx.x + (SHARED_SIZE_LIMIT / 2)];
d_DstVal[(SHARED_SIZE_LIMIT / 2)] = s_val[threadIdx.x + (SHARED_SIZE_LIMIT / 2)];
}
// Helper function (also used by odd-even merge sort)
extern "C" uint factorRadix2(uint *log2L, uint L)
{
if (!L) {
*log2L = 0;
return 0;
}
else {
for (*log2L = 0; (L & 1) == 0; L >>= 1, *log2L++)
;
return L;
}
}
extern "C" void bitonicSortShared(uint *d_DstKey,
uint *d_DstVal,
uint *d_SrcKey,
uint *d_SrcVal,
uint batchSize,
uint arrayLength,
uint sortDir)
{
// Nothing to sort
if (arrayLength < 2) {
return;
}
// Only power-of-two array lengths are supported by this implementation
uint log2L;
uint factorizationRemainder = factorRadix2(&log2L, arrayLength);
assert(factorizationRemainder == 1);
uint blockCount = batchSize * arrayLength / SHARED_SIZE_LIMIT;
uint threadCount = SHARED_SIZE_LIMIT / 2;
assert(arrayLength <= SHARED_SIZE_LIMIT);
assert((batchSize * arrayLength) % SHARED_SIZE_LIMIT == 0);
bitonicSortSharedKernel<<<blockCount, threadCount>>>(d_DstKey, d_DstVal, d_SrcKey, d_SrcVal, arrayLength, sortDir);
getLastCudaError("bitonicSortSharedKernel<<<>>> failed!\n");
}
////////////////////////////////////////////////////////////////////////////////
// Merge step 3: merge elementary intervals
////////////////////////////////////////////////////////////////////////////////
static inline __host__ __device__ uint iDivUp(uint a, uint b) { return ((a % b) == 0) ? (a / b) : (a / b + 1); }
static inline __host__ __device__ uint getSampleCount(uint dividend) { return iDivUp(dividend, SAMPLE_STRIDE); }
template <uint sortDir>
static inline __device__ void
ComparatorExtended(uint &keyA, uint &valA, uint &flagA, uint &keyB, uint &valB, uint &flagB, uint arrowDir)
{
uint t;
if ((!(flagA || flagB) && ((keyA > keyB) == arrowDir)) || ((arrowDir == sortDir) && (flagA == 1))
|| ((arrowDir != sortDir) && (flagB == 1))) {
t = keyA;
keyA = keyB;
keyB = t;
t = valA;
valA = valB;
valB = t;
t = flagA;
flagA = flagB;
flagB = t;
}
}
template <uint sortDir>
__global__ void bitonicMergeElementaryIntervalsKernel(uint *d_DstKey,
uint *d_DstVal,
uint *d_SrcKey,
uint *d_SrcVal,
uint *d_LimitsA,
uint *d_LimitsB,
uint stride,
uint N)
{
// Handle to thread block group
cg::thread_block cta = cg::this_thread_block();
__shared__ uint s_key[2 * SAMPLE_STRIDE];
__shared__ uint s_val[2 * SAMPLE_STRIDE];
__shared__ uint s_inf[2 * SAMPLE_STRIDE];
const uint intervalI = blockIdx.x & ((2 * stride) / SAMPLE_STRIDE - 1);
const uint segmentBase = (blockIdx.x - intervalI) * SAMPLE_STRIDE;
d_SrcKey += segmentBase;
d_SrcVal += segmentBase;
d_DstKey += segmentBase;
d_DstVal += segmentBase;
// Set up threadblock-wide parameters
__shared__ uint startSrcA, lenSrcA, startSrcB, lenSrcB, startDst;
if (threadIdx.x == 0) {
uint segmentElementsA = stride;
uint segmentElementsB = umin(stride, N - segmentBase - stride);
uint segmentSamplesA = stride / SAMPLE_STRIDE;
uint segmentSamplesB = getSampleCount(segmentElementsB);
uint segmentSamples = segmentSamplesA + segmentSamplesB;
startSrcA = d_LimitsA[blockIdx.x];
startSrcB = d_LimitsB[blockIdx.x];
startDst = startSrcA + startSrcB;
uint endSrcA = (intervalI + 1 < segmentSamples) ? d_LimitsA[blockIdx.x + 1] : segmentElementsA;
uint endSrcB = (intervalI + 1 < segmentSamples) ? d_LimitsB[blockIdx.x + 1] : segmentElementsB;
lenSrcA = endSrcA - startSrcA;
lenSrcB = endSrcB - startSrcB;
}
s_inf[threadIdx.x + 0] = 1;
s_inf[threadIdx.x + SAMPLE_STRIDE] = 1;
// Load input data
cg::sync(cta);
if (threadIdx.x < lenSrcA) {
s_key[threadIdx.x] = d_SrcKey[0 + startSrcA + threadIdx.x];
s_val[threadIdx.x] = d_SrcVal[0 + startSrcA + threadIdx.x];
s_inf[threadIdx.x] = 0;
}
// Prepare for bitonic merge by inversing the ordering
if (threadIdx.x < lenSrcB) {
s_key[2 * SAMPLE_STRIDE - 1 - threadIdx.x] = d_SrcKey[stride + startSrcB + threadIdx.x];
s_val[2 * SAMPLE_STRIDE - 1 - threadIdx.x] = d_SrcVal[stride + startSrcB + threadIdx.x];
s_inf[2 * SAMPLE_STRIDE - 1 - threadIdx.x] = 0;
}
//"Extended" bitonic merge
for (uint stride = SAMPLE_STRIDE; stride > 0; stride >>= 1) {
cg::sync(cta);
uint pos = 2 * threadIdx.x - (threadIdx.x & (stride - 1));
ComparatorExtended<sortDir>(s_key[pos + 0],
s_val[pos + 0],
s_inf[pos + 0],
s_key[pos + stride],
s_val[pos + stride],
s_inf[pos + stride],
sortDir);
}
// Store sorted data
cg::sync(cta);
d_DstKey += startDst;
d_DstVal += startDst;
if (threadIdx.x < lenSrcA) {
d_DstKey[threadIdx.x] = s_key[threadIdx.x];
d_DstVal[threadIdx.x] = s_val[threadIdx.x];
}
if (threadIdx.x < lenSrcB) {
d_DstKey[lenSrcA + threadIdx.x] = s_key[lenSrcA + threadIdx.x];
d_DstVal[lenSrcA + threadIdx.x] = s_val[lenSrcA + threadIdx.x];
}
}
extern "C" void bitonicMergeElementaryIntervals(uint *d_DstKey,
uint *d_DstVal,
uint *d_SrcKey,
uint *d_SrcVal,
uint *d_LimitsA,
uint *d_LimitsB,
uint stride,
uint N,
uint sortDir)
{
uint lastSegmentElements = N % (2 * stride);
uint mergePairs = (lastSegmentElements > stride) ? getSampleCount(N) : (N - lastSegmentElements) / SAMPLE_STRIDE;
if (sortDir) {
bitonicMergeElementaryIntervalsKernel<1U>
<<<mergePairs, SAMPLE_STRIDE>>>(d_DstKey, d_DstVal, d_SrcKey, d_SrcVal, d_LimitsA, d_LimitsB, stride, N);
getLastCudaError("mergeElementaryIntervalsKernel<1> failed\n");
}
else {
bitonicMergeElementaryIntervalsKernel<0U>
<<<mergePairs, SAMPLE_STRIDE>>>(d_DstKey, d_DstVal, d_SrcKey, d_SrcVal, d_LimitsA, d_LimitsB, stride, N);
getLastCudaError("mergeElementaryIntervalsKernel<0> failed\n");
}
}

119
cpp/0_Introduction/mergeSort/main.cpp Normal file
View File

@@ -0,0 +1,119 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <assert.h>
#include <cuda_runtime.h>
#include <helper_cuda.h>
#include <helper_functions.h>
#include <stdio.h>
#include <stdlib.h>
#include "mergeSort_common.h"
////////////////////////////////////////////////////////////////////////////////
// Test driver
////////////////////////////////////////////////////////////////////////////////
int main(int argc, char **argv)
{
uint *h_SrcKey, *h_SrcVal, *h_DstKey, *h_DstVal;
uint *d_SrcKey, *d_SrcVal, *d_BufKey, *d_BufVal, *d_DstKey, *d_DstVal;
StopWatchInterface *hTimer = NULL;
const uint N = 4 * 1048576;
const uint DIR = 1;
const uint numValues = 65536;
printf("%s Starting...\n\n", argv[0]);
int dev = findCudaDevice(argc, (const char **)argv);
if (dev == -1) {
return EXIT_FAILURE;
}
printf("Allocating and initializing host arrays...\n\n");
sdkCreateTimer(&hTimer);
h_SrcKey = (uint *)malloc(N * sizeof(uint));
h_SrcVal = (uint *)malloc(N * sizeof(uint));
h_DstKey = (uint *)malloc(N * sizeof(uint));
h_DstVal = (uint *)malloc(N * sizeof(uint));
srand(2009);
for (uint i = 0; i < N; i++) {
h_SrcKey[i] = rand() % numValues;
}
fillValues(h_SrcVal, N);
printf("Allocating and initializing CUDA arrays...\n\n");
checkCudaErrors(cudaMalloc((void **)&d_DstKey, N * sizeof(uint)));
checkCudaErrors(cudaMalloc((void **)&d_DstVal, N * sizeof(uint)));
checkCudaErrors(cudaMalloc((void **)&d_BufKey, N * sizeof(uint)));
checkCudaErrors(cudaMalloc((void **)&d_BufVal, N * sizeof(uint)));
checkCudaErrors(cudaMalloc((void **)&d_SrcKey, N * sizeof(uint)));
checkCudaErrors(cudaMalloc((void **)&d_SrcVal, N * sizeof(uint)));
checkCudaErrors(cudaMemcpy(d_SrcKey, h_SrcKey, N * sizeof(uint), cudaMemcpyHostToDevice));
checkCudaErrors(cudaMemcpy(d_SrcVal, h_SrcVal, N * sizeof(uint), cudaMemcpyHostToDevice));
printf("Initializing GPU merge sort...\n");
initMergeSort();
printf("Running GPU merge sort...\n");
checkCudaErrors(cudaDeviceSynchronize());
sdkResetTimer(&hTimer);
sdkStartTimer(&hTimer);
mergeSort(d_DstKey, d_DstVal, d_BufKey, d_BufVal, d_SrcKey, d_SrcVal, N, DIR);
checkCudaErrors(cudaDeviceSynchronize());
sdkStopTimer(&hTimer);
printf("Time: %f ms\n", sdkGetTimerValue(&hTimer));
printf("Reading back GPU merge sort results...\n");
checkCudaErrors(cudaMemcpy(h_DstKey, d_DstKey, N * sizeof(uint), cudaMemcpyDeviceToHost));
checkCudaErrors(cudaMemcpy(h_DstVal, d_DstVal, N * sizeof(uint), cudaMemcpyDeviceToHost));
printf("Inspecting the results...\n");
uint keysFlag = validateSortedKeys(h_DstKey, h_SrcKey, 1, N, numValues, DIR);
uint valuesFlag = validateSortedValues(h_DstKey, h_DstVal, h_SrcKey, 1, N);
printf("Shutting down...\n");
closeMergeSort();
sdkDeleteTimer(&hTimer);
checkCudaErrors(cudaFree(d_SrcVal));
checkCudaErrors(cudaFree(d_SrcKey));
checkCudaErrors(cudaFree(d_BufVal));
checkCudaErrors(cudaFree(d_BufKey));
checkCudaErrors(cudaFree(d_DstVal));
checkCudaErrors(cudaFree(d_DstKey));
free(h_DstVal);
free(h_DstKey);
free(h_SrcVal);
free(h_SrcKey);
exit((keysFlag && valuesFlag) ? EXIT_SUCCESS : EXIT_FAILURE);
}

537
cpp/0_Introduction/mergeSort/mergeSort.cu Normal file
View File

@@ -0,0 +1,537 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
/*
* Based on "Designing efficient sorting algorithms for manycore GPUs"
* by Nadathur Satish, Mark Harris, and Michael Garland
* http://mgarland.org/files/papers/gpusort-ipdps09.pdf
*
* Victor Podlozhnyuk 09/24/2009
*/
#include <assert.h>
#include <cooperative_groups.h>
namespace cg = cooperative_groups;
#include <helper_cuda.h>
#include "mergeSort_common.h"
////////////////////////////////////////////////////////////////////////////////
// Helper functions
////////////////////////////////////////////////////////////////////////////////
static inline __host__ __device__ uint iDivUp(uint a, uint b) { return ((a % b) == 0) ? (a / b) : (a / b + 1); }
static inline __host__ __device__ uint getSampleCount(uint dividend) { return iDivUp(dividend, SAMPLE_STRIDE); }
#define W (sizeof(uint) * 8)
static inline __device__ uint nextPowerOfTwo(uint x)
{
/*
--x;
x |= x >> 1;
x |= x >> 2;
x |= x >> 4;
x |= x >> 8;
x |= x >> 16;
return ++x;
*/
return 1U << (W - __clz(x - 1));
}
template <uint sortDir> static inline __device__ uint binarySearchInclusive(uint val, uint *data, uint L, uint stride)
{
if (L == 0) {
return 0;
}
uint pos = 0;
for (; stride > 0; stride >>= 1) {
uint newPos = umin(pos + stride, L);
if ((sortDir && (data[newPos - 1] <= val)) || (!sortDir && (data[newPos - 1] >= val))) {
pos = newPos;
}
}
return pos;
}
template <uint sortDir> static inline __device__ uint binarySearchExclusive(uint val, uint *data, uint L, uint stride)
{
if (L == 0) {
return 0;
}
uint pos = 0;
for (; stride > 0; stride >>= 1) {
uint newPos = umin(pos + stride, L);
if ((sortDir && (data[newPos - 1] < val)) || (!sortDir && (data[newPos - 1] > val))) {
pos = newPos;
}
}
return pos;
}
////////////////////////////////////////////////////////////////////////////////
// Bottom-level merge sort (binary search-based)
////////////////////////////////////////////////////////////////////////////////
template <uint sortDir>
__global__ void mergeSortSharedKernel(uint *d_DstKey, uint *d_DstVal, uint *d_SrcKey, uint *d_SrcVal, uint arrayLength)
{
// Handle to thread block group
cg::thread_block cta = cg::this_thread_block();
__shared__ uint s_key[SHARED_SIZE_LIMIT];
__shared__ uint s_val[SHARED_SIZE_LIMIT];
d_SrcKey += blockIdx.x * SHARED_SIZE_LIMIT + threadIdx.x;
d_SrcVal += blockIdx.x * SHARED_SIZE_LIMIT + threadIdx.x;
d_DstKey += blockIdx.x * SHARED_SIZE_LIMIT + threadIdx.x;
d_DstVal += blockIdx.x * SHARED_SIZE_LIMIT + threadIdx.x;
s_key[threadIdx.x + 0] = d_SrcKey[0];
s_val[threadIdx.x + 0] = d_SrcVal[0];
s_key[threadIdx.x + (SHARED_SIZE_LIMIT / 2)] = d_SrcKey[(SHARED_SIZE_LIMIT / 2)];
s_val[threadIdx.x + (SHARED_SIZE_LIMIT / 2)] = d_SrcVal[(SHARED_SIZE_LIMIT / 2)];
for (uint stride = 1; stride < arrayLength; stride <<= 1) {
uint lPos = threadIdx.x & (stride - 1);
uint *baseKey = s_key + 2 * (threadIdx.x - lPos);
uint *baseVal = s_val + 2 * (threadIdx.x - lPos);
cg::sync(cta);
uint keyA = baseKey[lPos + 0];
uint valA = baseVal[lPos + 0];
uint keyB = baseKey[lPos + stride];
uint valB = baseVal[lPos + stride];
uint posA = binarySearchExclusive<sortDir>(keyA, baseKey + stride, stride, stride) + lPos;
uint posB = binarySearchInclusive<sortDir>(keyB, baseKey + 0, stride, stride) + lPos;
cg::sync(cta);
baseKey[posA] = keyA;
baseVal[posA] = valA;
baseKey[posB] = keyB;
baseVal[posB] = valB;
}
cg::sync(cta);
d_DstKey[0] = s_key[threadIdx.x + 0];
d_DstVal[0] = s_val[threadIdx.x + 0];
d_DstKey[(SHARED_SIZE_LIMIT / 2)] = s_key[threadIdx.x + (SHARED_SIZE_LIMIT / 2)];
d_DstVal[(SHARED_SIZE_LIMIT / 2)] = s_val[threadIdx.x + (SHARED_SIZE_LIMIT / 2)];
}
static void mergeSortShared(uint *d_DstKey,
uint *d_DstVal,
uint *d_SrcKey,
uint *d_SrcVal,
uint batchSize,
uint arrayLength,
uint sortDir)
{
if (arrayLength < 2) {
return;
}
assert(SHARED_SIZE_LIMIT % arrayLength == 0);
assert(((batchSize * arrayLength) % SHARED_SIZE_LIMIT) == 0);
uint blockCount = batchSize * arrayLength / SHARED_SIZE_LIMIT;
uint threadCount = SHARED_SIZE_LIMIT / 2;
if (sortDir) {
mergeSortSharedKernel<1U><<<blockCount, threadCount>>>(d_DstKey, d_DstVal, d_SrcKey, d_SrcVal, arrayLength);
getLastCudaError("mergeSortShared<1><<<>>> failed\n");
}
else {
mergeSortSharedKernel<0U><<<blockCount, threadCount>>>(d_DstKey, d_DstVal, d_SrcKey, d_SrcVal, arrayLength);
getLastCudaError("mergeSortShared<0><<<>>> failed\n");
}
}
////////////////////////////////////////////////////////////////////////////////
// Merge step 1: generate sample ranks
////////////////////////////////////////////////////////////////////////////////
template <uint sortDir>
__global__ void
generateSampleRanksKernel(uint *d_RanksA, uint *d_RanksB, uint *d_SrcKey, uint stride, uint N, uint threadCount)
{
uint pos = blockIdx.x * blockDim.x + threadIdx.x;
if (pos >= threadCount) {
return;
}
const uint i = pos & ((stride / SAMPLE_STRIDE) - 1);
const uint segmentBase = (pos - i) * (2 * SAMPLE_STRIDE);
d_SrcKey += segmentBase;
d_RanksA += segmentBase / SAMPLE_STRIDE;
d_RanksB += segmentBase / SAMPLE_STRIDE;
const uint segmentElementsA = stride;
const uint segmentElementsB = umin(stride, N - segmentBase - stride);
const uint segmentSamplesA = getSampleCount(segmentElementsA);
const uint segmentSamplesB = getSampleCount(segmentElementsB);
if (i < segmentSamplesA) {
d_RanksA[i] = i * SAMPLE_STRIDE;
d_RanksB[i] = binarySearchExclusive<sortDir>(
d_SrcKey[i * SAMPLE_STRIDE], d_SrcKey + stride, segmentElementsB, nextPowerOfTwo(segmentElementsB));
}
if (i < segmentSamplesB) {
d_RanksB[(stride / SAMPLE_STRIDE) + i] = i * SAMPLE_STRIDE;
d_RanksA[(stride / SAMPLE_STRIDE) + i] = binarySearchInclusive<sortDir>(
d_SrcKey[stride + i * SAMPLE_STRIDE], d_SrcKey + 0, segmentElementsA, nextPowerOfTwo(segmentElementsA));
}
}
static void generateSampleRanks(uint *d_RanksA, uint *d_RanksB, uint *d_SrcKey, uint stride, uint N, uint sortDir)
{
uint lastSegmentElements = N % (2 * stride);
uint threadCount = (lastSegmentElements > stride) ? (N + 2 * stride - lastSegmentElements) / (2 * SAMPLE_STRIDE)
: (N - lastSegmentElements) / (2 * SAMPLE_STRIDE);
if (sortDir) {
generateSampleRanksKernel<1U>
<<<iDivUp(threadCount, 256), 256>>>(d_RanksA, d_RanksB, d_SrcKey, stride, N, threadCount);
getLastCudaError("generateSampleRanksKernel<1U><<<>>> failed\n");
}
else {
generateSampleRanksKernel<0U>
<<<iDivUp(threadCount, 256), 256>>>(d_RanksA, d_RanksB, d_SrcKey, stride, N, threadCount);
getLastCudaError("generateSampleRanksKernel<0U><<<>>> failed\n");
}
}
////////////////////////////////////////////////////////////////////////////////
// Merge step 2: generate sample ranks and indices
////////////////////////////////////////////////////////////////////////////////
__global__ void mergeRanksAndIndicesKernel(uint *d_Limits, uint *d_Ranks, uint stride, uint N, uint threadCount)
{
uint pos = blockIdx.x * blockDim.x + threadIdx.x;
if (pos >= threadCount) {
return;
}
const uint i = pos & ((stride / SAMPLE_STRIDE) - 1);
const uint segmentBase = (pos - i) * (2 * SAMPLE_STRIDE);
d_Ranks += (pos - i) * 2;
d_Limits += (pos - i) * 2;
const uint segmentElementsA = stride;
const uint segmentElementsB = umin(stride, N - segmentBase - stride);
const uint segmentSamplesA = getSampleCount(segmentElementsA);
const uint segmentSamplesB = getSampleCount(segmentElementsB);
if (i < segmentSamplesA) {
uint dstPos = binarySearchExclusive<1U>(
d_Ranks[i], d_Ranks + segmentSamplesA, segmentSamplesB, nextPowerOfTwo(segmentSamplesB))
+ i;
d_Limits[dstPos] = d_Ranks[i];
}
if (i < segmentSamplesB) {
uint dstPos = binarySearchInclusive<1U>(
d_Ranks[segmentSamplesA + i], d_Ranks, segmentSamplesA, nextPowerOfTwo(segmentSamplesA))
+ i;
d_Limits[dstPos] = d_Ranks[segmentSamplesA + i];
}
}
static void mergeRanksAndIndices(uint *d_LimitsA, uint *d_LimitsB, uint *d_RanksA, uint *d_RanksB, uint stride, uint N)
{
uint lastSegmentElements = N % (2 * stride);
uint threadCount = (lastSegmentElements > stride) ? (N + 2 * stride - lastSegmentElements) / (2 * SAMPLE_STRIDE)
: (N - lastSegmentElements) / (2 * SAMPLE_STRIDE);
mergeRanksAndIndicesKernel<<<iDivUp(threadCount, 256), 256>>>(d_LimitsA, d_RanksA, stride, N, threadCount);
getLastCudaError("mergeRanksAndIndicesKernel(A)<<<>>> failed\n");
mergeRanksAndIndicesKernel<<<iDivUp(threadCount, 256), 256>>>(d_LimitsB, d_RanksB, stride, N, threadCount);
getLastCudaError("mergeRanksAndIndicesKernel(B)<<<>>> failed\n");
}
////////////////////////////////////////////////////////////////////////////////
// Merge step 3: merge elementary intervals
////////////////////////////////////////////////////////////////////////////////
template <uint sortDir>
inline __device__ void merge(uint *dstKey,
uint *dstVal,
uint *srcAKey,
uint *srcAVal,
uint *srcBKey,
uint *srcBVal,
uint lenA,
uint nPowTwoLenA,
uint lenB,
uint nPowTwoLenB,
cg::thread_block cta)
{
uint keyA, valA, keyB, valB, dstPosA, dstPosB;
if (threadIdx.x < lenA) {
keyA = srcAKey[threadIdx.x];
valA = srcAVal[threadIdx.x];
dstPosA = binarySearchExclusive<sortDir>(keyA, srcBKey, lenB, nPowTwoLenB) + threadIdx.x;
}
if (threadIdx.x < lenB) {
keyB = srcBKey[threadIdx.x];
valB = srcBVal[threadIdx.x];
dstPosB = binarySearchInclusive<sortDir>(keyB, srcAKey, lenA, nPowTwoLenA) + threadIdx.x;
}
cg::sync(cta);
if (threadIdx.x < lenA) {
dstKey[dstPosA] = keyA;
dstVal[dstPosA] = valA;
}
if (threadIdx.x < lenB) {
dstKey[dstPosB] = keyB;
dstVal[dstPosB] = valB;
}
}
template <uint sortDir>
__global__ void mergeElementaryIntervalsKernel(uint *d_DstKey,
uint *d_DstVal,
uint *d_SrcKey,
uint *d_SrcVal,
uint *d_LimitsA,
uint *d_LimitsB,
uint stride,
uint N)
{
// Handle to thread block group
cg::thread_block cta = cg::this_thread_block();
__shared__ uint s_key[2 * SAMPLE_STRIDE];
__shared__ uint s_val[2 * SAMPLE_STRIDE];
const uint intervalI = blockIdx.x & ((2 * stride) / SAMPLE_STRIDE - 1);
const uint segmentBase = (blockIdx.x - intervalI) * SAMPLE_STRIDE;
d_SrcKey += segmentBase;
d_SrcVal += segmentBase;
d_DstKey += segmentBase;
d_DstVal += segmentBase;
// Set up threadblock-wide parameters
__shared__ uint startSrcA, startSrcB, lenSrcA, lenSrcB, startDstA, startDstB;
if (threadIdx.x == 0) {
uint segmentElementsA = stride;
uint segmentElementsB = umin(stride, N - segmentBase - stride);
uint segmentSamplesA = getSampleCount(segmentElementsA);
uint segmentSamplesB = getSampleCount(segmentElementsB);
uint segmentSamples = segmentSamplesA + segmentSamplesB;
startSrcA = d_LimitsA[blockIdx.x];
startSrcB = d_LimitsB[blockIdx.x];
uint endSrcA = (intervalI + 1 < segmentSamples) ? d_LimitsA[blockIdx.x + 1] : segmentElementsA;
uint endSrcB = (intervalI + 1 < segmentSamples) ? d_LimitsB[blockIdx.x + 1] : segmentElementsB;
lenSrcA = endSrcA - startSrcA;
lenSrcB = endSrcB - startSrcB;
startDstA = startSrcA + startSrcB;
startDstB = startDstA + lenSrcA;
}
// Load main input data
cg::sync(cta);
if (threadIdx.x < lenSrcA) {
s_key[threadIdx.x + 0] = d_SrcKey[0 + startSrcA + threadIdx.x];
s_val[threadIdx.x + 0] = d_SrcVal[0 + startSrcA + threadIdx.x];
}
if (threadIdx.x < lenSrcB) {
s_key[threadIdx.x + SAMPLE_STRIDE] = d_SrcKey[stride + startSrcB + threadIdx.x];
s_val[threadIdx.x + SAMPLE_STRIDE] = d_SrcVal[stride + startSrcB + threadIdx.x];
}
// Merge data in shared memory
cg::sync(cta);
merge<sortDir>(s_key,
s_val,
s_key + 0,
s_val + 0,
s_key + SAMPLE_STRIDE,
s_val + SAMPLE_STRIDE,
lenSrcA,
SAMPLE_STRIDE,
lenSrcB,
SAMPLE_STRIDE,
cta);
// Store merged data
cg::sync(cta);
if (threadIdx.x < lenSrcA) {
d_DstKey[startDstA + threadIdx.x] = s_key[threadIdx.x];
d_DstVal[startDstA + threadIdx.x] = s_val[threadIdx.x];
}
if (threadIdx.x < lenSrcB) {
d_DstKey[startDstB + threadIdx.x] = s_key[lenSrcA + threadIdx.x];
d_DstVal[startDstB + threadIdx.x] = s_val[lenSrcA + threadIdx.x];
}
}
static void mergeElementaryIntervals(uint *d_DstKey,
uint *d_DstVal,
uint *d_SrcKey,
uint *d_SrcVal,
uint *d_LimitsA,
uint *d_LimitsB,
uint stride,
uint N,
uint sortDir)
{
uint lastSegmentElements = N % (2 * stride);
uint mergePairs = (lastSegmentElements > stride) ? getSampleCount(N) : (N - lastSegmentElements) / SAMPLE_STRIDE;
if (sortDir) {
mergeElementaryIntervalsKernel<1U>
<<<mergePairs, SAMPLE_STRIDE>>>(d_DstKey, d_DstVal, d_SrcKey, d_SrcVal, d_LimitsA, d_LimitsB, stride, N);
getLastCudaError("mergeElementaryIntervalsKernel<1> failed\n");
}
else {
mergeElementaryIntervalsKernel<0U>
<<<mergePairs, SAMPLE_STRIDE>>>(d_DstKey, d_DstVal, d_SrcKey, d_SrcVal, d_LimitsA, d_LimitsB, stride, N);
getLastCudaError("mergeElementaryIntervalsKernel<0> failed\n");
}
}
extern "C" void bitonicSortShared(uint *d_DstKey,
uint *d_DstVal,
uint *d_SrcKey,
uint *d_SrcVal,
uint batchSize,
uint arrayLength,
uint sortDir);
extern "C" void bitonicMergeElementaryIntervals(uint *d_DstKey,
uint *d_DstVal,
uint *d_SrcKey,
uint *d_SrcVal,
uint *d_LimitsA,
uint *d_LimitsB,
uint stride,
uint N,
uint sortDir);
static uint *d_RanksA, *d_RanksB, *d_LimitsA, *d_LimitsB;
static const uint MAX_SAMPLE_COUNT = 32768;
extern "C" void initMergeSort(void)
{
checkCudaErrors(cudaMalloc((void **)&d_RanksA, MAX_SAMPLE_COUNT * sizeof(uint)));
checkCudaErrors(cudaMalloc((void **)&d_RanksB, MAX_SAMPLE_COUNT * sizeof(uint)));
checkCudaErrors(cudaMalloc((void **)&d_LimitsA, MAX_SAMPLE_COUNT * sizeof(uint)));
checkCudaErrors(cudaMalloc((void **)&d_LimitsB, MAX_SAMPLE_COUNT * sizeof(uint)));
}
extern "C" void closeMergeSort(void)
{
checkCudaErrors(cudaFree(d_RanksA));
checkCudaErrors(cudaFree(d_RanksB));
checkCudaErrors(cudaFree(d_LimitsB));
checkCudaErrors(cudaFree(d_LimitsA));
}
extern "C" void mergeSort(uint *d_DstKey,
uint *d_DstVal,
uint *d_BufKey,
uint *d_BufVal,
uint *d_SrcKey,
uint *d_SrcVal,
uint N,
uint sortDir)
{
uint stageCount = 0;
for (uint stride = SHARED_SIZE_LIMIT; stride < N; stride <<= 1, stageCount++)
;
uint *ikey, *ival, *okey, *oval;
if (stageCount & 1) {
ikey = d_BufKey;
ival = d_BufVal;
okey = d_DstKey;
oval = d_DstVal;
}
else {
ikey = d_DstKey;
ival = d_DstVal;
okey = d_BufKey;
oval = d_BufVal;
}
assert(N <= (SAMPLE_STRIDE * MAX_SAMPLE_COUNT));
assert(N % SHARED_SIZE_LIMIT == 0);
mergeSortShared(ikey, ival, d_SrcKey, d_SrcVal, N / SHARED_SIZE_LIMIT, SHARED_SIZE_LIMIT, sortDir);
for (uint stride = SHARED_SIZE_LIMIT; stride < N; stride <<= 1) {
uint lastSegmentElements = N % (2 * stride);
// Find sample ranks and prepare for limiters merge
generateSampleRanks(d_RanksA, d_RanksB, ikey, stride, N, sortDir);
// Merge ranks and indices
mergeRanksAndIndices(d_LimitsA, d_LimitsB, d_RanksA, d_RanksB, stride, N);
// Merge elementary intervals
mergeElementaryIntervals(okey, oval, ikey, ival, d_LimitsA, d_LimitsB, stride, N, sortDir);
if (lastSegmentElements <= stride) {
// Last merge segment consists of a single array which just needs to be
// passed through
checkCudaErrors(cudaMemcpy(okey + (N - lastSegmentElements),
ikey + (N - lastSegmentElements),
lastSegmentElements * sizeof(uint),
cudaMemcpyDeviceToDevice));
checkCudaErrors(cudaMemcpy(oval + (N - lastSegmentElements),
ival + (N - lastSegmentElements),
lastSegmentElements * sizeof(uint),
cudaMemcpyDeviceToDevice));
}
uint *t;
t = ikey;
ikey = okey;
okey = t;
t = ival;
ival = oval;
oval = t;
}
}

60
cpp/0_Introduction/mergeSort/mergeSort_common.h Normal file
View File

@@ -0,0 +1,60 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
////////////////////////////////////////////////////////////////////////////////
// Shortcut definitions
////////////////////////////////////////////////////////////////////////////////
typedef unsigned int uint;
#define SHARED_SIZE_LIMIT 1024U
#define SAMPLE_STRIDE 128
////////////////////////////////////////////////////////////////////////////////
// Extensive sort validation routine
////////////////////////////////////////////////////////////////////////////////
extern "C" uint
validateSortedKeys(uint *resKey, uint *srcKey, uint batchSize, uint arrayLength, uint numValues, uint sortDir);
extern "C" void fillValues(uint *val, uint N);
extern "C" int validateSortedValues(uint *resKey, uint *resVal, uint *srcKey, uint batchSize, uint arrayLength);
////////////////////////////////////////////////////////////////////////////////
// CUDA merge sort
////////////////////////////////////////////////////////////////////////////////
extern "C" void initMergeSort(void);
extern "C" void closeMergeSort(void);
extern "C" void
mergeSort(uint *dstKey, uint *dstVal, uint *bufKey, uint *bufVal, uint *srcKey, uint *srcVal, uint N, uint sortDir);
////////////////////////////////////////////////////////////////////////////////
// CPU "emulation"
////////////////////////////////////////////////////////////////////////////////
extern "C" void
mergeSortHost(uint *dstKey, uint *dstVal, uint *bufKey, uint *bufVal, uint *srcKey, uint *srcVal, uint N, uint sortDir);

363
cpp/0_Introduction/mergeSort/mergeSort_host.cpp Normal file
View File

@@ -0,0 +1,363 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "mergeSort_common.h"
////////////////////////////////////////////////////////////////////////////////
// Helper functions
////////////////////////////////////////////////////////////////////////////////
static void checkOrder(uint *data, uint N, uint sortDir)
{
if (N <= 1) {
return;
}
for (uint i = 0; i < N - 1; i++)
if ((sortDir && (data[i] > data[i + 1])) || (!sortDir && (data[i] < data[i + 1]))) {
fprintf(stderr, "checkOrder() failed!!!\n");
exit(EXIT_FAILURE);
}
}
static uint umin(uint a, uint b) { return (a <= b) ? a : b; }
static uint getSampleCount(uint dividend)
{
return ((dividend % SAMPLE_STRIDE) != 0) ? (dividend / SAMPLE_STRIDE + 1) : (dividend / SAMPLE_STRIDE);
}
static uint nextPowerOfTwo(uint x)
{
--x;
x |= x >> 1;
x |= x >> 2;
x |= x >> 4;
x |= x >> 8;
x |= x >> 16;
return ++x;
}
static uint binarySearchInclusive(uint val, uint *data, uint L, uint sortDir)
{
if (L == 0) {
return 0;
}
uint pos = 0;
for (uint stride = nextPowerOfTwo(L); stride > 0; stride >>= 1) {
uint newPos = umin(pos + stride, L);
if ((sortDir && (data[newPos - 1] <= val)) || (!sortDir && (data[newPos - 1] >= val))) {
pos = newPos;
}
}
return pos;
}
static uint binarySearchExclusive(uint val, uint *data, uint L, uint sortDir)
{
if (L == 0) {
return 0;
}
uint pos = 0;
for (uint stride = nextPowerOfTwo(L); stride > 0; stride >>= 1) {
uint newPos = umin(pos + stride, L);
if ((sortDir && (data[newPos - 1] < val)) || (!sortDir && (data[newPos - 1] > val))) {
pos = newPos;
}
}
return pos;
}
////////////////////////////////////////////////////////////////////////////////
// Merge step 1: find sample ranks in each segment
////////////////////////////////////////////////////////////////////////////////
static void generateSampleRanks(uint *ranksA, uint *ranksB, uint *srcKey, uint stride, uint N, uint sortDir)
{
uint lastSegmentElements = N % (2 * stride);
uint sampleCount = (lastSegmentElements > stride) ? (N + 2 * stride - lastSegmentElements) / (2 * SAMPLE_STRIDE)
: (N - lastSegmentElements) / (2 * SAMPLE_STRIDE);
for (uint pos = 0; pos < sampleCount; pos++) {
const uint i = pos & ((stride / SAMPLE_STRIDE) - 1);
const uint segmentBase = (pos - i) * (2 * SAMPLE_STRIDE);
const uint lenA = stride;
const uint lenB = umin(stride, N - segmentBase - stride);
const uint nA = stride / SAMPLE_STRIDE;
const uint nB = getSampleCount(lenB);
if (i < nA) {
ranksA[(segmentBase + 0) / SAMPLE_STRIDE + i] = i * SAMPLE_STRIDE;
ranksB[(segmentBase + 0) / SAMPLE_STRIDE + i] = binarySearchExclusive(
srcKey[segmentBase + i * SAMPLE_STRIDE], srcKey + segmentBase + stride, lenB, sortDir);
}
if (i < nB) {
ranksB[(segmentBase + stride) / SAMPLE_STRIDE + i] = i * SAMPLE_STRIDE;
ranksA[(segmentBase + stride) / SAMPLE_STRIDE + i] = binarySearchInclusive(
srcKey[segmentBase + stride + i * SAMPLE_STRIDE], srcKey + segmentBase, lenA, sortDir);
}
}
}
////////////////////////////////////////////////////////////////////////////////
// Merge step 2: merge ranks and indices to derive elementary intervals
////////////////////////////////////////////////////////////////////////////////
static void mergeRanksAndIndices(uint *limits, uint *ranks, uint stride, uint N)
{
uint lastSegmentElements = N % (2 * stride);
uint sampleCount = (lastSegmentElements > stride) ? (N + 2 * stride - lastSegmentElements) / (2 * SAMPLE_STRIDE)
: (N - lastSegmentElements) / (2 * SAMPLE_STRIDE);
for (uint pos = 0; pos < sampleCount; pos++) {
const uint i = pos & ((stride / SAMPLE_STRIDE) - 1);
const uint segmentBase = (pos - i) * (2 * SAMPLE_STRIDE);
const uint lenA = stride;
const uint lenB = umin(stride, N - segmentBase - stride);
const uint nA = stride / SAMPLE_STRIDE;
const uint nB = getSampleCount(lenB);
if (i < nA) {
uint dstPosA =
binarySearchExclusive(
ranks[(segmentBase + 0) / SAMPLE_STRIDE + i], ranks + (segmentBase + stride) / SAMPLE_STRIDE, nB, 1)
+ i;
assert(dstPosA < nA + nB);
limits[(segmentBase / SAMPLE_STRIDE) + dstPosA] = ranks[(segmentBase + 0) / SAMPLE_STRIDE + i];
}
if (i < nB) {
uint dstPosA =
binarySearchInclusive(
ranks[(segmentBase + stride) / SAMPLE_STRIDE + i], ranks + (segmentBase + 0) / SAMPLE_STRIDE, nA, 1)
+ i;
assert(dstPosA < nA + nB);
limits[(segmentBase / SAMPLE_STRIDE) + dstPosA] = ranks[(segmentBase + stride) / SAMPLE_STRIDE + i];
}
}
}
////////////////////////////////////////////////////////////////////////////////
// Merge step 3: merge elementary intervals (each interval is <= SAMPLE_STRIDE)
////////////////////////////////////////////////////////////////////////////////
static void merge(uint *dstKey,
uint *dstVal,
uint *srcAKey,
uint *srcAVal,
uint *srcBKey,
uint *srcBVal,
uint lenA,
uint lenB,
uint sortDir)
{
checkOrder(srcAKey, lenA, sortDir);
checkOrder(srcBKey, lenB, sortDir);
for (uint i = 0; i < lenA; i++) {
uint dstPos = binarySearchExclusive(srcAKey[i], srcBKey, lenB, sortDir) + i;
assert(dstPos < lenA + lenB);
dstKey[dstPos] = srcAKey[i];
dstVal[dstPos] = srcAVal[i];
}
for (uint i = 0; i < lenB; i++) {
uint dstPos = binarySearchInclusive(srcBKey[i], srcAKey, lenA, sortDir) + i;
assert(dstPos < lenA + lenB);
dstKey[dstPos] = srcBKey[i];
dstVal[dstPos] = srcBVal[i];
}
}
static void mergeElementaryIntervals(uint *dstKey,
uint *dstVal,
uint *srcKey,
uint *srcVal,
uint *limitsA,
uint *limitsB,
uint stride,
uint N,
uint sortDir)
{
uint lastSegmentElements = N % (2 * stride);
uint mergePairs = (lastSegmentElements > stride) ? getSampleCount(N) : (N - lastSegmentElements) / SAMPLE_STRIDE;
for (uint pos = 0; pos < mergePairs; pos++) {
uint i = pos & ((2 * stride) / SAMPLE_STRIDE - 1);
uint segmentBase = (pos - i) * SAMPLE_STRIDE;
const uint lenA = stride;
const uint lenB = umin(stride, N - segmentBase - stride);
const uint nA = stride / SAMPLE_STRIDE;
const uint nB = getSampleCount(lenB);
const uint n = nA + nB;
const uint startPosA = limitsA[pos];
const uint endPosA = (i + 1 < n) ? limitsA[pos + 1] : lenA;
const uint startPosB = limitsB[pos];
const uint endPosB = (i + 1 < n) ? limitsB[pos + 1] : lenB;
const uint startPosDst = startPosA + startPosB;
assert(startPosA <= endPosA && endPosA <= lenA);
assert(startPosB <= endPosB && endPosB <= lenB);
assert((endPosA - startPosA) <= SAMPLE_STRIDE);
assert((endPosB - startPosB) <= SAMPLE_STRIDE);
merge(dstKey + segmentBase + startPosDst,
dstVal + segmentBase + startPosDst,
(srcKey + segmentBase + 0) + startPosA,
(srcVal + segmentBase + 0) + startPosA,
(srcKey + segmentBase + stride) + startPosB,
(srcVal + segmentBase + stride) + startPosB,
endPosA - startPosA,
endPosB - startPosB,
sortDir);
}
}
////////////////////////////////////////////////////////////////////////////////
// Retarded bubble sort
////////////////////////////////////////////////////////////////////////////////
static void bubbleSort(uint *key, uint *val, uint N, uint sortDir)
{
if (N <= 1) {
return;
}
for (uint bottom = 0; bottom < N - 1; bottom++) {
uint savePos = bottom;
uint saveKey = key[bottom];
for (uint i = bottom + 1; i < N; i++)
if ((sortDir && (key[i] < saveKey)) || (!sortDir && (key[i] > saveKey))) {
savePos = i;
saveKey = key[i];
}
if (savePos != bottom) {
uint t;
t = key[savePos];
key[savePos] = key[bottom];
key[bottom] = t;
t = val[savePos];
val[savePos] = val[bottom];
val[bottom] = t;
}
}
}
////////////////////////////////////////////////////////////////////////////////
// Interface function
////////////////////////////////////////////////////////////////////////////////
extern "C" void
mergeSortHost(uint *dstKey, uint *dstVal, uint *bufKey, uint *bufVal, uint *srcKey, uint *srcVal, uint N, uint sortDir)
{
uint *ikey, *ival, *okey, *oval;
uint stageCount = 0;
for (uint stride = SHARED_SIZE_LIMIT; stride < N; stride <<= 1, stageCount++)
;
if (stageCount & 1) {
ikey = bufKey;
ival = bufVal;
okey = dstKey;
oval = dstVal;
}
else {
ikey = dstKey;
ival = dstVal;
okey = bufKey;
oval = bufVal;
}
printf("Bottom-level sort...\n");
memcpy(ikey, srcKey, N * sizeof(uint));
memcpy(ival, srcVal, N * sizeof(uint));
for (uint pos = 0; pos < N; pos += SHARED_SIZE_LIMIT) {
bubbleSort(ikey + pos, ival + pos, umin(SHARED_SIZE_LIMIT, N - pos), sortDir);
}
printf("Merge...\n");
uint *ranksA = (uint *)malloc(getSampleCount(N) * sizeof(uint));
uint *ranksB = (uint *)malloc(getSampleCount(N) * sizeof(uint));
uint *limitsA = (uint *)malloc(getSampleCount(N) * sizeof(uint));
uint *limitsB = (uint *)malloc(getSampleCount(N) * sizeof(uint));
memset(ranksA, 0xFF, getSampleCount(N) * sizeof(uint));
memset(ranksB, 0xFF, getSampleCount(N) * sizeof(uint));
memset(limitsA, 0xFF, getSampleCount(N) * sizeof(uint));
memset(limitsB, 0xFF, getSampleCount(N) * sizeof(uint));
for (uint stride = SHARED_SIZE_LIMIT; stride < N; stride <<= 1) {
uint lastSegmentElements = N % (2 * stride);
// Find sample ranks and prepare for limiters merge
generateSampleRanks(ranksA, ranksB, ikey, stride, N, sortDir);
// Merge ranks and indices
mergeRanksAndIndices(limitsA, ranksA, stride, N);
mergeRanksAndIndices(limitsB, ranksB, stride, N);
// Merge elementary intervals
mergeElementaryIntervals(okey, oval, ikey, ival, limitsA, limitsB, stride, N, sortDir);
if (lastSegmentElements <= stride) {
// Last merge segment consists of a single array which just needs to be
// passed through
memcpy(
okey + (N - lastSegmentElements), ikey + (N - lastSegmentElements), lastSegmentElements * sizeof(uint));
memcpy(
oval + (N - lastSegmentElements), ival + (N - lastSegmentElements), lastSegmentElements * sizeof(uint));
}
uint *t;
t = ikey;
ikey = okey;
okey = t;
t = ival;
ival = oval;
oval = t;
}
free(limitsB);
free(limitsA);
free(ranksB);
free(ranksA);
}

128
cpp/0_Introduction/mergeSort/mergeSort_validate.cpp Normal file
View File

@@ -0,0 +1,128 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#include <assert.h>
#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include "mergeSort_common.h"
////////////////////////////////////////////////////////////////////////////////
// Validate sorted keys array (check for integrity and proper order)
////////////////////////////////////////////////////////////////////////////////
extern "C" uint
validateSortedKeys(uint *resKey, uint *srcKey, uint batchSize, uint arrayLength, uint numValues, uint sortDir)
{
uint *srcHist;
uint *resHist;
if (arrayLength < 2) {
printf("validateSortedKeys(): arrays too short, exiting...\n");
return 1;
}
printf("...inspecting keys array: ");
srcHist = (uint *)malloc(numValues * sizeof(uint));
resHist = (uint *)malloc(numValues * sizeof(uint));
int flag = 1;
for (uint j = 0; j < batchSize; j++, srcKey += arrayLength, resKey += arrayLength) {
// Build histograms for keys arrays
memset(srcHist, 0, numValues * sizeof(uint));
memset(resHist, 0, numValues * sizeof(uint));
for (uint i = 0; i < arrayLength; i++) {
if ((srcKey[i] < numValues) && (resKey[i] < numValues)) {
srcHist[srcKey[i]]++;
resHist[resKey[i]]++;
}
else {
fprintf(stderr, "***Set %u source/result key arrays are not limited properly***\n", j);
flag = 0;
goto brk;
}
}
// Compare the histograms
for (uint i = 0; i < numValues; i++)
if (srcHist[i] != resHist[i]) {
fprintf(stderr, "***Set %u source/result keys histograms do not match***\n", j);
flag = 0;
goto brk;
}
// Finally check the ordering
for (uint i = 0; i < arrayLength - 1; i++)
if ((sortDir && (resKey[i] > resKey[i + 1])) || (!sortDir && (resKey[i] < resKey[i + 1]))) {
fprintf(stderr, "***Set %u result key array is not ordered properly***\n", j);
flag = 0;
goto brk;
}
}
brk:
free(resHist);
free(srcHist);
if (flag)
printf("OK\n");
return flag;
}
////////////////////////////////////////////////////////////////////////////////
// Value validation / stability check routines
////////////////////////////////////////////////////////////////////////////////
extern "C" void fillValues(uint *val, uint N)
{
for (uint i = 0; i < N; i++)
val[i] = i;
}
extern "C" int validateSortedValues(uint *resKey, uint *resVal, uint *srcKey, uint batchSize, uint arrayLength)
{
int correctFlag = 1, stableFlag = 1;
printf("...inspecting keys and values array: ");
for (uint i = 0; i < batchSize; i++, resKey += arrayLength, resVal += arrayLength) {
for (uint j = 0; j < arrayLength; j++) {
if (resKey[j] != srcKey[resVal[j]])
correctFlag = 0;
if ((j < arrayLength - 1) && (resKey[j] == resKey[j + 1]) && (resVal[j] > resVal[j + 1]))
stableFlag = 0;
}
}
printf(correctFlag ? "OK\n" : "***corrupted!!!***\n");
printf(stableFlag ? "...stability property: stable!\n" : "...stability property: NOT stable\n");
return correctFlag;
}

18
cpp/0_Introduction/simpleAWBarrier/.vscode/c_cpp_properties.json vendored Normal file
View File

@@ -0,0 +1,18 @@
{
"configurations": [
{
"name": "Linux",
"includePath": [
"${workspaceFolder}/**",
"${workspaceFolder}/../../../Common"
],
"defines": [],
"compilerPath": "/usr/local/cuda/bin/nvcc",
"cStandard": "gnu17",
"cppStandard": "gnu++14",
"intelliSenseMode": "linux-gcc-x64",
"configurationProvider": "ms-vscode.makefile-tools"
}
],
"version": 4
}

7
cpp/0_Introduction/simpleAWBarrier/.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,7 @@
{
"recommendations": [
"nvidia.nsight-vscode-edition",
"ms-vscode.cpptools",
"ms-vscode.makefile-tools"
]
}

41
cpp/0_Introduction/simpleAWBarrier/CMakeLists.txt Normal file
View File

@@ -0,0 +1,41 @@
cmake_minimum_required(VERSION 3.20)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/Modules")
project(simpleAWBarrier LANGUAGES C CXX CUDA)
find_package(CUDAToolkit REQUIRED)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(CMAKE_CUDA_ARCHITECTURES 75 80 86 87 89 90 100 110 120)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -Wno-deprecated-gpu-targets")
if(ENABLE_CUDA_DEBUG)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -G") # enable cuda-gdb (may significantly affect performance on some targets)
else()
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -lineinfo") # add line information to all builds for debug tools (exclusive to -G option)
endif()
# Include directories and libraries
include_directories(../../../Common)
# This sample is not supported on QNX
if(CMAKE_SYSTEM_NAME STREQUAL "QNX")
message(STATUS "simpleAWBarrier is not supported on QNX")
return()
endif()
# Source file
# Add target for simpleAWBarrier
add_executable(simpleAWBarrier simpleAWBarrier.cu)
target_compile_options(simpleAWBarrier PRIVATE $<$<COMPILE_LANGUAGE:CUDA>:--extended-lambda>)
target_compile_features(simpleAWBarrier PRIVATE cxx_std_17 cuda_std_17)
set_target_properties(simpleAWBarrier PROPERTIES CUDA_SEPARABLE_COMPILATION ON)
# Include installation configuration
include(${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/InstallSamples.cmake)
setup_samples_install()

36
cpp/0_Introduction/simpleAWBarrier/README.md Normal file
View File

@@ -0,0 +1,36 @@
# simpleAWBarrier - Simple Arrive Wait Barrier
## Description
A simple demonstration of arrive wait barriers.
## Key Concepts
Arrive Wait Barrier
## Supported SM Architectures
[SM 7.0 ](https://developer.nvidia.com/cuda-gpus) [SM 7.2 ](https://developer.nvidia.com/cuda-gpus) [SM 7.5 ](https://developer.nvidia.com/cuda-gpus) [SM 8.0 ](https://developer.nvidia.com/cuda-gpus) [SM 8.6 ](https://developer.nvidia.com/cuda-gpus) [SM 8.7 ](https://developer.nvidia.com/cuda-gpus) [SM 8.9 ](https://developer.nvidia.com/cuda-gpus) [SM 9.0 ](https://developer.nvidia.com/cuda-gpus)
## Supported OSes
Linux, Windows, QNX
## Supported CPU Architecture
x86_64, armv7l, aarch64
## CUDA APIs involved
### [CUDA Runtime API](http://docs.nvidia.com/cuda/cuda-runtime-api/index.html)
cudaStreamCreateWithFlags, cudaFree, cudaDeviceGetAttribute, cudaMallocHost, cudaFreeHost, cudaStreamSynchronize, cudaLaunchCooperativeKernel, cudaMalloc, cudaOccupancyMaxActiveBlocksPerMultiprocessor, cudaMemcpyAsync, cudaOccupancyMaxPotentialBlockSize
## Dependencies needed to build/run
[C++11 CUDA](../../../README.md#c11-cuda), [MBCG](../../../README.md#multi-block-cooperative-groups)
## Prerequisites
Download and install the [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) for your corresponding platform.
Make sure the dependencies mentioned in [Dependencies]() section above are installed.
## References (for more details)

248
cpp/0_Introduction/simpleAWBarrier/simpleAWBarrier.cu Normal file
View File

@@ -0,0 +1,248 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
// Includes, system
#include <stdio.h>
// Includes CUDA
#include <cooperative_groups.h>
#include <cuda/barrier>
#include <cuda_runtime.h>
// Utilities and timing functions
#include <helper_functions.h> // includes cuda.h and cuda_runtime_api.h
// CUDA helper functions
#include <helper_cuda.h> // helper functions for CUDA error check
namespace cg = cooperative_groups;
#if __CUDA_ARCH__ >= 700
template <bool writeSquareRoot>
__device__ void reduceBlockData(cuda::barrier<cuda::thread_scope_block> &barrier,
cg::thread_block_tile<32> &tile32,
double &threadSum,
double *result)
{
extern __shared__ double tmp[];
#pragma unroll
for (int offset = tile32.size() / 2; offset > 0; offset /= 2) {
threadSum += tile32.shfl_down(threadSum, offset);
}
if (tile32.thread_rank() == 0) {
tmp[tile32.meta_group_rank()] = threadSum;
}
auto token = barrier.arrive();
barrier.wait(std::move(token));
// The warp 0 will perform last round of reduction
if (tile32.meta_group_rank() == 0) {
double beta = tile32.thread_rank() < tile32.meta_group_size() ? tmp[tile32.thread_rank()] : 0.0;
#pragma unroll
for (int offset = tile32.size() / 2; offset > 0; offset /= 2) {
beta += tile32.shfl_down(beta, offset);
}
if (tile32.thread_rank() == 0) {
if (writeSquareRoot)
*result = sqrt(beta);
else
*result = beta;
}
}
}
#endif
__global__ void normVecByDotProductAWBarrier(float *vecA, float *vecB, double *partialResults, int size)
{
#if __CUDA_ARCH__ >= 700
#pragma diag_suppress static_var_with_dynamic_init
cg::thread_block cta = cg::this_thread_block();
cg::grid_group grid = cg::this_grid();
;
cg::thread_block_tile<32> tile32 = cg::tiled_partition<32>(cta);
__shared__ cuda::barrier<cuda::thread_scope_block> barrier;
if (threadIdx.x == 0) {
init(&barrier, blockDim.x);
}
cg::sync(cta);
double threadSum = 0.0;
for (int i = grid.thread_rank(); i < size; i += grid.size()) {
threadSum += (double)(vecA[i] * vecB[i]);
}
// Each thread block performs reduction of partial dotProducts and writes to
// global mem.
reduceBlockData<false>(barrier, tile32, threadSum, &partialResults[blockIdx.x]);
cg::sync(grid);
// One block performs the final summation of partial dot products
// of all the thread blocks and writes the sqrt of final dot product.
if (blockIdx.x == 0) {
threadSum = 0.0;
for (int i = cta.thread_rank(); i < gridDim.x; i += cta.size()) {
threadSum += partialResults[i];
}
reduceBlockData<true>(barrier, tile32, threadSum, &partialResults[0]);
}
cg::sync(grid);
const double finalValue = partialResults[0];
// Perform normalization of vecA & vecB.
for (int i = grid.thread_rank(); i < size; i += grid.size()) {
vecA[i] = (float)vecA[i] / finalValue;
vecB[i] = (float)vecB[i] / finalValue;
}
#endif
}
int runNormVecByDotProductAWBarrier(int argc, char **argv, int deviceId);
////////////////////////////////////////////////////////////////////////////////
// Program main
////////////////////////////////////////////////////////////////////////////////
int main(int argc, char **argv)
{
printf("%s starting...\n", argv[0]);
// This will pick the best possible CUDA capable device
int dev = findCudaDevice(argc, (const char **)argv);
int major = 0;
checkCudaErrors(cudaDeviceGetAttribute(&major, cudaDevAttrComputeCapabilityMajor, dev));
// Arrive-Wait Barrier require a GPU of Volta (SM7X) architecture or higher.
if (major < 7) {
printf("simpleAWBarrier requires SM 7.0 or higher. Exiting...\n");
exit(EXIT_WAIVED);
}
int supportsCooperativeLaunch = 0;
checkCudaErrors(cudaDeviceGetAttribute(&supportsCooperativeLaunch, cudaDevAttrCooperativeLaunch, dev));
if (!supportsCooperativeLaunch) {
printf("\nSelected GPU (%d) does not support Cooperative Kernel Launch, "
"Waiving the run\n",
dev);
exit(EXIT_WAIVED);
}
int testResult = runNormVecByDotProductAWBarrier(argc, argv, dev);
printf("%s completed, returned %s\n", argv[0], testResult ? "OK" : "ERROR!");
exit(testResult ? EXIT_SUCCESS : EXIT_FAILURE);
}
int runNormVecByDotProductAWBarrier(int argc, char **argv, int deviceId)
{
float *vecA, *d_vecA;
float *vecB, *d_vecB;
double *d_partialResults;
int size = 10000000;
checkCudaErrors(cudaMallocHost(&vecA, sizeof(float) * size));
checkCudaErrors(cudaMallocHost(&vecB, sizeof(float) * size));
checkCudaErrors(cudaMalloc(&d_vecA, sizeof(float) * size));
checkCudaErrors(cudaMalloc(&d_vecB, sizeof(float) * size));
float baseVal = 2.0;
for (int i = 0; i < size; i++) {
vecA[i] = vecB[i] = baseVal;
}
cudaStream_t stream;
checkCudaErrors(cudaStreamCreateWithFlags(&stream, cudaStreamNonBlocking));
checkCudaErrors(cudaMemcpyAsync(d_vecA, vecA, sizeof(float) * size, cudaMemcpyHostToDevice, stream));
checkCudaErrors(cudaMemcpyAsync(d_vecB, vecB, sizeof(float) * size, cudaMemcpyHostToDevice, stream));
// Kernel configuration, where a one-dimensional
// grid and one-dimensional blocks are configured.
int minGridSize = 0, blockSize = 0;
checkCudaErrors(
cudaOccupancyMaxPotentialBlockSize(&minGridSize, &blockSize, (void *)normVecByDotProductAWBarrier, 0, size));
int smemSize = ((blockSize / 32) + 1) * sizeof(double);
int numBlocksPerSm = 0;
checkCudaErrors(cudaOccupancyMaxActiveBlocksPerMultiprocessor(
&numBlocksPerSm, normVecByDotProductAWBarrier, blockSize, smemSize));
int multiProcessorCount = 0;
checkCudaErrors(cudaDeviceGetAttribute(&multiProcessorCount, cudaDevAttrMultiProcessorCount, deviceId));
minGridSize = multiProcessorCount * numBlocksPerSm;
checkCudaErrors(cudaMalloc(&d_partialResults, minGridSize * sizeof(double)));
printf("Launching normVecByDotProductAWBarrier kernel with numBlocks = %d "
"blockSize = %d\n",
minGridSize,
blockSize);
dim3 dimGrid(minGridSize, 1, 1), dimBlock(blockSize, 1, 1);
void *kernelArgs[] = {(void *)&d_vecA, (void *)&d_vecB, (void *)&d_partialResults, (void *)&size};
checkCudaErrors(cudaLaunchCooperativeKernel(
(void *)normVecByDotProductAWBarrier, dimGrid, dimBlock, kernelArgs, smemSize, stream));
checkCudaErrors(cudaMemcpyAsync(vecA, d_vecA, sizeof(float) * size, cudaMemcpyDeviceToHost, stream));
checkCudaErrors(cudaStreamSynchronize(stream));
float expectedResult = (baseVal / sqrt(size * baseVal * baseVal));
unsigned int matches = 0;
for (int i = 0; i < size; i++) {
if ((vecA[i] - expectedResult) > 0.00001) {
printf("mismatch at i = %d\n", i);
break;
}
else {
matches++;
}
}
printf("Result = %s\n", matches == size ? "PASSED" : "FAILED");
checkCudaErrors(cudaFree(d_vecA));
checkCudaErrors(cudaFree(d_vecB));
checkCudaErrors(cudaFree(d_partialResults));
checkCudaErrors(cudaFreeHost(vecA));
checkCudaErrors(cudaFreeHost(vecB));
return matches == size;
}

18
cpp/0_Introduction/simpleAssert/.vscode/c_cpp_properties.json vendored Normal file
View File

@@ -0,0 +1,18 @@
{
"configurations": [
{
"name": "Linux",
"includePath": [
"${workspaceFolder}/**",
"${workspaceFolder}/../../../Common"
],
"defines": [],
"compilerPath": "/usr/local/cuda/bin/nvcc",
"cStandard": "gnu17",
"cppStandard": "gnu++14",
"intelliSenseMode": "linux-gcc-x64",
"configurationProvider": "ms-vscode.makefile-tools"
}
],
"version": 4
}

7
cpp/0_Introduction/simpleAssert/.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,7 @@
{
"recommendations": [
"nvidia.nsight-vscode-edition",
"ms-vscode.cpptools",
"ms-vscode.makefile-tools"
]
}

37
cpp/0_Introduction/simpleAssert/CMakeLists.txt Normal file
View File

@@ -0,0 +1,37 @@
cmake_minimum_required(VERSION 3.20)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/Modules")
project(simpleAssert LANGUAGES C CXX CUDA)
find_package(CUDAToolkit REQUIRED)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(CMAKE_CUDA_ARCHITECTURES 75 80 86 87 89 90 100 110 120)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -Wno-deprecated-gpu-targets")
if(ENABLE_CUDA_DEBUG)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -G") # enable cuda-gdb (may significantly affect performance on some targets)
else()
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -lineinfo") # add line information to all builds for debug tools (exclusive to -G option)
endif()
# Removes -DNDEBUG For Print specific logs in this sample.
string(REPLACE "-DNDEBUG" "" CMAKE_CUDA_FLAGS_RELEASE "${CMAKE_CUDA_FLAGS_RELEASE}")
# Include directories and libraries
include_directories(../../../Common)
# Source file
# Add target for simpleAssert
add_executable(simpleAssert simpleAssert.cu)
target_compile_options(simpleAssert PRIVATE $<$<COMPILE_LANGUAGE:CUDA>:--extended-lambda>)
target_compile_features(simpleAssert PRIVATE cxx_std_17 cuda_std_17)
set_target_properties(simpleAssert PROPERTIES CUDA_SEPARABLE_COMPILATION ON)
# Include installation configuration
include(${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/InstallSamples.cmake)
setup_samples_install()

32
cpp/0_Introduction/simpleAssert/README.md Normal file
View File

@@ -0,0 +1,32 @@
# simpleAssert - simpleAssert
## Description
This CUDA Runtime API sample is a very basic sample that implements how to use the assert function in the device code. Requires Compute Capability 2.0 .
## Key Concepts
Assert
## Supported SM Architectures
[SM 5.0 ](https://developer.nvidia.com/cuda-gpus) [SM 5.2 ](https://developer.nvidia.com/cuda-gpus) [SM 5.3 ](https://developer.nvidia.com/cuda-gpus) [SM 6.0 ](https://developer.nvidia.com/cuda-gpus) [SM 6.1 ](https://developer.nvidia.com/cuda-gpus) [SM 7.0 ](https://developer.nvidia.com/cuda-gpus) [SM 7.2 ](https://developer.nvidia.com/cuda-gpus) [SM 7.5 ](https://developer.nvidia.com/cuda-gpus) [SM 8.0 ](https://developer.nvidia.com/cuda-gpus) [SM 8.6 ](https://developer.nvidia.com/cuda-gpus) [SM 8.7 ](https://developer.nvidia.com/cuda-gpus) [SM 8.9 ](https://developer.nvidia.com/cuda-gpus) [SM 9.0 ](https://developer.nvidia.com/cuda-gpus)
## Supported OSes
Linux, Windows
## Supported CPU Architecture
x86_64, armv7l
## CUDA APIs involved
### [CUDA Runtime API](http://docs.nvidia.com/cuda/cuda-runtime-api/index.html)
cudaDeviceSynchronize, cudaGetErrorString
## Prerequisites
Download and install the [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) for your corresponding platform.
## References (for more details)

130
cpp/0_Introduction/simpleAssert/simpleAssert.cu Normal file
View File

@@ -0,0 +1,130 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifdef _WIN32
#define WINDOWS_LEAN_AND_MEAN
#define NOMINMAX
#include <windows.h>
#else
#include <sys/utsname.h>
#endif
// Includes, system
#include <cassert>
#include <stdio.h>
// Includes CUDA
#include <cuda_runtime.h>
// Utilities and timing functions
#include <helper_functions.h> // includes cuda.h and cuda_runtime_api.h
// CUDA helper functions
#include <helper_cuda.h> // helper functions for CUDA error check
const char *sampleName = "simpleAssert";
////////////////////////////////////////////////////////////////////////////////
// Auto-Verification Code
bool testResult = true;
////////////////////////////////////////////////////////////////////////////////
// Kernels
////////////////////////////////////////////////////////////////////////////////
//! Tests assert function.
//! Thread whose id > N will print assertion failed error message.
////////////////////////////////////////////////////////////////////////////////
__global__ void testKernel(int N)
{
int gtid = blockIdx.x * blockDim.x + threadIdx.x;
assert(gtid < N);
}
////////////////////////////////////////////////////////////////////////////////
// Declaration, forward
void runTest(int argc, char **argv);
////////////////////////////////////////////////////////////////////////////////
// Program main
////////////////////////////////////////////////////////////////////////////////
int main(int argc, char **argv)
{
printf("%s starting...\n", sampleName);
runTest(argc, argv);
printf("%s completed, returned %s\n", sampleName, testResult ? "OK" : "ERROR!");
exit(testResult ? EXIT_SUCCESS : EXIT_FAILURE);
}
void runTest(int argc, char **argv)
{
int Nblocks = 2;
int Nthreads = 32;
cudaError_t error;
#ifndef _WIN32
utsname OS_System_Type;
uname(&OS_System_Type);
printf("OS_System_Type.release = %s\n", OS_System_Type.release);
if (!strcasecmp(OS_System_Type.sysname, "Darwin")) {
printf("simpleAssert is not current supported on Mac OSX\n\n");
exit(EXIT_SUCCESS);
}
else {
printf("OS Info: <%s>\n\n", OS_System_Type.version);
}
#endif
// This will pick the best possible CUDA capable device
findCudaDevice(argc, (const char **)argv);
// Kernel configuration, where a one-dimensional
// grid and one-dimensional blocks are configured.
dim3 dimGrid(Nblocks);
dim3 dimBlock(Nthreads);
printf("Launch kernel to generate assertion failures\n");
testKernel<<<dimGrid, dimBlock>>>(60);
// Synchronize (flushes assert output).
printf("\n-- Begin assert output\n\n");
error = cudaDeviceSynchronize();
printf("\n-- End assert output\n\n");
// Check for errors and failed asserts in asynchronous kernel launch.
if (error == cudaErrorAssert) {
printf("Device assert failed as expected, "
"CUDA error message is: %s\n\n",
cudaGetErrorString(error));
}
testResult = error == cudaErrorAssert;
}

18
cpp/0_Introduction/simpleAssert_nvrtc/.vscode/c_cpp_properties.json vendored Normal file
View File

@@ -0,0 +1,18 @@
{
"configurations": [
{
"name": "Linux",
"includePath": [
"${workspaceFolder}/**",
"${workspaceFolder}/../../../Common"
],
"defines": [],
"compilerPath": "/usr/local/cuda/bin/nvcc",
"cStandard": "gnu17",
"cppStandard": "gnu++14",
"intelliSenseMode": "linux-gcc-x64",
"configurationProvider": "ms-vscode.makefile-tools"
}
],
"version": 4
}

7
cpp/0_Introduction/simpleAssert_nvrtc/.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,7 @@
{
"recommendations": [
"nvidia.nsight-vscode-edition",
"ms-vscode.cpptools",
"ms-vscode.makefile-tools"
]
}

43
cpp/0_Introduction/simpleAssert_nvrtc/CMakeLists.txt Normal file
View File

@@ -0,0 +1,43 @@
cmake_minimum_required(VERSION 3.20)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/Modules")
project(simpleAssert_nvrtc LANGUAGES C CXX CUDA)
find_package(CUDAToolkit REQUIRED)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(CMAKE_CUDA_ARCHITECTURES 75 80 86 87 89 90 100 110 120)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -Wno-deprecated-gpu-targets")
if(ENABLE_CUDA_DEBUG)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -G") # enable cuda-gdb (may significantly affect performance on some targets)
else()
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -lineinfo") # add line information to all builds for debug tools (exclusive to -G option)
endif()
# Include directories and libraries
include_directories(../../../Common)
# Source file
# Add sample target executable
add_executable(simpleAssert_nvrtc simpleAssert.cpp)
target_compile_options(simpleAssert_nvrtc PRIVATE $<$<COMPILE_LANGUAGE:CUDA>:--extended-lambda>)
target_compile_features(simpleAssert_nvrtc PRIVATE cxx_std_17 cuda_std_17)
target_link_libraries(simpleAssert_nvrtc PRIVATE
CUDA::nvrtc
CUDA::cuda_driver
)
# Copy clock_kernel.cu to the output directory
add_custom_command(TARGET simpleAssert_nvrtc POST_BUILD
COMMAND ${CMAKE_COMMAND} -E copy_if_different
${CMAKE_CURRENT_SOURCE_DIR}/simpleAssert_kernel.cu ${CMAKE_CURRENT_BINARY_DIR}
)
# Include installation configuration
include(${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/InstallSamples.cmake)
setup_samples_install()

36
cpp/0_Introduction/simpleAssert_nvrtc/README.md Normal file
View File

@@ -0,0 +1,36 @@
# simpleAssert_nvrtc - simpleAssert with libNVRTC
## Description
This CUDA Runtime API sample is a very basic sample that implements how to use the assert function in the device code. Requires Compute Capability 2.0 .
## Key Concepts
Assert, Runtime Compilation
## Supported SM Architectures
[SM 5.0 ](https://developer.nvidia.com/cuda-gpus) [SM 5.2 ](https://developer.nvidia.com/cuda-gpus) [SM 5.3 ](https://developer.nvidia.com/cuda-gpus) [SM 6.0 ](https://developer.nvidia.com/cuda-gpus) [SM 6.1 ](https://developer.nvidia.com/cuda-gpus) [SM 7.0 ](https://developer.nvidia.com/cuda-gpus) [SM 7.2 ](https://developer.nvidia.com/cuda-gpus) [SM 7.5 ](https://developer.nvidia.com/cuda-gpus) [SM 8.0 ](https://developer.nvidia.com/cuda-gpus) [SM 8.6 ](https://developer.nvidia.com/cuda-gpus) [SM 8.7 ](https://developer.nvidia.com/cuda-gpus) [SM 8.9 ](https://developer.nvidia.com/cuda-gpus) [SM 9.0 ](https://developer.nvidia.com/cuda-gpus)
## Supported OSes
Linux, Windows, QNX
## Supported CPU Architecture
x86_64, aarch64
## CUDA APIs involved
### [CUDA Driver API](http://docs.nvidia.com/cuda/cuda-driver-api/index.html)
cuModuleGetFunction, cuLaunchKernel, cuCtxSynchronize
## Dependencies needed to build/run
[NVRTC](../../../README.md#nvrtc)
## Prerequisites
Download and install the [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) for your corresponding platform.
Make sure the dependencies mentioned in [Dependencies]() section above are installed.
## References (for more details)

121
cpp/0_Introduction/simpleAssert_nvrtc/simpleAssert.cpp Normal file
View File

@@ -0,0 +1,121 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
#ifdef _WIN32
#define WINDOWS_LEAN_AND_MEAN
#define NOMINMAX
#include <windows.h>
#else
#include <sys/utsname.h>
#endif
// Includes, system
#include <cassert>
#include <stdio.h>
// Includes CUDA
#include <cuda_runtime.h>
#include "nvrtc_helper.h"
// Utilities and timing functions
#include <helper_functions.h> // includes cuda.h and cuda_runtime_api.h
const char *sampleName = "simpleAssert_nvrtc";
////////////////////////////////////////////////////////////////////////////////
// Auto-Verification Code
bool testResult = true;
////////////////////////////////////////////////////////////////////////////////
// Declaration, forward
void runTest(int argc, char **argv);
////////////////////////////////////////////////////////////////////////////////
// Program main
////////////////////////////////////////////////////////////////////////////////
int main(int argc, char **argv)
{
printf("%s starting...\n", sampleName);
runTest(argc, argv);
exit(testResult ? EXIT_SUCCESS : EXIT_FAILURE);
}
void runTest(int argc, char **argv)
{
int Nblocks = 2;
int Nthreads = 32;
// Kernel configuration, where a one-dimensional
// grid and one-dimensional blocks are configured.
dim3 dimGrid(Nblocks);
dim3 dimBlock(Nthreads);
printf("Launch kernel to generate assertion failures\n");
char *cubin, *kernel_file;
size_t cubinSize;
kernel_file = sdkFindFilePath("simpleAssert_kernel.cu", argv[0]);
compileFileToCUBIN(kernel_file, argc, argv, &cubin, &cubinSize, 0);
CUmodule module = loadCUBIN(cubin, argc, argv);
CUfunction kernel_addr;
checkCudaErrors(cuModuleGetFunction(&kernel_addr, module, "testKernel"));
int count = 60;
void *args[] = {(void *)&count};
checkCudaErrors(cuLaunchKernel(kernel_addr,
dimGrid.x,
dimGrid.y,
dimGrid.z, /* grid dim */
dimBlock.x,
dimBlock.y,
dimBlock.z, /* block dim */
0,
0, /* shared mem, stream */
&args[0], /* arguments */
0));
// Synchronize (flushes assert output).
printf("\n-- Begin assert output\n\n");
CUresult res = cuCtxSynchronize();
printf("\n-- End assert output\n\n");
// Check for errors and failed asserts in asynchronous kernel launch.
if (res == CUDA_ERROR_ASSERT) {
printf("Device assert failed as expected\n");
}
testResult = res == CUDA_ERROR_ASSERT;
}

39
cpp/0_Introduction/simpleAssert_nvrtc/simpleAssert_kernel.cu Normal file
View File

@@ -0,0 +1,39 @@
/* Copyright (c) 2022, NVIDIA CORPORATION. All rights reserved.
*
* Redistribution and use in source and binary forms, with or without
* modification, are permitted provided that the following conditions
* are met:
* * Redistributions of source code must retain the above copyright
* notice, this list of conditions and the following disclaimer.
* * Redistributions in binary form must reproduce the above copyright
* notice, this list of conditions and the following disclaimer in the
* documentation and/or other materials provided with the distribution.
* * Neither the name of NVIDIA CORPORATION nor the names of its
* contributors may be used to endorse or promote products derived
* from this software without specific prior written permission.
*
* THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS ``AS IS'' AND ANY
* EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE
* IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR
* PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR
* CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL,
* EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO,
* PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR
* PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY
* OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
* (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
* OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
*/
////////////////////////////////////////////////////////////////////////////////
// Kernels
////////////////////////////////////////////////////////////////////////////////
//! Tests assert function.
//! Thread whose id > N will print assertion failed error message.
////////////////////////////////////////////////////////////////////////////////
extern "C" __global__ void testKernel(int N)
{
int gtid = blockIdx.x * blockDim.x + threadIdx.x;
assert(gtid < N);
}

18
cpp/0_Introduction/simpleAtomicIntrinsics/.vscode/c_cpp_properties.json vendored Normal file
View File

@@ -0,0 +1,18 @@
{
"configurations": [
{
"name": "Linux",
"includePath": [
"${workspaceFolder}/**",
"${workspaceFolder}/../../../Common"
],
"defines": [],
"compilerPath": "/usr/local/cuda/bin/nvcc",
"cStandard": "gnu17",
"cppStandard": "gnu++14",
"intelliSenseMode": "linux-gcc-x64",
"configurationProvider": "ms-vscode.makefile-tools"
}
],
"version": 4
}

7
cpp/0_Introduction/simpleAtomicIntrinsics/.vscode/extensions.json vendored Normal file
View File

@@ -0,0 +1,7 @@
{
"recommendations": [
"nvidia.nsight-vscode-edition",
"ms-vscode.cpptools",
"ms-vscode.makefile-tools"
]
}

35
cpp/0_Introduction/simpleAtomicIntrinsics/CMakeLists.txt Normal file
View File

@@ -0,0 +1,35 @@
cmake_minimum_required(VERSION 3.20)
list(APPEND CMAKE_MODULE_PATH "${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/Modules")
project(simpleAtomicIntrinsics LANGUAGES C CXX CUDA)
find_package(CUDAToolkit REQUIRED)
set(CMAKE_POSITION_INDEPENDENT_CODE ON)
set(CMAKE_CUDA_ARCHITECTURES 75 80 86 87 89 90 100 110 120)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -Wno-deprecated-gpu-targets")
if(ENABLE_CUDA_DEBUG)
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -G") # enable cuda-gdb (may significantly affect performance on some targets)
else()
set(CMAKE_CUDA_FLAGS "${CMAKE_CUDA_FLAGS} -lineinfo") # add line information to all builds for debug tools (exclusive to -G option)
endif()
# Include directories and libraries
include_directories(../../../Common)
# Source file
# Add target for simpleAtomicIntrinsics
add_executable(simpleAtomicIntrinsics simpleAtomicIntrinsics.cu simpleAtomicIntrinsics_cpu.cpp)
target_compile_options(simpleAtomicIntrinsics PRIVATE $<$<COMPILE_LANGUAGE:CUDA>:--extended-lambda>)
target_compile_features(simpleAtomicIntrinsics PRIVATE cxx_std_17 cuda_std_17)
set_target_properties(simpleAtomicIntrinsics PROPERTIES CUDA_SEPARABLE_COMPILATION ON)
# Include installation configuration
include(${CMAKE_CURRENT_SOURCE_DIR}/../../../cmake/InstallSamples.cmake)
setup_samples_install()

32
cpp/0_Introduction/simpleAtomicIntrinsics/README.md Normal file
View File

@@ -0,0 +1,32 @@
# simpleAtomicIntrinsics - Simple Atomic Intrinsics
## Description
A simple demonstration of global memory atomic instructions.
## Key Concepts
Atomic Intrinsics
## Supported SM Architectures
[SM 5.0 ](https://developer.nvidia.com/cuda-gpus) [SM 5.2 ](https://developer.nvidia.com/cuda-gpus) [SM 5.3 ](https://developer.nvidia.com/cuda-gpus) [SM 6.0 ](https://developer.nvidia.com/cuda-gpus) [SM 6.1 ](https://developer.nvidia.com/cuda-gpus) [SM 7.0 ](https://developer.nvidia.com/cuda-gpus) [SM 7.2 ](https://developer.nvidia.com/cuda-gpus) [SM 7.5 ](https://developer.nvidia.com/cuda-gpus) [SM 8.0 ](https://developer.nvidia.com/cuda-gpus) [SM 8.6 ](https://developer.nvidia.com/cuda-gpus) [SM 8.7 ](https://developer.nvidia.com/cuda-gpus) [SM 8.9 ](https://developer.nvidia.com/cuda-gpus) [SM 9.0 ](https://developer.nvidia.com/cuda-gpus)
## Supported OSes
Linux, Windows, QNX
## Supported CPU Architecture
x86_64, armv7l, aarch64
## CUDA APIs involved
### [CUDA Runtime API](http://docs.nvidia.com/cuda/cuda-runtime-api/index.html)
cudaStreamCreateWithFlags, cudaFree, cudaMallocHost, cudaFreeHost, cudaStreamSynchronize, cudaMalloc, cudaMemcpyAsync
## Prerequisites
Download and install the [CUDA Toolkit](https://developer.nvidia.com/cuda-downloads) for your corresponding platform.
## References (for more details)

Some files were not shown because too many files have changed in this diff Show More