GPU Programming with CUDA Training Course

Introduction

• Defining CUDA and its core purpose.
• Comparative analysis: CUDA vs. OpenCL vs. SYCL.
• Overview of key CUDA features and architectural components.
• Configuring the initial development environment.

Getting Started

• Initiating a new CUDA project within Visual Studio Code.
• Exploring the project structure and identifying key files.
• Processes for compiling and executing the program.
• Displaying output results using printf and fprintf functions.

CUDA API

• The role of the CUDA API within the host program.
• Querying device information and capabilities via the CUDA API.
• Managing device memory allocation and deallocation using the CUDA API.
• Transferring data between host and device using the CUDA API.
• Launching kernels and synchronizing threads with the CUDA API.
• Implementing error and exception handling mechanisms in the CUDA API.

CUDA C/C++

• The function of CUDA C/C++ within device-side programming.
• Writing kernels in CUDA C/C++ to execute on the GPU and manipulate data.
• Utilizing CUDA C/C++ data types, qualifiers, operators, and expressions.
• Applying built-in functions for math, atomic operations, and warp-level tasks.
• Accessing built-in variables such as threadIdx, blockIdx, and blockDim.
• Leveraging libraries like cuBLAS, cuFFT, and cuRAND for accelerated computations.

CUDA Memory Model

• Distinguishing between host and device memory models.
• Utilizing different CUDA memory spaces: global, shared, constant, and local.
• Working with CUDA memory objects including pointers, arrays, textures, and surfaces.
• Understanding memory access modes such as read-only, write-only, and read-write.
• Applying the CUDA memory consistency model and synchronization mechanisms.

CUDA Execution Model

• Differentiating host and device execution models.
• Defining parallelism using CUDA threads, blocks, and grids.
• Utilizing thread-specific functions and variables like threadIdx and blockIdx.
• Employing block-level synchronization functions such as __syncthreads and __threadfence_block.
• Managing grid-level operations using gridDim, gridSync, and cooperative groups.

Debugging

• Identifying common errors and bugs in CUDA applications.
• Inspecting variables, breakpoints, and call stacks using the Visual Studio Code debugger.
• Debugging CUDA programs on Linux using CUDA-GDB.
• Detecting memory errors and leaks with CUDA-MEMCHECK.
• Debugging and analyzing CUDA programs on Windows using NVIDIA Nsight.

Optimization

• Analyzing factors that influence CUDA program performance.
• Improving memory throughput using CUDA coalescing techniques.
• Reducing memory latency through caching and prefetching strategies.
• Optimizing memory access patterns and bandwidth using shared and local memory.
• Measuring and enhancing execution time and resource utilization via profiling tools.