CUDA Developer Tools | Memory Analysis with NVIDIA Nsight Compute

This tutorial video introduces memory workload analysis for CUDA applications with NVIDIA Nsight Compute. Memory bottlenecks can limit the performance of your GPU. This is especially true for content creation and other workloads that involve large amounts of data quickly streaming through memory. Use Nsight Compute memory workload analysis to maximize GPU memory bandwidth and optimize data access patterns. 

Highlights of this video tutorial include: 

Memory analysis chart: 
▫️ This chart visualizes hardware memory locality and memory type, including the amount of read or written bytes between physical units.

Overview of caches:
▫️ Memory requests in the kernel follow a hierarchy. L1 is checked first, then L2, and if the sector is not found, it is fetched from device memory.

Optimizing caches: 
▫️ Cache line allocation is crucial for optimal performance, ensuring efficient use of cache storage and reducing memory traffic between L1, L2, and device memory.

Live demonstration:
▫️ Walkthrough optimizing a simple CUDA program that converts 8-bit PNGs from RGBA to grayscale. We inspect the impact of aligned reads and vectorized loads on memory efficiency. 

Interpreting memory analysis:
▫️ Key tips for how to read memory profiles to address the balance between hardware limitations and algorithmic efficiency. 

▫️ Memory Management on Modern GPU Architectures: https://resources.nvidia.com/gtcd-2020/GTC2020cwe21754?lx=3X9y6T
▫️Sample code: https://github.com/NVIDIA/nsight-training/tree/master/cuda/nsight_compute/vlog_memory_workload