CUDA Graphs 101

CUDA Graphs is an asynchronous execution model that reduces the time taken to dispatch work on the GPU and improves GPU runtime. CUDA Graphs allows users to define the entire workflow up front, before submitting it to the GPU — enabling the workflow to be optimized in a way that's impossible within the immediate CUDA streams submission model. We'll talk about CUDA Graphs, from the basics of how the programming model works to how to add graphs to existing applications to new features available in CUDA 12.0. We assume you're familiar with basic CUDA.