The Good, the Bad, and the Ugly: Cost-Efficient LLM Inference With Quantization, Pruning, and Distillation

Dive into theoretical aspects and practical use cases for compression techniques used in creating more efficient versions of preferred models. Learn the main methods of compression — quantization, pruning, distillation — in detail, and compare trade-offs and options for jointly using multiple compression techniques. You'll become well-equipped to perform efficient inference and reduce costs when deploying LLMs. You'll understand the trade-offs in accuracy, latency, and hardware requirements for each method, and have hands-on expertise in deploying them using solutions from NVIDIA stack.
Prerequisite(s):

Python.
Basic knowledge of LLMs.
Familiarity with deep learning frameworks.