Snapchat: Applying GPUs to Very Large-scale Recommender Machine Learning Inference

At Snap, we apply the latest machine learning (ML) technology to find engaging content and relevant ads. ML plays a central role in delivering long-term value to Snapchatters, creators, and our advertisers. We'll share our experiences and insights in applying GPU technology to accelerate ML model inference. Inference refers to the computation-intensive process of calculating model predictions (such as the probability that a Snapchatter finishes watching a video) from input features (such as the number of videos that the Snapchatter watched in the past hour). This is particularly challenging at Snap because we serve our community of over 293 million Snapchatters daily, which creates over 10 trillion ML predictions daily. At this scale, we are constantly looking to improve our inference stack to keep up with growth and lower our cost.