Simplifying OCR Serving with Triton Inference Server

At Snap, we apply the latest computer vision technology to create exciting products and breakthrough interactive experiences for millions of Snapchat users around the world. We'll present one of the computer vision problems we tackled: the inference system for optical character recognition (OCR). We will share our experiences in using Triton Inference Server to simplify the OCR inference system. Serving and optimizing modern OCR inference is particularly interesting because custom logic has to be intermixed with deep learning model execution. We were able to 3x increase OCR inference throughput by adopting Triton Inference Server.