Advancing Automotive AI With Large Language Models and Vision Language Models on NVIDIA DRIVE

Large language and vision language models running on NVIDIA DRIVE today are redefining the next generation of automobiles. Vehicle cockpits are becoming a whole new experience: instead of touch screens and voice commands, there’ll be a much more intuitive natural language interaction with the vehicle, enabling the driver to manage their vehicle as if they had a human co-pilot. Beyond the cockpit, LLMs and VLMs show very promising results for advanced driver assistance and autonomous vehicle applications. Their capability to model traffic mechanisms based on vast amounts of text data is key to solving new, unseen driving scenarios. We'll explain the basics of LLM for automotive use cases and show how to deploy and run LLMs on NVIDIA DRIVE Orin using TensorRT-LLM.