Streamlining Investment Insights for Wealth Management with Generative AI

The collaboration between UBS and NVIDIA focuses on real-time risk assessment and monitoring of production retrieval augmented generation (RAG). Unlike holistic evaluations of RAG applications, real-time solutions must introduce minimal latency and offer a high degree of live adaptation and reasoning. To ensure the live observability of large language models (LLMs), we use "LLMs as a judge," where the reliability of the answers provided by one LLM is re-evaluated by another LLM before being returned as the system’s output. A common pitfall in "LLM as a judge" applications is that models tend to prefer their own answers, and thus undermine a system’s trustworthiness. To provide a first-of-its-class solution to this problem in the heavily-regulated banking industry, we adopt NVIDIA’s NIMs to deploy an open-source model as the LLM judge and extend it with observability tools from NeMo Guardrails and Evaluator.