Big NLP Demystified: Business Impact of Large Language Models

In the past year we've seen unprecedented growth of models and datasets that deal with natural language processing (NLP). Models such as GPT-3 or Megatron Turing are now two orders of magnitude larger than models that just recently were state of the art (like BERT, RoBerta). What drives the investment into this technology, and how does the successes of large NLP models transform businesses that rely on NLP? We'll discuss the rationale for developing large language models and demonstrate the impact they are already having with several practical examples. In particular, we'll show how their few-shot learning capability (the ability to perform well with just a handful of training examples) is transforming the way people interact with information. We'll conclude with an overview of resources required (in terms of skills, data, and infrastructure) to enable creation and use of the largest and most capable of NLP models.