The limits of large language models

In a new paper, researchers at OpenAI have revealed details about Codex, a deep learning model that generates software source code.

The paper is a fascinating read that explains the process through which the scientists at OpenAI managed to repurpose their flagship language model GPT-3 to create Codex. But more importantly, the paper also sheds much-needed light on how far you can trust deep learning in programming.

In my analysis of the paper, I discuss:

  • The “no free lunch” theorem and its application to large language models

  • The tradeoffs between size and cost in language models, and how it affects the business model of the products you build on top of them

  • The limits of deep learning models in understanding the source code they generate

  • What is the responsible use of models such as Codex

Read the full article on TechTalks.

For more on AI and programming: