In a new paper, researchers at OpenAI have revealed details about Codex, a deep learning model that generates software source code.
The paper is a fascinating read that explains the process through which the scientists at OpenAI managed to repurpose their flagship language model GPT-3 to create Codex. But more importantly, the paper also sheds much-needed light on how far you can trust deep learning in programming.
In my analysis of the paper, I discuss:
The “no free lunch” theorem and its application to large language models
The tradeoffs between size and cost in language models, and how it affects the business model of the products you build on top of them
The limits of deep learning models in understanding the source code they generate
What is the responsible use of models such as Codex
For more on AI and programming: