Why large language models are inherently undemocratic
In early May, Meta released Open Pretrained Transformer (OPT-175B), a large language model (LLM) that can perform various tasks. Large language models have become one of the hottest areas of research in artificial intelligence in the past few years.
OPT-175B is the latest entrant in the LLM arms race triggered by OpenAI’s GPT-3, a deep neural network with 175 billion parameters. GPT-3 showed that LLMs can perform many tasks without undergoing extra training and only seeing a few examples (zero- or few-shot learning). Microsoft later integrated GPT-3 into several of its products, showing not only the scientific but also the commercial promises of LLMs.
What makes OPT-175B unique is Meta’s commitment to “openness,” as the model's name implies. Meta has made the model available to the public (with some caveats). It has also released a ton of details about the training and development process. In a post published on the Meta AI blog, the company described its release of OPT-175B as “Democratizing access to large-scale language models.”
Meta’s move toward transparency is commendable. However, the competition over large language models has reached a point where it can no longer be democratized.
For more on large language models: