What we know about Yann LeCun vision for the future of AI

One of the most accomplished AI scientists is departing his long-time role at Meta. What do we know about Yann LeCun's vision for the future of AI?

Last week, news broke that Yann LeCun, Turin Award winner and one of the pioneers of modern artificial intelligence, is leaving his role as Meta’s chief AI scientist by the end of the year. LeCun will be starting a new AI startup with details to come later.

There were already rumors of LeCun being sidelined after Meta CEO Mark Zuckerberg assembled his new Superintelligence Lab, headed by Alexandr Wang, co-founder and former CEO of Scale AI. However, LeCun had already made clear long before his departure that he was not satisfied with the direction that the AI community is headed.

While most efforts are going into large language models (LLMs), LeCun has been very vocal about their limitations, particularly in their ability to solve real-world problems.

“We are not going to get to human-level AI just by scaling LLMs,” LeCun told Alex Kantrowitz’s Big Technology podcast in May, calling LLMs systems with gigantic memory and retrieval ability, “not a system that can invent solutions to new problems.”

But which direction will LeCun be going after his departure from Meta? We already know some of the areas that he is interested in and that the industry has mostly ignored.

In a post on LinkedIn in which he confirmed his departure, LeCun wrote: “I am creating a startup company to continue the Advanced Machine Intelligence research program (AMI) I have been pursuing over the last several years with colleagues at FAIR, at NYU, and beyond. The goal of the startup is to bring about the next big revolution in AI: systems that understand the physical world, have persistent memory, can reason, and can plan complex action sequences.”

LeCun has been consistent on how he believes we will achieve these goals. He has been a long-time advocate of self-supervised learning, and in recent years, he has been working on “world models” that can be trained through self-supervised learning.

In short, self-supervised learning is a process in which the model trains itself without the need for data labeled by humans. LeCun has been talking about self-supervised learning for more than a decade.

“If artificial intelligence is a cake, self-supervised learning is the bulk of the cake,” LeCun said at the AAAI conference in 2020. “The next revolution in AI will not be supervised, nor purely reinforced.”

LLMs do some form of self-supervised learning, where they’re given a sequence of text tokens (or other type of data) and told to predict the next token. (Since the next token is already present in the training corpus, there is no need for manual labeling, thus it is considered some form of self-supervised learning.)

However, LeCun (as well as other scientists such as Richard Sutton, another Turing Award winner), argue that LLMs don’t learn in the same way that humans do. In particular, LLMs have only been trained on data generated by humans (i.e., text) and can’t generalize to the unpredictability of the real world. They do not learn how the world works from sensory data like children do by observing and interacting with the world, discovering and internalizing things such as gravity, depth, object permanency, etc.

So they cannot predict consequences and counterfactuals the same way that humans or even animals do. And next-token prediction does not seem to generalize well to the unpredictable nature of the real world, which is partly evident in how much training data they need to learn basic tasks (while still failing to generalize beyond their training distribution).

So, if not LLMs, then what?