Going from behavior cloning to thought cloning in AI

Artificial intelligence systems are becoming increasingly good at cloning some aspects of human behavior. But without backing actions with thought, AI systems can become brittle and make unpredictable mistakes when faced with novel situations.

A new technique called “Thought Cloning” promises to make progress on this problem by training AI systems to think as well as behave like humans.

Thought cloning can enable deep learning models to generate a sort of reasoning process for their actions and convey that reasoning to human operators.

Key findings:

• Models trained only on behavior data don’t learn the reasoning behind actions

• The idea behind thought cloning is to train the model on multiple streams of data: behavior and thoughts

• Thought cloning enables models to learn with fewer examples because the see actions and thoughts at the same time

• By learning the reasoning of actions, TC-trained models are much better than behavior-based models in handling edge cases and out-of-distribution examples

• TC also makes models interpretable by enabling them to produce reasoning at the same time as actions

• One of the benefits of TC is safety—ML engineers can create safeguards to prevent harmful behavior based on the thought process of the model

• One of the biggest challenges of TC is gathering annotated training data (humans rarely record their thought processes when performing actions)

Read the full article on TechTalks.

For more on AI research:

• Why you should be careful about LLMs that imitate ChatGPT

• ChatGPT will make the web toxic for its successors

• Are the emergent abilities of LLMs like GPT-4 a mirage?

• Can you trust ChatGPT and other LLMs in math?