Cut the costs of GPT-4 by up to 98%

GPT-4 is a very capable model. But it is also very expensive. Using it for real-world applications can quickly amount to thousands of dollars in API costs per month.

In a recent study, researchers at Stanford University introduce “FrugalGPT,” a set of techniques that can considerably reduce the costs of using LLM APIs while maintaining accuracy and quality.

Key findings:

• The price of LLM APIs vary widely across different models

• For many prompts, the smaller and cheaper models can perform just as well as the more complex LLMs

• The FrugalGPT paper proposes three strategies to optimize LLM API usage

• Prompt adaptation: Reduce the size of your prompt or bundle several prompts together

• Model approximation: Cache LLM responses or use model imitation to reduce the number of API calls to large models

• LLM cascade: Create a list of LLM APIs from small to large; use the smallest model that can provide an acceptable answer to the user’s prompt

• FrugalGPT, an implementation of the cascade model, resulted in orders of magnitude cost reduction and even improved accuracy

Read the full article on TechTalks.

For more on AI research:

• Why you should be careful about LLMs that imitate ChatGPT

• Are the emergent abilities of LLMs like GPT-4 a mirage?

• To understand language models, we must separate “language” from “thought”

• The power of wide transformers models