Why you probably don't need your own LLM

With the hype surrounding large language models (LLM), many companies are asking whether they should have their own LLM or their own ChatGPT.

Probably not, says Jason Ly, co-founder and head of engineering at SpringBok AI, in an opinion article on TechTalks.

“If your goal is to create new products or cut costs by automating processes, you don’t need your own LLM. Even the traditional data science practice of taking an existing model and fine-tuning it is likely to be impractical for most businesses,” Ly writes. “Instead, consider what I call prompt architecting as an alternative that lets you borrow the power of an LLM, but allows you to fully control the chatbot’s processes, check for factual correctness, and keep everything on-brand.”

Key findings:

• Creating your own LLM from scratch requires millions of dollars in upfront investment

• Fine-tuning a foundation model with your own data can be cheaper, but is still complicated

• Use fine-tuning if you have strict data/privacy requirements and a hefty budget to continuously train and test your models

• “Prompt architecting” is similar to prompt engineering, but with a key difference: “Instead of engineering individual prompts that achieve a single goal, we create entire pieces of software that chain, combine, and even generate tens, if not hundreds, of prompts, on the fly to achieve a desired outcome”

• The architecture will be specialized for each problem, but it involves a few key steps:

  • Get user prompt

  • Embellish prompt with context about the user and the chatbot

  • Send prompt to an off-the-shelf LLM (ChatGPT, Claude, etc.)

  • Use verification techniques to automatically check output for errors

  • Send corrected answer to user

• Ly has already used prompt architecting for several clients. “The novel approach of 'prompt architecting', combining off-the-shelf LLMs with cleverly designed software, offers a more practical, cost-effective solution for most enterprises,” he writes

Read the full article on TechTalks.

For more on LLMs:

• How to write better code with ChatGPT

• The complete guide to LLM fine-tuning

• How to reduce the costs of using ChatGPT and GPT-4

• Why you should be careful about LLMs that imitate ChatGPT