Create your own local LLM that interacts with your docs

ChatGPT is a convenient tool, but it has downsides such as privacy concerns and reliance on internet connectivity. An alternative is to create your own private large language model (LLM) that interacts with your local documents, providing control over data and privacy.

In my latest article, I explore the key pieces and workflows of a private ChatGPT that runs on your own machine.

What you need: An open-source LLM, an embedding model, a store of documents, a vector database, and a user interface

• How it works: The user’s prompt is augmented with documents from the knowledge base before being sent to the LLM

• How to install it: You can assemble your own components or use PrivateGPT, a project that brings everything together as an installable package

Read the full article on TechTalks.

For more articles:

• How to customize LLMs like ChatGPT with your own data and documents

• What is low-rank adaptation (LoRA)?

• A look at open-source alternatives to ChatGPT

• What to know about augmented language models