How to Train a Custom AI Chatbot Using PrivateGPT Locally (Offline)

In an earlier tutorial, we demonstrated how you cantrain a custom AI chatbot using ChatGPTAPI. While it works quite well, we know that once your free OpenAI credit is exhausted, you need to pay for the API, which is not affordable for everyone. In addition, several users are not comfortable sharing confidential data with OpenAI. So if you want to create a private AI chatbot without connecting to the internet or paying any money for API access, this guide is for you. PrivateGPT is a new open-source project that lets you interact with your documents privately in an AI chatbot interface. To find out more, let’s learn how to train a custom AI chatbot using PrivateGPT locally.

Requirements to Set up PrivateGPT on Your PC

1. To runPrivateGPTlocally on your machine, youneed a moderate to high-end machine. You can’t run it on older laptops/ desktops. To give you a brief idea, I tested PrivateGPT on an entry-level desktop PC with an Intel 10th-gen i3 processor, and it took close to 2 minutes to respond to queries. Keep in mind, PrivateGPT does not use the GPU. Currently, it only relies on the CPU, which makes the performance even worse. Nevertheless, if you want to test the project, you can surely go ahead and check it out.

2. PrivateGPTcan be used offline without connecting to any online serversor adding any API keys from OpenAI or Pinecone. To facilitate this, itruns an LLM model locallyon your computer. So, you will have to download a GPT4All-J-compatible LLM model on your computer. I have added detailed steps below for you to follow.

Set Up the Environment to Train a Private AI Chatbot

1. First, you need to installPython 3.10 or lateron your Windows, macOS, or Linux computer. You can click onthis linkto download Python right away.

2. Next, run the setup file and make sure to enable the checkbox for “Add Python.exe to PATH.” After that, click on “Install Now” and follow the usual steps to install Python.

3. Next, you will need toinstall Visual Studio 2022if you are using Windows. This is done to get theC++ CMake tool and UWPcomponents. Click onthis linkand download the “Community” version for free.

4. Now, run the setup, and it will download another installer. A window will appear where you can choose the components. Scroll down and select “Desktop Development with C++” and “Universal Windows Platform development.”

5. Next, click on the “Install” button at the bottom right corner. After the installation, restart your computer. You don’t need to use Visual Studio thereafter, but keep it installed.

6. Finally, go ahead anddownload the default model(“groovy”) fromhere. The size is around 3.5GB. You can download other models fromthis linkif you have a more powerful computer.

How to Set Up PrivateGPT on Your PC Locally

1. First, open theGitHub linkof the privateGPT repository and click on “Code” on the right. Here, click on “Download ZIP.”

2. Now, open the ZIP file andextract the foldernamed “privateGPT-main”.

3. Next, open the “privateGPT-main” folder and create a new folder named “models“.

4. Under the “models” folder,paste the “groovy” modelyou downloaded above.

5. Now, go back to the main folder, and you will find an “example.env” file. Rename it as just the “.env” file.

6. Now, right-click on the “.env” file andopen it withNotepad.

7. If you have downloaded adifferent model, you can define it under “MODEL_PATH”. Since we are using the default model, no change is needed. You can close Notepad now.

8. Next, open the “source_documents” folder. Here, you can add all kinds of documents to train the custom AI chatbot. As an example, the developer has added a transcript of theState of the Union addressin TXT format. However, you can also add PDF, DOC, DOCX, CSV, EPUB, TXT, PPT, PPTX, ODT, MSG, MD, HTML, EML, and ENEX files here.

Train a Custom AI Chatbot Using PrivateGPT

1. Finally, it’s time to train a custom AI chatbot using PrivateGPT.Open Terminalon your computer. If you are using Windows, open Windows Terminal or Command Prompt.

2. Now, right-click on the “privateGPT-main” folder and choose “Copy as path“. This will copy the path of the folder.

3. Now, move back to the Terminal and typecd, add a space, andpaste the pathby right-clicking in the Terminal window. It will look something like below.

4. Next, hit Enter, and you will move to theprivateGPT-mainfolder.

5. Once you are in the folder, run the below command, and it will start installing all the packages and dependencies. It mighttake 10 to 15 minutesto complete the process, so please keep patience. If you get any error, run the below command again and make sure Visual Studio is correctly installed along with the two components mentioned above.

6. Once all the dependencies are installed, run the below command to create local embeddings and vectorstore. This process will take a few secondsdepending on the corpus of dataadded to “source_documents.” macOS and Linux users may have to usepython3instead ofpythonin the command below.

7. Finally, run PrivateGPT by executing the below command. You will be asked toenter your query. Enter your question and hit Enter.

8. On my Intel 10th-gen i3-powered desktop PC, it tookclose to 2 minutesto answer a query. After every answer, it will also display four sources from where it has got the context.

9. To stop the conversation, typeexitand hit Enter.

You can alsoturn off the internet, but the private AI chatbot will still work since everything is being done locally. PrivateGPT does not have a web interface yet, so you will have to use it in the command-line interface for now. Also, it currently does not take advantage of the GPU, which is a bummer. Once GPU support is introduced, the performance will get much better. Finally, to load up the PrivateGPT AI chatbot, simply runpython privateGPT.pyif you have not added new documents to the source folder.

Arjun Sha

Passionate about Windows, ChromeOS, Android, security and privacy issues. Have a penchant to solve everyday computing problems.

Add new comment

Name

Email ID

Δ

01

02

03

04

05