Run your own ChatGPT locally with Ollama! (Part 1)

Introduction

Ever since ChatGPT came out, it has thoroughly replaced how I scour the internet for information. A simple prompt and I save countless of hours/minutes clicking through links. However, there are a few issues with ChatGPT. One of them being the limited queries you have. Although now that the limits have been increased, and there are more available LLMs like Gemini, Meta AI, etc, you'll still eventually reach the limit on heavy usage. Secondly, privacy of your data. Pasting sensitive things like codes, personal information into ChatGPT might not exactly be the safest as well. Thus, to circumvent these issues, we can simply run our own "ChatGPT".

What exactly is ChatGPT?

For those that are new to the tech space, ChatGPT is essentially a Large Language Model (LLM). According to a LLM (yes, I prompted for the explanation), a LLM is a type of artificial intelligience designed to understand and generate human language. They are trained from a large amount of text data, where they learn grammar, context, meaning, etc. Afterwards, they can perform tasks like answering questions, writing essays, translating languages, etc. You can think of it as a very advanced text predictor. Pardon me if I butchered the essense of LLMs.

The ChatGPT that we see is essentially a wrapper around the LLM. Meaning to say, the underlying core/brain is a LLM, specifically one that is trained by OpenAI. Think of it as a whole package. The brain is the LLM, coupled with the User Interface (UI) the end product is ChatGPT. Of course, pardon me again if I butchered the soul of ChatGPT.

Where to find LLM?

Thankfully for us, as the race of AI progresses, many companies are churning out many LLMs. Each iteration of the model faster and better than the previous one. When big tech competes to see who is the best, all of us benefit. Trained LLMs can be downloaded on the internet. In fact, a good resource to checkout is Hugging Face , a platform in the machine learning community. Think of it as GitHub but for machine learning or a more layman analogy is a library that is filled with books (models) on the different machine learning techniques. These books (models) can solve different problems such as language translations, text analysis, image recognition, etc. As such, the BRAINS are everywhere!

How to run LLM locally?

And finally, the last ingredient, our secret sauce, Ollama! Ollama is a FREE and OPEN-SOURCE tool that allows you to run LLMs on your own computer! You just have to install and run a command! Isn't that amazing? The open-source community is truly one of the best!

Steps

Alright, now that the concepts are out of the way, let's put them all together!

1. Download and Install Ollama

Download the installer from here Ollama, choose the correct operating system as well. I.e. download the installer for Windows if you're on a Windows machine.

2. Run the following command in your terminal:

ollama run llama3.2

What happens is that ollama will download the LLM for you. In this instance, we are running Llama 3.2, a model from Meta. Fun fact, you can actually test out the model on WhatsApp! Once it has downloaded the model, it will start to run the model.

3. Start Chatting!

Your terminal will show "Send a message" on success. Start typing to converse with your LLM! To exit, press Ctrl + d

But wait...

Now you might be thinking, the terminal is a ugly interface! How is this ChatGPT!?

Well, first of all, the terminal is cool. Apologize!

Secondly, yes, you're right, we need a better UI!

And this can be done with another awesome FREE and OPEN-SOURCE tool, LibreChat. Did I mention that the Open-Source community is awesome yet? You can take a look at this guide to find out how to link LibreChat with Ollama!