This is going to be a very short article on how to run Ollama models like recently released llama3 or llama2 or even other models like mistral.

Recently I was looking for a way to play around with llama2 model when I saw the announcement by Mark Zuckerberg about the release of llama3. So I scoured the web and one github repo caught my eye.

And I’m going to show how we can use the llama3 and other models in local using Docker.

INSTALL DOCKER

So the first step for this is to install Docker Engine on your machine. – which ever it is – Mac/Windows/Linux.

There are a hundred million tutorials on how to do it – so I’m not going to go into it. I’ll leave a couple of good links for the same, which anyone who’s reading this can follow

  • Install docker on MacOS – Link
  • Install docker on Ubuntu Linux – Link , Link
  • Install docker on Windows – Link

Or you can follow this link for a comprehensive tutorial for all three platforms.

INSTALL OLLAMA

Once you’ve done the docker installation, you can now install Ollama on your system. It’s very easy process to do so.

Head over to Ollama website, and click on the Download button to download it. You can use between the Mac/Linux/Windows variants.

Once you’ve installed Ollama ( in my case on a Mac machine), just type this to run the latest llama3 model

ollama run llama3

To stop and exit from the prompt screen type /bye

USING A GUI

Now this is a good and easy way to use and run llama3 model, but having a GUI makes it much more easier. And also , if we can seamlessly switch in between different models, that would be great.

That’s where this Github repo came to my rescue – https://github.com/open-webui/open-webui

So this can be used to have a web interface ( instead of using terminal) to interact with llama3 model and also you can pull other models that are available in Ollama to seamlessly change or switch in between models like mistral or codegemma.

The installation is mentioned in the readme – but I’ll note it down anyways – if you’ve docker, you can run this command

docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main

This will use Ollama on your local machine and pull an image of llama3 and once the installation is done, simply open the http://localhost:3000 to see the UI.

It might ask you to register before allowing you to use it. So do it and then you’ll see this screen

At the starting, there will only be a single model llama3 by default , which you can use to provide prompts.

ADDING MORE MODELS

To add more models, click on the three dots (...) at the top right hand side of the screen

Click on Settings

From the left side menu, click on Models and then see there is a field name Pull a model from Ollama.com .

In this field, you need to enter a model name – for e.g – mistral for the latest mistral model. The eligible list is available here

Click on the download icon and then the model image will be pulled. Once this is done, refresh the page and then click on the + icon in the prompt window to select the newly added model

CUSTOM MODELS

You can also add custom models that community members have created by clicking on the Modelfiles from the left side menu. I will add another article about how to create this and use this.

So as you can see, with the help of docker and docker images of Ollama, you can use multiple LLM models in your local system.