This is going to be a very short article on how to run Ollama models like recently released llama3
or llama2
or even other models like mistral
.
Recently I was looking for a way to play around with llama2
model when I saw the announcement by Mark Zuckerberg about the release of llama3
. So I scoured the web and one github repo caught my eye.
And I’m going to show how we can use the llama3
and other models in local using Docker.
INSTALL DOCKER
So the first step for this is to install Docker Engine on your machine. – which ever it is – Mac/Windows/Linux.
There are a hundred million tutorials on how to do it – so I’m not going to go into it. I’ll leave a couple of good links for the same, which anyone who’s reading this can follow
- Install docker on MacOS – Link
- Install docker on Ubuntu Linux – Link , Link
- Install docker on Windows – Link
Or you can follow this link for a comprehensive tutorial for all three platforms.
INSTALL OLLAMA
Once you’ve done the docker installation, you can now install Ollama on your system. It’s very easy process to do so.
Head over to Ollama website, and click on the Download button to download it. You can use between the Mac/Linux/Windows variants.
Once you’ve installed Ollama ( in my case on a Mac machine), just type this to run the latest llama3
model
ollama run llama3
To stop and exit from the prompt screen type /bye
USING A GUI
Now this is a good and easy way to use and run llama3
model, but having a GUI makes it much more easier. And also , if we can seamlessly switch in between different models, that would be great.
That’s where this Github repo came to my rescue – https://github.com/open-webui/open-webui
So this can be used to have a web interface ( instead of using terminal) to interact with llama3
model and also you can pull other models that are available in Ollama
to seamlessly change or switch in between models like mistral
or codegemma
.
The installation is mentioned in the readme – but I’ll note it down anyways – if you’ve docker, you can run this command
docker run -d -p 3000:8080 --add-host=host.docker.internal:host-gateway -v open-webui:/app/backend/data --name open-webui --restart always ghcr.io/open-webui/open-webui:main
This will use Ollama on your local machine and pull an image of llama3
and once the installation is done, simply open the http://localhost:3000
to see the UI.
It might ask you to register before allowing you to use it. So do it and then you’ll see this screen
At the starting, there will only be a single model llama3
by default , which you can use to provide prompts.
ADDING MORE MODELS
To add more models, click on the three dots (...)
at the top right hand side of the screen
Click on Settings
From the left side menu, click on Models
and then see there is a field name Pull a model from Ollama.com
.
In this field, you need to enter a model name – for e.g – mistral
for the latest mistral
model. The eligible list is available here
Click on the download icon and then the model image will be pulled. Once this is done, refresh the page and then click on the +
icon in the prompt window to select the newly added model
CUSTOM MODELS
You can also add custom models that community members have created by clicking on the Modelfiles
from the left side menu. I will add another article about how to create this and use this.
So as you can see, with the help of docker and docker images of Ollama
, you can use multiple LLM models in your local system.
1.https://lmstudio.ai/
2.https://useanything.com/
These also trying to standarize the UI..
LikeLike