Skip to content

Run Ollama LLMs on Android

Ollama is an open source tool that allows you to run a wide range of Large Language Models (LLMs). This is great for the privacy conscious, with no input data being sent to the cloud. Meta's Llama 3.2 text only models feature three billion (3b) and one billion(1b) parameter models. Their small size makes them ideal for local operation on smartphones. Ollama can be run on Android, using Termux, which is a Unix/Linux based command line tool and utility.

Install Termux

Termux is available on the Google Play Store, and can be installed from there. Once installed, you now have access to a Linux command line shell.

Termux setup

Next you need to allow Termux storage access on your device, with the following command:

termux-setup-storage

Update and upgrade Termux packages

The Termux package manager is called pkg. The package repository needs to be updated and any installed programs needs to be upgraded before proceeding:

pkg update 
pkg upgrade

Install programs needed for Ollama

Three programs are essential for Ollama download and installation on your device - git, golang and cmake.

Git will allow us to clone the ollama repository from Github; Ollama is written in the Go language; and CMake is needed to build ollama:

pkg install git golang cmake

Download, install and build Ollama

Clone the Ollama respository

 git clone --depth 1 https://github.com/ollama/ollama.git
--depth 1 ensures that we clone just the most recent Ollama release, and not the whole repo.

The Ollama files should now be on your device. Move to the Ollama directory:

cd ollama

Build Ollama

Then generate the necessary Go code:

 go generate ./...
Then build ollama in the current directory:
go build .
The dot (.) at the end refers to the current directory. Please be patient here, it may take some time to build.

Start Ollama and run in the background

./ollama serve &

Find a LLM to run

Go to the Ollama website to view the Llama 3.2 models available:

https://ollama.com/library/llama3.2

Download and run Llama 3.2

Start (download and installation) with a smaller (1 billion parameter) model:

./ollama run llama3.2:1b
Currently, this one billion parameter model is 1.3GB in size.

Once installed, the model will open with the following prompt:

>>> Send a message (/? for help)
Enjoy your testing! Your mileage in terms of speed of response will vary, and is directly related to the power of your device and the size of model you are attempting to run. In general the smaller models will run faster, but may not be as accurate as the bigger but slower models.

To clear the shell of text in order to have an empty screen for the next question, use CTRL + D

Exit the current LLM session with the following command:

/bye

You can also move the Ollama executable binary file to your path, for global execution with the following command:

cp ollama /data/data/com.termux/files/usr/bin/

Please ensure you run this command in the ollama directory

Please ensure you are in the ollama directory or this command will not work. You can then call Ollama directly from anywhere within Termux with ollama run <name of model>

I found the Llama3.2 one billion model to be fast and generally responsive on my Android test device. 😄