Run Ollama LLMs on Android¶
Ollama is an open source tool that allows you to run a wide range of Large Language Models (LLMs). This is great for the privacy conscious, with no input data being sent to the cloud. Meta's Llama 3.2 text only models feature three billion (3b) and one billion(1b) parameter models. Their small size makes them ideal for local operation on smartphones. Ollama can be run on Android, using Termux, which is a Unix/Linux based command line tool and utility.
Install Termux¶
Termux is available on the Google Play Store, and can be installed from there. Once installed, you now have access to a Linux command line shell.
Termux setup¶
Next you need to allow Termux storage access on your device, with the following command:
Update and upgrade Termux packages¶
The Termux package manager is called pkg
. The package repository needs to be updated and any installed programs needs to be upgraded before proceeding:
Install programs needed for Ollama¶
Three programs are essential for Ollama download and installation on your device - git, golang and cmake
.
Git will allow us to clone the ollama repository from Github; Ollama is written in the Go language; and CMake is needed to build ollama:
Download, install and build Ollama¶
Clone the Ollama respository¶
--depth 1
ensures that we clone just the most recent Ollama release, and not the whole repo.
The Ollama files should now be on your device. Move to the Ollama directory:
Build Ollama¶
Then generate the necessary Go code:
Then build ollama in the current directory: The dot (.) at the end refers to the current directory. Please be patient here, it may take some time to build.Start Ollama and run in the background¶
Find a LLM to run¶
Go to the Ollama website to view the Llama 3.2 models available:
Download and run Llama 3.2¶
Start (download and installation) with a smaller (1 billion parameter) model:
Currently, this one billion parameter model is 1.3GB in size.Once installed, the model will open with the following prompt:
Enjoy your testing! Your mileage in terms of speed of response will vary, and is directly related to the power of your device and the size of model you are attempting to run. In general the smaller models will run faster, but may not be as accurate as the bigger but slower models.To clear the shell of text in order to have an empty screen for the next question, use CTRL + D
Exit the current LLM session with the following command:
You can also move the Ollama executable binary file to your path, for global execution with the following command:
Please ensure you run this command in the ollama directory
Please ensure you are in the ollama directory or this command will not work.
You can then call Ollama directly from anywhere within Termux with ollama run <name of model>
I found the Llama3.2 one billion model to be fast and generally responsive on my Android test device.