Android

A few months ago I deployed an LLM on an old smartphone. At the time I said that I was going to follow up later to share my experiences with various models and use cases and so here I am. What I didn’t know back in June is that decision models would appear. They represent a fantastic fit for a frugal phone like mine, and a good first use case to share.

This post continues from AI llama-server on a phone with a detailed walkthrough of every step needed to flash LineageOS, set up Termux, build llama.cpp, and serve a small language model over your local network.

Yesterday I made an AI server run on my broken 6 year old phone. In the same week as GitHub moved to usage based pricing and a week after we learnt that a company accidentally spent $500 million on Claude AI in one month alone, I dug out a discard piece of technology to host an LLM.