Footprints: I put a language model in an APK and deleted the internet

Every chat app you have talks to a datacentre. You type, it goes over the wire, something enormous thinks about it, and the answer comes back. That's a reasonable design — the models genuinely are too big for a phone — but it means every half-formed thought you type is somebody else's log line.

I wanted to know what the opposite extreme felt like. Not "privacy-respecting cloud AI". Not "we anonymise your prompts". A model that is physically incapable of transmitting anything, because the app it lives in has no network permission at all.

Footprints is that experiment. It's a chat app with Qwen 2.5 0.5B Instruct baked into the APK, running through llama.cpp on the CPU.

The numbers, honestly

I'd rather lead with the uncomfortable ones. Measured on a moto g57 power, arm64-v8a:

A half-gigabyte app that takes the better part of a minute to start is not something I'd defend as a mass-market product. On a Snapdragon 8 Gen 2 or 3 you'll see 15–20+ tok/s and it feels considerably better, but the cold start is structural.

What you get for it: an assistant that works in a tunnel, on a plane, in a village with no signal, with no account, no subscription, and no possibility of a prompt leaking. For a certain set of situations that's not a worse trade — it's the only acceptable one.

Why 0.5B, and not something impressive

This is the question everyone asks, so let me answer it properly.

A 7B model at Q4 wants 4–8 GB of RAM and delivers 2–5 tok/s on a phone. That's slower than you can read, and on most devices the OS kills you before you finish a paragraph. A 0.5B model fits in roughly 500 MB and runs 3–5× faster. It is, as far as I can tell, the smallest model that still holds a coherent conversation — and therefore the largest one that's actually usable in real time on hardware people already own.

Qwen 2.5 0.5B specifically because it's the strongest instruction-follower I found at that size, and it carries a full 32K context, which matters more than raw quality for a chat app where the conversation is the state.

Is it GPT-4? Obviously not. GPT-4 is somewhere around 6,000× larger. Footprints will not write your dissertation. It summarises, rephrases, drafts, brainstorms and answers ordinary questions, and it does that in your hand with the radio off.

How the pieces fit

The stack is deliberately shallow — eight Kotlin files, about 1,255 lines total:

Compose UI  →  ChatViewModel (StateFlow)  →  LlmEngine  →  JNI  →  llama.cpp  →  Qwen 2.5

The part I like most is the streaming. sendUserPrompt() returns a Flow<String>, the ViewModel collects tokens one at a time and pushes each into a MutableStateFlow, and Compose re-renders as they arrive. You watch the answer being written rather than waiting for a block of text to appear. On a slow model that difference is not cosmetic — it's the whole perceived-performance story. 12 tok/s that you can see is far more tolerable than 12 tok/s that you wait for in silence.

There's no DI framework. Manual injection through AndroidViewModel and an LlmEngine singleton, so the dependency graph is something you can hold in your head. One activity, one screen, no navigation library. For a project this size, every abstraction I skipped was one less thing between me and the bug.

Why CPU and not GPU

llama.cpp can use Vulkan or OpenCL. I stayed on CPU because GPU driver quality across Android OEMs is a lottery, and a model that runs on every arm64 device is worth more to me than one that's faster on a third of them and mysteriously broken on the rest. That may change as drivers improve; for now, universal beat fast.

Why the model is inside the APK

This is the 469 MB decision. I could ship a 40 MB app that downloads the model on first launch, and most projects do. But then first launch requires internet, which requires the INTERNET permission, which destroys the one property the whole project exists to demonstrate. Bundling the model is what lets the permission list stay empty. The APK size is the price of the guarantee.

What's actually in it

Beyond the chat loop it grew the things you inevitably want: conversation history persisted as JSON in app-internal storage, multiple conversations you can switch between and delete, system prompt presets (default, creative, professional, concise, or your own), markdown rendering for bold, italics, inline code and code blocks, export a conversation to Downloads, voice input via the platform SpeechRecognizer, and a dark/light toggle that overrides the system theme.

All of it local. The export writes a file; it does not upload one.

Things I'd do differently

The 50-second cold start is the real problem. Most of it is copying the model out of APK assets to a real path on first run, then llama.cpp loading it. Memory-mapping an uncompressed asset directly would remove the copy entirely, and it's the first thing I'd fix.

There are no tests yet. The dependencies are declared in build.gradle.kts and the ViewModel is the obvious first target. I'm not going to pretend that's fine — it's a gap, and it's next.

The model should be swappable. Bundling one model is what makes the offline guarantee airtight, but it also means you're stuck at 0.5B. An in-app picker that downloads 1.5B or 3B would be genuinely useful for people on better hardware — at the cost of needing the network permission, which reopens the exact door I welded shut. I haven't found a way to have both, and I'd rather keep the guarantee.

Was it worth building

As a product, it's rough. As a demonstration, I think it's the most convincing thing I've made: proof that a genuinely useful language model can live entirely inside an app that has no way to talk to anyone.

Every step you take online leaves a footprint somewhere. This one doesn't leave the device.