llamafile

Last reviewed Aug 26, 2026 · 800 words

Download one file, chmod +x it, run it, and a language model is answering in your browser with an OpenAI-compatible API on port 8080. No package manager, no Python, no driver dance, and the identical file runs on Linux, macOS, Windows, FreeBSD, OpenBSD, and NetBSD. That is the whole pitch of llamafile, Mozilla's packaging of llama.cpp into Cosmopolitan "actually portable executables", and for the thing it does there is nothing simpler. Where it fits next to Ollama, and where it does not, is the rest of this page.

The Windows 4 GB catch

Windows cannot execute a file larger than 4 GB, and any llamafile with a model bigger than a small quantised 3B baked in exceeds that. The workaround is built in: download the bare llamafile executable, which is a few tens of megabytes, keep the weights as a separate GGUF file, and run llamafile.exe -m model.gguf. You lose the one-file elegance on that platform and keep everything else. On Linux and macOS the combined files work as advertised, which is why the catalogue lists file size as the only con: a 7B Q4 model is around 4 to 5 GB, and a 70B one is 40 GB of executable sitting in your downloads folder.

Serving it on the LAN is two flags

By default the binary opens a browser tab with a chat page. For a headless box:

./Meta-Llama-3.1-8B-Instruct.Q4_K_M.llamafile --server --nobrowser \
  --host 0.0.0.0 --port 8080 -ngl 999

-ngl 999 offloads every layer it can to a GPU; on a box without one, drop it and the model runs on CPU, where llama.cpp's tinyBLAS kernels give respectable speed on a modern desktop for models up to about 8B. The server exposes /v1/chat/completions, so Open WebUI or any OpenAI-client library talks to it by changing a base URL. Put it in a systemd unit and it is a service. The 8 GB in our catalogue is the practical minimum for a 7B to 8B model in 4-bit quantisation with room for the OS; a 3B model runs in half that.

Where it beats Ollama, and where it loses

Ollama is a model manager: it pulls, stores, and swaps models, keeps them loaded on demand, and has a registry with a pull command. llamafile is a model: one file, one weight set, no library, no daemon. That makes llamafile the right shape for handing a colleague a working assistant on a USB stick, for an air-gapped machine, for a demo that must not depend on a download, or for pinning exactly one model on a small server. It is the wrong shape when you swap between five models a day, because each is a separate multi-gigabyte executable and nothing coordinates them. The llama.cpp versus Ollama comparison covers the same trade at the layer below; llamafile is llama.cpp with the packaging problem solved and the management problem ignored.

It tracks llama.cpp, with a lag

Because it is a build of llama.cpp with a portable runtime, llamafile inherits that project's model support and speed, and it also inherits its own release cadence, which is slower. A new model architecture that llama.cpp supports on Monday may take weeks to appear in a llamafile release. If you chase the newest weights, run llama.cpp or Ollama directly. If you want something that worked last month to keep working unchanged, the lag is a feature. Building your own llamafile from any GGUF is documented and takes one command with the llamafile tool and a .args file, so being stuck waiting for an official bundle of a particular model is rare.

The limits, stated plainly

It is not a chat product; the built-in web page is functional and plain. It is not a multi-model server, not a fine-tuning tool, and not faster than a well-configured llama.cpp on the same hardware, since it is the same code. The LLM runners category lays out the alternatives if any of those matter.

What I'd do

For a single-purpose box, a portable demo, or an offline machine: one llamafile, served with the flags above under systemd, Open WebUI in front if anyone other than you will use it. For a homelab where models change weekly: Ollama, and keep a llamafile or two on a drive for the day the network is down. It is the most reliable way I know to make a language model run somewhere unfamiliar in under 5 minutes, and that is worth a place in the toolkit even if it is not the daily driver.