LlamaGPT

Last reviewed Aug 26, 2026 · 861 words

LlamaGPT is a 2023 project, and in 2026 that is the most important thing about it. It still installs in 1 click on Umbrel and still gives you a ChatGPT-style chat window that never sends a byte off your machine, but its model menu stops at Llama 2 and Code Llama, and the repository has barely moved since its first year. If you are on Umbrel and want a private chatbot for an afternoon, install it. For anything you intend to keep using, run Ollama with Open WebUI, which is the same idea with current models and active maintenance.

What it actually is: llama.cpp with a chat front-end

Under the compose file there are 2 containers. One runs an OpenAI-compatible API server built on llama.cpp and listens on port 3001. The other is a fork of the open-source chatbot-ui project, listening on port 3000, pointed at the first. ./run.sh --model 7b pulls the quantised model on first start and brings both up; --with-cuda switches to a GPU build if you have an Nvidia card and the container toolkit installed. The model list is Llama 2 at 7B, 13B, and 70B, plus Code Llama at 7B, 13B, and 34B.

The OpenAI-compatible endpoint was the quietly useful part. Anything that speaks the OpenAI chat API can be pointed at http://your-box:3001/v1 and works offline, which in 2023 was not a given and today is table stakes for every local runner.

The RAM floor is 8 GB, and the speed is CPU speed

The catalogue's 8 GB minimum is for the 7B model with a quantised weight file that occupies roughly 4 GB on disk and more in memory once context is allocated. Plan on about 16 GB for 13B and north of 40 GB for 70B, and understand that on CPU those figures buy you speed measured in a few tokens per second for 7B and a crawl for anything larger. That is an estimate, and it varies with your CPU's memory bandwidth more than its clock speed.

GPU acceleration exists only on the path the run script knows about: Nvidia on Linux with CUDA. Docker Desktop on a Mac cannot pass Apple Silicon's GPU into a container, so an M-series machine runs LlamaGPT on CPU while llama.cpp run natively on the same Mac would use Metal and be several times faster. That gap is the practical reason the project lost its audience.

Why it stopped being the answer

Model releases since late 2023 have left Llama 2 far behind: the Llama 3 family, Qwen, Gemma, Mistral, and Phi models at the same parameter counts are better at instruction following, longer in context, and in several cases smaller for the same quality. LlamaGPT's run script and model registry do not know any of them. You can hand-edit the compose file to point at a different weight file, and people did, but at that point you are operating llama.cpp yourself behind a stale UI. The UI itself has no RAG over your documents, no web search, no multi-user accounts, no model switching without a restart, and no tool calling, all of which the AI chat UIs category now takes for granted.

Where it still earns a slot

Three cases. Umbrel users who want the app-store experience and a private chatbot in 1 click, with no terminal. A demo box for a talk about local AI, where "no network cable, still answers" is the point and the answer quality is not. And an air-gapped machine where the 2023 Llama 2 weights are what you already have on disk. In all 3, LlamaGPT does what it says: complete privacy, simple setup, and a familiar interface.

The replacement takes 10 minutes and the same hardware

Ollama installs with 1 command, runs on CPU, Nvidia, AMD, and Apple Silicon natively, and pulls current models by name. Open WebUI in a container gives you a chat interface that is better than LlamaGPT's in every respect: multiple users, document upload and retrieval, model switching per conversation, and an OpenAI-compatible endpoint of its own. The 8 GB machine that ran Llama 2 7B runs a current 7B or 8B model at the same speed with visibly better output. Hardware questions, including whether a used GPU is worth it, are covered in the hardware for self-hosted LLMs piece.

What I'd do

On Umbrel with no interest in the terminal: install LlamaGPT, enjoy it for a week, and then install the Ollama and Open WebUI apps from the same store, because they are there now too. Anywhere else: skip it. Ollama plus Open WebUI on the same 8 GB box, a current small model, and 10 minutes of setup gets you a private assistant that will still be maintained next year. LlamaGPT deserves credit for showing in 2023 that offline chat could be a 1-click install; the credit is the reason to remember it, not the reason to run it.