Tabby

Last reviewed Aug 26, 2026 · 977 words

A used 12 GB RTX 3060, about 200 USD at last check, runs Tabby with a 1.5B-parameter completion model and a 7B chat model loaded side by side, and that is roughly the point where self-hosted completions stop feeling worse than GitHub Copilot at about 10 USD a month. Below that (an 8 GB card, or CPU only) you get a working but visibly slower assistant that suits a laptop on a plane more than daily work. Above it, a 16 GB or 24 GB card lets you run a 7B completion model too, and the gap to the cloud product closes for everything except the frontier chat models. Buy or borrow the GPU first; the software part takes 10 minutes.

The hardware question, with numbers

Tabby runs its models through llama.cpp and its registry ships 8-bit quantisations, so budget roughly 1 GB of VRAM per billion parameters plus about 1 GB for context, per model. Completion latency is what matters: a 1.5B model on an RTX 3060 returns a suggestion in well under 200 ms, which is the threshold below which inline completions feel native. A 7B completion model on the same card is nearer 500 ms, and you notice. On Apple Silicon the unified memory means a 32 GB MacBook runs both a 3B and a 7B model at acceptable speed with --device metal. AMD works via ROCm, and there is a Vulkan backend for cards neither vendor blesses. The catalogue's 8 GB system RAM minimum is for the server process and the model files; VRAM is the real constraint. The hardware for self-hosted LLMs post covers the second-hand card market if you are choosing.

Serve it: docker, one device flag, two model flags

docker run -d --name tabby --gpus all -p 8080:8080 \
  -v /srv/tabby:/data \
  tabbyml/tabby serve \
  --model Qwen2.5-Coder-1.5B \
  --chat-model Qwen2.5-Coder-7B-Instruct \
  --device cuda

First start downloads both models into /data, which is a few GB and the reason to give it a real volume. Open http://host:8080, create the first account (it becomes the admin), and install the editor extension: VS Code and JetBrains from their marketplaces, Vim and Neovim via the tabby-agent package, and there is a community Emacs mode. Point the extension at the URL, paste the token the web UI issues, and completions appear as ghost text. Put a reverse proxy with TLS in front if the server and the laptop are not on the same trusted network; the token travels in every request.

Completion and chat are different jobs and different models

The completion model is small on purpose. It fills in the middle of the line you are typing and has to answer in the time it takes you to pause, so 1B to 3B parameters is the sweet spot and going bigger buys little. The chat model handles the side panel where you ask it to explain a function or write a test, has time to think, and benefits from every parameter you can afford; 7B is the floor for useful answers and 14B or more if the card allows. "Completion quality model-dependent" in the cons column is the honest version of this: Tabby is a serving layer, and a 2023-era 1B model gives 2023-era completions. The Qwen2.5-Coder family is the current sane default at both sizes and the registry updates as better open models appear.

Repository context is the feature that beats Copilot on your own code

In the admin UI, under repository providers, add any git URL, including a Forgejo or Gitea instance on the LAN, and Tabby indexes the code. Completions then draw on your actual functions and types rather than generic patterns, and the chat panel can answer "where do we validate upload sizes" with a file and line. This is where a local assistant earns its keep: the cloud product cannot index a private monorepo the way a box sitting next to it can, and nothing leaves the network. Indexing a few hundred thousand lines takes minutes and re-runs on a schedule.

Tabby against Continue plus Ollama

The main alternative is the Continue extension pointed at Ollama, which already runs on many self-hosters' boxes. Continue is more flexible (any model, any provider, per-feature routing) and lives entirely in the editor with no server to administer. Tabby is the integrated product: one server, a web admin, user accounts, repository indexing, and an extension that needs a URL and a token. For one person on one machine who already runs Ollama, Continue is less to maintain. For a household or a small team wanting a shared, indexed assistant with usage visible in one place, Tabby wins clearly, and it uses less VRAM because two right-sized models beat one general one.

The server is Apache-2.0 and the Community tier covers up to 5 seats at last check, with paid tiers above that. For a homelab that is effectively unlimited. Accounts, SSO and per-user tokens are in the web UI, and usage stats show which extension is generating the most completions, which is a small thing that Ollama-based setups simply do not have.

What I'd do

An RTX 3060 12 GB or better in the box that already runs Ollama, Tabby in Docker with Qwen2.5-Coder-1.5B for completion and Qwen2.5-Coder-7B-Instruct for chat, the LAN Forgejo added as a repository, and the VS Code extension on every machine. Keep the Copilot subscription for a month while you compare, then cancel it. If you are one person on a laptop with no GPU anywhere, use Continue with Ollama and revisit when hardware arrives.