Continue

Last reviewed Aug 26, 2026 · 792 words

Continue running against a local model is a completely different experience depending on one setting: which model handles autocomplete. Give it the same 7B chat model you use for questions and every keystroke waits 2 seconds for a suggestion that arrives after you have moved on. Give it a 1.5B fill-in-the-middle model and suggestions land in well under a second on a laptop with no GPU at all. Getting that split right is most of the setup; the rest is deciding where the models run.

Three jobs, three models

Continue is an extension for VS Code and JetBrains, not a server, so a self-hoster's question is what it talks to. It has three roles that want different models. Chat and edit want the largest model you can run at reading speed: qwen2.5-coder:7b on a 16 GB machine, qwen2.5-coder:14b or a 32B model on a 24 GB GPU. Autocomplete wants a small model trained for fill-in-the-middle, and qwen2.5-coder:1.5b-base is the one I would start with. Embeddings, which power the codebase index that lets you ask questions about your own repository, want nomic-embed-text. All three come from Ollama with one ollama pull each.

The config.yaml that works

Continue reads ~/.continue/config.yaml (older installs used config.json, and the extension can migrate it). A working local setup:

name: Local
version: 0.0.1
schema: v1
models:
  - name: Qwen chat
    provider: ollama
    model: qwen2.5-coder:7b
    roles: [chat, edit, apply]
  - name: Qwen autocomplete
    provider: ollama
    model: qwen2.5-coder:1.5b-base
    roles: [autocomplete]
  - name: Nomic embed
    provider: ollama
    model: nomic-embed-text
    roles: [embed]

Ollama defaults to http://localhost:11434, so no apiBase is needed when it runs on the same machine. The "config takes tweaking" caveat in the listing is fair: the first hour goes on context length (Ollama's default window is small for chat over whole files, so raise it per model), and on the autocomplete debounce and max tokens if suggestions feel late or long.

Put the models on the box with the GPU

The extension runs on your laptop; the models do not have to. Run Ollama on the machine with the GPU, set OLLAMA_HOST=0.0.0.0 so it listens on the network, reach it over Tailscale, and add apiBase: http://gpu-box:11434 to each model entry. A 12 GB GPU serving a 7B chat model and a 1.5B autocomplete model at the same time is comfortable and can be shared across a household or a small team. For more than a few developers hitting it at once, vLLM serves the same models with proper batching, and Continue talks to it through the openai provider with the vLLM base URL. The local LLM hardware guide covers which GPU buys which model size.

Continue also speaks to any OpenAI-compatible endpoint, which means one config can mix a local autocomplete model with a hosted frontier model for chat. That hybrid is what most people settle on once the novelty wears off: the high-frequency, low-value keystrokes stay private, and the hard questions go somewhere smarter.

Where Copilot still wins

Autocomplete quality from a 1.5B model is uneven. It is excellent at boilerplate, imports, and continuing the pattern from the line above, and poor at anything requiring knowledge of a distant file. Copilot's completions are trained on more and served from bigger hardware, and in a large codebase you notice. Agent mode, where Continue edits files and runs commands in a loop, works with local models but needs a strong one (32B and up) to avoid wandering; on a 7B it is a demo. What Continue gives back is zero per-seat cost, a full view of what context is sent, telemetry you can switch off (allowAnonymousTelemetry: false), and no dependency on a vendor's quota. Tabby is the alternative if you would rather run a purpose-built completion server than assemble one from Ollama.

What I'd do

Install Continue in VS Code, pull the three models above, paste the config, and turn off Copilot for a week. On a 16 GB laptop with no GPU, keep autocomplete local and point chat at whichever hosted model your budget allows. With a 24 GB GPU anywhere on your network, run everything local through Ollama over Tailscale and add a 14B or 32B chat model. Either way it is the best free entry to the local LLM runner stack for someone who lives in an editor, and the config is 20 lines you will keep for years.