Open-weight models in 2026 are the right default for classification, extraction, summarisation, RAG answering, and most internal tooling; frontier APIs still earn their price on hard multi-step reasoning, long agentic sessions, and anywhere the last 10% of quality is the product. The gap at the top is now measured in months rather than years, and it re-opens every time a frontier lab ships — so the durable skill isn't knowing which model is best this quarter, it's knowing which tier of task you're staffing and what a model at that tier costs to run.
The usable set, by task
Names below are current at time of writing; the tier structure outlives the names. Check Hugging Face for whatever has succeeded them by the time you read this — succession is the norm, roughly quarterly per family.
| Task | What to run | Size tier |
|---|---|---|
| Classification, routing, tagging | Qwen3 4B, Gemma 3 4B, Phi-4-mini | 1–4B |
| General chat, RAG answering | Qwen3 14B/32B, Gemma 3 27B, Llama 3.3 70B | 8–70B |
| Coding | Qwen2.5-Coder 32B, DeepSeek-V3-class MoE | 32B+ |
| Reasoning-heavy | DeepSeek-R1 distills, QwQ-32B | 32B–70B |
| Vision/document understanding | Qwen-VL family, Gemma 3 (multimodal) | 4–72B |
| Embeddings | bge-m3, nomic-embed-text | 0.1–0.6B |
Three observations that hold across generations. The 30B-class is the sweet spot — roughly frontier-of-18-months-ago quality on a single 24GB GPU at Q4, which is why the used RTX 3090 remains the community's workhorse (the hardware guide has the full tier table). Mixture-of-experts models punch above their active-parameter count but need the total parameters in memory, so they favour high-RAM Macs and multi-GPU rigs over single cards. And small models are the fastest-improving tier: today's 4B models beat 2024's 13B models on most instruction-following benchmarks, which quietly rewrites the economics of high-volume pipelines.
Licences: read before you build
"Open weight" is not one licence, and the differences bite at inconvenient times:
- Apache 2.0 / MIT (most Qwen releases, DeepSeek, Mistral's Apache drops, OLMo): genuinely permissive. Commercial use, modification, redistribution — no surprises. Prefer these when licensing review is expensive.
- Llama Community License: fine for almost everyone, but it's a custom licence with a 700M-MAU clause aimed at big tech, attribution requirements ("Built with Llama"), naming rules for derivatives, and an acceptable-use policy incorporated by reference. Your counsel will want to read it; that reading has a cost.
- Gemma Terms of Use: permissive in practice, but usage restrictions live in a policy document Google can update, which some compliance teams treat as a moving dependency.
- Research-only and non-commercial releases still appear regularly; reading only the model card headline has burned real teams.
Also check the licence of the training pipeline you use and whether a model's licence restricts using its outputs to train competing models — several do, which matters if you're generating synthetic data or distilling. The distinction between open-weight and OSI-open-source (training data and code included) is covered in the site's licence guide for self-hosters; for internal deployment the practical difference is small, for redistribution it isn't.
Where open weights beat APIs
Privacy and data residency. The clean argument: prompts containing patient records, contracts, or code under NDA never leave your hardware. This alone decides the question for a large share of deployments.
Unit cost at volume — with a utilisation asterisk. The arithmetic: a $1,800 RTX 4090 amortised over three years is about $0.07/hour; add ~$0.05 of electricity at 300W and $0.15/kWh. An 8B model served properly (batched, via vLLM) sustains thousands of tokens/second, putting marginal cost in the cents-per-million-tokens range against API prices of $0.10–0.60/M for comparable small models. The asterisk is utilisation: that math assumes the GPU is busy. At 5% utilisation your effective cost is 20× worse, and below roughly 1–2M tokens/day the API wins on cost alone. Batch pipelines and steady internal traffic amortise; a spiky chatbot doesn't.
Latency and control. On-prem inference has no network hop, no rate limits, and no surprise queueing during a provider incident.
Version stability. Nobody deprecates weights on your disk. API model retirements force migrations on the vendor's schedule; the fine-tune you own runs identically in five years.
Fine-tuning ownership. A LoRA on an open model is an asset you control and can serve anywhere; a fine-tune held via an API is a rental.
Where they don't
Long agentic chains still separate the tiers fastest — per-step error compounds, and frontier models' higher per-step reliability translates into disproportionately better 20-step outcomes. Frontier reasoning, low-resource languages, and the long tail of world knowledge remain API territory. So does low, spiky volume (the utilisation math above), and so does any team without appetite for owning GPU infrastructure — an idle MLOps burden is a real cost the API price includes.
Running them
For evaluation and development, Ollama gets any model on this page running in one command (ollama pull qwen3:32b) with sensible Q4_K_M defaults — what quantisation costs you is measurable and covered in the quantisation guide. For production serving, graduate to vLLM or similar for continuous batching; the difference is a full multiple of throughput once you have concurrent users.
Bottom line
Staff the task tier, not the leaderboard. Under 8B for high-volume classification and extraction; 30B-class on a 24GB GPU for chat, RAG, and coding assistance; frontier APIs for the hard reasoning and long-horizon agent work; and Apache/MIT models when legal review is a bottleneck. Do the utilisation math before buying hardware, and re-run the whole comparison every six months — the one prediction that's held since 2023 is that the open tier keeps absorbing tasks from the tier above it.