Ollama
Local LLM RunnersRun large language models locally with a simple CLI and API
Replaces ChatGPT, OpenAI API
Best Docker Local LLM Runners for self-hosting
The 12 best Docker local llm runners you can self-host, ranked by community traction.
vLLM
Local LLM RunnersHigh-throughput LLM serving engine with PagedAttention
Replaces OpenAI API
LiteLLM
Local LLM RunnersUnified proxy and gateway for 100+ LLM APIs
Replaces OpenRouter
New API
Local LLM RunnersNext-gen LLM gateway and AI asset management system
Replaces OpenRouter, OpenAI API
FastChat
Local LLM RunnersPlatform for serving and evaluating large language models
Replaces OpenAI API
One API
Local LLM RunnersUnified OpenAI-compatible gateway for many LLM providers
Replaces OpenAI API, OpenRouter
OpenLLM
Local LLM RunnersRun any open LLM as an OpenAI-compatible API endpoint
Replaces OpenAI API
LMDeploy
Local LLM RunnersToolkit for compressing and serving large language models
Replaces OpenAI API, Hugging Face Inference Endpoints
Text Embeddings Inference
Local LLM RunnersFast inference server for text embedding models
Replaces OpenAI Embeddings API
Harbor LLM Toolkit
Local LLM RunnersContainerized LLM toolkit to run a local AI stack with one CLI
Replaces OpenAI Platform
Aphrodite Engine
Local LLM RunnersHigh-throughput inference engine for large language models
Replaces OpenAI API
LLMKube
Local LLM RunnersKubernetes operator for llama.cpp-native LLM inference with GPU
No apps match these filters.
At a glance
| App | Deploy effort | Difficulty | Min. RAM | License | ★ Stars |
|---|---|---|---|---|---|
| Ollama | ≈5-minute setup | Easy | 8,192 MB | MIT | 183,072 |
| vLLM | Under-an-hour setup | Hard | 16,384 MB | Apache-2.0 | 93,739 |
| LiteLLM | Under-an-hour setup | Medium | 1,024 MB | MIT | 61,190 |
| New API | Under-an-hour setup | Medium | 512 MB | MIT | 49,901 |
| FastChat | Under-an-hour setup | Hard | 8,192 MB | Apache-2.0 | 39,549 |
| One API | Under-an-hour setup | Medium | 256 MB | MIT | 37,136 |
| OpenLLM | Under-an-hour setup | Medium | 8,192 MB | Apache-2.0 | 12,558 |
| LMDeploy | Under-an-hour setup | Hard | 16,384 MB | Apache-2.0 | 8,104 |
| Text Embeddings Inference | Under-an-hour setup | Medium | 2,048 MB | Apache-2.0 | 5,076 |
| Harbor LLM Toolkit | Under-an-hour setup | Easy | 8,192 MB | Apache-2.0 | 3,246 |
Quick picks
- Most adopted: Ollama (183,072 GitHub stars).
- Lightest footprint: One API (from 256 MB RAM, against 16,384 MB for the heaviest here).
- Fastest to deploy: Ollama (verified single-container compose).
- Licensing: 7 of 12 use Apache-2.0; check the rest individually if you plan to modify and redistribute.
Every option here is open-source and self-hostable, tagged Docker within the local llm runners category. Deploy effort is graded from published container recipes, not vendor claims — see the methodology.