vLLM
Local LLM RunnersHigh-throughput LLM serving engine with PagedAttention
Replaces OpenAI API
Best Kubernetes Local LLM Runners for self-hosting
The 4 best Kubernetes local llm runners you can self-host, ranked by community traction.
LiteLLM
Local LLM RunnersUnified proxy and gateway for 100+ LLM APIs
Replaces OpenRouter
OpenLLM
Local LLM RunnersRun any open LLM as an OpenAI-compatible API endpoint
Replaces OpenAI API
LMDeploy
Local LLM RunnersToolkit for compressing and serving large language models
Replaces OpenAI API, Hugging Face Inference Endpoints
No apps match these filters.
At a glance
| App | Deploy effort | Difficulty | Min. RAM | License | ★ Stars |
|---|---|---|---|---|---|
| vLLM | Under-an-hour setup | Hard | 16,384 MB | Apache-2.0 | 93,739 |
| LiteLLM | Under-an-hour setup | Medium | 1,024 MB | MIT | 61,190 |
| OpenLLM | Under-an-hour setup | Medium | 8,192 MB | Apache-2.0 | 12,558 |
| LMDeploy | Under-an-hour setup | Hard | 16,384 MB | Apache-2.0 | 8,104 |
Quick picks
- Most adopted: vLLM (93,739 GitHub stars).
- Lightest footprint: LiteLLM (from 1,024 MB RAM, against 16,384 MB for the heaviest here).
- Licensing: 3 of 4 use Apache-2.0; check the rest individually if you plan to modify and redistribute.
Every option here is open-source and self-hostable, tagged Kubernetes within the local llm runners category. Deploy effort is graded from published container recipes, not vendor claims — see the methodology.