SGLang
Self-Hosted AIFast serving framework for LLMs and vision-language models
Replaces OpenAI API
Best Kubernetes Self-Hosted AI for self-hosting
The 8 best Kubernetes self-hosted ai you can self-host, ranked by community traction.
Langfuse
Self-Hosted AIOpen-source observability and analytics for LLM apps
Replaces LangSmith
Triton Inference Server
Self-Hosted AIHigh-performance inference serving for any model framework
Replaces Amazon SageMaker
Xinference
Self-Hosted AIDistributed inference framework for LLMs and embeddings
Replaces OpenAI API, Hugging Face Inference Endpoints
BentoML
Self-Hosted AIFramework for building and serving AI model APIs
Replaces Amazon SageMaker, Vertex AI
GPUStack
Self-Hosted AIManage GPU clusters for running AI models
Replaces OpenAI API
RamaLama
Self-Hosted AIRun AI models using OCI containers
Replaces Ollama
BionicGPT
Self-Hosted AIOn-premise generative AI platform for teams
Replaces ChatGPT Enterprise
No apps match these filters.
At a glance
| App | Deploy effort | Difficulty | Min. RAM | License | ★ Stars |
|---|---|---|---|---|---|
| SGLang | Under-an-hour setup | Hard | 16,384 MB | Apache-2.0 | 37,106 |
| Langfuse | Under-an-hour setup | Medium | 2,048 MB | MIT | 35,795 |
| Triton Inference Server | Under-an-hour setup | Hard | 4,096 MB | BSD-3-Clause | 11,085 |
| Xinference | Under-an-hour setup | Medium | 8,192 MB | Apache-2.0 | 9,609 |
| BentoML | Under-an-hour setup | Medium | 2,048 MB | Apache-2.0 | 8,900 |
| GPUStack | Under-an-hour setup | Medium | 8,192 MB | Apache-2.0 | 5,825 |
| RamaLama | Under-an-hour setup | Medium | 8,192 MB | MIT | 3,084 |
| BionicGPT | Under-an-hour setup | Medium | 4,096 MB | Apache-2.0 | 2,388 |
Quick picks
- Most adopted: SGLang (37,106 GitHub stars).
- Lightest footprint: Langfuse (from 2,048 MB RAM, against 16,384 MB for the heaviest here).
- Licensing: 5 of 8 use Apache-2.0; check the rest individually if you plan to modify and redistribute.
Every option here is open-source and self-hostable, tagged Kubernetes within the self-hosted ai category. Deploy effort is graded from published container recipes, not vendor claims — see the methodology.