SHSelfHostIndex
OP

OpenLLM

Run any open LLM as an OpenAI-compatible API endpoint

Local LLM Runners ★ 12.6k stars Medium setup Apache-2.0

OpenLLM lets developers run open-source large language models as OpenAI-compatible API endpoints with a single command. It is built on BentoML and supports streaming, quantization, and easy deployment.

Official siteSource code

Key features

  • One-command model serving
  • OpenAI-compatible API
  • Built-in chat UI
  • Cloud deployment via BentoML

Strengths

  • Released under the Apache-2.0 license
  • First-class Docker support for quick deployment
  • Kubernetes-ready with Helm charts available
  • Mature project with 12.6k GitHub stars

Is OpenLLM secure?

1 known vulnerability affects OpenLLM's latest release (v0.6.30) according to OSV.dev; the most severe is rated low.

Affecting v0.6.30

AdvisorySeveritySummary
CVE-2026-15035Lowbentoml OpenLLM Model Repository Directory Name common.py async_run_command command injection

Source: OSV.dev, checked 2026-10-14. How this is measured.

OpenLLM replaces

Self-hosted OpenAI API alternatives

Health badge

Maintain OpenLLM? Add its health score to your README. The badge links back to this page.

SelfHostIndex: 82 · excellent

Markdown
[![SelfHostIndex health score for OpenLLM](https://selfhostindex.com/badge/openllm.svg)](https://selfhostindex.com/app/openllm/)
HTML
<a href="https://selfhostindex.com/app/openllm/"><img src="https://selfhostindex.com/badge/openllm.svg" alt="SelfHostIndex health score for OpenLLM"></a>
reStructuredText
.. image:: https://selfhostindex.com/badge/openllm.svg
   :alt: SelfHostIndex health score for OpenLLM
   :target: https://selfhostindex.com/app/openllm/

The score is recomputed on every nightly rebuild from public repository data — see how it is calculated.

Compare OpenLLM

15 head-to-head comparisons.

Similar local llm runners apps