SHSelfHostIndex
LM

LMDeploy

Toolkit for compressing and serving large language models

Local LLM Runners ★ 8.1k stars Hard setup Apache-2.0

LMDeploy is an inference and serving toolkit from the OpenMMLab ecosystem for deploying large language models efficiently. It offers quantization, a high-throughput serving engine, and an OpenAI-compatible API.

Official siteSource code

Key features

  • High-throughput inference engine
  • Weight quantization support
  • OpenAI-compatible serving
  • Multi-GPU deployment

Strengths

  • Released under the Apache-2.0 license
  • First-class Docker support for quick deployment
  • Mature project with 8.1k GitHub stars
  • Written in Python

Is LMDeploy secure?

No known vulnerabilities affect LMDeploy's latest release (v0.18.0). OSV.dev lists 13 past advisories for the project; none of them applies to this release. No known vulnerabilities is not proof of security: it means none have been publicly catalogued.

  • Most recent fixed advisory CVE-2025-66455, published 2026-09-18 (26 days ago), fixed in 0.16.0

Source: OSV.dev, checked 2026-10-14. How this is measured.

LMDeploy replaces

Self-hosted OpenAI API alternativesSelf-hosted Hugging Face Inference Endpoints alternatives

Health badge

Maintain LMDeploy? Add its health score to your README. The badge links back to this page.

SelfHostIndex: 93 · excellent

Markdown
[![SelfHostIndex health score for LMDeploy](https://selfhostindex.com/badge/lmdeploy.svg)](https://selfhostindex.com/app/lmdeploy/)
HTML
<a href="https://selfhostindex.com/app/lmdeploy/"><img src="https://selfhostindex.com/badge/lmdeploy.svg" alt="SelfHostIndex health score for LMDeploy"></a>
reStructuredText
.. image:: https://selfhostindex.com/badge/lmdeploy.svg
   :alt: SelfHostIndex health score for LMDeploy
   :target: https://selfhostindex.com/app/lmdeploy/

The score is recomputed on every nightly rebuild from public repository data — see how it is calculated.

Compare LMDeploy

15 head-to-head comparisons.

Similar local llm runners apps