SHSelfHostIndex
XI

Xinference

Distributed inference framework for LLMs and embeddings

Self-Hosted AI ★ 9.6k stars Medium setup Apache-2.0

Xorbits Inference (Xinference) is a framework for serving language, embedding, image, audio, and rerank models with a single command. It exposes OpenAI-compatible APIs and supports distributed deployment across multiple machines.

Official siteSource code

Key features

  • Serve LLMs, embeddings and images
  • OpenAI-compatible API
  • Distributed cluster support
  • Built-in model registry

Strengths

  • Released under the Apache-2.0 license
  • First-class Docker support for quick deployment
  • Kubernetes-ready with Helm charts available
  • Mature project with 9.6k GitHub stars

Is Xinference secure?

No known vulnerabilities affect Xinference's latest release (v4.0.0). OSV.dev lists 5 past advisories for the project; none of them applies to this release. No known vulnerabilities is not proof of security: it means none have been publicly catalogued.

  • Most recent fixed advisory CVE-2026-85668, published 2026-09-04 (40 days ago)

Source: OSV.dev, checked 2026-10-14. How this is measured.

Xinference replaces

Self-hosted OpenAI API alternativesSelf-hosted Hugging Face Inference Endpoints alternatives

Health badge

Maintain Xinference? Add its health score to your README. The badge links back to this page.

SelfHostIndex: 94 · excellent

Markdown
[![SelfHostIndex health score for Xinference](https://selfhostindex.com/badge/xinference.svg)](https://selfhostindex.com/app/xinference/)
HTML
<a href="https://selfhostindex.com/app/xinference/"><img src="https://selfhostindex.com/badge/xinference.svg" alt="SelfHostIndex health score for Xinference"></a>
reStructuredText
.. image:: https://selfhostindex.com/badge/xinference.svg
   :alt: SelfHostIndex health score for Xinference
   :target: https://selfhostindex.com/app/xinference/

The score is recomputed on every nightly rebuild from public repository data — see how it is calculated.

Compare Xinference

10 head-to-head comparisons.

Similar self-hosted ai apps