SHSelfHostIndex
TE

Text Embeddings Inference

Fast inference server for text embedding models

Local LLM Runners ★ 5.1k stars Medium setup Apache-2.0

Text Embeddings Inference is a toolkit for deploying and serving open-source text embeddings and sequence classification models. It provides high-performance extraction with dynamic batching and an OpenAI-compatible route.

Official siteSource code

Key features

  • Dynamic batching
  • GPU and CPU support
  • OpenAI-compatible endpoint
  • Low-overhead serving

Pros & cons

Strengths

  • Released under the Apache-2.0 license
  • First-class Docker support for quick deployment
  • Mature project with 5.1k GitHub stars
  • Written in Rust

Trade-offs

  • Embeddings only
  • Model compatibility limits

Is Text Embeddings Inference secure?

No known vulnerabilities affect Text Embeddings Inference's latest release (v1.9.4). OSV.dev lists 1 past advisory for the project; none of them applies to this release. No known vulnerabilities is not proof of security: it means none have been publicly catalogued.

  • Most recent fixed advisory CVE-2026-108857, published 2026-10-11 (3 days ago), fixed in 1.9.4

Source: OSV.dev, checked 2026-10-14. How this is measured.

Text Embeddings Inference replaces

Self-hosted OpenAI Embeddings API alternatives

Health badge

Maintain Text Embeddings Inference? Add its health score to your README. The badge links back to this page.

SelfHostIndex: 92 · excellent

Markdown
[![SelfHostIndex health score for Text Embeddings Inference](https://selfhostindex.com/badge/text-embeddings-inference.svg)](https://selfhostindex.com/app/text-embeddings-inference/)
HTML
<a href="https://selfhostindex.com/app/text-embeddings-inference/"><img src="https://selfhostindex.com/badge/text-embeddings-inference.svg" alt="SelfHostIndex health score for Text Embeddings Inference"></a>
reStructuredText
.. image:: https://selfhostindex.com/badge/text-embeddings-inference.svg
   :alt: SelfHostIndex health score for Text Embeddings Inference
   :target: https://selfhostindex.com/app/text-embeddings-inference/

The score is recomputed on every nightly rebuild from public repository data — see how it is calculated.

Similar local llm runners apps