SHSelfHostIndex
EX

ExLlama

Memory-efficient inference library for quantized Llama models

Self-Hosted AI ★ 2.9k stars Hard setup MIT

ExLlama is a standalone Python/C++/CUDA implementation for running quantized GPTQ Llama models with low VRAM use on modern GPUs. It is the predecessor to ExLlamaV2 and focuses on fast, memory-efficient local inference.

Official siteSource code

Key features

  • Low VRAM GPTQ inference
  • CUDA-accelerated
  • Standalone library
  • Fast token generation

Strengths

  • Released under the MIT license
  • Active community (2.9k GitHub stars)
  • Written in Python

Is ExLlama secure?

OSV.dev has no published security advisories for ExLlama's repository. No known vulnerabilities is not proof of security: it means none have been publicly catalogued.

Source: OSV.dev, checked 2026-10-14. How this is measured.

ExLlama replaces

Self-hosted OpenAI API alternatives

Health badge

Maintain ExLlama? Add its health score to your README. The badge links back to this page.

SelfHostIndex: 23 · at risk

Markdown
[![SelfHostIndex health score for ExLlama](https://selfhostindex.com/badge/exllama.svg)](https://selfhostindex.com/app/exllama/)
HTML
<a href="https://selfhostindex.com/app/exllama/"><img src="https://selfhostindex.com/badge/exllama.svg" alt="SelfHostIndex health score for ExLlama"></a>
reStructuredText
.. image:: https://selfhostindex.com/badge/exllama.svg
   :alt: SelfHostIndex health score for ExLlama
   :target: https://selfhostindex.com/app/exllama/

The score is recomputed on every nightly rebuild from public repository data — see how it is calculated.

Compare ExLlama

10 head-to-head comparisons.

Similar self-hosted ai apps