GPT4All
Local LLM RunnersPrivacy-first desktop chat with local language models
Replaces ChatGPT
Best Free Local LLM Runners for self-hosting
The 16 best Free local llm runners you can self-host, ranked by community traction.
New API
Local LLM RunnersNext-gen LLM gateway and AI asset management system
Replaces OpenRouter, OpenAI API
exo
Local LLM RunnersRun your own AI cluster across everyday devices
Replaces OpenAI API
Jan
Local LLM RunnersOpen-source offline ChatGPT alternative for the desktop
Replaces ChatGPT
FastChat
Local LLM RunnersPlatform for serving and evaluating large language models
Replaces OpenAI API
One API
Local LLM RunnersUnified OpenAI-compatible gateway for many LLM providers
Replaces OpenAI API, OpenRouter
llamafile
Local LLM RunnersDistribute and run LLMs with a single executable file
Replaces OpenAI API, ChatGPT
MLC LLM
Local LLM RunnersUniversal LLM deployment engine for any hardware
Replaces OpenAI API
Petals
Local LLM RunnersRun large language models collaboratively in a swarm
Replaces OpenAI API
LMDeploy
Local LLM RunnersToolkit for compressing and serving large language models
Replaces OpenAI API, Hugging Face Inference Endpoints
Text Embeddings Inference
Local LLM RunnersFast inference server for text embedding models
Replaces OpenAI Embeddings API
ik_llama.cpp
Local LLM RunnersPerformance-focused fork of llama.cpp with new quant types
Replaces OpenAI API
Harbor LLM Toolkit
Local LLM RunnersContainerized LLM toolkit to run a local AI stack with one CLI
Replaces OpenAI Platform
Aphrodite Engine
Local LLM RunnersHigh-throughput inference engine for large language models
Replaces OpenAI API
Wllama
Local LLM RunnersRun LLM inference directly in the browser with WebAssembly
Replaces OpenAI API
LLMKube
Local LLM RunnersKubernetes operator for llama.cpp-native LLM inference with GPU
No apps match these filters.
At a glance
| App | Deploy effort | Difficulty | Min. RAM | License | ★ Stars |
|---|---|---|---|---|---|
| GPT4All | Read-the-docs project | Easy | 8,192 MB | MIT | 77,370 |
| New API | Under-an-hour setup | Medium | 512 MB | MIT | 49,901 |
| exo | Read-the-docs project | Medium | 8,192 MB | GPL-3.0 | 47,870 |
| Jan | Read-the-docs project | Easy | 8,192 MB | AGPL-3.0 | 44,941 |
| FastChat | Under-an-hour setup | Hard | 8,192 MB | Apache-2.0 | 39,549 |
| One API | Under-an-hour setup | Medium | 256 MB | MIT | 37,136 |
| llamafile | Read-the-docs project | Easy | 8,192 MB | Apache-2.0 | 26,232 |
| MLC LLM | Read-the-docs project | Hard | 4,096 MB | Apache-2.0 | 23,239 |
| Petals | Under-an-hour setup | Hard | 8,192 MB | MIT | 10,629 |
| LMDeploy | Under-an-hour setup | Hard | 16,384 MB | Apache-2.0 | 8,104 |
Quick picks
- Most adopted: GPT4All (77,370 GitHub stars).
- Lightest footprint: One API (from 256 MB RAM, against 16,384 MB for the heaviest here).
- Licensing: 7 of 16 use Apache-2.0; check the rest individually if you plan to modify and redistribute.
Every option here is open-source and self-hostable, tagged Free within the local llm runners category. Deploy effort is graded from published container recipes, not vendor claims — see the methodology.
Keep exploring
Head-to-head: Free local llm runners
- Aphrodite Engine vs exo
- Aphrodite Engine vs FastChat
- Aphrodite Engine vs ik_llama.cpp
- Aphrodite Engine vs llamafile
- Aphrodite Engine vs LMDeploy
- Aphrodite Engine vs MLC LLM
- Aphrodite Engine vs New API
- Aphrodite Engine vs One API
- Aphrodite Engine vs Petals
- Aphrodite Engine vs Wllama
- exo vs FastChat
- exo vs ik_llama.cpp