FastChat
Platform for serving and evaluating large language models
FastChat is an open platform for training, serving, and evaluating large language model chatbots. It powers the Chatbot Arena and provides an OpenAI-compatible API server for self-hosted models.
Key features
- OpenAI-compatible API
- Web UI for model chat
- Distributed serving
- Model evaluation tools
Pros & cons
Strengths
- Released under the Apache-2.0 license
- First-class Docker support for quick deployment
- Mature project with 39.5k GitHub stars
- Written in Python
Trade-offs
- GPU recommended
- Steep learning curve
FastChat replaces
Compare FastChat
36 head-to-head comparisons.
- FastChat vs Ollama
- FastChat vs Hugging Face Transformers
- FastChat vs llama.cpp
- FastChat vs vLLM
- FastChat vs GPT4All
- FastChat vs GPT4Free
- FastChat vs LiteLLM
- FastChat vs LocalAI
- FastChat vs exo
- FastChat vs New API
- FastChat vs Jan
- FastChat vs One API
- FastChat vs Continue
- FastChat vs SGLang
- FastChat vs llamafile
- FastChat vs MLC LLM
- FastChat vs Guidance
- FastChat vs OpenLLM
- FastChat vs KoboldCpp
- FastChat vs Text Generation Inference
- FastChat vs Petals
- FastChat vs Xinference
- FastChat vs Llama Stack
- FastChat vs Page Assist
- FastChat vs LMDeploy
- FastChat vs MLX LM
- FastChat vs Enchanted
- FastChat vs Serge
- FastChat vs GPUStack
- FastChat vs LM Studio
- FastChat vs Text Embeddings Inference
- FastChat vs Harbor LLM Toolkit
- FastChat vs ik_llama.cpp
- FastChat vs Aphrodite Engine
- FastChat vs Wllama
- FastChat vs LLMKube
Similar local llm runners apps
Ollama
Local LLM RunnersRun large language models locally with a simple CLI and API
Replaces ChatGPT, OpenAI API
llama.cpp
Local LLM RunnersHigh-performance LLM inference in plain C/C++
Replaces OpenAI API
vLLM
Local LLM RunnersHigh-throughput LLM serving engine with PagedAttention
Replaces OpenAI API
GPT4All
Local LLM RunnersPrivacy-first desktop chat with local language models
Replaces ChatGPT
LiteLLM
Local LLM RunnersUnified proxy and gateway for 100+ LLM APIs
Replaces OpenRouter
exo
Local LLM RunnersRun your own AI cluster across everyday devices
Replaces OpenAI API