Harbor LLM Toolkit
Containerized LLM toolkit to run a local AI stack with one CLI
Harbor is a CLI and companion app that lets you run a complete local LLM stack of inference backends, web UIs, and supporting services with a single command. It bundles dozens of self-hostable AI tools into preconfigured Docker services.
Key features
- One-command local AI stack
- Dozens of bundled services
- Docker-based orchestration
- Preconfigured integrations
Strengths
- Released under the Apache-2.0 license
- Easy to set up — beginner-friendly
- First-class Docker support for quick deployment
- Active community (3.2k GitHub stars)
Harbor LLM Toolkit replaces
Compare Harbor LLM Toolkit
25 head-to-head comparisons.
- Harbor LLM Toolkit vs Ollama
- Harbor LLM Toolkit vs llama.cpp
- Harbor LLM Toolkit vs vLLM
- Harbor LLM Toolkit vs GPT4All
- Harbor LLM Toolkit vs LiteLLM
- Harbor LLM Toolkit vs exo
- Harbor LLM Toolkit vs New API
- Harbor LLM Toolkit vs Jan
- Harbor LLM Toolkit vs FastChat
- Harbor LLM Toolkit vs One API
- Harbor LLM Toolkit vs Continue
- Harbor LLM Toolkit vs llamafile
- Harbor LLM Toolkit vs MLC LLM
- Harbor LLM Toolkit vs OpenLLM
- Harbor LLM Toolkit vs KoboldCpp
- Harbor LLM Toolkit vs Text Generation Inference
- Harbor LLM Toolkit vs Petals
- Harbor LLM Toolkit vs Page Assist
- Harbor LLM Toolkit vs LMDeploy
- Harbor LLM Toolkit vs Enchanted
- Harbor LLM Toolkit vs Serge
- Harbor LLM Toolkit vs LM Studio
- Harbor LLM Toolkit vs Text Embeddings Inference
- Harbor LLM Toolkit vs ik_llama.cpp
- Harbor LLM Toolkit vs Aphrodite Engine
Similar local llm runners apps
Ollama
Local LLM RunnersRun large language models locally with a simple CLI and API
Replaces ChatGPT, OpenAI API
llama.cpp
Local LLM RunnersHigh-performance LLM inference in plain C/C++
Replaces OpenAI API
vLLM
Local LLM RunnersHigh-throughput LLM serving engine with PagedAttention
Replaces OpenAI API
GPT4All
Local LLM RunnersPrivacy-first desktop chat with local language models
Replaces ChatGPT
LiteLLM
Local LLM RunnersUnified proxy and gateway for 100+ LLM APIs
Replaces OpenRouter
exo
Local LLM RunnersRun your own AI cluster across everyday devices
Replaces OpenAI API