Harbor LLM Toolkit
Containerized LLM toolkit to run a local AI stack with one CLI
Harbor is a CLI and companion app that lets you run a complete local LLM stack of inference backends, web UIs, and supporting services with a single command. It bundles dozens of self-hostable AI tools into preconfigured Docker services.
Key features
- One-command local AI stack
- Dozens of bundled services
- Docker-based orchestration
- Preconfigured integrations
Strengths
- Released under the Apache-2.0 license
- Easy to set up — beginner-friendly
- First-class Docker support for quick deployment
- Active community (3.2k GitHub stars)
Harbor LLM Toolkit replaces
Similar local llm runners apps
Ollama
Local LLM RunnersRun large language models locally with a simple CLI and API
Replaces ChatGPT, OpenAI API
llama.cpp
Local LLM RunnersHigh-performance LLM inference in plain C/C++
Replaces OpenAI API
vLLM
Local LLM RunnersHigh-throughput LLM serving engine with PagedAttention
Replaces OpenAI API
GPT4All
Local LLM RunnersPrivacy-first desktop chat with local language models
Replaces ChatGPT
LiteLLM
Local LLM RunnersUnified proxy and gateway for 100+ LLM APIs
Replaces OpenRouter
New API
Local LLM RunnersNext-gen LLM gateway and AI asset management system
Replaces OpenRouter, OpenAI API