New API vs vLLM
A side-by-side comparison of two self-hosted local llm runners options — licensing, setup difficulty, resource needs, and what each one replaces.
vLLM
High-throughput LLM serving engine with PagedAttention
| Feature | New API | vLLM |
|---|---|---|
| Deploy effort | Under-an-hour setup | Under-an-hour setup |
| Health score | 100 · Excellent | 100 · Excellent |
| Category | Local LLM Runners | Local LLM Runners |
| License | MIT | Apache-2.0 |
| Language | Go | Python |
| Setup difficulty | Medium | Hard |
| Min. RAM | 512 MB | 16,384 MB |
| Deployment | docker | docker, kubernetes, bare-metal |
| GitHub stars | ★ 48,815 | ★ 92,565 |
| First released | 2024 | 2023 |
| Replaces | OpenRouter, OpenAI API | OpenAI API |
What are New API and vLLM?
New API
New API is a large-model interface management and distribution system derived from One API, providing an OpenAI-compatible gateway with richer billing, rendering and provider support. It is designed for running a self-hosted AI relay service.
- OpenAI-compatible relay
- Multi-provider channels
- Detailed billing
- Modern dashboard
vLLM
vLLM is a fast and memory-efficient inference and serving engine for large language models. Its PagedAttention algorithm delivers high throughput batching, and it exposes an OpenAI-compatible server for production deployments.
- PagedAttention memory management
- Continuous batching
- OpenAI-compatible server
- Tensor parallelism
New API vs vLLM: key differences
New API is written in Go, while vLLM is built with Python. Licensing differs — MIT for New API versus Apache-2.0 for vLLM. New API is the lighter option, starting around 512 MB of RAM against 16,384 MB for vLLM.
Why pick each one
Choose New API if…
- OpenAI-compatible gateway
- Broad provider support
- Built-in billing and quotas
Watch out for
- Docs primarily Chinese
- Niche relay use case
Choose vLLM if…
- Excellent serving throughput
- OpenAI-compatible API
- Efficient GPU memory use
Watch out for
- GPU practically required
- Complex tuning options
Frequently asked questions
Is New API or vLLM better?
Neither is universally better. vLLM has the larger community, while New API is simpler to set up (medium difficulty). Choose based on the comparison table above and your own setup.
Are New API and vLLM free and open-source?
Yes. New API is licensed under MIT and vLLM under Apache-2.0. Both can be self-hosted at no software cost.
Can I run New API and vLLM with Docker?
New API: yes. vLLM: yes.
Which is lighter on resources, New API or vLLM?
New API has the smaller minimum footprint at 512 MB of RAM, compared to about 16,384 MB for vLLM. Real-world usage depends on library size, user count, and enabled features.