ik_llama.cpp
Performance-focused fork of llama.cpp with new quant types
ik_llama.cpp is a fork of llama.cpp maintained by ikawrakow that adds optimized CPU and CUDA kernels, additional quantization formats, and improved performance for mixture-of-experts models. It targets users running large local models on commodity hardware.
Key features
- New quantization types
- Optimized MoE inference
- CPU and CUDA kernels
- OpenAI-compatible server
Strengths
- Released under the MIT license
- Active community (3k GitHub stars)
- Written in C++
ik_llama.cpp replaces
Compare ik_llama.cpp
25 head-to-head comparisons.
- ik_llama.cpp vs Ollama
- ik_llama.cpp vs llama.cpp
- ik_llama.cpp vs vLLM
- ik_llama.cpp vs GPT4All
- ik_llama.cpp vs LiteLLM
- ik_llama.cpp vs exo
- ik_llama.cpp vs New API
- ik_llama.cpp vs Jan
- ik_llama.cpp vs FastChat
- ik_llama.cpp vs One API
- ik_llama.cpp vs Continue
- ik_llama.cpp vs llamafile
- ik_llama.cpp vs MLC LLM
- ik_llama.cpp vs OpenLLM
- ik_llama.cpp vs KoboldCpp
- ik_llama.cpp vs Text Generation Inference
- ik_llama.cpp vs Petals
- ik_llama.cpp vs Page Assist
- ik_llama.cpp vs LMDeploy
- ik_llama.cpp vs Enchanted
- ik_llama.cpp vs Serge
- ik_llama.cpp vs LM Studio
- ik_llama.cpp vs Text Embeddings Inference
- ik_llama.cpp vs Harbor LLM Toolkit
- ik_llama.cpp vs Aphrodite Engine
Similar local llm runners apps
Ollama
Local LLM RunnersRun large language models locally with a simple CLI and API
Replaces ChatGPT, OpenAI API
llama.cpp
Local LLM RunnersHigh-performance LLM inference in plain C/C++
Replaces OpenAI API
vLLM
Local LLM RunnersHigh-throughput LLM serving engine with PagedAttention
Replaces OpenAI API
GPT4All
Local LLM RunnersPrivacy-first desktop chat with local language models
Replaces ChatGPT
LiteLLM
Local LLM RunnersUnified proxy and gateway for 100+ LLM APIs
Replaces OpenRouter
exo
Local LLM RunnersRun your own AI cluster across everyday devices
Replaces OpenAI API