whisper.cpp

Last reviewed Aug 26, 2026 · 932 words

A 142 MB model, a four-core laptop CPU, no GPU, no Python, no network: whisper.cpp with base.en turns an hour of clear English speech into text in about 5 minutes on that setup in my experience, and never sends a byte off the machine. The large model that matches cloud transcription quality is 3.1 GB and roughly 10 times slower on the same CPU, which is the whole decision you are making with this tool. Pick the model that fits your patience, build the binary once, and it will outlive every subscription transcription service you have used.

Build it; the packaged versions lag

Homebrew has whisper-cpp and some distros carry a package, but the project moves fast and the build takes about a minute, so I build from source everywhere:

git clone https://github.com/ggml-org/whisper.cpp
cd whisper.cpp
cmake -B build -DCMAKE_BUILD_TYPE=Release
cmake --build build -j --config Release
sh ./models/download-ggml-model.sh base.en
./build/bin/whisper-cli -m models/ggml-base.en.bin -f samples/jfk.wav

The binaries land in build/bin/: whisper-cli for files, whisper-server for the HTTP service, quantize for shrinking models. GPU builds are one flag: -DGGML_CUDA=1 for Nvidia, -DGGML_VULKAN=1 for most other cards. On Apple Silicon, Metal is on by default and the optional Core ML encoder (-DWHISPER_COREML=1) roughly triples speed after a one-time model conversion. There are no runtime dependencies; copy the binary and a model to any machine of the same architecture and it works.

Model choice is a size-versus-accuracy table, and .en matters

The .en variants are English-only and noticeably better at English than the multilingual model of the same size, so use them unless you need another language. Quantizing trims size and RAM with a small accuracy cost: ./build/bin/quantize models/ggml-large-v3-turbo.bin models/ggml-large-v3-turbo-q5_0.bin q5_0 roughly halves the file. Working memory is about the model size plus a few hundred MB, which is how a 1 GB machine can run base.en and why large-v3 wants 4 GB. If quantization levels are new to you, quantization explained covers what q5_0 and q8_0 mean in plain terms.

Feed it 16 kHz mono WAV or it will refuse

The single most common first-run failure is failed to read WAV file, because the CLI expects 16 kHz, 16-bit, mono PCM. Convert everything on the way in:

ffmpeg -i meeting.m4a -ar 16000 -ac 1 -c:a pcm_s16le meeting.wav
./build/bin/whisper-cli -m models/ggml-small.en.bin -f meeting.wav -otxt -osrt

-otxt and -osrt write meeting.wav.txt and meeting.wav.srt next to the input; -ovtt and -ocsv exist too. -l auto detects the language with a multilingual model, -tr translates to English, and -t 8 sets threads, which you should match to physical cores rather than leaving at the default. Word-level timestamps are behind -ml 1, useful for karaoke-style subtitles and not much else.

The server turns a CLI into a service

whisper-server is why this tool sits in a self-hosting directory rather than a developer-tools list. One process, one model loaded once, an HTTP endpoint any script can hit:

./build/bin/whisper-server -m models/ggml-small.en.bin --host 0.0.0.0 --port 8080 --convert
curl 127.0.0.1:8080/inference -F [email protected] -F response_format=srt

--convert lets the server call ffmpeg itself, so clients can post whatever they have. Two limits to design around: there is no authentication, so keep it on a LAN or behind a VPN, and requests are processed one at a time, so a queue in front (an n8n workflow, a folder watcher, a shell loop) is the pattern for batch jobs. A Raspberry Pi 5 running base.en this way is a perfectly serviceable household transcription box.

Where it loses: speakers and Nvidia cards

whisper.cpp does not tell you who is speaking. The --diarize flag only separates stereo channels, and the tinydiarize experiment covers small.en alone. For labelled speakers and accurate word timings, WhisperX is the tool, at the cost of a Python environment. And if you own an Nvidia GPU, faster-whisper generally beats whisper.cpp on throughput; Speaches wraps it in an OpenAI-compatible API, and Home Assistant's voice pipeline uses it rather than whisper.cpp. whisper.cpp's territory is CPUs, Apple Silicon, and machines where installing a CUDA toolchain is not on the table, which describes most homelabs.

What I'd do

Build from source with the GPU flag your hardware supports, download small.en and large-v3-turbo, and quantize the turbo model to q5_0. Run whisper-server with small.en as an always-on service on the LAN for quick jobs, and use the CLI with turbo for anything you will publish. Convert to 16 kHz WAV by reflex, set -t to your core count, and stop paying for transcription.