Text Generation WebUI

Last reviewed Aug 26, 2026 · 883 words

Run Text Generation WebUI when you want the knobs. Ollama hides the loader, the sampler settings, and the prompt template behind a one-line ollama run, and for most people that is the right amount of control. Oobabooga, as everyone calls this project, exposes all of it: llama.cpp, ExLlama, and Transformers loaders side by side, every sampling parameter as a slider, a notebook mode for raw completion, and a training tab. It is a workbench where Ollama is an appliance, and the price of the workbench is a multi-gigabyte Python environment, 8 GB of system RAM as the floor, and a GPU with at least 8 GB of VRAM before it is fun to use.

Who it is for, in one table

The middle row is the honest scope. If you never expect to change a loader or touch a sampler, the bulk is not worth carrying.

Install with the start script, and let it own its environment

Clone the repository and run the launcher for your platform:

git clone https://github.com/oobabooga/text-generation-webui.git
cd text-generation-webui
./start_linux.sh

On first run it asks which GPU vendor you have (NVIDIA, AMD, Apple Silicon, Intel, or CPU only) and builds a private conda environment under installer_files/. That directory is the entire install; when an update leaves things broken, and "bulky Python environment" in the catalogue is a polite description of why, deleting installer_files/ and re-running the script is a cleaner fix than any amount of pip surgery. The project also ships portable builds, zip archives with only the llama.cpp backend, that need no install at all; if GGUF is the only format you plan to run, those are the lower-maintenance choice. Docker compose files for each backend live in the repository too, but GPU passthrough adds its own layer of debugging, and I would only reach for them on a machine that is already all-containers.

Match the loader to the file format

Models go in the models directory (user_data/models/ in current releases) and the loader is chosen per file:

• GGUF files use the llama.cpp loader. It splits layers between GPU and CPU, so a model that does not fully fit in VRAM still runs, just slower. This is the loader for an 8 GB card.
• EXL2 and EXL3 files use ExLlama. GPU-only, and the fastest option when the whole model fits in VRAM.
• Full-precision safetensors use the Transformers loader. Slowest, largest, and the only one that supports the training tab.

The number to carry in your head: an 8B model at Q4 wants about 5 to 6 GB of VRAM with a modest context, a 32B model at Q4 wants about 20 GB, and every 1,000 tokens of context adds more on top. The VRAM math post has the working; run it before downloading a 20 GB file that will not load.

--api turns it into a backend

The web UI listens on port 7860. Start it with --api and it also serves an OpenAI-compatible endpoint on port 5000, which is the reason many people keep it around after they stop using the Gradio interface:

./start_linux.sh --api --listen --api-key change-me

--listen binds to all interfaces so other machines on the LAN can reach it, and --api-key is not optional once you do that. SillyTavern talks to it natively and so does anything that speaks the OpenAI chat format. Whatever model is loaded in the UI is what the API serves, and switching models through the API is supported but slow, so treat one instance as one model.

LoRA training is real, with one catch

The Training tab fine-tunes a LoRA on your own text and is the feature none of the lighter runners have. The catch is that it only works with the Transformers loader, so you need the unquantised model or one loaded with bitsandbytes 4-bit, and a 7B or 8B model is the practical ceiling on a 24 GB consumer card. Expect hours, not minutes, and read the LoRA fine-tuning on a consumer GPU post first for the dataset preparation that decides whether the result is any good.

What I'd do

If you have an NVIDIA card with 12 GB or more and you enjoy tuning, install it with the start script, run GGUF models through llama.cpp until you find one you like, then try the EXL2 version of the same model and notice the speed difference. Keep it running with --api and put Open WebUI or SillyTavern in front for daily chat. If you have 8 GB, use the portable build and stay on Q4 GGUF. If you just want a local model behind an API and never intend to open a Gradio tab, install Ollama instead and spend the saved afternoon on something else.