Prompt Optimizer

Last reviewed Aug 26, 2026 · 667 words

You have a system prompt that behaves 8 times in 10, and every attempt to fix the 2 failures breaks something else. Prompt Optimizer exists for that half hour: paste the prompt, let a model rewrite it against a set of optimisation templates, then run the original and the candidate side by side on the same input and see which one actually holds. It is a workbench, not an eval suite, and it runs entirely in your browser, which is both why it is trivial to self-host and why you should think about where your API keys end up.

The Docker image is a static site with a settings page

There is no backend to speak of. The container serves a TypeScript single-page app; your browser talks to the model providers directly. That means the server needs 256 MB of RAM at most, restarts are free, and there is no database to back up. The README's docker run line is all the deployment there is, and the same app is published as a browser extension, so "self-host" here mostly means "put it on a URL my whole team can open" rather than "run infrastructure". First released in 2024, it has already collected 33,357 GitHub stars, most of them from people who wanted something lighter than a full LLM ops platform.

Keys live in your browser unless you set them at the container

The default flow asks for provider API keys in the settings modal and stores them in browser local storage. On your own laptop that is fine. On a shared instance it is a problem, so the image accepts keys as environment variables at start, which the app reads instead of prompting. Put the optional access password (also an environment variable) in front of it and keep the whole thing on the LAN or behind Tailscale. Do not put a key-holding page on the public internet with no auth; it is a free API credit for whoever finds it.

Pointing it at your own models

The multi-provider support covers the hosted names and, more usefully, any OpenAI-compatible endpoint. An Ollama server at http://gpu-box:11434/v1 works as a custom provider, as does a vLLM or LiteLLM gateway. Two things to remember: enable CORS on the model server, because the browser is the client (OLLAMA_ORIGINS=* on the Ollama side, or a reverse proxy adding the headers), and note that "runs entirely client-side" cuts both ways: a locked-down corporate browser that blocks cross-origin requests will refuse silently. Running a local 8B model as the optimiser and a frontier model as the judge is a cheap way to iterate.

The actual loop, and where it stops being enough

The useful pattern is: optimise, then compare on 3 or 4 inputs you know are hard, not the one easy example. The side-by-side view makes regressions obvious in a way that flipping between tabs never did, and the history lets you walk back a step. What it does not do is run 200 cases and score them, track pass rates over time, or gate a deploy. That is Promptfoo territory, and the honest workflow is Prompt Optimizer for the exploratory 20 minutes and Promptfoo for the regression suite before shipping. The principles in prompt engineering for engineers are the ones the optimisation templates encode: explicit role, constraints before content, examples for format.

What I'd do

Run the container on the homelab, set the provider keys as environment variables so nobody pastes secrets into a browser, put it behind Tailscale with the access password on, and add an Ollama custom provider so the cheap iterations cost nothing. Use it every time a prompt misbehaves, and move the winning prompt into a Promptfoo test the same day. It is a single-purpose tool, and 33,357 stars say the single purpose is one a lot of people needed.