A prompt is a configuration artifact that ships to production, so treat it exactly like one: files in git, explicit template variables, a regression suite that runs in CI, and review on every diff. Teams that do this stop having "the model got worse" mysteries, because every behaviour change traces to a commit. Teams that don't are editing f-strings in a service file and deploying vibes. The tooling required is nearly nothing — a directory, a loader, and fifty test cases.

Prompts live in files, not string concatenation

The baseline sin is prompt fragments scattered across the codebase — half the instructions in a constant, some appended in a helper, a conditional paragraph glued on in the request handler. Nobody can answer "what exactly did the model see?", which makes every incident unreproducible.

One directory, one file per prompt, metadata up top:

---
# prompts/ticket-summary.md
version: 7
variables: [ticket_body, product_area]
---
Summarise the support ticket for an engineer triaging bugs.
Product area: {{product_area}}

<ticket>
{{ticket_body}}
</ticket>

Output exactly three bullet points: symptom, suspected component, severity.

The loader should be strict — declared variables only, and it raises on a missing or extra variable instead of silently rendering {{product_area}} into production traffic:

def render(name: str, **vars) -> str:
    tmpl = load(name)
    missing = set(tmpl.variables) - vars.keys()
    extra = vars.keys() - set(tmpl.variables)
    if missing or extra:
        raise PromptVariableError(name, missing, extra)
    return tmpl.render(**vars)

Twenty lines of loader buys you the property that matters: the rendered prompt is a pure function of (file version, variables), so any production request can be replayed exactly.

Version like code, attribute like code

Log a prompt identifier — name plus version, or a content hash — on every request, attached to the trace. When quality dips on Tuesday, the question "did a prompt change Monday?" becomes a query instead of an archaeology project. Tools like Langfuse will manage prompt versions and link them to traces for you, but git plus a hash in the log line gets you 80% of the value with zero new infrastructure.

Two habits complete it: pin versions per environment (prod runs v7 explicitly; staging runs latest), and keep a one-line changelog entry per version stating the intent of the change — "v7: stop severity inflation on cosmetic bugs" is what makes v9's regression diagnosable.

Regression tests: a golden set per prompt

Every prompt that matters gets a small eval: 30–100 real inputs with expected properties, run in CI whenever anything under prompts/ changes. Assertions come in tiers, cheapest first:

  1. Deterministic checks. Output parses, schema validates, has exactly three bullets, contains no PII, stays under the length cap. These catch a surprising majority of regressions and cost nothing.
  2. Content checks. Expected strings or labels for inputs with known answers.
  3. LLM-as-judge, only for genuinely open-ended quality, and only after you've calibrated the judge against human labels.

Cost is a non-issue: 100 cases against a small model is pennies per run. The discipline that makes it work is the same as any test suite — every production incident adds a case, and a prompt change that fails the suite doesn't merge, no matter how good it looks in the playground.

Review prompt diffs like code diffs

A prompt PR deserves a real review, and the reviewer checklist is short but specific:

  • Contradictions. The most common prompt bug at scale: line 12 says "always include severity", the new line 40 says "omit metadata for cosmetic issues". Models resolve contradictions arbitrarily, which presents as flakiness.
  • Stale examples. Few-shot examples that no longer match the updated instructions actively teach the old behaviour. Instructions changed → examples must change; treat it like updating tests with the code.
  • Edit position. Provider prompt caches match on exact prefixes — editing the top of a long, stable prompt invalidates the cached prefix and can multiply serving cost overnight. Volatile content (dates, user context, retrieved data) belongs at the end, after the stable instructions; the mechanics are covered in context window management. A cost-per-request graph in review catches what the eye misses.
  • Scope creep. Every incident adds a defensive sentence, and after a year the mega-prompt has 40 rules the model weighs equally. Prompts need refactoring — consolidate, delete rules the eval no longer proves necessary, and split genuinely different tasks into different prompts.

Anti-patterns worth naming

  • String soup: prompt fragments assembled across three modules. You can't test what you can't render standalone.
  • Timestamp interpolation: f"Today is {datetime.now()}" at the top of the system prompt breaks caching and reproducibility in one line. If the model needs the date, inject it at the end, truncated to the granularity you need.
  • Branching in code instead of variants: if premium: prompt += ... creates 2^n untested combinations. Named variants — ticket-summary-premium.md — are testable, diffable, and enumerable.
  • Playground-only iteration: a prompt validated on three hand-typed inputs, shipped to a distribution of thousands. The golden set exists precisely because the playground sample is unrepresentative.

What I'd do

This week: move prompts into prompts/, add the strict loader, log a version with every call. Next week: 50 golden cases for your highest-traffic prompt, wired into CI with deterministic assertions. After that it's culture, not tooling — prompt changes get PRs, PRs get the four-point review, incidents become test cases. None of this is novel; that's the argument. The industry spent a decade learning to treat config as code, and prompts are config with a particularly expensive interpreter.