DSPy

Last reviewed Aug 26, 2026 · 750 words

DSPy is not something you deploy. It is a Python library (pip install dspy) that sits where LangChain would in your stack: between your application code and whatever model server you run. What it changes is the unit of work. Instead of hand-editing a prompt string until the model behaves, you declare what goes in and what comes out, and DSPy writes the prompt, tests it against your examples, and rewrites it when your metric says it should. With 37,598 GitHub stars and an MIT licence it is the most-used framework built on this idea, and the reason it belongs on a self-hosting site is that its optimizers are the one tool I know that gets an 8B local model producing output you would previously have paid a frontier API for.

It runs against Ollama with one line

DSPy routes model calls through LiteLLM internally, so any OpenAI-compatible or Ollama endpoint works without an adapter:

import dspy
lm = dspy.LM("ollama_chat/llama3.1", api_base="http://localhost:11434", api_key="")
dspy.configure(lm=lm)

The library itself needs about 1 GB of RAM and no GPU; the model server carries the weight. If you already front your models with a LiteLLM proxy, point api_base there and every DSPy program inherits your routing and spend limits.

Signatures replace prompts, modules replace chains

A signature is a typed description of the task, and a module is the strategy for solving it:

class Triage(dspy.Signature):
    """Classify a support ticket."""
    ticket: str = dspy.InputField()
    priority: str = dspy.OutputField(desc="one of low, medium, high")

triage = dspy.ChainOfThought(Triage)
print(triage(ticket="Login page returns 500 for every user").priority)

dspy.Predict, dspy.ChainOfThought and dspy.ReAct are the three modules you will use 90% of the time. The mental shift takes about a day: you stop asking "what words make the model comply" and start asking "what examples and metric define correct". Modules compose like ordinary Python, so a retrieval step feeding a classifier feeding a summariser is a class with three attributes and a forward method.

The optimizer is the product, and it costs tokens

Everything above you could do with raw API calls. The reason to adopt DSPy is compilation: hand it 30 to 300 labelled examples and a metric function, and an optimizer such as BootstrapFewShot or MIPROv2 searches over few-shot demonstrations and instruction wording until the metric climbs. On a local model this is free in money and expensive in time; a MIPROv2 run over a few hundred examples can mean thousands of model calls, which on one consumer GPU is an afternoon. Against a paid API the same run is a real bill, so start with BootstrapFewShot (cheap, often good enough) and only escalate when it stalls. Save the compiled program with program.save("triage.json") and load it at runtime; the optimization happens once, offline.

Where it beats hand prompting and where it doesn't

It wins when you have a measurable task: classification, extraction, structured question answering over your own documents, anything where you can write def metric(example, prediction). It loses on open-ended generation with no metric, where you end up hand-tuning prompts anyway with an extra layer in the way. It also assumes you are comfortable in Python; there is no UI, no server, no YAML. Trace the calls with Langfuse once a pipeline reaches production, because DSPy's own dspy.inspect_history() is a debugging aid, not observability.

The conceptual tax is real

The project's own caveat is the honest one: the learning curve is conceptual, not syntactic. Signatures, modules, optimizers, metrics and example sets are five new nouns before you get a result, and earlier versions renamed things often enough that older tutorials mislead. Read the current docs at dspy.ai rather than a 2023 blog post, and expect the first useful pipeline to take a weekend rather than an hour.

What I'd do

Install it in a venv next to Ollama, rewrite one existing prompt as a signature plus ChainOfThought, and build a 50-example dev set from real inputs. Run BootstrapFewShot against that set with a local 8B model and compare the metric before and after. If the number moves, you have found the workflow; if it doesn't, the task probably lacks a metric and plain prompting is the right tool. Either way you will have spent one evening and zero dollars finding out.