Synthetic training data fails in one specific, measurable way: each generation round narrows the distribution, the tails disappear, and by round three your fine-tune is confidently mediocre at everything unusual. Three controls prevent it — engineered diversity at generation time, aggressive deduplication against both training and eval data, and an anchor of real human data that I keep above 20%. Teams that run all three ship useful models from 90% synthetic corpora; teams that skip one discover the problem in production, because they usually skipped the evals too.

Why collapse happens

Sampling from a model oversamples its modes. Generate 100,000 customer support conversations from one prompt and you get the same eight complaint archetypes in the same register with the same three names, because that's where the probability mass sits. Train on that and your model sharpens around the modes while the tails — the weird tickets, the angry-but-terse customers, the two-languages-in-one-message cases — get diluted toward zero. Recurse and it compounds; Shumailov et al.'s 2024 Nature paper on models collapsing when trained on recursively generated data demonstrated the mechanism formally, but you don't need the recursion to get hurt. One careless generation round already shifts your distribution, and production traffic lives in the tails.

The practical symptom, before any benchmark moves: falling variance. If the mean pairwise cosine similarity of a 1,000-example sample (any decent embedding model) is creeping up across generation batches, your generator is looping. I treat ~0.75 mean pairwise similarity as the alarm line for conversational data.

Diversity is engineered, not sampled

Temperature is not a diversity strategy — it adds noise within the modes, not coverage across them. Diversity comes from conditioning each generation on explicitly varied attributes. Build a seed grid and iterate over it:

axes = {
    "topic":      load_taxonomy("support_topics.yaml"),   # 120 leaf nodes
    "persona":    sample_personas(k=500),                 # age, register, patience
    "difficulty": ["routine", "edge-case", "adversarial"],
    "format":     ["short", "multi-turn", "code-attached"],
}
# 120 * 500 * 3 * 4 = 720k unique seed combinations
for seed in stratified_sample(axes, n=50_000):
    prompt = render(TEMPLATE, **seed)

The persona axis matters more than people expect — it's the insight behind the Persona Hub line of work (a billion synthetic personas as diversity seeds): varying who is speaking moves vocabulary, structure, and error patterns all at once. Ground generations in real material where you can (seed each sample with a real document, log line, or anonymised ticket fragment); grounded generation inherits the diversity of the source corpus instead of the generator's imagination. Then measure: distinct-2 n-gram ratio and embedding dispersion per batch, tracked like any other CI metric.

Dedup like you mean it

Three passes, in order of increasing subtlety, with typical removal rates from my last pipeline in parentheses:

  • Exact and near-exact: hash normalised text (3%).
  • Near-duplicate: MinHash/LSH at ~0.8 Jaccard on character shingles — synthetic data is far more self-similar than web text, so expect real losses here (22%).
  • Semantic: embedding clusters with a cap per cluster, catching paraphrase-loops MinHash misses (another 8%).

A third of the corpus gone before training is normal and good; you were about to pay GPU hours to teach the model the same sentence 40,000 times.

The non-negotiable pass is decontamination against your eval sets: 13-gram overlap (the convention popularised by GPT-3's cleanup) between every training candidate and every eval item, candidate loses. Synthetic pipelines are especially prone to this because the generator has often seen your eval benchmarks; a contaminated eval will tell you the fine-tune is wonderful right up until users disagree. If you don't yet have eval sets worth protecting, fix that first — evals before vibes is a prerequisite for this whole enterprise, not a nice-to-have.

Filter with a judge, but calibrate the judge

An LLM judge scoring each sample against a written rubric (factuality, instruction-adherence, realism, format) should be rejecting 30–50% of raw generations; a judge that passes 95% is rubber-stamping, not filtering. But judges drift and flatter, so calibrate before trusting: hand-label 100–200 samples, measure agreement (Cohen's kappa above ~0.6 is workable), and check the classic biases — length preference and self-preference (a judge from the same model family scores its sibling's output high). Use a different model family for judging than for generation, randomise presentation, and re-calibrate whenever you change either model.

The human anchor and where to spend it

Keep real human data above 20% of the training mix, and don't spend your human budget uniformly — spend it where synthetic data is weakest. Concretely: have humans write or correct the samples your judge rejected and your hardest eval slices, then feed those back as grounding seeds. This is active learning with a generation loop attached, and it's why a 90/10 synthetic/human corpus with well-placed humans beats a 70/30 one with random humans. If the end goal is a small specialist model, the same corpus discipline is what makes distillation work — a student model amplifies whatever bias the teacher corpus carries.

Measure before and after, on real data only

Evaluate exclusively on human-authored held-out data. Evaluating on held-out synthetic data measures how well you learned the generator, which is precisely the failure you're guarding against. Report variance and slice minimums, not just the mean — collapse shows up first as the worst slice getting worse while the average holds. For a LoRA fine-tune on consumer hardware, the eval run costs minutes; there is no excuse to skip the before/after.

What I'd do

Seed-grid generation with personas and grounding documents; distinct-2 and embedding-dispersion tracked per batch with an alarm at 0.75 mean pairwise similarity; exact, MinHash (0.8), and semantic dedup; 13-gram decontamination against every eval set; a cross-family judge calibrated on 200 hand labels rejecting at least 30%; real data above 20%, targeted at judge-rejects and weak slices; eval on human data only, watching the worst slice. It's a week of pipeline work, and it's the difference between synthetic data as a multiplier and synthetic data as a very expensive way to photocopy your model's biases.