When Google indexes 8,000 of your 48,000 pages, you do not have 40,000 separate page problems. Google classifies URL patterns, and it has decided that one or two of your templates are not worth indexing — a verdict it applies to every page sharing the template. The fix happens at the pattern level: add data the template genuinely lacks, or delete the pattern. Resubmitting URLs, re-pinging sitemaps, and mashing "Request indexing" do nothing, because none of them change the classification. This site generates roughly 54,000 pages from 3,550 app records, so partial indexation is an operational metric I watch weekly, and everything below comes from moving it.
Read the refusal codes correctly
The Pages report in Search Console (Indexing → Pages) buries the diagnosis in status names that sound interchangeable but are not:
| Status | What Google is telling you | The lever |
|---|---|---|
| Discovered — currently not indexed | Knows the URL, won't spend a crawl on it | Internal links, crawl priority, site reputation |
| Crawled — currently not indexed | Fetched it, judged it not worth keeping | Template quality — this is the thin-content signature |
| Duplicate without user-selected canonical | Clusters it with another page | Consolidate or genuinely differentiate |
| Alternate page with proper canonical | Working as intended | Nothing — leave it alone |
"Crawled — currently not indexed" stacking up under one URL pattern is the classic thin-content-at-scale symptom. Google paid the crawl cost, looked at the page, and declined. "Discovered — currently not indexed" at scale is a different disease — a crawl-priority problem you attack with internal links and honest sitemaps rather than with content changes.
Google judges the template, not the URL
The strongest evidence is how indexation moves: in whole sections at once. When I added three data fields to this site's comparison template (measured idle RAM, install-method matrix, a generated verdict line), indexation for that section went from roughly 31% to 58% over about ten weeks — tens of thousands of pages, one template change, no per-page work. Nothing else on the site moved. The corollary is blunt: your 40,000 ignored pages are probably three patterns, and you can fix three things.
Measure indexation per pattern, not per site
A site-wide indexation number hides everything useful. Two mechanics make per-pattern measurement nearly free:
- Shard sitemaps by template — one file for app profiles, one for comparisons, one for the blog. The Pages report filters by sitemap, so each shard becomes a per-template indexation dashboard. The setup is covered in sitemap sharding.
- For finer resolution, the URL Inspection API allows 2,000 requests per day per property. A random sample of 300–400 URLs per pattern gives you an indexation estimate within a few points, refreshed weekly:
# random per-template samples for the URL Inspection API
shuf -n 400 urls-comparisons.txt > sample-comparisons.txt
shuf -n 400 urls-apps.txt > sample-apps.txtThe threshold I use, same as in the programmatic SEO playbook: a section under ~30% indexation has a pattern problem, not a page problem.
Four levers that move a section
Non-derivable data. A page earns indexing when it contains facts a searcher cannot reconstruct from the two pages it aggregates. Restating two spec sheets side by side is derivable; measured resource usage, a difficulty rating, a generated verdict, and a migration note are not. Five or more unique data points per page is the working floor.
Internal links. Pages with fewer than three inbound internal links index at a fraction of the rate of well-linked ones, template quality held equal. This is the cheapest lever on the list — see internal linking at scale for the mesh that fixed it here.
An editorial layer on the head. Hand-write verdicts for the few hundred pages in a pattern that carry real search volume. Google's assessment of a pattern appears to weight its most-visited members, so lifting the head lifts the tail's odds.
Pruning. Delete the bottom of the pattern. A 410 frees crawl budget immediately; noindex still gets crawled occasionally for months. When I removed ~6,000 zero-demand comparison pages, indexation of the surviving pages in that section rose without any other change — Google reallocates attention within a site. Expect the effect over weeks, not days.
Levers that do not work
LLM paraphrase. Rewriting template output so each page "reads unique" is exactly what the scaled content abuse policy targets — Google's spam policies name scaled content generation regardless of how it was produced, and semantic clustering catches paraphrase that string matching misses. You spend money to look more like spam.
Fake freshness. Bumping lastmod and visible dates on unchanged pages teaches Google to distrust your dates site-wide. The honest pipeline is cheap — see what dateModified actually does.
Word-count padding. A 400-word intro above a spec table doesn't make the page less thin; it makes the answer harder to find. Thinness is about information, not length.
The Indexing API. It's for job postings and livestreams. Using it for regular pages does nothing durable and risks losing access.
What I'd do
Shard sitemaps by template this week so you can see per-pattern indexation at all. Then apply thresholds quarterly: a pattern under 10% after six months gets deleted; 10–40% gets the data-and-links treatment — new non-derivable fields plus a real internal-linking pass; above 60%, invest in hand-written verdicts for the head pages and leave the tail to compound. Track the numbers weekly, act quarterly, and budget a full quarter before judging any change — indexation is a slow instrument, but it is an instrument.