Thanos

Last reviewed Aug 26, 2026 · 873 words

Do not deploy Thanos in a homelab. It solves a problem you do not have, and it solves it with 6 cooperating services where you currently run 1. A single Prometheus with --storage.tsdb.retention.time=2y keeps 2 years of homelab metrics in a few GB of disk, with no object store, no compactor and nothing to page you at 3 am. Thanos earns its place when you have several Prometheus servers and need to query them as one, or when a single node's disk stopped being a sane place for the data.

The problem Thanos solves, precisely

Prometheus is deliberately a single-node system. Each instance scrapes, stores locally, and answers queries about what it scraped. That gives you 3 hard limits: retention is bounded by one disk, a query can only see one instance, and if the instance dies you lose the data with it. Thanos wraps Prometheus rather than replacing it. A Sidecar next to each Prometheus uploads finished 2-hour TSDB blocks to an S3-compatible bucket; a Store Gateway serves those blocks back; a Querier fans a PromQL query out to every sidecar and store, deduplicates replicas and returns one answer. Point Grafana at the Querier and it looks like a single, enormous Prometheus. That is the whole trick, and it is a good one.

Six components where you had one

Naming them makes the cost concrete. Sidecar (one per Prometheus), Query, Store Gateway, Compactor, plus the optional Ruler for alerts across the global view and Receive for a push model. Each is the same thanos binary with a different subcommand, each listens on gRPC 10901 and HTTP 10902 by default, and each has its own failure modes. The Compactor is the sharpest edge: exactly one may run against a bucket at a time, and it is the process that downsamples raw data to 5-minute and 1-hour resolutions so that a 1-year graph does not read a billion samples. Run two Compactors against one bucket and you corrupt it. Forget to run one and your bucket grows without bound and long-range queries crawl. The catalogue calls the difficulty "Hard" and the architecture "complex"; that is accurate, not cautious.

Object storage is the point, and the bill

Everything durable lives in the bucket, which is why retention becomes "unlimited": object storage costs roughly a cent or two per GB-month at the big providers and metrics compress well. For self-hosters the bucket is usually MinIO or SeaweedFS, which means you now also operate an S3 service, and its durability is your problem rather than Amazon's. Two Prometheus settings are mandatory before the Sidecar will upload anything: --storage.tsdb.min-block-duration=2h and --storage.tsdb.max-block-duration=2h so blocks are never locally compacted, and a distinct set of external_labels per instance so the Querier can tell replicas apart. Miss the labels and deduplication silently does the wrong thing.

What the same money buys elsewhere

If your actual need is "keep metrics longer on one machine", raise Prometheus retention or set --storage.tsdb.retention.size=50GB and stop. If it is "one query endpoint over several instances", Prometheus federation is crude but works for a couple of sources. If you genuinely need long retention plus a global view but only have a small team, VictoriaMetrics is a single binary that accepts remote-write from any number of Prometheus servers, uses less disk per sample, and needs no object store. Mimir is Grafana's own take on the same idea and is closer to Thanos in operational weight. The monitoring category lists the field; the Grafana vs Prometheus piece untangles which layer does what if the stack is new to you.

When Thanos is the right answer

Three signals. You already run Prometheus in more than one cluster or site and people keep asking for a cross-site dashboard. Your retention requirement is measured in years and audited, so "a bigger disk" is not a durable answer. And you have an object store you trust, whether that is a cloud bucket or a MinIO deployment someone is actually responsible for. With all three true, Thanos is the most battle-tested option there is: it has been a CNCF project since 2019, 14,000 stars on GitHub, and it is what a lot of the large Kubernetes shops run. The Helm charts are mature, and on Kubernetes with the Prometheus Operator the Sidecar is a few lines of manifest.

What I'd do

For one server or one rack: Prometheus alone, retention set to 2 years, restic the data directory nightly, done. For two or more sites that need one dashboard but no compliance-grade retention: VictoriaMetrics single-node receiving remote-write, because it is a tenth of the operational surface. Thanos only when I have Kubernetes, an object store I am not scared of, and a colleague who will learn the Compactor with me. In that last case, start with Sidecar plus Query only, prove the global view works, and add the Store Gateway and Compactor once the bucket has data worth serving.