Kuberhealthy

Last reviewed Sep 13, 2026 · 953 words

Prometheus tells you the pods are Running. Kuberhealthy tells you whether the cluster can still deploy one. That gap is the entire product: instead of scraping metrics about the system, it periodically does the things you care about (roll out a deployment, resolve a DNS name, pull an image, write to a volume) inside disposable check pods and reports pass or fail. On the day a CNI upgrade quietly breaks pod networking, every node metric stays green and only the synthetic check goes red. It is Kubernetes-only and 2,268 stars deep, so if you run Docker Compose you can stop reading, and I say more about that below.

What a check actually is

A check is a KuberhealthyCheck custom resource with three things in it: a run interval, a timeout, and a pod spec. On each interval the operator launches the pod, the pod does its work, and when finished it calls back to the Kuberhealthy API with ok: true or a list of error strings. The operator aggregates every check's latest result into a single JSON status page on its service and into Prometheus metrics on /metrics, so the cluster's health becomes one scrape target. The state is stored in the CR status, which means kubectl get khchecks -n kuberhealthy shows you the last outcome without opening a browser.

The elegance is that a check is just a container image. It has no SDK dependency beyond "POST a JSON body to an environment-provided URL before you exit". The project ships client libraries for Go, Python, and JavaScript that reduce this to two lines, but a shell script with curl counts.

Install it in ten minutes on k3s

The Helm chart is the supported path. At last check the repo lives on the project's GitHub Pages:

helm repo add kuberhealthy https://kuberhealthy.github.io/kuberhealthy/helm-repos
helm repo update
helm install kuberhealthy kuberhealthy/kuberhealthy \
  --namespace kuberhealthy --create-namespace

It runs as a small Go operator, 128 MB of memory is the realistic floor, and it creates its CRD on install. Confirm with kubectl -n kuberhealthy get pods and then port-forward the kuberhealthy service to see the status JSON. If you already run the Prometheus operator, enable the ServiceMonitor value in the chart and the metrics show up in Grafana with no further wiring; on a plain Prometheus install, add an annotation-based scrape or a static target.

The three checks worth running on a homelab cluster

The project ships a dozen or so checks, and most homelabs want exactly three. The deployment check creates a Deployment with a couple of replicas, waits for it to become ready, then deletes it: this is the one that catches broken scheduling, a wedged image registry, or a node that cannot start containers. The DNS resolution check confirms that pods can resolve both an in-cluster service and an external name, which is the failure mode behind most "everything is up but nothing works" evenings. The pod restarts check watches for containers crash-looping in a namespace and fails if any restart more than a threshold in the window.

Add the storage check only if you rely on a CSI driver like Longhorn; it provisions a PVC, writes, reads, and deletes. Skip the network connection and HTTP checks unless you have specific endpoints in mind, because Gatus or Uptime Kuma do outside-in endpoint checks better and from outside the failure domain, which matters. Kuberhealthy proves the cluster can do things; those tools prove the world can reach it.

Writing your own check is a small Go or Python program

The cons list on the catalogue page says writing checks takes effort, and it does, but the effort is one afternoon. A custom check is a container that reads KH_REPORTING_URL from its environment, does whatever business-specific thing you want (log in to your own app, run a database migration dry-run, verify a backup landed in MinIO), and reports. Build the image, push it to your registry, and reference it in a new KuberhealthyCheck with a sensible timeout so a hung check gets killed rather than silently never reporting. Keep the interval honest: a check that deploys workloads every 30 seconds is load, not monitoring. Two to five minutes is the range I use.

If you don't run Kubernetes, stop here

Kuberhealthy has no non-Kubernetes mode and no reason to want one. For a Docker host or a single VM, the same "do the thing, not just measure the thing" idea is covered by Gatus with a scheduled HTTP check against your own endpoints, or by a cron job that exercises the workflow and pushes a heartbeat. The homelab monitoring guide walks through that stack, and the wider monitoring category has the metric-side tools. Also worth saying plainly: on a three-node k3s cluster, Kuberhealthy is a nice-to-have on top of Prometheus, not a replacement. It does not store history, it does not graph, and it does not alert on its own; it emits metrics and expects Alertmanager to do the rest.

What I'd do

Install the chart, keep the deployment and DNS checks, set both to a 2-minute interval, and wire the kuberhealthy_check metric into an Alertmanager rule that fires after two consecutive failures. Put Gatus outside the cluster for the endpoint view. Write one custom check for the single workflow you would be most embarrassed to discover broken by a user, and leave it at that. That configuration has caught more real outages for me than any node-level dashboard, and it costs a container and 128 MB.