Concourse CI

Last reviewed Sep 13, 2026 · 948 words

Everything that is unusual about Concourse comes from one decision: a pipeline is not a script attached to a repository, it is a graph of resources (a Git repo, a container image, an S3 object, a timer, a semver counter) and jobs that consume some and produce others. Understand that and the rest of the system is consistent, elegant, and reproducible. Skip it and Concourse feels obstinate, because there is no .concourse.yml in your repo, no "just run this shell script on push" button, and no way to click a job into existence. It is the right CI for a platform team maintaining long-lived delivery pipelines; it is the wrong first CI for almost everyone else, and I say that as someone who likes it.

Resources are the whole model

A resource is anything with versions that can be checked, fetched, and optionally pushed. Concourse polls each resource on an interval, and a job that declares get: repo, trigger: true runs whenever a new version appears. The job's tasks run in fresh containers with only the inputs you named mounted in, so a build cannot depend on a file left over from the last run; that is the reproducibility guarantee, and it is real. Outputs flow to put: steps, which push a built image or a tag back out as a new version of some resource, which may trigger the next job. A pipeline of 12 jobs is 12 small containers with explicit data flow, drawn live as a graph in the web UI that has become the project's signature.

The price is that a resource type must exist for everything you touch. Git, registry-image, time, s3, semver, and pool ship in the box; anything else is a community resource type or one you write yourself as a container with three scripts.

Three containers and the fly CLI

The quickstart from concourse-ci.org is honest about the shape: PostgreSQL, a web node (API, scheduler, and the TSA that workers register with over port 2222), and at least one worker running privileged so it can create containers.

services:
  db:
    image: postgres
    environment:
      POSTGRES_DB: concourse
      POSTGRES_PASSWORD: concourse_pass
      POSTGRES_USER: concourse_user
  concourse:
    image: concourse/concourse
    command: quickstart
    privileged: true
    depends_on: [db]
    ports: ["8080:8080"]
    environment:
      CONCOURSE_POSTGRES_HOST: db
      CONCOURSE_POSTGRES_USER: concourse_user
      CONCOURSE_POSTGRES_PASSWORD: concourse_pass
      CONCOURSE_POSTGRES_DATABASE: concourse
      CONCOURSE_EXTERNAL_URL: http://localhost:8080
      CONCOURSE_ADD_LOCAL_USER: test:test
      CONCOURSE_MAIN_TEAM_LOCAL_USER: test
      CONCOURSE_WORKER_BAGGAGECLAIM_DRIVER: overlay

quickstart bundles web and worker into one container for evaluation. Real installs split them, generate keys with concourse generate-key, and put workers on separate hosts. Then everything happens through fly:

fly -t home login -c http://localhost:8080 -u test -p test
fly -t home set-pipeline -p site -c pipeline.yml
fly -t home unpause-pipeline -p site

There is no other way to create a pipeline. That is by design, and it means pipeline changes go through Git and a set-pipeline step in your own repo if you want them versioned, which most serious installs do.

The 2 GB is real and the workers are hungry

The catalogue's 2,048 MB minimum covers web plus Postgres plus one idle worker. Workers do the actual builds and want whatever your builds want, plus disk for the overlay volumes that every get and task creates; a worker on a 20 GB disk fills up in a week of image builds unless you let Concourse's garbage collection run and give it room. Postgres is not optional and should be one you back up, since every pipeline, build log, and resource version lives there; the Postgres for everything approach of a single well-tended instance suits it. The Helm chart is the vendor's preferred production path, and if you already run Kubernetes it is the least painful one.

Who runs it and why they stay

Concourse's users are platform teams at companies that ship the same artefact through many stages: build, test, publish image, deploy to staging, run smoke tests, promote. Long-lived pipelines with fan-in and fan-out, where the graph view earns its keep and where "every task in a clean container" prevents the drift that kills a five-year-old Jenkins. Cloud Foundry's ecosystem grew it, and it still shows in the resource-first thinking. For that audience, the learning curve buys a system that does not rot. If your need is "run the tests on every push to 15 repositories", you will spend the curve and receive nothing for it.

The alternative that fits most self-hosters

Woodpecker CI does container-per-step builds from a .woodpecker.yml in each repository, integrates with Gitea and Forgejo in ten minutes, and runs in 256 MB. For per-repository CI it is strictly simpler than Concourse and no less reproducible. Jenkins is the other direction: more plugins, more mutable state, more of the drift Concourse exists to prevent. The full field is in CI/CD, and the honest ranking for a homelab or small team is Woodpecker first, Concourse only when you find yourself drawing a delivery graph on a whiteboard.

What I'd do

Run Concourse only if you have 3 or more environments an artefact must move through and someone willing to think in resources. In that case: web and Postgres on one host, two workers on separate machines with 50 GB disks each, pipelines versioned in Git and applied by a set-pipeline job, and a time resource driving nightly rebuilds so images never go stale. For everyone else, install Woodpecker next to Gitea this afternoon and revisit Concourse when the pipeline stops fitting in one file.