git-annex

Last reviewed Sep 13, 2026 · 865 words

git-annex is a command-line tool, not a service. There is no container to run, no port to open and no web interface worth the name, and the reason it belongs on a self-hosting directory is that it solves a problem the sync tools do not: knowing, with certainty, how many copies of each large file exist and where. Git tracks the file names and a checksum; the content lives in whichever drives, servers and cloud buckets you have told it about, and git annex whereis lists them. For a 4 TB archive of photos, raw video, or research data spread across a NAS, two USB drives and an S3 bucket, that is the feature, and nothing else does it as well.

The mental model in one paragraph

In a normal Git repository, git add copies file content into the object store. In an annexed repository, git annex add moves the content into .git/annex/objects, keyed by checksum, and commits a small pointer in its place. The pointer is what gets pushed and pulled; the content moves only when you say git annex get, copy or move. Every repository, whether a laptop clone, a bare repository on a server, or a "special remote" such as an S3 bucket, is a place content can live, and a hidden git-annex branch records which places hold which files. Drop content from your laptop with git annex drop and the tool refuses unless enough other copies are verified to exist. That refusal is the whole safety story.

The six commands you will actually use

git init archive && cd archive && git annex init "laptop"
git annex add .                      # move content into the annex, commit pointers
git annex sync --content             # exchange pointers and content with remotes
git annex get video/2024/            # fetch content for a subtree
git annex drop --auto video/2023/    # free space, keeping numcopies satisfied
git annex whereis raw/IMG_4021.CR2   # list every place holding this file

Set git annex numcopies 2 early and git annex fsck periodically; fsck verifies checksums on the content you hold and flags any file with fewer known copies than the policy demands. Preferred content expressions let each repository state what it wants, so the NAS says "everything", the laptop says "only this year", and sync --content does the routing without you.

Special remotes are where self-hosting comes in

Any Git remote you host, on Gitea or a bare repository over SSH, carries the pointers and can carry content. Special remotes carry content only, and the list covers most storage a self-hoster owns: a plain directory on an external drive, rsync to any SSH host, S3-compatible buckets including a local MinIO, WebDAV against Nextcloud, and anything rclone can reach. Each can be encrypted:

git annex initremote b2 type=rclone target=b2-archive encryption=shared
git annex copy --to b2 --auto

With encryption=shared or hybrid, the remote holds encrypted chunks and never sees file names or contents, so a cloud bucket becomes an off-site copy you do not need to trust. That is the pattern that lets one tool be both the catalogue and the 3-2-1 backup for an archive that is too large and too slowly changing for Borg or restic to be the natural fit.

Where it hurts

The catalogue's "Hard" rating is earned. Annexed files are symlinks by default, which surprises every application that expects a plain file, and switching to the mode that stores plain files trades that for slower operations. Windows support exists and is second-class. The vocabulary (numcopies, preferred content, wanted, groups, the git-annex branch) is large, and mistakes at the start, such as adding a directory to Git rather than to the annex, are tedious to undo. The assistant and its browser interface try to hide all of this and mostly succeed for a two-machine sync, but people who reach for git-annex usually want the command line and the control it offers. Budget a weekend reading the walkthrough on the project site before trusting it with anything you cannot re-create.

What it is not

It is not Syncthing, which mirrors folders between devices continuously and is what most people wanting "my files on every machine" should run. It is not Git LFS, which stores every large file on one server and gives you no location tracking. And it is not a backup program: it will keep your copies counted, but versioning of changed content, deduplication and pruning are things Borg and restic do far better.

What I'd do

Use git-annex for one class of data: large, mostly write-once collections where losing a file is unacceptable and the copies are scattered by design. Photos and raw video, audio masters, datasets, a scanned document archive. Put the Git remote on your own forge, set numcopies 2, add an encrypted rclone or S3 special remote off-site, and run git annex fsck --from each remote twice a year. For everything that changes daily, keep Syncthing and a real backup tool, and let git-annex be the ledger for the archive.