Fluent Bit

Last reviewed Aug 26, 2026 · 941 words

Fluent Bit is the agent, not the destination. It tails files, reads journald and Docker output, tidies each line into a structured record, and hands it to something that stores and searches logs. It has no UI, no search, no retention and a resident footprint in the tens of megabytes. Once that role is clear it is the easiest part of a self-hosted logging stack. Treated as the stack itself, it disappoints, because there is nothing to look at.

Where it sits: one per host, feeding one store

Every log pipeline has three parts: a shipper on each machine, a store, and a place to query. Fluent Bit is the shipper. The store is your choice, and the output plugin you pick decides it: loki for Loki with Grafana on top, opensearch for an OpenSearch cluster, gelf for Graylog, forward for another Fluent daemon, stdout while you are debugging. You run one instance per host, either as a container with the host's log directories mounted read-only or as the distro package, and on Kubernetes as a DaemonSet. Real-world memory sits between 15 and 40 MB depending on how many inputs are buffering; the 32 MB listed minimum is a fair floor, not a marketing number.

Fluent Bit against the other three shippers

Fluentd is the older sibling and the reason Fluent Bit exists; the same foundation maintains both, and Fluent Bit is the one you want unless a specific Fluentd plugin is load-bearing for you. Vector's transform language is genuinely nicer for reshaping records, at the cost of 3 to 5 times the memory. Alloy matters mostly because Promtail, Loki's original agent, is deprecated in favour of it, so anyone who was going to reach for Promtail now picks between Alloy and Fluent Bit. For a Loki store on a small box, Fluent Bit wins on footprint and loses on nothing that matters.

The config that ships journald and Docker logs to Loki

The classic format is indentation-sensitive and terse, which is the honest complaint against it. This is the whole file for a Docker host:

[SERVICE]
    flush         5
    log_level     info
    storage.path  /var/lib/fluent-bit/storage

[INPUT]
    name           systemd
    tag            host.journal
    read_from_tail on
    storage.type   filesystem

[INPUT]
    name           tail
    tag            docker.*
    path           /var/lib/docker/containers/*/*-json.log
    parser         docker
    mem_buf_limit  50MB
    storage.type   filesystem

[OUTPUT]
    name         loki
    match        *
    host         loki
    port         3100
    labels       job=fluentbit, host=${HOSTNAME}
    line_format  json
    retry_limit  false

The docker parser ships in the default parsers.conf. Run the fluent/fluent-bit image with /var/log/journal, /run/log/journal and /var/lib/docker/containers mounted read-only and a named volume at /var/lib/fluent-bit/storage. One limitation of tailing Docker's JSON files: the record carries the container ID from the path, not the name. If you want names without extra filters, switch containers to Docker's fluentd log driver pointed at a Fluent Bit forward input on port 24224, which delivers container_name as a field. Newer releases also accept a YAML config format that is much easier to read; use it for anything longer than the file above.

Two lines decide whether you lose logs when Loki is down

By default every input buffers in memory only, and when mem_buf_limit is hit the input pauses. For tail that is harmless, because the file is still there when it resumes. For systemd and forward it means records arriving during the pause are gone. storage.type filesystem on each input, with storage.path set in the service section, spools chunks to disk instead, so a store that is unreachable for an hour costs you nothing. The second line is retry_limit false on the output. The default gives up on a chunk after a single retry and discards it; false retries until it succeeds. Set both before you trust the pipeline, and test it once by stopping Loki for 10 minutes and confirming the gap fills in.

Metrics and traces work, and I would still use them sparingly

The node_exporter_metrics input plus prometheus_remote_write output turns Fluent Bit into a host metrics agent, and the opentelemetry input and output make it a trace forwarder. Both are real and both are fine. On a home lab, though, Prometheus already scrapes node_exporter with less to configure, and traces are rare outside application development. I use the metrics path only on hosts where Fluent Bit is already present and I refuse to add a second agent; for everything else, logs are the job it does best.

What I'd do

Fluent Bit on every host, filesystem storage, retry_limit false, shipping to Loki with exactly two labels: host and job. Resist adding per-container labels in Loki, since every distinct label combination is a separate stream and cardinality is what makes Loki slow. Reach for Fluentd only when a plugin exists nowhere else, and for Vector only when you find yourself writing a fourth [FILTER] block to reshape the same record. For the store on the other end, Loki with Grafana if you want cheap and label-based, Graylog if you want fields parsed at ingest and alerts without query-time work.