Logstash

Last reviewed Aug 26, 2026 · 715 words

Do not run Logstash in a homelab unless you have a specific parsing job that Vector or Fluent Bit cannot do. It reserves 1 GB of JVM heap by default, takes 30 to 60 seconds to start, and the stack it was built to feed, Elasticsearch plus Kibana, is itself heavier than most self-hosted logging problems justify. That is the verdict for hobby scale. At work scale the picture flips: nothing else has its filter language, its plugin library, or 15-odd years of grok patterns for every log format ever invented, and about 14,900 GitHub stars say plenty of people still need exactly that.

What Logstash is in one paragraph

A pipeline with three stages. Inputs read from files, Beats agents, syslog, Kafka, JDBC databases, S3, HTTP and a hundred other sources. Filters transform each event: grok parses unstructured text into fields, date normalises timestamps, mutate renames and casts, geoip enriches addresses, and a ruby filter runs arbitrary code when nothing else fits. Outputs write to Elasticsearch, OpenSearch, Kafka, files, S3, or almost anywhere. Persistent queues and a dead-letter queue mean an output outage does not lose events. The core is Apache-2.0, written in Java, and Elastic publishes an OSS-tagged Docker image alongside the default one for people who care about the licence of the bundled extras.

A pipeline that works on first start

Put this in pipeline/logstash.conf and mount that directory into the container at /usr/share/logstash/pipeline/:

input {
  beats { port => 5044 }
}
filter {
  grok { match => { "message" => "%{COMBINEDAPACHELOG}" } }
  date { match => ["timestamp", "dd/MMM/yyyy:HH:mm:ss Z"] }
  geoip { source => "clientip" }
}
output {
  elasticsearch { hosts => ["http://elasticsearch:9200"] }
}

Filebeat on a web server ships access logs to 5044, grok turns each line into a dozen typed fields, and Elasticsearch indexes them for Kibana. The monitoring API on port 9600 tells you events per second per stage, which is how you find the filter that is eating CPU. Grok debugging is the skill to learn: Kibana has a built-in grok debugger and the pattern library on GitHub covers Nginx, Postfix, sshd, HAProxy and most of what a homelab emits.

The JVM cost is the number that decides it

jvm.options ships with -Xms1g -Xmx1g. You can lower it to 512 MB for a light pipeline, and it will run, but it will also spend more time in garbage collection and stall when a burst arrives. Compare the alternatives by resting memory in my experience: Vector around 50 MB, Fluent Bit around 20 MB, both starting in under a second and both configured in a few lines of TOML or YAML. Fluent Bit has a smaller filter vocabulary; Vector's VRL language covers most of what grok and mutate do, with better error messages. On a server with 8 GB shared across a dozen services, that 1 GB is the difference between fitting and not.

Where it still earns the memory

Three cases. You already run Elasticsearch or OpenSearch and want the officially supported ingest path with the same release cadence. You have genuinely nasty parsing, multi-line stack traces from three different frameworks, custom application formats, conditional routing by content, where grok plus a ruby filter is faster to write than anything else. Or you need an input the newer agents lack, such as reading rows from a database with the JDBC input on a schedule. In those cases Logstash is not legacy, it is the right tool, and the heap is the price of a language that has solved your problem before.

What I'd do

For a homelab, Loki with Grafana Alloy shipping the logs, or Vector if you want transformation on the way in, and Grafana as the front end; my monitoring and uptime piece walks through that stack, and it fits in the memory Logstash alone would consume. For a small business with an existing ELK deployment, keep Logstash, give it 2 GB, put Filebeat on the edges rather than Logstash itself, and run persistent queues so an Elasticsearch restart never costs you events. The rest of the monitoring category has plenty of lighter options; this one is for when heavy is what you need.