Fluentd VS vector

Compare Fluentd vs vector and see what are their differences.

Fluentd

Fluentd: Unified Logging Layer (project under CNCF) (by fluent)
Our great sponsors
  • PopRuby - Clothing and Accessories for Ruby Developers
  • WorkOS - The modern identity platform for B2B SaaS
  • InfluxDB - Power Real-Time Data Analytics at Scale
Fluentd vector
25 95
12,489 16,187
0.8% 5.1%
8.0 9.9
14 days ago 6 days ago
Ruby Rust
Apache License 2.0 Mozilla Public License 2.0
The number of mentions indicates the total number of mentions that we've tracked plus the number of user suggested alternatives.
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.

Fluentd

Posts with mentions or reviews of Fluentd. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2023-08-15.
  • Kubernetes Architecture
    4 projects | dev.to | 15 Aug 2023
    Currently, there is no cluster-wide logging. Fluentd can be used to have a unified logging layer for the cluster.
  • Top 11 Splunk Alternatives that you may consider in 2023
    4 projects | dev.to | 6 Feb 2023
    Fluentd is an open-source log management and data collection tool. Just like Logstash, Fluentd uses a pipeline-based architecture. This allows it to collect data from various sources and network traffic and forward it to various destinations.
  • 7 Open-Source Log Management Tools that you may consider in 2023
    4 projects | dev.to | 30 Jan 2023
    Fluentd is a powerful log management tool that offers organizations the flexibility and scalability required to handle large volumes of log data from a variety of sources and transport it to various destinations. Utilizing a flexible and modular architecture, Fluentd allows users to easily add new input and output plugins to integrate with a wide range of systems and applications. It supports a wide range of data sources and destinations, including databases, message queues, and data stores.
  • Substation: Data Pipeline and Transformation Toolkit Written in Go
    2 projects | dev.to | 23 Jan 2023
    Substation is an affordable alternative to products like Cribl (~10x cost savings) and is easier to manage than similar open-source projects such as Logstash and fluentd. It's been used in production by the security team at Brex for 2+ years and is ready for any scale, even beyond 100,000 events per second!
  • Simple way to centralize my server logs?
    8 projects | /r/selfhosted | 4 Jan 2023
    There are probably too many to chose from. Logstash, Promtail, Vector, Filebeat, FluentD, Logagent and probably many more
  • The Everything Guide to Data Collection in DevSecOps
    2 projects | dev.to | 4 Jan 2023
    To alleviate some of the pain, itโ€™s a good idea to use industry standards and tooling like OpenTelemetry (https://opentelemetry.io). For data collection specific to logs, open-source tools like LogStash and Fluentd are also popular.
  • Top 20 Observability Tools Every Startup Should Know About in 2022
    4 projects | dev.to | 21 Sep 2022
    Created and maintained by the creators of fluentd, fluentbit is a lightweight, fast, and scalable logging and metrics processor and forwarder. Built specifically for the cloud and containerized environments, it allows users to collect data from any source, enrich it with filters and forward it to the tool of their choice.
  • Top 5 Open-Source Log Shippers (alternatives to Logstash) in 2022
    3 projects | dev.to | 17 Aug 2022
    This one was built with the idea in mind to structure data in JSON as much as possible. FluentD claims that this method allows for a Unified Logging Layer (in other words, unifying logging infrastructure).
  • How to monitor nginx in Kubernetes with Prometheus
    4 projects | dev.to | 4 Jul 2022
    Fluentd, and its plugin for Prometheus.
  • The 12-Factor App Building Methodology
    3 projects | dev.to | 2 Jul 2022
    Example: Little Johnny's server went down... but why? He tries to check the system's logfile but it's so big and disorganized... maybe he should have treated logs as a continuous event stream and timestamp each event to at least know what happened when. After a while, he discovers that the server went down due to a lack of memory, as it was holding a 20 Gb log file... guess it would be better to keep logs as an external service. Little Johnny then configures his environment to use the open-source Fluentd to centralize different log sources and then route the logfile to an external data warehouse. Next time, debugging production issues will be a way less traumatic experience.

vector

Posts with mentions or reviews of vector. We have used some of these posts to build our list of alternatives and similar projects. The last one was on 2024-03-18.
  • FLaNK AI Weekly 18 March 2024
    39 projects | dev.to | 18 Mar 2024
  • Vector: A high-performance observability data pipeline
    5 projects | news.ycombinator.com | 17 Mar 2024
    Datadog bought Timber Technologies (creators of Vector) two years ago. https://www.datadoghq.com/blog/datadog-acquires-timber-techn...

    Timber definitely intended to just rock out & demolish everything else out there with their agent/forwarder/aggregator tech. But it wasn't a competitive play against OTel, in my humble opinion. Timber's whole shtick is that it integrates with everything, with really flexible/good glue logic in-between. A competent multi-system (logging, metrics, eventually traces) fluentd++. OTel - I want to believe - would have been part of that original vision.

    It's just taking a really really long time. One can speculate how direction & velocity might have changed since the Datadog acquisition. The lack of tracing (anywhere except Datadog, so far) materializing has been a hard hard hard & sad thing to see. OG https://github.com/vectordotdev/vector/issues/1444 and newer https://github.com/vectordotdev/vector/issues/17307

    5 projects | news.ycombinator.com | 17 Mar 2024
    Vector is fantastic software. Currently running a multi-GB/s log pipeline with it. Vector agents as DaemonSets collecting pod and journald logs then forwarding w/ vector's protobuf protocol to a central vector aggregator Deployment with various sinks - s3, gcs/bigquery, loki, prom.

    The documentation is great but it can be hard to find examples of common patterns, although it's getting better with time and a growing audience.

    My pro-tip has been to prefix your searches with "vector dev A recent contribution added an alternative to prometheus pushgateway that handles counters better: https://github.com/vectordotdev/vector/issues/10304#issuecom...

    5 projects | news.ycombinator.com | 17 Mar 2024
  • About reading logs
    2 projects | /r/sysadmin | 28 Sep 2023
    We don't pull logs, we forward logs to a centralized logging service.
  • Self hosted log paraer
    4 projects | /r/selfhosted | 20 Jun 2023
    opensearch - amazon fork of Elasticsearch https://opensearch.org/docs/latestif you do this an have distributed log sources you'd use logstash for, bin off logstash and use vector (https://vector.dev/) its better out of the box for SaaS stuff.
  • Show HN: Homelab Monitoring Setup with Grafana
    6 projects | news.ycombinator.com | 7 Jun 2023
    I think there's nothing currently that combines both logging and metrics into one easy package and visualizes it, but it's also something I would love to have.

    Vector[1] would work as the agent, being able to collect both logs and metrics. But the issue would then be storing it. I'm assuming the Elastic Stack might now be able to do both, but it's just to heavy to deal with in a small setup.

    A couple of months ago I took a brief look at that when setting up logging for my own homelab (https://pv.wtf/posts/logging-and-the-homelab). Mostly looking at the memory usage to fit it on my synology. Quickwit[2] and Log-Store[3] both come with built in web interfaces that reduce the need for grafana, but neither of them do metrics.

    - [1] https://vector.dev

  • Lightweight logging on RPi?
    4 projects | /r/selfhosted | 24 May 2023
    I would recommend that you run vector as a systems service so you don't have to worry about managing it. Here is a basic config to do that - https://github.com/vectordotdev/vector/blob/master/distribution/systemd/vector.service .
  • Monitoring traefik access logs easily
    2 projects | /r/selfhosted | 8 May 2023
    You could have a look at Grafana Loki, it's easy to run (single binary for a small setup). Shipping your logs can be done by Promtail or something like Vector. They're both lightweight log shippers with support for Loki.
  • Ask HN: How to build an image search service?
    2 projects | news.ycombinator.com | 1 Feb 2023

What are some alternatives?

When comparing Fluentd and vector you can also consider the following projects:

graylog - Free and open log management

zipkin - Zipkin is a distributed tracing system

agent - Vendor-neutral programmable observability pipelines.

syslog-ng - syslog-ng is an enhanced log daemon, supporting a wide range of input and output methods: syslog, unstructured text, queueing, SQL & NoSQL.

OpenSearch - ๐Ÿ”Ž Open source distributed and RESTful search engine.

Flume - Mirror of Apache Flume

Lograge - An attempt to tame Rails' default policy to log everything.

Semantic Logger - Semantic Logger is a feature rich logging framework, and replacement for existing Ruby & Rails loggers.

heka - DEPRECATED: Data collection and processing made easy.

fluent-bit - Fast and Lightweight Logs and Metrics processor for Linux, BSD, OSX and Windows

Logstash - Logstash - transport and process your logs, events, or other data

tracing - Application level tracing for Rust.