nifi
Logstash
| nifi | Logstash | |
|---|---|---|
| 35 | 5 | |
| 4,410 | 14,006 | |
| 1.8% | 0.3% | |
| 9.9 | 9.6 | |
| 7 days ago | 9 days ago | |
| Java | Java | |
| Apache License 2.0 | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
nifi
- FLaNK Stack Weekly 19 Feb 2024
- Ask HN: What are some unpopular technologies you wish people knew more about?
- FLaNK Stack Weekly for 13 November 2023
-
Ask HN: What low code platforms are worth using?
Apache NIFI (https://nifi.apache.org/).
It uses the concept of Flow-based programming. Also its so underacknolged but this tool is very flexible. I have used as an Event Bus all the 3rd-Party Integrations.
- Apache Nifi: easy to use, powerful, reliable system to process, distribute data
- Tool decision - What architecture would you choose and why?
-
Help with choosing techstack for a new DE team
Presently setting up Apache Nifi + Apache MiNiFi for the ETL portion of my work. NiFi was easy enough to figure out; but the docs for MiNiFi have been a pain due to differences between the Java and C++ versions. I then entirely configured it with the Java version so that it was easier to search for answers for the MiNiFi yaml syntax.
-
MS SQL Change Data Capture
Found it
-
Is there something like airflow but written in Scala/Java?
Apache Camel Apache Nifi Spring Cloud
-
Json splitting and Rerouting (new to nifi)
NIFI, like most Apache projects does most of its discussion on its mailing lists, but also has a slack.
Logstash
-
Simple way to centralize my server logs?
There are probably too many to chose from. Logstash, Promtail, Vector, Filebeat, FluentD, Logagent and probably many more
-
Help with parsing log in logstash into uniqe lines
https://github.com/elastic/logstash/issues/1650 See if this works.
-
Looking for open source projects that use data pipelines and big data flows
Is logstash the kind of project you are looking for? https://github.com/elastic/logstash
-
Top Observability tools for DevOps Engineers and SREs
Logstash is completely free and the source code is available freely on GitHub.
-
Disable Console Output in Logstash 7.10
What log level are you set to in logstash.yml? The default log level is "info" which is pretty chatty. I set mine to "error". See https://github.com/elastic/logstash/blob/master/config/logstash.yml#L248.
What are some alternatives?
superset - Apache Superset is a Data Visualization and Data Exploration Platform
Logback - The reliable, generic, fast and flexible logging framework for Java.
meltano
Apache Log4j 2 - Apache Log4j 2 is a versatile, feature-rich, efficient logging API and backend for Java.
meltano - Meltano: the declarative code-first data integration engine that powers your wildest data and ML-powered product ideas. Say goodbye to writing, maintaining, and scaling your own API integrations.
graylog - Free and open log management
Pandas - Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more
Flume - Mirror of Apache Flume
Metabase - The simplest, fastest way to get business intelligence and analytics to everyone in your company :yum:
Logbook - An extensible Java library for HTTP request and response logging
Apache Cassandra - Mirror of Apache Cassandra
SLF4J - Simple Logging Facade for Java