|3 days ago||3 days ago|
|Apache License 2.0||GNU General Public License v3.0 or later|
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
How to Serve Massive Computations Using Python Web Apps.
1 project | dev.to | 23 Nov 2021
In this demo, we use the request itself as the trigger and begin computation immediately. But it may vary according to the nature of your application. Often, you might have to use a separate pipeline as well. In such scenarios, you may need technologies such as Apache Airflow or Prefect.
Apache Airflow In EKS Cluster
1 project | dev.to | 10 Nov 2021
Airflow is one of the most popular tools for running workflows espeically data-pipeline.
Distributed computing in python??
2 projects | reddit.com/r/learnpython | 9 Nov 2021
AWS MWAA and AWS SES integration
1 project | dev.to | 2 Nov 2021
This problem was already reported in a few Airflow issues and PRs. The fix didn't make the cut for Airflow 2.2 and will be probably there in version 2.3, but because we are talking about MWAA (version 2.0.2), we don't really know when this will be fixed on AWS.
Noobie who is trying to use K8s needs confirmation to know if this is the way or he is overestimating Kubernetes.
3 projects | reddit.com/r/kubernetes | 20 Oct 2021
The Data Engineer Roadmap 🗺
12 projects | dev.to | 19 Oct 2021
Anything Comparable to power automate or flow for Linux?
2 projects | reddit.com/r/sysadmin | 17 Oct 2021
I never used Power Automate, but it looks like a workflow orchestrator. So checkout https://airflow.apache.org/
Airflow with different conda environments
1 project | reddit.com/r/dataengineering | 13 Oct 2021
If Airflow is the way to go then try DockerOperators (https://github.com/apache/airflow/blob/main/airflow/providers/docker/example_dags/example_docker.py). It's not the easiest set up but will do what you from what I get from your question.
Databricks jobs and Airflow on Kubernetes
1 project | reddit.com/r/dataengineering | 2 Oct 2021
I have not used databricks but it is something we are looking into integrating into our infrastructure in the future. Since Databricks is a service that does not run locally, I would use the databricks Operators/Hooks that come with airflow, rather than trying to build out anything of my own. https://github.com/apache/airflow/blob/main/airflow/providers/databricks/hooks/databricks.py
what do you think about airflow?
2 projects | reddit.com/r/dataengineering | 2 Oct 2021
I think one of the main design problems I have with Airflow is the fact that it tends to tightly couple processing/transform code with data movement code which makes debugging tricky. The way I have solved this is by building a command line interface to all the processing code so I can debug the processing code outside of any airflow infrastructure (which can be painful to get running locally if one does not use Airflow Breeze.
Building an Employee Survey Dashboard with Supabase and N8N
4 projects | dev.to | 3 Dec 2021
You should also go-ahead to set up N8N. You can signup for the cloud version or self-host it. Create a new workflow, and we’ll come back to it in a bit.
Stock market and crypto tracking with open source tools
4 projects | dev.to | 1 Dec 2021
How to automatically manage contributions to open-source projects 🏷️
3 projects | dev.to | 26 Nov 2021
Contribute to n8n: You can work on open issues, create nodes, improve our docs, or write a blog post.
Selfhosted dashboard suggestions
1 project | reddit.com/r/selfhosted | 25 Nov 2021
Look at n8n https://n8n.io/ and roll your own automation for checking webpages and sending/publishing alerts/messages.
Low-Code and the Democratization of Programming
8 projects | news.ycombinator.com | 17 Nov 2021
Developing automation for my support team
1 project | reddit.com/r/webdev | 16 Nov 2021
look into https://n8n.io/ or similar. this is very easy to do in python or with this tool. you might be overthinking this :-)
The Modern Stack to Build Internal Tools: Supabase, Appsmith, n8n
3 projects | dev.to | 8 Nov 2021
n8n: An extendable workflow automation tool. With a fair-code distribution model, n8n will always have visible source code, be available to self-host, and allow you to add your custom functions, logic and apps.
Tools that display the latest posts from your favorite communities or groups in a notion dashboard, all grouped together?
2 projects | reddit.com/r/Notion | 8 Nov 2021
Here is link number 1 - Previous text "n8n"2 projects | reddit.com/r/Notion | 8 Nov 2021
Hey! I did something similar but for the newsletters that I am subscribed to. I used n8n to manage this. Here's the video: https://twitter.com/harshil1712/status/1443307263909892104
How can I integrate Notion with Google Calendar and iCloud Calendar without a premium service like Zapier or Automate.io?
1 project | reddit.com/r/Notion | 7 Nov 2021
n8n https://n8n.io/ recently launched a desktop app that could work for you if you’ve got a machine that you can keep on 24/7 then you could roll your own solutions with their tool.
What are some alternatives?
Node RED - Low-code programming for event-driven applications
Kedro - A Python framework for creating reproducible, maintainable and modular data science code.
Huginn - Create agents that monitor and act on your behalf. Your agents are standing by!
luigi - Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
dagster - An orchestration platform for the development, production, and observation of data assets.
Dask - Parallel computing with task scheduling
Pandas - Flexible and powerful data analysis / manipulation library for Python, providing labeled data structures similar to R data.frame objects, statistical functions, and much more
robotjs - Node.js Desktop Automation.
StackStorm - StackStorm (aka "IFTTT for Ops") is event-driven automation for auto-remediation, security responses, troubleshooting, deployments, and more. Includes rules engine, workflow, 160 integration packs with 6000+ actions (see https://exchange.stackstorm.org) and ChatOps. Installer at https://docs.stackstorm.com/install/index.html. Questions? https://forum.stackstorm.com/.
Home Assistant - :house_with_garden: Open source home automation that puts local control and privacy first.
Apache Camel - Apache Camel is an open source integration framework that empowers you to quickly and easily integrate various systems consuming or producing data.
Numba - NumPy aware dynamic Python compiler using LLVM