Airflow
d3
Our great sponsors
Airflow | d3 | |
---|---|---|
169 | 277 | |
34,485 | 107,634 | |
2.3% | 0.3% | |
10.0 | 8.0 | |
about 18 hours ago | 17 days ago | |
Python | Shell | |
Apache License 2.0 | ISC License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Airflow
-
Building in Public: Leveraging Tublian's AI Copilot for My Open Source Contributions
Contributing to Apache Airflow's open-source project immersed me in collaborative coding. Experienced maintainers rigorously reviewed my contributions, providing constructive feedback. This ongoing dialogue refined the codebase and honed my understanding of best practices.
-
Navigating Week Two: Insights and Experiences from My Tublian Internship Journey
In week Two, I contributed to the Apache Airflow repository.
-
Airflow VS quix-streams - a user suggested alternative
2 projects | 7 Dec 2023
-
Best ETL Tools And Why To Choose
Apache Airflow is an open-source platform to programmatically author, schedule, and monitor workflows. The platform features a web-based user interface and a command-line interface for managing and triggering workflows.
-
Simplifying Data Transformation in Redshift: An Approach with DBT and Airflow
Airflow is the most widely used and well-known tool for orchestrating data workflows. It allows for efficient pipeline construction, scheduling, and monitoring.
-
Share Your favorite python related software!
AIRFLOW This is more of a library in my opinion, but Airflow has become an essential tool for scheduling in my work. All our ML training pipelines are ordered and scheduled with Airflow and it works seamlessly. The dashboard provided is also fantastic!
-
Ask HN: What is the correct way to deal with pipelines?
I agree there are many options in this space. Two others to consider:
- https://airflow.apache.org/
- https://github.com/spotify/luigi
There are also many Kubernetes based options out there. For the specific use case you specified, you might even consider a plain old Makefile and incrond if you expect these all to run on a single host and be triggered by a new file showing up in a directory…
- "Você veio protestar para ter acesso ao código fonte da urnas. O que é o código fonte?" "Não sei" 🤡
- Cómo construir tu propia data platform. From zero to hero.
-
Is it impossible to contribute to open source as a data engineer?
You can try and contribute some new connectors/operators for workflow managers like Airflow or Airbyte
d3
-
A visual guide to Vision Transformer – A scroll story
Yes this was done with a combination of GSAP Scrolltrigger https://gsap.com/docs/v3/Plugins/ScrollTrigger/ and https://d3js.org/
- Ask HN: Tips to get started on my own server
-
Full Stack Web Development Concept map
d3 - very power visualization library enabling dynamic visualizations. docs
-
Observable 2.0, a static site generator for data apps
Yep, Evidence is doing good work. We were most directly inspired by VitePress; we spent months rewriting both D3’s docs (https://d3js.org) and Observable Plot’s docs (https://observablehq.com/plot) in VitePress, and absolutely loved the experience. But we wanted a tool focused on data apps, dashboards, reports — observability and business intelligence use cases rather than documentation. Compared to Evidence, I’d say we’re trying to target data app developers more than data analysts; we offer a lot of power and expressiveness, and emphasize custom visualizations and interaction (leaning on Observable Plot or D3), as well as polyglot programming with data loaders written in any language (Python, R, not just SQL).
-
Using Deno with Jupyter Notebook to build a data dashboard
D3.js: A robust library to visualize your data and create interactive data-driven visualizations.
-
What is the technology stack used to create these live charts?
They are images so it could be any number of things, datawrapper, charts.js, d3.js to name a few options.
-
Animated map showing frequency and location of births around the world [OC]
I made this interactive visualization that attempts to show the real-time frequency and location of births around the world. A country’s annual births (i.e. the country’s population times its birthrate) were distributed across all of the populated locations in each country, weighted by the population distribution (i.e. more populated areas got a greater fraction of the births). Data Sources and Tools Population and birthrate data for 2023 was obtained from Wikipedia (Population and birth rates). Population distribution across the globe was obtained from Socioeconomic Data and Applications Center (sedac) at Columbia University. Data is processed and visualized at a 1 degree x 1 degree resolution, each of which has a different probability of a birth occurring in a specific time period. D3.js was used to create the map elements and html, css and javascript were used to create the user interface.
-
How do you implement library types?
When I go to the homepage of types/d3 the only hint for any kind of documentation is what seems to be the main github page of d3. It's highly possible I'm missing something here, so sorry if I am but I can't find any documentation of how you are supposed to type these library objects.
-
The top 11 React chart libraries for data visualization
Website: D3.js official site
-
Frontend development roadmap
D3js
What are some alternatives?
Kedro - Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.
echarts - Apache ECharts is a powerful, interactive charting and data visualization library for browser
dagster - An orchestration platform for the development, production, and observation of data assets.
GoJS, a JavaScript Library for HTML Diagrams - JavaScript diagramming library for interactive flowcharts, org charts, design tools, planning tools, visual languages.
n8n - Free and source-available fair-code licensed workflow automation tool. Easily automate tasks across different services.
vis
luigi - Luigi is a Python module that helps you build complex pipelines of batch jobs. It handles dependency resolution, workflow management, visualization etc. It also comes with Hadoop support built in.
d4 - A friendly reusable charts DSL for D3
Apache Spark - Apache Spark - A unified analytics engine for large-scale data processing
svg.js - The lightweight library for manipulating and animating SVG
Dask - Parallel computing with task scheduling
sigma.js - A JavaScript library aimed at visualizing graphs of thousands of nodes and edges