PyFunctional
mara-pipelines
Our great sponsors
| PyFunctional | mara-pipelines | |
|---|---|---|
| 4 | 3 | |
| 2,332 | 2,054 | |
| - | 0.4% | |
| 5.6 | 6.0 | |
| about 2 months ago | 4 months ago | |
| Python | Python | |
| MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
PyFunctional
-
Python: Uncovering the Overlooked Core Functionalities
If you actually think this code is better there's a real library that does this: https://github.com/EntilZha/PyFunctional.
-
Kotlin/Java/Javascript/Scala users do you miss the ability to chain functional operators in Python?
I often endup using [pyfunctional](https://github.com/EntilZha/PyFunctional) which gives the ability of using chained functional operators, but I am still kind of sad this is not a builtin solution in Python (please note I am not involved in the pyfunctional project and I do not know the author)
- PyFunctional makes creating data pipelines easy by using chained functional operators
- The pipe-operator to python |>
mara-pipelines
-
How to keep track of the different Transformations done in an ETL pipeline?
The closest I've found is Mara but not what I'm after.
-
Using PostgreSQL as a Data Warehouse
The tooling behind the approach has been built as a set of python package named Mara. It is available at GitHub:
https://github.com/mara/mara-pipelines
And additional packages can be found at the Mara org:
https://github.com/mara
-
Build your own “data lake” for reporting purposes
Minio and nifi, require machines by themselves. Better off pure python and if obe wants sonething lighweight and visually pleasing Mara [0] or Dagster with Dagit [1] will do the job
[0] https://github.com/mara/mara-pipelines
[1] https://docs.dagster.io/tutorial/execute
What are some alternatives?
awesome-functional-python - A curated list of awesome things related to functional programming in Python.
abcd-hcp-pipeline - bids application for processing functional MRI data, robust to scanner, acquisition and age variability.
RxPY - ReactiveX for Python
kuwala - Kuwala is the no-code data platform for BI analysts and engineers enabling you to build powerful analytics workflows. We are set out to bring state-of-the-art data engineering tools you love, such as Airbyte, dbt, or Great Expectations together in one intuitive interface built with React Flow. In addition we provide third-party data into data science models and products with a focus on geospatial data. Currently, the following data connectors are available worldwide: a) High-resolution demographics data b) Point of Interests from Open Street Map c) Google Popular Times
modin - Modin: Scale your Pandas workflows by changing a single line of code
pybaseball - Pull current and historical baseball statistics using Python (Statcast, Baseball Reference, FanGraphs)
fn.py - Functional programming in Python: implementation of missing features to enjoy FP
dbt-core - dbt enables data analysts and engineers to transform their data using the same practices that software engineers use to build applications.
etl-markup-toolkit - ETL Markup Toolkit is a spark-native tool for expressing ETL transformations as configuration
alkymi - Pythonic task automation
dremio-oss - Dremio - the missing link in modern data