-
hamilton
Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage and metadata. Runs and scales everywhere python does.
-
hamilton
Discontinued A scalable general purpose micro-framework for defining dataflows. THIS REPOSITORY HAS BEEN MOVED TO www.github.com/dagworks-inc/hamilton (by stitchfix)
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
When we ran the ML platform team at Stitch Fix, we dealt with a lot of monolithic, messy pandas scripts. We built hamilton to solve this problem. A programmer represents transforms on dataframes as a series of python functions. The parameter names are used to specify upstream dependencies, and the whole thing gets wired into a dependency graph.
And find the repository here: https://github.com/dagworks-inc/hamilton/
Related posts
-
Show HN: Hamilton's UI – observability, lineage, and catalog for data pipelines
-
Using IPython Jupyter Magic commands to improve the notebook experience
-
Show HN: On Garbage Collection and Memory Optimization in Hamilton
-
Show HN: Declarative Spark Transformations with Hamilton
-
Free access to beta product I'm building that I'd love feedback on