dlt
| dlt | Spooq | |
|---|---|---|
| 6 | 1 | |
| 1,758 | 8 | |
| 9.0% | - | |
| 9.9 | 7.4 | |
| 5 days ago | about 2 months ago | |
| Python | Python | |
| Apache License 2.0 | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
dlt
-
Ask HN: Freelancer? Seeking freelancer? (December 2023)
SEEKING FREELANCER | REMOTE | GERMANY
dltHub is looking for a freelance help in the following repos:
- https://github.com/dlt-hub/dlt
-
Show HN: Data load tool(dlt)-Python library to automate the creation of datasets
You can use pydantic models to define schemas, validate data (we also load instances of the models natively): https://dlthub.com/docs/general-usage/resource#define-a-sche...
We have a PR (https://github.com/dlt-hub/dlt/pull/594) that is about to merge that makes the above highly configurable, between evolution and hard stopping:
- Data load tool (dlt) – open-source Python library that makes data loading easy
-
[Discussion] How to implement Data Contracts generically? Seeking advice from data contract users.
customize pipeline with hooks
- Which ETL/ELT tools do you think have future in data engineering space?
Spooq
-
Using Spooq to load a large scale of data
the link to the project: https://github.com/Breaka84/Spooq/blob/master/spooq/loader/hive_loader.py
What are some alternatives?
airbyte - The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.
Proxmox-load-balancer - Designed to constantly maintain the Proxmox cluster in balance
pyspark-example-project - Implementing best practices for PySpark ETL jobs and applications.
grablinks - A simple and streamlined Python script to extract and filter links from a remote HTML resource.
data-retrieval - Data extraction and transformation for the animated graph
Udacity-Data-Engineering-Projects - Few projects related to Data Engineering including Data Modeling, Infrastructure setup on cloud, Data Warehousing and Data Lake development.
workshop-realtime-data-pipelines - You will inspect and run a sample architecture making use of Apache Pulsar™ and Pulsar Functions for real-time, event-streaming-based data ingestion, cleaning and processing.
data-engineering-zoomcamp - Free Data Engineering course!
data-science-ipython-notebooks - Data science Python notebooks: Deep learning (TensorFlow, Theano, Caffe, Keras), scikit-learn, Kaggle, big data (Spark, Hadoop MapReduce, HDFS), matplotlib, pandas, NumPy, SciPy, Python essentials, AWS, and various command lines.
DXY-COVID-19-Data - 2019新型冠状病毒疫情时间序列数据仓库 | COVID-19/2019-nCoV Infection Time Series Data Warehouse
hamilton - Hamilton helps data scientists and engineers define testable, modular, self-documenting dataflows, that encode lineage and metadata. Runs and scales everywhere python does.