Skytrax-Data-Warehouse
jaydebeapi
Our great sponsors
| Skytrax-Data-Warehouse | jaydebeapi | |
|---|---|---|
| 1 | 1 | |
| 131 | 355 | |
| - | - | |
| 0.0 | 0.0 | |
| about 4 years ago | 21 days ago | |
| Python | Python | |
| MIT License | GNU Lesser General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Skytrax-Data-Warehouse
-
Open source contributions for a Data Engineer?
Always open to accept contributions to my project (Skytrax Data Warehouse). If you are into data stuff support my work at youtube as well (One Developer Pirate), I mostly make data-oriented videos. These days I'm making a SQL course from a data analysis perspective that is expected to release in next week.
jaydebeapi
-
Bulk load Pandas DataFrames into SQL databases using Jaydebeapi
Loading Pandas DataFrames into SQL databases of all names is a common task between all developers working on building data pipelines for their environments or trying to automate ETL jobs generally. And for that, Pandas DataFrame class has the built-in method pandas.DataFrame.to_sql that allows to do so very quickly, for SQLite and all the databases supported by SQLAlchemy library, and when it comes to those who doesn’t have a good support by it ( in my case was IBM DB2 ), developers find themselves forced to think twice for some work around in order to get the job done. Jaydebeapi introduces himself as a good alternative, and it’s particularly seen thus by all developers coming from a Java background and having some familiarities of working with JDBC API to access the database. Let’s start first by creating the database connection. for that reason I will be creating a simple function that takes in params all the informations required and it will give a connection to DB2 as a return.
What are some alternatives?
dbd - dbd is a database prototyping tool that enables data analysts and engineers to quickly load and transform data in SQL databases.
snowflake-connector-python - Snowflake Connector for Python
sqlfluff - A modular SQL linter and auto-formatter with support for multiple dialects and templated code.
dataset - Easy-to-use data handling for SQL data stores with support for implicit table creation, bulk loading, and transactions.
dbt-spotify-analytics - Containerized end-to-end analytics of Spotify data using Python, dbt, Postgres, and Metabase
sqllex - The most pythonic ORM (for SQLite and PostgreSQL). Seriously, try it out!
airflow-api-tests - This is a collection of Pytest for the 2.0 Stable Rest Apis for Apache Airflow. I have another repo where you could setup airflow locally and play around with these. I am used to RestAssured, but trying out pytest here.
PyPika - PyPika is a python SQL query builder that exposes the full richness of the SQL language using a syntax that reflects the resulting query. PyPika excels at all sorts of SQL queries but is especially useful for data analysis.
dagster - An orchestration platform for the development, production, and observation of data assets.
cinemagoer - Cinemagoer is a Python package useful to retrieve and manage the data of the IMDb (to which we are not affiliated in any way) movie database about movies, people, characters and companies
DataGristle - Tough and flexible tools for data analysis, transformation, validation and movement.
airbyte - The leading data integration platform for ETL / ELT data pipelines from APIs, databases & files to data warehouses, data lakes & data lakehouses. Both self-hosted and Cloud-hosted.