SaaSHub helps you find the best software and product alternatives Learn more →
Top 23 Python SQL Projects
-
devops-exercises
Linux, Jenkins, AWS, SRE, Prometheus, Docker, Python, Ansible, Git, Kubernetes, Terraform, OpenStack, SQL, NoSQL, Azure, GCP, DNS, Elastic, Network, Virtualization. DevOps Interview Questions
-
Scout Monitoring
Free Django app performance insights with Scout Monitoring. Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.
-
-
pandas-ai
Chat with your database (SQL, CSV, pandas, polars, mongodb, noSQL, etc). PandasAI makes data analysis conversational using LLMs (GPT 3.5 / 4, Anthropic, VertexAI) and RAG.
In this blog, we will build a powerful IDE agent for PandasAI using Dash Agent. Then later on, we'll understand how using RAG can significantly improve LLM responses.
-
vanna-ai is a pretty cool project that combines AI with database knowledge. SQLAI seems to be a small German company which specializes in this area. I will start exploring these in my free time.
-
-
-
We will define our database schema using SQLAlchemy. First, we will store a subset of the information in a single table called documents. This is to test that our database configuration is correct and avoid storing nested data now. The code is fairly simple to create a SQLite3 database with SQLAlchemy:
-
InfluxDB
Purpose built for real-time analytics at any scale. InfluxDB Platform is powered by columnar analytics, optimized for cost-efficient storage, and built with open data standards.
-
I wrote an async wrapper around SQLite in Python - I'm using a thread pool: https://github.com/simonw/datasette/blob/main/datasette/data...
I have multiple threads for reads and a single dedicated thread for writes, which I send operations to via a queue. That way I avoid ever having two writes against the same connection at the same time.
-
Mage
🧙 The modern replacement for Airflow. Mage is an open-source data pipeline tool for transforming and integrating data. https://github.com/mage-ai/mage-ai
Mage AI is a data transforming and integrating framework that allows data scientists and ML engineers to build and automate data pipelines without extensive coding. Data scientists can easily connect to their data sources, ingest data, and build production-ready data pipelines within Mage notebooks.
-
sqlfluff
A modular SQL linter and auto-formatter with support for multiple dialects and templated code.
Project mention: 🐍🐍 23 issues to grow yourself as an exceptional open-source Python expert 🧑💻 🥇 | dev.to | 2023-10-19Repo : https://github.com/sqlfluff/sqlfluff
-
Project mention: The Future of MySQL is PostgreSQL: an extension for the MySQL wire protocol | news.ycombinator.com | 2024-04-26
This is probably referring to "zero changes to your driver code" and not "zero changes to the SQL you send over this driver".
Translating between SQL dialects is notoriously hard and attempts to translate [1] are working in 95% of cases. But the last 5% would require 5x amount of work. That's because "SQL dialect" also includes weird edge cases of type inference of things like COALESCE(5, FALSE) and emulation of system catalogs (pg_catalog, information_schema).
[1] https://github.com/tobymao/sqlglot
-
There is is still much to do especially on large table formats (iceberg/delta) and memory management when running on bigger boxes on cloud. Eg the elusive "Failed to allocate ..." bug[1] is an inhibitor to the claim that big data is dead[2]. As it is, we tried and abandoned DuckDB as a cheaper replacement for some databricks batch jobs.
[0] https://github.com/ibis-project/ibis
-
dataset
Easy-to-use data handling for SQL data stores with support for implicit table creation, bulk loading, and transactions.
-
Flask-AppBuilder
Simple and rapid application development framework, built on top of Flask. includes detailed security, auto CRUD generation for your models, google charts and much more. Demo (login with guest/welcome) - http://flaskappbuilder.pythonanywhere.com/
GitHub https://github.com/dpgaspar/Flask-AppBuilder GitHub Stars 4.6k Most Recent Update on GitHub 2 days ago Open Source License BSD-3-Clause Number of Active Contributors This Year 11 Acceptance of External PRs Yes Official Website / Documentation https://flask-appbuilder.readthedocs.io/en/latest/
-
-
ethereum-etl
Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ
Project mention: Blockchain transactions decoding: making wallet activity understandable | dev.to | 2023-10-27Event is a log entity which EVM smart contracts can emit during transaction execution. Events are very good at signalling that an some action has taken place on-chain. Applications can subscribe and listen to events to trigger some off-chain logic or they can index, transform and store events in some off-chain storage (look at The Graph protocol or Ethereum ETL).
-
django-sql-explorer
SQL reporting that Just Works. Fast, simple, and confusion-free. Write and share queries in a delightful SQL editor, with AI assistance.
Project mention: Show HN: SQL Explorer – Open-source reporting tool that just works | news.ycombinator.com | 2024-07-02You got it! You'll need three values set in settings.py:
https://github.com/explorerhq/django-sql-explorer/blob/64170...
```
-
The ORM is built on top of this Core foundation. It expands upon it through a domain object model that is mapped to a database schema. This means you can have classes that are mapped to the database tables and columns, and implement different operations in an object-oriented way. This can be further expanded with external tools such as Alembic, which is a migration tool, or marshmallow, which is a serialization tool.
-
PyPika
PyPika is a python SQL query builder that exposes the full richness of the SQL language using a syntax that reflects the resulting query. PyPika excels at all sorts of SQL queries but is especially useful for data analysis.
-
-
fugue
A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.
-
-
-
SaaSHub
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
NOTE:
The open source projects on this list are ordered by number of github stars.
The number of mentions indicates repo mentiontions in the last 12 Months or
since we started tracking (Dec 2020).
Python SQL discussion
Python SQL related posts
-
Show HN: An open source project in Python to inspect databases through CLI fast
-
Show HN: SQL Explorer – Open-source reporting tool that just works
-
Analyzing Svenskalag Data using DBT and DuckDB
-
Track Errors in Your Python Flask Application with AppSignal
-
Show HN: SQLFrame – I ran PySpark without Spark on a SQL database
-
Setting up a standalone SQLAlchemy 2.0 ORM application
-
The Future of MySQL is PostgreSQL: an extension for the MySQL wire protocol
-
A note from our sponsor - SaaSHub
www.saashub.com | 16 Sep 2024
Index
What are some of the best open-source SQL projects in Python? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | devops-exercises | 65,830 |
| 2 | sqlmodel | 14,032 |
| 3 | pandas-ai | 12,553 |
| 4 | vanna | 10,730 |
| 5 | q | 10,183 |
| 6 | modin | 9,744 |
| 7 | SQLAlchemy | 9,428 |
| 8 | datasette | 9,389 |
| 9 | Mage | 7,703 |
| 10 | sqlfluff | 7,647 |
| 11 | sqlglot | 6,390 |
| 12 | ibis | 5,061 |
| 13 | dataset | 4,752 |
| 14 | Flask-AppBuilder | 4,628 |
| 15 | dataherald | 3,304 |
| 16 | ethereum-etl | 2,920 |
| 17 | django-sql-explorer | 2,738 |
| 18 | alembic | 2,724 |
| 19 | PyPika | 2,479 |
| 20 | pg_activity | 2,454 |
| 21 | fugue | 1,957 |
| 22 | djongo | 1,874 |
| 23 | learning | 1,836 |
Sponsored
Free Django app performance insights with Scout Monitoring
Get Scout setup in minutes, and let us sweat the small stuff. A couple lines in settings.py is all you need to start monitoring your apps. Sign up for our free tier today.
www.scoutapm.com