Python SQL

Open-source Python projects categorized as SQL
Edit details
Topics: #Python #Database #Sqlite #ETL #Python3
Tired of breaking your main and manually rebasing outdated pull requests?
Managing outdated pull requests is time-consuming. Mergify's Merge Queue automates your pull request management & merging. It's fully integrated to GitHub & coordinated with any CI. Start focusing on code. Try Mergify for free.
blog.mergify.com
Sponsored

Top 23 Python SQL Projects

  • devops-exercises

    Linux, Jenkins, AWS, SRE, Prometheus, Docker, Python, Ansible, Git, Kubernetes, Terraform, OpenStack, SQL, NoSQL, Azure, GCP, DNS, Elastic, Network, Virtualization. DevOps Interview Questions

    Project mention: DevOps Exercises | news.ycombinator.com | 2023-08-07

  • sqlmodel

    SQL databases in Python, designed for simplicity, compatibility, and robustness.

    Project mention: SQLModel with the same relationship column twice | /r/FastAPI | 2023-02-06

    Seems like this is a known bug in SQLModel: https://github.com/tiangolo/sqlmodel/issues/10

  • InfluxDB

    Collect and Analyze Billions of Data Points in Real Time. Manage all types of time series data in a single, purpose-built database. Run at any scale in any environment in the cloud, on-premises, or at the edge.

  • q

    q - Run SQL directly on delimited files and multi-file sqlite databases (by harelba)

    Project mention: I wrote this iCalendar (.ics) command-line utility to turn common calendar exports into more broadly compatible CSV files. | /r/commandline | 2023-03-24

    CSV utilities (still haven't pick a favorite one...): https://github.com/harelba/q https://github.com/BurntSushi/xsv https://github.com/wireservice/csvkit https://github.com/johnkerl/miller

  • modin

    Modin: Scale your Pandas workflows by changing a single line of code

    Project mention: The Distributed Tensor Algebra Compiler (2022) | news.ycombinator.com | 2023-06-15

  • datasette

    An open source multi-tool for exploring and publishing data

    Project mention: Seeking Help to Preserve Rare WWII Database on Windows 98 | news.ycombinator.com | 2023-09-18

    Looks like it's an Access database. Perhaps convert it to SQLite and publish with something like https://datasette.io/?

    I think the problem is, thread author doesn't know how to rip an ISO of the CD or move the database out; looks like they are getting help already though.

  • SQLAlchemy

    The Database Toolkit for Python

    Project mention: Unlocking efficient authZ with Cerbos’ Query Plan | dev.to | 2023-09-06

    To simplify this process, Cerbos developers have come up with adapters for popular Object-Relational Mapping (ORM) frameworks. You can check out for more details on the query plan repo - which also contains adapters for Prisma and SQLAlchemy - as well as a fully functioning application using Mongoose as its ORM.

  • sqlfluff

    A modular SQL linter and auto-formatter with support for multiple dialects and templated code.

    Project mention: Show HN: Postgres Language Server | news.ycombinator.com | 2023-08-06

    It has tons of annoying quirks, but I couldn't imagine running a DBT project without it: https://github.com/sqlfluff/sqlfluff

  • Mergify

    Tired of breaking your main and manually rebasing outdated pull requests?. Managing outdated pull requests is time-consuming. Mergify's Merge Queue automates your pull request management & merging. It's fully integrated to GitHub & coordinated with any CI. Start focusing on code. Try Mergify for free.

  • Mage

    🧙 The modern replacement for Airflow. Mage is an open-source data pipeline tool for transforming and integrating data. https://github.com/mage-ai/mage-ai

    Project mention: A mage on the Hero’s Journey: a fantasy epic on how a startup rose from the ashes | dev.to | 2023-06-12

    In the coming years, Mage will create a cooperative experience so that developers can build data pipelines with their team and level up together. After that journey, Mage will go on an epic quest to create the 1st open world community experience in the data universe.

  • dataset

    Easy-to-use data handling for SQL data stores with support for implicit table creation, bulk loading, and transactions.

    Project mention: What's the best process for scraping w/ the process of: Webpage -> List of Items -> Scrape Each Item Page | /r/webscraping | 2023-01-31

    I use the package 'dataset' (https://github.com/pudo/dataset), which lets you write a list of dictionaries as a database table. Easy peasy. :)

  • Flask-AppBuilder

    Simple and rapid application development framework, built on top of Flask. includes detailed security, auto CRUD generation for your models, google charts and much more. Demo (login with guest/welcome) - http://flaskappbuilder.pythonanywhere.com/

    Project mention: Pretty simple question, could be rewarding if enough people answer. | /r/flask | 2023-06-06

    This one might be interesting as well: https://github.com/dpgaspar/Flask-AppBuilder

  • sqlglot

    Python SQL Parser and Transpiler

    Project mention: Need help in converting Presto SQLs to Hive SQL | /r/SQL | 2023-05-08

  • ibis

    The flexibility of Python with the scale and performance of modern SQL.

    Project mention: A LLM+OLAP Solution | news.ycombinator.com | 2023-09-11

    Ibis could also be a target. It compiles queries written in python to multiple dataframe libraries, and SQL targets.

    https://ibis-project.org/

  • ethereum-etl

    Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQ

  • data-diff

    Compare tables within or across databases

    Project mention: How to Check 2 SQL Tables Are the Same | news.ycombinator.com | 2023-07-26

    If the issue happen a lot, there is also: https://github.com/datafold/data-diff

    That is a nice tool to do it cross database as well.

    I think it's based on checksum method.

  • pg_activity

    pg_activity is a top like application for PostgreSQL server activity monitoring.

    Project mention: pg_activity | /r/devopspro | 2022-11-08

  • PyPika

    PyPika is a python SQL query builder that exposes the full richness of the SQL language using a syntax that reflects the resulting query. PyPika excels at all sorts of SQL queries but is especially useful for data analysis.

    Project mention: any recommendations for a good query builder library with good support? | /r/learnpython | 2023-07-11

    I recently started using drizzle orm and I am now looking for something similar in python, my goal is to be as close to sql syntax as possible without just passing dml commands as strings, type safety would be cool as well, I saw this one pypika but it ha a lot of open issues and no commits for a year, is there anything similar but more stable?

  • django-sql-explorer

    Easily share data across your company via SQL queries. From Grove Collab.

    Project mention: Saving Filtered Querysets for Future Access | /r/django | 2022-10-04

  • alembic

    A database migrations tool for SQLAlchemy.

  • djongo

    Django and MongoDB database connector

    Project mention: Problems with DjOngo | /r/django | 2023-05-05

  • fugue

    A unified interface for distributed computing. Fugue executes SQL, Python, Pandas, and Polars code on Spark, Dask and Ray without any rewrites.

    Project mention: Daft: A High-Performance Distributed Dataframe Library for Multimodal Data | news.ycombinator.com | 2023-06-07

    Please integrate it with Fugue.

    https://github.com/fugue-project/fugue

  • Cubes

    [NOT MAINTAINED] Light-weight Python OLAP framework for multi-dimensional data analysis

  • learning

    Learning Shell，Python，Golang，System，Network (by anzhihe)

  • jet-bridge

    Jet Admin – No-code Business App builder

  • Sonar

    Write Clean Python Code. Always.. Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2023-09-18.

Python SQL related posts

Index

What are some of the best open-source SQL projects in Python? This list will help you:

Project Stars
1 devops-exercises 57,969
2 sqlmodel 10,737
3 q 9,937
4 modin 8,967
5 datasette 8,272
6 SQLAlchemy 7,748
7 sqlfluff 6,530
8 Mage 5,516
9 dataset 4,631
10 Flask-AppBuilder 4,311
11 sqlglot 3,747
12 ibis 3,096
13 ethereum-etl 2,658
14 data-diff 2,512
15 pg_activity 2,228
16 PyPika 2,146
17 django-sql-explorer 2,142
18 alembic 2,118
19 djongo 1,776
20 fugue 1,723
21 Cubes 1,490
22 learning 1,471
23 jet-bridge 1,460

Sponsored
Write Clean Python Code. Always.
Sonar helps you commit clean code every time. With over 225 unique rules to find Python bugs, code smells & vulnerabilities, Sonar finds the issues while you focus on the work.
www.sonarsource.com