Top 23 Python SQL Projects
Linux, Jenkins, AWS, SRE, Prometheus, Docker, Python, Ansible, Git, Kubernetes, Terraform, OpenStack, SQL, NoSQL, Azure, GCP, DNS, Elastic, Network, Virtualization. DevOps Interview QuestionsProject mention: Questions you would get asked on an interview? | reddit.com/r/devops | 2021-01-28
I think the link you're looking for is https://github.com/bregman-arie/devops-exercises
q - Run SQL directly on CSV or TSV files (by harelba)Project mention: Q – Run SQL Directly on CSV or TSV Files | reddit.com/r/patient_hackernews | 2021-06-07
Scout APM - Leading-edge performance monitoring starting at $39/month. Scout APM uses tracing logic that ties bottlenecks to source code so you know the exact line of code causing performance issues and can get back to building a great product faster.
Modin: Speed up your Pandas workflows by changing a single line of codeProject mention: How to Speed Up Pandas with 1 Line of Code | reddit.com/r/Python | 2021-03-03
An open source multi-tool for exploring and publishing dataProject mention: Help me find a specific HackerNews article | reddit.com/r/node | 2021-06-17
Not sure for the HN article but the guy is called Simon Willison and the tool is Datasette: https://datasette.io/
Easy-to-use data handling for SQL data stores with support for implicit table creation, bulk loading, and transactions.
The Database Toolkit for PythonProject mention: How can I update data on my live website using Python? | reddit.com/r/learnpython | 2021-06-10
Simple and rapid application development framework, built on top of Flask. includes detailed security, auto CRUD generation for your models, google charts and much more. Demo (login with guest/welcome) - http://flaskappbuilder.pythonanywhere.com/Project mention: Splitting flask app into multiple files | reddit.com/r/flask | 2021-03-12
You can also try FlaskAppbuilder: https://github.com/dpgaspar/Flask-AppBuilder. Some of the large project like Apache Airflow, Apache Superset are built on top of it.
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Easily share data across your company via SQL queries. From Grove Collab.Project mention: Show HN: Django SQL Dashboard | news.ycombinator.com | 2021-05-10
Very cool! I wrote Django SQL Explorer, and this looks very similar in spirit, but an emphasis on visualization that Explorer does not have (to the extent it has a focus, it's more on providing a reasonable way to write complex queries and re-use them).
These types of tools are extremely handy.
Light-weight Python OLAP framework for multi-dimensional data analysisProject mention: Building data analysis apps | reddit.com/r/Python | 2021-04-16
I'm looking for materials and tools to learn. I'm reading up on OLAP and cubes. I found cubes python package but it hasn't been updated in years. Could you give me some tips on what to learn in 2021?
Django and MongoDB database connectorProject mention: How to properly set djongo timeout | reddit.com/r/django | 2021-06-22
PyPika is a python SQL query builder that exposes the full richness of the SQL language using a syntax that reflects the resulting query. PyPika excels at all sorts of SQL queries but is especially useful for data analysis.Project mention: Migrating to SQLAlchemy 2.0 | news.ycombinator.com | 2021-02-18
There is a middle-ground between writing SQL statement strings in your code, and a full-blown ORM: query builders. At least in my experience with small to medium projects, these have far fewer footguns while keeping the code composable and readable. Here's one for Python: https://github.com/kayak/pypika
Python scripts for ETL (extract, transform and load) jobs for Ethereum blocks, transactions, ERC20 / ERC721 tokens, transfers, receipts, logs, contracts, internal transactions. Data is available in Google BigQuery https://goo.gl/oY5BCQProject mention: Trying To Recover Old ETH | reddit.com/r/ethereum | 2021-01-01
You can use https://github.com/blockchain-etl/ethereum-etl
Jet Bridge – Admin Panel Framework for your application
Python library for using dplyr like syntax with pandas and SQLProject mention: R / Tidyverse User -> Python | How to Make it Hurt Less | reddit.com/r/rprogramming | 2021-05-21
Check out siuba
Unofficial API for finviz.comProject mention: A few Github repositories that I 'm planning to go through | reddit.com/r/RKSP | 2021-04-18
An interpreted relational query language that compiles to SQL.Project mention: Preql: A relational language that compiles to SQL | reddit.com/r/SQL | 2021-03-25
Hi everyone, I'm happy to introduce Preql.
🌈Rainbow CSV - Vim plugin: Highlight columns in CSV and TSV files and run queries in SQL-like languageProject mention: Any recommendations for a cli CSV editor? | reddit.com/r/commandline | 2021-03-24
Rainbow CSV, for visual color highlighting and has an SQL-like language for running data queries.
A dynamic FastAPI router that automatically creates CRUD routes for your modelsProject mention: FastAPI framework, high perf, easy to learn, fast to code, ready for production | news.ycombinator.com | 2021-02-01
Thanks, that's a really helpful example.
Where I think this could be taken to the next level of reusability is in modularising the front-end into API-specific components. For example, the login behaviour could depend on FastAPI-Users, with a sibling frontend library containing components that implement the same login flow. Adding user behaviour is then a matter of using the same third-party library on the front and back end.
This approach could be extended to other components such as an admin panel (perhaps using https://github.com/awtkns/fastapi-crudrouter), or a blogging component.
JayDeBeApi module allows you to connect from Python code to databases using Java JDBC. It provides a Python DB-API v2.0 to that database.Project mention: Bulk load Pandas DataFrames into SQL databases using Jaydebeapi | dev.to | 2021-05-09
Loading Pandas DataFrames into SQL databases of all names is a common task between all developers working on building data pipelines for their environments or trying to automate ETL jobs generally. And for that, Pandas DataFrame class has the built-in method pandas.DataFrame.to_sql that allows to do so very quickly, for SQLite and all the databases supported by SQLAlchemy library, and when it comes to those who doesn’t have a good support by it ( in my case was IBM DB2 ), developers find themselves forced to think twice for some work around in order to get the job done. Jaydebeapi introduces himself as a good alternative, and it’s particularly seen thus by all developers coming from a Java background and having some familiarities of working with JDBC API to access the database. Let’s start first by creating the database connection. for that reason I will be creating a simple function that takes in params all the informations required and it will give a connection to DB2 as a return.
Django app for building dashboards using raw SQL queriesProject mention: Why is uncoupled documentation bad? | news.ycombinator.com | 2021-06-06
I use documentation systems that publish the documentation from the repo to a website. Most of my projects use Sphinx and reStructuredText for this, but I recently tried MyST (Markdown for Sphinx) and I like that a lot.
Splitgraph command line client and python libraryProject mention: Cloudera taken private for $5.3b, acquires Datacoral and Cazena | news.ycombinator.com | 2021-06-01
The data industry continues to hype this idea of “multi-cloud,” but then the “modern data stack” is centralized around a single warehouse and nobody sees any irony in that.
The big bet we’re making at Splitgraph  is that the next wave of data engineering will take a more decentralized, “data mesh” type approach to enterprise architecture. “Data gravity” really does exist -expensive to move, in terms of both cost and operational complexity. So instead of bringing the data to the query, why not bring the query to the data? All we need for that is a set of read only credentials.
Cloudera mentions they bought DataCoral to help with data integration and connectors. They’ve correctly identified the problem - data sprawl and fragmentation will inevitably grow - but I’m not sure they have the right solution.
Data integration is important, but it’s a moving target, which is why it calls for a collaborative open source solution. This is why so many new startups, like AirByte most recently, are coalescing around the Singer taps that Stitch left behind after its acquisition by Talend.
We also support using Singer taps to ingest data into versioned Splitgraph images , so we’re excited to see more collaboration on maintenance of taps. For us it’s a useful feature, but it should be just that — a feature. Is there really a need to replicate all of your data before you can even query it? Or would you rather experiment by directly querying its source?
 unreleased and undocumented atm, but it does work. We’re hiring, especially on the frontend if you want to help build the web UI. See profile.
Snowflake Connector for PythonProject mention: Loading a file-like object to Snowflake via Python? | reddit.com/r/snowflake | 2021-03-26
I actually just found a useful comment on https://github.com/snowflakedb/snowflake-connector-python/issues/317 saying that this isn't supported yet. Its fine, I can just go through S3 in the meantime.
Fork of django-postgres that focuses on maintaining and improving support for Postgres SQL Views.Project mention: How to speed up Django when querying large data? | reddit.com/r/django | 2021-03-25
Ive used https://github.com/mypebble/django-pgviews for the same purpose. I like the idea that it adds the sql into git directly. Only works for postgres though.
What are some of the best open-source SQL projects in Python? This list will help you: