Top 23 SQL Open-Source Projects
-
Apache Spark is one of the most actively developed open-source projects in big data. The following code examples require that you have Spark set up and can execute Python code using the PySpark library. The examples also require that you have your data in Amazon S3 (Simple Storage Service). All this is set up on AWS EMR (Elastic MapReduce).
-
tidb
TiDB is an open-source, cloud-native, distributed, MySQL-Compatible database for elastic scale and real-time analytics. Try free: https://tidbcloud.com/signup
As the database supporter for ossinsight.io, I also use this to get many insights for TiDB itself, see https://ossinsight.io/analyze/pingcap/tidb.
-
SonarQube
Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.
-
Project mention: Most popular PostgreSQL GUIs in 2022: the (almost) scientific list | dev.to | 2022-05-17
OmniDB is more complex than a Postico or TablePlus but less advanced than a DataGrip or Navicat. It's a good free option but maybe less complete than DBeaver, which has a better documentation and a seemingly larger community.
-
Sequelize
An easy-to-use and promise-based multi SQL dialects ORM tool for Node.js | Postgres, MySQL, MariaDB, SQLite, MSSQL, Snowflake & DB2
-
CRDB implements the algorithm described in a 2013 paper called "On the correct and complete enumeration of the core search space". It is a bottom-up DP approach that is able to handle inner, left, anti, semi, full, etc. joins (many papers only cover inner joins). In addition, CRDB extends it in a few ways that enable reorderings in some important practical cases that the paper doesn't cover (a full description can be found here). As a result, it's able to reorder in a number of cases that other optimizers don't even consider.
-
devops-exercises
Linux, Jenkins, AWS, SRE, Prometheus, Docker, Python, Ansible, Git, Kubernetes, Terraform, OpenStack, SQL, NoSQL, Azure, GCP, DNS, Elastic, Network, Virtualization. DevOps Interview Questions
This guy https://github.com/bregman-arie/devops-exercises made an awesome practice material for DevOps tasks
-
Project mention: Ask HN: Free and open source distributed database written in C++ or C | news.ycombinator.com | 2022-05-16
-
Scout APM
Less time debugging, more time building. Scout APM allows you to find and fix performance issues with no hassle. Now with error monitoring and external services monitoring, Scout is a developer's best friend when it comes to application development.
-
It took me some time to get a good grasp of the power of SQL; and it really kicked in when I learned about optimization rules. It's a program that you rewrite, just like an optimizing compiler would.
You state what you want; you have different ways to fetch and match and massage data; and you can search through this space to produce a physical plan. Hopefully you used knowledge to weight parts to be optimized (table statistics, like Java's JIT would detect hot spots).
I find it fascinating to peer through database code to see what is going on. Lately, there's been new advances towards streaming databases, which bring a whole new design space. For example, now you have latency of individual new rows to optimize for, as opposed to batch it whole to optimize the latency of a dataset. Batch scanning will be benefit from better use of your CPU caches.
And maybe you could have a hybrid system which reads history from a log and aggregates in a batched manner, and then switches to another execution plan when it reaches the end of the log.
If you want to have a peek at that here are Flink's set of rules [1], generic and stream-specific ones. The names can be cryptic, but usually give a good sense of what is going on. For example: PushFilterIntoTableSourceScanRule makes the WHERE clause apply the earliest possible, to save some CPU/network bandwidth further down. PushPartitionIntoTableSourceScanRule tries to make a fan-out/shuffle happen the earliest possible, so that parallelism can be made use of.
[1] https://github.com/apache/flink/blob/5f8fb304fb5d68cdb0b3e3c...
-
osquery, with or without fleetdm or kolide.
-
How about batch insert, updates, and deletes? I had to fix a broken MyBatis project recently and was surprised that this feature doesn't even seem to be implemented, at least according to this GitHub issue.
-
shardingsphere
Ecosystem to transform any database into a distributed database system, and enhance it with sharding, elastic scaling, encryption features & more
Project mention: Apache ShardingSphere -Proxy Performance with PostgreSQL is Improved 26.8% with Version 5.1.0 | dev.to | 2022-04-25Relevant PR：https://github.com/apache/shardingsphere/pull/11459/files
-
Knex
A query builder for PostgreSQL, MySQL, CockroachDB, SQL Server, SQLite3 and Oracle, designed to be flexible, portable, and fun to use.
Knex.js as Query Builder
-
Project mention: i get a "Unable to cast object of type 'System.Byte' to type 'System.Int32'.'" error. I have looked on google but I can't really find a solution | reddit.com/r/csharp | 2022-04-15
Dude, cant use dapper? https://github.com/DapperLib/Dapper
-
Project mention: Let's write a compiler, part 5: A code generator | news.ycombinator.com | 2021-08-19
-
TimescaleDB
An open-source time-series SQL database optimized for fast ingest and complex queries. Packaged as a PostgreSQL extension.
If you need persistent storage, TimescaleDB is a PostgreSQL extension designed for time-series data.
-
In this piece, we will teach you how to leverage the power of Cube to build an analytics dashboard that aggregates data from a relational database and displays it using Bubble.
-
go-sql-driver/mysql
Go MySQL Driver is a MySQL driver for Go's (golang) database/sql package (by go-sql-driver)
go-sql-driver/mysql 12.1k Stars, Used by 72.4k
-
For those of you new to Dolt (https://github.com/dolthub/dolt). Dolt is like Git and MySQL had a baby. We built a SQL database from the storage engine up to support branch, merge, clone, diff and all the other Git idioms.
-
Project mention: Let's build a distributed Postgres proof of concept | news.ycombinator.com | 2022-05-17
Every node in the cluster "knows" the network address of the Leader. It knows because a) it's part of the Raft cluster configuration, and every follower "heartbeats" to the Leader periodically.
So it's actually pretty simple. If a node receives a request which must be performed on the leader, and that node is not itself the Leader, it can do one of the following things:
1) reject the request with an error, but this isn't really a production-viable option.
2) reject the request with an error, but tell the client where the leader can be found, so the leader can retry the request.
3) transparently forward the request to the leader, wait for the leader to execute the request, get the respond, and return the response to the client. In this case the client doesn't even know the forwarding to the leader happened.
rqlite supports mode 2 and 3, client can choose which behavior it wants, on a request-by-request basis. Option 3 is the default.
https://github.com/rqlite/rqlite/blob/master/DOC/DATA_API.md...
-
The reason I originally looked into this was to see if there's a way to use sqlite in wasm with indexeddb as the storage backend, getting a full SQL engine in the browser (like WebSQL, but actually using web standard API). Here is sqlite compiled to wasm, but only with the in-memory database, which is only of limited use. My conclusion was that it might be possible, but needs a ton of work in C interfacing with web APIs through emscripten, which is not fun at all.
-
This has been answered already, but until the patch comes out a password manager like bitwarden can trivialize the issue.
-
beekeeper-studio
Modern and easy to use SQL client for MySQL, Postgres, SQLite, SQL Server, and more. Linux, MacOS, and Windows.
-
Project mention: OpenOffice. Necesito ver una hoja de cálculo que tiene 6 millones de columnas, pero no puedo | reddit.com/r/AskArgentina | 2022-05-15
SQL related posts
Index
What are some of the best open-source SQL projects? This list will help you:
| Project | Stars | |
|---|---|---|
| 1 | Apache Spark | 32,903 |
| 2 | tidb | 31,319 |
| 3 | dbeaver | 26,450 |
| 4 | Sequelize | 26,139 |
| 5 | cockroach | 24,577 |
| 6 | devops-exercises | 24,205 |
| 7 | ClickHouse | 23,738 |
| 8 | Apache Flink | 18,920 |
| 9 | OSQuery | 18,896 |
| 10 | MyBatis | 17,176 |
| 11 | shardingsphere | 15,869 |
| 12 | Knex | 15,799 |
| 13 | Dapper | 14,895 |
| 14 | Presto | 13,478 |
| 15 | TimescaleDB | 13,060 |
| 16 | cube.js | 12,964 |
| 17 | go-sql-driver/mysql | 12,233 |
| 18 | dolt | 10,633 |
| 19 | rqlite | 10,322 |
| 20 | sql.js | 10,119 |
| 21 | Bitwarden | 9,776 |
| 22 | beekeeper-studio | 9,677 |
| 23 | q | 9,059 |
Are you hiring? Post a new remote job listing for free.
