Top 23 Database Open-Source Projects
-
prometheus
The Prometheus monitoring system and time series database.
It worked! Stopping Prometheus broke the active connection that Prometheus previously had with NewRelic. I wasn't aware that remote_write keeps an active connection; I was sure it just sends the data and closes the connection. Apparently, it is documented in the official changelog of Prometheus - 1.8.0 / 2017-10-06 - "..Remote storage connections use HTTP keep-alive..
-
etcd
Distributed reliable key-value store for the most critical data of a distributed system
etcd-io/etcd (Go): Distributed reliable key-value store for the most critical data of a distributed system
-
tidb
TiDB is an open source distributed HTAP database compatible with the MySQL protocol
pingcap/tidb (Go): TiDB is an open source distributed HTAP database compatible with the MySQL protocol
-
SheetJS js-xlsx
:green_book: SheetJS Community Edition -- Spreadsheet Data Toolkit
-
Metabase
The simplest, fastest way to get business intelligence and analytics to everyone in your company :yum:
-
Sequelize
An easy-to-use multi SQL dialect ORM tool for Node.js
Latest mention: How to work with MySQL more like on array? (To update it easily, edit and so on) | reddit.com/r/learnjavascript | 2021-01-12You can write these yourself, or just use one that already exists - a popular option for Node is Sequelize: https://github.com/sequelize/sequelize/
-
LevelDB
LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.
I decided to dig through a database source code to answer that question. There are plenty of databases that use mmap. Some of them decided to not use anymore. Some examples: SQLite has an option of accessing disk content directly using memory-mapped I/O[1], it seems LevelDB used to use but it changed it[2], Lucene has an option with MMapDirectory[3], LMDB uses mmap[4], a simple key/value in-memory database from Counchbase called moss uses mmap for durability of in-memory data[5] and MongoDB removed mmap storage engine for WiredTiger[6].
-
TypeORM
ORM for TypeScript and JavaScript (ES7, ES6, ES5). Supports MySQL, PostgreSQL, MariaDB, SQLite, MS SQL Server, Oracle, SAP Hana, WebSQL databases. Works in NodeJS, Browser, Ionic, Cordova and Electron platforms.
Latest mention: Any ORMs that don't use classes for entities? | reddit.com/r/typescript | 2021-01-26That's fair! FWIW, I don't think you get that kind of certainty with any open source tools? TypeORM had a crazy boost in popularity when it was released and after ~2 years of intense development it was basically dropped and now has a very unclear future. And I don't have a crystal ball to see what's going to happen to Prisma, I just know that we have an amazing team and things are looking incredibly good at the moment. But can I guarantee you that the Prisma open source tools with be around in their current form in 10 years? Nope, in the same way that I can't make these guarantees about any other OSS tools.
-
Mongoose
MongoDB object modeling designed to work in an asynchronous environment.
Latest mention: The right way to make advanced and efficient MongoDB pagination | dev.to | 2021-01-25Because we use mongoose, as ODM, first of all, we take a look at its plugins. The most popular of them, that related to pagination are mongoose-paginate *Pagination plugin for Mongoose Note: This plugin will only work with Node.js >= 4.0 and Mongoose >= 4.0. Add plugin to…*www.npmjs.com mongoose-paginate-v2 *A cursor based custom pagination library for Mongoose with customizable labels.*www.npmjs.com mongoose-aggregate-paginate *mongoose-aggregate-paginate is a Mongoose plugin easy to add pagination for aggregates. This plugin can be used in…*www.npmjs.com mongoose-aggregate-paginate-v2 *A cursor based custom aggregate pagination library for Mongoose with customizable labels. If you are looking for basic…*www.npmjs.com
-
InfluxDB
Scalable datastore for metrics, events, and real-time analytics
Latest mention: Cannot set the Window Period when creating a check in the Alerts page. Anybody else having this issue? Just greyed out on the auto setting... Using v2.0.3 docker image | reddit.com/r/influxdb | 2021-01-19Can you file an issue on https://github.com/influxdata/influxdb/issues? That way I can have an engineer look into it
-
cockroach
CockroachDB - the open source, cloud-native distributed SQL database.
However if you ever fancy going for a newer tech, CockroachDB is a recent take on distributed SQL databases that's built with scalability in mind. Though from i gathered, deploying it is kind of an experiment in itself.
-
SQLMap
Automatic SQL injection and database takeover tool
sqlmap does have OS detection capabilities. Official documentation:
-
MongoDB
The MongoDB Database
Latest mention: Official packages for MongoDB and its MongoDB Compass | reddit.com/r/ManjaroLinux | 2020-12-29 -
RocksDB
A library that provides an embeddable, persistent key-value store for fast storage.
W.r.t. standard dictionaries, it's something we're interested in, but the fundamental reality of dictionaries is that their effectiveness is strongly tied to their specificity. Put another way, a universal dictionary is a self-contradiction.
And yes, totally, I know at least RocksDB supports exactly that behavior [0].
[0] https://github.com/facebook/rocksdb/blob/12f11373554af219c51...
-
prisma1
💾 Database Tools incl. ORM, Migrations and Admin UI (Postgres, MySQL & MongoDB)
Going with a full rewrite was a painful decision in various ways, but I still think it was necessary (and the steam that Prisma 2 is currently picking up seems to prove it right as well). This rewrite of course had cost a lot of resources and we did our best to provide support and maintenance for Prisma 1 while building out Prisma 2. Once the initial release of Prisma 2 was out, we officially put Prisma 1 in maintenance mode which means that we're still looking into critical and security-related issues with Prisma 1 when they're brought to our attention.
-
postgrest
REST API for any Postgres database
Latest mention: PostgREST: REST API for any Postgres database | reddit.com/r/coolgithubprojects | 2021-01-15 -
MySQL
A pure node.js JavaScript Client implementing the MySQL protocol.
If you're using javascript on the back end in node you'll probably want a node module like mysql on npm. This will give you an api to connect to the database and execute queries. Their readme is very informative so I suggest starting there: https://www.npmjs.com/package/mysql#install
-
RxDB
🔄 A realtime Database for JavaScript Applications
-
dgraph
Native GraphQL Database with graph backend
Latest mention: Ask HN: Which are the best go repositories to read to learn the language? | news.ycombinator.com | 2021-01-17I believe the best of the best use of Golang is being done by Dgraph.
They are the The world’s most advanced native GraphQL database with a graph backend.
Checkout their GitHub repo.
-
awesome-cheatsheets
👩💻👨💻 Awesome cheatsheets for popular programming languages, frameworks and development tools. They include everything you should know in one single file.
Awesome cheat sheets
-
Lowdb
⚡️ lowdb is a small local JSON database powered by Lodash (supports Node, Electron and the browser)
lowdb MIGHT be a good choice. It stores everything in a JSON file and uses Lodash for querying. I have used on a couple of personal projects but it isn't very well maintained (over 100 issues). Personally I really like it! I would never use it in anything serious though lol
-
HikariCP
光 HikariCP・A solid, high-performance, JDBC connection pool at last.
Related, about sizing DB connection (counterintuitively): https://github.com/brettwooldridge/HikariCP/wiki/About-Pool-...
-
ClickHouse
ClickHouse® is a free analytics DBMS for big data
Latest mention: Technology for storing/querying very large time series data (~1billion points per day) with ~80 columns | reddit.com/r/Database | 2021-01-14Another solution could be to use a columnar analytical database like ClickHouse. But i dont have any experience with it.
Index
What are some of the best open-source Database projects? This list will help you:
Project | Stars | |
---|---|---|
1 | prometheus | 34,970 |
2 | etcd | 34,514 |
3 | tidb | 26,617 |
4 | SheetJS js-xlsx | 24,345 |
5 | Metabase | 23,621 |
6 | Sequelize | 23,593 |
7 | LevelDB | 22,844 |
8 | TypeORM | 22,792 |
9 | Mongoose | 22,078 |
10 | InfluxDB | 20,340 |
11 | cockroach | 19,804 |
12 | SQLMap | 19,230 |
13 | MongoDB | 19,153 |
14 | RocksDB | 18,658 |
15 | prisma1 | 16,885 |
16 | postgrest | 16,460 |
17 | MySQL | 16,132 |
18 | RxDB | 15,036 |
19 | dgraph | 14,720 |
20 | awesome-cheatsheets | 14,714 |
21 | Lowdb | 14,673 |
22 | HikariCP | 14,477 |
23 | ClickHouse | 14,282 |