#Database

Open-source projects categorized as Database | Edit details
Related topics: #SQL #Postgresql #MySQL #ORM #Golang

Top 23 Database Open-Source Projects

  • GitHub repo prometheus

    The Prometheus monitoring system and time series database.

    Project mention: It's taken way too long, but part 4 on my home network setup is now available! | reddit.com/r/homelab | 2021-05-15

    Two raspberry pi's running PiHole with gravity-sync and pihole-exporter (for prometheus.io monitoring)

  • GitHub repo etcd

    Distributed reliable key-value store for the most critical data of a distributed system

    Project mention: Automatic Configuration Reloading in Java Applications on Kubernetes | dev.to | 2021-05-02

    If you need your configuration changes to be rolled out more immediate, there are other options as well. Rather than reading from a properties file, you could use a key-value store such as Consul, etcd, or AWS Systems Manager Parameter Store. While this gives you more direct control of configuration changes, it introduces new challenges. First, managing your configuration as code might require additional tooling, such as defining them as Terraform resources. Additionally, your application will have to know how to speak to the configuration services, including a proper authentication mechanism.

  • GitHub repo tidb

    TiDB is an open source distributed HTAP database compatible with the MySQL protocol

    Project mention: TiGraph: 8,700x Computing Performance Achieved by Combining Graphs + the RDBMS Syntax | dev.to | 2021-04-05

    The three hackers on the TiGraph team are all top developers in the TiDB community:

  • GitHub repo SheetJS js-xlsx

    :green_book: SheetJS Community Edition -- Spreadsheet Data Toolkit

    Project mention: Export to Excel | reddit.com/r/webdev | 2021-04-27
  • GitHub repo Metabase

    The simplest, fastest way to get business intelligence and analytics to everyone in your company :yum:

    Project mention: Open source contributions for a Data Engineer? | reddit.com/r/dataengineering | 2021-04-16

    If you want to work more on the visualization side maybe Metabase, Superset and Streamlit.

  • GitHub repo LevelDB

    LevelDB is a fast key-value storage library written at Google that provides an ordered mapping from string keys to string values.

    Project mention: Hosting SQLite databases on GitHub Pages (or any static file hoster) | news.ycombinator.com | 2021-05-02

    If you just do an occasional key/value lookup, you don't need 1.2 MiB of WebAssembly. [1]

    I'd solve it via sharding: divide the database into N pieces via range- or mod-sharding. [1] Choose an N that's large enough for each piece to be reasonably small. When you look up a key, fetch the shard of interest.

    You can put each piece into separate files (a little simpler to code, and most static servers will use pre-gzipped files for "Content-Encoding: gzip requests" easily, but you waste more disk space due to internal fragmentation) or one file (with range serving and an index of the byte range offset for each piece).

    The format for each piece can be anything, eg json (simple) or an sstable-like format (more efficient). [3]

    [1] Content-Length of https://phiresky.github.io/youtube-sponsorship-stats/sql-was...

    [2] mod-sharding means: piece[i] has all the keys where hash(key) % N = i.

    [3] https://github.com/google/leveldb/blob/master/doc/table_form...

  • GitHub repo Sequelize

    An easy-to-use multi SQL dialect ORM tool for Node.js

    Project mention: Dialect differences between SQLite and Postgres in Sequelize | dev.to | 2021-05-10

    Grouparoo uses the Sequelize Object Relational Mapper, or ORM, along with sequelize-typescript so we can work with the same Objects in our codebase, regardless of the database providing persistence. Sequelize does a great job of abstracting away the differences between the database types... most of the time. In this blog post, I’ll be sharing the times when the differences in the SQL implementations of Postgres and SQLite matter.

  • GitHub repo TypeORM

    ORM for TypeScript and JavaScript (ES7, ES6, ES5). Supports MySQL, PostgreSQL, MariaDB, SQLite, MS SQL Server, Oracle, SAP Hana, WebSQL databases. Works in NodeJS, Browser, Ionic, Cordova and Electron platforms.

    Project mention: Any Laravel Eloquent like ORM for node.js? | reddit.com/r/node | 2021-05-08

    You could also checkout https://typeorm.io/

  • GitHub repo Mongoose

    MongoDB object modeling designed to work in an asynchronous environment.

    Project mention: Five conductive - and five innovative npm packages | dev.to | 2021-05-15

    If you've seen Knex in the upper section and thought: No way I'll give up my NoSQL or GraphQL way or writing queries - this module means good news. Prisma uses its own .prisma filetype to declare its schemata. The way these can be used to query data looks a lot like the MongoDB's Mongoose. While being relatively big in size, it offers a bunch of useful interfaces and tools for fast prototyping, migration and development.

  • GitHub repo InfluxDB

    Scalable datastore for metrics, events, and real-time analytics

    Project mention: Clarification on bundling with influxdb 2.0? | reddit.com/r/influxdb | 2021-04-10

    Am I missing something? Is the all-in-one TICK stack only a feature of pre-compiled binaries? Should I expect a build from https://github.com/influxdata/influxdb.git to contain a full TICK stack?

  • GitHub repo cockroach

    CockroachDB - the open source, cloud-native distributed SQL database.

    Project mention: #30DaysofAppwrite : Appwrite’s building blocks | dev.to | 2021-05-03

    Appwrite uses MariaDB as the default database for project collections, documents, and all other metadata. Appwrite is agnostic to the database you use under the hood and support for more databases like Postgres, CockroachDB, MySQL and MongoDB is currently under active development! 😊

  • GitHub repo SQLMap

    Automatic SQL injection and database takeover tool

    Project mention: sqlmap cannot find inject on the testfire.net | reddit.com/r/Kalilinux | 2021-05-12

    I found the same question on GitHub, but the payload given in the answer doesn't work.

  • GitHub repo RocksDB

    A library that provides an embeddable, persistent key-value store for fast storage.

    Project mention: Has anyone managed to upgrade to v16 with Rook? | reddit.com/r/ceph | 2021-05-07

    Fix RocksDB SIGILL error on Raspberry PI 4

  • GitHub repo MongoDB

    The MongoDB Database

    Project mention: 5 Reasons to use Node.js along with React.js | dev.to | 2021-05-15

    The web applications that use mongoDB, express.js, react.js and node.js. Using this we can create applications like e-commerce websites, banking websites and many more. React plus node will help us building such applications with ease.

  • GitHub repo awesome-cheatsheets

    👩‍💻👨‍💻 Awesome cheatsheets for popular programming languages, frameworks and development tools. They include everything you should know in one single file.

    Project mention: Github repositories for Web Development via twitter Pratham. | reddit.com/r/webdev | 2021-03-15

    Awesome cheat-sheets for popular programming languages, frameworks and development tools

  • GitHub repo postgrest

    REST API for any Postgres database

    Project mention: Going full-stack with Flutter and Supabase - Part 1: Authentication | dev.to | 2021-05-14

    It's all open source: Supabase is built on top of other popular open-source packages, such as PostgREST for accessing your database directly from the client, and GoTrue for user authentication. All that, along with a nice admin interface so you can manage everything. If you wanted, you could host and manage these services on your own. If PostgREST does not fit your use case, you could host a server that will communicate with your Postgres database directly. And if you ever decide that Supabase is not for you, you could always migrate all your data somewhere else. At the end of the day, it's all in a Postgres database.

  • GitHub repo prisma1

    💾 Database Tools incl. ORM, Migrations and Admin UI (Postgres, MySQL & MongoDB)

    Project mention: Journey to the real world by cloning DEV.to backend server. | dev.to | 2021-05-12

    Prisma is a Next-Gen javascript and typescript ORM . It generate types for your model and provide highlight during development to work faster. You can explore prisma 2 here

  • GitHub repo MySQL

    A pure node.js JavaScript Client implementing the MySQL protocol.

    Project mention: Update on the malicious commits to PHP codebase | reddit.com/r/programming | 2021-04-07

    And if you wonder who else does this, well... For instance, the most popular mysql driver for NodeJs out there https://github.com/mysqljs/mysql#escaping-query-values still doesn't support prepared statements, and still does client-side character escaping.

  • GitHub repo dgraph

    Native GraphQL Database with graph backend

    Project mention: Need help in choosing a database - Postgres or BadgerDB | reddit.com/r/Database | 2021-05-08

    Dgraph is a highly scalable hyper fast graph database that is distributed, and is built on top of Badger. For consensus, it Raft protocol. (Git repo https://github.com/dgraph-io/dgraph)

  • GitHub repo ClickHouse

    ClickHouse® is a free analytics DBMS for big data

    Project mention: Little Analyst in a Big Data Pond | reddit.com/r/datascience | 2021-04-29

    As many have already mentioned this is more of data engineering than Data Science one. Try to build ETL pipelines for storing the data to a data lake or data warehouse(more organised). Make sure the pipelines are reliable and fall back mechanism to ensure consistency. Check out open source DBs https://clickhouse.tech (open-source OLAP database management system) or you can get started with Postgres as well. https://airbyte.io is an open source project which provides data integrations/pipelines.

  • GitHub repo RxDB

    🔄 A realtime Database for JavaScript Applications

    Project mention: Web App running on Offline mode | reddit.com/r/webdev | 2021-05-11
  • GitHub repo Lowdb

    ⚡️ lowdb is a small local JSON database powered by Lodash (supports Node, Electron and the browser)

    Project mention: Group chat bot that assign points to members | reddit.com/r/Telegram | 2021-04-17

    you can use lowdb for database system. It is simple and easy. lowdb link npm

  • GitHub repo HikariCP

    光 HikariCP・A solid, high-performance, JDBC connection pool at last.

    Project mention: How big should your connection pool be? | news.ycombinator.com | 2021-04-30
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2021-05-15.

Index

What are some of the best open-source Database projects? This list will help you:

Project Stars
1 prometheus 36,619
2 etcd 35,802
3 tidb 27,784
4 SheetJS js-xlsx 25,473
5 Metabase 24,780
6 LevelDB 24,674
7 Sequelize 24,268
8 TypeORM 24,154
9 Mongoose 22,630
10 InfluxDB 21,418
11 cockroach 20,497
12 SQLMap 20,057
13 RocksDB 19,733
14 MongoDB 19,707
15 awesome-cheatsheets 19,672
16 postgrest 17,003
17 prisma1 16,861
18 MySQL 16,413
19 dgraph 16,001
20 ClickHouse 15,867
21 RxDB 15,558
22 Lowdb 15,318
23 HikariCP 14,984