Rust SQL

Open-source Rust projects categorized as SQL

Top 23 Rust SQL Projects

  • surrealdb

    A scalable, distributed, collaborative, document-graph database, for the realtime web

    Project mention: Which database should I use? | reddit.com/r/learnprogramming | 2022-11-23

    DB page: https://surrealdb.com

  • cube.js

    📊 Cube — Headless Business Intelligence for Building Data Applications

    Project mention: Is dimensional modelling dead ? | reddit.com/r/dataengineering | 2022-11-20

    This isn't at all a solved problem, and we as an industry are still struggling with the tradeoffs. Handling semantics at the BI layer provides immense flexibility and rapid iteration, but provides little governance and multiple versions of the truth. Consolidating semantics further upstream (dbt's semantic layer, cube.dev) allows for that fabled "single source of truth," but it also means slower development and a loss of business agility.

  • Scout APM

    Truly a developer’s best friend. Scout APM is great for developers who want to find and fix performance issues in their applications. With Scout, we'll take care of the bugs so you can focus on building great things 🚀.

  • diesel

    A safe, extensible ORM and Query Builder for Rust

    Project mention: Using Rust at a startup: A cautionary tale | reddit.com/r/programming | 2022-11-23

    There's also Diesel and SeaORM for higher level interfaces.

  • sqlx

    🧰 The Rust SQL Toolkit. An async, pure Rust SQL crate featuring compile-time checked queries without a DSL. Supports PostgreSQL, MySQL, SQLite, and MSSQL. (by launchbadge)

    Project mention: What's new in SeaQuery 0.27.0 | dev.to | 2022-11-09

    Upgrade sqlx to 0.6.1

  • prql

    PRQL is a modern language for transforming data — a simple, powerful, pipelined SQL replacement

    Project mention: It's not Ruby that's slow, it's your database | news.ycombinator.com | 2022-11-08

    > Take PRQL for example: https://prql-lang.org. It looks nice, but the examples are very basic. What about window functions, grouping sets, lateral, DML, recursive SQL, pattern matching, pivot/unpivot etc.

    [PRQL core-dev here]

    Window functions are very much supported! Check out the examples on the home page & in the docs.

    The others aren't yet, but not because of a policy — we've started with the most frequently used features and adding features as they're needed.

  • databend

    A powerful cloud data warehouse. Built for elasticity and efficiency. Free and open. Also available in the cloud: https://app.databend.com

    Project mention: Sqllogictest Illustrated | reddit.com/r/DatafuseLabs | 2022-09-16

    Here is a quick guidehttps://github.com/datafuselabs/databend/blob/main/tests/logictest/suites/select_0.The supported handlers: MySQL handler, HTTP handler, Clickhouse handlerAnnotation: The use of '--' to annotate specific lines is supported.Statement types:

  • materialize

    The Fastest Way to Build the Fastest Data Products. Build data-intensive applications and services in SQL — without pipelines or caches — using materialized views that are always up-to-date. (by MaterializeInc)

    Project mention: Show HN: Logical Replication with Rust | news.ycombinator.com | 2022-11-24

    Postgres Logical Replication (the ability to listen to the write-ahead-log) is an extremely useful tool that opens up a lot of use cases which has been put to great effect by companies like Materialize[0] and Redhat with Debezium[1].

    Recently there was a discussion here on 'push-based output patterns'[2] where I saw it was not immediately obvious how to implement this pattern with existing tooling.

    This project seeks to demonstrate how to implement logical replication using Rust that can be easily set up and experimented with.

    [0] https://materialize.com/

  • InfluxDB

    Build time-series-based applications quickly and at scale.. InfluxDB is the Time Series Data Platform where developers build real-time applications for analytics, IoT and cloud-native services in less time with less code.

  • toydb

    Distributed SQL database in Rust, written as a learning project

    Project mention: Any ideas for resume | reddit.com/r/rust | 2022-11-26

    Build something you’d like to learn about. Things I’ve considered replicating: A distributed database (see https://github.com/erikgrinaker/toydb), an interpreter (crafting interpreters is a good book), a Ray tracer (http://raytracerchallenge.com/), an RPC compiler and framework, a simpler neural network framework ( https://github.com/pjreddie/darknet)…

  • risingwave

    RisingWave: the next-generation streaming database in the cloud.

    Project mention: Streaming data in Postgres to 1M clients with GraphQL | news.ycombinator.com | 2022-10-16
  • fselect

    Find files with SQL-like queries

    Project mention: A list of new(ish) command line tools – Julia Evans | news.ycombinator.com | 2022-04-12

    Shameless plug: a tool I wrote to manage downloads directory :)

    https://github.com/jhspetersson/fselect

  • sea-orm

    🐚 An async & dynamic ORM for Rust

    Project mention: Using Rust at a startup: A cautionary tale | reddit.com/r/programming | 2022-11-23

    There's also Diesel and SeaORM for higher level interfaces.

  • arrow-datafusion

    Apache Arrow DataFusion SQL Query Engine

    Project mention: Command-line data analytics made easy | news.ycombinator.com | 2022-11-03

    It could be the NDJSON parser (DF source: [0]) or could be a variety of other factors. Looking at the ROAPI release archive [1], it doesn't ship with the definitive `columnq` binary from your comment, so it could also have something to do with compilation-time flags.

    FWIW, we use the Parquet format with DataFusion and get very good speeds similar to DuckDB [2], e.g. 1.5s to run a more complex aggregation query `SELECT date_trunc('month', tpep_pickup_datetime) AS month, COUNT(*) AS total_trips, SUM(total_amount) FROM tripdata GROUP BY 1 ORDER BY 1 ASC)` on a 55M row subset of NY Taxi trip data.

    [0]: https://github.com/apache/arrow-datafusion/blob/master/dataf...

    [1]: https://github.com/roapi/roapi/releases/tag/roapi-v0.8.0

    [2]: https://observablehq.com/@seafowl/benchmarks

  • rust-postgres

    Native PostgreSQL driver for the Rust programming language

    Project mention: Push-Based Outbox Pattern with Postgres Logical Replication | news.ycombinator.com | 2022-10-28
  • roapi

    Create full-fledged APIs for slowly moving datasets without writing a single line of code.

    Project mention: Command-line data analytics made easy | news.ycombinator.com | 2022-11-03

    It could be the NDJSON parser (DF source: [0]) or could be a variety of other factors. Looking at the ROAPI release archive [1], it doesn't ship with the definitive `columnq` binary from your comment, so it could also have something to do with compilation-time flags.

    FWIW, we use the Parquet format with DataFusion and get very good speeds similar to DuckDB [2], e.g. 1.5s to run a more complex aggregation query `SELECT date_trunc('month', tpep_pickup_datetime) AS month, COUNT(*) AS total_trips, SUM(total_amount) FROM tripdata GROUP BY 1 ORDER BY 1 ASC)` on a 55M row subset of NY Taxi trip data.

    [0]: https://github.com/apache/arrow-datafusion/blob/master/dataf...

    [1]: https://github.com/roapi/roapi/releases/tag/roapi-v0.8.0

    [2]: https://observablehq.com/@seafowl/benchmarks

  • gobang

    A cross-platform TUI database management tool written in Rust

    Project mention: How to use Vim or Neovim for SQL? | reddit.com/r/SQLServer | 2022-09-07
  • pg_graphql

    GraphQL support for PostgreSQL

    Project mention: GraphJin – An Instant GraphQL to SQL Compiler | news.ycombinator.com | 2022-05-28

    Check out some of the generated queries this extension [1] pumps out and you might have an answer.

    [1] https://github.com/supabase/pg_graphql

  • gluesql

    GlueSQL is quite sticky. It attaches to anywhere.

    Project mention: GitHub - gluesql/gluesql: SQL Database Engine as a Library | reddit.com/r/programming | 2022-10-23
  • rusqlite

    Ergonomic bindings to SQLite for Rust

    Project mention: Embedded SQL database | reddit.com/r/rust | 2022-07-19

    As far as I know, the only option for an embedded SQL database is SQLite. The most actively maintained one, for rust, seems to be rusqlite (https://github.com/rusqlite/rusqlite).

  • readyset

    ReadySet is a lightweight SQL caching engine written in Rust that helps developers enhance the performance and scalability of existing applications.

    Project mention: PlanetScale Boost serves your SQL queries instantly | news.ycombinator.com | 2022-11-15

    :wave: Author of the paper this work is based on here.

    I'm so excited to see dynamic, partially-stateful data-flow for incremental materialized view maintenance becoming more wide-spread! I continue to think it's a _great_ idea, and the speed-ups (and complexity reduction) it can yield are pretty immense, so seeing more folks building on the idea makes me very happy.

    The PlanetScale blog post references my original "Noria" OSDI paper (https://pdos.csail.mit.edu/papers/noria:osdi18.pdf), but I'd actually recommend my PhD thesis instead (https://jon.thesquareplanet.com/papers/phd-thesis.pdf), as it goes much deeper about some of the technical challenges and solutions involved. It also has a chapter (Appendix A) that covers how it all works by analogy, which the less-technical among the audience may appreciate :) A recording of my thesis defense on this, which may be more digestible than the thesis itself, is also online at https://www.youtube.com/watch?v=GctxvSPIfr8, as well as a shorter talk from a few years earlier at https://www.youtube.com/watch?v=s19G6n0UjsM. And the Noria research prototype (written in Rust) is on GitHub: https://github.com/mit-pdos/noria.

    As others have already mentioned in the comments, I co-founded ReadySet (https://readyset.io/) shortly after graduating specifically to build off of Noria, and they're doing amazing work to provide these kinds of speed-ups for general-purpose relational databases. If you're using one of those, it's worth giving ReadySet a look to get these kinds of speedups there! It's also source-available @ https://github.com/readysettech/readyset if you're curious.

  • sqlparser-rs

    Extensible SQL Lexer and Parser for Rust

    Project mention: Parsing / Recursive Descent Parser | news.ycombinator.com | 2022-05-09

    That code could be copied directly from some real-world examples - sqlparser-rs code looks pretty much exactly the same.

    https://github.com/sqlparser-rs/sqlparser-rs

  • skytable

    Skytable is a fast, secure and reliable realtime NoSQL database with keyspaces, tables, data types, authn/authz, snapshots and more to build powerful apps

    Project mention: skytable / skytable : | reddit.com/r/DevOpsLinks | 2022-08-29
  • cnosdb

    An Open Source Distributed Time Series Database with high performance, high compression ratio and high usability.

  • connector-x

    Fastest library to load data from DB to DataFrames in Rust and Python

    Project mention: Querying Postgres Tables Directly from DuckDB | news.ycombinator.com | 2022-09-30

    I was trying https://github.com/sfu-db/connector-x and hacking around with this https://github.com/spitz-dan-l/postgres-binary-parser but it turned out that a COPY to csv using asyncpg and then converting to parquet was the fastest.

  • Zigi

    Workflow assistant built for devs & their teams. Automate the mundane part of your day, with live actionable messages for your GitHub & Jira tasks.

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2022-11-26.

Rust SQL related posts

Index

What are some of the best open-source SQL projects in Rust? This list will help you:

Project Stars
1 surrealdb 15,642
2 cube.js 14,165
3 diesel 9,530
4 sqlx 7,349
5 prql 5,336
6 databend 4,832
7 materialize 4,657
8 toydb 4,318
9 risingwave 3,561
10 fselect 3,381
11 sea-orm 3,246
12 arrow-datafusion 2,792
13 rust-postgres 2,720
14 roapi 2,515
15 gobang 2,079
16 pg_graphql 2,076
17 gluesql 2,001
18 rusqlite 1,810
19 readyset 1,502
20 sqlparser-rs 1,498
21 skytable 1,376
22 cnosdb 1,093
23 connector-x 891
Clean code begins in your IDE with SonarLint
Up your coding game and discover issues early. SonarLint is a free plugin that helps you find & fix bugs and security issues from the moment you start writing code. Install from your favorite IDE marketplace today.
www.sonarlint.org