duckdb

Open-source projects categorized as duckdb
Edit details
Language: + Python + TypeScript + Rust + Go + Vue + C# + PLpgSQL + C + C++
Topics: SQL Database Python Postgresql Bigquery
Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured

Top 23 duckdb Open-Source Projects

  • sqlglot

    Python SQL Parser and Transpiler

    • Project mention: The Future of MySQL is PostgreSQL: an extension for the MySQL wire protocol | news.ycombinator.com | 2024-04-26

    This is probably referring to "zero changes to your driver code" and not "zero changes to the SQL you send over this driver".

    Translating between SQL dialects is notoriously hard and attempts to translate [1] are working in 95% of cases. But the last 5% would require 5x amount of work. That's because "SQL dialect" also includes weird edge cases of type inference of things like COALESCE(5, FALSE) and emulation of system catalogs (pg_catalog, information_schema).

    [1] https://github.com/tobymao/sqlglot

  • ibis

    the portable Python dataframe library

    • Project mention: Show HN: Hashquery, a Python library for defining reusable analysis | news.ycombinator.com | 2024-04-23

    I really don't understand the appeal of dbt vs a proper programming language. The templating approach leads to massive spaghetti. I look forward to trying out something like Ibis [0]

    0: https://ibis-project.org/

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo

  • lance

    Modern columnar data format for ML and LLMs implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, with more integrations coming..

    • Project mention: The Nimble File Format by Meta | news.ycombinator.com | 2024-04-25

  • tad

    A desktop application for viewing and analyzing tabular data

    • Project mention: Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL | news.ycombinator.com | 2024-03-15

    Very impressive project and vision! Love the demo!

    I am also ex-GS and worked on what I am fairly sure is the table display tool you're describing. I tried to carry the essential aspects of that work (multi-level pivots, with drill-down to the leaf level, and all interactive events and analytics supported by db queries) to Tad (https://www.tadviewer.com/, https://github.com/antonycourtney/tad), another open source project powered by DuckDb.

    An embeddable version of Tad, powered by DuckDb WASM, is used as the results viewer in the MotherDuck Web UI (https://app.motherduck.com/).

    If you're interested in embedding Tad in Pretzel, or leveraging pieces of it in your work, or collaborating on other aspects of DuckDb WASM powered UIs, please get in touch!

  • ingestr

    ingestr is a CLI tool to copy data between any databases with a single command seamlessly.

    • Project mention: FLaNK 04 March 2024 | dev.to | 2024-03-04

  • pretzelai

    Open-source, browser-local data exploration using DuckDB-Wasm and PRQL

    • Project mention: Using Google Sheets as the back end/APIs of your app | news.ycombinator.com | 2024-04-12

  • rill

    Rill is a tool for effortlessly transforming data sets into powerful, opinionated dashboards using SQL. BI-as-code. (by rilldata)

    • Project mention: Governments on GitHub | news.ycombinator.com | 2023-06-09

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo

  • splink

    Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends

    • Project mention: Splink: Fast, accurate, scalable probabilistic data linkage | news.ycombinator.com | 2024-03-13

  • dbt-duckdb

    dbt (http://getdbt.com) adapter for DuckDB (http://duckdb.org)

  • jupysql

    Better SQL in Jupyter. 📊

    • Project mention: Show HN: JupySQL – a SQL client for Jupyter (ipython-SQL successor) | news.ycombinator.com | 2023-12-06

    Hey, HN community!

    We're stoked to launch JupySQL today! JupySQL is an open-source library that brings a modern SQL experience to Jupyter. JupySQL is compatible with all major databases, such as Snowflake, Redshift, PostgreSQL, MySQL, MariaDB, DuckDB, SQL Server, Clickhouse, Trino, and more!

    To get started, check out our tutorial: https://jupysql.ploomber.io/en/latest/quick-start.html

    SQL is the defacto language for data analysis; however, analysis often requires a mix of SQL and Python. JupySQL bridges this gap, allowing users to execute SQL queries seamlessly in Jupyter and continue their analysis in Python. Add %%sql to the top of your cell and start writing SQL.

    Here are some of JupySQL's main features:

    - Syntax highlighting

  • vulcan-sql

    Data API Framework for AI Agents and Data Apps

    • Project mention: Shout out to Appsmith developers to check out this new tool! | /r/lowcode | 2023-07-09

    I am one of the members of an open-source project VulcanSQL, a Data API Framework for data applications that helps data folks create and share data APIs faster.

  • inline-sql

    🪄 Inline SQL in any Python program

  • WhatTheDuck

    WhatTheDuck is an open-source web application built on DuckDB. It allows users to upload CSV files, store them in tables, and perform SQL queries on the data.

    • Project mention: Show HN: WhatTheDuck – open-source, in-browser SQL on CSV files | news.ycombinator.com | 2024-03-26

  • duckdb-rs

    Ergonomic bindings to duckdb for Rust

  • puffin

    Serverless HTAP cloud data platform powered by Arrow × DuckDB × Iceberg (by sutoiku)

  • DuckDB.NET

    Bindings and ADO.NET Provider for DuckDB

    • Project mention: Announcing DuckDB 0.8.0 | /r/programming | 2023-05-18

    0.8.0 version of DuckDB provider for .NET was released too.

  • duckdb_fdw

    DuckDB Foreign Data Wrapper for PostgreSQL

  • sqlite_scanner

    DuckDB extension to read and write to SQLite databases

  • WrenAI

    WrenAI makes Text-to-SQL simple and accurate. Natural Language Interface to Your Data.

    • Project mention: WrenAI: Open-Source Natural Language Interface to Your Data | news.ycombinator.com | 2024-04-25

  • quack-reduce

    A playground for running duckdb as a stateless query engine over a data lake

    • Project mention: quack-reduce: duckdb as a stateless query engine over a data lake | news.ycombinator.com | 2024-01-27

  • portable-data-stack-dagster

    A portable Datamart and Business Intelligence suite built with Docker, Dagster, dbt, DuckDB, PostgreSQL and Superset

    • Project mention: Portable Data Stack | news.ycombinator.com | 2023-09-20

  • cuallee

    Possibly the fastest DataFrame-agnostic quality check library in town.

    • Project mention: Show HN: Snowflake Data Quality Checks in Python | news.ycombinator.com | 2024-02-11

  • h3-duckdb

    Bindings for H3 to DuckDB

    • Project mention: PostGEESE? Introducing the DuckDB Spatial Extension | news.ycombinator.com | 2023-07-06

  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

    SaaSHub logo
NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

duckdb related posts

  • WrenAI: Open-Source Natural Language Interface to Your Data

    1 project | news.ycombinator.com | 25 Apr 2024

  • Using Google Sheets as the back end/APIs of your app

    11 projects | news.ycombinator.com | 12 Apr 2024

  • Show HN: WhatTheDuck – open-source, in-browser SQL on CSV files

    5 projects | news.ycombinator.com | 26 Mar 2024

  • Show HN: Open-source, browser-local data exploration using DuckDB-WASM and PRQL

    11 projects | news.ycombinator.com | 15 Mar 2024

  • quack-reduce: duckdb as a stateless query engine over a data lake

    1 project | news.ycombinator.com | 27 Jan 2024

  • JupySQL: Connecting to a SQL database from Jupyter

    1 project | /r/SQL | 9 Sep 2023

  • GitHub - ploomber/jupysql: Better SQL in Jupyter. 📊

    1 project | /r/coolgithubprojects | 6 Sep 2023
  • A note from our sponsor - InfluxDB
    www.influxdata.com | 3 May 2024
    Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →

Index

What are some of the best open-source duckdb projects? This list will help you:

Project Stars
1 sqlglot 5,511
2 ibis 4,208
3 lance 3,275
4 tad 3,016
5 ingestr 2,331
6 pretzelai 1,432
7 rill 1,348
8 splink 1,091
9 dbt-duckdb 736
10 jupysql 605
11 vulcan-sql 594
12 inline-sql 412
13 WhatTheDuck 405
14 duckdb-rs 365
15 puffin 277
16 DuckDB.NET 278
17 duckdb_fdw 235
18 sqlite_scanner 185
19 WrenAI 185
20 quack-reduce 122
21 portable-data-stack-dagster 116
22 cuallee 107
23 h3-duckdb 107

Sponsored
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com