tree-sitter-sql VS duckdb

(use-package treesit :ensure nil :custom ;; Some stuff taken from here: https://robbmann.io/posts/emacs-treesit-auto/ (treesit-extra-load-path '("/usr/lib64/")) (treesit-language-source-alist '((bash . ("https://github.com/tree-sitter/tree-sitter-bash")) (c . ("https://github.com/tree-sitter/tree-sitter-c")) (c++ . ("https://github.com/tree-sitter/tree-sitter-cpp")) (csharp . ("https://github.com/tree-sitter/tree-sitter-c-sharp")) (css . ("https://github.com/tree-sitter/tree-sitter-css")) (elixir ("https://github.com/elixir-lang/tree-sitter-elixir")) (html . ("https://github.com/tree-sitter/tree-sitter-html")) (java . ("https://github.com/tree-sitter/tree-sitter-java")) (javascript . ("https://github.com/tree-sitter/tree-sitter-javascript")) (json . ("https://github.com/tree-sitter/tree-sitter-json")) (lua . ("https://github.com/Azganoth/tree-sitter-lua")) (makefile . ("https://github.com/alemuller/tree-sitter-make")) (org . ("https://github.com/milisims/tree-sitter-org")) (python . ("https://github.com/tree-sitter/tree-sitter-python")) (tsx . ("https://github.com/tree-sitter/tree-sitter-typescript" "master" "tsx/src")) (typescript . ("https://github.com/tree-sitter/tree-sitter-typescript" "master" "typescript/src")) (ruby . ("https://github.com/tree-sitter/tree-sitter-ruby")) (rust . ("https://github.com/tree-sitter/tree-sitter-rust")) (sql . ("https://github.com/m-novikov/tree-sitter-sql")) (toml . ("https://github.com/tree-sitter/tree-sitter-toml")) (yaml . ("https://github.com/ikatyang/tree-sitter-yaml")))) (major-mode-remap-alist '((c-mode . c-ts-mode) (c++-mode . c++-ts-mode) (csharp-mode . csharp-ts-mode) (css-mode . css-ts-mode) (html-mode . html-ts-mode) (java-mode . java-ts-mode) (js-mode . js-ts-mode) (json-mode . json-ts-mode) (makefile-mode . makefile-ts-mode) ;; (org-mode . org-ts-mode) ;; not mature yet (python-mode . python-ts-mode) (typescript-mode . typescript-ts-mode) (ruby-mode . ruby-ts-mode) (rust-mode . rust-ts-mode) (toml-mode . toml-ts-mode) (yaml-mode . yaml-ts-mode))) (treesit-auto-fallback-alist '((toml-ts-mode . conf-toml-mode) (typescript-ts-mode . nil) (tsx-ts-mode . nil))) (treesit-font-lock-settings t) (treesit-simple-indent t) (treesit-defun-type-regexp t)) (use-package treesit-auto :demand t :config (setq treesit-auto-install t) (global-treesit-auto-mode))

Can something like this https://github.com/m-novikov/tree-sitter-sql help?

I'm not sure if the fix is reassuring or not: https://github.com/duckdb/duckdb/pull/9411/files

It's an interesting question!

The problem is that the data is spread everywhere - no choice about that. So with that in mind, how do you query that data? Today, the idea is that you HAVE to put it into a central location. With tools like Bacalhau[1] and DuckDB [2], you no longer have to - a single query can be sharded amongst all your data - EFFECTIVELY giving you a lot of what you want from a data lake.

It's not a replacement, but if you can do a few of these items WITHOUT moving the data, you will be able to see really significant cost and time savings.

[1] https://github.com/bacalhau-project/bacalhau

[2] https://github.com/duckdb/duckdb

[4] Switch to Push-Based Execution Model by Mytherin · Pull Request #2393 · duckdb/duckdb (github.com)

it depends on your query obviously.

In general, I did very deep benchmarking of pg, clickhouse and duckdb, and I sure didn't make stupid mistakes like this: https://news.ycombinator.com/item?id=36990831

My dataset has 50B rows and 2tb of data, and I think columnar dbs are very overhiped and I chose pg because:

- pg performance is acceptable, maybe 2-3x times slower than clickhouse and duckdb on some queries if pg is configured correctly and run on compressed storage

- clickhouse and duckdb start falling apart very fast because they specialized on very narrow type of queries: https://github.com/ClickHouse/ClickHouse/issues/47520 https://github.com/ClickHouse/ClickHouse/issues/47521 https://github.com/duckdb/duckdb/discussions/6696

This action installs duckdb with the version provided in input.

Duckdb: https://github.com/duckdb/duckdb - seems pretty popular, been keeping an eye on this for close to a year now.

The Parquet-Go library is very complex, not yet success to use it. So I ask whether DuckDB can provide API https://github.com/duckdb/duckdb/issues/7776