Our great sponsors
-
prql
PRQL is a modern language for transforming data — a simple, powerful, pipelined SQL replacement
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
2. Point and click through the IDE treeview.
Honestly, I don't think it helps a whole lot beyond this functionality, but I can see why folks who are accustomed to thinking in functional pipelines (from -> select -> map -> filter -> collect) can prefer this way of querying.
I think PRQL is one attempt at building something this way.[1]
I think these are great suggestions.
It seems like you're suggesting that someone could design a functional-style programming language that compiles to SQL.
2 & 3 are my biggest pain points. I can't just extract functions like I can with a regular programming language. Instead, SQL queries get increasingly complex with no great tools to manage that.
For 3, products like https://materialize.com/ look interesting for being able to create derived materialized views that can efficiently be kept up to date.
Cool! I do this, but I haven't seen anyone else do it. Is any of your code public?
I wrote about it at https://sive.rs/pg
and posted my SQL shopping cart at https://github.com/sivers/store
Please contact me if you'd like to share tips: https://sive.rs/contact
Ibis might be an option. It has syntax similar to pandas and can compile to a number of types of sql, pyspark, or dask.
Related posts
- How moving from Pandas to Polars made me write better code without writing better code
- I used multiprocessing and multithreading at the same time to drop the execution time of my code from 155+ seconds to just over 2+ seconds
- The Data Engineer Roadmap 🗺
- FLaNK AI Weekly 18 March 2024
- SQLite + Rust: Building a CLI Password Vault 🦀