zillion
sqlglot
Our great sponsors
zillion | sqlglot | |
---|---|---|
11 | 56 | |
154 | 5,511 | |
- | - | |
7.2 | 9.9 | |
3 months ago | 2 days ago | |
Python | Python | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
zillion
-
Let's Talk about Joins
I've also been frustrated when testing out tools that kinda keep you locked into one predetermined view, table, or set of tables at a time. I made a semantic data modeling library that puts together queries (and of course joins) for you as it uses a drill-across querying technique, and can also join data across different data sources in a secondary execution layer.
https://github.com/totalhack/zillion
Disclaimer: this project is currently a one man show, though I use it in production at my own company.
-
Ask HN: Show me your half baked project
https://github.com/totalhack/zillion
Semantic data warehousing and analytics tool written in python. It has experimental/half-baked NLP features to query your warehouse by interacting with the semantic layer with AI, instead of the normal approach of having an LLM write SQL and needing to know your entire schema.
- So I watched a few videos about Fabric, and started to cry a little...
-
Zillion - Semantic data modeling and analytics with a sprinkle of AI
Hey All, I wanted to share Zillion -- an open source Python data modeling and analytics library with experimental natural language features powered by OpenAI, LangChain, and Qdrant. Zillion acts as a semantic layer on top of your data, writes SQL so you don't have to, and easily bolts onto existing database infrastructure via SQLAlchemy Core.
-
Ask HN: Most interesting tech you built for just yourself?
Built it for me, but available to all -- Zillion: a python data modeling and analytics library.
https://github.com/totalhack/zillion
-
Zillion - Data modeling and analytics with a sprinkle of AI
More details/docs can be found in the GitHub repo: https://github.com/totalhack/zillion
-
πΌπ¬ BabyDS: An AI powered Data Analysis pipeline
Nice work. I had considered implementing something similar in https://github.com/totalhack/zillion down the road, probably as a layer on top.
-
Ask HN: Those making $0/month or less on side projects β Show and tell
Zillion: https://github.com/totalhack/zillion
A python data warehousing / modeling / analytics library that can unify multiple datasources and writes SQL for you. It's alpha level at the moment and I just slowly chip away when time allows, though I'm using it in production in another project (which does make money).
-
Replacing a SQL analyst with 26 recursive GPT prompts
This seems fun, but certainly unnecessary. All of those questions could be answered in seconds using a warehouse tool like Looker or Metabase or https://github.com/totalhack/zillion (disclaimer: I'm the author and this is alpha-level stuff, though I use it regularly).
-
PRQL a simple, powerful, pipelined SQL replacement
At first glance this seems more confusing, particularly the grouping/aggregation syntax, though I suppose that's something I'd just get used to. Some of the syntactic sugar is nice, but some things are also unlike SQL for no apparent reason which just makes adoption harder than necessary (join syntax for example).
IMO the main selling point would be the "database agnostic" part, but I already achieve that through SQLAlchemy Core and/or a warehouse layer like https://github.com/totalhack/zillion (disclaimer: I'm the author and this is alpha-level stuff, though I use it regularly). It seems like many newer DB technologies/services I'd want to use either speak PostgreSQL or MySQL wire protocol anyway.
The roadmap is worth a read, as it notes some limitations and expected challenges supporting the wide variety of DBMS features and syntax. That said, I can see where this might be useful in the cases where I do have to jump into direct SQL, but want the flexibility to easily switch the back end DB for that code -- that's assuming it can cover the use cases that forced me to write direct SQL in the first place though.
sqlglot
-
The Future of MySQL is PostgreSQL: an extension for the MySQL wire protocol
This is probably referring to "zero changes to your driver code" and not "zero changes to the SQL you send over this driver".
Translating between SQL dialects is notoriously hard and attempts to translate [1] are working in 95% of cases. But the last 5% would require 5x amount of work. That's because "SQL dialect" also includes weird edge cases of type inference of things like COALESCE(5, FALSE) and emulation of system catalogs (pg_catalog, information_schema).
[1] https://github.com/tobymao/sqlglot
- FLaNK AI Weekly 18 March 2024
- SQLGlot: No-dependency SQL parser, transpiler, optimizer for 21 SQL dialects
-
Transpile Any SQL to PostgreSQL Dialect
Recommend checking out https://github.com/tobymao/sqlglot if you are interested in this capability for other SQL dialects
Tools like this are helpful for:
- Rendering SQL in a consistent way, eg for snapshot testing
-
This Week In Python
sqlglot β Python SQL Parser and Transpiler
- SQLglot: Python SQL Parser and Transpiler
-
Build the dependency graph of your BigQuery pipelines at no cost: a Python implementation
In the project we used Python lib networkx and a DiGraph object (Direct Graph). To detect a table reference in a Query, we use sqlglot, a SQL parser (among other things) that works well with Bigquery.
- A Primer on SQLGlot's Abstract Syntax Tree
-
Show HN: SQL Polyglot
Cool! Is this built with sqlglot[1] on the back end?
[1] https://github.com/tobymao/sqlglot
-
sqlglot - Amazing SQL parsing library
Wanted to give sqlglot a shoutout as it saved me a ton of time.
What are some alternatives?
endoflife.date - Informative site with EoL dates of everything
sqloxide - Python bindings for sqlparser-rs
scikit-learn-intelex - Intel(R) Extension for Scikit-learn is a seamless way to speed up your Scikit-learn application
JSqlParser - JSqlParser parses an SQL statement and translate it into a hierarchy of Java classes. The generated hierarchy can be navigated using the Visitor Pattern
objectiv-analytics - Open-source product analytics infrastructure for data teams that want full control. Built for high quality data collection and ready to use for advanced analytics & ML.
Transcrypt - Python 3.9 to JavaScript compiler - Lean, fast, open! -
nature - π The Nature Programming Language, may you be able to experience the joy of programming.
zetasql - ZetaSQL - Analyzer Framework for SQL
Skytrax-Data-Warehouse - A full data warehouse infrastructure with ETL pipelines running inside docker on Apache Airflow for data orchestration, AWS Redshift for cloud data warehouse and Metabase to serve the needs of data visualizations such as analytical dashboards.
duckdb - DuckDB is an in-process SQL OLAP Database Management System
DB-GPT-Hub - A repository that contains models, datasets, and fine-tuning techniques for DB-GPT, with the purpose of enhancing model performance in Text-to-SQL
criterion.rs - Statistics-driven benchmarking library for Rust