SQLGlot: SQL parser, transpiler, optimizer – translate to Presto, Spark, Hive

This page summarizes the projects mentioned and recommended in the original post on news.ycombinator.com

Our great sponsors
  • InfluxDB - Power Real-Time Data Analytics at Scale
  • WorkOS - The modern identity platform for B2B SaaS
  • SaaSHub - Software Alternatives and Reviews
  • sqlglot

    Python SQL Parser and Transpiler

  • You should try it out and see. Here are the test cases

    https://github.com/tobymao/sqlglot/blob/main/tests/fixtures/...

    I can even parse and optimizer TPC-H https://github.com/tobymao/sqlglot/blob/main/tests/fixtures/....

    SQLGlot's parser handles a superset of SQL, so it's much more flexible than any one engine.

    SQLGlot is great. We've used it to build a FOSS probabilistic data linking library[1] that's capable of executing against a variety of SQL backends (Spark, Presto, DuckDB, Sqlite), significantly widening our potential user base.

    We implement all our logic in SQL, and then use SQLGlot to transpile to the target execution engine. One big motivation was to futureproof our work - we're no longer tied down to Spark, and as the 'next big thing' (GPU accelerated SQL for analytics?) comes along, it should be relatively straightforward to support it by writing another adaptor.

    Working on this has highlighted some of the really tricky problems associated with translating between SQL engines, and we haven't hit any major problems, so kudos to the author!

    [1] https://github.com/moj-analytical-services/splink/tree/splin...

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • duckdb

    DuckDB is an in-process SQL OLAP Database Management System

  • zetasql

    ZetaSQL - Analyzer Framework for SQL

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts