Top 13 spark-sql Open-Source Projects

Redash

38 24,948 9.5 Python

Make Your Company Data Driven. Connect to any data source, easily visualize, dashboard and share your data.

Project mention: Redash: Connect to data source, easily visualize, dashboard and share your data | news.ycombinator.com | 2024-03-20

spark

3 1,997 0.0 C#

.NET for Apache® Spark™ makes Apache Spark™ easily accessible to .NET developers. (by dotnet)
InfluxDB

www.influxdata.com sponsored

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
kyuubi

1 1,928 9.8 Scala

Apache Kyuubi is a distributed and multi-tenant gateway to provide serverless SQL on data warehouses and lakehouses.
Jupyter Scala

6 1,562 9.0 Scala

A Scala kernel for Jupyter
LearningSparkV2

1 1,059 0.0 Scala

This is the github repo for Learning Spark: Lightning-Fast Data Analytics [2nd Edition]
incubator-gluten

2 976 9.9 Scala

Gluten is a middle layer responsible for offloading JVM-based SQL engines' execution to native engines.

Project mention: FLaNK Stack for 04 December 2023 | dev.to | 2023-12-04

jupysql

8 598 9.3 Python

Better SQL in Jupyter. 📊

Project mention: Show HN: JupySQL – a SQL client for Jupyter (ipython-SQL successor) | news.ycombinator.com | 2023-12-06

Hey, HN community!
We're stoked to launch JupySQL today! JupySQL is an open-source library that brings a modern SQL experience to Jupyter. JupySQL is compatible with all major databases, such as Snowflake, Redshift, PostgreSQL, MySQL, MariaDB, DuckDB, SQL Server, Clickhouse, Trino, and more!
To get started, check out our tutorial: https://jupysql.ploomber.io/en/latest/quick-start.html
SQL is the defacto language for data analysis; however, analysis often requires a mix of SQL and Python. JupySQL bridges this gap, allowing users to execute SQL queries seamlessly in Jupyter and continue their analysis in Python. Add %%sql to the top of your cell and start writing SQL.
Here are some of JupySQL's main features:
- Syntax highlighting

WorkOS

workos.com sponsored

The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
ngods-stocks

3 354 0.0 Jupyter Notebook

New Generation Opensource Data Stack Demo
cuelake

2 284 0.0 JavaScript

Use SQL to build ELT pipelines on a data lakehouse.
qbeast-spark

12 190 8.6 Scala

Qbeast-spark: DataSource enabling multi-dimensional indexing and efficient data sampling. Big Data, free from the unnecessary!
opaque-sql

2 176 1.1 Scala

An encrypted data analytics platform
Sparkplug

0 28 0.0 Scala

Spark package to "plug" holes in data using SQL based rules ⚡️ 🔌
iceberg-intro-workshop

1 10 10.0 Shell

Hands-on workshop with Apache Iceberg

Project mention: FLaNK Stack Weekly for 13 November 2023 | dev.to | 2023-11-13

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).

Index

What are some of the best open-source spark-sql projects? This list will help you:

	Project	Stars
1	Redash	24,948
2	spark	1,997
3	kyuubi	1,928
4	Jupyter Scala	1,562
5	LearningSparkV2	1,059
6	incubator-gluten	976
7	jupysql	598
8	ngods-stocks	354
9	cuelake	284
10	qbeast-spark	190
11	opaque-sql	176
12	Sparkplug	28
13	iceberg-intro-workshop	10