Top 6 Java Parquet Projects

Apache Parquet

4 2,407 9.2 Java

Apache Parquet
Apache Drill

9 1,894 8.1 Java

Apache Drill is a distributed MPP query layer for self describing data (by apache)

Project mention: Git Query Language (GQL) Aggregation Functions, Groups, Alias | /r/ProgrammingLanguages | 2023-06-30

Also are you familiar with apache drill . The idea is to put an SQL interpreter in front of any kind of database just like you are doing for git here.

InfluxDB

www.influxdata.com sponsored

Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
parquet-format

4 1,637 7.4 Java

Apache Parquet

Project mention: Summing columns in remote Parquet files using DuckDB | news.ycombinator.com | 2023-11-16

Right, there's all sorts of metadata and often stats included in any parquet file: https://github.com/apache/parquet-format#file-format
The offsets of said metadata are well-defined (i.e. in the footer) so for S3 / blob storage so long as you can efficiently request a range of bytes you can pull the metadata without having to read all the data.

rumble

1 207 8.5 Java

⛈️ RumbleDB 1.21.0 "Hawthorn blossom" 🌳 for Apache Spark | Run queries on your large-scale, messy JSON-like data (JSON, text, CSV, Parquet, ROOT, AVRO, SVM...) | No install required (just a jar to download) | Declarative Machine Learning and more (by RumbleDB)
parquet-floor

1 36 3.4 Java

A lightweight Java library that facilitates reading and writing Apache Parquet files without Hadoop dependencies

Project mention: Welcome to mwmbl, the free, open-source and non-profit search engine | news.ycombinator.com | 2023-09-18

ChatGPT has other failure modes. When a question doesn't have an answer written down somewhere, it really struggles. A case is something like "how do I write a parquet file in Java without using Hadoop".
This not at all trivial but quite possible[1], but ChatGPT will in 100% of the time either hallucinate APIs, disregard the instructions to not use Hadoop or give otherwise plausible but incorrect-looking answers.
The trick is that it isn't doable by simply finding the correct dependencies and API calls, you need extract and override filesystem classes from the Hadoop project to cut those ties.
[1] https://github.com/strategicblue/parquet-floor

delta-fetch

1 1 5.9 Java

HTTP API on Delta Lake tables

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020).