Top 8 Java Bigdata Projects

• shardingsphere

  23 19,425 10.0 Java

  Distributed SQL transaction & query engine for data sharding, scaling, encryption, and more - on any database.

  

Project mention: Managing Data Residency - the demo | dev.to | 2023-05-25

Opposite to what the documentation tells, the full prefix is jdbc:shardingsphere:absolutepath. I've opened a PR to fix the documentation.

• hudi

  20 5,053 9.9 Java

  Upserts, Deletes And Incremental Processing on Big Data.

  

Project mention: Getting Started with Flink SQL, Apache Iceberg and DynamoDB Catalog | dev.to | 2023-12-18

Apache Iceberg is one of the three types of lakehouse, the other two are Apache Hudi and Delta Lake.

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• Apache Avro

  22 2,764 9.7 Java

  Apache Avro is a data serialization system.

  

Project mention: Open Table Formats Such as Apache Iceberg Are Inevitable for Analytical Data | news.ycombinator.com | 2024-01-18

Apache AVRO [1] is one but it has been largely replaced by Parquet [2] which is a hybrid row/columnar format

[1] https://avro.apache.org/

• odd-platform

  33 1,108 8.8 Java

  First open-source data discovery and observability platform. We make a life for data practitioners easy so you can focus on your business.

  

Project mention: OpenDataDiscovery 0.15 with Data Deprecation and Metadata Stale | news.ycombinator.com | 2023-08-04

• dataCompare

  1 234 3.7 Java

  big data comparison and data profiling platform: low code，data comparison and data profiling

  

• big-data-pipeline-lambda-arch

  1 161 1.5 Java

  A full big data pipeline (Lambda Architecture) with Spark, Kafka, HDFS and Cassandra.

• hadoopcryptoledger

  7 141 1.8 Java

  Hadoop Crypto Ledger - Analyzing CryptoLedgers, such as Bitcoin Blockchain, on Big Data platforms, such as Hadoop/Spark/Flink/Hive

  

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• rapiddweller-benerator-ce

  3 128 8.5 Java

  BENERATOR is a leading software solution to generate, obfuscate, pseudonymize and migrate data for development, testing, and training purposes with a model-driven approach.

  

Java Bigdata related posts

• For those of you with Lakehouse Architectures, how do you handle duplicate records?
  1 project | /r/dataengineering | 16 Apr 2023
• AWS ACID data lakehouse
  1 project | /r/dataengineering | 30 Jan 2023
• hadoopcryptoledger: NEW Data - star count:139.0
  1 project | /r/algoprojects | 21 Jan 2023
• hadoopcryptoledger: NEW Data - star count:139.0
  1 project | /r/algoprojects | 20 Jan 2023
• hadoopcryptoledger: NEW Data - star count:139.0
  1 project | /r/algoprojects | 19 Jan 2023
• hadoopcryptoledger: NEW Data - star count:139.0
  1 project | /r/algoprojects | 18 Jan 2023
• hadoopcryptoledger: NEW Data - star count:139.0
  1 project | /r/algoprojects | 17 Jan 2023
• A note from our sponsor - InfluxDB
  www.influxdata.com | 26 Apr 2024

  Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality. Learn more →

Index

Sponsored

SaaSHub - Software Alternatives and Reviews

SaaSHub helps you find the best software and product alternatives

www.saashub.com

Do not miss the trending Java projects with our weekly report!