One Billion Row Challenge in Golang - From 95s to 1.96s

This page summarizes the projects mentioned and recommended in the original post on dev.to

InfluxDB - Power Real-Time Data Analytics at Scale
Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
www.influxdata.com
featured
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com
featured
  • 1brc

    1️⃣🐝🏎️ The One Billion Row Challenge -- A fun exploration of how quickly 1B rows from a text file can be aggregated with Java

  • Given that 1-billion-line-file is approximately 13GB, instead of providing a fixed database, the official repository offers a script to generate synthetic data with random readings. Just follow the instructions to create your own database.

  • InfluxDB

    Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

    InfluxDB logo
  • swiss

    Golang port of Abseil's SwissTable

  • Time to deal with the large elephant in the room, the runtime.mapaccess2_fast64 map lookup. Despite spending some hours of research, I couldn't found any viable way to optimize the builtin map. However, there is a community alternative called Swiss Map, which sells itself as faster and more memory efficient than the builtin one. Replacing it is almost a drop-in, with just some syntax changes:

NOTE: The number of mentions on this list indicates mentions on common posts plus user suggested alternatives. Hence, a higher number means a more popular project.

Suggest a related project

Related posts

  • Golang Port of Abseil's SwissTable

    1 project | news.ycombinator.com | 16 Feb 2024
  • [Std] An enhanced std library write with generics

    2 projects | /r/golang | 13 May 2022
  • Go is not an easy language (2021)

    1 project | news.ycombinator.com | 14 Jan 2024
  • v3.1.0 of ttlcache - an in-memory cache with item expiration and generics

    1 project | /r/golang | 25 Aug 2023
  • Functional Programming Library for Golang by IBM

    8 projects | news.ycombinator.com | 17 Aug 2023