1brc
mmap-go
1brc | mmap-go | |
---|---|---|
5 | 3 | |
69 | 889 | |
- | - | |
7.2 | 0.0 | |
21 days ago | over 1 year ago | |
C | Go | |
- | BSD 3-clause "New" or "Revised" License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
1brc
-
The One Billion Row Challenge in CUDA: from 17 minutes to 17 seconds
There are some good ideas for this type of problem here: https://github.com/dannyvankooten/1brc
After you deal with parsing and hashes, basically you are IO limited so mmap helps. A reasonable guess is that even for the optimal CUDA implementation, because there is no compute to speak of other than a hashmap, the starting of kernels and transfer of data to the GPU would likely add a noticeable bottleneck and make the optimal CUDA code slower than this pure C code.
-
The One Billion Row Challenge in Go: from 1m45s to 4s in nine solutions
c dominates every other language again...https://github.com/dannyvankooten/1brc#submitting
-
The One Billion Row Challenge
You can run the bin/create-sample program from this C implementation here: https://github.com/dannyvankooten/1brc
Itβs just the city names + averages from the official repository using a normal distribution to generate 1B random rows.
mmap-go
-
The One Billion Row Challenge in Go: from 1m45s to 4s in nine solutions
Well, I guess it's more that the standard library doesn't have a cross-platform way to access them, not that memory-mapped files themselves can't be done on (say) Windows. It looks like there's a fairly popular 3rd party package that supports at least Linux, macOS, and Windows: https://github.com/edsrzf/mmap-go
-
llama.go - Meta's LLaMA GPT inference in pure Golang
Can you use https://github.com/edsrzf/mmap-go to speed up loading
-
Discovering and exploring mmap using Go
We're going to explore more mmap functionalities from the point of view of the API provided by mmap-go. There are probably more features that the native syscall provides that this library does not implement.
What are some alternatives?
1brc - 1οΈβ£πποΈ The One Billion Row Challenge -- A fun exploration of how quickly 1B rows from a text file can be aggregated with Java
llama.go - llama.go is like llama.cpp in pure Golang!
nodejs - 1οΈβ£πποΈ The One Billion Row Challenge with Node.js -- A fun exploration of how quickly 1B rows from a text file can be aggregated with different languages.
JDK - JDK main-line development https://openjdk.org/projects/jdk
1brc - 1BRC in .NET among fastest on Linux