adix
countwords
adix | countwords | |
---|---|---|
4 | 43 | |
38 | 209 | |
- | - | |
7.2 | 5.9 | |
11 days ago | about 2 years ago | |
Nim | Rust | |
ISC License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
adix
-
I/O is no longer the bottleneck
Note: Just concatenating the bibles keeps your hash map artificially small...which matters because as you correctly note the big deal is if you can fit the histogram in the L2 cache as noted elsewhere and this really matters if you go parallel where N CPUsL2 caches can speed things up a lot -- until* your histograms blow out CPU-private L2 cache sizes. https://github.com/c-blake/adix/blob/master/tests/wf.nim (or a port to your favorite lang) might make it easy to play with these ideas.
-
A Cost Model for Nim
which is notably logarithmic - not unlike a B-Tree.
When these expectations are exceeded you can at least detect a DoS attack. If you wait until such are seen, you can activate a "more random" mitigation on the fly at about the same cost as "the next resize/re-org/whatnot".
All you need to do is instrument your search to track the depth. There is some example such strategy in Nim at https://github.com/c-blake/adix for simple Robin-Hood Linear Probed tables.
-
Performance comparison: counting words in Python, Go, C++, C, Awk, Forth, Rust
Knuth-McIlroy comes up a lot. Previous discussion [1]. For this example I can make a Nim program run almost exactly the same speed as `wc -w`, yet the optimized C program runs 1.2x faster not 3.34x slower - a whopping 4x discrepancy - much bigger than many of the ratios in the table. So, people should be very cautious about conclusions from any of this.
[1] https://news.ycombinator.com/item?id=24817594
[2] https://github.com/c-blake/adix/blob/master/tests/wf.nim
countwords
-
How fast is really ASP.NET Core?
"dang, I didn't know that was 50x faster than the idiomatic way" or "hey, I didn't know that this implementation in the stdlib prioritized this over that and made this so slow, that's interesting" -- .e.g, there's some kinda neat language details to be found in something like Ben Hoyt's community word count benchmarks repo and 'simple' vs 'optimal' code: https://github.com/benhoyt/countwords
-
Correct name for word matching problem
It benchmarks programs that count the total number of unique words in some input. It's not exactly equivalent to your problem, but it's similarish. All of the programs used some kind of hash map for lookups, but I contributed a program that used a trie. Its performance in my experience varies depending on the CPU interestingly enough. On my old CPU (i7-6900K) it was a little slower, but on my new cpu (i9-12900KS) it was faster.
-
Performance comparison: counting words in Python, C/C++, Awk, Rust, and more
Why not read the source code? :-)
I wrote comments explaining things: https://github.com/benhoyt/countwords/blob/8553c8f600c40a462...
-
do you guys prefer functional programming style when using rust?
My own code example of a drastic speed up (~25%) simply replacing a couple of for loops with iters: https://github.com/benhoyt/countwords/pull/115
- Ripen scripting engine (Similar to RetroForth, but tiny)
- Performance comparison: counting words in Python, Go, C++, C, AWK, Forth, and Rust
-
The difference between Go and Rust
And yet Go was faster than Rust in a simple app that count words: https://benhoyt.com/writings/count-words/
-
How to Rapidly Improve at Any Programming Language
> but the performance profiles & characteristics that we must know about in order to make a choice on which tool to use. And it shouldn't be that each user has to figure it out on their own, dig into PR's or whatever.
That's an interesting take – I like the idea of a catalog of standard tasks with implementations in several languages as well as their performance characteristics. I suppose Rosetta Code gets the ball rolling with this, but it's missing some performance metrics. It reminds me of [Ben Hoyt's piece](https://benhoyt.com/writings/count-words/) on counting unique words in the KJV Bible in different languages.
-
Faster string keyed maps in Go
This article shows that map lookups can be optimized by using the (unintuitive) pattern:
- Go beats out several top languages including Rust in this performance matchup
What are some alternatives?
RAMCloud - **No Longer Maintained** Official RAMCloud repo
CPython - The Python programming language
wordcount - Counting words in different programming languages.
coreutils - upstream mirror
KindleClippingsTranslator - Czytacz slowek
llfio - P1031 low level file i/o and filesystem library for the C++ standard
tiny_sqlite - A thin SQLite wrapper for Nim
parallel-hashmap - A family of header-only, very fast and memory-friendly hashmap and btree containers.
word_frequency_nim - The word frequency program, written in simple nim.
securitytxt.org - Static website for security.txt.
wyhash - The FASTEST QUALITY hash function, random number generators (PRNG) and hash map.