river
bolt
Our great sponsors
river | bolt | |
---|---|---|
17 | 6 | |
4,715 | 2,444 | |
2.0% | - | |
9.2 | 0.0 | |
5 days ago | over 1 year ago | |
Python | C++ | |
BSD 3-clause "New" or "Revised" License | Mozilla Public License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
river
-
🔍Underrated Open Source Projects You Should Know About 🧠
River is a Python library for online machine learning. Online machine learning can dynamically adapt to new patterns in the data, or when the data itself is generated as a function of time, e.g., stock price prediction, content personalization.
- Ask HN: What Underrated Open Source Project Deserves More Recognition?
-
Unexpected Expected Thriller: A Tale of Coding Curiosity
Today, I'm going to take you on a thrilling coding adventure inspired by a LinkedIn code snippet, where I tangled with FastAPI, River, Watchdog, and Tenacity. Ready? Buckle up!
-
Elevate Your Python Skills: Machine Learning Packages That Transformed My Journey as ML Engineer
Complimentary: river and skorch
-
What are your favorite tools or components in the Kafka ecosystem?
River - https://github.com/online-ml/river (Online machine learning, best used with Bytewax for Kafka integration)
-
Show HN: Want something better than k-means? Try BanditPAM
Hey, great work. Do you think this algorithm would be amenable to be done online? I'm the author of River (https://riverml.xyz) where we're looking for good online clustering algorithms.
-
Python's “Disappointing” Superpowers
If you don't know Rust, but know Python, you can install Python libraries written in Rust with pip. Like, pip install polars or pip install robyn. In this case you follow the two bottom links. But then you don't write your own libraries and stuff so.. I guess that's not what you want.
But, if you want to learn Rust, you probably wouldn't start out with pyo3. You first install Rust with https://rustup.rs/ and then check out the official book, and the book rust by example, that you can find here https://www.rust-lang.org/learn - and maybe write some code on the Rust playground https://play.rust-lang.org/ - then, you use pyo3 to build Python libraries in Rust, and then use maturin https://www.maturin.rs/ to build and publish them to Pypi.
But if you still prefer to begin with Rust by writing Python libraries (it's a valid strategy if you are very comfortable with working with multiple stacks), the Maturin link has a tutorial that setups a program that is half written in python, half written in Rust, https://www.maturin.rs/tutorial.html (well the pyo3 link I sent also has one too. You should refer to the documentation of both, because you will use the two together)
After learning Rust, the next step is looking for libraries that you could leverage to make Python programs ultra fast. Here https://github.com/rayon-rs/rayon is an obvious choice, see some examples from the Rust cookbook https://rust-lang-nursery.github.io/rust-cookbook/concurrenc... - when you create a parallel iterator, it will distribute the processing to many threads (by default, one per core). The rust cookbook, by the way, is a nice reference to see the most used crates (Rust libraries) in the Rust ecosystem.
Anyway there are some posts about pyo3 on the web, like this blog post https://boring-guy.sh/posts/river-rust/ (note: it uses an outdated version of pyo3, and doesn't seem to use maturin which is a newer tool). This post was written by the developers of https://github.com/online-ml/river - another Python library written in Rust
-
[D] How do you deal with covariate shift and concept drift in production?
Have a look at here: https://github.com/online-ml/river
-
Integration test: Complexity of privacy-preserving bird call bio-sensor for distributed ecological monitoring?
Some of the technologies which could be integrated include differential privacy, distributed online machine learning, misinformation resilience and multi-party computation, all within the context of smart contracts and bioinformatics.
bolt
-
Show HN: Want something better than k-means? Try BanditPAM
> frown on that sort of dataset
That example was definitely contrived and designed to strongly illustrate the point. I'll counter slightly that non-peaky topologies aren't uncommon, but they're unlikely to look anything that would push KMedoids to a pathological state rather than just a slightly worse state ("worse" assuming that KMeans is the right choice for a given problem).
> worth pointing out .. data reference
Totally agreed. I hope my answer didn't come across as too negative. It's good work, and everyone else was talking about the positives, so I just didn't want to waste too much time echoing again that while getting the other points across.
> bolt reference
https://github.com/dblalock/bolt
They say as much in their paper, but they aren't the first vector quantization library by any stretch. Their contributions are, roughly:
1. If you're careful selecting the right binning strategy then you can cancel out a meaningful amount of discretization error.
2. If you do that, you can afford to choose parameters that fit everything nicely into AVX2 machine words, turning 100s of branching instructions into 1-4 instructions.
3. Doing some real-world tests to show that (1-2) matter.
Last I checked their code wasn't very effective for the places I wanted to apply it, but the paper is pretty solid. I'd replace it with a faster KMeans approximation less likely to crash on big data (maybe even initializing with KMedoids :) ), and if the thing you're quantizing is trainable with some sort of gradient update step then you should do a few optimization passes in the discretized form as well.
- 10x faster matrix and vector operations
What are some alternatives?
alibi-detect - Algorithms for outlier, adversarial and drift detection
python-tidal - Python API for TIDAL music streaming service
wayfire - A modular and extensible wayland compositor
PySyft - Perform data science on data that remains in someone else's server
edl - Inofficial Qualcomm Firehose / Sahara / Streaming / Diag Tools :)
makinage - Stream Processing Made Easy
frameworks - Sample code and build environments for MPC frameworks
composer - Supercharge Your Model Training
halutmatmul - Hashed Lookup Table based Matrix Multiplication (halutmatmul) - Stella Nera accelerator
AugLy - A data augmentations library for audio, image, text, and video.
rxsci - ReactiveX for data science
draco - Draco is a library for compressing and decompressing 3D geometric meshes and point clouds. It is intended to improve the storage and transmission of 3D graphics.