river VS nimpy

River is a Python library for online machine learning. Online machine learning can dynamically adapt to new patterns in the data, or when the data itself is generated as a function of time, e.g., stock price prediction, content personalization.

Today, I'm going to take you on a thrilling coding adventure inspired by a LinkedIn code snippet, where I tangled with FastAPI, River, Watchdog, and Tenacity. Ready? Buckle up!

Complimentary: river and skorch

River - https://github.com/online-ml/river (Online machine learning, best used with Bytewax for Kafka integration)

Hey, great work. Do you think this algorithm would be amenable to be done online? I'm the author of River (https://riverml.xyz) where we're looking for good online clustering algorithms.

If you don't know Rust, but know Python, you can install Python libraries written in Rust with pip. Like, pip install polars or pip install robyn. In this case you follow the two bottom links. But then you don't write your own libraries and stuff so.. I guess that's not what you want.

But, if you want to learn Rust, you probably wouldn't start out with pyo3. You first install Rust with https://rustup.rs/ and then check out the official book, and the book rust by example, that you can find here https://www.rust-lang.org/learn - and maybe write some code on the Rust playground https://play.rust-lang.org/ - then, you use pyo3 to build Python libraries in Rust, and then use maturin https://www.maturin.rs/ to build and publish them to Pypi.

But if you still prefer to begin with Rust by writing Python libraries (it's a valid strategy if you are very comfortable with working with multiple stacks), the Maturin link has a tutorial that setups a program that is half written in python, half written in Rust, https://www.maturin.rs/tutorial.html (well the pyo3 link I sent also has one too. You should refer to the documentation of both, because you will use the two together)

After learning Rust, the next step is looking for libraries that you could leverage to make Python programs ultra fast. Here https://github.com/rayon-rs/rayon is an obvious choice, see some examples from the Rust cookbook https://rust-lang-nursery.github.io/rust-cookbook/concurrenc... - when you create a parallel iterator, it will distribute the processing to many threads (by default, one per core). The rust cookbook, by the way, is a nice reference to see the most used crates (Rust libraries) in the Rust ecosystem.

Anyway there are some posts about pyo3 on the web, like this blog post https://boring-guy.sh/posts/river-rust/ (note: it uses an outdated version of pyo3, and doesn't seem to use maturin which is a newer tool). This post was written by the developers of https://github.com/online-ml/river - another Python library written in Rust

Oh they definitely can be incrementally updated, there is just added complexity. Online learning has been used with more classical machine learning methods in real-time analytics for a while now. River is a library that handles that.

We have both bandits and FTRL implemented in River (https://riverml.xyz) if that helps.

I mean honestly, the closest language to Mojo really is Nim. In the latest Lex Fridman interview [0] when he talks about his ideas behind Mojo it pretty much sounds like he's describing Nim. Ok fair, he wants Mojo to be a full superset of Python, but honestly with nimpy [1] our Python interop is about as seamless as it can really be (without being a superset, which Mojo clearly is not yet). Even the syntax of Mojo looks a damn lot like Nim imo. Anyway, I guess he has the ability to raise enough funds to hire enough people to write his own language within ~2 years so as not have to follow random peoples whim about where to take the language. So I guess I can't blame him. But as someone who's pretty invested in the Nim community it's quite a shame to see such a hyped language receive so much attention by people who should really check out Nim. ¯\_(ツ)_/¯

[0]: https://youtu.be/pdJQ8iVTwj8?si=LfPSNDq8UKKIsJd3

[1]: https://github.com/yglukhov/nimpy

You can also do this in Nim, which basically means you can write any program you could in Python with libraries in Nim. https://github.com/yglukhov/nimpy

Ones that have not been mentioned so far:

nlvm is an unofficial LLVM backend: https://github.com/arnetheduck/nlvm

npeg lets you write PEGs inline in almost normal PEG notation: https://github.com/zevv/npeg

futhark provides for much more automatic C interop: https://github.com/PMunch/futhark

nimpy allows calling Python code from Nim and vice versa: https://github.com/yglukhov/nimpy

questionable provides a lot of syntax sugar surrounding Option/Result types: https://github.com/codex-storage/questionable

ratel is a framework for embedded programming: https://github.com/PMunch/ratel

cps allows arbitrary procedure rewriting to continuation passing style: https://github.com/nim-works/cps

chronos is an alternative async/await backend: https://github.com/status-im/nim-chronos

zero-functional fixes some inefficiencies when chaining list operations: https://github.com/zero-functional/zero-functional

owlkettle is a declarative macro-oriented library for GTK: https://github.com/can-lehmann/owlkettle

A longer list can be found at https://github.com/ringabout/awesome-nim.

I use Python daily for its massive momentum for scientific stuff, but I also use Nim for everything else. Nim compiles to C, and making Python native modules with Nim is easy with Nimpy.

If are a data scientist and come from python take a look at nimpy, a great way to just import python libraries and use them! https://github.com/yglukhov/nimpy Numpy, pandas, pytorch all usable in Nim.

Nim is the ultimate glue language, use libraries from anything: python, c, js, objc.

I've come to really enjoy programming in Nim. Note that Nim is very different language despite sharing a similar syntax. However, I feel it keeps a lot of the "feel" of Python 2 days of being a fairly simple neat language but that lets you do things at compile time (like compile time duck typing).

There's a good Python -> Nim bridge: https://github.com/yglukhov/nimpy

See this nimpy issue about it: https://github.com/yglukhov/nimpy/issues/43

It has pretty good integration with python, either for having your main code in python and writing small hot functions as nim and importing via nimporter or using python libraries in nim via nimpy.

Related: Nimpy[0] provides an easy way to write Python extensions in Nim, which manages the ABI side very well.

Python 2 is now gone, but until it was, Nimpy was an easy way to write Python extension modules that only needed to be compiled once, and would work with any of your installed Python 2 and Python 3. Magic.

[0] https://github.com/yglukhov/nimpy