A list of publicly available datasets with real-time data maintained by the team at bytewax.io
Rust bindings for the Python interpreter
Hey HN! I am one of the people working on Bytewax. Bytewax came out of our experience working with ML infrastructure at GitHub. We wanted to use Python because we could move fast, the team was very fluent in it, and the rest of our tooling was Python-native already. We didn't want to introduce JVM-based solutions into our stack because of the lack of experience and the friction we had trying to get Python-centric tooling working with existing solutions like Flink.
In our research, we found Timely Dataflow (https://timelydataflow.github.io/timely-dataflow/, https://news.ycombinator.com/item?id=24837031) and the Naiad project (https://www.microsoft.com/en-us/research/project/naiad/) as well as PyO3 (https://github.com/PyO3/pyo3) and we thought we found a match made in heaven :). Bytewax leverages both of these projects and builds on them to provide a clean API (at least we think so) and table stakes features like connectors, state recovery, and cloud-native scaling. It has been really cool to learn about the dataflow computation model, Rust, and how to wrangle the GIL with Rust and Python :P.
Would love to get your feedback :).
`pip install bytewax` to get started. We have a page of guides (https://www.bytewax.io/guides) with ready-to-run examples.
Learn any GitHub repo in 59 seconds. Onboard AI learns any GitHub repo in minutes and lets you chat with it to locate functionality, understand different parts, and generate new code. Use it for free at www.getonboard.dev.
Calling Rust from Python
2 projects | news.ycombinator.com | 8 Oct 2023
Documenting Native Python Extensions Made With Rust and PyO3
2 projects | /r/rust | 19 Jun 2023
My first superoptimizer
1 project | /r/programming | 31 May 2023
My PyO3 library dosent have auto completion
1 project | /r/learnrust | 9 May 2023
Announcing sequence_align: an open-source Python + Rust toolkit for efficient sequence alignment
2 projects | /r/rust | 6 Apr 2023