Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression. Learn more →
Top 23 Rust Python Projects
Fast multi-threaded, hybrid-out-of-core DataFrame library in Rust | Python | Node.jsProject mention: Any job processing framework like Spark but in Rust? | reddit.com/r/dataengineering | 2023-03-23
For data frames built on Apache Arrow and: https://github.com/pola-rs/polars/
An extremely fast Python linter, written in Rust.Project mention: The Ruff python linter is insanely good | reddit.com/r/Python | 2023-03-16
I mean considering this is their pylint compatibility I doubt they would start implementing random requests https://github.com/charliermarsh/ruff/issues/970
Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.
Sampling profiler for Python programsProject mention: Has anyone switched from numpy to Rust? | reddit.com/r/rust | 2023-03-11
So as a first step you'll want to profile your program to figure out where it's slow, and hopefully that'll also tell you why it's slow. I'm the (biased) author of the Sciagraph profiler which is designed for this sort of application (https://sciagraph.com) but you can also try py-spy, which isn't as well designed for data processing/analysis applications (e.g. it won't visualize parallelism at all) but can still be informative (https://github.com/benfred/py-spy). Both are written in Rust ;)
Rust bindings for the Python interpreterProject mention: no good UI toolkit/framework | reddit.com/r/rust | 2023-03-18
I use Qt but I use PyO3 to set up a "Python/PyQt/PySide frontend on top of Rust backend" architecture for the QWidget APIs similar to the "QML frontend on top of C++ backend" that they officially promote for Qt Quick.
Apache Arrow DataFusion SQL Query EngineProject mention: Bridging Async and Sync Rust Code - A lesson learned while working with Tokio | reddit.com/r/rust | 2023-03-10
Problem comes when you want to do this inside an async context since we couldn't block an async task. https://users.rust-lang.org/t/sync-function-invoking-async/43364/6 You might need to do it in another runtime/thread. It is not recommended to do this, but sometimes it is unavoidable while implementing a third-party trait. https://github.com/apache/arrow-datafusion/issues/3777 However, I believe this isn't a problem particular to tokio, or any specific runtime.
Oso is a batteries-included framework for building authorization in your application.Project mention: Show HN: ILLA is an Open-source alternative to Retool | news.ycombinator.com | 2022-11-19
Not OP but Authentication is easy, authorization is a cross-cutting concern that often requires custom code. E.g., there are people and teams, both of which can have different kinds of access to something (read/write). Sometimes teams have sub-teams. Do the sub-teams have access to the parent teams' resources and/or vice versa? Also what kind of sharing are you going to support? Do people have to have an account to view stuff shared to them or can you just send a link? There are some efforts to make custom DSLs for describing authorization policies, to avoid cross-cutting code.
Computed fields require different treatment at every level of the stack. This isn't inherently hard, but it is an extra feature these low-code/no-code platforms need. Where things get difficult is inn migrations. It's common for a field that is computed at the beginning to become customizable, or for the computation to change. When that happens, what should the value be for old columns? Computed fields also often pull data from multiple other tables, which may require some combination of custom queries and database optimization.
PostgresML is an end-to-end machine learning system. It enables you to train models and make online predictions using only SQL, without your data ever leaving your favorite database.Project mention: How to store hugging face model in postgreSQL | reddit.com/r/LanguageTechnology | 2023-02-05
I'd encourage you to do inference outside of PostgreSQL (use TF serving and make requests against it, or do batch inference), but if you're determined to do so, they have an extension that integrates with the transformers library and allows for calling models directly from SQL.
Access the most powerful time series database as a service. Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression.
A statically typed language that can deeply improve the Python ecosystemProject mention: This Week in Python | dev.to | 2022-08-19
erg – A Python-compatible statically typed language
Build and publish crates with pyo3, rust-cpython and cffi bindings as well as rust binaries as python packagesProject mention: Is Rust faster than Python out of the box | reddit.com/r/rust | 2023-03-12
Lastly if you're willing to introduce Rust, I'd consider a gradual approach using native libraries built in rust with PYO3. Check the maturin guide that helps you to streamline the build process of native libraries : https://github.com/PyO3/maturin . From there you could try to find hotspots in your python app and replace those with a native implementation.
The Universal Plug-in System. Extend anything with WebAssembly.Project mention: Announcing Scale: A WebAssembly runtime for running Go functions in any other language | reddit.com/r/golang | 2023-03-14
Also I discovered https://github.com/extism/extism recently but I don’t quite understand how their API works.
🐍🕸 WebAssembly runtime for PythonProject mention: WebAssembly: Adding Python Support to WASM Language Runtimes | news.ycombinator.com | 2023-01-31
PyOdide isn't currently supported outside of browsers, though that might change.
Either way, I couldn't figure out how to do the above sequence of steps with any of the available Python WASM runtimes - they're all very under-documented at the moment, sadly. I tried all three of these:
Rust <-> Python bindingsProject mention: How does Rust Python ffi work? | reddit.com/r/rust | 2022-05-10
I've never used pyo3, just cpython, but the latter at least let me do things like:
Log images, point clouds, etc, and visualize them effortlessly. Built in Rust using egui (by rerun-io)Project mention: Leveraging Rust and the GPU to render user interfaces at 120 FPS | news.ycombinator.com | 2023-03-09
Egui seems well suited to render even tens of thousands pointcloud with wgpu backend.
Modern columnar data format for ML implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, with more integrations coming..Project mention: Any job processing framework like Spark but in Rust? | reddit.com/r/dataengineering | 2023-03-23
For Feature Stores check out: https://github.com/eto-ai/lance
The enterprise-ready webhooks service 🦀Project mention: Open source webhook service | reddit.com/r/rust | 2023-03-11
Fastest library to load data from DB to DataFrames in Rust and PythonProject mention: Querying Postgres Tables Directly from DuckDB | news.ycombinator.com | 2022-09-30
I was trying https://github.com/sfu-db/connector-x and hacking around with this https://github.com/spitz-dan-l/postgres-binary-parser but it turned out that a COPY to csv using asyncpg and then converting to parquet was the fastest.
Cross-platform command-line AV1 / VP9 / HEVC / H264 encoding framework with per scene quality encodingProject mention: Automating av1 encoding? | reddit.com/r/AV1 | 2023-02-18
That might be because it doesn't work.
A native Rust library for Delta Lake, with bindings into PythonProject mention: Create/write Iceberg with Python? | reddit.com/r/dataengineering | 2023-03-18
If you absolutely have to use only basic Python for setting up a data lake, the only thing that comes close is delta-rs for delta lake, another open-source lakehouse format. It is still very new, but it does work.
A Rust HTTP server for Python applicationsProject mention: Robyn: a fast and extensible async Python web server with a Rust runtime | reddit.com/r/rust | 2023-01-16
Recently I found this ASGI compatible server written in rust, but i haven’t tried it yet https://github.com/emmett-framework/granian
A simple, cross-platform GUI automation module for Python and Rust.
A fun safe language for rapid prototyping and high performance applications (by astrolang)
Chromepass - Hacking Chrome Saved PasswordsProject mention: Auto-Import to Value? | reddit.com/r/Bitwarden | 2022-12-28
Apache Arrow Ballista Distributed Query EngineProject mention: Any job processing framework like Spark but in Rust? | reddit.com/r/dataengineering | 2023-03-23
SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives
Rust Python related posts
web service framework and OpenAPI spec
1 project | reddit.com/r/rust | 24 Mar 2023
Any job processing framework like Spark but in Rust?
4 projects | reddit.com/r/dataengineering | 23 Mar 2023
Create/write Iceberg with Python?
1 project | reddit.com/r/dataengineering | 18 Mar 2023
no good UI toolkit/framework
4 projects | reddit.com/r/rust | 18 Mar 2023
Is Rust faster than Python out of the box
2 projects | reddit.com/r/rust | 12 Mar 2023
I use dwm btw
1 project | reddit.com/r/linuxmasterrace | 13 Mar 2023
An example for OpenCL 3.0?
4 projects | reddit.com/r/OpenCL | 11 Mar 2023
A note from our sponsor - InfluxDB
www.influxdata.com | 25 Mar 2023
What are some of the best open-source Python projects in Rust? This list will help you: