Rust Python

Open-source Rust projects categorized as Python

Top 23 Rust Python Projects

  • polars

    Fast multi-threaded, hybrid-out-of-core DataFrame library in Rust | Python | Node.js

    Project mention: Any job processing framework like Spark but in Rust? | reddit.com/r/dataengineering | 2023-03-23

    For data frames built on Apache Arrow and: https://github.com/pola-rs/polars/

  • ruff

    An extremely fast Python linter, written in Rust.

    Project mention: The Ruff python linter is insanely good | reddit.com/r/Python | 2023-03-16

    I mean considering this is their pylint compatibility I doubt they would start implementing random requests https://github.com/charliermarsh/ruff/issues/970

  • SonarQube

    Static code analysis for 29 languages.. Your projects are multi-language. So is SonarQube analysis. Find Bugs, Vulnerabilities, Security Hotspots, and Code Smells so you can release quality code every time. Get started analyzing your projects today for free.

  • py-spy

    Sampling profiler for Python programs

    Project mention: Has anyone switched from numpy to Rust? | reddit.com/r/rust | 2023-03-11

    So as a first step you'll want to profile your program to figure out where it's slow, and hopefully that'll also tell you why it's slow. I'm the (biased) author of the Sciagraph profiler which is designed for this sort of application (https://sciagraph.com) but you can also try py-spy, which isn't as well designed for data processing/analysis applications (e.g. it won't visualize parallelism at all) but can still be informative (https://github.com/benfred/py-spy). Both are written in Rust ;)

  • PyO3

    Rust bindings for the Python interpreter

    Project mention: no good UI toolkit/framework | reddit.com/r/rust | 2023-03-18

    I use Qt but I use PyO3 to set up a "Python/PyQt/PySide frontend on top of Rust backend" architecture for the QWidget APIs similar to the "QML frontend on top of C++ backend" that they officially promote for Qt Quick.

  • arrow-datafusion

    Apache Arrow DataFusion SQL Query Engine

    Project mention: Bridging Async and Sync Rust Code - A lesson learned while working with Tokio | reddit.com/r/rust | 2023-03-10

    Problem comes when you want to do this inside an async context since we couldn't block an async task. https://users.rust-lang.org/t/sync-function-invoking-async/43364/6 You might need to do it in another runtime/thread. It is not recommended to do this, but sometimes it is unavoidable while implementing a third-party trait. https://github.com/apache/arrow-datafusion/issues/3777 However, I believe this isn't a problem particular to tokio, or any specific runtime.

  • oso

    Oso is a batteries-included framework for building authorization in your application.

    Project mention: Show HN: ILLA is an Open-source alternative to Retool | news.ycombinator.com | 2022-11-19

    Not OP but Authentication is easy, authorization is a cross-cutting concern that often requires custom code. E.g., there are people and teams, both of which can have different kinds of access to something (read/write). Sometimes teams have sub-teams. Do the sub-teams have access to the parent teams' resources and/or vice versa? Also what kind of sharing are you going to support? Do people have to have an account to view stuff shared to them or can you just send a link? There are some efforts to make custom DSLs for describing authorization policies, to avoid cross-cutting code[1].

    Computed fields require different treatment at every level of the stack. This isn't inherently hard, but it is an extra feature these low-code/no-code platforms need. Where things get difficult is inn migrations. It's common for a field that is computed at the beginning to become customizable, or for the computation to change. When that happens, what should the value be for old columns? Computed fields also often pull data from multiple other tables, which may require some combination of custom queries and database optimization.

    [1] https://github.com/osohq/oso

  • postgresml

    PostgresML is an end-to-end machine learning system. It enables you to train models and make online predictions using only SQL, without your data ever leaving your favorite database.

    Project mention: How to store hugging face model in postgreSQL | reddit.com/r/LanguageTechnology | 2023-02-05

    I'd encourage you to do inference outside of PostgreSQL (use TF serving and make requests against it, or do batch inference), but if you're determined to do so, they have an extension that integrates with the transformers library and allows for calling models directly from SQL.

  • InfluxDB

    Access the most powerful time series database as a service. Ingest, store, & analyze all types of time series data in a fully-managed, purpose-built database. Keep data forever with low-cost storage and superior data compression.

  • erg

    A statically typed language that can deeply improve the Python ecosystem

    Project mention: This Week in Python | dev.to | 2022-08-19

    erg – A Python-compatible statically typed language

  • maturin

    Build and publish crates with pyo3, rust-cpython and cffi bindings as well as rust binaries as python packages

    Project mention: Is Rust faster than Python out of the box | reddit.com/r/rust | 2023-03-12

    Lastly if you're willing to introduce Rust, I'd consider a gradual approach using native libraries built in rust with PYO3. Check the maturin guide that helps you to streamline the build process of native libraries : https://github.com/PyO3/maturin . From there you could try to find hotspots in your python app and replace those with a native implementation.

  • extism

    The Universal Plug-in System. Extend anything with WebAssembly.

    Project mention: Announcing Scale: A WebAssembly runtime for running Go functions in any other language | reddit.com/r/golang | 2023-03-14

    Also I discovered https://github.com/extism/extism recently but I don’t quite understand how their API works.

  • wasmer-python

    🐍🕸 WebAssembly runtime for Python

    Project mention: WebAssembly: Adding Python Support to WASM Language Runtimes | news.ycombinator.com | 2023-01-31

    PyOdide isn't currently supported outside of browsers, though that might change.

    Either way, I couldn't figure out how to do the above sequence of steps with any of the available Python WASM runtimes - they're all very under-documented at the moment, sadly. I tried all three of these:

    - https://github.com/wasmerio/wasmer-python

    - https://github.com/bytecodealliance/wasmtime-py

    - https://github.com/wasm3/pywasm3

  • rust-cpython

    Rust <-> Python bindings

    Project mention: How does Rust Python ffi work? | reddit.com/r/rust | 2022-05-10

    I've never used pyo3, just cpython, but the latter at least let me do things like:

  • rerun

    Log images, point clouds, etc, and visualize them effortlessly. Built in Rust using egui (by rerun-io)

    Project mention: Leveraging Rust and the GPU to render user interfaces at 120 FPS | news.ycombinator.com | 2023-03-09

    Egui seems well suited to render even tens of thousands pointcloud with wgpu backend.

    [0] https://github.com/rerun-io/rerun

  • lance

    Modern columnar data format for ML implemented in Rust. Convert from parquet in 2 lines of code for 100x faster random access, vector index, and data versioning. Compatible with Pandas, DuckDB, Polars, Pyarrow, with more integrations coming..

    Project mention: Any job processing framework like Spark but in Rust? | reddit.com/r/dataengineering | 2023-03-23

    For Feature Stores check out: https://github.com/eto-ai/lance

  • svix-webhooks

    The enterprise-ready webhooks service 🦀

    Project mention: Open source webhook service | reddit.com/r/rust | 2023-03-11
  • connector-x

    Fastest library to load data from DB to DataFrames in Rust and Python

    Project mention: Querying Postgres Tables Directly from DuckDB | news.ycombinator.com | 2022-09-30

    I was trying https://github.com/sfu-db/connector-x and hacking around with this https://github.com/spitz-dan-l/postgres-binary-parser but it turned out that a COPY to csv using asyncpg and then converting to parquet was the fastest.

  • Av1an

    Cross-platform command-line AV1 / VP9 / HEVC / H264 encoding framework with per scene quality encoding

    Project mention: Automating av1 encoding? | reddit.com/r/AV1 | 2023-02-18

    That might be because it doesn't work.

  • delta-rs

    A native Rust library for Delta Lake, with bindings into Python

    Project mention: Create/write Iceberg with Python? | reddit.com/r/dataengineering | 2023-03-18

    If you absolutely have to use only basic Python for setting up a data lake, the only thing that comes close is delta-rs for delta lake, another open-source lakehouse format. It is still very new, but it does work.

  • granian

    A Rust HTTP server for Python applications

    Project mention: Robyn: a fast and extensible async Python web server with a Rust runtime | reddit.com/r/rust | 2023-01-16

    Recently I found this ASGI compatible server written in rust, but i haven’t tried it yet https://github.com/emmett-framework/granian

  • autopy

    A simple, cross-platform GUI automation module for Python and Rust.

  • astro

    A fun safe language for rapid prototyping and high performance applications (by astrolang)

  • chromepass

    Chromepass - Hacking Chrome Saved Passwords

    Project mention: Auto-Import to Value? | reddit.com/r/Bitwarden | 2022-12-28
  • arrow-ballista

    Apache Arrow Ballista Distributed Query Engine

    Project mention: Any job processing framework like Spark but in Rust? | reddit.com/r/dataengineering | 2023-03-23
  • SaaSHub

    SaaSHub - Software Alternatives and Reviews. SaaSHub helps you find the best software and product alternatives

NOTE: The open source projects on this list are ordered by number of github stars. The number of mentions indicates repo mentiontions in the last 12 Months or since we started tracking (Dec 2020). The latest post mention was on 2023-03-23.

Rust Python related posts

Index

What are some of the best open-source Python projects in Rust? This list will help you:

Project Stars
1 polars 15,640
2 ruff 11,045
3 py-spy 9,882
4 PyO3 7,981
5 arrow-datafusion 3,365
6 oso 2,955
7 postgresml 2,395
8 erg 2,145
9 maturin 1,890
10 extism 1,783
11 wasmer-python 1,736
12 rust-cpython 1,710
13 rerun 1,570
14 lance 1,311
15 svix-webhooks 1,263
16 connector-x 1,164
17 Av1an 1,027
18 delta-rs 956
19 granian 755
20 autopy 717
21 astro 677
22 chromepass 657
23 arrow-ballista 644
SaaSHub - Software Alternatives and Reviews
SaaSHub helps you find the best software and product alternatives
www.saashub.com