Numba: A High Performance Python Compiler

• Numba

  124 9,432 9.9 Python

  NumPy aware dynamic Python compiler using LLVM

  

• cij

  1 13 4.0 Python

  Semiemperical quasiharmonic thermal elasticity

  

Software from our group (cij[1], qha[2]) were developed when numba seems to be the best option for JIT. It generates more pain in the hindsight. It generates a lot of depreciated warning due to unstable API, locked numpy to a certain version (i remember 1.21) due to compatibility issues, and when M1 Mac comes out, there were for a long time lack of llvmlite porting to the new platform, so cannot run on these new Macs.

If I had to do it again I would just use plain numpy or use the JAX from Google if JIT is really necessary.

[1]: https://github.com/MineralsCloud/cij

[2]: https://github.com/MineralsCloud/qha

• WorkOS

  workos.com sponsored

  The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.

  

• qha

  1 28 8.3 Python

  A Python package for calculating thermodynamic properties under quasi-harmonic approximation, using data from ab-initio calculations

  

Software from our group (cij[1], qha[2]) were developed when numba seems to be the best option for JIT. It generates more pain in the hindsight. It generates a lot of depreciated warning due to unstable API, locked numpy to a certain version (i remember 1.21) due to compatibility issues, and when M1 Mac comes out, there were for a long time lack of llvmlite porting to the new platform, so cannot run on these new Macs.

If I had to do it again I would just use plain numpy or use the JAX from Google if JIT is really necessary.

[1]: https://github.com/MineralsCloud/cij

[2]: https://github.com/MineralsCloud/qha

• hn-search

  1,617 524 2.9 TypeScript

  Hacker News Search

  

• PyO3

  147 10,997 9.8 Rust

  Rust bindings for the Python interpreter

  

As a side note, now it is easy to write Rust code, which can be directly used in Python - https://github.com/PyO3/pyo3.

It cannot use NumPy and other libraries (since it is Rust), but at the same time, I see its potential in creating high-performance code to be used in Python numerical environment.

• pure_numba_alias_sampling

  1 3 10.0 Python

  Pure numba version of Alias sampling algorithm from L. Devroye's, "Non-Uniform Random Random Variate Generation"

It’s not suitable for all use cases.

But I highly highly recommend it if you need to do somewhat complex calculations iterating over numpy arrays for which standard numpy or scipy functions don’t exist. Even then, often we were surprised that we could speed up some of those calculations by placing them inside numba.

Edit: ex of a very small function I wrote with numba that speeds up an existing numpy function (note - written years ago and numba has undergone quite some amount of changes since!): https://github.com/grej/pure_numba_alias_sampling

Disclosure - I now work for Anaconda, the company that sponsors the numba project.

• codon

  34 13,819 7.9 C++

  A high-performance, zero-overhead, extensible Python compiler using LLVM

  

I am really intrigued by the Codon project, which aims to be a JIT compiler for Python with Numba/JAX decorator syntax: https://github.com/exaloop/codon

• InfluxDB

  www.influxdata.com sponsored

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• fbpic

  2 165 7.6 Python

  Spectral, quasi-3D Particle-In-Cell code, for CPU and GPU

  

When I wrote my bachelor thesis years back I worked on a particle-in-cell code [1] that makes heavy use of numba for GPU kernels. At the time it was the most convenient way to do that from python. I remember spending weeks to optimizing these kernels to eek out every last bit of performance I could (which interestingly enough did eventually involve using atomic operations and introducing a lot of variables[2] instead of using arrays everywhere to keep things in registers instead of slower caches).

I remember the team being really responsive to feature requests back then and I had a lot of fun working with it. IIRC compared to using numpy we managed to get speedups of up to 60x for the most critical pieces of code.

[1]: https://github.com/fbpic/fbpic

• rust-numpy

  10 1,015 6.7 Rust

  PyO3-based Rust bindings of the NumPy C-API

  

On the contrary, it can use and interface with numpy quite easily: https://github.com/PyO3/rust-numpy

• ideas4

  26 89 4.6

  An Additional 100 Ideas for Computing https://samsquire.github.io/ideas4/

https://github.com/samsquire/ideas4#31-algebraic-materialise...

• autograd

  6 6,787 6.0 Python

  Efficiently computes derivatives of numpy code.

  

XLA is "higher level" than what Numba produces.

You may be able to get the equivalent of jax via numba+numpy+autograd[1], but I haven't tried it before.

IMHO, jax is best thought of as a numerical computation library that happens to include autograd, vmapping, pmapping and provides a high level interface for XLA.

I have built a numerical optimisation library with it, and although a few things became verbose, it was a rather pleasant experience as the natural vmapping made everything a breeze, I didn't have to write the gradients for my testing functions, except for special cases that involved exponents and logs that needed a bit of delicate care.

[1] https://github.com/HIPS/autograd

Suggest a related project