LibTIFF.jl VS Lux.jl

Are there solid C interfaces that can be used?

A large part of why I started using Julia is because calling into other languages through the C FFI is pretty easy and efficient. Most of the wrappers are a single line. If there is not existing driver support, I would pass the C headers through Clang.jl, which automatically wraps the C API in a C header.

https://github.com/JuliaInterop/Clang.jl

I most recently did this with libtiff. Here is the Clang.jl code to generate the bindings. It's less than 30 lines of sterotypical code.

https://github.com/mkitti/LibTIFF.jl/tree/main/gen

The generated bindings with a few tweaks is here:

https://github.com/mkitti/LibTIFF.jl/blob/main/src/LibTIFF.j...

There is also the less known Lux.jl package: https://github.com/avik-pal/Lux.jl

Lots of things are being rewritten. Remember we just released a new neural network library the other day, SimpleChains.jl, and showed that it gave about a 10x speed improvement on modern CPUs with multithreading enabled vs Jax Equinox (and 22x when AVX-512 is enabled) for smaller neural network and matrix-vector types of cases (https://julialang.org/blog/2022/04/simple-chains/). Then there's Lux.jl fixing some major issues of Flux.jl (https://github.com/avik-pal/Lux.jl). Pretty much everything is switching to Enzyme which improves performance quite a bit over Zygote and allows for full mutation support (https://github.com/EnzymeAD/Enzyme.jl). So an entire machine learning stack is already seeing parts release.

Right now we're in a bit of an uncomfortable spot where we have to use Zygote for a few things and then Enzyme for everything else, but the custom rules system is rather close and that's the piece that's needed to make the full transition.