Transformers.jl
DaemonMode.jl
Transformers.jl | DaemonMode.jl | |
---|---|---|
7 | 22 | |
504 | 269 | |
- | - | |
6.9 | 4.7 | |
3 months ago | 5 months ago | |
Julia | Julia | |
MIT License | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Transformers.jl
-
Julia 1.10 Released
Flux is quite a nice lower level library:
https://github.com/FluxML/Flux.jl
On top of that there are many higher level libraries such as Transformers.jl
https://github.com/chengchingwen/Transformers.jl
- How is Julia Performance with GPUs (for LLMs)?
-
Load a transformer model with julia
Check out Transformers.jl. It’s a library that implements transformer based models in Julia using Flux.jl. They have support for some of the huggingface transformers.
-
Ask HN: Why hasn't the Deep Learning community embraced Julia yet?
https://github.com/chengchingwen/Transformers.jl but I have not had any personal experience with.
All of this is build by the community and your mileage may vary.
In my rather biased opinion the strengths of Julia are that the various ML libraries can share implementations, e.g. Pytorch and Tensorflow contain separate Numpy derivatives. One could say that you can write an ML framework in Julia, instead of writting a DSL in Python as part of your C++ ML library. As an example Julia has a GPU compiler so you can write your own layer directly in Julia and integrate it into your pipeline.
-
Help on Differentiable Programming
I think you might have some luck with looking at a transformers implementation in flux, e.g: https://github.com/chengchingwen/Transformers.jl/tree/master/src/basic
-
Fastai.jl: Fastai for Julia
Having tried fastai for a "serious" research project and helped (just a bit) towards FastAI.jl development, here's my take:
> motivation behind this is unclear.
Julia currently has two main DL libraries. Flux, which is somewhere between PyTorch and (tf.)Keras abstraction wise, and Knet, which is a little lower level (think just below PyTorch/around where MXNet Gluon sits). Frameworks like fastai, PyTorch Lightning and Keras demonstrate that there's a desire for higher-level, more batteries included libraries. FastAI.jl is looking to fill that gap in Julia.
> Since FastAI.jl uses Flux, and not PyTorch, functionality has to be reimplemented. FastAI.jl has vision support but no text support yet.
This is correct. That said, FastAI.jl is not and does not plan to be a copy of the Python API (hence "inspired by"). One consequence of this is that integration with other libraries is much easier, e.g. https://github.com/chengchingwen/Transformers.jl for NLP tasks.
> What is the timeline for FastAI.jl to achieve parity?
-
Julia Update: Adoption Keeps Climbing; Is It a Python Challenger?
If NLP primitives are all that's keeping you from testing the waters, have a look at https://github.com/chengchingwen/Transformers.jl.
DaemonMode.jl
-
Potential of the Julia programming language for high energy physics computing
Thats for an entry point, you can search `Base.@main` to see a little summary of it. Later it will be able to be callable with `juliax` and `juliac` i.e. `~juliax test.jl` in shell.
DynamicalSystems looks like a heavy project. I don't think you can do much more on your own. There have been recent features in 1.10 that lets you just use the portion you need (just a weak dependency), and there is precompiletools.jl but these are on your side.
You can also look into https://github.com/dmolina/DaemonMode.jl for running a Julia process in the background and do your stuff in the shell without startup time until the standalone binaries are there.
-
Julia 1.9.0 lives up to its promise
> If I were to use e.g. Rust with polars, load time would be virtually none.
Because you're compiling...
And if you need to do the same in Julia, you should also pre-compile or some other method like https://github.com/dmolina/DaemonMode.jl (their demo shows loading a database, with subsequent loads after the first one taking roughly ~0.2% of the first)
- Administrative Scripting with Julia
- GNU Octave 8.1
-
Ask HN: Why is Julia so underrated?
Well, not nicely certainly, but:
https://github.com/dmolina/DaemonMode.jl
> portable
Neither is python - it just relies on universal availability. Over time…
-
Is Julia suitable today as a scripting language?
You can get around a lot of these problems with DaemonMode.jl though.
-
Julia performance, startup.jl, and sysimages
You might want DaemonMode.jl
-
Can I execute code in Julia REPL if I'm connected to a remote server?
https://github.com/dmolina/DaemonMode.jl can possibly help in the future. Leaving it here so that people know this is planned.
- Ask HN: Why hasn't the Deep Learning community embraced Julia yet?
-
Compile for faster execution?
If you strongly prefer to run scripts though, then you can use the package https://github.com/dmolina/DaemonMode.jl in order to re-use a Julia session between multiple scripts, saving you recompilation time.
What are some alternatives?
Flux.jl - Relax! Flux is the ML library that doesn't make you tensor
julia - The Julia Programming Language
PackageCompiler.jl - Compile your Julia Package
Makie.jl - Interactive data visualizations and plotting in Julia
model-zoo - Please do not feed the models
HTTP.jl - HTTP for Julia
DataLoaders.jl - A parallel iterator for large machine learning datasets that don't fit into memory inspired by PyTorch's `DataLoader` class.
FromFile.jl - Julia enhancement proposal (Julep) for implicit per file module in Julia
Chain.jl - A Julia package for piping a value through a series of transformation expressions using a more convenient syntax than Julia's native piping functionality.
julia-numpy-fortran-test - Comparing Julia vs Numpy vs Fortran for performance and code simplicity
StatsPlots.jl - Statistical plotting recipes for Plots.jl
DataFramesMeta.jl - Metaprogramming tools for DataFrames