flow
eqwalizer
flow | eqwalizer | |
---|---|---|
2 | 11 | |
1,479 | 500 | |
0.5% | 0.4% | |
3.4 | 8.2 | |
10 months ago | 13 days ago | |
Elixir | Scala | |
Apache License 2.0 | Apache License 2.0 |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
flow
-
Switching to Elixir
You can actually have "background jobs" in very different ways in Elixir.
> I want background work to live on different compute capacity than http requests, both because they have very different resources usage
In Elixir, because of the way the BEAM works (the unit of parallelism is much cheaper and consume a low amount of memory), "incoming http requests" and related "workers" are not as expensive (a lot less actually) compared to other stacks (for instance Ruby and Python), where it is quite critical to release "http workers" and not hold the connection (which is what lead to the creation of background job tools like Resque, DelayedJob, Sidekiq, Celery...).
This means that you can actually hold incoming HTTP connections a lot longer without troubles.
A consequence of this is that implementing "reverse proxies", or anything calling third party servers _right in the middle_ of your own HTTP call, is usually perfectly acceptable (something I've done more than a couple of times, the latest one powering the reverse proxy behind https://transport.data.gouv.fr - code available at https://github.com/etalab/transport-site/tree/master/apps/un...).
As a consequence, what would be a bad pattern in Python or Ruby (holding the incoming HTTP connection) is not a problem with Elixir.
> because I want to have state or queues in front of background work so there's a well-defined process for retry, error handling, and back-pressure.
Unless you deal with immediate stuff like reverse proxying or cheap "one off async tasks" (like recording a metric), there also are solutions to have more "stateful" background works in Elixir, too.
A popular background job queue is https://github.com/sorentwo/oban (roughly similar to Sidekiq at al), which uses Postgres.
It handles retries, errors etc.
But it's not the only solution, as you have other tools dedicated to processing, such as Broadway (https://github.com/dashbitco/broadway), which handles back-pressure, fault-tolerance, batching etc natively.
You also have more simple options, such as flow (https://github.com/dashbitco/flow), gen_stage (https://github.com/elixir-lang/gen_stage), Task.async_stream (https://hexdocs.pm/elixir/1.12/Task.html#async_stream/5) etc.
It allows to use the "right tool for the job" quite easily.
It is also interesting to note there is no need to "go evented" if you need to fetch data from multiple HTTP servers: it can happen in the exact same process (even: in a background task attached to your HTTP server), as done here https://transport.data.gouv.fr/explore (if you zoom you will see vehicle moving in realtime, and ~80 data sources are being polled every 10 seconds & broadcasted to the visitors via pubsub & websockets).
-
An opinionated map of incremental and streaming systems (2018)
Elixir has a few interesting abstractions for that: GenStage, Flow, Broadway.
https://github.com/dashbitco/flow
eqwalizer
-
Switching to Elixir
I don't think the implementation itself is at fault, but yes, I do think that the design of dialyzer makes it an (at times) faulty type checker. The unfortunate reality of a type checker that fails sometimes is that it makes it mostly useless because you can never trust that it'll do the job.
To be clear, I've had it fail in a function where I've literally specced that very function to return a `binary` but I'm returning an `integer` in one of the cases. This is a very shallow context but it can still fail. Now add more functions, maybe one more `case`.
I think an entire rethink of type checking on the BEAM had to be done and that's why eqWalizer[0] was created and why Elixir is looking to add an actual sound, well-developed type checker. Gleam[1] I would assume is just a Hindley-Milner system so that's completely solid. `purerl`[2] is just PureScript for the BEAM so that's also Hindley-Milner, meaning it's solid. `purerl` has some performance issues caused by it compiling down to closures everywhere but if you can pay that cost it's actually pretty fantastic. With that said my bet for the best statically typed experience right now on the BEAM would be `gleam`.
0 - https://github.com/WhatsApp/eqwalizer
1 - https://gleam.run
2 - https://github.com/purerl/purerl
- Unpacking Elixir: Concurrency
-
eqwalizer VS Gradualizer - a user suggested alternative
2 projects | 17 Apr 2023
- Erlang: The coding language that finance forgot
-
Phoenix 1.7 is View-less
While it's not static-typing, compile-time type checking for Erlang have come a long way: Eqwalizer works pretty well - but I may be biased since my employer sponsors the project.
1. https://github.com/WhatsApp/eqwalizer
-
[New] How do you verify program correctness in Elixir?
Note there is also research happening in this area by the Elixir team. The WhatsApp is also working on static types for Erlang, which I am certain will be available for Elixir too at some point.
- Eqwalizer: A Type-Checker for Erlang
- Eqwalizer: WhatsApp’s Erlang Type Checker
-
Elixir Livebook now as a desktop app
From the discord blog posts it seems that elixir powers the chat system, with rust and python as the other two main languages in their stack.
As for whatsapp, they are mainly a erlang shop and yesterday they open sourced a type checker for erlang:
https://github.com/WhatsApp/eqwalizer
What are some alternatives?
parallel_stream - A parallelized stream implementation for Elixir
gradient - Gradient is a static typechecker for Elixir
MapDiff - Calculates the difference between two (nested) maps, and returns a map representing the patch of changes.
erllambda - AWS Lambda in Erlang
fsm - Finite State Machine data structure
explorer - Series (one-dimensional) and dataframes (two-dimensional) for fast and elegant data exploration in Elixir
graphmath - An Elixir library for performing 2D and 3D mathematics.
kino - Client-driven interactive widgets for Livebook
witchcraft - Monads and other dark magic for Elixir
FunkyABX - Audio blind tests
matrex - A blazing fast matrix library for Elixir/Erlang with C implementation using CBLAS.
Gradualizer - A Gradual type system for Erlang