gen_stage
ramda
gen_stage | ramda | |
---|---|---|
2 | 80 | |
1,451 | 23,584 | |
0.3% | 0.2% | |
3.8 | 6.8 | |
about 2 months ago | 5 days ago | |
Elixir | JavaScript | |
- | MIT License |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
gen_stage
-
Switching to Elixir
You can actually have "background jobs" in very different ways in Elixir.
> I want background work to live on different compute capacity than http requests, both because they have very different resources usage
In Elixir, because of the way the BEAM works (the unit of parallelism is much cheaper and consume a low amount of memory), "incoming http requests" and related "workers" are not as expensive (a lot less actually) compared to other stacks (for instance Ruby and Python), where it is quite critical to release "http workers" and not hold the connection (which is what lead to the creation of background job tools like Resque, DelayedJob, Sidekiq, Celery...).
This means that you can actually hold incoming HTTP connections a lot longer without troubles.
A consequence of this is that implementing "reverse proxies", or anything calling third party servers _right in the middle_ of your own HTTP call, is usually perfectly acceptable (something I've done more than a couple of times, the latest one powering the reverse proxy behind https://transport.data.gouv.fr - code available at https://github.com/etalab/transport-site/tree/master/apps/un...).
As a consequence, what would be a bad pattern in Python or Ruby (holding the incoming HTTP connection) is not a problem with Elixir.
> because I want to have state or queues in front of background work so there's a well-defined process for retry, error handling, and back-pressure.
Unless you deal with immediate stuff like reverse proxying or cheap "one off async tasks" (like recording a metric), there also are solutions to have more "stateful" background works in Elixir, too.
A popular background job queue is https://github.com/sorentwo/oban (roughly similar to Sidekiq at al), which uses Postgres.
It handles retries, errors etc.
But it's not the only solution, as you have other tools dedicated to processing, such as Broadway (https://github.com/dashbitco/broadway), which handles back-pressure, fault-tolerance, batching etc natively.
You also have more simple options, such as flow (https://github.com/dashbitco/flow), gen_stage (https://github.com/elixir-lang/gen_stage), Task.async_stream (https://hexdocs.pm/elixir/1.12/Task.html#async_stream/5) etc.
It allows to use the "right tool for the job" quite easily.
It is also interesting to note there is no need to "go evented" if you need to fetch data from multiple HTTP servers: it can happen in the exact same process (even: in a background task attached to your HTTP server), as done here https://transport.data.gouv.fr/explore (if you zoom you will see vehicle moving in realtime, and ~80 data sources are being polled every 10 seconds & broadcasted to the visitors via pubsub & websockets).
-
I've made a simple Discord bot to learn Elixir, and would really like some feedback and tips
I think if the bot gets a lot of users or streams being tracked you might run into issues with how sync_streams works, you might want to look at something like GenStage or maybe Oban to send messages in batches. In a similar note you'd want to do the same thing for cases where you are doing Repo.all.
ramda
-
Tacit Programming
JavaScript is great for point-free programming! Make sure you check out Ramda.js https://ramdajs.com/
It’s fun in the sense that solving a puzzle is fun, but I avoid it for anything I need to maintain long-term.
But it’s good practice for understanding combinators which is useful for some kinds of problems.
-
Pipeline-Oriented Programming [video]
This is very cool. I remember I got sucked into things like Ramda going down this functional programming rabbit hole :-)
https://ramdajs.com/
-
Level up your Typescript game, functionally - Part 2
To create our pipeline, I'm going to use the pipe function from the NodeJS ramda library instead of building my own.
-
Level up your Typescript game, functionally - Part 3
Other libraries to check out are pratica and ramda
- Ramda: A practical functional library for JavaScript programmers
-
FP and JavaScript/TypeScript
I recently took ownership of the new types/ramda repo. This repo is re-exported by @types/ramda and is the first step to bringing type definitions for ramda in-house. We're already hard at work correcting major issues, adding full currying support, and general bug fixes
- [AskJS] Auto-Generated Documentation from JSDoc comments, nice modern themes?
-
When to use currying in JavaScript
I'm going to be honest. You probably don't need to use currying in JavaScript. In fact, trying to fit it in your code is going to do more harm than good, unless it's just for fun. Currying only becomes useful when you fully embrace functional programming, which, in JavaScript, means using a library like Ramda instead of the standard built-in functions.
-
No Lodash
Lodash gets so many things wrong I’d rather not see it in most projects. I appreciate a good utility library for JS projects but my go-to choice has to be Ramda[1]. Every function it exports is curried and works great with pipe which enables me to write highly reusable and composable functions in pointfree notation. I have never been as productive with lodash, and I find the functional style easier to read
[1] https://ramdajs.com/
-
Snap.js - A competitor to Lodash
Do note though that ramda is different from rambda. 👍 (Granted they are very similar!)
What are some alternatives?
botchini - Discord bot with Twitch integration
lodash - A modern JavaScript utility library delivering modularity, performance, & extras.
transport-site - Rendre disponible, valoriser et améliorer les données transports
Rambda - Faster and smaller alternative to Ramda
eqwalizer - A type-checker for Erlang
RxJS
flow - Computational parallel flows on top of GenStage
immutable-js - Immutable persistent data collections for Javascript which increase efficiency and simplicity.
rustic_maybe - Maybe monad for Elixir inspired by Rust Option type
fp-ts - Functional programming in TypeScript
broadway - Concurrent and multi-stage data ingestion and data processing with Elixir
lazy.js - Like Underscore, but lazier