Concurrent Ruby
Sidekiq
Concurrent Ruby | Sidekiq | |
---|---|---|
14 | 91 | |
5,629 | 12,950 | |
0.3% | 0.3% | |
7.4 | 8.9 | |
27 days ago | 5 days ago | |
Ruby | Ruby | |
MIT License | GNU Lesser General Public License v3.0 only |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
Concurrent Ruby
- A Tour of Go Examples in Ruby
-
Exploring concurrent rate limiters, mutexes, semaphores
After this, I took a look at the semaphore class in the popular library, concurrent-ruby to see how they implement it, and I learnt about something new: condition variables. And Ruby comes with this included!
-
My Adventure with Async Ruby
https://github.com/ruby-concurrency/concurrent-ruby has great docs if someone is looking for alternatives.
-
My Adventure With Async Ruby
I wonder how this would compare to using concurrent-ruby under ruby 2.7, especially in a real-world setting (where the calls are actually to external services that return and buffer data, instead of just sleep). The author says that he's felt that ruby threads "feel easy to mess up," but I've found that concurrent-ruby makes it pretty simple, and performant enough even with the GIL.
-
Using Concurrent::Promise while rescuing exceptions in Ruby
As I could not find a clear example about how to rescue exceptions from Concurrent::Promises (part of the Concurrent Ruby gem ) I read through the documentation and here are two examples: one that documents success case and one that shows what is happening when there is an error.
-
Ask HN: Any efforts to remove the GIL for Ruby?
In a sense the GIL (or actually GVL as it's called in current ruby versions) has already been removed for ruby.
It's only the original MRI Ruby that still has it several over Ruby implementations already removed it. e.g. JRuby.
Concurrent-Ruby[1] is probably a good place to start if you want to work with GVL free ruby on JRuby. It's quite well supported and is currently used by Rails.
If you just want async or non-blocking IO I'd take a look at the Async Gem[2]. It looks pretty solid in Ruby > 3.0 and it's been invited by Matz to be part of the stdlib, which I think is a pretty good endorsement.
For MRI itself I don't think it's likely they'll ever remove the GVL. Ractors are probably a better solution for CPU concurrency in the long run, although I think they're pretty experimental currently.
1. https://github.com/ruby-concurrency/concurrent-ruby
-
Intro to Thread-safety in Ruby on Rails
I like how the article exposes you to tools to prove/disprove the problem. I would have hoped it introduced to tools like concurrent ruby and the use of atomics like u/Freeky already mentioned though.
-
How to get results from Concurrent::Promise::all?
Using conccurrent-ruby, how can I execute a set of promises and then get the results?
-
Ruby 3.1.0 Released
I’d highly recommend the concurrent-ruby gem that has implementations of various metaphors of concurrency, from async to promises, as well as edge features such as actors.
-
Using Thread.new
You may want to consider using something like concurrent-ruby that provides nice abstractions over multithreading.
Sidekiq
-
solid_queue alternatives - Sidekiq and good_job
3 projects | 21 Apr 2024
I'd say Sidekiq is the top competitor here.
-
Valkey Is Rapidly Overtaking Redis
There's something wrong at Redislabs, it took them over a year to get RESP3 rolled out into their hosted service, you'd expect a rollout of that to be a bit quicker when they're the owner of Redis.
It affected us when upgrading Sidekiq to version 7, which dropped support for older Redis, and their Envoy proxy setup didn't support HELLO and RESP3: https://github.com/sidekiq/sidekiq/issues/5594
-
Redis Re-Implemented with SQLite
That depends on how the `maxmemory-policy` is configured, and queue systems based on Redis will tell you not to allow eviction. https://github.com/sidekiq/sidekiq/wiki/Using-Redis#memory (it even logs a warnings if it detects your Redis is misconfigured IIRC).
-
3 one-person million dollar online businesses
Sidekiq https://sidekiq.org/: This one started as an open source project, once it got enough traction, the developer made a premium version of it, and makes money by selling licenses to businesses.
-
Choose Postgres Queue Technology
Sidekiq will drop in-progress jobs when a worker crashes. Sidekiq Pro can recover those jobs but with a large delay. Sidekiq is excellent overall but it’s not suitable for processing critical jobs with a low latency guarantee.
https://github.com/sidekiq/sidekiq/wiki/Reliability
-
We built the fastest CI in the world. It failed
> I'm not sure feature withholding has traditionally worked out well in the developer space.
I think it's worked out well for Sidekiq (https://sidekiq.org). I really like their model of layering valuable features between the OSS / Pro / Enterprise licenses.
-
Exploring concurrent rate limiters, mutexes, semaphores
I was studying Sidekiq's page on rate limiters. The first type of rate limiting mentioned is the concurrent limiter: only n tasks are allowed to run at any point in time. Note that this is independent of time units (e.g. per second), or how long they take to run. The only limitation is the number of concurrent tasks/requests.
- Ask HN: What are some of the most elegant codebases in your favorite language?
- Sidekiq and managing resumable jobs?
-
Organize Business Logic in Your Ruby on Rails Application
The code above isn't idempotent. If you run it twice, it will create two copies, which is probably not what you intended. Why is this important? Because most backend job processors like Sidekiq don't make any guarantees that your jobs will run exactly once.
What are some alternatives?
Async Ruby - An awesome asynchronous event-driven reactor for Ruby.
Resque - Resque is a Redis-backed Ruby library for creating background jobs, placing them on multiple queues, and processing them later.
Celluloid - Actor-based concurrent object framework for Ruby
Sneakers - A fast background processing framework for Ruby and RabbitMQ
EventMachine - EventMachine: fast, simple event-processing library for Ruby programs
Shoryuken - A super efficient Amazon SQS thread based message processor for Ruby
Polyphony - Fine-grained concurrency for Ruby
Sucker Punch - Sucker Punch is a Ruby asynchronous processing library using concurrent-ruby, heavily influenced by Sidekiq and girl_friday.
render_async - render_async lets you include pages asynchronously with AJAX
Apache Kafka - Mirror of Apache Kafka
ruby-vips - Ruby extension for the libvips image processing library.
celery - Distributed Task Queue (development branch)