-
Someone else pointed the same thing on bluesky, so I guess this part of my post isn't very clear.
First, I/O bound isn't necessarily a very well defined term, but generally speaking, it implies that you can't substantially speedup such system by speeding up code execution. The fact that YJIT did yield substantial performance improvement in the real world suggest many Rails apps aren't in fact strictly I/O bound.
Now about the 15-30% vs 2-3x, what I mean by that is the benchmark where YJIT yield this much are mostly micro-benchmarks, on more complex and heterogenous code, the gains are much smaller, hence we can assume the "Ruby" part of these applications wasn't improved by this much.
So for YJIT to be able to yield `15-30%` latency gains, this latency must have been in large part composed of Ruby execution. One last thing to note is that YJIT can only speedup pure Ruby code, in a Rails application, a large part of the CPU time isn't in pure Ruby code, but in various methods implemented in C that YJIT can't speedup.
Ultimately the lobste.rs maintainer kindly offered to run an experiment in production, so we should soon see if my assumptions hold true or if I was way off, at least for that particular app: https://github.com/lobsters/lobsters/pull/1442
-
Judoscale
Save 47% on cloud hosting with autoscaling that just works. Judoscale integrates with Rails, Sidekiq, Solid Queue, and more to make autoscaling easy and reliable. Save big, and say goodbye to request timeouts and backed-up job queues.
-
The discussion in the referenced Rails issue "Set a new default for the Puma thread count"[0] is much more telling of IO vs CPU than this simplified post.
There are many assumptions here especially not considering anything about the database itself. The Rails issue also considers benchmarks which when benchmarking Rails would configure the database to not be the bottleneck. That's not true of real systems. The advice to use async queries for an IO-bound app could backfire if the reason the queries are slow is because the database is overloaded--adding concurrent queries only increases its thrashing and latency.
The best thing to do is consider the whole system. Don't throw the everything that's not the Rails app as IO. Is it actually doing network IO, or is it CPU, memory, or IO bound in the database? Maybe its not even a lack of CPU proper on the database but that it's being wasted on write contention/locks. Only then will you be able to choose the right course. Another way to go is blindly try different configurations and use what works well without full understanding, which is fine until you have an outage you can't explain then scale everything just-in-case.
[0] https://github.com/rails/rails/issues/50450
