Ruby on Rails VS yjit

Here's a real life example: Imagine a Ruby on Rails app on which a team of developers are working. The code is hosted on GitLab and all the work is coordinated using GitLab issues. In other words: For every commit, there's an associated issue and the issue number acts as a sort of primary key for documentation, time reporting and so forth. This convention has a few advantages, most notably the ability to easily learn more about how, when and by whom features were implemented as well as how this implementation came to be.

Ruby on Rails is regarded as one of the best ruby frameworks. It was the primary language in developing big projects such as Twitter and helped the language boost the community. Often referred to as “Rails,” Ruby on Rails is a web development framework with an MVC control structure and currently running its 6.1 version. The 16-year-old language has dramatically influenced the web development structures and managing databases, web pages, and other components on a web application.

Rails isn't super opinionated about database writes, its mostly left up to developers to discover that for relational DBs you do not want to be doing a bunch of small writes all at once.

That said it specifically has tools to address this that started appearing a few years ago https://github.com/rails/rails/pull/35077

The way my team handles it is to stick Kafka in between whats generating the records (for us, a bunch of web scraping workers) and and a consumer that pulls off the Kafka queue and runs an insert when its internal buffer reaches around 50k rows.

Rails is also looking to add some more direct background type work with https://github.com/basecamp/solid_queue but this is still very new - most larger Rails shops are going to be running a second system and a gem called Sidekiq that pulls jobs out of Redis.

Here is what strict_loading does (source):

Kamal is a wonderfully simple way to deploy your applications anywhere. It will also be included by default in Rails 8. Kamal is trivial, but I don’t recommend using it on your development machine.

I think that you're conflating correlation with causation. I think it's more plausible to assume it was the early numbers that are skewed and non-representative.

The fact that GitHub itself was is a killer app of the Ruby on Rails, and that the Rails project itself changed to being hosted on GitHub somewhat very early on it's history [1] had a disproportionate effect on the early community that gathered there.

Now GitHub attracts a much more diverse portfolio of projects, so the numbers you see there are less statistically biased towards early Ruby on Rails adopters.

[1] Commit history on the main branch of rails/rails via github goes as far as Apr 10, 2008 https://github.com/rails/rails/commit/c67e985994362290308073...

Done! https://github.com/rails/rails/pull/50295

releaseCycle: "6.1" releaseDate: 2020-12-09 eol: 2024-06-01 # https://github.com/rails/rails/pull/46895#issuecomment-1673353127 latest: "6.1.7.6" latestReleaseDate: 2023-08-22

You might have luck. It does look like docs changes are being accepted into 7.1-stable branch: https://github.com/rails/rails/commits/7-1-stable/

Also this doesn't show how database access is handled which is the hard part. If you are not touching the database, you can run Rails on falcon and get fiber based concurrency.

If you run falcon on rails and access database, then you have to explicitly checkin/checkout a connection to be safe. Details here - https://github.com/rails/rails/issues/42271.

> I’m curious how the impact affects development, deployment, etc.

YJIT is pretty much transparent in production, if not it's likely a bug.

When we tried MJIT in production to compare it against YJIT, it causes lots of request timeouts on deploy, because the JIT warmup would take 10 to 20 minutes and it's much slower during that phase.

But YJIT warms ups extremely fast and with a much lower overhead, it's seemless on deploy.

The only thing you may need to tweak is `--yjit-exec-mem-size`, it defaults to `--yjit-exec-mem-size=256` (MB) which is not quite enough for larger apps.

As for development, it would work, but with code reloading enabled, you'd likely exhaust the executable memory allocation pretty fast, because for now YJIT doesn't GC generated code [0]. It will come soon, hopefully before the 3.1.0 release, but that's one of the reason why it's not enabled by default.

[0] https://github.com/Shopify/yjit/issues/87

Just want to temper expectations because YJIT is still new. But if you run into crashes or bugs, please open an issue with as much detail as you can: https://github.com/Shopify/yjit

You raised a point that the compiler only does a subset. That's actually what I would expect from a new project. I don't expect a full implementation to start. It takes time for a compiler to be mature enough to be general purpose. Here is another Ruby compiler in its infancy: https://github.com/Shopify/yjit.

We allocate our own chunk of executable memory and append/rewrite the end of it as we compile new blocks. We have our own in-memory assembler that's implemented here. It's x86 only right now, totally not portable, but over the course of the summer we're going to work on a backend that can open up the possibility of ARM64 support and some lower-level optimizations.

Yes. I put some suggestions here. I realize that not all of them are practical, but refactoring specific hot methods could make a difference.

Yes we are in touch with the Ruby core devs. They seem open to collaborating. k0kubun (working on MJIT) has contributed to the project: https://github.com/Shopify/yjit/pull/60