Kiba VS normandy

I started https://github.com/thbar/kiba#kiba-etl to scratch my own itch & be able to write properly structured ETL jobs in Ruby. It was a blank-slate rewrite of something larger (activewarehouse-etl) which I could not maintain anymore.

This landed me not strictly a job, but long term consulting gigs with a number of companies in EU, UK & US.

The job was directly related to the project: companies wanted the expertise of data engineering & ETL, often with Kiba directly, but also in general.

This "side project" was totally worth it :-)

Definitely! As a matter of fact, this is the default data structure I use when writing Ruby ETL code (e.g. https://github.com/thbar/kiba/wiki).

Methods like "except" (https://docs.ruby-lang.org/en/3.2/Hash.html#method-i-except) or "fetch" (raising an error on missing key) are very convenient to write defensive data processing code!

Similarly, in Elixir, I use Maps a lot for the same type of jobs (https://hexdocs.pm/elixir/1.15.4/Map.html), with similar properties.

Ruby has a very cool ETL library named Kiba that fits wonderfully with Ruby's strengths.

Though it might be overkill for your problem, but have you had a look at [kiba-etl](https://github.com/thbar/kiba/blob/master/README.md)?

Kiba

If you don’t use Kiba, but work with data, check it out.

Some years ago I was on a shitty job - not technically, but the company turned out to be inhumane - at a Ruby shop, and on the side I was toying with mini_racer and I just upgraded to some macOS beta where it failed to build. A shitty +1-1 hack† for a compiler flag later and it was back flying.

A month later I received a cold email from a CTO to chat a bit about that PR, turns out they were using mini_racer heavily and forked it for their own purpose, and also created PyMiniRacer for the Python side of things. Next thing I know I got hired. Two years later the company got acquired.

Of course conditionally adding a compiler flag wasn't what got me hired per se, it only got my profile noticed. Probably side projects such as porting go by example to Ruby by implementing a ~1:1 CSP channel API[1], an Electron desktop client for Mattermost basically on a dare[2], ex mode for the Atom editor so that I could have that frackin' `:w`[3], leveraging Blocks to bolt on object-oriented-ness onto C because "closures are a poor man's object"[4], or reverse-engineering the Xbox One USB gamepad and writing a kext to turn it into a HID device on macOS from scratch on a lonely 7+h train ride with passengers judgementally staring at me sideways[4] probably contributed to it a bit.

My takeaway: luck is when preparation meets opportunity; but don't to side projects to get hired, because if you don't get hired then that time is lost. Rather, of all things, scratch your itch, have fun, embrace whatever quirkiness you fancy; no one can take that away from you.

[0]: https://github.com/rubyjs/mini_racer/commit/2086db1bbf2b5de4...

[1]: https://github.com/lloeki/normandy

[2]: https://github.com/lloeki/matterfront

[3]: https://github.com/lloeki/ex-mode

[4]: https://github.com/lloeki/cblocks-clobj/blob/master/main.c

[5]: https://github.com/lloeki/xbox_one_controller

Do learn Ruby. You seem to answer your own question! You’re curious about it, worst case you’ll have opened your mind to something else which is only a good thing.

But do not be fooled by Rails, Ruby is quite something else, of which Rails is a very small, opinionated part.

Tips: look at MiniTest source code, Sinatra and Rack source code are quite interesting too.

Shameless plug, a couple of idiomatic Ruby repos of mine:

https://github.com/lloeki/normandy

https://github.com/lloeki/rebel