chronicle-etl
Kiba
| chronicle-etl | Kiba | |
|---|---|---|
| 1 | 8 | |
| 194 | 1,773 | |
| 0.0% | 0.0% | |
| 7.6 | 3.2 | |
| about 2 years ago | 5 months ago | |
| Ruby | Ruby | |
| MIT License | GNU General Public License v3.0 or later |
Stars - the number of stars that a project has on GitHub. Growth - month over month growth in stars.
Activity is a relative number indicating how actively a project is being developed. Recent commits have higher weight than older ones.
For example, an activity of 9.0 indicates that a project is amongst the top 10% of the most actively developed projects that we are tracking.
chronicle-etl
-
Memex like products/communities for gathering personal data?
- Chronicle ETL : a cli for ETL'ing from different data silos https://github.com/chronicle-app/chronicle-etl
Kiba
-
Sequel: The Database Toolkit for Ruby
Hell yes :-)
For the record, I am a deep Sequel fan, and wrote ETL connectors that use it:
https://www.kiba-etl.org/kiba-pro
https://github.com/thbar/kiba/wiki#kiba-pro (full doc with snippets)
It is one of the most underrated gems of the Ruby universe IMO!
-
Ask HN: What side projects landed you a job?
I started https://github.com/thbar/kiba#kiba-etl to scratch my own itch & be able to write properly structured ETL jobs in Ruby. It was a blank-slate rewrite of something larger (activewarehouse-etl) which I could not maintain anymore.
This landed me not strictly a job, but long term consulting gigs with a number of companies in EU, UK & US.
The job was directly related to the project: companies wanted the expertise of data engineering & ETL, often with Kiba directly, but also in general.
This "side project" was totally worth it :-)
-
Ruby's Hash Is a Swiss-Army Knife
Definitely! As a matter of fact, this is the default data structure I use when writing Ruby ETL code (e.g. https://github.com/thbar/kiba/wiki).
Methods like "except" (https://docs.ruby-lang.org/en/3.2/Hash.html#method-i-except) or "fetch" (raising an error on missing key) are very convenient to write defensive data processing code!
Similarly, in Elixir, I use Maps a lot for the same type of jobs (https://hexdocs.pm/elixir/1.15.4/Map.html), with similar properties.
-
Thinking in learn Ruby
Ruby has a very cool ETL library named Kiba that fits wonderfully with Ruby's strengths.
- What ETL tool do you use?
-
Massive SQL import from csv file, nulls, best practices.
Though it might be overkill for your problem, but have you had a look at [kiba-etl](https://github.com/thbar/kiba/blob/master/README.md)?
-
My favorite Ruby gems
Kiba
-
Ruby ETL Strategies: Organizing block-based Kiba Pipelines
If you don’t use Kiba, but work with data, check it out.
What are some alternatives?
YouPlot - A command line tool that draw plots on the terminal.
lru_redux - An efficient optionally thread safe LRU Cache
ISO-3166-Countries-with-Regional-Codes - ISO 3166-1 country lists merged with their UN Geoscheme regional codes in ready-to-use JSON, XML, CSV data sets
activerecord-jdbc-adapter - JRuby's ActiveRecord adapter using JDBC.
friends - Spend time with the people you care about. Introvert-tested. Extrovert-approved.
ALX - ALX is a command line tool for Linux, macOS and Windows to export and import the game data of Skies of Arcadia, Skies of Arcadia Legends, Eternal Arcadia (エターナルアルカディア), and Eternal Arcadia Legends (エターナルアルカディアレジェンド).