MLFlow users, what would you want from an integration with GitLab?

• incubation-engineering

  18 - -

I took a look at few different options, the main issue is that GitLab is ruby, while most options (like nbdime) are in python. It also needs to work by default, so zero effort for the user. What I did was create a markdown from each version, cleaning up a bit metadata and some noisy outputs, and diff them (https://gitlab.com/gitlab-org/incubation-engineering/mlops/rb-ipynbdiff). It's an MVP, but it works well enough and allows for diffing output as well (I will be adding some metadata back soon too). The next step is create a semantic diff algorithm over the JSON tree, and actually render the diffs per cell.

• databooks

  2 103 6.6 Python

  A CLI tool to reduce the friction between data scientists by reducing git conflicts removing notebook metadata and gracefully resolving git conflicts.

  

If you're working on diffs for Jupyter Notebooks, it's worth looking into this: https://github.com/datarootsio/databooks

• InfluxDB

  www.influxdata.com featured

  Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.

  

• orchest

  44 4,022 4.5 TypeScript

  Build data pipelines, the easy way 🛠️

  

• ploomber

  121 3,380 7.4 Python

  The fastest ⚡️ way to build data pipelines. Develop iteratively, deploy anywhere. ☁️

  

I recommend checking out ploomber, great integration with notebooks and Git!

Suggest a related project