-
cookiecutter-data-science
A logical, reasonably standardized, but flexible project structure for doing and sharing data science work.
-
InfluxDB
Power Real-Time Data Analytics at Scale. Get real-time insights from all types of time series data with InfluxDB. Ingest, query, and analyze billions of data points in real-time with unbounded cardinality.
While data science is not SE, it's fundamental to have some structure in your projects since you want the work to be somewhat reproducible. I recommend you start here https://github.com/drivendata/cookiecutter-data-science Since it's a cookie cutter it will be easier to implement at first since they can create the structure by running a short command, after some time you will tailor it to your specific company needs :) For notebooks it's kind of hard, they can't be peer reviewd that easily since cells are editable even after code has been run, keeping the old result... I recommend tools like deepnote, but I'm not sure how well they work for collaboration in notebooks because I never used them yet, I just know they are working on solving these problems. I hope these things help!
At the institute I work at intermediate results are also often not documented. It is sometimes even worse than you described so that the researcher have difficulties to reproduce their own work. Therefore they hired me to develop shournal in order to have at least some safety net to reconstruct command-line work on the shell. Maybe it can help your team too? https://github.com/tycho-kirchner/shournal