Our great sponsors
-
-
WorkOS
The modern identity platform for B2B SaaS. The APIs are flexible and easy-to-use, supporting authentication, user identity, and complex enterprise features like SSO and SCIM provisioning.
-
OpenRefine
OpenRefine is a free, open source power tool for working with messy data and improving it
On the other hand, some of the techniques to set the ground for data analysis are equally valuable in other situations. The two installments about regular expressions on programming historian Understanding Regular Expressions and Cleaning OCR’d text with Regular Expressions, for example. They have no relevance to handling chemicals in the lab, yet since then, I find myself working with data files more efficiently, than earlier because of grep, an utility in Linux to crawl across data files. Or AWK, actually picking up theses "regexes", which I find generally useful since Benjamin Porter's "Hack the planet's text" (presentation video, and exercise video) with its link back to chem/bio e.g., to bioawk (btw, there equally is biopython, too).
Or that you enter into counseling, accreditation: there already are processes somewhat working, and your expertise in (statistical) design of experiments (example entry on CRAN, a blog post) recommends a set of experiments. Your clients perform then the experiments in the lab, and you analyze the data collected. Eventually, the yield of product X is increased, with lower consumption of energy in a shorter time. You can complement R, or Python for this (there is an 101 on learnxinyminutes, too), of course with GUI programs you know and like (e.g., JMP, minitab; orange etc). There are some closer related to chemistry (e.g., DataWarrior.
Prior to analysis of the data, the data have to be collected. Especially if those who collect are not (yet) aware that their data collected equally are interesting for an audience different than to the one anticipated, working with data from different source requires data consolidation. This can be fixing typos, cross-validation with other sources about the same topic, etc. There are routines and programs out there to help here (openrefine an example), but relying on them as prêt-a-porter / ready-to-go likely would limit you.