fairlearn VS model-card-toolkit

I'm involved in the Fairlearn project, and we always love getting new contributors. We have a bunch of open issues, ranging from new functionality to writing documentation, so feel free to take a look and see if there is something you would like to work on.

I'm involved with the Fairlearn project, so once I figure out what's necessary from a company policy-side, my plan is to incorporate these methods into Fairlearn one day.

Besides the existing monitoring solution mentioned in the section above, we were also took inspiration from continuous integration and continuous delivery (CI/CD) testing tools like Jenkins and Circle CI, on the engineering front, and existing fairness libraries like Microsoft's Fairlearn and IMB's Fairness 360, on the machine learning side of things.

Another commenter mentioned contributing to open-source tools. If you're interested in going that route, I'm involved in the Fairlearn project, and we could always benefit from a good data engineer.

We explored various solutions in the machine learning space as well as in the neighbouring software development space for inspiration and learning. Some notable ones include Git and Github, Google Model Cardsand IBM's Factsheets. One of the main design decisions we faced was whether to go with a plain-text git-based solution or a structured schema approach. Eventually, we decided to build on top of Google Model Card (structured protobuf schema). This sacrifices immediate readability of the file, but makes it easy to be processed across different systems in a predictable manner.