imodels VS shap

- all domains requiring high interpretability absolutely ignore deep learning at all, and put all their research into traditional ML; see e.g. counterfactual examples, important interpretability methods in finance, or rule-based learning, important in medical or law applications

Next, this variable will be your target and you can use various supervised learning models to answer your question. Since interpretation is key, you can use something from here: https://github.com/csinva/imodels or do some black box models and use shab to understand which features contributed most.

Option 2) fit a model from https://github.com/csinva/imodels on the predicted values of the RF

Despite recent breakthroughs in the formulation and fitting of interpretable models, implementations are frequently challenging to locate, utilize, and compare. imodels solves this void by offering a single interface and implementation for a wide range of state-of-the-art interpretable modeling techniques, especially rule-based methods. imodels is basically a Python tool for predictive modeling that is simple, transparent, and accurate. It gives users a straightforward way to fit and use state-of-the-art interpretable models, all of which are compatible with scikit-learn (Pedregosa et al., 2011). These models can frequently replace black-box models while boosting interpretability and computing efficiency without compromising forecast accuracy. Continue Reading

Our package imodels is expanding our sklearn-compatible set of interpretable models and always looking for new contributors!

something like GitHub - shap/shap: A game theoretic approach to explain the output of any machine learning model.?

A less than 100 lines of code implementation of KernelSHAP because I had a hard time understanding shap's code.

We use tools like SHAP to explain what the model is doing to stakeholders.

https://github.com/slundberg/shap this or https://github.com/marcotcr/lime would be relevant to you, especially if you want to look at explaining a single prediction.

See this closed topic for more detail: https://github.com/slundberg/shap/issues/29

Dr. Molnar recently had a semi-viral post on LinkedIn and on Twitter, where he essentially highlights the booming popularity [and power] of using SHAP for explainable AI (which I agree with), but that it also comes with problems; i.e., the open source implementation has thousands of pull requests, bugs, and issues and yet there is no permanent or significant funding to go in and fix them.

Option 4) create SHAP values https://github.com/slundberg/shap to better understand what the RF did.

txtai pipelines are wrappers around Hugging Face pipelines with logic to easily integrate with txtai's workflow framework. Given that, we can use the SHAP library to explain predictions.