interpret VS shap

Maybe InterpretML? It's developed and maintained by Microsoft Research and consolidates a lot of different explainability methods.

You may also find Explainable Boosting Machines interesting: https://github.com/interpretml/interpret

They're a bit like a best of both worlds between linear models and random forests (generalized additive models fit with boosted decision trees)

Disclosure: I helped build this open source package

Microsoft Explainable Boosting Machine (which is a Gaussian Additive Model and not a Gradient Boosted Trees 🙄 model) is a step in that direction https://github.com/interpretml/interpret

Also I'd recommend everyone who works with xgboost to give EBM's a try! They perform comparably (except in the case of extreme interactions) but are actually interpretable! https://github.com/interpretml/interpret/ Beside that they since on runtime they're practically a lookup table they're very quick (at the cost of longer training time).

Open source code by Microsoft: https://github.com/interpretml/interpret (called EBM in this implementation).

If it's not an image, have a go at Microsoft's Explainable Boosting Maching) https://github.com/interpretml/interpret which is not a GBM but a GAM (Gradient Boosting Machine vs Gradient Additive Model). This will also give you explanation via SHAP or LIME values.

something like GitHub - shap/shap: A game theoretic approach to explain the output of any machine learning model.?

A less than 100 lines of code implementation of KernelSHAP because I had a hard time understanding shap's code.

We use tools like SHAP to explain what the model is doing to stakeholders.

https://github.com/slundberg/shap this or https://github.com/marcotcr/lime would be relevant to you, especially if you want to look at explaining a single prediction.

See this closed topic for more detail: https://github.com/slundberg/shap/issues/29

Dr. Molnar recently had a semi-viral post on LinkedIn and on Twitter, where he essentially highlights the booming popularity [and power] of using SHAP for explainable AI (which I agree with), but that it also comes with problems; i.e., the open source implementation has thousands of pull requests, bugs, and issues and yet there is no permanent or significant funding to go in and fix them.

Option 4) create SHAP values https://github.com/slundberg/shap to better understand what the RF did.

txtai pipelines are wrappers around Hugging Face pipelines with logic to easily integrate with txtai's workflow framework. Given that, we can use the SHAP library to explain predictions.