CARLA VS sagemaker-explaining-credit-decisions

Abstract: Counterfactual explanations provide means for prescriptive model explanations by suggesting actionable feature changes (e.g., increase income) that allow individuals to achieve favourable outcomes in the future (e.g., insurance approval). Choosing an appropriate method is a crucial aspect for meaningful counterfactual explanations. As documented in recent reviews, there exists a quickly growing literature with available methods. Yet, in the absence of widely available open–source implementations, the decision in favour of certain models is primarily based on what is readily available. Going forward – to guarantee meaningful comparisons across explanation methods – we present CARLA (Counterfactual And Recourse Library), a python library for benchmarking counterfactual explanation methods across both different data sets and different machine learning models. In summary, our work provides the following contributions: (i) an extensive benchmark of 11 popular counterfactual explanation methods, (ii) a benchmarking framework for research on future counterfactual explanation methods, and (iii) a standardized set of integrated evaluation measures and data sets for transparent and extensive comparisons of these methods. We have open sourced CARLA and our experimental results on GitHub, making them available as competitive baselines. We welcome contributions from other research groups and practitioners.

4 Min Read| Paper | Github

Have looked at: https://sagemaker-examples.readthedocs.io/en/latest/advanced_functionality/scikit_bring_your_own/scikit_bring_your_own.html https://docs.aws.amazon.com/sagemaker/latest/dg/docker-containers-create.html https://sagemaker-immersionday.workshop.aws/lab3/option1.html The above don't specify LightGBM, but the concept of Bring Your Own Container/Algorithm is the same. I think this article might be more than what you need, buy it does reference LightGBM https://github.com/awslabs/sagemaker-explaining-credit-decisions And also this one https://github.com/aws-samples/amazon-sagemaker-script-mode/blob/master/lightgbm-byo/lightgbm-byo.ipynb

Yes, SageMaker is an end to end service covering data labeling, data preparation, model training, model deployment, model monitoring, etc. This recent video will give you a grand hands-on tour of SageMaker: https://www.twitch.tv/aws/video/929163653. Training and deployment is based on Docker containers, either built-in (algorithms and open source frameworks) or your own. With respect to LightGBM, you can easily start from the built-in scikit-learn container, and add LightGBM to it. Here's a complete example: https://github.com/awslabs/sagemaker-explaining-credit-decisions.