Explainable Artificial Intelligence for Trustworthy Decision-Making in Intelligent Computing Systems
Keywords:
Explainable artificial intelligence (XAI), trustworthy AI, interpretability, Shapley values, LIME, integrated gradients, counterfactual explanations, faithfulness, calibration, selective prediction, shortcut learning.Abstract
Intelligent computing systems increasingly make or support high-stakes decisions, and regulations now require that such decisions be transparent and open to human oversight. Explainable AI (XAI) methods are widely used to meet these demands, yet they are rarely evaluated quantitatively, and different methods often disagree. This paper proposes a trustworthy decision framework that couples predictive models with explanation quality assurance, calibrated confidence, and actionable recourse, and evaluates it on three real benchmark datasets (breast-cancer diagnosis, diabetes progression, handwritten digits) and a synthetic dataset with known ground truth. Five local explanation methods (SHAP, LIME, Integrated Gradients, Gradient×Input, and Occlusion) are implemented from first principles and assessed for correctness, faithfulness, stability, agreement, and sanity, across glass-box and black-box models. Interpretable models match black-box accuracy on the clinical datasets (logistic regression AUC 0.840 vs. 0.809 for the best black box on diabetes). Shapley values recover the true feature relevance most accurately (top-3 agreement 0.85 for gradient boosting), whereas gradient-based methods collapse on tree ensembles and perform no better than random deletion. Methods agree on only 41% of their top-5 features for gradient boosting. Signed attribution maps pass a model-randomization sanity check, whereas absolute maps largely track input intensity. Explanations exposed an injected shortcut artefact as the top-ranked feature, and removing it raised deployment AUC from 0.951 to 0.978. Actionability-constrained counterfactuals required changes to 2.4 features on average, and calibrated selective prediction raised accuracy on diabetes from 71.3% to 76.6% at 80% coverage. The paper concludes with evidence-based guidelines for deploying XAI in trustworthy decision systems.
References
[1] European Parliament and Council, “Regulation (EU) 2016/679 (General Data Protection Regulation),” Off. J. Eur. Union, vol. L 119, pp. 1–88, 2016.
[2] European Parliament and Council, “Regulation (EU) 2024/1689 laying down harmonised rules on artificial intelligence (Artificial Intelligence Act),” Off. J. Eur. Union, 2024.
[3] National Institute of Standards and Technology, “Artificial Intelligence Risk Management Framework (AI RMF 1.0),” NIST AI 100-1, Gaithersburg, MD, USA, 2023.
[4] F. Doshi-Velez and B. Kim, “Towards a rigorous science of interpretable machine learning,” arXiv:1702.08608, 2017.
[5] A. Adadi and M. Berrada, “Peeking inside the black-box: A survey on explainable artificial intelligence (XAI),” IEEE Access, vol. 6, pp. 52138–52160, 2018.
[6] A. Barredo Arrieta et al., “Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI,” Inf. Fusion, vol. 58, pp. 82–115, 2020.
[7] D. Gunning and D. W. Aha, “DARPA’s explainable artificial intelligence (XAI) program,” AI Mag., vol. 40, no. 2, pp. 44–58, 2019.
[8] M. T. Ribeiro, S. Singh, and C. Guestrin, “‘Why should I trust you?’: Explaining the predictions of any classifier,” in Proc. 22nd ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2016, pp. 1135–1144.
[9] S. M. Lundberg and S.-I. Lee, “A unified approach to interpreting model predictions,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 30, 2017, pp. 4765–4774.
[10] M. Sundararajan, A. Taly, and Q. Yan, “Axiomatic attribution for deep networks,” in Proc. 34th Int. Conf. Mach. Learn. (ICML), 2017, pp. 3319–3328.
[11] S. Wachter, B. Mittelstadt, and C. Russell, “Counterfactual explanations without opening the black box: Automated decisions and the GDPR,” Harvard J. Law Technol., vol. 31, no. 2, pp. 841–887, 2018.
[12] C. Rudin, “Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead,” Nature Mach. Intell., vol. 1, no. 5, pp. 206–215, 2019.
[13] W. Samek, A. Binder, G. Montavon, S. Lapuschkin, and K.-R. Müller, “Evaluating the visualization of what a deep neural network has learned,” IEEE Trans. Neural Netw. Learn. Syst., vol. 28, no. 11, pp. 2660–2673, 2017.
[14] D. Alvarez-Melis and T. S. Jaakkola, “On the robustness of interpretability methods,” arXiv:1806.08049, 2018.
[15] J. Adebayo, J. Gilmer, M. Muelly, I. Goodfellow, M. Hardt, and B. Kim, “Sanity checks for saliency maps,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 31, 2018, pp. 9505–9515.
[16] D. Slack, S. Hilgard, E. Jia, S. Singh, and H. Lakkaraju, “Fooling LIME and SHAP: Adversarial attacks on post hoc explanation methods,” in Proc. AAAI/ACM Conf. AI, Ethics, Soc. (AIES), 2020, pp. 180–186.
[17] S. Krishna et al., “The disagreement problem in explainable machine learning: A practitioner’s perspective,” arXiv:2202.01602, 2022.
[18] T. Hastie and R. Tibshirani, Generalized Additive Models. London, U.K.: Chapman & Hall, 1990.
[19] Y. Lou, R. Caruana, J. Gehrke, and G. Hooker, “Accurate intelligible models with pairwise interactions,” in Proc. 19th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining, 2013, pp. 623–631.
[20] E. Štrumbelj and I. Kononenko, “Explaining prediction models and individual predictions with feature contributions,” Knowl. Inf. Syst., vol. 41, no. 3, pp. 647–665, 2014.
[21] M. D. Zeiler and R. Fergus, “Visualizing and understanding convolutional networks,” in Proc. Eur. Conf. Comput. Vis. (ECCV), 2014, pp. 818–833.
[22] M. Ancona, E. Ceolini, C. Öztireli, and M. Gross, “Towards better understanding of gradient-based attribution methods for deep neural networks,” in Proc. Int. Conf. Learn. Represent. (ICLR), 2018.
[23] L. S. Shapley, “A value for n-person games,” in Contributions to the Theory of Games II, H. W. Kuhn and A. W. Tucker, Eds. Princeton, NJ, USA: Princeton Univ. Press, 1953, pp. 307–317.
[24] T. Laugel, M.-J. Lesot, C. Marsala, X. Renard, and M. Detyniecki, “Inverse classification for comparison-based interpretability in machine learning,” arXiv:1712.08443, 2017.
[25] S. Hooker, D. Erhan, P.-J. Kindermans, and B. Kim, “A benchmark for interpretability methods in deep neural networks,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 32, 2019.
[26] C. Guo, G. Pleiss, Y. Sun, and K. Q. Weinberger, “On calibration of modern neural networks,” in Proc. 34th Int. Conf. Mach. Learn. (ICML), 2017, pp. 1321–1330.
[27] Y. Geifman and R. El-Yaniv, “Selective classification for deep neural networks,” in Proc. Adv. Neural Inf. Process. Syst. (NeurIPS), vol. 30, 2017.
[28] S. Lapuschkin, S. Wäldchen, A. Binder, G. Montavon, W. Samek, and K.-R. Müller, “Unmasking Clever Hans predictors and assessing what machines really learn,” Nature Commun., vol. 10, Art. no. 1096, 2019.
[29] R. Geirhos et al., “Shortcut learning in deep neural networks,” Nature Mach. Intell., vol. 2, no. 11, pp. 665–673, 2020.
[30] T. Miller, “Explanation in artificial intelligence: Insights from the social sciences,” Artif. Intell., vol. 267, pp. 1–38, 2019.
[31] W. N. Street, W. H. Wolberg, and O. L. Mangasarian, “Nuclear feature extraction for breast tumor diagnosis,” in Proc. SPIE, vol. 1905, Biomedical Image Processing and Biomedical Visualization, 1993, pp. 861–870.
[32] B. Efron, T. Hastie, I. Johnstone, and R. Tibshirani, “Least angle regression,” Ann. Statist., vol. 32, no. 2, pp. 407–499, 2004.
[33] E. Alpaydin and C. Kaynak, “Optical recognition of handwritten digits,” UCI Machine Learning Repository, 1998, doi: 10.24432/C50P49.
[34] L. Breiman, “Random forests,” Mach. Learn., vol. 45, no. 1, pp. 5–32, 2001.
[35] J. H. Friedman, “Greedy function approximation: A gradient boosting machine,” Ann. Statist., vol. 29, no. 5, pp. 1189–1232, 2001.
[36] F. Pedregosa et al., “Scikit-learn: Machine learning in Python,” J. Mach. Learn. Res., vol. 12, pp. 2825–2830, 2011.
[37] J. C. Platt, “Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods,” in Advances in Large Margin Classifiers. Cambridge, MA, USA: MIT Press, 1999, pp. 61–74.
Downloads
Published
Data Availability Statement
no
Issue
Section
License
Copyright (c) 2026 Suresh Babu Narra (Author)
This work is licensed under a Creative Commons Attribution 4.0 International License.
This journal is licensed under the Creative Commons Attribution 4.0 International License (CC BY 4.0). Authors retain copyright in their published work and permit others to share and adapt the material for any purpose, provided appropriate credit is given to the original author(s) and source, a link to the license is provided, and any changes made are indicated.
