The Role of Machine Learning Model Interpretability in Improving the Quality of Strategic Decision-Making
Keywords:
Machine learning interpretability, Explainable artificial intelligence, Strategic decision-making, Human-centered ai, Decision qualityAbstract
This study aims to explore the role of machine learning model interpretability in improving the quality of strategic decision-making. The study addresses the growing use of machine learning in organizational decision processes and the problem of limited transparency in AI-generated recommendations. A qualitative case study approach was used to examine how decision-makers understand, evaluate, and apply model explanations in strategic contexts. Data were collected through semi-structured interviews, non-participant observation, and document analysis involving managers, strategic planners, data analysts, data scientists, AI project leaders, risk officers, and policy decision-makers who had direct experience with machine learning-based decision support systems. Thematic analysis identified four main themes: interpretability as a basis for trust formation, interpretability as a bridge between technical analysis and managerial judgment, interpretability as a tool for reducing uncertainty, and interpretability as a mechanism for accountability and justification. The findings show that decision-makers use model explanations not merely to understand algorithmic outputs, but also to negotiate meaning, validate recommendations, and align AI-supported insights with organizational priorities. This study contributes to human-centered explainable AI and strategic decision-making theory by showing that interpretability functions as a social, managerial, and organizational process. The findings imply that organizations should develop AI systems and decision policies that prioritize clear explanations, contextual relevance, uncertainty disclosure, and accountability. Future research should compare interpretability practices across sectors and examine how data literacy and organizational culture influence AI-supported strategic decisions.
References
Adeoye-Olatunde, O. A., & Olenik, N. L. (2021). Research and scholarly methods: Semi-structured interviews. JACCP: Journal of the American College of Clinical Pharmacy, 4(10), 1358–1367. https://doi.org/10.1002/jac5.1441
Ahmed, S. K. (2024). The pillars of trustworthiness in qualitative research. Journal of Medicine, Surgery, and Public Health, 2, 100051. https://doi.org/10.1016/j.glmedi.2024.100051
Ali, S., Abuhmed, T., El-Sappagh, S., Muhammad, K., Alonso-Moral, J. M., Confalonieri, R., Guidotti, R., Del Ser, J., Díaz-Rodríguez, N., & Herrera, F. (2023). Explainable artificial intelligence (XAI): What we know and what is left to attain trustworthy artificial intelligence. Information Fusion, 99, 101805. https://doi.org/10.1016/j.inffus.2023.101805
Angelov, P. P., Soares, E. A., Jiang, R., Arnold, N. I., & Atkinson, P. M. (2021). Explainable artificial intelligence: An analytical review. Wiley Interdisciplinary Reviews: Data Mining and Knowledge Discovery, 11(5), e1424. https://doi.org/10.1002/widm.1424
Barredo Arrieta, A., Díaz-Rodríguez, N., Del Ser, J., Bennetot, A., Tabik, S., Barbado, A., García, S., Gil-López, S., Molina, D., Benjamins, R., Chatila, R., & Herrera, F. (2020). Explainable artificial intelligence (XAI): Concepts, taxonomies, opportunities and challenges toward responsible AI. Information Fusion, 58, 82–115. https://doi.org/10.1016/j.inffus.2019.12.012
Busetto, L., Wick, W., & Gumbinger, C. (2020). How to use and assess qualitative research methods. Neurological Research and Practice, 2, 14. https://doi.org/10.1186/s42466-020-00059-z
Christou, P. A. (2023). How to use thematic analysis in qualitative research. Journal of Qualitative Research in Tourism, 3(2), 79–95. https://doi.org/10.4337/jqrt.2023.0006
Coussement, K., Abedin, M. Z., Kraus, M., Maldonado, S., & Topuz, K. (2024). Explainable AI for enhanced decision-making. Decision Support Systems, 184, 114276. https://doi.org/10.1016/j.dss.2024.114276
Csaszar, F. A., Ketkar, H., & Kim, H. (2024). Artificial intelligence and strategic decision-making: Evidence from entrepreneurs and investors. Strategy Science, 9(4), 322–345. https://doi.org/10.1287/stsc.2024.0190
Doshi, A. R., Bell, J. J., Mirzayev, E., & Vanneste, B. S. (2025). Generative artificial intelligence and evaluating strategic decisions. Strategic Management Journal, 46(3), 583–610. https://doi.org/10.1002/smj.3677
Ehsan, U., Liao, Q. V., Muller, M., Riedl, M. O., & Weisz, J. D. (2021). Expanding explainability: Towards social transparency in AI systems. Proceedings of the 2021 CHI Conference on Human Factors in Computing Systems, 1–19. https://doi.org/10.1145/3411764.3445188
Enholm, I. M., Papagiannidis, E., Mikalef, P., & Krogstie, J. (2022). Artificial intelligence and business value: A literature review. Information Systems Frontiers, 24, 1709–1734. https://doi.org/10.1007/s10796-021-10186-w
Hamilton, A. B., & Finley, E. P. (2020). Reprint of: Qualitative methods in implementation research: An introduction. Psychiatry Research, 283, 112629. https://doi.org/10.1016/j.psychres.2019.112629
Hase, P., & Bansal, M. (2020). Evaluating explainable AI: Which algorithmic explanations help users predict model behavior? Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, 5540–5552. https://doi.org/10.18653/v1/2020.acl-main.491
Johnson, J. L., Adkins, D., & Chauvin, S. (2020). A review of the quality indicators of rigor in qualitative research. American Journal of Pharmaceutical Education, 84(1), 7120. https://doi.org/10.5688/ajpe7120
Kaur, H., Nori, H., Jenkins, S., Caruana, R., Wallach, H., & Wortman Vaughan, J. (2020). Interpreting interpretability: Understanding data scientists’ use of interpretability tools for machine learning. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–14. https://doi.org/10.1145/3313831.3376219
Keding, C. (2021). Understanding the interplay of artificial intelligence and strategic management: Four decades of research in review. Management Review Quarterly, 71, 91–134. https://doi.org/10.1007/s11301-020-00181-x
Kim, J., Liao, Q. V., & Wortman Vaughan, J. (2024). Human-centered evaluation of explainable AI applications: A systematic review. Frontiers in Artificial Intelligence, 7, 1456486. https://doi.org/10.3389/frai.2024.1456486
Liao, Q. V., Gruen, D., & Miller, S. (2020). Questioning the AI: Informing design practices for explainable AI user experiences. Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems, 1–15. https://doi.org/10.1145/3313831.3376590
Lim, W. M. (2025). What is qualitative research? An overview and guidelines. Australasian Marketing Journal, 33(2), 199–229. https://doi.org/10.1177/14413582241264619
Longo, L., Brcic, M., Cabitza, F., Choi, J., Confalonieri, R., Del Ser, J., Guidotti, R., Hayashi, Y., Herrera, F., Holzinger, A., Jiang, R., Khosravi, H., Lecue, F., Malgieri, G., Páez, A., Samek, W., Schneider, J., Speith, T., & Stumpf, S. (2024). Explainable artificial intelligence (XAI) 2.0: A manifesto of open challenges and interdisciplinary research directions. Information Fusion, 106, 102301. https://doi.org/10.1016/j.inffus.2024.102301
Magaldi, D., & Berler, M. (2020). Semi-structured interviews. In V. Zeigler-Hill & T. K. Shackelford (Eds.), Encyclopedia of personality and individual differences (pp. 4825–4830). Springer. https://doi.org/10.1007/978-3-319-24612-3_857
Mohseni, S., Zarei, N., & Ragan, E. D. (2021). A multidisciplinary survey and framework for design and evaluation of explainable AI systems. ACM Transactions on Interactive Intelligent Systems, 11(3–4), Article 24. https://doi.org/10.1145/3387166
Naeem, M., Ozuem, W., Howell, K., & Ranfagni, S. (2023). A step-by-step process of thematic analysis to develop a conceptual model in qualitative research. International Journal of Qualitative Methods, 22, 1–18. https://doi.org/10.1177/16094069231205789
Nauta, M., Trienes, J., Pathak, S., Nguyen, E., Peters, M., Schmitt, Y., Schlötterer, J., van Keulen, M., & Seifert, C. (2023). From anecdotal evidence to quantitative evaluation methods: A systematic review on evaluating explainable AI. ACM Computing Surveys, 55(13s), Article 295. https://doi.org/10.1145/3583558
O’Connor, C., & Joffe, H. (2020). Intercoder reliability in qualitative research: Debates and practical guidelines. International Journal of Qualitative Methods, 19, 1–13. https://doi.org/10.1177/1609406919899220
Rai, A. (2020). Explainable AI: From black box to glass box. Journal of the Academy of Marketing Science, 48, 137–141. https://doi.org/10.1007/s11747-019-00710-5
Raisch, S., & Krakowski, S. (2021). Artificial intelligence and management: The automation-augmentation paradox. Academy of Management Review, 46(1), 192–210. https://doi.org/10.5465/amr.2018.0072
Rong, Y., Leemann, T., Nguyen, T. T., Fiedler, L., Qian, P., Unhelkar, V., Seidel, T., Kasneci, G., & Kasneci, E. (2024). Towards human-centered explainable AI: A survey of user studies for model explanations. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(4), 2104–2122. https://doi.org/10.1109/TPAMI.2023.3331846
Vilone, G., & Longo, L. (2021). Notions of explainability and evaluation approaches for explainable artificial intelligence. Information Fusion, 76, 89–106. https://doi.org/10.1016/j.inffus.2021.05.009
Downloads
Published
Issue
Section
License
Copyright (c) 2026 Intan Putri, Slamet Hidayat, Vina Saputra (Author)

This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.




