Counterfactual Explanations for Incorrect Predictions Made by AI Models

2023 | International Conference on Information Systems | Citations: 0

Authors: Asrzad, Amir; Li, Xiaobai

Abstract: Advanced AI models are powerful in making accurate predictions for complex prob ... Expand

Abstract: Advanced AI models are powerful in making accurate predictions for complex problems. However, these models often operate as black boxes. This lack of interpretability poses significant challenges, especially in high-stakes applications such as finance, healthcare, and criminal justice. Explainable AI seeks to address the challenges by developing methods that can provide meaningful explanations for humans to understand. When black box models are used for prediction, they inevitably produce errors. It is important to appropriately explain incorrect predictions. This problem, however, has not been addressed in the literature. In this study, we propose a novel method to provide explanations for misclassified cases made by black box models. The proposed method takes a counterfactual explanation approach. It builds a decision tree to find the best counterfactual examples for explanations. Incorrect predictions are rectified using a trust score measure. We validate the proposed method in an evaluation study using real-world data. Collapse

Topics: artificial intelligence explainable artificial intelligence local interpretable model agnostic explanation decision making enterprise information system

Methods: machine learning decision tree classification literature study random forest classification experiment

Rumor Classification through a Multimodal Fusion Framework and Ensemble Learning

2023 | Information Systems Frontiers | Citations: 0

Authors: Azri, Abderrazek; Favre, Cécile; Harbi, Nouria; Darmont, Jérôme; Noûs, Camille

Abstract: The proliferation of rumors on social media has become a major concern due to it ... Expand

Abstract: The proliferation of rumors on social media has become a major concern due to its ability to create a devastating impact. Manually assessing the veracity of social media messages is a very time-consuming task that can be much helped by machine learning. Most message veracity verification methods only exploit textual contents and metadata. Very few take both textual and visual contents, and more particularly images, into account. Moreover, prior works have used many classical machine learning models to detect rumors. However, although recent studies have proven the effectiveness of ensemble machine learning approaches, such models have seldom been applied. Thus, in this paper, we propose a set of advanced image features that are inspired from the field of image quality assessment, and introduce the Multimodal fusiON framework to assess message veracIty in social neTwORks (MONITOR), which exploits all message features by exploring various machine learning models. Moreover, we demonstrate the effectiveness of ensemble learning algorithms for rumor detection by using five metalearning models. Eventually, we conduct extensive experiments on two real-world datasets. Results show that MONITOR outperforms state-of-the-art machine learning baselines and that ensemble models significantly increase MONITOR’s performance. Collapse

Topics: Twitter social network Python microblog logistics management

Methods: computational algorithm machine learning experiment meta learning ensemble learning

Modern Centaurs: How Humans and AI Systems Interact in Sales Forecasting

2023 | European Conference On Information Systems | Citations: 0

Authors: Beese, Jannis; Fahse, Tobias

Abstract: Recent achievements of artificial intelligence (AI) have caused organizations t ... Expand

Abstract: Recent achievements of artificial intelligence (AI) have caused organizations to increasingly bring AI capabilities into their core business processes. Such AI-supported business processes often result in human-AI hybrid systems, which consist of an AI system, which performs most of the execution, and humans, who monitor this execution and occasionally provide additional inputs and overrides. Using sales data from Walmart, we conduct an online study to investigate if human supervision can improve upon state-of-the-art AI forecasts. Furthermore, we analyze the perceptions and behavioral intentions of the human participants over time. We find that human interventions consistently lead to less accurate forecasts and that participants initially underestimate the AI system's accuracy and overestimate their own potential to improve upon AI forecasts. However, perceptions quickly shift over the course of the study, causing the participants to perceive the AI system increasingly favorably, which also leads to behavioral changes and better hybrid system performance. Collapse

Topics: artificial intelligence IT productivity information system use task productivity perceived usefulness

Methods: regression analysis method survey LightGBM conceptual modelling experiment

Theories: technology acceptance model

Applying Random Forests in Federated Learning: A Synthesis of Aggregation Techniques

2023 | International Conference on Business Informatics | Citations: 0

Authors: Bodynek, Mattis; Leiser, Florian; Thiebes, Scott; Sunyaev, Ali

Abstract: ... Expand

Abstract: Collapse

Topics: privacy decision making evaluation criteria

Methods: literature study decision tree classification structured literature research

AI-Enhanced Hybrid Decision Management

2023 | Business & Information Systems Engineering | Citations: 0

Authors: Bork, Dominik; Ali, Syed Juned; Dinev, Georgi Milenov

Abstract: The Decision Model and Notation (DMN) modeling language allows the precise speci ... Expand

Abstract: The Decision Model and Notation (DMN) modeling language allows the precise specification of business decisions and business rules. DMN is readily understandable by business users involved in decision management. However, as the models get complex, the cognitive abilities of humans threaten manual maintainability and comprehensibility. Proper design of the decision logic thus requires comprehensive automated analysis of e.g., all possible cases the decision shall cover; correlations between inputs and outputs; and the importance of inputs for deriving the output. In the paper, the authors explore the mutual benefits of combining human-driven DMN decision modeling with the computational power of Artificial Intelligence for DMN model analysis and improved comprehension. The authors propose a model-driven approach that uses DMN models to generate Machine Learning (ML) training data and show, how the trained ML models can inform human decision modelers by means of superimposing the feature importance within the original DMN models. An evaluation with multiple real DMN models from an insurance company evaluates the feasibility and the utility of the approach. Collapse

Topics: artificial intelligence explainable artificial intelligence perceived usefulness price management extensible markup language

Methods: computational algorithm decision tree classification experiment conceptual modelling simulation

Augmenting Social Bot Detection with Crowd-Generated Labels

2023 | Information Systems Research | Citations: 0

Authors: Benjamin, Victor; Raghu, T. S.

Abstract: Social media platforms are facing increasing numbers of cyber-adversaries seekin ... Expand

Abstract: Social media platforms are facing increasing numbers of cyber-adversaries seeking to manipulate online discourse by using social bots (i.e., social media software robots) to help automate and scale their attacks. Likewise, some social media users can identify social bot activity at varying degrees of confidence. In this research, human reactions to social bot messages are used to augment existing social bot detection capabilities. Speech act theory is used to inspire a framework for assessing the credibility of instances where users identify potential bot activity, as not all user responses are of equal credibility for assisting with the bot detection task. The framework is then operationalized through deep learning methodologies to develop a computational system for identifying social bots. Real-world performance and practicality of the developed framework is demonstrated on a live, crowd-sourced data set collected from a real-world social media platform. Results show that consideration of crowd reactions to suspected bots can significantly improve bot detection performance. Furthermore, consideration of speech acts to evaluate crowd reactions can even further augment the system’s performance, although speech acts themselves are not necessary to observe performance boost through crowd intelligence. This study serves as a grounding point for future work that can explore an augmented model for detecting other forms of algorithmically generated content within social media platforms.History: Olivia Sheng, Senior Editor; Jesse Bockstedt, Associate Editor.Supplemental Material: The online appendix is available at https://doi.org/10.1287/isre.2022.1136. Collapse

Topics: Reddit social media crowdsourcing IT productivity social media analytics

Methods: experiment BERT machine learning support vector machine random forest classification

Theories: speech act theory

Business Inferences and Risk Modeling with Machine Learning; The case of Aviation Incidents

2023 | HICSS | Citations: 0

Authors: Cankaya, B.; Topuz, K.; Glassman, A.

Abstract: Machine learning becomes truly valuable only when decision-makers begin to depe ... Expand

Abstract: Machine learning becomes truly valuable only when decision-makers begin to depend on it to optimize decisions. Instilling trust in machine learning is critical for businesses in their efforts to interpret and get insights into data, and to make their analytical choices accessible and subject to accountability. In the field of aviation, the innovative application of machine learning and analytics can facilitate an understanding of the risk of accidents and other incidents. These occur infrequently, generally in an irregular, unpredictable manner, and cause significant disruptions, and hence, they are classified as "highimpact, low-probability" (HILP) events. Aviation incident reports are inspected by experts, but it is also important to have a comprehensive overview of incidents and their holistic effects. This study provides an interpretable machine-learning framework for predicting aircraft damage. In addition, it describes patterns of flight specifications detected through the use of a simulation tool and illuminates the underlying reasons for specific aviation accidents. As a result, we can predict the aircraft damage with 85% accuracy and 84% in-class accuracy. Most important, we simulate a combination of possible flight-type, aircraft-type, and pilot-expertise combinations to arrive at insights, and we recommend actions that can be taken by aviation stakeholders, such as airport managers, airlines, flight training companies, and aviation policy makers. In short, we combine predictive results with simulations to interpret findings and prescribe actions.A machine-learning (ML) algorithm was chosen for its fast performance, short run-time, and interpretability. The ML algorithms were compared and evaluated on a confusion matrix for their mean receiver operating characteristic curve (ROC), Collapse

Topics: decision making decision support system graphical user interface usability missing data

Methods: simulation machine learning multinomial logistic regression computational algorithm support vector machine

Deploying Artificial Intelligence to Combat Covid-19 Misinformation on Social Media: Technological and Ethical Considerations

2023 | HICSS | Citations: 0

Authors: Cartwright, B.; Frank, R.; Weir, G.; Padda, K.; Strange, S.

Abstract: This paper reports on AI research into online misinformation pertaining to the ... Expand

Abstract: This paper reports on AI research into online misinformation pertaining to the COVID-19 pandemic within the Canadian context. This is part of our longerterm goal, i.e., development of a machine-learning tool to assist social media platforms, online service providers and government agencies in identifying and responding to misinformation on social media. We report on predictive accuracies accomplished by applying a combination of technologies, including a custom-designed web-crawler, The Dark Crawler, the Posit toolkit, and four different machine-learning models based on Naïve Bayes, Support Vector Machines, LibLinear and LibShortText. Overall, we found that Posit and LibShortText models showed higher levels of correlation to the pre-determined (manual and machine-driven) data classifications than the other machine-learning algorithms tested. We further argue that the harms associated with COVID-19 misinformatione.g., the social and economic damage, and the deaths and severe illnessesoutweigh the right to personal privacy and freedom of speech considerations. Collapse

Topics: misinformation Twitter Reddit social media pandemic

Methods: machine learning computational algorithm support vector machine decision tree classification random forest classification

The Relationship Between Twitter Sentiment and Stock Performance: A Decision Tree Approach

2023 | HICSS | Citations: 0

Authors: Chen, Rongjuan; Dong, Ruoxi

Abstract: Social media has become a communication tool, but also a valuable database for r ... Expand

Abstract: Social media has become a communication tool, but also a valuable database for researchers and practitioners to gather information, share knowledge, as well as express opinions about stock performance. The sentiment embedded in social media content can be analyzed to predict stock performance. Although numerous past studies have attempted to predict stock price movement using social media sentiment, some emerging analytical tools, like existing lexicons, may require further testing and validation in a financial decision making context. In this study, we develop and test predictive models for stock price and trend forecasting. By using a large-scale sample of tweets collected from Twitter, related to four companies, Apple, Google, Microsoft, and Netflix, we propose a novel decision tree approach to stock performance prediction. Based on our findings, we then provide theoretical and practical implications and discuss the directions for future work. Collapse

Topics: Twitter social media decision making digital service Netflix

Methods: sentiment analysis decision tree classification regression analysis method multiple linear regression NLP tool

Theories: behavioral economics

Understanding Emojis for Financial Sentiment Analysis

2023 | International Conference on Information Systems | Citations: 0

Authors: Chen, Siyi; Xing, Frank

Abstract: ... Expand

Abstract: Collapse

Topics: application programming interface social media website cryptocurrency Python

Methods: sentiment analysis machine learning BERT deep learning experiment