Benchmarking the Robustness of Phishing Email Detection Systems

2023 | Americas Conference on Information Systems | Citations: 0

Authors: Ampel, Benjamin; Gao, Yang; Hu, James; Samtani, Sagar; Chen, Hsinchun

Abstract: Social engineering attacks are currently the most cited cybersecurity threat to ... Expand

Abstract: Social engineering attacks are currently the most cited cybersecurity threat to organizations. Phishing emails are the most salient form of social engineering attacks. Organizations are increasingly implementing AI-enabled systems to detect phishing emails. However, AI-enabled systems are often susceptible to textual perturbations, where an adversary makes a small change to cause a misclassification. In this study, we sought to identify the performance of prevailing phishing email detection systems (PEDS) against character, word, sentence, and multi-level adversarial text perturbations. Through a principled benchmarking framework, we quantitatively demonstrated the lack of robustness prevailing PEDS have to specific types of text-based adversarial perturbations (e.g., character, word, sentence, multi-level). The results of this study provide new insights into the robustness of AI-based PEDS and highlight the need for organizations to adopt a multi-layered approach to phishing protection. Additionally, organizations can implement our benchmark framework to test their PEDS against adversarial perturbations. Collapse

Topics: artificial intelligence electronic mail email phishing phishing IT security

Methods: word2vec random forest classification naïve bayes literature study pre-trained language model

The motivation paradox: Understanding contradictory effects of awards on contribution quantity versus quality in virtual community

2023 | Information & Management | Citations: 0

Authors: Banerjee, Shankhadeep; Bhattacharyya, Samadrita; Bose, Indranil

Abstract: We use the lens of the intrinsic/extrinsic motivation theory to holistically mod ... Expand

Abstract: We use the lens of the intrinsic/extrinsic motivation theory to holistically model the effects of intangible awards on member contribution in virtual communities. Using multiple studies on Yelp, we validate that the influence of award stimuli on contribution outcomes become significantly moderated by the aspect of contribution (quantity versus quality) and the novelty of stimuli (time and repetition of awards) to interestingly form divergent (downward versus upward) yet theoretically explicable curvilinear trajectories of contribution. This study contributes to IS literature on virtual communities and social-cognitive literature on work motivation and offers insights to community managers to design better incentive systems. Collapse

Topics: online community intrinsic motivation electronic health blockchain online review

Methods: theory development ANOVA parametric test regression analysis method experimental group

Theories: motivation theory self determination theory cognitive evaluation theory

Modern Centaurs: How Humans and AI Systems Interact in Sales Forecasting

2023 | European Conference On Information Systems | Citations: 0

Authors: Beese, Jannis; Fahse, Tobias

Abstract: Recent achievements of artificial intelligence (AI) have caused organizations t ... Expand

Abstract: Recent achievements of artificial intelligence (AI) have caused organizations to increasingly bring AI capabilities into their core business processes. Such AI-supported business processes often result in human-AI hybrid systems, which consist of an AI system, which performs most of the execution, and humans, who monitor this execution and occasionally provide additional inputs and overrides. Using sales data from Walmart, we conduct an online study to investigate if human supervision can improve upon state-of-the-art AI forecasts. Furthermore, we analyze the perceptions and behavioral intentions of the human participants over time. We find that human interventions consistently lead to less accurate forecasts and that participants initially underestimate the AI system's accuracy and overestimate their own potential to improve upon AI forecasts. However, perceptions quickly shift over the course of the study, causing the participants to perceive the AI system increasingly favorably, which also leads to behavioral changes and better hybrid system performance. Collapse

Topics: artificial intelligence IT productivity information system use task productivity perceived usefulness

Methods: regression analysis method survey LightGBM conceptual modelling experiment

Theories: technology acceptance model

Empowering Users to Create Augmented Reality-Based Solutions – Deriving Design Principles for No-Code AR Authoring Tools

2023 | International Conference on Information Systems | Citations: 0

Authors: Bräker, Julia; Hertel, Julia; Semmann, Martin

Abstract: Grounded on an experimental study with 18 participants, we derive 15 design pri ... Expand

Abstract: Grounded on an experimental study with 18 participants, we derive 15 design principles for no-code AR authoring tools in an organizational setting. The study consists of two distinct treatments that aim to augment lightweight processes with AR. The outcomes are two interactive tutorials utilizing AR instructions. Following the no-code approach, the participants were empowered to create relevant AR content using a reduced interface and no need for advanced configurations or coding. The study thus combines two research streams with the aim of better understanding mechanisms for AR use in a professional context. As prior work has shown, despite the potential benefits, the adoption of AR authoring tools is limited because ramping up AR to productive use is heavily dependent on consulting and custom software solutions. Our novel approach bears the potential to broaden application domains and empower professionals to apply AR. Collapse

Topics: mobile application usability user experience information system use augmented reality

Methods: design principle design artifact survey cognitive walkthrough design science

Algorithm-Human-Algorithm: A New Classification Approach to Integrating Judgemental Adjustments

2023 | International Conference on Information Systems | Citations: 0

Authors: Chen, Christopher; Jain, Nitish; Karamshetty, Varun

Abstract: Modern-day firms face the predicament of blending the comparative advantages of ... Expand

Abstract: Modern-day firms face the predicament of blending the comparative advantages of their two core resources: machines and humans. When forecasting demand (e.g. for a product), extant literature documents that always permitting (or prohibiting) human revision of a machine forecast is beneficial if the humans' private information role is larger (or smaller) than that of machine-accessible public information. We propose and test a complementary framework that shifts the focus to the regulation of each human revision; and in doing so, adjusts for human vulnerability to systematic biases. We collaborate with a European retailer to compile a large dataset (∼ 1.1 mn transactions) on machine-led demand forecasts. Humans revise nearly 38% of these forecasts, but revisions do not always yield an improved final forecast. Compared to the always permit or prohibit strategies, the ideal regulation of each of these revisions could reduce the absolute deviation of the final forecast from realized sales in close to 50% of instances. Any regulation to obtain the best of the machine and human-revised forecasts requires an ability to predict revision quality. In addition to private information indicators, we show that indicators of systematic bias in human judgement (e.g., indicators of cognitive load) can improve the prediction of revision quality. In an out-of-sample analysis, our revision-level regulation approach picks the best of available forecasts in 14% more instances, compared to an always-permit or prohibit strategy at the product-store level. Our paper provides practitioners with a novel approach to combining the machine-human output to improve demand forecasting performance. Collapse

Topics: privacy logistics management supply chain management analytical information system

Methods: computational algorithm machine learning descriptive statistic XGBoost gradient boosting

Prediction of hotel booking cancellations: Integration of machine learning and probability model based on interpretable feature interaction

2023 | Decision Support Systems | Citations: 0

Authors: Chen, Shuixia; Ngai, Eric W.T.; Ku, Yaoyao; Xu, Zeshui; Gou, Xunjie; Zhang, Chenxi

Abstract: Reliable hotel cancellation prediction can help establish appropriate operationa ... Expand

Abstract: Reliable hotel cancellation prediction can help establish appropriate operational strategies for hotel manage­ ment. In this sector, personal name records (PNR) data may be the most representative information source for prediction tasks. Despite the popularity of PNR, its inherent lack of availability has been commonly disregarded in the literature. Existing studies have directly input PNR into high-dimensional machine learning (ML) models to achieve cancellation predictions. Another type of model generates cancellation prediction based on the proba­ bility modeling of samples. In this study, we propose an interpretable feature interaction method to enrich the existing PNR information. Thereafter, we empirically assess the prediction performance of the two model classes. This study specifically determines whether or not the two methods can cross-fertilize each other to improve cancellation prediction. To do so, we propose a model integrating Bayesian networks (BNs) and Lasso regression for this prediction task. This study utilizes BNs for the probability model consistent with our correlated variables and dichotomous prediction setting. Moreover, we use a linear ML model (i.e., Lasso regression), given its ad­ vantages in reducing ineffective predictors and transparency for ranking feature importance. Empirical results show that the proposed integration model has better prediction performance, and the obtained BN estimators and interactive features are the most important predictors. This study contributes to the booking cancellation liter­ ature by proposing an interpretable feature interaction and a prediction method integrating two types of effective models. The obtained accurate and interpretable cancellation prediction further contributes to offering practical implications to hoteliers in managerial decision-making. Collapse

Topics: missing data decision making competitive advantage price management marketing management

Methods: lasso regression experiment computational algorithm robustness check parametric test

Promise not fulfilled: FinTech, data privacy, and the GDPR

2023 | Electronic Markets | Citations: 0

Authors: Dorfleitner, Gregor; Hornuf, Lars; Kreppmeier, Julia

Abstract: This article analyzes how the General Data Protection Regulation (GDPR) has affe ... Expand

Abstract: This article analyzes how the General Data Protection Regulation (GDPR) has affected the privacy practices of FinTech firms. We study the content of 276 privacy statements respectively before and after the GDPR became binding. Using text analysis methods, we find that the readability of the privacy statements has decreased. The texts of privacy statements have become longer and use more standardized language, resulting in worse user comprehension. This calls into question whether the GDPR has achieved its original goal—the protection of natural persons regarding the transparent processing of personal data. We also link the content of the privacy statements to FinTech-specific determinants. Before the GDPR became binding, more external investors and a higher legal capital were related to a higher quantity of data processed and more transparency, but not thereafter. Finally, we document mimicking behavior among FinTech industry peers with regard to the data processed and transparency. Collapse

Topics: legal context General Data Protection Regulation privacy electronic finance website

Methods: parametric test paired t-test descriptive statistic Student's t-test qualitative content analysis

Theories: privacy calculus theory

Incorporating Multiple Knowledge Sources for Targeted Aspect-based Financial Sentiment Analysis

2023 | ACM Transactions on Management Information Systems | Citations: 0

Authors: Du, Kelvin; Xing, Frank; Cambria, Erik

Abstract: Combining symbolic and subsymbolic methods has become a promising strategy as re ... Expand

Abstract: Combining symbolic and subsymbolic methods has become a promising strategy as research tasks in AI grow increasingly complicated and require higher levels of understanding. Targeted Aspect-based Financial Sentiment Analysis (TABFSA) is an example of such complicated tasks, as it involves processes like information extraction, information specification, and domain adaptation. However, little is known about the design principles of such hybrid models leveraging external lexical knowledge. To fill this gap, we define anterior, parallel, and posterior knowledge integration and propose incorporating multiple lexical knowledge sources strategically into the fine-tuning process of pre-trained transformer models for TABFSA. Experiments on the Financial Opinion mining and Question Answering challenge (FiQA) Task 1 and SemEval 2017 Task 5 datasets show that the knowledge-enabled models systematically improve upon their plain deep learning counterparts, and some outperform state-of-the-art results reported in terms of aspect sentiment analysis error. We discover that parallel knowledge integration is the most effective and domain-specific lexical knowledge is more important according to our ablation analysis. Collapse

Topics: microblog domain-specific language Twitter social media knowledge graph

Methods: sentiment analysis RoBERTa transformer model BERT XLNet

Promoting Learning Through Explainable Artificial Intelligence: An Experimental Study in Radiology

2023 | International Conference on Information Systems | Citations: 0

Authors: Ellenrieder, Sara; Kallina, Emma; Pumplun, Luisa; Gawlitza, Joshua Felix; Ziegelmayer, Sebastian; Buxmann, Peter

Abstract: The deployment of machine learning (ML)-based decision support systems (DSSs) i ... Expand

Abstract: The deployment of machine learning (ML)-based decision support systems (DSSs) in high-risk environments such as radiology is increasing. Despite having achieved high decision accuracy, they are prone to errors. Thus, they are primarily used to assist radiologists in their decision making. However, collaborative decision making poses risks to the decision maker, e.g. automation bias and long-term performance degradation. To address these issues, we propose combining findings of the research streams of explainable artificial intelligence and education to promote human learning through interaction with ML-based DSSs. We provided radiologists with explainable vs nonexplainable decision support that was high-vs low-performing in a between-subject experimental study to support manual segmentation of 690 brain tumor scans. Our results show that explainable ML-based DSSs improved human learning outcomes and prevented false learning triggered by incorrect decision support. In fact, radiologists were able to learn from errors made by the low-performing explainable ML-based DSS. Collapse

Topics: decision support system decision support explainable artificial intelligence decision making artificial intelligence

Methods: experiment qualitative interview magnetic resonance imaging parametric test ANOVA

Live streaming recommendations based on dynamic representation learning

2023 | Decision Support Systems | Citations: 0

Authors: Gao, Ge; Liu, Hongyan; Zhao, Kang

Abstract: As an emerging form of social media, live streaming services (e.g., Twitch and C ... Expand

Abstract: As an emerging form of social media, live streaming services (e.g., Twitch and Clubhouse) allow users to interact with hosts and peers in real time while enjoying shows or participating in discussions. These platforms are also dynamic, with shows or discussions changing quickly inside a room and users frequently switching between rooms. To improve user engagement and experience on such platforms, we design a new recommendation model named Dynamic Representations for Live Streaming Rooms (DRIVER) to provide room recommendations. Guided by the Integrated Framework for Consumer Path Modeling and the social affordance theory, DRIVER infers dynamic representations of live streaming rooms by leveraging users’ behavior paths in entering, staying in, and leaving rooms. One contribution of our model is a new and efficient dynamic learning framework to model instantaneous and ever-changing inter-room relationships by considering individual users’ behavior paths after leaving a room. Also supported by social affordance theory, another methodological novelty of our model is to capture dynamic characteristics of a room by incorporating features of the current audience inside the room. Experiments on real-world datasets from two different types of live streaming platforms demonstrate that DRIVER outperforms state-of-the-art representation learning methods and sequential recommender systems. The proposed method also has implications for recommender system design in other contexts, in which items are characterized by users’ dynamic behavior paths and ongoing social interactions. Collapse

Topics: live streaming platform recommender system user experience knowledge sharing systems design

Methods: experiment sensitivity analysis ablation analysis descriptive statistic deep learning

Theories: theory of affordance