Benchmarking the Robustness of Phishing Email Detection Systems

2023 | Americas Conference on Information Systems | Citations: 0

Authors: Ampel, Benjamin; Gao, Yang; Hu, James; Samtani, Sagar; Chen, Hsinchun

Abstract: Social engineering attacks are currently the most cited cybersecurity threat to ... Expand

Abstract: Social engineering attacks are currently the most cited cybersecurity threat to organizations. Phishing emails are the most salient form of social engineering attacks. Organizations are increasingly implementing AI-enabled systems to detect phishing emails. However, AI-enabled systems are often susceptible to textual perturbations, where an adversary makes a small change to cause a misclassification. In this study, we sought to identify the performance of prevailing phishing email detection systems (PEDS) against character, word, sentence, and multi-level adversarial text perturbations. Through a principled benchmarking framework, we quantitatively demonstrated the lack of robustness prevailing PEDS have to specific types of text-based adversarial perturbations (e.g., character, word, sentence, multi-level). The results of this study provide new insights into the robustness of AI-based PEDS and highlight the need for organizations to adopt a multi-layered approach to phishing protection. Additionally, organizations can implement our benchmark framework to test their PEDS against adversarial perturbations. Collapse

Topics: artificial intelligence electronic mail email phishing phishing IT security

Methods: word2vec random forest classification naïve bayes literature study pre-trained language model

Augmenting Social Bot Detection with Crowd-Generated Labels

2023 | Information Systems Research | Citations: 0

Authors: Benjamin, Victor; Raghu, T. S.

Abstract: Social media platforms are facing increasing numbers of cyber-adversaries seekin ... Expand

Abstract: Social media platforms are facing increasing numbers of cyber-adversaries seeking to manipulate online discourse by using social bots (i.e., social media software robots) to help automate and scale their attacks. Likewise, some social media users can identify social bot activity at varying degrees of confidence. In this research, human reactions to social bot messages are used to augment existing social bot detection capabilities. Speech act theory is used to inspire a framework for assessing the credibility of instances where users identify potential bot activity, as not all user responses are of equal credibility for assisting with the bot detection task. The framework is then operationalized through deep learning methodologies to develop a computational system for identifying social bots. Real-world performance and practicality of the developed framework is demonstrated on a live, crowd-sourced data set collected from a real-world social media platform. Results show that consideration of crowd reactions to suspected bots can significantly improve bot detection performance. Furthermore, consideration of speech acts to evaluate crowd reactions can even further augment the system’s performance, although speech acts themselves are not necessary to observe performance boost through crowd intelligence. This study serves as a grounding point for future work that can explore an augmented model for detecting other forms of algorithmically generated content within social media platforms.History: Olivia Sheng, Senior Editor; Jesse Bockstedt, Associate Editor.Supplemental Material: The online appendix is available at https://doi.org/10.1287/isre.2022.1136. Collapse

Topics: Reddit social media crowdsourcing IT productivity social media analytics

Methods: experiment BERT machine learning support vector machine random forest classification

Theories: speech act theory

Deploying Artificial Intelligence to Combat Covid-19 Misinformation on Social Media: Technological and Ethical Considerations

2023 | HICSS | Citations: 0

Authors: Cartwright, B.; Frank, R.; Weir, G.; Padda, K.; Strange, S.

Abstract: This paper reports on AI research into online misinformation pertaining to the ... Expand

Abstract: This paper reports on AI research into online misinformation pertaining to the COVID-19 pandemic within the Canadian context. This is part of our longerterm goal, i.e., development of a machine-learning tool to assist social media platforms, online service providers and government agencies in identifying and responding to misinformation on social media. We report on predictive accuracies accomplished by applying a combination of technologies, including a custom-designed web-crawler, The Dark Crawler, the Posit toolkit, and four different machine-learning models based on Naïve Bayes, Support Vector Machines, LibLinear and LibShortText. Overall, we found that Posit and LibShortText models showed higher levels of correlation to the pre-determined (manual and machine-driven) data classifications than the other machine-learning algorithms tested. We further argue that the harms associated with COVID-19 misinformatione.g., the social and economic damage, and the deaths and severe illnessesoutweigh the right to personal privacy and freedom of speech considerations. Collapse

Topics: misinformation Twitter Reddit social media pandemic

Methods: machine learning computational algorithm support vector machine decision tree classification random forest classification

Understanding Emojis for Financial Sentiment Analysis

2023 | International Conference on Information Systems | Citations: 0

Authors: Chen, Siyi; Xing, Frank

Abstract: ... Expand

Abstract: Collapse

Topics: application programming interface social media website cryptocurrency Python

Methods: sentiment analysis machine learning BERT deep learning experiment

A Theory-based Deep-Learning Approach to Detecting Disinformation in Financial Social Media

2023 | Information Systems Frontiers | Citations: 0

Authors: Chung, Wingyan; Zhang, Yinqiang; Pan, Jia

Abstract: The spreading of disinformation in social media threatens cybersecurity and unde ... Expand

Abstract: The spreading of disinformation in social media threatens cybersecurity and undermines market efficiency. Detecting disinformation is challenging due to large volumes of social media content and a rapidly changing environment. This research developed and validated a theory-based, novel deep-learning approach (called TRNN) to disinformation detection. Grounded in social and psychological theories, TRNN uses deep-learning and data-centric augmentation to enhance disinformation detection in financial social media. Temporal and contextual information is encoded as specific knowledge about human-validated disinformation, which was identified from our unique collection of 745,139 financial social media messages about four U.S. high-tech company stocks and their fine-grained trading data. TRNN uses multiple series of long short-term memory (LSTM) recurrent neurons to learn dynamic and hidden patterns to support disinformation detection. Our experimental findings show that TRNN significantly outperformed widely-used machine learning techniques in terms of precision, recall, F-score and accuracy, achieving consistently better classification performance in disinformation detection. A case study of Apple Inc.’s stock price movement demonstrates the potential usability of TRNN for secure knowledge management. The research contributes to developing novel approach and model, producing new information systems artifacts and dataset, and providing empirical findings of detecting online disinformation. Collapse

Topics: disinformation social media Twitter fake news usability

Methods: machine learning experiment design artifact deep learning recurrent neural network

Theories: psychological theory

Dissecting AI-Generated Fake Reviews: Detection and Analysis of GPT-Based Restaurant Reviews on Social Media

2023 | International Conference on Information Systems | Citations: 0

Authors: Gambetti, Alessandro; Han, Qiwei

Abstract: Recent advances in generative models such as GPT may be used to fabricate indis ... Expand

Abstract: Recent advances in generative models such as GPT may be used to fabricate indistinguishable fake customer reviews at a much lower cost, posing challenges for social media platforms to detect this kind of content. This study addresses two research questions: (1) the effective detection of AI-generated restaurant reviews generated from high-quality elite authentic reviews, and (2) the comparison of out-of-sample predicted AI-generated reviews and authentic reviews across multiple dimensions of review, user, restaurant, and content characteristics. We fine-tuned a GPT text detector to predict fake reviews, significantly outperforming existing solutions. We applied the model to predict non-elite reviews that already passed the Yelp filtering system, revealing that AI-generated reviews typically score higher ratings, users posting such content have less established Yelp reputations and AI-generated reviews are more comprehensible and less linguistically complex than human-generated reviews. Notably, machine-generated reviews are more prevalent in low-traffic restaurants in terms of customer visits. Collapse

Topics: fake review artificial intelligence application programming interface open source pandemic

Methods: machine learning survey computational algorithm sensitivity analysis ANOVA

How Best to Hunt a Mammoth - Toward Automated Knowledge Extraction From Graphical Research Models

2023 | International Conference on Business Informatics | Citations: 0

Authors: Huettemann, Sebastian; Mueller, Roland M.; Larsen, Kai R.; Dinter, Barbara; Chiny, Joshua Campos

Abstract: ... Expand

Abstract: Collapse

Topics: knowledge repository database system knowledge base application programming interface Python

Methods: design science metamodel computational algorithm machine learning literature study

Automated Knowledge Extraction from IS Research Articles Combining Sentence Classification and Ontological Annotation

2023 | International Conference on Business Informatics | Citations: 0

Authors: Huettemann, Sebastian

Abstract: Manually analyzing large collections of research articles is a time-and resourc ... Expand

Abstract: Manually analyzing large collections of research articles is a time-and resource-intensive activity, making it difficult to stay on top of the latest research findings. Limitations of automated solutions lie in limited domain knowledge and not being able to attribute extracted key terms to a focal article, related work, or background information. We aim to address this challenge by (1) developing a framework for classifying sentences in scientific publications, (2) performing several experiments comparing state-of-the-art sentence transformer algorithms with a novel few-shot learning technique and (3) automatically analyzing a corpus of articles and evaluating automated knowledge extraction capabilities. We tested our approach for combining sentence classification with ontological annotations on a manually created dataset of 1,000 sentences from Information Systems (IS) articles. The results indicate a high degree of accuracy underlining the potential for novel approaches in analyzing scientific publications. Collapse

Topics: knowledge representation open access search engine database system open source

Methods: machine learning ontological modelling experiment transformer model domain ontology

Vulnerability-based Cyber-Risk Management: A Text-mining Approach

2023 | Americas Conference on Information Systems | Citations: 0

Authors: Jain, Swati; Mukhopadhyay, Arunabha

Abstract: Vulnerable information systems inside an organization make it prone to cyber-at ... Expand

Abstract: Vulnerable information systems inside an organization make it prone to cyber-attacks, leading to loss of reputation, financial loss, customer churn, and loss of future prospects. In our study, we assess, quantify, and mitigate the cyber-attack risk generated due to the vulnerable information technology assets using our proposed Vulnerability-based Cyber-Risk Management Model (VCRMM). We leverage Protection Motivation Theory and cyber-kill chain to assess the cyber risk based on specific characteristics of vulnerabilities. We perform text mining using the topic modelling technique, Latent Dirichlet Allocation, find a correlation between the topics, and then classify the severity rating of vulnerabilities. The higher the severity rating of any vulnerability, the greater the probability of cyber-attack (p) any organization faces. Next, we quantify the cyber-attack risk in terms of expected losses. Finally, based on Rational Choice Theory and NIST-guided Vulnerability Management Process, we propose mitigation strategies to reduce, accept, or transfer the cyber-attack risk. Collapse

Topics: IT security threat risk management IT security Microsoft Windows Chief Information Officer

Methods: topic model decision tree classification naïve bayes machine learning experiment

Theories: rational choice theory protection motivation theory

Detecting Feature Requests of Third-Party Developers through Machine Learning: A Case Study of the SAP Community

2023 | HICSS | Citations: 0

Authors: Kauschinger, M.; Vieth, N.; Schreieck, M.; Krcmar, H.

Abstract: The elicitation of requirements is central for the development of successful so ... Expand

Abstract: The elicitation of requirements is central for the development of successful software products. While traditional requirement elicitation techniques such as user interviews are highly labor-intensive, data-driven elicitation techniques promise enhanced scalability through the exploitation of new data sources like app store reviews or social media posts. For enterprise software vendors, requirements elicitation remains challenging because app store reviews are scarce and vendors have no direct access to users. Against this background, we investigate whether enterprise software vendors can elicit requirements from their sponsored developer communities through data-driven techniques. Following the design science methodology, we collected data from the SAP Community and developed a supervised machine learning classifier, which automatically detects feature requests of third-party developers. Based on a manually labeled data set of 1,500 questions, our classifier reached a high accuracy of 0.819. Our findings reveal that supervised machine learning models are an effective means for the identification of feature requests. Collapse

Topics: enterprise information system requirements elicitation requirements analysis Fiverr cloud computing

Methods: machine learning supervised learning design science naïve bayes classification method