Towards Designing a NLU Model Improvement System for Customer Service Chatbots

2023 | International Conference on Business Informatics | Citations: 0

Authors: Schloß, Daniel; Gnewuch, Ulrich; Maedche, Alexander

Abstract: Current customer service chatbots often struggle to meet customer expectations. ... Expand

Abstract: Current customer service chatbots often struggle to meet customer expectations. One reason is that despite advances in artificial intelligence (AI), the natural language understanding (NLU) capabilities of chatbots are often far from perfect. In order to improve them, chatbot managers need to make informed decisions and continuously adapt the chatbot's NLU model to the specific topics and expressions used by customers. Customer-chatbot interaction data is an excellent source of information for these adjustments because customer messages contain specific topics and linguistic expressions representing the domain of the customer service chatbot. However, extracting insights from such data to improve the chatbot's NLU, its architecture, and ultimately the conversational experience requires appropriate systems and methods, which are currently lacking. Therefore, we conduct a design science research project to develop a novel artifact based on chatbot interaction data that supports NLU improvement. Collapse

Topics: customer service knowledge base

Methods: chatbot design science usage data case study language model

Theories: communication theory

What Do Citizens Think of AI Adoption in Public Services? Exploratory Research on Citizen Attitudes through a Social Contract Lens

2023 | HICSS | Citations: 0

Authors: Schmager, Stefan; Grøder, Charlotte Husom; Parmiggiani, Elena; Pappas, Ilias; Vassilakopoulou, Polyxeni

Abstract: The adoption of Artificial Intelligence (AI) by the public sector has the potent ... Expand

Abstract: The adoption of Artificial Intelligence (AI) by the public sector has the potential to improve service delivery. However, the risks related to AI are significant and citizen concerns have halted several AI initiatives. In this paper we report findings from an empirical study on citizens´ attitudes towards AI use in public services in Norway. We found a generally positive attitude and identified three factors contributing to this: a) the high level of trust in government; b) the reassurance provided by having humans in the loop; c) the perceived transparency into processes, data used for AI models and models´ inner workings. We interpret these findings through the lens of social contract theory and show how the introduction of AI in public services is subject to the social contract power dynamics. Our study contributes to research by foregrounding the government-citizen relationship and has implications for public sector AI practice. Collapse

Topics: artificial intelligence information system use user interface decision making usability

Methods: design artifact survey personal interview design science cognitive walkthrough

Theories: contract theory

Understanding Information-Limiting Environments in Personalized News Platforms—A Systems Perspective

2023 | International Conference on Information Systems | Citations: 0

Authors: Schmalenbach, Kian; Brechtelsbauer, Bastian; Schötteler, Sebastian; Laumer, Sven

Abstract: ... Expand

Abstract: Collapse

Topics: user attitude collaborative filtering digital platform Facebook Twitter

Methods: experiment conceptual modelling design artifact computational algorithm theory development

Unraveling Information-Limiting Environments: An Empirical Review of Individual, Social, and Technological Filters in Social Media

2023 | International Conference on Business Informatics | Citations: 0

Authors: Schmalenbach, Kian; Gengler, Eva; Schötteler, Sebastian; Laumer, Sven

Abstract: Social media platforms offer a convenient way for people to interact and exchan ... Expand

Abstract: Social media platforms offer a convenient way for people to interact and exchange information. However, there are sustained concerns that filter bubbles and echo chambers create information-limiting environments (ILEs) for their users. Despite a well-developed conceptual understanding, the empirical evidence regarding the causes and supporting conditions of these ILEs remains inconclusive. This paper addresses this gap by applying the triple-filter-bubble model developed by Geschke et al. (2019) to analyze empirical literature on the individual, social, and technological causes of ILEs. While we identify some factors that increase the probability of ILEs under certain conditions, our findings do not suffice to thoroughly validate conceptual models that explain why ILEs emerge. Therefore, we call for future research to investigate the causes of ILEs with higher external validity to develop a more comprehensive understanding of this phenomenon. Collapse

Topics: social media filter bubble database system Twitter TikTok

Methods: literature sample literature study conceptual modelling structured literature research focus group

Theories: big five model

Building Trust in AI: A new Understanding of the Role of Reliability

2023 | Americas Conference on Information Systems | Citations: 0

Authors: Schmid, Amelie; Wiesche, Manuel

Abstract: It is well known that trust represents a crucial success factor for the adoptio ... Expand

Abstract: It is well known that trust represents a crucial success factor for the adoption of a technology. Among others, the reliability of the technology has a direct impact on the process of building trust. In the present paper, it is claimed that there is a contradiction related to the impact of reliability on building trust between former technologies and AI with its specific characteristics. Moreover, the static relationship between the process of building trust in AI and the corresponding measures, as reported in the literature, require a new approach. To close this gap, an analysis of an AI implementation and operation within an international automotive supplier was conducted. Based on 17 interviews -as main outcome -workers accept a lower level of reliability without a negative impact on the process of building trust. In order to explain and understand this finding the importance of organizational measures for trust calibration are discussed. Collapse

Topics: artificial intelligence research and development explainable artificial intelligence weak AI

Methods: qualitative interview qualitative coding personal interview grounded theory business process modeling

Theories: failure mode and effects analysis

How Engineers perform Data Science Work: Designing Hybrid Roles

2023 | International Conference on Business Informatics | Citations: 0

Authors: Schmid, Amelie Lena; Wiesche, Manuel

Abstract: As of today, organizations are still struggling to derive consistent value from ... Expand

Abstract: As of today, organizations are still struggling to derive consistent value from data science projects. The basic relevance of domain knowledge for data science work can be considered as common sense. Engineers, in particular, offer a unique view on emerging data science work, based on their critical role within traditional industries. As a constraint, current studies on data science work consider domain experts as rather passive and engineering-related studies are rare. To further explore these challenges, the present study analyses data science work of 30 engineers at an international automotive supplier. Investigating three cases, the evolvement of hybrid data science work can be derived, which combines two perspectives: engineering and data science. Thus, engineers actively incorporate the data science perspective, particularly when development activities involve minimal participation of data scientists. This contribution significantly enhances existing knowledge by demonstrating how engineers embrace the data scientists' perspective and perform hybrid data work. Collapse

Topics: decision making

Methods: qualitative interview qualitative coding personal interview data modeling grounded theory

ACCOUNTABILITY INCONGRUENCE AND ITS EFFECTS ON AI DEVELOPERS’ JOB SATISFACTION

2023 | European Conference On Information Systems | Citations: 0

Authors: Schmidt, Jan-Hendrik; Bartsch, Sebastian; Adam, Martin; Benlian, Alexander

Abstract: Developers of Artificial Intelligence (AI)-based systems are increasingly urged ... Expand

Abstract: Developers of Artificial Intelligence (AI)-based systems are increasingly urged to assume accountability for their development decisions, referring to the degree to which they must justify underlying algorithms and their outcomes on demand. Thereby, AI developers often have to juxtapose how much accountability they self-attribute to them and how much accountability they perceive others attribute to them, creating intrapersonal perceptual accountability (in)congruence with unknown consequences for their job satisfaction. Building on perceptual congruence research and algorithmic accountability literature, we conducted an online survey of 87 AI developers about their experiences in AI-based systems development projects. Our results show that the lower the incongruence between self-attributed and others-attributed accountability, the higher the job satisfaction of AI developers. Moreover, we find that AI developers' role ambiguity mediates this effect. Our study contributes to a more nuanced understanding of AI developers' perceived accountability, with essential insights for defining job roles and understanding AI developers. Collapse

Topics: artificial intelligence job satisfaction IT development project algorithmic transparency IT manager

Methods: survey linear regression analysis nonparametric test Wilcoxon signed-rank test statistical power analysis

The Role of Process and Outcome Accountability Claims for Shaping AI Developers’ Perceived Accountability

2023 | International Conference on Information Systems | Citations: 0

Authors: Schmidt, Jan-Hendrik; Bartsch, Sebastian; Adam, Martin

Abstract: As accountability becomes increasingly important for developers of artificial i ... Expand

Abstract: As accountability becomes increasingly important for developers of artificial intelligence (AI)-based systems, governance mechanisms such as AI principles or audits are often criticized for not sufficiently influencing AI developers. Therefore, we examine how visualized arguments in user interfaces (UIs) of integrated development environments (IDEs) can increase AI developers' perceived accountability. Combining construal level theory and Toulmin's model of argumentation, four UI design artifacts were developed, each containing a claim of process or outcome accountability with or without monitoring and evaluation tools that act as claim-supporting data. Results of an online experiment with 164 AI developers show that claiming process accountability increases AI developers' perceived accountability more than claiming outcome accountability, both without supporting data. However, when supporting data are available, both claims increase AI developers' perceived accountability comparably effectively. The study's results highlight the theoretical and practical usefulness of visualized arguments in UIs of IDEs to promote AI developers' accountability. Collapse

Topics: artificial intelligence individual characteristic data quality crowdsourcing job satisfaction

Methods: design artifact experimental group experiment qualitative interview survey design

Theories: argumentation theory construal level theory

Introduction to the Minitrack on Enterprise Ecosystem: Extending and Integrating Technology Serving the Enterprise

2023 | HICSS | Citations: 0

Authors: Schmidt, Pamela; Freeze, Ronald; Narasimhan, Sathya

Abstract: ... Expand

Abstract: Collapse

Topics: enterprise resource planning cloud computing organizational value business process management enterprise information system

Methods: modeling language business process model and notation experiment design artifact experimental group

Introduction to the Minitrack on Artificial Intelligence-based Assistants

2023 | HICSS | Citations: 0

Authors: Schmidt, Rainer; Alt, Rainer; Zimmermann, Alfed

Abstract: ... Expand

Abstract: Collapse

Topics: artificial intelligence explainable artificial intelligence

Methods: machine learning chatbot design science