Designing a Conversation Mining System for Customer Service Chatbots

2023 | European Conference On Information Systems | Citations: 0

Authors: Schloß, Daniel; Espitia, Juan; Gnewuch, Ulrich

Abstract: As chatbots are gaining popularity in customer service, it is critically import ... Expand

Abstract: As chatbots are gaining popularity in customer service, it is critically important for companies to continuously analyze and improve their chatbots' performance. However, current analysis approaches are often limited to the question-answer level or produce highly aggregated metrics (e.g., conversations per day) instead of leveraging the full potential of the large volume of conversation data to provide actionable insights for chatbot developers and chatbot managers. To address this challenge, we developed a novel chatbot analytics approachconversation miningbased on concepts and methods from process mining. We instantiated our approach in a conversation mining system that can be used to visually analyze customer-chatbot conversations at the process level. The results of four focus group evaluations suggest that conversation mining can help chatbot developers and chatbot managers to extract useful insights for improving customer service chatbots. Our research contributes to research and practice with novel design knowledge for conversation mining systems. Collapse

Topics: process mining customer service SWOT analysis data quality MySQL

Methods: chatbot design artifact design science event log focus group

Context Modeling for the Adaption of Mobile Business Processes – An Empirical Usability Evaluation

2022 | Information Systems Frontiers | Citations: 0

Authors: Dörndorfer, Julian; Seel, Christian

Abstract: Nowadays, the Internet of Things (IoT) and mobile devices are ubiquitous. Both, ... Expand

Abstract: Nowadays, the Internet of Things (IoT) and mobile devices are ubiquitous. Both, the IoT and mobile devices contain sensors and thus can provide data about the device’s environment. The sensor data can be used to infer the current context. However, for this purpose, the sensor data have to be aggregated. In this aggregation process, several different sensors and data provided by other sources, such as databases, can be used. In order to facilitate this, the paper presents a modeling language for context modeling based on sensors. Moreover, a detailed usability evaluation of the presented context modeling language is shown. This evaluation is based on three hypotheses regarding learnability, time expenditure and effectiveness. An experiment involving an experimental group and a control group was conducted to test these three hypotheses, and the results were interpreted. Collapse

Topics: business process management systems development usability mobile application user satisfaction

Methods: modeling language experiment design science experimental group descriptive statistic

How Inconsistencies Between Multiple Conceptual Modeling Scripts Affect Readers’ Understanding

2022 | Americas Conference on Information Systems | Citations: 0

Authors: Jabbari, Mohammad; Werder, Karl

Abstract: IS professionals often use multiple conceptual modeling scripts to develop an u ... Expand

Abstract: IS professionals often use multiple conceptual modeling scripts to develop an understanding of a domain. However, using multiple scripts introduces potential inconsistencies between scrips which can reduce script readers' cognitive ability to develop an understanding. While there are computational methods to avoid or detect inconsistencies, there is a lack of studies on how individuals deal with inconsistencies when they are performing different tasks. We developed a 2x2 between-subject experimental design to investigate the effects of syntactic vs semantic inconsistency on two different systems analysis and design tasks. We expect to contribute to conceptual modeling research, by investigating the effect of inconsistencies, comparing the effects of two tasks, and by elaborating the on role of a pragmatic factor, domain familiarity. Collapse

Topics: task productivity systems development problem solving systems development life cycle accounting

Methods: design science class diagram activity diagram experimental design conceptual modelling

How Do Individuals Understand Multiple Conceptual Modeling Scripts?

2022 | Journal of the Association for Information Systems | Citations: 5

Authors: Jabbari, Mohammad; Recker, Jan; Green, Peter; Werder, Karl

Abstract: Because most real-world domains intended to be supported by an information syst ... Expand

Abstract: Because most real-world domains intended to be supported by an information system are complex, practitioners often use multiple different types of conceptual modeling scripts to understand them. We performed two experiments to examine how two theoretical factors of multiple scriptscombined ontological completeness and ontological overlap-influence how users develop an understanding of a real-world domain from multiple scripts. Results of the first experiment show that to some degree, ontological overlap improves participants' understanding of a domain, more so than combined ontological completeness. In the second experiment, we tracked the eye movement data of participants to understand how ontological overlap between scripts impacts users' information search and cognitive integration processes. We found that some occurrence of semantically similar constructs between scripts helps individuals to identify and relate constructs presented in different scripts. Users, therefore, can identify and focus on script areas that are relevant to their problem tasks. However, a high level of ontological overlap decreases the attention paid by participants to relevant task-specific areas because they spend more time searching for relevant information. Together, our findings both refine and extend existing conceptual modeling theory. We clarify the dialectics between the full and parsimonious real-world representations offered through multiple scripts and the individual's understanding of the domain that is represented by those scripts. Collapse

Topics: problem solving task productivity systems development IS education information storage and retrieval system

Methods: experiment design science conceptual modelling unified modeling language class diagram

Theories: theory of combined ontological completeness and overlap representation theory

Industry, Quality, and Social Issues (IQSI) Mini-track

2022 | HICSS | Citations: 0

Authors: Mead, Nancy R; Hochmüller, Elke

Abstract: This is an overview of the IQSI Mini-track within the special track on Software ... Expand

Abstract: This is an overview of the IQSI Mini-track within the special track on Software Engineering Education and Training. The IQSI mini-track includes 7 accepted papers on a diverse set of topics, including industry studies, cybersecurity, social issues including remote teaching during Covid and gender, and other topics related to quality. Collapse

Topics: systems development information and communication technology for development IS discipline

Methods: chatbot class diagram

Toward a Metric Catalog for Large-Scale Agile Development

2022 | Americas Conference on Information Systems | Citations: 0

Authors: Philipp, Pascal; Tobisch, Franziska; Menzel, Leon; Matthes, Florian

Abstract: Nowadays, organizations use agile software development to remain competitive in ... Expand

Abstract: Nowadays, organizations use agile software development to remain competitive in their frequently changing business environment. Inspired by the success of agile methods on a small scale, organizations have started to apply them in larger contexts. However, the limited scalability of agile methods is a problem. Metrics can be a success factor for achieving agility at scale, thus adopting them is promising. Most scaling agile frameworks provide few recommendations regarding metrics. Likewise, research on metrics in largescale agile development lacks concrete guidance for metrics or their organization-specific adoption. To fill this gap, we propose two artifacts. We present the design of a minimalistic metric management fact sheet (MMFS) for large-scale agile development to support practitioners in using metrics in their organizationspecific development environment. Furthermore, the MMFS is the basis for our metric catalog documenting 196 metrics identified in an expert study to provide a comprehensive metric set for scaling agile environments. KeywordsAgile software development, large-scale agile development, metrics. Collapse

Topics: agile software development IT project organization structure web application systems development

Methods: qualitative interview design methodology survey data modeling design science

Organizing Self-Organizing Systems: A Terminology, Taxonomy, and Reference Model for Entities in Cyber-Physical Production Systems

2021 | Information Systems Frontiers | Citations: 0

Authors: Berger, Stephan; Häckel, Björn; Häfner, Lukas

Abstract: Ongoing digitalization accelerates the transformation and integration of physica ... Expand

Abstract: Ongoing digitalization accelerates the transformation and integration of physical production and traditional computing systems into smart objects and their interconnectivity, forming the Internet of Things. In manufacturing, the cross-linking of embedded systems creates adaptive and self-organizing Cyber-Physical Production Systems (CPPSs). Owing to ever-increasing cross-linking, rapid technological advances, and multifunctionality, the complexity and structural opacity of CPPSs are rapidly increasing. The development of urgently needed modeling approaches for managing such complexity and structural opacity, however, is impeded by a lack of common understanding of CPPSs. Therefore, in this paper, we contribute to a common understanding of CPPSs by defining and classifying CPPS entities and illustrating their relations. More precisely, we present a terminology, a taxonomy, and a reference model for CPPS entities, created and evaluated using an iterative development process. Thereby, we lay the foundation for future CPPS modeling approaches that make CPPS complexity and structural opacity more manageable. Collapse

Topics: computer hardware manufacturing information system network infrastructure robotic enterprise architecture

Methods: qualitative interview focus group literature study group interview design artifact

Cloud computing ecosystem model: evaluation and role clusters

2021 | Electronic Markets | Citations: 2

Authors: Floerecke, Sebastian; Lehner, Franz; Schweikl, Sebastian

Abstract: Several significantly different attempts have been undertaken to visualize the c ... Expand

Abstract: Several significantly different attempts have been undertaken to visualize the complex and opaque cloud computing ecosystem by a descriptive model. Recently, scholars have developed a new role-based cloud ecosystem model (Passau Cloud Computing Ecosystem Model (PaCE model)) based on a systematic analysis and synthesis of the previous models. However, its scientific evaluation is still outstanding. The main goal is therefore to conduct an evaluation regarding its structural equivalence and completeness compared to the real cloud business according to the design science research paradigm. The second goal is to demonstrate the model’s usefulness by using it as an instrument for the identification of role clusters, which are frequently covered by organizations. To this end, a cross-sectional analysis is conducted, in which 758 organizations are manually analysed and matched with the model’s current roles. As a result, roles have been newly added, removed, broken down into sub-roles, abstracted to a meta-role and renamed. The cluster analysis unveiled five role clusters. Whereas some roles were predominantly found as single role, further roles were exclusively or very frequently found in combination with others. Overall, the revised model creates, compared to the common rather technically orientated market overviews by market research institutes, a detailed categorization schema helping to understand and investigate the organizations belonging to the cloud ecosystem. Collapse

Topics: cloud computing cloud computing provider computer hardware data center digital ecosystem

Methods: cluster analysis cross sectional research computational algorithm descriptive statistic design science

How do individuals decide which modeling scripts to use during systems analysis and design?

2021 | Decision Support Systems | Citations: 0

Authors: Jabbari, Mohammad; Recker, Jan; Green, Peter

Abstract: Because most information systems in real-world domains are complex, practitioner ... Expand

Abstract: Because most information systems in real-world domains are complex, practitioners often use different types of conceptual modeling scripts to understand them. Modeling methods such as UML, for example, provide more than a dozen scripts for practitioners to use. We study how script readers decide which of several different scripts to use during systems analysis and design tasks. We carried out a free-simulation experiment to test how users select scripts based on two factors: combined ontological completeness and ontological overlap. We find that participants indeed decided to select more than one script to achieve a more complete domain representation. But, when they selected more than three scripts, they decided to remove scripts and reduce combined completeness to increase the clarity of the combined representation. Our results indicate that script readers prefer relatively less complete scripts with high levels of clarity over more complete, but more overlapping, script combinations. We detail the implications these findings have for the theory and the practice of conceptual modeling. Collapse

Topics: problem solving systems development logistics management decision making use case modeling

Methods: design science experiment unified modeling language conceptual modelling descriptive statistic

Theories: theory of combined ontological completeness and overlap representation theory

Quarry: A User-centered Big Data Integration Platform

2021 | Information Systems Frontiers | Citations: 0

Authors: Jovanovic, Petar; Nadal, Sergi; Romero, Oscar; Abelló, Alberto; Bilalli, Besim

Abstract: Obtaining valuable insights and actionable knowledge from data requires cross-an ... Expand

Abstract: Obtaining valuable insights and actionable knowledge from data requires cross-analysis of domain data typically coming from various sources. Doing so, inevitably imposes burdensome processes of unifying different data formats, discovering integration paths, and all this given specific analytical needs of a data analyst. Along with large volumes of data, the variety of formats, data models, and semantics drastically contribute to the complexity of such processes. Although there have been many attempts to automate various processes along the Big Data pipeline, no unified platforms accessible by users without technical skills (like statisticians or business analysts) have been proposed. In this paper, we present a Big Data integration platform (Quarry) that uses hypergraph-based metadata to facilitate (and largely automate) the integration of domain data coming from a variety of sources, and provides an intuitive interface to assist end users both in: (1) data exploration with the goal of discovering potentially relevant analysis facets, and (2) consolidation and deployment of data flows which integrate the data, and prepare them for further analysis (descriptive or predictive), visualization, and/or publishing. We validate Quarry’s functionalities with the use case of World Health Organization (WHO) epidemiologists and data analysts in their fight against Neglected Tropical Diseases (NTDs). Collapse

Topics: data integration data quality big data data management usability

Methods: computational algorithm design science meta learning literature study class diagram