Too Good to Be True? An Empirical Study of ChatGPT Capabilities for Academic Writing and Implications for Academic Misconduct

2023 | Americas Conference on Information Systems | Citations: 0

Authors: Busch, Peter André; Hausvik, Geir Inge

Abstract: This study evaluated the capabilities of ChatGPT, a conversational artificial i ... Expand

Abstract: This study evaluated the capabilities of ChatGPT, a conversational artificial intelligence (AI) agent, to generate quality answers to exam questions that are indistinguishable from human-generated text, normally requiring high-level cognitive abilities. This potential capacity is a major threat to academic integrity since ChatGPT may be used for academic misconduct. To determine its capabilities, we asked 15 faculty members to assess a total of ten answers (two each) to an actual IS exam question, of which students wrote five, and ChatGPT generated five. The study found that ChatGPT can generate answers of generally good quality that may pass as human-written text by examiners. The contributions of the study lay in providing empirical results in a novel research area, providing recommendations for teachers, and suggesting ways forward for research. Collapse

Topics: artificial intelligence smartphone evaluation criteria smartphone addiction mobile system

Methods: chatGPT machine learning descriptive statistic correlation analysis personal interview

Value-driven IT Project Portfolio Management: Process Model, Evaluation Framework, and Decision Support

2022 | International Conference on Information Systems | Citations: 0

Authors: Karrenbauer, Christin; Breitner, Michael H.

Abstract: Companies must optimize their information technology (IT) project portfolio to ... Expand

Abstract: Companies must optimize their information technology (IT) project portfolio to achieve goals. However, IT projects often exceed resources and do not create their promised value, for example, because of missing structured processes and evaluation methods. Continuous IT portfolio management is thus of importance and a critical business activity to reach value-driven goals. Guided by Design Science Research with literature reviews and expert interviews, we develop, evaluate, and adjust an IT project portfolio management process model, a holistic IT project evaluation framework, and implement a decision support system prototype. Our results and findings synthesize and extend previous research and expert opinions and guide decision-makers to make more informed and objective IT project portfolio management decisions aligned with optimal value creation. Furthermore, we deduce new research opportunities for IT project portfolio management process models, decision tools, and evaluation frameworks. Introduction and MotivationInformation technology (IT) impacts a company's long-term performance and competitiveness and forms a critical success factor (Bezdrob et al. 2020). Thereby, IT projects are characterized by complexity, crossfunctionality, dynamics, non-routine, temporality, and uncertainty. These make IT project portfolio management (ITPPM) a challenging task (Chiang and Nunez 2013;Kester et al. 2011). Selecting the "right" IT projects is essential to create optimal value. Nevertheless, ITPPM is often unstructured and decisions are made ad hoc instead of long-term planning. Thus, many IT projects deviate from their defined objectives, are not completed, or completely fail (Varajão and Trigo 2016). According to the Project Management Institute (PMI 2017a), roughly $97 million US Dollars per $ 1 billion investments in IT projects are wasted. Similarly, Lee et al. (2021) refer that on average 66% of implemented IT projects are more expensive and 33% require longer as planned. Failed IT projects due to weak ITPPM processes lead to resource losses and exceedances (e.g., Hershey: $150 in lost sales, 19% drop in earnings), project abandonment (e.g., Dell: $200 million), or bankruptcies (e.g., FoxMeyer) (Fadlalla and Amani 2015; Hughes et al. 2017). Thus, companies need adequate and resilient methods for the critical business activity of ITPPM. Considering existing interdependencies and constraints, these methods ensure that selected IT projects fit the company's strategy and create value (Chiang and Nunez 2013;Kester et al. 2011). If departments and functions are aligned to strategy, IT projects are more likely to be completed successfully ITPPM: Process Model, Evaluation Framework, and DSS Forty-Third International Conference on Information Systems, Copenhagen 2022 2 (PMI 2017a). Additionally, uniform IT guidelines stabilize the performance of IT projects (Martin 2006). However, many decisions are motivated by subjectivity, personal experiences and perceptions. Often uniform ITPPM methods, evaluation and selection criteria, and their consistent usage are missing (Varajão and Trigo 2016). An ITPPM tool can further support decision-makers (DM) to ensure efficient, transparent, and consistent decisions more rapidly (Caniëls and Bakens 2012;Killen et al. 2020).Even though ITPPM is considered to be a structured process, besides best practices for PPM (e.g., PMI 2017b) only few ITPPM process models exist in literature which are rather perfunctory. Mostly, existing models encompass three (Ajjan et al. 2016) to five main phases (Archer and Ghasemzadeh 1999). Typically, they include activities such as the definition of baseline conditions (Alaeddini and Mir-Amini 2020), IT project identification (Montgomery 2007), evaluation (Chiang and Nunez 2013), selection (Ajjan et al. 2016), and monitoring with adaptions (Miller 2002). Other than that, existing models vary in their phases and activities. Moreover, the procedure is mostly sequential with no or few re-cycles between and within phases. To address this and better meet IT project characteristics, we propose an integrated ITPPM process model that allows an objective and value-driven ITPPM. To do this, we synthesize existing process models and expand them with expert interviews. It comprises activities from different models combined within an integrated one. Unlike many existing models, it also allows re-cycles between and within individual phases and allows to address peculiarities of IT projects. It enables, e.g., a flexible reaction to changes and uncertainties and a discussion between stakeholders. In addition, IT project evaluation, prioritization, and selection are critical activities (Varajão and Trigo 2016). Several methods have been researched on this topic ranging from rather simple financial project selections (Rosacker and Olson 2009), and balance scorecard approaches (Asosheh et al. 2010), to more advanced optimization models (Cho and Shaw 2013) and fuzzy programming (Heidary et al. 2020), among others (Mohagheghi et al. 2019). In rather simple methods, IT project decisions are often based only on the evaluations. In more advanced methods, several constraints are considered, however, the project evaluation is often already available and not described in more detail (Archer and Ghasemzadeh 1999). We combine both approaches and develop a holistic scoring framework that enables a quantification of subjective estimations and thus an objective evaluation of IT projects of different types and sizes. The scoring results then serve as an input for our decision support system (DSS) prototype. It allows to select from a large number of IT project proposals those that maximize the value proposition, considering interdependencies and limitations. In doing so, we mitigate disadvantages resulting from the sole use of scoring approaches (Mohagheghi et al. 2019) and decisions are not made on an individual project level but consider other factors such as resource constraints and dependencies. Our main contributions are our three artifacts, an integrated ITPPM process model, a holistic scoring framework, and our DSS prototype. These Level 1 and Level 2 artifacts (Gregor and Hevner 2013) support ITPPM and enhance and contribute to the knowledge base. We provide knowledge to increase transparency and objectivity to make more informed IT portfolio decisions and thus decrease IT project failure rates, minimize subject manipulations, increase value creation, and goal achievement. We follow a Design Science Research (DSR) process (Hevner et al. 2004) to address the following research questions (RQ):RQ 1: What activities constitute a value-driven ITPPM process? RQ 2: How can IT projects (and proposals) be uniformly evaluated to generate a value-driven IT portfolio? RQ 3: How can a DSS support IT project selection and scheduling?First, we describe the theoretical background of ITPPM. Then, we motivate our DSR-oriented research design. Afterwards, we present our results and findings of our literature reviews and expert interviews, including an ITPPM process model and a scoring framework. Then, we introduce our DSS prototype and provide an applicability check. Finally, we discuss our results and findings, their implications, deduce recommendations for theory and practice and a research agenda, present limitations, and conclusions. Collapse

Topics: IT project project management decision support system value creation knowledge base

Methods: qualitative interview business process modeling design artifact literature study design science

Information System Quality Judgment for Continued E-Government Use: Theorizing the Role of Positive and Negative Affect

2021 | Communications of the Association for Information Systems | Citations: 0

Authors: Srivastava, Shirish; Teo, Thompson

Abstract: Affect and emotions play an important role in how individuals form judgments. Y ... Expand

Abstract: Affect and emotions play an important role in how individuals form judgments. Yet, the literature on technological judgments has primarily relied on the cognitive belief perspective. By segregating emotions into positive and negative affect, we incorporate affect in addition to cognitions to understand what drives perceptions about IS quality and, specifically, e-government website quality. Grounding our discussion in the affect infusion model (AIM) and prospect theory, we examine the mechanisms through which positive and negative affect infuse into IS quality judgments. We also theorize that both positive and negative affect have a moderating role in the relationships between cognitions and IS quality perceptions. We tested the model via surveying e-government website users and found that affect had a significant direct role in how they judged IS quality. While negative affect significantly moderated the relationship between experienced usefulness and how individuals perceived the three IS quality measures (i.e., information quality, system quality, and service quality), positive affect did not moderate this relationship. Finally, we theorize about the differential role that affect has on how individuals perceive the three IS quality measures depending on their affect infusion potential. We conclude by discussing our study's theoretical and practical implications.AIM describes how affect-laden information infuses into judgments via individuals' cognitive and behavioral processes. This infusion plays an important role in individuals' thoughts and constructive deliberations in that it colors their judgments in an affect-congruent direction (Forgas, 1995). Individuals who experience positive affective states will generally make positive judgments, whereas individuals who experience negative affective states will make negative judgments. AIM, therefore, suggests that affective states will likely be associated with what individuals recall and, ultimately, the kind of judgments they make, which will generally concur with the valence of their affective state.The traditional view on quality judgment primarily takes a cognitive perspective. For example, Juran and Bingham (1974) define quality as fitness for use. Emanating from this concept, recent IS research has also conceptualized "usefulness" as a factor that predicts IS quality judgments (e.g., Hu, Brown, Thong, Chan, & Tam, 2009;Tate, 2010;Yang et al., 2011). In this paper, we move beyond conceptualizing quality in general and IS quality in particular from a cognitive perspective and use the AIM to incorporate "affect" into the nomological network that relates cognitions to IS quality evaluations in order to further explain the mechanisms through which IS quality perceptions manifest. To more clearly understand affect's role, we distinguish and aggregate emotions as positive and negative affect. We propose that, when individuals interact with an IS (especially in a continued usage context), they experience affect, which infuses into their quality judgments about the IS. We further propose that the negative affect experienced when using 391 Information System Quality Judgment for Continued E-Government Use: Theorizing the Role of Positive and Negative Affect Volume 49 10.17705/1CAIS.04916 Paper 16an IS interacts with individuals' cognitions and, thereby, moderates the relationship between cognitions and quality judgments when they subsequently use the IS.In particular, we ask two key research questions (RQ): RQ1:In the continued usage context, what relationship do positive and negative affect have with IS quality? RQ2:In the continued usage context, how do positive and negative affect moderate the relationship between cognitions (defined based on usefulness) and IS quality?With this study, we make several contributions. First, prior research has primarily conceptualized quality in general, including IS quality, from a cognitive perspective. In contrast, we conceptualize affect infusion as playing a key role in how individuals perceive IS quality. In particular, situating our arguments in the AIM, we theorize the mechanisms through which affect infuses into IS quality judgments and, thereby, contribute to the literature on IS quality. Second, we contribute to the IS literature on affect. Prior literature in this stream has tended to focus on a limited set of emotions (such as anxiety and enjoyment) generally in the IT adoption and usage context. Relatively few IS studies have examined a wider range of emotions and affect (e.g., Beaudry & Pinsonneault, 2010;Zhang, 2013) or the joint impact of cognitions and affect (Kim, Chan, & Chan, 2007;Park, Hill, & Bonds-Raacke, 2015). Using the AIM, we add to this research stream by aggregating emotions into positive and negative affect and examining the role that affect plays in IS quality judgments. In addition to examining the different mechanisms through which affect infuses into IS quality judgments, we theorize that positive and negative affect moderate the relationship between cognitions and quality judgments. Consistent with prospect theory (Kahneman, 2011;Barberis, 2013), which proposes that negative outcomes (e.g., losses) are stronger than positive outcomes (e.g., gains), we empirically show that negative affect (but not positive affect) has a moderating effect-a significant contribution because few prior studies have distinguished between the role of positive and negative affect (e.g., Cenfetelli, 2004). Third, we emphasize the need to examine cognition (usefulness) and affect as antecedents to the specific IS quality variables (i.e., information quality, system quality, and service quality) in order to understand the different ways that affect infuses into quality deliberations. We complement and augment the existing work on affect in IS research by examining the differences in affect infusion potentials among the three IS quality variables. Further, we also fill a gap in existing research since researchers have rarely combined cognition with affect as antecedents to IS quality dimensions. In fact, relatively few papers have examined antecedents to IS quality dimensions as Petter, DeLone, and McLean (2013) have highlighted. Collapse

Topics: system quality service quality government system data quality website

Methods: theory development survey descriptive statistic statistical hypothesis test survey design

Theories: prospect theory

The Potential of Technology-Mediated Learning Processes: A Taxonomy and Research Agenda for Educational Process Mining

2021 | International Conference on Information Systems | Citations: 0

Authors: Wambsganss, Thiemo; Schmitt, Anuschka; Mahnig, Thomas; Ott, Anja; Soellner, Sigitai; Ngo, Ngoc Anh; Geyer-Klingeberg, Jerome

Abstract: Educational process mining (EPM) offers new possibilities to discover, monitor, ... Expand

Abstract: Educational process mining (EPM) offers new possibilities to discover, monitor, improve, or predict students' learning processes using data about their learning activities captured in technology-mediated information systems (IS). Although EPM has recently attracted considerable research interest, there is still limited shared knowledge about the distinctive design characteristics of EPM from an integrative perspective. To address this Taxonomy and Research Agenda for Educational Process Mining Forty-Second International Conference on Information Systems, Austin, Texas 2021 2 gap, we conducted a systematic literature review to identify EPM characteristics. Building on a technology-mediated learning perspective, we develop a taxonomy that classifies EPM characteristics into four major categories (i.e., purpose, user, input, analysis). We evaluate and refine our taxonomy with ten domain experts, identified three clusters in the reviewed literature, and derived six archetypes of EPM scenarios based on our categorization. Finally, we formulate a novel research agenda to guide researchers in systematizing and synthesizing research on different technological embeddings of EPM in a students' learning process. Collapse

Topics: process mining online learning system usability human computer interaction database system

Methods: qualitative interview cluster analysis literature study literature sample structured literature research

Theories: socio technical theory

Capturing the complexity of gamification elements: a holistic approach for analysing existing and deriving novel gamification designs

2020 | European Journal of Information Systems | Citations: 41

Authors: Schöbel, Sofia Marlena; Janson, Andreas; Söllner, Matthias

Abstract: Gamification is a well-known approach that refers to the use of elements to incr ... Expand

Abstract: Gamification is a well-known approach that refers to the use of elements to increase the motivation of information systems users. A remaining challenge in gamification is that no shared understanding of the meaning and classification of gamification elements currently exists. This impedes guidance concerning analysis and development of gamification concepts, and often results in non-effective gamification designs. The goal of our research is to consolidate current gamification research and rigorously develop a taxonomy, as well as to demonstrate how a systematic classification of gamification elements can provide guidance for the gamification of information systems and improve understanding of existing gamification concepts. To achieve our goal, we develop a taxonomic classification of gamification elements before evaluating this taxonomy using expert interviews. Furthermore, we provide evidence as to the taxonomy’s feasibility using two practical cases: First, we show how our taxonomy helps to analyse existing gamification concepts; second, we show how our taxonomy can be used for guiding the gamification of information systems. We enrich theory by introducing a novel taxonomy to better explain the characteristics of gamification elements, which will be valuable for both gamification analysis and design. This paper will help guide practitioners to select and combine gamification elements for their gamification concepts. Collapse

Topics: gamification intrinsic motivation mobile application online learning system accounting

Methods: qualitative interview literature study case study theoretical contribution theory development

Progressive Web Apps: the Definite Approach to Cross-Platform Development?

2018 | HICSS | Citations: 0

Authors: Majchrzak, Tim A; Biørn-Hansen, Andreas; Grønli, Tor-Morten

Abstract: Although development practices for apps have matured, cross-platform development ... Expand

Abstract: Although development practices for apps have matured, cross-platform development remains a prominent topic. Typically, apps should always support both Android and iOS devices. They ought to run smoothly on various hardware, and be compatible with a host of platform versions. Additionally, device categories beyond smartphone and tablets have emerged, which makes multi-platform support even trickier. Truly developing an app once and serving the multitude of possible targets remains an issue despite having crossplatform frameworks that are acknowledged by practice and research. The technology unifier remains to be found, but Progressive Web Apps (PWA) might be a step towards it. In this paper, we analyse the foundations of PWAs in cross-platform development and scrutinize the status quo of current possibilities. Based on our observations, we investigate unified development, and discuss open questions. We seek to stimulate interest and narrow the immense gap that has arisen since industry started to embrace PWAs. Collapse

Topics: mobile application mobile system Google+ internet of things website

Methods: literature study synthesis

More than providing ‘solutions’: towards an understanding of customer-oriented citizenship behaviours of IS professionals

2015 | Information Systems Journal | Citations: 0

Authors: Deng, Xuefei (Nancy); Wang, Tawei; Galliers, Robert D.

Abstract: Information systems (IS) support in organizations has undergone dramatic changes ... Expand

Abstract: Information systems (IS) support in organizations has undergone dramatic changes over the years. IS professionals in the support function have become an important knowledge source to colleagues who seek assistance with their IS usage. Our understanding of IS professionals' customer-oriented behaviours is limited, however. Focusing on IS post-implementation support and drawing upon organizational citizenship behaviour (OCB) theory, this paper seeks to understand IS professionals' citizenship behaviours in supporting colleagues. Our analysis of 630 support tasks performed by IS professionals with regard to two systems at three periods reveals five types of customer-oriented OCB: anticipation, education, justification, personalization-technology and personalization-business. Our results also show different associations between four contextual factors of IS support (i.e. system, user, task and problem) and the OCBs. In instances of user deficiency, more personalization-business and anticipation OCBs were observed across all the four problem domains (functionality, data, workflow and role). By contrast, in instances of system deficiency, more personalization-technology OCBs were observed among the two problem domains of data and functionality. Moreover, the occurrence of OCBs revealed a temporal pattern such that personalization-business OCBs are more pronounced in early post-implementation periods whereas anticipation OCBs and personalization-technology OCBs become more dominant later. The categorization scheme of the customer-oriented OCB, the OCB dynamics and the patterns between OCB types and the contextual factors advance our understanding of the evolving and challenging work of organizational IS support. Our findings extend the OCB literature on customer orientation and enrich the limited studies on knowledge-intensive IS support work. Practical implications of the findings on IS management and policies are discussed. Collapse

Topics: system support IT workforce usability personalization information system use

Methods: mixed method qualitative interview multivariate analysis of covariance qualitative coding survey

A Self-Service MSS Design from a New-Generation Manager Perspective

2014 | DESRIST | Citations: 0

Authors: Mayer, Jörg H.; Röder, André; Hartwig, Jens; Quick, Reiner

Abstract: More and more digital natives are populating the management levels of organizati ... Expand

Abstract: More and more digital natives are populating the management levels of organizations. As such managers have higher expectations toward information systems (IS) accommodating their user self-service preferences, a more business-driven requirements analysis for management support systems (MSS) plays an increasingly dominant role. The objective of this paper is to develop both a set of MSS requirements that is more business-driven than the state of the art and—applying this approach—initial design guidelines for a new self-service MSS design. We demonstrate utility with a single-case study, evaluate our approach against the state of the art, and propose avenues for future research. The findings should lead to a better MSS design and be applicable to other IS domains as well. Collapse

Topics: evaluation criteria user requirement management accounting smart device systems design

Methods: design guideline literature study single case study case study structural equation modeling

Theories: technology acceptance model information systems success model

Current Research in Risk-aware Business Process Management―Overview, Comparison, and Gap Analysis

2014 | Communications of the Association for Information Systems | Citations: 0

Authors: Suriadi, Suriadi; Weiß, Burkhard; Winkelmann, Axel; ter Hofstede, Arthur H.M.; Adams, Michael; Conforti, Raffaele; Fidge, Colin; La Rosa, Marcello; Ouyang, Chun; Pika, Anastasiia; Rosemann, Michael; Wynn, Moe

Abstract: The management of risk in business processes has been the subject of active res ... Expand

Abstract: The management of risk in business processes has been the subject of active research in the past few years. Potentially, many benefits can be obtained by integrating the two traditionally separated fields of risk management and business process management, including the ability to minimize risks in business processes by design and to mitigate such risks at run time. While there has been an increasing amount of research aimed at delivering such an integrated system, these research efforts vary in terms of scope, goals, and functionality. Through the systematic collection and evaluation of relevant literature, this article compares and classifies current approaches in the area of risk-aware business process management in order to expose and explain current research gaps. The process through which relevant literature was collected, filtered, and evaluated is also detailed. Finally, a research agenda is proposed. Collapse

Topics: business process management risk management business process management system evaluation criteria database system

Methods: business process modeling literature study literature sample design science design artifact

Theories: fuzzy set theory failure mode and effects analysis computer science theory

Towards the Design of Evidence-based Mental Health Information Systems: A Preliminary Literature Review

2014 | International Conference on Information Systems | Citations: 0

Authors: Wahle, Fabian; Kowatsch, Tobias

Abstract: Mental disorders belong to a significant and serious disease pattern with an inc ... Expand

Abstract: Mental disorders belong to a significant and serious disease pattern with an increasing prevalence worldwide. Due to limited health personnel and financial resources, scalability of mental health services and tailored care of individuals are two key barriers for a successful treatment. Mental health information systems (MHIS) are expected to address these barriers and thus, it is of utmost importance to understand the underlying technological rules that guide the design of evidence-based MHIS interventions. However, up till now, there is no systematic literature review on the anatomy of MHIS that quantitatively tests the effects of MHIS service configurations on treatment success. We therefore conducted, as a very first step a preliminary review on MHIS in this research-in-progress. This review has not only the objective to present state-of-the-art on MHIS but also to propose a set of fine-grained evaluation criteria relevant for future work on the design of evidence-based MHIS interventions. Collapse

Topics: evaluation criteria database system smartphone website electronic mail

Methods: literature study literature sample experiment structured literature research qualitative coding