Could Gamification Designs Enhance Online Learning Through Personalization? Lessons from a Field Experiment

2023 | Information Systems Research | Citations: 1

Authors: Leung, Alvin Chung Man; Santhanam, Radhika; Kwok, Ron Chi-Wai; Yue, Wei Thoo

Abstract: The COVID-19 pandemic has highlighted the importance of online learning. As lear ... Expand

Abstract: The COVID-19 pandemic has highlighted the importance of online learning. As learner autonomy is relatively high in online environments, learners must engage in self-regulated learning (SRL) to achieve optimal learning outcomes. Because most learners are unable to consistently engage in SRL, gamification interventions are being implemented to improve SRL engagement; however, mixed results cast doubt on the efficacy of this approach. Massively open online courses (MOOCs), a type of online learning environment, are currently experiencing rapid growth due to widespread adoption by many institutions. In MOOCs, there is no instructor intervention; hence, students have difficulty regulating their own learning and are easily distracted. Therefore, this study investigates whether mixed-research results regarding the efficacy of gamification can be attributed to lack of attention to individual learner traits during design. For this purpose, the study analyzes MOOCs as an instance of online learning by applying SRL theory and gamification principles. We altered a traditional MOOC platform to provide different types of gamified performance feedback to facilitate learners’ SRL engagement. We then examined whether this matched with goal orientation, an individual learner trait to influence SRL and learning outcomes. Using learning-analytics tools, we tracked 760 college students’ SRL engagement on a MOOC platform over five weeks. As theorized, SRL engagement and learning outcomes of participants who had a strong performance-avoidance goal orientation increased with positively framed performance feedback that involved no social comparisons; however, the same feedback had a negative impact on participants with a strong mastery goal orientation. Our findings add to SRL theory by demonstrating that gamification designs can enhance SRL engagement and learning outcomes in online learning, but with a caveat—this occurs only when there is a match with learner traits—confirming the gamification principle stating that task improvements and meaningful engagement can only occur through thoughtful gamification design.History: Raghu Santanam, Senior Editor; J. J. Hsieh, Associate Editor.Funding: Financial supported from the University Grants Committee’s (UGC’s) Special Grant for the Strategic Development of Virtual Teaching and Learning [Project Number 6430900], the UGC Teaching and Learning Funding [Project Numbers 6391211 and 6391001], and the Digital Innovation Laboratory of Department of Information Systems, City University of Hong Kong is gratefully acknowledged.Supplemental Material: The online appendix is available at https://doi.org/10.1287/isre.2022.1123. Collapse

Topics: gamification online learning system electronic mail IT skill business intelligence

Methods: experimental group pilot experiment survey experiment digital trace data

Theories: learning theory

UNDERSTANDING THE DETERMINANTS OF BLOCKCHAIN ADOPTION IN SUPPLY CHAINS: AN EMPIRICAL STUDY IN CHINA

2023 | European Conference On Information Systems | Citations: 0

Authors: Leung, Wilson; Chang, Man Kit; Cheung, Man Lai; Shi, Si; Chan, Paddy

Abstract: This study adopted an affordance approach to examine how supply chain managers ... Expand

Abstract: This study adopted an affordance approach to examine how supply chain managers interpret the possible benefits that can be obtained in blockchain enabled supply chain applications. With a focus on governance efficiency improvement, the effects of traceability and transparency affordance on uncertainty reduction were examined from the perspective of transaction cost theory in the supply chain industry. Partial least squares-structural equation modelling was used to analyse the data collected from 364 experienced supply chain managers in China. The results revealed that traceability affordance exerted larger significant effects on environmental and trading-partner uncertainty than transparency affordance, which, in turn, affected the adoption intention. This study contributes to the literature by embedding transaction cost in blockchain affordances. The findings are useful in guiding practitioners to improve blockchain system design to reduce uncertainties in the supply chain environment, leading to a higher rate of blockchain technology adoption. Collapse

Topics: supply chain management blockchain affordance user experience usability

Methods: survey partial least squares path modeling case study PLS tool survey design

Theories: theory of affordance transaction cost economics

Utilizing the omnipresent: Incorporating digital documents into predictive process monitoring using deep neural networks

2023 | Decision Support Systems | Citations: 0

Authors: Levich, Sergej; Lutz, Bernhard; Neumann, Dirk

Abstract: Predictive process monitoring (PPM) allows companies to improve the efficiency o ... Expand

Abstract: Predictive process monitoring (PPM) allows companies to improve the efficiency of their business processes by predicting aspects such as the process outcome, the next event, or the time until the next event. So far, existing studies have mainly focused on developing novel predictive models while using features solely from event logs. In this study, we aim to go beyond log data and increase the focus of PPM research towards external context information. To this end, we consider digital documents as they are omnipresent in many business processes and their inclusion can often be justified by a business rationale. However, incorporating digital documents into PPM models poses considerable challenges as they present unstructured data that can contain visual and textual cues of future process behavior, while manual feature extraction is generally not feasible. Therefore, we propose an approach that processes digital documents based on automated visual and textual feature extraction methods. Furthermore, we design a tailored integration module which transforms the extracted features from multiple document pages into a fixed-size representation that subsequently serves as input for the predictive models. Our evaluation, based on a real-world dataset of insurance claims from a mid-sized German insurance company, featuring 5131 process instances with 32,058 events and 39,242 document pages, shows that incorporating digital documents improves the performance by significant margins in predicting the damage type, the next event, and the time until the next event. Finally, we analyze how digital documents contribute to the model’s predictions in terms of Shapley additive explanations. Collapse

Topics: shapley additive explanation business process management artificial intelligence explainable artificial intelligence

Methods: BERT machine learning descriptive statistic artificial neural network business process modeling

Operationalizing Ethics for Information Systems Design- A Tool for ethical software assessment

2023 | Americas Conference on Information Systems | Citations: 0

Authors: Levina, Olga

Abstract: Propagation of information technology in the increasingly diverse domains, rais ... Expand

Abstract: Propagation of information technology in the increasingly diverse domains, raises question concerning moral implications. While the number of ethical principles in the IT design is growing, they do not necessarily apply to specific products, which means developers and e.g. purchasing agent need to contextualize them for each specific IT product. This paper describes an actionable artifact that allows assessment of ethical non-functional requirements of an information system through operationalization of ethical values, Ethical Software Assessment Tool (ESAT). It is based on a survey of values that are currently considered in research and practice as well as in legal initiatives in IT context. The ethical approach and principles operationalized in ESAT are presented and discussed. KeywordsInformation systems development, ethical assessment, usable artifact. Collapse

Methods: survey

Primer for Experimental Methods in Organization Theory

2023 | Organization Science | Citations: 0

Authors: Levine, Sheen S.; Schilke, Oliver; Kacperczyk, Olenka; Zucker, Lynne G.

Abstract: Experiments have long played a crucial role in various scientific disciplines an ... Expand

Abstract: Experiments have long played a crucial role in various scientific disciplines and have been gaining ground in organization theory, where they add unique value by establishing causality and uncovering theoretical mechanisms. This essay provides an overview of the merits and procedures of the experimental methodology, with an emphasis on its application to organization theory. Drawing on the historical roots of experiments and their impact across science, we argue the method holds immense potential for furthering organization theory. We highlight key advantages of experimental methods, including high internal and construct validity, vividness in communicating findings, the capacity to examine complex and understudied phenomena, and the identification of microfoundations and theoretical mechanisms. We alleviate some concerns about external validity and offer guidance for designing and conducting sound, reproducible experimental research. Ultimately, we contend that the current experimental turn holds the potential to reorient organization theory.History: This paper has been accepted for the Organization Science Special Issue on Experiments in Organizational Theory.Funding: This work was supported by the Otto Moensted Foundation, which granted the Otto Moensted Visiting Professorship to S. S. Levine, and a National Science Foundation CAREER Award from the Directorate for Social, Behavioral and Economic Sciences [Grant 1943688] to O. Schilke.Supplemental Material: The online appendix is available at https://doi.org/10.1287/orsc.2023.18093. Collapse

Topics: electronic mail behavioral intention innovation management anonymity social network

Methods: experiment survey statistical hypothesis test post-hoc analysis theory development

Theories: organizational theory behavioral economics

EXPLORING VIRTUAL INFLUENCERS' POTENTIAL TO REDUCE INTERETHNIC INTOLERANCE IN ONLINE YOUTH COMMUNICATION

2023 | European Conference On Information Systems | Citations: 0

Authors: Levkov, Nikola; Santa, Mijalche; Kitanovikj, Bojan

Abstract: ... Expand

Abstract: Collapse

Topics: social media societal evaluation criteria social influence video conference usability

Methods: focus group group interview design artifact qualitative content analysis design artifact testing

Leveraging the Potential of Conversational Agents: Quality Criteria for the Continuous Evaluation and Improvement

2023 | HICSS | Citations: 2

Authors: Lewandowski, Tom; Poser, Mathis; Kučević, Emir; Heuer, Marvin; Hellmich, Jannis; Raykhlin, Michael; Blum, Stefan; Böhmann, Tilo

Abstract: Contemporary organizations are increasingly adopting conversational agents (CAs) ... Expand

Abstract: Contemporary organizations are increasingly adopting conversational agents (CAs) as intelligent and natural language-based solutions for providing services and information. CAs promote new forms of personalization, speed, cost-effectiveness, and automation. However, despite their hype in research and practice, organizations fail to sustain CAs in operations. They struggle to leverage CAs’ potential because they lack knowledge on how to evaluate and improve the quality of CAs throughout their lifecycle. We build on this research gap by conducting a design science research (DSR) project, aggregating insights from the literature and practice to derive a validated set of quality criteria for CAs. Our study contributes to CA research and guides practitioners by providing a blueprint to structure the evaluation of CAs to discover areas for systematic improvement. Collapse

Topics: database system usability conversational agent user interface robotic

Methods: design artifact qualitative interview literature sample design science chatbot

Multi-tier Supply Chain Complexity and Buyer Performance in ICT Industry

2023 | Americas Conference on Information Systems | Citations: 0

Authors: Li Liu, Kara; Ozpolat, Koray; Shin, Seung Kyoon

Abstract: The characteristics of suppliers in multi-tier supply chains have drawn more at ... Expand

Abstract: The characteristics of suppliers in multi-tier supply chains have drawn more attention recently in the postpandemic age. Due to the increasing reliance on suppliers, information-communication technology (ICT) buying firms expect to map out their upstream suppliers to identify the characteristics of those suppliers, which might affect their performance. In this paper, we utilize horizontal, vertical, and spatial complexities to examine how complex characteristics of the first-and second-tier suppliers affect the ICT buying firms' financial performance. Our empirical test shows that ICT buying firms' sales growth is affected by multitier suppliers from different industries. Our results bring insights into the role of multi-tier suppliers and their effects on ICT buying firms' financial performance. The research findings contribute to the literature on the complexity of multi-tier supply chains. We also provide practical implications for managers from ICT firms to improve their decision-making and collaboration with multi-tier suppliers. Collapse

Topics: financial performance supply chain management computer hardware decision making database system

Methods: Capital IQ network analysis

Theories: complexity theory

Managing Missing Preference Values Through Consistency and Consensus in Distributed Linguistic Preference Relations: A Two-stage Method Based on Personalized Individual Semantics

2023 | Group Decision and Negotiation | Citations: 0

Authors: Li, Cong-Cong; Gao, Yuan; Dong, Yucheng

Abstract: Linguistic distribution, as a flexible representation structure, is often used i ... Expand

Abstract: Linguistic distribution, as a flexible representation structure, is often used in linguistic decision-making. Several approaches have been proposed for managing the missing elements in incomplete preference relations when decision makers cannot provide all the linguistic comparisons between alternatives in preference relations. However, in linguistic decision-making, words can mean different things to different people, which implies the necessity of personalizing individual semantics. This study proposes a two-stage method, including a personalized individual semantics (PIS)-based consistency model and a PIS-based consensus model, to manage PIS and missing elements in incomplete distributed linguistic preference relations (DLPRs). The proposed method combines the characteristics of the personalization of linguistic expression and estimates the missing values, guaranteeing optimum consistency and consensus at the same time in group decision-making with incomplete DLPRs. Numerical and comparison analyses were performed to illustrate the use of the two-stage method, and demonstrates the various numerical meanings of words and the features of the obtained complete DLPRs from the proposed model. Collapse

Topics: missing data usability personalization decision making

Methods: optimization model cross sectional research linear optimization

The optimal asset trading settlement based on Proof-of-Stake blockchains

2023 | Decision Support Systems | Citations: 0

Authors: Li, Chao; Wang, Li; Yang, Haijun

Abstract: The Proof-of-Stake (PoS) protocol is booming in blockchain networks because of e ... Expand

Abstract: The Proof-of-Stake (PoS) protocol is booming in blockchain networks because of excessive energy consumption and slow block generation associated with the Proof-of-Work (PoW) protocol. In terms of transaction settlement, does the PoS protocol perform better than PoW? We build a transaction settlement model based on the PoS blockchain, describing how the staking income and costs affect settlement performance, and put forward a Successful Settlement Factor (SSF) to determine whether the settlement will be successful given the block size and block time. In addition, the system developers can adjust block size and block time according to SSF to achieve the best settlement performance. Finally, using historical data for a real-world blockchain network, our model demonstrates that the optimal block time, average settlement time, average settlement fees, and total settlement lag of the PoS blockchain are 23.41%, 23.29%, 23.23%, 23.31%, respectively, less than that of the PoW blockchain. Collapse

Topics: blockchain systems design ADA cryptocurrency decentralized finance

Methods: robustness check theoretical contribution experiment sensitivity analysis economic model