Using Eye Tracking to Expose Cognitive Processes in Understanding Conceptual Models

2019 | Management Information Systems Quarterly | Citations: 17

Authors: Bera, Palash; Soffer, Pnina; Parsons, Jeffrey

Abstract: Conceptual models are used to communicate information about a domain during the ... Expand

Abstract: Conceptual models are used to communicate information about a domain during the development of information systems. In two experimental studies using business process models, we demonstrate how eye tracking can contribute to understanding the cognitive processes by which readers use conceptual modeling scripts to perform problem solving tasks. In the first study, we compare scripts generated using two process modeling grammars and demonstrate how attention paid to specific parts of scripts generated using grammar variations, and differences in visual association between parts of a diagram, account for task performance. In the second study, we use a combination of eye tracking and verbal protocol analysis to examine how visual association between parts of conceptual modeling scripts can indicate cognitive integration while performing problem solving tasks. The studies show that task performance can be explained with different mental processes, reflected in specific eye tracking behavior, where scripts developed following different rules invoke different cognitive processes. We show that attention can be measured by eye tracking and can explain task performance. In addition, we show that visual association (which is observable) between parts of a modeling script involves cognitive integration (which is not observable). This finding can be used to improve conceptual modeling grammars in several ways, including understanding the effects of alternative visual arrangements of models on how effectively they communicate domain knowledge for particular tasks, and guiding the design of visual modeling notations. Collapse

Topics: task productivity problem solving decision making business process management

Methods: oculometry conceptual modelling modeling language business process model and notation protocol analysis

Theories: cognitive fit theory cognitive theory cognitive load theory

Enhancing User-Game Engagement Through Software Gaming Elements

2014 | Journal of Management Information Systems | Citations: 0

Authors: Li, Mengxiang; Jiang, Qiqi; Tan, Chuan-Hoo; Wei, Kwok-Kee

Abstract: User-game engagement is vital for building and retaining a customer base for sof ... Expand

Abstract: User-game engagement is vital for building and retaining a customer base for software games. However, few studies have investigated such engagement during gameplay and the impact of gaming elements on engagement. Drawing on the theoretical foundation of engagement, we meticulously deduced two cognitive-related gaming elements of a software game, namely, game complexity and game familiarity, and argued that these elements have individual and joint effects on user-game engagement. This research adopted multimethod empirical investigations to validate our conceptions. The first investigation used electroencephalography and a self-report survey to study quantitatively the cognitive activities of user-game engagement. The second investigation adopted the qualitative interview method to triangulate the findings from the quantitative data. This research contributes to theory in two ways, namely, conceptualizing and empirically examining user-game engagement as well as theorizing and demonstrating how the two gaming elements affect user-game engagement. This work contributes to the gaming practice by providing a set of design principles for gaming elements. Collapse

Topics: smartphone mobile game mobile system marketing management desktop computer

Methods: survey qualitative interview parametric test triangulation descriptive statistic

Deriving knowledge representation guidelines by analyzing knowledge engineer behavior

2012 | Decision Support Systems | Citations: 2

Authors: Chua, Cecil Eng Huang; Storey, Veda C.; Chiang, Roger H. L.

Abstract: Knowledge engineering research has focused on proposing knowledge acquisition te ... Expand

Abstract: Knowledge engineering research has focused on proposing knowledge acquisition techniques, developing and evaluating knowledge representation schemes and engineering tools, and testing and debugging knowledge-based systems. Few formal studies have been conducted on understanding the behaviors and roles of knowledge engineers. Applying the theory of mental models, this paper describes a think aloud verbal protocol study to determine an empirical basis for understanding: (1) how knowledge engineers extract domain knowledge from textual sources; and (2) the cognitive mechanisms by which they engage various knowledge representation schemes to represent that knowledge acquired. The results suggest that knowledge representation is not simply a translation of acquired knowledge to a knowledge representation. Instead, it is an iterative process of selective querying of acquired knowledge, and continuous refinement of a model leveraging, not only on acquired knowledge from domain experts, but also from the knowledge engineer. From the findings of empirical studies, a set of guidelines is derived to support the training and development of better knowledge representation schemes, representation processes, and knowledge engineering tools. Collapse

Topics: knowledge representation decision support system

Methods: protocol analysis qualitative coding experiment design science cognitive walkthrough

Theories: mental model theory learning theory

Minimizing Method Bias Through Programmatic Research

2009 | Management Information Systems Quarterly | Citations: 0

Authors: Burton-Jones, Andrew

Abstract: Researchers have long known that research methods influence construct measuremen ... Expand

Abstract: Researchers have long known that research methods influence construct measurements and that this influence, or method bias, can lead to false conclusions. Despite much work in the methodological literature on specific aspects of method bias, such as common method bias and self-report bias, the meaning of method bias remains unclear, and there is no comprehensive approach for dealing with it. This paper offers a clear definition of method bias, proposes a more comprehensive approach for dealing with it, and describes a demonstration exercise applying the approach in an empirical study of how individual system use and task performance relate. The demonstration suggests that the approach is feasible and illustrates how it can help researchers test theories and identify new research opportunities Collapse

Topics: information system use spreadsheet task productivity accounting

Methods: self reported survey survey qualitative coding descriptive statistic partial least squares regression

The Effects of Decomposition Quality and Multiple Forms of Information on Novices' Understanding of a Domain from a Conceptual Model

2008 | Journal of the Association for Information Systems | Citations: 0

Authors: Burton-Jones, Andrew; Meso, Peter N.

Abstract: Individuals can often use conceptual models to learn about the business domain t ... Expand

Abstract: Individuals can often use conceptual models to learn about the business domain to be supported by an information system. We investigate the extent to which such models can help novices (i.e., individuals who lack knowledge in the business domain and in conceptual modeling) to obtain an understanding of the domain codified in the model. We focus on two factors that we predict will influence novices' understanding: (1) decomposition quality: whether the conceptual model manifests a good decomposition of the domain, and (2) multiple forms of information: whether the conceptual model is accompanied by information in another form (e.g., a textual narrative). We hypothesize that both factors will have positive effects on understanding and that these effects depend on whether the individual seeks a surface or deep understanding. Our results are largely in line with our predictions. Moreover, our results suggest that while novices are generally aware that having multiple forms of information affects their understanding, they are unaware that decomposition quality affects their understanding. Based on these results, we recommend that practitioners include complementary forms of information (such as a textual narrative) along with conceptual models and be careful to ensure that their conceptual models manifest a good decomposition of the domain. Collapse

Topics: problem solving extended reality task productivity job satisfaction structured query language

Methods: design science conceptual modelling experiment unified modeling language ANOVA

Theories: cognitive theory of multimedia learning cognitive fit theory

The Differential Use and Effect of Knowledge-Based System Explanations in Novice and Expert Judgment Decisions

2006 | Management Information Systems Quarterly | Citations: 0

Authors: Arnold, Vicky; Clark, Nicole; Collier, Philip A.; Leech, Stewart A.; Sutton, Steve G.

Abstract: Explanation facilities are considered essential in facilitating user interaction ... Expand

Abstract: Explanation facilities are considered essential in facilitating user interaction with knowledge-based systems (KBS). Research on explanation provision and the impact on KBS users has shown that the domain expertise affects the type of explanations selected by the user and the basis for seeking such explanations. The prior literature has been limited, however, by the use of simulated KBS that generally provide only feedback explanations (i.e., ex post to the recommendation of the KBS being presented to the user). The purpose of this study is to examine the way users with varying levels of expertise use alternative types of KBS explanations and the impact of that use on decision making. A total of 64 partner/ manager-level and 82 senior/staff-level insolvency professionals participated in an experiment involving the use of a fully functioning KBS to complete a complex judgment task. In addition to feedback explanations, the KBS also provided feedforward explanations (i.e., general explanations during user input about the relationships between information cues in the KBS) and included definition type explanations (i.e., declarative-level knowledge). The results show that users were more likely to adhere to recommendations of the KBS when an explanation facility was available. Choice patterns in using explanations indicated that novices used feedforward explanations more than experts did, while experts were more likely than novices to use feedback explanations. Novices also used more declarative knowledge and initial problem solving type explanations, while experts used more procedural knowledge explanations. Finally, use of feedback explanations led to greater adherence to the KBS recommendation by experts--a condition that was even more prevalent as the use of feedback explanations increased. The results have several implications for the design and use of KBS in a professional decision-making environment. Collapse

Topics: problem solving decision making knowledge acquisition hypertext usability

Methods: analysis of covariance multivariate analysis of variance parametric test ANOVA experiment

Theories: theory of technology dominance

Conceptualizing Systems for Understanding: An Empirical Test of Decomposition Principles in Object-Oriented Analysis

2006 | Information Systems Research | Citations: 0

Authors: Burton-Jones, Andrew; Meso, Peter N.

Abstract: During the early phase of systems development, systems analysts often conceptual ... Expand

Abstract: During the early phase of systems development, systems analysts often conceptualize the domain under study and represent it in one or more conceptual models. One of the most important, yet elusive roles of conceptual models is to increase analystś understanding of a domain. In this paper, we evaluate the ability of the good decomposition model (GDM) (Wand and Weber 1990) to explain the degree to which conceptual models communicate meaning about a domain to analysts. We address the question, 䄙Do unified modeling language (UML) analysis diagrams that manifest better decompositions increase analysts¿ understanding of a domain?” GDM defines five conditions (minimality, determinism, losslessness, weak coupling, and strong cohesion) deemed necessary to decompose a domain in such a way that the resulting model communicates meaning about the domain effectively. In our evaluation, we operationalized each of these conditions in a set of UML diagrams and tested participantś understanding of those diagrams. Our results lend support to GDM across measures of actual understanding. However, the impact on participantś perceptions of their understanding was equivocal. Collapse

Topics: problem solving systems analysis systems design ease of use

Methods: experiment design science unified modeling language conceptual modelling qualitative interview

Theories: social network theory

Differences Between Novice and Expert Systems Analysts: What Do We Know and What Do We Do?

1998 | Journal of Management Information Systems | Citations: 39

Authors: Schenk, K. D.; Vitalari, Nicholas P.; Davis, K. Shannon

Abstract: Due to increased organizational dependence on information systems technology, th ... Expand

Abstract: Due to increased organizational dependence on information systems technology, the effectiveness of the systems analyst is more crucial today than ever. Despite technological advances that have increased the number and quality of tools available to systems analysts, the individual analyst's problem-solving skills remain key to defining good systems requirements. Understanding how expert analysts approach the requirements analysis task may provide insights for improving the effectiveness of novice analysts. This paper examines differences in the problem-solving approaches of novice and expert systems analysts during an information requirements task. Specific weaknesses that set novice and expert analysts apart are identified, and techniques that may be used to strengthen novice skills are suggested. This research supports the current literature on creativity techniques as a strategy for strengthening system analysis skills. Collapse

Topics: problem solving expert system requirements analysis participatory design systems analysis

Methods: design methodology statistical hypothesis test qualitative coding protocol analysis descriptive statistic