Sources

.

.

.

.

.

.

.

Volume 38, Issue 2, April-June 2020, Page 87-115
.

Volume 38, Issue 2, April-June 2020, Page 224-263
.

Volume 38, Issue 2, April-June 2020, Page 116-152
.

.

.

.

.

.

.

.

.

.

Volume 41, Issue 1, January 2020, Page 1-5
.

Volume 41, Issue 1, January 2020, Page 86-105
.

Volume 41, Issue 1, January 2020, Page 70-85
.

Volume 41, Issue 1, January 2020, Page 106-127
.

Volume 41, Issue 1, January 2020, Page 26-47
.

Volume 41, Issue 1, January 2020, Page 6-25
.

Volume 41, Issue 1, January 2020, Page 48-69
.

Volume 41, Issue 1, January 2020, Page 128-147
.

Volume 41, Issue 1, January 2020, Page 148-152
.

Abstract

Many professional development programs aim to improve student outcomes by enhancing teacher competencies. Effective evaluation of these programs requires a clear delineation of the competencies to be gained. A competency model was developed to evaluate the impact of a teacher professional program that aimed to improve teachers’ ability to effectively implement technologically engaged modules in a flipped classroom setting. Competencies were identified via participatory evaluation techniques and assessments were aligned to the competencies. The competency of teachers in the knowledge, skills, and abilities needed for creation and delivery of effective flipped lessons can be tracked using a radar graph to guide tailored professional development.

Abstract

This study developed a learning system that allows teachers to edit assignments designed to teach students the text structure strategy through the use of four phases: instructing, modeling, practicing, and reflecting. A 7-week instructional experiment was conducted in which 84 12th-grade students learned the text structure strategy using this system. The results produced several significant findings. First, the students demonstrated very different patterns of accessing time when using this system, making it possible to classify them into 3 categories or clusters of users: “low-practice-low-reflection students”, who spent little time using the system; “low-practice-high-reflection students” who spent most of their time in the reflecting phase, during which they constructed graphic organizers and wrote summaries primarily by referring to the teacher’s examples; and “high-practice-low-reflection students” who spent most of their time in the practicing phase, during which they constructed graphic organizers and wrote summaries without referring to the teacher’s examples. Second, the students’ patterns of accessing time were related to their prior knowledge, Web experience, and learning performance. In particular, the “low-practice-low-reflection students” had the lowest learning performance. The “low-practice-high-reflection students” had a higher frequency of using PCs to access the Web, a lower level of prior knowledge, and higher gain scores than the “high-practice-low-reflection students”. These findings are discussed, and several suggestions are proposed for future research work.

Abstract

This study investigated and compared the effectiveness of both digital game-based learning (DGBL) and static e-learning material for Newton’s laws of motion on students’ learning attention, affective experiences, cognitive load and academic achievement. Physiological signals and affective techniques were adopted to measure students’ learning affective states and cognitive load. After learning, a post-test was then conducted to discover the differences in academic achievement between DGBL and static e-learning. The results showed that the DGBL group displayed greater variance in positive emotion and attention than the traditional e-learning group during the learning process, as well as a greater cognitive load. Based on the timeline measurement of attention and positive emotion patterns in the DGBL and e-learning groups, the largest gap in both attention and positive emotion patterns was found when the DGBL group members were about to finish playing the game. The findings of this study revealed that emotional well-being and increased attention are the key advantages that DGBL learning provides when compared with traditional e-learning approaches.

Abstract

Self-regulated learning (SRL) is associated with increased academic achievement and improved learning outcomes for students. Thus, it is import to find ways to improve SRL, such as through training. Face-to-face training, discipline-dependent training, and paper-and-pencil diaries are limited in the number of students they can reach. The current randomised control study implemented discipline-independent online training and novel mobile-app based diaries and tested SRL motivation and perceived strategy use in 73 University students from mixed disciplines and study mode. Results showed that participants in the combined condition (training with diaries) improved more than other conditions. Specifically, they improved on SRL knowledge, metacognitive strategies, cognitive strategies (elaboration, organisation and critical thinking), and resources management strategies (time-management and effort regulation). The present study extends previous findings, showing that positive effects can be found for SRL when a discipline-independent approach is used coupled with online training and a mobile-app based daily diary.

Abstract

The purpose of this study was to identify types and patterns of peer scaffolding that occur during inquiry-based learning (IBL) group activities. It employed a single instrumental case approach that integrated quantitative and qualitative analyses of data gathered from 21 students in a ninth grade biology course. A verbal analysis, a content analysis, and a social network analysis (SNA) were performed to identify patterns in group interactions and refine emergent themes. First, nine types of peer scaffolding were identified and found to serve the goals of direction maintenance, cognitive structuring, and simplification. Second, three different patterns related to the high, mixed, and low prior knowledge levels of each group were identified. The high prior knowledge group provided peer scaffolding that focused attention on considerations key to developing their arguments, and this scaffolding may have improved the group’s work. In the mixed prior knowledge group, the students with greater prior knowledge were likely to support those with less prior knowledge. Together, these findings indicate the way students are grouped may impact observable patterns in peer scaffolding. Identifying the difficulties that learners face and the assistance they seek could help instructional designers and teachers identify areas in which students need support during IBL group activities. This study informs educators and practitioners of effective strategies for designing and implementing peer scaffolding to assist inquiry activities in technology-enhanced classroom settings.

Abstract

In this paper, we synthesize research on the nature and development of expertise to propose a developmental model that describes four main areas of expert knowledge: procedural, conditional, and conceptual knowledge, along with knowledge generation. We propose that these types of expert knowledge map onto and promote the development of four types of expert performance: procedural, functional, adaptive, and generative expertise. Further, we propose that expertise develops in terms of a fluency dimension consisting of execution, repertoire, and automaticity. We propose that this model highlights a potential opportunity for educators and instructional designers to target the appropriate level of expertise through teaching specific knowledge types in progression and providing practice and feedback to improve fluency. At a minimum, graduates would possess a degree of functional fluency and be better able to enter the workforce. Being aware of the need, and also knowing how, to conditionalize their own knowledge should also accelerate their continued acquisition of expertise throughout their career.

Abstract

An important element of good design, instructional or otherwise, is awareness of and being empathetic to the needs, wants, interests, values, and opinions of the intended audience. Typical approaches to learning about the subjective viewpoints of a particular audience have included survey instruments based on Likert-type items and open-ended questions, interviews, and focus groups. This article presents an overview of another approach using Q methodology. Q methodology is specifically designed to reveal and study subjectivity within a group of people in a systematic way using both quantitative and qualitative data. Q methodology offers designers with the means to identify a small number of profiles representing distinct points of view among the intended audience on a given topic. After presenting an overview of the historical and theoretical underpinnings of Q, several examples are provided to illustrate Q’s potential to improve design within the field of learning, design, and technology. Critiques of Q methodology are also described.

Abstract

The present study was an attempt to help us reveal the characteristics and complexity of today’s first-time online students in a higher education setting. Data were collected from undergraduate and graduate students enrolled in fully online courses for the first time during spring semester in the 2016–2017 academic year at a Southern university in the United States. Primarily, path analysis was conducted to investigate the impacts of flexible thinking, mindsets, and self-efficacy on the 254 first-time online students’ online learning engagement. The results of the path analysis supported six out of the eight hypotheses and all standardized path coefficients have values between 0.14 and 0.31. In conclusion, growth mindset and learning self-efficacy appear to be important variables for first-time online students and have a positive relation to online engagement. The practical implications and future research are discussed.

Abstract

Education is characterized by at least the following features: (a) it offers learning environments that help students to achieve preset goals, (b) it induces learners to engage in relevant learning tasks, and (c) it offers support while learners execute learning tasks. Offering learning environments builds on the assumption that learners will actually engage in the learning tasks and use the support provided. There is, however, growing evidence, that in a lot of cases students do not comply with that assumption. By not engaging as expected in the learning tasks and/or by not (adequately) using the support, learners reveal the phenomenon of what could be called—at least from the perspective of the (designer of the) learning environment—‘instructional disobedience’. ‘Instructional disobedience’ occurs when learners do not act as expected from them in a learning environment. While the literature has already referred to faulty assumptions and specified conditions for the effectiveness of instructional interventions, it seems the phenomenon of instructional disobedience has not yet attracted systematic research attention. In this contribution, we want therefore further unravel the nature and relevance of the phenomenon. This is done by analyzing the occurrence of instructional disobedience, possible explanations and ways to deal with it. As a start we illustrate the phenomenon by providing some examples.

Abstract

Recent research on multimedia learning has considered the integration of cognitive and affective aspects of media processing. The literature suggests that learners’ emotions influence the effectiveness of multimedia learning, which is explained by the cognitive-affective theory of learning with media (CATLM). A multimedia design that changes learners’ emotional status can facilitate or suppress learning. Individual difference, which suggests that learners with different expertise levels respond differently to an emotional design, is an assumption of CATLM. However, how learner expertise influences the effectiveness of emotional designs remains unclear. This study investigated the effects of learner expertise (novice vs advanced) and an emotional design incorporating a face-like shape and warm colours (with vs without) on developing skills in remembering and understanding in mathematics learning. The novice group comprised younger learners who had no prior knowledge of the topic; the advanced group comprised older learners who had studied the topic previously. We randomly allocated 122 primary school students to four experimental groups to see how they learned geometrical patterns from videos with different designs. These results showed that (1) the emotional design group performed better in remembering, and (2) the emotional design benefited the advanced group, but not the novice group, in understanding. A plausible explanation is that the benefits of the emotional design do not outweigh its drawback in the novice group when developing understanding. Further analysis revealed that learner expertise and learning outcomes influence the designs’ effects. Our findings suggested that using emotional design can effectively facilitate lower-order thinking skills such as remembering, identifying and procedural skills, and drawing students interests and motivation may not lead to better learning outcomes.

Abstract

Massive Open Online Courses (MOOCs) can be enhanced with the so-called learning-by-doing, designing the courses in a way that the learners are involved in a more active way in the learning process. Within the options for increasing learners’ interaction in MOOCs, it is possible to integrate (third-party) external tools as part of the instructional design of the courses. In MOOCs on computer sciences, there are, for example, web-based Integrated Development Environments (IDEs) which can be integrated and that allow learners to do programming tasks directly in their browsers without installing desktop software. This work focuses on analyzing the effect on learners’ engagement and behavior of integrating a third-party web-based IDE, Codeboard, in three MOOCs on Java programming with the purpose of promoting learning-by-doing (learning by coding in this case). In order to measure learners’ level of engagement and behavior, data was collected from Codeboard on the number of compilations, executions and code generated, and compared between learners who registered in Codeboard to save and keep a record of their projects (registered learners) and learners who did not register in Codeboard and did not have access to these extra features (anonymous learners). The results show that learners who registered in Codeboard were more engaged than learners who did not register (in terms of number of compilations and executions), spent more time coding and did more changes in the base code provided by the teachers. The main implication of this study suggests the need for a trade-off between designing MOOCs that allow a very easy and anonymous access to external tools aimed for a more active learning, and forcing learners to give a step forward in terms of commitment in exchange for benefitting from additional features of the external tool used.

Abstract

Instructional Design Knowledge (IDK) can inform technology integration decisions and Technology Pedagogy and Content Knowledge (TPACK) can help instructional design processes. As a means to understand how teachers may draw from their TPACK and IDK as they design instructions and develop technology-enhanced learning activities, we examined the final projects of two groups of teachers enrolled in graduate-level instructional design and technology courses. By using both content and social network analysis methods, we identified the IDK and TPACK components exemplified in teachers’ projects. While the content analysis revealed differences between the two groups, some findings were common across the courses such as teachers minimally connecting technology to their content areas, exhibiting limited knowledge on learning needs, and having difficulties in engaging in design thinking processes. Furthermore, the social network analysis identified various communities of the knowledge components, highlighting when teachers tended to use their IDK and TPACK as they planned technology-enhanced learning activities and were engaged in instructional design respectively.

Abstract

This study focused on the use of casual games to motivate learners in a corporate retail settings to engage in a online learning platform. The study analyzed two populations of learners who were using the learning platform. One group of learners was in a game condition. In this condition, the learners were provided the option to play a casual game (lasting no more than five minutes) every time they logged into the software. The second group of learners was in a non-game condition. They did not have the option of playing games at any time on the platform. Over a 12 month period of time, the study compared the level of engagement of the two groups as measured by the number of times learners returned to the platform and the proportion of time the learner spent browsing non-required areas of the platform. The results provided evidence that playing a casual game produced significant levels of engagement with the learning platform. It was found that learners in the game condition logged in significantly more often than those in the non-game condition. The evidence supports the conclusion that adult learners are more motivated to come back and engage with the learning platform when they can play a casual game first. In terms of learning, it was found learners in the game condition answered significantly more questions correctly and had significantly longer correct answer streaks than learners in the nongame condition.

Abstract

We examined how social antecedents impact students’ use of explanations versus evidence to justify arguments using statistical discourse analysis on 2028 postings from 87 graduate students in five courses, each participating in four online debates. The results show that students overall were much more likely to justify arguments with explanations than with evidence. Explanations were more likely than evidence to be used in postings by women, when students posted responses to messages that conveyed greater social proximity (using he/she/they and using you instead of we) or directed attention (there), when making posts in early parts of a discussion thread and in the opening argument. Evidence was more likely to be used when responding to messages from men and when making posts towards the end of each discussion thread.

Abstract

This study is a review of video games studies completed in South Africa. This paper looked at existing research on video games in South Africa in order to understand the research approaches used, and the learning outcomes found in respect to findings from studies in the western world. Interestingly, the study shows similarities in the learning outcomes, yet a great emphasis on educational games targeted to address issues relevant to South Africa. This research also reveals that context, as argued in sociocultural theory, shaped the content of the games designed, populations studied, type of game to associate with video games, and learning.

Abstract

We proposed to investigate whether properly calibrated e-learning environments can efficiently promote computational thinking of both sciences- and humanities-oriented people. We invited two groups of students (sciences- vs. humanities-oriented members) to participate in a six-stage learning session: to watch a folk-dance illustration (s1) and an animation (s2) of the bubble-sort algorithm; to reconstruct the algorithm on the same input (s3); to orchestrate the algorithm on a random input stored in a white(s4)/black(s5) array (visible/invisible sequence) and to watch a parallel simulation of several sorting algorithms as they work side-by-side on different color-scale bars (s6). To assess the current motivation of students we created nine specific questionnaires (Q1–9). The experiment we conducted included the following task sequence: Q1–2, s1, Q3, s2, Q4, s3, Q5, s4, Q6, s5, Q7, s6, Q8–9. We focused on assessing the motivational contributions of the generated (situational factors) emotions, challenge and active involvement during the e-learning experience. Research results revealed that there are no unbridgeable differences in the way these two groups relate to e-learning processes that aim to promote computational thinking. Although sciences-oriented students’ motivational-scores were consistently superior to their humanities-oriented colleagues, there was strong correlation between them; furthermore, differences diminished as both groups advanced with their learning tasks.

Abstract

The nature of knowledge and how it is developed have been debated in philosophy and research for centuries. In the literature on teachers’ knowledge, two perspectives have been particularly visible. One perspective stresses cognitive processes and deliberate knowledge acquisition. Another perspective stresses the situated nature of teachers’ learning and knowledge development through awareness. This theoretical article proposes that uniting both epistemological perspectives is beneficial for developing teachers’ contextualized knowledge of how to teach at all phases of career development, and especially early on. In so doing, action and reflection are positioned as central to the development of teachers’ knowledge, and affordances from both the deliberate and the aware perspectives are articulated. Specifically, this article explains why uniting the two perspectives supports better sense-making, more refined instructional planning, and more responsive teaching, before offering a united reflection model. These processes are then discussed in the context of video coaching interventions for early-career teachers. After presenting a blueprint for video-based reflection and key design features that could support teachers’ learning, important differences compared to other reflection models are discussed and implications for (the design of) teachers' professional development based upon this united perspective are presented.

Abstract

Complex problem solving is an effective means to engage students in disciplinary content while also furnishing critical non-cognitive and life skills. Despite increased adoption of complex problem-solving methods in K-12 classrooms today (e.g., case-, project-, or problem-based learning), we know little about how to make these approaches accessible to linguistically and culturally diverse (LCD) students. In this paper, we promote a conceptual framework, based on an ethos of intentionality, that supports culturally responsive teaching (CRT). We provide specific questions to guide teachers’ implementation of an ethos of intentionality, through critical reflection and meaningful action, and discuss a framework for culturally relevant practice that operationalizes key central tenets (e.g., high expectations, cultural competence, and critical consciousness). Finally, we include strategies that can help teachers and designers translate the principles of the CRT framework into action with a specific focus on complex problem solving in classrooms.

Abstract

This article explores the complex question of how instruction should be framed (i.e., contextualized). Reports from the US National Research Council reveal a broad consensus among experts that most instruction should be framed with problems, examples, cases, and illustrations. Such framing is assumed to help learners connect new knowledge to broader “real world” knowledge, motivate continued engagement, and ensure that learners can transfer their new knowledge to subsequent contexts. However, different theories of learning lead to different assumptions about when such frames should be introduced and how such frames should be created. This article shows how contemporary situative theories of learning argue that frames should be (a) introduced before instructional content, (b) generated by learners themselves, (c) used to make connections with people, places, topics, and times beyond the boundaries of the course, and (d) used to position learners as authors who hold themselves and their peers accountable for their participation in disciplinary discourse. This expansive approach to framing promises to support engagement with disciplinary content that is productive (i.e., increasingly sophisticated, raising new questions, recognizing confusion, making new connections, etc.) and generative (i.e., supporting transferable learning that is likely to be useful and used in a wide range of subsequent educational, professional, achievement, and personal contexts). A framework called Participatory Learning and Assessment (PLA) is presented that embeds expansively framed engagement within multiple levels of increasing formal assessments. This paper first summarizes PLA as theory-laden design principles. It then presents PLA as fourteen more prescriptive steps that some may find easier to implement, allowing them to learn as they go. Examples are presented from several courses from an extended program of design-based research using this approach in online and hybrid secondary, undergraduate, graduate, and technical courses.

Presents the 2019 subject/author index for this publication.

Our educational project has three primary goals. First, we want to provide a robust vision of how hardware and software interplay, by integrating the design of an instruction set (through microprogramming) and using that instruction set for assembly programming. Second, we wish to offer a versatile and interactive tool where the previous integrated vision could be tested. The tool we have developed to achieve this is called WepSIM and it provides the view of an elemental processor together with a microprogrammed subset of the MIPS instruction set. In addition, WepSIM is flexible enough to be adapted to other instruction sets or hardware components (e.g., ARM or x86). Third, we want to extend the activities of our university courses, labs, and lectures (fixed hours in a fixed place), so that students may learn by using their mobile device at any location, and at any time during the day. This paper presents how WepSIM has improved the teaching of Computer Architecture courses by empowering students with a more dynamic and guided learning process. In this paper, we show the results obtained during the experience of using the simulator in the Computer Structure course of the Bachelor's Degree in Computer Science and Engineering, University Carlos III of Madrid.

Children with autism spectrum disorder (ASD) are characterized by deficits in social communication, partly attributed to the inability to pick up cues from social partners using joint attention (JA) skill. These deficits have cascading adverse effects on language acquisition and the development of cognitive skills. Therapist-mediated JA interventions are labor intensive. Robot-facilitated skill training is expensive, need specialized knowledge to operate, etc. In contrast, computer-based JA skill training platforms are affordable, offers flexibility to the designer, but lack individualization. Individualization is critical for effective skill training. To bridge this gap, we have developed virtual reality based JA task platform augmented with hierarchical prompt protocol (using eye, head turn, finger pointing, and sparkling cues). It was adaptive to individualized performance and autonomously increased level of prompting on demand. Results of a study with 20 pairs of age-matched ASD and typically developing (TD) participants indicate the potential of our system to identify JA skill deficits. Participants with ASD showed impairment in following eye cue. However, their ability to pick up finger pointing was the best, with some of them being able to pick up only the sparkling cue. However, all TD participants were able to pick up eye cue with none requiring other cues.

Bullying is a serious social problem at schools, very prevalent independently of culture and country, and particularly acute for teenagers. With the irruption of always-on communications technology, the problem, now termed cyberbullying, is no longer restricted to school premises and hours. There are many different approaches to address cyberbullying, such as school buddies, educational videos, or involving police in counseling; however, awareness continues to be insufficient. We have developed Conectado, a serious game to be used in the classroom to increase awareness on bullying and cyberbullying in schools. While playing the game, students gain a first-hand immersive experience of the problem and the associated emotions, fostering awareness and empathy with victims. This paper describes Conectado and presents its validation with actual students using game analytics.

Intelligent learning environments can be designed to support the development of learners’ cognitive skills, strategies, and metacognitive processes as they work on complex decision-making and problem-solving tasks. However, the complexity of the tasks may impede the progress of novice learners. Providing adaptive feedback to learners who face difficulties requires learner modeling approaches that can identify learners’ proficiencies and the difficulties they face in executing required skills, strategies, and metacognitive processes. This paper discusses a multilevel hierarchical learner modeling scheme that analyzes and captures learners’ cognitive processes and problem-solving strategies along with their performance on assigned tasks in a game-based environment called UrbanSim that requires complex decision making for dealing with counterinsurgency scenarios. As the scenario evolves in a turn-by-turn fashion, UrbanSim evaluates the learners’ moves using a number of performance measures. Our learner modeling scheme interprets the reported performance values by analyzing the learners’ activities captured in log files to derive learners’ proficiencies in associated cognitive skills and strategies, and updates the learner model. We discuss the details of the learner modeling algorithms in this paper, and then demonstrate the effectiveness of our approach by presenting results from a study we conducted at Vanderbilt University.

This paper studies students engagement in e-learning environments in which students work independently and solve problems without external supervision. We propose a new method to infer engagement patterns of users in such self-directed environments. We view engagement as a continuous process in time, measured along chosen axes that are derived from student data in the system using unsupervised learning (Principal Component Analysis). We construct a trajectory of user activity by projecting the user's scores along the selected PCs at regular time intervals. This approach is applied to a popular e-learning software for K12 math education that is used by thousands of students worldwide. We identify cohorts of users according to the way their trajectory changes over time (e.g., monotone up, monotone down, and constant). Each of the cohorts exhibits distinct behavioral dynamics and differed substantially in the amount of time users spent in the e-learning system. Specifically, one cohort included students that dropped out of the system after choosing very difficult problems that they were not able to complete, while another cohort included students users that chose more diverse problems and stayed longer in the system. In future work, these results can be used by teachers or intelligent tutors to track students’ engagement in the system and decide whether and how to intervene.

Learning design in a massive open online course (MOOC) intends to promote creativity, autonomy, and social networked learning, amongst other things. Students in a MOOC are required to self-regulate their learning to properly self-monitor their learning process and effectiveness of the adopted learning strategies. This paper presents the results of a study among 279 students enrolled in a MOOC that was enriched with a set of scaffolding interventions for social mirroring. The mirroring interventions supported social awareness and social embeddedness of learners. Associations between the use of the interventions and microlevel self-regulated learning processes were measured and analyzed. The extent to which those associations are affected by learner demographics and motivational characteristics was also investigated. Findings show that interventions that provide students, throughout the course, with learning updates and progress of peers are associated with the students' engagement with learning tasks and applying changes in strategies for completing those tasks. Social awareness scaffold influenced more students low in need for cognition, with a higher education degree, high in performance-approach orientation and low in grit, to engage with their learning tasks, while its effect on the change in learning strategies was higher with those early and towards the end of their careers and high in performance-approach strategy. The social comparison scaffold affected more students low in mastery goal orientation and high in grit to work on their learning tasks.

Problem solving, worked examples, and erroneous examples have proven to be effective learning activities in Intelligent Tutoring Systems (ITSs). However, it is generally unknown how to select learning activities adaptively in ITSs to maximize learning. In the previous work of A. Shareghi Najar and A. Mitrovic, alternating worked examples with problem solving (AEP) was found to be superior to learning only from worked examples or only from problem solving. In our first study, we investigated whether the addition of erroneous examples further improves learning in comparison to AEP. The results indicated that erroneous examples prepared students better for problem solving in comparison to worked examples. Explaining and correcting erroneous examples also led to improved debugging and problem-solving skills. In the second study, we introduced a novel strategy that adaptively decided what learning activity (a worked example, a 1-error erroneous example, a 2-error erroneous example, or a problem to be solved) is appropriate for a student based on his/her performance. We found the adaptive strategy resulted in comparative learning improvement in comparison to the fixed sequence of worked/erroneous examples and problem solving, but with a significantly lower number of learning activities.

We present the design of a citizen science system that uses machine learning to guide the presentation of image classification tasks to newcomers to help them more quickly learn how to do the task while still contributing to the work of the project. A Bayesian model for tracking volunteer learning for training with tasks with uncertain outcomes is presented and fit to data from 12,986 volunteer contributors. The model can be used both to estimate the ability of volunteers and to decide the classification of an image. A simulation of the model applied to volunteer promotion and image retirement suggests that the model requires fewer classifications than the current system.

Maintaining and cultivating student engagement is critical for learning. Understanding factors affecting student engagement can help in designing better courses and improving student retention. The large number of participants in massive open online courses (MOOCs) and data collected from their interactions on the MOOC open up avenues for studying student engagement at scale. In this work, we develop an interpretable statistical relational learning model for understanding student engagement in online courses using a complex combination of behavioral, linguistic, structural, and temporal cues. We show how to abstract student engagement types of active, passive, and disengagement as meaningful latent variables using logical rules in our model connecting student behavioral signals with student success in MOOCs. We demonstrate that the latent formulation for engagement helps in predicting two measures of student success: performance, their final grade in the course, and survival, their continued presence in the course till the end, across seven MOOCs. Further, in order to initiate better instructor interventions, we need to be able to predict student success early in the course. We demonstrate that we can predict student success early in the course reliably using the latent model. We also demonstrate the utility of our models in predicting student success in new courses, by training our models on one course and testing on another course. We show that the latent abstractions are helpful in predicting student success and engagement reliably in new MOOCs that haven't yet gathered student interaction data. We then perform a closer quantitative analysis of different features derived from student interactions on the MOOC and identify student activities that are good indicators of student success at different points in the course. Through a qualitative analysis of the latent engagement variable values, we demonstrate their utility in understanding students&- x0027; engagement levels at various points in the course and movement of students across different types of engagement.

With the wide spread large-scale e-learning environments such as MOOCs, peer assessment has been popularly used to measure the learner ability. When the number of learners increases, peer assessment is often conducted by dividing learners into multiple groups to reduce the learner's assessment workload. However, in such cases, the peer assessment accuracy depends on the method of forming groups. To resolve that difficulty, this study proposes a group formation method to maximize peer assessment accuracy using item response theory and integer programming. Experimental results, however, have demonstrated that the method does not present sufficiently higher accuracy than a random group formation method does. Therefore, this study further proposes an external rater assignment method that assigns a few outside-group raters to each learner after groups are formed using the proposed group formation method. Through results of simulation and actual data experiments, this study demonstrates that the proposed external rater assignment can substantially improve peer assessment accuracy.

In this paper, we studied different automatic short answer grading (ASAG) systems to provide a comprehensive view of the feature spaces explored by previous works. While the performance reported in previous works have been encouraging, systematic study of the features is lacking. Apart from providing systematic feature space exploration, we also presented ensemble methods that have been experimentally validated to exhibit significantly higher grading performance over the existing papers in almost all the datasets in ASAG domain. A comparative study over different features and regression models toward short-answer grading has been performed with respect to evaluation metrics used in evaluating ASAG. Apart from traditional text similarity based features like WordNet similarity, latent semantic analysis, and others, we have introduced novel features like topic models suited for short text, relevance feedback based features. An ensemble-based model has been built using a combination of different regression models with an approach based on stacked regression. The proposed ASAG has been tested on the University of North Texas dataset for the regression task, whereas in case of classification task, the student response analysis (SRA) based ScientsBank and Beetle corpus have been used for evaluation. The grading performance in case of ensemble-based ASAG is highly boosted from that exhibited by an individual regression model. Extensive experimentation has revealed that feature selection, introduction of novel features, and regressor stacking have been instrumental in achieving considerable improvement in performance over the existing methods in ASAG domain.

Humor and fear appeals are widely employed in traditional communication for educational purposes, but their exploitation in animated pedagogical agents has been scarcely explored. We studied the use of humor and fear appeals by a three-dimensional animated pedagogical agent that taught the same procedural knowledge in four conditions: i) humor appeal, ii) fear appeal, iii) humor, and fear appeal together, and iv) no emotion appeals. The agent delivered and illustrated the knowledge in an application for aviation safety education. The results of our study show that using the educational application had overall positive learning effects, regardless of the appeals used by the agent. Resort to humor and fear appeals by the agent affected instead participants’ mood, liking of the application, and perception of the agent. The paper includes a discussion of the advantages and disadvantages of using humor and fear appeals in animated pedagogical agents.

In most cultural heritage courses, students physically visit several renowned heritage sites for educational purposes. However, because of time and manpower limitations, many teachers use traditional outdoor instruction methods to transmit vital information regarding these sites and buildings. This approach could result in students merely memorizing knowledge for test-taking purposes, rather than engaging in active thinking. Therefore, this study proposed using a ubiquitous guide-learning system to enhance students’ performance and active participation in cultural heritage courses. To evaluate the effectiveness of the proposed system, a trial experiment was conducted to visit eight heritage sites featured in the cultural heritage course offered by a local university. In total, 62 students were randomly assigned to either an experimental group testing the proposed system or a control group using traditional outdoor instruction methods. The experimental results indicated that our ubiquitous guide-learning system significantly improved students’ learning achievements. The outcome of the Instructional Materials Motivation Survey questionnaire indicated that the experimental group participants were motivated to learn and interested in using the proposed system. Moreover, the two groups had statistically significant results for the attention, confidence, and satisfaction factors. Therefore, implementing our proposed system in conjunction with cultural heritage education courses may yield a significant learning advantage for students by improving overall learning performance and motivation.

Learning in computer-mediated setting represents a complex, multidimensional process. This complexity calls for a comprehensive analytical approach that would allow for understanding of various dimensions of learner generated discourse and the structure of the underlying social interactions. Current research, however, primarily focuses on manual or, more recently, supervised methods for discourse analysis. Moreover, discourse and social structures are typically analyzed separately without the use of computational methods that can offer a holistic perspective. This paper proposes an approach that addresses these two challenges, first, by using an unsupervised machine learning approach to extract speech acts as representations of knowledge construction processes and finds transition probabilities between speech acts across different messages, and second, by integrating the use of discovered speech acts to explain the formation of social ties and predicting course outcomes. We extracted six categories of speech acts from messages exchanged in discussion forums of two MOOCs and each category corresponded to knowledge construction processes from well-established theoretical models. We further showed how measures derived from discourse analysis explained the ways how social ties were created that framed emerging social networks. Multiple regression models showed that the combined use of measures derived from discourse analysis and social ties predicted learning outcomes.

Adaptive e-learning can be used to personalize learning environment for students to meet their individual demands. Individual differences depend on the students’ personality traits. Numerous studies have indicated that understanding the role of personality in the learning process can facilitate learning. Hence, personality identification in e-learning is a critical issue in education. In this study, we propose the enhanced extended nearest neighbor (EENN) algorithm to automatically identify two of the Big Five personality traits from students’ behavior in online learning: openness to experience and extraversion. The performance of the proposed method is evaluated using a fivefold cross-validation approach on data from 662 senior high school students. The experimental results indicate that the EENN method can automatically recognize personality at an average accuracy of 0.758. The optimized method that combines EENN with particle swarm optimization significantly improves the identification, resulting in an average accuracy of 0.976. The results can benefit students by increasing the accuracy of personalization based on their personality traits, while simultaneously allowing them to be better understood and possibly allowing their instructors to provide more appropriate learning interventions.

Automatic multiple choice question (MCQ) generation from a text is a popular research area. MCQs are widely accepted for large-scale assessment in various domains and applications. However, manual generation of MCQs is expensive and time-consuming. Therefore, researchers have been attracted toward automatic MCQ generation since the late 90's. Since then, many systems have been developed for MCQ generation. We perform a systematic review of those systems. This paper presents our findings on the review. We outline a generic workflow for an automatic MCQ generation system. The workflow consists of six phases. For each of these phases, we find and discuss the list of techniques adopted in the literature. We also study the evaluation techniques for assessing the quality of the system generated MCQs. Finally, we identify the areas where the current research focus should be directed toward enriching the literature.

We present a system that automatically generates deictic gestures for animated pedagogical agents (APAs). The system takes audio and text as input, which define what the APA has to say, and generates animated gestures based on a set of rules. The automatically generated gestures point to the exact locations of elements on a whiteboard nearby the APA, which are calculated by searching for keywords mentioned in the speech. We conducted a study with 100 subjects, in which we compared lecture videos containing gestures automatically-scripted by the system to videos of the same lecture containing manually-scripted gestures. The study results show that the manually-scripted and automatically-scripted lectures had comparable number of gestures, and that the gestures were timed equally well.

Presents a listing of the editorial board, board of governors, current staff, committee members, and/or society editors for this issue of the publication.

Presents the table of contents for this issue of the publication.

Abstract

Conditions under which group work leads to learning have been studied in collaborative settings. Little is known, however, about whether and how the interplay between collaboration and cooperation impinges on group learning. In this paper, we study this interplay in the context of mathematical problem-solving. We focus on how training students to learn together influences this interplay, and on the relations of this interplay with mathematical problem-solving. Five groups of Grade 8 students participated in a course aimed at fostering learning to solve mathematical problems in small groups. Before and after the course, they solved a mathematical problem. An increase in the ratio of cooperation episodes out of total group work time was observed, as well as advancements in mathematical problem-solving. In addition, we found a mid-high correlation between instances of cooperation and mathematical activity: up to a certain threshold, cooperating more in a group yielded an increase in the individual generation of mathematical claims and arguments. We identified the critical role of coordination: for group learning to be productive, students should continuously negotiate and adjust their goals through communication before or while they cooperate on different tasks. We conclude that teachers aiming at fostering group work should encourage the diversification of modes of group work, for the advancement of mathematical problem-solving or of any case in which individual settings are too challenging.

Abstract

Prior literature has begun to demonstrate that even young children can learn about complex systems using participatory simulations. This study disentangles the impacts of third-person perspectives (offered by traditional simulations) and first-person perspectives (offered by participatory simulations) on children’s development of such systems thinking in the context of the emergent complexity of honeybee nectar foraging. Specifically, we worked with three first-grade classrooms assigned to one of three conditions—instruction through use of a first-person perspective only, third-person perspective only, and integrated instruction—to engage ideas of complex systems thinking. In each condition, systems concepts were targeted through instruction and assessment. The integrated and third-person classrooms demonstrated significant gains while the first-person classroom showed gains that were not statistically significant, suggesting that third-person perspectives play a critical role in how children learn systems thinking. This work also puts forth a novel assessment design for young children using multiple-choice questions.

Abstract

Classroom activities using an inquiry approach often feature students working in small groups to reduce teacher-centeredness and maximize student autonomy. Within science classrooms, group work may mirror modern scientific research: successful interaction among team members (social/relational) that engages probing questioning and creativity (cognitive/content) with emotional attachment to their work (affective). Previous research on small group work in school science focused either on single dimensions of group work—mostly on needed cognitive resources, e.g., knowledge and skills for understanding and addressing the problem—or on the interplay between cognitive and social resources (e.g., science knowledge and capacity to foster group interactions), while the role of affects is relatively unexplored. We propose that group work demands the collective construction of a “triple problem solving space” in which all three dimensions—cognitive/content (the problem to be solved), social/relational (the challenges based on social interactions within the group), and affective (the emotional life of the group)—are developed on a moment-by-moment basis. Assessing whether and to what extent students collectively construct a positive triple problem solving space, we videotaped small groups’ interactions (3–4 students per group) during inquiry-based activities in three ninth grade science classes. Results showed that when a group collectively positions itself positively in terms of social and affective dynamics, it tends to engage effectively in the cognitive aspects of the assigned tasks. The qualitative analysis further highlights the socially-shared regulation processes that involve an ongoing negotiation between intra- and inter-individual resources and which are the result of each group member deploying individual resources along each dimension, monitoring and evaluating their peers’ processes, and adjusting their processes accordingly through integration of information from self and others.

Abstract

Many learners possess misconceptions regarding instructional content; toward this aim, educational practitioners employ teaching practices that support learners’ efforts to restructure their existing knowledge structures—a process known as conceptual change. The Cognitive Reconstruction of Knowledge Model emphasizes the importance of conceptual dissatisfaction, topic interest, and learners’ need for cognition in the process of conceptual change. However, most conceptual change studies have failed to differentiate the contribution of dispositional and situational interest to the revision of conceptual understanding. The current study was designed to test key predictions of the Cognitive Reconstruction of Knowledge Model while also exploring the influence of dispositional and situational interest on the conceptual change process. Participants (N = 360) recruited from two universities in the United States completed measures assessing pre- and post-test knowledge, individual and situational interest, need for cognition, dissatisfaction, and cognitive engagement. Further, participants read a refutational text designed to address commonly endorsed misconceptions regarding HIV/AIDS. Results of a path analysis indicated situational interest exerted a significant indirect effect on conceptual change scores through cognitive engagement. Contrary to the predictions of the Cognitive Reconstruction of Knowledge Model, our findings indicated that the need for cognition, individual interest, and cognitive conflict were not significant predictors of conceptual change. We believe the findings of our investigation highlight the importance of fostering situational interest when attempting to promote knowledge reconstruction among learners.

Abstract

While previous classroom studies of status hierarchies tell us who has low status and how to increase those learners’ participation in small group contexts via teacher-led interventions, we know little about how one becomes low status, or the role peers play in legitimating or delegitimating inequitable relations. This study used the sociocultural concept of marginal non-participation to describe interactional moves learners use to navigate status hierarchies in an inquiry science context where student authority may permit learners to obstruct peers’ participation. Participants were three collaborative groups of 3–4 learners in 7th grade science classrooms where a series of inquiry curriculum units were being implemented. Interviews were used alongside a microgenetic analysis of video-recorded group work observations to identify interactions that legitimated and delegitimated status hierarchies. Legitimation involved communicating acceptance of differential belonging and competence while delegitimation involved challenging differential reward by fostering widespread participation. Low- and high-status group members were active in both processes. Results suggest that diffuse status characteristics and science capital inform how status hierarchies are negotiated and that learners adapt disciplinary norms for status legitimating and delegitimating ends. Implications for learners’ participation in scientific practices and identification with science are discussed.

Abstract

Digital drawing can foster learning, but only if the drawing is of sufficient quality. Hence, the focus of the present study was to investigate whether and how two types of drawing support may foster drawing quality and, in turn, learning outcomes. To this end, participants (N = 156) were randomly assigned to one of four conditions, in which they either just read text (control), were prompted to make a free-hand representational drawing (unsupported drawing), or they were additionally supported in their drawing efforts because a background (global support) or single elements for the drawing (local support) were already provided. Learning outcomes were assessed by means of recognition, transfer, and a drawing test. Results revealed that students from all three drawing conditions (unsupported, global, and local support) scored better on the transfer and drawing tests than the control condition. Both types of drawing support did neither increase drawing quality nor learning in comparison to unsupported drawing. Reasons for the latter findings are discussed.

Abstract

This paper presents a secondary analysis of data collected during an intervention study in which students learnt to synthesise pairs of texts presenting opposite views on controversial issues. The original intervention study included two treatments and examined the effects of two instruction conditions when instructional materials and tasks were held constant. The participants were 114 undergraduate psychology students. The object of the instruction was a guide on strategies for writing an argumentative synthesis text. However, the instruction varied between explicit strategy instruction, consisting of explaining each of the process’s four phases (exploring and identifying arguments and counterarguments, contrasting positions, drawing an integrative conclusion, and organising and revising the final draft), modelled via videos, versus self-study of the written strategy guide. After the initial instruction session, the students in both groups practiced collaboratively writing synthesis texts over two sessions with access to the strategy guide. The primary study compared the individually written pre- and posttest syntheses and found statistically significant differences favouring explicit instruction in both dependent variables: the argumentation coverage and the level of integration. The secondary analysis reported in the current paper involved scoring additional written syntheses produced during two practice sessions and then analysing the data for all time points (pretest, posttest, and the two practice sessions) using structural equation modelling (SEM) to test whether explicit instruction directly or indirectly affected the two indicators of good argumentative synthesis texts—argument coverage and integration—via the following collaborative practice. The results suggested two different learning paths for both dependent variables: explicit instruction is effective for both variables, while collaborative practice only has an additional indirect effect on argument coverage.

Abstract

Education is under a radical transformation in the current innovation-driven knowledge age. The instructor-student collaborative partnership has the potential to transform education from traditional instructor-directed, transmissive teaching to active, participatory student-centered learning. However, relevant inquiry indicates the conceptual, analytical, and practical gaps on the instructor-student collaborative partnership. This study aims to conceptualize, analyze, and foster the instructor-student collaborative partnership in higher education contexts. To achieve this purpose, we empirically investigate the instructor-student collaborative partnership in an online course where the instructor uses a learning-community approach to foster learning. Using mixed methods, we examine the instructor-student collaborative partnership from the participation frequency, engagement move, and participant perception perspectives. Results show that the instructor and students not only actively participate in learning, instruction, and social environment building processes, but also maintain mutual interactions, communications, and actions to construct knowledge, to design and facilitate discussions, and to build a social learning environment. In addition, most participants perceive a sense of an online learning community in this online course. Based on the results, we provide theoretical, analytical, and pedagogical implications to advance the theory, analysis, and practice of the instructor-student collaborative partnership.

Abstract

Multimedia learning scenarios in which a picture is the main focus often use combinations of verbal and visual cueing. Based on models of picture processing and multimedia learning, the present study examined the effect of verbal and visual cueing on two basic aspects of pictorial learning: retention and localization of pictorial elements. Videos of three paintings were presented with verbal cueing (naming of pictorial elements), either alone or in combination with visual frames (explicit cues) or zoom-ins (implicit cues), in a 2 × 3 × 3 mixed design (n = 86) with the factors verbal cueing (uncued vs. cued elements, within-subjects) × visual cueing (no vs. explicit vs. implicit, between-subjects) × film (Mantegna vs. Rubens vs. Marsh, within-subjects). The three films were used to check whether our results are generalizable across different pictorial contents. The retention of pictorial elements was measured by open questions, and the localization of the pictorial elements was measured by asking the participants to place picture snippets at the correct location on an area representing the dimensions of the respective painting. The combination of verbal and visual cueing increased the difference between the cued and the less well retained uncued elements and compensated a disadvantage of verbal cueing for localization performance. This was compensated by both types of visual cueing. Regarding retention and localization, explicit and implicit cueing were equally effective. The study provides a differentiated insight into the interplay of verbal and visual cueing regarding cognitive processing in multimedia learning scenarios in which pictures are the main learning focus.

Abstract

Training researchers represents a substantially deeply international activity for higher education, and yet the transition into independence, a critical aim of doctoral education, remains a challenge for both supervisors and doctoral students, especially those from different cultural backgrounds. Interactions between Chinese doctoral students and their supervisors at Dutch universities exemplify the challenges in such an intercultural context. Interviews with 21 Chinese doctoral students and 16 supervisors from three Dutch universities reveal three potential challenges to fostering independence: (1) misalignment in supervisors’ and students’ conceptualizations of independence due to implicit diversity; (2) misalignment between supervisory support and students’ zone of proximal development (ZPD) of independence, as derived from the broader ZPD concept, especially in the first year of the doctoral study; and (3) a gap between supervisors’ interpretation of students’ visible learning behavior and students’ actual concerns. We provide a rich description of these hidden challenges and conclude with a framework outlining the relationships among the three layers of challenges. In so doing, we provide detailed information and a practical tool for supervisors to increase students’ awareness and skills, accurately diagnose students’ ZPD, recognize and reduce any potential misalignments in time, and thereby support students’ transition into independence. We conclude by discussing the practical and theoretical implications of our findings for supervisors and students in other intercultural contexts to reflect on their own practices and explore new ways of promoting international students’ transition into independence.

Abstract

Research on productive failure suggests that attempting to solve a problem prior to instruction facilitates conceptual understanding compared to receiving instruction prior to problem solving. The assumptions are that during the problem-solving phase, students activate their prior knowledge, become aware of their knowledge gaps, and discover deep features of the target content, which prepares them to better process the subsequent instruction. Unclear is whether this effect results from merely changing the order of the learning phases (i.e., instruction or problem solving first) or from additional features, such as presenting problem-solving material in the form of cases that differ in one feature at a time. Contrasting such cases may highlight the deep features and provide grounded feedback to students’ problem-solving attempts. In addition, the effect of the order of instruction and problem solving on procedural fluency is still unclear. The present experiment (N = 181, mean age = 14.53) investigated in a 2 × 2 design the effects of order (instruction or problem solving first) and of contrasting cases in the problem-solving material (yes/no) on conceptual understanding and procedural fluency. Additionally, the quality and quantity of students’ solution attempts from the problem-solving phase were coded. Regarding the learning outcomes, the ANOVA results suggest that for procedural fluency instruction prior to problem solving was more beneficial than problem solving prior to instruction. Merely delaying instruction did not increase conceptual understanding. The contrasting cases did not affect the quality of solution attempts, nor the posttest results. As expected, students who received instruction first generated fewer, but higher-quality solution attempts.

.

Volume 28, Issue 4, June 2020, Page 383-384
.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

.

Abstract

This paper proposes the Learning in Embodied Activity Framework (LEAF) which aims to synthesize across individual and sociocultural theories of learning to provide a more robust account of how the body plays a role in collaborative learning, particularly when students are learning about a collective phenomenon where coordination between and across students is important to their learning. To demonstrate the uses and limits of LEAF, we apply it to data from several iterations of the Science through Technology Enhanced Play (STEP) project. This project involved first and second-grade students using an embodied, mixed-reality simulation to learn about the particulate nature of matter. We use data from this project to demonstrate the ways in which students’ embodied actions serve as a resource in understanding their embodied activity both individually and collectively. We demonstrate how both dimensions provide insight into student cognition and learning. Supported by this analysis, we present LEAF as a useful tool to help researchers, designers, and instructors thoughtfully design collective activities.

Abstract

Over the last decade, there has been a renewed interest in capturing twenty-first century skills using new data collection tools. In this article, we leverage an existing dataset where electrodermal activity (EDA) was used to identify markers of productive collaboration. The data came from 42 pairs of participants (N = 84) who had no coding experience and were asked to program a robot to solve a variety of mazes. Because little is known on how physiological synchrony relates to collaborative learning, we explored four different measures of synchrony: Signal Matching (SM), Instantaneous Derivative Matching (IDM), Directional Agreement (DA) and Pearson’s Correlation (PC). Overall, we found PC to be positively associated with learning gains (r = 0.35) and DA with collaboration quality (r = 0.3). To gain further insights into these results, we also qualitatively analyzed two groups and identified situations with high or low physiological synchrony. We observed higher synchrony values when members of a productive group reacted to an external event (e.g., following instructions, receiving a hint), oscillations when they were watching a video or interacting with each other, and lower values when they were programming and / or seem to be confused. Based on these results, we developed a new measure of collaboration using electrodermal data: we computed the number of cycles between low and high synchronization. We found this measure to be significantly correlated with collaboration quality (r = 0.57) and learning gains (r = 0.47). This measure was not significantly correlated with the measures of physiological synchrony mentioned above, suggesting that it is capturing a different construct. We compare those results with prior studies and discuss implications for measuring collaborative process through physiological sensors.

Abstract

Scripting computer-supported collaborative learning has been shown to greatly enhance learning, but is often criticized for hindering learners’ agency and thus undermining learners’ motivation. Beyond that, what makes some CSCL scripts particularly effective for learning is still a conundrum. This meta-analysis synthesizes the results of 53 primary studies that experimentally compared the effect of learning with a CSCL script to unguided collaborative learning on at least one of the variables motivation, domain learning, and collaboration skills. Overall, 5616 learners enrolled in K-12, higher education, or professional development participated in the included studies. The results of a random-effects meta-analysis show that learning with CSCL scripts leads to a non-significant positive effect on motivation (Hedges’ g = 0.13), a small positive effect (Hedges’ g = 0.24) on domain learning and a medium positive effect (Hedges’ g = 0.72) on collaboration skills. Additionally, the meta-analysis shows how scaffolding single particular collaborative activities and scaffolding a combination of collaborative activities affects the effectiveness of CSCL scripts and that synergistic or differentiated scaffolding is hard to achieve. This meta-analysis offers the first counterevidence against the widespread criticism that CSCL scripts have negative motivational effects. Furthermore, the findings can be taken as evidence for the robustness of the positive effects on domain learning and collaboration skills.

Abstract

This research examines small group collaboration on the Augmented Reality (AR) Sandbox, an interactive, real-time topographical simulator that provides a color layer of augmentation showing depths and height, contour lines, and hydrology vis-a-vis the terrain of sand in a box. Prior research has focused on AR Sandbox activity designs, outcome measures, and user’s perceptions of different usability functions. No research to date has examined the situated processes by which groups engage in CSCL activities on the AR Sandbox as they participate in authentic forms of topographical studies. Taking a dialogic stance to examine CSCL using AR, in this study we draw on previous scholarship about distributed spatial sensemaking to analyze the way groups interact over material, social, and activity contexts. Based on an Interaction Analysis methodology, our findings point to the different resources that are coordinated with the use of the AR Sandbox; the different ways that turn-taking during distributed spatial sensemaking occurs; and the intricacy and speed by which multimodal resources are used to advance spatial thinking. The implications of this research broaden views of distributed spatial sensemaking, provide novel methodological tools to examine this phenomena, and suggest different levels of analysis and expectations for studies on the AR Sandbox.