ETR&D

Predicting the level of student learning engagement in online learning is crucial for student success, especially for asynchronous courses. While digital traces can track students’ activity on the platform and help to measure the engagement level, they could provide contradictory results, so it is crucial to incorporate complementary methods which can triangulate the findings obtained from digital traces. This study aimed to develop and validate a model to determine the level of learning engagement in adult learners on an asynchronous online platform using a mixed-method approach. Data from digital traces, surveys, and interviews were combined. The study involved 2234 students and employed Extreme Gradient Boosting and Logistic Regression with L2 regularisation models to predict the level of engagement. The Extreme Gradient Boosting model more accurately predicted students in the low engagement group, providing crucial support for potentially vulnerable students. The number of finished homework assignments and attempts were found to increase the probability of high engagement. The diversity of activities, such as access to text materials, played a pivotal role in sustaining engagement. Interviews corroborated these results, suggesting the model effectively reflects engagement levels. The article discusses implications for constructing similar models in future research.

The presence of on-screen instructors in educational videos, as well as the contextual conditions surrounding their use, constitutes a critical aspect of instructional video design. Variables such as the type of instructor – whether a human presenter or a pedagogical agent – and the characteristics of the knowledge type affect learning outcomes. However, the literature remains inconclusive regarding how the presence and presentation style of on-screen instructors influence learning outcomes across different knowledge types. Therefore, this study investigates the impact of an instructor’s presence in educational videos on learning outcomes, mental effort, and visual attention allocation, with a focus on the knowledge domain. A three-by-two between-subjects factorial design was employed, with video type (no on-screen instructor, human instructor, animated pedagogical agent) and knowledge type (declarative, procedural) as the independent variables. A total of 160 university students participated in the study. Results indicated that instructor presence influenced retention and visual attention allocation depending on the knowledge domain. Procedural knowledge videos led to higher transfer scores and mental effort than declarative ones. Importantly, however, the presence of an on-screen instructor – whether human or a pedagogical agent – did not produce differences in mental effort or learning transfer. Both human and animated pedagogical agent drew learners’ visual attention, potentially dividing it between the instructor and the learning content, whereas videos without instructors directed visual attention more exclusively toward the content itself. These findings highlight the importance of knowledge type in determining the effectiveness of on-screen instructors, suggesting pedagogical agents as viable alternatives to human instructors.

This paper details the design and development of MathFlowLens, a visualization tool that illustrates students’ procedural pathways in algebraic problem solving and provides valuable insights into various mathematical strategies they use. MathFlowLens was built using the middle-school student (N = 1,649) log data from a gamified learning platform, From Here To There! (FH2T), and was developed in two phases. First, by using pathfinding algorithms, we identified four distinct types of students problem solving pathways in the platform: optimal, suboptimal, dead-end, and incomplete pathways. Second, we created sequential network visualizations based on the identified classifications to present these distinct procedural pathways. Furthermore, we tested the applicability of this tool by examining the relations between the identified classifications and students’ performance on a posttest assessing three facets of algebraic knowledge: conceptual knowledge, procedural knowledge, and procedural flexibility. To examine the relations with algebraic knowledge, we focused on the subset of students who completed both the pre- and post-test (N = 778). The results indicated that students who took dead-end pathways more frequently, which we posited as exploratory behavior, had higher conceptual and procedural knowledge scores than those who did not. This finding highlights the importance of fostering the exploration of multiple procedural pathways, regardless of failure, to bolster the acquisition of algebraic knowledge. This study demonstrates that MathFlowLens, a novel method for visualizing students’ solution pathways, can provide valuable insights into their solution strategies and mathematical problem solving processes.

Despite the potential benefits offered by GenAI technologies to provide innovative solutions to address distinct challenges faced by working adult learners (ALs) in higher education and beyond, there is limited understanding of how best to structure AI-powered learning for this population while ensuring their distinct needs and perspectives are considered. Hence, this study aimed to determine what curriculum and student-AI interaction would be required by situating ALs’ views. Through analyzing 48 e-portfolios and in-depth interviews with 20 ALs from diverse educational and professional backgrounds, the study found that ALs perceived content mastery and developing a lifelong habit of learning as the optimal learning goals for AI-powered learning. AI-powered learning can be facilitated through personalized mastery-based learning and collaborative performance-based tasks, in tandem with scenario-based assessment, unobtrusive gamified assessment, and competency-based assessment. Along this line, AL articulated various necessary supports to foster AL-AI interactions. While AL identified metacognition and developing diverse and high-quality questions as crucial to support AL-AI cognitive interaction, they also highlighted that building ethical AL-AI relationships is important for enhancing AL-AI socio-emotional interaction. In addition, AL perceived immersive game-based platforms and interactive interfaces could serve as effective mediums for enhancing student-AI interactions. These findings can provide a more comprehensive understanding of AI-powered adult learning and implications for the design of educational AI, as well as instructional design to improve the educational experience for ALs.

This study explores the interactive characteristics of learning objects used in online courses and design intent of instructional designers. The study adopts the "Window of Interaction" (WoI) framework, drawnfrom Human–Computer Interaction (HCI), to critically examine the interactive characteristics of learning objects in the context of the designers’ intent and learning goals. This study provides research-based evidence to document: (1) the interactive characteristics of learning objects used in online courses; (2) designers’ intent and its manifestation in the learning objects they have designed; and (3) the connection between learning goals and the interactive characteristics of learning objects. The application of the WoI framework allowed us to identify the link between the interactive features of the design objects and the design intent guided by specific learning goals. With more advanced technologies, such as various AI-driven tools, the analysis of interactive features of technologies and learning objects becomes critical for designing more intentional learning experiences.

Computational Thinking (CT) capabilities are crucial for students’ future development. As a pivotal mode of thought, CT extends beyond mere programming skills, representing a methodology and strategy for problem-solving that empowers students to address complex challenges across diverse domains. In this context, this study aims to investigate the impact of a teaching strategy combining Project-Based Learning with Pair Programming Instructional Strategy (PBL-PPIS) on high school students’ CT capabilities and metacognitive skills. Conducted in a public high school in H City, Central China, this quasi-experimental design spanned one academic term and involved 90 first-year high school students aged 14 to 16. These students were divided into an experimental group and a control group, with the former utilizing the PBL-PPIS strategy and the latter adhering to conventional Project-Based Learning methods (PBL). To comprehensively assess the impact, this study utilized specialized scales for Computational Thinking and metacognitive abilities, and employed detailed analyses through paired sample t-tests and univariate ANCOVA. Through pre- and post-experiment surveys, we analyzed and compared the performance differences in CT and metacognitive skills between the two groups. The findings indicate significant enhancements in the experimental group across the five core competencies of Computational Thinking (Creativity, Algorithmic Thinking, Cooperativity, Critical Thinking, Problem Solving) and in their metacognitive abilities (planning, monitoring, evaluating). These results validate the effectiveness of the PBL-PPIS strategy in integrating the advantages of project-based learning and pair programming, underscoring its significant role in enhancing students’ CT and metacognitive abilities. This study contributes novel insights to the field of educational practice, offering fresh inspiration and direction for educators in designing and implementing programming education strategies.

The purpose of this study was to examine the effects and predictive power of the Diagnostic Assessment and Achievement of College Skills (DAACS) on student success. DAACS is an open-source diagnostic assessment tool designed to measure newly enrolled college students’ reading, writing, mathematics, and self-regulated learning skills, and to provide individualized feedback and learning resources that students can use to become better prepared for college. A randomized control trial was performed at two online colleges (n = 23,467) to test the effects of DAACS on credit acquisition and retention. The results indicate an overall null effect of treatment, but post hoc analyses reveal two important findings: 1) Students who not only received the assessment results but also accessed the feedback were significantly more likely to earn credits and be retained for a second term than students who only accessed the assessment results; 2) some students who only accessed the assessment results without reading the feedback, particularly those with low scores on the assessments, low self-efficacy, or high test anxiety, had worse outcomes than the control group. We speculate that feedback mitigates the potentially negative effects of testing on student success. In addition, an examination of the predictive power of DAACS indicated that DAACS data significantly strengthen predictions of academic outcomes.

The education sector is undergoing a rapid digital transformation, accelerated by the challenges posed by COVID-19. The pandemic has disrupted traditional teaching methods that prompt teachers to explore new approaches, such as the use of serious games, to keep learners engaged and learning. Serious games have proved to be an effective solution for bridging the gaps in distance learning and meeting the expectations of the new generation of learners. However, there are still obstacles to their adoption and implementation, particularly with teachers who have difficulty selecting games that suit the specific needs of their students. This paper presents the results of a study aimed at designing and developing a tool to help teachers better understand serious games, their components, and how they work, in order to integrate them effectively into their teaching practices. For this study, we used an approach that combines the unified version of the adoption and use of technologies (Venkatesh et al., 2003), the Jakob Nielsen System Acceptability Model (Nielsen, 1994b), and the analysis method of structure, interface, and use (Bouroumane et al., 2022). We tested our tool through two studies. One assessed its use in a real context, and the other utilized a questionnaire to evaluate its usefulness, usability, and acceptability. According to 80% of users, this tool is extremely useful for understanding the pedagogical dimensions of serious games, which can significantly and positively impact their adoption in education.