ETR&D

Developing digital materials for learning purposes, or e-learning materials, is not as straightforward as it may seem. The traditional approach is multimedia instruction, the presentation of both pictures and words in an e-learning format to foster learning. However, multimedia instruction, to develop meaningful learning, needs to support the active processing of the learner who engages with those materials. The Cognitive Theory of Multimedia Learning (CTML) informs the development of effective multimedia e-learning materials. However, given the dearth of CTML studies directed at students of diverse and underrepresented populations, such as d/Deaf and hard-of-hearing students (DHH), it is unclear as to the extent that CTML principles may inform effective multimedia instruction for this group. We conducted a mixed methods study in which we recruited DHH students to view an asynchronous, online multimedia tutorial covering a general chemistry concept and complete a brief questionnaire that inquired about their e-learning perspectives. Students characterized their use of the tutorial’s features as broadly serving three different functions and suggested additional improvements to the tutorial. Within the context of both these self-described behaviors and feedback, findings suggest the CTML principle of segmenting may have strong applicability for the e-learning needs of this student population while the principle of redundancy likely conflicts with those needs. Additional feedback more broadly focused on the tutorial’s formatting and visuals suggests other CTML principles may be important to consider for multimedia designed for DHH students.

Computational thinking (CT) is a core skill for students in the digital age, which refers to the use of fundamental concepts of computer science to analyze and solve real-world problems. However, it is a challenging task to accurately measure the CT levels of the students. The goal of this study is to explore a new CT assessment paradigm that combines a data-driven approach with evidence-based reasoning, and to verify the reliability and validity of this paradigm through an empirical study. Specifically, we examined middle school students’ CT by adopting an Evidence-Centered Design (ECD) approach. We first developed a CT evaluation framework based on ECD which included student, task, and evidence models. We then created several simulation-based tasks to gather evidence of CT levels; with these tasks, we extracted the observable variables in ECD model from clickstream data during the students’ answer process. Finally, a Bayesian network model was used for CT level prediction. Findings suggest that students’ clickstream data provide fine-grained, time-variant information on their interactions with tasks, thus promising more objective and richer insight into the performances of students with different CT levels. This study provides a paradigm reference for future CT measurement studies and proposes ideas for using process data in teaching to promote students’ CT development.

This paper reports on a study aiming at examining whether online argumentation enhance the teacher candidates’ scientific habits of mind (SHOM). The study was carried out as a pre-experimental design including pre and post-test. Scientific Habits of Mind scale, interviews and written arguments were used as data collection tools. Google Classroom, sentence openers, prompt questions and videos were used to facilitate the online argumentation process. Five different SSIs were given as discussion and argumentation tasks for the participants who worked in online groups. The results indicated that, the SHOM scores of the teacher candidates were slightly decreased while the argumentation skills were significantly increased. In addition, there was a medium, positive correlation between the levels of scientific habits of mind and levels of argumentation skills. Participants’ perspectives also showed the contributions of virtual argumentation strategy implemented in this study. Sentence openers, prompt questions, videos and feedback were the prominent factors shaped the online argumentation process. We believe that the findings of this study would assist in future design and implementation of online tools for enhancing argumentation.

Problem-posing has been regarded as a potential method to engage students in deeper thinking; however, without sufficient supports, most students could encounter difficulties in posing quality problems. In this study, a concept mapping-based online cooperative problem-posing (CM-OCPP) approach is proposed to guide students to complete problem-posing tasks in online collaborative learning contexts. To verify the effectiveness of the proposed approach, a quasi-experiment was conducted in a natural science class at a junior high school with two classes of seventh graders. Among them, one class with 29 students was assigned as the experimental group conducting the CM-OCPP approach; the other class with 29 students was the control group conducting the conventional online cooperative problem-posing (C-OCPP) approach. The research findings showed that the CM-OCPP group had better learning achievement, problem-solving skills, and problem-posing quality in terms of topic relevance and depth of questions than the C-OCPP group. The interactive pattern analysis results further showed that, with the guidance of the CM-OCPP approach, the students engaged more in knowledge sharing, discussion, knowledge co-construction, and diverse thinking.

In science education, the abstraction and complexity of scientific concepts are usually stumbling blocks that prevent students from learning science. Recently, augmented reality (AR) has offered transformative potential to support scientific concept learning by visualizing scientific phenomena and enhancing students' experiences. However, the lack of appropriate pedagogical scaffolds might not ensure effective learning in the AR learning environment (ARLE). In this study, we developed an AR-based learning tool (PeachBlossom) to support students' scientific concept learning and integrated the concept map strategy into AR learning activities. We conducted a quasi-experiment to examine the educational effectiveness of the concept map strategy on students' mental models and cognitive load in an ARLE. Eighty-five seventh graders (aged 12–14) from Central China were assigned into two groups (AR and AR with a concept map [ARCM]). The results showed that when considering students' prior mental models, the positive effect of the concept map strategy was found only in students with low and medium levels of prior mental models. In addition, the concept map strategy reduced students' mental effort but did not significantly affect students' mental load. This study emphasises the importance of considering students' prior mental models when implementing the concept map strategy in ARLEs.

Conversational agents have been designed to improve instruction quality and support student learning. In addition to their instructional use, they can be incorporated into assessment—conversation-based assessment (CBA). This study primarily introduces a CBA with selected-response and constructed-response tests as a formative assessment tool for higher education students, while also reporting student attitudes. CBA was designed using Rasa and deployed to Google Chat for students in two sections of an undergraduate-level course to use. Results show that CBA with both formats produces high standard performance measures and confidence scores for each student response. Most students reported positive attitudes toward CBA via the survey. Overall, accurate dialogue moves within CBA underscore its effectiveness in measuring higher education students’ knowledge and skill, while positive student attitudes toward CBA indicate the promise of conversational agents to enhance student assessment experiences by providing a more interactive assessment environment.

In the context of process-oriented writing instruction, the significance of engaging students in draft revision is widely acknowledged (McGarrell and Verbeem, ELT Journal 61:228–236, 2007). Nevertheless, L2 learners often exhibit limited motivation for writing, leading to inadequate revision efforts. This quasi-experimental study investigates the use and efficacy of technology-enhanced peer feedback workshops (PFWs) in comparison to traditional teacher oral feedback (TOF) in promoting student motivation for draft revision and revision quality. Over a 10-week academic English course, 18 EFL business freshmen received TOF for 3 writing tasks while they participated in PFWs facilitated by PeerMark for another 3 writing tasks in the first and second halves of the term respectively. Analysis of survey responses, interviews, peer feedback, students’ written works, and the teacher’s field notes reveals that PeerMark-based PFWs were well-received by students and had a positive impact on their motivation for draft revision and revision quality. Implications for pedagogical practices are discussed.

This paper explores the potential of new, technology enriched teaching simulations (Simulations) to support learning. Using the principles of evidence-centered design (ECD; Mislevy et al., Measurement: Interdisciplinary Research and Perspectives. 1:3–62, 2003), we provide a roadmap for the design of: (a) simulation-based assessments (SBAs) and (b) instruction-focused Simulations, and argue that the ECD framework can foster coherence between an instructional simulation and the larger educational efforts it is designed to support. This approach also helps ensure that Simulations target intended cognitions that might not otherwise be developed. Using teacher preparation as an example, we discuss how Simulations can be used to create focused approximations of practice opportunities that are tightly controlled and that target difficult to capture aspects of teaching, including responsiveness to student ideas through teacher noticing. Using our own work developing Simulations to foster teacher noticing skills, the paper outlines the stages of an ECD approach to Simulation design and also the role of various stakeholders in development processes. We conclude by highlighting the affordances of incorporating Simulations into educational efforts, both as instructional and assessment tools, and identify areas in need of further study.

Gesture recognition can create an interactive environment in which to train children to control their thoughts, gestures, and body postures when performing learning activities. This study integrated gesture recognition and a memory strategy to develop a physical learning activity for preschoolers to improve their English learning performance, motor skills (namely, stability and movement), and executive functions (namely, working memory). The study selected 67 kindergarten level 3 preschoolers. They were divided into either the experimental group (gesture recognition with memory strategy) or the control group (traditional activity learning with memory strategy). The experiment was conducted over a period of six stages for a total duration of 360 min. The results showed that the participants who used gesture recognition with the memory strategy demonstrated better learning performance, motor skills, and executive functions, and the statistics showed a significant deviation between the two approaches. With gesture recognition, the instructor enhanced the learning process of the children’s physical motions and cognitive knowledge by strengthening the connection between life experiences and specific tasks, and the children’s executive functions also improved. The research contribution of this study is a pedagogy manuscript for teachers who want to create a physical activity with gesture technology for preschoolers.

The current monument guide teaching has proven to have many challenges, as the learning effectiveness may be reduced by external interference in the real guide activities, and online approaches may cause the problem of insufficient interaction. This study integrated role-playing and multi-dimensional scaffolding theories to develop a remote scaffolding-based educational game that involved a role-playing mechanism, in which guides and learners play the roles of historical characters and participate in historical events in an authentic game environment, with the aim of improving the problems of existing monument teaching and of enhancing the motivation to visit. The results showed that this learning model has advantages in flow and motivation compared to the traditional online guide. It was demonstrated that the multi-dimensional scaffolding formed by the guides, the game environment, and the peers could keep the learners quite engaged in the game, while the dynamic scaffolding of the guides could also be effective and even keep the learners interested in the monument after the activity.

Self-regulated learning (SRL) is an approach to learning which aims to improve learners' learning outcomes. In the SRL cycle, the quality of students' reflections is a critical factor in SRL performance that can improve learning outcomes. The feedback provided by teachers often has a significant impact on the quality of students' reflections. However, research has shown that teachers are often confronted with many students and so are unable to give individual feedback. To address this problem, this study proposed chatbot-assisted SRL, which provides personal and immediate feedback during the SRL process to promote the quality of students' reflection. This study applied this approach in the "Website Design" unit of an Introduction to Multimedia course. Two classes of second-year university students were the participants. One class with 23 students was the experimental group using the chatbot-assisted SRL mode, whereas the other class with 24 students was the control group using the conventional SRL mode. The experiment results showed that the proposed approach improved the students' performance, motivation, SRL, reflection, and meta-cognition tendency more than the conventional SRL approach.

Engagement in digital reading assessments has been underexplored in spite of its importance for assessment and instructional design. In this study, we used different engagement indicators in two ePIRLS tasks (n1 = 14,324, n2 = 15,026) to identify engagement profiles, and interpreted them using the three-dimensional model of engagement: affective, behavioral, and cognitive. In both tasks, we found profiles representing high, moderate, and low levels of engagement, as well as profiles with high behavioral-cognitive engagement but low affective engagement, high cognitive engagement, and high behavioral but low cognitive engagement. The students who took both tasks (n3 = 2658) were likely to be grouped into similar profiles in both of them, demonstrating the profiles’ stability and generalizability. These profiles significantly predicted students’ performance, self-efficacy, and reading engagement outside of school (all p < .001), with especially strong effects on performance (η12 = .22, η22 = .17). While most groups’ performance was as expected, high behavioral-cognitive and low affective engagement students had the lowest reading engagement outside of school. This study reveals the unique reading engagement patterns that readers exhibit in digital contexts, contributing to the literature on reading engagement. From the perspective of large-scale assessments, the study demonstrates the complex relationship between different forms of engagement and performance, suggesting that response time alone might not be enough to identify disengaged participants.

This pilot study aimed to evaluate the development of writing skills among Master of Public Health (MPH) students after completing a public health writing assignment designed with a transparent framework. The assignment incorporated Lettersmith, a free educational technology tool that supports metacognition and the writing process. A secondary aim was to assess students′ perceptions  of the usefulness of Lettersmith in supporting their writing process. After using Lettersmith to complete their assignments, students reported feeling more confident in their writing skills, as well as in their ability to write more concisely and express ideas clearly. Most students indicated that the assignment expectations were clear and that Lettersmith was helpful during the writing process. A transparent, designed-focused approach to writing, paired  with the Lettersmith tool, may support students in becoming competent writers by enhancing their awareness of the writing process. Lettersmith can be applied to writing assignments in fields beyond public health.

Metacognition, which involves the deliberate awareness and analysis of one’s own learning and thought processes, has gained significant traction among educational researchers. The burgeoning volume of metacognition studies underscores the importance of examining its current status and evolving trends. Leveraging topic modeling and bibliometrics on a dataset comprising 2568 papers spanning from 2000 to 2023, this study seeks to address questions like “What are the prevailing research themes in metacognition?” and “How has the level of research attention to these themes evolved over time?” This study also scrutinizes major journals, countries/regions, academic institutions, and collaborative networks, presenting a visual representation of their interconnections. Considering the analyses conducted, this study proffers several recommendations for the future of metacognition research. Firstly, it suggests the integration of metacognitive instruction, assessment, and feedback mechanisms into various educational domains, encompassing design, healthcare, language, teacher training, as well as special and early childhood education. Secondly, it advocates for the exploration and utilization of diverse metacognitive instructional strategies and analytics technologies to effectively bolster students’ metacognitive processes. Lastly, it underscores the significance of interdisciplinary collaborations between metacognition experts, educators, psychologists, computer scientists, and data scientists. Such collaborative efforts are envisioned to harness the potentials of big data and learning analytics technologies to inform pedagogical practices that nurture metacognitive skills. This study offers a comprehensive overview of metacognition research in the realm of education, shedding light on emerging trends in metacognitive instructional practices and providing valuable insights for charting the course of future investigations in this field.

The Flipped Classroom Model (FCM) has gained widespread acceptance in higher education as an effective pedagogical strategy. Despite its success, the FCM still faces persistent concerns, including a lack of personalized interaction, limited application to introductory courses, and insufficient analysis of the learning process. The integration of generative artificial intelligence (AI) with the Metaverse may significantly enrich and revitalize FCM by introducing affordances that support personalized and immersive learning experiences. This study examined the potential of FCM enhanced by an AI-Generated Metaverse (AIGM) to improve learning outcomes in higher education. Through introducing an AI-powered virtual tutor, this study aimed to address existing concerns and shed light on the nature of knowledge-building interactions between students and the AI tutor. The study involved 94 university graduates who participated in an experimental FCM session. Data on students’ learning performance were collected through pre/posttests, pre-class video engagement, and analyzed dialogic interactions between students and the AI virtual tutor. The findings indicated that the AIGM-FCM positively influenced learners’ academic performance. Interaction patterns revealed that students predominantly engaged with the AI virtual tutor at the onset and at the end of their learning sessions, primarily for the purpose of asking questions. However, the educational dialogues suggested that the depth of collaborative knowledge construction between students and the AI tutor remained at a basic level. Notably, students with lower academic performance were more likely to initiate inquiry-based dialogues with the AI tutor compared to their higher-achieving peers exhibiting a more focusing-oriented approach. In light of these discussions, the implications of this study are also deliberated.

Flipped learning has been recognized as a promising pedagogical model for methodological progress in educational environments. Studies have shown the effectiveness of flipped learning when used with appropriate educational technologies and learning strategies. On the other hand, comprehensive guidance is required to combine this pedagogical model with new technologies. The current study investigates how discussion tools integrated into online flipped learning environments affect students’ engagement and self-regulation. Participants targeted for this study consisted of 121 first-year students randomly assigned to one of four experimental conditions formed by a 2 (synchronous online discussions vs. asynchronous online discussions) × 2 (online flipped learning vs. online learning) factorial pretest/post-test design. The results revealed that there was a significant interaction effect between the online flipped learning approach and synchronous online discussions on behavioral and cognitive engagement. While no interaction effects on self-regulation between the learning approach and online discussions were found, the results revealed that the online flipped learning approach and synchronous online discussions separately increased self-regulation. The findings contribute to the online flipped learning approach by adding a new perspective, namely synchronous online discussions, and thereby proposing a new interactive learning experience for a more engaging and deeper exploration of online learning. The implications of designing an online flipped learning environment with synchronous online discussion tools are discussed.

Abstract

Professional vision (PV), which includes the ability to perceive and interpret classroom events, is important for classroom behaviour management, particularly in the training of student teachers. However, to date, few assessment systems that can assess student teachers’ professional vision by immersing them in a realistic classroom environment (presence) from their own perspective (first-person perspective) have been developed. To address this gap, this study employs a design-based research approach to develop an immersive virtual reality (IVR) technology-based professional vision assessment system (IVR-based PVAS). The research outlines the four-stage process through which the system was constructed to meet the requirements of professional vision assessment and address the core challenges of the current assessment approach. Twenty-four student teachers were enrolled as participants in an implementation case study in which their perceptions and interpretations of seven classroom events related to student misbehaviour were assessed via an IVR-based PVAS. The participants’ problems with perception and interpretation were diagnosed with multiple assessment indices. The benefits of the IVR-based PVAS are summarised, and recommendations are provided for the use of this assessment system in both instruction and research.

Abstract

Digital game-based learning (DGBL) has emerged as an effective strategy to enhance students’ learning effectiveness. Concept mapping, recognized as a valuable tool for knowledge construction, has been widely implemented in educational settings. However, research indicates challenges when integrating concept maps into games, particularly when the learning content is extensive or complex. In such cases, the use of concept maps may increase cognitive load and negatively impact learning effectiveness. Consequently, this study addressed the issues of reducing learners’ cognitive load and increasing their motivation within a DGBL context. To address these issues, this study developed a progressive concept map-based game approach. This approach integrates mathematical concepts with real-life situations and employs progressive concept maps to facilitate knowledge construction. The aim is to help students understand the relationships between mathematical concepts and guide them in solving learning tasks and achieving higher learning outcomes during gameplay. A quasi-experimental design was employed, with the experimental group using the progressive concept map-based game approach and the control group using a conventional concept map-based game approach. The results indicated that the experimental group significantly outperformed the control group in learning achievement, learning motivation, problem-solving tendencies, and self-efficacy. These findings offer valuable insights for future DGBL research, particularly in the development of progressive strategies.

Abstract

Collaboration scripts are widely employed in online collaborative learning to enhance student engagement and facilitate collaboration. However, the optimal level of scripting remains a subject of debate. This study aims to address this issue by designing and developing different types of collaborative scripts implemented through conversational agents and supported by WeChat. Utilizing interventional studies, we investigate the effects of these different collaboration scripts on student engagement during online collaborative learning. A total of 54 college students participates in the study, divided into six adaptable scripts teams, six maximal script team, and six minimal script team, with each team consisting of three students. Both quantitative and qualitative data are collected and meticulously analyzed. The results reveal that the maximal collaboration script significantly enhances cognitive interactions, whereas the minimal collaboration script fosters high-quality cognitive engagement. In terms of socio-emotional engagement, the adaptable collaboration script effectively promotes positive socio-emotional engagement, while the maximal collaboration script facilitates greater socio-emotional interactions. Furthermore, thematic analysis demonstrates that all three types of collaboration scripts support student engagement by providing time reminders, facilitating planning, clarifying ideas, and promoting task reflection. These findings have important implications for improving group learning engagement in online collaborative learning environments.

Abstract

This study explores the roles of students’ help-seeking profiles when seeking help from AI chatbots, specifically ChatGPT, in a digital game-based learning environment, Summon of Magicrystal. The study involved 102 middle school students who played an online game with the provision of ChatGPT and sought help from ChatGPT while solving physics problems. The results revealed that students’ help-seeking profiles, help-seeking threats, help-seeking avoidance, and instrumental help-seeking were positively correlated. Students’ instrumental help-seeking profile has a positive effect on game performance/engagement, while students’ avoidance help-seeking profile has a positive effect on the number of game attempts. The findings highlight the importance of students’ help-seeking profiles when considering designing AI-assisted game-based learning environments to better support students’ science learning.