ETR&D

The purpose of this study was to examine the effects and predictive power of the Diagnostic Assessment and Achievement of College Skills (DAACS) on student success. DAACS is an open-source diagnostic assessment tool designed to measure newly enrolled college students’ reading, writing, mathematics, and self-regulated learning skills, and to provide individualized feedback and learning resources that students can use to become better prepared for college. A randomized control trial was performed at two online colleges (n = 23,467) to test the effects of DAACS on credit acquisition and retention. The results indicate an overall null effect of treatment, but post hoc analyses reveal two important findings: 1) Students who not only received the assessment results but also accessed the feedback were significantly more likely to earn credits and be retained for a second term than students who only accessed the assessment results; 2) some students who only accessed the assessment results without reading the feedback, particularly those with low scores on the assessments, low self-efficacy, or high test anxiety, had worse outcomes than the control group. We speculate that feedback mitigates the potentially negative effects of testing on student success. In addition, an examination of the predictive power of DAACS indicated that DAACS data significantly strengthen predictions of academic outcomes.

The education sector is undergoing a rapid digital transformation, accelerated by the challenges posed by COVID-19. The pandemic has disrupted traditional teaching methods that prompt teachers to explore new approaches, such as the use of serious games, to keep learners engaged and learning. Serious games have proved to be an effective solution for bridging the gaps in distance learning and meeting the expectations of the new generation of learners. However, there are still obstacles to their adoption and implementation, particularly with teachers who have difficulty selecting games that suit the specific needs of their students. This paper presents the results of a study aimed at designing and developing a tool to help teachers better understand serious games, their components, and how they work, in order to integrate them effectively into their teaching practices. For this study, we used an approach that combines the unified version of the adoption and use of technologies (Venkatesh et al., 2003), the Jakob Nielsen System Acceptability Model (Nielsen, 1994b), and the analysis method of structure, interface, and use (Bouroumane et al., 2022). We tested our tool through two studies. One assessed its use in a real context, and the other utilized a questionnaire to evaluate its usefulness, usability, and acceptability. According to 80% of users, this tool is extremely useful for understanding the pedagogical dimensions of serious games, which can significantly and positively impact their adoption in education.

Computer programming is an essential, cross-disciplinary skill relevant to learners beyond merely computer scientists. This paper introduces parallel courseware, a novel approach to courseware design that contributes to adaptable learning in introductory programming education. Adaptability is achieved through the implementation of parallel, aligned courseware versions, designed along key customization dimensions, such as programming language and application domain, resulting in numerous courseware permutations to meet diverse student needs. These versions are delivered through LearnCoding, an online platform that enables seamless switching between permutations while maintaining a consistent structure and layout, thereby supporting personalized learning, facilitating comparative exploration of content, and enhancing skill transfer across contexts. The paper also reports findings from a study conducted over two offerings of a first-year programming course, exploring the impact of the proposed approach on students’ understanding of and interest in the learning material. Results indicate a positive student experience, suggesting promising implications for the design of adaptable courseware and the development of flexible e-learning platforms.

Generalist primary school computer science (CS) teachers are often reluctant to introduce CS activities that go beyond CS unplugged tasks. To address this challenge, we drew from constructive alignment principles to implement a new programming modality for primary school: the handwriting programming language (HPL). HPL brings programming activities closer to existing teaching practices by enabling students to write instructions on paper, take a picture, and have an agent execute them. HPL’s applicability in classrooms was investigated in two stages. First, 49 primary school teachers evaluated two alternative programming modalities—HPL and an equivalent paper-based Tangible Programming Language—using the technology acceptance model (TAM). As teachers preferred HPL, we then conducted a 3-session quasi-experimental study with 143 public school students (aged 9–10) to compare HPL’s acceptance (with the TAM) and learning outcomes (with a validated test) to the established Scratch programming language. The findings indicate that: (i) over 80% of teachers were willing to use HPL to teach CS in class, irrespective of gender or prior experience; (ii) HPL-students exhibited less trial-and-error behaviour than Scratch-students (fewer attempts, more time between attempts); (iii) students perceived HPL as positively as Scratch; (iv) HPL-students learned as much as Scratch-students. In conclusion, HPL is an accessible, accepted, and pedagogically meaningful means of teaching CS that is as efficient as Scratch to teach CS in primary school. HPL’s efficiency and acceptance by teachers and students suggests that handwriting-based programming languages may help shift primary school CS teaching practices, and make CS education more widespread, bringing us closer to CS for all.

The reading ability of hearing-impaired students is essential for their participation in mainstream society. However, previous studies have shown that they may encounter obstacles due to a lack of interest or limitations in reading communication systems. Augmented reality (AR) has been noted to provide immersive learning environments, collaborative assistance, and in-time resources for improving reading experience and motivation. While previous studies have developed mobile reading environments tailored for hearing-impaired students, the incorporation of pedagogical approaches within immersive AR reading environments remains unexplored. Accordingly, this study investigated the effects of immersive AR environments on hearing-impaired students’ reading with a quasi-experiment. Sixty-five hearing-impaired students’ reading achievement, perceptions, and behaviours were analysed with one-way analysis of covariance and lag sequential analysis. The result indicated that the immersive AR learning environment incorporating the DEEP reading strategy (i.e., Developing self-regulated reading, Experimental exploration, Express and creative construction, and Pluralism) improved hearing-impaired students’ reading achievement, perceptions, and behaviours. Students with the immersive AR reading strategy exhibited more interactive-oriented behaviours and high levels of cognitive attainment (e.g., experiential exploration, creative construction, and problem-solving). This study contributed to existing hearing-impaired teaching practices by revealing what essential behaviour teachers should consider and how to design an immersive AR learning environment.

The generalization of the cognitive theory of multimedia learning to immersive virtual reality (IVR) learning contexts has been increasingly examined; for example, there has been an exploration of redundancy effects on science learning using IVR technology. However, studies on how well multimedia principles such as redundancy work in IVR learning with humanities-related content has been limited. This work therefore aimed to explore the redundancy effects of immersive virtual literary reading on students’ humanities learning by evaluating their empathy, cognitive load, and reading comprehension with consideration of the moderating effects of their learning backgrounds. A between-subjects design with two modes of presentation format (auditory narrative only vs. auditory narrative and on-screen text) involved 70 higher education students who were randomly assigned to two groups, with 35 students in each condition. Results showed that the students’ learning background moderated their perceived empathy but not their cognitive load and reading performance when reading redundant virtual information. Redundancy design of instructional content in IVR learning environments was verified as reducing cognitive load. Moreover, reading an immersive virtual literary novel with redundant information design was likely not detrimental to the students’ higher-level reading comprehension performance, but this was not the case for their lower-level reading comprehension.

AI literacy is becoming increasingly popular in education, yet there has been limited focus on reviewing empirical research on AI literacy assessment. The purpose of this study was to synthesize existing empirical studies on AI literacy assessment, with the aim to understand how AI literacy has been assessed and to inform future AI literacy assessment development. This scoping review evaluates and synthesizes 36 studies on AI literacy assessment published between 2019 and 2024, involving assessment tools, forms of assessment, and reliability and validity evidence related to AI literacy assessment. The review proposes four aspects (i.e., knowledge of AI, AI ethics, affect towards AI, and use of AI) for assessing AI literacy. The results showed that (1) most research focused on assessing primary and secondary school students’ AI literacy; (2) most studies used questionnaires, followed by surveys; (3) most studies used computer-based tests, followed by paper-based tests; (4) most studies assessed participants’ AI knowledge, followed by AI ethics; and (5) only a few studies reported evidence of the reliability and effectiveness of their assessments. Based on the reviewed literature, this study develops an AI literacy framework for people of all ages and from all countries. The findings and directions for future research are also discussed.

There is a growing global interest in the inclusion of coding as early as preschool. However, most coding languages involve reading and writing, leaving non-writing children deprived of the opportunity to develop any kind of coding proficiency. Moreover, many preschool teachers lack the knowledge and self-efficacy necessary for teaching coding. This case study examines the subjective experiences of preschool teachers with no early coding knowledge in implementing the ScratchJr-based Coding as Another Language (CAL) curriculum. Following a four-hour professional development session, two preschool teachers taught coding to 26 children (M = 5.72, SD = 0.33) using the CAL curriculum over seven months. Thematic analysis of semi-structured interviews with teachers identified eight key themes, including initial CS-resistance to coding, pedagogical and cultural curriculum adjustments, overcoming resistance, learning with and from children, enhanced teacherstudent relationship, inter-syllabi learning opportunities, coding-driven community partnership, and potential for self-expression. Quantitative results showed significant improvement in children’s coding knowledge, positioning most at the Fluency stage of CAL proficiency. Results highlighted the effectiveness of the CAL curriculum in building teachers’ coding self-efficacy, despite its relatively brief duration. They further emphasize the importance of curriculum adaptability to diverse educational contexts. These findings support overcoming technology integration barriers in early childhood education. Additionally, the study suggests using coding as a tool for creative self-expression, highlighting its potential beyond technical skill-building. This direction may be further explored to promote emotional and personal growth in both teachers and children, contributing to individual well-being improvement while supporting the integration of STEM-related subject learning in formal curricula.

Digital distraction in education describes the interruption of learner’s concentration during academic tasks. With the increase in digital learning, there is a need for a review to synthesize research on digital distraction. This systematic review examined 26 articles using the DISCAR process (design, inclusion/exclusion, searching and screening, coding, analyzing/synthesizing, reporting) and was guided by the Technology-Personal-Environment (TPE) framework. The review examined causes and consequences, and strategies to prevent/reduce digital distraction. Measures used, modalities studied, and devices used in digital distraction research were also synthesized. Causes for digital distraction were technology distractors (51.95%), personal needs (37.66%), and instructional environment (10.39%) factors. Consequences for digital distraction included personal performance issues (66.67%), ineffective classroom instruction (23.33%), and problematic technology use (10%). Prevention strategies included classroom environment regulations (41.03%), technology controls (30.77%), and personal behavioral interventions (28.21%). The findings have implications for instructors, students, administrators, instructional designers and researchers. This systematic review adopted a multi-faceted approach to effectively mitigate digital distractions.

Good teaching requires a professional vision of the relevant dimensions of teaching quality and their interrelationships. For example, classroom management is necessary but insufficient for providing effective instructional support. Thus, teacher education should foster a multiperspective professional vision of these dimensions of teaching quality as a basis for implementing appropriate teaching actions. Research shows that professional vision can be promoted when preservice teachers analyze classroom videos. However, acquiring a multiperspective professional vision is more complex than a single perspective. Furthermore, preservice teachers have different entry levels and developmental trajectories. Individual learning requirements and the more complicated task demands can potentially be met through virtual learning environments and additional support tools implemented during video analysis. We used a video-based assessment with an open response format and investigated (1) the effect of a video-based virtual learning environment on promoting multiperspective professional vision in elementary science education and (2) the effect of additional support tools (modeling videos and prompts) implemented during video analysis. A quasi-experimental pre-post-follow-up study with 145 preservice teachers showed that a basic virtual learning environment improved participants’ multiperspective professional vision compared to an untreated control group in the short and long term. The additional support tools in the enriched virtual learning environment did increase preservice teachers’ performance even more while training professional vision but not in the post- and follow-up tests. Therefore, teacher education programs should consider the benefits of video-based virtual learning environments for an individualized promotion of professional vision. Further research on effective digital support tools is needed.

Hybrid Flexible (HyFlex) instruction offers a promising approach to enhancing flexibility and student engagement in undergraduate education. Yet, challenges related to technology, faculty readiness, and equity remain. This systematic review serves to explore current trends in the peer-reviewed literature on HyFlex learning between 2013 and 2023, specifically within the context of undergraduate education. The PRISMA principles were used as a guide to complete this review. Researchers conducted a broad search of HyFlex instruction research using five electronic databases. A total of 1,512 articles were screened as part of the systematic review. A total of 46 articles met the inclusion criteria. The results of the systematic review revealed that between 2013 and 2021 there were very few publications per year related to HyFlex instruction in undergraduate education. However, the number of publications increased significantly in 2022 and 2023. The results of the systematic review also revealed that research in HyFlex instruction in undergraduate education is a global and highly collaborative endeavor. In terms of major research trends, the systematic review also served to better understand the context in which the implementation of HyFlex instruction in undergraduate education was explored including the subject–matter of the instruction, number of participants, and research methodology. Of the 46 studies reviewed, the majority reported neutral findings, indicating that HyFlex instruction had neither a distinctly positive nor negative impact. Key findings from the research focused on HyFlex instruction design and learning strategies in undergraduate education are also discussed.

The research focus on teacher questions is justified by previous research emphasizing the essential role of questions in facilitating meaningful learning in science. Analysis of teacher questions has traditionally been based mainly on manual coding, which is extremely labour intensive. In this study we explore how both machine learning and large language models can be used for this purpose. Whereas machine learning approaches involve supervised training with extensive data, pre-trained large language models operate through prompt engineering. ​​The​ automatic speech recognition text outputs of 23 physics lessons on the same topic from 23 science teachers were analysed with variety of techniques. The results revealed that the large language model approaches improved with few-shot approaches compared to zero-shot ones. Furthermore, few shot approaches outperformed the supervised machine learning approaches, yet human- and hand-crafted approaches continue to demonstrate their relevance. ​​Implications for​ science teaching and learning are discussed.

Native language (L1) reading studies have established that text type, or genre, strongly influences reading comprehension, and narrative, or story-based, texts are easier to recall and understand than expository, or informational, texts, indicating that the comprehensibility of the content may differ depending on the genre in which it is presented. However, the effects of text genres on reading comprehension have rarely been the focus of target language (L2) reading mainly due to methodological difficulty because because L2 reading is a highly complex process that involves additional cognitive demands beyond those of L1 reading. For this problem, this investigation proposes a recent “knowledge structure (KS)” network analysis approach to visually describe and distinguish the reading processes and outcomes that may be triggered by the use of text genre in L2 reading. University mixed proficiency Korean English language learners (n = 616) were randomly assigned to one of 8 conditions that all involved a pre-reading task in L1 or L2 (as a sorting task), reading a text (either narrative or expository), then from memory a post-reading task in L1 or L2 (as a summary writing), and finally a comprehension posttest. All of the participants’ sorting and essay artifacts were converted into Pathfinder Networks, a graph-theoretic psychometric networks scaling measure, that were visually and statistically compared with each lesson text’s Pathfinder Networks they read. The findings have practical implications for L2 reading instruction. Narrative texts would be more beneficial to L2 readers who do have lower L2 proficiency because of its greater easy of processing, while expository texts would be more beneficial to L2 readers who have enough prior knowledge of to-be-learned content because of its tendency to integrate content with prior knowledge.

This paper describes the design principles and impact of an online asynchronous short course “Key Ideas in Mentoring Mathematics Teachers”, contributing to the professional development (PD) of prospective and practicing school-based mentors of mathematics teachers. The course was designed to empower mentors with knowledge about research informed practice and instil in them a welcoming stance towards mathematics education research. An Architecture of Online Engagement and a Vignette Activity Sequence approaches were employed in the design of the course, as means of supporting the participants to critically reflect on their teaching practices through engagement with topic-specific mathematics education research and on the implications for their mentoring practices. With a focus on the Vignette Activity Sequence (VAS) in particular, we discuss the potential impact and value of this approach to designing an online asynchronous professional development course in general, but also in the context of mentors of mathematics teachers. This paper’s contribution lies in the discussion of design principles of a PD course that successfully engage mentors with mathematics education research literature, while allowing them to reflect on their own practices and experiences.

This article presents the results of a multiple case study that explored the ways learning, design, and technology (LDT) scholars negotiate issues of social justice in their practice and research. Using Stake’s (2009) multiple case study, we examined the design practices of 4 scholars, looking across contexts to understand participants’ design practices, as well as the connections between those practices, their perspectives, and the local context. We found that attending to social justice requires a reflexive, reflective approach situated within understandings related to designer positionality and power. Design practices were employed to address justice-based tensions that emerged around project goals, perceived roles, and power dynamics. This study provides insight into the ways current scholars are taking up issues of social justice by engaging in a critical, responsive approach to design.

To promote deeper cognitive interactions and positive socio-emotional interactions among group members, thereby achieving high-quality collaborative outcomes, researchers have endeavored to develop conversational agents (CAs) that provide adaptive support to small groups. However, existing CAs for supporting collaborative learning struggled to integrate and analyze multi-source learning data throughout the collaborative learning process and to offer comprehensive and personalized scaffolding based on diagnostic results. To address these issues, this study designed a CA named CollaBot, based on the contingent teaching model, which has been used in the past to guide teachers on how to provide adaptive scaffolding to small groups. CollaBot integrates AI technologies, including retrieval-based models, generative AI models, and retrieval-augmented generation techniques, to offer adaptive cognitive, metacognitive, and social scaffolding to groups engaged in online collaborative learning. A randomized controlled experimental design was employed, recruiting 78 undergraduate students who were randomly divided into two groups: the experimental group (n = 39) utilized a co-writing platform with CollaBot, while the control group (n = 39) used the platform with task scripts. Results indicate that students supported by CollaBot demonstrated significantly better learning performance. In addition, both CollaBot and task scripts significantly enhanced students’ self-efficacy for writing. Furthermore, analysis of the interview data revealed both positive perceptions about CollaBot, such as aiding group members in regulating their own and the group’s learning processes and supporting the development of writing skills, as well as negative perceptions, including causing anxiety and providing ambiguous feedback. This research provides guidance for the design of CAs and offers insights into harnessing hybrid intelligence between teachers and GAI to support collaborative learning.

Badminton is one of the most popular student sports, but it is challenging to increase learning efficiency by observing learners using the naked eye without assistive tools. Therefore, this study proposes an auto-feedback badminton teaching app integrated with an auto-feedback-based WISER model. Learners could conduct self-learning with the functions of the badminton teaching app, including automatic grading, automatic feedback, and professional player demonstration videos. This study adopts a quasi-experimental design. The proposed App and model were applied in the experimental group, while the control group used traditional teaching and mobile devices with a video recording function. Both groups of learners learned serve and clear skills for 6 weeks each. The increase in post-test scores of the experimental group was significantly higher than that of the control group, though post-test scores of both groups were significantly higher than pre-test scores. The proposed method demonstrates its efficiency for self-learning, as confirmed by interviews. Future work can apply the integration of motion recognition and the auto-feedback-based WISER model across diverse educational disciplines to personalize students’ learning.

Experiential learning and online learning platforms are increasingly being recognized as important in contemporary vocational language education and research. However, vocational language education has been criticized for its lack of connection with current technological developments. To address this issue, the study aims to determine the effects of “Experience-Based Cyclical E-Curriculum Design” on learners’ vocational second language performance and language learning experiences. The study utilized a design-based research approach. Quantitative data from the Vocational Second Language Skills Assessment Test were analyzed using ANCOVA, and qualitative data from Reflection Form were analyzed using Strauss and Corbin’s coding technique. The results indicate that the experiential learning curriculum, developed using the E-Curriculum Design framework, had a positive impact on learners’ vocational second language performance and vocational second language learning experiences. Thus, this study suggests that E-Curriculum Design, when integrated with an online learning environment, can effectively support learners in constructing knowledge through practical experience.

This study aimed to examine writing performance, critical thinking tendency, and research writing reflection among postgraduates with different reading levels across different subject professional backgrounds. A problem-based flipped classroom learning environment was implemented to enhance graduates' research writing performance and perceptions. A total of 28 first-year postgraduates participated. The result revealed significant differences in research writing performance and critical thinking tendency between students with low reading levels (LRL) and those with high reading levels (HRL). Additionally, an interaction effect was observed between reading level and subject professional background on graduates' critical thinking tendencies. In the LRL group, the critical thinking tendency of students in this major is significantly higher than that of students in non-majors. Furthermore, Epistemic Network Analysis (ENA) indicated significant differences in research writing reflections between students with different reading levels and pre-test and post-test. However, no significant differences were found across different subjects professional backgrounds. These findings demonstrate that problem-based flipped classroom learning can improve research writing performance and perception. From the teaching perspective, it provides a reference for writing instruction.

Advances in educational research and technology provide educators and learning and development (L&D) specialists with powerful tools to innovate health professions education (HPE) and address the relentless growth of content information. Yet, traditional practices often hinder the critical appraisal and application of evidence-based educational methods. While systematic models ground the instructional design of learning experiences in research and theory, they are frequently perceived as too rigid, time consuming, and resource intensive for widespread adoption in clinical education settings. This article introduces an agile, evidence-informed approach to instructional design tailored for HPE. The Agile eVidence-Informed Design (AVIDesign) model was developed to streamline the design and continuous improvement of educational interventions. AVIDesign aligns with evidence-based medicine by offering a systematic yet flexible process for designing instruction, incorporating design sprints, transdisciplinary collaboration, and contextually relevant appraisal of research evidence. We present the core principles and practices of AVIDesign, including strategies to right-size instructional design initiatives, formulate targeted LICO (Learner, Intervention, Context, Outcome) questions, appraise evidence using QSR (Quality, Strength, Relevance) criteria, and evaluate outcomes through formative and summative assessments. The application of AVIDesign is illustrated through three real-world design projects in HPE involving curriculum development, branching scenarios, and ePortfolio systems. Each case highlights how AVIDesign promotes compatibility, trialability, and reduced complexity to facilitate adoption. Key lessons learned from the projects are distilled to inform future applications of AVIDesign and guide continued research on agile, evidence-informed instructional design practices in health professions education.