ETR&D

Interpersonal online interactions are key to digital learning pedagogies and student experiences. Researchers use learner log and text data collected by technologies that mediate learner interactions online to provide indicators about interpersonal interactions. However, analytical approaches used to derive these indicators face conceptual, methodological, and practical challenges. Existing analytical approaches are not well aligned with the theories of digital learning, lack rigor, and are not easily replicable. To address these challenges, we put forward a multi-level framework linking indicators of individual posting with group-level communication and emergent relational structures. We exemplify the use of the framework by analyzing twenty online and blended courses. Empirical insights demonstrate how indicators at these three levels relate to each other and to potential instructor decisions. Our conclusion highlights current gaps in the framework and the areas for future work.

Researchers tasked with understanding the effects of educational technology innovations face the challenge of providing evidence of causality. Given the complexities of studying learning in authentic contexts interwoven with technological affordances, conducting tightly-controlled randomized experiments is not always feasible nor desirable. Today, a set of tools is available that can help researchers reason about cause-and-effect, irrespective of the particular research design or approach. This theoretical paper introduces such a tool, a simple graphical formalism that can be used to reason about potential sources of bias. We further explain how causal graphs differ from structural equation models and highlight the value of explicit causal inference. The final section shows how causal graphs can be used in several stages of the research process, whether researchers plan to conduct observational or experimental research.

In the field of educational technology and instructional design, research methods are emerging that aim to curate different forms of knowledge and insights beyond traditional research studies, or what Reigeluth and An (in Reigeluth and Carr-Chellman (eds) Instructional-design theories and models: Building a common knowledge base, Lawrence Erlbaum Associates, Mahwah, 2009) refer to as “research to prove.” As a result of evolving efforts in this area, editors of research journals in the field are receiving increased submissions employing these methods but have detected some persistent confusion among authors surrounding them. This has resulted in authors submitting articles with muddled methodologies and to outlets that may not be a fit for the work an author seeks to share. It can even be unclear whether authors intentionally employed a specific design-related method prior to reporting. In this piece, we will cover four methods—instructional design cases, case studies, design-based research, and formative evaluation of designs/products—to provide clarity for both graduate students and researchers. For each of these, we will provide definitions, discuss exemplars and features of exemplars, summarize key features that should be present in such a study and its reporting, and provide guidance on front-end intentional design and planning for research studies that employ these methodologies. Additional clarity on these methods can better support scholars and emerging scholars in their roles as researchers, authors, and reviewers.

Emergent technologies present platforms for educational researchers to conduct randomized controlled trials (RCTs) and collect rich data to study students’ performance, behavior, learning processes, and outcomes in authentic learning environments. As educational research increasingly uses methods and data collection from such platforms, it is necessary to consider the most appropriate ways to analyze this data to draw causal inferences from RCTs. Here, we examine whether and how analysis results are impacted by accounting for multilevel variance in samples from RCTs with student-level randomization within one platform. We propose and demonstrate a method that leverages auxiliary non-experimental “remnant” data collected within a learning platform to inform analysis decisions. Specifically, we compare five commonly-applied analysis methods to estimate treatment effects while accounting for, or ignoring, class-level factors and observed measures of confidence and accuracy to identify best practices under real-world conditions. We find that methods that account for groups as either fixed effects or random effects consistently outperform those that ignore group-level factors, even though randomization was applied at the student level. However, we found no meaningful differences between the use of fixed or random effects as a means to account for groups. We conclude that analyses of online experiments should account for the naturally-nested structure of students within classes, despite the notion that student-level randomization may alleviate group-level differences. Further, we demonstrate how to use remnant data to identify appropriate methods for analyzing experiments. These findings provide practical guidelines for researchers conducting RCTs in similar educational technologies to make more informed decisions when approaching analyses.

The emergence of immersive digital technologies, such as shared augmented reality (shAR), virtual reality (VR) and motion capture (MC) offers promising new opportunities to advance our understanding of human cognition and design innovative technology-enhanced learning experiences. Theoretical frameworks for embodied and extended cognition can guide novel ways in which learning in these environments can be understood and analyzed. This conceptual paper explores a research method in Educational Technology—multimodal analysis for embodied technologies—and provides examples from shAR, VR, and MC projects that use this approach. This analysis involves tracking learners’ gestures, actions on physical and virtual objects, whole body movements and positions, and their talk moves, in addition to other relevant modalities (e.g., written inscriptions), over time and across space. We show how this analysis allows for new considerations to arise relating to the design of educational technology to promote collaboration, to more fully capture students’ knowledge, and to understand and leverage the perspectives of learners.

How is the academic community conceptualizing and approaching the integration of AI in education, considering its potential, complexities, and challenges? This study addresses this fundamental question by employing a multifaceted approach that combines co-occurrence network analysis, latent Dirichlet allocation (LDA), and sentiment analysis on a corpus of abstracts from academic publications from 2018 to 2024. The findings reveal key themes in the scholarly discourse, including the centrality of ethical considerations, the impact of global events on AI adoption, and the practical applications of AI in educational management and policymaking. Moreover, the study identifies the main factors discussed in literature as influencing successful AI integration, the challenges and opportunities associated with AI in education, and the evolving academic perspectives on AI’s role in educational settings. This comprehensive analysis of academic literature provides valuable insights into the current state of AI in education research, highlighting trends, challenges, and sentiments as they have evolved over time. By mapping the landscape of scholarly thought on this topic, this study aims to inform future research agendas, contribute to policy discussions, and provide a foundation for evidence-based decision-making in the development and implementation of AI technologies in educational contexts.

The increased availability of low-cost, standalone and immersive virtual reality (VR) can facilitate adoption in autism education. An immersive VR implementation of the individual work system (IWS) from the TEACCH® approach has the potential to be a safe and predictable environment for autistic learners with or without intellectual disability. This study is a multi-site usability and safety trial examining an immersive VR implementation of the IWS co-designed with autistic pupils and their teachers from three educational centers in the UK, Spain, and Turkey. Twenty-one autistic students aged between 6 and 17 years were involved in the study, six of whom had an intellectual disability. The students tested a total of 164 customized tasks. All participants were able to finish all the tasks. No significant safety issues were identified. The student’s average score on the SUS Usability Scale was 85.36 points. A linear regression analysis showed that autistic children with intellectual disability scored significantly lower on feasibility than children without intellectual disability (p < 0.01) across all locations. This study concluded that an immersive VR-IWS proved usable and safe for the 21 students. However, our findings highlight the need for further adaptations and further research on those with an intellectual disability before recommending universal use. Our findings also have implications for game design for learners with special educational needs.

This study advances the emerging research on shared metacognition through the lens of the community of inquiry framework. It seeks components and utterances of the community of inquiry and shared metacognition in online collaborative learning environments to bring an instructional design model to the fore. A three-cycle design-based research method was followed in two cases of university students by triangulating the quantitative and qualitative data sources. A coding scheme, the shared metacognition questionnaire, the community of inquiry questionnaire and one-to-one and focus-group interview protocols were used as data collection tools. Quantitative data were interpreted through descriptive, inferential statistics, and an open/selective coding process interpreted qualitative data. The findings pointed out that the community of inquiry framework presented a powerful theoretical ground to investigate and distinguish cognitive, social, and teaching presence episodes from shared metacognition episodes. Orientation-planning, monitoring, and evaluation-reflection were proved as three main components of the shared metacognition construct in online collaborative learning settings. This study further advances the specification of each shared metacognition component from the group-related regulative actions and task-related regulatory actions. Moreover, a set of six instructional design principles within an instructional design model that combines the components of shared metacognition were put forward. These guidelines are intended to aid practitioners and instructional designers in the development of online collaborative learning activities.

E-textiles provide an interesting field of research as they “blend traditional craft with modern science” (Peppler, 2016) and help learners “broaden their own perceptions of computing” (Searle et al., 2016). Despite the promising findings by primarily long-term interventions structured around e-textiles, educational curriculum reform has been slow to materialize. Educators who embrace a STEAM philosophy are more likely to endorse short workshops, integrating them in existing courses or initiatives; this could serve as a steppingstone for longer interventions and bottom-up curriculum reform. This study examines whether shorter e-textile workshops (lasting four hours) can result in significant gains in understanding. We present an investigation of e-textiles with 22 young children who have no prior experience with e-textiles or working with microprocessors. We present details of our learning design, as well as findings related to circuitry knowledge and computational making skills. We find that the children advanced their circuitry knowledge and practice a range of computational making skills. We further document a series of emerging challenges, including the children’s unwillingness to engage or lack of adeptness with software, a tension between aesthetics and construction, creativity limited by samples of previous e-textile projects, and the difficulty in grasping the materiality of e-textiles. We propose that some direct instruction and facilitation is not incompatible with the making ethos; the approach can help address these challenges, allowing young children to benefit from their participation in short-duration e-textile workshops.

Despite the efforts of instructional design (ID) to solve real-life problems, it remains challenging to adapt and be flexible in such situations. In particular, problems that require simultaneous knowledge of multiple domains and contexts are more challenging to solve because real-life problems do not reconstruct the learned experience. This is generally thought to stem from differences between learning and real-life practice, but it also stems from instructional designs that fail to reflect the problem's structure and cognitive processes. This study is based on the 4C/ID model and proposes an instructional design for developing and connecting cognitive processes across multidimensional domains and contexts. It employs a simple-to-complex method that combines emphasis manipulation sequencing with simplifying condition sequencing, exposing students to the entire domain and context from the beginning of the learning process to develop a holistic cognitive process. A quasi-experiment was conducted with 34 sophomore college students majoring in education who were asked to create a lesson plan using different teaching styles. The groups consisted of students learning using emphasis manipulation sequencing and single sequencing (emphasis + condition), and the experimental procedure consisted of a total of five sessions, with between-group and within-group analyses of the effects of cognitive strategies and structural models. In the between-group analysis, cognitive strategies and structural models using the single sequencing method were effective from sessions 2 through 5, while in the within-group analysis, the development of cognitive strategies and structural models occurred from sessions 1 through 3, when the simplifying condition sequencing principle was maintained. While the proposed instructional designs are not a foolproof way to develop cognitive processes, a combined approach that considers the nature of the task provides a starting point that can enhance real-life training.

The relationship between technology and educational processes is a complex one. At this moment, increased digitization as well as efforts to limit the use of digital tools can be observed. In view of (a) deepening our understanding of the relationship between technology and educational processes and (b) strengthening the productive educational use of technological tools, elements of a relevant and comprehensive agenda for educational technology research agenda are proposed. The agenda refers to the following elements: educational goals, learners, technological tools, a tripartite relationship learner-tool-teacher, context, and designing learning environments. Recognizing the need for diverse methods and consistent descriptive frameworks can help make the research agenda more cumulative and coherent.

Artificial Intelligence (AI) is driving ecological shifts and systemic reforms in education. As practitioners of educational reform, teachers’ behavioral intention to experience and accept the effectiveness of AI technologies will affect the quality of educational change. From an educational ecology perspective, this study explores the impact of core elements within three dimensions—technologies, pedagogies, and cultures—on teachers’ behavioral intention to use AI in an AI-enabled educational ecosystem (AI-e3) environment. The study uses a multi-level mediation model to analyze data of 4349 teachers from 189 primary and secondary schools from a western province of China. The results indicated that school-level dimensional elements, directly or indirectly, influenced teachers’ behavioral intention to use AI, mediated by teacher-level dimensional elements. The findings are relevant to school administrators and policy makers, who should consider the key influences on teachers’ behavioral intentions to use AI and promote the effective application of AI science for educational change.

During COVID, HyFlex gained popularity and became a "new normal" that educators need to consider as an effective instructional approach. Previous research offers conflicting findings related to the impact of HyFlex instruction on students' basic psychological needs and academic performance. Our investigation provides insight into a specific variation of HyFlex we call "Interactive Synchronous HyFlex" as it is situated in a highly collaborative active learning environment. The investigation aimed to clarify relationships between students' academic performance, basic psychological needs, and demographics of a pre-pandemic face-to-face offering of an undergraduate project-based design course and the same course using an Interactive Synchronous HyFlex approach at the end of the pandemic. Demographic data were collected from university databases; academic performance was measured by end-of-semester grades; and a survey measured basic psychological needs. The findings revealed that students in the HyFlex offering perceived their basic psychological needs as being met as effectively or significantly more so compared to students in the face-to-face offering. Significant predictors of student academic success were different for face-to-face environments compared to predictors that were significant in HyFlex environments. In the HyFlex environment, relatedness to the instructor was a significant predictor of academic success as was class rank and gender. These findings point to the importance of instructor presence as a key factor in student success in the HyFlex model. Overall, the results indicate that the HyFlex environment is a viable educational model for the post-pandemic era.

Artificial intelligence (AI) competence in education is a set of skills that enable teachers to ethically and responsibly develop, apply, and evaluate AI for learning and teaching processes. While AI competence becomes a key competence for teachers, current research on the acceptance and use of AI in classroom practice with a specific focus on the required competencies of teachers related to AI is scarce. This study builds on an AI competence model and investigates predispositions of AI competence among N = 480 teachers in vocational schools. Results indicate that AI competence can be modeled as combining six competence dimensions. Findings suggest that the different competence dimensions are currently unequally developed. Pre- and in-service teachers need professional learning opportunities to develop AI competence.

Based on the assumptions of cognitive load theory, this study aims to utilize the eye movement data collected from multiple experts to scaffold novice graphic designers. The study has two main stages. In the first stage, eye tracking was used to record the eye movements of 7 experts, who covered eight topics explaining how to use Photoshop. The areas of interest and fixation durations were analyzed for each topic to extract the common patterns. Instructional videos were produced with the guidance of both eye movement patterns and multimedia design principles. In the second stage of this study, those videos were delivered to 30 university students. The experiment group watched the guided videos, whereas the control group watched straight videos, i.e., content without guidance. The guidance elements were reduced (faded) as students got familiar with the interface. All participants were assigned to perform tasks as soon as they completed the videos. The findings showed that the knowledge scores did not significantly differ between groups. Still, the average performance scores of the experiment group were considerably higher than that of the control group. Considering the transition process, the experiment group outperformed the others. The topics in which experts’ eye movements highly matched each other resulted in high-performance gaps among groups.

The COVID-19 pandemic has accelerated digitization, access to IT resources, and digital inclusion in the Italian school system. This paper presents D-UDA (i.e., “unità didattica di apprendimento digitale”, in Italian), a digital didactic unit for learning mathematics concepts. The presented approach combines teaching methodologies and game-based activities (e.g., the escape room) with a multisensory approach to designing and developing digital and multimodal technologies. D-UDA is divided into two parts: the first part consists of logic puzzles that adhere to the guidelines set by INVALSI (the Italian Istituto Nazionale per la Valutazione del Sistema educativo di Istruzione e di formazione) for mathematics learning, while the second part involves a series of multisensory games designed to promote the development of transversal competencies, such as cooperation and engagement. Moreover, D-UDA encourages children to create their own adventure using the same tools employed by the designers to develop the experience. The children who participated in testing D-UDA in June 2020 were asked to complete usability questionnaires after the experience. Preliminary results indicate the effectiveness of the educational intervention presented, which integrates recent pedagogical theories and teaching methodologies with a multisensory perspective and a technological design.

Group awareness tools have garnered significant interest within the realm of computer-supported collaborative learning (CSCL), as they foster collaborative learning behaviors. However, in the context of a CSCL environment devoid of rich technologies, supporting group awareness is challenging. Contextualized in a teacher professional development course in a large normal university in southwestern China, this study followed a minimalist approach to develop group awareness support through collaborating authoring software. An 8-week-long quasi-experiment was conducted to explore whether supporting group awareness in project-based learning (PBL) through a minimalist approach could (1) improve the quality of group project, and (2) improve students’ participation in collaborative learning. The sample consisted of 125 junior students. Students received regular instruction in stage 1 (week 1 to week 4) and instruction with group awareness support in stage 2 (week 5 to week 8). After each stage, students’ group projects and self-reported surveys were analyzed qualitatively and quantitatively. It was found that supporting group awareness led to significantly better group performance but did not significantly influence student participation. Group awareness support also contributed to equal involvement among group members. Furthermore, cross-group awareness encouraged students to co-construct knowledge across groups. These findings might help practitioners in economically disadvantaged and technologically underprivileged areas to design group awareness support. It might also benefit instructors who prefer a light-tech solution to support CSCL.

Technological advancements in recent years have accelerated the development of information and communication technologies, introducing numerous innovations. One prominent innovation is the concept of the metaverse, which has gained significant popularity and is increasingly influencing various sectors, including the economy, art, entertainment, and education. Despite its growing relevance, there is a practical gap in understanding how management information systems students in Türkiye perceive the use of the metaverse for software education. This study aims to address this gap by exploring students’ perceptions and identifying the factors that influence their intentions to use the metaverse. The conceptual model includes adoption characteristics such as traibility, observability, compatibility, and complexity, as well as user satisfaction, personal innovativeness, and the structures of the technology acceptance model. The data of the study were obtained from 877 students, and the collected data were analyzed utilizing the structural equation modeling technique. The results indicate that personal innovativeness positively influences perceived usefulness and perceived ease of use. However, perceived observability, user compatibility, and perceived traibility did not significantly impact user satisfaction. Conversely, perceived usefulness, user satisfaction, and perceived ease of use positively affect students’ intentions to use the metaverse for software education. This study offers valuable contributions to the metaverse literature, educators, field experts, and researchers.

The purpose of the present study was to analyze the impact of technology on student foreign language learning, as it has been widely used to enhance language instruction over the past few decades. This multivariate meta–meta-analysis study aimed to examine the effects of technology on various aspects of language learning, including listening, speaking, reading, writing and vocabulary, and explore how factors like educational level and technology type influenced these impacts. The researchers conducted a meta-analysis of 10 studies published prior to May 2023, using both qualitative and quantitative methods. They analyzed the descriptive and methodological characteristics of each study, and found a statistically significant overall effect size (g = .068, p < .001 with a 95% confidence interval of .595–.860) indicating that technology positively impacted language learning outcomes compared to traditional learning methods. The researchers identified educational level and technology type as important factors contributing to the variability in effect size. Specifically, both higher education and K-12 settings, as well as VR tools and computing resources, had positive impacts on students’ foreign language learning. Overall, the results suggest that using technology is an effective way to improve foreign language learning for students, and provide valuable recommendations for future research and practical applications in this area.

Computational thinking is accepted today as a collection of cognitive and social skills required for functioning in the 21st century. The paper presents a conceptual view at computational thinking that encompass concepts, problem-solving skills, application skills, and social skills. To impart those perceptions and skills the paper proposes the Four Pedagogies for Developing Computational Thinking (4P4CT) framework, which relies on active learning, project-based learning, product-based learning, and context-based learning, and advocates implementing computational thinking across all the education system in all subject matters at all ages by all teachers. The framework is presented and its implementation in an academic course for preservice K-12 teachers, taught so far in 16 classes attended by 409 preservice K-12 teachers, is described in detail. To support the effective development of the expected competences among preservice teachers, two types of empirical qualitative evidence, related to student outcomes, are presented: (a) simulations of computational processes, and (b) reflections that indicate a change in preservice teachers' perceptions and the application of computational thinking in their future teaching work.