ETR&D

Teachers’ use of data from digital learning platforms for instructional design: a systematic review

1 week 2 days ago
Abstract

Data-based decision-making is a well-established field of research in education. In particular, the potential of data use for addressing heterogeneous learning needs is emphasized. With data collected during the learning process of students, teachers gain insight into the performance, strengths, and weaknesses of their students and are potentially able to adjust their teaching accordingly. Digital media are becoming increasingly important for the use of learning data. Students can use digital learning platforms to work on exercises and receive direct feedback, while teachers gain data on the students’ learning processes. Although both data-based decision-making and the use of digital media in schools are already widely studied, there is little evidence on the combination of the two issues. This systematic review aims to answer to what extent the connection between data-based decision-making and the use of digital learning platforms has already been researched in terms of using digital learning data for further instructional design. The analysis of n = 11 studies revealed that the use of data from digital learning platforms for instructional design has so far been researched exploratively. Nevertheless, we gained initial insights into which digital learning platforms teachers use, which data they can obtain from them, and how they further use these data.

Unlocking the Everdoor: analyzing the serious game Spiritfarer

1 week 2 days ago
Abstract

This qualitative research study aims to examine the potential of the commercially available serious game, Spiritfarer. The study focuses on the game's unique approach to serious themes and its ability to facilitate discussions about grief. A grounded theory approach was used to analyze lived experience descriptions from 54 participants. Findings indicate that Spiritfarer can impact gamers' outlook, knowledge, and behaviors, particularly related to grief, empathy, and loss management. The game achieves this through effective game design principles that generate emotional reactions and establish a connection between the narrative, characters, and players.

Instructional design students’ design judgment development

1 week 2 days ago
Abstract

This qualitative multi-case study explores the exercise and development of the design judgment of eight instructional design (ID) students working on design projects over one semester in graduate programs at four different institutions in the USA. Their design processes were explored through interviews and their design documents using the concepts of design judgment as reported by Nelson and Stolterman (The design way: Intentional change in an unpredictable world, MIT Press, Cambridge, 2012) and reflection-in-action as reported by Schön (The reflective practitioner: How professionals think in action, Basic Books, New York, 1983) to characterize the exercise and development of their design judgment across the semester. Findings reveal that these ID students make multiple design judgments concurrently and constantly. Their prior design experience, the course requirements and their design practice in class influence both the exercise and development of their design judgment. Lastly, the development of their design judgment is both perceptible and personal. In light of these findings, recommendations are made for how to support the development of ID students’ design judgment.

Virtual reality and gamification in education: a systematic review

1 week 3 days ago
Abstract

This study aims to analyze the use of virtual reality and gamification in education by examining the existing literature. In addition to virtual reality, this study focuses on gamified virtual reality learning environments which refer to virtual reality learning environments that integrate gamification elements and mechanisms. Based on the Preferred Reporting Items for Systematic Reviews and Meta-Analyses (PRISMA) statement, a systematic literature review was carried out. No limitations were set regarding educational level, type of study, subject, and publication year. The related articles were retrieved from 5 databases (ERIC, Google Scholar, IEEE, SCOPUS, and Web of Science). A total of 112 articles were included, 16 research questions were explored, and a thematic analysis was conducted. To evaluate the quality of the articles included, the Mixed Methods Appraisal Tool (MMAT) was used. According to the findings, gamification and virtual reality support several pedagogical theories and approaches. Their adoption to and integration into education can enrich and transform traditional teaching and learning and were assessed positively by students and teachers. Gamification elements significantly affected students’ achievements. In comparison to traditional learning environments, gamified virtual reality learning environments were more motivating, engaging, and interactive and offered more opportunities for personalized and collaborative learning. Through the realistic and interactive experiences offered, students’ immersion and social presence can be enhanced, knowledge acquisition can be improved, and material comprehension can be facilitated. Positive changes in student attitude, behavior, and mentality as well as improved cognitive, physical, and social–emotional development were observed. When using learning environments that integrate both virtual reality and gamification, students’ learning outcomes, motivation, engagement, and self-efficacy were increased. Additionally, students’ academic performance, active involvement, and satisfaction were improved. Students’ curiosity, imagination, focus, and interest were enhanced and their skills and competences were developed. Finally, gamified virtual reality emerged as an effective educational tool that can improve learning at all educational levels, subjects, and contexts.

A systematic review of educational online peer-review and assessment systems: charting the landscape

1 week 3 days ago
Abstract

Over the past two decades, there has been an explosion of innovation in software tools that encapsulate and expand the capabilities of the widely used student peer assessment. While the affordances and pedagogical impacts of traditional in-person, “paper-and-pencil” peer assessment have been studied extensively and are relatively well understood, computerized (online) peer assessment introduced not only shifts in scalability and efficiency, but also entirely new capabilities and forms of social learning interactions, instructor leverage, and distributed cognition, that still need to be researched and systematized. Despite the ample research on traditional peer assessment and evidence of its efficacy, common vocabulary and shared understanding of online peer-assessment system design, including the variety of methods, techniques, and implementations, is still missing. We present key findings of a comprehensive survey based on a systematic research framework for examining and generalizing affordances and constraints of online peer-assessment systems. This framework (a) provides a foundation of a design-science metatheory of online peer assessment, (b) helps structure the discussion of user needs and design options, and (c) informs educators and system design practitioners. We identified two major themes in existing and potential research—orientation towards scaffolded learning vs. exploratory learning and system maturity. We also outlined an agenda for future studies.

Effects of robot-based multiple low-stakes assessments on students’ oral presentation performance, collective efficacy, and learning attitude

1 week 3 days ago
Abstract

Low-stakes assessment has gained attention in recent years due to its link to enhancing learning effects and its essential role in learning evaluation. Unlike high-stakes assessments, low-stakes assessments have little or no consequences for learners’ academic performance, and are designed to support the feedback-oriented learning process. Providing multiple low-stakes assessments to students yields significantly greater long-term retention of knowledge and skills. However, learners may not give their best efforts when taking low-stakes assessments, which could lead to poorer learning outcomes. Using emerging technologies such as social robots in the learning environment could foster interactive learning, engagement, and motivation for learning assessments. Therefore, integrating low-stakes assessments and robots might encourage students to exert greater effort while performing learning tasks. This study aimed to discover the impacts of robot-based multiple low-stakes assessments on students’ oral presentation performance, collective efficacy, and learning attitude. A quasi-experiment was conducted in two sixth-grade classes of elementary students. The Robot-based Multiple Low-Stakes Assessment (Robot-MLSA) was randomly assigned to one class, while the Computer-based Multiple Low-Stakes Assessment (C-MLSA) was assigned to another class. The findings showed that the Robot-MLSA could enhance students’ oral presentation performance, support their collective efficacy, and improve their learning attitude toward robots. Furthermore, an in-depth discussion of students’ learning perceptions and experience is provided to explore the effectiveness of the Robot-MLSA.

Effects of a concept mapping-guided virtual laboratory learning approach on students’ science process skills and behavioral patterns

1 week 4 days ago
Abstract

Laboratory courses can help students learn in a meaningful way. In the past, students encountered difficulties in chemistry laboratory courses due to limited access to equipment and space for practicing experimental operations. In recent years, virtual laboratories have allowed students to repeatedly practice in order to achieve their experimental goals. However, even when students follow the experimental protocol, some still cannot fully understand the principles and meaning of the experimental procedures. Therefore, when they encounter unexpected situations in experiments, they may not know how to handle them. Thus, this study incorporated a concept mapping-guided learning approach to facilitate students' understanding of the concepts of the experimental process and the relationship between experimental procedures before virtual laboratory practice. Using a quasi-experimental approach, 51 middle school students aged 12 to 13 years were assigned to an experimental group (M = 12, F = 14) using the concept mapping-guided virtual laboratory learning approach, and a control group (M = 12, F = 13) using the conventional virtual laboratory learning approach. The experimental group improved their science process skills, problem-solving awareness, and creative thinking tendency. Teachers can employ guided virtual laboratory learning approaches to help students experiment and understand concepts.

A meta-analysis of technology-delivered literacy instruction for elementary students

1 week 4 days ago
Abstract

Technology to support student learning has become increasingly prevalent in schools and homes during the last few decades. Several recent reviews have examined aspects of technology-based literacy instruction, but they have not focused on the full array of literacy skills that can be addressed during technology-delivered instruction for elementary students, nor have they utilized meta-analytic methods to rigorously examine effects of such instruction. Therefore, the purpose of this meta-analysis was to describe and evaluate recent research on technology-delivered literacy instruction for students in Grades K-5. A total of 53 experimental or quasi-experimental studies were analyzed. Results demonstrated a statistically significant main effect of instruction on norm- or criterion-referenced literacy outcomes (g = 0.24, p < .001), indicating that elementary students are likely to benefit from technology-delivered literacy instruction. Although the effects of several moderator variables representing study, participant, instruction, and outcome characteristics were explored, analyses revealed no statistically significant moderators of effects of instruction. Overall, findings indicate that further research on technology-based literacy instruction is needed to determine what works for whom and under what conditions.

Improving part-time instructors’ student failure rate with an educational engagement information system

2 weeks 1 day ago
Abstract

Higher education institutions rely on part-time faculty to teach a large percentage of courses in a range of disciplines. Existing research indicates that instruction by part-time faculty can negatively impact academic outcomes, student retention, and subsequent interest in a field of study. This study investigates if using a motivational information system can help close this performance gap. It introduces a type of motivational information system called an educational engagement information system (EEIS). EEISs draw on principles of behavioral economics, motivation theory, and learning cognition theory to help students want to improve their performance. The study compared student failure status in course sections taught by part- and full-time instructors both with and without an EEIS. Results suggest that using an EEIS can help improve student failure rates in courses taught by part-time faculty members and bring students’ performance to parity with the performance of students taught by a full-time instructor. Features of an EEIS, such as the additional structure, grade and outcome trackers, and motivational elements, can augment a part-time instructor’s approach and may compensate for limitations sometimes associated with part-time instruction including limitations of expertise, time, or rewards that can have a negative impact on students’ academic outcomes. As a result, students can experience higher completion rates and better performance.

How do Canadian faculty members imagine future teaching and learning modalities?

2 weeks 2 days ago
Abstract

This study, originally prompted by the impact of the COVID-19 pandemic on educational practices, examined Canadian faculty members’ expectations of teaching and learning modalities in the year 2026. Employing a speculative methodology and thematic analysis, interview responses of 34 faculty members led to the construction of three hypothetical scenarios for future teaching and learning modalities: a hybrid work model, a high tech and flexible learning model, and a pre-pandemic status quo model. In contrast to radical education futures described in the literature, the findings do not depart significantly from dominant modes of teaching and learning. Nevertheless, these findings offer insights into the expectations that Canadian faculty members have with respect to future teaching and learning modalities, the contextual issues and concerns that they face, the use of speculative methodologies in educational technology research, and the potential impacts remote learning trends have on the future of higher education in Canada.

Entangled eclecticism: a sociotechnical-pedagogical systems theory approach to learning experience design

2 weeks 4 days ago
Abstract

This paper explores the adoption of an entangled eclecticism perspective in Learning Experience Design (LXD), integrating a sociotechnical-pedagogical systems approach. It emphasizes the significance of considering the sociocultural, technological, and pedagogical dimensions of learning as a cohesive, interconnected ecology to design effective learning experiences. The authors discuss the implications of this perspective on learning outcomes, methodologies, and the selection of relevant theories for design. The aim is to guide the creation of learning experiences that are not only effective, efficient, and appealing but also deeply empathetic, culturally sensitive, and responsive to the evolving dynamics of learning environments. Concluding that LXD is a complementary approach to traditional learning design, the paper underscores its evolution, drawing from multiple traditions to offer a holistic framework for educational design and technology. This approach strives for learning experiences that are not just effective and efficient, but also culturally sensitive and empathetic, adapting to the dynamic nature of learning environments. Broader implications of this approach and directions for future research are discussed.

What I think I learned

3 weeks 1 day ago
Abstract

A retrospective review of 50 years of research and development experience showing the connectedness of the author’s theoretical ideas to practical application. An effort to show designers how over the span of a career new ideas begin as work-related insights and discoveries that by problem solving flow together to create a unique personal view of design and designing. Encouragement for individual designers to be willing to experiment with new ideas that may step beyond received practice and to learn from those experiences, even to the extent of testing and adopting new worldviews that may differ from the general view. Encouragement for revisiting foundational documents of the field of educational and instructional technology to examine the intent of the founders and to build possible alternative interpretations of their meaning. A recommendation of topics the field should consider to maintain relevance within a rapidly changing theoretical and technical landscape.

Investigating the design, participation and experience of teaching and learning facilitated by user-generated microgames on an open educational platform

3 weeks 4 days ago
Abstract

Although user-generated microgames, defined as very simple games made by non-professionals on open platforms, are popular and appear to have considerable advantages in facilitating learning, further exploration is needed to establish their potential in instructional practices. The present study investigates the design, participation and experience of teaching and learning facilitated by user-generated microgames on an open educational platform. Through an exploratory experiment research method, four elementary school teachers designed and implemented microgame-based learning utilising these very small games on GeoGebra Classroom attended by 129 students. Data were gathered from lesson plans, classroom activity records and self-reflection questionnaires. This study revealed that teachers designed learning with various user-generated microgames and debriefing methods respecting learning content, but they shared comparatively similar scenarios by inserting microgame-based learning into the middle of the main session. The completion rate for the debriefing activity is minimum although the total joining times overshoot the number of students. Teachers found that user-generated microgames are acceptable to orchestrate short serious gaming sessions even though they are limited to one player with basic interfaces. Notwithstanding several disadvantages of these microgames recognised by students, such as missing learning instructions and inadequate interfaces, they so far enjoy learning by playing the games. The most critical implication of this study is to provide sufficient instructions and additional time for microgaming sessions in elementary schools to ensure sustainable completion of the briefing, playing and debriefing activities.

Making for science: a framework for the design of physical materials for science learning

1 month ago
Abstract

Recent work in science education has emphasized the importance of the conceptual, social, and epistemic dimensions in science learning. But what about the material dimension—the actual physical materials that students use? The way their characteristics and affordances limit or determine instruction has thus far not been as thoroughly explored. Despite the vast body of work that exists on science and engineering practices, there are relatively few examples of how physical science materials are linked to and can support engagement with such practices for sensemaking. In this paper, we propose a framework for analyzing the design of these physical materials and the activities associated with them that can serve as lenses for educators and researchers to consider when planning and reflecting on classroom instruction. To validate our framework, we apply it to analyze activities developed and implemented by teachers during a four-year research-practice partnership project conducted in a city in Brazil, and investigate potential connections between the design of physical materials and students’ epistemic agency. We conclude with recommendations for the development of physical resources in labs and makerspaces for science learning.

Sparring with technology: collaborating with coaches, mentors, and academic staff to develop culturally responsive computing education for a youth boxing program

1 month ago
Abstract

In the United States, culturally responsive computing is a framework that aims to support broadening the participation of racially and linguistically diverse children in computing and technology education through bottom-up interventions that are community-oriented, technology rich, and culturally dynamic. Despite the important role that youth sports play in many local neighborhoods and communities across demographic groups there is only a small amount of research on developing culturally responsive computing education that incorporates these activities. We report findings from a culturally responsive computing collaboration between computing education researchers and coaches, mentors, and academic staff who ran a youth boxing program that predominantly served African American children. The purpose of the collaboration was to learn about and represent the adults’ expertise and knowledge in the co-development of culturally responsive computing activities. Using an emergent mixed methods research design, we collected qualitative data (i.e., interviews and group discussions) and quantitative data (i.e., pre- and post-surveys) throughout the collaboration. We analyzed these data to study how coaches, mentors, and staff members brought their knowledge and expertise to bear on the co-development of culturally responsive computing activities for the youth boxing program. Our findings show how the coaches, mentors, and academic staff used their expertise and knowledge in ways that leveraged boxing culture to go beyond boxing itself in the co-development of the activities. In addition, even when connections between computing and boxing did not appear authentically motivated this did not negate the adults’ engagement with the culturally responsive computing project. These findings have implications for anti-deficit theorizing about authenticity and inauthenticity in the co-development of culturally responsive computing in youth sports contexts. The construction of authenticity in culturally responsive computing might be less understood as a direct translation from community into education and more so as a negotiation between locally defined demarcations of what is considered authentic and inauthentic.

Impact of an instructional guide and examples on the quality of feedback: insights from a randomised controlled study

1 month 1 week ago
Abstract

While the provision of peer feedback has been widely recommended to enhance learning, many students are inexperienced in this area and would benefit from guidance. This study therefore examines the impact of instructions and examples on the quality of feedback provided by students on peer-developed learning resources produced via an online system, RiPPLE. A randomised controlled experiment with 195 students was conducted to investigate the efficacy of the approach. While the treatment group had access to instructions and examples to support their provision of feedback, the control group had no such assistance. Students’ feedback comments were coded using an adaptation of the S.P.A.R.K. (Specific, Prescriptive, Actionable, Referenced, Kind) model. The results indicate that the instructional guide and examples led to students writing more comprehensive comments. The intervention notably enhanced the presence of feedback traits matching the S.P.A.R.K. model and increased instances where multiple traits of quality were observed in a single comment. However, despite the guide’s impact, the students’ ability to provide actionable feedback was limited. These findings demonstrate the potential of developing and integrating structured guidance and examples into online peer feedback platforms.

Taking adaptive learning in educational settings to the next level: leveraging natural language processing for improved personalization

1 month 1 week ago
Abstract

Educational technology plays an increasingly significant role in supporting Self-Regulated Learning (SRL), while the importance of Adaptive Learning Technology (ALT) grows due to its ability to provide personalized support for learners. Despite recognizing the potential of ALT to be influential in SRL, effectively addressing pedagogical concerns about using ALT to enhance students’ SRL remains an ongoing challenge. Consequently, learners can develop perceptions that ALT is not customized to their specific needs, resulting in critical or dismissive attitudes towards such systems. This study therefore explores the potential of combining Natural Language Processing (NLP) to enhance real-time contextual adaptive learning within an ALT to support learners’ SRL. In addressing this question, our approach consisted of two steps. Initially, we focused on developing an ALT that incorporates learners’ needs. Subsequently, we explored the potential of NLP to capture pertinent learner information essential for providing adaptive support in SRL. In order to ensure direct applicability to pedagogical practice, we engaged in a one-year co-design phase with a high school. Qualitative data was collected to evaluate the implementation of the ALT and to check complementary possibilities to enhance SRL by potentially adding NLP. Our findings indicate that the learning technology we developed has been well-received and implemented in practice. However, there is potential for further development, particularly in terms of providing adaptive support for students. It is evident that a meaningful integration of NLP and ALT holds substantial promise for future enhancements, enabling sustainable support for learners SRL.

Factors contributing to teachers’ acceptance intention to gamified EFL tools: a scale development study

1 month 3 weeks ago
Abstract

This study is to develop and validate a scale that measures the acceptance intention of secondary school teachers toward gamified English-as-a-foreign-language (EFL) tools. Drawing upon the technology acceptance models (TAM) and related frameworks, we synthesized a framework that includes six main factors contributing to teachers’ acceptance intention of gamified learning tools. These factors are perceived enjoyment, perceived usefulness (PU), perceived ease of use (PEoU), perceived risks, facilitating conditions, and control variables. The study involved 361 and 512 secondary school teachers who participated in qualitative interviews and quantitative surveys, respectively. Surprisingly, in the context of implementing gamified tools for the teaching of English-as-a-second-language, the results indicate that only two factors can be retained in the proposed framework (PU and PEoU). However, PU and PEoU in the current research contexts brought new connotations, which is of significance for future studies. Eventually, a five-item scale measuring PU (PU-gamification-EFL) and a six-item scale measuring PEoU (PEoU-gamification-EFL) have been validated. Implications and limitations were discussed, as well as suggestions for future studies.

Should We account for classrooms? Analyzing online experimental data with student-level randomization

1 month 3 weeks ago
Abstract

Emergent technologies present platforms for educational researchers to conduct randomized controlled trials (RCTs) and collect rich data to study students’ performance, behavior, learning processes, and outcomes in authentic learning environments. As educational research increasingly uses methods and data collection from such platforms, it is necessary to consider the most appropriate ways to analyze this data to draw causal inferences from RCTs. Here, we examine whether and how analysis results are impacted by accounting for multilevel variance in samples from RCTs with student-level randomization within one platform. We propose and demonstrate a method that leverages auxiliary non-experimental “remnant” data collected within a learning platform to inform analysis decisions. Specifically, we compare five commonly-applied analysis methods to estimate treatment effects while accounting for, or ignoring, class-level factors and observed measures of confidence and accuracy to identify best practices under real-world conditions. We find that methods that account for groups as either fixed effects or random effects consistently outperform those that ignore group-level factors, even though randomization was applied at the student level. However, we found no meaningful differences between the use of fixed or random effects as a means to account for groups. We conclude that analyses of online experiments should account for the naturally-nested structure of students within classes, despite the notion that student-level randomization may alleviate group-level differences. Further, we demonstrate how to use remnant data to identify appropriate methods for analyzing experiments. These findings provide practical guidelines for researchers conducting RCTs in similar educational technologies to make more informed decisions when approaching analyses.

Trends, tensions, and futures of maker education research: a 2025 vision for STEM+ disciplinary and transdisciplinary spaces for learning through making

1 month 3 weeks ago
Abstract

This special issue aims to sketch the present state of maker learning research, reveal possible tensions, and present future possibilities to articulate principles for learning through design in the era of maker education. The special issue was announced in 2022 in ETR&D, a leading academic journal in educational technology. Of the 50 submissions to the special issue, eighteen (18) were accepted for publication. The editors favored a robust inclusion of papers to help define the contours of the field at present. Four clusters of topics are identified in this collection of papers: (i) STEM+ disciplinary and transdisciplinary learning spaces; (ii) Digital technologies in making, opportunities and challenges; (iii) Assessment practices and frameworks; (iv) Representation, inclusion, and tensions around maker-centered initiatives and reforms. The editors of the special issue believe that these clusters reflect the current state-of-the-art in the field as well as significant questions to guide near future research. Reflecting on these papers but also the overall editorial process, the editors identified several opportunities and provide suggestions on how the field might expand moving forward.