ETR&D

Conceptual and theoretical frameworks for leveraging makerspaces to encourage and retain underrepresented populations in STEM through learning by design

1 month 3 weeks ago
Abstract

Amidst this era of rapid technological advancement, the impact of White dominance in STEM causes inequity throughout the design, implementation, and function of modern technologies. Evidence of this includes AI systems that perpetuate racial and gender biases, medical devices that are incompatible with non-White medical needs, and hiring algorithms that prioritize the White male experience. Though not a panacea, greater representation of traditionally marginalized groups in the STEM workforce will help reduce and safeguard against digital racism, sexism, and ableism. Advocates of greater representation in STEM fields suggest that makerspace pedagogy and design that is rooted in equity and inclusivity can attract students from traditionally marginalized groups and make STEM more accessible and welcoming to all. To this end, this paper proposes a modification of the TPACK theoretical framework (Koehler and Mishra in Contemp Issues Tech Teach Educ 9(1):60–70, 2009) that centers knowledge of technological and inclusive practices in Makerspaces, giving rise to the Maker Technology, Pedagogy, Inclusion, and Content Knowledge (MakerTPICK) theoretical framework. Additionally, this paper presents the Makerspace Planning, Implementation, Establishment, and Reassessment (PIER) conceptual framework. This framework outlines the process for makerspace leaders to create and sustain an inclusive makerspace through the MakerTPICK framework, be they teachers in a school setting or makerspace coordinators outside of the field of K-12 education. The paper describes future implications for these frameworks in terms of practical applications for makerspaces and applied to research settings.

Analyzing properties of success for assessment development in maker-based learning

1 month 3 weeks ago
Abstract

Public libraries have embraced the popularity of maker education and makerspaces by integrating maker education in their program offerings, and by developing makerspaces that enable patrons to tinker and create products. But less attention has been paid to supporting librarians and maker educators in assessing the impact of these spaces. To expand assessment scholarship and practices related to public library makerspaces, we offer two contributions. First, we share findings from a qualitative research study in which we analyzed how 17 library staff and maker educators define success and identify evidence of success in their maker programs. The findings from that study, in conjunction with our collective experience as research partners working with public library makerspaces, laid the foundation for a series of analysis tools we developed to help stakeholders identify the assessment needs of such learning environments. The Properties of Success Analysis Tools (PSA Tools) represent our second contribution; these tools invite library staff and maker educators to reflect on and unpack their definitions of success in order to identify what features a relevant assessment tool should have.

An exploratory case study of the use of a digital self-assessment tool of 21st-century skills in makerspace contexts

1 month 3 weeks ago
Abstract

Maker education provides the perfect context for young learners to develop 21st-century skills. However, research is inconclusive on how these skills could be assessed. Namely, the complex nature of 21st-century skills requires different types of assessments, not necessarily relying on paper-and-pencil or multiple-choice tests, but rather drawing on the learners’ perspective in the form of self-assessment and reflection. Prior studies highlighted several challenges of situating self-assessment in makerspace contexts, such as the lack of dedicated technology for documentation, distractions caused by noise or group work, and the lack of skills and motivation to practice self-assessment. This paper presents an exploratory case study aimed at an in-depth investigation of the use of a digital self-assessment tool of 21st-century skills in makerspace contexts. The authors converged qualitative data collected mainly from interviews with teachers and students. Researcher observations and tool log files (e.g., student work in the digital tool) were used as triangulation sources. Although challenges emerged, the study presents encouraging findings regarding the use of the digital tool for raising students’ awareness of their development of 21st -century skills and engaging them in self-assessment and reflection. The results of the study provide rich insights to guide future research on the topic.

Making in virtual reality environments: a case study of K-12 teachers’ perceptions on the educational affordances of virtual reality for maker-centered learning

1 month 3 weeks ago
Abstract

The emergence of immersive VR technology in K-12 educational spaces has created a need for research examining the affordances and constraints of this technology for student learning. The current study uses a case-study methodology to illustrate K-12 secondary science teachers’ perceived affordances and constraints of using immersive VR tools to develop maker-centered learning experiences which align with curricular goals following a professional development experience. Findings suggest teachers were able to design student-centered maker learning experiences using immersive VR devices that addressed K-12 science content. In addition, participants discovered ways to overcome reported challenges, and develop teaching artifacts as well. This study offers a model for teacher educators in this area, examples of curriculum aligned learning activities, and provides a foundation for future research on integration of VR in K-12 contexts.

Weaving in: shifts in youth mathematical engagement through weaving

1 month 3 weeks ago
Abstract

Despite recent efforts to support learners from traditionally minoritized backgrounds in mathematics, inequities in math achievement and participation still exist, particularly for women and people of color. Additionally, much of how math is taught in schools aligns with a particular epistemology that comes from western mathematicians and philosophers.  While a strong link between weaving and mathematics has been established, fewer studies explore the possibilities of this link for supporting youth’s pluralistic engagement with mathematics in educational settings. Thus, we ask: In a making-focused intervention designed to expose youth to the mathematical practices inherent in weaving, how do student-created artifacts showcase learning? In this paper, four cases demonstrate how middle-school youth learning to weave and learning through weaving experimented or leveled up their planned or implemented designs. Their learning processes showcase engagement with mathematics that mirrors that of more experienced weavers. Making visible the mathematical engagement that youth undertake through weaving may be a step toward building frameworks and classroom activities through making that work to combat these disciplinary issues.

The impact of teachers’ pedagogical beliefs, self-efficacy, and technology value beliefs on 3D printing integration in K-12 science classrooms

1 month 3 weeks ago
Abstract

Teachers’ beliefs such as pedagogical beliefs, self-efficacy, and technology value beliefs are influential to technology integration practice. This study aims to investigate teachers’ beliefs and the impact on their 3D printing integration in science classrooms. A total of 26 K-12 teachers across six states in the U.S. participated in a nationally funded project. Teachers’ STEM education lesson plans were analyzed to assess their 3D printing and STEM integration levels. Teachers’ beliefs were collected through a survey with rating scales adapted from previously validated surveys and several open-ended questions. Correlation analysis was conducted to investigate how teacher beliefs were associated with their 3D printing integration. Thematic analysis of the open-ended questions provided a detailed view on teachers’ experiences and perceptions, which further explained teachers’ beliefs and the impact on their 3D printing integration practice. This study revealed that teachers’ pedagogical, self-efficacy, and technology value beliefs were generally not correlated with their 3D printing integration practices except for a negative correlation between teachers’ self-efficacy in pedagogical content knowledge and their STEM integration levels. Teachers perceived 3D printing integration as beneficial for students, but they encountered a number of challenges including logistic and technical issues, lack of time and resources, insufficient ability to use 3D printers and connect 3D printing with curriculums, and challenges in teaching students with individual differences. Implications for practice and future research are discussed.

Girls Design with Code Club

1 month 3 weeks ago
Abstract

In this manuscript, we describe a coding club we created and implemented during the COVID-19 pandemic. We were purposeful in creating the club to: (a) focus on design and problem solving as the basis for learning computer coding and (b) include elements to improve the engagement of girls. We ran multiple iterations of a Girls Design with Code Club that involved over 100 girls from 22 countries. We reviewed various sources of data to evaluate how our design and implementation of the coding clubs impacted the girls who participated. In an effort to share our learnings with other researchers and program providers, we share evidence of choices that we believe had positive impacts and others that we can improve in future iterations.

Evaluating educational robotics as a maker learning tool for pre-service teacher computer science instruction

1 month 3 weeks ago
Abstract

Computer science teaching standards for grades K-8 have been implemented in nearly all U.S. states, and the core subject area teachers (e.g., math, science, English, social studies) have been asked to integrate these standards into their instruction. Thus, it is important that K-8 pre-service teachers of all subjects are both prepared and motivated to teach computer science concepts—such as programming—upon entering the field. However, little is known about how pre-service teachers learn and retain programming knowledge or obtain and sustain their motivation related to programming. Maker-focused educational robotics activities have the potential to both reduce abstract cognitive load and work as motivational tools for STEM learning. The purpose of this study was to examine pre-service teachers’ motivational persistence and retention of programming concepts after learning with educational robotics through maker-focused computer science activities. Hands-on maker robotics programming activities were used to teach and motivate pre-service teachers. This quantitative study utilized repeated measures through pre-, post-, and 6-month follow-up surveys and tests. The findings indicated the pre-service teachers’ programming comprehension gains exhibited on the posttest deteriorated substantially to near-baseline levels within 6 months of instruction. Conversely, pre-service teachers’ motivation related to programming continued to rise after the instruction had concluded. Both the retention of comprehension of programming concepts and motivational persistence findings imply that educator preparation providers should integrate programming instruction throughout their pre-service teacher curricula and support curricular initiatives that call for the integration of computer science instruction across pre-service teacher methods courses to reinforce computer science learning.

Design tensions in developing and using observation and assessment tools in makerspaces

1 month 3 weeks ago
Abstract

Makerspaces, especially in their diverse proliferating forms, support a broad variety of learning outcomes. There is rich work in attempting to understand and describe these learning goals. Yet, there is a lack of support for practitioners and educators to assess the learning in events and programming at makerspaces (and similar environments) without extensive videorecording and documentation. In this paper, we present our design iterations at adapting the Tinkering Studio’s Learning Dimensions Framework (LDF) into tools usable by makerspace facilitators. These tools are intended to support recording observations, to inform the design of events they organize. Coupling an activity theory perspective (Cole and Engeström in The Cambridge handbook of sociocultural psychology. Cambridge University Press, Cambridge, 2007) with Tatar’s (2007) Design Tensions framework, we highlight key categories of considerations that emerge in creating and implementing such an assessment system, namely, tools, terminology, and practice. These interlinked categories foreground the following tensions which expand our considerations for the practice of assessment in makerspaces: supporting real-time, informative observation increases granularity of data collected, but also imposes a cost on facilitator attention; using a common assessment framework across different facilitators requires developing and establishing shared vocabulary and understanding; and tool-driven assessments need repeated adaptation and responsiveness to different facilitator practices. Additionally, this analysis also surfaces the learning for facilitators themselves in such a co-design process of creating and implementing tools to understand, recognize and assess learning experiences through the lenses of personal and shared values around productive learning.

Understanding the educators’ practices in makerspaces for the design of education tools

1 month 3 weeks ago
Abstract

Makerspaces persist as formal and informal spaces of learning for youth, promoting continued interest in studying how design can support the variety of learning opportunities within these spaces. However, much of the current research examining learning in makerspaces neglects the perspectives of educators. This not only hinders our understanding of educators’ goals and how educators navigate makerspaces but also constrains how we frame the design space of the learning experiences and environments. To address this, we engaged in a set of semi-structured interviews to examine the contexts, goals, values, and practices of seven educators across five makerspaces. A thematic analysis of the data identified six key categories of competencies that these educators prioritize including a range of skills, practices, and knowledge, such as technical proficiency, communication, and contextual reflection. The analysis also identified five categories of strategies to accomplish certain goals, such as scaffolding, collaboration, and relationship building. Last, it also shed light on three categories of challenges faced at the student-level, teacher-level, and institutional level. We conclude with a discussion on our insights into how we can broaden the problem space in the design of educational technologies to support learning in makerspaces.

Using eye-tracking in education: review of empirical research and technology

2 months ago
Abstract

This study aims to provide a systematic review of recent eye-tracking studies conducted with children and adolescents in learning settings, as well as a scoping review of the technologies and machine learning approaches used for eye-tracking. To this end, 68 empirical studies containing 78 experiments were analyzed. Eye-tracking devices as well as the ever-evolving mechanisms of gaze prediction endorsed in the prior and current research were identified. The review results indicated a set of salient patterns governing the employment of eye-tracking measures and the inferred cognitive constructs in learning, along with the common practices in analyzing and presenting the eye-tracking data. Eye-tracking has been used to track engagement, learning interactions, and learning-relevant cognitive activities mainly in a research lab or a highly-controlled learning setting. The mechanisms of gaze capturing and prediction with learners in a dynamic and authentic learning environment are evolving.

How do they define design failure? An investigation of design failure in instructional design practice from the practitioners’ perspective

2 months 1 week ago
Abstract

In this paper, we report findings from a larger study that investigated design failure, as a phenomenon, in instructional design (ID) practice from the perspective of ID practitioners. Following an interpretive phenomenological study design, we interviewed 17 ID practitioners working in diverse settings, seeking their stories of design failure. Throughout the interviews and the analysis of the practitioners’ stories of design failures, we found that ID practitioners define design failure in different ways that are mostly not captured in design literature, that is: design failure as “failure during use of design,” design failure as “failure during process of design,” design failure as “an opportunity for reflection on design actions,” design failure as “an outcome or an event that needs to be avoided/prevented during the process of design,” and design failure as “an outcome or an event that could not be avoided/prevented during the process of design.” These findings and our discussions of them emphasize the generative role design failure plays in ID practice, the richness and the complexity of this phenomenon, and point to implications for IDT scholarship and ID education. In an upcoming paper, we report what ID practitioners attribute design failure to and what is the essence of design failure in ID practice.

Gamification enhances student intrinsic motivation, perceptions of autonomy and relatedness, but minimal impact on competency: a meta-analysis and systematic review

2 months 1 week ago
Abstract

Although many studies in recent years have examined the use of gamification as a motivational strategy in education, evidence regarding its effects on intrinsic motivation is inconsistent. To make the case for or against the adoption of gamification in education, this study examines its effects on students’ intrinsic motivation and the underlying motivational factors: perceived competence, autonomy, and relatedness. In this review, we analyzed the results of studies comparing gamified learning with non-gamified learning published between 2011 and 2022. The results of our meta-analysis of 35 independent interventions (involving 2500 participants) indicated an overall significant but small effect size favoring gamified learning over learning without gamification (Hedges’ g = 0.257, 95% CI [0.043, 0.471], p = .019) with no evidence of publication bias. Gamification also exerted a positive and significant effect on the students’ perceptions of autonomy (Hedges’ g = 0.638, 95% CI [0.139, 1.136], p = .012) and relatedness (Hedges’ g = 1.776, 95% CI [0.737, 2.814], p = .001), but minimal impact on competence (Hedges’ g = 0.277, 95% CI [0.001, 0.553], p = .049). To further investigate the possible reasons for the small impact on intrinsic motivation, a systematic review of 31 studies was conducted. The findings revealed two major challenges encountered in the adoption of gamification to increase students’ intrinsic motivation: students’ lack of perceived competence and lack of perceived autonomy in gamified classes.

Automated feedback on discourse moves: teachers’ perceived utility of a professional learning tool

2 months 2 weeks ago
Abstract

Technological tools that provide automated feedback on classroom teaching afford a unique opportunity for educators to engage in self-reflection and work towards improvement goals, in particular to ensure that their instructional environment is equitable and productive for students. More information is needed about how teachers experience automated professional learning tools, including what they perceive as relevant and impactful for their everyday teaching. This mixed-methods study explored the perceptions and engagement of 21 math teachers who used an AI-based tool that generates information about their discourse practices from classroom recordings. Findings indicate that teachers perceived the tool to have a high utility value, especially those who elected to use it over two school years. These teachers increased their use of talk moves over time, suggesting that they were making intentional changes due to their review and uptake of the personalized feedback. These results from this study speak to promising directions for developing AI-based professional learning tools that can support teacher learning and instructional improvement, particularly tools with robust perceived utility.

The effects of unplugged programming activities on K-9 students’ computational thinking: meta-analysis

2 months 2 weeks ago
Abstract

Due to its task-based, gamified, and interactive features, unplugged programming activity has been widely employed in education and teaching as an activity away from electronic screens and other digital devices. There is ongoing debate over how to help K-9 pupils develop their computational thinking through unplugged programming activities. Based on findings from 15 experimental and quasi-experimental studies conducted domestically and abroad between 2006 and 2023, this study adopts meta-analysis to quantitatively analyze the influence of unplugged programming activities on K-9 students’ computational thinking, focusing on an in-depth analysis of variables such as different students’ grade, gender, experimental period, research discipline, and unplugged programming activities categories. The results show that unplugged programming activity has a positive effect on the computational thinking of K-9 students (Hedges’ g = .631, 95% CI .463, .799, P < .001), and its promotion effect is affected by the variables such as students’ grade, learning time, subjects studied, and unplugged programming activities categories. Based on the conclusions of the meta-analysis, this paper proposes several suggestions. For example, it suggests paying attention to the cultivation of computational thinking among middle school students, and mastering the appropriate timing for unplugged programming activities to avoid the plateau effect. Additionally, it recommends strengthening the integration of unplugged programming activities across different disciplines, as well as promoting the use of storytelling and collaborative unplugged programming activities in the classroom.

Successful and unsuccessful mapping behaviors for learning procedural-type knowledge

2 months 3 weeks ago
Abstract

Mind mapping is a powerful technique that is often used for teaching declarative knowledge, but seldom implemented to record procedural knowledge. The present study focused on the latter. During a 12-week public presentation course, self-developed mind mapping software was utilized as a learning tool and an instrument to collect and analyze user behavior logs while summarizing and revising procedural knowledge. The participants were 53 working adults. They were divided into successful and unsuccessful mapping profiles based on their improvement. The pre- and post-tests on presentation skills, lag sequential analysis on log data, and interviews suggested that participants showing successful mapping behavior prioritized readability and ease of navigation of their maps. Their counterparts with unsuccessful mapping behavior tended to overload their maps and overuse highlighting. The discovery of actions and behavior patterns during the creation and revision of mind maps corresponding to successful/unsuccessful mind mapping profiles provides important suggestions to enhance existing digital mind mapping tools and to diagnose students who are falling behind. The implementation of mind mapping for procedural learning expands the area of mind mapping research and enlarges our understanding of teaching procedural knowledge.

A two-staged SEM: artificial neural network approach for understanding and predicting the factors of students’ satisfaction with emergency remote teaching

2 months 3 weeks ago
Abstract

This study seeks to address knowledge gaps regarding the role of self-regulated learning as a mediator in the relationship between interactions, internet self-efficacy, and student satisfaction. We conducted a survey of 1590 students from north Indian universities about their level of satisfaction, self-regulated learning, internet self-efficacy, and different interactions (learner-learner interaction, learner-content interaction, and learner-instructor interaction) during emergency remote teaching. By employing a two-stage SEM-ANN approach, this study contributes to methodological advancements and provides a comprehensive analysis of complex relationships. According to the findings, the identified factors are significant predictors of students’ satisfaction with online education in synchronous settings. Our research also shows that self-regulated learning fully mediates the effect of internet self-efficacy on student satisfaction during emergency remote teaching. This suggests that internet self-efficacy alone may not guarantee student satisfaction unless accompanied by self-regulated learning skills.

Block-based versus text-based programming: a comparison of learners’ programming behaviors, computational thinking skills and attitudes toward programming

2 months 3 weeks ago
Abstract

In the current era where computational literacy holds significant relevance, a growing number of schools across the globe have placed emphasis on K-12 programming education. This field of education primarily comprises two distinct modalities—the block-based programming modality (BPM) and the text-based programming modality (TPM). Previous research may not have provided a complete understanding of the differences between these two modalities as it did not take into account both the learning process and learning outcomes. This study aimed to compare secondary students’ programming behaviors, computational thinking skills, and attitudes toward programming between the two modalities through a quasi-experimental design in a Chinese secondary school. The findings showed that (1) learners in TPM encountered more syntactical errors and spent more time between two clicks of debugging, while learners in BPM had more code-changing behaviors by adjusting programming blocks, made more attempts of debugging, and had more irrelevant behaviors; (2) learners in BPM achieved a higher level of computational thinking skills; (3) learners in both modalities experienced a slight decrease in confidence and enjoyment, while learners in BPM had higher interest levels in programming. (4) Code Changer, Minimal Debugger, Maximal Debugger, Distracted Coder and Average Coder were identified through students’ programming behavior in the two programming modalities, and differences in their CT skills and attitudinal data were revealed. Lastly, pedagogical implications based on the findings are also discussed.

The effectiveness of design thinking on K-12 school students’ creativity in a maker curriculum

3 months ago
Abstract

The role of creativity has been well recognized in wholesome development of children’s personality and attitude. Among various approaches to foster creativity in children, design thinking (DT) has emerged as a significant approach. The study presented in this paper explores the implementation of DT into maker education to help K-12 students acquire creativity effectively. Majority of the studies in the literature have measured students’ creativity using a traditional creative ability scale, instead of work performance or both. We proposed a Three-Stage DT Framework (TSDTF) suitable for maker education, which has three stages, namely pre-design, in-design and post-design. TSDTF provides more specificity to indicate students’ learning process iteratively. Then we applied TSDTF to the 3D model-design curriculum to see the impact on the students’ work performance, learning achievement, and creativity ability. The study engaged two sixth-grade classes, with a total of 90 students, wherein the experimental group (45 students) was taught using TSDTF and the control group (45 students) was taught with traditional pedagogy. The results showed that there was no significant difference in basic knowledge of learning achievements for the students in both groups, while the students in the experimental group improved their work performance, operational skill of learning achievements, and creative ability more significantly than those in the control group. The qualitative analysis of the interview data revealed that the DT approach helped the students to be creative, thus enabling them to develop innovative works. The study showed that it is important for teachers to utilize the DT learning strategy in maker education in developing the students’ creative thinking.

Detector-driven classroom interviewing: focusing qualitative researcher time by selecting cases in situ

3 months ago
Abstract

In this paper, we propose a new method for selecting cases for in situ, immediate interview research: detector-driven classroom interviewing (DDCI). Published work in educational data mining and learning analytics has yielded highly scalable measures that can detect key aspects of student interaction with computer-based learning in close to real-time. These measures detect a variety of constructs and make it possible to increase the precision and time-efficiency of this form of research. We review four examples that show how the method can be used to study why students become frustrated and how they respond, how anxiety influences how students respond to frustration, how metacognition interacts with affect, and how to improve the design of an adaptive learning system. Lastly, we compare DDCI to other mixed-methods approaches and outline opportunities for detector-driven classroom interviewing in research and practice, including research opportunities, design improvement opportunities, and pedagogical opportunities for teachers.