ETR&D

Academic Communities of Engagement: an expansive lens for examining support structures in blended and online learning

4 days 14 hours ago
Abstract

In this article we share the Academic Communities of Engagement (ACE) framework, which describes a student’s ability to engage affectively, behaviorally, and cognitively in an online or blended course independently and with support. Based on Vygotsky’s (Mind in society: the development of higher psychological processes, Harvard University Press, Cambridge, 1978) zone of proximal development, the framework examines how a student’s ability to engage in online or blended courses increases with support from two types of communities. The course community is organized and facilitated by those associated with the course or program. The personal community is comprised of actors not officially associated with the course who have typically formed relationships with the student before the course or program began and may extend well beyond its boundaries. Actors within each community have varying skills and abilities to support student engagement, and a student is most likely to reach the necessary engagement for academic success with active support from both. The framework identifies the community actors most likely to provide specific support elements, aligning them to the different types of student engagement. The article outlines implications for practice and research, concluding with illustrative examples.

Utilizing online learning data to design face-to-face activities in a flipped classroom: a case study of heterogeneous group formation

1 week 1 day ago
Abstract

This study investigates the possibility of utilizing online learning data to design face-to-face activities in a flipped classroom. We focus on heterogeneous group formation for effective collaborative learning. Fifty-three undergraduate students (18 males, 35 females) participated in this study, and 8 students (3 males, 5 females) among them joined post-study interviews. For this study, a total of 6 student characteristics were used: three demographic characteristics obtained from a simple survey and three academic characteristics captured from online learning data. We define three demographic group heterogeneity variables and three academic group heterogeneity variables, where each variable is calculated using the corresponding student characteristic. In this way, each heterogeneity variables represents a degree of diversity within the group. Then, a two-stage hierarchical regression analysis was conducted to identify the significant group heterogeneity variables that influence face-to-face group achievement. The results show that the academic group heterogeneity variables, which were derived from the online learning data, accounted for a significant proportion of the variance in the group achievement when the demographic group heterogeneity variables were controlled. The interviews also reveal that the academic group heterogeneity indeed affected group interaction and learning outcome. These findings highlight that online learning data can be utilized to obtain relevant information for effective face-to-face activity design in a flipped classroom. Based on the results, we discuss the advantages of this data utilization approach and other implications for face-to-face activity design.

Faculty acceptance of the peer assessment collaboration evaluation tool: a quantitative study

1 week 6 days ago
Abstract

The problem this study sought to address was faculty reluctance to use new online peer-assessment tools. The purpose of this study was to examine the motivational factors that influence the acceptance of the Peer Assessment Collaboration Evaluation (PACE) Tool among faculty employed at a mid-sized university in the Southeastern United States. This study used Davis’s (1986) technology acceptance model (TAM) and motivational constructs “attitude toward using, perceived usefulness and perceived ease of use” (p. 44). The researcher used simple linear regression and standard multiple regression to determine if there was a significant relationship, if any, between the motivational constructs. The results of the linear regressions denoted positive, significant relationships between perceived ease of use of the PACE Tool and attitude toward using the PACE Tool, perceived usefulness of the PACE Tool and attitude toward using the PACE Tool; and perceived ease of use of the PACE Tool and perceived usefulness of the PACE Tool. The results of the multiple regression indicated that both perceived ease of use and perceived usefulness of the PACE Tool were positively, significantly related to attitude toward using the PACE Tool. Through faculty members’ speculations, the researcher was able to measure their motivation to use the PACE Tool. The results of this study demonstrated faculty members are motivated to use the PACE Tool, which indicates high acceptability and potential usage in the future. By understanding how faculty members perceive the PACE Tool, designers may be able to develop online peer-assessment tools that are more acceptable.

Investigating the feasibility of using assessment and explanatory feedback in desktop virtual reality simulations

2 weeks 3 days ago
Abstract

There is great potential in making assessment and learning complementary. In this study, we investigated the feasibility of developing a desktop virtual reality (VR) laboratory simulation on the topic of genetics, with integrated assessment using multiple choice questions based on item response theory (IRT) and feedback based on the cognitive theory of multimedia learning. A pre-test post-test design was used to investigate three research questions related to: (1) students’ perceptions of assessment in the form of MC questions within the VR genetics simulation; (2) the fit of the MC questions to the assumptions of the partial credit model (PCM) within the framework of IRT; and (3) if there was a significant increase in intrinsic motivation, self-efficacy, and transfer from pre- to post-test after using the VR genetics simulation as a classroom learning activity. The sample consisted of 208 undergraduate students taking a medical genetics course. The results showed that assessment items in the form of gamified multiple-choice questions were perceived by 97% of the students to lead to higher levels of understanding, and only 8% thought that they made the simulation more boring. Items within a simulation were found to fit the PCM and the results showed that the sample had a small significant increase in intrinsic motivation and self-efficacy, and a large significant increase in transfer following the genetics simulation. It was possible to develop assessments for online educational material and retain the relevance and connectedness of informal assessment while simultaneously serving the communicative and credibility-based functions of formal assessment, which is a great challenge facing education today.

Using feedback to promote student participation in online learning programs: evidence from a quasi-experimental study

2 weeks 3 days ago
Abstract

How should learner analytics and different media be used to optimize feedback to increase students’ motivation and sense of learning community in online learning programs? This study was designed to examine the usage of feedback delivery methods (text only, video only, or both) and learner analytics (individual vs. class average) to answer the above question. Two consecutive surveys were administrated to the students of a series of online courses over four semesters which resulted in a sample of 96. Using this quasi-experimental design, we aimed to capture changes in students’ perceived feedback quality, motivation, and sense of learning community when different feedback delivery methods and learner analytics were introduced. The findings revealed that students who received both video and text feedback were least motivated and lowest in their sense of online learning community when compared with students who received just video or text feedback. No significant differences were found between students who received video or text feedback regarding motivation and their sense of learning community. The findings also showed that when sharing class average, students’ motivation decreased. This study provides insights into how instructors might use media and learner analytics when designing feedback to motivate and promote student learning in online learning programs.

Sources of teachers’ self-efficacy for technology integration from formal, informal, and independent professional learning

2 weeks 3 days ago
Abstract

Teachers do not consistently maximize the potential of classroom technologies for enhancing student learning. As their self-efficacy is positively associated with technology integration, developing teachers’ self-efficacy could increase high-quality integration. We investigate how a holistic system of professional learning about technology integration including formal, informal, and independent professional learning might allow for access to and prompt reflection on self-efficacy source information. We interviewed six middle school math and science teachers from two schools where leadership teams participated in a leadership development program aiming to ultimately support teachers’ technology integration. To gather the teachers’ perspectives, we asked them to reflect on and explain visual displays of their reported confidence in technologies and frequency of professional learning. Through core assertions, we suggest a holistic system of learning supports teachers’ access to three sources of self-efficacy information (verbal persuasion, vicarious experiences, and mastery experiences) and prompts teachers’ reflection on source information such that it can impact their self-efficacy. We make recommendations for leaders to operate on these findings, as leaders are responsible for fostering the conditions necessary for teachers’ effective technology integration, as well as recommendations for future research to increase the generalizability and depth of understanding.

Towards emotion awareness tools to support emotion and appraisal regulation in academic contexts

2 weeks 3 days ago
Abstract

This paper studies learners’ emotion awareness in university level academic contexts as a first step to help learners regulate their emotions. Existing emotion awareness tools offer little information on learners’ emotions and their antecedents. This study created an emotion-reporting grid for university students based on the emotions they experienced daily. Students were interviewed based on their self-reported grid. A quantitative descriptive analysis of these retrospective interviews was conducted based on Pekrun’s control-value theory of achievement emotions. Student transcripts were analyzed based on the focus of their emotions (retrospective, activity, or prospective), the causes they attribute to their emotions (agent or external circumstances) and how they appraised the situation in which they experienced the emotions (value and control). We discuss the results with regard to the types of emotion-oriented and appraisal-oriented regulation strategies used in learning contexts and draw implications for the design of emotion awareness tools to support emotion regulation processes.

Managing items and knowledge components: domain modeling in practice

2 weeks 3 days ago
Abstract

Adaptive learning systems need large pools of examples for practice—thousands of items that need to be organized into hundreds of knowledge components within a domain model. Domain modeling and closely related student modeling are intensively studied in research studies. However, there is a gap between research studies and practical issues faced by developers of scalable educational technologies. The aim of this paper is to bridge this gap by connecting techniques and notions used in research papers to practical problems in development. We put specific emphasis on scalability—on techniques that enable relatively cheap and fast development of adaptive learning systems. We summarize conceptual questions in domain modeling, provide an overview of approaches in the research literature, and discuss insights based on the development and analysis of a widely used system. We conclude with recommendations for both developers and researchers in the area of adaptive learning systems.

Asynchronous discussion forum design to support cognition: effects of rubrics and instructor prompts on learner’s critical thinking, achievement, and satisfaction

2 weeks 3 days ago
Abstract

The purpose of this study was to test the effects of two metacognitive scaffolds on learners’ cognition by evaluating student critical thinking skills performance in an asynchronous discussion board and achievement in a blended learning module. The two scaffolds included a systematic protocol for individualized facilitation prompts and an analytic rubric with three criteria (critical thinking, participation frequency, and writing quality) along with four levels of achievement for each criterion. This research study employed a quasi-experimental, two-by-two factorial design. The study participants (N = 257) were assigned to one of four different conditions. Those provided with the rubric scaffold demonstrated significant differences with respect to their performances. However, the combination of both metacognitive scaffolds had a detrimental effect on learner performance. Additionally, learners reported higher satisfaction levels with the module when presented only with the rubric scaffold. Based on these results, the implications are discussed for those who design, facilitate, and manage asynchronous discussions and blended learning environments.

How research informs educational technology decision-making in higher education: the role of external research versus internal research

2 weeks 3 days ago
Abstract

Research use in educational decision-making has been encouraged and well documented at the K-12 education level in the United States but not in higher education, or more specifically for educational technology. We conducted a qualitative study to investigate the role of research in decisions about acquiring and using educational technology for teaching and learning in higher education. Results from 45 interviews of decision-makers in higher education show that they engage in different types of research activities throughout the decision-making process, but that in most cases the research is lacking in methodological rigor. Externally-produced, scientifically-rigorous research was mentioned in less than 20% of interviews. Decision-makers often conduct their own internal investigations on educational technology products and strategies producing locally-relevant, but usually less-than rigorous, evidence to inform decisions about continuing use of the technology or scaling up.

Enhancing pre-service teachers’ technological pedagogical content knowledge (TPACK): a mixed-method study

2 weeks 3 days ago
Abstract

The main aim of this two-step mixed-method study was to explore the effectiveness of the strategies used to prepare pre-service teachers for technological pedagogical content knowledge (TPACK). Specifically, we focused on the strategies included in the synthesis of qualitative evidence (SQD) model: (1) using teacher educators as role models, (2) reflecting on the role of technology in education, (3) learning how to use technology by design, (4) collaboration with peers, (5) scaffolding authentic technology experiences, and (6) providing continuous feedback. To explore the relation between the perceived occurrences of the SQD-strategies and TPACK (controlled for pre-service teachers’ general attitudes towards technology), survey data were collected from a sample of 688 final-year pre-service teachers in Belgium. In a next step, 16 telephone interviews and 6 in-depth interviews were conducted to gain a more in-depth insight into the nature of the 6 strategies and their influences on TPACK. The quantitative analyses indicated positive correlations between the SQD-strategies and TPACK, controlled for general attitudes towards technology. The findings from the qualitative analyses showed that teachers acknowledged the importance of the six strategies. However, the respondents emphasized that some of the six strategies are often underutilized. Based on the quantitative and qualitative results, the discussion provides recommendations to improve the potential of pre-service training to enhance future teachers’ TPACK.

The use of open data as a material for learning

2 weeks 3 days ago
Abstract

Open data has potential value as a material for use in learning activities. However, approaches to harnessing this are not well understood or in mainstream use in education. In this research, early adopters from a diverse range of educational projects and teaching settings were interviewed to explore their rationale for using open data in teaching, how suitable activity designs could be achieved, and the practical challenges of using open data. A thematic analysis was conducted to identify patterns and relationships in these open data-based practices that have already emerged. A document analysis of teaching materials and other related artefacts was used to augment and validate the findings. Drawing on this, common approaches and issues are identified, and a conceptual framework to support greater use of open data by educators is described. This paper also highlights where existing concepts in education and educational technology research, including inquiry-based learning, authenticity, motivation, dialogue, and personalisation can help us to understand the value and challenges of using open data in education.

Spatial task solving on tablets: analysing mental and physical rotation processes of 12–13-year olds

2 weeks 3 days ago
Abstract

Spatial skill assessment and training are promising fields of application for tablets, as touch-based interaction can prime and support mental transformations of spatial knowledge. We report on a study with 49 secondary school students who used our iPad app to solve mental and physical rotation tasks. During physical rotation, students were able to rotate 3D stimuli using touch interaction. Results show specific similarities (e.g., regarding angular disparity effects) as well as differences between mental and physical conditions, such as for task success, mental effort, efficiency; all to the advantage of the physical condition. 12–13-year olds can benefit from these advantages without previous task training, whereas previous research showed this to be different for younger students. In a second step, our analysis compares low and high achievers regarding physical rotation behaviour and motivational variables, including expected success. The results lay grounds for constructing individualized, tablet-based training apps for spatial skills.

Cognitive load theory and educational technology

2 weeks 3 days ago
Abstract

Cognitive load theory provides instructional recommendations based on our knowledge of human cognition. Evolutionary psychology is used to assume that knowledge should be divided into biologically primary information that we have specifically evolved to acquire and biologically secondary information that we have not specifically evolved to acquire. Primary knowledge frequently consists of generic-cognitive skills that are important to human survival and cannot be taught because they are acquired unconsciously while secondary knowledge is usually domain-specific in nature and requires explicit instruction in education and training contexts. Secondary knowledge is first processed by a limited capacity, limited duration working memory before being permanently stored in long-term memory from where unlimited amounts of information can be transferred back to working memory to govern action appropriate for the environment. The theory uses this cognitive architecture to design instructional procedures largely relevant to complex information that requires a reduction in working memory load. Many of those instructional procedures can be most readily used with the assistance of educational technology.

Computational thinking in compulsory education: a survey study on initiatives and conceptions

2 weeks 3 days ago
Abstract

This article communicates the results of a Danish survey study conducted in 2018 that aimed to examine initiatives relating to computational thinking in primary and lower-secondary schools, as well as the professional development of teachers and the perceptions of school principals in this area. The context is an increasing interest in this field, motivated by a sense that it is important for children to learn computational thinking skills. However, educators struggle with questions regarding what computational thinking in education actually is—and consequently, how they should teach and assess it. In this survey, we wanted to explore existing practices and current situations to find out what school principals regard as important; thus, we designed an electronic questionnaire on this topic. 98 principals started the survey, and 83 completed it. Our analysis suggests that many initiatives connected to computational thinking are currently being implemented, but according to the principals taking part, teachers are not trained to teach this subject. The principals have inclusive views and focus on broad aspects of what computational thinking involves. According to them, computational thinking is not about pushing students into computing careers; rather it is about supporting the well-rounded development of human beings in a free and democratic society. However, the principals do report limited understanding of this subject, which suggests that teachers are not the only ones in need of training—principals also need help to develop a culture and mindset around this subject and implement it efficiently into schools.

An exploratory quantitative case study of critical thinking development through adult distance learning

2 weeks 3 days ago
Abstract

Critical thinking is a metacognitive process that, through purposeful, self-regulatory reflective judgment; skills of analysis, evaluation and inference; and a disposition towards thinking, increases the chances of producing a logical conclusion to an argument or solution to a problem. Critical thinking is vital for not only educational achievement, but also continuous professional development, as it is necessary in social and interpersonal contexts, where adequate decision-making and problem-solving are necessary on a daily basis—which is of particular relevance to adult distance learners, many of whom return to learning to further their professional development. Though a large body of extant research focuses on traditional BA student’s critical thinking performance, there is a dearth of research conducted on more mature adults’ critical thinking. ADL provides a unique opportunity to explore these abilities and the effects of critical thinking instruction through distance learning on such performance. Given the potential benefits linked with critical thinking and associated with adult distance learning, the aim of the current, retrospective, exploratory case study was to examine the effects of an adult distance learning critical thinking module, taught through a BA Training and Education programme, on critical thinking performance. A series of six paired samples t-tests were conducted in order to assess the performance of 95 ADLs from pre-to-post-testing on overall critical thinking performance and the critical thinking sub-scales of hypothesis-testing; verbal reasoning; argument analysis; judging likelihood an uncertainty; and problem-solving. Correlational analysis was also conducted. Results revealed a significant increase from pre-to-post-intervention on overall critical thinking performance, as well as all critical thinking sub-scale performances. However, there were no effects of active engagement with the module, disposition towards thinking or motivation towards learning on critical thinking performance. Results are discussed in light of theory and research on critical thinking.

Typology of motivation and learning intentions of users in MOOCs: the MOOCKNOWLEDGE study

2 weeks 3 days ago
Abstract

Participants in massive open online courses show a wide variety of motivations. This has been studied with the elaboration of classifications of the users according to their behavior throughout the course. In this study, we aimed to classify the participants in the MOOCs according to the initial motivations and intentions, before long interaction with the online device. Using a survey of 1768 participants in 6 MOOCs, we classify the participants according to: internal motives, external motives and intention of persistence. Three profiles of involvement in the course were identified: poorly motivated (16.7%), self referential (28.8%) and highly committed (54.5%). All three profiles showed significant differences in self-reported learning experiences at the end of the course. The intensity of the initial motivation was positively related to the satisfaction and perceived quality of the training experience. According to our analysis, identifying motivational profiles before starting the course allows to diagnose in advance the educational use and the diversity of individual training itineraries.

Effects of robotics programming on the computational thinking and creativity of elementary school students

2 weeks 3 days ago
Abstract

Around the world, programming education is actively promoted by such factors as economic and technical requirements. The use of a robot in programming education could help students understand computer-science concepts more easily. In this study we designed a course in programming a robot for elementary school students and investigated its effectiveness by implementing it in actual classes. We further examined the effects of students’ prior skills and of gender on the outcomes. In addition, we reviewed the applicable teaching and learning strategies in the field of robotics programming. Our course in programming a robot was implemented for 155 Korean elementary school students in the fifth and sixth grades. The course was conducted for 11 weeks. Our results show that teaching programming by using a robot significantly improved computational thinking and creativity. Computational thinking, however, was not significantly improved in the group that initially showed high scores. Further, creativity was improved more in girls than in boys, and the mean difference was statistically significant, but the difference in computational thinking was not. The implication of this study is that the best approach is to design a course in programming a robot and apply it in actual classrooms in order to discuss teaching and learning strategies according to students’ prior skills and their gender.

Ask Dr. Discovery: the impact of a casual mobile game on visitor engagement with science museum content

2 weeks 3 days ago
Abstract

This study examines the impact of a mobile game app on science museum visitors’ level of engagement with exhibit content, compared to a non game-based version of the same app. Ask Dr. Discovery (Dr. D) is a question-asking app containing two versions: a Game Mode employing casual game mechanics and an Ask Mode providing a baseline version. We implemented both versions of Dr. D at two science museums located in the Southwestern United States with 1539 participants. In both conditions, participants could type or speak questions to receive vetted answers about museum content, but only Game Mode embedded question-asking within a simple game. Participants’ level of engagement was represented by the number of questions asked about exhibit content in Dr. D. Additionally, we explored the relationship between app engagement and participants’ self-reported level of interest in science. All participants completed pre- and post-questionnaires with questions related to science interest, impressions of the Dr. D app, and demographic information. Results in both museums indicated that users of Game Mode asked nearly twice as many questions on average as participants using Ask Mode. Science interest predicted engagement at one of two sites. Demographic variables, including gender, age, and race/ethnicity were not found to influence the rate of question asking in either mode. These results indicate that employing simple game mechanics in apps for museum visitors may lead to strong positive impacts on visitor engagement with museum content.

Examining the role of sentence openers, role assignment scaffolds and self-determination in collaborative knowledge building

2 weeks 3 days ago
Abstract

Scaffolds establish a cognitive connection with the students and what they want to express. Supporting the collaborative knowledge building process with scaffolds is crucial for the participation and continuity in the online discussions. In this research, where a quasi-experimental design is used, the contributions of the students in the online collaborative knowledge building process are examined in terms of role assignment, sentence opener scaffolds, and self-determination. 77 teacher candidates, who are registered to Computer II course, are assigned to 4 groups, in three of which scaffolds are used, and in the remaining one of which scaffolds are not used. The students contribute to the knowledge building process in the first group by using the sentence openers, in the second group by being assigned with roles, in the third group by both being assigned with roles, and using the sentence openers appertaining to the respective roles, and in the fourth group by not making use of any scaffold. Using content analysis and MANOVA, the research results reveal that using scaffolds, especially the combination of sentence openers and role assignment scaffolds encouraged higher cognitive levels of knowledge building. Significant differences with high effects were found between the groups for the dimensions of self-determination: self-awareness and perceived choice. The research points out some suggestions for future research.