This paper reports on a study aiming at examining the development of science process skills through the implementation of experiments in a science course which was performed in flipped classroom design. Flipped classroom model was implemented for a series of experimentation activities in a science classroom in which 38 science teacher candidates were enrolled. Both quantitative and qualitative data were used to acquire a better understanding of the improvement in science process skills. A test of Integrated Process Skills, participants’ experiment plans and interviews were used as data gathering tools. The results indicated that flipped learning in the experimentation process positively influenced especially identifying and stating hypotheses and operationally defining science process skills. It was observed that the provision of the required prior conceptual and procedural knowledge via watching videos as out of-school activities was an effective way in flipping the experimentation process. The affordances of videos and the guidance of the instructor played a key role on the teacher candidates’ laboratory oriented learning journey. Along with the results, some implications were provided for flipping the courses including such practical sessions as experiments.
Video lectures are one of the primary learning resources embedded in e-learning environments. Many universities prepare video lectures and share them as open course materials with students. In order to make the learning process more efficient and effective, it is very important to design and personalize those learning resources for learners, taking into their needs and cognitive differences. The literature regarding how cognitively diverse users benefit from which media design is scarce. Therefore, the purpose of this research is to explore the effect of video lectures types (voice over type, picture-in-picture type, and screencast type) and learners’ sustained attention levels (low, and high) on their learning performance in an e-learning environment. The research was designed as a 2 × 3 factorial design. In addition, learners’ eye movements have been recorded during study sessions. Results indicate that main effect of learners’ sustained attention levels and video lecture types on learning performance were significant. Furthermore, it was observed that both for the students with high level of sustained attention and for the students with low level of sustained attention, the use of picture-in-picture types of video lectures led higher learning performance scores. Similarly, the eye tracking analyses also supported this finding.
Problem based learning (PBL) supports the development of transversal skills and could underpin the training of a workforce competent to withstand the constant generation of new information. However, the application of PBL is still facing challenges, as educators are usually unsure how to structure student-centred courses, how to monitor students’ progress and when to provide guidance. Recently, the analysis of educational data, namely learning analytics (LA), has brought forth new perspectives towards informative course monitoring and design. However, existing research shows that limited studies have combined PBL with LA to explore their potential in offering data-driven, student-centred courses. This paper presents a framework, termed PBL_LA, that aims to address this gap by combining PBL with LA. The framework is populated from the literature and discussions with PBL and LA experts. The paper also presents results from redesigning, delivering and assessing ten courses in different disciplines and countries using the proposed framework. Results showed positive feedback on all different testing settings, exhibiting reliability of the framework and potential across countries, disciplines and sectors.
Developing computer-aided diagramming tools to mine, model and support students’ reasoning processes
Despite the last 40 years of research showing that computer-aided diagramming tools improve student learning, very little research reveals the cognitive processes that explain why and how diagramming tools and specific features of the tools affect learning. This study developed a tool that graduate students used to diagram and analyze arguments as the tool mined each students’ actions. The mined data was used to develop algorithms to operationalize and measure the use of backward, forward, breadth, depth-first reasoning. Regression models were then compared to identify which algorithm produced measures that best predicted diagram scores, and to determine the combined and relative impact of each reasoning process on diagram scores. The findings show that observing the placement and location of the first five nodes moved and positioned on screen in relation to the location of the previously moved node provides sufficient data to generate backward/forward and breadth/depth-first ratio scores that predict diagram scores, while individual frequency counts of each of the four processes do not predict scores. The best-fit regression model using the ratio scores show that students using more backward and depth-first processing construct diagrams with higher scores – scores based on criteria that gauge the depth of analysis and not just the number of correct diagram links. This study presents new tools, methods, and new lines of inquiry to advance research on ways to integrate learning analytics into computer-aided diagramming tools.
Effect of behavior patterns on the death of threads in asynchronous discussion forums: a study of informal learners from China
Asynchronous discussions have been widely used in education to support learning. Exploring the reasons why threads in asynchronous discussion forums die may help us improve our understanding of asynchronous discussions and lead to a more effective learning process. This study explored why threads in asynchronous discussion forums shut down. Two analytical methods, namely, content analysis and statistical analysis, were employed. The discussion postings were coded into various behavior patterns based on the work of Lu et al. (Computers in Human Behavior 27:946–955, 2011). Differences of behavior patterns between long lifespan threads and short lifespan threads were investigated and reasons why some threads die quickly was explored. The shortest lifespan threads that died after one or two comments were statistically analyzed. The results showed that there was a significant difference in the disagreement with added justification between long lifespan threads and short lifespan threads. Explanation oriented questions, disagreement against justification, fact oriented questions, new claim, thumbs up and thanks, to some extent, were significantly different between long lifespan threads and short lifespan threads. Agreement played a large part in shortest lifespan threads. Based on the results and further content analysis, many educational implications have emerged for designing asynchronous discussion activities and asynchronous discussion rules. For example, it is necessary to continuously encourage learners to express fresh ideas; rewards can be a way of encouraging high quality replies; learners need to give the reason or explanation why they agree or disagree with others’ postings; learners should be encouraged to ask more explanatory questions or factual questions; and off topic comments should not be forbidden.
Recognizing that digital citizenship includes the capacity to foster growth opportunities and also, can help to minimize risks and divides from the use of digital technology is essential for today’s society and education. This on-going study intends to identify core aspects that help or hinder youth participation and contribute on how to educate youth and improve on digital citizenship through the development of digital and socio-civic skills. The aim of this study is to develop a valid and reliable instrument that invites young people, aged 16–35, to assess their digital and socio-civic skills. The development process of the instrument involves a literature review, designing and constructing the dimensions and items of the questionnaire and finally, assessing the content and construct validity as well as the reliability of the instrument. We have obtained an instrument in English and Spanish with acceptable scientific guarantees; including statistical data regarding a confirmatory factor analysis (CFA) and favorable reliability coefficients using cronbach’s alpha.
Is too much help an obstacle? Effects of interactivity and cognitive style on learning with dynamic versus non-dynamic visualizations with narrative explanations
The aim of this study was to investigate the role of visual/verbal cognitive style and interactivity level in dynamic and non-dynamic multimedia learning environments. A group of 235 biology students learned about photosynthesis either from a computer-based animation or a series of static pictures with spoken explanatory text. Participants were randomly assigned to one of two conditions: with or without the possibility to pause, to play, or to fast-forward/rewind the learning environment (self-paced versus system-paced condition). Participants obtained better results when learning with the system-paced environment than with the self-paced one. A significant triple interaction between cognitive style, type of pacing, and type of visualization showed that highly developed visualizers learned poorer with self-paced static pictures than with system-paced static pictures. There were no significant effects regarding verbal cognitive style. Results shed more light on the relation between different levels of interactivity and visual cognitive style, when learning from static pictures.
An exploration of how learning design and educational technology programs prepare instructional designers to evaluate in practice
The purpose of this qualitative study was to examine how learning design and educational technology degree programs prepare students to evaluate in practice. This study involved the curricular mapping of 16 graduate instructional design programs and 29 semi-structured interviews with program faculty and recent postgraduates. Based on information shared during the interviews, this study produced nine themes according to three metathemes: (a) the role of problem-solving in evaluation, (b) alignment of evaluation activities in instructional design, and (c) relevancy of evaluation in instructional design. The findings revealed that evaluation was not prioritized for most programs, due to lack of time, client resources, employer lack of interest, and limited faculty experience in evaluation. Other recommendations are provided for how to enhance evaluative practices in instructional design coursework.
From massive open online courses (MOOC) to the smaller scale use of learning management systems, many students interact with online platforms that are meant to support learning. Investigations into the use of these systems have considered how well students learn when certain approaches are employed. However, we do not fully understand how course type or system design influence student actions and experiences, meaning prior findings cannot be properly interpreted and used because we do not understand how these factors influence those findings. Accordingly, we conducted a study to compare student experiences and behaviors across learning management systems and courses from a learning analytics perspective. Differences in student behaviors and experiences highlight how system design and the nature of the course interact: Students reported increased learning support when using a system that foregrounds student interaction through discussion forums, but this relationship did not hold across all course types. Similarly, students from the content-delivery focused system spent more time online while feeling less supported regardless of which type of course they were taking. This newly found evidence for the often-interrelated influence that the course and system have on student experiences and behaviors should therefore be considered when selecting a system to meet particular pedagogical goals.
Evaluating the usability and instructional design quality of Interactive Virtual Training for Teachers (IVT-T)
Simulation offers unique affordances over traditional training (e.g., remote access, mastery learning experiences, immediate feedback) relevant to teacher training in behavior management. This study describes a user-based evaluation of Interactive Virtual Training for Teachers (IVT-T). The study involved observing representative users (seven advanced education majors) perform benchmark tasks with the system, complete rating scales, and participate in interviews to evaluate the usability and instructional design quality of IVT-T. Global usability ratings based on established usability rating scales suggested IVT-T was adequately usable while observations of user performance and semi-structured interviews revealed design shortcomings that impeded effective user performance and informed ways to improve the interface. Observations of user performance, for example, identified 36% of usability problems related to learning, 19% = screen design, 17% = terminology; 3% = system capabilities and 25% = other problems. Cross analysis of user semi-structured interviews pointed to the system’s ability to convey believable, visually appealing, realistic characters and classrooms. More contextual cues, multiple challenging behaviors featured at the same time, and changes to the visual appearance of the classroom would enhance realism. Revisions made to enhance the usability and instructional design elements of IVT-T are discussed. In addition, implications for teacher educators and researchers involved in the development of instructional technologies are summarized along with the potential value of including simulation in teacher training for behavior management.
Instructional strategy is defined in abstract terms independent of instructional theories, philosophies, or standard formulas. Strategies share dimensions in common that allow instructional designers to communicate about them in theory-agnostic terms. These dimensions supply a common reference for comparing design viewpoints and discussing their relative strengths and applicability. This definition simplifies the design of strategies. It provides a stable framework for the instructional designer, regardless of loyalty to a philosophy or theory. Examples show how theories about strategy compare with regard to the framework’s dimensions.
Syntactic complexity in individual, collaborative and E-collaborative EFL writing: mediating role of writing modality, L1 and sustained development in focus
This study investigates the potential of individual, collaborative, and E-collaborative writing modalities on the development of syntactic complexity (SC), their sustained effect on SC development, and the potential meditating role of SC of L1 (First Language) in SC of L2 (Second Language). To this end, 90 Iranian intermediate EFL learners participated in individual, collaborative, and E-collaborative writing treatments for 10 sessions. L2 SC on three measures of pre, post, and delayed posttests in three writing modalities were assessed using multidimensional SC measures. Besides, the role of L1 syntactic complexity preferences in L2 SC was examined. The results of ANOVA (analysis of variance) and Repeated Measure ANOVA indicated that individual writing had the highest potential in developing SC of L2 in comparison with other writing modalities. As far as sustainable development is concerned, only the effect of collaborative writing was sustained in delayed posttest. Regression through mediation analysis indicated that the degree of syntactic complexity in L1 significantly mediated L2 SC under individual writing and its mediating effect faded away in collaborative and E-collaborative writing. Analysis of students' perceived efficacy of writing modalities through theme elicitation analysis of students' reflective essay suggests that the centrality of the role of instructors, group leadership and environmental supports in computer mediated learning need to be revisited. Implications and future research directions were discussed.
The study presents results of a tablet-based math game intervention to enhance early numeracy skills of children in Tanzania. Standard level 1 children (n = 122), attending a rural primary school, were randomly allocated to either intervention or control group. The intervention group participated in a daily intervention session for 46 days. Children’s performances in number identification, quantity discrimination, addition, subtraction, and missing number tasks were measured before and after the intervention with randomly selected children from both groups (treatment = 30, control = 31). Score gains in the intervention group were substantially greater than those in the control group. In particular, statistically significant effects of the intervention were identified in quantity discrimination, addition, and subtraction tasks. Item-level analyses using Item Response Theory showed that addition and subtraction items involving regrouping and most missing number items were too difficult even after the intervention. The study also identified which games were played the most or least during the sessions from play-log data and analyzed associations between children’s test performances and gameplays.
Science identity development: how multimodal composition mediates student role-taking as scientist in a media-rich learning environment
Science identity has been widely discussed in recent years; however, research on its development in multimodal composing environments, especially in formal classroom settings, has yet to be fully investigated. This qualitative study unraveled the science identity development of sixth-grade students as they created multimodal science fiction stories in a STEAM course. Thirty-two students enrolled in the course and worked in groups of 3–5, and each student self-selected one of three roles: designer, scientist, or writer. This study focused on the students (n = 9) who took the role of scientist and examined their science identity development. Data sources include digital surveys, semi-structured group interviews, and multimodal artifacts. Our qualitative analysis suggests that (a) composing with modes of choices could drive interests in science; (b) students connected science practices in classrooms with those in professional domains through taking the role of scientist; (c) taking hybrid roles (i.e., a combination of scientist and other roles) while composing with multiple modes contributed to the recognition of science in non-science careers. Based on these findings, we discuss the implications for cultivating positive science identities and engaging early adolescents in career exploration.
Exploring indicators of engagement in online learning as applied to adolescent health prevention: a pilot study of REAL media
Engagement is central to the effectiveness of online health messages and the related educational programs that aim to deliver these messages to the intended audience (Li et al. in Educ Tech Res Dev. https://doi.org/10.1007/s11423-019-09709-9, 2019). Drawing from health communication and social learning theories, the Theory of Active Involvement (TAI) (Greene in Health Commun 28:644–656. https://doi.org/10.1080/10410236.2012.762824, 2013) posits that an online prevention program’s impact depends on how engaged participants are. In practice, measuring engagement in this context has relied primarily on self-report measures (e.g., Hamutoglu et al. in Educ Tech Res Dev. https://doi.org/10.1007/s11423-019-09705-z, 2019). However, the emergence and growth of online learning platforms to deliver health-specific information offers other options for assessing engagement. This includes program analytics that capture interaction with content and facilitate examination of patterns via multiple indicators such as responses to interactive questions and time spent in the program (Herodotou et al. in Educ Technol Res Devel 67:1273. https://doi.org/10.1007/s11423-019-09685-0, 2019; Li et al. 2019; van Leeuwen in Educ Tech Res Dev 67:1043. https://doi.org/10.1007/s11423-018-09639-y, 2019). However, little is known about the relationships between these different indicators of engagement as it applies to health curricula. This study uses self-report, observational, and program analytic data collected on a small (N = 38) sample using REAL media, an online substance use prevention program, to examine relationships among various indicators of engagement. Findings suggest a cluster of indicators across the three modalities that provide a useful way of measuring engagement. A cluster centered around complexity suggests a separate factor to be considered when designing engaging interventions.
Classic studies of educational media have demonstrated that children can engage in quick, incidental word learning on the basis of a single exposure of a program. Since most words are learned from context, a lingering question has been whether the kind of contextual support affects word learning. Using a within-subjects design this study examined 102 low-income preschoolers’ word learning of digital episodes in three contextual settings: participatory, expository, and narrative contexts. Across three rounds, children’s word knowledge was assessed through researcher-developed measures. Results indicated that target word learning occurred most frequently in the participatory followed by the expository context, with narrative being the most challenging for children. In all cases, however, children with lower receptive language scores acquired fewer words than their higher language peers, suggesting that without additional supports, educational media might exacerbate rather than close the word gap.
The purpose of this study is to develop a standardized measurement tool that can be used to determine the participation styles of individuals participating in online instructional discussions. The scale consists of two dimensions called ‘Why’ and ‘How.’ The Why dimension comprises the main motivators of the participants’ participation in online instructional environments, whereas the How dimension comprises items expressing participation behaviors or forms. Two separate datasets were used for exploratory factor analysis (450 participants) and confirmatory factor analysis (693 participants), and the scale was applied to both undergraduate and graduate students. The scale consisted of eight factors, with four in the Why dimension and four in the How dimension. The results of confirmatory factor analysis showed that the scale was able to identify four different patterns of participation. Expressed as participation styles, these patterns are To Socialize/Connective, To Get Information/Analytical, To Discuss/Innovative, and To Fulfill Requirements/Practical. According to these results, the Participation Style Scale for Online Instructional Discussions was assumed to be a valid and a reliable measurement tool for determining the participation styles of participants of online instructional discussions. These four styles are thought to contribute to the instructors, the researchers, and the learners who want to benefit from effective learning in online environments. Instructors and researchers can determine learners’ participation styles before organizing discussion environments, and learners who think and gain awareness about their own participation styles can manage their discussion and learning processes more effectively.
Providing high-quality courses is of utmost importance to drive successful learning. This compels course authors to continuously review their contents to meet learners’ needs. However, it is challenging for them to detect the reading barriers that learners face with content, and to identify how their courses can be improved accordingly. In this paper, we propose a learning analytics approach for assisting course authors performing these tasks. Using logs of learners’ activity, a set of indicators related to course reading activity are computed and used to detect issues and to suggest content revisions. The results are presented to authors through CoReaDa, a learning dashboard empowered with assistive features. We instantiate our proposals using the logs of a major European e-learning platform, and validate them through a study. Study results show the effectiveness of our approach providing authors with more awareness and guidance in improving their courses, to better suit learners’ requirements.
There have been many waves of emerging learning technologies over the past few decades. Some of these waves are extended, some waves are connected, and other waves are repeated. The authors discuss the special journal issue from the standpoint of their personal involvement in many such waves during their careers. They also detail the evolution of this special issue and the potential audiences and stakeholders for it. In the end, they pose several questions and points to ponder in looking toward the future.
In this introduction to the special issue on systematic reviews on emerging learning environments and technologies, we introduce best practices for conducting systematic reviews and meta-analysis and discuss the need for a systematic review on emerging learning environments and technologies. We synthesize research on seven primary areas of emerging learning environments and technologies that include: (1) social media, (2) massive open online courses, (3) special education technology, (4) mobile learning, (5) game-based learning and gamification, (6) adaptive learning, and (7) learning analytics and introduce the thirteen articles that were included in this special issue. This article also provides implications for the readers on using and conducting systematic reviews.