Peer evaluation consists of the evaluation of students by their peers following criteria or rubrics provided by the teacher, where the way to evaluate students is specified so that they achieve the desired competencies. The quality of the measurement instrument must meet two essential criteria: validity and reliability. In this research, we explored the educational value of peer evaluation rubrics by analyzing the quality of the rubric through the study of content validity, reliability, and internal consistency. Our main purpose was to design an appropriate rubric to grade tasks in the field of information engineering, as well as performing content validation through a group of experts. It was carried out in three phases: 1) construction of a rubric, with its criteria, characteristics, and levels of achievement; 2) content validation by five experts in the field, and 3) application of the rubric to ascertain students' perceptions and satisfaction with its validity. The relevance of the criteria and the definition of their characteristics obtained a score higher than 3.75/4 on a Likert scale. The content validity values (CVR), content validity index (CVI), and general content validity index (GIVC) gave maximum values of + 1. The results obtained indicate that the rubric is adequate, with Aiken’s V higher than V 0.87 in all its criteria. The rubric was applied to 326 students of 4 subjects. Cronbach's alpha was used to calculate the reliability of the rubric, obtaining a value of 0.839. The students' perception of validity and satisfaction with the rubric was higher than 0.78. As future work, we intend to design a rubric validation engine according to the applied procedure.
Journal of Computing in Higher Education
Depicting the reason for the mismatch between instructor expectations of students’ performance in advanced courses and their actual performance has been a challenging issue for a long time, which raises the question of why such a mismatch exists. An implicit reason for this mismatch is the student’s weakness in prerequisite course skills. To solve this challenge, this research proposes a new graph mining algorithm combined with statistical analysis to reveal the dependency relationships between Course Learning Outcomes (CLOs) of prerequisite and advanced courses. In addition, a new model is built to predict students’ performance in the advanced courses based on prerequisites. The contributions of this research are threefold: (1) Modeling: three models are built based on bipartite graphs. The first model is a bipartite graph to model the relationships between the CLOs of different courses. This bipartite graph is constructed using a new calculated dependency measure based on a statistical analysis of students’ grades. Then the relationships between the Learning Outcomes are discovered by extracting the maximal bipartite cliques in the graph. The second model is built to model the relationships between students and the prerequisite CLOs. The maximal bipartite cliques are then extracted from this graph to discover the maximal set of students who share the same study weaknesses. The third model is built to model the relationships between students and the advanced CLOs. The maximal bipartite cliques are then extracted to discover the maximal set of students who are expected to share the same study weaknesses in the advanced course. Therefore, the same remedial actions can be used towards this group of students. (2) Algorithm: A new maximal bipartite cliques enumeration algorithm is proposed to extract the targeted patterns and relationships between CLOs themselves and between students and CLOs. (3) Applicability: The proposed models and algorithm have been applied using a real educational data set collected from one university. Other real datasets are used to conduct an empirical evaluation to assess the maximal bipartite enumeration algorithm’s correctness, the running time of the inflation and enumeration steps, and the overhead of the inflation algorithm on the size of the generated general graph. The evaluation proves that the proposed algorithm is accurate, efficient, effective, and applicable to real-world graphs more than the traditional algorithm.
Automated text detection from big data scene videos in higher education: a practical approach for MOOCs case study
Automated text detection and analysis holds incredible potential for research in higher education. It is challenging because higher education institutes produce an enormous amount and variety of texts, letters, articles, books, reports etc. Futuristic E-learning based education replaces the difficulty of understanding the semantic meaning of the learning content from videos which is most prominent source used by the leaners to acquire knowledge. Therefore, Content Based Video retrieval has become the challenging research area under pattern recognition and computer vision in higher education through Massive Open Online Courses (MOOCs). Text plays a dynamic role in understanding the true meaning of behavior of the video. Hence, it is challenging to detect and identify the text in video due to variable complex background, low contrast, blur, poor illumination, font size, font-style, occlusions. The traditional approach of end-to-end convolution neural network (CNN) performs satisfactory in detecting video text. However, it is also important to deal with the video size, therefore, we have adopted Map Reduce technique to store the video content and utilize it efficiently by parallel computing. Followed by this, we employed novel approach to clean up the video frames to feed to neural network model based on region proposal network (RPN) with CNN by finding appropriate anchor ratios to extract the text candidates. Finally, we train our model with extracted frames to predict for the test videos. The proposed method is evaluated on ICDAR Video text benchmark datasets and few publicly available test datasets to achieve high recall.
Data Analytics has become an essential part of the Internet of Things (IoT), mainly text analytics-related applications, since they can be utilized to benefit educational institutions, consumers, and enterprises. Text Analytics is excessively used in Smart Education after the emerging technologies such as personal computers, tablets, and even smartphones transformed the education system and improved the teaching methods by helping the teachers to evaluate the students' performance or determine the degree of similarity between a lecturer’s and the students’ posts in the discussion forum, and by collecting the students’ feedback on the teaching method, in order to categorize each feedback into positive or negative, which will help the lecturers in optimizing their way of teaching. In this paper, we highlight the main components of IoT analytics, along with a comprehensive background of text analytics used techniques and applications. This paper provides a comprehensive survey and comparison of the leveraged IoT Text Analytics models and methods in Smart Education and many other applications.
Does one size fit all? Investigating the effect of group size and gamification on learners’ behaviors in higher education
Gamification—the use of game elements in serious contexts, has been prevalent to enhance users’ motivation and engagement in difficult activities. In the literature related to higher education, the use of gamification has emerged as a new pedagogical approach in order to improve students’ learning behaviors. On the other hand, traditional education research suggested that working in groups can enhance students’ learning behaviors. However, no study has been found in the literature that investigates these two distinct concepts in education domain. Therefore, this research aims to explore the effect of different group sizes and gamification on students’ learning behaviors. For this purpose, the study has explored the comparison between gamification and traditional classroom settings on students’ learning behavior with different group sizes: individual, small group, and large group settings. Further, the comparison of students’ learning behaviors in gamification environment within different group settings over time has also been investigated in this research. The analysis suggests that different group sizes can have varying impacts on students’ perception of the course in gamification environment over time. Moreover, it was observed that group size only affects students’ interest, comparison, and discouragement in gamification environment, but does not affect their effort, perceived choice, perceived competence, tension, or motivation. Also, it was found that gamification does not affect the perceived competence of students in any of the group settings. These results can be useful in future decisions about the optimal classroom size, group activities, and group sizes in other activities in larger classrooms.
The flipped classroom model has gained prominence as advances in technology afford increasing opportunities for ubiquitous access to a variety of online resources. Despite the benefit of the flipped classroom model, flipped classrooms are not equally advantageous to all students due to its self-regulated nature. To address the issues in flipped learning, we explored principles for supporting self-regulated learning in flipped learning by synthesizing suggestions provided in previous research. We also conducted an empirical study to validate the identified principles by implementing a self-regulated learning support that combined a learner dashboard with a reflection interface in a real flipped classroom setting. While the dashboard interface utilized students’ learning traces to support students’ self-monitoring and evaluation, the reflection interface facilitated their follow-up reflection, which contributed to the cyclical process of self-regulated learning. The results indicated that the experimental group that used the support for self-regulated learning exhibited higher levels of self-regulated learning skills, behavioral engagement in pre-class sessions, cognitive engagement in in-class sessions, emotional engagement in both pre- and in-class session, learning performance than the control group. Implications for future research and directions for design and implementation of self-regulated learning supports are described.
Gaining insight from survey data: an analysis of the community of inquiry survey using Rasch measurement techniques
This article presents the results of evaluating a dataset collected with the Community of inquiry (CoI) survey (Arbaugh, The International Review of Research in Open and Distributed Learning 9:1–21, 2008) using Rasch psychometric techniques to evaluate instrument functioning. Data were collected over a two-year period yielding a sample of 704 survey responses from students who were enrolled in a blended online graduate program. The purpose of this article is to present a Rasch analysis of the CoI survey to provide insight into the functioning of the instrument beyond other statistical analyses of the CoI that have been conducted to date. The results of the analysis provide new insights into the functioning of this measurement instrument and demonstrate the usefulness of Rasch techniques. The rationale for using Rasch techniques as well as the implications of this technique when using the CoI survey when conducting research or evaluations of practices in blended online courses are discussed.
The purpose of this study was to investigate the effect of mobile phone usage policies on college students’ learning. Based on quasi-experimental research, with pretest–posttest nonequivalent group design, two pre-existing groups were randomly assigned treatment conditions, namely the removal of students’ mobile phones (Restricted Phone Access), and the allowance for students’ mobile phone usage (Unrestricted Phone Access) during class lectures. Data were collected from 63 college students, of which 25 were in the Restricted Phone Access group and 38 in the Unrestricted Phone Access group, using pretest and posttest. The results of a mixed analysis of variance test showed that the change in students’ scores from pretest to posttest was significantly greater for the Restricted Phone Access group than the Unrestricted Phone Access group, although there was a statistically significant increase seen in the students’ test scores from pretest to posttest regardless of any policy on mobile phone usage. This study discusses the theoretical and practical implications, and then recommendations were put forward with regards to future studies in this area.
An investigation of under-represented MOOC populations: motivation, self-regulation and grit among 2-year college students in Korea
Educators have raised concerns that massive open online courses (MOOCs) mainly serve the interests of advantaged groups. In response, this study examined underrepresented MOOC learners; namely, 2-year college students in South Korea in terms of their perceptions of MOOCs and learning readiness for MOOCs. A total of 119 Korean 2-year college students participated in the survey and their responses were analyzed. Research findings revealed that approximately 90% of participants were unaware of MOOCs, and few students had previously taken a MOOC. These results indicate that it is necessary to advertise MOOCs effectively to underrepresented learners. Importantly, most participants were optimistic about the effects of MOOCs for individual development. Some learners, however, were concerned about their lack of commitment or low self-regulation to complete MOOCs. In terms of intrinsic and extrinsic motivation, grit, and self-regulation, the learning readiness of 2-year college students for MOOCs was moderate. Additionally, 2-year college students preferred MOOCs with practical content offered in short study periods, and they emphasized extrinsic motivators over intrinsic ones.
This study implements a design-based research approach to design and evaluate different scaffolding strategies for supporting student learning as well as promoting student agency within a computational science course. The course introduces computational methods and tools in the context of disciplinary problems for materials science and engineering students. Initial course offerings suggested that students were overwhelmed by the interdisciplinary nature of the course. Therefore, the research team evaluated different scaffolding strategies for supporting students’ learning, and how those may have provided students with agency to self-scaffold when needed. Three rounds of data collection included 17 students who participated in individual semi-structured interviews to explore how they used (or not) different scaffolds. Five of the participants were recruited for the first iteration; six of them were recruited in the second iteration, and six more in the third one. The iterative process allowed us to adapt the scaffolding procedures for the third iteration from the data collected in iterations 1 and 2. The purpose of this study is to understand how students used different scaffolds, and what implementation strategies were effective according to student uses of these scaffolds in the context of computational science. The results suggest that students developed agency to self-scaffold when needed, as they benefited from multiple scaffolds at different steps of the problem-solving process. Moreover, providing worked examples without engaging students in their active exploration can be ineffective, but this engagement can be achieved using written explanations. Additional support may be needed at an early stage of skill development, so students have an idea of how to validate their model.
Exploring the effects of corpus-based business English writing instruction on EFL learners’ writing proficiency and perception
This exploratory research presents the implementation and evaluation of the effects of integrating corpus consultation with business English writing instruction. The subjects consisted of English as a foreign language (EFL) learners enrolled in two undergraduate business English writing classes. Two groups of EFL students were randomly assigned, one group (n = 49) receiving corpus-based writing instruction constructed on a Moodle course management system (CMS), while the other group (n = 58) was given traditional lecture-based instruction. A mixed methods design combining qualitative and quantitative approaches has been chosen to investigate the overall effect of the corpus-based intervention on the improvement of business letter writing performance in aspects of lexical and syntactic complexity, as well as learners’ perceptions. The comparison of the pre- and post-tests of writing revealed a significant difference between the experimental and control groups after the instruction. Significant differences in students’ lexical and syntactic complexity were found between the pre- and post-test of the experimental group. Further, in response to a questionnaire survey and interview, the students stated they improved their writing skills regarding vocabulary, syntactic structure and content in general, and their writing confidence and linguistics awareness were also enhanced. The results suggest that the corpus provides useful resources to supplement existing materials.
Does project focus influence challenges and opportunities of open online education? A sub-group analysis of group-concept mapping data
Openness in education is not a consistent term or value since “open” is used to describe various things and often means different things to different individuals. In a research context, it is important to identify the many interpretation(s) and perspectives of openness being investigated, especially since the underlying ideas behind these different interpretations and contexts can yield different results. Not much empirical research on the implementation aspects of open education exists, especially comparing open educational resources (OER) and open online education (OOE). This empirical study addresses this gap, exploring identification and prioritization of organizational challenges and opportunities of two subgroups of projects (i.e. OER focused or OOE focused) within various higher education institutions in The Netherlands. The main research question in this study is: Does the project character (OER focus vs. OOE focus) of innovation projects lead to perceived differences by actors involved in their implementation? Findings indicate that there are differences in conceptual as well as practical representation between the two groups. These findings imply that higher education institutions need to internally adapt to the needs of various manifestations of “openness” to be able to fully benefit from opportunities and overcome challenges.
Partial versus full captioning mode to improve L2 vocabulary acquisition in a mobile-assisted language learning setting: words pronunciation domain
Video captioning has been investigated extensively in the Computer-Assisted Language Learning (CALL) literature to aid second language vocabulary acquisition. However, a little is known about how video captioning could foster learners’ pronunciation, which is a component of second language vocabulary acquisition proposed by Nation (Nation, Learning vocabulary in another language, Cambridge University Press, 2001), when attending to video captioning. Therefore, this study aims to investigate the effect of two types of video captioning, namely, full versus partial captioning, on mastery of word pronunciation. Furthermore, we tested the magnitude of the cognitive load imposed by video captioning types using NASA TLX. A total of 55 Arab English as a Foreign Language learners watched videos with full, partial or no captioning. Their perceptions about learning with captioning were also surveyed. Results of the pre–post-tests indicated that the captioning groups’ performance in the pronunciation tests outscored the no captioning group. In turn, the partial captioning group’s scores were slightly higher than those of the full captioning group. However, this difference was statistically insignificant. Cognitive load was found higher in full captioning and no captioning than that in the partial captioning mode. The participants showed highly positive attitudes towards learning with captions.
Procedural problem solving is an important skill in most technical domains, like programming, but many students reach problem solving impasses and flounder. In most formal learning environments, instructors help students to overcome problem solving impasses by scaffolding initial problem solving. Relying on this type of personalized interaction, however, limits the scale of formal instruction in technical domains, or it limits the efficacy of learning environments without it, like many scalable online learning environments. The present experimental study explored whether learners’ self-explanations of worked examples could be used to provide personalized but non-adaptive scaffolding during initial problem solving to improve later performance. Participants who received their own self-explanations as scaffolding for practice problems performed better on a later problem-solving test than participants who did not receive scaffolding or who received expert’s explanations as scaffolding. These instructional materials were not adaptive, making them easy to distribute at scale, but the use of the learner’s own explanations as scaffolding made them effective.
Gamification in education: a mixed-methods study of gender on computer science students’ academic performance and identity development
Underrepresentation of women in computer science (CS) increasingly demands the necessity to find and enhance current learning engagement approaches to bring more women into computing fields. Some researchers have been exploring the influence of gamification on female students as one of these possible learning engagement strategies. Gamification refers to the introduction of video game elements into non-game activities to enhance engagement and motivation. Previous studies have reported mixed results of the impact of gamification on women. In this study, we introduce SEP-CyLE (Software Engineering and Programming Cyberlearning Environment), an online gamified tool that was designed to provide supplemental computing content to students. This paper presents a convergent mixed-methods study guided by social identity theory and self-efficacy to understand women's experiences with this gamified tool. More specifically, this study explores virtual points' and leaderboards' effects on CS identity development, self-efficacy, and performance. The results show that virtual points and the leaderboard contributed to improved performance for students of all genders, suggesting that gamification is a gender-neutral learning engagement strategy that improves female students' performance as much as male students. Regardless of improved performance, most women did not actively enjoy or were motivated by the virtual points or leaderboard in SEP-CyLE. Additionally, gamification had no significant impact on CS identity development or self-efficacy constructs and had little to no impact on women's interest and engagement in the field of computing.
Multilayered-quality education ecosystem (MQEE): an intelligent education modal for sustainable quality education
Sustainable quality education is a big challenge even for the developed countries. In response to this, education 4.0 is gradually expanding as a new era of education. This work intends to unfold some hidden parameters that are affecting the quality education ecosystem (QEE). Academic loafing, unawareness, non-participation, dissatisfaction, and incomprehensibility are the main parameters under this study. A set of hypothesis and surveys are exhibited to study the behavior of these parameters on quality education at the institution level. The bidirectional weighted sum method is deployed for precise and accurate results regarding boundary value analysis of the survey. The association between parameters understudy and quality education is illustrated with correlation and scatter diagrams. Academic loafing, the hidden and unintended rudiment that affects the QEE is also defined, intended and explored in this work. The study exhibits that the average percentage association between quality education and all the parameters under study is 93.32%, whereas awareness has the least association (82.63%) and academic loafing has the highest association (99.35%) with quality education. The paper proposes a cognitive-IoT (internet of things) based multilayered QEE as a remedial solution for sustainable quality education. The emerging demand of real-time data processing for the education 4.0 environment, makes MQEE suitable for education 4.0 environment. The IoT enabled heterogeneous-data preprocessing, integration, and analysis to foster the proposed model with robustness, scalability, and flexibility. The proposed abstraction mechanism, public/private reporting, and IoT-based data preprocessing system are rich enough to handle data management issues under education 4.0 environment.
Learning and development roles and competency domains in higher education: a content analysis of job announcements
Learning and Development (L&D) roles are important to organizations for improving employee’s knowledge and skills. This study examined various roles and competency domains required of learning and development professionals in higher education. Ten different roles of L&D professionals were examined for 20 competency domains through a qualitative coding process. We extracted and analyzed 294 unique postings from the job board, higheredjobs.com. Results indicated that designers and directors are the most advertised L&D postings. The top five competency domains required of L&D professionals were collaboration, communication, content development, project management, and assessment and evaluation. Communication and collaboration skills were required for most of the roles. Leadership and people management were ranked highest for executives and directors. In addition, competency domains aggregated by roles are provided. Implications are provided for employees, L&D graduate programs and professionals, and researchers.
Improve teaching with modalities and collaborative groups in an LMS: an analysis of monitoring using visualisation techniques
Monitoring students in Learning Management Systems (LMS) throughout the teaching–learning process has been shown to be a very effective technique for detecting students at risk. Likewise, the teaching style in the LMS conditions, the type of student behaviours on the platform and the learning outcomes. The main objective of this study was to test the effectiveness of three teaching modalities (all using Online Project-based Learning -OPBL- and Flipped Classroom experiences and differing in the use of virtual laboratories and Intelligent Personal Assistant -IPA-) on Moodle behaviour and student performance taking into account the covariate "collaborative group". Both quantitative and qualitative research methods were used. With regard to the quantitative analysis, differences were found in student behaviour in Moodle and in learning outcomes, with respect to teaching modalities that included virtual laboratories. Similarly, the qualitative study also analysed the behaviour patterns found in each collaborative group in the three teaching modalities studied. The results indicate that the collaborative group homogenises the learning outcomes, but not the behaviour pattern of each member. Future research will address the analysis of collaborative behaviour in LMSs according to different variables (motivation and metacognitive strategies in students, number of members, interactions between students and teacher in the LMS, etc.).
The design and development of an open educational resources intervention in a college course that manifests in open educational practices: a design-based research study
Shifting from open educational resources (OER) to open educational practices (OEP) is the next stage in the OER movement, but there have been few attempts to understand how the next phase in this movement will be achieved. Addressing this question can uncover the potential benefits of OER besides cost reduction. The current study represents the Enactment Phase (design and development) of a larger design-based research study that sought to design an integrative OER intervention in a college course to promote OEP. The Enactment Phase resulted in developing the design principles that describe the integration of OER use and creation into a college course; the development of the components of the OER intervention prototype; and designing the OER intervention prototype in a college course at a mid-Atlantic research university. The significant results that emerged from this Enactment Phase reside in that OER should be integrated into a course using learner-centered pedagogical models with constructivist approaches to teaching, and it should be integrated as an integral part of the syllabus of the course. The main idea for integrating the 5Rs into the Advanced Instructional Design was threading across assignments to make a connection between knowledge and skills students have learned throughout the course.