Journal of Computing in Higher Education

This paper presents an analysis of learning analytics practices which aimed to achieve personalised learning. It addresses the need for a systematic analysis of the increasing amount of practices of learning analytics which are targeted at personalised learning. The paper summarises and highlights the characteristics and trends in relevant learning analytics practices, and illustrates their relationship with personalised learning. The analysis covers 144 related articles published between 2012 and 2019 collected from Scopus. The learning analytics practices were analysed from the dimensions of what (learning context, learning environment, and data collected), who (stakeholder), why (objective of learning analytics, and personalised learning goal), and how (learning analytics method), as well as their outcomes and limitations. The results show the diversified contexts of learning analytics, with the major ones being tertiary education and online learning. The types of data for learning analytics, which have been increasingly collected from online and emerging learning environments, are mainly related to the learning activities, academic performance, educational background and learning outcomes. The most frequent types of learning analytics objectives and personalised learning goals are enhancing learning experience, providing personal recommendations and satisfying personal learning needs. The learning analytics methods have commonly involved the use of statistical tests, classification, clustering and visualisation techniques. The findings also suggest the areas for future work to address the limitations revealed in the practices, such as investigating more cost-effective ways of offering personalised support, and the transforming role of teachers in personalised learning practices.

Since its inception in 2011, Learning Analytics has matured and expanded in terms of reach (e.g., primary and K-12 education) and in having access to a greater variety, volume and velocity of data (e.g. collecting and analyzing multimodal data). Its roots in multiple disciplines yield a range and richness of theoretical influences resulting in an inherent theoretical pluralism. Such multi-and interdisciplinary origins and influences raise questions around which learning theories inform learning analytics research, and the implications for the field should a particular theory dominate. In establishing the theoretical influences in learning analytics, this scoping review focused on the Learning Analytics and Knowledge Conference (LAK) Proceedings (2011–2020) and the Journal of Learning Analytics (JLA) (2014–2020) as data sources. While learning analytics research is published across a range of scholarly journals, at the time of this study, a significant part of research into learning analytics had been published under the auspices of the Society of Learning Analytics (SoLAR), in the proceedings of the annual LAK conference and the field’s official journal, and as such, provides particular insight into its theoretical underpinnings. The analysis found evidence of a range of theoretical influences. While some learning theories have waned since 2011, others, such as Self-Regulated Learning (SRL), are in the ascendency. We discuss the implications of the use of learning theory in learning analytics research and conclude that this theoretical pluralism is something to be treasured and protected.

The purpose of this study was to identify educational, technological, and professional consequences of ePortfolio use that were intended, unintended, positive and negative. Ethical approval was obtained from seven universities in accordance with institutional guidelines established in accordance with Australia’s National Statement on Ethical Conduct in Human Research. A mixed methods design was employed to survey staff and invite them to participate in a focus group or interview. Online questionnaires were distributed across seven universities to 22 volunteer respondents drawn from academic teachers in both disciplines. Following completion and quantification of questionnaire data, respondents were invited to participate in a either a focus group or interview with one of the researchers. Merton’s sociological model was adopted in the study as it informed a conceptual framework used to examine the impact of unintended consequences that emerged when building a professional online presence. This theory was used to identify the full range of ePortfolio learning and teaching practices emerging as a multi-function, complex system, within domains and purpose of ePortfolios such as information systems, health or education disciplines. Results indicated a significant challenge, namely there is no single set of guidelines that would support teachers and students when engaging in learning and teaching activities to improve protection of private data relating to vulnerable people. Digital literacy of academics and students is variable and may impact on data privacy. Data gathered will be used to inform development of guidelines on the use and publication of information collected in ePortfolios to protect vulnerable people.

Learning with hypertexts require learners to navigate in a non-linear environment and build a coherent representation of the informational content. The expansion of digital technologies and hypertext use in higher education has emphasized the need to examine how technological tools may foster quality learning. This study examined how three types of interactive content organizers (COs) used to represent the main concepts and to navigate in the content pages can impact information processing and learning outcome. The COs designed for the experiment varied in terms of conceptual and navigation support they provide and format. Ninety-three undergraduates used a list of concepts that only provided conceptual support, a summary or a concept map that provided both conceptual and navigation support either with text or graphic format. Results showed that highly coherent COs such as summaries or concept maps improved navigation coherence. However, the summary focused the attention of learners and decreased hypertext exploration (i.e. longer time on the CO and fewer content pages accessed). Longer reading times of the summary also increased text-based outcome, whereas longer processing of the concept map improved inference-based outcome. Implications for the design of interactive COs used to access, navigate and a learn with hypertexts are discussed.

Higher education may benefit from investigating alternative evidence-based methods of online learning to understand students’ learning behaviors while considering students’ social cognitive motivational traits. Researchers conducted an in situ design-based research (DBR) study to investigate learner experience design (LXD) methods, deploying approaches of asynchronous video, course dashboards, and enhanced user experience. This mixed-methods study (N = 181) assessed associations of students’ social cognitive motivational traits (self-efficacy, task-value, self-regulation) influencing their learning behaviors (engagement, elaboration, critical thinking) resulting from LXD. Social cognitive motivational traits were positively predictive of learning behaviors. As motivational factors increased, students’ course engagement, usage of elaboration, and critical thinking skills increased. Self-efficacy, task-value, and self-regulation explained 31% of the variance of engagement, 47% of the explained variance of critical thinking skills, and 57% of the explained variance in the usage of elaboration. As a predictor, task-value beliefs increased the proportion of explained variance in each model significantly, above self-efficacy and self-regulation. Qualitative content analysis corroborated these findings, explaining how LXD efforts contributed to motivations, learning behaviors, and learning experience. Results suggest that mechanisms underpinning LXD and students’ learning behaviors are likely the result of dynamically catalyzing social cognitive motivational factors. The discussion concludes with the LXD affordances that explain the positive influences in students’ social cognitive motivational traits and learning behaviors, while also considering constraints for future iterations.

Understanding the relationship between interactive behaviours and discourse content has critical implications for instructors' design and facilitation of collaborative discussion activities in the online discussion forum (ODF). This paper adopts social network analysis (SNA) and epistemic network analysis (ENA) methods to jointly investigate the relationships between students’ network characteristics, discussion topics, and learning outcomes in a course discussion forum. Discourse data from 207 participants were included in this study. The findings indicated that (1) the interactive network generated in the collaborative discussion activities was sparsely connected, and there was limited information exchange between instructors and students; (2) students’ discussion topics were mainly related to the learning content; (3) compared with the isolated group, students in the leader, mediator, and animator groups were more concerned about topics related to the learning content; and (4) students who discussed more topics related to the learning content performed better than the students who discussed more topics related to learning methods and social interactions. The learning outcomes of the influencer and leader groups were significantly higher than those of the peripheral and isolated groups. However, there was no significant correlation between students’ individual centrality and their learning outcomes. The findings enrich the ODF research on the comprehensive identification of interactive behaviours and discourse content in the process of collaborative discussion activities and on the discussion topic differences between different role groups. The study findings also have practical implications for instructors to design effective instructional interventions aimed at improving the quality of collaboration in the ODF.

Social presence is an important concept for understanding psychosocial processes in learning scenarios that make extensive use of mediated communication like online distance learning. Despite this centrality, a coherent and nuanced theory of social presence is yet to emerge from the literature. Past research has shown associations with desirable affective variables like satisfaction and perceived learning, yet our knowledge as to when and for whom these effects are expected is still very limited. By introducing two contextual explanatory variables, we provide the means toward a more mature theory of social presence. The first variable, social presence divergence, relates students experiences to their preferences, yielding three distinct scenarios: too little, too much, and just the right amount of social presence. The second variable, interaction integration, considers the centrality of social interaction in the learning scenario, suggesting that this functions as a moderator. In a sample of teacher education students (N = 305), we find evidence that these variables interact with social presence and affective dependent variables as expected. These results add nuance and context to the discussion about the practical relevance of social presence. The implications of these findings as well as limitations of this study are discussed.

Using a mixed methods design, this study examines graduate students’ perceptions of attitudinal learning regarding the use of case-based learning (CBL) methods in an online instructional design course. One hundred and thirty-three students completed a survey taken from the Attitudinal Learning Inventory (ALI) in weeks three and seven of the eight-week course. Using Latent Profile Analysis (LPA), three classes of student types were identified: Skeptics who perceived low and declining attitudinal learning throughout the course, Adopters who perceived high and rising, and Advocates, who perceived very high and rising attitudes. Student reflection assignments on developing instructional design expertise using CBL, and a post-course questionnaire sent out five months after the course concluded were analyzed to gain further insights into student experiences and perspectives of both the course and CBL. Results indicated the majority of students perceived positive attitudinal learning throughout the course and had positive views of CBL as an instructional method. Students who struggled had less prior ID experience and seemed to exhibit a more negative perspective towards the course and CBL despite reporting less external challenges that impacted their learning in the course than the positive groups. Implications highlight the importance of facilitating SRL development and supplementing more novice CBL and ID students with additional support.

The present study aims to conceptualize service quality and perceived value in the context of blended learning by redefining and modifying the existing SERVQUAL model, reviewing prior marketing literature on perceived value, and examining the relationships between service quality, perceived value, and student satisfaction. The sample was restricted to colleges in South Korea, where blended learning programs have started to receive much attention. We examined our hypotheses by using regression analysis via the statistical programs Amos 22.0 and SPSS 23.0. The following results are produced. First, the conceptualization of service quality and perceived value was confirmed. Second, the different effects of online and offline service quality on each perceived value are confirmed. Offline service quality is more effective in generating perceived epistemic value, perceived social value, and perceived emotional value than online service quality, whereas online service quality is more effective in triggering perceived conditional value than offline service quality. Finally, perceived emotional value and perceived conditional value are the important determinants of student satisfaction. We address the theoretical implications that (1) service quality and perceived value are conceptualized through modification, refinement, and empirical testing and develop a multidimensional scale for service quality and perceived value, and (2) the sequential and causal relationships among service quality, perceived value, and student satisfaction are confirmed. Practically, we expect that our measurement scales for service quality and perceived value, which have high validity and reliability, can serve as diagnostic tools for blended learning program evaluation from students’ perspective.

The modality and redundancy principles are two fundamental principles used to inform the design of multimedia instruction. They are based on a variety of experimental studies that utilized different types of multimedia lessons to compare input modes of graphics+audio, graphics+text, and graphics+audio+text with each other. However, a lack of control of multimedia lessons in previous studies creates a threat to validity because a single case scenario without following certain principles is not sufficient to represent a construct. Therefore, this study addressed this inherent validity threat and reinvestigated the applicability of the modality and redundancy principles when students learned during a controlled multimedia lesson. In this study the multimedia lesson was developed to follow a series of multimedia learning principles. These principles ensured that the lesson was representative of different types of multimedia lessons. Additionally, they ensured that the multimedia lesson was conducive to learning, since those that were not helpful would not be utilized for instruction in the first place. Eighty-six students in a research university in the US took a prior knowledge survey. They were then randomly assigned to the three input mode conditions and watched the multimedia lesson about the formation of lightning. Subsequent retention and transfer tests revealed that there were no statistically significant differences among the three input mode conditions. Therefore, both the redundancy and modality effects disappeared. This study provided an updated understanding of the applicability of the two important principles for multimedia instruction. Limitations and implications were discussed.

Engagement is critical in learning, including computer-supported collaborative learning (CSCL). Previous studies have mainly measured engagement using students’ self-reports which usually do not capture the learning process or the interactions between group members. Therefore, researchers advocated developing new and innovative engagement measurements to address these issues through employing learning analytics and educational data mining (e.g., Azevedo in Educ Psychol 50(1):84–94, 2015; Henrie in Comput Educ 90:36–53, 2015). This study responded to this call by developing learning analytics to study the multifaceted aspects of engagement (i.e., group behavioral, social, cognitive, and metacognitive engagement) and its impact on collaborative learning. The results show that group behavioral engagement and group cognitive engagement have a significantly positive effect on group problem-solving performance; group social engagement has a significantly negative effect; the impact of group metacognitive engagement is not significant. Furthermore, group problem-solving performance has a significant positive effect on individual cognitive understanding, which partially mediates the impact of group behavioral engagement and fully mediates the impact of group social engagement on individual cognitive understanding. The findings have important implications for developing domain-specific learning analytics to measure students’ sub-constructs of engagement in CSCL.

College graduates, especially teachers, often feel alienated when starting their careers as they may lack robust support systems to help them address daily difficulties. Researchers study online learning communities as a model to address this missing support system; however, very little research exists on the benefits of embedding these learning communities into Science, Technology, Engineering, and Mathematics (STEM) teacher candidates’ (TC) coursework. In this design case, we identified that a sustained Reddit social media learning community (SMLC) embedded within a technology integration course created a robust support system for STEM TCs preparing for their careers. In this SMLC, TCs discussed and solved technological and professional problems, kept up to date with new technologies and methodologies, and accessed a network of peers with similar interests. The potential of such SMLCs goes far beyond technology integration and STEM, and applying the insights from this design case might address a wide range of problems many college students, not only STEM TCs, face when beginning their careers. The design decisions we implemented in crafting our course and SMLC are documented in this paper. We also present our findings and recommendations to understand the impact and practical implications for designing effective and sustainable SMLCs.

Considering the rise of online education and an increasing number of students with disabilities in higher education, examining the validity of the Self-efficacy Questionnaire for Online Learning (SeQoL) for students with disabilities is warranted. The purpose of this study is to examine the reliability and validity of (SeQoL; Shen et al., 2013) for students with disabilities in higher education. We analyzed the internal structure, convergent validity, criterion validity, and reliability of SeQoL. A sample of 278 students with disabilities responded to an online survey in Spring 2021. Most of our sample were female, White, and undergraduate students. We used confirmatory factor analysis, correlation, multivariate analysis of variance, and Cronbach’s alpha to analyze the data. Our results indicated that data fit the five factors model with 25 items. Students who preferred online or hybrid courses had significantly higher online learning self-efficacy than face-to-face courses. Limitations and future research were discussed.

The study explored Engineering students’ and their instructors’ knowledge and use of tools, applications, and websites; their interests, abilities, and attitudes to using technology; and their views about the advantages and challenges of using technology in the academic listening classes. The participants were 184 undergraduate students of Engineering and seven academic listening instructors at Iran University of Science and Technology (IUST). Mixed methods research was used and the instruments included a questionnaire on knowledge, use, and attitudes to technology, followed by seven open-ended questions. A semi-structured interview was also conducted with the listening instructors. Engineering students’ less knowledge and use were related to the tools of e-Portfolio, Rosetta Stone, and Quick Time and the applications of English listening and speaking and BBC learning English. Students’ knowledge and use of technological types could be hierarchically ranked as tools, applications, and websites. Audio files, podcasts, videos, and academic websites were the technologies mostly used by the instructors inside the class, while Edmodo, podcasts, TED Talks, and recording voice to receive feedback from instructors were used for assignments outside the class. The areas of instruction the instructors mostly emphasized included the techniques and strategies for using technologies, how to use the tools for educational purposes, and the technological literacy. Disorder of technological types, inability to use the tools, using outdated tools in educational contexts, and low Internet speed were the students’ challenges, while difficulty in finding appropriate online tools and materials, rigid schedule, lack of time, and technological failure were the instructors’ challenges.

Research on academic performance in online education proliferated in the last decade; however, our current understanding of the relationship between self-regulated learning and academic performance in online learning environments remains scattered. This meta-analysis provides a holistic view of the relationship between learners’ use of self-regulated learning strategies and their online academic performance in higher education. We investigated the conditions that moderate the strength of the correlations. Results revealed (1) a small positive correlation between overall self-regulated learning strategy and online academic performance (r = .21); (2) small positive correlations between online academic performance and metacognitive, behavioral, and environmental self-regulated learning strategies (r = .07; r = .23; r = .14, respectively); and (3) three conditions that moderate the correlation between self-regulated learning strategy and online academic performance (i.e., participant, self-regulated learning measurement, and academic performance measurement). Taken together, this updated holistic review will deepen our understanding of the relationship between SRL and online academic performance, enhancing teaching and learning practices in online education.

In recent years, gamification has seen a rise in usage for cyber-security training, with some of the most used elements being story and narrative. Yet there seems to be a lack of research on story-based gamification as well as how this type of gamification affects a training’s effectiveness. The goal of this study was to test the effectiveness of story-driven gamification in the prevention of USB-based attacks among students. To do this a story-driven gamified training was designed and deployed among students. It was found that story-elements were perceived to have a positive impact on the effectiveness of gamification. Testing the game showed that adding elements such as a guide and introducing characters caused the game to be more engaging and participants to learn more. Even to the extent that it was stated to be better than other gamification attempts lacking a focus on narrative. Because this research focuses on just story-elements and not other often used gamification elements such as rewards and leaderboards, it creates a clear image of how they should be used. It aids the design of a gamified training by showing what questions need to be answered to make story-elements work as effectively as possible towards the goal of the training. This also provides a foothold for future frameworks that could be made in regard to the usage of story-elements in gamification and education. During the research, there were also some questions raised that could be researched further such as a difference in results between participants with different study backgrounds.

Interdisciplinary collaboration is widely used in research, industry, and education. Understanding the differences in cognitive processes between cross-discipline and same-discipline groups can improve instruction in collaborative learning. In this study, students volunteered to participate in cross-discipline or same-discipline collaborative learning groups to collaboratively complete three assignments. Epistemic network analysis was employed to identify the differences in cognitive engagement, the roles of group leaders, and the trajectory differences between these two kinds of groups. The results showed that compared to same-discipline groups, cross-discipline groups have lower-order cognitive engagement. Group leaders in cross-discipline groups invested most of their energy into inquiring or starting the discussion. In addition, cognitive engagement of cross-discipline groups dropped sharply over time in collaborative learning. These differences imply that to achieve better collaboration in cross-discipline groups, teachers should provide more interventions, such as evaluation and analytical support, to help students reach high-level cognitive engagement.

The study of pedagogical design and development for virtual laboratory in science education in higher education is limited. Few tailored pedagogical approaches have been adopted for developing students higher order thinking (i.e. reflective thinking skills) in virtual laboratory enabled learning in the university level. For addressing these, this study borrowed the merits of virtual laboratory and Know-Want-Learn (KWL) reflective thinking approach, with considering the development of students reflective thinking, to support university students to learn physical chemistry concepts. To examine the impacts of this virtual lab with KWL reflective thinking approach on students’ conceptual understanding and reflections, a quasi-experimental research was conducted among 30 students in experimental group and 28 students in control group in a university located in the north of China. Positive learning outcomes including were received based on quantitative and qualitative data analysis of pre-and post-test scores, questionnaire, reflection journals and selected interview. The study could inform the pedagogical design and the implementation of virtual lab for developing students’ higher order thinking skills in the university level.

The adoption of online teaching and learning in educational settings poses a notable challenge to teachers as they are required to implement student-centered pedagogy in online learning environments. This is often hindered by teachers' insufficient preparation in this area. The current study hinges on two underexplored aspects that may explain this challenge among faculty teaching in teacher training colleges, both related to the link between professionals and their workplace: innovative work behavior and personal accountability. This study sought to examine higher education teachers’ perceived innovative behavior and personal accountability as precursors of their reported distance learning practices during the COVID-19 pandemic period. Data were gathered from 303 faculty members from public teacher training colleges. Data were analyzed by using Partial Least Squares—Structural Equation Modeling. Based on our findings, innovative teachers reported using more student-centered approaches in distance learning than traditional practices. Another finding showed that high levels of external accountability were associated with traditional teaching routines focused on setting clear rules for student behavior in online environments and enforcing these rules. Hence, teachers who placed an emphasis on delivering timely reports to their managers and believed they should be evaluated by their students’ final grades, were more inclined towards traditional approaches to teaching. Internal accountable teachers tended to employ adaptive instructional practices, underscoring the quality of the learning process. Implementations and directions for future research are discussed.

In this qualitative study we investigated the experiences of instructional designers as they sought to build quality into online courses. Through semi-structured interviews, we explored what enabled and hindered their pursuit of quality, how they experienced their efforts in this regard, what mattered to them, and complexities that accompanied this pursuit. Our analysis of participant experiences suggested four themes: (1) connections between quality and designers’ ability to act autonomously; (2) connections between quality and collaborative, team-based relationships; (3) ambivalence due to tensions between autonomy and collaboration; and (4) ways of coping with limits on autonomy and collaboration. We conclude our report with implications for instructional design practice, suggesting that the pursuit of quality often requires creative work arounds and is informed by affective judgements that lie beyond the purview of traditional instructional design processes.