https://doi.org/10.53656/math2023-6-4-qua

2023/6, стр. 607 - 623

QUALITY OF BLENDED LEARNING COURSES: STUDENTS’ PERSPECTIVE

Резюме: This research paper focuses on studying students' perspectives on the quality of online course developed to support traditional face-to-face learning, specifically exploring their satisfaction levels. The study aims to identify the factors influencing student satisfaction and their impact on academic performance. A questionnaire was developed, consisting of four evaluated areas: course content and design, organization and preparation of training, communication and support in the learning process, and evaluation. The questionnaire was administered to 51 students who completed an Object-Oriented Programming course. The collected data was analysed using statistical techniques, including skewness and kurtosis indexes, Cronbach's alpha coefficients, average variance extracted (AVE), composite reliability (CR), and principal component analysis (PCA). The findings revealed a generally positive perception of course content, organization, communication, and evaluation with specific areas identified for improvement. The findings emphasize the importance of addressing students' satisfaction to enhance the overall quality of blended learning courses. The study contributes to the existing literature on student satisfaction with learning courses for blended learning. It highlights the need for institutions to prioritize course quality to meet students' expectations and needs.

Ключови думи: blended learning; student satisfaction; survey; quality

1. Introduction

Ensuring the quality of online courses is of utmost importance for Higher Education Institutions (HEIs). Worldwide, organizations develop standards, conduct external evaluations of online courses and award quality labels. Many institutions develop and implement at the local level course quality standards, recognizing the important role of students as primary users of the educational services. Student satisfaction is considered so crucial to the quality of online courses that some organizations include it in their quality framework. Although student satisfaction surveys are usual practice in many universities, some critics question the validity of these questionnaires (Dziuban et al. 2015). They test whether students can reliably evaluate the learning experience or whether they complete the questionnaire based on emotional reactions or the lack of focus on significant points such as interaction, assessment, feedback, student engagement, etc. Because student satisfaction is complicated and varies among contexts, each HEI should develop own questionnaire and carefully examine their students' satisfaction.

The results of studies on student satisfaction with the quality of online courses and training can be used by different stakeholders in HEIs. Teachers can use the results of such an assessment to pinpoint their strengths and identify areas for improvement to provide a better learning experience for students (Younas et al. 2022). They help higher education decision-makers understand the factors affecting student satisfaction and take the necessary actions to improve the quality of online courses offered in HEI according to the needs of students (Nikou & Maslov 2023).

This paper focuses on studying students' perspectives on the quality of online courses for blended learning, specifically exploring their satisfaction levels. It investigates the factors influencing student satisfaction and their impact on academic performance. To conduct the study, after a detailed review of the research in the field, an author's questionnaire was developed, which consists of 31 questions divided in four areas: course content and design, organization and preparation of training, communication and support in the learning process, and evaluation. The questionnaire was administered to 51 students who completed an Object-Oriented Programming course. The collected data was analysed using statistical techniques, including skewness and kurtosis indexes, Cronbach's alpha coefficients, average variance extracted (AVE), composite reliability (CR), and principal component analysis (PCA). The findings revealed a generally positive perception of course content, organization, communication, and evaluation with specific areas identified for improvement.

2. Methodology

The study´s method is based on an empirical approach – an exploratory survey using a questionnaire for data collection. The questionnaire was developed after in-depth studying of factors influencing students’ satisfaction determined in previous research (see Table 1).

Based on the reviews of literature concerning quality of online learning, in this study, the authors regard e-learning quality as a multidimensional construct of four components influencing student satisfactions: Course content and design; Organization, preparation and conduct of training; Communication and support in the learning process; Evaluation.

The questionnaire contains 31 mandatory questions divided into 4 areas.

The nine questions in the first area Course content and design aim to determine how much students think the course is high quality in terms of clear learning objectives (Q1), list of literature sources is available for students' self-preparation (Q2), availability of information about the roles of the team conducting and providing the training and contact data (Q3), overall presentation and a variety of the learning content (level of difficulty, level of interactivity, type of target knowledge/skills, individual/teamwork, etc.) (Q4), consistency of learning content for theoretical training with the learning objectives (Q5), educational content for practical training (Q6), learning content for self-preparation and self-assessment (materials, projects, assignments, tests, etc.) (Q7), learning content for assessing knowledge and forming the final grade (Q8), overall design and easy use and navigation of the e-course (Q9).

The second area Organization, preparation and conduct of training contains 11 questions that aim to quantify the students' perceptions of the organization, preparation and conduct of the training. Students appreciate the availability of information about the organization and conducting the training (Q10), training schedule (Q11) and requirements for completing the course, assessment methods, forming the final grade, and their feasibility (Q12). Other evaluated factors in this area provided opportunities for preliminary technological preparation for working with the e-learning system (Q13), personalized determination of the time, place and pace of training (Q14), interactive tools to track the student's progress in the learning process (Q15) and measures taken during the training to verify students’ identity (Q16) and prevent plagiarism and exam fraud, e.g. informing students, using plagiarism detection software, etc. (Q17). This section also includes questions that assess whether educational content is presented sufficiently comprehensively and allows successful completion of the training and the formation of the final grade (Q18), familiarization with the current level of knowledge and achievements in the field (Q19) and whether students acquire knowledge and skills that contribute to their professional training (Q20).

The third area Communication and support in the learning process includes seven questions which aim to measure students' satisfaction with the means of communication and the support provided. Students evaluate whether appropriate means of synchronous (Q21) and asynchronous communication (Q22) are used in the e-course, to what extent they have constant access to means of communication and interaction with peers (Q23), system administrator (Q24) and teacher(s) (Q25), as well as whether the teacher(s) (Q26) and administrator (Q27) provide timely support when difficulties are encountered.

The last area Evaluation includes 4 questions assessing the feasibility of the questions/tasks/topics/projects for (self)assessment of the knowledge (Q28), the sufficiency of time provided for conducting evaluations (Q29), the usefulness of the provided feedback for assessment results (Q30) and implementation of activities for self-preparation and self-assessment, e.g. materials, projects, assignments, tests, etc. (Q31).

All questions require students to state the extent of their agreement with formulated statements on the 5-point Likert-type scale in which 1 means Strongly Disagree (SD), 2-Disagree (D), 3-Neutral (N), 4-Agree (A) and 5-Strongly Agree (SA).

The initial version of the questionnaire was evaluated by two experts in distance learning quality assurance and two experts in the organization and conduct of surveys. The experts in quality assurance of distance learning assessed whether the questions successfully covered the topic. Based on their feedback, the questionnaire was revised with 10 questions removed and 5 corrected. Experts in survey conduction reviewed the updated version and ensured that it did not contain common errors and confusing questions. The final version of the questionnaire has been added in Moodle – the learning management system used at the university. For this reason, the questionnaire did not contain filter questions to exclude students who did not meet the inclusion criteria for studying the course during the academic year nor questions collecting demographic data (gender, academic year, study programme, etc.).

Based on the developed questionnaire, five hypotheses were formulated:

H1: Course content and design positively affects students’ satisfact ion with online course.

H2: Organization, preparation and conduct of training positively affects students’ satisfaction with online course.

H3: Communication and support in the learning process positively affects students’ satisfaction with online course.

H4: Evaluation positively affects students’ satisfaction with online course.

H5: Satisfaction regarding the course affects the final grade.

The study on student satisfaction with the quality of the e-course on "Objectoriented programming" was conducted in the period 25.05.2023-31.05.2023 after the completion of the training and the conduct of the final exam in the discipline. The system administrator added the questionnaire to the e-course, which the students could use throughout their training in addition to the traditional training (weekly lectures and exercises during the semester). All 86 second-year students (12 women and 74 men) aged \(20-21\) years from 3 undergraduate majors at the University of Plovdiv – Information and Computer Engineering (69 students), Telecommunication and Information Systems (14 students), and Bioengineering (3 students), all of whom studied the course in the academic year 2022/2023, were invited to participate in the survey. All students received an email with clear information about the purpose of the study and a request to complete the online questionnaire. Students were informed that the collected empirical data would only be used for research and to improve the course quality and were asked to complete a consent to participate in the study. Participation in this study was voluntary, and students could opt out without consequence. Of all invited students, 51 (9 women and 42 men) completed the questionnaire (59.30%) – Information and Computer Engineering (\(n=41 ; 59.42 \%\) ), Telecommunication and Information Systems (\(n=7 ; 50 \%\) of in=7; 50% of invited students), Bioengineering (\(n=3 ; 100 \%\) of the invited students). Because the survey was organized and conducted through the Moodle Feedback activity, each student could complete the questionnaire only once. The last excluded the chances for duplicated responses.

Data were extracted from the system database along with students' final grades and then exported to Excel to process and analyse the results using a query. Placing them in a matrix in an Excel worksheet allowed the data to be analysed using SPSS and AMOS. Since all questions in the questionnaire were mandatory, there was no need to perform a data check and use statistical techniques to estimate missing data values. Cronbach's alpha, Average Variance Extracted (AVE) and Composite Reliability (CR) were employed to assess the internal reliability and validity of each scale. Hypotheses were tested using a one-sample t-test, comparing sample means to a known or hypothesized population mean. Additionally, the association between grade and students' satisfaction level was evaluated using a chi-square test.

3. Results

The calculated Skewness and Kurtosis indexes show the normality of data (see Table 2). The results showed the deviation of data from normality was not severe as the absolute value of skewness and kurtosis index were below 3.10, respectively, and hence were appropriate for parametric analysis. The internal consistency of each scale was carefully examined via Cronbach's alpha coefficients to ensure the reliability of the measurements. A recommended threshold of 0.70 or higher was employed to assess the scale reliability. Cronbach's alpha coefficients for different scales ranged from 0.88 to 0.94, which suggests good to excellent internal consistency according to the classification proposed by George & Mallery (George & Mallery 2010). The calculated coefficients indicated that the items within each scale were highly correlated, demonstrating the reliability and consistency of the measurements. Moreover, additional measures, such as Average Variance Extracted (AVE) to represent the variance captured by the constructs relative to the measurement error and Composite Reliability (CR) to assess the reliability of the measurement model by considering both the shared and unique variance captured by the constructs, were calculated. In this study, all constructs exhibited AVE values exceeding 0.5 (the commonly recommended threshold for AVE), indicating that the constructs explain more variance than measurement error. On the other hand, all constructs demonstrated CR values above 0.7 (generally recommended value for CR), suggesting strong internal consistency and reliability.

Based on the high Cronbach's alpha coefficients, AVE values greater than 0.5, and CR values exceeding 0.7, we can conclude that the scales used in the questionnaire are reliable for measuring the various dimensions under investigation. These findings provide confidence in the consistency and accuracy of the collected data, enabling us to draw meaningful conclusions and make valid inferences based on the measured constructs.

Construct validity, a crucial aspect of scale evaluation focusing on the accuracy with which a scale measures its intended construct was meticulously assessed through a principal component analysis (PCA) utilizing Varimax rotation and Kaiser normalization techniques in this study. The analysis of the four scales employed to measure student satisfaction revealed significant outcomes, as evidenced by Barlett's test (chi-square \(=1815.203 ; \mathrm{df}=456 ; p \lt 0.001\) ), indicating substantial variability among the scale items. Additionally, the high Kaiser-MeyerOlkin (KMO) measure of sampling adequacy (0.835) further validated the suitability of the data for factor analysis, affirming the quality of the dataset. The resulting factor structure (see Table 3), illustrates the loading of 31 items onto four dimensions, successfully capturing the underlying constructs that the questionnaire aimed to assess concerning student satisfaction. This comprehensive examination of construct validity, employing robust statistical procedures, not only reinforces the credibility of the measurement instruments but also ensures the reliability and validity of the collected data for meaningful interpretation and analysis.

The findings depicted in fig. 1 provide insights into the distribution of student satisfaction across different areas. The analysis revealed that students strongly agree with the Course design and content area overall. Especially, Questions 3, 8, and 9 garnered a significant percentage of Strongly agree responses (approximately \(60 \%\) ), indicating that students highly appreciated the content and design of the course. Alike, in the Area 2. Organization, preparation and conduct of training Question 12 received a notably high percentage of Strongly agree responses (more than \(70 \%\) ), suggesting that students were very pleased with the organization and preparation of their training sessions. In addition, in Area 3. Communication and support in the learning process, a substantial proportion of students (over 40%) expressed Strongly agree with all the questions, indicating their positive perception of the communication and support provided during their learning experience.

The trend continued in Area 4. Evaluation, in which a significant number of students demonstrated strong agreement. However, it is crucial to note that many students maintained a neutral stance regarding the statements across all areas, implying some level of indecisiveness or lack of clear opinion among these individuals. These findings shed light on the areas where students are very pleased and highlight the need for further investigation into the factors contributing to neutral responses to enhance the online learning experience.

Table 4 displays the results of the one-sample t-test, which aimed to determine whether the areas encompassing course content and design, organization, preparation and conduct of training, communication and support, and evaluation process benefit the students. The specific hypothesis under examination is as follows:

Null (H): There exist no significant associations in student satisfaction within the domains of course content and design, organization, preparation and conduct of training, communication and support, and the evaluation process.

Alt (H): There exist significant associations in student satisfaction within the domains of course content and design, organization, preparation and conduct of training, communication and support, and the evaluation process.

The findings unveiled significant positive associations between all four areas and student satisfaction, indicating their substantial impact. Specifically, there existed a positive effect of course content and design (\(M=4.20, S E=0.10,95 \%\) CI [3.98, 4.40], \(p \lt 0.001\) ), demonstrating that students perceived the course content and course design as valuable, thereby fostering higher levels of satisfaction. Similarly, the organization, preparation and conduct of training exhibited a significant positive effect on student satisfaction \((M=4.17, S E=0.11\), \(95 \% C I[3.94,4.40], p \lt 0.001)\), highlighting the effectiveness and importance of well-organized and prepared training sessions in enhancing overall satisfaction. Additionally, the communication and support in the learning process yielded a significant positive effect on student satisfaction (\(M=4.25, S E=0.12,95 \% C I\) [4.02, 4.49], \(p \lt 0.001\) ), underscoring the influential role of effective communication and adequate support throughout the learning journey in fostering student satisfaction. Moreover, the evaluation demonstrated a significant positive effect on student satisfaction \((M=4.24, S E=0.13,95 \% C I[3.99,4.49], p \lt 0.001)\), suggesting that a well-designed evaluation process positively contributed to the overall satisfaction of students. Together, these results from the one-sample t-test provide robust empirical evidence to support the idea that the examined areas, encompassing course content and design, organization and preparation of training, communication and support, and evaluation process, significantly and positively influence student satisfaction.

4. Findings

Based on the one sample t-test, hypotheses 1, 2, 3 and 4 were supported.

We also have the information on the student’s final grade, categorized as Fail, Satisfactory, Good, Very Good and Excellent (the highest result). For the analysis purpose, we utilized this variable and categorized answer to each question into two parts. If a student agrees or strongly agree with the statement, we interpret this as an indication of satisfaction with the module, whether it pertains to course content and design, organization, preparation and conduct of training, communication and support in the learning process, or evaluation. That is, as below:

\[ \begin{gathered} \text { Satisfaction status }\left\{\begin{aligned} \text { Satisfied, } & \text { if the student is A or SA with the statement } \\ & \text { Not satisfied,otherwise } \end{aligned}\right. \end{gathered} \] Fig. 2 presents a comparative analysis of student grades based on four areas: Area 1. Course content and design, Area 2. Organization, preparation and conduct of training, Area 3. Communication and support in the learning process, Area 4. Evaluation. The results indicate that satisfaction with Area 1 is associated with a higher prevalence of ach ieving “Good” grades (alm ost 40%) and “Ve ry Good” grades (approximately \(21 \%\) ) compared to students who did not find satisfaction in the course content and design. However, students expressed dissatisfaction with the course content and design show a higher prevalence of receiving “Excellent” grades. Similar patterns emerge when examining the satisfaction level of communication and support. The analysis suggests that students who expressed satisfaction with the communication and support had a higher prevalence of achieving a good grade (approximately 40%). In addition, the organization, preparation and conduct of training reveals that students who were satisfied in this area had a higher percentage of “Good” grades (approximately 40%). In contrast, Area 4. Evaluation demonstrates that the prevalence of receiving “Very Good” and “Excellent” grades was higher among students who expressed satisfaction with the evaluation.

These findings collectively highlight the importance of student satisfaction in different evaluated areas. Satisfaction with the course content and design, communication and support, and evaluation appears to positively influence students' grades, particularly in achieving “Good” and “Very Good” grades. On the other hand, in the Organization, preparation and conduct of training area, there is a positive association primarily with “Good ” grades. It is ev ident that add ressing student satisfaction and ensuring the effectiveness of the evaluated areas can play a pivotal role in enhancing overall academic performance.

We also performed a contingency table Chi-square to assess the association between the grade and satisfaction level of the student. Table 5 represents the results, showing no significant association between the grade and the level of student satisfaction \((P \geq 0.05)\). Therefore, our findings are not able to support the hypothesis H5.

5. Discussion

This paper emphasizes the importance of addressing areas for improvement in blended learning courses, such as self-preparation materials, personalized learning flexibility, and communication tools used. It highlights that student satisfaction is influenced by factors beyond academic performance, underscoring the need to prioritize course quality and meet student expectations. The paper also emphasizes the significance of considering students' subjective experiences and perceptions of learning. The findings reveal some common themes and insights regarding student satisfaction and the factors influencing it in blended learning environments with various other studies. For example, some authors (Zeqiri et al. 2021) support the importance of teacher-student interaction in student satisfaction and performance improvement. It emphasizes the significance of communication channels and the instructor's role in creating a positive learning experience. They also support the findings of this paper by highlighting the positive impact of course management and student interaction on student satisfaction in blended learning (Zeqiri et al. 2021). The focus on course management aligns with this study's emphasis on the organization of training, and the importance of student interaction corresponds to the underlining of communication channels here in this paper. Authors of (Zein et al. 2023) further strengthen the importance of interaction, highlighting the role of instructor-student and student-peer interaction in online learning satisfaction. The finding aligns with the emphasis on communication channels and the instructor's facilitating ability in our study. It also supports the notion that well -designed courses and technology play vital roles in increasing student satisfaction. Also, Ismail (Ismail 2018) explores the impact of an improved blended learning strategy on student satisfaction and reveals high satisfaction levels among graduate students. These findings underscore the significance of creating a rich learning environment and using appropriate instructional methods to enhance student satisfaction. Overall, these studies collectively emphasize the importance of various factors such as course quality, instructor-student interaction, course management, technology support, and a rich learning environment in promoting student satisfaction in blended learning. The findings highlight the need for educators and course designers to address areas of improvement, foster effective communication, and prioritize student experiences to enhance satisfaction and ultimately improve learning outcomes.

6. Conclusion

These finding suggests that factors associated with course content, organization, communication and assessment consistently affect student satisfaction, regardless of their grades. This finding emphasizes the importance of considering students’ subjective experiences and perceptions of learning, no matter their academic performance. While academic performance remains a vital indicator of learning outcomes, it is crucial to understand that many factors besides grades affect student satisfaction. The results of this study highlight the need for HEI to prioritize the quality of online courses and ensure that they meet the expectations and needs of students. Improving course content, organization, communication channels, and assessment methods can improve student satisfaction and learning outcomes. However, it is crucial to acknowledge the limitations of this study. Results are based on a specific sample and may not be generalizable to the entire student population. Further research with more students is needed to confirm these findings and provide a more comprehensive understanding of the relationship between grades and satisfaction in online learning environments.

In conclusion, this study contributes to the growing literature on student satisfaction with online courses. The findings highlight the importance of prioritizing the quality of course planning, organization, communication and assessment to increase student satisfaction. Future research should continue to explore the multifaceted nature of student satisfaction with online learning and identify other factors that may influence this construct. By understanding and addressing the factors influencing student satisfaction, institutions can better support efficient online learning experiences and promote positive educational outcomes for all students.

Acknowledgements

This paper is financed by the European Union-NextGenerationEU, through the National Recovery and Resilience Plan of the Republic of Bulgaria, project \(\mathrm{N}^{\circ}\) BG-RRP-2.004-0001-C01. The paper reflects only the author’s view and the Agency is not responsible for any use that may be made of the information it contains.

REFERENCES

ALAM, F.A., 2022. The Survey on Students’ Satisfaction Degree Towards Online Learning During Covid-19 Pandemic Condition. JELITA, vol. 3, no. 1, pp. 44 – 56.

BISMALA, L., MANURUNG, Y.H., 2021. Student Satisfaction in e-Learning along the COVID-19 Pandemic with Importance Performance Analysis. International Journal of Evaluation and Research in Education, vol. 10, no. 3, pp. 753 – 759.

CHEN, T., PENG, L., YIN, X., RONG, J., YANG, J., CONG, G., 2020. Analysis of User Satisfaction with Online Education Platforms in China during the COVID-19 Pandemic. Healthcare, vol. 8, no. 3, p. 200.

DZIUBAN, C., MOSKAL, P., THOMPSON, J., KRAMER, L., DECANTIS, G., HERMSDORFER, A., 2015. Student Satisfaction with Online Learning: Is It a Psychological Contract? Journal of Asynchronous Learning Network, vol. 19, no. 2.

FLEMING, J., BECKER, K., NEWTON, C., 2017. Factors for successful e-learning: does age matter? Education + Training, 2017, vol. 59, no. 1, pp. 76 – 89.

GEORGE, D., MALLERY, P., 2010. SPSS for Windows Step by Step: A Simple Guide and Reference 17.0 Update. 10th Edition, Pearson, Boston

GHADERIZEFREH, S., HOOVER, M., 2018. Student Satisfaction with Online Learning in a Blended Course. International Journal of Digital Society, vol. 9, no. 3, pp. 1393 – 1398.

ISMAIL, A., 2018. Empowering your students’ satisfaction with blended learning: A lesson from the Arabian Gulf University distance teaching and training program. International Journal of Information and Education Technology, vol. 8, no. 2, pp. 81 – 94.

KUO, Y. C., WALKER, A.E., BELLAND, B.R., SCHRODER, K.E., 2013. A predictive study of student satisfaction in online education programs. The International Review of Research in Open and Distance Learning, vol. 14, no. 1, pp. 16 – 39.

MOORE, J., 2014. Effects of online interaction and instructor presence on students’ satisfaction and success with online undergraduate public relations course. Journalism & Mass Communication Educator, vol. 69, no. 3, pp. 271 – 288

MORRIS, C., CHIKWA, G., 2014. Screencasts: How effective are they and how do students engage with them? Active Learning in Higher Education, vol. 5, no. 3, p. 25.

MTEBE, J., RAPHAEL, C., 2018. Key factors in learners’ satisfaction with the e-learning system at the University of Dares Salaam, Tanzania. Australas. J. Educ. Technol., vol. 34, no. 4, pp. 107 – 122, doi: 10.1111/j.1747- 1567.2009.00538.x

NIKOU, S., MASLOV, I., 2023. Finnish university students' satisfaction with e-learning outcomes during the COVID-19 pandemic. International Journal of Educational Management, vol. 37, no. 1, pp. 1 – 21.

ROACH, V., LEMASTERS, L., 2006. Satisfaction with Online Learning: A Comparative Descriptive Study. Journal of Interactive Online Learning, vol. 5, no. 3, pp. 317 – 332.

SUN, P.C., TSAI, R.J., FINGER, G., CHEN, Y.Y. YEH, D., 2008. What drives a successful e-Learning? An empirical investigation of the critical factors influencing learner satisfaction. Computers and Education, vol. 50, no. 4, pp. 1183 – 1202.

THOO, A., LEE, Y., TAN, L., 2021. Students’ satisfaction using e as a supplementary tool. International Journal of Emerging Technologies in Learning (iJET), vol. 16, no. 15, pp. 6 – 30.

YOUNAS, M., NOOR, U., ZHOU, X., MENHAS, R., QINGYU, X., 2022. COVID-19, students’ satisfaction about e-learning and academic achievement: Mediating analysis of online influencing factors. Frontiers in psychology, vol. 13, doi.org/10.3389/fpsyg.2022.948061

YUNUSA, A.A., UMAR, I.N., 2021. A scoping review of Critical Predictive Factors (CPFs) of satisfaction and perceived learning outcomes in E-learning environments. Educ. Inf. Technol., vol. 26, pp. 1223 – 1270.

ZEIN, A.E., HILAL, N., JIBAI, B., ATTIEH, L.,2023. Factors Influencing Students’ Satisfaction in Online Learning Amid the Challenging COVID 19 Pandemic: Case Study for Lebanese Educational Sector. ResМilitaris, vol. 13, no. 1, pp. 2923 – 2934.

ZEQIRI, J., KAREVA, V., ALIJA, S., 2021. Blended Learning and Student Satisfaction: The Moderating Effect of Student Performance. Business Systems Research. International journal of the Society for Advancing Innovation and Research in Economy, vol. 12, no. 2, pp. 79 – 94.