https://doi.org/10.53656/str2022-5-5-sel

2022/5, стр. 512 - 525

SELF-ASSESSMENT – A COMPONENT OF THE COMPETENCE-BASED TRAINING IN THE PROFESSION “APPLIED PROGRAMMER”

Резюме: The article discusses the idea of competency-based education (CBE), which is increasingly used to improve training for the modern knowledge-based economy. A general description of such training has been made, in the center of which is the learner. The ideas for ADDIE were used – a training model that meets the objectives of the UNCCD. The article focuses on self-assessment as one of the characteristic forms of CBE assessment. The results of the self-assessment of students participating in the training in the profession “Applied Programmer” are presented. The study involved students studying under the National Program “IT Career Training” (NPITC), with which the CBE was conducted and a group of students in which traditional training in the profession was conducted. The prepared self-assessment questionnaire includes questions related to the development of basic and specific competencies. The questions on specific competencies are based on the competency models used in the IT sector. The results of the survey show that students who participated in the CBE have a higher self-esteem than students trained in the traditional approach. It is necessary to conclude that the CBE is suitable for use in modern education, despite the need for serious preparation before its implementation.

Ключови думи: competence based approach; competency model; software engineering; programming; self-assessment; education

Introduction

The concepts of the competence approach in education have been enshrined in the legislation of the Bulgarian educational system since 2015, and since 2018 the State Educational Standard (SES) \({ }^{1)}\) is a fact for the training on the new for our educational system profession “Applied Programmer” (AP). SES was established by experts from the National Agency for Vocational Education and Training (NAVET) \({ }^{2}\) ), specialists from the Ministry of Education and Science and representatives of the ICT sector. Specialists from the branch have formulated and submitted to the Ministry of Education and Science their Strategic Requirements to the educational system for the implementation of a competency-based approach in the field of ICT training \({ }^{3)}\). They recommend the implementation of a competency-based approach to training. The requirements are described in detail in the competency models for the various professions in the field of computer science (CS) developed by the Association for Computing Machinery \((\mathrm{ACM})^{4}\), together with recommended programs and standards for software engineering (SI) training for high school students \({ }^{5)}\), as well as for bachelors \({ }^{6)}\), masters \({ }^{7)}\) and doctors. The strategic requirements are formulated according to the European Qualifications Framework (EQF) for key competences for lifelong learning \({ }^{8)}\) and the Digital Competence Framework DigComp \({ }^{9)}\). The e-\(\mathrm{CF}^{10)}\) framework, based on the competency models proposed by ACM, is recommended as a reference framework in the field of professional ICT skills. In 2018, the Bulgarian Chamber of Commerce (BCC) offers 12 competency models to those employed in ICT \({ }^{11)}\), according to the proposed strategic requirements and the aforementioned frameworks.

It is important to note the difference between competence and competency – while the requirements for acquired knowledge and skills are focused on what needs to be known, those on competence are related to how it is done, what behavior the practitioner demonstrates, what attitude he has, how much he works independently, and to what extent he takes responsibility for dealing with the activity for which a standard of competence is set. Such an approach to training, selecting and identifying essential competencies that performers must possess is known as a competency-based approach (CBA), and teaching, learning and training aimed at developing such competencies is known as CBE.

Evaluation of learning outcomes at the CBE

The ideas of the CBA can be found in the middle of the last century – Robert White (White 1959) defines competence as the main motive for acquiring knowledge and skills. McClelland (McClelland 1973) uses the term “competence” as a successful combination of knowledge, skills, attitudes and behaviors of employees to achieve results. Boyatzis (1982) proposes a new strategy in the field of human resource management to increase the efficiency and potential of companies. L& M. Spencer (1993) developed a generic “vocabulary of competencies” for excellence.

The concepts of competency and competence have prevailed in almost the same, but often different in separate nuances meaning. According to the American school, competency is a quality, an ability inherent to the best ones, a standard to which everyone should strive. The British school uses the concept of competence as a minimum requirement, a standard that all practitioners must have.

According to the growing interpretation of the two concepts among scientists, competency should be seen as a requirement, a standard by which to compare, to which specialists should strive, while competence is an integral concept characteristic of the individual's set of abilities that meet certain requirements.

White, McCleland, Boyatsis give their definitions of the concept of competence and/or competency. For the purposes of this article we will use the definition of the European Commission (EC):

Competence – a set of knowledge, skills, attitudes and behaviors of employees needed to achieve certain results.

Competence is most often associated with the ability to understand something that can be done, ie. knowledge-based skills. Competences are abilities, but not innate, but “those that are developed through quality learning, in an appropriate pedagogical environment and through the acquisition of serious practical experience” (Zwell 2000). Knowledge is a valuable resource that is acquired with a lot of effort, self-control and work aimed at self-improvement of the individual. The earlier the age of children working in this direction, the higher the level of competence over time. The knowledge and skills acquired by the students reflect on the degree of competence in each activity by interests and activities.

The CBE paradigm has its roots in the constructivist theory of pedagogy, whose first representatives are considered to be Lev Vigotskiy (2005) and Jean Piaget (1969) \({ }^{12)}\). According to the constructivist theory, it is based on the understanding of the activity of the knowing subject (Kasavin 2009; Lebedev 2004; Mikeshina 2005). Georgieva (2014) highlights the weaknesses of traditional learning: “Many researchers, teachers and students admit that the traditional approaches to 'Teaching as a transfer of knowledge' have serious shortcomings.” In it, students are often unable to apply what they have learned in real life outside school (Boaler 1998; Von Glasersfeld 1984) or simply forget what they have learnt by heart over time (Von Glasersfeld 1984). Boaler (1998) shares the conclusions of mathematics teachers that students are unable to use the methods and rules learned in traditional schools due to their lack of understanding. There is a need to transform traditional learning to one in which the learner is at the center of learning and develop the learner's ability to solve practical problems individually or in a team. Learning that is 'active', aimed at enhancing students' cognitive activity throughout the learning process (Gyurova 2006).

At the center of the CBE is the student. The aim is to achieve results described as expected behaviors in the successful implementation of the learning activity, the formation of competencies, rather than teaching certain learning materials.

In creating curricula, the constructivist principle is decisive – the student to be a researcher and to participate in the evaluation of their own achievements, discovering and overcoming gaps. The teacher is to diagnose difficulties and gaps in a timely manner and to adapt the learning content, personalizing it for the needs of the learner. Assessment does not aim at quantifying, but serves to correct pedagogical interaction to improve learning outcomes. Suitable for this purpose is the so-called. ADDIE (Analyze, Design, Develop, Implement, Evaluate) model \({ }^{13)}\) for curriculum development and follow-up training. The main phases are: Needs analysis – students communicate their interests, and the teacher gets acquainted with the level of students by making an initial diagnosis of the level of knowledge, skills and competencies that he aims to develop in them. The design includes Planning (design) Analyze of appropriate learning content and teaching methodology. Development involves the creation of lesson units. Implementation implementation of training. Evaluate – evaluation of learning outcomes of training.

In the CBE, assessment is formative (FD) \({ }^{14)}\), also known by the term “education assessment” and summarizing (summative). The combination of which leads to better results (Dante 2016; Connors 2021). FD is aimed at feedback at each stage, which aims to determine how much new knowledge has been mastered and what difficulties are encountered.

According to (Grangeat 2021) such professional moves are complex because implementing formative and summative assessments in a complementary manner remains a challenge for teachers and researchers.

In the subsequent analysis in the cycle, if necessary, personalization of the teaching is made in order to achieve the target ROs. Given that this assessment takes place during the learning process and is used to support learning, it is also known as 'ongoing assessment'. This can be done by a teacher or other learner who checks and reports the results of the students' work, without having to make an assessment. The purpose of the ongoing assessment is to direct the learner and the teacher to the problematic part of the learning process in which the learner has difficulty in mastering it, and not in making assessments. Students define their own work and are encouraged to ask questions about the material they are studying. In the current assessment of the modules related to programming, it is appropriate to use automated assessment systems (Judge systems), (Wasik 2016, 2018; Yadav 2017).

In the evaluation of tasks related to writing program code, evaluation systems are a kind of tool for automated code testing. Given the various possible strategies for action, the developed such a way assessment systems offer a number of didactic opportunities to facilitate the learning and self-preparation of learners. The evaluation systems can be adapted to support automatically, with varying degrees of assistance, guidance in cases where programs are not working properly. Such systems allow the learner to receive timely feedback on their progress.

The tools for checking the achieved learning outcomes include: tests, practical tasks, self-assessments, expert assessments, \(360^{\circ}\) assessments. The latter is a mutual evaluation between learners \({ }^{15)}\).

For the purposes of the study we will focus in more detail on the tool “Selfassessment”.

Self-esteem is the judgment we give ourselves. It is an assessment of one's own qualities, skills and way of doing the job. Self-assessment is related to identifying our strengths and weaknesses and guidelines for further training and professional development. The ability to self-assess is a skill of a high order and it must be developed and improved. According to Staribratov, self-assessment is not an isolated method unknown to the teacher, but the idea is not to use it sporadically, but to be a system that is purposefully used throughout the year, throughout the training period. For which the teacher must build a strategy and actively involve students in its implementation (Staribratov 2021). A very large part of high school students use self-assessment as a building block for acquiring new knowledge. The teacher is the engine and motivating factor for their commitment, and this leads to high results and a sense of satisfaction with the work done.

Self-esteem can also be complete, generalized, also called holistic, authentic. Such self-assessment is an assessment obtained as a result of all self-assessments on individual topics and sub-topics that one gives for themselves. Such self-assessment concerns the extent to which a person is considered significant on the basis of standards that he or she considers authentic. Such self-esteem is psychological in nature and is not the goal of specific research, despite its importance to the individual.

The CBE works especially successfully and effectively to build a positive selfesteem, where mistakes are seen as a natural stage of the learning process. "Pointing out failures and mistakes is measured, respectful and related to opportunities and prospects. Incentive-based training is supported. In this way, students develop a sense of significance, confidence and satisfaction. (Brankova 2021). The application of innovative methods of teaching and assessment is a key factor for increasing knowledge and skills, for developing and expanding the competencies of students.

Self-assessment (Staribratov 2021) is a way of assessing, diagnosing and reflecting on one's own knowledge, a method that modern students do not master at a good enough level.

Given the proposed definitions and clarifications in the sense of the term "selfassessment", we present you the study.

Purpose of the study

To check whether the methodological approach used for CBE has a positive effect on the results of students' self-assessment.

Materials and tools for the study

For the purpose of the research, a questionnaire was created with questions selected on the basis of the expected learning outcomes set out in the SES for AP, as well as the expected skills for writing quality program code described in the competency models and profiles. Each competence (competence according to the SES terminology for AP) is described by behavioral indicators – suitable for the activity verbs from Bloom's taxonomy, which the learner performs during his work.

Other dimensions of competence include: Degree of responsibility, degree of independence, compliance with a quality standard, compliance with a safety standard.

Course of the researc

The students are divided into 2 groups:

– Experimental group (EG) – 53 students trained at the centers of Plovdiv and Burgas by NPITC, for whom a methodological approach for CBТ has been applied, – Control Group (CG) – 116 students from the regular form of education, in which traditional education in the same schools was used.

After training in half of the AP modules, a survey was conducted in both groups for self-assessment of the results of students' learning with the help of the Google Forms tool. The questions explore key and specific competencies related to Algorithms and Data Structures, Functional and Object Oriented Programming, skills for writing quality program code. For each question, students indicate the degree of mastery of the specific competence in 5 possible levels – from the lowest first level – 1, to the highest fifth level – 5. Each of the levels is described in advance what it means, through relevant indicators of mastery and behavior which students demonstrate in solving problems related to competence. Each level after the first includes as mastered competencies the previous levels shown in Figure 1.

Research results

The processing of the results goes through several steps. After data collection, a summary of the results was made and their frequency distributions were compiled. For each level, the absolute frequency was calculated, showing the number of responses for a given level, as well as its percentage, by its reference frequency. The calculated cumulative frequency shows how much of the students cover this and the lower levels, given the principle of including the lower levels. The results are presented in Table 1,2,3:

Of interest is the test of the hypothesis whether the students of the new methodology have a higher self-esteem at a significance level of 0.05.

As the two samples are derived from different general populations, there is a case of two independent samples with different variances of the general populations. The following hypotheses are formulated for solving the problem:

\[ H_{0}: \mu_{1}-\mu_{2}=0 \text { и } H_{a}: \mu_{1}-\mu_{2} \gt 0 \] To find the test statistics and their critical values at a predetermined level of significance, we will apply t-Test: Two-Sample Assuming Unequal Variances. The results are presented in Table 4, 5, 6:

All three test statistics \(4.78 ; 3.73\) and 4.82222 exceed the critical value for the one-sided test 1.75, therefore the null hypothesis is rejected. This means that it can be assumed that there is a statistically significant difference between the level of self-esteem of students in the two groups at a significance level of 0.05.

Discussion

With regard to key competences, there is an interesting result of Question 2. Which is an exception to the overall trend. Further study of the data obtained can be done to find an explanation for the better result in the control group. In addition, the other type of assessment will be conducted with the trained students – Expert assessment, which will provide more data for analysis of the cause.

With regard to the CBE, we can say that it requires additional preliminary work, few teachers are ready to implement it, but the results are encouraging.

Conclusion

The CBE is a successful form of education that is increasingly used to improve the training of the modern knowledge-based economy. At the center of the CBE is the learner. ADDIE – the training model meets the objectives of the CBE.

A key component of the CBE is the feedback that trainers and trainees exchange with one another. Students' self-assessment is a complex of: information about their own achievements, attitude to the value of personal and professional abilities, as well as most importantly - psychological self-perception of one's own personality and its significance. In the context of the CBE, the process of building a true selfassessment of personal and professional competencies is a key factor for both successful education at school and the subsequent realization at the labor market. Building such reflective ability in learners should be a priority of education. Authentic self-assessment can most successfully indicate and “suggest” where the gaps are and what difficulties the person is facing. It is this autonomous element in the personality, without which it is impossible to fully integrate it into social and public life, which in turn is the meaning of education.

From the presented results of the research it follows that the CBE leads to better results in learners in self-assessment of key and professional competencies. It is necessary to conclude that the CBE is suitable for use in modern education, despite the need for serious preparation before its implementation.

Acknowledgements

The work on the article is partially supported by the MU21-FMI-011 project of the Fund “Scientific research” at the University of Plovdiv “Paisii Hilendarski”.

NOTES

1. NES PAP https://www.mon.bg/upload/14210/dos_481030.pdf [Last visited: 13.04.2022].

2. NAPOO https://www.navet.government.bg/bg/sashtnost-na-spisaka-naprofesiite-za-pr/ [Last visited: 13.04.2022].

3. Strategicheski iziskvania na softuernata industria za reforma na obrazovatelnata sistema https://basscom.org/RapidASPEditor/MyUploadDocs/SoftwareIndustry-Requirements-for-Educational-Ref.pdf [Last visited: 13.04.2022].

4. ACM https://www.acm.org/ [Last visited: 13.04.2022].

5. CSTA K-12 https://www.doe.k12.de.us/cms/lib/DE01922744/Centricity/ Domain/176/CSTA%20Computer%20Science%20Standards%20Revised%20 2017.pdf [Last visited: 13.04.2022]

6. Software Engineering 2014 Curriculum Guidelines for Undergraduate Degree Programs in Software Engineering https://www.acm.org/binaries/content/ assets/education/se2014.pdf [Last visited: 13.04.2022].

7. Graduate Software Engineering 2009(GSwE2009) Curriculum Guidelines for Graduate Degree Programs in Software Engineering https://www.acm.org/ binaries/content/assets/education/gsew2009.pdf [Last visited: 13.04.2022].

8. European-qualifications-framework https://europa.eu/europass/bg/europeanqualifications-framework-eqf [Last visited: 13.04.2022].

9. The digital competence framework for citizens with eight proficiency levels and examples of use https://op.europa.eu/bg/publication-detail/-/ publication/3c5e7879-308f-11e7-9412-01aa75ed71a1 [Last visited: 13.04.2022].

10. e-CF https://itprofessionalism.org/about-it-professionalism/competences/thee-competence-framework/ [Last visited: 13.04.2022].

11. Sektoren model s vklyucheni 12 klyuchovi dlazhnosti https://www.basscom.org/ RapidASPEditor/MyUploadDocs/SCM_IT12_final.pdf [Last visited: 13.04.2022].

12. Konstruktivizmat kato teoria za prepodavane i uchene https://bg.larafornm. com/constructivism-128 [Last visited: 13.04.2022].

13. ADDIE https://en.wikipedia.org/wiki/ADDIE_Model [Last visited: 13.04.2022].

14. Formative and Summative assesment https://web.archive.org/ web/20090312035549/http://www.ukcle.ac.uk/resources/assessment/formative. html [Last visited: 13.04.2022].

15. MyCompetence https://mycompetence.bg/bg/static/10 [Last visited: 13.04.2022].

REFERENCES

BOALER, J. 1998. Open and Closed Mathematics: Student Experiences and Understandings. Journal for Research in Mathematics Education, 29(1), 41 – 62. Available at: https://doi.org/10.2307/749717

BOYATZIS, R., 1982. The Competent Manager: A Model for Effective Performance. NY: John Wiley & Sons.

BRANKOVA, Z., 2021. Aktivnoto uchene kato sredstvo za razvivane na klyuchovi kompetentnosti. Prodalzhavashto obrazovanie TOM 16/2021 Available at: https://diuu.bg/emag/9052/.

CONNORS, C. B., 2021. Summative and Formative Assessments: An Educational Polarity. Kappa Delta Pi Record. 57(2), 70 – 74. DOI: 10.1080/00228958.2021.1890441.

DANTE D., WORRELL, F., 2016. Formative and Summative Assessment in the Classroom. Theory Into Practice. 55(2), 153 – 159, DOI: 10.1080/00405841.2016.1148989

GEORGIEVA, D., 2014, Rolyata na konstruktivizma v obuchenieto po matematika. Nauchni trudove na rusenskia universitet – 2014, 53, seria 6.2., 173 – 179, ISSN 1311-3321. (In Bulgarian).

GRANGEAT M., HARRISON, C., DOLIN, J. 2021. Exploring assessment in STEM inquiry learning classrooms, International Journal of Science Education, 43(3), 345 – 361. DOI: 10.1080/09500693.2021.1903617

GYUROVA, V.; DERMENDZHIEVA, G.; VARBANOVA S. & BOZHILOVA, V., 2006. Priklyuchenieto ucheben protses. Sofia: Agentsia Evropres. (In Bulgarian).

KASAVIN, I., 2009. Entsiklopedia epistemologii i filosofii nauki. Available at: http://epistemology_of_science.academic.ru/

LEBEDEV, S., 2004. Filosofia nauki: Slovary osnovnыh terminov. Moskow: Akademicheskiy Proekt. Available at: http://terme.ru/dictionary/905/ word/konstruktivizm-filosofskii.

MCCLELLAND, D., 1973. Testing for competence rather than for intelligence. American Psychologist.28(4), 1 – 14. [viewed 15 April 2022]. Available from: http://dx.doi.org/10.1037/h0034092.

MIKESHINA, L., 2005. Filosofia nauki. Sovremennaya эpistemologia. Nauchnoe znanie dinamike kulyturы. Metodologia nauchnogo issledovania. Uchebnoe posobie. Moskva: Progress-Traditsia; Moskovskiy psihologo-sotsialynыy institut Izdatelystvo “Flinta”.

PIAGET, J., 1969. Psychologie et pédagogie. La réponse du grand psychologue aux problèmes de l'enseignement. – Paris, Denoël, 1969. – \(20 \mathrm{~cm}, 264\) p. (Médiations).

SPENCER, L., SPENCER, S., 1993. Competence atWork: Models for Superior Performance. New York: John Wiley & Sons.

STARIBRATOV, I., 2021. Samootsenyavaneto – element ot kompetetnostnia model na obuchenie. Pedagogika na obuchenieto po matematika \(i\) informatika 6, sbornik dokladi. ISBN 2534-8795.

Vigotski, L., 2005. Izbrani psihologicheski proizvedenia, Sofia: PSIDO EOOD.

VON GLASERSFELD, E., 1984. An introduction to radical constructivism. In:

PAUL WATZLAWIK (Ed.). The Invented Reality. (pp. 1 – 29). Norton, New York (Reprinted from Die Erfindung der Wirklichkeit. Piper, Munich, 1981).

ZWELL, M., 2000. Creating a Competence. New York, Published by John Wiley & Sons Inc, New York, ISBN 10: 0471350745 / ISBN 13: 9780471350743