https://doi.org/10.53656/math2022-3-3-eff

2022/3, стр. 259 - 275

EFFECTS OF SHORT-TERM STEM INTERVENTION ON THE ACHIEVEMENT OF \(9^{\text {TH }}\) GRADE STUDENTS IN MATHEMATICS

Резюме: The aim of this study is to examine the effect of the STEM approach on students’ achievements in processing teaching topics from Algebra. A quasiexperimental design was used with an experimental and control group. The research was conducted on a sample of 73 ninth-grade students (aged between 14 and 15). STEM approach was performed in teaching the experimental group, while the control group was taught in a traditional way. The obtained results showed that the STEM intervention had a positive impact reflected in significantly better results on the achievement tests. The girls performed better than boys on all tests.

Ключови думи: algebra; gender differences; STEM activities; STEM approach

Introduction

STEM is an approach to education that enables the integration of science, technology, engineering, and mathematics, and enables the application of theoretical knowledge in practice (Benek & Akcay 2019). The STEM approach aims to develop with students the ability of interdisciplinary cooperation, openness to communication, development of ethical values, research skills, creativity and proper problem solving (Buyruk & Korkmaz 2016). The STEM approach improves student achievement in mathematics and science, increases motivation and interest in learning, and problem-solving ability (Furner & Kum 2007; Stinson et al. 2009).

Increasingly rapid changes and progress in the world need to be accompanied by innovations, which require the so-called \(21{ }^{\text {st }}\) century skills: critical thinking, problem-solving ability, collaboration, leadership ability, adaptability, communication ability, information literacy, curiosity and imagination (Buyruk & Korkmaz 2016). The development of these skills is closely linked to the STEM approach in education and research.

The time in which we live is such that new knowledge is acquired quickly, that new occupations replace existing ones, and that the demands within the occupations that survive are changing and thus become more complex. Therefore, new generations of students need to be qualified to cope with the challenges that await them in life as successfully as possible, which should be a priority task of education. That is why the STEM approach in education and research is of public interest to societies around the planet (Avdispahić 2018).

In order for a country to become a developed nation, it is necessary for its education system to put emphasis on STEM and that STEM is integrated into the curriculum at the earliest level of education (Washington STEM Study Group 2011; according to Abdullah N. et al. 2014). Given the problem of declining number of students who want to continue their education in the fields of science, mathematics and engineering (Bakırcı & Karışan 2018), it is necessary to change and upgrade curricula constantly in order to arouse interest in STEM areas.

Mathematics and natural sciences (physics, biology and chemistry) can be connected the most in the educational system. They are so closely related that it would be best to use an interdisciplinary approach to teach them (Berlin & White 1994). According to Bossé et al. (2010), mathematics and the natural sciences are linked because they are taught in a similar way, and other natural sciences provide mathematics with the context and specific examples so that learning mathematics could be easier and more interesting. On the other hand, mathematics serves as a useful tool for a deeper understanding and analysis of phenomena in those other areas (Basista & Mathews 2002; McBride & Silverman 1991).

Soylu (2016) considers that the early introduction of mathematics and natural sciences in education is of great importance for children’s future academic success as well as for the development of critical thinking. Research shows that the introduction of mathematics in the earliest stages of education has a positive impact on later achievements in mathematics (Duncan et al. 2007; Clementset et al. 2011). Previous research has shown that the STEM approach also exerts a positive impact on academic achievement in primary and secondary school students (Han et al. 2016; Judson 2014). McCaslin’s (2015; as cited in Siregar et al. 2019) experimental study with fourth-graders showed a positive effect of the STEM approach on student achievements in the following areas of mathematics: number and operations, measurement, analysis, geometry and algebra. Research conducted by Çaycı and Tabaru (2019) with fourth graders showed that there is a statistically significant difference in academic achievement between the control and experimental group (STEM approach).

Regarding gender differences in achievements, previous research on mathematical achievements has shown that differences in achievements in mathematics in boys and girls are declining (Halpern et al. 2007; Hedges & Nowell 1995; Hydeet et al. 2008; Lindberg et al. 2010).

The development of STEM competencies has been a preoccupation of most countries for years, while in Bosnia and Herzegovina (B&H), unfortunately, only recently. In B&H, the PISA survey was conducted for the first time in 2018, and the TIMSS survey in 2007 and 2019. The results of the PISA survey showed that the minimum level of functional literacy does not reach 58 percent of students in mathematics, 54 in reading and 57 in science. In both TIMSS surveys (2007 and 2019), students from B&H accomplished achievements that are below the average of the TIMSS scale.

The introduction of STEM education in B&H is at an early stage. The Agency for Pre-Primary, Primary and Secondary Education (APOSO) began to develop Common Core Curriculum defined on learning outcomes (CCC) in 2012. In this process, APOSO has defined eight subjects of education, and one of them is the subject of mathematics.

In 2015, APOSO published the CCC for mathematics. In it, mathematics is divided in four areas: Sets, Numbers and operations; Algebra; Geometry and measurements; Data and probability. Each area was divided into two or three components, and for each component learning outcomes were defined. In the end, indicators were defined for each learning outcome in accordance with the developmental age of a child: at the end of preschool education (age 5 – 6), at the end of the third grade (age 8 – 9), at the end of the sixth grade (age 11 – 12), at the end of the ninth grade (age 14 – 15) and at the end of the secondary education (age 18 – 19).

For example, the area of algebra is divided into components:

Component 1: Algebraic expressions, functions, proportions, and applications.

Component 2: Equations, inequalities, and their representation.

Component 3: Elements of logic.

For Component 1, the learning outcomes are as follows:

1. Analyze laws, relationships, dependencies, connections, and functions in mathematics and the real world.

2. Analyze and display mathematical situations and structures using algebraic symbols and various notations (records), graphs and diagrams, and generalize based on them.

3. Apply mathematical models to represent and interpret quantitative relationships.

4. Analyze and formulate assumptions of change in different contexts.

For example, for learning outcome 2, the indicators at the end of the ninth grade are:

2a. Display graphically direct and inverse proportionality as well as a linear function.

2b. Apply powers with the whole exponent.

2c. Apply operations with polynomials.

For each of the six STEM subjects – Biology, Chemistry, Physics, Geography, Mathematics, Technics and Information Technology, and for each developmental age of students, a Draft of Operational Teaching Curriculum (OTC) for STEM competencies has been prepared. OTC for mathematics provides numerous links between mathematics and other STEM disciplines. The document is divided into four parts according to the age of the students (Grades 1 – 3, 4 – 6, 7 – 9, and 10 – 13). Each of these four parts is further divided into four areas of mathematics, which are then divided into components, then into outcomes and indicators. Outcomes and indicators are related to the areas of a knowledge-based economy in which they are applicable and to outcomes/ indicators from other STEM disciplines. The document also offers examples of interdisciplinary STEM projects.

Given that the STEM approach increases motivation and interest in STEM subjects, and considering that the introduction of STEM education in B&H is only in its initial stage, this motivated us to conduct research described in this paper. Therefore, the main goal of this study is to determine the impact of the STEM approach/ STEM activities on the achievements of ninth grade students in mathematics. The following specific objectives ensue from the basic objective:

1. Examine the differences in mathematics achievement between the experimental and control groups before and after intervention.

2. Examine differences in math achievements between boys and girls before and after intervention.

Methods

Participants

The research was conducted in two phases, in which a quasi-experimental design was used, with the experimental (E) and control group (C) and with pretest-posttest. A total of 73 ninth-grade students participated.

Prior to this research, students were taught in the traditional way and through this research they were introduced to STEM for the first time. They studied STEM subjects (natural sciences, mathematics, informatics, technical culture) separately because very often the curricula are not harmonized.

Instruments

Before this research, students did an achievement test (Pre-test) which included teaching material from the 8th grade, which is related to the teaching material done during the research. A pre-test was carried out to determine if there was a statistically significant difference between the groups. Based on the results of the pre-test, two groups were formed: control and experimental.

After the completion of the first phase of the research, the students did an achievement test (Post-test 1), which included the material done during the research. During the second phase of the research, students did another achievement test (Post-test 2) which included material processed in the second phase up to the moment of testing (problems with one unknown and linear equations with two unknowns). However, due to the interruption of classes caused by the coronavirus, the students did not do a final test that was supposed to include systems of linear equations with two unknowns.

For each of the tests (Pre-test, Post-test 1 and Post-test 2) the reliability was tested using the Cronbach’s Alpha coefficient and the following values were obtained: \(\alpha=0.74, \alpha=0.85\) and \(\alpha=0.80\) respectively, which indicates that the tests had good reliability. It is important to note that all achievement tests were classic tests in terms of that they did not benefit the students of the experimental group by setting of tasks or work methodology. Tasks from one test (Post-test 1) can be seen at: https://drive. google.com/file/d/1O32TH8VRw-SXR7ZaHr51EAPogboR7wue/view?usp=sharing.

STEM intervention

The research took place in two phases. The first phase of the research included teaching topics: Graphs of the function of direct and inverse proportionality and Linear function which belong to Component 1 of the area Algebra. The second phase included the teaching topic: Linear equations and inequalities with one unknown – Problems with one unknown and Systems of linear equations with two unknowns which belong to Component 2 of the area Algebra. The first phase of the research was conducted in the first semester, and the second phase in the second semester in the 2019/20 school year.

In the experimental group the teaching was conducted using the STEM approach, where certain STEM activities were carried out with the attempt to connect the teaching contents in mathematics with the contents from other subjects and examples from everyday life. Some of the examples that have been done with students are listed below.

1. The students were introduced with the graph and the direction of homogeneous linear function \(y=k x\) and the linear function \(y=k x+n\), as well as the meaning of the parameters \(k\) and \(n\), by using GeoGebra software. Also, the graphical method of solving a system of linear equations with two unknowns was explained to students by applying GeoGebra software.

2. A linear motion is described with the equation \(s=v \cdot t\), where for a constant velocity we have a homogeneous linear function. This is an example of connecting a homogeneous linear function with the subject Physics. On the made path, the students marked the length of the road of \(1 \mathrm{~m}, 2 \mathrm{~m}\) and 3 m and measured the time (with stopwatches on mobile phones) that a car needed to cross the length of the road of \(1 \mathrm{~m}, 2 \mathrm{~m}\) and 3m. The results obtained were noted in a prepared table on the basis of which they were supposed to conclude that the car was moving at a constant speed. They obtained \(s=0.5 \cdot t\). Thus, we have a homogeneous linear function \(y=k x\), where \(k=0.5, y\) is the distance traveled, and \(x\) is the time.

3. In groups, similar to example 2, the students determined the mass and volume of one, two, three and four of the same cubes, and based on the obtained results, they should have come to the conclusion that the mass increases with increasing volume. Thus, they should have concluded that the function \(m=\rho \cdot V\) which describes the dependence of mass on volume is an example of the direct proportionality function in the case where the density is constant (for more information, see the example of this lesson plan in the following address: https://drive.google.com/file/d/1O32TH8VRwSXR7ZaHr51EAPogboR7wue/view?usp=sharing.

4. A worksheet, on which the Celsius and Kelvin temperature scales were given, was distributed to the students. Their task was to express K as a function of \({ }^{\circ} \mathrm{C}\), and to express \({ }^{\circ} \mathrm{C}\) as a function of K. Then, they had to convert the specific values of temperature in \({ }^{\circ} \mathrm{C}\) into K and thus check the obtained equations.

5. Students were given the task to make a pattern of squares measuring \(1 \times 1 \mathrm{~cm}\) cut from collage paper in two different colors. Previously, students had to solve a textual problem by applying a system of linear equations with two unknowns and, based on the solution of the system, to determine how many total squares they can use and how many for each color.

6. By working in groups, students were given the task to solve a textual problem using a system of linear equations with two unknowns. Based on the solution of the system, they had to choose the number of long and short sticks and make the tower as high as possible. This task was of a competitive character as the team that built the tallest tower won.

7. The students made pencil holders from rolls of paper and straws. Since a circle can be inscribed in a square/equilateral triangle, students make squares/ equilateral triangles out of straws, which are placed around a roll of paper.

Before that, they had to determine the dimension of the side of the square/ equilateral triangle.

The way in which the STEM approach is applied in classroom, and/or how the existing unit is integrated with OTC, is visible in the lesson plan available at the address https://drive.google.com/file/d/1O32TH8VRw-SXR7ZaHr51EAPogboR7wue/ view?usp=sharing.

In the control group, teaching took place in the traditional way in which the frontal form of work dominates, teaching without the use of technology, without experiments/independent activities of students.

Analysis of results

The data were analyzed using the SPSS 20.0 program.

The first specific goal of the research was to examine the differences in mathematics achievements between the experimental and control groups before and after the intervention. Whether the data were normally distributed in both populations (E and C groups) was checked using the Shapiro-Wilk test (\(n \lt 50\) ), skewness and kurtosis, and \(z\)-score. The results are shown in Table 2.

From the Table 2, it is evident that the Shapiro-Wilk test indicates a statistically significant deviation of distributions from normal ones in some populations (\(p \lt 0.05\) ). However, by analysis of skewness and kurtosis, that is \(z\)-score, it was concluded that the values are within acceptable limits \((|\mathrm{z}| \lt 1.96)\) and data have a normal distribution (Kim 2013; Mishra et al. 2019). Therefore, a \(t\)-test for independent sample was used to examine the differences in the results of math achievement tests between the experimental and control groups. The results are given in Table 3.

Table 3 demonstrates that there is no statistically significant difference in achievement on the pre-intervention math test. After the intervention, the difference in achievement on the math test between the experimental and control groups was statistically significant on both tests (Post-test 1 and Post-test 2), whereby the students of the experimental group did both tests much better. These results suggest that STEM activities had a positive effect on the achievement of the students in the experimental group.

We examined differences in achievements on a math test between boys and girls before and after the intervention, which was our second specific goal of the study. We first checked whether the data were normally distributed in both populations (girls and boys) using the Shapiro-Wilk test \((n \lt 50)\), skewness and kurtosis, that is \(z\)-score, and the results are shown in Table 4.

It ensues from the Table 4 that the Shapiro-Wilk test indicates a statistically significant deviation of distributions from normal in some populations for some measurements (\(p \lt 0.05\) ). By analysis of skewness and kurtosis, that is \(z\)-score, it was concluded that in one population the \(z\)-score is not within acceptable limits (\(|z| \gt 1.96\) ), and accordingly we conclude that the data do not have a normal distribution. Therefore, the Mann Whitney-U test was used to examine the differences in math achievement scores between boys and girls as an alternative to the \(t\)test when the data did not have a normal distribution and when it concerned a small sample (Leech & Onwuegbuzie 2002; Kitchen 2009). The results are given in Table 5.

As it can be seen from Table 5, the difference in achievements on the math test with respect to gender before and after the intervention is not statistically significant for either group: control and experimental. From the table it can also be seen that the girls achieved better results in both groups (E and C), but the difference is not statistically significant.

Discussion and conclusion

The main objective of this research was to investigate the effect of the STEM approach/activities on mathematics achievements in ninth grade students, in relation to learning teaching topics: Graphs of direct and inverse proportionality functions, Linear function, Linear equations and inequalities with one unknown – Problems with one unknown and Systems of linear equations with two unknowns. Differences in achievement between the experimental and control groups were examined before and after the intervention.

The results obtained before the intervention tell us that the difference in achievements on the mathematics test (Pre-test) between the experimental and control groups is not statistically significant. After the Pre-test, students were assigned to a control and experimental group, so there was no statistically significant difference between the two groups on the pre-test. At the end of the first phase of the research, we reexamined the differences in achievement on the test in mathematics (Post-test 1) between the experimental and control groups and the difference between the groups was statistically significant: \(p=0.020\). In addition, the results obtained after the second phase of the research (Post-test 2) indicate that the difference in achievements between the experimental and control groups is statistically significant in favor of the experimental group (\(p=0.009\) ). Thus, in both tests the students of the experimental group achieved significantly better results. The results showed that the use of STEM approach/STEM activities in mathematics teaching is more effective in improving student achievement in mathematics compared to traditional teaching methods. The obtained results are in agreement with the results of previous research which showed that STEM activities have a positive effect on increasing student achievement (Çaycı & Tabaru 2019; Olivarez 2012; Tank 2014; Yıldırım & Altun 2015).

In this research, the teaching materials used in the experimental group have an important effect on the achievements of students in mathematics and significantly enhance the practical abilities of students from the experimental group. Such materials allow students to try different solutions, research and build their own knowledge. During the research, GeoGebra software was used, which offers a number of advantages: it encourages students’ intellectual curiosity, creativity and the ability to think logically. Some previous research has shown that GeoGebra has a positive impact on student achievement in geometry, algebra, and computation (Botana et al. 2015; Liburd & Jen 2021; Zengin et al. 2012). The results of numerous studies have shown that computer-assisted teaching is more effective in improving student achievement in mathematics compared to traditional teaching methods. Birgin et al. (2015), in a study aimed at determining the impact of computer-assisted teaching on achievement and attitudes toward mathematics in seventh-grade students, showed that there is a significant difference in mathematics achievement between the experimental and control groups. Işıksal and Aşkar (2005) investigated the effect of computer-assisted teaching, which consisted of the use of Excel and a computer program for dynamic geometry (Autograph), and compared it with traditional teaching. The topic was “Coordinates of a point in a plane, symmetry and a graph of a linear function”. The results of the research showed that the mathematical achievements of the group taught with the help of the computer program for dynamic geometry were significantly better than the achievements of the group that was taught in the traditional way and with the Excel computer program. A study by Kurt and Benzer (2020), in which \(6^{\text {th }}\) grade students participated, also showed a positive effect of the STEM approach on increasing student achievement in mathematics.

One of the goals of this study was to examine the achievements in mathematics in boys and girls before the intervention and to examine the achievements in mathematics in boys and girls after the intervention. Based on the results shown in Table 5, it is seen that the difference in achievement in mathematics in boys and girls before and after the intervention is not statistically significant for either group: experimental and control. However, it is important to mention that in all three tests the girls achieved better results.

Research by Yaratan and Kasapoğlu (2012) showed that girls have more positive attitudes towards mathematics and higher mathematical achievements than boys. Possible reasons for better results in achievements are more positive atti tudes of girls towards mathematics. Developing students’ positive attitudes toward mathematics increases achievement, while students’ negative attitudes and poor motivation reduce academic achievement. Students’ involvement in mathematical activities depends on their attitudes towards mathematics. The obtained results are in line with the conclusions derived from previous research that STEM intervention has an equal effect on the attitudes, and thus on the achievements, of students of different genders (Bararović et al. 2018, Rosenzweig & Wigfield 2016).

Research by Ajai and Imoko (2015) in which quasi-experimental design was applied also showed that teaching algebra through activities involving problem-solving learning (PBL activities) did not lead to a statistically significant difference in achievement between boys and girls. Some previous research has shown that girls are just as successful as boys, and sometimes even more successful, in most of school subjects – including mathematics and other natural sciences (Burušić et al. 2012; Duckworth & Seligman 2006; Hydeet al. 2008; Lindberg et al. 2010; Mullis et al. 2016; OECD 2016; Stoet & Geary 2013; Voyer & Voyer 2014; as cited in Blažev et al. 2017).

The results of the international PISA survey (OECD 2016, 2019) demonstrate that in most countries there are no gender differences in achievements in the natural sciences and mathematics. There exist gender differences in achievements in a smaller number of countries, more precisely in some countries in favor of boys and in some in favor of girls. However, these gender differences are generally small. In general, there is no evidence that boys outperform girls in achievement in the STEM field. In the OECD countries in the 2018 PISA survey, boys’ scores in math are on average five points higher than girls’ scores. There is also a difference in B&H, but it is not statistically significant because it is slightly more than two points in favor of boys. In mathematics, the biggest difference in achievements by gender is among Austrian students, specifically 13 points in favor of boys (APOSO 2019).

This research showed that the use of STEM approach/STEM activities in mathematics teaching led to a significant difference in the development of mathematics achievements in ninth grade students in the processing of certain topics in the field of Algebra. This research has shown that teaching in which the STEM approach is applied contributes to better student achievements in mathematics. Therefore, it would be useful to examine the effect of the STEM approach in mathematics teaching for various topics from different areas of mathematics and for different grades. The obtained results show that the intervention had an equal effect on students of different gender because in the tests done after the intervention, the difference in achievements between girls and boys was not statistically significant. The girls achieved better results on all three achievement tests (but not statistically significantly better).

The STEM approach develops a positive attitude of students towards mathematics, and by developing a positive attitude success also increases (Yaratan & Kasapoğlu 2012). This research was limited because of COVID-19, but also because the STEM approach to teaching has recently been introduced in B&H. There will be shorter projects within the traditional teaching, but not everything will change. Therefore, our goal was to see if such short interventions have an impact on student achievement in mathematics. This research was conducted with 73 ninth grade students. New research can be conducted with a larger sample, and in different grades, and may include some other teaching topics or areas of mathematics such as Geometry and measurement.

Regardless of the above mentioned limitations, the results of this study show that short-term (two-weeks - first phase) implementation of teaching by using the STEM approach has led to a significant difference in student achievements in mathematics (Post-test 1) compared to traditional teaching methods. Thus, we can recommend the introduction of STEM in schools. We hope that research will serve as a motivation for all teachers to use the STEM approach in mathematics teaching at least occasionally (when dealing with some specific teaching topics) – and adopt STEM curriculum.

NOTES

1. Agencija za predškolsko, osnovno I srednje obrazovanje, 2015. [Agency for pre-primary, primary and secondary education, 2015]. ZJNPP za matematičko područje definisana na ishodima učenja [Common Core Curriculum for Mathematics defined on learning outcomes]. https://aposo.gov.ba/sadrzaj/ uploads/ZJNPP-matemati%C4%8Dko-podru%C4%8DjeBOSANSKI.pdf.

2. Agencija za predškolsko, osnovno i srednje obrazovanje obrazovanje, 2019. [Agency for pre-primary, primary and secondary education, 2019]. PISA 2018: izvješće za Bosnu i Hercegovinu [PISA 2018 report for Bosnia and Herzegovina]. https://mo.ks.gov.ba/sites/mo.ks.gov.ba/files/pisa_-_dokument.pdf.

3. OECD., 2016. PISA 2015 Results (Volume I): Excellence and Equity in Education, PISA, OECD Publishing, Paris, Available from: doi. org/10.1787/9789264266490-en.

4. OECD., 2019. PISA 2018 results (Volume II): Where All Students Can Succeed, PISA, OECD Publishing, Paris, Available from: doi.org/10.1787/b5fd1b8f-en.

REFERENCES

ABDULLAH, N., HALIM, L. & ZAKARIA, E., 2014. VStops: A Thinking Strategy and Visual Representation Approach in Mathematical Word Problem Solving toward Enhancing STEM Literacy. Eurasia Journal of Mathematics, Science & Technology Education, 10(3), 165 – 174. https://doi.org/10.12973/eurasia.2014.1073a.

AJAI, J. T., & IMOKO, I. I., 2015. Gender differences in mathematics achievement and retention scores: A case of problem-based learning method. International Journal of Research in Education and Science (IJRES), 1(1), 45 – 50. https://www.ijres.net/index.php/ijres/article/ view/16.

AVDISPAHIĆ, M., 2018. Uvođenje STEM pristupa u osnovno I srednje obrazovanje u Bosni I Hercegovini [Introduction of STEM approach in primary and secondary education in Bosnia and Herzegovina]. Seminar for teachers and professors of mathematics in the Federation of \(B \& H\), Association of Mathematicians “Algorithm”, Mostar, 11 – 12 January 2018.

AVDISPAHIĆ, M., SMAJLOVIĆ, L. & MILLER, L., 2018. Nacrt operativnog nastavnog plana I programa (ONPP) za STEM kompetencije zasnovanog na ZJNPP definiranoj na ishodima učenja[Draft of Operational Teaching Curriculum for STEM competencies based on the CCC defined on learning outcomes- Mathematics], Save the Children International, Sarajevo. http://enablebih.org/doc_PPDM_STEM/STEM/ HRV/ONPP%20HRV/HRV-ONPP-MAT.pdf.

BAKIRCI, H. & KARIŞAN, D., 2018. Investigating the Preservice Primary School, Mathematics and Science Teachers’ STEM Awareness. Journal of Education and Training Studies, 6(1), 32 – 42. Available from: doi. org/10.11114/jets.v6i1.2807.

BABAROVIĆ, T., PALE, P. & BURUŠIĆ, J., 2018. Effects of STEM Intervention Program on Student Attitudes, Interests and Motivation in Primary School: Applicability of Propensity Score Matching Technique. Društvena Istraživanja [Journal for General Social Issues]. 23(2), 583 – 604. Available from: doi.org/10.5559/di.27.4.01.

BASISTA, B. & MATHEWS, S., 2002. Integrated science and mathematics professional development programs . School Science and Mathematics, 102(7), 359 – 370. Available from: doi.org/10.1111/j.1949-8594.2002. tb18219.x.

BENEK, I. & AKCAY, B., 2019. Development of STEM attitude scale for secondary school students: Validity and reliability study. International Journal of Education in Mathematics, Science and Technology (IJEMST), 7(1), 32 – 52. https://www.ijemst.net/index.php/ijemst/article/view/567.

BERLIN, D. F. & WHITE, A. L., 1994. The Berlin-White integrated science and mathematics model. School Science and Mathematics, 94(1), 2 – 4. Available from: doi.org/10.1111/j.1949-8594.1994.tb12280.x.

BIRGIN O., BOZKURT E., GÜREL R. & DURU A., 2015, The Effect of Computer-Assisted Instruction on 7th Grade Students’ Achievement and Attitudes toward Mathematics: The Case of the Topic “Vertical Circular Cylinder”. Croatian Journal of Education, 17(3), 783 – 813. Available from: doi:10.15516/cje.v17i3.1075.

BLAŽEV, M., KARABEGOVIĆ, M., BURUŠIĆ, J. & SELIMBEGOVIĆ, L., 2017. Predicting gender - STEM stereotyped beliefs among boys and girls from prior school achievement and interest in STEM school subjects, Social Psychology of Education, 20, 831 – 847. Available from: doi.org/10.1007/s11218-017-9397-7.

BOSSÉ, M. J., LEE, T. D., SWINSON, M. & FAULCONER, J., 2010. The NCTM process standards and the five Es of science: Connecting math and science. School science and mathematics, 110(5), 262 – 276. Available from: doi.org/10.1111/j.1949-8594.2010.00033.x.

BOTANA, F., HOHENWARTER, M., JANIČIĆ P, KOVÁCS, Z., PETROVIĆ, I., RECIO, T. & WEITZHOFER S., 2015. Automated Theorem Proving in GeoGebra: Current Achievements. J Autom Reasoning, 55, 39 – 59. Available from: doi.org/10.1007/s10817-0159326-4.

BUYRUK, B. & KORKMAZ, Ö., 2016. Teacher Candidates’ STEM Awareness Levels. Participatory Educational Research (PER), Special Issue (III), 272 – 279. http://www.perjournal.com/archieve/spi_16_3/ per_16_spi_3_28.pdf.

CLEMENTS, D. H., SARAMA, J., SPITLER, M. E., LANGE, A. A. & WOLFE, C. B., 2011. Mathematics learned by young children in an intervention based on learning trajectories: A large-scale cluster randomized trial. Journal for Research in Mathematics Education, 42(2), 127–166. Available from: doi.org/10.5951/jresematheduc.42.2.0127.

DUNCAN, G. J., DOWSETT, C. J., CLAESSENS, A., MAGNUSON, K., HUSTON, A. C., KLEBANOV, P., PAGANI, L. S., FEINSTEIN, L., ENGEL, M., BROOKS-GUNN, J., SEXTON, H., DUCKWORTH, K. & JAPEL, C., 2007. School readiness and later achievement. Developmental Psychology, 43(6), 1428 – 1446. Available from: doi.org/10.1037/00121649.43.6.1428.

FURNER, J. & KUMAR, D., 2007. The mathematics and science integration argument: a stand for teacher education. Eurasia Journal of Mathematics, Science & Technology, 3(3), 185 – 189. Available from: doi.org/10.12973/ejmste/75397.

HAN, S., ROSLI, R., CAPRARO, M. M. & CAPRARO, R. M., 2016. The effect of science, technology, engineering and mathematics (STEM) project based learning (PBL) on students’ achievement in four mathematicstopics. Journal of Turkish Science Education, 13, 3 – 29. Available from: doi.org/10.12973/tused.10168a.

HALPERN, D. F., BENBOW, C. P., GEARY, D. C., GUR, R. C., HYDE, J. S. & GERNSBACHER, M. A., 2007.The science of sex differences in science and mathematics. Psychological Science in the Public Interest, 8(1), 1 – 51. Available from: doi.org/10.1111%2Fj.15291006.2007.00032.x.

HEDGES, L. V. & NOWELL, A., 1995. Sex differences in mental test scores, variability, and numbers of high-scoring individuals. Science, 269(5220), \(41-45\). https://psycnet.apa.org/doi/10.1126/science.7604277.

HYDE, J. S., LINDBERG, S. M., LINN, M. C., ELLIS, A. B. & WILLIAMS, C. C., 2008. Gender similarities characterize math performance. Science, 321(5888), 494 – 495. http://dericbownds.net/uploaded_images/hyde. pdf.

IŞIKSAL, M. & AŞKAR, P., 2005. The effects of spread sheet and dynamic geometry software on the achievement and self-efficacy of \(7^{\text {th }}\) grade students. Educational Research, 47(3), 333 –– 350. Available from: doi. org/10.1080/00131880500287815.

JUDSON, E., 2014. Effects of transferring to STEM-focused charter and magnet schools on student achievement. The Journal of Educational Research, 107(4), 255 – 266. Available from: doi.org/10.1080/0022067 1.2013.823367.

KIM, HY., 2013. Statistical notes for clinical researchers: assessing normal distribution (2) usingskewness and kurtosis. Open lecture on statistics (NA), \(52-54\). Available from: doi.org/10.5395/rde.2013.38.1.52.

KITCHEN C. M., 2009. Nonparametric vs parametric tests of location in biomedical research. American journal of ophthalmology, 147(4), 571 – 572. Available from: doi.org/10.1016/j.ajo.2008.06.031.

KURT M. & BENZER S., 2020. An Investigation on the Effect of STEM Practices on Sixth Grade Students’ Academic Achievement, Problem Solving Skills, and Attitudes towards STEM. Journal of Science Learning, 3(2), 79 – 88. Available from: doi.org/10.17509/jsl.v3i2.21419.

LEECH, N. L. & ONWUEGBUZIE, A.J., 2002. A call for greater use of nonparametic Stataistics. Paper presented at the annual meeting of the Mid – South Educational Research Association, Chattanooga,TN, 6 – 8 November 2002. https://files.eric.ed.gov/fulltext/ED471346.pdf.

LIBURD, K. K. D. & JEN, H.-Y., 2021. Investigating the Effectiveness of Using a Technological Approach on Students’ Achievement in Mathematics–Case Study of a High School in a Caribbean Country. Sustainability, 13(10), 5586. Available from: doi.org/10.3390/ su13105586.

LINDBERG, S. M., HYDE, J. S., PETERSEN, J. L. & LINN, M. C., 2010. New trends in gender and mathematics performance: a meta-analysis. Psychological bulletin, 136(6), 1123. https://www.ncbi.nlm.nih.gov/ pmc/articles/PMC3057475/.

MCBRIDE, J. W. & SILVERMAN, F. L., 1991. Integrating elementary/ middle school science and mathematics. School Science and Mathematics, 91(7), 285 – 292. Available from: doi.org/10.1111/j.1949-8594.1991. tb12102.x.

MISHRA, P., PANDEY, C. M., SINGH, U., GUPTA, A., SAHU, C. & KESHRI, A., 2019. Descriptive statistics and normality tests for statistical data. Annals of cardiac anaesthesia, 22(1), 67–72. Available from: doi. org/10.4103/aca.ACA_157_18.

OLIVAREZ, N., 2012. The impact of a STEM program on academic achievement of eighth grade students in a South Texas middle school. Doctoral Thesis. Texas A and M University. https:// tamuccir.tdl.org/bitstream/handle/1969.6/417/Olivarez_dissertation. pdf?sequence=1&isAllowed=y.

ROSENZWEIG, E. Q. &WIGFIELD, A., 2016. STEM motivation interventions for adolescents: A promising start, but further to go. Educational Psychologist, 51(2), 146 – 163. Available from: doi.org/10. 1080/00461520.2016.1154792.

SIREGAR, N. C., ROSLI, R., MAAT, S. M. & CAPRARO, M. M., 2019. The Effect of Science, Technology, Engineering and Mathematics (STEM) Program on Students’ Achievement in Mathematics: A Meta-Analysis. International Electronic Journal of Mathematics Education, 1(1), 1 – 12. Available from: doi.org/10.29333/iejme/5885.

SOYLU, S., 2016. STEM education in early childhood in Turkey. Journal of educational and instructional studies in the world, 6(1), 38 – 47. https://arastirmax.com/en/system/files/dergiler/116392/makaleler/6/1/ arastirmax-stem-education early-childhood-turkey.pdf.

STINSON, K., SHEATS HARKNESS, S., MEYER, H. & STALLWORTH, J., 2009. Mathematics and science integration: models and characterizations. School Science and Mathematics, 109(3), 153 – 161.

TANK, K. M., 2014. Examining the effects of integrated science, engineering, and nonfiction literature on student learning in elementary classrooms. Doctoral Dissertation. University of Minnesota, Mankato. https://conservancy.umn.edu/handle/11299/165090.

ÇAYCI, B. & TABARU, Ö. G., 2019. Effect of Stem Based Activities Conducted in Science Classes on Various Variables. Asian Journal of Education and Training, 5(1), 260 – 268. Available from: doi.org/10.20448/journal.522.2019.51.260.268.

ZENGIN, Y., FURKAN, Z. & KUTLUCA, T., 2012. The effect of dynamic mathematics software geogebra on student achievement in teaching of trigonometry. Procedia - Social and Behavioral Sciences, 31, 183 – 187. Available from: doi.org/10.1016/j.sbspro.2011.12.038.

YARATAN, H. & KASAPOĞLU, L., 2012. Eight grade students’ attitude, anxiety, and achievement pertaining to mathematics lessons. Procedia - Social and Behavioral Sciences, 46, 162 – 171. Available from: doi.:10.1016/j.sbspro.2012.05.087.

YILDIRIM, B., &ALTUN, Y., 2015. Examination of the effects of STEM education and engineering applications on science course laboratory course. El-Cezeri Journal of Science and Engineering, 2(2), 28 – 40. https://dergipark.org.tr/en/download/article-file/56981.