The Association of Intended and Attained Curriculum in Science with Program for International Students Assessment

International examination results have already influenced many countries to make radical reforms in education system. According to these results countries have been categorized as high, middle and low achievement in education. Turkey has also taken these results into consideration quite seriously and started to investigate to what extent there are similarities among the nationwide examination questions and the international examination questions. This paper presents the findings obtained from the analysis of intended and attained curriculum in Turkish science context. Secondly, the paper analyses Level Determination Examination science questions according to Piaget cognitive development theory. Finally, the paper discusses the differences and similarities between the science questions in Level Determination Examination and Program for International Students Assessment. The results reveal that most Turkish students could not attain learning outcomes in intended curriculum. Level Determination Examination science questions are usually compatible with formal operational abilities of Piaget’s theory of cognitive development and they are highly similar with the questions of Program for International Students Assessment.


Introduction
Education systems in each country all over the world but especially in the developing countries have been shaped mostly by international perspectives and global economic trends.Program for International Students Assessment (PISA) is one of the most important of international comparative assessments which compare the science, mathematics and reading skills of the students who completed their basic education in different countries.The results of these examinations provide some clues about the strengths and weaknesses of the countries' education systems.Moreover, these international assessments contribute to the education reforms of the countries (Edwards, 2010).Turkey does not have a good ranking among the countries of organisation for Economic Co-operation and Development in PISA.Science achievement of 15-year-old Turkish students is low in their age group in international assessments.Table 1 summarizes science achievement of Turkey in these assessments.Turkey has undergone radical changes in education system to change this negative situation to positive.
Curriculum which aims at setting up a science and technology literate society has been developed and implemented since 2006.The new science curriculum aims at fostering the students' high level knowledge and skills, and promoting positive attitudes and values (Ministry of National Education [MNE], 2006).After a few years of implementation of the new curriculum, the government has announced a new national examination called Level Determination Examination (LDE).The children who complete their secondary education sit for this examination.The LDE scores of the students are used for their high school admission.The families spend a lot of money on private courses and tutors for their children to get prepared for this examination.Besides the school courses, the schools and the teachers implement extra classes to promote student achievement in this examination.
Large scale examinations have had important effects on education systems not only in Turkey but also in most European, Asian, and African countries and the USA (Akınbobolo & Afolobi, 2010;Amadola, Maiyo, & Amingo, 2009;Stanley, 2012;Tienken, 2011).The students begin to take large scale examinations while they are still at the primary education level.
A big debate has also started concerning the effects of central examinations on teaching and learning process.
According to Bishop (1998) and Wößmann (2005), central examinations have a positive effect on students' performance.Moreover, these examinations provide useful benefits for the teachers and schools to renew themselves and increase the quality of education they give.Contrary to this belief, some researchers (Bjork & Tsuneyoshi, 2005;Lisle, Smith, & Jules, 2005;Youell, 2005) argue that central examinations may have negative effects on teachers' teaching methods and students' learning.Teachers give up student-centred teaching and active learning techniques in order to make their students answer more questions correctly and teach them test techniques because teachers' achievements are directly related to how many questions their students answer correctly at the central examinations.However, many teachers know that without understanding real meaning of the concepts, students are able to answer the test questions by memorizing the certain rules.They really do not use critical thinking skills and they may not do things in order to solve these questions either.Large scale examinations are one of the obstacles for their implementation in the class because teachers and students spend most of their time and energy to get prepared for central examinations (Kasanen & Raty, 2008).In Turkey many curricula have been accepted unsuccessful with the same or similar reasons (Ayas, Çepni, & Akdeniz, 1993;Çepni & Çil, 2009).The problem is "how we convince them about the new curriculum which is mostly consistent with the nature of central examinations." The purpose of this study consists of three parts: (1) to compare intended and attained curricula in Turkish science context; (2) to analyze the LDE science questions using Piaget's cognitive development theory; and (3) to examine the similarities and differences between the science questions of LDE and PISA.The guiding research questions were as follows: 1) Are attained curriculum and intended curriculum compatible with each other in science education in Turkey?
2) Do the LDE science questions align with formal operational stage abilities of Piaget's cognitive development theory?
3) What are the differences and similarities between LDE and PISA science questions?
The findings obtained from the study might be important for the curriculum developers and the teachers.The similarities and differences between the intended curriculum and attained curriculum can enable the countries to revise their own instructions.The teachers can revise their in-class teaching strategies, assessment tools, and more.Moreover, the context of this study is Turkey, yet still this study can be valuable for the other developing countries because the developing countries suffer from similar educational problems.For example, the achievement scores of the majority of the developing countries in international assessments are usually low (PISA, 2009).
The countries which struggle with the common educational problems might benefit from each other's experience to increase the quality of the external examinations and develop the performance of the students in international assessment.This paper can provide an understanding of what happens in different countries and an opportunity for comparisons between the countries.

Theoretical Framework
Tri-partite model of curriculum conceptualized by International Association for the Evaluation of Educational Achievement (IEA), Turkish science curriculum, LDE and Piaget's cognitive development theory are the main components of the conceptual framework of this study.

Intended Curriculum
It is the educational level.

Science Curriculum in Turkey
The Turkish education system is highly bureaucratized.All curricula are designed by a committee of experts at the Ministry of National Education.These curricula are implemented across the nation.In Turkey, the schooling consists of four main components: Basic Education (age 6-9), Secondary Education (age 10-13), High School (age 14-17) and Higher Education.Students who complete their basic education continue their secondary education depending on their choices.After secondary school, the scores of large scale examination are the indicators of in which schools students are going to study.Science curriculum of Turkish secondary education is composed of four learning areas: content areas, scientific process skills (SPS), science technology society and environment (STSE) and attitudes and values (AV).The content areas are related to the science concepts, principles, theories and/or laws which the students have to understand.SPS are tools scientists use to identify problems, solve problems, and answer questions about world.SPS include observing, measuring, predicting, experimenting, formulating models, interpreting data.STSE emphasize the teaching of scientific and technological developments in their cultural, economic, social and political contexts.AV is an affective domain.

Level Determination Examination
There are two types of schools at high school education level in Turkey.One of them is the schools which give academic education and the other one is the vocational training schools.Most vocational training schools are not popular between the students and their families.Due to the limited capacities of the high schools which give academic education, a competition among students who want to enter these schools have been continuing for years.In order to be accepted to these schools, the students sit for a national examination called LDE and they must get high scores from this examination.LDE consists of multiple choice questions in the areas of Turkish (23 questions), Maths (20 questions), Science (20 questions), and Foreign Languages (17 questions).Students are responsible for only the eighth grade instructional objectives of these lessons.The test takers must complete the examination in two hours.

Piaget's Cognitive Development Theory
Teachers must be aware of the students' cognitive development levels in order to decide the content of instruction and design effective learning environments (Hinde & Perry, 2007).In addition, teaching carried out in the classroom by the teachers must make contributions to the students' development of higher level thinking skills.The cognitive demands of assessments tools affect the students' minds (Endler & Bond, 2008;Karamustafaoğlu, Sevim, Karamustafaoğlu, & Çepni, 2003;Kuhn, 2008;Webb, 1980).Thus, it is beneficial to understand that which cognitive abilities are assessed by large scale examinations and in-class examinations such as pen and paper tests.Some aspects of Piaget's cognitive development theory are criticized.However, Piaget's theory is frequently used to assess and develop the student's cognitive skills in the 20 th century (Endler & Bond, 2008;Gray, 1978;Hinde & Perry, 2007).
Piaget divides cognitive development into four stages.The two of them, which are sensorimotor (0-2 years) and preoperational (2-7 years), befit the preschool stage.Children are at concrete operational stage from ages 7 to 11.The students aged between 11 and 12 are expected to go through formal operational stage (Shayer & Adhami, 2008).A lot of students aged between 11 and 12 may not be able to show their formal operational stage abilities (Endler & Bond, 2008;Ojose, 2008;Sutherland, 1999;Webb, 1980).The cognitive abilities expected from the students at concrete and formal operational stages are given in Table 2 (Çepni & Çil, 2009).

Classification
It is the ability to classify objects in an order according to size, shape, and any other characteristics.Similarities and differences between objects are determined with classification.Concept formation is developed with classification.

Ordered
Students may pair objects one to one or sort them in an order according to a series and organize the objects.

Conservation
Although the quantity and the shapes of the matter change in some situations, it is the understanding that the quantity will remain the same.

Hypothetical thinking
It is a thinking process which develops possible aspects to solve a problem encountered and provides them to be done in a specific order.It is expressed with general sentence structures like if …..and what if …. .

Proportional thinking
It is a cognitive skill used to compare and contrast the relationships between the variables.

Identification of variables
It is to understand the factors which are effective in an event.It is required to identify these factors as dependent, independent and fixed variables.

Probabilistic thinking
It is the ability of being able to think of every possible probability in all stages from the beginning of an event or a hypothesis to the result stage.

Combinational thinking
It is to take into account all the theoretical and experimental relationships in a systematic way.

Correlational thinking
It is to associate a changing object with another object.Two variables are needed for a correlation.

Research Method
Document analysis, one of the qualitative research methods, was used in the study because in this study Turkish science curriculum, science questions in LDE and PISA are examined and interpreted in order to elicit meaning, gain understanding, and develop empirical knowledge (Corbin & Strauss, 2008).

Research Instruments
The data of this study were obtained from four different documents: Moreover, PISA released item document was examined to compare the differences and similarities between LDE and PISA science questions.In this study, a national examination questions in Turkey and PISA questions were compared.There are a few international science assessment programs.But, PISA is the most prominent program which assesses the science achievement of the fifteen year old children all over the world.The researchers decided the documents which were going to be examined according to the needs of the research.Therefore, purposeful sampling, one of the qualitative research sampling types, was used in the study (Morse, 1991).

Data Analyses
Task analysis of the documents was carried out throughout the study.Task analysis for instructional design is "a process of analyzing and articulating the kind of learning that you expect the learners to know how to perform" (Jonassen, Tessmer, & Hannum, 1999, p. 3).This study tries to understand the following: Turkish science curriculum includes what the students should know and do with LDE and PISA science questions.Thus, the study is compatible with the nature of task analysis.
To reveal the intended curriculum, the analysis of the standards described in science curriculum was carried out with regard to SPS, STSE, AV and cognitive learning areas.Curriculum explicitly demonstrated the standards for science course and which learning areas of SPS, STSE, and AV these standards were related to.For example, a standard is stated as follows in the curriculum: "Students can give examples of chemical change of a transformation of a matter into a different matter (SPS-8)."SPS 8 makes an inference which is a science process skill.However, some standards are addressed with expressions such as define, know, explain and these are treated as cognitive standards.The authors counted all the SPS, STSE, AV and cognitive standards in the 8 th grade science curriculum one by one.Some standards were intended for more than one learning area.For example, "the variable in physical changes emphasizes that matter does not change identity (SPS-6, 8, 9; AV-2)".
In this condition, the standard was treated both as SPS and AV and added to the total standard number.
In order to reveal attained curriculum, firstly the compatibility of LDE questions with the instructional goals of science curriculum were reviewed.Thus, the researchers asked three experts in the field which objectives of science curriculum the LDE questions aimed at evaluating.After the learning area which each question focused on was decided, the percentages of SPS, STSE, AV and cognitive learning areas were calculated.After the researchers made sure that curriculum and national examination included similar learning goals, LDE science achievement of the students was investigated.The data obtained from the result reports were presented in Table 5.
The same experts were asked to decide which skills of Piaget's cognitive development theory each LDE science questions tested in order to find an answer to the second research question.Some questions could be answered only by remembering basic science concepts, principles, and theories and so on.Such questions were accepted at memorizing level.The percentile distribution of questions according to cognitive abilities was shown in the Table6.
The above-mentioned experts analysed the PISA released item documents as the last issue of the purpose of this study.Firstly, the SPSS, STSE, AV, and cognitive learning areas of PISA science questions like the LDE questions were asked to the experts.Then, they decided about the cognitive ability required to answer each examination question.
All the experts were science teachers and at the same time they continued their studies in science education PhD.program.They were trained about science curriculum and cognitive development theories.First of all, reviewers analysed the questions individually.Then, their analyses were compared under the supervision of a professor from science education area.It was found that analyses were usually (90%) compatible with each other.Disagreements were resolved by discussion and they reached 98% consensus.

Findings
First, the findings obtained about the compatibility between the intended and attained curriculum in science education in Turkey were presented in this section.Next, the findings obtained from the analysis of LDE questions according to Piaget's theory of cognitive development were presented.The similarities and differences between LDE questions and PISA questions were covered in the discussion section and the questions were given in the appendix as comparatively.

Findings Related to Compatibility of Intended and Attained Curriculum in Science Education in Turkey
We analysed the standards of science curriculum in learning areas (SPS, STSE, AV and cognitive) to understand the intended curriculum in Turkey.The average Turkey score was 5.25 out of 20 science questions in LDE held in 2009.There was a slight increase in the average test scores in the LDE carried out in the ensuing years.The average science test score increased to 6.76 in 2010 and 7.13 in 2011.

Findings Related to Analysis of LDE Questions Regarding Piaget Cognitive Abilities
The distribution of twenty questions belonging to the eighth grade regarding cognitive ability was given in Table 6.
Table 6.According to Table 6, in each examination, less than half of the questions are intended for the assessment of concrete operational stage.The classification is the most focused cognitive ability in concrete operational stage because nearly one-fourth of the questions require using classification ability in all the examinations.In each examination, more than half of the questions assess formal operational cognitive abilities.One-fourth of the questions and more than that assess students' correlational thinking ability.

Discussion and Conclusion
Discussion and results, parallel to the research questions, were presented under three headings in this section.

Discussion and Conclusion about Compatibility of Intended and Attained Curriculum in Science Education in Turkey
When the eighth grade science curriculum is analysed, it is discovered that standards are mostly SPS.One of the basic goals of science education is that students can solve the problems they come across with scientific methods.Thus, it is quite satisfactory that SPS learning area is dominant in the curriculum.5% of STSE objectives exist in some chapters of the eighth grade science curriculum and this ratio goes up to 20 % with some chapters.Science curriculum frequently emphasises technology.It suggests that students should build bridges between each science topic taught and the technological developments in the field.Today developing countries suffer from poverty and in order to reduce this pressure, these countries should integrate science and technology in their curriculum.So, in parallel with this opinion, United Nations Educational, Scientific and Cultural Organization supports the developing countries to develop innovative programs which integrate both science and technology (Stears, Mckay, & Bentham, 2011).Less than 10% of the standards in five out of eight units in science curriculum take part in AV learning area.The promotion of positive attitudes towards science has always been emphasized in the reform documents which shape education in the world (American Association for the Advancement of Science [AAAS], 1993; National Research Council [NRC], 1996).The vision of Turkish basic education science curriculum is to make all children science and technology literate.In order to achieve this vision, it is emphasized in curriculum that AV learning area is very important (NME, 2006).However, very few standards in the curriculum are related to this learning area.Therefore, it can be stated that students like and appreciate science and make what they have learned in science courses a lifestyle, they are ignored.It is suggested that Turkish science curriculum standards should be revised to include affective learning more.The researchers in other countries can conduct studies on the intended curriculum of their own country which can give opportunities for the comparison of international curriculum.
More than half of the LDE science questions assess the students' scientific process skills.Very few of the LDE science questions aim at assessing STSE and AV learning areas.It can be stated that LDE science questions are parallel with the instructional goals of the science curriculum.This condition can affect the implementation of science curriculum in the class positively since it is known that national and/or international examinations such as exit and entrance examinations which affect the future of the students lead the teaching strategies and activities of the teachers (Locke, Vuuiamy, Webb, & Hill, 2005;Riffert, 2005;Webb, Vulliamy, Hamalainen, Sarj, Kimonen, & Nevalainen, 2004).Since LDE science questions have the potential to assess the instructional goals of science curriculum, student's science achievement in LDE can be used to make decisions about the attained curriculum.However, the Turkish children's LDE science achievement is not very satisfactory.Most Turkish students can only attain less than half of the skills and abilities aimed by the curriculum.However, in addition to this, the students' LDE science achievement increased from 2009 to 2011.There are some reasons for this condition: (1) the teachers' curriculum experience enhanced in time and accordingly the teachers could understand and implement science curriculum better in their classes; (2) the teachers' understood the LDE structure and questions better.A research focusing on the intended, implemented and attained curriculum should be conducted to find the reasons for this condition.

Discussion and Conclusion about LDE Science Questions' Alignments with Formal Operational Stage Abilities of Piaget's Cognitive Development Theory
More than 60 % of the national examination science questions are answered with cognitive abilities of formal operational stage.These results are similar to the findings of many studies in the literature (Azar, 2005;Çepni, Özsevgeç, & Gökdere, 2003;Edwards, 2010;Karamustafaoğlu et al., 2003).The questions of large scale examinations affect the students' mind because the students get prepared for the examination by studying the questions of the previous years and their similar.While low order questions lead the students to memorization, high order questions can support the students to analyse, synthesise, and transfer knowledge (Brualdi, 1998;Liang & Yuan, 2008;Özsevgeç & Çepni, 2006;Vendlinski et al., 2007).In this context it can be stated that LDE science questions have the potential to provide contribution to the students' cognitive development.Most of the formal operational stage questions evaluate correlational thinking cognitive ability.The hypothetical thinking and identification variables are secondarily focused on formal operational cognitive abilities.Naidoo and Ranjeeth (2007) discuss that these cognitive abilities are late formal stage abilities.In other words, these abilities come out as high level cognitive abilities.The students are ranked according to their LDE scores.According to this achievement rating, the students are accepted to the eligible high schools in Turkey.Due to all these reasons, it is not surprising that the questions which require higher level cognitive skills outweigh in this national examination.This situation may help the teachers focus on the implementations in their classrooms which will draw the students' attention from memorization and ensure conceptual learning.The teachers can prepare high level questions for their own assessments.Studies which analyse the questions in the assessment tools prepared by the teachers according to the Piaget's theory can be designed.

Discussion and Conclusion about Differences and Similarities between LDE and PISA Science Questions
There are questions in PISA which evaluate scientific method knowledge.The mutual and sophisticated interactions between science, technology, society and environment are the subjects of many questions in PISA.
In 2006 PISA searched students' attitudes towards science with surveys and tests with mixed questions.Finally, most of the PISA questions require the students to use their high level cognitive abilities.PISA science test assesses scientific literacy which is accepted to be one of the basic goals of science education in almost every country in the world.Therefore, the context of PISA science questions is related to daily life and /or based on technological issues.PISA questions require the students to solve the problems they encounter and take a decision by using the scientific methods.PISA is an examination which is developed for fifteen-year old students.So, it is not surprising that the students are expected to use formal operational stage skills in order to cope with the questions in PISA.
It can be stated that Turkish national examinations show some similarities with the PISA.This similarity is that SPS learning area and formal operational stage skills of Piaget's theory of cognitive development are emphasized in two documents.While AV learning objective of science education exists commonly in PISA, it is rather scarce in Turkish national examination (Question samples Appendix-1).While the science scores of Turkish students were 424 in PISA 2006, they reached to 454 in PISA 2009(PISA, 2009).Although Turkey's science achievement shows tendency to increase, its science average is still below the international average.The science curriculum analysed in this study was implemented in 2006.15-year-old children have been taking national examination since 2009.Both the science curriculum and the entrance examination are new.But, it is very well-known by the educational environments that education is a slow process and it takes time.We believe that when the curriculum and the national examination are understood and implemented better in the near future in Turkey, our students can improve their performances in international indicators.
There are certainly many factors which have an effect on the achievement of the countries in the international examinations.However, educators should understand the structure of these examinations and analyse which abilities are assessed by the examination questions.Moreover, they should promote their in-class teaching to teach these abilities so that their students can attain them and this will lead to answering the questions in the examinations.Therefore, they can help their students to get satisfying scores in international assessments.

Question
Teacher: Let's look at the contents of the rubbish bin and decide on the solution, and give examples about the solution.
Mehmet: Different wastes are mixed.We can put paper, glass, plastic and food in different baskets in order to use them by recycling.
Teacher: Which points should we give to Mehmet's ideas from points explained in the

Question
Tobacco is smoked in cigarettes, cigars and pipes.Research shows that tobacco related diseases kill nearly 13 500 people worldwide every day.It is predicted that, by 2020, tobacco-related diseases will cause 12% of all deaths globally.Tobacco smoke contains many harmful substances.The most damaging substances are tar, nicotine and carbon monoxide.Some people use nicotine patches to help them to give up smoking.The patches are put on skin and release nicotine into the blood.This helps to relieve cravings and withdrawal symptoms when people have stopped smoking.To study the effectiveness of nicotine patches, a group of 100 smokers who want to give up smoking is chosen randomly.The group is to be studied for six months.The effectiveness of the nicotine patches is to be measured by finding out how many people in the group have not resumed smoking by the end of the study.
Which one of the following is the best experimental design?
A. All the people in the group wear the patches.Which must be done about this research question in the 3 rd stage by the students who observe the experiment stages given below?Experiment stages: 1.One of the leaves of the green plant is covered with a black paper so that its getting light is prevented.
2. The black paper is removed a week later.A) The plant whose black paper on its leaf is removed waits in a lightened environment for a week B) The leaf whose black paper is removed is picked up and it is made to wait in a lightened environment.
C) Another leaf of this plant is covered with a black paper D) This plant is made to wait in a dark environment for a week.

Question
The stronger the wind blows, the faster the windmill blades rotate and the greater the electric power output is.However, there is not a direct relationship between wind speed and electric power in a real setting.Below are four working conditions of electricity generation in a real wind farm.
• The windmill blades start rotating when the wind speed reaches V1.
• The electric power output reaches a maximum (W) when the wind speed is V2.
• For safety reasons, the blades are prevented from rotating faster than they do when the wind speed is V2.
• The blades stop rotating when the wind speed reaches V3.
Which one of the following graphs best represents the relationship between the wind speed and electric power output under these working conditions?PISA 2006 correlational thinking question

B
. All wear patches except one person who tries to give up smoking without them.C.People choose whether or not they will use patches to help give up smoking.D. Half are randomly chosen to use patches and the other half do not use them.PISA 2006 SPS questionQuestionResearch question: Are some of the changes which occur in plants because of the effect of environment permanent?

Table 1 .
Turkish Students' Science Achievement in PISA

Table 2 .
Piaget's stage of cognitive development Turkish science curriculum, LDE science test booklet, LDE result reports and PISA released items document.Three different documents were examined to find an answer to the first research question.The science curriculum which has been used in Turkey since 2006 was analysed to understand the intended curriculum(MNE, 2006).Science curriculum explicitly describes the content and the standards.Therefore, it is a good tool to seek an answer to the first question of the research.LDE only included science curriculum of the eighth grade.Therefore, only the eighth grade science curriculum was examined in this study.To understand attained curricula, two types of documents, LDE science questions and LDE result reports, were examined.These documents can be obtained from the official website of Ministry of National Education.LDE is a national examination and more than one million students take LDE in Turkey.Therefore, if LDE questions are compatible with the learning objectives of science curriculum, the scores of this examination might be a good data source about attained curriculum.Thus, this paper presents the analysis results of LDE science questions from 2009 to 2011 because LDE was first administered in 2009, but this paper was written in 2012.When this paper was written, LDE was not administered in 2012.LDE science test booklet was examined to question the compatibility of LDE science questions with formal operational stage skills of Piaget's cognitive development theory.

Table 3 .
Percentile distribution of the science curriculum standards according to learning areas unit based for eighth grade Table3presents that cognitive and SPS learning areas are emphasized in the curriculum.The cognitive objectives are dominant especially in four chapters, which are Cell Division and Heredity, Force and Motion, Living Things and Their Relation to Energy, Electricity in Our Life.The ratio of SPS objectives change nearly from 25% to 50% in each chapter of science curriculum.20 % of the learning outcomes in three out of eight chapters want the students to comprehend the interaction between science, technology, society and environment.The ratio of AV objectives is below the 10% in five chapters.In the other chapters AV learning areas objectives are nearly 15%.First, LDE science questions were examined for science curriculum learning areas, which are SPS, STSE and AV, to bring out attained curriculum.The findings obtained from this analysis were presented in Table4.

Table 4 .
Percentile distribution of LDE science questions according to learning areas

Table 4
presents that more than half of the questions in all examinations from 2009 to 2012 evaluate SPS objectives.The cognitive learning area is the secondary dominant.Only 10% and/or 15% of the questions assess STSE objectives.There aren't any questions to assessing AV learning area in the LDE.The learning areas which LDE science questions tested are parallel with the learning areas of intended curriculum.Thus, we can use the LDE science test achievement scores to understand attained curriculum.Table5presents LDE science test achievement and these findings were obtained from more than one million students.