Nº 2 - abril 2011
Ines Elezović , Sociology of Education, Educational assessments, Methodology, National Centre for External Evaluation of Education, ines.elezovic@ncvvo.hr
Jasminka Buljan Culej, Education, Educational assessments, Standard setting, Natural Science, National Centre for External Evaluation of Education, jbculej@ncvvo.hr
Abstract: Project State Matura is one of the most comprehensive educational changes that have been undertaken in Croatian educational system for decades. It includes all main actors in high education: Ministry of Science, Education and Sports, high schools, universities, national centers and agencies. One example of positive interaction between State Matura implementation and changes in higher education institutions’ enrollment is setting up the National Informatical System for Applying to Universities (NISAU), as one of most interesting and technically advanced features. During school year 2009/2010, students were obliged to take State Matura exams at the end of their secondary education (high school) as a passage to higher education institutions: polytechnics, higher professional schools, faculties and art academies (altogether called universities in this article). Upon entering the State Matura process students registered with personal electronical codes and signatures which they acquired from school coordinator and can monitor their progress up until they are signed in for some of Croatian tertiary education institutions. In this article this nine-step procedure, from making the decision of becoming a student to actually becoming one, is described. Taking into consideration that approximately 35 000 students in one school year enter the system and every one of them take at least three exams, obligatory assessment in Croatian language, Mathematics and foreign language, it was challenging work to secure the stability of this process for the first time. An additional advantage for the students during this computerization of enrollment process is the technical possibility to access their scanned and graded exams when the process of scoring is finished. They can also monitor their status on the ranking lists and (re)choose the programme to which they wish to enroll to. In the same time, described method of tracking student information proved to be beneficial for institutions organizing the State Matura because of the systems’ high speed of tracing and displaying changes. This example of good practice is considered to be one of the most advanced national student information system in Southern European education systems.
Keywords: higher education enrollment; large scale assessment; State Matura, informatical system
Introduction
Education, as a vital part of social life in every society, is undergoing through some substantial changes, in the past decade, regarding the development of informational and communicational technology (ICT) and changing labor market demands. Croatian educational system has experienced very propulsory period since establishing an independent national government in the early 1990’s. Nevertheless, real motivation and guidance toward shared European body of knowledge and skills, standardized quality assurance in education and various forms of internal and external evaluation of education came, in the last ten years, throughout common goal of becoming the part of European Union. The Ministry of Education, Science and Sports produced and started implementing some extremely important documents such as Development plan for educational system 2005-2010 (2005) and Strategy for developing national curricula for pre-school, elementary and high school education (2010) as well as establishing new institutions to straighten the process of introducing changes. As stated in these documents, four key priority areas in educational system were defined: improvement in quality and efficiency of education; encouraging lifelong professional development of teachers and other employees in education; development of strategies of management in educational system and ensuring its efficiency and its capability to foster social cohesion as well as economic growth and development. “Notion that in today’s very competitive economic and social context, education (and corresponding work productivity) can be increased by setting high national educational standards, leads towards standardization of educational achievements, emphasizing the need for their accomplishment on national scale and evaluation of objective results.” (The Strategy, 2005: 11).
Ongoing public debate about new school curricula for elementary and high schools, introducing State Matura exams at the end of the secondary education and implementation of Bologna process in the tertiary educational institutions, faculties and polytechnics, all came at the same time. Of course, all of these modifications had to be carefully and timely introduced because of their impact on everybody involved – students, their parents, teachers, principals and schools, faculties, and on the opposite end, employers. As concluded in many studies: “In some cases, such as with high schools exit tests, dissemination of information about the test requirements, and appropriate curricural match, may be needed years in advance of the testing event. In order to maintain appropriate opportunity to learn, curricural changes need to be kept synchronized with tests and standards.” (Yen et. al., 2002: 136). To accomplish all this, a new network of institutions and responsibilities was established in only a few years, so that this comprehensive vertical reform of education can be carried out successfully.
This article describes how in this new proactive science and education environment all main actors in high education (focusing particularly on connections between secondary and tertiary educational institutions) met to fulfill one goal: implement changes in high school education in a way that State Matura exams can be administered to all students in optimal and standardized manner that will not only serve the purpose of finishing secondary level education, but also to enroll to one of the institutions of tertiary education through organized and secured procedure. Following this path, experimental Project State Matura was introduced in school year 2008/2009 to all high schools. The next school year, 2009/2010, State Matura exams were obligatory and obtained results were used to rank students at Croatian universities. State Matura exams is uniformed process, meaning that all students have to take compulsory exams in Mathematics, Croatian language and one of the foreign languages at basic or advanced level. There is some difference in procedure regarding the students of minority groups. Students attending classes in a language and script of national minorities are obliged, in addition to the Croatian language exam, to take exam in the national minority language in which they are educated. They choose either Mathematics or a foreign language as third exam. All students can also choose as many optional subjects as they like according to the propositions of the programme they would like to apply to.
Technical support for the described procedure is based upon National Informatical System of Applying to Universities in Croatia (hereinafter NISAU) which was designed by the expert teams gathered around Faculty of Electrical Engineering and Computing.
Croatian education system
In Croatian educational system, children begin attending school at the age of 6 or 7. Elementary education in Croatia is compulsory and consists of eight years. From 1st through 4th grade, one teacher per class is teaching children all eight subjects. From 5th through 8th grades, different teachers teach different subjects, and some of new subjects are being introduced (such as Biology, Geography and Informatics). There are currently 848 elementary schools in Croatia, with total of 1 225 adjoined satellite schools and/or departments. The teacher/pupil ratio in regular education in the 2008/2009 school year was 1:11.9.[1]
After completion of elementary school, there follows the secondary education (high schools) that are not compulsory yet, but there are initiatives in that direction for some years now. Secondary schools are subdivided according to the programme into:
(1) Gymnasiums with four major educational streams: Natural science and mathematics gymnasiums (emphasize on informatics, mathematics and science), Foreign languages gymnasiums (with at least three foreign compulsory languages), Classical gymnasium (with additional Latin and Ancient Greek) and General gymnasiums (not a specific emphasize on any subject area).
(2) Vocational, technical or trade and related schools (technical, industrial, trade, craft, arts and others) are defined by the type of instructional programme they offer. Their educational programmes last from one to five years. Upon completion of this type of schools students have a secondary professional degree (for programmes from one to three years) and a basic professional degree (for programmes of four years).
(3) Art schools (music, dance, visual arts and others). At the end of 2008/2009 school year, 429 active upper secondary schools, that include various types of secondary education schooling facilities, were enlisted. There are currently around 90 gymnasiums and total of around 300 vocational schools in Croatia.
The secondary education is regarded finished after taking final graduation exams (State Matura exams) which are compulsory by the law since last school year. After that students can enroll into two basic kinds of higher education institutions: (1) polytechnic schools, so called “higher level of education” and (2) universities, “the highest level of education”. Professional study programmes provide students with an appropriate level of knowledge, skills and competences to work in applied professions, and to involve in any work process immediately after graduation. University study programmes allow students to work, for example, in science and higher education, private and public sectors. Graduates from university study programmes are also educated to apply and develop scientific and professional knowledge at the appropriate level[2]. Two major distinctions between the programmes available at Croatian higher education institutions, universities and polytechnics, used to be: final denomination and general duration of specific study progamme. These differences have been somewhat equaled by the on-going implementation of the Bologna process (since 2005). Newly introduced levels of expertise after completing one of the levels of tertiary education are: (1) Bachelor of Science and Bachelor of Arts; (2) Master of Science and Master of Arts and (3) Doctor of Science and Doctor of Arts.
State Matura at glance
Project State Matura is one of the most comprehensive educational changes that have been undertaken in Croatian educational system for decades. It includes all main actors in high education: Ministry of Science, Education and Sports, high schools, universities, polytechnics, different national centers and agencies. There are two primary goals that are going to be met by introducing these high stake exams into our system – upgrading the quality and standardization in high education (1) and simplifying the procedures of enrollment to universities (2). “Pursuant to the Act on Primary and Secondary Education (Official Gazette, 87/08), secondary education of pupils within grammar school educational programmes completes upon passing the state graduation exam. Secondary education of pupils within vocational and art educational programmes, lasting for at least four years, finishes upon the final paper creation and defense, organized and conducted by the school. Pupils of four-year vocational and art high schools may also take the state graduation exams if they wish so. The provisions on taking the state graduation exam refer to the pupils who enrolled in the first grade of grammar school or four-year vocational and art school in the school year of 2006/2007.”[3] This year the complete cycle from graduating in high schools through taking State Matura exams and enrolling to universities via NISAU system was conducted for the first time in Croatia. It seems that similar process or their advanced versions amazed experts in many countries, “After very little change in the university sector for several centuries, there has been something of a ‘mini-revolution’ in the last quarter of a century or so. The learner profile has changed socially, culturally and economically as new universities have emerged, enrolling an increasingly diverse population of students (…) and there has been spectacular change in the area of educational technology.” (Williams et al., 2009: 227)
In its essence the State Matura represents a group of exams, which are under the same circumstances and criteria conducted for all students at the same time. This process enables receiving comparative results for all students at the national level. The goal of this testing is to assess students’ acquired knowledge and ability of the subject curricula at this particular point in their education, when they leave high schools. Mandatory exams, taken within the State Matura, are those general subjects that students were taught during their elementary and high school education: Croatian language, Mathematics and one foreign language. There are few additional guidelines regarding foreign language as an obligatory exam. Students in classical gymnasiums can take an exam in Latin or Ancient Greek language as part of mandatory exams instead of, or in addition to, one of standard foreign languages. Students attending classes in a language and script of national minorities (Serbian, Hungarian, Italian or Czech) are obliged, in addition to taking Croatian language exam, to take an exam in the national minority language in which they are educated. In this case, their third exam can be either Mathematics or a foreign language exam. There are two levels of the subject matter testing in mandatory part – basic and advanced – and one unified level in second part. The second part of the State Matura exams is optional and is comprised of all remaining subjects that students had in school.
As in many other countries this type of large scale assessment testing requires standardized procedures such as administering the same exam materials at the same time and in the same way for all students in the Republic of Croatia. This is intended to produce the results that are mutually comparable, making the selection of students for higher education more objective and transparent. Overall goal is to encourage teachers and pupils to study harder in elementary and high schools and to generally increase the quality of school education on the national level.
NISAU at glance
National Informatical System of Applying to Universities (NISAU) is informatical system which enable students who are finishing high schools (vocational and gymnasiums) to continue their education path in more organized and secure way. The process includes few important general stages: defining student’s educational priorities (1), taking the appropriate State Matura exams (2) and applying to preferred tertiary educational institution – university or polytechnics (3). From the first step of the process students are using computers and Internet. In terms of computer science and data management, hardware and software background system is up to date with the most advanced systems, similar to those used for the same purpose in Sweden or Ireland. The following scheme is graphical representation of the system in whole:
Picture 1 – Position of National Informatical System of Applying to Universities (Central Applications Office, AZVO)
Three institutions at the top of the picture (Croatian Academic Network – CARNet, Ministry of Science, Education and Sports – MZOŠ and Agency for Vocational Education and Training – AZSO) along with the four institutions at the bottom (Central State Office for Governance – SDUU, National Centre for External Evaluation of Education – NCVVO, Institutions of higher education and Agency for Science and Higher Education – AZVO) are all mutual consignators, users and monitors of the entire NISAU system. Each institution has an obligation to obtain and/or monitor certain peace of student information and to execute their activities through institution’s personalized interface. As far as students are concerned, they are always using only one web page and one interface for all their activities, symbolically named “Become a student!” (www.postani-student.hr).
Picture 2 represents nine basic steps that each applicant must go through on their way to becoming a student. To log-in to the system student must get a user name and password, together with PIN and TAN numbers from the school administrator (step 2). The PIN number is used for additional protection of students’ personal data and TAN number is used for final confirmation of enrollment to the chosen university. For the first log-in student needs to use his user name, password and PIN to enter to his/her personal space in the system (step 3). There is a toolbar which contains: Home Page, My Data, My Choice, My Results and Settings options. Student is obligated to check his/hers personal data and to notify the school coordinator if something is wrong with their current data (step 4). At this point students need to make a choice whether to only register for State Matura exams or register for the State Matura exams plus enrolling to preferable study programmes (step 5).
Picture 2 – Nine steps from making a decision to becoming a student
After taking the State Matura exams, students can see their results with detail explanation about their grades and appointed scores (step 6). Additional advantage for the students is the possibilities to access their scanned and graded exams when the process of scoring is finished. For each applied programme, student can see his/her ranking and when one programme is chosen, student’s name is being automatically deleted from the lists of other nine choices (step 7). This procedure allows quicker and up-to date flow of information for other students to move up the ranking list and make an informed decision. Lists of students that have right to enroll are processed by the central computer for each higher education institution based on the results of the State Matura exams and, in some cases, additional testing at the institution in question.
To complete the process, it is necessary to confirm the intent of enrollment for one of the higher educational programmes that student gained right to enroll to (step 8). To do so, the students need to write personal and secret TAN number and press ‘confirm’. This must be done in the period of 48 hours after the student’s name appears within the planned enrollment quota and this deadline is clearly visible on ones’ personalized web page. When everything described is done, further information are accessible directly at the institution that student has enrolled to. Although this system was introduced for the first time this year, there were no significant student discontents during the admission process.
Discussion: two problem areas from two countries
There are many fields in which education is widening its research capability span, as well as implementing state-of-the-art technical solutions to improve information flow within the system. However, there are also many learning possibilities from more advanced educational systems. We will mention two examples of countries that are experiencing and dealing with similar methodological issues that Croatian education is faced with.
American Diploma Project (ADP), launched only seven years ago, had great impact on American education, especially making changes in transition from high schools to universities. The project is focusing on achieving five major objectives on the national level: national consensus about alignment for standards (1), equal graduation requirements (2), assessments aligned to college- and career-ready expectations (3) implementing P-20 Data System (4) and social accountability (5). As the American educational experts concluded in the project’s annual report Closing the expectations gap 2010: Fifth annual 50-state progress report on the alignment of high school policies with the demands of college and careers, after many years of experience in assessments, “There was a sizeable gap between what students knew leaving high school and the actual knowledge and skills they need to be successful in college and careers”. Very strong point was also made about state assessments at the high school level which must do a better job of measuring the real-world knowledge and skills that students need to be successful after high school. As one part of this project, Data Quality Campaign was launched with a clear notice of common interest of expanding access to student information so that in the future better understanding and use of state longitudinal data by policymakers, education administrators, teachers, parents, students and researchers can be expected. This fosters the idea of simultaneously coordinated changes throughout educational system and sets forth in the mission to make stronger connections between high school expectations and tertiary education institution’s demands. “The goal of college and career readiness for all high school graduates is no longer radical reform idea promulgated by a handful of states: It has emerged as the new norm throughout the nation.” (ADP, 2010: 23). Although most of the researchers agree on the idea of common high standards for educational assessments, there are some rising doubts about ‘too much leveling or standardizing’ in this matter. D’Agostino and Bonner (2009) investigated this alignment process at the Arizona University and came to a conclusion that “Endeavors such as Standards for Success and the American Diploma Project aim not only to enhance secondary standards but increase the rigor of college expectations so that both systems work in unison to prepare students for increased life and work demands. Consequently, those studies focused on idealized expectations.” (D’Agostino et al., 2009: 35). It could be interpreted that unwanted consequences of over idealized legislative regulation may as well lead to only superficial reforms and changes, nicely formulated phrases that are not manageable in the real educational environment. In one earlier study, also focused on researching knowledge and skills aligning with universities’ demands, four alignment criteria were analyzed – categorical concurrence, depth of knowledge, range of knowledge and balance of representation – on the basis of Mathematics and English assessments in 20 American states. “The results of the study indicate that state high school assessments and the knowledge and skills necessary for university readiness align in areas that might be characterized as more basic and do not align as well in areas requiring more sophisticated cognitive functioning.” (Brown et al., 2007: 152). Even after only few years of administering large scale national assessments, psychometric experts in National Centre for External Evaluation of Education reached the same conclusion about the lack of testing higher cognitive levels with fact-oriented tests and are training the expert groups to investigate more complex testing designs and forms of assessment questions and tasks.
Secondly, the Netherlands, that has a similar predominantly state governed and monitored education system with rich and gainful history of educational assessments and research, addressed recently another important dimension in higher education. Different lines of implementing quality assurance policies in secondary and tertiary education were undertaken in last two decades. In charge of large scale educational assessments are Netherlands’ National Ministry of Education and Science together with six institutes that provide support and technical assistance directly to schools (through 64 regional education service agencies). Although many European countries look up to this educational system, there is still capacity for further improvements of internal and external quality control management (Fredriks et al., 1994; Jeliazkova et al., 2002). One can generally argue that collision in multitude of changes across the European educational systems – new school curricula for elementary and high schools, introducing graduation exit exams at the end of the secondary education and implementation of Bologna process in tertiary educational institutions, faculties and polytechnics – is more than a single system can bear in just few years. Critical approach researchers in this country, have questioned the quality of transition methods to tertiary education programmes as well as ‘flattening’ higher education programmes area by the Bologna process. “The field of higher education, complex as it may seem, present policy-makers with a moderately structured problem – the debate is not about the necessity of good (accessible, flexible, ect.), higher education, nor even about the feasibility of this goal, but about the means and the costs to achieve it.” (Jeliazkova, 2002: 443). Implementation of quality assurance policies, such as multidimensional systems of external and internal institutional and programme (self)evaluations, require more than well trained procedures of knowledge assessment or enrollment testing. As many country-case studies showed, responsible initiation of new elements in education requires joined effort of government bodies along with expert groups of educators and scientists during extensive time period so all the institutions have time to adapt.
Conclusion
When rethinking Croatian experience, on the school and faculty level there might had been some discontents with the timing and rushing pace of introducing educational changes. Nevertheless, state national bodies, responsible for coordinating these projects, reached high level of agreement and collaboration. Institutions such as National Centre for External Evaluation of Education and different agencies, together with the university community, have undertaken some national-wide projects of implementing educational changes. This is crucial for keeping pace with European secondary and tertiary educational systems in terms of educating competitive young professionals. It is not so surprising that the substance of knowledge requirements around the world is changing, but the readiness of the state in following those shifting educational needs must be fostered equally fast. “While the content of education is changing, the expectation is that the general ‘outcome level’, whatever that should be, should not change, or at least should not fall.” (Alberts, 2001: 354). Much effort and knowledge, combined with experience of other countries is required to ensure satisfactory level of quality when determining state policy on high risk assessment exams such as Project State Matura. Continuance in achievement assessments in education, as well as unbiased self-evaluation of one’s research methods, as Brown (2007) have concluded, can help second-generation state assessments achieve clearer goals and provide better information than the first first-generation.
Although a very important aspect in every day functioning, technical support such as student information systems of different kinds, here represent just one dimension of the whole process – and for students probably the most important one. After devoting so much intellectual strength and efforts in conducting large scale assessment, people tend to expect that informatical system, perceived just as some hardware asset in the shadow, must work flawless. This was the case in the Croatia during the first year of enrolling students leaving high schools to higher education institutions via NISAU system. Maintaining the high quality service is for sure one aspect that needs to be taken into account when planning future educational investments and development plans. Valuable lessons can also be learned by researching the use of Student Information Systems (SIS), widely used in American educational system at the level of individual schools or school districts. Some practical conclusions in this matter also came from the pioneer implementation of web-based testing. As research showed, the most frequent asked features of a new information system for students are: “demands on wider spectrum of information of the academic field, periodic updates of information and of functions at all.” (Charvatova et al., 2007: 244). These are all the needs that can be successfully met at the present state of knowledge and skills that educational systems and policy-makers possess today.
References
American Diploma Project. (2010). Closing the expectations gap 2010: Fifth annual 50-state progress report on the alignment of high school policies with the demands of college and careers. Washington, DC: Achieve, Inc., (Web: http://www.achieve.org/ClosingtheExpectationsGap2010)
ALBERTS, René V.J. (2001). Equating exams as a prerequisite for maintaining standards: experience with Dutch centralized secondary examinations. Assessment in Education, 8 (3), 353-367.
BROWN, Richard S., CONLEY, David, T. (2007). Comparing state high school assessments to standards for success in entry-level university courses. Educational Assessment, 12 (2), 137-160.
CHARVATOVA, D., VONDRUS, J. (2007). Users’ demands in students’ information system. Proceedings of World Academy of Science, Engineering and Technology, 26, 239-244.
D’AGOSTINO, Jerome V., BONNER, Sarah M. (2009). High school exit exam scores and university performance. Educational Assessment, 14, 25-37.
FREDERIKS, M., WESTERHEIJDEN, D., WEUSTHOF, J. (1994). Effects of Quality Assessment in Dutch Higher Education, European Journal of Education, 29, 2, 181-199.
JELIAZKOVA, M., WESTERHEIJDEN, D. (2002). Systemic adaptation to a changing environment: Towards a next generation of quality assurance models. Higher Education, 44, 433-448.
WILLIAMS, Jeremy B., WONG, A. (2009). The efficacy of final examinations: A comparative study of closed–book, invigilated exams and open–book, open–web exams. British Journal of Educational Technology, 40 (2), 227-236.
YEN, Wendy M., HENDERSON, Dianne L. (2002). Professional standards related to using large-scale state assessments in decision for individual students. Measurement and Evaluation in Counseling and Development, 35, 132-143.
Državna matura 2009./2010. i prijave za upis na studijske programe. State matura exams 2009/2010 and enrolling to tertiary study programmes. (2009). Zagreb: Ministry of Education, Science and Sports, Agency for Science and Higher Education and National Centre for External Evaluation of Education, (Web: http://public.mzos.hr/lgs.axd?t=16&id=16058)
Nacionalni informacijski sustav prijava na visoka učilišta – korisničke upute. National Informatical System of Applying to Universities in Croatia – Users Guide. (2010). Zagreb: Ministry of Education, Science and Sports (Web: http://public.mzos.hr/fgs.axd?id=16197)
Plan razvoja sustava odgoja i obrazovanja 2005. – 2010. Development plan for educational system 2005-2010. (2005). Zagreb: Ministry of Education, Science and Sports (Web: http://public.mzos.hr/fgs.axd?id=14193)
Strategija za izradbu i razvoj nacionalnoga kurikuluma za predškolski odgoj, opće obavezno i srednjoškolsko obrazovanje. Strategy for developing national curricula for pre-school, elementary and high school education. (2010). Zagreb: Ministry of Education, Science and Sports (Web: http://public.mzos.hr/lgs.axd?t=16&id=14190)
[1]Central Bureau of Statistics, Basic schools: End of 2008/2009 school year and beginning of 2009/2010 school year (Web: http://www.dzs.hr/Hrv_Eng/publication/2010/08-01-02_01_2010.htm)
[2]For more detail information, please visit Croatian Ministry of Science, Education and Sports web-page (www.mzos.hr)
[3]For more detail information, see http://public.mzos.hr/Default.aspx?art=8750&sec=2504
Autores: Ines Elezović e Jasminka Buljan