Are We Ready to Pilot a New Test Format for English Language Matura Exam ?

Much of the debate about school improvement and raising of educational standards in Albania revolves around the issue of providing external standard tests which cannot allow students to be discriminated but allowed to receive equal opportunity to be assessed professionally in varying areas of English language. Being part of the National Agency of Exams, and part of standard testing service staff, it is obvious that taking the new challenges that education system is facing in consideration would be indispensable. The questions discussed are: What is to be changed; Based on what; What new features should the State Matura Exams (the foreign languages exams) have in Albania; Are the teachers and students prepared for this? This research paper will be focused on answering such questions and giving a clearer view on what has happened and what will happen with Matura Exams in Albania, especially with English Language Test. The aim of this paper is to see whatever has been applied and will be applied in future in State Matura Exams (external national standard assessment) based on concrete evidence.


Introduction
How did State Matura Exams start in Albania?(The political aspect) State Matura represents not only an important element of the educational system reformation, but it has urged and laid down positively the reformation of some other links of the system, their harmonization to ensure quality and high standards as permanent objective of the education.
Which were the factors that urged and accelerated the reform of State Matura?
The results of the entrance examinations for universities lost the reliability, because in many cases they didn't correspond to the level of the acquired knowledge in high school.
The well known and unacceptable situations of letting out the secret and the test trade years ago were the top of the iceberg of the corruption size which had spread into some academic institutions, which had shaken their integrity.Such a corruptive climate favored the informal business of tutoring private unlicensed courses and which did not pay taxes, from the professors known for the preparation of the university entrance exams.
The internal assessment, without denying the honest and devoted work of the most of the Albanian teachers, did not respond to the results of the inspecting, and sometimes, it represented some conflict of interests between the teachers and their evaluation (grades).
State Matura didn't express directly the government's responsibility and commitment, the veracity of the evaluation of the results of our educational service, creating in this way an equalizing system of evaluation which didn't take in consideration the differences of the schools' levels and the differences between different regions of the country.
All above mentioned, and the lack of the transparency shook the confidence of the public and the tax-payers not only to the educational system, but to the whole system itself.
Right for this reason, the creation and the application of State Matura represented a major and big step to stop the crisis and to get out of it; it helped to regain the public's confidence to the system based in work and merits.
State Matura Exams represented a system of examinations (2 plus 2) which means two compulsory exams and two optional ones.They were performed in written form to avoid the subjectivism in evaluation as much as possible.
For the first time in the system of examination, were applied optional examinations, in the form of a group of subjects that allow each candidate to make his own choice to continue the university based on his wishes, vocations and the gained grades.
The external assessment was the core of the novelties and the reform of State Matura which represented an objective assessment was central and impartial which expelled the conflict of interests and corruptive tendencies.
Though State Matura in its gist was a system of examinations with external assessment, it didn't expel, but included the school evaluation which is an average synthesis of all the results during all the school years of the high school and this one forms 20% of the general evaluation, and its specific weigh could grow with the consolidation of the system.
State Matura 2006-2010 was enriched with "Merit-Preference" element for a better connection of the offer with the level of each student preparation, so to avoid the endless turns of registrations in preceding Matura Exams that let the doors open to the corruption.
The optional exams included five foreign languages (English, French, German, Spanish, and Italian).
To design tests for these languages, tests specialists have been based on National Curriculum for foreign languages and on the Framework of the European Languages.The priority of our government has been decisive to make English language an official language, so its weight in schools programs has been the same with that of compulsory subjects such as Albanian language and literature and Mathematics.In 2008-2009, the Ministry of Education in collaboration with the National Agency of Exams decided to change the format of the National Matura Exams.Open-ended questions would be included in the tests too.That was seen as a better way to really diagnose students' proficiency level in reading and writing; because the previous format resulted as not reliable because of a great number of students punished for cheating during the test-taking.So, this was a good way to reduce the possibility to cheat.The test contained 10 items of multiple-choice format (10 points) and 10 open-ended ones (30 points).There were 40 points in total, not 50 as in the previous Matura Exams.

What changed? A better test administration More reliable test results
The psychometric analysis showed that some open-ended in Reading Comprehension were not practiced at schools at the level they had to.So, teachers got their message: they had to treat such questions (critical thinking questions) more seriously and thoroughly.This issue had a profound effect on the training needs of in-service teachers.
The changes resulted necessary, but at the same time costly.NAE engaged 60 -100 teachers to be a part of test correction procedures; while in previous Matura Exams they were corrected automatically.
Tests specialists needed to be trained on construction of open-ended questions and their scoring.
From 2010-2013, except English language and French language which were the only ones to be tested on Matura Exams (optional exams), other foreign languages were included in National Matura Exams (optional exams), German, Italian, Spanish, Greek, and Turkish languages were included too.The table below shows some important aspects of National Matura Exams in foreign languages, in Albania.In State Matura 2011-2013, English language started to be treated in two tests:

National Exams (optional exams) during 2010-2013 Foreign Languages Exams
English Language as a first language (a test which had to be chosen by students that had English language as a first foreign language in their schools); Its format was kept the same as the previous one; 20 items 10 items of multiple-choice format -10 points 10 items of open-ended format -30 points 3 sections (reading comprehension / use of language / writing) Level B1-B2 (CEFR) English Language as e second language (a test which had to be chosen by students that had English language as a second foreign language in their schools) Its format was kept the same as the previous one; 20 items 10 items of multiple-choice format -10 points 10 items of open-ended format -30 points 3 sections (reading comprehension / use of language / writing) Level A2-B1 (CEFR) In State Matura 2014-2015, English language was set for the first time as a compulsory exam, the same as Albanian language and literature and Mathematics; at the same time it was an optional exam with another difficulty level.
Item bank was completely rebuilt.The agency asked collaboration by English Testing specialists, and it really had very good results.Tests were accepted by most of students and teachers very well.Tests results were better than the previous years ones.Being an assessment specialist in this agency for 8 years, and taking part in all procedures of exams preparation, production, administration, correction, etc, has helped me to have a continuous and clear mirror of problems related with all procedures above-mentioned.

English language as a compulsory exam
As far as this paper is focused with problems of assessment in English language, here are some issues related with teaching, learning and assessment of English language in Albania.
The first thing which is quite a problem is that examination reform has to be integrated with other reforms in curricula and teacher training as well.Curricula are well-integrated with assessment, but teacher training is not uniform all over the country.Well-trained teachers are concentrated in big cities, while in the other sub-urban areas; the training reform is nearly inexistent.This creates a gap to complete a regular circle: curricula -teacher training -students' preparation -assessment.This leads to a different students' preparation for Matura Exams and to different results in final exams.
The non-inclusion of other language skills (listening and speaking) in the final exam leaves an unclear mirror of reality about the level of achievements and performances of students and schools in foreign languages.We cannot have complete "scanning" results without having these important skills included in the tests of foreign languages.
A network of competent item writers, test designers, moderators, and examiners must be created to have a better presentation with foreign language tests in State Matura Exams.This has its price, but it must be done to have good results in this aspect.
Not enough work with reading comprehension is another issue that affects Matura Exams results.What happens?Where are the problems?
The students' results and psychometric analysis of the tests show that one of the weakest points in teaching and learning English language is being not equally focused on every aspect of reading comprehension.The aspects which should be treated more seriously by the teachers and students at school are:

Vocabulary
New words are seen by students as a great obstacle to comprehend a text.It is necessary for students to properly comprehend the words or the vocabulary of a written passage in order to be able to decode the message given in the text.

Ability to memorize
The students show features of the fact that they cannot recall all the information in the text.They need to hold the information in working memory long enough for the information to be more extensively processed, and often some of them lack it.

Students read too little
They have to be encouraged to read as much as possible from different types of texts.If they don't do so, this will always be an obstacle for students to comprehend well a variety of written texts and have good results in the test.

Dealing with scanning details items
It has been years since we introduced for the first time open-ended items in tests of foreign languages in State Matura Exams.Since then, it is a repeated problem; students mistake in such items.Maybe the problem is that they don't concentrate too much in the question posed to them.They often give answers expressing their general opinion on what is asked, but not try to find and give details on what is asked.This is an issue that should be treated at schools by teachers carefully.Teachers and students must pay more attention to reading exercises that contain such types of questions.

They do better in grammar items than in vocabulary ones
Much more work needed for vocabulary activities, exercises, quizzes, tests, that will improve students' results in this language aspect.

Cheating still remains an issue
Even these last years (2014-2015), when Ministry of Education and Sports together with its dependent institutions have taken measures for a strict and fair test administration, there have been cases of students punished for cheating.But, to say the truth, the number of students who have cheated in State Matura Exams and punished for it has been reduced significantly compared with previous years.

New items formats asked to be applied, so a new test format for foreign languages tests
What is the existing test format like?

B1 Level Examination
This examination aims to test candidates on their English language competence at B1 level of the Common European Framework of Reference for Languages (CEFR).
The examination comprises a Reading Section, a Use of Language Section and a Writing Section.Listening and speaking skills are not tested.
The overall descriptors of the B1 level of the CEFR have been used to guide the content and level of the examination.The descriptors are as follows: Can understand the main points of clear standard input on familiar matters regularly encountered in work, school, leisure, etc.
Can deal with most situations likely to arise whilst travelling in an area where the language is spoken.
Can produce simple connected text on topics, which are familiar, or of personal interest.
Can describe experiences and events, dreams, hopes & ambitions and briefly give reasons and explanations for opinions and plans.

B1 Level Examination (MATURA EXAMS 2014-2015)
This examination aims to test candidates on their English language competence at B1 level of the Common European Framework of Reference for Languages (CEFR).
The examination comprises a Reading Section, a Use of Language Section and a Writing Section.Listening and speaking skills are not tested.
The overall descriptors of the B1 level of the CEFR have been used to guide the content and level of the examination.The descriptors are as follows: Can understand the main points of clear standard input on familiar matters regularly encountered in work, school, leisure, etc.
Can deal with most situations likely to arise whilst travelling in an area where the language is spoken.
Can produce simple connected text on topics, which are familiar, or of personal interest.
Can describe experiences and events, dreams, hopes & ambitions and briefly give reasons and explanations for opinions and plans.

Some figures on English Language Tests for both years:
Nr of students (test-takers) in English Language Test, B2 level(CEFR) -11852; Average grade -6,99; Average points -23,05 (test carries 40 points in total); Girls have had a better performance than boys; Urban regions have performed better than rural ones; Public schools have performed better than private ones.

Some issues related to Matura Exams (English Language Test, B2 level (CEFR)
Difficulties with scanning details items in Reading Comprehension section.Students have problems to read the questions very carefully and attentively; they always give answers giving general information without being concentrated on details.This aspect still remains an issue to be improved.
Prepositional phrases result to be difficult for a considerable number of students, they know when to use the phrase but misuse the preposition; for example: I was really impressed by his dedication to the project.

Made
His dedication to the project really _______________________ me.
Key: His dedication to the project really made an impression on me.
Most of the students give answers as follow: His dedication to the project really made an impression to me.
His dedication to the project really made an impression at me.
There were just few students who give the correct answer.
The number of the students with spelling mistakes is increasing.This can be explained with teachers' efforts to encourage the speaking aspect of the language more than the other ones.

Piloting a new format
National Agency of Exams has planned a new format for all exams (compulsory and optional ones).
Foreign languages (English, French, Italian, and German) are to be presented with a try-out test format which includes only reading and writing skills again.The test will carry 70 points in total; 20 multiple choice items which carry 20 points, and 15 open-ended items which carry 35 points.This is a project that will help students and teacher with new item formats which have never been applied before.
The new format will include some of the previous years' item formats, but at the same time will have new ones.
In the following test format, items in cursive will be the new ones to be experimented in this try-out test.The agency with its specialists and collaborators have been trying to include all possible item formats to make students and teachers be prepared for what will come next year.

Are teachers and students prepared for this new test format?
Nearly all the item formats described at the second table, are items practiced at school.They have worked with these types of questions all the time; maybe they haven't practiced them in tests before, but they have worked several exercises related to grammar, vocabulary, and reading, in which are use such items formats.
So, it is expected that students will do well in this new test format.

What else will change except items' formats?
One of the biggest complaints students have with the current version of State Matura Exams is that there is not enough time to take two exams in a single day.That will change with the new version of the test; they have to take it as a single one in three hours.They won't have two tests in three hours any more.So, there will be more time per section on the new version, and students will even have double the amount of time to write their essay.
The test of English Language, Level B1, will be designed in a way to no longer use tricks and uncomfortable items to students (eliminating texts related to family problems, crime, drug, racial issues, etc).
The try-out test will evaluate how much improvement can be expected from the change.
The agency will see the Try-out Test results to decide whether the proposed changes will work in the actual environment of interest.
This test will help the test specialists to decide which combinations of changes will have the desired effects on the important measures of quality.

Conclusions
Testing is typically seen as an evaluation of what students have learned, and indeed this is true.After taking the new test format, teachers will pay attention to how students will perform on tests and will use that knowledge to inform their teaching in the future.If many students will fail a particular topic on the test, it may be a sign to spend more time covering that material next time or use a different approach to teaching the materials.Teachers will also learn how individual students perform and what the students' respective strengths and weaknesses are.In turn, teachers can use that information to guide further instruction.
Today's students need to know not only the basic reading and writing skills, but also skills that will allow them to face a world that is continually changing.This is the way the new test format is conceptualized.They must be able to think critically, to analyze, and to make inferences.Changes in the skills base and knowledge our students need require new learning It hasn't been easy to design standard tests for different types of schools and students levels.The level of teaching English differs from a region to another one.It depends on the qualifications the teachers of English language have.Since 2006, State Matura Exams in Albania have been applied in different formats.(Thetechnical aspect) In 2006-2007, all tests (Albanian Language and Literature, Mathematics, English language, and other subjects such as Physics, Biology, History, etc) were multiple-choice format (50 questions -50 points) In 2007-2008 was kept the same test format; 50 multiple-choice questions -50 points.
Its format changed 25 items 12 items of multiple-choice format -10 points 13 items of open-ended format -40 points 3 sections (reading comprehension / use of language / writing) Level A2 (CEFR) English language as an optional exam Its format was kept the same as the previous one; 20 items 10 items of multiple-choice format -10 points 10 items of open-ended format -30 points 3 sections (reading comprehension / use of language / writing) Level B2 (CEFR) Some important issues that English Language and other foreign languages are facing; Language aspects that have resulted problematic in these tests during the years

Task
Questions 18-21 -Each question tests ability to choose the multiple choice, 4 correct vocabulary item to complete a gap 4 2 answer choices in a sentence Tests ability to use conversational Task English, e.g."Could you tell me…" Question Question 22 -cloze 22 has one short dialogue.Candidates 3 3 Use of complete a gap in a sentence.No answers 20 12 5 Language are provided.Question 23 -Tests ability to choose the appropriate Task choosing vocabulary to complete gaps in a dialogue, the correct option from 4 4 choosing from a list of 8 options.The a list dialogue has four gaps.Tests ability to decide the appropriate Task word to complete gaps in a text in terms of Question 24 -cloze grammar, vocabulary, and linking 4 5 words.The text has four gaps.No answers are provided.Tests ability to write a short text.Candidates choose 1 writing option from a .Ideas are given to help with 10 short text writing.The options may be a short email, story or a description Candidates write 100-125 words.

facts on English Language Test as compulsory and optional tests in Matura Exams from 2006-2014 In
Note: The Passing grade is 5; 10 is the highest.The other languages are not included in the table because of the minimal number of students who has taken these tests.
State Matura 2006, the English Language Test contained 20 questions (20 points) on reading comprehension, 25 questions (25 points) on Grammar and Vocabulary, and 5 questions on Writing.English Language Test was an optional test at Level B1 (CEFR).In State Matura 2007-2010, the English Language Test contained 9 questions of multiple-choice and open-ended format on reading comprehension which carried 16 points; 10 questions of multiple-choice and open-ended format on grammar and vocabulary, which carried 18 points and 1 question on Writing.English Language Test was an optional test at Level B1 (CEFR).

Table 2 )
This school year (2015-2016), ETS produced tests for our State Matura Exams (English Language Test, B2 level).So, this test will pave the way for the new format we are planning to apply.
1.Description of experiences, impressions, plans, ambitions;A short story, the content of a favourite book or a film; Writing one's personal opinion