In the field of applied linguistics and language teaching, Swain (1984) put forward a key principle for testing – to “bias for best”. Essentially, she is suggesting that tests or exams should aim to elicit a student’s best performance and this can be accomplished by providing the following:
- more than adequate time to complete a task,
- an opportunity to review and change work,
- access to reference materials,
- checking to see that examinees are on task,
- clear instructions including information about what is being assessed,
- useful suggestions about how to do the task.
All of these suggestions aim to create an assessment environment that enables us to find out what students do know, rather than what they don’t know.
A review of a wide range of past exam papers from the Oxford Brookes library illustrates that a lot of good practice in writing clear exam papers already exists. Nevertheless, there were still many cases where improvements could be made. This section aims to provide practical examples of small changes that can help to reduce the potential for negative bias caused by the language or layout of exam papers. Sample rubrics and questions have been taken from these public exam papers. No disrespect is intended to their authors.
Rubrics and Layout
It is very important that rubrics are clear, because you want to ensure that students are being assessed on their ability to demonstrate their understanding of your subject rather than on their ability to make sense of your instructions. This section aims to provide some simple suggestions for getting your rubrics right.
Begin by writing task instructions in clear, accessible language. Then organise the instructions so that students can easily follow what they are supposed to do. Here’s an example of an effective rubric.
Examination Rubric
The examination will last 2 ¼hours. The first ¼ hour is reading time only.
Section A consists of Question No. 1. You must answer this question. The question is worth 40 marks.
Section B consists of four questions. You must answer any two questions of your choice. Each question is worth 30 marks.
Some questions have more than one part. The marks given to each part of a question are shown in square brackets, like this [20].
Calculators will be provided.
Below you will see some examples of less effective rubrics along with some suggestions on how they might be improved.
Sample 1
Examination Rubric
The duration of the examination is 2 hours.
ANSWER ALL THE QUESTIONS
You must answer all three questions shown below. Questions 1 and 3 are divided into sub-questions which have marks allocated, as shown below. Answers should be in essay format. Examples should be used to illustrate and support your answers and use of diagrams is encouraged where appropriate. Close reference to second language acquisition theory must be made in all answers. Credit will be given for evidence of appropriate wider reading and citation of appropriate authors.
The questions are based on general second language acquisition theory and the case A VSO course for Adult English language learners in Malawi. You are allowed to bring your own annotated copy of the case into the examination, together with four sides of A4 notes to help you answer the questions.
Can you identify some potential problems with this rubric?
The changes made to Sample 1 include:
- Adding reading time for review of the case study.
- Moving sentences around so that the instructions are not interrupted.
- Changing passive sentences to active.
- Replacing vocabulary that is likely to be unfamiliar with simpler vocabulary.
- Breaking up one long dense paragraph into shorter ones.
- Replacing a phrase that could easily be misinterpreted with less ambiguous instructions.
- Using bold and CAPITAL LETTERS to highlight key words
Sample 2
Examination Rubric
The duration of the examination is 3 1/4 hours; of which the first ¼ hour is reading time only.
The students would be expected to answer all three questions.
You are allowed to bring into the examination the notes and calculations that you have made relating to the case study. These must not exceed four A4 or equivalent sides of paper. They must be numbered, clipped together and identifiable as your notes. Your notes may include financial ratios, analysis of marketing data, other statistical trends or predictions and any other data which is time consuming to collate under examination. You may also bring into the examination your annotated copy of the case study. You should not include any written narrative that can be imported directly into your answer in either your notes or your annotated case study. You should hand in your notes, calculations and annotated copy of the case study at the end of the examination.
Can you identify some potential problems with this rubric?
The changes made to Sample 2 include:
- Cutting the length of the instructions down from 159 words to 103 words, by cutting out instructions relating to actions students should have taken before the exam, but cannot do anything about in the exam.
- Changing passive sentences to active.
- Replacing or cutting vocabulary that is likely to be unfamiliar with simpler vocabulary.
- Replacing a phrase that could easily be misinterpreted with less ambiguous instructions.
- Limiting the use of bold to highlighting key words and phrases.
- Highlighting a key instruction by separating it from the main paragraph.
Other considerations
- Some exams provide an estimate of how much time should be allotted to each question.
- Some exams point out to students that they may not need the full amount of time scheduled to complete the exam.
- Some exams give word count guidelines for answers. (If you ask for word counts, e.g. no more than 250 words or no less than 250 words, it useful to advise students to write out 250 words in advance so they know what this looks like in terms of length. It is important that students don’t waste time counting words during the exam.)
Ask yourself
- What are the repercussions of an answer being over or under your specified word count?
- What might be some advantages of specifying time or length guidelines? Are there any disadvantages?