A number of the exams sampled include multipart questions. For these items, the beginnings and ends must be signalled very clearly. Among the exam scripts reviewed, the numbering and formatting of these varies considerably. In several cases, it is quite difficult to differentiate between main questions and sub-questions. This introduces the possibility of some students not completing enough of the exam paper.
Multi-item questions need to be labelled clearly. For example:
Examination Rubric
This exam has four questions. You must answer Q1 (worth 40 marks). You must answer any two from Q2, Q3, and Q4 (worth 30 marks each).
Each question is made up of several sub-questions. The marks given to each part of a question are shown in square brackets, like this [5 marks].
Section A
Question 1 (You must answer this question.)
This question has 5 parts. Be sure to answer all parts of the question.
Section B
There are three questions, you may answer any two.
Question 2
This question has 4 parts. Be sure to answer all parts of the question.
Etc.
Why write a multi-part question?
In most of the exams reviewed, the sub-questions in multi-part questions are clearly related by topic or to one problem. On some exams, however, the sub-questions are unrelated. Because the formatting of multi-part questions can be confusing to examinees, your questions will be clearer if they are set out as clearly separate items wherever possible.
Number of questions
Guides to writing good exams normally recommend offering students some choice of questions. This advice holds equally true when examining EFL students, with one caveat. When deciding the total number of question choices to offer, consider how long each question is and how much reading is required to review the questions carefully enough to make an informed choice about which to answer.
If your questions are long, consider whether it might be better to offer fewer choices to cut down on the reading. Or ensure that your paper includes designated reading time and that the amount of time allotted is sufficient for all students to read through the exam thoroughly. Current reading time allotments and total exam duration have probably been based on the requirements of native-speaker home students. If you have a cohort of students that has a high proportion of EFL students, you may need to readjust these timings.
Other considerations
Do not crowd your exam paper. Leave plenty of white space between and around questions.
Question design
Question words
The standard question words used in exams are problematic because many of them can be interpreted in a number of different ways. Even within one exam paper, the same question words can require quite different responses. Here are some examples.
Sample 1
Discuss the relationships between classification, ritual and taboo.
According to Blaug “ Malthus invented a game that we will meet again and again: it might be labelled the “apocalyptic fallacy”, denoting the habit of making predictions with open ended time horizons…his theory is tautology masquerading as theory.”Discuss.
Discuss the main challenges for companies in the ecological context. Illustrate your answer with specific examples from companies and industries.
Sample 2
Outline the main factors which affect the value of European call options according to the Black-Scholes model of option pricing. What assumptions does this model depend on and can it be extended to other kinds of options?
You are a financial manager of a French company operating in the USA which expects to receive income in dollars at the end of next year as a result of a long term contract. Outline how you could use hedging strategies in order to safeguard your profits from this contract when measured in Euros. Are there any problems with relying on hedging in this way to eliminate foreign currency risk?
Becker’s 1965 ‘Economics of the Family’ caused great controversy at the time by extending economic theory to what had been regarded as something falling outside the remit of economics. Outline Becker’s approach and critically evaluate its contribution to our understanding of economic relations within the household.
Sample 3
In the early years of this decade there were a number of high profile corporate scandals in the USA. To what extent do these developments indicate a widespread failure of corporate governance in market-based financial systems?
To what extent can the 1870s be correctly considered as a revolutionary period in the history of economic thought?
Can a student answer “not at all” to these questions? Is there an implied “yes” in these questions? It is not clear how much leeway students have in answering “to what extent” questions. If you intend to use them in your exam, ensure that you have discussed this type of problem in a lecture or seminar first.
Click to see some definitions for common exam question words from a popular study skills handbook that your students might turn to when preparing for their exams.
How closely do these generic definitions match the way the question words discuss, outline, and to what extent have been used in the examples above?
How well do they match your own use of question words?
If you notice even subtle differences between these definitions and your own use, then it is important that you do one or both of the following:
- select alternative question words or question phrasing that more clearly states what you require.
- take time to talk through past exam questions with your students. Model expected responses to your question types.
Other considerations
Avoid overly long questions. (See also the page on case studies.)
Sample question
1. Knowledge intensive tasks can be divided into different task types, “planning”, being one such type. Various task templates are associated with these task types and can be used to help create systems that address the corresponding tasks. This question is concerned with task types and task templates.
(i) Briefly describe what is meant by a planning task. [4 marks]
(ii) Task types can be divided into “synthetic” and “analytic” types. Explain the difference between these two types. [3 marks]
(iii) A task template contains various elements, one of which is a description of the goal of the task. Briefly describe THREE other elements that a task template may contain. [6 marks]
Revised sample question
1. This question is concerned with knowledge intensive task types and task templates.
(i) Briefly describe what is meant by a planning task. [4 marks]
(ii) Task types can be divided into “synthetic” and “analytic” types. Explain the difference between these two types. [3 marks]
(iii) A task template contains various elements, one of which is a description of the goal of the task. Briefly describe THREE other elements that a task template may contain. [6 marks]
Highlight key words. Sample question
OUTLINE the main concepts in EITHER currency substitution OR portfolio balance models of exchange rate. For the model you have chosen, CRITICALLY ASSESS the extent to which it can explain recent exchange movements.
Avoid ambiguity Sample question
It is sometimes argued that in their writings about the proper role of government the classical economists advocated that the government should, ideally, be little more than a ‘nightwatchman’. Comment on this view.
What is being argued here? That the economists made these statements? Or that classical economists advocated a ‘nightwatchman’ approach to government?
What are you looking for with a comment?
Avoid colloquialisms. Sample question
A high tech company has developed a revolutionary mobile phone that has many new functions that make it a truly innovative product. All the magazines are suggesting customers will flock topurchase them as soon as they are available.
Also, which magazines?
Break up complex sentences or complex terminology. Sample question
Using an appropriate model, compare and contrast the implications of relatively high interest-elasticity of investment and consumption demand with relatively low interest-elasticity of investment and consumption demand for expansionary monetary policy.
Revised question
Using an appropriate model, compare and contrast the implications of
- relatively high interest-elasticity of investment and consumption demand
with
- relatively low interest-elasticity of investment and consumption demand
for expansionary monetary policy.
Avoid culturally biased language or examples.
Avoid examples relating to alcohol and gambling.
Also avoid reference to specific British people, thing or events that international students may not be familiar with. This is important even when this knowledge is not required to answer the question.
Sample question
As an anthropologist, how would you study Eurovision?
A useful solution to this problem was found in an exam which framed a question inside a cheese scenario. This examiner added the following line to this question.
You do not need to know about cheese to answer this question.
Use frequent vocabulary in place of infrequent vocabulary. Sample question
Close inspection of the sample data reveals that while 18 of the plants are of modern vintage, the remaining 12 are somewhat older.
Revised question
The sample data also shows that 18 of the plants are modern, but the remaining 12 are somewhat older.
Sample question
You are required to provide a draft report for the group’s senior management, which clearly articulates a strategy for improving the organization’s approach to managing its diverse workforce across the whole portfolio of properties.
Revised question
You are required to provide a draft report for the group’s senior management, which clearly presents a strategy for improving the organization’s approach to managing its diverse workforce across the whole portfolio of properties.
Avoid double negatives
It’s not impossible that …
Proofread your exam
EFL learners may feel less comfortable questioning the language of an exam and may be less able to work out on their own how any mistakes should be corrected.
Case studies
Case studies are probably the most likely question type to bias against EFL students. A number of the exams reviewed use case studies; the length of these ranges from one paragraph to 12 pages. There is a variety of practice in relation to case study questions. On some exams case studies are:
- read for the first time in the exam.
- given to students in advance of the exam. Clean copies of the case study are issued with the exam paper. Students are not allowed to bring their own copies of the case or any notes into the exam.
- given to students in advance of the exam. Students are allowed bring their annotated case studies and accompanying notes into the exam and are required to turn these in with the exam paper.
The reading load of case studies is very likely to have a negative impact on EFL students’ performance on an exam due to their slower reading speed. This is even true if they have been given the case study to read in advance. Case study questions require that students repeatedly refer back to the case study and their notes while they write. EFL students will require more time for this reading than will native speaker students.
Marking criteria
It is always easier to complete a task when you know what is expected of you and how your achievement will be evaluated. The same is true for students sitting exams. Rather than keeping marking criteria under lock and key assessment specialists generally agree that examinees should be made aware of these as early as possible in the teaching and learning cycle.
What can you do?
- Make your marking criteria public.
- Indicate clearly whether or not language, grammar and spelling will affect the mark and how. Will you take points off for poor grammar? If yes, how many?
- Discuss what the exam will look like in class and indicate how the various sections or tasks will be weighted and marked.
- Explain how the exam fits in with any other assessment in the module or programme.
- For first year students in particular, ensure that they understand the marking system, e.g. 70+ is a distinction and 40 is a pass mark. The numbers in the UK marking scheme represent very different achievement levels than in most other countries.
After the exam
The following are some suggestions for mini-action research reviews of your exams and students’ performance. While the suggestions below are not intended to be particularly rigorous or scientific, the hope is that they may help you identify areas where changes or research might be beneficial.
Time
You may wish to examine whether time and speed are a factor affecting your students’ results. When marking, monitor the length of answers in relation to the quality of answers. If a pattern emerges in which poor answers are clearly associated with shorter answers, you may need to consider whether allowing students more time to write longer (more contentful) answers might result in more marks being awarded.
You could also check answer length against the amount of time students spend on the exam by asking the students or the invigilator to record the time that each student finishes the exam and leaves the room. (Assuming they are allowed to leave when they are finished.)
Language difficulty
Several exam papers allow or require students to bring in annotated case studies, pre-prepared notes and/or calculations. Students are then required to hand these in with their exam paper when they leave the room.
Review the annotated texts from international students asking the following questions. How marked up are the texts? Are they covered with first language translations? If so, this is a key signal that students may be having difficulty with basic reading comprehension. Students reading heavily translated texts are often reading something quite different in meaning and tone than students reading entirely through English.
If it is technical terms in particular that are marked up, consider that many of the translations may be incorrect or off the mark as technical vocabulary often is not included or not well defined in bilingual dictionaries.
These types of reading and vocabulary problems cannot be solved by making changes to the exam paper. They can be alleviated by actions that you and students can take prior to the exam. Some examples of these are outlined in the next section.