Sue Bloxham

  • Assessment: helping diverse students understand the tacit demands of studying in higher education

    Overview: Sue Bloxham (Convenor)

    This symposium will present research from three related projects all funded by a UK Institute for Learning and Teaching in Higher Education ‘Making a Difference’ Award.

    UK national quality initiatives have created a drive towards transparency and consistency in student assessment including robust approaches to marking and moderating student work. This drive has generated a range of institutional and departmental practices including stated learning outcomes and assessment criteria for each assignment and common grade descriptors. However, this information has its limitations because the academic language employed only develops meaning through use. Furthermore there are often differences in interpretation of language between disciplines, indeed between tutors. Interdisciplinary courses and the diversification of assessment create demands to produce work that conforms to a variety of different academic conventions.

    This topic has become particularly important because an increasingly diverse student body suggests that we cannot rely on shared language resources amongst those entering University. Haggis and Pouget’s (2002) research indicates that the greater heterogeneity of students in contemporary higher education means that we need greater clarity and explicitness about what students need to do in order to deal with their confusion and disorientation in the context of specific disciplines. However, the ‘academic literacy’ (Lea & Stierer 2000) approach would argue that one cannot transmit clarity in these matters, they can only be understood as a gradual process involving participation in order to absorb and be absorbed into a ‘community of practice’ (Lave & Wenger, 1999).

    The three papers presented in this symposium all explore interventions designed to raise the achievement and aid retention of non-traditional students through active engagement with the demands of assessment. The groups involved are:

    • mature, part-time students on an ‘open access’ foundation degree
    • International Confusion Heritage Culture students
    • Students with low entry qualifications

    They are based in the same HEI which has recently developed clear assessment protocols for staff and students including making explicit links between learning outcomes and assessment tasks, assessment criteria and common grade descriptors. The latter is a series of statements, for each credit level, which describe what a student should demonstrate in order to achieve the various grades in the academic marking scale.

    Each paper describes and evaluates a planned intervention to assist students in understanding the language and demands of assessment, in particular the interpretation of assessment criteria and grade descriptors. In each case, tutors have actively engaged students in using these statements to assess their own or others work through the use of exemplars and peer assessment activities. Significant emphasis has also been place upon students generating and making sense of feedback.

    The research methodology has incorporated both quantitative and qualitative methods with the latter experimenting with various tools that combine a resource for learning with a technique for data collection. The symposium will offer an opportunity for participants to critically analyse the function of common grade descriptors and explore both the practical and theoretical implications of the projects for the emerging research field of ‘academic literacy’.

    Themes addressed:

    This symposium directly addresses the overall theme of ISL 12 in terms of ‘Diversity and Inclusivity’ with particular emphasis on:

    • Assessment
    • Supporting learners
    • Learning and Teaching methods

    Paper 1: Developing reflective skills: enhancing the use of assessment portfolios in a foundation degree

    Pete Boyd

    In the UK, the next decade will see a large increase in students enrolling on Foundation degrees. These programmes, leading to a qualification equivalent to the first two years of a traditional degree, are designed to have a strong vocational and employment-based focus. They are seen as an important tool in increasing participation rates in higher education, particularly amongst non-traditional groups. The intervention reported here was carried out on an open access, part-time, Foundation degree for teaching assistants in schools; predominantly mature, female students.

    The assessment of professional learning, particularly in the paramedic professions, has a tradition of using portfolios both to demonstrate and reflect on vocational learning. Although characteristically student-centred, the requirements and conventions required in the construction of portfolios are far from transparent for many students and this had proved to be the case for first year students on this course. Previous students found the assignment difficult to understand and operationalise so that many achieved only a mediocre standard with detailed but largely descriptive evidence and only a poorly developed rationale.

    In order to assist students a marking scheme was devised. This scheme interprets the College grade descriptors against the learning outcomes for the portfolio in order to create clear criteria for students and staff to judge attainment against. However, Rust et al (2003) suggest that such guidelines are insufficient on their own and that students need to be actively helped to understand the tacit requirements of assessment. Thus an experimental intervention was included in the course programme to address this challenge.

    Within an action research framework, formative anonymous peer assessment was introduced followed by a task designed to translate peer feedback and exposure to other students’ work into an action plan for effective completion of the portfolio. This intervention was aimed at increasing the students’ understanding of the marking scheme. Naturally occurring qualitative evidence including observation, peer feedback sheets, an action-planning task and the portfolios themselves were used to gain understanding of the students’ perspectives. Semi-structured interviews with a sample of the students and with the tutors involved provided additional data.

    The data identified that the students’ emotional investment was very high as the portfolios consisted of evidence gathered about their school-based practice. The ethical issues and power relations involving tutors and students were a critical aspect of the process but there were potential rewards in terms of promoting reflection on practice. Students and their tutors started from a range of different positions in terms of self-awareness and confidence and a key question emerging from the research is whether the ethical and emotional barriers to peer assessment are too high for such emerging professionals. However if the aim is to develop such students as reflective practitioners, then developing professional skills, such as those involved in peer review, at an early stage may be justifiable.

    The session will discuss the findings of this small scale case study and provide an opportunity to explore the implications for the enhancement of students’ perceptions of assessment within professional education contexts.

    Back to top

    Paper 2: Enhancing the academic cultural awareness of international students on an economics programme

    Amanda Chapman

    High numbers of students from Confusion Heritage Cultures (CHCs) are choosing to study in the UK. For the majority of these students the expectation is that they can ‘slot in’ to existing programmes. Many assumptions are made about teaching International students. Some of these assumptions are valid: many are not. The one factor that remains however is that these students need particular help when embarking on an undergraduate programme. Academic cultural awareness is both explicit and implicit. There is evidence to suggest that such students can cope extremely well given the right support and induction.

    A key area of this cultural awareness is their understanding of assessment requirements. Assessment criteria and grade descriptors have been introduced in the drive to make assessment demands more transparent to students. However, the academic language involved is unfamiliar for all students and particularly complicated when English is not a first language. Therefore, the project reported here is an experiment designed to ease the process of assimilation into UK higher education, particularly assessment techniques. It involved giving guided tuition with assessment criteria and the use of exemplars.

    The students chosen for this action research project are level 1 economics students. The institution hosting this experiment has embraced the widening participation agenda, and to that end, the course group comprised approximately half CHC students with the other half drawn from the UK and the rest of Europe. The majority of the UK students are first generation users of higher education.

    The data was collected via classroom discussion and task sheets designed to help the students reflect on the process of constructing an assignment, analysing the feedback they received and action-planning for future assignments. Analysis of the data focused on a comparison between UK and CHC students.

    The results indicate that the students found the exercise extremely useful although they were more concerned with structure than content, reinforcing Rust et al’s view that students focus on ‘visible’ criteria. However, students’ self-assessment tended to privilege effort over actual attainment. They acknowledged that they would not have considered referring to the grade descriptors and, to a lesser extent, the assessment criteria if they had not taken part in the exercise. CFC students particularly appreciated the activity; just seeing HE level work was identified as valuable. This finding confirms the sense that these students do not lack motivation to do well, but have tended to lack the knowledge of what ‘well’ means in the UK higher education context.

    For home students, the financial reality of higher education today means that they often have other commitments and attend university for contact time only, thus lacking the sense of belonging associated with retention. Indeed the research indicates that CHC students have a more coherent peer group with which to discuss academic matters.

    The session will critically analyse the methodology against its usefulness to the student. Attention will also be drawn to the impact of the exercise on students’ use of feedback in constructing future assignments and the relationship of feedback with reflection.

    Back to top

    Paper 3: Helping Sport Studies students understand the rules of the game!

    Amanda West & Sue Bloxham

    Students need to understand the assessment process in order to be effective learners in higher education (Elwood and Klenowski, 2002). And, significantly, the QAA has called for greater transparency to ensure that students have access to, for example, assessment criteria and grade descriptors. Such developments have been accepted somewhat uncritically as ‘good practice’. Recent research, however, suggests that students need to be actively inducted into the academic community in order to realise the real benefits of transparent assessment information (Orsmond, Merry and Reiling, 2002; Rust, Price and O’Donovan, 2003 ). This issue may be particularly pertinent in relation to the widening participation agenda where students may enter higher education less prepared for work at this level.

    This paper reports the results of a two-phase project designed to engage Sports Studies students, in the assessment process. The students had relatively low entry qualifications, averaging 160 UCAS tariff points. .An action research methodology was employed in both phases. In the first phase, guided by the tutor, students on a Level 1 module in Sport Sociology, practiced marking using assessment criteria and the College-wide grade descriptors. Subsequently, students undertook double blind peer assessment of their colleagues’ summative assessment; a poster. Unlike other studies of peer assessment, the peer marking was itself tutor-marked as an extra incentive for the students to think about the assessment criteria and the writing of feedback.

    Quantitative data, in the form of marks, awarded by students and tutors provided a measure of the relationship between students’ ability to mark and the quality of their own work. Questionnaires, designed to help them analyse their learning from the activity but also to provide research data on the perceived usefulness of the exercise, were administered.

    Phase one provided a variety of positive outcomes. Student marks were improved, they utilised the assessment criteria in writing and marking, a high proportion were able to accurately predict the grade they had achieved and it appeared to help students to generate and understand feedback more effectively. At the end of this process they were able to articulate what they needed to do to improve and were very positive about their intention to make changes in the future.

    However, the long term impact of the exercise was unknown. Phase two of the project commenced one year on from the end of phase one. A sample of students who had participated in phase one were invited to participate in a study to explore their perspectives on, and experiences of, the assessment process. A similar study took place with matched students who had not participated in the initial activity. None of the participants were told that this project was a follow up to the Level One study at the outset of the interview so we could explore their involvement in the assessment process in the first instance without reminding them about the Level One exercise.

    The results of both phases of the research will be reported together with the implications for further research and future practice in this area.

    Back to top

    References

    • Elwood, J & Klenowski, V (2002) ‘Creating Communities of Shared Practice: the challenges of assessment use in learning and teaching’ in Assessment and Evaluation in Higher Education 27 (3) Pp. 243 – 256.
    • Haggis, T. and Pouget, M. (2002) Trying to be motivated: perspectives on learning from younger students accessing higher education. Teaching in Higher Education 7 (3) Pp. 323-336
    • Lea, M.R & Stierer, B. (Eds) (2000) Student Writing in Higher Education Buckingham, Open University Press
    • Lave, J & Wenger, E (1999) Legitimate Peripheral Participation in Communities of Practice in McCormick, R & Paechter, C Learning and Knowledge London, Paul Chapman
    • Orsmond, P. Merry, S & Reiling, K (2002) The Use of Exemplars and Formative Feedback when Using Student Derived marking Criteria in Peer and Self Assessment Assessment and Evaluation in Higher Education 27 (4) Pp. 309 – 322.
    • Rust, C, Price, M & O’Donovan, B. (2003) Improving Students’ Learning by developing their Understanding of Assessment Criteria and Processes Assessment & Evaluation in Higher Education 28 (2) Pp. 147- 164