Susan Orr

  • Justify 66 to me! An investigation in to the social practice of agreeing marks in an HE Art and Design department.

    Susan Orr, York St John College of the University of Leeds

    Research seminar

    Theme: departmental strategies

    Yorke (2000) and Johnson (2004) comment that assessment in higher education is under researched and under theorised. Why this might be becomes clear if we consider the methodological difficulties inherent in researching assessment practice in what is, in many disciplines, a very individualistic practice carried out at home or work.

    In spite of this challenge, a few researchers are investigating lecturer judgement. For example, Baume et al (2004) asked assessors to mark archived student portfolios while completing a written commentary. Orrell (2003) asked lecturers to record their thoughts while they marked essays. Hand and Clews (2000) used mixed methods to explore the ways that lecturers utilise criteria to mark work.

    These researchers are developing theory concerning the process of judgment and their results are important because, collectively, they point to the complex and diverse ways that lecturers draw on assessment criteria, tacit practice and their own experience to mark student work.

    This paper reports on research into lecturer judgment in art and design education. Art and design offers a rich terrain in which to study the social practice of assessment. The centrality of the ‘group crit’1 and the location of student work in the studio has created a context where dialogue is a cornerstone of assessment. Adopting a constructivist approach I argue that lecturers’ judgements are co-constructed through this dialogue.

    As a semi participant observer I observed a series of moderation meetings in an HE art and design department. Using discourse analysis I noted distinctive discourse features that were employed by the lecturers to accommodate difference and establish consensus whilst moderating marks. For example, the process of mark agreement was characterised by an open ended discussion focusing on qualities of the student work followed by a distinct change in register where the communication shifted to agreeing the actual number to be recorded on the mark sheet. Where there was agreement the discussion resembled a verbal dance and when disagreement was evident the dance became a verbal skirmish. In each case this dance/skirmish moved to resolution through a series of rhetorical moves that culminated in an agreed mark.

    Drawing on Wenger’s (2003) work on communities of practice and on research into tacit practice (Torff 1999, Sambell and Mc Dowell 1998) my findings reveal that whilst marks are agreed through dialogue they are also agreed through silence. As Canatella (2001) points out, in the field of art and design, words do not always enable lecturers to verbalise the ways that judgements are made; assessment in art and design is a multi sensory practice.

    I have been able to build up a thick description (Geertz 1973) that offers insights into the ways that assessment work is done. For example, the lecturers repeatedly used certain terms against which student work was judged. These are terms that students often find hard to understand (for example critical analysis, visual research). My analysis of the ways that these terms are used in assessment dialogues goes some way to explicate the multiple meanings of these terms.

    1 Where students and lecturers discuss student work together