Reporting evidence of graduate outcomes to the national agency. How did Swedish institutions manage?

  • Asa Lindberg-Sand, Centre for Educational Development, Lund University, Lund, Sweden

    In 2011, a new national system for outcome-based quality assurance in higher education was launched in Sweden (Gov. bill 2009/10:139), marking a shift at the national level from a system oriented towards enhancement to one with its main focus on control. The outcomes of all degrees are now assessed by large groups of external experts using two sources: A sample of students’ theses and a self-evaluation for each degree. The experts collect samples of students’ theses to make direct assessments of to which extent they indicate that the degree outcomes are achieved. The theses, though, are not meant to fulfil all the degree outcomes. Therefore departments responsible for programmes leading to the degree, also have to produce a self-evaluation showing evidence on how the full range of degree outcomes are achieved, f.i. by reporting outcomes from student assessments and examinations (National Agency, 2011).

    This paper deals with the question of how higher education institutions have managed this reporting task. The analysis is founded in a social-cultural perspective on learning as an integrated aspect of social practices (Wenger, 1998). Quality assurance methods, learning outcomes and student assessment practices are viewed as different knowledge methodologies, appropriating tools for the classification of knowledge (Bowker & Star, 1999). In the self-evaluation different classifications are comprised to claim the fulfilment of the intended outcomes. The aim of the paper is to describe the variation both in the interpretations of the national guidelines informing the self-evaluation task, and in the evidence of the achieved outcomes provided by the institutions. Considering the loose regulation and decentralised nature of student assessment in Swedish HE, a somewhat disturbing question is lurking beneath the surface: Is the task possible to fulfil satisfactory at all?

    The results are acquired along two pathways: The first is a conceptual analysis concerning the possible relations between the national degree outcomes, the intended learning outcomes in the syllabi, eventually assessed, and institutional documentation practices (Lindberg-Sand, 2011). The second, ongoing analysis consists of an empirical content analysis of nine self-evaluations for three degrees from different higher education institutions. Preliminary conclusions show that the material provided to the panels by the self-evaluations is varying, fragmented and not sufficient for a fair assessment of the extent to which the degree outcomes are fulfilled, especially when degrees at different institutions are compared. Academic trust in the outcomes from the new quality assurance system will probably be difficult to build.