Margaret Kiley

  • What examiners' comments tell us about the postgraduate learning environment

    Margaret Kiley, University of Canberra

    Research seminar

    Theme addressed: Assessment

    In 2002 the University of Canberra's Higher Degrees Committee expressed concern that there might be themes that were noted frequently in postgraduate thesis Examiners' Reports but that might not be recognised within current practices. A previous analysis of University of Canberra reports was undertaken by Johnston (1997) where the reports on 16 theses were analysed. The most common comments related to editorial and written presentation with the view that poor presentation implied poor quality of work. While the university-specific outcome of the 2002 research was to ensure that the University's practices take into account the comments of external examiners of PhD theses, more general lessons will be drawn for a wider audience and will be located within research undertaken regarding the examination of theses and conceptions of research (Kiley & Mullins, 2001; Mullins & Kiley, 2002) and the work of others in the field (Ballard, 1996; Hansford & Maxwell, 1993; Holdbrook, Bourke, Farley, & Carmichael, 2001; Marsh, 1998; Winter, Griffiths, & Green, 2000). This presentation will report on the following which arose from the research:

    • The examiner's conception of the purpose of research candidature and the production of a research thesis.
    • The implied perspective of the examiner regarding her/his role in the examination process
    • Comments regarding the appropriateness of the research question, the structure of the thesis, written expression, technical accuracy (including correct referencing) and the overall quality of the research and to whom the examiners directed these comments; i.e. the university, the supervisor or the student.
    • The perceived role of the supervisor, the quality of supervision and how these comments were expressed in the text.
    • The examiner's perception of the role of publications in the decision made about the quality of the thesis.
    • Anomalies between the grading given and the text of the report and how they might be explained.

    References

    Ballard, B. (1996, 18-19 April). Contexts of judgment: An analysis of some assumptions identified in examiners' reports on 62 successful PhD theses. Paper presented at the Quality in Postgraduate Research: Is it happening? Adelaide.
    Bogdan, R., & Biklen, S. K. (1982). Qualitative research for education: An introduction to theory and methods. Boston: Allyn and Bacon.
    Bryman, A., & Burgess, R. (Eds.). (1994). Analyzing qualitative data. London: Routledge.
    Denzin, N., & Lincoln, Y. (Eds.). (1994). Handbook of qualitative research. Thousand Oaks: Sage. 
    Hansford, B. C., & Maxwell, T. W. (1993). A masters Degree Program: Structural components and Examiners' comments. Higher Education Research and Development, 12(2), 171-187.
    Holdbrook, A., Bourke, S., Farley, P., & Carmichael, K. (2001, February). Analysing PhD examination reports and the links between PhD candidate history and examination outcomes: Methodology. Paper presented at the HERDSA Conference Committee, Newcastle.
    Johnston, S. (1997). Examining the examiners: An analysis of examiners' reports on doctoral theses. Studies in Higher Education, 22(3), 333-346.
    Kiley, M., & Mullins, G. (2001). Supervisors' and students' conceptions of research: A comparison. Paper presented at the Bridging instruction to learning: 9th European conference of the European Association for Research on Learning and Instruction, Fribourg, Switzerland.
    Marsh, H. (1998). Improving the Australian PhD examination process (Paper to DoGS). Townsville: James Cook University.
    Mullins, G., & Kiley, M. (2002). 'It's a PhD, not a Nobel Prize': How experienced examiners assess research theses. Studies in Higher Education, 27(4), pp. 369-386.
    Strauss, A., & Corbin, J. (1990). Basics of qualitative research: Grounded theory procedures and techniques. Newbury Park: Sage.
    Winter, R., Griffiths, M., & Green, K. (2000). The 'academic' qualities of practice: What are the criteria for a practice-based PhD? Studies in Higher Education, 25(1), 25-37.