Chris Cope

  • Improving teaching about threshold concepts

    Chris Cope, La Trobe University, Australia

    Session 2d, Tuesday 09.00

    Research paper

    Theme addressed:

    • Teaching methods

    The notion of threshold concepts is a relatively new one in the student learning research literature. A threshold concept has been described by Meyer and Land (2005) as a portal, opening up a new and previously inaccessible way of thinking about something. A key feature of threshold concepts is that they are troublesome for students. It follows that teaching about threshold concepts could be improved. While the literature describes the nature of threshold concepts in detail it does not report research-based projects which have successfully improved teaching. This paper describes a project which successfully identified and implemented more appropriate approaches to teaching about information systems (IS), a threshold concept in the IS discipline. While a threshold concept specific to one discipline was the focus of the research, the paper describes a research approach which can be used to identify and evaluate more effective teaching approaches for any threshold concepts.

    To improve teaching about IS an action research cycle based on a modified version of a lesson study was used. A lesson study is a model developed for Japanese teachers to continuously evaluate their teaching (Lo et al, 2004). The modified learning study had the following stages:

    • Conduct a phenomenographic study to empirically identify distinctly different ways of experiencing an IS
    • Analyse the outcomes of the phenomenographic study using a structure of awareness as an analytical framework to identify the educationally critical aspects of the concept of an IS
    • Identify teaching approaches more likely to assist students to address the educationally critical aspects
    • For a group of students, determine their levels of understanding of an IS prior to experiencing the teaching approaches (using the outcomes of the phenomenographic study)
    • Apply the teaching approaches
    • Evaluate the impact of the teaching approaches by determining the levels of understanding of the group of students after they have experienced the teaching
    • Statistically analyse the evaluation responses
    • Iterate to the teaching approach identification stage.

    In a detailed phenomenographic study, Cope (2002, 2006), identified multiple educationally critical aspects of the concept of an IS. Two of these aspects became the focus of selecting appropriate teaching approaches. First, students need to conceptualise people as part of an IS. Secondly, students need to understand the role of people in an IS. People attribute meaning to the output of the IT and make organisational decisions based on the meaning.

    Two teaching approaches were devised to encourage 30 students in a third year undergraduate unit called IS Practice to address these educationally critical aspects. First, the students were asked to study a Bureau of Meteorology (BOM) information system. The weather forecasts and warnings issued by the BOM are important to students in their everyday lives. Yet the output of the IT aspect of the BOM information system (predicted pressure maps) is incomprehensible to most students, at least in terms of predicting a day’s weather and maximum temperature. These predictions can only be made by experienced meteorologists interpreting the predicted weather maps in the light of experience, historical data and direct observations. Without people the BOM information system would be ineffectual.

    The second teaching approach was a role-played case study. The case study concerned a business in a regional town of 80,000 people which ran a bus service to and from the nearest main airport. The lecturers role-played the owners and employees of the business. The business’s IS was proving ineffectual and some poor business decisions had been made on the basis of inappropriate attribution of meaning to the output of the IS. The students were required to analyse the owners’ requirements and design a new IS.

    A short written answer questionnaire was devised on the basis of the outcomes of Cope’s phenomenographic study to investigate students’ levels of understanding of the concept of an IS. The questionnaire contained 13 items which required students to reflect on their understanding of IS from different perspectives. The questionnaires were completed by the students at the beginning and end of the unit. Thirty paired questionnaires were collected. The questionnaires were initially analysed qualitatively to categorise responses as representing the beginning stages of the development of an understanding of an IS as a social system, the consolidating stages or the established stage. Sixteen out of the 30 students showed an improved understanding of people being part of an IS (McNemar’s test, MC = 16, p < 0.001). Seven out of thirty showed an improved understanding of the process of attribution of meaning (McNemar’s test, MC = 7, p < 0.01). Despite the statistically significant improvement in students’ understanding of the process of attribution of meaning, 7 out of 30 was considered a poor result that possibly reflected regression towards the mean (Trochim 2002).

    The second iteration of the action research cycle of the learning study is currently underway. The teaching approaches will be modified on the basis of the analysis of the questionnaires, in particular the failure of the teaching approaches to improve many students’ understanding of the process of attribution of meaning. The questionnaire responses indicated that students found it difficult to isolate the concept of an IS from the process of developing an IS. For instance a common response to a question asking students to describe the components of an IS was to describe the stages in the process of developing an IS. The revised teaching approaches need to put more emphasis on students studying examples of real IS before encountering the role-played case study.

    References

    • Cope CJ (2002). Educationally critical aspects of the concept of an information system.Informing Science Journal, 5, 2, 67-78.
    • Cope CJ (2006). Beneath the Surface: The Experience of Learning about Information Systems, Santa Rosa, CA: Informing Science Press.
    • Lo ML, Marton F, Pang MF, & Pong WY (2004). Toward a pedagogy of learning. In F Marton & ABM Tsui (Eds), Classroom Discourse and the Space of Learning (pp 189-225). Mahwah, NJ: Erlbaum.
    • Meyer JHF & Land R (2005). Threshold concepts and troublesome knowledge (2): Epistemological considerations and a conceptual framework fro teaching and learning,Higher Education, 49, 3 373-388.
    • Trochim W (2002). Regression to the mean. Retrieved 25 January 2006 fromhttp://www.socialresearchmethods.net/kb/regrmean.htm