The following resources illustrate approaches to the calibration of academic standards. Social moderation, which is the main method used for calibration, is used as a process for obtaining consistency in understanding and applying academic standards through discussion of examples (e.g. of student work) and drawing on reference points (e.g. subject benchmark statements).
Case study: Summary of a social moderation process to improve the reliability of dissertation assessment in a university department
This case study describes measures taken by a university department to improve the reliability of their assessment of dissertations. Despite having established assessment criteria, there was still concern that these on their own had not established a common set of standards. The case study describes the two workshops devised and the social moderation processes involved in establishing greater common understanding of the criteria and greater reliability in the standards applied.
Achievement Matters project: Calibration in accounting education in Australia
In this project, the calibration activities focused on a specific national threshold standard, ‘written communication skills’. Each participating university contributed assessed work by final year accounting students, from which a random sample was selected and de-identified. Individual participants graded three pieces of work on a continuous scale from 0 to 100 from ‘meeting’ to ‘not meeting’ the standard, coupled with written reasons for the grade awarded, suggestions for ways in which the work could be improved and ratings for the validity of the task. This was done anonymously and facilitated by an electronic repository which revealed and analysed all assessors’ responses prior to a face-to-face workshop. During the workshop, the collated responses were used as a basis for discussion of the task validity and the sample of student work, both in small groups and in plenary, with the ultimate purpose of achieving a shared understanding of the threshold standard and a consensus on each piece of student work.