In this project, the calibration activities focused on a specific national threshold standard, ‘written communication skills’. Each participating university contributed assessed work by final year accounting students, from which a random sample was selected and de-identified. Individual participants graded three pieces of work on a continuous scale from 0 to 100 from ‘meeting’ to ‘not meeting’ the standard, coupled with written reasons for the grade awarded, suggestions for ways in which the work could be improved and ratings for the validity of the task. This was done anonymously and facilitated by an electronic repository which revealed and analysed all assessors’ responses prior to a face-to-face workshop. During the workshop, the collated responses were used as a basis for discussion of the task validity and the sample of student work, both in small groups and in plenary, with the ultimate purpose of achieving a shared understanding of the threshold standard and a consensus on each piece of student work.
Welcome to the Geography Toolkit, which is designed to assist you as a geography academic in calibrating your academic standards with those of your subject community. The toolkit:
The toolkit provides all the resources required to run a calibration workshop with for use in a department or across institutions to help discuss and create a shared understanding of the criteria used to assess student performances, leading to greater consistency in understanding and application of academic standards.
This resource contains:
Examples of final-year student assignments:
Welcome to the Musical Recitals Toolkit, which is designed to assist you as a music academic and external examiner in calibrating your academic standards with those of your subject community.
The toolkit provides all the resources required to run a calibration workshop for use in a department or across institutions to help discuss and create a shared understanding of the criteria used to assess student performances, leading to greater consistency in understanding and application of academic standards.
This resource contains:
Hartpury College is a specialist sports and land-based college, with over 1,500 students undertaking qualifications in equine, sport, veterinary nursing, animal and land studies. The college started to award its own degrees from September 2018. Involvement in the Degree Standards project was timely as the college was undergoing significant strategic change, and the project provided professional development to an institution that describes itself as ‘relatively young in higher education terms.’
Approximately 35 staff members participated in professional development for external examiners in the 2017-18 academic year, mostly aspiring external examiners. The course has been used as a development opportunity for a range of staff members responsible for developing newly required institutional policies and practices. Participants report increased knowledge and understanding of degree standards and the role of external examining in ensuring consistency across the sector. The opportunity to engage in the course is providing the institution with increased confidence that the approach to external examining reflects best practice, as well as ideas to support the development of institutional external examiner induction due to be delivered in 2018-2019.
The induction will be broader in scope as a result, to include the role of an external examiner in relation to academic standards and discussion of external reference points with the focus less on critical friend and more on degree standards. In addition, feedback from academic staff members suggests that they are able to identify changes to their own practice, for example improvements in the module files provided to examiner examiners. Promoting staff externality is an explicit aim of the college’s Teaching and Research Excellence Strategy, and external examining has been identified as one of the most effective ways to provide staff, many of whom are relatively early in their careers, with an opportunity to observe practices beyond their own institutional context. Levels of participation in the course, and numbers of staff engaged as external examiners have been set as measures within the action plan, which has helped to drive progress.
However, prioritising the time needed for preparation and participation in the developer programme has been difficult, in a small institution where the absence of two strategic level staff from one team has been hard to manage. In future, because of the size of the institution, opportunities for institutional facilitators to deliver the course, and therefore maintain their levels of competency, will be fairly restricted, and staff will probably need to invest significant time prior to course delivery revisiting the content and messages.
The college is interested in working more closely with Advance HE on recruiting participants to the course and would like to invite academics from other institutions to courses being delivered internally, as a way of adding value to course delivery.
The University of West London is a post-1992 institution with just under 9,000 students at undergraduate level. The Academic Quality and Standards Office led the institution’s involvement in the Degree Standards project. A facilitated professional development course was held in November 2017 and subsequently two further courses have been delivered, by institutional staff participating in the develop the developer process.
The second course offered at the University was targeted not just at staff members but also at external examiners appointed to the University. Engagement with the professional development course prompted a review of both induction content and delivery style. The revised institutional external examiner induction, trialed in Spring 2018, contains input and discussion over the role of externals in maintaining academic standards, which is expected to drive more critical engagement by external examiners and strengthen the institution’s approach to maintaining standards.
There is a desire within the university to improve the dialogue about degree standards between staff and appointed externals including maximising the impact and usefulness of external examiner reports, and the professional development course is one of the mechanisms being used by the institution to address this. The university is particularly interested in developing this dialogue to enhance student awareness of how disciplinary progression on both an academic and professional basis build during their time at university. Therefore, the university has positioned the course as a development opportunity for academic staff, particularly programmer leaders, which has allowed colleagues to reflect on how they currently work with external examiners and to identify how changes to external examining arrangements might strengthen delivery and standards within their own provision.
The current develop the developer model is resource intensive for the institution and requires a relatively long lead-in period, however the quality team will explore possible collaboration with other institutions as a way to resolve the issue in the short term. The university is keen to trial different approaches to the delivery of the professional development course over the next academic year and will offer the course not only to staff who are aspiring and existing externals, but also to academic staff with a general interest in academic standards. External examiners appointed by the university will be offered two different induction experiences; they will self-select whether to attend the newly developed institutional induction or attend the professional development course plus a shortened institutional induction. It is hoped to offer four professional development courses in total, with two being aimed at staff and two for externals appointed by the university.
A review will take place at the end of the academic year to assess patterns of take up and to reflect on course feedback. Ultimately, the university will measure the success of the course and other changes through the annual synthesis of external examiner reports and specifically the team is looking for change in how staff respond to external examiner findings and comments.