Invitation to quote

Modelling of HEA survey data

Deadline: 5 June

1  About the HEA

The Higher Education Academy (HEA) is a national body for learning and teaching in higher education. We work with universities and other higher education providers to help bring about change in learning and teaching. We do this to improve the experience that students have while they are studying, and to support and develop those who teach them. Our activities focus on rewarding and recognising excellence in teaching, bringing together people and resources to research and share best practice, and by helping to influence, shape and implement policy - locally, nationally and internationally.

Further information on the work of the HEA can be found at www.heacademy.ac.uk.

2  Background

The HEA is the leading provider of postgraduate student surveys in the UK, and has recently developed a new national undergraduate survey. The primary purpose of the HEA’s student surveys is to support institutions in enhancing the student learning experience. A secondary aim is to inform the sector about national- and discipline-level findings. Over 130 higher education providers from across the UK run HEA student surveys, with well over 100,000 responses from students expected in 2015. The surveys are run through an online survey platform.

The current round of HEA student surveys comprises the following three surveys:

  • United Kingdom Engagement Survey (UKES)

  • Postgraduate Taught Experience Survey (PTES)

  • Postgraduate Research Experience Survey (PRES)

The surveys run from February-March until May-June. Alongside data on students’ experiences and engagement, the surveys also collect data on a range of student characteristics (e.g. gender, subject of study, domicile, etc.). Data on institutional characteristics is also available (e.g. institution size, type, etc.)

All three surveys collect sufficient data to present a rich picture of students’ experiences and engagement. In their last administrations PRES had around 50,000 responses from 122 institutions, PTES had 67,000 responses from 100 institutions, and UKES had 25,000 responses from 32 institutions. These large datasets allow us to perform a range of analyses. We publish annual reports for each of the surveys, which contain relatively simple analyses. In addition, institutions receive their own confidential benchmarking reports.

The purpose of the annual reporting is twofold: a) to influence policy debates about the enhancement of learning and teaching, and b) to support institutions in making sense of their own data. The analyses we publish in the annual reports are limited by the hierarchical nature of the data; while we may present a difference between male and female students for a particular item or scale, this may be due to the way in which male and female students differ in their subject mix, for example. To draw conclusions about the real influence of student and institutional characteristics, it is necessary to perform multilevel modelling. This will allow for the nesting of students within disciplines within institutions.

Previous examples of such work with large-scale student surveys do exist. In 2008 the HEA commissioned researchers to investigate the National Student Survey (NSS) using multilevel modelling; specifically, the variance explained student and institutional characteristics (Marsh and Cheng 2008, Cheng and Marsh 2010). Multilevel modelling of the NSS has also been done recently by HEFCE (2014).[1]

The purpose of the proposed work, for which we are inviting quotations, is to undertake this form of analysis on the three surveys run by the HEA. This will be the first time that multilevel modelling on these surveys has been undertaken. The intention is to understand the impact of student and institutional variables on the experiences and engagement of PGR, PGT and UG students; to inform the sector about key issues; and to support institutions in making sense of their own data.

The key research questions that need to be addressed, for each of the three surveys, are:

  1. What proportion of the variance in experience and engagement is explained by student and institutional characteristics?

  2. Can the survey reliably distinguish between institutions, and between courses?

References

For information about the HEA’s surveys, please visit: http://www.heacademy.ac.uk/surveys

Cheng, J. and Marsh, H. (2010) “National Student Survey: Are differences between universities and courses reliable and meaningful?” Oxford Review of Education 36(6): 693-712

HEFCE (2014) National Student Survey results and trends analysis 2005-2013 (Bristol, Higher Education Funding Council for England). Available at: http://www.hefce.ac.uk/media/hefce/content/pubs/2014/201413/HEFCE2014_13%20-%20corrected%2012%20December%202014.pdf

Marsh, H. and Cheng, J. (2008) National Student Survey of teaching in UK universities: Dimensionality, multilevel structure, and differentiation at the level of university and discipline – preliminary results (York, Higher Education Academy). Available at: http://www.islamicstudiesnetwork.ac.uk/assets/was%20York%20-%20delete%20this%20soon/documents/ourwork/nss/NSS_herb_marsh-28.08.08.pdf

 

[1] The HEA also published a non-technical guide to interpreting NSS data using multilevel modelling: http://jisctechdis.ac.uk/assets/documents/nss/NSS_interpreting_data_nontechnical_guide.pdf

3  Aims and objectives

Aims:

  • To determine the contribution of important student and institutional factors to students’ experiences and engagement

Objectives:

  • To develop models for each of the three HEA surveys, that take into account the hierarchical structure of the data

  • To generate coefficients for important student and institutional characteristics

  • To consider the implications of these findings for sector and institutional priorities

  • To draw conclusions about the ability of each of the three surveys to make useful distinctions at institution and course (discipline within institution) levels

4  Intended outputs

  • A report of the research undertaken, describing the methodology, the findings for each of the surveys, and the implications of the findings for sector and institutional policy

  • Suitable dissemination, including presentation at conference/s, blog posts for the HEA, etc.

Timescales

Description Date

Deadline for submission of quotations

5 June

Presentations of short-listed quotations

11 June

Successful contractor selected and informed

12 June

Project commences

12 June

Project initiation meeting

16 June

Submission of interim report

10 July

Submission of final research report

31 July

Quote

Please supply an outline quote for the work reflecting the programme of development above. Potential contractors may then be invited to present on their experience and outline their approach to this piece of work.

Queries relating to this invitation to quote

Any queries relating to this invitation must be sent to Alex Buckley at surveys@heacademy.ac.uk