Arrangements for the publication of results

April 2014

Dear Vice-Chancellor or Principal

1.    I am writing to provide details of the arrangements for publishing the Research Excellence Framework (REF) results. No response is required to this letter.

Format and timing of publication

2.    On Thursday 18 December 2014, the REF team on behalf of the four UK higher education funding bodies will publish the results on the REF website and in a printed publication.

3.    We will send 10 hard copies of the publication to reach each higher education institution (HEI) on Thursday 18 December 2014. This publication will contain the overall quality profile awarded to each submission, listed by unit of assessment (UOA). The overall quality profiles will present the proportions, rounded to one per cent, of research activity judged to have met each of the quality levels from 4* to unclassified. The full-time equivalent (FTE) number of Category A staff included in the submission will be shown alongside the overall quality profile.

4.     The results on the REF website published on Thursday 18 December will comprise the information as described in paragraph 3 as well as the sub-profiles for each of the three elements of the assessment (outputs, impact and environment). The sub-profiles will be rounded to 1 decimal place. In the 'Guidance on submissions' we had indicated that the sub-profiles would be published in spring 2015. The funding bodies have since decided that the sub-profiles should be published at the same time as the overall quality profiles, given the widespread interest in the assessment of impact.

5.     The results on the REF website will be listed by UOA and by HEI. Downloadable files containing all of the results will also be made available on the REF website.

6.    The results for joint submissions will be listed separately against each institution involved in the submission. The FTE figure will reflect the FTE of staff submitted by that HEI.

7.     We will not publish sub-profiles for any submission whose headcount of submitted staff (Category A plus Category C) is three or fewer, because of the inferences that could be drawn about the quality of the published output of individual researchers in very small submissions. With the exception of submissions with a headcount of three or fewer, we will publish the impact sub-profiles for all submissions, regardless of the number of case studies which were submitted.

8.     The Higher Education Statistics Agency will publish contextual data for the REF on 18 December (www.hesa.ac.uk/ref2014)

Release of embargoed results

9.     On the morning of Tuesday 16 December 2014, the REF team will release individual institutions' results to the head of each HEI, in confidence and under embargo until Thursday 18 December 2014. The embargoed results released on 16 December will contain the same data as the website publication: the overall quality profile, the three sub-profiles and the FTE number of Category A staff, for each submission made by that institution. We will advise HEIs how we will disseminate these confidential results.

10.    To assist institutions in interpreting their own results, on Tuesday16 December we will also provide comparative data. This will comprise, for each UOA, the average profiles for all submissions to the UOA (mean average, weighted for staff volume); and the minimum, lower quartile, median, upper quartile and maximum percentage for each quality level from 4* to unclassified. These comparative data will also be published on 18 December.

11.     At 9:00 am on Wednesday 17 December 2014, the REF team will provide HEIs with all results, under embargo. The results will also be released under embargo to the press on Wednesday 17 December, on the basis that no approach should be made to institutions for comment, until the afternoon of that day.

12.     By October 2014 we intend to provide institutions with an example of the format in which the embargoed results will be provided.

Confidential feedback on submissions

13.     In January 2015, the REF team will provide confidential feedback to the head of each HEI. This will comprise, for each submission, concise feedback summarising the reason for the quality profiles awarded, with reference to the published assessment criteria. In the case of joint submissions, we will provide this feedback confidentially to the heads of all institutions involved.

14.     Concise feedback will also be provided to the institution on its individual staff circumstances, at institutional level.

Publication of other REF material

15.     In early 2015, we will publish the submissions on the REF website. For each submission we will publish: a separate list of staff and outputs; the submitted case studies and impact template; and the submitted environment data and template. We will remove personal and contractual details of staff, and any other data that the HEI has indicated should not be published for commercial sensitivity or other reasons. HEIs have had the opportunity to provide redacted versions of case studies, impact templates and environment templates for the purpose of publication.

16.     Any adjustments made to submissions as a result of audit will be reflected in the published submissions. We are keeping HEI's audit contacts informed of all such data adjustments. The submission system will remain available for institutions to access their submissions in a read-only format, until the submissions are published in 2015. Data adjustments made as a result of audit are visible on the submission system.

17.     In early 2015 we will publish panel overview reports. A report by each main panel, including sections from each sub-panel, will detail how the assessment was carried out, provide an overview of the panels’ observations about the state of research in the areas falling within its remit, and general reflections on the submissions and their assessment.

18.     A report by the REF Equalities and Diversity Panel (EDAP) will also be published in early 2015. The report will provide details of EDAP's work and observations about the equality aspects of the REF.

19.     In spring 2015 we will also publish the following on the REF website:

• Minutes of the sub-panel and main panel meetings for the assessment phase of the REF. 
• A report by the REF manager, detailing how the process was managed in operational terms, and reflections on how the implementation of the REF might be improved in future.

20.     During 2015 the funding bodies will also publish the impact case studies as a searchable database.

Yours sincerely,

Graeme Rosenberg
REF Manager.