Assessment criteria and level definitions

The assessment criteria and the definitions of the starred levels that were used in the assessment are set out below. These are further explained at Annexes A and B of ‘guidance on submissions’. The ‘panel criteria’ explains how the sub-panels would apply the assessment criteria and interpret the level definitions.

The primary outcome of the assessment was to award an overall quality profile to each submission, showing the proportion of the submission that meets each of the following starred levels:

Overall quality profile: Definitions of starred levels

Four star	Quality that is world-leading in terms of originality, significance and rigour.
Three star	Quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence.
Two star	Quality that is recognised internationally in terms of originality, significance and rigour.
One star	Quality that is recognised nationally in terms of originality, significance and rigour.
Unclassified	Quality that falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of this assessment.

In forming the overall quality profile, sub-panels developed a sub-profile for each of the three elements of the assessment – outputs, impact and environment. The weighting of each element in forming the overall quality profile, and the assessment criteria and definitions of the starred levels for the sub-profiles, are set out below.

Outputs sub-profile (65%)

The criteria for assessing the quality of outputs were ‘originality, significance and rigour’.

Four star	Quality that is world-leading in terms of originality, significance and rigour
Three star	Quality that is internationally excellent in terms of originality, significance and rigour but which falls short of the highest standards of excellence.
Two star	Quality that is recognised internationally in terms of originality, significance and rigour.
One star	Quality that is recognised nationally in terms of originality, significance and rigour.
Unclassified	Quality that falls below the standard of nationally recognised work. Or work which does not meet the published definition of research for the purposes of this assessment.

Impact sub-profile (20%)

The criteria for assessing impacts were ‘reach’ and ‘significance’:

In assessing the impact described within a case study, the panel formed an overall view about its ‘reach and significance’ taken as a whole, rather than assess ‘reach and significance’ separately.
In assessing the impact template (REF3a) the panel considered the extent to which the unit’s approach described in the template was conducive to achieving impacts of ‘reach and significance’.

Four star	Outstanding impacts in terms of their reach and significance.
Three star	Very considerable impacts in terms of their reach and significance.
Two star	Considerable impacts in terms of their reach and significance.
One star	Recognised but modest impacts in terms of their reach and significance.
Unclassified	The impact is of little or no reach and significance; or the impact was not eligible; or the impact was not underpinned by excellent research produced by the submitted unit.

Environment sub-profile (15%)

The research environment was assessed in terms of its ‘vitality and sustainability’. Panels considered both the ‘vitality and sustainability’ of the submitted unit, and its contribution to the ‘vitality and sustainability’ of the wider research base.

Four star	An environment that is conducive to producing research of world-leading quality, in terms of its vitality and sustainability.
Three star	An environment that is conducive to producing research of internationally excellent quality, in terms of its vitality and sustainability.
Two star	An environment that is conducive to producing research of internationally recognised quality, in terms of its vitality and sustainability.
One star	An environment that is conducive to producing research of nationally recognised quality, in terms of its vitality and sustainability.
Unclassified	An environment that is not conducive to producing research of nationally recognised quality.