Contextual citation data

Citation counts depend partly on the field of research and a publication’s age. Therefore, where sub-panels are making use of citation data the REF team will provide contextual data to assist in the sub-panel's interpretation of the citation counts. This will consist of information about the citation behaviour of groups of papers worldwide, published in a similar subject area and of a similar age.

Elsevier have been selected to provide citation data for the 2014 REF. Elsevier own the Scopus database which covers almost 18,000 titles from over 5,000 publishers. Each journal in the database is assigned to one or more subject classifications, using their ‘All Science Journal Classification’ (ASJC) codes. A list of ASJC codes, alongside the journals included in each category is available on the SciVerse web-site (see the link ‘Key Links: List of Titles (xlsx)’). 

The REF team will provide the following information for each publication year in the period 2008 to 2012, and for each relevant ASJC code:

• The average (mean) number of times that journal articles and conference proceedings published worldwide in that year, in that ASJC code, were cited
• The number of times that journal articles and conference proceedings in that ASJC code would need to be cited to be in the top 1 per cent, 5 per cent, 10 per cent and 25 per cent of papers published worldwide in that year.

The REF team will also provide these contextual data at the level of the unit of assessment (UOA). This will be done by assigning each ASJC code to one or more UOAs, and providing the contextual data for these grouped ASJC codes.  We will assign each ASJC code to one or more UOAs by looking at the proportion of journals articles assigned to each ASJC code that has been submitted to each UOA.

Where very few journal articles and conference proceedings have been published in an AJSC category in a particular year, we will not return contextual information for that ASJC. We will decide on the threshold for providing contextual data in consultation with Elsevier.

The contextual data will be provided in the form of stand-alone tables, for the sub-panels to refer to when considering citation counts for individual outputs. We will not provide a ‘benchmark’ for each individual output.

Citation counts and these contextual data will be calculated shortly after the submissions deadline. In order to ensure that all outputs are treated on a fair and equitable basis, this will be the only citation information provided to sub-panels and will not be updated during the assessment phase.

Contextual information for institutions

When these data are provided to sub-panels in early 2014 they will also be made available to the REF contact at all institutions, for information.

To help institutions understand the contextual data when preparing submissions, in early 2013 we provided analogous data to REF contacts in the form of an Excel workbook. This is contextual data for the period 2007 to 2011 (data for one year earlier than what will be provided to institutions and sub-panels after the submissions deadline in late 2013).

UOA level data will not be provided to institutions during the submissions phase.  This is because we cannot assign ASJC codes to UOAs until all submission information has been collected.