Research outputs (REF2)

The journal article should still be submitted as output type 'D' = journal article. The ISSN field should state that this information is not available.

All research outputs should be submitted against the output type which an institution considers to best represent the research being submitted for assessment. The choice of output type should not be dictated by the metadata available for that output. If mandatory metadata does not exist for a particular output you should enter 'n/a' in the required submission system field. Wherever the DOI is not available for journal articles, this field should be left blank. 

When submitting web content as a research output, URLs alone should not be submitted and you should provide a portfolio with material sufficiently substantial to constitute evidence which allow sub-panel members to assess the research dimensions of the work. This may mean a combination of media are used which could include, for example, screen shots from the website, images and photographs, a written description of the structure of the section and how it interacts with other sections, descriptive or contextual information to evidence the research process. You will need to decide whether it is necessary to provide a copy of the whole website or a representative selection. You should also provide evidence that this was indeed first publically available within the publication period.

All additional information for REF2 will be published as part of the submission information. It is not possible for HEIs to redact elements of the additional information field. Therefore, if it is not appropriate for the name and contact details of the individual to be published, you should instead state the individuals' position and company. If audited, the institution will be expected to provide contact details for the individual.

If you are submitting two different chapters within a book as two separate outputs, you should provide two copies of the book.

No. Each output must be provided either as a PDF upload or deposited as a physical output; not as a mixture of the two. A physical output may include multiple items to represent one output. It is not possible to provide some of the output as a PDF upload and the send another part of the same output as a 'physical' output.

The submission system requires that a reserve output is associated with a specific output for which double-weighting has been requested. If the request is declined, the panel will assess the associated reserve output. However, where two double-weighting requests are made for an individual and the institution wishes to rank the two reserve outputs, it may use the additional information field in REF2 to indicate which of the two reserves should be assessed if only one of the double-weighting requests is accepted.

Unless specifically requested by the REF team, artefacts, devices, products and items of great rarity or value should not be deposited. For these types of outputs, a representation of the output and sufficient information about the research it embodies should be provided to enable the panel to assess it, without providing the item itself.

The REF team will reimburse HEIs for outputs that are lost during the assessment phase. Arrangements will be made directly with the institution concerned where this is required.

The numbering of research outputs has no significance and will play no role in the assessment of output quality. It is simply a means to identify the outputs and any associated information within the REF submission system.

In the disciplines covered by Main Panel A, many (but not all) journals have well established conventions for the order in which the lead and/or corresponding author are listed - typically as first or last author. Corresponding authors are also often indicated by a typographic mark and/or a footnote. Where these conventions are well established, the sub-panel will accept them as identifying the lead or corresponding author. If in doubt about the journal’s conventions, then it would be safest to use the 'additional information' field to state what the author’s role was.

Outputs of type A (Authored book) and type B (Edited book) cannot be uploaded to the submission system as a PDF and must be provided in physical form. For e-books, institutions may choose to submit the output on a CD, DVD or USB, or provide a printed hardcopy. For output type C (Chapter in book), a PDF of the book chapter (e-book or otherwise) may be uploaded to the REF submission system. In these instances the PDF should include a copy of the page(s) of the book that bear the title, publisher, editor and publication date.

The REF team will not source e-books submitted to the REF. They must be provided by the HEI as described above and it is not sufficient to provide only the URL for an e-book within REF2.

Main Panel A have requested standard statements about the individual's contribution to a co-authored output, where required (see paragraphs 34–37 of Part2A of the 'Panel criteria'). In addition, to assure themselves about an author's substantial contribution, the sub-panel may request further details of the specific contribution of the author, through an audit query. Main Panel A have indicated that such audit queries are especially likely for outputs with 15 or more authors (and where the individual is not the lead or corresponding author).

The edited work could be submitted as a single output or alternatively, could be submitted separately if they are coherent as separate outputs. If returning them as separate outputs, you should consider whether they have significant material in common. (Further guidance on how outputs with significant material in common will be assessed is available in Part2 of the 'panel criteria'.)

Where Scopus returns multiple matches for different formats of the same output, you should not confirm the match and should contact Scopus user support (via the REF submission system). Scopus user support will be able to create a single record for the output which consolidates citations for both formats. You will then be able to match this new record against the output within the submission system.

Yes. As stated at paragraph 79f of the 'guidance on submissions', an individual may be returned as Category A by more than one HEI if they have a contract with and receive a salary from more than one HEI. The same research outputs may, but need not be, listed in each submission.

Online first journal articles should be returned as output type 'D - Journal article'. The REF team recognises that some of the meta-data may not be available for an online first publication, but sufficient information must be provided to enable the REF team to identify the output.

There may be some fields for which data is not available, that are mandatory for type D outputs (e.g. first page, volume number). These fields should be completed with ‘n/a’ to allow the submission to be made. You will not be able to submit without completing these mandatory fields.

If submitting a product (output type = ‘P’) the date that the product was produced would be considered the date it became publicly available in the form in which it is submitted (see the ‘Output information requirements’ spreadsheet, available for download on the 'Submission system data requirements' page ).

Separately published papers are discrete outputs and cannot be grouped together as a single output. Substantial dictionary or encyclopaedia entries and groups of short items including groups of entries may be submitted as a single output (see paragraph 51, Part2D of the ‘panel criteria’).

The following additional guidance was added in November 2013:

Where research has been split by a publisher and would only be considered a single coherent work when viewed together, these should be returned as one research output. 

A portfolio is not an output type in itself but is the format in which the evidence of certain outputs can be submitted (see paragraph 59 of Part2C and paragraph 71d, Part2D of the ‘panel criteria’.) For example, an output that is a performance or exhibition may be submitted in the form of a portfolio of evidence. Where submitting a portfolio, the most applicable output type should be used, and the ‘media’ field should be used to state that a portfolio will be submitted (see the ‘Output information requirements’ spreadsheet, available for download on the 'Submission system data requirements' page).

Where a research output is submitted multiple times to the same UOA, it will normally be allocated to the same individuals on the sub-panel and reviewed only once (although the additional information may differ). Please note, an output may only be submitted a maximum of 2 times within one submission.

If a research output is submitted to multiple UOAs, it will be assessed by each of the sub-panels according to their respective criteria, including in terms of its significance to the disciplines concerned. The output will not, therefore, necessarily receive the same quality score.

If the sub-panel considers that the individual may not have made a substantial research contribution to the output and there is not sufficient evidence in the submission to make an informed judgement about this, the sub-panel will raise an audit query. The HEI will then be asked to explain the individual’s research contribution, and provide any appropriate evidence. The sub-panel will consider all the information provided before deciding. (Note that in situations where this information has already been requested as part of the submission, for example for sub-panel 9 where there are more than 10 co-authors, the sub-panel will normally decide on the basis of the submitted information without any further audit query.) If the sub-panel does not accept that the individual made a substantial research contribution, the output will be graded as ‘unclassified’ (see paragraph 126e, Part 1 of the ‘panel criteria’.)

See paragraph 43 of the ‘panel criteria’. The final proviso should be interpreted to mean that such an output is eligible for submission to the REF, provided that the ‘pre-published’ output was not submitted to the 2008 RAE by the same institution, or was not listed against the same member of staff (if they were in a different institution at the time).

Yes, 'online first' and other pre-published outputs that are in the public domain during the publication period (1 January 2008 – 31 December 2013) are eligible for submission.

Both a granted patent and a published patent application are eligible as research outputs. If you are submitting a granted patent you should include the patent number and date of the granted patent; if you are submitting a published patent application you should include the publication number and the date that this was published. Where a patent application was submitted to the 2008 RAE, the granted patent is not eligible for return to the REF.

No sub-panel will make any use of journal impact factors, rankings, lists or the perceived standing of publishers in assessing the quality of research outputs. An underpinning principle of the REF is that all types of research and all forms of research outputs across all disciplines shall be assessed on a fair and equal basis.

A working paper is often written in the style of a journal article, is usually made freely available on line, may not have been peer reviewed, and has not yet been formally published (typically, in a journal). Working papers will not have a Digital Object Identifier. The use of working papers is more prevalent within certain research communities, for example ArXiv is often used in particular disciplines. The output must still meet the eligibility definitions described in 'guidance on submissions'.

Yes.