Reporting of Patient-Reported Outcomes in Randomized Trials:  The CONSORT PRO Extension FREE

The CONSORT (Consolidated Standards of Reporting Trials) Statement, first published in 1996 and most recently revised in 2010,^1- 2 provides evidence-based recommendations to improve the completeness of reporting of randomized controlled trials (RCTs). The statement focuses on parallel-group trials, but a number of extensions for reporting other trial designs (cluster, noninferiority, and equivalence), interventions (nonpharmacologic and herbal therapies), and for specific data, such as harms have been developed.³ The CONSORT Statement is endorsed by major journals and editorial groups, such as the International Committee of Medical Journal Editors, and its use has been associated with improved reporting of trials.⁴

Patient reported outcomes (PROs; Box include health-related quality of life (HRQL), symptoms, utilities, and satisfaction ratings and are defined as assessments that are patient reported rather than observer reported. PRO data from RCTs are increasingly used to inform patient-centered care, clinical decision making, and health policy or reimbursement decisions.^5- 7

CONSORT Extension

Additional checklist items regarding guidance for reporting patient-reported outcomes (PROs) (supplementing the CONSORT 2010 Statement)

Elaboration of the CONSORT Statement

Further details on the existing CONSORT 2010 guidance as applied to a specific context; in this instance, as applied to randomized controlled trials assessing PROs

Patient-Reported Outcome

An outcome reported directly by patients themselves and not interpreted by an observer; PROs may include patient assessments of health status, quality of life, satisfaction with care or symptoms, or patient-reported adherence to medication

Proxy-Reported Outcome

Proxy reports from caregivers or clinicians cannot be viewed as PROs⁵; it is recommended that the standards for reporting of proxy outcomes are similar to those recommended for PROs

Primary or Principal Trial Publication

The first publication of the trial results: this will include the results of all primary outcomes prespecified in the protocol for all trial participants; prespecified secondary PROs will also be reported in the primary publication

Secondary Publication of PRO Results

A detailed exploration of PRO results may be provided in a secondary trial publication (published following the primary article).

Health-Related Quality of Life

Health-related quality of life (HRQOL) is a multidimensional concept that usually includes self-report of the way in which physical, emotional, social, or other domains of well-being are affected by a disease or its treatment

Primary Outcome

The most important outcome in a trial, “providing the most clinically relevant and convincing evidence directly related to the primary objective of the trial.”

Secondary Outcomes

These are outcomes prespecified in the protocol to assess additional effects of the intervention and often include PROs; within the secondary PROs, some may be identified as important or key

Important or Key Secondary Patient-Reported Outcomes

Some PRO measures (particularly HRQOL measures) are multidimensional, producing several domain-specific scales. For any particular trial, it is likely that some domains will be more relevant than others, reflecting the expected effect(s) of the trial intervention(s) in the target patient population. These relevant domains constitute the important or key secondary PROs. Ideally, these will be identified a priori, specified as such in the trial protocol, statistical analysis plan, or both and will be the focus of the principal PRO analysis. Because these are linked with hypotheses (see CONSORT PRO Extension 2b), they may be subject to P value adjustment (or α-spending)

The applications of PROs require accurate, valid, and accessible reporting, but this is uncommonly observed. In a review of 794 RCTs only 56% provided a rationale for the selected PRO, 50% included a PRO hypothesis, 28% provided information about missing PROs data, and 64% discussed the PRO findings in the context of the other trial outcomes.⁸

The aim of this work was to develop an evidence-based extension of the CONSORT statement for reporting patient reported outcomes in RCTs (extensions) and to elaborate on the existing CONSORT 2010 statement specifically as applied to PROs (elaborations). This article describes the methods used to gain consensus on the extensions and elaborations and provides the rationale for each new item and examples of good reporting.

The extension was based on the methodological framework for guideline development proposed by the EQUATOR Network.⁹ Initial work was led by the International Society for Quality of Life Research (ISOQOL) Reporting Guidelines Task Force and focused on establishing and developing guidance for RCTs with HRQL as an outcome.¹⁰ Following feedback from stakeholders the scope of the guidance was expanded to include PROs. The University of Birmingham Ethical Review Board approved the survey and consensus meeting.

A detailed description of the systematic review has been previously reported.¹⁰ Briefly, the review identified then existing guidelines (until April 2011) for HRQL reporting in RCTs. Titles identified by a Medline literature search were reviewed independently by 2 task force members (M.D.B. and Brenda Bass, MBA, Queens University, Kingston, Ontario), and supplemented by literature identified by a review of article bibliographies. Candidate standards for reporting HRQL outcome data in RCTs were abstracted from eligible articles.¹⁰ The literature search was replicated, updating it to January 2013, to identify additional candidate reporting standards that may have been published since the original systematic review.

This process identified 6 potentially relevant articles, although no proposed reporting standards beyond those considered at the CONSORT extension Delphi meeting were identified. The research leading to the development of the reporting standards was originally HRQL focused. The task force received feedback from ISOQOL and other stakeholder groups that the consensus process should be extended to include all PROs in order to explicitly evaluate the applicability of reporting standards to PROs overall, thus allowing generalization of the reporting standards to all clinical trials that included PROs.

An online survey was created using candidate reporting items taken from the systematic review.¹⁰ The survey was first distributed using the membership listings of ISOQOL¹⁰ and subsequently to additional stakeholder groups. Survey respondents were asked to rate the importance of each reporting item when HRQL was a primary outcome and a secondary outcome in an RCT of a biomedical intervention. (The survey instrument, stakeholder groups surveyed, and results are available in eAppendix 1).

The survey results and comments were synthesized into draft reporting guidance by the task force. The draft guidance was sent to all ISOQOL members and debated at the annual conference in Denver, October 2011. Written feedback after the meeting was encouraged. Following feedback the scope of the guidance was broadened to include all PROs and revised draft guidance was produced for discussion at the CONSORT PRO consensus meeting.

Twenty-nine participants attended the 2-day meeting in London, England, in January 2012. The meeting, which was designed to obtain consensus on the content of the CONSORT PRO extension, included journal editors, methodologists, clinical trialists, policymakers, clinicians, knowledge translation experts, representatives of UK and US funding bodies, industry, and patients. An overview of the consensus process is described in eAppendix 2 with examples of the survey results, and the voting process provided in eFigure 1 and eFigure 2, respectively.

The final CONSORT PRO guidance identifies 5 items to be reported in all RCTs in which PROs are a primary or important secondary outcome. Definitions for terms such as important secondary outcome are contained in the glossary (the Box, and exemplified in the eBox). Table 1 lists the 25 items of the CONSORT 2010 checklist (left column) and the 5-item extension relating to PROs (right-hand column prefaced by the letter P). The items specific to PROs are (1) that the PROs be identified as a primary or secondary outcome in the abstract; (2) that a description of the hypothesis and relevant domains be provided (if a multidimensional PRO tool has been used); (3) that evidence of instrument validity and reliability be provided or cited; (4) that the statistical approaches for dealing with missing data be explicitly stated; and (5) that PRO–specific limitations of study findings and generalizability of results to other populations and clinical practice be discussed. Although an extension was deemed unnecessary for a number of existing CONSORT checklist items, an elaboration of items applied to PRO was recommended (Table 1, right-hand column, plain text). Below we provide the rationale for each PROs specific entry in Table 1, with examples of good practice.

CONSORT 2010: structured summary of trial design, methods, results, and conclusions.

PRO Extension: The PRO should be identified in the abstract as a primary or secondary outcome.

Example. “The primary outcome was the change in COPD specific quality of life at 24 months as measured with the chronic respiratory questionnaire total score.”¹¹

Explanation. If a PRO is prespecified as a primary or important secondary outcome in the trial, it should be explicitly stated in the abstract to facilitate indexing and identification of studies to inform clinical care and evidence synthesis.

CONSORT 2010: Scientific background and explanation of rationale.

PRO Elaboration: The relevant background and rationale for why PROs were assessed in the RCT should be briefly described.

Example. “Migraine causes severe impairment or bed rest in more than half (57%) of affected people, markedly impairs quality of life both during and between attacks, increases absenteeism and reduces productivity at work, and is associated with increased health care costs^(referenced).”¹²

Explanation. Given the increasing literature on PROs, and the increasing number of validated instruments available to assess them, the Background or Methods section should briefly establish the rationale for including PROs and why the specific outcomes were selected, thus providing appropriate context for the PRO–specific objectives and hypotheses (see item 2b below). When a PRO is a primary study outcome, a more detailed summary of the existing literature regarding its assessments relevant to the study purpose and objectives is helpful.

CONSORT 2010: Specific objectives or hypotheses.

PRO Extension: The PROs hypothesis should be stated and relevant domains identified, if applicable.

Example. “Potential survival benefit needs to be weighed against the burden of treatment. For this reason, HRQOL, a multidimensional construct^(referenced) was included as a secondary end point in the EORTC 18991 study . . . The protocol hypothesized that there would be a difference in global HRQOL scale between both arms, showing worse HRQOL in the PEG-IFN-α-2b arm. The remaining HRQOL variables were then examined on an exploratory basis.”¹³

Explanation. Patient-reported outcome measures may be multidimensional or unidimensional assessing either one or several aspects of health (eg, physical and social function, or symptoms such as fatigue). In addition, PRO measures may assess global health or HRQL at several time points during an RCT. Without a prespecified hypothesis there is a risk of multiple statistical testing and selective reporting of PROs based on statistically significant results. It is recommended that authors report the rationale for the selection of specific patient-reported outcomes and the time frames of interest, including biological or psychosocial evidence for the proposed anticipated benefits or harms where relevant.

CONSORT 2010: Completely defined prespecified primary and secondary outcome measures, including how and when they were assessed.

PRO Extension: Evidence of PRO instrument validity and reliability should be provided or cited, if available.

Example. “The DLQI [Dermatology Life Quality Index] has well-established reliability and validity when used in a dermatology setting^(referenced) and is used frequently in clinical trials of psoriasis^(referenced).”¹⁴

Explanation. Ideally, the validity of all PROs used in RCTs should be established in relation to the study target population and a brief rationale for the choice of PRO instrument in the trial should be provided. This rationale may also include the validity of translated or otherwise culturally specific versions of the instrument where relevant.

There are currently more than 700 PRO measures available for use in trials.¹⁵ Clinical use of PRO data requires that the trial results are robust, which depends on a valid and reliable PRO instrument being used appropriately. Evidence should be cited of the reliability and validity of the PRO measure used in the trial so that readers can access this information. If an RCT uses a PRO instrument with psychometric properties that have not yet been published (eg, a new instrument developed for the trial), the authors should provide information on item content of the instrument and evidence regarding its reliability and validity, in an appendix if the article does not allow for such details.¹⁶

CONSORT 2010: Completely defined prespecified primary and secondary outcome measures, including how and when they were assessed.

PRO Elaboration: Details of the mode of PRO completion (in particular if a proxy completed the questionnaire on behalf of the patient), and the method of data collection (paper, telephone, electronic, other) should also ideally be provided particularly when the PRO is the primary outcome.

Example. “Participants were asked to provide data at three time points; four, eight, and 12 months post-randomization, using a self completion questionnaire to eliminate any observer bias.”¹⁷

Explanation. In some instances it may not be possible for the PRO to be completed directly by the patient. If the outcome has been completed by a proxy, this should be reported so that readers can assess any potential bias or effect on the results. Different methods of data collection may also affect the results and lead to potential bias if used differentially between intervention groups. For example, collecting PROs by telephone or in a face-to-face interview may cause patients to respond in a way that differs from what they would self-report on paper in private.

CONSORT 2010: Statistical methods used to compare groups for primary and secondary outcomes.

PRO Extension: Statistical approaches for dealing with missing data should be explicitly stated for PROs prespecified as primary or important secondary outcomes.

Example. “Analysis of complete cases, last observation carried forward, and imputation of expected and worse scores per time point were provided to check the robustness of the main results.”¹⁸

Explanation. Missing trial outcome data leads to reduced power, is a potential source of bias, and can result in misleading results. The level of missing PRO data are often relatively high. In a review of a random selection of RCTs (n = 61) published in leading international journals with HRQL as an outcome, only 10% of studies reported no missing data; in 21%, the level of missing data was unclear; and 36% had in excess of 10% missing data.¹⁹ Importantly, PRO data often are not missing at random but in relation to the outcome of interest, for example, improvement or deterioration in health status. Different statistical approaches to dealing with missing data have respective strengths and limitations.^2,20 In the example shown above,¹⁸ for instance, the “last observation carried forward” method has been criticized by some authors (who offer guidance on these methods for interested readers).²¹ Thus, in order to allow adjudication of the methods used by the authors, the approach taken should be clearly described and the potential effect on the validity of the PRO findings should be discussed when relevant.^2,22- 23

CONSORT 2010: For each group, the numbers of participants who were randomly assigned, received intended treatment, and were analyzed for the primary outcome.

PRO Elaboration: The number of participants reporting PRO data at baseline and at subsequent time points should be made transparent.

Example. eFigure 3 is a CONSORT flow diagram that includes the number of participants providing PRO data (available at http://www.jama.com)

Explanation. The CONSORT flow diagram provides readers with an overview of the progress of participants through the phases of an RCT (enrolment, intervention allocation, follow-up, and data analysis). Authors are encouraged to consider how best to report the flow of participants through the trial in relation to PROs, including information on the reason for missing PRO forms, such as lack of questionnaire return, translations unavailable, or other reasons if known (eFigure 3).^24- 27 This information will help readers to interpret the PRO results and assess the potential for bias, particularly when missing data are due to deterioration of health status. Authors may also consider providing this information in a tabulated form,²⁸ produced for each treatment group, or in footnotes of the flow diagram.

CONSORT 2010: A table showing baseline demographic and clinical characteristics for each group.

PRO Elaboration: Including baseline PRO data when collected.

Example. See Table 2: Example presentation of baseline PROs data in an RCT.²⁹

Explanation. Baseline PROs data may be used by clinicians and policy makers to assess the relevance and generalizability of trial findings.³⁰

CONSORT 2010: For each primary and secondary outcome, results for each group, and the estimated effect size and its precision (such as 95% confidence interval).

PRO Elaboration: for multidimensional PROs, results from each domain and time point specified for analysis.

Example. See Table 3: Example of treatment effects on quality of life outcomes, taken from the report of an RCT comparing 2 interventions for drug-resistant temporal lobe epilepsy.³¹

Explanation. The potential for selective reporting of PROs is increased because study instruments often contain multiple scales and items. In general, it is recommended that all PRO results should be presented alongside other outcome data typically in tabular form. The important PRO secondary outcomes should be presented in the main publication in order to facilitate the clinical integration of the important findings with other prespecified outcomes. Additional PROs or the components of composite PRO scores should be presented in the main publication where possible, as an eAppendix or expanded secondary report to reduce selective reporting of significant results and to ensure that PRO evidence is available to inform clinical practice and evidence synthesis.^32- 33

CONSORT 2010: Trial limitations, addressing sources of potential bias, imprecision, and, if relevant, multiplicity of analyses.

CONSORT 2010: Generalizability (external validity, applicability) of the trial findings.

PRO Extension: PRO–specific limitations and implications for generalizability of study findings and clinical practice.

Examples. “A potential source of bias was the overall amount of missing HRQL forms over the course of the assessment period, with more missing data in the Gemcitibine arm . . . this problem tempers our ability to generalize these longer-term effects to future patients.”³⁴

“Non-attenders at one year, however, might have had a different symptom profile and overall quality of life than attenders, and therefore some degree [of] selection bias is possible.”³⁵

Explanation. In addition to the design and conduct issues relevant to the generalizability of the RCT overall, several PRO–specific limitations (including both patient- and center-level characteristics) may affect generalizability of the PRO results. For example, if PRO assessments are limited to a subgroup of the main trial population, it is recommended to provide reasons why patients were excluded from the PRO study (such as where appropriate translations were unavailable). If PRO data are missing, it is particularly important to discuss the potential reasons in relation to the clinical context and implications for interpretation, as well as the interpretation of any supportive (eg, sensitivity) analyses undertaken. Furthermore, because many of the previously described methodological details of PROs assessment may affect the RCT results, the potential influence of these details on the interpretation of the PRO findings is recommended where suspected to be important.

CONSORT 2010: Interpretation consistent with results, balancing benefits and harms, and considering other relevant evidence.

PRO Elaboration: Patient reported outcome data should be interpreted in relation to clinical outcomes including survival data, where relevant.

Example. “Patients who received cetuximab experienced significantly less HRQL deterioration and a longer time before clinically significant deterioration occurred. These results are important, because . . . although cetuximab monotherapy . . . results in improved overall survival, progression free survival, recurrence rates and disease control rate . . . the magnitude of these benefits . . . was not large.”

Conclusion: “[C]etuximab offers important HRQL benefits and survival benefits for pre-treated patients with advanced CRC.”³⁶

Explanation. The clinical significance of PRO results is often not discussed in RCT reports but should be interpreted in relation to other important clinical outcomes such as survival, especially in trials for which there are clinically relevant trade-offs between PROs and survival outcomes.³⁷ Further interpretation of PRO results may include discussion of a minimal important change or a responder definition (if validated for the particular PRO instrument used in the study), comparison with other similar RCTs, or linking the clinical significance of the PRO results to the other trial outcomes such as toxicity rates.

CONSORT PRO aims to promote transparent reporting of RCTs in which PROs are primary or important secondary outcomes. Improved reporting will facilitate interpretation of PRO results for use in clinical practice, as described in User Guides^23,38 and inform evidence synthesis and health policy. Transparent reporting will facilitate comprehension of limitations of the data and potential sources of bias. The primary trial publication is often the only opportunity to report PRO data such that it can be interpreted in the context of the other clinical trial findings. Presentation of PRO data in stand-alone articles, which may be published months or years after the main trial report, can be a barrier to patient reported outcome data uptake. Therefore, we recommend that authors report primary or important secondary outcome for PRO results according to the 5 new items described in this article in the primary publication and that journals provide appropriate mechanisms to facilitate optimal PRO reporting (for example templates and online appendices). The CONSORT PRO checklist also elaborates how specific components of the existing CONSORT 2010 Statement may be implemented in relation to PROs.

We encourage authors, peer reviewers, and readers to use CONSORT PRO in conjunction with the CONSORT 2010 Statement and other explanation and elaboration articles (appropriate for the trial design, intervention, and outcomes).^1- 3 The CONSORT Group is considering consolidating some of the guidance statements, to facilitate uptake by authors who may find the number of available checklists difficult to implement.

In this extension we make reference to important or key secondary outcomes, which may be defined as prespecified PRO domains in the protocol that have hypothesized effects or for which the statistical power and sample size may have been taken into account. It is recognized that these definitions are not widely used and more work is needed to provide standardized terms for secondary outcomes (clinical and PROs)(eBox). A further issue that arose during the consensus meeting is that the PRO extension to item P12a, “Statistical approaches for dealing with missing data explicitly stated,” is relevant for other RCT outcomes. We encourage authors to consider reporting this type of information for all outcomes.

We developed CONSORT PRO rigorously according to current standards for developing reporting guidelines.⁹ Like other reporting guidelines, the development of CONSORT PRO is a work in progress. We will continue to monitor the literature to help guide us in any further development of this CONSORT extension. Similarly, we encourage readers to provide feedback regarding this reporting guidance and how it might be further refined.

We encourage journals to modify their “Instructions for Authors” to endorse the CONSORT PRO extension. We plan to disseminate CONSORT PRO to journals currently known to endorse CONSORT 2010³⁹ and other relevant groups, such as the EQUATOR Network.⁴⁰ We plan to evaluate whether the CONSORT PRO extension is having its intended effect, namely improved completeness of reporting RCTs in which PROs are a primary or important secondary outcome.

Finally, although these guidelines focus on PRO reporting, the design of trials assessing PROs may also be improved. We recommend that trialists consider the useful guidance from the US Food and Drug Administration (FDA) on the development, validation, and implementation of PRO measures and their analysis in RCTs.⁵ Trialists should consider PRO-specific protocol requirements in relation to the FDA guidance and more general recommendations on RCT design from the Standard protocol Items: Recommendations for Interventional Trials (SPIRIT) initiative.^40- 41