Developing a core outcome set for chronic rhinosinusitis: a systematic review of outcomes utilised in the current literature

Background A core outcome set (COS) is an agreed standardised collection of outcomes that should be measured and reported by all trials for a specific clinical area, in this case chronic rhinosinusitis. These are not restrictive and researchers may continue to explore other outcomes alongside these that they feel are relevant to their intervention. The aim of this systematic review was to identify the need for a COS for chronic rhinosinusitis. Methods A sensitive search strategy was used to identify all published Cochrane systematic reviews and randomised control trials of intervention for adult patients with chronic rhinosinusitis. Two independent authors reviewed these to obtain a list of outcomes and outcome measures reported by each clinical trial. Results Sixty-nine randomised control trials and eight Cochrane systematic reviews were included in this study. They reported 68 individual outcomes and outcome measures, with an average of four to ten outcomes per clinical trial. These outcomes were mapped to 23 subcategories belonging to eight core categories. Conclusions The key finding of this review was the heterogeneity of outcomes reported and measured by clinical trials of patients with chronic rhinosinusitis, precluding meaningful meta-analysis of data. This review supports the need for development of a COS, to be used in future trials on adult patients with chronic rhinosinusitis.


Background
Chronic rhinosinusitis (CRS) is a disease in which patients develop inflammation of their nose and paranasal sinuses, with symptoms persistent for longer than 12 weeks. It may exist with or without nasal polyps [1]. It represents a common and widespread source of ill health in the UK with 11% of UK adults reporting symptoms of CRS [2]. It is also a source of substantial economic burden, through both direct health care costs and indirect societal costs secondary to lost economic productivity [3,4].
Symptoms of CRS include nasal obstruction, nasal discharge, facial pain, loss of smell and sleep disturbance. When measured using the generic quality of life outcome measure, the Short-form-36, these symptoms have a major impact on patient's quality of life, worse in some domains than other chronic diseases including angina and chronic respiratory disease [5]. Acute exacerbations, inadequate symptom control and respiratory disease exacerbation are common amongst this population. Complications are rare, but may include visual impairment and intracranial infection.
There is considerable variation in the way that CRS is managed. This relates, in part, to the lack of strong recommendations in treatment guidelines. There are a number of Cochrane reviews evaluating the effectiveness of treatments in CRS, but they are limited both by a paucity of high-quality randomised trials, and the heterogeneity of outcomes in those that have been reported which precludes meaningful meta-analysis. In order to overcome this, both the European Position Paper on Rhinosinusitis and Nasal Polyps, and the recently revised Cochrane systematic reviews, recognise the need to define a core outcome set for CRS [1,[6][7][8][9][10].
A COS is an agreed standardised collection of outcomes that should be measured and reported by all trials for a specific clinical area, in this case CRS [11]. The outcomes defined in a COS are not restrictive and trialists may continue to explore other outcomes alongside these that they feel are relevant to their intervention. However, the primary outcome should be one contained within the COS and if the COS is not being implemented then researchers should explain this decision in their findings [12].
The use of a COS in future trials of intervention in adult patients with CRS serves to minimise heterogeneity between outcomes reported by trials, allowing research data to be pooled for more meaningful meta-analysis, increasing both numbers of patients available and statistical power. The other advantage of a COS is that the outcomes are reflective of all health service users, including patients. The use of COS is supported by the World Health Organisation [13] and the Cochrane reviews of effects of health care interventions and have already been developed and adopted by multiple medical and surgical specialties [14].
The aim of this systematic review is to summarise the reporting standards of surgical outcomes in trials of intervention in adult patients with CRS with or without nasal polyposis, providing evidence in support of further development of a COS. It also aims to identify a list of outcomes required in the development of a COS.

Methods
This systematic review was conducted in accordance with the published Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) guidelines [15]. The study protocol was registered with the Core Outcome Measures in Effectiveness Trials (COMET) initiative (www.comet-initiative.org/) [16].

Literature search
At the suggestion of COMET, high-quality randomised control trials for inclusion in this systematic review were obtained through a systematic search of the Cochrane Database of Systematic Reviews (16 Aug 2015) and the Cochrane Central Register of Controlled Trials (CEN-TRAL) (16 Aug 2015). The search was performed using the search terms, rhinosinusitis or ENT or Otolaryngology. Language, age and date restrictions were not applied to the searches.

Inclusion criteria
Systematic reviews of trials of intervention in patients, 18 years or older, with CRS with or without nasal polyps were included. Those systematic reviews that had included patients with asthma, aspirin sensitivity and allergic fungal rhinosinusitis were also included.

Exclusion criteria
Studies looking at CRS in a paediatric population, defined as age less than 18 years, studies of patients with allergic rhinitis or those with patients suffering with secondary CRS (patients with cystic fibrosis, granulomatosis with polyangiitis and other systemic diseases other than the ones listed above) were excluded. Published Cochrane systematic review protocols of on-going reviews were excluded.

Study selection
The full-text articles for the Cochrane systematic reviews were obtained and analysed for eligibility by two independent reviewers, with a third reviewer available to adjudicate discrepancies.
As each systematic review presents the results of a number of pooled clinical trials, the published papers for each individual trial were obtained. The bibliographies of the trial papers were further evaluated to identify additional studies for inclusion, not included within the Cochrane systematic reviews themselves. If we could not find the published full-text article of each randomised control trial, the authors were contacted directly to see if they could provide a copy of their original research paper or thesis.

Data extraction
Two independent reviewers extracted data from the randomised controlled trials (RCTs). Two independent spreadsheets were created and the primary outcome (if defined), secondary outcome and the Outcome Measure Instrument (OMI) used to measure both the primary and secondary outcome in each trial listed. These outcomes and outcome measures were obtained from the method and results section of each paper. If the paper did not explicitly mention the outcomes that they measured, then the reviewer inferred these from the given data. At the end of the data extraction process, a final list of outcomes was compiled by consensus between both reviewers and a single spreadsheet created, containing details of each study, including sample size, study type, outcome and validity of outcome measure.
Previous work perfumed as part of the OMIPP project, commissioned by Cochrane UK, surveyed Public, Patients and Practitioners to identify a list of outcomes that they felt were important in Cochrane reviews and trials of intervention for patients with CRS. We employed the categories identified as part of this project, using them to map our outcomes and OMIs, as appropriate [17]. The OMIPP project identified 23 predetermined core categories belonging to eight main categories, namely 'Changes in patient-related symptom severity' , 'Quality of life' , 'Physiological assessments' , 'Microbiological' , 'Biomarkers' , 'Lower airway disease' , 'Side effects' and ' Acceptability of treatment'.
Each individual outcome from this study was mapped to a core subcategory. These subcategories were then grouped into main categories which were then placed into the Outcome Measures in Rheumatoid Arthritis Clinical Trials (OMERACT) filter 2.0. The OMERACT filter 2.0 has been developed by a group of international health care professionals involved in improving outcomes reporting in rheumatology and one of the first groups to develop a COS. They have provided a framework to be considered when developing a COS which includes death, life impact and resource use/emotional impact, pathophysiological manifestations and adverse events [18].

Results
Following a systematic search of the Cochrane database, using the PRISMA guidelines, a total of 49 systematic Cochrane review articles were identified Once the inclusion and exclusion criteria were applied, this number reduced to 18. Duplicate entries and published protocols of on-going studies were excluded to leave ten systematic reviews which were included in this study. A summary flowchart is provided in Fig. 1

below.
A total of 83 RCT papers were included within these ten Cochrane systematic review papers. Of these, 14 were excluded as they were trials of paediatric patients, those with allergic rhinitis or published conference abstracts. A total of 69 RCTs were finally obtained from these systematic reviews and included within this study. No further references were identified through examination of the bibliography of the published systematic reviews or trials. All included articles were published in the English language. The included clinical trials were published between the years 1975 and 2012. Table 1 provides the details of the systematic reviews and the number of trials extracted from each of these.
Three hundred and sixty-five individual outcomes were extracted from the clinical trials. The trials overlapped in their use of outcomes and between them had used 68 different outcomes and OMIs. Initially these were divided into two broad categories, namely category 1 'outcome' and category 2 'Outcome Measure Instrument (OMI) used to assess the specific outcome'. This produced an extensive list, with multiple OMIs used to measure individual outcomes ( Table 2). We employed the categories identified as part of the OMIPP project (previously discussed in the 'Methods' section), using them to map our outcomes and OMIs, as appropriate [17]. Hence, our long-list was mapped onto 23 predetermined 'core categories' , belonging to eight 'main categories' , namely 'Changes in patient-related symptom severity' , 'Quality of life' , 'Physiological assessments' , 'Microbiological' , 'Biomarkers' , 'Lower airway disease' , 'Side effects' and ' Acceptability of treatment' (Table 3).
These main categories were then mapped into the five OMERACT subheadings, 'Life Impact' , 'Pathophysiology' , 'Resource use/Economic impact' , ' Adverse Events' and 'Death' (Table 4). No clinical trial in our study had included an outcome for the OMERACT core area 'Death' [18].
Most trials reported between four and ten outcomes each. The most frequently reported outcome was the patient's individual perception of their health and severity, measured using either a Visual Analogue Scale (VAS) or the patient's history or patient diary (n = 51 RCTs). The most frequently used outcome measure was an endoscopic assessment of the patient's sinonasal cavity (n = 55 RCTs).

Discussion
The COMET initiative was supportive of this study and, at the recommendation of their lead biostatistician, we extracted outcomes from RCTs that had been included in Cochrane systematic reviews, hopefully reflecting high-quality, well-designed research studies.
The key finding of this systematic review was the heterogeneity of outcomes reported and measured by Cochrane systematic reviews and clinical trials of adult patients with CRS. Most trials had not explicitly defined the primary outcome for their study or provided definitions for the outcomes that they were measuring, with many simply providing a list of outcome measures, such as nasoendoscopy or radiology, that were being used by them to evaluate effectiveness of treatment in their study. Sixty-nine RCTs had employed 68 individual outcomes between them to report effectiveness of various treatments being studied on patients with CRS. The biggest drawback of this is the inability to pool patient data and perform meaningful meta-analysis.
A positive finding was that most studies had included a measure of 'life impact' reflecting the patient's perception of their own health and its impact on their quality of life. However this was achieved using a variety of VASs or simply based on the patients history, hence making comparison between studies more difficult.
The most commonly reported subgroup of outcomes was 'Pathophysiology'. Some of the outcomes included in this group were the outcome measures 'nasoendoscopy' , 'radiology' , 'mucociliary function' , 'nasal volume' and 'resistance'. This is unsurprising as these are frequently used clinical measures that allow clinicians to assess objective response to treatment. However, 16 different outcomes measures had been used in this category alone, hence once again making direct comparison between studies difficult.
None of the included studies had assessed 'resource use/economic impact' of disease and only a handful had asked about patient preference and acceptability of the treatment to the patient. These may be of importance when considering compliance and longevity of treatment. They may also be important to stakeholders other than physicians, including patients and third parties such as health care providers, insurance companies and drug companies.
The OMERACT category 'death' was not reported by any of our trials. Patients with CRS are generally fit patients undergoing elective surgery or medical therapy and hence we would not expect that patients should die as a direct result of the treatment being offered [18]. We must remember that the OMERACT filter 2.0 was developed for patients with rheumatoid arthritis, and although the breadth of outcomes it offers is excellent, it must perhaps be modified for our cohort of patients with CRS, with the exclusion of the death category.
We found that historical trials reported fewer outcomes, rarely defined the outcome and used simpler OMIs such as nonvalidated VAS scores and facial roentgenograms. After the late nineties these evolved to include betterdefined outcomes, with multiple validated outcome The key strength of this study is the large number of clinical trials included, providing an exhaustive list of outcomes that have been reported in the academic literature. However, a limitation of this study is that we have only included outcomes that are important to researchers and clinicians. We have not captured outcomes important to patients themselves or other stakeholders such as primary care physicians.
To improve the quality of trials included in this review, with minimal bias, only those studies published within Cochrane systematic reviews were included in this study. Hence, a study of other search engines, as would be routine practice when researching a systematic review, was not performed. This does risk missing some good trials published independently of the Cochrane reviews.
In order to overcome the two limitations discussed, the outcomes obtained through this review, will be combined with (a) outcomes obtained through patient focus groups, (b) outcomes obtained through Public, Patient and Practitioner surveys [17] and (c) outcomes obtained through interviews with family practitioners and otolaryngologists. It is this final long-list of outcomes, representing all stakeholders, that will be used in the development of a COS via the Delphi process [12,19].
The Cochrane Collaboration has recently updated a few of the published systematic reviews included in this study [6-8, 10, 20] . For completeness, full-text articles of these updated reviews and any trials not previously reviewed were analysed. This has not revealed any adjunctive outcomes, for inclusion in the COS development process.
Once a final list of outcomes has been formulated, we will ask patients and clinicians to define the outcomes that have been extracted from these trials, providing standardised definitions that we, both clinicians and patients, understand. The involvement of patients in contributing and then defining outcomes will ensure that the target audience for the treatment is an active participant in the development of the COS. As previously explored, developing a COS for this subgroup of patients will allow meaningful comparisons to be made between future clinical trials of novel and existent therapies, with the ability to combine results and produce meaningful, high-quality research data, with an overall positive impact on patient management.

Conclusion
This systematic review supports the need for development of a COS in CRS and has identified a potential list of outcomes to be used in this process.