- Study protocol
- Open Access
Core Rehabilitation Outcome Set for Single Sided Deafness (CROSSSD) study: protocol for an international consensus on outcome measures for single sided deafness interventions using a modified Delphi survey
Trials volume 21, Article number: 238 (2020)
- The Correction to this article has been published in Trials 2020 21:272
Single-sided deafness (SSD) describes the presence of a unilateral severe to profound sensorineural hearing loss. SSD disrupts spatial hearing and understanding speech in background noise. It has functional, psychological and social consequences. Potential options for rehabilitation include hearing aids and auditory implants. Benefits and harms of these interventions are documented inconsistently in the literature, using a variety of outcomes ranging from tests of speech perception to quality of life questionnaires. It is therefore difficult to compare interventions when rehabilitating SSD. The Core Rehabilitation Outcome Set for Single Sided Deafness (CROSSSD) study is an international initiative that aims to develop a minimum set of core outcomes for use in future trials of SSD interventions.
The CROSSSD study adopts an international two-round online modified Delphi survey followed by a stakeholder consensus meeting to identify a patient-centred core outcome domain set for SSD based on what is considered critical and important for assessing whether an intervention for SSD has worked.
The resulting core outcome domain set will act as a minimum standard for reporting in future clinical trials and could have further applications in guiding the use of outcome measures in clinical practice. Standardisation will facilitate comparison of research findings.
‘Single-sided deafness’ (SSD) is the name given to the condition in which there is normal or near-normal hearing in one ear and a severe to profound hearing impairment in the other ear . SSD can be congenital, sudden or progressive. The most common causes of SSD in adulthood are sudden and idiopathic, including vestibular schwannoma  and associated surgery , Ménière’s disease , and sudden-onset sensorineural hearing loss . The incidence of SSD in the United Kingdom is estimated to be approximately 9000 new cases per year .
Good hearing in both ears helps people to deal with everyday listening tasks . These include understanding speech in noisy environments and locating where sounds, such as the telephone or car traffic, are coming from [8, 9]. In adults with SSD, both these abilities are compromised [10,11,12,13,14] and can lead to functional , psychological and social consequences [16, 17]. The multi-dimensional burden on overall health is indicated by reductions in health-related quality of life  in individuals with a diagnosis of SSD.
The most commonly used treatments for SSD restore two-sided (bilateral) access to sounds by re-routing sounds from the impaired ear to the hearing ear . This can be achieved with the help of a specialised hearing aid system known as the CROS (contralateral routing of signals) aid . Bone-anchored hearing aids (BAHA) have also been used as interventions for SSD to achieve signal re-routing . Alternatively, an auditory prosthesis such as a cochlear implant can deliver information about sounds directly to the auditory pathway on the side of the impaired ear, thus creating a sensation of true ‘binaural’ hearing .
Existing literature has highlighted inconsistencies in what benefits and risks (side effects) are assessed when evaluating these interventions . The different sorts of benefits and risks are collectively called ‘outcomes’ . For example, researchers have measured aspects or outcomes such as speech understanding in quiet [24,25,26,27,28] or noise [12, 24, 26,27,28,29,30,31,32,33,34,35,36,37,38,39,40,41,42,43,44,45], sound localisation [12, 20, 24, 27,28,29, 32, 36,37,38,39,40,41, 43, 44, 46,47,48,49,50], the impact on the recipients’ quality of life [29, 47, 51] or tinnitus effects [12, 31, 52,53,54,55,56,57,58,59,60]. These inconsistencies in outcomes used in the field of SSD and the variety of methods used to measure them have been identified as a major barrier to synthesising evidence across trials . This diversity in outcomes and instruments used also hinders researchers in making decisions about the choice of outcome measures for health and social care trials of clinical efficacy [23, 62, 63].
The importance of using valid instruments that effectively measure the intended audiological outcomes has been highlighted by Hall et al. . Triallists should ideally base the choice of outcome measures on what is important and of interest to people making decisions about healthcare [65,66,67], not on what outcome instruments are available or most commonly used . If evidence is lacking for an important outcome, this should be acknowledged rather than ignoring the outcome . A core outcome set (COS) developed from the perspectives of healthcare users [69, 70], healthcare professionals and other relevant stakeholders would overcome this problem [71,72,73,74].
A COS is defined by COMET (Core Outcome Measures for Effectiveness Trials) as ‘an agreed minimum set of outcomes or outcome measures’ . A COS comprises a standardised collection of outcome domains that should be measured and reported worldwide, at minimum, in all controlled trials within a research area [63, 76,77,78], as well as a recommended measurement instrument for each outcome domain. An internationally adopted COS allows study findings in a specific health area or condition to be combined, compared and contrasted across trials. It also reduces potential for reporting bias and ensures that the data are useful and usable, which is essential for making well-informed healthcare choices .
One of the earliest examples of an attempt to standardise outcomes is an initiative by the World Health Organisation in the 1970s relating to cancer trials . More recent projects are the IMMPACT (Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials) study for chronic pain , the OMERACT (Outcome Measures in Rheumatoid Arthritis Clinical Trials) consensus initiative for many rheumatologic conditions , the HOME (Harmonizing Outcome Measures for Eczema) framework in dermatology , the GASTROS (Standardising Outcome Reporting in Gastric Cancer Surgery Research) study for reporting outcomes in gastric cancer surgery , and the COMiT’ID (Core Outcome Measures in Tinnitus International Delphi) initiative for chronic subjective tinnitus . It is vital that all stakeholders, such as healthcare users, with lived experience of the condition, as well as healthcare professionals, commercial representatives or budget holders, are involved in the development of relevant COS [23, 85].
The primary aim of the Core Rehabilitation Outcome Set for Single Sided Deafness (CROSSSD) study is to develop an agreed minimum set of outcome domains relevant to both patients and professionals that should be measured and reported in all future trials examining SSD interventions, regardless of whether the intervention restores two-sided (bilateral) access to sound via the better ear or delivers sound information directly to the impaired ear.
The primary objective is as follows:
To develop an international consensus on a COS for SSD interventions using a long-list list of candidate outcomes, a two-round modified electronic Delphi survey and a subsequent face-to-face consensus meeting with relevant stakeholders
To assist in identifying potential measurement instruments for each core outcome domain identified in the first objective, two secondary objectives are as follows:
To synthesise the evidence on the available outcome measurement instruments for measuring the construct outcomes (e.g., speech perception, localisation) in the COS for SSD interventions
To follow up on any issues raised during the Delphi process concerning the concept definition of any of the outcome domains in a subsequent face-to-face workshop involving healthcare users and healthcare professionals
This study will adopt recommendations by the COMET Initiative [63, 72, 86,87,88] and the COMET Handbook version 1.0  '(see Additional file 2 COS-STAP checklist). We will use a modified e-Delphi process to achieve a consensus of opinion among broadly representative and international expert stakeholder groups . This study uses an observational design and is sponsored by the University of Nottingham and managed by the National Institute for Health Research (NIHR) Nottingham Biomedical Research Centre (BRC). A prospective study protocol was registered on the COMET database in January 2018 . This paper describes protocol version 2.0 (dated 6th July 2019) that was approved by the Proportionate Review Nottingham 2 Research Ethics Committee (REC reference 19/EM/0222, IRAS Project ID 239750) on the 6th of August 2019.
Research steering group
A research steering group was appointed in October 2017 to guide the protocol development and oversee the CROSSSD study. The group comprises international colleagues who are experts in SSD research methodologies and intervention approaches (PVH, JBF, IAB); a patient and public involvement (PPI) and engagement manager (AH); two healthcare users with lived experience of SSD, referred to as public research partners (NB, NH); and the study management team (RK, DAH, PTK).
The roles of the research steering group are as follows:
Support the development of the study protocol, specifically commenting on the feasibility of the modified Delphi process, reviewing study documentation (e.g., advertisements, information leaflets, supporting video explanations of the survey and intended advertisements, website content) and participating in a pilot of round 1 of the e-Delphi survey
Review the initial list of outcome domains and associated descriptions, specifically commenting on the readability of the outcome descriptions, the appropriateness of the grouping of outcomes into categories and providing any additional outcomes that they believe should be included in round 1 of the e-Delphi survey
Assist with participant recruitment and engage in dissemination activities, such as contributing to publications
Consider any necessary revisions to the protocol which may inadvertently arise while the study is underway
Eligibility criteria for Delphi panels
A range of expertise within the panel is an important quality criterion for development of a core outcome domain set . Specific inclusion criteria have been defined for three key stakeholder groups:
Healthcare users who have experience of living with SSD for 12 months or more and have received or have considered receiving an SSD intervention
Healthcare practitioners who have a clinical qualification, are currently employed by a public or private institution that provides SSD interventions to patients and have experience of assessing, diagnosing or managing SSD in adults
Clinical researchers who have an academic qualification, are currently employed by a research organisation, have current or ‘recent past’ experience with studies that focus on questions of clinical efficacy (benefit) of SSD interventions in humans (i.e., co-author on a relevant peer-reviewed journal publication in the past 3 years)
Other participants will be invited to participate, including those commercial representatives who are currently employed by a company that develops, manufactures or sells product(s) that may be used as an SSD intervention, as well as funders who are currently employed by an organisation that funds SSD research and have experience of reviewing funding applications for SSD intervention research in the last 3 years. However, we do not anticipate recruiting sufficient numbers from among these stakeholders to form distinct stakeholder groups in their own right, because these pods of stakeholders are small.
General eligibility for participation includes men and women aged 18 years or older who are computer-literate; possess sufficient command of English to read, understand and independently complete the questionnaires; and have the ability to give informed consent. All enrolled e-Delphi panellists will be eligible to register their interest in attending a 1-day face-to-face consensus meeting and/or a follow-up workshop. However, allocation of places will be limited to those respondents who complete both rounds of the e-Delphi survey. None of the research steering group members will be allowed to vote on domains in the consensus meeting, because this risks inadvertently introducing a power differential across participants; however, they can enrol in the e-Delphi surveys.
Panel size and justification
There is no agreed method to statistically calculate a sample size for e-Delphi surveys or consensus meetings . However, one of the key deciding factors is that the participant panel membership should adequately represent corresponding stakeholder groups. Adult SSD is a relatively rare hearing disorder, with approximately 9000 new cases diagnosed in the United Kingdom each year . SSD intervention is also a relatively new field, especially cochlear implantation, which has been used in this population only in the last decade . Therefore, the number of professionals and members of the public with knowledge and experience of these interventions is limited. The aim is therefore to recruit a sufficient number of participants so that a minimum of 20 participants complete the two rounds of the e-Delphi survey in each of the key stakeholder groups (healthcare users, healthcare practitioners and clinical researchers). This target is consistent with our previous work .
The consensus meeting and follow-up workshop require in-depth discussions, and therefore up to 20 participants will be recruited for each. Enrolment will be balanced across stakeholder groups when possible.
Effective recruitment methods similar to the ones described by Hall et al. in 2018  will be used. For example, adopting an explicit marketing plan and engaging with charities or participants to act as ‘champions’ were successful strategies that helped recruitment for both healthcare users and professionals in the COMiT’ID study.
Generally, the CROSSSD study recruitment plan includes e-promotion routes, which include a study webpage (www.nottingham.ac.uk/research/groups/hearingsciences/projects/crosssd/index.aspx) and regular updates on the study’s progress via social media platforms (e.g., Twitter @CROSSSD_, @hearingnihr). A video advertisement promoting the study will also be developed.
Healthcare users will be targeted using various routes, including promotion of the study during the Ménière’s Society Balance Awareness Week (September 2019) and the British Acoustic Neuroma Association annual conference (October 2019). Moreover, healthcare users in the United Kingdom and Ireland will be targeted using a traditional National Health Service (NHS) recruitment route with 18 audiology and ear, nose and throat (ENT) departments whose members specialise in provision of interventions for SSD. These will be designated as participant identification centres (PICs) and will be in addition to the lead site in Nottingham. An application will also be submitted for adoption of the CROSSSD study into the NIHR Clinical Research Network (CRN) portfolio, through which other NHS sites can express their interest to support the study by being a PIC.
PICs will display study posters in the audiology and ENT clinic waiting rooms and hand out participant information leaflets, as appropriate. If feasible, participant invitation letters will be posted by local PIC clinicians to their database of patients diagnosed with SSD. Specific e-promotion routes include several organisations that have agreed to support the project by publishing newsletter articles and announcements to their members (e.g., Manchester Hearing BRC volunteers, Ménière’s Society). Finally, the lead study site, the NIHR Nottingham BRC, has a participant database containing email contacts for approximately 70 healthcare users who have been diagnosed with SSD and will be invited to participate.
For healthcare professional recruitment, we are aiming to recruit experts who maximise the international relevance of the study findings . A number of professional networks and organisations will be approached to circulate invitations to their membership (e.g., British Society of Audiology Adult Rehabilitation Interest Group, HEARRING Network, Hearing Australia). When possible, the expert members of the CROSSSD steering group will be asked to approach their networks to make the approach more personal. Parallel routes for recruiting healthcare professionals will also involve personal invitation via email or face-to-face contact (e.g., presentations at teams’ monthly journal clubs or research meetings).
The study management team has created a long list of potential participants with relevant expertise via existing connections with the NIHR Nottingham BRC, manual searches of relevant hearing-related organisations (e.g., UHealth Ear Institute at the University of Miami), corresponding authors of the relevant research publications identified by the systematic review , manual searches of relevant conference proceedings in the last 3 years (e.g., UK Implantable Acoustic Devices Conference, International Conference on Cochlear Implants and Other Implantable Technologies [Ci2018.org], OSSEO International Congress on Bone Conduction Hearing and Related Technologies), and email queries sent to representatives from each additional stakeholder organisations from commercial sectors (e.g., clinical research managers for relevant device companies) and funding bodies asking for recipients to nominate any colleagues with expertise in SSD interventions.
Eligible professionals will be identified and invited to participate and will be asked a number of questions which will confirm both their stakeholder group (i.e., job role or medical specialty) and that they meet the eligibility criteria for their stakeholder group (see ‘Eligibility criteria for Delphi panels’ section above). Professional stakeholder groups include individuals involved in management or research in the field of SSD. These include healthcare professionals, clinical researchers, commercial representatives, funders, and journal editors. These groups have been identified as those representing the main professional categories in SSD research and clinical trials.
An international Delphi survey will be managed online using DelphiManager software maintained by the COMET Initiative . Each panellist will receive a unique identification code and an e-link to the webpage. A video explanation will illustrate how to use the online tool. A flowchart of the study is shown in Fig. 1.
Delivering this study online allows us to capture the opinions of a diverse population of stakeholders with an interest in shaping outcome measures for SSD interventions. If any healthcare users do not have home access to a computer or a tablet, then they will be offered the option of visiting the NIHR Nottingham BRC to complete the two online rounds using one of the centre’s computers.
The e-Delphi technique can minimise response bias because individual feedback is anonymised and not affected by views of influential individuals . Surveys can also be perceived as intimidating by members of the public as a result of the long number of outcomes included in some e-Delphi surveys that lay participants would have to go through and score at every round . Methodological features highlighted by Smith et al. , such as shortening and renaming the long list of domains and plain language descriptions, will be adhered to when possible. Evaluations discussed by Hall et al.  will be considered to ensure robust recruitment and retention of healthcare users.
Part 1: Preparatory work to generate the long list of candidate outcomes
Potentially important outcomes were first gathered from a systematic review of the literature which identified those outcome domains and outcome instruments reported in studies investigating interventions that seek to restore hearing in adults with SSD , as well as by considering published qualitative data  derived from group interviews examining subjective psychological and social effects of highly asymmetric hearing loss. A workshop with members of the research steering group reviewed this long list of candidate outcomes with the following objectives:
Exclude outcomes that are deemed outside the scope of this COS
Identify any missing outcomes
Consider the choice of language used to define each outcome
Generate plain-language descriptions of each outcome
The CROSSSD study management team collated all primary and secondary outcomes that were identified by the systematic review , which equated to 216 outcome domain terms. In preparation for the workshop, these were categorised into six preliminary groupings: (1) adverse effects or harms, (2) performance in a test situation, (3) patient outcome, (4) resource use, (5) satisfaction and (6) other/cannot code. All individual outcome domains were printed on cards in preparation for a 2-day workshop that took place in July 2019 with members of the research steering group (RK, DAH, AH, PTK) and the two public research partners (NH, NB).
During the workshop, each member first performed an independent rapid review of the individual outcome domains and marked those that they thought did not fit within the scope of core domain set for SSD interventions. It was agreed that an outcome domain would be excluded for not fitting within the scope if all six members unanimously agreed to exclude or there was no more than one dissenting opinion. During this rapid review, 83 outcome domain terms were excluded, and examples of those domains were phrases describing ‘how’ to measure such as thresholds, audiologic, tonotopy, informational masking, cortical changes, and brain activity; or outcome domains that were deemed too broad, generic or ill-defined, such as qualities, hearing, therapy, background noise, mental health, and cognitive distress. Further group discussion led to consolidation of an additional 23 outcome domain terms into a smaller number of outcome domain labels. For example, hearing disability was considered to be synonymous with residual disability, perceived hearing disability, hearing disability at everyday life, and auditory disability. Another 17 outcome domain terms were deemed to be duplicates or descriptions of already-included outcome domains (e.g., subjective assessment of handicap, disability, use, benefit, and satisfaction). This left 93 outcome domains for the long list.
The workshop team next systematically reviewed and discussed the findings published by Lucas et al.  to determine whether qualitative interviews might have identified any other candidate outcome domains. This process added three new outcome domains which had not been assessed explicitly in previous quantitative studies: personal safety (e.g., road safety, independent living), motivation (e.g., to engage in challenging listening situations) and mood (e.g., general sense of well-being). The resulting 96 outcome domains were consolidated further by grouping domains together that were considered by the group to describe the same domain. For example, the outcome domains sound localisation, localisation, localisation performance, azimuthal sound localisation, auditory localisation, localisation ability, source localisation, localisation testing, and ability to judge direction of sound were consolidated into an outcome domain labelled ‘sound localisation (telling where a sound is coming from)’. This consolidation resulted in a final list of 43 outcome domains, which were subsequently organised thematically into ten categories: (1) psychological effects, (2) factors related to the treatment being tested, (3) health-related quality of life, (4) hearing disability, (5) spatial hearing, (6) physical effects, (7) self, (8) sound quality, (9) tinnitus and (10) other effects (Fig. 2).
Some of the outcome domains were the same as had been defined in our previous work on tinnitus  and hearing loss , so we used the same plain-language descriptors when appropriate. Others required plain-language descriptors to be developed through interactive discussion during the workshop.
The long list of outcome domains, labels and plain-language descriptors were subsequently circulated electronically to the CROSSSD study steering group for feedback and cross-checking. This was done to ensure that the outcome domain concepts were explained in ways that are understandable and meaningful to an international audience, especially to those whose first language is not English. The research steering group was also prompted to suggest any missing candidate outcome domains. The study management group used this feedback and, following further revisions to the long list, prepared a final list of 44 candidate outcome domains that will be incorporated into the e-Delphi survey (see Additional file 1 for the final list of outcome domains and their definitions). The randomisation feature of the DelphiManager software (version 4.0) will be used to avoid potential weighting ; that is, presentation of each outcome domain will be randomised as per current recommendations [23, 99].
Part 2: Outcome prioritisation and consensus decision-making
The modified e-Delphi survey comprises a series of two sequential questionnaires or ‘rounds’ aiming to obtain a consensus of opinion from professional and healthcare user stakeholder groups. Each Delphi survey will be managed using a bespoke online e-management system (DelphiManager software, version 4.0) maintained by the COMET Initiative . Both survey rounds will contain a questionnaire that includes the final long list of categorised outcome domains (n = 44) developed in part 1. International healthcare users and professionals with experience in receiving or managing SSD interventions will be identified and invited to take part (see ‘Recruitment methods’ section above for details).
Upon entering the online survey webpage, an introductory page will reiterate key information previously provided in the participant information sheet, including an embedded link to a video explanation. Participants will then be asked to give informed consent, and a unique identification code will be generated to allow tracking of individual responses in round 2. Video explanations will guide participants through the round. Following this, participants will complete a checklist of relevant personal characteristics. These include personal and/or professional experience with SSD interventions; treatments trialled, if applicable (e.g., BAHA, CROS); the group they primarily identify with (e.g., healthcare users, healthcare practitioners, clinical researchers, commercial representatives, funders); age range; gender; country of residence; primary language used for communication; professional role (if applicable); length of SSD diagnosis and interventions primarily used (for healthcare users); and email address.
The survey will be piloted by the study management team and public research partners for face validity, understanding and acceptability. Following this, if needed, modifications will be made before finalising and launching the questionnaire. When the first round of the e-Delphi survey is launched, participant recruitment will commence immediately, and the recruitment period will be for at least 2 months. Participant response rates will be monitored throughout, and the study management team will keep clearly defined records of the number of participants who have completed the rounds and those who have not.
For each of the 44 outcome domains, participants will be asked to think about the importance of each and indicate how important it is to measure when deciding if an intervention is working. Participants will be asked to assign a score to each of the 44 candidate outcome domains. A 9-point Likert scoring system will be used, with a score of 1 to 3 signifying that an outcome domain is of limited importance, 4 to 6 indicating important but not critical, and 7 to 9 meaning critical and important . Participants will be made aware that an outcome domain will be considered for inclusion in the COS only if 70% or more of the participants in each of the stakeholder groups select scores 7–9 on the scale. If a participant feels that (s)he did not understand a particular outcome, (s)he will be able to select ‘unable to score’.
Following each outcome and at the end of the questionnaire, each participant will be offered an open-text box to add any comments about particular outcome domains. This is optional, but participants will be encouraged to provide a reason for their scores on individual outcomes as recommended by the COMET Handbook . These comments will be summarised as part of the feedback after the first round.
In round 1, participants will be able to propose additional outcome domains. These additional outcome domains will be reviewed and coded by the study management team members, with appropriate plain-language concept definitions, to ensure that they represent new items for inclusion in round 2. When uncertainty exists, the research steering group will be consulted, and all new outcome domain terms, concept definitions and category labels will be reviewed. Adhering to current recommendations, reporting of the e-Delphi surveys will describe any new outcomes introduced into the consensus process at the end of round 1, with reasons .
Participants will be eligible to continue to round 2 if they have scored at least 22 (50%) of the outcome domains in round 1. Corresponding data from those participants who responded to fewer than this will be removed. In round 2, all participants will receive the same list of outcomes with feedback tailored according to their key group allocation (healthcare users, healthcare professionals, clinical researchers). Participants who identify themselves as commercial representatives or funders when they register will be considered collectively, and feedback on their scores will be reported separately from the three stakeholder groups.
The purpose of round 2 is to enable participants to reflect on their scores in light of the viewpoint of their stakeholder group and the other stakeholder groups in the e-Delphi survey. Results will be presented graphically as well as numerically to be readily understood by participants. Participants will be asked to re-score the same list of outcome domains, considering this new information. To help give meaning to the 9-point Likert scale , participants will be reminded that individual outcome domains will be considered for inclusion in the COS only if 70% of all participants select scores of 7–9 on the scale. The distribution of the new scores for each outcome domain will then be calculated for each stakeholder group. Other aspects of design and analysis are the same as for round 1. After completion of the second round of the e-Delphi survey, a questionnaire, anticipated to take up to 10 min to complete, will be emailed to all participants to collect feedback on their experience of being a participant.
The aim of the consensus meeting is to integrate healthcare users and professional perspectives on outcomes, as well as to provide final recommendations on an agreed COS for SSD interventions. Participants who have completed the two rounds of the e-Delphi survey, responded to at least 90% of the outcome domains in round 2, and register an interest in participating in the consensus meeting will be eligible to participate. Places will be allocated on a first-come, first-served basis. Recruitment will be guided by methods successfully adopted by Fackrell et al. ; that is, as far as possible, allocated places will maintain a 50/50 balance across healthcare users and professionals and will aim to include non-UK, non–native English language speakers. As far as possible, the COMET guidance for designing an accessible COS consensus meeting will be followed .
After confirming their attendance to the meeting, participants will be sent an email with information on how to get to the meeting and what to expect, as well as the participant information sheet again as a reminder. At the consensus meeting, the research team will discuss with the participants the aim of the meeting and what will happen, ensuring that all participants understand the purpose before consenting and starting the meeting.
An experienced independent moderator will be recruited to facilitate the consensus meeting discussions to agree to a final COS. Discussion within the meeting will include anonymised voting on each outcome as either ‘in’ or ‘out’ (e.g., using electronic keypads which will create histograms and descriptive statistics ‘live’, to be displayed in the meeting). Participants will be given materials summarising the anonymised round 2 results.
Consensus recommendations will be guided by round 2 results. The ‘70/15%’ consensus approach as described by Williamson et al. , and Williamson et al.  and successfully used by Harman et al.  and Hall et al.  will be employed, as follows:
For outcomes recommended to be included on the basis of round 2 analysis (70% scored 7–9), the moderator will establish whether anyone has a major reason to want any to be excluded. The moderator will focus the discussion and voting on these outcomes. Domains will be included if at least 70% of participants vote ‘in’. All other outcomes recommended for inclusion will be ‘in’, without further discussion.
For outcomes in which at least 50% of more than one stakeholder group scored 7–9 on the round 2 analysis, the moderator will focus the discussion and voting. Domains will be included if at least 70% of participants vote ‘in’.
For outcomes in which less than 50% of the participants in all stakeholder groups scored 7–9 on the round 2 analysis, the moderator will establish whether anyone has a major reason to want any to be included. Domains will be included only if at least 70% of participants vote ‘in’.
If consensus is not reached after two rounds of voting, a ‘majority rules’ approach will be applied. Because time for discussion will be limited, there will be no discussion about outcomes whereby the round 2 data meet the criteria for exclusion based on the pre-defined consensus definition.
The final consensus meeting will be audio recorded and transcribed to facilitate reporting. These will be classed as source data and will be retained in the study archives using unique identifier codes for each talker. Reporting of the Delphi surveys will list the outcomes in the final COS .
Finally, we will evaluate the participants’ experience of the consensus meeting using a short evaluation form. This is adapted and modified from the recommended template developed by the COMET Initiative . It is anticipated that completion of this will take approximately 10 min, and completion is entirely voluntary.
This workshop will be convened only if considered necessary by the study management team, a judgement that will be made in consultation with the research steering group. The aim of the workshop would be to discuss in more detail any of the outcome domains that were voted into the core outcome domain set but for which there might have been some unresolved debate about what the exact concept of the domain was or how it was defined in the plain-language descriptor. This will be important underpinning information to have before seeking to identify suitable measurement instruments for each of the outcome domains in the COS.
Compliance in the e-Delphi survey will be defined according to the number of participants completing rounds 1 and 2. Participation within each stakeholder group will be assessed, including (1) numbers who were directly contacted, (2) numbers who registered in the e-Delphi system, (3) numbers enrolled, and (4) numbers completing each round. Similar to the methods used by the mOMEnt (management of Otitis Media with Effusion in children with cleft palate) team , if a reduced number of responders (n < 10) is observed for one or more stakeholder groups, the round 2 Delphi survey will be reviewed and revised. For example, we may consider amalgamating stakeholder groups.
Other analysis will incorporate participant characteristics, such as gender, country, region and native English language speaker (or not). We will analyse the shifts in scores between rounds 1 and 2 for each outcome domain and stakeholder group as a consequence of considering the anonymised feedback from other participants.
Attrition, referring to the percentage of participants who withdraw or drop out between rounds, will be analysed and reported using methods similar to those employed by the mOMEnt team  and the COMiT’ID team . For example, attrition bias which might occur if participants who do not respond in round 2 have different views from their stakeholder group peers who participate in both rounds  will be considered and analysed. To achieve this, methods used by Bruce et al.  can be adopted: Response distributions of withdrawn and completing participants can be drawn. Graphical representations by stakeholder group (healthcare users, healthcare professionals, clinical researchers) can be drawn, too, as presented by Hall et al. , to indicate if attrition bias is likely to have affected the outcome domain recommendations.
Round 2 score distributions for each outcome domain will be considered at the final consensus meeting using a nominal group technique to evaluate individual perspectives. For example, like the methods adopted by Harman et al. , the results of the stakeholder group responses will be compared with the whole group’s response, and percentage agreement will be considered to plan the focus of the consensus meeting . The data derived from the Delphi feedback questionnaire and consensus meeting evaluation form will comprise open-text responses, and these will be analysed using a thematic analytic approach.
The project proposal is registered in the COMET Initiative database . Data derived from the final analysis of the e-Delphi survey, consensus meeting and follow-up workshop will be presented at relevant national and international conferences such as the British Society of Audiology e-conference and the Implantable Acoustic Devices Conference in Oxford (September 2020). Peer-reviewed publications resulting from the research are also planned. We intend to publish the final COS addressing all primary objectives in the summer of 2020. This research will be further disseminated to members of the public and clinicians through specialist magazine articles and support groups. Participants will not be identified in any publications.
This paper describes the design of a Delphi process to develop a COS for SSD interventions, comprising an agreed minimum set of outcome domains relevant to both patients and professionals. This will be applicable to all future trials examining SSD interventions, regardless of whether the intervention restores two-sided (bilateral) access to sound via the better ear or delivers sound information directly to the impaired ear.
During the study protocol development, a question arose whether to consider developing a single COS for the two intervention approaches (re-routing/restoring) or whether to develop separate COSs, one for each intervention approach. The CROSSSD study research steering group was asked to consider the advantages and disadvantages and to help make a decision.
Advantages of considering both intervention approaches together were as follows:
A single COS would set a standard for outcomes that are critical and important for any of the common intervention strategies. This would facilitate comparisons across intervention methods.
A single minimum reporting standard may encourage uptake, minimise the cost and time resources required, and reflect the fact that SSD is a relatively small field in otology with a limited number of clinical trials whose designs and methodological quality are highly variable .
A single Delphi survey improves confidence with regard to adequate numbers of stakeholders recruited internationally to ensure that the decision-making represents a wider view.
Disadvantages of considering both intervention approaches together were as follows:
Outcomes common to both intervention approaches might be less specific to the unique benefits of either intervention. The chosen outcomes might not be optimally sensitive to detecting treatment-related change.
Seeking a single COS might reduce the potential for reaching consensus criteria on individual outcome domains because the different intervention approaches can be very different in their intended effects.
Few participants are likely to have expertise in both intervention approaches.
Two separate Delphi surveys would potentially deliver a stronger message because they would be tailored to individual approaches: re-routing and restoring interventions.
Following thorough consideration of the arguments put forward, it was agreed to proceed with a single consensus process to develop one COS that would be applicable to both intervention approaches. Similar issues have previously been considered by the IMMPACT (Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials) group  and led to the same decision. The IMMPACT team proposed that development of a single core set of domains and measurement procedures would facilitate the comparison and pooling of data while leaving investigators free to augment the core domains with other outcomes of their choice.
To our knowledge, this is the first time that the Delphi technique using these consensus decision-making methods has been used for developing a COS in SSD. This study also heavily incorporates patient and public involvement throughout all stages of the project. An agreement on a set of outcome domains for what is critical and important for deciding whether an intervention is efficacious will drive up the quality and relevance of research by ensuring that the most relevant outcomes are consistently measured and reported in every clinical trial relating to SSD. This would make it much easier for people with SSD and their intervening clinicians to make sense of all the knowledge produced and consequently minimise bias when making decisions about healthcare. This should subsequently lead to improvements in SSD interventions and in turn the management and clinical outcomes of patients with SSD. On the basis of the recommended outcome domains, further research will then be needed to identify measurement instruments that assess the outcome domains in the minimum set.
Availability of data and materials
Upon the completion of the study, supporting data will be available upon request.
Bone-anchored hearing aids
Biomedical Research Centre
Core Outcome Measures for Effectiveness Trials
Core Outcome Measures in Tinnitus, International Delphi
Core outcome set
Clinical Research Network
Contralateral routing of signals
Core Rehabilitation Outcome Set for Single Sided Deafness study
Core Outcome Set-STAndardised Protocol
Ear, nose, and throat
Standardising Outcome Reporting in Gastric Cancer Surgery Research
Harmonising Outcome Measures for Eczema
Initiative on Methods, Measurement, and Pain Assessment in Clinical Trials
Management of Otitis Media with Effusion in children with cleft palate
National Health Service
National Institute for Health Research
Outcome Measures in Rheumatoid Arthritis Clinical Trials
Participant identification centre
Patient and public involvement
van de Heyning P, Tavora-Vieira D, Mertens G, Van Rompaey V, Rajan GP, Muller J, Hempel JM, Leander D, Polterauer D, Marx M, Usami SI, Kitoh R, Miyagawa M, Moteki H, Smilsky K, Baumgartner WD, Keintzel TG, Sprinzl GM, Wolf-Magele A, Arndt S, Wesarg T, Zirn S, Baumann U, Weissgerber T, Rader T, Hagen R, Kurz A, Rak K, Stokroos R, George E, Polo R, Medina MDM, Henkin Y, Hilly O, Ulanovski D, Rajeswaran R, Kameswaran M, Di Gregorio MF, Zernotti ME. Towards a unified testing framework for single-sided deafness studies: a consensus paper. Audiol Neurootol. 2016;21(6):391–8.
Daniels RL, Swallow C, Shelton C, Davidson HC, Krejci CS, Harnsberger HR. Causes of unilateral sensorineural hearing loss screened by high-resolution fast spin echo magnetic resonance imaging: review of 1,070 consecutive cases. Otol Neurotol. 2000;21(2):173–80.
Staecker H, Nadol JB Jr, Ojeman R, Ronner S, McKenna MJ. Hearing preservation in acoustic neuroma surgery: middle fossa versus retrosigmoid approach. Otol Neurotol. 2000;21(3):399–404.
Wu Q, Li X, Sha Y, Dai C. Clinical features and management of Meniere’s disease patients with drop attacks. Eur Arch Otorhinolaryngol. 2019;276(3):665–72.
Schreiber BE, Agrup C, Haskard DO, Luxon LM. Sudden sensorineural hearing loss. Lancet. 2010;375(9721):1203–11.
Baguley DM, Bird J, Humphriss RL, Prevost AT. The evidence base for the application of contralateral bone anchored hearing aids in acquired unilateral sensorineural hearing loss in adults. Clin Otolaryngol. 2006;31(1):6–14.
Dwyer NY, Firszt JB, Reeder RM. Effects of unilateral input and mode of hearing in the better ear: self-reported performance using the speech, spatial and qualities of hearing scale. Ear Hear. 2014;35(1):126–36.
Hawley ML, Litovsky RY, Culling JF. The benefit of binaural hearing in a cocktail party: effect of location and type of interferer. J Acoust Soc Am. 2004;115(2):833–43.
Levitt HR, Rabiner L. Predicting binaural gain in intelligibility and release from masking for speech. J Acoust Soc Am. 1967;42(4):820–9.
Agterberg MJ, Snik AF, Hol MK, Van Wanrooij MM, Van Opstal AJ. Contribution of monaural and binaural cues to sound localization in listeners with acquired unilateral conductive hearing loss: improved directional hearing with a bone-conduction device. Hear Res. 2012;286(1–2):9–18.
Hoth S, Rosli-Khabas M, Herisanu I, Plinkert PK, Praetorius M. Cochlear implantation in recipients with single-sided deafness: audiological performance. Cochlear Implants Int. 2016;17(4):190–9.
Arndt S, Aschendorff A, Laszig R, Beck R, Schild C, Kroeger S, Ihorst G, Wesarg T. Comparison of pseudobinaural hearing to real binaural hearing rehabilitation after cochlear implantation in patients with unilateral deafness and tinnitus. Otol Neurotol. 2011;32(1):39–47.
Harford E, Barry J. A rehabilitative approach to the problem of unilateral hearing impairment: the contralateral routing of signals. J Speech Hear Disord. 1965;30:121–38.
Giolas TG, Wark DJ. Communication problems associated with unilateral hearing loss. J Speech Hear Disord. 1967;32(4):336–43.
Welsh LW, Welsh JJ, Rosen LF, Dragonette JE. Functional impairments due to unilateral deafness. Ann Otol Rhinol Laryngol. 2004;113(12):987–93.
Lucas L, Katiri R, Kitterick PT. The psychological and social consequences of single-sided deafness in adulthood. Int J Audiol. 2018;57(1):21–30.
Carlsson PI, Hall M, Lind KJ, Danermark B. Quality of life, psychosocial consequences, and audiological rehabilitation after sudden sensorineural hearing loss. Int J Audiol. 2011;50(2):139–44.
Kitterick PT, Lucas L, Smith SN. Improving health-related quality of life in single-sided deafness: a systematic review and meta-analysis. Audiol Neurootol. 2015;20(1):79–86.
Peters JP, Smit AL, Stegeman I, Grolman W. Review: bone conduction devices and contralateral routing of sound systems in single-sided deafness. Laryngoscope. 2015;125(1):218–26.
Leterme G, Bernardeschi D, Bensemman A, Coudert C, Portal JJ, Ferrary E, Sterkers O, Vicaut E, Frachet B, Grayeli AB. Contralateral routing of signal hearing aid versus transcutaneous bone conduction in single-sided deafness. Audiol Neurootol. 2015;20(4):251–60.
Lin LM, Bowditch S, Anderson MJ, May B, Cox KM, Niparko JK. Amplification in the rehabilitation of unilateral deafness: speech in noise and directional hearing effects with bone-anchored hearing and contralateral routing of signal amplification. Otol Neurotol. 2006;27(2):172–82.
van de Heyning P, Rajan G, Arndt S, Skarzynski P. Extended indications of cochlear implantation [abstract]. J Laryngol Otol. 2016;130(Suppl 3):S40.
Williamson PR, Altman DG, Bagley H, Barnes KL, Blazeby JM, Brookes ST, Clarke M, Gargon E, Gorst S, Harman N, Kirkham JJ, McNair A, Prinsen CAC, Schmitt J, Terwee CB, Young B. The COMET Handbook: version 1.0. Trials. 2017;18(Suppl 3):280.
Niparko JK, Cox KM, Lustig LR. Comparison of the bone anchored hearing aid implantable hearing device with contralateral routing of offside signal amplification in the rehabilitation of unilateral deafness. Otol Neurotol. 2003;24(1):73–8.
Wazen JJ, Spitzer JB, Ghossaini SN, Fayad JN, Niparko JK, Cox K, Brackmann DE, Soli SD. Transcranial contralateral cochlear stimulation in unilateral deafness. Otolaryngol Head Neck Surg. 2003;129(3):248–54.
Desmet JB, Wouters K, De Bodt M, van de Heyning P. Comparison of 2 implantable bone conduction devices in patients with single-sided deafness using a daily alternating method. Otol Neurotol. 2012;33(6):1018–26.
Jacob R, Stelzig Y, Nopp P, Schleich P. Audiological results with cochlear implants for single-sided deafness [in German]. HNO. 2011;59(5):453–60.
Firszt JB, Holden LK, Reeder RM, Waltzman SB, Arndt S. Auditory abilities after cochlear implantation in adults with unilateral deafness: a pilot study. Otol Neurotol. 2012;33(8):1339–46.
Härkönen K, Kivekäs I, Kotti V, Sivonen V, Vasama JP. Hybrid cochlear implantation: quality of life, quality of hearing, and working performance compared to patients with conventional unilateral or bilateral cochlear implantation. Eur Arch Otorhinolaryngol. 2017;274(10):3599–604.
Krempaska S, Koval J, Schmid C, Pfiffner F, Kurz A, Kompis M. Influence of directionality and maximal power output on speech understanding with bone anchored hearing implants in single sided deafness. Eur Arch Otorhinolaryngol. 2014;271(6):1395–400.
Mertens G, Kleine Punte A, De Ridder D, Van de Heyning P. Tinnitus in a single-sided deaf ear reduces speech reception in the nontinnitus ear. Otol Neurotol. 2013;34(4):662–6.
Grossmann W, Brill S, Moeltner A, Mlynski R, Hagen R, Radeloff A. Cochlear implantation improves spatial release from masking and restores localization abilities in single-sided deaf patients. Otol Neurotol. 2016;37(6):658–64.
Pfiffner F, Kompis M, Flynn M, Asnes K, Arnold A, Stieger C. Benefits of low-frequency attenuation of Baha® in single-sided sensorineural deafness. Ear Hear. 2011;32(1):40–5.
Popelka GR. SoundBite hearing system by Sonitus Medical: a new approach to single-sided deafness. Semin Hear. 2010;31(4):393–409.
Laske RD, Roosli C, Pfiffner F, Veraguth D, Huber AM. Functional results and subjective benefit of a transcutaneous bone conduction device in patients with single-sided deafness. Otol Neurotol. 2015;36(7):1151–6.
Moore BC, Popelka GR. Preliminary comparison of bone-anchored hearing instruments and a dental device as treatments for unilateral hearing loss. Int J Audiol. 2013;52(10):678–86.
Snapp HA, Hoffer ME, Xuezhong L, Rajguru SM, Liu X. Effectiveness in rehabilitation of current wireless CROS technology in experienced bone-anchored implant users. Otol Neurotol. 2017;38(10):1397–404.
Snapp HA, Holt FD, Liu X, Rajguru SM. Comparison of speech-in-noise and localization benefits in unilateral hearing loss subjects using contralateral routing of signal hearing aids or bone-anchored implants. Otol Neurotol. 2017;38(1):11–8.
Weber BA, Roush J, McElveen JT. Application of an implantable bone conduction hearing device to patients with unilateral sensorineural hearing loss. Laryngoscope. 1992;102(5):538–42.
Arndt S, Laszig R, Aschendorff A, Hassepass F, Beck R, Wesarg T. Cochlear implant treatment of patients with single-sided deafness or asymmetric hearing loss - German version [in German]. HNO. 2017;65(7):586–98.
Döge J, Baumann U, Weissgerber T, Rader T. Single-sided deafness: impact of cochlear implantation on speech perception in complex noise and on auditory localization accuracy. Otol Neurotol. 2017;38(10):e563–9.
Linstrom CJ, Silverman CA, Yu GP. Efficacy of the bone-anchored hearing aid for single-sided deafness. Laryngoscope. 2009;119(4):713–20.
Newman CW, Sandridge SA, Wodzisz LM. Longitudinal benefit from and satisfaction with the Baha system for patients with acquired unilateral sensorineural hearing loss. Otol Neurotol. 2008;29(8):1123–31.
Saliba I, Nader ME, El Fata F, Leroux T. Bone anchored hearing aid in single sided deafness: outcome in right-handed patients. Auris Nasus Larynx. 2011;38(5):570–6.
Schafer EC, Baldus N, d’Souza M, Algier K, Whiteley P, Hill M. Behavioral and subjective performance with digital CROS/BiCROS hearing instruments. J Acad Rehabil Audiol. 2013;46:62–93.
Grantham DW, Ashmead DH, Haynes DS, Hornsby BWY, Labadie RF, Ricketts TA. Horizontal plane localization in single-sided deaf adults fitted with a bone-anchored hearing aid (Baha). Ear Hear. 2012;33(5):595–603.
Ryu NG, Moon IJ, Byun H, Jin SH, Park H, Jang KS, Cho YS. Clinical effectiveness of wireless CROS (contralateral routing of offside signals) hearing aids. Eur Arch Otorhinolaryngol. 2015;272(9):2213–9.
Távora-Vieira D, De Ceulaer G, Govaerts PJ, Rajan GP. Cochlear implantation improves localization ability in patients with unilateral deafness. Ear Hear. 2015;36(3):e93–8.
Bosman AJ, Hol MKS, Snik AFM, Mylanus EAM, Cremers CWRJ. Bone-anchored hearing aids in unilateral inner ear deafness. Acta Otolaryngol. 2003;123(2):258–60.
Wazen JJ, Ghossaini SN, Spitzer JB, Kuller M. Localization by unilateral BAHA users. Otolaryngol Head Neck Surg. 2005;132(6):928–32.
Schmerber S, Deguine O, Marx M, Van de Heyning P, Sterkers O, Mosnier I, Garin P, Godey B, Vincent C, Venail F, Mondain M, Deveze A, Lavieille JP, Karkas A. Safety and effectiveness of the Bonebridge transcutaneous active direct-drive bone-conduction hearing implant at 1-year device use. Eur Arch Otorhinolaryngol. 2017;274(4):1835–51.
Härkönen K, Kivekäs I, Rautiainen M, Kotti V, Sivonen V, Vasama JP. Single-sided deafness: the effect of cochlear implantation on quality of life, quality of hearing, and working performance. ORL J Otorhinolaryngol Relat Spec. 2015;77(6):339–45.
Kleine Punte A, Vermiere K, Hofkens A, De Bodt M, De Ridder D, Van de Heyning P. Cochlear implantation as a durable tinnitus treatment in single-sided deafness. Cochlear Implants Int. 2011;12(Suppl 1):S26–9.
Kleine Punte A, De Ridder D, Van de Heyning P. On the necessity of full length electrical cochlear stimulation to suppress severe tinnitus in single-sided deafness. Hear Res. 2013;295:24–9.
Song JJ, Kim K, Sunwoo W, Mertens G, Van de Heyning P, De Ridder D, Vanneste S, Lee SY, Park KJ, Choi H, Choi JW. A quantitative electroencephalography study on cochlear implant-induced cortical changes in single-sided deafness with tinnitus. Front Hum Neurosci. 2017;11:210 A published erratum appears in Front Hum Neurosci. 2018;12:46.
Song JJ, Kleine Punte A, De Ridder D, Vanneste S, Van de Heyning P. Neural substrates predicting improvement of tinnitus after cochlear implantation in patients with single-sided deafness. Hear Res. 2013;299:1–9.
Távora-Vieira D, Marino R, Acharya A, Rajan GP. The impact of cochlear implantation on speech understanding, subjective hearing performance, and tinnitus perception in patients with unilateral severe to profound hearing loss. Otol Neurotol. 2015;36(3):430–6.
Indeyeva YA, Diaz A, Imbrey T, Gao G, Coelho DH. Tinnitus management with percutaneous osseointegrated auditory implants for unilateral sensorineural hearing loss. Am J Otolaryngol. 2015;36(6):810–3.
Holder JT, O’Connell B, Hedley-Williams A, Wanna G. Cochlear implantation for single-sided deafness and tinnitus suppression. Am J Otolaryngol. 2017;38(2):226–9.
Mertens G, De Bodt M, Van de Heyning P. Cochlear implantation as a long-term treatment for ipsilateral incapacitating tinnitus in subjects with unilateral hearing loss up to 10 years. Hear Res. 2016;331:1–6.
Kitterick PT, Smith SN, Lucas L. Hearing instruments for unilateral severe-to-profound sensorineural hearing loss in adults: a systematic review and meta-analysis. Ear Hear. 2016;37(5):495–507.
Clarke M, Williamson PR. Core outcome sets and systematic reviews. Syst Rev. 2016;5:11.
Williamson P, Clarke M. The COMET (Core Outcome Measures in Effectiveness Trials) initiative: its role in improving Cochrane reviews. Cochrane Database Syst Rev. 2012;(5):ED000041. https://doi.org/10.1002/14651858.ED000041.
Hall DA, Kitterick PT, Heffernan E, Fackrell K, Lucas L, Ferguson M. How do we know that our patients have benefitted from our ENT/audiological interventions? Presented at the annual meeting of ADANO 2016 in Berlin. Otol Neurotol. 2019;40(4):e474–81.
Clarke M. Standardising outcomes for clinical trials and systematic reviews. Trials. 2007;8:39.
Sinha IP, Williamson PR, Smyth RL. Outcomes in clinical trials of inhaled corticosteroids for children with asthma are narrowly focussed on short term disease activity. PLoS One. 2009;4(7):e6276.
Gandhi GY, Murad MH, Fujiyoshi A, Mullan RJ, Flynn DN, Elamin MB, Swiglo BA, Isley WL, Guyatt GH, Montori VM. Patient-important outcomes in registered diabetes trials. JAMA. 2008;299(21):2543–9.
Gargon E, Gurung B, Medley N, Altman DG, Blazeby JM, Clarke M, Williamson PR. Choosing important health outcomes for comparative effectiveness research: a systematic review. PLoS One. 2014;9(6):e99111.
Serrano-Aguilar P, Trujillo-Martin MM, Ramos-Goni JM, Mahtani-Chugani V, Perestelo-Perez L. Posada-de la Paz M. Patient involvement in health research: a contribution to a systematic review on the effectiveness of treatments for degenerative ataxias. Soc Sci Med. 2009;69(6):920–5.
Kirwan JR, Hewlett SE, Heiberg T, Hughes RA, Carr M, Hehir M, Kvien TK, Minnock P, Newman SP, Quest EM, Taal E, Wale J. Incorporating the patient perspective into outcome assessment in rheumatoid arthritis -progress at OMERACT 7. J Rheumatol. 2005;32(11):2250–6.
Williamson PR, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E, Tugwell P. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13:132.
Sinha IP, Smyth RL, Williamson PR. Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med. 2011;8(1):e1000393.
Mease PJ, Arnold LM, Crofford LJ, Williams DA, Russell IJ, Humphrey L, Abetz L, Martin SA. Identifying the clinical domains of fibromyalgia: contributions from clinician and patient Delphi exercises. Arthritis Rheum. 2008;59(7):952–60.
Sinha IP, Gallagher R, Williamson PR, Smyth RL. Development of a core outcome set for clinical trials in childhood asthma: a survey of clinicians, parents, and young people. Trials. 2012;13:103.
Core Outcome Measures in Effectiveness Trials (COMET). Glossary: Core outcome set (COS). http://www.comet-initiative.org/. Accessed 23 Oct 2019.
Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG. CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869.
Chan AW, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin JA, Dickersin K, Hrobjartsson A, Schulz KF, Parulekar WR, Krleza-Jeric K, Laupacis A, Moher D. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. BMJ. 2013;346:e7586.
Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes B, Oxman AD, Moher D. Improving the reporting of pragmatic trials: an extension of the CONSORT statement. BMJ. 2008;337:a2390.
Miller AB, Hoogstraten B, Staquet M, Winkler A. Reporting results of cancer treatment. Cancer. 1981;47(1):207–14.
Dworkin RH, Turk DC, Farrar JT, Haythornthwaite JA, Jensen MP, Katz NP, Kerns RD, Stucki G, Allen RR, Bellamy N, Carr DB, Daniel B, Chandler J, Cowan P, Dionne R, Galer BS, Hertz S, Jadad AR, Kramer LD, Manning DC, Martin S, McCormick CG, McDermott MP, McGrath P, Quessy S, Rappaport BA, Robbins W, Robinson JP, Rothman M, Royal MA, Simon L, Stauffer JW, Stein W, Tollett J, Wernicke J, Witter J. Core outcome measures for chronic pain clinical trials: IMMPACT recommendations. Pain. 2005;113(1):9–19.
Boers M, Kirwan JR, Wells G, Beaton D, Gossec L, d’Agostino MA, Conaghan PG, Bingham CO 3rd, Brooks P, Landewé R, March L, Simon LS, Singh JA, Strand V, Tugwell P. Developing core outcome measurement sets for clinical trials: OMERACT Filter 2.0. J Clin Epidemiol. 2014;67(7):745–53.
Schmitt J, Apfelbacher C, Spuls PI, Thomas KS, Simpson EL, Furue M, Chalmers J, Williams HC. The Harmonizing Outcome Measures for Eczema (HOME) roadmap: a methodological framework to develop core sets of outcome measurements in dermatology. J Invest Dermatol. 2015;135(1):24–30.
Alkhaffaf B, Glenny AM, Blazeby JM, Williamson P, Bruce IA. Standardising the reporting of outcomes in gastric cancer surgery trials: protocol for the development of a core outcome set and accompanying outcome measurement instrument set (the GASTROS study). Trials. 2017;18:370.
Hall DA, Smith H, Hibbert A, Colley V, Haider HF, Horobin A, Londero A, Mazurek B, Thacker B, Fackrell K, Core Outcome Measures in Tinnitus (COMiT) initiative. The COMiT’ID study: developing core outcome domains sets for clinical trials of sound-, psychology-, and pharmacology-based interventions for chronic subjective tinnitus in adults. Trends Hear. 2018;22:2331216518814384..
Hall DA, Smith H, Heffernan E, Fackrell K, Core Outcome Measures in Tinnitus International Delphi (COMiT’ID) Research Steering Group. Recruiting and retaining participants in e-Delphi surveys for core outcome set development: evaluating the COMiT’ID study. PLoS One. 2018;13(7):e0201378..
Brookes ST, Macefield RC, Williamson PR, McNair AG, Potter S, Blencowe NS, Strong S, Blazeby JM. Three nested randomized controlled trials of peer-only or multiple stakeholder group feedback within Delphi surveys during core outcome and information set development. Trials. 2016;17:409..
Kirkham JJ, Gorst S, Altman DG, Blazeby J, Clarke M, Devane D, Gargon E, Williamson PR. COS-STAR: a reporting guideline for studies developing core outcome sets. Trials. 2015;16:373..
Kirkham JJ, Davis K, Altman DG, Blazeby JM, Clarke M, Tunis S, Williamson PR. Core Outcome Set-STAndards for Development: the COS-STAD recommendations. PLoS Med. 2017;14(11):e1002447.
Keeley T, Williamson P, Callery P, Jones LL, Mathers J, Jones J, Young B, Calvert M. The use of qualitative methods to inform Delphi surveys in core outcome set development. Trials. 2016;17:230.
Core Outcome Measures in Effectiveness Trials (COMET). Towards a consensus on outcome measures for interventions that seek to restore bilateral and binaural hearing in adults with unilateral severe-to-profound hearing loss: the CROSSSD (Core Rehabilitation Outcome Set for Single Sided Deafness) study. http://www.comet-initiative.org/studies/details/1084?result=true. Accessed 23 Oct 2019.
Van de Heyning P, Vermeire K, Diebl M, Nopp P, Anderson I, De Ridder D. Incapacitating unilateral tinnitus in single-sided deafness treated by cochlear implantation. Ann Otol Rhinol. 2008;117(9):645–52.
Core Outcome Measures in Effectiveness Trials (COMET). DelphiManager. http://www.comet-initiative.org/delphimanager/. Accessed 25 Oct 2019.
Keeney S, Hasson F, McKenna HP. A critical review of the Delphi technique as a research methodology for nursing. Int J Nurs Stud. 2001;38(2):195–200.
Young B, Bagley H. Including patients in core outcome set development: issues to consider based on three workshops with around 100 international delegates. Res Involv Engagem. 2016;2:25.
Smith H, Horobin A, Fackrell K, Colley V, Thacker B, Hall DA, Core Outcome Measures in Tinnitus (COMiT) initiative. Defining and evaluating novel procedures for involving patients in Core Outcome Set research: creating a meaningful long list of candidate outcome domains. Res Involv Engagem. 2018;4:8.
Hall DA, Kitterick PT, Katiri, R. Systematic review of outcome domains and instruments used in designs of clinical trials for interventions that seek to restore bilateral and binaural hearing in adults with unilateral sensorineural severe to profound hearing loss ('Single Sided Deafness'). PROSPERO: International prospective register of systematic reviews. 2018. Available from: https://www.crd.york.ac.uk/prospero/display_record.php?RecordID=84274.
Vas V, Akeroyd MA, Hall DA. A data-driven synthesis of research evidence for domains of hearing loss, as reported by adults with hearing loss and their communication partners. Trends Hear. 2017;21:2331216517734088.
Blackwood B, Ringrow S, Clarke M, Marshall J, Rose L, Williamson PR, McAuley D. Core Outcomes in Ventilation Trials (COVenT): protocol for a core outcome set using a Delphi survey with a nested randomised trial and observational cohort study. Trials. 2015;16:368.
Gargon E, Crew R, Burnside G, Williamson PR. Higher number of items associated with significantly lower response rates in COS Delphi surveys. J Clin Epidemiol. 2019;108:110–20.
Guyatt GH, Oxman AD, Kunz R, Atkins D, Brozek J, Vist G, Alderson P, Glasziou P, Falck-Ytter Y, Schunemann HJ. GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin Epidemiol. 2011;64(4):395–400.
Kirkham JJ, Gorst S, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E, Moher D, Schmitt J, Tugwell P, Tunis S, Williamson PR. Core Outcome Set–STAndards for Reporting: the COS-STAR statement. PLoS Med. 2016;13(10):e1002148.
Fackrell K, Smith H, Colley V, Thacker B, Horobin A, Haider HF, Londero A, Mazurek B, Hall DA. Core outcome domains for early phase clinical trials of sound-, psychology-, and pharmacology-based interventions to manage chronic subjective tinnitus in adults: the COMIT’ID study protocol for using a Delphi process and face-to-face meetings to establish consensus. Trials. 2017;18:388.
Core Outcome Measures in Effectiveness Trials (COMET). Tips for designing an accessible core outcome set consensus meeting. http://www.comet-initiative.org/assets/downloads/Tips%20for%20Designing%20an%20Accessible%20Core%20Outcome%20Set%20Consensus%20Meeting%20final%2026-10-17.pdf. Accessed 26 Oct 2019.
Harman NL, Bruce IA, Kirkham JJ, Tierney S, Callery P, O’Brien K, Bennett AM, Chorbachi R, Hall PN, Harding-Bell A, Parfect VH, Rumsey N, Sell D, Sharma R, Williamson PR. The importance of integration of stakeholder views in core outcome set development: otitis media with effusion in children with cleft palate. PLoS One. 2015;10(6):e0129514..
Core Outcome Measures in Effectiveness Trials (COMET). Evaluating participant experience of consensus meetings. http://www.comet-initiative.org/assets/downloads/exampleevaluationformCOS.docx. Accessed 26 Oct 2019..
Harman NL, Bruce IA, Callery P, Tierney S, Sharif MO, O’Brien K, Williamson PR. MOMENT – Management of Otitis Media with Effusion in Cleft Palate: protocol for a systematic review of the literature and identification of a core outcome set using a Delphi survey. Trials. 2013;14:70.
Bruce I, Harman N, Williamson P, Tierney S, Callery P, Mohiuddin S, Payne K, Fenwick E, Kirkham J, O’Brien K. The management of Otitis Media with Effusion in children with cleft palate (mOMEnt): a feasibility study and economic evaluation. Health Technol Assess. 2015;19(68):1–374.
Hall DA, Hibbert A, Smith H, Haider HF, Londero A, Mazurek B, Fackrell K, Core Outcome Measures in Tinnitus (COMiT) initiative. One size does not fit all: developing common standards for outcomes in early-phase clinical trials of sound-, psychology-, and pharmacology-based interventions for chronic subjective tinnitus in adults. Trends Hear. 2019;23:2331216518824827.
Turk DC, Dworkin RH. What should be the core outcomes in chronic pain clinical trials? Arthritis Res Ther. 2004;6(4):151–4.
An independent review of the protocol has been conducted by Professor Paula R. Williamson, University of Liverpool, as part of the research ethics committee approval process. The CROSSSD study group acknowledges the support of the National Institute of Health Research Clinical Research Network (NIHR CRN) in participant recruitment. We acknowledge contributions from PhD students Catherine Killan and Pattarawadee Prayuenyong and research assistant Sandra Smith at the NIHR Nottingham BRC for their help with the systematic review. Many thanks to Henryk Faas, Assistant Professor in Neuroimaging, Sir Peter Mansfield Imaging Centre, School of Medicine, University of Nottingham, who has assisted with the systematic review of publications in German. The views expressed in this article are those of the author(s) and not necessarily those of the NHS, the NIHR, or the Department of Health and Social Care.
At the time of manuscript submission, the e-Delphi survey was open to recruitment as per revised and approved final study protocol version 2.0 (dated 6th July 2019). Study recruitment commenced 9th of September 2019. At the time of proof review, the e-Delphi survey was still open to recruitment with study completion anticipated around the end of April 2020.
The main body of work for this project is funded by the National Institute for Health Research (NIHR) Nottingham Biomedical Research Centre (BRC), funding reference number BRC-1215-20003. Additional grants obtained are as follows: Graham Fraser Foundation Travel Grant to attend the 15th International Conference on Cochlear Implants and Other Implantable Auditory Technology, where the study was first launched. Oticon Medical™ provided funding to purchase the DelphiManager software from the COMET Initiative, University of Liverpool. DAH is an NIHR Senior Investigator. The funding bodies had no role in the study design and implementation, writing of the report, or the decision to submit the report for publication.
Ethics approval and consent to participate
Ethics approval and consent to participate will be sought for each aspect of the study for which ethical approval is required. Ethical approval has been authorised by the Nottingham 2 Research Ethics Committee (IRAS Project ID 239750), Health Research Authority and Health and Care Research Wales (Reference 19/EM/0222). Informed consent will be obtained from all study participants as per approved final study protocol version 2.0 (dated 6th July 2019).
Consent for publication
PTK declares receiving research grants and support in kind awarded to his institution for research projects in the field of SSD from manufacturers of hearing devices. All other authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The original version of this article was revised: a typo in the third bullet point of the Consensus Criteria section has been corrected
Table of domain categories, SSD intervention-related outcome domains, and concept definitions for all 44 outcome domains co-produced with collaborators with lived experience of SSD at the pre-Delphi stage.
Core Outcome Set-STAndardised Protocol Items (COS-STAP) Checklist.
About this article
Cite this article
Katiri, R., Hall, D.A., Buggy, N. et al. Core Rehabilitation Outcome Set for Single Sided Deafness (CROSSSD) study: protocol for an international consensus on outcome measures for single sided deafness interventions using a modified Delphi survey. Trials 21, 238 (2020). https://doi.org/10.1186/s13063-020-4094-9
- Consensus methods
- Core outcome set
- Delphi technique
- Single-sided deafness