- Study protocol
- Open Access
A protocol for the Development of Core Outcome Sets for Endodontic Treatment modalities (COSET): an international consensus process
Trials volume 22, Article number: 812 (2021)
The outcome of endodontic treatment is generally assessed using a range of patient and clinician-centred, non-standardised clinical and radiographic outcome measures. This makes it difficult to synthesise evidence for systematic analysis of the literature and the development of clinical guidelines. Core outcome sets (COS) represent a standardised list of outcomes that should be measured and reported in all clinical studies in a particular field. Recently, clinical researchers and guideline developers have focussed on the need for the integration of a patient-reported COS with clinician-centred measures. This study aims to develop a COS that includes both patient-reported outcomes and clinician-centred measures for various endodontic treatment modalities to be used in clinical research and practice.
To identify reported outcomes (including when and how they are measured), systematic reviews and their included clinical studies, which focus on the outcome of endodontic treatment and were published between 1990 and 2020 will be screened. The COSs will be defined by a consensus process involving key stakeholders using semi-structured interviews and an online Delphi methodology followed by an interactive virtual consensus meeting. A heterogeneous group of key ‘stakeholders’ including patients, general dental practitioners, endodontists, endodontic teachers, clinical researchers, students and policy-makers will be invited to participate. Patients will establish, via interactive interviews, which outcomes they value and feel should be included in a COS. In the Delphi process, other stakeholders will be asked to prioritise outcomes identified from the literature and patient interviews and will have the opportunity at the end of the first round to add outcomes that are not included, but which they consider relevant. Feedback will be provided in the second round, when participants will be asked to prioritise the list again. If consensus is reached, the remaining outcomes will be discussed at an online meeting and agreement established via defined consensus rules of outcome inclusion. If consensus is not reached after the second round, a third round will be conducted with feedback, followed by the online meeting. Following the identification of a COS, we will proceed to identify how and when these outcomes are measured.
Using a rigorous methodology, the proposed consensus process aims to develop a COS for endodontic treatment that will be relevant to stakeholders. The results of the study will be shared with participants and COS users. To increase COS uptake, it will also be actively shared with clinical guideline developers, research funders and the editors of general dental and endodontology journals.
COMET 1879. 21 May 2021.
Endodontic treatment aims to retain teeth, by maintaining pulp vitality and/or health of periapical tissues by preventing and treating infection, and comprises a fundamental part of everyday dental practice. The scope of the treatment includes, but is not limited to, non-surgical and surgical root canal treatment (RCTx) and vital pulp treatment (VPT).
RCTx is usually indicated in teeth in which the dental pulp is severely inflamed (irreversible pulpitis) or when the pulp becomes necrotic with or without infection, but occasionally elective removal of healthy pulp tissue may be indicated for restorative reasons . The treatment involves removing infection primarily from the root canal system, by cleaning, shaping and filling of the root canals. Although complex, the treatment is highly successful when performed to a high standard [2, 3]. The outcome of RCTx is generally assessed by clinical and radiographic examination using a range of criteria. These assessment tools are largely clinician-centred, rather than patient-centred. If primary RCTx fails, teeth can be retreated conventionally or surgically. The outcomes of retreatment are assessed in the same way as primary treatment, with clinical and radiographic examination using various criteria . Some studies reporting outcomes of surgical and non-surgical RCTx use clinical and radiographic criteria individually or as a combined outcome [2, 5]. Others have reported tooth survival or need for further treatment as the outcome . Within the clinical criteria, there is often marked variation in the way clinical outcomes are reported. Radiographic examination is often carried out using conventional imaging, but recently outcomes are also reported using cone beam tomography (CBCT) . There is marked heterogeneity in the reporting of outcomes in published studies and a lack of consistency in reporting the outcomes of randomised controlled trials (RCT).
VPT is a re-emerging minimally invasive endodontic treatment that aims to maintain the vitality of all or part of the dental pulp. There are two main VPT modalities for treating teeth with deep caries and compromised pulp; pulp capping and pulpotomy. Pulp capping (direct or indirect) is a type of VPT generally indicated for teeth with deep caries in which the pulp is not pathologically exposed or is mildly symptomatic . Pulpotomy is indicated for teeth in which the pulp is cariously exposed, and the procedure involves removal of a small part of the inflamed pulp (partial pulpotomy) or amputation of the coronal pulp (complete pulpotomy) depending on the extent of the inflammation and patient symptoms . The outcome of VPT depends on the treatment modality and, as with RCTx, various outcomes including clinical and radiographic outcomes are reported to indicate its success, but with a lack of standardised outcome measures.
A biologically based endodontic therapy known as revascularization or revitalization that aims to promote normal physiological development in immature permanent teeth with pulpal necrosis is an emerging endodontic treatment modality . Like other endodontic treatments, various outcomes are reported for revascularisation procedures that are mainly clinician focussed with no consensus .
The absence of standardised outcomes is reflected in the quality of many systematic reviews reporting the effects of endodontic treatment on permanent teeth [2, 11, 12] and has been acknowledged in the European Society of Endodontology (ESE) position statement on the management of deep caries and exposed pulp . For RCTx, the principal outcome reported is periapical health, diagnosed clinically and radiographically. The same also applies to surgical endodontics where healing of periapical tissue is generally regarded as an important outcome. In VPT, maintenance of pulp vitality is considered the primary outcome. There are, however, other outcomes that are often inconsistently reported including tooth survival [6, 13], integrity of restoration, pathological changes (calcification, resorption, etc.)  and patient-reported outcomes such as symptoms and oral health-related quality of life measures (OHRQoL).
Why are core outcome sets important in endodontics?
Core outcome sets (COS) are defined as an agreed standardised set of outcomes that should be measured and reported as a minimum in all trials in a particular field . COS should be registered to support the selection of outcomes and outcome measures in clinical trials, the delivery of routine care  and the quality of systematic reviews .
The selection of appropriate outcomes is essential if clinical studies are to enable direct comparison between the effects of different interventions with minimal bias. Development and implementation of COS minimise bias in the selection and reporting of outcomes, thereby facilitating evidence synthesis for systematic reviews [15, 18]. Indeed, the bulk of the systematic reviews on the outcome of endodontic treatment reported problems when comparing outcome measures [2, 11, 19]. Outcome reporting bias is likely to affect not only systematic reviews but applies to published research in general, with a resultant negative impact on clinical guideline development and ultimately patient care. The GRADE (Grading of Recommendations Assessment, Development and Evaluation) group, supported by Cochrane  and the World Health Organization (WHO)  for the development of guideline recommendations, recognises the need to identify a relevant COS for use in the research that will underpin those recommendations.
In dentistry, COS has been or is being developed for various dental treatments including traumatic dental injuries , periodontology  and orthodontics . Endodontology is an important field in dentistry for which COS are needed. Recently, the ESE developed a focused but limited COS for S3 level clinical practice guidelines . This process was limited to the guideline steering group and was not representative of all stakeholders particularly, service users and patients. There is therefore a need for the development of COS which include full representation of stakeholders so that clinicians, researchers, patients, the general public, policymakers and public health professionals will benefit.
The aim of this study is to develop COS suitable for assessing endodontic treatment outcomes in permanent teeth after any form of endodontic treatment including non-surgical and surgical RCTx, VPT and revitalisation procedures.
The COS is designed for use in both research and clinical practice, including any healthcare setting in which endodontic treatment for adults with pulpal and periapical disease is carried out.
Perform scoping reviews to identify a list of potentially important outcomes from published studies investigating outcomes of various endodontic treatment modalities.
Identify a list of potentially important outcomes reported by patients in semi-structured interviews to augment the list generated from the scoping review process.
To reach consensus regarding the most important outcomes from the perspective of patients and clinicians by using consensus.
To identify appropriate outcome measurement instruments (OMIs) to be used in the reporting of the COS and the appropriate time points at which the outcomes should be measured.
Determine existing outcomes used for endodontic treatment
A scoping review of previous systematic reviews and their included studies will be carried out to identify all outcomes that are reported in endodontic treatment research, including studies evaluating outcomes of different endodontic treatment modalities or prevalence of different types of endodontic treatments. A systematic literature search will be carried out using specific search terms and the following sources: PubMed, MEDLINE, OVID, Embase, Cochrane Database of Systematic Reviews, Web of Science citations and grey literature. Searches will be carried out for reviews published from 1 January 1990 to 31 December 2020. Specific search strategy for each endodontic treatment modality will be developed using appropriate search terms for non-surgical and surgical RCTx, VPT and revitalisation procedures performed in human permanent teeth. Inclusion criteria will be clinical studies in humans involving permanent teeth only and that reported on outcomes of any specific endodontic treatment modality. A minimum of 1-year follow-up and a maximum of as long as possible will be required for reported long term outcomes. Studies not published in the English language or in full text will be excluded. Two independent assessors will carry out study selection and data extraction for each endodontic treatment modality. Data to be collected will include type of intervention, outcomes measured and outcomes reported (clinical, radiographic, patient-reported or any other) and how the outcome is measured and duration of follow-up. A list of outcomes will be generated from these data for use in the subsequent consensus process.
Patients will be recruited to participate in semi-structured qualitative interviews to establish which outcomes they value and report to be the most important conclusion of endodontic treatment. All participants will be adults aged 18 years or older and include those who enrolled in a clinical trial investigating different endodontic treatment modalities and are willing to participate. Purposive sampling will ensure diversity in terms of demographics and type of endodontic treatment experience. This will aim at capturing potential participant differences as previously described for qualitative research . It is planned to recruit and interview the minimum number of participants required to achieve data saturation. The researchers will decide when saturation has been reached.
Semi-structured telephone interviews, utilising open-ended questions will be conducted. Conducting the interviews on a 1:1 basis on the telephone should improve accessibility and maximise participant engagement. The questions will be based on the list of outcomes generated from the literature review with patients being asked to prioritise and rank those outcomes which they feel are the most important and should be included in the COS. Interviews will be audio-recorded, transcribed verbatim and a thematic analysis undertaken.
If the findings are to influence policy and practice, then the outcomes in the COS need to be relevant and important to key stakeholders. Appropriate key stakeholders include patients, general dental practitioners, specialist endodontists, policy makers, students, endodontic teachers, industry representatives and clinical researchers. These will all be included in the consensus panel with adequate representation from each group. As there is no agreement on appropriate panel size for achieving consensus with the Delphi approach , a minimum recruitment of 50 participants will be attempted; however, if saturation of data is not reached with this number, further participants will be recruited. The aim is to recruit a heterogeneous sample, representative of all stakeholders.
Inclusion of patients is critical for the project, as patient contributions help to identify outcomes which have not previously been considered by the other stakeholders , and patient-reported outcomes are often ignored in endodontic research. Patient representatives will be identified from those attending their dentist for endodontic treatment. Endodontists, general dental practitioners, students, policy makers and researchers will be identified and contacted through their employers or professional bodies.
A diverse and geographically representative group of senior academics, endodontists and general practitioners will form the steering group for this project. All members of the group will fulfil the following criteria for eligibility: (i) working for at least 5 years in the field of endodontics or another dental science, (ii) have previously published in the area of evidenced-based dentistry, (iii) have a minimum of 10-year academic experience post-qualification and (iv) have no conflict of interest in developing a COS. The members of the group will be invited to participate in the process of COS development as outlined in this document.
Definition of the consensus
The first phase in the consensus process is to agree what to measure, i.e. the principal outcome. In the second phase, decisions will be made about how and when the outcome should be measured. The aim is to build solid consensus from a group of diverse users using a rigorous methodology.
Consensus will be reached using Delphi surveys as previously recommended . This process comprises sequential questionnaires answered anonymously by the participants. The Delphi process has advantages of independent opinion gathering and feasibility for wide geographical representation. An invitation will be sent to potential participants with information on the COS and the consensus process. Those who reply and agree to participate will be invited to contribute to round one of the Delphi surveys. The surveys will be hosted in Google forms and piloted for validation prior to distribution. The aim is to represent all stakeholders to achieve international consensus and an anticipated minimum of 50 participants. The outcomes will be described in lay terms with the help of patient and public involvement (PPI) representatives. Participants will be asked to score each outcome using the scale proposed by the GRADE group, in which 1 to 3 signifies an outcome of relatively little importance, 4 to 6 signifies important but not critical, and 7 to 9 a critical outcome. Participants will also be asked to suggest any outcome measures that they would consider relevant but which were not included in the original list.
For an outcome to qualify for the second round of the Delphi process, ≥70% of responders should score it 7–9 and < 15% should score it 1–3. Participants who complete round 1 will be invited to complete round 2. They will be provided with feedback on the previous round for each outcome and given the opportunity to rescore each of the outcomes remaining in the e-Delphi process, if they wish.
It is expected that three rounds may be required to complete the Delphi process to identify candidate outcomes for the consensus meeting. However, if consensus is clear after two rounds, the third round will not take place. All consensus critical outcomes (rated as 7–9) by ≥70% of responders and < 15% of responders scored the outcome of little importance (rated as 1–3) will be put forward to the COS consensus meeting. To reduce attrition, the researchers will send a reminder via text or email at the beginning of every week.
Online consensus meeting
Participants who complete all rounds of the Delphi survey will be invited to attend the online consensus meeting. To ensure effective discussion, up to 20 participants will be chosen at random from those expressing willingness to attend, but ensuring representation from the key stakeholder groups. The meeting will be facilitated with the help of an expert in group consensus meeting methodology and two clinicians.
Selection of instruments to measure the core outcomes
Following completion of the identification of COS, we will then identify a set of core outcome measures. The identified outcome measures will be evaluated according to the nine measurement properties identified by the COnsensus-based Standards for the selection of health status Measurement INstruments (COSMIN) initiative, including internal consistency, reliability, measurement error, content validity, structural validity, hypotheses testing, cross-cultural validity, criterion validity and responsiveness . A consensus meeting will also be organised to agree on the developed outcome measures and the appropriate time points for their measurement.
Ethics approval was obtained from the Queen’s University Belfast Faculty Research Ethics Committee and the Office of Research Ethics Committee N. Ireland. Informed consent will be obtained from all participants.
The development, dissemination and implementation of a COS for endodontic treatment will ensure standardised use of outcomes in research and clinical practice. This study will apply sound methodology and include key stakeholders in the selection of outcomes and their measurement. The addition of patient-reported outcomes is a strength of this proposal and ensures a focus on treatment outcomes that are relevant for patients. The results of this study will be shared with participants and stakeholders including publication in peer-reviewed international journals and conference presentations. To increase uptake, the COS will also be actively shared with clinical guideline developers, research funders, specialist endodontology and other dental journal editors.
The study is registered on the COMET website, registration number 1879. The study timeline is provided in Table 1.
Availability of data and materials
Cumulative Index of Nursing and Allied Health Literature
- COMET Initiative:
Core Outcome Measures in Effectiveness Clinical Trials
Core outcome set
COnsensus-based Standards for the selection of health Measurement INstruments
Excerpta Medica database
European Society of Endodontology
Grading of Recommendations Assessment, Development and Evaluation
Mini-Mental State Examination
Oral health-related quality of life measures
Patient and public involvement
Randomised controlled trial
Root canal treatment
Vital pulp treatment
Levin LG, Law AS, Holland GR, Abbott PV, Roda RS. Identify and define all diagnostic terms for pulpal health and disease states. J Endod. 2009;35(12):1645–57. https://doi.org/10.1016/j.joen.2009.09.032.
Ng YL, Mann V, Rahbaran S, Lewsey J, Gulabivala K. Outcome of primary root canal treatment: systematic review of the literature - Part 1. Effects of study characteristics on probability of success. Int Endod J. 2007;40(12):921–39. https://doi.org/10.1111/j.1365-2591.2007.01322.x.
Ng YL, Mann V, Rahbaran S, Lewsey J, Gulabivala K. Outcome of primary root canal treatment: systematic review of the literature - Part 2. Influence of clinical factors. Int Endod J. 2008;41(1):6–31. https://doi.org/10.1111/j.1365-2591.2007.01323.x.
Setzer FC, Kohli MR, Shah SB, Karabucak B, Kim S. Outcome of endodontic surgery: A meta-analysis of the literature - Part 2: Comparison of endodontic microsurgical techniques with and without the use of higher magnification. J Endod. 2012;38:1–10.
Fong W, Heidarifar O, Killough S, Lappin MJ, El Karim IA. An audit on technical quality of root fillings performed by undergraduate students. Int Endod J. 2018;51:e197–203. https://doi.org/10.1111/iej.12803.
Lumley PJ, Lucarotti PSK, Burke FJT. Ten-year outcome of root fillings in the General Dental Services in England and Wales. Int Endod J. 2008;41(7):577–85. https://doi.org/10.1111/j.1365-2591.2008.01402.x.
Karabucak B, Bunes A, Chehoud C, Kohli MR, Setzer F. Prevalence of apical periodontitis in endodontically treated premolars and molars with untreated canal: a cone-beam computed tomography study. J Endod. 2016;42(4):538–41. https://doi.org/10.1016/j.joen.2015.12.026.
Duncan HF, Galler KM, Tomson PL, Simon S, El-Karim I, Kundzina R, et al. European Society of Endodontology position statement: management of deep caries and the exposed pulp. Int Endod J. 2019;52(7):923–34. https://doi.org/10.1111/iej.13080.
Hargreaves KM, Diogenes A, Teixeira FB. Treatment options: biological basis of regenerative endodontic procedures. J Endod. 2013.
Saoud TMA, Zaazou A, Nabil A, Moussa S, Lin LM, Gibbs JL. Clinical and radiographic outcomes of traumatized immature permanent necrotic teeth after revascularization/revitalization therapy. J Endod. 2014;40(12):1946–52. https://doi.org/10.1016/j.joen.2014.08.023.
Cushley S, Duncan HF, Lappin MJ, Tomson PL, Lundy FT, Cooper P, et al. Pulpotomy for mature carious teeth with symptoms of irreversible pulpitis: a systematic review. J Dent. 2019;88:103158. https://doi.org/10.1016/j.jdent.2019.06.005.
Cushley S, Duncan H, Lappin M, Chua P, Elamin A, Clarke M, et al. Efficacy of direct pulp capping for management of cariously exposed pulps in permanent teeth: a systematic review and meta-analysis. Int Endod J. 2020;54(4):556–71. https://doi.org/10.1111/iej.13449.
Tickle M, Milsom K, Qualtrough A, Blinkhorn F, Aggarwal VR. Summary of: The failure rate of NHS funded molar endodontic treatment delivered in general dental practice. Br Dent J. 2008;204(5):254–5. https://doi.org/10.1038/bdj.2008.133.
Brodén J, Heimdal H, Josefsson O, Fransson H. Direct pulp capping procedures versus root canal treatment in young permanent vital teeth with pulp exposure due to caries. A systematic review. Am J Dent. 2016;29:201–7.
Williamson PR, Altman DG, Blazeby JM, Clarke M, Devane D, Gargon E, et al. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13(1). https://doi.org/10.1186/1745-6215-13-132.
Davis K, Gorst SL, Harman N, Smith V, Gargon E, Altman DG, et al. Choosing important health outcomes for comparative effectiveness research: an updated systematic review and involvement of low and middle income countries. PLoS One. 2018;13(2):e0190695. https://doi.org/10.1371/journal.pone.0190695.
Clarke M, Williamson PR. Core outcome sets and systematic reviews. Syst Rev. 2016;5(1):11. https://doi.org/10.1186/s13643-016-0188-6.
Sinha IP, Smyth RL, Williamson PR. Using the Delphi technique to determine which outcomes to measure in clinical trials: recommendations for the future based on a systematic review of existing studies. PLoS Med. 2011;8(1):e1000393. https://doi.org/10.1371/journal.pmed.1000393.
Elmsmari F, Ruiz XF, Miró Q, Feijoo-Pato N, Durán-Sindreu F, Olivieri JG. Outcome of partial pulpotomy in cariously exposed posterior permanent teeth: a systematic review and meta-analysis. J Endod. 2019;45:1296–1306.e3.
Rosenbaum SE, Glenton C, Oxman AD. Summary-of-findings tables in Cochrane reviews improved understanding and rapid retrieval of key information. J Clin Epidemiol. 2010;63(6):620–6. https://doi.org/10.1016/j.jclinepi.2009.12.014.
El Karim IA, McCrudden MTC, McGahon MK, Curtis TM, Jeanneau C, Giraud T, et al. Biodentine reduces tumor necrosis factor alpha-induced TRPA1 expression in odontoblastlike cells. J Endod. 2016;42(4):589–95. https://doi.org/10.1016/j.joen.2015.12.017.
Kenny KP, Day PF, Sharif MO, Parashos P, Lauridsen E, Feldens CA, et al. What are the important outcomes in traumatic dental injuries? An international approach to the development of a core outcome set. Dent Traumatol. 2018;34(1):4–11. https://doi.org/10.1111/edt.12367.
Lamont TJ, Clarkson JE, Ramsay CR. Outcome measures in Cochrane reviews and protocols for the prevention and treatment of periodontal disease. Trials. 2015;16(S1). https://doi.org/10.1186/1745-6215-16-S1-P32.
Tsichlaki A, O’Brien K, Johal A, Marshman ZZ, Benson PP, Colonio Salazar FB, et al. Development of a core outcome set for orthodontic trials using a mixed-methods approach: Protocol for a multicentre study. Trials. 2017;18(1). https://doi.org/10.1186/s13063-017-2098-x.
Duncan HF, Nagendrababu V, El-Karim IA, Dummer PMH. Outcome measures to assess the effectiveness of endodontic treatment for pulpitis and apical periodontitis for use in the development of European Society of Endodontology (ESE) S3 level clinical practice guidelines: a protocol. Int Endod J. 2021. https://doi.org/10.1111/iej.13627.
Kirkham JJ, Gorst S, Altman DG, Blazeby JM, Clarke M, Devane D, et al. Core Outcome Set–STAndards for Reporting: The COS-STAR Statement. PLoS Med. 2016;13(10):e1002148. https://doi.org/10.1371/journal.pmed.1002148.
Akins RB, Tolson H, Cole BR. Stability of response characteristics of a Delphi panel. Application bootstrap data expansion. BMC Med Res Methodol. 2005;5:37.
Sanderson T, Morris M, Calnan M, Richards P, Hewlett S. What outcomes from pharmacologic treatments are important to people with rheumatoid arthritis? Creating the basis of a patient core set. Arthritis Care Res. 2010;62(5):640–6. https://doi.org/10.1002/acr.20034.
Mokkink LB, Terwee CB, Patrick DL, Alonso J, Stratford PW, Knol DL, et al. The COSMIN checklist for assessing the methodological quality of studies on measurement properties of health status measurement instruments: An international Delphi study. Qual Life Res. 2010;19(4):539–49. https://doi.org/10.1007/s11136-010-9606-8.
No funding received for this study
Ethics approval and consent to participate
Ethics approval was obtained from the Queen’s University Belfast Faculty Research Ethics Committee and the Office of Research Ethics Committee N. Ireland. Informed consent will be obtained from all participants.
Consent for publication
The authors declare that they have competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
El Karim, I.A., Duncan, H.F., Cushley, S. et al. A protocol for the Development of Core Outcome Sets for Endodontic Treatment modalities (COSET): an international consensus process. Trials 22, 812 (2021). https://doi.org/10.1186/s13063-021-05764-x
- Root canal treatment
- Vital pulp treatment
- Clinician-centred outcomes
- Clinical endpoints
- Surgical endodontics
- Patient-reported outcomes