This study will address previously identified barriers to the adoption of AD in child health CTs (incorrect use, lack of knowledge, few cases, lack of support, ethical challenges, regulatory evaluation, methodological and statistical issues) [8] through a systematic review and mixed-methods study. To increase appropriate use of AD in CTs, this study will include a comprehensive IKT component.
Systematic review
The systematic review has been designed in accordance with the Preferred Reporting Items for Systematic Review and Meta-Analysis Protocols (PRISMA P) [22]. Specifically, the review will evaluate where, when, and how AD has been used in CTs in children and adults. Findings in pediatric and adult trials will be compared to investigate potential differences in application of AD in pediatric and adult populations.
Eligibility criteria
Published CT protocols, reports, and a secondary analysis using an adaptive method will be included. ADCTs will be defined based on the FDA description provided earlier [10], allowing also for formal incorporation of relevant external information that becomes available. Interventions will be limited to drugs and vaccines, as there are separate regulatory guidance and ethical considerations for device and behavioral studies in Canada and elsewhere. CTs of all diseases and populations (children and adults) will be included, as long as they involve at least one pharmacological agent. CT trial registrations (e.g., ClinicalTrials.gov) will not be included at this stage, as these registries rarely provide any useful information on methods or justification of methods that would identify a trial as an ADCT. The time of publication will be restricted from 2010 to 2018, for feasibility purposes. Also, because FDA guidance [10] on ADCTs was first issued in 2010, we expect to find more rigorous and consistent ADCT application after this time.
Search strategy
An experienced research librarian (CJN) will design a preliminary search strategy for ADCTs using Ovid MEDLINE. Search results will be limited to English, French, and Dutch (the languages in which our project team is fluent) and articles published between 01January 2010 and the 2018 search date. No other search filters will be applied. The search strategy will be peer-reviewed by a second, independent librarian according to Peer Review of Electronic Search Strategies (PRESS) guidelines [23], and the finalized search strategy will be translated for use in additional bibliographic databases (see the Appendix).
Sources and selection process
The literature search will incorporate the following bibliographic databases: MEDLINE (Ovid), Embase (Ovid), CENTRAL (Wiley), International Pharmaceutical Abstracts (Ovid), and MathSciNet. All titles and abstracts will be evaluated by two independent reviewers for eligibility. Full-text review, conducted in duplicate, will determine the nature of any relevant adaptations made in each CT. Any studies where it is unclear if an adaptation was made will be reviewed by the senior author and study team. Furthermore, references from recent reviews on AD from 2016 forward will be reviewed for inclusion.
Data management and collection
EndNote will be used to manage abstract screening and function as part of the study database. Data will be extracted using standardized forms in REDCap [24] following the appropriate training and approvals by the University of Manitoba. The data to be extracted will include characteristics of the CT population (age group, disease, location of recruitment), study objective, key characteristics of the CT design and statistical analysis plan (nature of the intervention(s), use of any of the control groups’ primary and secondary outcomes, planned sample size and sample size methodology, intended analyses and how closely they reflect the study design) type of adaptation(s), name of applied AD method, rationale for adaptation, reported challenges, or study limitations categorized as regulatory, ethical, logistical, statistical, or other. Data extraction with REDCap will also allow us to capture random distributions of sample sizes, conditional on individual study assumptions. Data collection procedures will include the use of detailed screening logs for recording details of studies warranting exclusion. If data are available from both a protocol and a report for the same study, the methods applied will be compared. Modifications and the justification for the ADCT modifications will be sought from corresponding study authors (if not reported). The work of two individual data extractors will be assessed for agreement.
Risk of bias
The methodological quality of individual studies will be evaluated using the Cochrane risk of bias tool [25]. The domains of Cochrane’s tool for assessing risk of bias are selection bias (random sequence generation and allocation concealment), performance bias (blinding of participants and personnel), detection bias (blinding of outcome assessment), attrition bias (incomplete outcome data for each of the main outcomes), reporting bias (selective reporting), and other sources of bias including the identification, attribution, and resolution of adverse events.
Data synthesis and analysis
The strength of the body of evidence will not be evaluated, as the analysis will be primarily descriptive, and there will be no pooling of results in a meta-analysis. Subgroup analyses will include CTs with children (only participants younger than 18 years), mixed populations of both children/adults, and only adults. The rationale for evaluating AD trials with adults and children separately is to compare the type of adaptation and frequency of AD use between the two groups. A separate analysis comparing registration/protocol/publication discrepancies for the same study will be conducted to evaluate what modifications were made and if there were unanticipated challenges before the study was conducted. A potential exploratory subgroup analysis is planned for neonatal studies (participants younger than 28 days at the time of recruitment). Results from this review will inform the instruments developed for the second stage of this research: the mixed-methods study.
Mixed-methods study
The objective of the mixed-methods study is to elicit input from a broader set of interested stakeholders to evaluate the importance and scope of regulatory, ethical, logistical, and statistical issues related to ADCTs involving children. Key priorities related to ADCTs in children will be distinguished from AD facilitators and challenges relevant in all populations. As the review may be unable to identify regulatory challenges, the mixed-methods study will obtain practical feedback from a regulatory and health technology assessment perspective. We will elicit this input using both an online survey and a 2-day face-to-face meeting. Approval for this study will be obtained prospectively from the University of Manitoba Health Research Ethics Board.
Online survey
The aim of the online survey is to have participants evaluate the importance of regulatory, ethical, logistical, and statistical features of ADCT, identify additional potential challenges not identified in the review, and classify which aspects are specific in a pediatric context relative to AD in general (all populations).
Target study population and recruitment
We have partnered with several CT networks that will help recruit our mixed-methods study population. We aim to recruit participants with a broad set of expertise, understanding, and familiarity with CTs as well as a knowledge of the challenges involved in applying AD in trials enrolling children. Specifically, our partner CT networks will use their membership lists to recruit the following types of participant groups: regulators, health technology assessors, research ethics board members, biostatisticians, clinicians, researchers, and members of an interested public (i.e., recruited from patient/parent/family groups). We will also invite authors of included studies from the systematic review. To ensure a broad representation, we will use social media and other strategies to recruit additional participants with experience in pediatric CTs.
Our intent is to recruit 100 participants from these different strategies, with an aim to have maximum variation purposive sampling [26]. Namely, we will seek to recruit 15–20 members from each participant group category identified above. We recognize that recruitment for some participant group categories will be more difficult to achieve; that is, recruiting regulators or health technology assessors may be harder than recruiting biostatisticians, clinicians, and researchers. Thus, we will seek to recruit at least 10 people who identify with each category. The stakeholder type will be self-selected, acknowledging that many individuals may represent multiple groups. Individuals who represent multiple stakeholder categories will be placed in a group with fewer participants. This redistribution will take place at the end of recruitment, according to patterns observed during the recruitment period. We will attempt to extend survey invitations until at least 10 individuals in each category have participated, or until a period of 2 months has passed.
The online survey will ask specific questions regarding the importance (using a numerical scale) of regulatory, ethical, logistical, and statistical considerations identified in the systematic review and will also solicit any additional concerns relating to various adaptations in CTs involving children. Participants will also be asked if each consideration is pediatric-specific or would be generally applicable in all CTs. At the end of the survey, participants will be asked if they are interested in attending a 2-day facilitated meeting involving focus group discussions. We would seek to recruit approximately 24 participants from the survey sample to participate in the 2-day facilitated meeting. Recognizing that there may be challenges in recruiting equal numbers of members for each participant group, we would continue recruiting until we had a good representation for each participant group type. As we aim to recruit a representative sample within our broad survey participants, each survey will be assigned a numeric identifier in order to facilitate a combined analysis of input from the online survey and the 2-day facilitated meeting.
Two-day facilitated meeting
The aim of the facilitated meeting is to arrive at a stronger rationale focused on priorities, preferences, implications, acceptability, and consequences surrounding AD in pediatric CTs. We will also pilot and seek feedback on four developed case examples to be used for teaching important concepts related to ADCTs. Specifically, using a combination of strategies — small group discussions, opportunities to assess group cohesion around certain concepts using “group voting” technology — the 2-day meeting will assess areas of convergence around priority elements and possible barriers to the use of ADCT and will present new ideas the research team may not have identified in previous project stages [27].
Two-day meeting process
One week prior to the meeting, participants will receive a summary report of findings from the systematic review and a descriptive summary of the survey results. The first day of the meeting will consist of a discussion of the systematic review and survey results through a combination of facilitated group discussions to assess within- and between-group perspectives. Discussion groups will be separated by stakeholder type in the morning, and randomly assigned mixed stakeholder groups (a maximum of eight people) will meet in the afternoon to assess how participant perspectives change when exposed to the priorities and opinions of other stakeholder group types. At the end of the first day, the project team will also assess priority elements to bring forward for discussions on the second day. Using clickers, participants will present anonymized responses to questions surrounding the priorities, preferences, implications, acceptability, and consequences of AD presented in the cases. The goal on the second day will be to elicit feedback on appropriate AD application in child health research using four distinct case examples. Case examples based on the literature review and survey findings will be created by the project team to further elicit preferences and considerations regarding the application of AD and will be presented in one of the educational video deliverables. Educational materials will be made available online and open access (see the section on knowledge translation).
Analysis
Small and large group discussions from over the 2 days will be recorded after obtaining written consent from participants, transcribed verbatim, audio-verified, and analyzed using NVivo 11™ [28]. Transcripts will be coded using a constant comparative and concept development approach [29, 30]. Two team members will develop a coding framework of descriptive surface content by independently reviewing transcripts and developing draft coding domains with definitions and refining through consensus. We will develop coding domains inductively from the data, and qualitative content will be sorted along the five prespecified consideration domains: regulatory, ethical, logistical, statistical, and other. By focusing on descriptive surface content, research assistants will be trained to systematically apply the coding guide to the dataset (after achieving inter-rater reliability scores of 0.85 or higher) [31]. We will compare and contrast early attitudes from the survey responses to participant discussions over the 2-day meeting. This will enable a more fulsome understanding of participant attitudes and an assessment of how these attitudes may have shifted (or not). Results from the qualitative study will be reported following conventional guidelines [32, 33].
Ensuring data quality
Standard metrics of reliability and validity do not apply in the same way when blending quantitative and qualitative data in mixed-methods designs. To ensure interpretation validity of study findings, we will involve our diverse knowledge users and partner organizations to assist in the analysis. Moreover, as we will be tapping into different stakeholder participant groups, their engagement in the study by completing the survey and/or participating in the 2-day face-to-face meeting will provide a broader perspective to contextualize the analysis. Study participants who agree will be sent a draft report of the mixed-methods results (aggregate results so as to maintain confidentiality) and requested to comment on the overall credibility of our interpretation as well as on any concerns they may have regarding major issues or errors. These data will be included in the final analysis. Data on the application of AD in pediatric CTs will be triangulated in two ways. First, the findings from the systematic review will inform the scope of regulatory, ethical, logistical, and statistical issues to be assessed through the surveys, facilitated group discussions over the course of the 2-day meeting, researcher reflections, and participant feedback as described above to ensure confirmability of findings [31, 34]. Second, the different stakeholder types involved in participant groups for the mixed-methods portion of the research, when combined with the interdisciplinary backgrounds of the full research team and knowledge user partners, will ensure that different perspectives inform the analysis.
Knowledge translation
All study materials will be made freely accessible through publication in open access journals and disseminated through the chimb.ca website as well as national and international conferences. We will develop educational materials, tailored to the needs of our stakeholders, which will be shared through social media, websites, newsletters, and meetings. Stakeholders for this project include clinicians, academic researchers (including methodologists and statisticians), ethicists, ethics board members, regulators, journal editors, and members of the public. We will ensure meaningful engagement with the public through collaboration with the provincial Strategy for Patient-Oriented Research (SPOR) support unit platforms. Through a diverse group of partnerships, including the Pediatric Trials Network (USA), the Pediatric Clinical Research Infrastructure Network (Europe), KidsCAN Trials (Canada), and Clinical Trials Ontario, we will have access to a wide audience in both Canada and abroad. To ensure the ongoing relevance of our results, we will also work with these partners to obtain feedback on the utility and function of our educational materials through the administration of usability surveys. The goal of our partnerships is to harmonize methodological approaches and increase the efficiency of pediatric CTs globally. Biannual teleconferences will be set to discuss ongoing AD trial challenges and methodological barriers and to share training materials. This opportunity will also be used to discuss and address regulatory hurdles in various jurisdictions globally.
An educational portal will be created and will be openly available on the chimb.ca website to share the case examples, webinars, and training videos. We will use the knowledge gained through the systematic review and mixed-methods study to create online webinars relating to specific adaptations and create a downloadable training manual supported by short videos. Each of these components will be tailored for specific stakeholder audiences based on the mixed-methods study. Metrics related to web traffic will be collected as a proxy of the uptake of the educational materials. By breaking topics into small digestible pieces, we will create opportunities for the community to utilize the portal to answer specific questions or for in-depth training. By creating an annotated bibliography of all CTs using AD identified from our systematic review, we will promote acceptance and familiarity with ADCTs. We are committed to providing ongoing support for AD through the Centre for Healthcare Innovation, which houses experts in CT methods and biostatistics. Support will be offered to researchers through free 1-h consultations, followed by a fee for service or opportunity for collaboration model.