Table 3 Process areas for performing scientific design reviews of clinical trials

Informativeness-centric

A scientific design review where the main focus is on identifying and reducing risks that the CT will end without definitively answering its research question. An informativeness-centric review leaves as secondary any design concerns tied exclusively to regulatory, bioethics, and clinical operations topics. Focus is on evidence-based drivers of informativeness, such as sample size methods, use of local up-to-date epidemiological data as input variables, conservative effect estimates, use of biostatistical simulation, and use of common endpoints

Breadth of review expertise

Every CT has a variety of attributes that might make it distinctive. These attributes may appear across a range of CT elements, such as the intervention, stage of the disease, CT phase, CT site(s), or design characteristics. Breadth of review expertise means the expert review panel includes, for most or all unique attributes, a reviewer who has implemented, provided oversight for, designed, or critiqued that attribute in the past. This represents how complete the application of reviewer expertise to all details of a CT can be. This is often correlated with more, rather than fewer, reviewer individuals on a panel

Depth of reviewer expertise

Depth of expertise means the review panel includes, for key attributes of a CT and its design, reviewers who have designed and implemented, participated in, or provided oversight for related CTs. The reviewer is known to others in the field as being a well-known or famous resource or author on intricacies, advanced methods, or the corpus of work in a specific topic; typically, this requires decades of experience

Iterative

There are multiple rounds of analysis, edits, and collaboration in the review. Each expert sub-panel or working group iterates its findings and recommendations. Sub-panels consolidate and submit their review to a higher-level panel, which iterates with the sub-panels and within itself. The higher-level panel iterates the review with the PI. The iterations ensure each critique and recommendation has been refined, prioritized, and understood

Information-enhanced

There is a wide variety of information beyond the protocol that could indicate the risk level and riskiest attributes of a CT’s design. This information, if curated, and put in the hands of reviewers, makes for a richer review. An information-enhanced review means one where the protocol is accompanied by information incremental to the protocol requested by reviewers and sourced from PI or internally that provides risk insights to the reviewer

Solution-oriented

Solution-oriented means reviews ought to focus on solutions to multiple stakeholders’—but especially PIs’—challenges as well as the challenges inherent in design attributes. The solutions offered ought to be specific, timely, feasible, and informativeness-forward. Solutions could include links to other experts, additional funding such as CT planning grants, data, or other resources

Software-enabled

Software-enabled scientific design review means all relevant portions of the process that can be reliably enhanced with technology would be. This ranges from basic mechanics such as scheduling, communication, and secure document sharing, all the way to the use of artificial intelligence for prediction and data mining. Software could be used to support other process areas, such as measuring time spans, or for scouring registries, databases, historical protocols, and publications toward information enhancement

Collaborative

A collaborative review process is one that is increasingly communicative within and across stakeholder groups. This communication and collaboration could be flexible enough to adjust to changes in context. Collaborations could range from enabling quick scheduling and correspondence to partnering more deeply in-person, telephonically, or with other real-time engagement. Reviewers speaking to PIs about protocol review findings and recommendations is a crux of collaboration

Rich in data and analytics

A review program rich in data and analytics is one that collects, cleans, curates, and enriches information about all parts of reviews and uses analysis and visualization to communicate more richly with stakeholders, answer questions, and aid in actionable decision-making, as well alerting to trends and finding opportunities

Reliability and quality

The platform and approaches to delivering reviews perform their intended function. Reviews and the mechanics of delivering them are dependable. The team and platform sustain a level of quality over time. There is an increasingly lower number of fails and defined approaches to fix failed reviews. Stakeholders perceive quality and value in reviews. There is a consistency of delivery over time; costs are maintainable

Time appropriate

The review approach considers time sensitivities of disease urgency and current context and needs of the funders, PIs, and other stakeholders. Each segment in the review process may consider, relative to different facets of time, the attributes of sustainability, routineness, elasticity, rigidity, or fragility. Organizations are precise around trade-offs related to timing and deliberate in their application of time aids and boundaries