This activation exercise confirmed that the ASAP trial is in a state of readiness and can be ‘aroused’ from its current hibernation phase to active recruitment within 4 weeks of activation, as planned [9]. Important practical issues that had the potential to delay the time from trial activation to first patient recruitment were identified and addressed.
A need for greater trial resilience was identified in relation to absenteeism of key personnel important to trial delivery. During a pandemic, it is estimated that up to 50% of staff may be absent at some point due to illness, or the need to care for family members who fall ill [12]. Trial Coordinating Centres need to have suitable pandemic plans in place, including approved cover arrangements for the chief investigator and other key personnel. Considerations of resilience extend to the membership and functioning of the Trial Steering Committee, the Data Monitoring Committee, and to elements potentially outside the immediate influence of the trial team, such as the IMP/placebo product supplier and QPs. Although the trial team had already developed what were thought to be appropriate pandemic resilience plans, the simulation exercise revealed further areas for attention.
In the design of the ASAP trial, it was recognised that conducting a trial during a pandemic would be highly challenging due to demands on clinical staff and healthcare services. Specifically in relation to randomisation, it was determined that a web-based or telephone-based process which required a clinician to contact the trial team for the randomisation schedule would be too time-consuming and would deter clinicians recruiting potential participants. Therefore, it was designed that IMP would be pre-packed in distribution cartons according to a computer-generated random sequence. The distributions cartons would allow for easy storage at hospital admission points and crucially, would allow clinicians to simply take the next IMP pack from the distribution carton as part of the randomisation process. However, the well-established Stock Control System used by the CTU and which was designed primarily for a web-based randomisation process, required the local pharmacy to remove individual IMP packs from the distribution cartons in order to confirm receipt of IMP; with the potential to disturb the randomisation sequence. That this potential problem was not identified during trial set-up is likely due to the mismatch between ‘usual’ versus ‘unique’ trial-specific processes.
The costs for the simulation exercise were relatively modest compared to the overall costs of the trial. We were unable to reliably estimate likely cost savings arising from improvements made to trial procedures as a result of the simulation exercise. Anticipated cost savings are dependent on the ‘cost’ of slow trial accrual. In the case of pandemic and public health emergency trials, slow accrual may result in failure to complete a trial with enormous attendant costs in economic and human terms [13].
The single site design of the simulation exercise is a study limitation. Potentially, involvement of different sites might identify different areas for improvement. However, the majority of issues identified during the simulation exercise were relevant to the entire trial and were not site-specific. Hence, repeated simulation exercises involving multiple sites within a single trial may be associated with decreasing amounts of learning and be inappropriate. On the other hand, some of the major benefits of a simulation exercise derive simply from enabling local sites to rehearse their trial processes; hence identifying outdated local processes or new obstacles to trial delivery which may have arisen during the hibernation period. These benefits are not manifest as improvements to the trial but as the equally important requirement to maintain local site trial readiness and interest. Performing simulation exercises across all sites, whether in clusters or in totality, will require resources which need to be recognised by all parties, including trial funders. Inevitably, the simulation exercise included elements that were artificial, such as the involvement of a junior physician as a ‘simulated patient’. Future improvements to the simulation design would include employing professional actors as ‘patients’, as is common in simulated medical training exercises. In addition, a large number of ‘patients’ could be asked to present at the same time, thus simulating the crowd pressures that would exist during a pandemic. These improvements would increase the veracity of the simulation exercise but would also be associated with higher costs. Ultimately, awareness of those areas where simulation biases towards improved performance, or artificiality, enables avoidance of unjustified conclusions.
Implications
A major challenge in the conduct of studies during public health emergencies is the requirement to rapidly recruit sufficient participants before the end of the emergency; there is no option to extend the recruitment period because of obstacles encountered during trial delivery or slow accrual [14]. During the 2009 influenza pandemic, a phase III trial of corticosteroids in critically ill patients was initiated 2 to 3 weeks following the peak of the pandemic and only managed to recruit 26 participants before the end of the pandemic; the required sample size was 438 participants [15]. Similarly, during the 2014 Ebola outbreak in east Africa, at least three clinical trials were initiated without managing to recruit the required number of participants before the end of the outbreak [13, 16]. These uncompleted trials leave important clinical questions unanswered. Simulation exercises allow potential procedural delays to trial delivery to be identified and tackled prior to trial activation. This is a major benefit particularly for public health emergency trials.
Simulation exercises may also be useful for trials with complex or unusual trial processes. The importance of trial systems and procedures for efficient trial conduct is well recognised [17]. In most circumstances, an experienced clinical trial team should be able to pre-emptively identify all relevant potential hurdles to trial delivery and address these prior to trial activation. However, when trial processes deviate from standard patterns, it can be more difficult to anticipate potential problems.
