We started evaluating our trial focusing on the assumptions about the design that we had at the start of our study. We reflected on our intentions for the execution of the trial and how these compared to what we in fact realized and from this comparison we drew the lessons learned. The contemplation and description of the experiences and problems faced during the execution of the trial and the lessons we learned were derived from extensive surveys and structured discussions among the co-investigators,These investigators were experienced researchers regarding complex interventions, with various backgrounds including methodological as well as content expertise. Below, the design of the HELP trial is described to facilitate understanding of the experiences, problems and lessons learned.
Design of the pragmatic, stepped wedge trial
The methods and design of the randomized, stepped wedge trial assessing the effects of the HELP in the Dutch setting have been comprehensively described in a previously published design paper [11]. The intervention study used a stepped wedge design according to the scheme displayed in Fig. 1. Eight units of two hospitals located in the center of The Netherlands were enrolled in the study from the start (cardiology, geriatrics, internal medicine and orthopedics and surgery at both hospitals). In one of the hospitals the internal unit and the surgery unit failed to include patients. To make up for the loss of these units, two units (cardiology and geriatrics), in a university hospital, were added to the trial in a later stage of the study. In an order randomly assigned using Excel, the units participating in our study consecutively started using the intervention during the study period.
Study population
Eligibility to participate was assessed in all patients aged 70 years and over who were admitted to the participating units of the hospitals. Further inclusion criteria were the absence of delirium at the time of hospital admission and being considered at increased risk for delirium according to the three questions of the mandatory Dutch Hospital Safety Program (see below). Exclusion criteria were an expected hospital stay of less than 24 h or having a life-threatening condition, suffering from serious cognitive limitations, being legally incapable of participating, unable to communicate verbally, or receiving palliative care at the time of admission.
The a priori sample size needed was 940 patients equally distributed over the control group “pre-HELP” and the intervention group “the HELP condition” (see design paper [11] for the full sample size calculations) [8]. The sample size calculation was based on reduction of 10% in the primary endpoint incidence of delirium. To demonstrate this difference, using a two-sided test with an alpha of 0.05 and a power of 0.90, two groups of 470 patients were required, indicating a study population of 940 patients. Taking into account 15% not willing to participate, the total number of eligible patients required was estimated at 1081 patients. The study achieved inclusion of 518 patients, and was thus underpowered to assess the study outcomes. When designing this study, not much was known yet about sample size calculations for stepped wedge designs. So, the power calculation was done for a simple randomized controlled trial, not taking clustering into account, making the study even more underpowered. When analyzing the data, we conducted a state-of-the-art analysis by taking the clustering into account in multilevel analysis. This way, the influence of a specific unit on the outcome of an individual patient is separately modeled, resulting in a less biased effect estimation of the intervention in the individual patient.
Assessment of risk for delirium
As part of routine care, the risk for delirium was assessed by a nurse within 24 h after hospital admission using the three questions of the Hospital Safety Program [12]. This program was launched in The Netherlands in 2009 as part of mandatory hospital care for patients aged 70 years and older. Patients are considered to be at risk for delirium if one of the following questions is answered positively: “Do you have memory problems?”; “During the past 24 h, did you need assistance with your daily self-care?”; and “Have you ever been confused during earlier hospital admissions or illnesses?” If a patient was considered to be at risk for delirium, the patient was approached for study participation by a NP specializing in the care for older patients. The NPs were explicitly instructed to apply the same criteria for approaching and including patients in the pre-HELP and the HELP periods.
Intervention
For an extensive description of the HELP intervention, we refer to the design article of our study [11] and to the articles in which the development of the original HELP is described [7, 13]. In close collaboration with the developer of the HELP, Dr. Sharon Inouye, all materials were adapted and translated into Dutch. In short, the components of the program were the provision of standardized protocols targeting risk factors for delirium, the introduction of elderly care NPs, bedside interventions conducted by trained volunteers, and regular reassessment of enrolled patients to keep personalized interventions matching the changing needs throughout the course of hospitalization. If, during the course of hospitalization, patients requested no volunteers, they did not receive further bedside interventions, but were not withdrawn from the study.
Measurements
As much as possible, we relied on measurements done as part of routine clinical practice which were retrieved from electronic patient records at the end of the study. Telephone interviews were used to collect follow-up data on rehospitalization and admission to residential care.
Primary outcome: delirium
According to Dutch guidelines, nurses had to administer the Delirium Observation Screening Scale (DOSS) [14] as part of routine care three times daily for three consecutive days after a patient screened positive for an increased risk for delirium. When a positive DOSS occurred, incident delirium was confirmed by the NP or geriatrician using the Confusion Assessment Method (CAM) [15]. The CAM includes a four-item diagnostic instrument for delirium assessing the acute onset and fluctuating course of inattention (i.e., distractibility), and either disorganized thinking (i.e., illogical or unclear ideas) or an alteration in consciousness.
Secondary outcomes
The secondary outcomes were hospital length of stay, change in health-related quality of life [16], hospital readmission, and admission to an institution assessed monthly for 3 months after discharge in a random subset of study patients. Hospital length of stay was determined as a secondary outcome from the electronic patient records. Additionally, patients were contacted monthly by telephone for 3 months after discharge to assess readmission to the hospital or admission to an institution (nursing home or rehabilitation facility). If patients were not able to participate, a close family member was asked for information on admission to a hospital or institution.
Covariates assessed at baseline
From the electronic patient records, level of education and Activities of Daily Living (ADL) functioning were extracted. ADL function at baseline was assessed using the Katz Index on independence in ADL [17], a six-item instrument to assess independence. Trained NPs additionally assessed cognitive function, using the Six-item Cognitive Impairment Test (6CIT), and self-rated health using a Visual Analog Scale asking patients to rate their own health on a scale of 0 to 100 [18, 19].
Statistical analysis
Using multilevel regression models, the incidence of delirium (logistic), the change in health-related quality of life during hospital stay (linear), the length of stay (Poisson), (re-)hospitalization (logistic) and admittance to an institution following discharge (logistic) were compared between the pre-HELP (control) and HELP (intervention) patient groups. All analyses were adjusted for the clustering of patients within units, period effect and for baseline characteristics (age, sex and ADL function at baseline). All statistical analyses were performed using the lme4 package [20] for R, version 3.1.3 [21].