Table 1 Overview of the lessons learned from the execution of our pragmatic, cluster randomized, stepped wedge trial

Cluster randomized (stepped wedge) controlled trial design

The design promotes hospitals to participate because all participating hospitals will receive the intervention (which was shown effective in previous studies)

Participation of eight units in three (locations of) different hospitals, all starting with a (control) pre-HELP period

Two of the eight units were excluded from participation because of problems with the inclusion of patients during the pre-Hospital Elder Life Program (pre-HELP) period

Emphasize that the study starts before implementation of the intervention and the equal importance of accurate data collection both before and after the intervention is implemented

The start of the trial was delayed. It took more time than expected to make all necessary arrangements, because not all hospitals were familiar with the regulations for performing a scientific study in the clinic

Allow for enough time to explain and familiarize participating hospitals with rules and regulations of scientific studies and make timely arrangements accordingly

Because of the delay, the time frame available to complete our research project was no longer sufficient to stick to the original scheme. Therefore, we decided to start the intervention in period 1 in the first two units to make up for the delay

A stepped wedge, randomized trial takes more time than a regular cluster randomized trial. Be sure the time frame available is comfortably sufficient, even if difficulties occur

Individual recruitment and informed consent of patients is not necessary when using routinely collected data for a quality improvement trial

Analyses of the effects according to the intention-to-treat principle in all eligible patients admitted to the participating units using data from the electronic patient record systems to reduce the risk of bias

The Medical Ethical Review Board required informed consent of every patient included and thereby made individual recruitment inevitable

When using a novel and/or rarely used study design, consider the need to discuss the assumptions underlying the design with the members of the Medical Ethical Review Board before submitting the research proposal

Be sure to have a timely decision of the Medical Ethical Review Board to be able to adapt procedures to the need for individual recruitment

The study design enables a gradual recruitment of volunteers, implementation of the intervention and inclusion of sufficient numbers of patients in each period in each unit

Recruit and educate volunteers in four cycles, to enable two new units to start the intervention every 3 months

The recruitment and education of volunteers went accordingly to plan

In the months before starting the intervention, nurses on the participating units include patients in the pre-HELP condition applying the same criteria to be used during the HELP period. Each unit had to include both pre-HELP and HELP patients to be compared and to avoid empty cells in the stepped wedge matrix

Several units had difficulties including patients in the pre-HELP condition. Two units were excluded from the study after a couple of months in which no patients were included. The ultimate result is the presence of empty wedges in the stepped wedge matrix

Strongly emphasize the importance of including patient during the preintervention period in every participating unit

Inclusion of eligible patients

No need for informed consent

We intended to study the effects of a safe, noninvasive quality improvement intervention

The Medical Ethical Review Board decided informed consent was required. Inclusion of patients in our study severely lagged behind the expected numbers. The requirement of informed consent is likely to have diminished the number of patients willing to participate

Be sure to have a timely decision of the Medical Ethical Review Board to be able to adapt procedures

The Ethical Committee permitted the use of a delayed informed consent procedure to improve the participation rate in a later stage.

No need for individual recruitment of patients

By applying the intervention to all patients at risk and using routinely collected data to monitor the effects, results of the intention-to-treat analyses are maximally generalizable

The need for informed consent and, therefore, individual recruitment in our nonblinded study, made the inclusion more prone to selection bias and the results less generalizable.

Reconsider the use of a stepped wedge, cluster randomized trial when there is a need for individual consent

The need for individual inclusion of the eligible patients made the analyses of the effects according to the intention-to-treat principle on cluster level to reduce the risk of bias impossible

Applying the same inclusion criteria in the control condition and in the intervention condition will create a balanced study population without empty cells in the stepped wedge matrix

Similar (numbers of) patients will be included in the wedges in the control period as compared to the intervention period

Comparing the baseline characteristics of patients included in the control versus the intervention period showed that patients included in the HELP condition seem slightly healthier. This may imply selective inclusion favoring positive effects of the HELP

Especially when the recruitment is done by different persons over the units, education on using the same inclusion criteria when considering patients for inclusion in both the control and the intervention condition is crucial

Each unit will include both pre-HELP and HELP patients to be compared and no empty cells will occur in the stepped wedge matrix

We failed to include a similar number of participating patients in the control condition versus the intervention condition. This caused the dropout of two units, empty cells in the stepped wedge matrix and an unequal distribution of patients in the control and the intervention groups. These phenomena would partly be concealed in a regular cluster randomized trial

Constant monitoring and motivational activities should be executed to enhance inclusion of patients, especially during the control period

Use of routinely collected data

Helps to avoid selection bias (especially in the absence of the need to individually recruit patients) and enhances external validity because of the optimal reflection of usual situation

Routinely collected data from electronic patient record systems would be consecutively made available over the time frames of the stepped wedge design.

Data from the electronic patient records were not accessible during the study, but only provided after the study period was finished

Check the availability, consistency and quality of routinely collected data well before the study and monitor data quality throughout the study

According to the study protocol, nurse practitioners (NPs) were to confirm the diagnosis of delirium using the Confusion Assessment Method (CAM) if the results of the routinely collected data on the Delirium Observation Screening Scale (DOSS) were positive

When receiving the data, a lot of data on the outcome appeared to be missing. As a consequence, an important share of the patients included in the study could not be included in the analyses, causing more empty cells in the stepped wedge matrix

Data crucial for the assessment of the effectiveness of the HELP were considered to be present in all patients eligible for the study (who all were at increased risk for delirium)

Assure timely accessibility of the data to be able to do early checks on the quality and completeness of the data during the study period