Skip to main content

Performance feedback on the quality of care in hospitals performing thrombectomy for ischemic stroke (PERFEQTOS): protocol of a stepped wedge cluster randomized trial



Although the provision of performance feedback to healthcare professionals based on data from quality registries is common practice in many fields of medicine, observational studies of its effect on the quality of care have shown mixed results. The objective of this study is to evaluate the effect of performance feedback on the quality of care for acute ischemic stroke.


PERFEQTOS is a stepped wedge cluster randomized trial in 13 hospitals in the Netherlands providing endovascular thrombectomy for ischemic stroke. The primary outcome is the hospital’s door-to-groin time. The study starts with a 6-month period in which none of the hospitals receives the performance feedback intervention. Subsequently, every 6 months, three or four hospitals are randomized to cross over from the control to the intervention conditions, until all hospitals receive the feedback intervention. The feedback intervention consists of a dashboard with quarterly reports on patient characteristics, structure, process, and outcome indicators related to patients with ischemic stroke treated with endovascular thrombectomy. Hospitals can compare their present performance with their own performance in the past and with other hospitals. The performance feedback is provided to local quality improvement teams in each hospital, who define their own targets on specific indicators and develop performance improvement plans. The impact of the performance feedback and improvement plans will be evaluated by comparing the primary outcome before and after the intervention.


This study will provide evidence on the effectiveness of performance feedback to healthcare providers. The results will be actively disseminated through peer-reviewed journals, conference presentations, and various stakeholder engagement activities.

Trial registration

Netherlands Trial Register NL9090. Registered on December 3, 2020

Peer Review reports


Background and rationale

In 2019, more than 30,000 ischemic stroke patients were admitted to Dutch hospitals and this number is increasing every year [1]. Ischemic stroke is a major cause of death and long-term disability [2]. Randomized trials showed that early endovascular thrombectomy (EVT) substantially improves the 3-month functional outcome with an absolute risk reduction in terms of death or permanent disability of 19.5% [3,4,5]. With EVT, the neuro-interventionalist advances a catheter through the femoral artery up to the occluded cerebral artery to remove the occlusion of blood clots. The effectiveness of EVT is highly time-dependent. Every hour delay in the initiation of EVT results in death or permanent disability in 1 out of every 19 patients [6]. Several trials and observational studies noted an association between an efficient workflow to achieve fast recanalization and a stronger treatment effect of EVT resulting in better clinical outcomes [7,8,9,10,11]. Therefore, having the right infrastructure and an efficient process of care is of utmost importance to be able to treat every patient as fast as possible.

Performance feedback has been defined as “a summary of clinical performance on a specific indicator, e.g. process measure, with benchmarking against performance of other providers over a specified period of time with or without recommendations for action” [12,13,14,15]. Providing performance feedback regarding process indicators to healthcare professionals has become quite common in healthcare [12, 16]. However, performance feedback has no firm empirical basis and consensus on how this feedback is best to be provided is lacking [12]. A Cochrane systematic review of 140 studies on the effectiveness of audit and performance feedback on the quality of care reported a median 4.3% absolute improvement in patient outcomes (interquartile range 0.5–16%) [12]. However, effects tend to be very heterogeneous, and success factors for the design and delivery of effective performance feedback have not been identified. Understanding heterogeneity of effect has been limited in part by the lack of appropriate design of the feedback interventions and the absence of a quantitative evaluation of effectiveness [17,18,19,20]. The aim of this study is to evaluate the effect of providing performance feedback to healthcare providers in individual EVT hospitals to improve the quality of stroke care.


Since stroke logistics and time metrics vary considerably between centers and within centers, and are strongly associated with outcome [21], this study aims to assess the extent to which performance feedback to healthcare providers in individual hospitals providing EVT for ischemic stroke, resulting in action plans and targets based on this feedback, reduces time from arrival at the hospital to initiation of EVT and thereby improves the quality of care.


Trial design

We designed a stepped wedge cluster randomized trial (CRT) of performance feedback on the quality of care in hospitals performing EVT for ischemic stroke (PERFEQTOS) to determine the impact of a performance feedback intervention accompanying hospital-specific action plans and improvement targets. This is a specific form of CRT in which initially all clusters (in this study: hospitals) serve as controls. The intervention is rolled out sequentially but randomly at different time points, such that at the end of the study all clusters have crossed over to the intervention condition (Fig. 1) [22]. In the present study, the design includes an initial 6-month period in which none of the hospitals receives performance feedback. Subsequently, every 6 months, three or four randomly chosen hospitals will cross over from control to intervention conditions, which implies a total study duration of 30 months given that 13 hospitals participate. The Standard Protocol Items: Recommendation for Interventional Trials (SPIRIT) checklist [23] is shown in Additional file 1.

Fig. 1
figure 1

SPIRIT figure. Schematic visualization of stepped wedge cluster randomized trial control and intervention allocation, follow-up, and reporting of the study

Study setting and participants

The study will be carried out in specialized neuro-intervention hospitals performing EVT for ischemic stroke. In the Netherlands, EVT is concentrated in 17 such specialized hospitals [24]. All EVT hospitals are invited to participate in this trial with no specific inclusion or exclusion criteria. Thirteen out of these 17 hospitals agreed to participate in the study (Fig. 2). The participating hospitals include all admitted adult patients with acute ischemic stroke who underwent EVT.

Fig. 2
figure 2

Specialized EVT hospitals in the Netherlands which participate in this study

Randomization and intervention allocation

Randomization will be performed according to a lottery method of sampling by the Study Coordinator in the presence of the Principal Investigator and an independent observer and will be videotaped. Once the hospitals provided consent to participate in the study, each enrolled hospital will start in the control condition. Every 6 months, three to four hospitals will be randomized to cross over to the intervention condition. All other hospitals stay in the control condition. This procedure continues every 6 months until all hospitals are crossed over to the intervention condition. Participating hospitals will be blinded to the allocation sequence, and those hospitals not yet in the intervention condition will not be aware of the time at which they will cross over to the intervention condition. When a hospital is crossed over to the intervention, this is communicated to all participating hospitals.


Dashboard design

When it comes to performance feedback, it is necessary to visualize and communicate the information content in such a way that care providers can use this information for improvement and to maximally reduce the risk of misinterpretation of the results [25]. With this in mind, we developed a dashboard containing quality of care measures per hospital, benchmarked against the average/median performance of the other hospitals for the same time period as well as their own performance over time. Prior to designing this dashboard, we reviewed the general empirical literature on dashboard design [25,26,27,28,29,30,31,32]. Based on the insights from previous studies, we composed a dashboard containing a compositional qualitative and quantitative (graphical display with textual explanation) visualization of quality of care data. Specifically, the PERFEQTOS dashboard provides an overview of results on quality of care indicators and patient characteristics (Fig. 3). The performance feedback is presented in a way that clearly highlights the key message (i.e., improvement is recommended or not), limiting the amount of extra information to increase actionability while still allowing the recipient hospital to view more detailed comparative information.

Fig. 3
figure 3

Screenshots of the PERFEQTOS dashboard (generated from anonymous data)

Indicator measurement for the dashboard

Evaluation of healthcare providers’ quality performance is commonly done using a combination of structure, process, and outcome indicators (Table 1) [33, 34]. Our dashboard follows that approach.

Table 1 Quality of care indicators included in the dashboard

Structure indicators are the total number of patients treated with EVT in each hospital (henceforth referred to as “center volume”), the number of direct (i.e., non-transferred) and transferred stroke patients from other hospitals, and completeness of data.

Process indicators are time from arrival at the emergency department of the intervention hospital to start of the intravenous thrombolytics (IVT) administration (door-to-needle time) for non-transferred patients and time from arrival at the emergency department of the intervention hospital to initiation of EVT (door-to-groin time) for all patients. In case a patient has an ischemic stroke while already admitted to the hospital, then the arrival time is defined as the time the neurologist first assessed the patient. We stratified the indicators by whether patients are transferred or non-transferred from another hospital.

Outcome indicators are post-EVT reperfusion grade of the Extended Thrombolysis in Cerebral Infarction (eTICI) score, ranging from 0 (no reperfusion) to 3 (full reperfusion), and the National Institute of Health Stroke Scale (NIHSS) score at 24 h (± 12 h), which quantifies the neurological deficit caused by a stroke (range 0–42). In addition, the modified Rankin Scale (mRS) score is used as a measure of patients’ functional outcome after acute ischemic stroke, ranging from 0 (no symptoms) to 6 (death). The mRS score is assessed 3 months after admission. Since the outcome is strongly associated with baseline patient characteristics [35], patients’ age, sex, NIHSS score on arrival, location of the proximal intracranial occlusion, and time from onset to arrival at the intervention hospital (onset-to-door time) are also collected. Stroke onset is defined as the time point when stroke symptoms were first noticed by the patient or an observer. In cases the time of first symptoms is unknown, onset is defined as the moment the patient was last seen well. Additionally, we adjusted the outcomes per hospital for differences in these baseline characteristics.

Strategy for quality improvement

The performance improvement cycle is depicted in Fig. 4 to reflect the procedures through which hospitals aim to improve their clinical performance. Each hospital randomized to the intervention will install a local quality improvement team. This team consists at least of a neurologist, an interventionalist, a resident in neurology, and a (stroke) nurse. It can be expanded with representatives of other relevant disciplines. Each quality improvement team is trained by one of the PERFEQTOS investigators to explain the dashboard functionalities, interpretation of data, and how action plans can be developed. Quality improvement teams of hospitals in the intervention cluster periods receive quarterly performance feedback reports, which they can use to set improvement targets pertaining to specific indicators and develop performance improvement plans to achieve those targets. The assumption is that if a center’s performance is below that of the comparator, an improvement target will be set accordingly, and subsequent improvement actions would be aimed at reaching the target and remove the discrepancy [16, 36, 37]. The impact of these actions can then be evaluated based on the next feedback report(s).

Fig. 4
figure 4

The performance feedback cycle to improve each hospital’s performance. Adapted from Carver and Scheier’s Control Theory [36]

In addition, during workshops organized every 6 months for those hospitals randomized to the intervention, both the best performing hospital and most improved hospital share their best practices with the other hospitals.

Data collection and management

Data on performance indicators and patient characteristics are routinely collected in each hospital and reported to the Dutch Acute Stroke Audit (DASA) from the Dutch Institute of Clinical Auditing (DICA). DASA is the main prospective clinical auditing tool for stroke in The Netherlands since 2014, with the aim to assure the quality of patient care and to aid in improving outcomes [38]. DICA is an independent organization, founded by medical specialists, that facilitates national audits for various medical professions, including the DASA. Hospitals are free to decide who carries out the data registration (e.g., nurses, data managers, neurologists), but the final responsible person is a neurologist. Medical Research Data Management (MRDM) processor, a trusted third party, is involved to pseudonymize the data to comply with privacy legislation. Hospitals can provide the collected data to MRDM in three ways. First, an online survey through a secured web environment is available for hospitals to record the data. Second, hospitals can provide the data in batches, i.e., data files in which large amounts of data can be transferred directly to MRDM. Third, to minimize the registration burden, some hospitals took the initiative to implement data linkage, i.e., extracting the data from their individual electronic patient health record to be automatically forwarded to MRDM [38]. Participating hospitals approved with a written agreement that MRDM send pseudo raw data about the patients treated in their hospital to the PERFEQTOS Study Coordinator at Erasmus MC. The Study Coordinator then aggregates the data into hospital-level medians and percentages. Data collection continues throughout the study, so each hospital contributes to both control and intervention cluster periods, which will be compared. Next, the aggregated data are summarized in a report and fed back via the dashboard. All hospitals randomized to the intervention group will receive a performance feedback report on a quarterly basis and will start and re-iterate the performance improvement cycle (Fig. 4).

Study outcomes

The primary outcome is door-to-groin time. Secondary outcomes are door-to-needle time, eTICI score, NIHSS at 24 h, and mRS at 3 months (Table 1). These outcomes are collected for every individual patient (Table 1).

Power calculation

The power calculation was based on mean differences in door-to-groin time as a primary outcome. We used a parametric power estimation methodology for stepped wedge designs put forward by Hemming and Girling [39] in the Stata function stepped wedge, derived from Hussy and Hughes [40]. We assumed 13 EVT centers were randomized in four clusters per time step (three clusters of three hospitals and one cluster of four hospitals) that treated an average of 30 patients per center per time period of 3 months [21]. We used a mean door-to-groin time of 77 min (standard deviation 47 min) and ICC 0.37, both obtained from the MR CLEAN Registry data [21, 24]. Assuming a significance level of 5%, this would result in 88% power to detect a clinically relevant reduction of door-to-groin time of 10 min.

Statistical methods

The effect of the intervention on the primary and secondary outcomes will be assessed by comparing median/percentage outcomes between control and intervention cluster periods and also in the time trends of intervention cluster periods using a non-parametric Kruskal-Wallis test and Pearson’s chi-square statistic, respectively.

Generalized linear mixed modeling at the individual patient level will be used for analyzing the effect of the performance feedback intervention on the primary outcome (door-to-groin time). This will allow us to appropriately account for the different observation cluster periods (both control and intervention) and the hierarchical structure of the data [40]. The model will contain fixed effects for intervention (yes/no), calendar time (month) to account for autonomous time trends, patient characteristics (i.e., patients’ age, sex, NIHSS score at arrival, location of the proximal intracranial occlusion, and onset-to-door time) [40], and a random effect for the hospital. This model enables us to estimate the variance of the outcome of interest (door-to-groin time) at the hospital level (inter-cluster variation) and individual level (intra-cluster variation), which in turn enables us to estimate the total effect of the performance feedback intervention. The increase in the variance due to the clustering will be quantified by the variance inflation factor. In sensitivity analyses, we will estimate the impact of the duration of exposure to the performance feedback intervention, the effect of moving from control to the intervention condition, and/or interaction of both factors on the primary outcome [41].

Multivariable normal models will be used for imputation of missing values, using available data on patient characteristics, structure, and care processes.

Oversight and data monitoring

The trial Executive Committee consists of one Principal Investigator, one junior Study Coordinator, two researchers affiliated to DICA, three vascular neurologists, one neuro-interventionalist, one radiologist, and two trial methodologists. The Trial Steering Committee is the main decision-making body. It consists of members of the Executive Committee and all local Principal Investigators. The Steering Committee meets at least once a year.

All incoming data are reviewed by the Study Coordinator at the central trial office. All pseudo data from DASA are stored on the Erasmus MC’s secure server, which is only accessible by the Study Coordinator and Principal Investigator.

This study will be conducted in accordance with the principles of Good Clinical Practice, the Dutch Agreement on Medical Treatment Act (WGBO), and the European General Data Protection Regulation. All patients will receive the best medical treatment according to national and local guidelines and current insights. We use data collection exclusively for improving quality of care purposes. To protect patients’ privacy, the trusted third party MRDM processes the data on behalf of the healthcare providers in such a way that the Study Coordinator receives only pseudonymized patient data, meaning that personal data has been processed in such a way that they can no longer be linked to a specific person. These additional data are stored separately, and technical and organizational measures are taken to ensure that personal data can never be linked to a specific person.

This trial does focus on (performance feedback about) the treatment of acute ischemic stroke. No reason to assume that the ancillary or post-trial care varies between participating hospitals or is influenced by the intervention tested in this trial. Therefore, we do not expect this to confound the trial’s results.

Dissemination policy

The Executive Committee forms the Writing Committee for the trial. Publications will be made on behalf of all investigators. The main study results will be disseminated via publication in an international peer-reviewed journal and presentation at international conferences for health provider specialists. Representatives of the participating hospitals will be given the opportunity to comment on the manuscript and participate as co-authors. We plan to disseminate the results of the planned secondary analyses on the feasibility and effectiveness of performance feedback in one or more separate papers.


PERFEQTOS is a stepped wedge cluster randomized trial about the effect of performance feedback on the quality of acute ischemic stroke care. This performance feedback consists of a multifaceted intervention, including the implementation of a quality indicator dashboard, quality improvement teams, and performance improvement plans. The primary outcome is a process measure (door-to-groin time) which is actionable and strongly associated with clinical outcomes [6].

Previously, it has been noticed that a stepped wedge CRT design is more efficient when the intra-cluster correlation (ICC) is moderate or high (ICC ≥ 0.1) [42, 43]. Generally, ICC tends to be higher for process indicators in comparison to clinical outcomes, which is related to the fact that processes can be more easily influenced by providers than outcomes [43]. Therefore, since our main focus is on improving care processes through performance feedback, a stepped wedge CRT design is an efficient design for this study. Another strength of the stepped wedge CRT design is that it provides an opportunity to measure the effect of duration of exposure to the intervention as well as of underlying temporal changes. In order to take advantage of this strength, the longitudinal time intervals should be sufficiently broad [40]. We hypothesize 6 months is long enough to achieve an effect based on treatment outcome measurements (e.g., functional outcome at 3 months after stroke). This design helps us also to tackle difficulties of implementation of the intervention at all hospitals at once. Additionally, the intervention effect in all hospitals can be evaluated with different time exposures (6–18 months).

The (design of the) current study has some limitations. First, given that our intervention is multifaceted, it may be difficult to disentangle the relative impact of the different aspects of the intervention. However, an intervention with a combination of different strategies is more effective than approaches using a single intervention [44]. Second, the quality improvement policies of hospitals in the control phase are heterogeneous and this may influence the effect of our intervention. To facilitate the interpretation of our results, we will use semi-annual questionnaires to obtain insight into the quality improvement policies of the hospitals in the control arm. Third, outcome assessment is not blinded in our trial. However, the primary outcome (door-to-groin time) is an objective measure and therefore unlikely to bias the trial results. The assessment of the secondary outcomes NIHSS at 24 h and mRS at 3 months is not blinded as well, but because these outcomes are routinely collected through a structured algorithm, we expect the risk of bias to be limited.

Findings from this study will provide insight into the feasibility and effectiveness of structured performance feedback in reducing the time to treatment of patients with ischemic stroke treated with endovascular thrombectomy and, thereby, in improving patient outcomes. If the expected results are realized, the developed method for performance feedback is ready for wider implementation on a national and international level and can be adapted for use in other diseases.

Trial status

Protocol version number: 2, August 2020. PERFEQTOS has started in January 2020, and the anticipated study duration is 30 months.

Availability of data and materials

Data cannot be made available, as no patient approval has been obtained for sharing coded data. However, syntax files and output of statistical analyses will be made available upon request after the trial is finalized.



Cluster randomized trial


Dutch Acute Stroke Audit


Dutch Institute of Clinical Auditing


Extended thrombolysis in cerebral infarction


Endovascular thrombectomy


Intra-cluster correlation coefficient


Intravenous thrombolytics


Medical Research Data Management


Modified Rankin Scale


National Institute of Health Stroke Scale


  1. De Boer AR, Van Dis I, Wimmers RH, Vaartjes I, Bots ML. Cardiovascular diseases in the Netherlands. Dutch Heart Foundation. 2020.

  2. Donkor ES. Stroke in the 21(st) century: a snapshot of the burden, epidemiology, and quality of life. Stroke Res Treat. 2018;2018:3238165.

    PubMed  PubMed Central  Google Scholar 

  3. Vidale S, Agostoni E. Endovascular treatment of ischemic stroke: an updated meta-analysis of efficacy and safety. Vasc Endovascular Surg. 2017;51(4):215–9.

    Article  PubMed  Google Scholar 

  4. Goyal M, Menon BK, van Zwam WH, Dippel DW, Mitchell PJ, Demchuk AM, et al. Endovascular thrombectomy after large-vessel ischaemic stroke: a meta-analysis of individual patient data from five randomised trials. Lancet (London, England). 2016;387(10029):1723–31.

    Article  Google Scholar 

  5. Berkhemer OA, Fransen PS, Beumer D, van den Berg LA, Lingsma HF, Yoo AJ, et al. A randomized trial of intraarterial treatment for acute ischemic stroke. N Engl J Med. 2015;372(1):11–20.

    Article  CAS  Google Scholar 

  6. Saver JL, Goyal M, van der Lugt A, Menon BK, Majoie CB, Dippel DW, et al. Time to treatment with endovascular thrombectomy and outcomes from ischemic stroke: a meta-analysis. Jama. 2016;316(12):1279–88.

    Article  PubMed  Google Scholar 

  7. Goyal M, Almekhlafi MA, Fan L, Menon BK, Demchuk AM, Yeatts SD, et al. Evaluation of interval times from onset to reperfusion in patients undergoing endovascular therapy in the Interventional Management of Stroke III trial. Circulation. 2014;130(3):265–72.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  8. Menon BK, Almekhlafi MA, Pereira VM, Gralla J, Bonafe A, Davalos A, et al. Optimal workflow and process-based performance measures for endovascular therapy in acute ischemic stroke: analysis of the Solitaire FR thrombectomy for acute revascularization study. Stroke. 2014;45(7):2024–9.

    Article  PubMed  Google Scholar 

  9. Demchuk AM, Goyal M, Menon BK, Eesa M, Ryckborst KJ, Kamal N, et al. Endovascular treatment for Small Core and Anterior circulation Proximal occlusion with Emphasis on minimizing CT to recanalization times (ESCAPE) trial: methodology. Int J Stroke. 2015;10(3):429–38.

    Article  PubMed  Google Scholar 

  10. Mulder M, Jansen IGH, Goldhoorn RB, Venema E, Chalos V, Compagne KCJ, et al. Time to endovascular treatment and outcome in acute ischemic stroke: MR CLEAN registry results. Circulation. 2018;138(3):232–40.

    Article  PubMed  Google Scholar 

  11. Fransen PS, Berkhemer OA, Lingsma HF, Beumer D, van den Berg LA, Yoo AJ, et al. Time to reperfusion and treatment effect for acute ischemic stroke: a randomized clinical trial. JAMA Neurol. 2016;73(2):190–6.

    Article  PubMed  Google Scholar 

  12. Ivers N, Jamtvedt G, Flottorp S, Young JM, Odgaard-Jensen J, French SD, et al. Audit and feedback: effects on professional practice and healthcare outcomes. Cochrane Database Syst Rev. 2012;6:CD000259.

    Article  Google Scholar 

  13. Davis DA, Mazmanian PE, Fordis M, Van Harrison R, Thorpe KE, Perrier L. Accuracy of physician self-assessment compared with observed measures of competence: a systematic review. Jama. 2006;296(9):1094–102.

    Article  PubMed  CAS  Google Scholar 

  14. Porter ME. What is value in health care? N Engl J Med. 2010;363(26):2477–81.

    Article  PubMed  CAS  Google Scholar 

  15. Porter ME, Baron JF, Chacko JM, Tang RJ. The UCLA Medical Center: kidney transplantation. Harvard: Business School Case 9-711-410 Boston: Harvard Business School Publishing. 2012.

  16. van der Veer SN, de Keizer NF, Ravelli AC, Tenkink S, Jager KJ. Improving quality of care. A systematic review on how medical registries provide information feedback to health care providers. Int J Med Inform. 2010;79(5):305–23.

    Article  PubMed  Google Scholar 

  17. Colquhoun HL, Brehaut JC, Sales A, Ivers N, Grimshaw J, Michie S, et al. A systematic review of the use of theory in randomized controlled trials of audit and feedback. Implement Sci. 2013;8:66.

    Article  Google Scholar 

  18. Sales A, Smith J, Curran G, Kochevar L. Models, strategies, and tools. Theory in implementing evidence-based findings into health care practice. J Gen Intern Med. 2006;21(Suppl 2):S43–9.

    PubMed  PubMed Central  Google Scholar 

  19. Ivers NM, Sales A, Colquhoun H, Michie S, Foy R, Francis JJ, et al. No more ‘business as usual’ with audit and feedback interventions: towards an agenda for a reinvigorated intervention. Implement Sci. 2014;9:14.

    Article  Google Scholar 

  20. Gude WT, Brown B, van der Veer SN, Colquhoun HL, Ivers NM, Brehaut JC, et al. Clinical performance comparators in audit and feedback: a review of theory and evidence. Implement Sci. 2019;14(1):39.

    Article  Google Scholar 

  21. Amini M, van Leeuwen N, Eijkenaar F, MJHL M, Schonewille W, Lycklama à Nijeholt G, et al. Improving quality of stroke care through benchmarking center performance: why focusing on outcomes is not enough. BMC Health Serv Res. 2020;20(1):998.

    Article  Google Scholar 

  22. Brown CA, Lilford RJ. The stepped wedge trial design: a systematic review. BMC Med Res Methodol. 2006;6(1):54.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Chan A-W, Tetzlaff JM, Gøtzsche PC, Altman DG, Mann H, Berlin JA, et al. SPIRIT 2013 explanation and elaboration: guidance for protocols of clinical trials. Br Med J. 2013;346:e7586.

    Article  Google Scholar 

  24. Jansen IGH, Mulder M, Goldhoorn RB, investigators MCR. Endovascular treatment for acute ischaemic stroke in routine clinical practice: prospective, observational cohort study (MR CLEAN Registry). BMJ (Clinical research ed). 2018;360:k949.

    Article  Google Scholar 

  25. Brust-Renck PG, Royer CE, Reyna VF. Communicating numerical risk: human factors that aid understanding in health c. Rev Hum Factors Ergon. 2013;8(1):235–76.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Bantug ET, Coles T, Smith KC, Snyder CF, Rouette J, Brundage MD, et al. Graphical displays of patient-reported outcomes (PRO) for use in clinical practice: what makes a pro picture worth a thousand words? Patient Educ Couns. 2016;99(4):483–90.

    Article  PubMed  Google Scholar 

  27. Garcia-Retamero R, Cokely ET. Designing visual aids that promote risk literacy: a systematic review of health research and evidence-based design heuristics. Hum Factors. 2017;59(4):582–627.

    Article  PubMed  Google Scholar 

  28. Kurtzman ET, Greene J. Effective presentation of health care performance information for consumer decision making: a systematic review. Patient Educ Couns. 2016;99(1):36–43.

    Article  PubMed  Google Scholar 

  29. Okan Y, Galesic M, Garcia-Retamero R. How people with low and high graph literacy process health graphs: evidence from eye-tracking. J Behav Decis Mak. 2016;29(2-3):271–94.

    Article  Google Scholar 

  30. Okan Y, Garcia-Retamero R, Galesic M, Cokely ET. When higher bars are not larger quantities: on individual differences in the use of spatial information in graph comprehension. Spatial Cognition Computation. 2012;12(2-3):195–218.

    Article  Google Scholar 

  31. Stone ER, Gabard AR, Groves AE, Lipkus IM. Effects of numerical versus foreground-only icon displays on understanding of risk magnitudes. J Health Commun. 2015;20(10):1230–41.

    Article  PubMed  Google Scholar 

  32. Hallgreen CE, Mt-Isa S, Lieftucht A, Phillips LD, Hughes D, Talbot S, et al. Literature review of visual representation of the results of benefit-risk assessments of medicinal products. Pharmacoepidemiol Drug Saf. 2016;25(3):238–50.

    Article  PubMed  Google Scholar 

  33. Lilford R, Mohammed MA, Spiegelhalter D, Thomson R. Use and misuse of process and outcome data in managing performance of acute medical care: avoiding institutional stigma. Lancet (London, England). 2004;363(9415):1147–54.

    Article  Google Scholar 

  34. Donabedian A. The quality of care. How can it be assessed? JAMA. 1988;260(12):1743–8.

    Article  PubMed  CAS  Google Scholar 

  35. Lingsma HF, Dippel DW, Hoeks SE, Steyerberg EW, Franke CL, van Oostenbrugge RJ, et al. Variation between hospitals in patient outcome after stroke is only partly explained by differences in quality of care: results from the Netherlands Stroke Survey. J Neurol Neurosurg Psychiatry. 2008;79(8):888–94.

    Article  PubMed  CAS  Google Scholar 

  36. Carver CS, Scheier MF. Control theory: a useful conceptual framework for personality-social, clinical, and health psychology. Psychol Bull. 1982;92(1):111–35.

    Article  PubMed  CAS  Google Scholar 

  37. Gardner B, Whittington C, McAteer J, Eccles MP, Michie S. Using theory to synthesise evidence from behaviour change interventions: the example of audit and feedback. Soc Sci Med. 2010;70(10):1618–25.

    Article  PubMed  Google Scholar 

  38. Kuhrij LS, Wouters MWJM, van den Berg-Vos RM, de Leeuw F-E, Nederkoorn PJ. The Dutch Acute Stroke Audit: benchmarking acute stroke care in the Netherlands. Eur Stroke J. 2018;3(4):361–8.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Hemming K, Girling A. A menu-driven facility for power and detectable-difference calculations in stepped-wedge cluster-randomized trials. Stata J. 2014;14(2):363–80.

    Article  Google Scholar 

  40. Hussey MA, Hughes JP. Design and analysis of stepped wedge cluster randomized trials. Contemp Clin Trials. 2007;28(2):182–91.

    Article  PubMed  Google Scholar 

  41. Nickless A, Voysey M, Geddes J, Yu LM, Fanshawe TR. Mixed effects approach to the analysis of the stepped wedge cluster randomised trial-Investigating the confounding effect of time through simulation. PLoS One. 2018;13(12):e0208876.

    Article  PubMed  PubMed Central  Google Scholar 

  42. Campbell MK, Fayers PM, Grimshaw JM. Determinants of the intracluster correlation coefficient in cluster randomized trials: the case of implementation research. Clin Trials (London, England). 2005;2(2):99–107.

    Article  Google Scholar 

  43. Hemming K, Taljaard M. Sample size calculations for stepped wedge and cluster randomised trials: a unified approach. J Clin Epidemiol. 2016;69:137–46.

    Article  PubMed  PubMed Central  Google Scholar 

  44. Kilkenny MF, Bravata DM. Quality improvement. Stroke. 2021;52(5):1866–70.

    Article  PubMed  Google Scholar 

Download references


This study protocol was published on behalf of the PERFEQTOS Investigators: Bob Roozenbeek; Sanne J. den Hartog; Diederik W.J. Dippel; Aad van der Lugt; Hester F. Lingsma; Nikki van Leeuwen; Laurien S. Kuhrij, Lotte J. Stolze; Paul J. Nederkoorn; Adriaan C.G.M. van Es; Ido R. van den Wijngaard; Paula M. Janssen; Pieter-Jan van Doormaal; Yvo B.W.E.M. Roos; Bart J. Emmer; Suzanne M. Silvis; Wouter Dinkelaar; Michel J.M. Remmers; Douwe L.D. Vos; Rob A.R. Gons; Lonneke S.F. Yo; Julia H. van Tuijl; Hans Kortman; Jelis Boiten, Geert J. Lycklama à Nijeholt; Jurgen R. Piet; Wouter Stomp; Heleen M. den Hertog; H. Zwenneke Flach; Nyika D. Kruyt; Marianne A.A. van Walderveen; Wim H. van Zwam; Robert J. van Oostenbrugge; Jasper M. Martens, Jeannette Hofmeijer; H. Bart van der Worp; Rob T.H. Lo.

Sponsor contact information

Trial sponsor: Erasmus MC, University Medical Center Rotterdam

Contact name: Bob Roozenbeek

Address: Department of Neurology

Erasmus MC, University Medical Center Rotterdam

Dr. Molewaterplein 40

3015 GD Rotterdam, the Netherlands



This work is supported by a grant from the Erasmus University Rotterdam, The Netherlands (Erasmus Initiative “Smarter Choices for Better Health,” Action Line “Value-Based Health Care”). Award/Grant number: not applicable.

This funding source had no role in the design of this study and will not have any role during its execution, analyses, interpretation of the data, or decision to submit results.

Author information

Authors and Affiliations




BR and HFL initially conceptualized the methodology. MA, SJdH, NvL, FE, LSK, LS, PJN, AMvE, IvdW, AvdL, and DWJD aided the development of the methodology. SJdH is collecting data and undertaking data analysis. MA and SJdH contributed to writing this manuscript. All authors contributed to the refinement of the study protocol and approved the final manuscript.

Corresponding author

Correspondence to Bob Roozenbeek.

Ethics declarations

Ethics approval and consent to participate

The Institutional Review Board of Erasmus MC University Medical Center has reviewed the study protocol and confirmed that the Dutch Medical Research Involving Human Subjects Act (WMO) is not applicable.

Consent for publication

Not applicable.

Competing interests

The authors and principal investigators of the trial declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

SPIRIT Checklist.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Amini, M., den Hartog, S.J., van Leeuwen, N. et al. Performance feedback on the quality of care in hospitals performing thrombectomy for ischemic stroke (PERFEQTOS): protocol of a stepped wedge cluster randomized trial. Trials 22, 870 (2021).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: