Skip to main content

Treatment fidelity monitoring, reporting and findings in a complex aphasia intervention trial: a substudy of the Very Early Rehabilitation in SpEech (VERSE) trial

Abstract

Background

Treatment fidelity is inconsistently reported in aphasia research, contributing to uncertainty about the effectiveness of types of aphasia therapy following stroke. We outline the processes and outcomes of treatment fidelity monitoring in a pre-specified secondary analysis of the VERSE trial.

Methods

VERSE was a 3-arm, single-blinded RCT with a 12-week primary endpoint comparing Usual Care (UC) to two higher intensity treatments: Usual Care-Plus (UC-Plus) and VERSE, a prescribed intervention. Primary outcome results were previously reported. This secondary analysis focused on treatment fidelity. Video-recorded treatment sessions in the higher intensity study arms were evaluated for treatment adherence and treatment differentiation. Treatment components were evaluated using a pre-determined fidelity checklist. Primary outcome: prescribed amount of therapy time (minutes); secondary outcomes: (i) adherence to therapy protocol (%) and (ii) treatment differentiation between control and high intensity groups.

Results

Two hundred forty-six participants were randomised to Usual Care (n=81), Usual Care-Plus (n=82), and VERSE (n=83). One hundred thirty-five (82%) participants in higher intensity intervention arms received the minimum prescribed therapy minutes. From 10,805 (UC 7787; UC-Plus 1450; VERSE 1568) service events, 431 treatment protocol deviations were noted in 114 participants. Four hundred thirty-seven videos were evaluated. The VERSE therapists achieved over 84% adherence to key protocol elements. Higher stroke and aphasia severity, older age, and being in the UC-Plus group predicted more treatment deviations.

Conclusions

We found high levels of treatment adherence and differentiation between the intervention arms, providing greater confidence interpreting our results. The comprehensive systems for intervention fidelity monitoring and reporting in this trial make an important contribution to aphasia research and, we argue, should set a new standard for future aphasia studies.

Trial registration

ACTRN 12613000776707

Peer Review reports

Background

Clinical practice guidelines draw on the highest levels of existing evidence to provide recommendations for evidence-based practice (EBP). To evaluate a clinical intervention, treatment fidelity, in addition to effectiveness, must be considered prior to the adoption of the intervention within standard care [1] as it alters the strength and interpretation of study findings. Although the importance of treatment fidelity monitoring and reporting is recognised [2], the quality of reporting of complex interventions in clinical trials is poor [3]. This directly leads to an evidence base that struggles to guide clinical practice [3] as both intervention detail and real-world application is lacking. To address this, the Template for Intervention Description and Replication (TIDieR) checklist [3], an extension of the Consolidated Standards of Reporting Trials (CONSORT) statement [4], recommends that treatment fidelity is included as an essential interventional trial reporting component.

Treatment fidelity monitoring and reporting is critical in trial evaluations and refers to the strategies and methodological practices used to monitor the reliability and validity of behavioural interventions in a research study [5, 6]. Without the rigour that treatment fidelity brings, conclusive statements about the effectiveness of interventions cannot be made [6]. In establishing the design of an interventional study, the theoretical model underpinning the intervention should be clear and the active ingredients of the intervention should be identified prior to trial commencement. Each ingredient should be defined and subsequently monitored during the implementation of the intervention [7].

Of frameworks established to assist the application of treatment fidelity to behavioural intervention research, the Behavioural Change Consortium (BCC) [5] is the most widely adopted [8]. The BCC outlined treatment fidelity monitoring and reporting recommendations designed to link the theory of a complex behavioural intervention to the application of that intervention. The five areas identified by the BCC were (1) study design, i.e. strategies to test theory driven hypotheses; (2) training providers, i.e. methods used to ensure therapists are adequately trained to deliver the intervention; (3) delivery of treatment, i.e. strategies to determine if the treatment is delivered as planned; (4) receipt of treatment, i.e. ability of patients to demonstrate during the intervention that they understand and can perform the behavioural skills in the intervention; and (5) enactment of treatment skills, i.e. demonstration by the participants of the ability to use the intervention in real life settings.

In a recent review of 93 studies, less than 30% reported actual or planned fidelity in aphasia treatment studies with information on where the treatment was provided, individual treatment tailoring, and modification rarely reported [9]. Reviews of treatment fidelity processes confirm limited treatment fidelity reporting in aphasia treatment studies [10,11,12]. Authors of more recent aphasia trials have published standalone treatment fidelity protocols [13,14,15,16] reflecting an increased focus on reporting treatment fidelity in aphasia research and incorporation of more rigorous procedures in study design and evaluation.

Aim

We aim to outline the treatment fidelity monitoring processes and results of the large, multi-centre VERSE trial.

Methods

Background to the VERSE trial

VERSE [17] was a three-armed RCT consisting of UC, UC-Plus and ‘VERSE’ aphasia therapy with therapy provided as follows:

  • UC therapy was provided by a qualified speech pathologist, a qualified therapy assistant or a speech pathology student, being directly supervised by a qualified speech pathologist. Aphasia treatment was delivered on a 1:1 basis or in small groups.

  • UC-Plus therapy was defined as per UC with the additional intensity requirement of 20 sessions of 45–60 min (15–20 h of direct aphasia therapy within a maximum of 50 days post stroke. This allowed an extra 7 days to complete treatment to account for illness or other factors).

  • VERSE was a prescribed intervention for treatment type and provided at the same intensity as UC-Plus.

UC and UC-Plus sessions were used as the “active concurrent control” for treatment intensity (H1) and type (H2) comparisons in the trial. A full description of therapy including setting, participant inclusion and exclusion criteria, intervention description and planned data analysis is provided in the VERSE trial manual in the main trial publication supplement [17]. Differences in the type of treatment provided in UC and UC-Plus were not controlled for unless the intervention was deemed outside the above description. The results of the primary analysis showed that early, intensive aphasia therapy did not improve recovery of communication, as measured by the Western Aphasia Battery-Revised Aphasia Quotient (AQ) at 12 weeks after stroke [17].

Treatment fidelity adherence monitoring for the three arms of the trial was documented for the prescribed study phases as per TIDieR [3]. Assessment fidelity is reported in the supplement. UC was monitored only for compliance to the main study protocol which was necessary to maintain the control arm of the trial. Further monitoring of the control arm was not undertaken to avoid a Hawthorne effect bias, or change in behaviour due to being watched [18]. Therefore, this paper contains reporting and comparisons only for the UC-Plus and VERSE arms in the trial. The VERSE treatment fidelity analysis focused on two key areas that aligned to the primary and secondary hypotheses of the trial:

  1. 1.

    Primary: the amount of therapy provided to participants in the intensive arms to confirm the prescribed amount of therapy met protocol requirements

  2. 2.

    Secondary: the nature of the therapy provided to verify that VERSE therapy was adherent to the protocol and different to the UC arms of the study.

Data extraction

Data were extracted from the VERSE trial’s data management system (REDCap® [19]) containing therapy session logs, protocol deviations and therapist and participant details. A secured electronic data transfer system compliant with international data safety requirements (CloudstorTM) was used for the transfer of video-recorded sessions. The trial data monitor downloaded the files to a central database, monitored by dedicated security systems.

Treatment fidelity procedures

An independent treatment fidelity monitor was employed to complete compliance checks related to commencement of treatment (on or before day 15 post stroke) and that the intervention period did not exceed 50 days post stroke. The treatment fidelity monitor was responsible for providing feedback to all therapists (UC, UC-Plus and VERSE), about any deviations from the therapy protocol and responded to general questions about the treatment and assessment procedures. They also reviewed the recorded treatment sessions to determine if the prescribed VERSE protocol was adhered to. A treatment fidelity co-ordinator supervised this process and provided specific feedback to VERSE therapists in the event of reduced compliance reported by the treatment fidelity monitor. A database that recorded adherence and differentiation of data was maintained. An independent research assistant cross checked and summarised these data. Trial investigators were blinded to the fidelity processes and results during the trial and received only summary data to ensure the trial was progressing as prescribed. The treatment fidelity co-ordinator whose role was to oversee the treatment integrity processes was the exception to this.

Table 1 details the way treatment fidelity was conceptualised in the study design phase as per TIDieR Item 11 ‘How well planned’ [3] and the BCC recommendations [5]. TIDieR Item 12 ‘Actual’ [3] is addressed in the ‘Results’ section of this paper. In line with recommendations, the theoretical underpinnings of the intervention were considered and specified to allow monitoring. The prescribed VERSE intervention was founded on principles of promoting neuroplasticity through targeted early intensive language therapy based on the patient’s impairment(s). The main principles that guided the VERSE prescribed intervention were (i) massed practice, (ii) error-free learning, (iii) task complexity, (iv) salience and (v) maximising communicative success through interactive functional tasks. Adherence to these components was monitored using the therapy integrity monitoring form (Supplement 1).

Table 1 VERSE reporting for TIDieR and Behaviour Change Consortium treatment fidelity recommendations

Results

TIDieR Item 12 ‘Actual’ stipulates the monitoring and evaluation of intended treatment fidelity aspects as they eventuated in the trial [3]. We have conceptualised this as the results of the planned treatment fidelity procedures reported in the ‘Methods’ section. Results are presented as per the Study Design, Training Providers and Delivery of Treatment areas of the Behaviour Change Consortium recommendations [5]. The receipt of treatment and enhancement of treatment skills areas of the recommendations were planned into the study as per Table 1 TIDieR Item 11 however, not monitored further due to constraints of the trial design.

Study design

A total of 10,805 speech pathology sessions (UC 7787; UC-Plus 1450; VERSE 1568) were recorded. Reported UC services include all services for communication and swallowing, assessment and treatment for all groups. UC-Plus and VERSE sessions were for aphasia treatment only, and these form the basis for the analysis related to ensuring a difference between therapy types.

Within the UC-Plus arm, 61 of 81 (75%) participants received the minimum intensity of greater than 15 h of intervention, while, within the VERSE intervention, 72 of 83 (86%) received this dose.

Based on the number of participants per intensive intervention arm (UC-Plus n=82; VERSE n=83) and the recommendation that each therapist submit four videos over the therapy period, 660 videos were planned while 437 were received. Two videos were received from the UC arm of the trial where video recording was not mandated and have not been included in this analysis. Figure 1 describes videos received and analysed and outlines the reasons for non-analysis.

Fig. 1
figure 1

Videos received in the trial

Training providers

Therapy was delivered by 430 therapists in total (UC 200, UC-Plus 142, VERSE 98) across 17 acute care and 45 subacute participating hospital sites. Some therapists worked across multiple sites. Therapists were predominantly female (n=418, 97.2%). In the higher intensity arms of the trial, 100% of therapists received systematised training (2–3 h) and training materials (manuals, therapy materials, recording sheets). Therapists in the VERSE arm received additional VERSE-specific training (2 h). Quarterly meetings were held for UC-Plus and VERSE therapists for the duration of the intervention. To encourage recruitment and adherence to protocol, 43 newsletters and 24 mid-monthly updates were sent to therapists from 15 July 2014 to 30 April 2018.

Delivery of treatment

A standard therapy protocol was written and disseminated to therapists. Two complaints were received from participants during the trial regarding their therapist. Six potential treatment contamination deviations were logged in the VERSE treatment arm (a VERSE trained therapist completed usual care therapy with a study participant as part of their routine employment). However, the therapists’ treatment deviations were reported as not using the VERSE treatment protocol to treat language but instead providing Usual Care for dysarthria and or apraxia of speech. No other reports of contamination were received.

From 10,805 total intervention episodes in the trial, 3077 (28.5%) were related directly to aphasia intervention in the UC-plus (1505) and VERSE (1572) groups during the intervention period. Of these treatment sessions, 431 therapy deviations were reported. There were 36% more deviations reported in the UC-plus group compared to the VERSE group. The quantity and reasoning are provided in Table 2.

Table 2 Therapy deviations recorded from REDCap logged deviation forms

Therapy deviations were recorded for 114 participants across all sites. Multiple regression was used to determine if baseline patient and or stroke factors influenced the likelihood of having a therapy deviation. Results showed that 10% of the variance in reported therapy deviations can be collectively accounted for by five factors, F(5,432)=9.83), p=.000. Age (β=.282, t=2.377, p=.018), lower AQ score (β=−.351, t=−4.66, p=.000) and being in the UC-Plus group (β=−8.883, t=−2.749, p=0.006) positively predicted treatment deviations. A higher stroke severity score (NIHSS) (β=−1.582, t=−5.726, p=0.000) predicted more treatment deviations.

Therapy adherence and differentiation

Videoed therapy sessions in the VERSE arm were marked as adherent or non-adherent by the treatment fidelity monitor according to key therapeutic elements. Sessional adherence data is in Table 3. There was no indication of contamination between UC-Plus and VERSE treatment arms in the videos using the treatment fidelity monitoring form (Supplement 1). No UC-Plus treatment session included the key VERSE therapeutic elements.

Table 3 Therapy adherence by session in VERSE arm (n=252)

Discussion

Post-stroke aphasia treatment fidelity in clinical trials has been commonly reported as ‘planned’, but lacks comprehensiveness for actual treatment fidelity outcomes reported [11]. We demonstrated that detailed treatment fidelity data collection and reporting against recommended frameworks is possible in a large-scale aphasia randomised control trial, and that fidelity reporting and analysis facilitated interpretation of how closely the planned intervention was delivered.

The relatively low rates of protocol deviation in VERSE give confidence that overall, the treatment(s) we intended to test were realised in this trial. We reported protocol and therapy deviations separately (see Supplement 2 for protocol deviations) as they represent different elements of the trial design. Therapy deviations were classified as deviations in the specific delivery of the intervention. Whilst 431 therapy deviations may seem a large number, it is important to note that with a trial of this size (3018 UC-Plus and VERSE service events) these deviations represent 1.4% of the overall services. Nevertheless, we do not know the threshold of acceptable deviations for efficacy or whether the number of deviations in our study is similar to other aphasia RCTs. We note the significant but small contribution of multiple patient, stroke and aphasia factors contributing to the number of therapy deviations in this study. This is likely to represent ‘real-life’ reasons for therapy not going ahead as per the planned protocol. These factors (increasing age, higher stroke and aphasia severity) are not surprising when considering their role in early stroke recovery. The fact that more therapy deviations occurred in the UC-Plus group may be related to patient-specific issues and or staffing-related issues. Our data were not coded to determine site-specific therapy deviations, in that all deviation data were linked via the original participant number which recorded the de-identified recruiting site only. The trial participant number did not reflect transfer of participants between healthcare sites. This point may be of use to the planning of future treatment fidelity studies.

We believe the reporting of all intervention protocol deviations is essential to provide deeper understanding of trial results. When planning the VERSE trial, we used pilot data from our previous research regarding challenges faced concerning dose and type of treatment delivery, to help design treatment fidelity processes in VERSE. A central tenant of treatment fidelity reporting is that it should be based on factual observation of the treatment, and not simply whether treatments were planned [5, 6, 11]. Given the multifactorial nature of complex interventions in aphasia trials, a level of deviation from the treatment plan is anticipated. How much deviation from intervention protocol constitutes a complete break from the original therapy intent is unknown.

Our main method of direct observation of the intervention was the use of video-recorded therapy sessions. Two-thirds of the expected video-recorded sessions were received (66%). We argue that this number of recordings is reasonable and a valid representation of the intensive intervention in the trial, given (a) there were over 240 therapists providing intensive treatment, (b) not every participant received every prescribed session and (c) significant technological challenges of involved in transferring large confidential data files. The VERSE intervention group therapists submitted more recordings than the UC-Plus therapists which may have been due to additional training received by these therapists. Four hundred thirty-seven videos (14.5% of the total number of treatment sessions) in the intensive arms of the trial were reviewed in their entirety for treatment fidelity, reflecting a logistically feasible number given the size of the VERSE trial. The issue of resourcing therapy monitoring in any study is significant and needs to be considered in the overall funding of intervention studies.

There is no accepted minimum level of integrity required in a complex, behavioural RCT; however, the literature suggests protocol adherence of 80% or greater to be considered a high-fidelity level [6, 7, 20]. An important finding from our study is the 98% protocol adherence to the VERSE treatment a priori defined key intervention ingredients, consistent with high protocol adherence in another study [13]. The VERSE treatment group reported less than half the number of therapy deviations compared to UC-Plus (137 compared to 294, respectively). This is likely due to the detailed and prescribed VERSE therapy regimen, facilitating adherence. Overall, these data indicate the higher intensity, clearly defined VERSE intervention was delivered as planned with few deviations, and treatment types between the different arms were clearly differentiated. Results can therefore be interpreted with confidence.

In complex behavioural interventions, the multiple planned elements of the intervention are interconnected, such that isolating and describing the therapeutic elements of a complex behavioural intervention is challenging. While study design and treatment tasks are frequently reported in aphasia trials, the theoretical underpinnings and potential active ingredients within the intervention are reported much less often [9, 11]. When delivering and evaluating an intervention such as aphasia therapy, conceptualising and developing the therapeutic elements into a measurable protocol can seem overwhelming. It is therefore not surprising that measuring protocol adherence in aphasia intervention is poorly executed. The proposed macrostructure for measuring treatment fidelity, outlined by TIDieR [3] and Bellg [5], that reinforces the use of a measurable theoretical approach, the mapping of this to planned treatment tasks, and measuring the implementation of the overall study protocol, offers a robust guide for including this critical dimension of intervention research.

We note gaps in reporting requirements between CONSORT [4], Standard Protocol Items: Recommendations for Intervention Trials SPIRIT [21], TIDieR [3] and Bellg et al. [5] related to treatment fidelity. The CONSORT and SPIRIT statements provide a broad overview of intervention reporting standards, designed to be supported by the TIDieR statement. To report a level of detail that allows true replication of a complex behavioural intervention, we found the BCC framework an essential addition to reporting aphasia treatment fidelity.

Study limitations

Limitations of the reporting of treatment fidelity in this trial related to limited video recordings in the UC arm. Whilst it was mandatory for therapists in the intensive arms of the trial to video-record sessions, the UC therapists were only encouraged to record sessions, and only two sessions were received, one of which was corrupted. Therefore, limited data was available for UC [17].The addition of treatment fidelity data for the UC arm would have provided valuable information about therapy adherence and differentiation between all therapy arms in the trial. The 66% of expected videos received were assumed to be representative of treatment as a whole; however, the possibility exists that it was not. The Hawthorne effect [18] may play an undetermined role whereby therapists changed their behaviour once aware they were being recorded. We believe this to be an unavoidable bias in the reporting of fidelity currently. Further, the treatment fidelity reported here was completed at the macrostructure level. Previously published work [22, 23] presents finer grained, utterance level analyses and adds nuanced therapeutic information to the efficacy picture. Finally, the receipt of treatment and enactment of treatment skills areas of Bellg et al. [5] were not measured in this study and so cannot be commented on.

Future directions

A main finding of this study indicated greater treatment protocol adherence in the arm of the trial that received the highest protocol training and more detailed treatment resources (including manualised treatment plans and within treatment session reporting). This suggests that greater detail in a treatment protocol coupled with increased training may result in increased protocol adherence. We hope that future studies will incorporate fidelity treatment processes similar to those outlined to build robust reporting systems for aphasia research to increase and improve reporting of treatment fidelity. The detailed level of treatment reporting is not exclusive to randomised controlled trials and should be reflected in all treatment research with this population. The critical nature of full treatment fidelity reporting when interpreting evidence highlights the importance of this level of detailed planning in aphasia research. The importance of explicit reporting of the theoretical rationale and the potential active ingredients as part of the study design is stressed here as the first step to measuring treatment fidelity. This will promote identification and subsequent measurement of hypothesised therapeutic elements, enabling their examination when assessing the fidelity of the study treatment being evaluated. Exploring ways to conceptualise and measure treatment enactment is an area of need in future research. Investigating the measurement of trained-skills carryover to naturalistic settings presents a significant challenge to aphasia intervention design and evaluation and inches us forward in the measurement of treatment efficacy.

Conclusion

We devoted substantial financial, logistical and intellectual resources to treatment fidelity, heeding comments from the BCC. The overall value of the detailed fidelity reporting in this clinical trial allows for strengthened interpretation of the ‘null’ VERSE trial results, in that, the early aphasia recovery does not appear to be enhanced by an intensive therapy regimen. Further research is urgently required to determine dose-related responses to treatment in early recovery. Commencing aphasia effectiveness research with a detailed treatment fidelity plan and reporting structure is essential. Vigilant monitoring and reporting of all intervention components will only enhance the aphasia research discovery pipeline [24].

Availability of data and materials

The datasets used and/or analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

VERSE:

Very Early Rehabilitation in SpEech

RCT:

Randomised controlled trial

UC:

Usual Care

UC-Plus:

Usual Care-Plus

TIDieR:

Template for Intervention Description and Replication

CONSORT:

Consolidated Standards of Reporting Trials

BCC:

Behaviour Change Consortium

AQ:

Aphasia Quotient

SPIRIT:

Standard Protocol Items: Recommendations for Intervention Trials

References

  1. Ang K, Hepgul N, Gao W, Higginson IJ. Strategies used in improving and assessing the level of reporting of implementation fidelity in randomised controlled trials of palliative care complex interventions: a systematic review. Palliat Med. 2018;32(2):500–16.

    Article  Google Scholar 

  2. Gearing RE, El-Bassel N, Ghesquiere A, Baldwin S, Gillies J, Ngeow E. Major ingredients of fidelity: a review and scientific guide to improving quality of intervention research implementation. Clin Psychol Rev. 2011;31(1):79–88.

    Article  Google Scholar 

  3. Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, et al. Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. Br Med J. 2014;348:g1687.

    Article  Google Scholar 

  4. Schulz KF, Altman DG, Moher D. CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. BMJ. 2010;340:c869. https://doi.org/10.1136/bmj.c869.

  5. Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, et al. Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH behavior change consortium. Health Psychol. 2004;23(5):443–51.

    Article  Google Scholar 

  6. Borrelli B. The assessment, monitoring, and enhancement of treatment fidelity in public health clinical trials. J Public Health Dent. 2011;71(s1):S52–63.

    Article  Google Scholar 

  7. Perepletchikova F, Kazdin AE. Treatment integrity and therapeutic change: issues and research recommendations. Clin Psychol Sci Pract. 2005;12(4):365–83.

    Article  Google Scholar 

  8. O’Shea O, McCormack R, Bradley J, O’Neill B. Fidelity review: a scoping review of the methods used to evaluate treatment fidelity in behavioural change interventions. Phys Ther Rev. 2016;21(3-6):207–14.

    Article  Google Scholar 

  9. Dipper LT, Franklin S, de Aguiar V, Baumgaertner A, Brady M, Best W, et al. An umbrella review of aphasia intervention description in research: the AsPIRE project. Aphasiology. 2021;36(4):1–26.

    Google Scholar 

  10. Hinckley J, Douglas NF. Treatment fidelity: its importance and reported frequency in aphasia treatment studies. Am J Speech-Language Pathol. 2013;22(2):S279–S84.

    Google Scholar 

  11. Brogan E, Ciccone N, Godecke E. Treatment fidelity in aphasia randomised controlled trials. Aphasiology. 2019;33(7):759–79.

    Article  Google Scholar 

  12. Hickin J, Cruice M, Dipper L. A systematically conducted scoping review of the evidence and fidelity of treatments for verb deficits in aphasia: verb-in-isolation treatments. Am J Speech-Language Pathol. 2020;29(1s):530–59.

    Article  Google Scholar 

  13. Conlon EL, Braun EJ, Babbitt EM, Cherney LR. Treatment fidelity procedures for an aphasia intervention within a randomized controlled trial: design, feasibility, and results. Am J Speech-Language Pathol. 2020;29(1S):412–24.

    Article  Google Scholar 

  14. Carragher M, Ryan B, Worrall L, Thomas S, Rose M, Simmons-Mackie N, et al. Fidelity protocol for the action success knowledge (ASK) trial: a psychosocial intervention administered by speech and language therapists to prevent depression in people with post-stroke aphasia. Br Med J Open. 2019;9(5):e023560.

    Google Scholar 

  15. Behn N, Hilari K, Marshall J, Simpson A, Northcott S, Thomas S, et al. SUpporting well-being through PEeR-befriending (SUPERB) trial: an exploration of fidelity in peer-befriending for people with aphasia. Aphasiology. 2018;32(sup1):21–3.

    Article  Google Scholar 

  16. Spell LA, Richardson JD, Basilakos A, Stark BC, Teklehaimanot A, Hillis AE, et al. Developing, implementing, and improving assessment and treatment fidelity in clinical aphasia research. Am J Speech-Language Pathol. 2020;29(1):286–98.

    Article  Google Scholar 

  17. Godecke E, Armstrong E, Rai T, Ciccone N, Rose ML, Middleton S, et al. A randomized control trial of intensive aphasia therapy after acute stroke: the very early rehabilitation for SpEech (VERSE) study. Int J Stroke. 2020;0(0):1747493020961926.

    Google Scholar 

  18. McCambridge J, Witton J, Elbourne DR. Systematic review of the Hawthorne effect: new concepts are needed to study research participation effects. J Clin Epidemiol. 2014;67(3):267–77.

    Article  Google Scholar 

  19. Harris PA, Taylor R, Thielke R, Payne J, Gonzalez N, Conde JG. Research electronic data capture (REDCap)—a metadata-driven methodology and workflow process for providing translational research informatics support. J Biomed Inform. 2009;42(2):377–81.

    Article  Google Scholar 

  20. Borrelli B, Sepinwall D, Ernst D, Bellg AJ, Czajkowski S, Breger R, et al. A new tool to assess treatment fidelity and evaluation of treatment fidelity across 10 years of health behavior research. J Consult Clin Psychol. 2005;73(5):852–60.

    Article  Google Scholar 

  21. Chan A-W, Tetzlaff JM, Altman DG, Laupacis A, Gøtzsche PC, Krleža-Jerić K, et al. SPIRIT 2013 statement: defining standard protocol items for clinical trials. Ann Intern Med. 2013;158(3):200–7.

    Article  Google Scholar 

  22. Brogan E, Ciccone N, Godecke E. An exploration of aphasia therapy dosage in the first six months of stroke recovery. Neuropsychol Rehabil. 2020;34(10):1–35.

    Google Scholar 

  23. Brogan E, Godecke E, Ciccone N. Behind the therapy door: what is “usual care” aphasia therapy in acute stroke management? Aphasiology. 2020;6(10):1–23.

    Google Scholar 

  24. Dalton E, Churilov L, Lannin NA, Corbett D, Hayward KS. Dose articulation in preclinical and clinical stroke recovery: refining a discovery research pipeline and presenting a scoping review protocol. Front Neurol. 2019;10(1148):1291–313.

    Google Scholar 

Download references

Acknowledgements

We would like to thank all participants and their families for taking part in this trial, the therapists who allowed us to video-record the therapy sessions and our treatment fidelity team (Fiona Ellery, Sanita Kratina, Crystal Ladzinski, Oriana Borschmann, Katie Ilich, Robynne Stevenson, Gemma Hughes, Jan Chamberlain) for the hours of monitoring and analysis that went into this publication.

Study data were collected and managed using REDCap® electronic data capture tools [19] hosted at The Florey Institute of Neuroscience and Mental Health, Victoria, Australia [1]. REDCap (Research Electronic Data Capture) is a secure, web-based application designed to support data capture for research studies, providing (1) an intuitive interface for validated data entry, (2) audit trails for tracking data manipulation and export procedures, (3) automated export procedures for seamless data downloads to common statistical packages and (4) procedures for importing data from external sources.

Funding

This study was funded by the National Health and Medical Research Council (NHMRC) (#1044973), 2013–2017; The Tavistock Trust for Aphasia (UK), 2017–2018; and Edith Cowan University, Australia, 2017–2018. Author DAC was supported by a National Health and Medical Research Council fellowship (#1063761 co-funded by Heart Foundation). Author JB was supported by a National Health and Medical Research Council fellowship (# 1154904).

Author information

Authors and Affiliations

Authors

Contributions

EG, EA, TR, NC, MLR, AW, AH, SM, GJH, DAC, JB. Conceived and developed the study; secured funding. EG, FE, EA. Co-ordinated the study. EG, EA, TR, NC, MLR, AW, AH, SM, GJH, DAC, JB. Drafted the main study protocol with input from listed authors. EG, EA, NC, MLR, AW, AH, FE. Developed and drafted the VERSE intervention protocol. EG, NC, EB. Collated and analysed the data. EG, NCWrote the first draft of the manuscript. EB Restructured the treatment fidelity reporting and results sections of the manuscript. EG, EB, EA, TR, NC, MLR, AW, AH, SM, GJH, DAC, JB, FE. Provided input and approved the final version of this paper.

Corresponding author

Correspondence to Erin Godecke.

Ethics declarations

Ethics approval and consent to participate

The trial was approved by hospital ethics committees and all participants gave written informed consent using aphasia friendly consent processes.

This study had Nationals Ethics approval via National Mutual Acceptance through The Austin Hospital (HREC13/Austin/101). All sites had site-specific agreements ethics approval based on the National Mutual Acceptance.

Consent for publication

Not applicable as individual data not used.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

 Therapy integrity monitoring form.

Additional file 2.

VERSE Prescribed treatment arm goal levels.

Additional file 3.

Study protocol deviations. Table S3. Study protocol—assessment deviations by treatment arm.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Godecke, E., Brogan, E., Ciccone, N. et al. Treatment fidelity monitoring, reporting and findings in a complex aphasia intervention trial: a substudy of the Very Early Rehabilitation in SpEech (VERSE) trial. Trials 23, 501 (2022). https://doi.org/10.1186/s13063-022-06433-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13063-022-06433-3

Keywords