Using mixed methods to evaluate efficacy and user expectations of a virtual reality–based training system for upper-limb recovery in patients after stroke: a study protocol for a randomised controlled trial
Trials volume 15, Article number: 350 (2014)
In recent years, virtual reality has been introduced to neurorehabilitation, in particular with the intention of improving upper-limb training options and facilitating motor function recovery.
The proposed study incorporates a quantitative part and a qualitative part, termed a mixed-methods approach: (1) a quantitative investigation of the efficacy of virtual reality training compared to conventional therapy in upper-limb motor function are investigated, (2a) a qualitative investigation of patients’ experiences and expectations of virtual reality training and (2b) a qualitative investigation of therapists’ experiences using the virtual reality training system in the therapy setting. At three participating clinics, 60 patients at least 6 months after stroke onset will be randomly allocated to an experimental virtual reality group (EG) or to a control group that will receive conventional physiotherapy or occupational therapy (16 sessions, 45 minutes each, over the course of 4 weeks). Using custom data gloves, patients’ finger and arm movements will be displayed in real time on a monitor, and they will move and manipulate objects in various virtual environments. A blinded assessor will test patients’ motor and cognitive performance twice before, once during, and twice after the 4-week intervention. The primary outcome measure is the Box and Block Test. Secondary outcome measures are the Chedoke-McMaster Stroke Assessments (hand, arm and shoulder pain subscales), the Chedoke-McMaster Arm and Hand Activity Inventory, the Line Bisection Test, the Stroke Impact Scale, the MiniMentalState Examination and the Extended Barthel Index. Semistructured face-to-face interviews will be conducted with patients in the EG after intervention finalization with a focus on the patients’ expectations and experiences regarding the virtual reality training. Therapists’ perspectives on virtual reality training will be reviewed in three focus groups comprising four to six occupational therapists and physiotherapists.
The interviews will help to gain a deeper understanding of the phenomena under investigation to provide sound recommendations for the implementation of the virtual reality training system for routine use in neurorehabilitation complementing the quantitative clinical assessments.
Cliniclatrials.gov Identifier: NCT01774669 (15 January 2013)
Virtual reality (VR)–based training is a fast-developing field of rehabilitation and has its origins in the gaming industry. In recent years, VR has been introduced into neurorehabilitation with the intention of improving upper-limb training options and facilitating motor function recovery. Virtual reality is defined as the ‘use of interactive simulations created with computer hardware and software to present users with opportunities to engage in environments that appear and feel similar to real-world objects and events’ .
So far, few studies have evaluated the efficacy of VR training in chronic stroke rehabilitation . In 2011, Laver et al. analysed 19 randomised controlled trials in their systematic literature review, which included only seven RCTs that focused on upper-limb motor function recovery . A meta-analysis of five trials revealed a moderate effect (Cohen’s d = 0.53) of VR compared to conventional therapy. However, six different commercial or customised VR training systems were used in the analysed trials. No conclusion could be reached regarding the effect of VR training on grip strength. Furthermore, no recommendations could be suggested regarding dosage, type or programme of the VR training.
Over the past few decades, VR technology and its applications have changed. Besides its potential to trigger external stimulation, it is hypothesized that VR induces use-dependent plastic changes in response to internal stimulation of higher motor cortical areas that recruit the motor memory system, which consists of stored motor programs. This so-called VR-based interactive cognitive intervention is based on the idea that stimulation of the action-processing system in turn activates downstream cortical areas involved in movement execution. There a population of neurons—‘mirror neurons’—is purported to play a key role, as these neurons discharge during both action execution and action observation or imagery . With the recruitment of a widespread movement network normally involved in movement execution, VR-based cognitive therapy offers a potential to specifically promote and/or enhance functional movement recovery. You et al. demonstrated important neuroplasticity changes in the primary sensorimotor cortex after VR training in stroke patients . The observed changes were associated with improved locomotion.
Other researchers have shown mirror-like visuomotor activity in the posterior parietal lobe in humans during object-related hand actions . Because task-oriented rehabilitation is known to be beneficial , this finding suggests that VR-based cognitive therapy may induce cortical plasticity and promote recovery by using goal-directed arm- and hand movements.
It is hypothesized that a system combining two elements—movement observation with intent to imitate and visualization of mirrored movements of the nonparetic limb—may optimally induce cortical plasticity and functional recovery in both acute and chronic stroke patients. The first element is based on the observation that mirror neurons discharge during goal-directed hand actions and also during observation of another individual performing a similar action. It has been proposed that mirror neurons constitute a vocabulary of hand actions . Their activation leads to recruitment of functionally interconnected cortical structures coupling action execution and observation. The execution–observation system has also been found in humans . Moreover, there is evidence that action observation may facilitate motor activity  and induce cortical plasticity . In addition to action execution and observation, mirror neurons and motor-planning areas are known to be activated during voluntary motor imagery, which selectively modulates muscle excitability .
In the evaluation of novel technologies developed for neurorehabilitation, it is important to evaluate efficacy and the user’s perspective to enhance successful implementation of the technology. In their literature review, Laver et al. suggested assessing patients’ motivation, engagement in the therapy and level of enjoyment . Therefore, a mixed-methods approach will be used in the study proposed here. Together with the randomised efficacy trial it is intended to conduct semistructured patient interviews and focus group interviews with the treating therapists. The interviews will help to provide more information and a deeper understanding of the phenomena under investigation to be used in developing sound recommendations for the implementation of the VR training system for routine use in neurorehabilitation.
The current study includes two parts: (1) a quantitative investigation of the efficacy of virtual reality training compared to conventional therapy in upper-limb motor function are investigated, (2a) a qualitative investigation of patients’ experiences and expectations of virtual reality training and (2b) a qualitative investigation of therapists’ experiences using the virtual reality training system in the therapy setting.
Methods and design
This phase III study consists of two parts and envisages a mixed-methods design comprising quantitative and qualitative research methods. The research questions, objectives, hypotheses and methodology of each part are described separately. Figure 1 illustrates the study design.
All study parts will conform to the guidelines of good clinical practice and the Declaration of Helsinki. Ethical approval was obtained from two responsible Swiss ethics committees (ECs): EC Aarau (2012/065) and EC Bern (220/12).
After study implementation, agreed-upon eligibility criteria or therapy content may limit the progress of the study. Therefore, each participating centre or therapist can suggest a modification. However, these modifications can be employed only if all centres agree to accept and apply the modifications and if the responsible ethics committees approve the modifications.
Data collection will be performed in the outpatient centre of a community hospital, an academic hospital and a rehabilitation clinic in the German-speaking part of Switzerland.
Part 1: randomised controlled trial
Study design and measurement events
The first part is designed as a randomised, controlled, assessor-blinded, multicentre trial with two parallel groups and repeated measurement events (MEs). Patients will be evaluated by a blinded assessor on five occasions: twice within 2 weeks at baseline, before intervention start (BL, T0); once after eight treatment sessions (T1); once immediately after the intervention (T2); and once after a 2-month follow-up period (FU).
Aim: The aim in the first part is to implement a multicentre, single-blind, randomised controlled trial to investigate the efficacy of VR training compared to conventional therapy. The first part will be implemented at three research sites in the German-speaking part of Switzerland and has two study arms: an experimental group (EG, VR training) and a control group (CG, conventional therapy).
Research question: The first part is designed to test the hypothesis that patients after stroke in EG show higher postintervention performance on the Box and Block Test (BBT) compared to CG patients.
Hypothesis: It is hypothesised (H0) that there will be no group differences after 16 training sessions or after the FU period. It is also hypothesised (H1) that there will be a group difference after the 16 training sessions and after the FU period.
Patient selection criteria and recruitment
Patients after stroke will be eligible for study participation if they fulfil the selection criteria listed in Table 1.
The patient recruitment strategy employs different approaches:
Patients will be recruited from the clinics’ inpatient or outpatient departments by physicians, therapists and nurses.
Patients will be recruited from the clinics’ patient database. Datasets will be screened for study selection criteria by the involved study personnel. If patients are eligible, they will receive a letter describing the study and including patient information. If patients are interested in participating, they can contact the study personnel in the responsible clinic by telephone, postal mail or email.
Patients will be recruited via a study information flyer provided on each clinic’s homepage and through patient self-help groups. If patients are interested in participating, they can contact the study personnel in the responsible clinic by telephone, postal mail or email.
Randomisation and group allocation
After the patient has received written and oral study information, written informed consent will be obtained from each patient. The procedure will be performed by the clinics’ local study coordinator or the blinded assessors before patient inclusion. Documents (patient information and consent forms) can be obtained from the first author. Patients will be randomly allocated to either the EG or the CG after the second ME (T0). Group allocation will be based on a computer-generated randomisation list (one for all centres), (MATLAB release 2007b; MathWorks, Natick, MA, USA) created by a researcher not involved in the study. The randomisation list will be stored at one clinic’s pharmacy. After the second ME (T0), the treating therapist will contact the pharmacist to disclose the group allocation and the current patient will start with the respective therapy. Group allocation will remain concealed from the independent assessor until study finalisation. Patients and treating therapists will be reminded not to talk with other therapists or participants about group allocations.
As the study is single-blind, the treating therapist will be informed about the patients group allocation. If any unexpected events (severe or nonsevere) occur, the treating therapist will inform the physician on duty and the responsible study organisation personnel to initiate all necessary procedures.
In intervention studies, it is challenging to keep the assessors blinded. To ensure an objective outcome evaluation, the following procedures will be incorporated:
Both interventions will take place in the same room.
All therapists can apply both interventions.
All patients and therapists will be advised not to talk to the assessor about therapy or training content.
Intervention documentation will not be accessible by the assessor.
If in any case blinding is uncovered, either a different blinded assessor will perform the assessments or the complete assessment session will be videotaped to allow for an objective evaluation by an assessor from a different participating centre.
Both study interventions are described in Table 2 using the Template for Intervention Description and Replication checklist and guide, known as TIDieR . Figure 2 shows the training set up for the experimental group.
Outcome parameters and related outcome measures
Table 3 provides an overview of all outcome measures and measurement events. All data will be collected on a case report form (CRF) that will be stored in a locked cabinet and will not be accessible by the treating therapist. The standardised CRF can be obtained from the first author (CSA).
Any patient, who leaves the trial will still be invited to all planned measurement events so that the recover process can be followed. The measurement event will include the same assessments planned for study participants (Table 3).
Primary outcome: hand dexterity
Change in hand dexterity between T0 and T2 is the primary outcome of interest. It will be measured with the BBT, which was described by Mathiowetz et al. in 1985 . The test is easy to administer and can be quickly performed. It has been used in patients after stroke in patients with multiple sclerosis and traumatic brain injury . Patients will be asked to grasp small wooden cubes and move them from one side of the box to the other as fast as possible within 60 seconds. The BBT is a reliable and valid assessment tool and provides normative data for healthy individuals in age groups ranging from 20 years to older than 75. A change of five or six cubes before and after an intervention seems to be the smallest real difference .
Secondary outcomes: upper-limb activities of daily living and motor and cognitive function
Upper-limb ADL and motor and cognitive function are secondary outcomes. They will be assessed objectively with the Chedoke-McMaster Stroke Assessment (CMSA), the Chedoke Arm and Hand Activity Inventory (CAHAI) and the Line Bisection Test (LBT) and subjectively with the Stroke Impact Scale (SIS).
The CMSA was developed by Gowland et al. in 1995 for the evaluation of physical impairment and activity level of stroke patients . In the present study, we will use the impairment subscales for hand and arm function. Patients will be scored on a seven-point scale (1 = hypoactive or absent muscle reflexes, 7 = no functional impairment detectable anymore, prestroke status) according to seven stages of motor recovery . Additionally, shoulder pain of the affected body side will be evaluated on the same seven-point scale using another subscale. The CMSA has been shown to be a valid and reliable assessment tool [15, 17].
The CAHAI was developed by Barreca et al. in 2004 [18–21]. It contains 13 real-life items scored from 1 to 7 (highest score). For example, one item is putting toothpaste on a toothbrush. Scores represent the patient’s relative ability to independently perform stabilisation or manipulation in ADL with the affected upper limb. A score of 1 represents total dependence on another person, and a score of 7 indicates patient independence without time or safety concerns or necessary splints or devices. The test’s reliability and validity have been evaluated [20, 21].
The LBT is a paper-and-pencil test used to evaluate the presence of unilateral spatial neglect (USN) . Patients are asked to mark the centre of a drawn line on paper with a pencil. This is repeated for 18 lines on a sheet of paper. A deviation of more than 6 mm from the centre in the upper and lower 9 lines indicates the presence of USN.
The SIS is a questionnaire comprising questions regarding the impact of stroke on physical function, emotion, memory, communication and social participation. The SIS was developed by Duncan and colleagues in 2001 and has been modified in recent years . The current version, 3.0, consists of eight subscales (strength, hand function, mobility, ADL, emotion, memory, communication and participation) comprising a total of 59 questions that should be administered in a one-to-one interview. Patients can rate the level of their stroke’s impact on a 5-point Likert scale. The reliability, validity and sensitivity to change of this instrument have been evaluated for version 2.0 . Official translations have been produced for 14 languages. The higher the score, the less affected the patient perceives his or her current status to be. A clinically important difference is indicated by changes of 9.2, 5.9, 4.5 and 17.8 points for the strength, ADL, mobility and hand function subscales, respectively .
Further assessments and evaluations
At study entry, the MiniMental State Examination , the Edinburgh Handedness Inventory  and the Extended Barthel Index  will be used for patient evaluation and descriptive purposes at BL only. Furthermore, patients’ personal information regarding age, gender, marital status, education and profession will be recorded. All currently received therapy sessions will be documented by the examiners during BL assessment. If patients receive another mode of therapy, they will be asked to reduce or suspend it for the duration of the study.
Sample size and statistical analyses
On the basis of a study on the efficacy of an earlier study in which the investigators tested the VR system in children with cerebral palsy , a power analysis and sample size calculation for the present study were performed with G*Power software version 3.1.5 . In the cited study of children, the BBT (primary outcome measure) showed an effect size (Cohen’s d) of 0.98. Assuming a similar effect size for adult stroke patients, we would need (two-tailed test, power = 0.9, significance level P-value = 0.05) a total of 46 patients (23 per group). Assuming a dropout rate of 20%, we will recruit a total of 60 patients across the participating centres.
Data will be analysed qualitatively and quantitatively for each patient separately. Changes will be provided as a total number or percentage. Group comparisons between EG and CG will be analysed using analysis of covariance after assessment of normal distribution. We intend to perform an intention-to-treat analyses. If necessary, an additional per protocol analysis will be carried out. Furthermore, subgroup analyses will be performed to investigate any possible study site effects. Patients’ training settings (nominal and interval data) will be analysed with frequencies and percentages to describe therapy content and provide recommendations for further therapy settings.
All study personnel will try to avoid patient dropouts. However, patients can leave the study at any time without reason. Collected patient data until the point of dropout will be included in the final analysis. Missing data will be replaced using two different approaches: (1) with the last available value carried forward method and (2) by adding or subtracting the mean difference of other patients in the respective group. Analysis with both approaches will be performed.
Study design and measurement events
It is highly relevant to consider the patients’ and therapists’ perspectives, in particular if new therapy systems will be evaluated regarding their effectiveness. Part 2 comprises two qualitative investigations to explore patients’ and therapists’ experiences with the VR training system. The gained knowledge can be used to improve the training system and to formulate treatment guidelines.
Aim: The aim of the investigations is the exploration and description of the patients’ and treating therapists’ experiences and expectations during the intervention with the VR training system. Semistructured interviews will be conducted with patients in EG and focus groups will be performed with therapists, who have trained patients with the VR training system.
Main research question: How do patients and therapists experience the intervention with the VR training system?
Semistructured interviews with patients
Patients allocated to EG will be offered the opportunity to participate in one semistructured interview after intervention finalisation. They will be informed about and asked for participation during the third or fourth week of the VR training intervention. The face-to-face interview will take between 30 to 60 minutes and will be conducted at the place of the patient’s convenience, either at the treating clinic or at the patient’s home. To provide a high level of privacy, all interviews will take place in a separate and quiet room. Participation will be voluntary, and all interviews will be voice-recorded.
A purposive sampling method will be employed according to the patients’ experienced phenomenon of interest . It is proposed to include a sample of about ten patients to explore and to describe in detail experiences and new knowledge gained in a particular field .
Interview guide semistructured interviews
Semistructured interviews consist of open-ended questions . The loose structure allows the opportunity to explore and follow an idea deeper without a preimposed interview structure . It is intended to provide an in-depth description of the explored phenomenon . On the basis of existing literature, an interview guide has been developed . The guide divides the interview into three parts (see Additional file 1):
Welcoming, information about the interview procedure, questions regarding the rehabilitation process and querying the patient’s current condition.
Questions regarding the main research question, such as, What do you think about the VR intervention? How did you feel during the intervention? What were your concerns related to the training system? How did you experience the role of the therapist during the VR training?
Interview summary by the interviewer, final remarks of the patient and farewell.
Focus groups with therapists
Therapists treating patients with the VR training system will be offered the opportunity to participate in a focus group session that will last from 1 to 2 hours. Therapists will be informed about and asked for participation after they have completed at least one series of patient training with the VR training system, including 16 training sessions or a comparable number of training sessions with different patients. The focus group sessions will be conducted at one of the participating clinics, and three to six therapists will participate in one focus group. Participation will be voluntary, and all interviews will be voice-recorded.
Interview guide focus groups
Welcoming, information about the interview procedure, introduction of focus group participants and taking general questions on the VR training experience.
Questions regarding the main research question, such as, How did you experience the patient during the VR training? How did you experience the patient’s motivation? How did you feel during the training with VR?
Interview summary of the interviewer, final remarks of the participants and farewell.
Semistructured interview and focus group questions were tested with two patients and therapists beforehand to check for comprehensibility and clarity and were reviewed by the responsible ethics committees.
Data analysis of semistructured interviews and focus groups
In addition to the voice-recording of the interviews and focus groups, interviewers will write field notes to describe the interview situation: the way the patient acts during the interview, the patient’s mood and feelings, the course of the interview, unexpected events or statements, and the feelings and impressions of the interviewer. Field notes and the interview and focus group content will provide the basis for the data analysis, which will be based on a descriptive phenomenological approach without data or opinion interpretation . Data analysis will include three steps:
TRANSCRIPTION (step 1): Interview content will be transcribed verbatim.
CONDENSATION (step 2): Patients’ and therapists’ quotes will be summarised to highlight the main statement.
CODING and CATEGORISATION (step 3): On the basis of the thematic analysis, categories and codes will be created to sort patients’ and therapists’ statements . This can be done with the help of a qualitative data analysis tool such as NVivo  or ATLAS.ti .
Overall ethical considerations and reporting of adverse events
The VR training system is a commercial product sold under the name YouGrabber. It uses VR to train upper-limb motor function. It has been available (initially as a beta version) for testing purposes in clinics since 2010 and has been used in rehabilitation institutions and pilot clinical studies with no safety-related incidents. Nevertheless, patients might experience negative emotions or have the impression that they cannot perform the training tasks because of their reduced motor function. It is expected that study participation will take a lot of patients’ time in both groups. The following measures will be utilized in exchange for the patients’ time exposure in the current study:
Patients will be assessed on a regular basis to evaluate their rehabilitation process.
Patients will have an intensive therapy schedule with four therapy sessions per week over the course of 4 weeks.
Patients will have the opportunity to learn about an innovative VR training system.
Patients can withdraw from the study at any time without giving a reason.
The study will be carried out in accordance with the protocol, with the guidelines for good clinical practice and current national and international valid legal provisions. Any adverse event will be reported to the responsible ethics committee directly and will be mentioned in intermediate and final study reports. The study is registered in the Clinicaltrials.gov trials database with the identifier NCT01774669.
Quality control and quality assurance
To achieve and maintain a high quality standard during study preparation and implementation, the following measures will be employed:
Comprehensive training for all study therapists to use the VR training system before study start.
Comprehensive training for all study therapists to perform all necessary outcome measures before study start.
Three refresher training days for VR system training and assessment performance throughout the study
Study management updates for all centres on a regular basis (monthly).
Daily telephone and email access for technical support.
Furthermore, up to three monitoring events will be performed for each participating study centre. An authorised person from the sponsor, which is not involved in patient treatment or assessments, will visit the clinics and inspect data handling and patient organisation. Up to five randomly selected patient data files will be inspected.
All study-related patient data will be entered directly onto the anonymised case report forms or assessment scoring sheets. Study-related original patient documents will remain in the responsible clinic and will be archived for 10 years. After that period, patient documents will be destroyed in accordance with the clinic’s data destruction guidelines. Only anonymised electronic data or paper copies will be sent to the study coordinator for data analyses.
Training data collected via the VR training system will be saved in anonymised form under the patient’s study ID, with no other identifying information. Data backup and synchronization will be carried out using a secure, encrypted data transfer and storage framework.
The study personnel will adhere to an open access policy:
The trialists intend to publish the study protocol and the study results in international open access journals to provide easy access to the study documents for all interested readers.
The study is registered in an international open access clinical trial database (Cliniclatrials.gov Identifier: NCT01774669).
As the study progresses, its methods and preliminary results will be presented at national and international congresses and workshops.
After study finalisation and data analyses, all study patients will receive a plain language summary of the study results, and the results will be presented in each participating clinic.
Involvement of professional writers is not intended. No restrictions will be placed on the publication of positive or negative results. The study results will be reported in accordance with the guidelines set forth in the 2010 Consolidated Standards of Reporting Trials (CONSORT) .
Criteria for halting the trial
At present, the commercial YouGrabber system has been used for more than 2 years with over 100 patients in different acute hospitals and rehabilitation clinics. So far, no adverse events have been reported. However, this study will be halted if any of the following criteria are fulfilled:
More than three EG patients report a sudden onset of or increase in shoulder pain during or just after therapy that is highly likely to be attributable to the use of YouGrabber, and which does not immediately cease after stopping therapy (evaluated as ≤3 on the Chedoke-McMaster Stroke Assessment subscale pain).
More than 50% of EG patients (minimum of five patients) report severe cybersickness during YouGrabber training which persists after training is halted.
EG patients show an unexpected decrease in motor function (change of three or more levels in the Chedoke-McMaster Stroke Assessment) indicating a highly significant difference between therapies with YouGrabber compared to conventional therapy.
Epileptic seizures in at least two patients are induced directly while using YouGrabber.
Patients reporting the criteria mentioned above will be evaluated by the physician on duty and will be assessed and followed up for the originally planned study duration.
Unexpected events will be categorised as severely unexpected events (harmful) and unexpected events (nonharmful). Each event will be documented on a predefined form that is available from the first author (CSA) and will be sent to the overall study coordinator (DK). He will decide if the study insurance provider has to be notified about the event. All events will be reported to the responsible ethics committee independently from the study insurance provider.
The aim of this multicentre trial is to determine the efficacy of a VR-based upper-limb training programme compared to conventional occupational therapy and physiotherapy. Additionally, the users’ perspectives will be evaluated by conducting semistructured patient interviews and therapist focus groups.
In the rapidly evolving VR training industry, it is important to develop a detailed perspective on different aspects of the training interventions used, based on the patients’ and therapists’ perceptions. This goal will be achieved by using a mixed-methods design combining quantitative and qualitative research methods. In particular, from the qualitative data, new hypotheses for further research and adaptations will emerge, including training recommendations and system modifications.
In addition, the knowledge gained in the present study will contribute to the trends described in the review by Laver et al. . On the basis of their systematic review of the literature reaching back to March 2010 or earlier, they claimed that the most beneficial patient and VR system characteristics are still undetermined.
The design of a multicentre study offers several advantages and challenges. The shared patient recruitment and treatment methodology provides the opportunity to conduct a sufficiently powered trial to reduce the risk of type I error . Also, therapists and patients from various clinics and different parts of Switzerland can evaluate the VR training options according to personal preferences and local conditions.
In the present study, we decided to use only one randomisation list that will be stored in the pharmacy at one of the three clinics. Therefore, the risk of unblinding the group allocation is reduced, and patient study entry, measurement events and treatment sessions will be under control of one clinic that is responsible for study management.
The planned study is single-blinded only. In general, a sham or placebo intervention in therapeutic disciplines is not always possible. In this study, the treating therapists and patients cannot be blinded to the study group interventions. However, the assessor and the researchers performing the data analyses will be blinded to increase the study’s internal validity and to ensure a low detection bias .
Furthermore, each clinic will assign a responsible therapist, who will coordinate patient recruitment and study organisation and conduct therapist training. Regularly scheduled team meetings and study newsletters will help to ensure employment of homogeneous procedures regarding study management, patient treatment and communication with the overall study leader and among all involved clinics.
This study is currently recruiting participants. It is anticipated that the trial will take 18 to 36 months for completion, including all follow-up assessments. The first patient was randomised in February 2013.
The authors are thankful to the model in our figure, who illustrates the equipment of the VR training system. Written informed consent was obtained from the model for publication of this manuscript and the accompanying image. A copy of the written consent is available for review by the Editor-in-Chief of this journal.
Activities of daily living
Box and Block Test
Baseline measurement 2 weeks before intervention start
Chedoke McMaster Arm and Hand Activity Inventory
Chedoke-McMaster Stroke Assessment
Extended Barthel Index
Edinburgh Handedness Inventory
Follow-up measurement event
Line Bisection Test
Stroke Impact Scale
Measurement event before intervention start
Measurement event after 8 intervention sessions
Measurement event after 16 intervention sessions
Weiss P, Kizony R, Feintuch U, Katz N: Virtual reality in neurorehabilitation. Textbook of Neural Repair and Rehabilitation. Edited by: Selzer M, Cohen LG, Gage F, Clarke S, Duncan P. 2006, Cambridge University Press, 182-197.
Laver KE, George S, Thomas S, Deutsch JE, Crotty M: Virtual reality for stroke rehabilitation. Cochrane Database Syst Rev. 2011, 9: CD008349
A holistic approach to stroke care how a therapeutic group was set up to help patients come to terms with the psychological effects of stroke. Nurs Times. 2002, 98: 33-35.
You SH, Jang SH, Kim YH, Hallett M, Ahn SH, Kwon YH, Kim JH, Lee MY: Virtual reality–induced cortical reorganization and associated locomotor recovery in chronic stroke: an experimenter-blind randomized study. Stroke. 2005, 36: 1166-1171. 10.1161/01.STR.0000162715.43417.91.
Centers for Disease Control and Prevention: Prevalence of stroke—United States, 2005. MMWR Morb Mortal Wkly Rep. 2007, 56: 469-474.
The Innovative Medicines Initiative (IMI) Strategic Research Agenda: 2006, [http://ec.europa.eu/research/fp6/pdf/innovative_medicines_sra_final_draft_en.pdf] (accessed 27 August 2014)
Aben I, Denollet J, Lousberg R, Verhey F, Wojciechowski F, Honig A: Personality and vulnerability to depression in stroke patients: a 1-year prospective follow-up study. Stroke. 2002, 33: 2391-2395. 10.1161/01.STR.0000029826.41672.2E.
Hoffmann TC, Glasziou PP, Boutron I, Milne R, Perera R, Moher D, Altman DG, Barbour V, Macdonald H, Johnston M, Lamb SE, Dixon-Woods M, McCulloch P, Wyatt JC, Chan AW, Michie S: Better reporting of interventions: template for intervention description and replication (TIDieR) checklist and guide. BMJ. 2014, 348: g1687-10.1136/bmj.g1687. doi:10.1136/bmj.g1687
Eng K, Siekierka E, Pyk P, Chevrier E, Hauser Y, Cameirao M, Holper L, Hägni K, Zimmerli L, Duff A, Schuster C, Bassetti C, Verschure P, Kiper D: Interactive visuo-motor therapy system for stroke rehabilitation. Med Biol Eng Comput. 2007, 45: 901-907. 10.1007/s11517-007-0239-1.
Luke C, Dodd K, Brock K: Outcomes of the Bobath concept on upper limb recovery following stroke. Clin Rehabil. 2004, 18: 888-898. 10.1191/0269215504cr793oa.
Barreca S, Wolf SL, Fasoli S, Bohannon R: Treatment interventions for the paretic upper limb of stroke survivors: a critical review. Neurorehabil Neural Repair. 2003, 17: 220-226. 10.1177/0888439003259415.
Mathiowetz V, Volland G, Kashman N, Weber K: Adult norms for the box and block test of manual dexterity. Am J Occup Ther. 1985, 39: 386-391. 10.5014/ajot.39.6.386.
Connell LA, Tyson SF: Clinical reality of measuring upper-limb ability in neurologic conditions: a systematic review. Arch Phys Med Rehabil. 2012, 93: 221-228. 10.1016/j.apmr.2011.09.015.
Chen HM, Chen CC, Hsueh IP, Huang SL, Hsieh CL: Test-retest reproducibility and smallest real difference of 5 hand function tests in patients with stroke. Neurorehabil Neural Repair. 2009, 23: 435-440. 10.1177/1545968308331146.
Gowland C, Van Hullenaar S, Torresin W, Moreland J, Vanspall B, Barreca S, Ward M, Huijbregts M, Stratford P, Barclay-Goddard R: Chedoke-McMaster Stroke Assessment: Development, Validation, and Administration Manual. 1995, Hamilton, ON, Canada: School of Rehabilitation Science, McMaster University
Gowland CA: Staging motor impairment after stroke. Stroke. 1990, 21 (9 Suppl): II19-II21.
Gowland C, Stratford P, Ward M, Moreland J, Torresin W, Van Hullenaar S, Sanford J, Barreca S, Vanspall B, Plews N: Measuring physical impairment and disability with the Chedoke-McMaster Stroke Assessment. Stroke. 1993, 24: 58-63. 10.1161/01.STR.24.1.58.
Barreca S, Gowland CK, Stratford P, Huijbregts M, Griffiths J, Torresin W, Dunkley M, Miller P, Masters L: Development of the Chedoke Arm and Hand Activity Inventory: theoretical constructs, item generation, and selection. Top Stroke Rehabil. 2004, 11: 31-42. 10.1310/JU8P-UVK6-68VW-CF3W.
Barreca S, Stratford P, Masters L, Lambert CL, Griffiths J, McBay C: Validation of three shortened versions of the Chedoke Arm and Hand Activity Inventory. Physiother Can. 2006, 58: 148-156. 10.3138/ptc.58.2.148.
Barreca SR, Stratford PW, Lambert CL, Masters LM, Streiner DL: Test-retest reliability, validity, and sensitivity of the Chedoke Arm and Hand Activity Inventory: a new measure of upper-limb function for survivors of stroke. Arch Phys Med Rehabil. 2005, 86: 1616-1622. 10.1016/j.apmr.2005.03.017.
Schuster C, Hahn S, Ettlin T: Objectively-assessed outcome measures: a translation and cross-cultural adaptation procedure applied to the Chedoke McMaster Arm and Hand Activity Inventory (CAHAI). BMC Med Res Methodol. 2010, 10: 106-10.1186/1471-2288-10-106.
Plummer P, Morris ME, Dunai J: Assessment of unilateral neglect. Phys Ther. 2003, 83: 732-740.
Duncan PW, Wallace D, Studenski S, Lai SM, Johnson D: Conceptualization of a new stroke-specific outcome measure: the Stroke Impact Scale. Top Stroke Rehabil. 2001, 8: 19-33. 10.1310/BRHX-PKTA-0TUJ-UYWT.
Duncan PW, Wallace D, Lai SM, Johnson D, Embretson S, Laster LJ: The Stroke Impact Scale Version 2.0: evaluation of reliability, validity, and sensitivity to change. Stroke. 1999, 30: 2131-2140. 10.1161/01.STR.30.10.2131.
Lin KC, Fu T, Wu CY, Wang YH, Liu JS, Hsieh CJ, Lin SF: Minimal detectable change and clinically important difference of the Stroke Impact Scale in stroke patients. Neurorehabil Neural Repair. 2010, 24: 486-492. 10.1177/1545968309356295.
Folstein MF, Folstein SE, McHugh PR: “Mini-Mental State”: a practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975, 12: 189-198. 10.1016/0022-3956(75)90026-6.
Oldfield RC: The assessment and analysis of handedness: the Edinburgh Inventory. Neuropsychologia. 1971, 9: 97-113. 10.1016/0028-3932(71)90067-4.
Jansa J, Pogacnik T, Gompertz P: An evaluation of the Extended Barthel Index with acute ischemic stroke patients. Neurorehabil Neural Repair. 2004, 18: 37-41. 10.1177/0888439003262287.
van Hedel HJA, Wick K, Eng K, Meyer-Heim A: Improving dexterity in children with cerebral palsy: preliminary results of a randomised trial evaluating a glove based VR-system. Proceedings of the 2011 International Conference on Virtual Rehabilitation (ICVR): 2011 June 27–29; Zurich. 2011, 127-132. doi:10.1109/ICVR.2011.5971872, International Conference on Virtual Rehabilitation (ICVR): 2011 June 27–29; Zurich
Faul F, Erdfelder E, Lang AG, Buchner A: G*Power 3: a flexible statistical power analysis program for the social, behavioural, and biomedical sciences. Behav Res Methods. 2007, 39: 175-191. 10.3758/BF03193146.
Starks H, Trinidad SB: Choose your method: a comparison of phenomenology, discourse analysis, and grounded theory. Qual Health Res. 2007, 17: 1372-1380. 10.1177/1049732307307031.
Kvale S, Brinkmann S: InterViews: Learning the Craft of Qualitative Research Interviewing. 2009, Thousand Oaks, CA: Sage, 2
Britten N: Qualitative interviews in medical research. BMJ. 1995, 311: 251-253. 10.1136/bmj.311.6999.251.
Finlay L: Phenomenology for Therapists: Researching the Lived World. 2011, Chichester, UK: Wiley-Blackwell
Hughes AM, Burridge J, Freeman CT, Donnovan-Hall M, Chappell PH, Lewin PL, Rogers E, Dibb B: Stroke participants’ perceptions of robotic and electrical stimulation therapy: a new approach. Disabil Rehabil Assist Technol. 2011, 6: 130-138. 10.3109/17483107.2010.509882.
Speziale HS, Carpenter DR: Qualitative Research in Nursing: Advancing the Humanistic Imperative. 2007, Philadelphia: Lippincott Williams & Wilkins, 4
Braun V, Clarke V: Using thematic analysis in psychology. Qual Res Psychol. 2006, 3: 77-101. 10.1191/1478088706qp063oa.
Wiltshier F: Researching with NVivo. Forum Qual Soc Res. 2011, 12 (1):
Mühlmeyer-Mentzel A: The logical structure of data in ATLAS.ti and its advantage for grounded theory studies. Forum Qual Soc Res. 2011, 12 (1):
Moher D, Hopewell S, Schulz KF, Montori V, Gøtzsche PC, Devereaux PJ, Elbourne D, Egger M, Altman DG: CONSORT 2010 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ. 2010, 340: c869-10.1136/bmj.c869.
Munro BH: Statistical Methods for Health Care Research. 2005, Philadelphia: Lippincott Williams & Wilkins, 5
Higgins JP, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, Savovic J, Schulz KF, Weeks L, Sterne JA: The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ. 2011, 343: d5928-10.1136/bmj.d5928.
The authors are grateful to Dr Edith Chevrier and Dr Pawel Pyk for providing technical support in the preparation of the VR training systems for the study. We would like to thank the pharmacists, who will take over the responsibility for the randomisation list. The study is funded by the Swiss Commission of Technology and Innovation (CTI: email@example.com, +41 (0)31 322 21 29) and YouRehab Ltd (firstname.lastname@example.org, +41 44 5861386). YouRehab Ltd provides the virtual reality–based training systems for all three participating centres.
CSA, IL, LS, IT, MV, AH, SS and MM declare that they have no competing interests. KE, NK and DK declare that they have commercial interests in the VR system being evaluated in this study. The study funders (Commission for Technology and Innovation (CTI)) will have no authority over and will not be involved in the study design; the collection, management, analysis or interpretation of data; the writing of the report; and the decision to submit the report for publication. The study sponsor (YouRehab Ltd) will be involved in the study design and data analysis and interpretation. They will not write the study report and will not have authority over the decision to submit the study report for publication.
DK, KE, CSA, IL and LS initiated the study design. IL, IT, MLV, NK, AH, MM and SS helped with implementation of the study design. DK is the grant holder. DE, KE and CSA provided statistical expertise in the development of the clinical trial design and sample size calculation. All authors contributed to refinement of the study protocol. All authors read and approved the final manuscript.
About this article
Cite this article
Schuster-Amft, C., Eng, K., Lehmann, I. et al. Using mixed methods to evaluate efficacy and user expectations of a virtual reality–based training system for upper-limb recovery in patients after stroke: a study protocol for a randomised controlled trial. Trials 15, 350 (2014). https://doi.org/10.1186/1745-6215-15-350