The promotion of well-being is an important goal of recovery oriented mental health services. No structured, evidence-based intervention exists that aims to increase the well-being in people with severe mental illness such as psychosis. Positive psychotherapy (PPT) is a promising intervention for this goal. Standard PPT was adapted for use with people with psychosis in the UK following the Medical Research Council framework for developing and testing complex interventions, resulting in the WELLFOCUS Model describing the intended impact of WELLFOCUS PPT. This study aims to test the WELLFOCUS Model, by piloting the intervention, trial processes, and evaluation strategy.
This study is a non-blinded pragmatic pilot RCT comparing WELLFOCUS PPT provided as an 11-session group therapy in addition to treatment as usual to treatment as usual alone. Inclusion criteria are adults (aged 18–65 years) with a main diagnosis of psychosis who use mental health services. A target sample of 80 service users with psychosis are recruited from mental health services across the South London and Maudsley NHS Foundation Trust. Participants are randomised in blocks to the intervention and control group. WELLFOCUS PPT is provided to groups by specifically trained and supervised local therapists and members of the research team. Assessments are conducted before randomisation and after the group intervention. The primary outcome measure is well-being assessed by the Warwick-Edinburgh Mental Well-being Scale. Secondary outcomes include good feelings, symptom relief, connectedness, hope, self-worth, empowerment, and meaning. Process evaluation using data collected during the group intervention, post-intervention individual interviews and focus groups with participants, and interviews with trial therapists will complement quantitative outcome data.
This study will provide data on the feasibility of the intervention and identify necessary adaptations. It will allow optimisation of trial processes and inform the evaluation strategy, including sample size calculation, for a future definitive RCT.
Current Controlled Trials ISRCTN04199273 – WELLFOCUS study: an intervention to improve well-being in people with psychosis, Date registered: 27 March 2013, first participant randomised on 26 April 2013.
Developing a recovery orientation in mental health services is a policy goal in the UK and internationally . Recovery is an individual process of gradual restoration from the illness, focusing on the resources and abilities of the individual instead of exclusively treating symptoms. Hence, supporting recovery from severe mental illness involves, in addition to treating symptoms, an increased emphasis on personal strengths , positive identity development , and the promotion of well-being .
Well-being is not only a central component of recovery from mental illness , its importance is further supported by research showing an association between well-being and improved functioning [6, 7], increased resilience and life satisfaction , and suggesting its protective value against the onset or re-occurrence of mental illness [8, 9]. There is strong evidence that well-being is not only a desirable outcome in its own right, but also a statistically significant predictor of symptomatic response in the treatment of people with psychosis [10, 11] and strongly associated with medication compliance in this group . Despite the relevance of well-being research to recovery , no structured intervention based on an empirically-defensible theory exists which targets well-being in people with severe mental illnesses such as psychosis .
The promotion of well-being is a focus of the academic discipline of positive psychology; one intervention based on this body of knowledge is positive psychotherapy (PPT). PPT was originally developed for depression as the target condition, following the hypothesis that depression can not only be treated effectively by reducing its negative symptoms but also by directly and primarily building positive emotions, character strengths, engagement, and meaning .
Evidence from intervention studies suggests that standard PPT is beneficial for people suffering distress or common mental disorders. Randomised controlled trials (RCTs) comparing PPT with no treatment show increased positive affect and life satisfaction and decreased depressive symptoms in undiagnosed students , in a self-referred online sample , in undiagnosed middle school students , and in students with a diagnosis of mild to moderate or severe depression [15, 18]. PPT also showed statistically significant well-being gains and depressive symptom relief as compared to placebo therapy (e.g., recording early memories) in RCTs involving students with mild to moderate depression and in an undiagnosed community sample [15, 19]. Further small studies from Iran (RCT) and Chile (quasi-experimental control group design) comparing PPT to behavioural therapy in patients with diagnosed depression showed PPT to be more effective in increasing happiness and decreasing depression [20, 21]. Moreover, a meta-analysis of RCT evidence demonstrated that positive psychology interventions in general, including mindfulness, positive writing, hope, gratitude, forgiveness, kindness therapies, Fordyce’s Happiness Program, and PPT, statistically significantly improve well-being and decrease depressive symptoms for people with depression . A dose–response effect was also present, with positive effect increasing with the number of different positive exercises in a therapy programme. Recently, a small uncontrolled feasibility study of PPT yielded promising results for people with psychosis in the US, statistically significantly increasing participants well-being, savouring beliefs, hope, self-esteem, and personal recovery scores .
Based on the above preliminary evidence, we adapted standard PPT  for use with people with psychosis in a UK context. The scientific framework for the study is the Medical Research Council (MRC) Framework for Evaluating Complex Health Interventions . This involved a systematic review , a qualitative study , and an expert consultation to adapt the intervention and develop an intervention model to be tested in the present pilot RCT.The new intervention, i.e., WELLFOCUS PPT, has four target areas: increasing positive experiences, amplifying strengths, fostering positive relationships, and creating a more meaningful self-narrative. The WELLFOCUS model describes the intended impact of WELLFOCUS PPT, and is shown in Figure 1.
The aim of this pilot RCT is to test the WELLFOCUS model. The three objectives are:
Objective 1: Piloting the intervention
To identify whether WELLFOCUS PPT is feasible and acceptable and determine any necessary modifications.
Objective 2: Piloting the trial processes
To test procedures for a future definitive RCT, especially in relation to eligibility criteria, randomisation procedures, allocation processes, and recruitment and retention rates.
Objective 3: Piloting the evaluation strategy
To test approaches to assessing fidelity, process evaluation, and outcome evaluation, to inform the design of a future definitive RCT including choice of primary and secondary outcome measures, and sample size calculation.
While we shall conduct hypothesis testing, this is not the main objective of the study, as this is the purpose of the future definitive RCT. The reporting of the study will not therefore emphasise the results of hypothesis testing over the reporting of whether the objectives were met .Our hypotheses are derived from the WELLFOCUS Model shown in Figure 1. Regarding the effect of the intervention, we hypothesise that participants receiving the intervention will experience, compared to the control group, an improvement in well-being (primary outcome) and good feelings, symptoms, connectedness, hope, self-worth, empowerment, and meaning in life (secondary outcomes).
This is a single centre pilot RCT to test WELLFOCUS PPT in a group format in a convenience sample of people with psychosis. Patients are block-randomised to receive either WELLFOCUS PPT in addition to treatment as usual, or to continue to receive treatment as usual only. The design of this pilot RCT has been informed by recommendations for the conduct of pilot trials by Tabane et al.  and Lancaster et al. . The Consort flow diagram  is shown in Figure 2.
Ethical approval for the WELLFOCUS trial was obtained from the Camberwell St Giles Research Ethics Committee (reference 12/LO/1960). R&D approval was obtained from South London and Maudsley NHS Foundation Trust. Participants receive verbal and written information, and written informed consent is obtained from participants before they enter the study. The research is conducted in compliance with the Declaration of Helsinki .
Patients are recruited from seven teams and two research registers across the South London and Maudsley NHS Foundation Trust (SLaM). SLaM employs 4,500 staff in 296 teams, works with 34,128 service users, and provides adult mental health services across four Boroughs (Croydon, Lambeth, Lewisham, and Southwark). These services are provided through Clinical Academic Groups organised according to diagnoses; these groups bring together clinical services, research, education, and training for the benefit of patient care. This study takes place in the SLaM Psychosis Clinical Academic Group.
Patients are eligible to participate if they are aged 18–65 years, have a primary clinical diagnosis of psychosis, are using specialist mental health services, are not currently in prison, speak and understand English, and, in the opinion of their key clinician, are sufficiently well to participate in a group therapy.
Patients with serious cognitive impairment preventing meaningful participation in a group intervention, and those who in the opinion of their key clinician are unable to give consent or are too unwell to be interviewed.
A sample size of n = 30 per arm has been recommended for pilot studies to estimate location (mean) and variability (standard deviation) of candidate outcome measures to inform later sample size calculation . WELLFOCUS PPT is conducted as a group intervention with about 8 participants in each group. We aim to conduct five waves of PPT groups versus control, each starting with 16 participants (n = 8 in the intervention and n = 8 in the control arm), giving a total sample of 80. Allowing for a drop-out rate of 20%, consistent with attrition in a feasibility study of a similar intervention with service users with psychosis , this gives an analysable sample of n = 32 per arm to generate estimates for well-being as our primary outcome. However, we will try to minimize dropouts in our study as suggested by Little et al. .
The Warwick-Edinburgh Mental Well-being Scale (WEMWBS) was shown to have a one-week re-test reliability of 0.83 . The correlation between baseline and 3-months follow-up can be assumed to be below that. Assuming a correlation of 0.7 the power to detect an effect size of 0.5 on the WEMWBS at follow-up, adjusted for baseline, would be at 0.80. A more conservative estimate of a correlation between baseline and follow-up of 0.6 would allow us to detect the same effect size with a power of 0.71. Higher effect sizes would provide higher power. For example, an effect size of 0.6 could be detected with a power of 0.85 assuming a correlation between baseline and follow-up of 0.6. All power calculations are based on a significance level of 0.05 and were calculated using the STATA 12.
Control group participants receive treatment as usual, consistent with the Care Programme Approach . This includes systematic arrangements for assessing the health and social needs of people accepted into specialist mental health services, the formation of a care plan which identifies the health and social care required from a variety of providers, the appointment of a key worker to keep in close touch with the service user and to monitor and co-ordinate care, and regular review of and, where necessary, agreed changes to the care plan. Control group participants receive care from a multidisciplinary mental health team, and treatments may include medication, social or other individual or group-based psychological interventions. No psychological intervention based on positive psychology principles is currently provided.
All intervention group participants receive treatment as usual, as described for the control group, and in addition receive WELLFOCUS PPT.
WELLFOCUS PPT is an 11-session intervention which aims to improve well-being in people with psychosis. All sessions begin and close with a music savouring exercise. Over the course of the 11 sessions, 10 exercises adapted from standard PPT are covered: positive introductions, savouring, good things, identifying personal strengths, personal strength activity, strength activity with significant other, forgiveness, one door closes another door opens, gratitude, and positive responding. All intervention exercises target at least one of the four target areas identified in the WELLFOCUS model. Every session develops an ‘ongoing exercise’, comprising an exercise to be completed, repeated, or reflected on in participants’ own time.
WELLFOCUS PPT is provided by two facilitators who follow the WELLFOCUS PPT manual (unpublished, on request from first author). Groups last approximately 90 minutes, with a 10 minute break in the middle. All facilitators have experience of facilitating therapeutic groups and of working with people with a diagnosis of psychosis. Facilitators are encouraged to show warmth, empathy, and genuineness in their interactions. Facilitators participate in all exercises themselves, they share personal examples from their own lives with the group, and are encouraged to do the ‘ongoing exercise’ in their own time. Participants are not prohibited from sharing distressing, unpleasant, or negative states and experiences in the group. Negative contributions are validated but not focused on. Instead, facilitators establish a link between the negative experience and one or more of the intervention’s target areas. Participants in the intervention group receive a phone call between each session to offer support with the ‘ongoing exercise’. All facilitators are invited to attend a monthly peer supervision meeting and we plan to report on the rate of attendance.
Treatment fidelity strategies
We use principles for enhancing treatment fidelity outlined by the NIH Behavior Change Consortium , comprising strategies to increase treatment fidelity in five areas: study design, training providers, delivery of treatment, receipt of treatment, and enactment of treatment skills. Compliance with fidelity strategies will be assessed using checklists.
Fidelity strategies at the design level aim to ensure that a study can adequately test its hypothesis by ensuring the design is theory driven, and establishing procedures to standardise the dose and intensity, prevent contamination and deal with implementation setbacks. We used three fidelity strategies at study design level:
We use an evidence-based testable intervention model, i.e., the WELLFOCUS model.
To ensure the same treatment dose in all groups across the treatment arm, we send treatment reminders, work with fixed length group sessions, and a fixed number of mid-week telephone contacts per participant. We use a treatment manual which is followed in each group and provide the same information material to every participant.
To address possible implementation setbacks, we maintain a pool of trained facilitators, to avoid facilitator absence leading to group cancellations.
Fidelity strategies at the level of provider training aim to ensure adequate skill acquisition and skill maintenance of facilitators. We use three fidelity strategies at the level of provider training:
We provided two days of standardised training for all facilitators, led by the co-developer of standard PPT and the WELLFOCUS research team.
We provide monthly peer supervision sessions, which mirrors the group’s facilitator style and the focus on the four target areas as training boosters.
Following each session, both facilitators jointly write PPT-style notes on participants’ achievements in the four target areas, to avoid therapeutic drift.
Fidelity strategies at the level of treatment delivery aim to ensure provision of identical content across the individual groups in the intervention arm and to minimise contamination between arms. We use two fidelity strategies at treatment delivery level:
Facilitators met before each session to rehearse the session content as described in the manual, to minimise between-group differences.
Fidelity strategies at the level of treatment receipt aim to ensure that participants understand and perform the cognitive and behavioural skills delivered during treatment. This is an important and often neglected component of fidelity. If participants do not understand or are not able to implement the new skills during treatment, an otherwise effective intervention may be deemed ineffective. We use two fidelity strategies at the treatment receipt level:
To ensure participant comprehension we use concise and plain English information materials supported by pictures to avoid a focus on literacy.
The ‘ongoing exercise’ is supported and monitored by weekly calls and reviewed in the subsequent group session to increase both understanding and implementation of the involved skills.
Fidelity strategies at the level of treatment skills enactment aim to ensure that skills learned during the therapy are adequately applied in real-life settings. Enactment is different from treatment adherence and treatment efficacy. Treatment adherence relates to whether a participant performs a task definitive of a specific treatment (e.g., actually notes down a good thing that has happened) and treatment efficacy relates primarily to whether an intervention influences the hypothesised endpoint. Treatment enactment specifically relates to the extent to which a participant actually implements a new skill in their everyday life. It is possible that a study shows excellent enactment but poor adherence and poor efficacy. This would provide a good test of the intervention because the treatment skills are used but are not effective. By contrast, in a study with poor enactment neither adherence nor efficacy is likely to be high. We use two fidelity strategies at the level of skills enactment:
Participants are repeatedly reminded throughout the duration of the group of skills learned during past group sessions to encourage their ongoing application, e.g., savouring small things in everyday life or noticing good things and collecting them in a ‘good things’ box.
The last manualised session focuses on reflecting on the implementation of learned skills in everyday life.
The primary outcome is the WEMWBS, a 14-item measure of positive well-being originally developed for the UK general population which frames well-being as a multi-dimensional construct . The overall score is the sum of all items, varying from 14 (low well-being) to 70 (high well-being). The overall scale has proven feasible, reliable, and sensitive to change in people with various mental health problems, including psychosis . Chronbach’s α for the scale lies between 0.87 and 0.91 and the one-week test-retest reliability at r = 0.83 [32, 36].
Eight secondary outcomes are used.
Positive Psychotherapy Inventory (PPTI)
The PPTI consists of 25 questions rated on a 5-point Likert scale varying from “not at all like me” to “very much like me”. The overall score is the sum of all items, varying from 25 (low) to 125 (high). The scale contains five factors designed to assess the positive emotions, engagement, relationships, meaning, and accomplishment (each with a total score between 5 (low) and 25 (high)). It was specifically designed to capture changes due to PPT in clients . The internal consistency of the scale was found at α = 0.80, its test-retest reliability at r = 0.81 in a sample of healthy participants .
Savouring Beliefs Inventory
The Savouring Beliefs Inventory is a 24-item scale that assesses individuals’ perceptions of their ability to derive pleasure through anticipating upcoming positive events, savouring positive moments in the present, and reminiscing about past positive experiences. It uses a 7-point Likert scale between “strongly disagree” and “strongly agree”. The overall score is computed by summing responses to the 12 positively-anchored items and subtracting responses to the 12 negatively-anchored items. Total scores can vary between −72 (low savouring beliefs) and +72 (high savouring beliefs) for the overall scale. Scores for the three subscales are calculated according to the same principle and can vary from −24 (low savouring beliefs on subscale) to +24 (high savouring beliefs on subscale). The overall scale showed an internal consistency of between α = 0.88 and 0.94 in different studies and a test-retest reliability of r = 0.84 . It was also found feasible to use in people with psychosis .
Short Depression-Happiness Scale
The Short Depression-Happiness Scale measures affect on a bipolar continuum between depression and happiness . It uses a 4-point Likert scale between “never” and “often”. The 6-item short version was found to have an internal consistency of between α = 0.77 and 0.92 in different studies, and a test-retest reliability of r = 0.68. The 3 positive items are reverse-scored which leads to overall scores varying from 6 (more depression) to 24 (more happiness). Preliminary normative data suggest that a score of 9 or less may be indicative of mild but clinically relevant depression .
Brief Psychiatric Rating Scale
The Brief Psychiatric Rating Scale is an 18-item observer-rated measure of psychiatric symptom severity . It uses a 7-point Likert scale from “not present” to “extremely severe”. The overall score is the sum of all items, varying between 18 (low symptoms) and 126 (severe symptoms). The internal consistency of the overall scale lies between α = 0.65 and 0.79, its diagnostic sub-scales for withdrawal/retardation, thinking disorder, anxiety/depression, and activation vary from α = 0.766 to 0.879 [42, 43]. Inter-rater reliabilities for this scale have been reported between 0.87 and 0.97. In terms of clinical interpretation, a total score of 31 was found to correspond to a clinical global impression rating of “mildly ill”, 41 to “moderately ill”, and 53 to “markedly ill” .
Integrative Hope Scale
The Integrative Hope Scale captures a comprehensive concept of hope and has been validated specifically for people with psychosis [45, 46]. The scale contains 23 items that are rated on a 6-point Likert scale from “strongly agree” to “strongly disagree”. The 6 negatively anchored items are reverse-scored. The overall score is the sum of all items, varying between 23 (low hope) and 138 (high hope). The scale’s internal consistency lies at Cronbach’s α = 0.92 for the overall scale and the test-retest reliability at r = 0.84 in people with psychosis .
Rosenberg Self-Esteem Scale
The Rosenberg Self-Esteem Scale contains 10 items which are rated on a 4-point Likert scale from “strongly agree” to “strongly disagree”. The scale measures self-esteem by asking the respondents to reflect on their current feelings, with the 5 negatively anchored items being reverse-scored. The overall score is the sum of all items, varying between 0 (low self-esteem) and 30 (high self-esteem). Its psychometric properties have been repeatedly established in various client groups, including people with psychosis, and languages with the internal consistency found between α = 0.77 and 0.88, and the test-retest reliability between r = 0.82 and 0.88 [47, 48].
Rogers Empowerment Scale
The Rogers Empowerment Scale is a 28-item instrument designed to measure subjective feelings of empowerment on a 4-point Likert scale varying from “strongly agree” to “strongly disagree”. After reverse-scoring of the 9 negatively framed items the sum of all items forms the overall score which can vary between 28 (low empowerment) and 112 (high empowerment). Its psychometric properties have been confirmed for people with psychiatric conditions, including psychosis, where the scale’s internal consistency was found at α = 0.86, and 6 of the 7 factors of the scale showed a re-test reliability of r >0.75 [49, 50].
Sense of Coherence Scale
The Sense of Coherence Scale contains 29 questions to measure a person’s global orientation to view the world and the individual environment as comprehensible, manageable, and meaningful. It uses a 7-point scale Likert scale between “very often” and “very seldom or never”. Thirteen items are formulated negatively and have to be reversed for scoring. The overall score is the sum of all items, varying between 29 (low sense of coherence) and 203 (high sense of coherence). In different samples and translations, the scale showed an internal consistency of between α = 0.70 and 0.95, and a 1 year test-retest reliability from r = 0.69 to 0.78. .
In addition, five further measures are used. The Manchester Short Assessment of Quality of Life is an established 12-item measure that frames well-being in terms of subjective health-related quality of life and includes a specific item asking for satisfaction with life as a whole , which has been regarded as a measure of well-being in its own right . The overall score is the mean of all item scores which can vary between 1 (low satisfaction with life) and 7 (high satisfaction with life). The scale’s internal consistence was found between α = 0.74  and 0.81  in different samples of people with mental illness. The Health of the Nation Outcome Scale is a widely used 12-item measure of social disability. Items cover a range of problem areas rated on a 5-point scale between 0 (no problem) and 4 (serious problem) with a resulting overall score of up to 48. Cronbach’s α for the scale varies between 0.59 and 0.76 . The test-retest reliability was found to be mixed for all items, varying between r = 0.65 and 0.40 for seven items, and 0.31–0.32 for three items .
The Global Assessment of Functioning is an equally well accepted measure of functioning, which is used in a 2-item version assessing social and psychological functioning, each rated from 1 (most serious symptoms or disability) to 90 (no symptoms or disability) . The items show an internal consistency of between α = 0.61 and 0.91, and a test-retest reliability of between r = 0.66 and 0.92 .
The CORE10 is a 10-item measure of symptoms. It is scored on a 5-point Likert scale from 1 “not at all” to 5 “most or all of the time” resulting in overall scores between 10 and 50. The scale’s internal consistency is α = 0.90 .
The Sociodemographics Form – Service User is a non-standardised measure modified from another RCT , which records the service user’s date of birth, gender, ethnicity, languages spoken, country of birth, education, employment, marital status, and housing. The rationale for inclusion of each outcome measure is shown in Table 1. Those in the intervention group additionally receive a process evaluation form at follow-up which asks to rate the effect of WELLFOCUS PPT on the hypothesised intervention model variables using a 10-point Likert scale.
Outcome measures used in the WELLFOCUS study
WELLFOCUS model component
Measures overall well-being, i.e., enhanced sense of self
Savouring Beliefs Inventory
Assesses pleasure in the past present and future, which is a form of good feeling addressed in the intervention
PPTI positive emotions
Assesses enjoyment and happiness, which is a form of good feeling addressed in the intervention
Short Depression-Happiness Scale
Measures the reduction of depression, which is an intended outcome of PPT
Brief Psychiatric Rating Scale
Measures general symptom severity including psychosis-specific symptoms
Measures general symptom severity
Measures the presence of supportive relationships as a form of connectedness addressed in the intervention
Integrative Hope Scale
Measures hope, which is an indicator of well-being
Rosenberg Self-Esteem Scale
Measures self-worth, which is an indicator of well-being
Rogers Empowerment Scale
Measures empowerment, which is an indicator of well-being
Sense of Coherence Scale
Measures meaning, which is an indicator of well-being
Provides a PPT-specific measure of meaning
Quality of Life
Manchester Short Assessment of Quality of Life
Quality of life is a form of well-being measure, allowing triangulation with the WEMWBS
Health of the Nation Outcome Scale
To give opportunity to compare changes in well-being with social disability
Global Assessment Functioning
To give opportunity to compare changes in well-being with functioning
Corresponding with the strategies to improve treatment fidelity, we use fidelity measures at three of the five levels of fidelity, i.e., study design, provider training, and treatment delivery. The other two levels of fidelity – treatment receipt and skills enactment – will be assessed as part of the process evaluation.
To assess fidelity at the study design level, deviations from session lengths and the length of mid-week calls is recorded (to establish treatment dose) and group cancellations and any changes to group times are recorded (to identify implementation setbacks). To assess fidelity at the provider training level, facilitators’ presence at training and supervision sessions is recorded. To assess fidelity at the treatment delivery level, adherence to treatment protocol is rated by the co-facilitator for each session on a fidelity evaluation scale developed for WELLFOCUS PPT.
The aim of the process evaluation is to test the intermediate processes outlined in the WELLFOCUS model between the intervention (tested by fidelity assessment) and outcome. Process evaluation strategies include ratings and notes taken by the co-facilitators directly after each session and phone call for each participant and each week, and additional qualitative interviews and focus groups after the end of the therapy.
The co-facilitator collected process evaluation includes five assessment strategies at treatment receipt and skills enactment level which map onto the WELLFOCUS intervention model:
Presence of participants in group sessions
Engagement of participants in exercises during treatment groups and a facilitator rating of whether this was perceived as beneficial
Facilitator rating of behavioural intent to undertake the ‘ongoing exercise’, based on the session in which the ‘ongoing exercise’ was set
Facilitator rating of engagement and benefit from the ‘ongoing exercise’, based on the session in which the ‘ongoing exercise’ was reviewed
Length and content of the weekly telephone calls.
Qualitative process evaluation comprises individual interviews and focus groups with trial participants, and individual interviews with group facilitators. Participants will be asked to take part in either an individual interview (50%) or a focus group (50%).
Semi-structured interviews will ask about participants’ experience of taking part in the study, how the intervention was delivered, and what impact the intervention has had on their well-being and life generally. Questions will also relate to the WELLFOCUS model and processes facilitated by the therapy. Suggestions for further improving the intervention and trial processes will be collected. As far as possible, those who dropped out of the intervention arm or attended irregularly will be asked for reasons including contextual factors influencing attendance and for suggestions to improve attendance rates in the definitive RCT. Focus groups will also allow discussion of any suggestions for improvements of the intervention. Hence, topics that come up in interviews, particularly about problematic issues in relation to therapy content, delivery, and study procedures, will feed into focus groups.
All clinicians involved in providing the therapy will be interviewed to obtain feedback on the WELLFOCUS manual, the intervention delivery, and the trial procedures.
All researchers are trained in the use of all standardised outcome assessments and in semi-structured interviews or focus groups for the process evaluation. Ongoing supervision is provided from experienced clinical researchers.
Recruitment and randomisation
The study is introduced to all staff members in the participating teams. Care coordinators, key nurses or other appropriate staff members are asked to identify potentially eligible service users from their case load. Potentially eligible participants receive information about the study from the clinician. In addition, information about the study is sent to eligible service users on two research registers (which involves consent to be contacted by researchers). These are followed-up for recruitment via telephone by the research team. An appointment is scheduled for assenting participants from all sources with a member of the research team to provide further information on the study. In this meeting, the study is explained, written information is provided, and an opportunity is given to ask any questions. If the participant wishes to have time to consider participation, a second meeting is scheduled. Once the participant has given informed consent, baseline assessments are completed. Participants receive an ID number which is entered in the web-based randomisation system together with age (for validation). Some teams may have fewer than 16 eligible participants, and the minimum number of participants will be 8 (i.e., 4 per arm), with block for randomisation. Where possible we will over-recruit to a maximum of 20 participants in other teams. When participants from a participating clinical team have all been entered into the database, randomisation is conducted. Randomisation involves independent allocation to one of the two trial arms on a 1:1 basis. The use of stratification was considered disproportionate for a pilot RCT. The generation and implementation of the randomisation sequence is conducted independently by the King’s Clinical Trials Unit (registration number 053).
Outcomes are measured pre-randomisation and at the end of the intervention (i.e., 3 months after baseline). Every effort will be made to include patients who drop out of treatment in the follow-up assessments to enable intention-to-treat analysis . Process and fidelity measures are collected by the co-facilitators for each participant after each group session. Qualitative process evaluation is conducted with those in the intervention arm, who are asked to participate either in an individual interview or in a focus group (with alternating allocation based on finishing order of groups). Participants are offered £20 as compensation for their time after completing each assessment and either the interview or focus group (i.e., £60 in total for intervention, £40 for control). The £20 difference in payment between the groups might increase follow-up rates for the process evaluation data from intervention group participants, but remuneration for the outcome evaluation is the same for both groups.
Approaches to minimise bias
It is not possible for service user participants or facilitators to be blind to allocation status. The resources needed to ensure blinding in research assessors would be disproportionate in a pilot study; therefore, this trial is unblinded. However, several approaches will be used to minimise bias.
Addressing bias at participant selection
This study uses a convenience sample of service users considered by team leaders to be fit to participate. This may lead to selection bias. However, internal validity is protected due to the random allocation strategy and not revealing the block size.
Addressing bias at allocation
All randomisation is undertaken by the independent King’s Clinical Trials Unit.
Addressing bias in baseline data
Baseline data are collected before allocation, to reduce assessment bias due to inadequate concealment.
Addressing bias in the intervention
Fidelity to the WELLFOCUS manual is monitored to ensure comparability of the intervention between local groups.
Addressing bias in follow-up data
All participating services users are followed-up and included in the analysis using an intention-to-treat approach to reduce the impact of selective attrition.
Addressing bias in outcome data collection
Bias in the outcome data is minimised by the use of standardised objective assessments and by rater training and supervision. Post-treatment outcomes are, where possible, assessed by researchers who were not involved in the participant’s baseline assessment or their group.
Addressing bias in process evaluation
As far as possible the focus group and interview data are collected by a researcher not involved in delivering the intervention to the respective participants, to minimise social desirability bias.
Addressing bias in analysis
No data regarding the allocation status are stored in the data entry database, following Good Clinical Practice . The primary data analysis is undertaken blind to allocation status. Allocation status is coded as A and B in all data requests.
All study data will be input by the research workers who collected the data. Rigorous approaches to validating and verifying the data will be used, including rater training to achieve an acceptable concordance in administering standardised assessments, development of a database with allowable values for variables as a validation check, double rating for 10% of the data to identify transcription errors, and data cleaning and preliminary analysis led by a statistician to identify invalid responses, checking for outliers, and unexpected patterns in the relationships between variables.
Protected data storage with clear access protocols in line with Good Clinical Practice Guidelines is used to store allocation and outcome data separately. All data are stored in either a locked filing cabinet or in an electronic password-protected database.
All qualitative data will be audio taped, fully transcribed, and quality checked. Transcripts and notes will be subjected to rigorous thematic analysis . Results from the qualitative analysis of process evaluation data will be integrated with results from quantitative statistical analysis to address the three study objectives. Overall, the results of this pilot study will inform the design of a future definitive RCT.
Objective 1: Piloting the intervention
Acceptability of the intervention will be assessed by statistically exploring compliance patterns. This will involve the calculation of patterns of adherence, i.e., attendance rates, completion rates, and proportion of homework completed. Descriptive statistics will be used to evaluate whether levels of adherence and variability in adherence depend on therapy group and basic participant characteristics, such as gender, age, and symptom severity. Rate of loss to follow-up and its dependence on adherence level and on basic participant characteristics will be explored. The quantitative results will be combined with results from the qualitative process evaluation to suggest recommendations for specific adaptations to the intervention components and delivery. Qualitative and quantitative measures will be used to investigate whether the processes and changes subjectively experienced by participants correspond with the processes outlined in the WELLFOCUS model.
Objective 2: Piloting the trial processes
Descriptive statistics will be used to assess recruitment rates in relation to referrals received by clinical staff. Mean time between referral and consent will be calculated. Attrition between referral, consent, and randomisation will be described. Waiting times will be analysed in relation to attrition rates. The proportion of those receiving the intervention as allocated will be calculated. Those receiving and not receiving the intervention will be compared on participant characteristics. Qualitative data will supplement quantitative data to explain discrepancies between allocation and receipt of intervention. Reasons for facilitator drop-out will be explored in the qualitative analysis. Qualitative data from clinician feedback and quantitative data on recruitment rates will be used to formulate recommendations for adapting recruitment procedures, eligibility criteria, participant information, and seeking consent. Any issues arising with regards to randomisation procedures from the qualitative process evaluation and researcher feedback will be used to suggest improvements. Feedback from researchers and participants will be used to suggest improvements to the feasibility of data collection forms.
Objective 3: Piloting the evaluation strategy
The usefulness of fidelity and process evaluation approaches will be determined by exploring whether researcher ratings of fidelity to the manual and process factors, such as engagement and beneficial effect of exercises, helps to improve prediction of outcome and future attendance as compared to recording group attendance only. The confidence interval approach will be used to help determine the suitability of the outcome measures and to inform the sample size calculation for a definitive RCT. This approach involves estimating the confidence interval of the difference between intervention and control group at follow-up, adjusted for baseline. The inclusion of zero (i.e., point of no effect) and the inclusion of a clinically relevant effect into the confidence interval will help to estimate the suitability of measures for a definitive RCT . Further considerations to be taken into account for the choice of outcome measure for the definitive RCT will include the acceptability of the scale (as measured by the number of missing responses), the magnitude of correlation between baseline and follow-up on this scale, and the overall variance of the scale. The group means, standard deviations, and correlations between baseline and follow-up will be used to inform the sample size calculation for the definitive RCT.
In addition, outcomes will be analysed to estimate effectiveness of the intervention. This analysis will be done on an intention-to-treat basis, with participants analysed in the group to which they were randomised irrespective of their compliance with the assigned trial arm. To estimate effectiveness of the intervention, WEMWBS (primary outcome) at pre-treatment (baseline) and 3-month follow-up will be analysed using a random effects regression of outcome on trial arm adjusted for baseline score including a random effect to allow for between group variations in the intervention arm. Where relevant, we will also consider using other approaches to address missing data such as complier-average causal effect approaches .
Divergence of normality will be assessed for the distribution of all variables as will linearity of correlations and equality of variance. The choice of statistical methods will be adapted accordingly. P values of 0.05 will be considered significant in all cases. All analyses will be carried out in SPSS Version 20 and Stata Version 12.
Prof Mike Slade (principal investigator; PI) has overall responsibility for the trial. The Trial Manager is Dr Beate Schrank, who is responsible for co-ordination.
Risk and adverse events
Relevant trust policies relating to potential areas of risk, such as risk management and medication, will be adhered to. Serious adverse events will be monitored by the Trial Manager, who will report these to the PI and where there is a possibility that they are linked to the trial, the Trial Steering Committee will be informed.
An independent Trial Steering Committee (TSC) has been convened, with membership comprising Prof Stefan Priebe (Professor of Social Psychiatry, Queen Mary and Westfield College, University of London) (Chair), Jan Wallcraft (independent user researcher), and Michael Wright (funder representative). At the first TSC meeting the need for a Data Monitoring and Ethics Committee, interim analyses, and stop rules for the trial were discussed. Any serious untoward incident is reported to the TSC Chair.
Data handling and record keeping
The research workers will enter data into a secure password-controlled database. Data entry will include validation checks.
Interim analysis will be performed, as requested by the TSC and as needed by the PI to monitor progress. The only documentation which will contain identifying material are the participants’ contact details and consent forms, which will be stored separately from other study data with linkage through an ID number. Any audiotape recordings will be destroyed once the transcription has been checked for accuracy. All paper forms of these data will be stored in locked filing cabinets at the Institute of Psychiatry. Only the research team will have access to these filing cabinets. All other data will be identified by a participant identification number only. A file linking the participant identification number and personal data will be password-protected and stored on a secure server at the Institute of Psychiatry. Only the research team will have access to these data. Electronic and paper data will be retained for 10 years. All members of the study team will receive MRC Good Clinical Practice training in RCTs and we shall follow Research Governance arrangements.
This study will generate qualitative data comprising interview transcripts and associated analyses, and quantitative data from questionnaires and process evaluation forms. Exclusive use for primary research by the research team is envisaged for no more than 3 years following the study, to meet dissemination goals. Both the quantitative and qualitative data will be shared in anonymised form only. It is anticipated that the data may be used for secondary re-analysis as well as contributing to larger datasets of routinely collected outcome data. Archiving and curating (including data sharing agreements and management of access rights) will be undertaken within the framework used by King’s College London, with due attention to issues of ethical (including consent and confidentiality aspects), legal, and institutional regulatory permissions.
Reporting of trial
The trial data will be reported in line with the extension of CONSORT guidance for trials assessing non-pharmacological treatments and pragmatic trials [64–66].
The results of the research will be targeted for publication in peer-reviewed journals of general and special interest, and presented at international conferences. Lay summaries of the results will be published on the research section webpage (researchintorecovery.com).
Recruitment for the trial began on 27 March 2013 and is on-going.
Medical research council
Randomised controlled trials
South London and Maudsley NHS foundation trust
Trial Steering Committee
Warwick-Edinburgh Mental Well-being Scale.
The WELLFOCUS study is funded by Guy’s & St Thomas’ Charity (Ref G101016). Mike Slade receives salary support from the National Institute for Health Research (NIHR) Biomedical Research Centre for Mental Health at South London and Maudsley NHS Foundation Trust and the Institute of Psychiatry, King’s College London. The views expressed are those of the author(s) and not necessarily those of the NHS, the NIHR, or the Department of Health.
We thank the participating trial facilitators (Nicola Gawn, Zara Kanji, Steven Livingstone, Matthew Richardson, and Marieke Wrigley) and Sherry Clarke for their contributions to the development of WELLFOCUS PPT. Data was also collected by Tamsin Brownell, Agnes Chevalier, Zivile Jakaite, Charley Larkin, and Judy Tchikaya.
Health Service and Population Research Department, Institute of Psychiatry, King’s College
South London and Maudsley NHS Foundation Trust, Mental Health Promotion
Health & Wellness Centre, University of Toronto
Department of Psychiatry and Psychotherapy, Medical University of Vienna
Slade M, Williams J, Bird V, Leamy M, Le Boutillier C: Recovery grows up.J Ment Health 2012, 21:99–103.PubMedView Article
Bird VJ, Le Boutillier C, Leamy M, Larsen J, GO L, Williams J, Slade M: Assessing the strengths of mental health consumers: a systematic review.Psychol Assess 2012, 24:1024–1033.PubMedView Article
Slade M: The contribution of mental health services to recovery.J Ment Health 2009, 18:367–371.View Article
Slade M, Leamy M, Bacon F, Janosik M, Le Boutillier C, Williams J, Bird V: International differences in understanding recovery: systematic review.Epidemiol Psychiatr Sci 2012, 21:353–364.PubMedView Article
Leamy M, Bird V, Le Boutillier C, Williams J, Slade M: Conceptual framework for personal recovery in mental health: systematic review and narrative synthesis.Br J Psychiatry 2011, 199:445–452.PubMedView Article
Frisch MB: Quality of Life Therapy: Applying a Life Satisfaction Approach to Positive Psychology and Cognitive Therapy. Hoboken, NJ: Wiley; 2006.
Schueller SM, Parks AC: Disseminating self-help: positive psychology exercises in an online trial.J Med Internet Res 2012, 14:e63.PubMed CentralPubMedView Article
Meyer PS, Johnson DP, Parks A, Iwanski C, Penn DL: Positive living: a pilot study of group positive psychotherapy for people with schizophrenia.J Posit Psychol 2012, 7:239–248.View Article
Schennach-Wolff R, Jager M, Obermeier M, Schmauss M, Laux G, Pfeiffer H, Naber D, Schmidt LG, Gaebel W, Klosterkötter J, Heuser I, Kühn KU, Lemke MR, Rüther E, Klingberg S, Gastpar M, Seemüller F, Möller HJ, Riedel M: Quality of life and subjective well-being in schizophrenia and schizophrenia spectrum disorders: valid predictors of symptomatic response and remission?World J Biol Psychiatry 2010, 11:729–738.PubMedView Article
Cohn MA, Fredrickson BL: In search of durable positive psychology interventions: Predictors and consequences of long-term positive behavior change.J Posit Psychol 2010, 5:355–366.PubMed CentralPubMedView Article
Kristjansson K: Positive psychology, happiness, and virtue: the troublesome conceptual issues.Rev Gen Psychol 2010, 14:296–310.View Article
Slade M: Mental illness and well-being: the central importance of positive psychology and recovery approaches.BMC Health Serv Res 2010, 10:26.PubMed CentralPubMedView Article
Schrank B, Bird V, Tylee A, Coggins T, Rashid T, Slade M: Conceptualising and measuring the well-being of people with psychosis: systematic review and narrative synthesis.Soc Sci Med 2013, 92:9–21.PubMedView Article
Lu W, Wang Z, Liu Y: A pilot study on changes of cardiac vagal tone in individuals with low trait positive affect: the effect of positive psychotherapy.Int J Psychophysiol 2013, 88:213–217.PubMedView Article
Rashid TAA: Positive psychotherapy for children and adolescents. In Handbook of Depression in Children and Adolescents. Edited by: Abela JZ, Hankin BL. New York, NY: Guilford Press; 2008:250–287.
Positive psychotherapy: building a model of empirically supported self-help
. In Dissertation AAI3363580. Philadelphia, PA: University of Pennsylvania, Psychology; 2009.
Seligman ME, Steen TA, Park N, Peterson C: Positive psychology progress: empirical validation of interventions.Am Psychol 2005, 60:410–421.PubMedView Article
Asgharipoor N: A comparative study on the effectiveness of positive psychotherapy and group cognitive-behavioral therapy for the patients suffering from major depressive disorder.Iran J Psychiat Behav Sci 2012, 6:33–41.
Cuadra-Peralta A, Veloso-Besio C, Ibergaray Pérez M, Rocha Zúñiga M: [Positive psychotherapy results in patients with depression].Ter Psicol 2010, 28:127–134. In SpanishView Article
Sin NL, Lyubomirsky S: Enhancing well-being and alleviating depressive symptoms with positive psychology interventions: a practice-friendly meta-analysis.J Clin Psychol 2009, 65:467–487.PubMedView Article
Rashid T: Positive psychotherapy. In Positive Psychology: Exploring the Best in People. Edited by: Lopez SJ. Westport CT: Greenwood Publishing Company; 2008.
Craig P, Dieppe P, Macintyre S, Michie S, Nazareth I, Petticrew M, Medical Research Council Guidance: Developing and evaluating complex interventions: the new Medical Research Council guidance.BMJ 2008, 337:a1655.PubMed CentralPubMedView Article
Schrank B, Riches S, Bird V, Murray J, Tylee A, Slade M: A conceptual framework for improving well-being in psychosis.Epidemiol Psychiatr Sci 2013, 20:1–11.
Arain M, Campbell M, Cooper C, Lancaster G: What is a pilot or feasibility study? A review of current practice and editorial policy.BMC Med Res Methodol 2010, 10:67.PubMed CentralPubMedView Article
Thabane L, Ma J, Chu R, Cheng J, Ismaila A, Rios LP, Robson R, Thabane M, Giangregorio L, Goldsmith CH: A tutorial on pilot studies: the what, why and how.BMC Med Res Methodol 2010, 10:1.PubMed CentralPubMedView Article
Lancaster GA, Dodd S, Williamson PR: Design and analysis of pilot studies: recommendations for good practice.J Eval Clin Pract 2004, 10:307–312.PubMedView Article
Moher D, Schulz KF, Altman DG: The CONSORT statement: revised recommendations for improving the quality of reports of parallel group randomized trials.BMC Med Res Methodol 2001, 1:2.PubMed CentralPubMedView Article
Little RJ, D’Agostino R, Cohen ML, Dickersin K, Emerson SS, Farrar JT, Frangakis C, Hogan JW, Molenberghs G, Murphy SA, Neaton JD, Rotnitzky A, Scharfstein D, Shih WJ, Siegel JP, Stern H: The prevention and treatment of missing data in clinical trials.N Engl J Med 2012, 367:1355–1360.PubMed CentralPubMedView Article
Tennant R, Hiller L, Fishwick R, Platt S, Joseph S, Weich S, Parkinson J, Secker J, Stewart-Brown S: The Warwick-Edinburgh Mental Well-being Scale (WEMWBS): development and UK validation.Health Qual Life Outcomes 2007, 5:63.PubMed CentralPubMedView Article
Department of Health: Effective Care Co-Ordination in Mental Health Services. London: HMSO; 1999.
Bellg AJ, Borrelli B, Resnick B, Hecht J, Minicucci DS, Ory M, Ogedegbe G, Orwig D, Ernst D, Czajkowski S, Treatment Fidelity Workgroup of the NIHBCC: Enhancing treatment fidelity in health behavior change studies: best practices and recommendations from the NIH Behavior Change Consortium.Health Psychol 2004, 23:443–451.PubMedView Article
Margrove KL, Heydinrych K, Secker J, SE-SURG (South Essex Service User Research Group): Waiting list-controlled evaluation of a participatory arts course for people experiencing mental health problems.Perspect Public Health 2012, 133:28–35.PubMedView Article
Clarke A, Friede T, Putz R, Ashdown J, Martin S, Blake A, Adi Y, Parkinson J, Flynn P, Platt S, Stewart-Brown S: Warwick-Edinburgh Mental Well-being Scale (WEMWBS): Validated for teenage school students in England and Scotland. A mixed methods assessment.BMC Public Health 2011, 11:487.PubMed CentralPubMedView Article
Guney S: The Positive Psychotherapy Inventory (PPTI): reliability and validity study in Turkish population.Soc Behav Sci 2011, 29:81–86.View Article
Bryant F: Savoring Beliefs Inventory (SBI): a scale for measuring beliefs about savouring.J Ment Health 2003, 12:175–196.View Article
Joseph S, McCollam P: A bipolar happiness and depression scale.J Genet Psychol 1993, 154:127–129.PubMedView Article
Joseph S, Linley PA, Harwood J, Lewis CA, McCollam P: Rapid assessment of well-being: the Short Depression-Happiness Scale (SDHS).Psychol Psychother 2004, 77:463–478.PubMedView Article
Overall JE, Gorham DR: The Brief Psychiatric Rating Scale (BPRS): recent developments in ascertainment and scaling.Psychopharmacol Bull 1988, 24:97–99.
Crippa JAS, Sanches RF, Hallak JEC, Loureiro SR, Zuardi AW: Factor structure of Bech’s version of the Brief Psychiatric Rating Scale in Brazilian patients.Braz J Med Biol Res 2002, 35:1209–1213.PubMedView Article
Sanchez R, Ibanez MA, Pinzon A: Factor analysis and validation of a Spanish version of the brief Psychiatric Rating Scale in Colombia.Biomedica 2005, 25:120–128.PubMed
Leucht S, Kane JM, Kissling W, Hamann J, Etschel E, Engel R: Clinical implications of Brief Psychiatric Rating Scale scores.Br J Psychiatry 2005, 187:366–371.PubMedView Article
Schrank B, Woppmann A, Grant Hay A, Sibitz I, Zehetmayer S, Lauber C: Validation of the Integrative Hope Scale in people with psychosis.Psychiatry Res 2011, 198:395–399.View Article
Schrank B, Woppmann A, Sibitz I, Lauber C: Development and validation of an integrative scale to assess hope.Health Expect 2011, 14:417–428.PubMedView Article
Blaskovich J, Tomaka J: Measures of self-esteem. In Measures of Personality and Social Psychological Attitudes Volumes 1: Measures of Psychological Attitudes. Edited by: Robinson JP, Shaver PR, Lawrence S. San Diego, CA: Wrightsman Academic Press Inc; 1991:121–123.
Wowra SA, McCarter R: Validation of the empowerment scale with an outpatient mental health population.Psychiatr Serv 1999, 50:959–961.PubMed
Rogers ES, Ralph RO, Salzer MS: Validating the empowerment scale with a multisite sample of consumers of mental health services.Psychiatr Serv 2010, 61:933–936.PubMedView Article
Corrigan PW, Faber D, Rashid F, Leary M: The construct validity of empowerment among consumers of mental health services.Schiz Res 1999, 38:77–84.View Article
Eriksson M, Lindstrom B: Antonovsky’s sense of coherence scale and the relation with health: a systematic review.J Epidemiol Community Health 2006, 60:376–381.PubMed CentralPubMedView Article
Priebe S, Huxley P, Knight S, Evans S: Application and results of the Manchester Short Assessment of Quality of Life (MANSA).Int J Soc Psychiatry 1999, 45:7–12.PubMedView Article
Cummins RA: On the trail of the gold standard for subjective well-being.Soc Ind Res 1995, 35:179–200.View Article
Björkman T, Svensson B: Quality of life in people with severe mental illness. Reliability and validity of the Manchester Short Assessment of Quality of Life (MANSA).Nord J Psychiatry 2005, 59:302–306.PubMedView Article
Pirkis J, Burgess P, Kirk P, Dodson S, Coombs T, Williamson M: A review of the psychometric properties of the Health of the Nation Outcome Scales (HoNOS) family of measures.Health Qual Life Outcomes 2005, 3:76.PubMed CentralPubMedView Article
Orrell M, Yard P, Handysides J, Schapira R: Validity and reliability of the Health of the Nation Outcome Scales in psychiatric patients in the community.Br J Psychiatry 1999, 174:409–412.PubMedView Article
Wing JK, Beevor AS, Curtis RH, Park SB, Hadden S, Burns A: Health of the Nation Outcome Scales (HoNOS). Research and development.Br J Psychiatry 1998, 172:11–18.PubMedView Article
Woldoff S: Reliability of the Global Assessment of Functioning Scale. Philadelphia, PA: Drexel University; 2004.
Barkham M, Bewick B, Mullin T, Gilbody S, Connell J, Cahill J, Mellor-Clark J, Richards D, Unsworth G, Evans C: The CORE-10: A short measure of psychological distress for routine use in the psychological therapies.Couns Psychother Res 2013, 13:3–13.View Article
Slade M, Bird V, Le Boutillier C, Williams J, McCrone P, Leamy M: REFOCUS Trial: protocol for a cluster randomised controlled trial of a pro-recovery intervention within community based mental health teams.BMC Psychiatry 2011, 11:185.PubMed CentralPubMedView Article
Council MR: MRC Guidelines for Good Clinical Practice in Clinical Trials. London: MRC; 1998.
Braun V, Clarke V: Using thematic analysis in psychology.Qual Res Psychol 2006, 3:77–101.View Article
Dunn G: Pragmatic trials of complex psychosocial interventions: methodological challenges.Epidem Psychiatr Sci 2013, 22:105–109.View Article
Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P, Group C: Methods and processes of the CONSORT Group: example of an extension for trials assessing nonpharmacologic treatments.Ann Intern Med 2008, 148:W60-W66.PubMed
Boutron I, Moher D, Altman DG, Schulz KF, Ravaud P, Group C: Extending the CONSORT statement to randomized trials of nonpharmacologic treatment: explanation and elaboration.Ann Intern Med 2008, 148:295–309.PubMedView Article
Zwarenstein M, Treweek S, Gagnier JJ, Altman DG, Tunis S, Haynes RB, Oxman AD, Moher D: Improving the reporting of pragmatic trials: an extension of the CONSORT statement.BMJ 2008, 337:a2390.PubMedView Article
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited.