- Study protocol
- Open Access
- Open Peer Review
A realist process evaluation of Enhanced Triple P for Baby and Mellow Bumps, within a Trial of Healthy Relationship Initiatives for the Very Early years (THRIVE): study protocol for a randomized controlled trial
Trialsvolume 20, Article number: 351 (2019)
THRIVE is a three-arm randomised controlled trial (RCT) that aims to evaluate whether antenatal and early postnatal interventions, Enhanced Triple B for Baby (ETPB) plus care as usual (CAU) or Mellow Bumps (MB) plus CAU (versus CAU alone), can: 1) improve the mental health and well-being of pregnant women with complex health and social care needs; 2) improve mother-infant bonding and interaction; 3) reduce child maltreatment; and 4) improve child language acquisition. This paper focuses on THRIVE’s realist process evaluation, which is carefully monitoring what is happening in the RCT.
Realistic evaluation provides the theoretical rationale for the process evaluation. We question: 1) how faithfully are MB and ETPB implemented? 2) What are the mechanisms by which they work, if they do, and who do they work for and how? 3) What contextual factors are necessary for the programmes to function, or might prevent them functioning?
The mixed-methods design includes quantitative measures, which are pre- and post-training/intervention questionnaires for facilitators and mothers-to-be, and post-session evaluation forms. Qualitative data collection methods include participant observation of facilitator training and the delivery of a series of antenatal sessions in selected intervention groups (n = 3 for ETPB and n = 3 for MB), semi-structured interviews with facilitators, pregnant women, partners, and referring facilitators, and telephone interviews examining the content of the postnatal components of ETPB and MB.
The findings of this process evaluation will help researchers and decision makers interpret the outcomes of THRIVE. It will provide a greater understanding of: how the interventions work (if they do); the extent and quality of their implementation; contextual factors facilitating and constraining intervention functioning; variations in response within and between subgroups of vulnerable parents; and benefits or unintended consequences of either intervention. Few studies to date have published detailed research protocols illustrating how realist process evaluation is designed and conducted as an integral part of a randomised controlled trial.
ISRCTN, ISRCTN21656568. Registered on 8 November 2013.
This paper describes the planned process evaluation component of a Trial of Healthy Relationship Initiatives for the Very Early years (THRIVE), a three-arm randomised controlled trial (RCT) of two parenting interventions for vulnerable pregnant women. THRIVE is investigating whether receiving antenatal and early postnatal parenting interventions, in addition to care as usual (CAU), can improve maternal mental health, mother-infant relationships, and child language development compared with receiving routine antenatal care alone. The protocol for the trial and research questions for the outcomes evaluation can be found in the study documentation on the funder’s website . The process evaluation will examine how faithfully Mellow Bumps (MB) and Enhanced Triple P for Baby (ETPB) are implemented, the mechanisms by which ETPB and MB work (if they work), who the interventions work for and how, and what contextual factors are necessary for the programmes to function as intended or prevent them functioning. The rigorous approach to evaluation outlined here is important and timely, given that it is current UK policy to invest significant resources in psychosocial parenting interventions in the antenatal and early postnatal period. The publication of protocols for process evaluations has particular utility for researchers who may be aware of recent guidelines for best practice  but lack examples of how to design and execute work of this complexity .
Women who are more vulnerable in pregnancy, due to domestic abuse, mental health problems, addictions, or a combination of complex social factors , are more likely to suffer from stress, depression, and/or anxiety, and produce higher levels of stress-related hormones . Stress and mental health problems during pregnancy may disrupt a mother’s capacity to subsequently interact sensitively with her baby . Poor quality interaction between mother and child, and maternal mental health problems, are also strong predictors of child maltreatment. Children who experience adversity (e.g. maltreatment) during the very early years demonstrate reduced language skills compared with their peers [7,8,9,10,11,12,13,14,15,16]. Other effects may last well into adulthood, reducing opportunities for educational attainment  and have negative impacts on emotional and mental well-being [18,19,20,21]. There is currently a paucity of evidence about whether and how parenting programmes targeting vulnerable families in the very early years are effective (or not) at improving maternal mental health and outcomes for children.
THRIVE is recruiting ‘vulnerable’ pregnant women (as defined by the Special Needs in Pregnancy protocol (SNiPS) of the Glasgow Child Protection Committee) at 20–30 weeks of pregnancy . Participants will be referred through their antenatal booking clinics and randomly assigned to receive ETPB plus CAU, MB plus CAU, or CAU. The aims of the RCT are to evaluate whether ETPB plus CAU or MB plus CAU can: 1) improve maternal mental health and well-being; 2) improve mother-infant bonding and interaction; 3) reduce child maltreatment; and 4) improve child language acquisition. Children will also be followed up at 30 months to assess socio-emotional development and educational outcomes. The recent guidance of the Medical Research Council (MRC) highlights the importance of incorporating mixed-methods process evaluations into RCTs for the testing of complex interventions [2, 23]. The recommendation is that process evaluations include three key elements in their design: 1) data gathering to capture, and describe in detail, what happens during implementation; 2) identifying and investigating the mechanisms through which interventions operate; and 3) explaining the influence of ‘context’ (“which may include anything external to the intervention which impedes or strengthens its effects”) . Reports of process evaluations have been criticised for not being explicit about theoretical influences and for lacking sufficient explanation of 1) what counts as context and 2) how intervention mechanisms and contextual elements interact [24, 25]. The broad principles of realist evaluation [26, 27] provides realist process evaluators with a framework that may make it easier to articulate, and test, the theoretical underpinnings of interventions like MB and ETPB (i.e. key components, intended mechanisms of action, and anticipated benefits) and make more explicit theorising about context-mechanism-outcome relationships (CMO configurations). Table 1 summarizes the approach of Pawson and Tilley  to realistic evaluation which has informed the design of THRIVE’s process evaluation.
Although many researchers present them as diametrically opposed [28,29,30], experimental social science is viewed by some as being highly compatible with the methodological principles and epistemological assumptions of critical realism [31,32,33]. Recent discussions have highlighted the possibilities of designing ‘realist RCTs’ and suggest ways that realist principles might be integrated across all phases of the MRC framework . Although there are excellent examples of how realist evaluation might be used for evaluation in different healthcare contexts (e.g. [35,36,37,38,39]), there have been very few protocols of realist process evaluations embedded within RCTs published to date.
THRIVE’s realist process evaluation aims to provide a greater understanding of: how the interventions work (if they do); the extent and quality of their implementation; contextual factors facilitating and constraining intervention functioning; variations in response within and between subgroups of vulnerable parents; and will explore benefits and/or unintended consequences of either intervention. In practice, it is a huge challenge, both methodologically and with the resources available, to design and execute a realist process evaluation that allows us to explore and document such complexities. We present a detailed protocol for THRIVE’s realist process evaluation, which explains how and why we designed this study the way we have, which may be of utility to other researchers designing realist process evaluations of RCTs.
THRIVE is a 5-year RCT with a stratified randomised sample that ensures intervention groups will have a mix of identified vulnerabilities (see Additional files 1 and 2 for an overview of the main activities of the outcome and process evaluation). The trial and the mixed-methods realist process evaluation is led and managed at the MRC/Chief Scientist Office (CSO) Social and Public Health Sciences Unit, University of Glasgow, UK, following MRC guidelines. The lead researcher for the realist process evaluation is based at the School of Health and Life Sciences, Glasgow Caledonian University, UK. There is regular communication between the outcome and process evaluation teams, although we take advisement from a Data Monitoring Committee and a process evaluation advisory group about data management. Ethical approval was granted to conduct this research by the NHS’s West of Scotland Research Ethics Service, reference 13/WS/0163.
Stage I of process evaluation design: understanding the theory of how ETPB and MB work
An important starting point for THRIVE’s realist process evaluation was to map the theory of change for each of the interventions (see Figs. 1 and 2 for ETPB and MB theory of change models), in other words to model how ETPB and MB are theorized to work and produce beneficial change (in line with Pawson and Tilley’s approach to realistic evaluation, as described under Aim 1 of Table 1). ETPB and MB were developed by researchers and facilitators with considerable expertise in designing parenting interventions. Both interventions are for women from 20 weeks of pregnancy and aim to reduce maladaptive responses to stress, maltreatment of children, and improve socio-developmental outcomes for children. However, there are fundamental differences in focus and mechanisms between ETPB and MB.
Developing theory of change models
In the developmental, planning, and organizational stages of THRIVE the team reviewed evidence on Triple P and Mellow Parenting interventions and intervention materials (where available). The Chief Investigator and Co-Investigators on THRIVE liaised with key stakeholders, including the developers of the interventions, to produce models of the theory of change underpinning ETPB and MB.
Enhanced Triple P for Baby (ETPB)
ETPB is the most recently developed variant of the University of Queensland’s Positive Parenting Program’s (Triple P) five-tiered standardized system of family support (see Fig. 3), which will be delivered and evaluated for the first time as part of THRIVE. ETPB, devised by Clinical Psychologists and underpinned by cognitive and behavioural theory, offers enhanced (Level 5) support to parents with ‘additional needs’.
The Triple P research group has published a wealth of studies and meta-analyses that report on the efficacy and effectiveness of other variants of Triple P (e.g. [40,41,42,43,44,45,46], see also: http://www.triplep.net/glo-en/home/). Triple P was appraised by NICE as being particularly effective for the treatment of conduct disorder based on five RCTs that delivered Standard (Level 3) and Self-directed and Enhanced levels of Triple P (Level 5), suggesting that Triple P may have particular benefits for families trying to address severe behavioural problems [47, 48].
Qualitative studies have largely been absent from evaluations of Triple P to date, which limits understanding of how and why these programmes work, for whom, and in what context, although there are a few very recent exceptions [49, 50]. Qualitative work has begun to make more explicit the behavioural changes that take place following participation in Triple P, for example praising children more often who, in turn, appeared more responsible and better at following instructions . Qualitative research has also helped to identify necessary adaptations that are needed when transporting Triple P to a different social and cultural context .
Positive trial results have been questioned by researchers who were critical of ‘small sample sizes’ and who have suggested there is a ‘high risk of bias’ due to ‘lack of independence’ of Triple P evaluations, ‘poor reporting’, and ‘conflicts of interest’ [51,52,53]. However, Triple P evaluators argue that critical reviews (e.g. ) overlook the multiple levels, modes, and intensities of Triple P and fail to consider the broad range of possible outcomes generated by different variants of the programme . A systematic review and meta-analysis of a wider sample of Triple P programmes found that there were short-term effects on a wide range of child, parent, and family outcomes, including children’s social, emotional, and behavioural outcomes, relationships, parenting satisfaction, and efficacy, as identified in parent observational data .
While still to be tested, ETPB aims to improve parents’ maladaptive psychological coping strategies for managing stress (see Fig. 1 for theory of change endorsed by Triple P International). Based on our observation of what was presented at facilitator training, and reading materials that were distributed about the programme, we know that parents will be offered four antenatal group sessions, each lasting 2 h, covering four ‘universal’ issues that commonly affect new parents. These include: 1) ‘positive parenting’; 2) ‘responding to your baby’ (education about infant development); 3) ‘survival skills’ (raising awareness of the connection between feelings and behaviour and how these impact on parents’ interactions with their baby); and 4) ‘partner support’ (development of skills to improve partner relationships and manage the stresses of parenting).
These sessions are theorized to improve psychological coping by offering new knowledge about infant development, new parenting skills (e.g. how to manage sleep, feeding, and soothing the baby), improved self-awareness of feelings and behaviour when interacting with the baby, and skills to improve and sustain the partner relationship. Postnatal support (up to three one-to-one sessions of 1 h and one group session) is introduced from 6 weeks after birth. Parents use this time to practice positive parenting skills and strategies with their baby in a naturalistic setting and facilitators provide them with feedback. Facilitators are trained to use a guided participation model, based on self-regulation theory, which encourages parents to identify their own solutions to problems. Therefore, self-management, self-efficacy, personal agency and problem solving appear to be implicit drivers of change. It is theorized that the outcomes of the ETPB include more positive and sensitive interaction with baby, greater self-confidence and efficacy as parents, fewer postnatal adjustment and mood disorders, greater perceived social support, and less discord in the partner relationship, which all result in a significantly improved mother-infant relationship.
A recent evaluation of another new Triple P programme, Baby Triple P, reports that mothers with postnatal depression (albeit a very small sample) found the delivery of an early postnatal parenting intervention highly acceptable . The pilot RCT showed more favourable improvements in the intervention group on measures of depression, happiness, and perceptions of parenting a new baby, postpartum bonding, and parenting beliefs, although results were not significant. Another study of staff at a Mother and Baby Unit treating mothers presenting with severe mental illness also suggested that it would be feasible, and likely to be acceptable, to deliver Baby Triple P to women in this setting . It remains to be seen how a wider ranging sample of vulnerable women respond to a similar parenting intervention and whether or not the enhanced component of Baby Triple P has any additive effects.
Mellow Bumps (MB)
MB, developed in Scotland, UK, is an antenatal intervention developed by Mellow Parenting (a group that is led by a research-active Clinical Psychologist). The social and cultural context of the area in which Mellow Parenting is based, an area of very high deprivation, was certainly influential to the initial development of support for women, although programmes are now delivered in a variety of settings and social contexts (as is the case for Triple P, which has international reach). MB was designed in response to the high levels of stress vulnerable women reported experiencing during and after pregnancy, as identified through early evaluation of Mellow Babies.
Current evidence on Mellow Parenting programmes includes a mix of peer-reviewed publications and grey literature [58,59,60,61] (also see http://www.mellowparenting.org/). Mellow Babies reportedly improved mother and child well-being and quality of the mother-child interaction, and was associated with an increased uptake of relevant services, including education, mental health, and social services . Mothers who participated in Mellow Babies groups described a positive change in their perception of their children and an increased self-confidence in handling ‘difficult’ child behaviour . Recent findings of a feasibility trial that compared Mellow Bumps, Chill out In Pregnancy (CHIP), and CAU suggest that MB may be effective at lowering maternal anxiety and mothers’ outwardly directed irritability . However, there was a need for a definitive and well-powered  RCT to fully test these promising findings. The feasibility trial has been particularly informative about the procedures and processes needed to support THRIVE’s RCT.
MB offers six 2 h antenatal group sessions for pregnant women and an optional seventh session for fathers delivered in Week 5. The final group postnatal session is convened 6 weeks after birth. The principal aims of MB are to improve: 1) the reflective functioning and self-care (e.g. poorly managing stress) of pregnant women; and 2) parenting skills. MB is theorized to work by providing nurturance (e.g. improved nurturing and relaxation and support in expressing and dealing with emotions) and guided reflection to improve capacity to reflect on parenting (e.g. by improving maternal representations of early attachment relationships) . Facilitators are trained to foster the nurturing and relaxing ethos of the group by conducting a ‘meet and greet’ session with participants in their homes prior to group attendance, inviting women to join a ‘closed’ and ‘safe’ group , and taking care of participants’ needs while at the group (e.g. welcoming them and offering refreshments on arrival). Facilitators devote time within each session to stress reduction (guided relaxation), which is intended to bring benefits to both mother-to-be and baby. Mothers-to-be are usually given a small gift during the session, for example bubble bath or massage oil to encourage self-care at home. The social aspect of the group (i.e. potentially gaining support from other mothers-to-be and non-judgemental facilitators) was found to be of therapeutic benefit in Mellow Babies groups , and may well have similar effects for participants of MB.
The theorized aims are to improve attachment and the quality of interaction between mother and baby; mothers-to-be are shown educational videos and play interactive games to learn about their baby’s capacity to respond to them whilst in the womb and during the very early postnatal period. Mothers-to-be are encouraged to interact with their bumps by being given torches to shine on their bumps to track movement of the baby and talking or singing to their bumps at home. The anticipated benefits of the intervention are reduced stress, anxiety, and irritability, greater self-acceptance, and better self-care and improved parenting skills and positive intentions, which reduce risks of child maltreatment and increase mother-infant attachment. See Fig. 2 for Mellow Bumps’ theory of change.
Stage II of process evaluation design: key questions
We sought to address three overarching questions in Stage II of the realist process evaluation. These questions, outlined in Table 2, were designed to examine implementation fidelity and any contextual factors that influence delivery, as recommended by MRC guidelines on process evaluation . We also questioned how theorized mechanisms of ETPB and MB were activated, amongst whom, and in what contexts, informed by the approach of Pawson and Tilley to realistic evaluation (see Aim 2, Table 1).
How faithfully are MB and ETPB implemented?
We question how faithfully ETPB and MB are implemented in relation to the developer manuals or protocols and will explore the wider contextual factors that may influence programme fidelity. Prior knowledge of, or experience of, delivering different levels or variants of Triple P programmes and/or Mellow Parenting programmes are two possible factors that could affect how faithfully interventions are delivered. Other influences may include the clarity and consistency of training and intervention materials and the amount of work that facilitators feel they need to undertake in order to familiarize themselves with content and deliver the interventions.
What are the mechanisms by which ETPB and MB work, if they work, who do they work for and how?
We seek to examine how specific mechanisms of ETPB and MB work for this particular population of pregnant women. We will compare the responses of participants and facilitators to the skills-based approach taken in ETPB with the nurturing and therapeutic approach taken by MB. We question the extent to which experiences within particular groups (e.g. in terms of vulnerabilities) result in the firing, or inhibition, of change mechanisms. We will also examine the decision making of referring facilitators about what to deliver, adapt, or leave out, and their sensitivity towards pregnant women.
What contextual factors are necessary for the programmes to function, or might prevent them functioning?
It is important to explore the wider contextual factors that may influence the functioning of ETPB and MB, given that the effects of intervention come about through the interaction of their mechanisms with people and contexts . We will examine the extent to which the professional backgrounds of facilitators, and prior knowledge of ETPB and/or MB, or previous training in these interventions, influence the way interventions are delivered and how they engage with participants. We also aim to locate the accounts of participants of their participation in ETPB or MB within a wider context of their lives and their experiences of parenting and of being parented. We question the extent to which group dynamics might be affected by the particular mix of vulnerabilities (e.g. homogenous or heterogeneous group composition). We aim to explore the nature of participants’ vulnerabilities in detail and question the extent to which ETPB, MB, or individual mechanisms within each are perceived to help. Figure 4 summarizes the main functions and methods of THRIVE’s realist process evaluation.
Multiple methods, including interviews, observations, and questionnaires, will be used to investigate the relationships between context, mechanism, and outcomes, thus allowing methodological triangulation .
We anticipate that the interview and ethnographic data collected as part of the realist process evaluation will be crucial in terms of understanding the complexities of the relationship between mechanisms, context, and outcomes, and will help explain how different characteristics of participants and facilitators, and different contexts, may mediate the intended benefits of MB and ETPB. Few qualitative studies have been published to date on Triple P or Mellow Parenting programmes that offer detailed insights into these critical processes. Ethnography in particular, it is argued, has an ‘untapped’ potential to yield new insights into the processes that influence the functioning of interventions , although it is still relatively rarely utilised. An ethnographic approach to process evaluation has recently been suggested as being particularly helpful for trials being conducted with vulnerable groups or challenging social contexts and where building theory might be important .
Recruitment to the RCT
Facilitators will be appointed by Greater Glasgow & Clyde (GGC) and Ayrshire & Arran (A&A) and allocated to ETPB or MB based on their stated preferences. It is planned that the facilitators will be line managed by their respective NHS Community Health Partnerships (GGC and A&A respectively). All facilitators in NHS A&A are likely to be recruited from Midwifery or Health Visiting background as this is the sustainable way NHS A&A envisage working with early parenting interventions in the future. NHS GGC have indicated they are facing too much demand for midwives’ time and wish to involve a wider range of professionals to undertake delivery of ETPB and MB. This contextual difference between the two NHS areas affords an opportunity to explore the impact of different strategic and managerial decision making on the rollout of the THRIVE interventions.
Mothers-to-be will be recruited from the same two NHS regions, enabling us to explore variations in the experiences of participants based on levels of deprivation and urban and rural localities. We plan to interview a sample of referring practitioners, to explore the referral process (e.g. how ‘vulnerability’ is conceptualized; how they identify mothers-to-be as suitable or ineligible for referral; what they think of ETPB and MB). Once a referral is made, mothers-to-be will be contacted by a member of the trial team who will obtain full consent to trial. This will cover consent for the process researcher to make initial contact with mothers-to-be, and partners, about interviews. Mothers-to-be are then randomized to MB plus CAU, ETPB plus CAU or CAU.
We aim to conduct 18–20 semi-structured interviews with a sample of referring facilitators from clinic- and community-based settings. The sample design was responsive to any recruitment issues that came to the fore in the early stages of the trial, as reported by THRIVE field workers or the outcomes evaluation team in field notes. We purposively sampled the highest and lowest practitioners, from a range of clinic settings, to understand why recruitment was working well in some settings and not in others. We planned interviews with the Heads of Midwifery at Greater Glasgow and Clyde and Ayrshire and Arran to gain further insights into the ways in which the local context and organisational culture might have impacted recruitment.
Facilitators implementing ETPB or MB
Questionnaires (Facilitator Questionnaire 1) will be administered to all facilitators before they embark on training in MB or ETPB (to be delivered by Mellow Parenting and Triple P, respectively) to examine prior knowledge of, and views on, the intervention and their expectations of training. A second questionnaire (Facilitator Questionnaire 2) will be completed at the end of training. This examines views on course content, intervention materials, and perceived preparedness to deliver interventions groups.
Facilitators will be purposively sampled by area, intervention allocation, professional roles, and prior knowledge and experience of the interventions to be delivered. Pre-training interviews (Facilitator Time 1) will examine facilitators’ backgrounds in detail, motivations for applying for their post, and knowledge of, and thoughts on, the particular intervention they will be trained to deliver. The topic guide in Table 3 provides an initial framework for interviews, with flexibility to incorporate other relevant subject areas identified within and between interviews.
Facilitators will be invited to participate in a second interview (Facilitator Time 2) once they have acquired experience of delivering the intervention (‘experienced’ meaning they have delivered three or more groups). Follow-up interviews will explore their experiences of delivering ETPB and MB and examine the influence of peer support (in the case of ETPB) or supervision (for MB). The topic guide in Table 4 will used as a starting point for these discussions.
Semi-structured interviews will be conducted after intervention delivery has begun to examine the experiences of MB developers in providing supervision.
Participant observation of facilitator training
Participant observation will be conducted during ETPB (4 days) and MB (1 day) facilitator training, and implementation materials will be reviewed. Particular attention will be paid in training to the way key mechanisms are presented to, and understood by, facilitators. There may be opportunities during the course of training and other facilitator events (e.g. refresher training) to engage with facilitators informally (e.g. during lunch and coffee breaks) and discuss their initial thoughts about the programmes, check understandings of the theory of the interventions, and how they anticipate delivering groups. There may be further opportunities to gather ‘informal’ data (e.g. experiences of delivering groups) as the project team are likely to have ongoing contact with facilitators throughout the implementation period in order to arrange training updates and facilitate group organisation. Any relevant observations will be recorded in field notes.
Facilitators undertaking training in ETPB are given a detailed manual, with training and intervention materials and other materials (e.g. DVDs) that they need to deliver groups. The Triple P for Baby workbook, included with these materials, will be given to parents, along with copies of a number of shorter booklets (e.g. ‘Enhanced Triple P for Baby’ booklet). All materials will be shared with the research team following the delivery of facilitator training. Mellow Parenting will also provide facilitators and the research team access to all programme materials, which includes the MB manual and weekly packs prepared in advance to support the delivery of each session.
Pregnant women will be asked to complete a questionnaire at the first ‘meet and greet’ sessions (delivered as part of MB and ETPB) or at the beginning of the first group session (Mother’s Questionnaire, Time 1). The first questionnaire is designed to explore views on the intervention, thoughts on group participation, and anticipated outcomes of participation for themselves and their partners. A second questionnaire, administered to pregnant women at end of the last antenatal group session, examines the views of participants on the groups and what aspects of the intervention they think works (Mother’s Questionnaire Time 2). Facilitators will post questionnaires to participants who miss the relevant sessions. Participants who do not respond to postal questionnaires will be followed-up by telephone. Post-trial quantitative analyses will use the data on participant characteristics to investigate which participants benefitted most from the interventions.
Semi-structured individual interviews will be conducted at the end of the antenatal phase of ETPB and MB before the birth (Mothers Time 1). Interviews will be used to gather detailed background information about the pregnant women, including the nature of their additional health and social care needs, the circumstances surrounding their pregnancy, and their experiences of being a parent and of being parented. Other topics likely to be covered include experiences of recruitment to the trial, understanding of and responses to key mechanisms, and thoughts on, and experiences of, participating within the group (see Table 5 for topic guide).
Mothers will be re-contacted 6 months after the birth of their babies and invited to take part in a second individual interview (Mothers Time 2). Follow-up interviews will explore how their lives have been since their babies were born and whether or not there have been any sustained benefits of their involvement in THRIVE, or indeed any negative effects (see Table 6 for topic guide).
Interviews with partners/fathers-to-be
MRC funding was secured for a complementary PhD studentship to study the experiences of marginalised fathers who will be sampled from the trial. Interviews will be conducted as part of the process evaluation to explore: 1) how partners might influence the participation of mothers-to-be in the trial; 2) the involvement of partners (fathers, same-sex parents, sisters, and mothers can accompany the participant to each session in ETPB); and 3) fathers’ engagement with MB, if any (there is an optional ‘Dad’s session’ for one of seven possible sessions). Table 7 summarises our procedures for collecting and analysing qualitative data.
Implementation of ETPB and MB
Participant observation of MB and ETPB
Participant observation was suggested to MB programme developers who found that other data collection methods (e.g. videoing group sessions or a researcher attending to observe) were too intrusive and not a good fit with the programme ethos. However, a researcher fully participating in sessions was considered more sensitive and was accepted by both MB and ETPB developers as a way to understand how the programmes work. The precise role the researcher takes within these groups is likely to be negotiated with the intervention developers, facilitators, and group participants. The process researcher will participate in three complete series of antenatal sessions with three different MB groups and three different ETPB groups. Groups will be selected at the early, mid-point, and end stages of the trial to ensure we capture facilitators with a range of different experiences of implementing groups (e.g. implementing their first group through to experienced facilitator). Table 8 indicates the subjects of interest that are most likely to be noted. Data will take the form of field notes, which will be written from memory immediately after each group session. Researchers will be reflexive in their written accounts about the roles and the influence their presence has within groups.
Post-session evaluation questionnaires
Facilitators and participants will be asked to complete post-session evaluation questionnaires after each intervention session. These invite respondents to review the content covered in each session and to reflect on what they think worked well and what was less successful. Responses are likely to inform the facilitator and participant interviews (Facilitator Questionnaire Time 2 and Mothers Questionnaire Time 2).
ETPB offer up to three postnatal home visits to each parent, or couple, from 6 weeks following the birth of the baby, and a final group session. Facilitators and participants will be asked to complete post-session evaluation forms for these home-based sessions just as they do for the group sessions above. We will include an additional measure of a short telephone interview (10–15 min) to further explore, with selected participants, the content of home-based postnatal sessions. Up to three women from each ETPB group will be contacted and asked to describe one of three sessions. Data will take the form of research notes recorded on a pro forma observation sheet.
All process data will be analysed independently of the outcome data and, importantly, documented before the outcomes are actually known . Pawson and Tilley state that the aim of analysis of realist evaluation data is to ‘draw closer to explaining the complex signature of outcomes’ that an intervention may produce . They suggest the starting point for analysis is to examine whether the theories about how each of the programmes work are supported or refuted by the data. They suggest that analysis is an ‘ever-repeating cycle’ of examining recognisable outcome variations, disentangling those that are less clear, and identifying unanticipated consequences of the intervention(s). This will involve constant comparison of emergent findings from the quantitative and qualitative studies outlined above.
Descriptive accounts of the data will be prepared in the order of completion of each key ‘project’ (e.g. pre-training interviews; observations; interviews with pregnant women) as far as is practicable. Report writing is likely to be a critical part of the process evaluation since it will enable us to: 1) keep clear records of which substantive themes were identified and when; 2) present what we thought about the data at particular stages of the evaluation (as opposed to re-interpreting data retrospectively in light of later analyses); and 3) compare data more easily. Dissemination activities (e.g. presentations and publication) will be carefully timed to avoid contamination of the RCT.
This trial is of great social importance since the overarching aim is to establish the most effective way to break the cycle of maladaptive coping leading to maltreatment of infants. The evaluation as a whole will make a substantial contribution to the evidence base for very early interventions appropriate to vulnerable pregnant women. The different theoretical perspectives of the interventions provide the opportunity to assess which theoretical approach has greatest empirical support. The mixed-methods realist process evaluation described here will help disentangle which components of each intervention work best, for whom, and why, and help explain the effects of antenatal and postnatal components of the intervention. The independence of the THRIVE team is important given that Triple P and Mellow Parenting’s involvement in the development and evaluation of their own programmes has attracted heavy criticism .
A trial of this scale and complexity inevitably presents practical and methodological challenges to both the trial and process teams. We have outlined here the work we hope to be able to do for THRIVE’s realist process evaluation. However, it will be important to remain flexible about the research design and be responsive to what happens within the RCT. It may be that emergent issues, or additional research problems, come to light during the course of implementation, meaning that planned methods have to be re-designed or that new measures have to be incorporated.
Henderson M, Wittkowski A, McIntosh E, Calam R, McConnachie A, Buston K, Wilson P, Minnis H, Thompson L, O’Dowd J, Law J, McGee E, Wight D, THRIVE Trial Research Team. Trial of Healthy Relationship Initiatives for the Very Early Years (THRIVE): a three-arm randomised controlled trial for mothers identified as vulnerable in pregnancy and their babies who are at risk of high maltreatment. THRIVE trial detailed project proposal submitted to NIHR PHR available at http://www.nets.nihr.ac.uk/projects/phr/11300201. Accessed 11 Jan 2019.
Moore G, Audrey S, Barker M, Bonnell C, Hardeman W, Moore L, O’Caithain A, Tinati T, Wight D & Baird J. Process evaluation of complex interventions: UK Medical Research Council (MRC) guidance. 2014. https://mrc.ukri.org/documents/pdf/mrc-phsrn-process-evaluation-guidance-final/. Accessed 15 Jan 2019.
Ellard D, Taylor S, Parsons S, Thorogood M. The OPERA trial: a protocol for the process evaluation of a randomised trial of an exercise intervention for older people in residential and nursing accommodation. Trials. 2011;12:28.
National Institute for Clinical Excellence (NICE). NICE guidelines: [CG110] pregnancy and complex social factors. September 2010 https://www.nice.org.uk/guidance/cg110. Accessed 11 Jan 2019.
Radtke KM, Ruf M, Gunter H, Dohrmann K, Schauer M, Meyer A, Elbert T. Trans generational impact of intimate partner violence on methylation in the promoter of the glucocorticoid receptor. Transl Psychiatry. 2011;1(7):e21.
Pearson R, Cooper RM, Penton-Voak IS, Lightman SL, Evans J. Depressive symptoms in early pregnancy disrupt attentional processing of infant emotion. Psychol Med. 2010;40(04):621–31.
Brown J, Cohen P, Johnson JG, Salzinger S. A longitudinal analysis of risk factors for child maltreatment: findings of a 17-year prospective study of officially recorded and self-reported child abuse and neglect. Child Abuse Negl. 1998;22(11):1065–78.
Morton N, Browne KD. Theory and observation of attachment and its relation to child maltreatment: a review. Child Abuse Negl. 1998;22(11):1093–104.
Barlow J, Stewart-Brown S. Child abuse and neglect. Lancet. 2005;365(9473):1750–2.
Barlow J, Davis H, McIntosh E, Jarrett P, Mockford C, Stewart-Brown S. Role of home visiting in improving parenting and health in families at risk of abuse and neglect: results of a multicentre randomised controlled trial and economic evaluation. Arch Dis Child. 2007;92(3):229–33.
Law J, Conway J. Effect of abuse and neglect on the development of children's speech and language. Dev Med Child Neurol. 1992;34(11):943–8.
Roulstone S, Law J, Rush R, Clegg J & Peters T. Investigating the role of language in children’s early educational outcomes. 2011. Research Report DFE-RR134, Department for Education https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/181549/DFE-RR134.pdf. Accessed 15 Jan 2019.
Paulson JF, Keefe HA, Leiferman JA. Early parental depression and child language development. J Child Psychol Psychiatry. 2009;50(3):254–62.
Sylvestre A, Mérette C. Language delay in severely neglected children: a cumulative or specific effect of risk factors? Child Abuse Negl. 2010;34(6):414–28.
Zubrick SR, Taylor CL, Rice ML, Slegers DW. Late language emergence at 24 months: an epidemiological study of prevalence, predictors, and covariates. J Speech Lang Hear Res. 2007;50(6):1562–92.
Shonkoff J, Garner A, Siegel B, Dobbins M, Earls M, McGuinn L, Pascoe J, Wood D. The lifelong effects of early childhood adversity and toxic stress. Pediatrics. 2012;129(1):232–46.
Law J, Rush R, Schoon I, Parsons S. Modeling developmental language difficulties from school entry into adulthood: literacy, mental health, and employment outcomes. J Speech Lang Hear Res. 2009;52(6):1401–16.
Mantymaa M, Kaija P, Luoma I, Raili SK, Tamminen T. Early mother-infant interaction, parental mental health and symptoms of behavioural and emotional problems in toddlers. Infant Behav Dev. 2004;27(2):134–49.
O'Connor TG, Heron J, Golding J, Beveridge M, Glover V. Maternal antenatal anxiety and children's behavioural/emotional problems at 4 years. Report from the Avon Longitudinal Study of Parents and Children. Br J Psychiatry. 2002;180:502–8.
UNICEF. Child poverty in perspective: an overview of child well-being in rich countries 2007. The United Nations Children’s Fund. http://www.unicef-irc.org/publications/pdf/rc7_eng.pdf. Accessed 11 Jan 2019.
World Health Organization’s Commission on the Social Determinants of Health prepared by Irwin L, Siddiqi A & Hertzman C. Early Child Development: a powerful equalizer. June 2007. http://www.who.int/social_determinants/resources/ecd_kn_report_07_2007.pdf. Accessed 11 Jan 2019.
Glasgow Child Protection Committee Inter-Agency procedural guidance for vulnerable women during pregnancy. Glasgow City Council Social Work Services. August 2008. https://www.glasgowchildprotection.org.uk/CHttpHandler.ashx?id=12921&p=0 Accessed 11 Jan 2019.
Moore G, Audrey S, Barker M, Bonnell C, Hardeman W, Moore L, O'Caithain A, Tinati T, Wight D. Process evaluation of complex interventions: Medical Research Council guidance. BMJ. 2015;350:h1258.
Wells M, Williams B, Treweek S, Coyle J, Taylor J. Intervention description is not enough: evidence from an in-depth multiple case study on the untold role and impact of context in randomised controlled trials of seven complex interventions. Trials. 2012;13(1):95–111.
Murdoch J. Process evaluation for complex interventions in health services research: analysing context, text trajectories and disruptions. BMC Health Serv Res. 2016;16:407.
Pawson R, Tilley N. An introduction to scientific realist evaluation. London: Sage Publications; 1997.
Pawson R, Tilley N. Realistic evaluation. London: Sage; 1997.
Marchal B, van Belle D, van Olmen J, Hoerée T, Kegels G. Is realist evaluation keeping its promise? A review of published empirical studies in the field of health systems research. Evaluation. 2012;18(2):192–212.
Marchal B, Westhorp G, Wong G, Van Belle S, Greenhalgh T, Kegels G, Pawson R. Realist RCTs of complex interventions—an oxymoron. Soc Sci Med. 2013;94:124–8.
Van Belle S, Wong G, Westhorp G, Pearson M, Emmel N, Manzano A, Marchal B. Can "realist" randomised controlled trials be genuinely realist? Trials. 2016;17(1):313.
Bonell C, Fletcher A, Morton M, Lorenc T, Moore L. Realist randomised controlled trials: a new approach to evaluating complex public health interventions. Soc Sci Med. 2012;75(12):2299–306.
Bonell C, Fletcher A, Morton M, Lorenc T, Moore L. Methods don't make assumptions, researchers do: a response to Marchal et al. Soc Sci Med. 2013;94:81–2.
Bonell C, Warren E, Fletcher A, Viner R. Realist trials and the testing of context-mechanism-outcome configurations: a response to Van Belle et al. Trials. 2016;17(1):478.
Jamal F, Fletcher A, Shackleton N, Elbourne D, Viner R, Bonell C. The three stages of building and testing mid-level theories in a realist RCT: a theoretical and methodological case-example. Trials. 2015;16:466.
Greenhalgh T, Humphrey C, Hughes J, Macfarlane F, Butler C, Pawson R. How do you modernize a health service? A realist evaluation of whole-scale transformation in London. Milbank Q. 2009;87(2):391–416.
Mukumbang FC, Van Belle S, Marchal B, et al. Realist evaluation of the antiretroviral treatment adherence club programme in selected primary healthcare facilities in the metropolitan area of Western Cape Province, South Africa: a study protocol. BMJ Open. 2016;6:e009977.
Mirzoev T, Etiaba E, Ebenso B, Uzochukwu B, Manzano A, Onwujekwe O, Huss R, Ezumah N, Hicks J, Newell J, Ensor T. Study protocol: realist evaluation of effectiveness and sustainability of a community health workers programme in improving maternal and child health in Nigeria. Implementation Sci. 2016;11:83.
Mehdipanah R, Manzano A, Borrell C, Malmusi D, Rodriguez-Sanz M, Greenhalgh J, Muntaner C, Pawson R. Exploring complex causal pathways between urban renewal, health and health inequality using a theory-driven realist approach. Soc Sci Med. 2015;124:266–77.
Abhyanka P, Cheyne H, Maxwell M, Harris F, McCourt C. A realist evaluation of a normal birth programme. Evidence-based midwifery: December 2013. https://dspace.stir.ac.uk/handle/1893/17839#.XOa06hZKiUk. Accessed 23 May 2016.
Henderson M, Calam, R, McConnachie A, McGee E, McIntosh E, O’Dowd J, Wight D, Wilson P, Wittkowski A, Minnis H, Law J, Thompson L & Buston K. THRIVE trial theory of change diagram. Unpublished images submitted with funding application to NIHR PHR in Henderson, M and THRIVE Trial Co-investigators (2012) THRIVE Trial Detailed Project Proposal http://www.nets.nihr.ac.uk/projects/phr/11300201. Accessed 11 Jan 2019.
De Graaf I, Speetjens P, Smit F, de Wolff M, Tavecchio L. Effectiveness of the Triple P positive parenting program on behavioral problems in children: a meta-analysis. Behav Modif. 2008;32(5):714–35.
Nowak C, Heinrichs N. A comprehensive meta-analysis of Triple P-positive parenting program using hierarchical linear modeling: effectiveness and moderating variables. Clin Child Fam Psychol Rev. 2008;11(3):114–44.
De Graaf I, Speetjens P, Smit F, De Wolff M, Tavecchio L. Effectiveness of the Triple P positive parenting program on parenting: a meta-analysis. Fam Relat. 2008;57(5):553–66.
Sanders MR. Development, evaluation, and multinational dissemination of the Triple P-positive parenting program. Annu Rev Clin Psychol. 2012;8:345–79.
Morawska A, Sanders MR. Self-administered behavioral family intervention for parents of toddlers: Part I. Efficacy. J Consult Clin Psychol. 2006;74(1):10.
Sanders MR, Markie-Dadds C, Turner KM. Theoretical, scientific and clinical foundations of the Triple P-positive parenting program: a population approach to the promotion of parenting competence. Queensland: Parenting and Family Support Centre, The University of Queensland; 2003.
Turner K, Mihalopoulos C, Murphy-Brennan M, Sanders M. Triple-P positive parenting program. Submission for technology appraisal by the National Institute for Clinical Excellence and the Social Care Institute for Excellence. 2004. https://www.nice.org.uk/guidance/ta102/documents/triple-p2. Accessed 23 May 2019.
Turner K, Mihalopoulos C, Murphy-Brennan M, Sanders M. Comments on the assessment report for the appraisal of parent training/education programs for children with conduct disorders commissioned by the National Institute of Clinical Excellence and the Social Care Institute for Excellence; 2004.
Mejia A, Ulph F, Calam R. Preventing interpersonal violence in Panama: is a parenting intervention developed in Australia culturally appropriate? Int J Public Health. 2016;61(8):915–922.
Mejia A, Ulph F, Calam R. Exploration of mechanisms behind changes after participation in a parenting intervention: a qualitative study in a low-resource setting. Am J Community Psychol. 2016;57(1–2):181–9.
Wilson P, Rush R, Hussey S, Puckering C, Sim F, Allely S, Doku P, McConnachie A.& Gillberg C. How evidence-based is an ‘evidence-based parenting program’? A PRISMA systematic review and meta-analysis of Triple P. BMC Med. 2012;10:130.
Coyne JC, Kwakkenbos L. Triple P-positive parenting programs: the folly of basing social policy on underpowered flawed studies. BMC Med. 2013;11:11.
Eisner M. The South Carolina Triple P System Population trial to prevent child maltreatment: seven reasons to be sceptical about the study results. 25 May 2014. Working Paper. https://www.vrc.crim.cam.ac.uk/PDFs/workingtriplep. Accessed 23 May 2019.
Sanders MR, Pickering JA, Kirby JN, Turner KM, Morawska A, Mazzucchelli T, Ralph A, Sofronoff K. A commentary on evidenced-based parenting programs: redressing misconceptions of the empirical support for Triple P. BMC Med. 2012;10:145.
Sanders MR, Kirby JN, Tellegen CL, Day JJ. The Triple P-positive parenting program: a systematic review and meta-analysis of a multi-level system of parenting support. Clin Psychol Rev. 2014;34(4):337–57.
Tsivos ZL, Calam R, Sanders M, Wittkowski A. A pilot randomised controlled trial to evaluate the feasibility and acceptability of the Baby Triple P positive parenting programme in mothers with postnatal depression. Clin Child Psychol Psychiatry. 2015;20(4):532–54.
Butler-Coyne H, Hare D, Walker S, Wieck A, Wittkowski A. Acceptability of a positive parenting programme on a mother and baby unit: Q-methodology with staff. J Child Fam Stud. 2017;26:623.
Puckering C, McIntosh E, Hickey A, Longford J. Mellow Babies: a group intervention for infants and mothers experiencing postnatal depression. Couns Psychol Rev. 2010;25(1):28–40.
Puckering C. Getting in early: a good start in life doesn’t just mean beginning at birth. Children in Scotland; 2011. Available at http://www.mellowparenting.org. Accessed 11 Jan 2019.
Puckering C, Mills M, Cox A, Maddox H, Evans J, Rogers J. Improving the quality of family support: an intensive parenting programme: mellow parenting. London: The Department of Health; 1999.
Puckering C. Mellow parenting: an intensive intervention to change relationships. Signal. 2004;12(1):1–5.
Fargie J & Waugh H. A Scotland-wide trial of Mellow Bumps with teenage mums-to-be. Poster. https://www.mellowparenting.org/our-evidence/posters/a-scotland-wide-trial-of-mellow-bumps-with-teenage-mums-to-be-2/. Accessed 15 Jan 2019.
Breustedt S, Puckering C. A qualitative evaluation of women's experiences of the Mellow Bumps antenatal intervention. Br J Midwifery. 2013;21(3):187–94.
White J, Thompson L, Puckering C, Waugh H, Henderson M, MacBeth A, Wilson P. Antenatal parenting support for vulnerable women: an exploratory randomised controlled trial of mellow bumps versus chill-out in pregnancy or care as usual. Br J Midwifery. 2015;23(10):724-732.
Denzin N. Sociological methods: a sourcebook. 5th ed: Aldine Transaction; 2006.
Greenhalgh T, Swinglehurst D. Studying technology use as social practice: the untapped potential of ethnography. BMC Med. 2011;9:45.
Morgan-Trimmer S, Wood F. Ethnographic methods for process evaluations of complex health behaviour interventions. Trials. 2016;17:232.
We thank the NHS GGC and NHS A&A for participating in and supporting the research, and NHS GGC for sponsoring the research. We also thank the MRC/CSO Social and Public Health Sciences Unit, University of Glasgow and Glasgow Caledonian University for creating a supportive environment to conduct the research. Beyond the authors, others who have commented on the process evaluation development and progress include: THRIVE’s co-investigators—Dr Anja Wittkowski (The University of Manchester), Professor Emma McIntosh (University of Glasgow), Professor Rachel Calam (The University of Manchester), Dr Alex McConnachie (University of Glasgow), Dr Katie Buston (University of Glasgow), Professor Philip Wilson (University of Aberdeen), Professor Helen Minnis (University of Glasgow), Dr Lucy Thompson (University of Aberdeen), Dr John O'Dowd (NHS GGC), Professor James Law (University of Newcastle upon Tyne), Dr Elizabeth McGee (Glasgow Caledonian University), Professor Daniel Wight (University of Glasgow); the THRIVE research team beyond the authors, based at University of Glasgow—Dr Kathleen Boyd, Karen Crawford, Dr Alice MacLachlan, Dr Karen Maxwell, Dr Catherine Nixon, Shona Shinwell and Helen Thompson; the team is supported by eleven field workers, also based at the University of Glasgow. We also thank Triple P International, Mellow Parenting and Martin McKee who contributed to work on the theory of change models presented in this article and Alice MacLachlan for helpful comments on the manuscript.
We are grateful to the National Institute for Health Research Public Health Research Programme (NIHR PHR, project number 11/3002/01) for funding the evaluation and to the CSO and the Scottish Government for funding the intervention costs (GN12KH589 THRIVE). In addition, MRC/CSO Social and Public Health Sciences Unit, University of Glasgow, have supported Dr. Marion Henderson’s use of time regarding THRIVE via MRC/CSO Quinquennial funding of the Relationships and Health Improvement Programme and Settings and Organisation's Programme, which is part of the Social and Public Health Sciences Unit, based at the University of Glasgow, MC_UU_12017_11 and MC_UU12017_12 and CSO SPHSU11 and SPHSU12.
The views expressed are those of the authors and not necessarily those of the NHS, the NIHR or the Department of Health and Social Care. Neither the trial funders nor sponsor (NHS Greater Glasgow & Clyde Health Board) will have any involvement in the implementation of the study design or the analysis of the data.
Availability of data and materials
This publication is based on protocol version 3.0 31.10.17 found on the funder’s website. Recruitment for the trial was conducted between January 2014 and May 2018.
Ethics approval and consent to participate
Ethical approval was granted to conduct this research by the NHS’s West of Scotland Research Ethics Service, reference 13/WS/0163. Letters of access have been provided by NHS Greater Glasgow and Clyde and Ayrshire and Arran.
THRIVE is sponsored by NHS GGC Health Board (Reference GN12KH589; NHS Greater Glasgow & Clyde, Research and Development Management Office, West Glasgow Ambulatory Care Hospital, Dalnair Street, Glasgow), which is responsible for ensuring that THRIVE is managed “according to the Research Governance Framework for Health and Community Care (Second edition, 2006) and World Medical Association Declaration of Helsinki Ethical Principles for Medical Research Involving Human Subjects”.
Informed written consent will be obtained from all participants by research staff prior to all data collection.
Consent for publication
Written consent has been obtained from research participants to publish anonymised data gathered as part of THRIVE, although these data are not presented in this particular manuscript. All authors gave their consent for publication of this manuscript prior to submission.
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
SPIRIT checklist. Standard Protocol Items: Recommendations for Interventional Trials (SPIRIT) checklist indicating the location of relevant information with this publication. (DOC 123 kb)
SPIRIT Schedule of activities. Schedule of activities for women participating in THRIVE and for MB and ETPB facilitators. Activities relating to the realist process evaluation of THRIVE are shown with bold text and shading. (DOCX 22 kb)
About this article
- Process evaluation
- Realist evaluation
- Parenting interventions
- Maternal mental health
- Early years
- Vulnerable populations