This study uses an experimental design with randomization of parents seeking behavioral treatment for their 2- to 5-year old children at a mental health clinic in Baltimore, MD. Equivalence or non-inferiority studies are used to determine whether a novel treatment or health delivery model, that might be less costly or more acceptable to the treatment population, is at least as effective as another more established treatment . In addition to effectiveness and cost, we are comparing the social validity of these two PT program as assessed by the clinicians delivering the program and parents being served since client and practitioner acceptance is necessary for effective implementation. Finally, subgroup analyses will be conducted to identify groups that would most and least benefit from each PT intervention based on baseline child behavior problems scores and degree of family social risk.
A modified intent-to-treat design will be used. In a superiority trial, all randomized participants, regardless of whether they attended any PT sessions, would be included in analyses using an intent-to-treat design. However, such analyses reduce statistical power, increase the chance of finding no differences, and threaten the validity of results in equivalence and non-inferiority studies . Conversely, a per protocol analysis, which would include only parents who attend all recommended sessions, would diminish external validity based on research showing that low attendance rates are common among economically disadvantaged treatment populations [15, 34]. Therefore, in this study, participants who attend at least one PT session are included in the analyses and in estimates of attrition. A dose-response analysis will be conducted to understand the effects of attendance and hours of treatment exposure on outcomes.
This study was first reviewed and approved by the Institutional Review Board of the Johns Hopkins University School of Medicine on 7 September 2010. A Certificate of Confidentiality was obtained on 26 July 2011.
Sampling design: parent-child dyads
The trial setting is a large mental health clinic located in Baltimore, MD. Approximately 80% of the clinic population is African-American or multi-racial and 97% receives Medicaid. Based on a power analysis, we are recruiting 262 parent-child dyads over 5 years (131/condition; allowing 20% attrition). A block randomization procedure is being used to facilitate the initiation of CPP groups without lengthy waiting periods. Each block consists of 10 parent-child dyads.
Inclusion criteria are: (1) ‘parent’ is the biological or adoptive parent or the legal guardian for the target child; (2) parent is English-speaking; (3) child is aged 2 to 5 years; (4) child is brought into treatment by the parent because of externalizing (for example, aggression, inattention) or internalizing (for example,, anxiety, depression) behavior problems; and (5) parent is willing to be randomly assigned to PCIT or CPP. Exclusion criteria are: (1) parent has severe mental illness that would interfere with their ability to participate in PT; (2) parent is actively using drugs or alcohol, based on clinician assessment at intake; and (3) parent has significant cognitive impairment that would affect their ability to understand the PT information presented. Child exclusion criteria include: (1) actively suicidal, psychotic, or have significant developmental delay; (2) congenital, genetic, or sensory anomalies that would interfere with participating in PT; and (3) diagnosis of autism or pervasive developmental disorder. Excluded children are referred for appropriate treatment services or hospitalization. Only one parent and one child per family are eligible to participate in the study.
Parent-child recruitment strategy
Parent-child participants are primarily recruited from the population of active patients at the mental health clinic (n = 150 to 180 new referrals per year). Parents of children aged 2 to 5.5 years are informed about the study following their intake and diagnostic interview. The diagnostic interview clarifies the mental health and psychosocial needs of the family and helps to determine whether the parent and child meet study eligibility criteria. Parents who expressed interest in the study are contacted by a research team member to inform them about the details of the study, answer study related questions, and obtain informed consent. Randomization and scheduling to complete parent-child baseline assessment is completed at that time. Additional recruitment is also taking place in the community using flyers posted in early childhood agencies and presentations to providers serving families with young children.
In this study, attrition is defined as a parent who: (1) consents to participate; (2) is randomized to CPP or PCIT; (3) completes baseline assessments; (4) attends at least one PT session; but (5) does not complete any of the follow-up assessments.
Sampling design and recruitment: mental health clinicians
Study clinicians are recruited from the participating mental health clinic. Criteria for inclusion are: (1) the clinician is employed as a therapist at the participating mental health clinic, (2) the clinician has successfully completed either PCIT or CPP training; (3) the clinician is willing to have their PT sessions recorded and submitted to fidelity monitoring; and (4) the clinician is willing to complete research measures. To minimize contamination, clinicians are assigned only to the PCIT or CPP condition based on their training. To date, we have consented 13 mental health clinicians (9 CPP and 4 PCIT).
The primary outcomes of interest are reductions in child behavior problems, improvements in parenting quality, perceived value of the PT treatment from the perspective of parents and clinicians (that is, social validity), and cost. In addition, we will assess indicators of parent distress and family social risk as modifiers of treatment effectiveness.
Child behavior problems will be measured using a parent-report survey and rater observation. The Child Behavior Checklist 1½ -5 (CBCL) is a widely used 99-item parent-report measure of externalizing and internalizing behavior problems in preschool children . The CBCL is a standardized measure with demonstrated reliability and validity across racial and ethnic groups . Child behavior problems will also be assessed by independent observers from video recorded parent-child interactions during a structured play and clean-up situation using the Dyadic Parent-Child Interaction Coding System (DPICS) . The DPICS measures frequencies of observed child non-compliance, destructive behavior, crying, whining, yelling, and negative behavior and affect. Child behavior problems are measured at baseline, post intervention, and 4-month follow-up.
Parenting quality will be assessed over time using multiple indices, including parent discipline strategies, parenting self-efficacy, and parenting behavior. The Parenting Questionnaire [27, 38] assesses parents’ use of warmth (22 items; alpha = 0.82), following through on discipline (6 items; alpha = 0.78), and corporal punishment (4 items; alpha = 0.59) on a scale of 1 (almost never) to 5 (very often). We will also measure parenting self-efficacy using the Toddler Care Questionnaire [30, 39], a Likert-type measure of the extent to which parents feel confident about managing a range of tasks and behaviors specific to parenting young children (38 items; alpha = 0.94). Finally, frequency of parents’ use of praise, commands, critical statements, and positive or negative affect with their children will be assessed from the video recorded parent-child play and clean-up sessions using the DPICS. These measures of parenting quality will be obtained at baseline, post intervention, and 4-month follow-up.
Social validity is measured from the perspective of parents (that is, parent satisfaction with treatment and attendance rates) and clinicians (that is, clinician satisfaction with the treatment session and perception of parents’ engagement in treatment). Post intervention, parents complete an overall Program Satisfaction form assessing the extent to which they believe they and their children’s behavior have improved as a result of the PT program, the extent to which aspects of the PT program were difficult for them to do, and whether they would recommend the PT program to other parents. Clinicians assess the extent to which parents actively engaged in the PT sessions using the 7-item clinician-rated Engagement Form . Alpha reliability of the Engagement Form is 0.89 and validity has been supported by its significant associations with improvements in teachers’ and parents’ ratings of child behavior problems  and independent ratings of implementation fidelity . Since parents are unlikely to attend mental health treatments they do not find it helpful, parent attendance at the PT sessions is also documented.
Since we were unable to find a brief, valid measure of clinician satisfaction that was appropriate for this study, we created a 3-item measure for clinicians to complete after each PT session. This brief Clinician Satisfaction Form asks clinicians to rate the extent to which (1) they felt positive about their clinical performance in the session they had just led, (2) they believed the PT session assigned to their client was the best PT intervention for them at this time, and (3) they wished they could have done things differently in the PT session but could not because of the requirements of PT protocol assigned to their client. Since there are no data on the reliability of this measure, we will evaluate each item individually.
Moderators of treatment effectiveness will also be examined. These include baseline indicators of parent distress (that is, depressive symptoms, parenting stress) using the Center for Epidemiologic Studies Depression Scale-Revised (CESD-R)  and the Parenting Stress Index Short Form (PSI-SF) . In addition, family social adversity is estimated using a composite of whether the participating parent has been exposed to each of the following 10 parent indicators: less than a high school education, currently unemployed, annual household income below $20,000, living with four or more people in the same home, current or past history of homelessness, aged under 20 years at study entry, aged under 20 years at birth of first child, history of substance abuse, history of domestic violence, and baseline CESD-R score of 16 or higher. Social adversity items are scored as yes/no; scores range from 0 to 10.
We will compare the costs of the PT treatments including costs associated with clinician time required to administer PT, administrative time, childcare, food (CPP only), and space. Parent costs will also be assessed including time spent in PT, time missed from work if parent left work early to attend PT, and additional childcare parent may have needed to provide for other children not brought to the clinic. Parents will complete a Weekly Effort Form at each PT session indicating whether they left work early to attend PT (and if so, how early) and whether arrangements were made for childcare.
We anticipate that many participants in this study will require additional therapeutic services beyond PT (for example, individual therapy, medication). Type, amount, and cost of additional therapies provided to the participating parent or child while receiving PT and during the 4 months immediately after completing PT are being collected.
Data collection procedures
Following consent, parents and children complete baseline measures of child behavior problems and parenting quality. Parents randomized to the CPP condition are informed of the CPP start date; CPP groups meet at the same time each week. Parents randomized to the PCIT condition are contacted by their assigned study clinician to schedule their first PCIT session.
Post-intervention assessments occur after the 12th CPP session (approximately 4 months post baseline) or after the parent and child are discharged from PCIT. These assessments are scheduled only for parents who attended at least one PT session. A second follow-up assessment is scheduled for parents and children in both conditions 4 months after the post-treatment assessment to capture longer-term changes in parents and children. Following each of the three assessment phases, parents receive $30 and the child receives an age-appropriate book valued at $5 or less.
Treatment fidelity procedures
Both PT programs are manualized with established procedures for monitoring fidelity. Consistent with their established protocols, all CPP groups are audio recorded and all PCIT sessions are video recorded and a random selection of 20% of these treatment sessions are sent to independent raters to assess quality and adherence to protocol.
Hypotheses and analysis
Initial analyses will examine data for baseline differences, assumptions of normality, and determination of successful randomization. Violations of normality among continuous dependent variables will be addressed by data transformation. Any measures that cannot be transformed successfully will be analyzed using rank-order statistical models rather than the parametric models described below.
H1: CPP will be at least as effective as PCIT for reducing child behavior problems based on parent-report and observation
This hypothesis will be analyzed using a one-sided two-sample t-test at alpha level 0.05, along with a 90% two-sided confidence interval or a 95% one-sided confidence bound. If the outcome variable appears to deviate from approximate normal distribution, a Wilcoxon rank-sum test will be used. Dependent measures associated with child behavior problems will include the CBCL and observed child behaviors derived from the DPICS.
H2: CPP will be at least as effective as PCIT for improving parenting discipline, self-efficacy, and observed parenting behavior
This hypothesis will be analyzed using similar methods described above in H1. Dependent measures include parent warmth, follow through on discipline, use of corporal punishment, and observed parent behaviors derived from the DPICS.
H3: Parents in the CPP condition will (1) attend a higher percent of PT sessions, (2) be more engaged, and (3) report higher satisfaction than parents in PCIT
Percent attendance by treatment will be compared using 2 × 2 contingency tables and a Fisher’s exact test. Two-sample t-tests will be used to test superiority of CPP over PCIT in Engagement Scale scores by condition. Non-parametric tests will be used to compare differences on the Parent Satisfaction Scale items, and also for Engagement Scale scores if the distribution appears to depart from normality.
H4: CPP will yield comparable effects but at less cost and greater satisfaction
We will conduct a cost-consequence analysis (CCA) to compare the costs and consequences of PCIT versus CPP. The analysis will be conducted from a societal perspective and separated into the parents’ and clinicians’ perspectives. The dollar value assigned to clinician time will be based on their salaries; in sensitivity analyses we will use data from the Bureau of Labor Statistics to apply an average wage for individuals with similar levels of training. To assign a dollar value to parent time, we will use age-sex average wages for individuals from under-represented racial and ethnic backgrounds as suggested by the United States Panel on Cost-Effectiveness in Health and Medicine , data that are also available from the Bureau of Labor Statistics. We will calculate the average cost of clinician time only, of both clinician and parent time, and of all costs per parent to bootstrap the difference between PCIT and CPP to ascertain whether the confidence range overlaps with zero (since data are unlikely to be normally distributed). Since some parents wish to meet individually with their child’s therapist in addition to the PT session, costs associated with all additional therapies will be included in the analyses over two time intervals: while parents are engaged in PT (that is, from baseline to post-intervention) and after PT has ended (that is, from post-intervention to the 4-month follow-up).
Exploratory aim: Is PCIT more effective than CPP for some families based on severity of baseline child behavior problems, parent distress, or degree of family social risk?
A secondary analysis of Aims 1 and 2 will be conducted, adding baseline CBCL, CESDR, PSI, and family social risk scores as covariates.
A limitation of the study design is that the two PT treatments, though roughly equal in the number of treatment hours, require different lengths of time to complete. Treatment duration cannot be made identical without also changing how these treatments are delivered - a key aspect of this study. Nonetheless, we acknowledge that differences in lengths of treatment and child maturation during treatment could confound the results. Treatment duration will be added as a covariate in outcome analyses. A second study limitation is the use of a single urban mental health clinic serving predominantly low-income, African-American families, which reduces the generalizability of results to other populations and clinics. Although employing multiple clinics would have enhanced generalizability, it could also have introduced substantial site effects and increased study cost.