Design
The Good Schools Study consists of a two-arm cluster randomised controlled trial, an embedded qualitative study, a process evaluation and an economic evaluation. The study is a partnership between the London School of Hygiene and Tropical Medicine, Raising Voices, Makerere University, and the Institute of Education. This publication describes the design of the trial component, which will measure the impact of the Good School Toolkit intervention. The trial will involve two cross-sectional surveys, one at baseline and one at endline (Figure 1). Our main analysis will involve a cross-sectional comparison of end line data; here we report procedures for our end line survey. The same procedures were used at baseline.
Intervention
The experimental group is receiving the Good School Toolkit (http://raisingvoices.org/download-good-school-toolkit/) and implementation support over an 18-month period. The Toolkit is an established intervention, which has been popular in Uganda since it was developed 6 years ago by Raising Voices. This implementation period was chosen based on Raising Voices programmatic experience and is the period during which the Toolkit is designed to produce changes in a school.
The Toolkit is designed to be implemented with minimal cost, appropriate for low-resource settings. Development was in close collaboration with six Ugandan schools over 18 months, and the Toolkit has had two rounds of extensive revisions based on in-depth feedback from 40 schools to increase acceptability and effectiveness. Raising Voices also conducted interviews with 200 teachers, and 91% of those teachers reported that both teachers and students in their schools were using the materials, and nearly 100% reported that the materials were useful for their school. The Toolkit is currently being used in approximately 450 Ugandan schools, and in all of these cases, schools or their NGO partners have sought the Toolkit from Raising Voices, thereby showing demand. No schools that have been offered the Toolkit have declined to use it.
The intervention content is based on well-established behavioural change techniques that have been shown to be effective in a variety of fields [25] and have been included in interventions that reduce intimate partner violence perpetration [26] and change teacher behaviour and discipline methods in primary schools [22, 27]. Drawing on the Transtheoretical Model [28], the Toolkit uses a six-step process to engage teachers, students, administration, and parents to reflect on how they can promote quality of education in their school. The intervention support materials consist of booklets, posters and facilitation guides for 60 different activities. These activities are related to creating a better learning environment, to respecting each other, to understanding power relationships, to using non-violent discipline, and to improving teaching techniques.
The process begins by selection of a school-based ‘protagonists’, usually two motivated members of staff and two students in each school, to engage other staff, students, and the administration to set school-wide goals [25] and to develop action plans [25] with specific dates for deliverables [25]. Activities are facilitated by the protagonist and other school personnel, and use written materials to encourage empathy by facilitating reflection [25] on experiences of violence [26], to provide new knowledge [25–27] on alternative non-violent discipline, and to provide opportunities to practice new behavioural skills [25–27]. Students are also encouraged to reflect on use of physical, sexual and emotional violence in relationships with each other. Schools are encouraged to self-monitor [25] their progress according to their action plans on a termly basis, and are prompted [25] to do so initially by Raising Voices. Reinforcement of new information and ideas [25, 26] feedback on progress and modeling of new techniques and behaviours [25, 27] is provided by visits from the Raising Voices team, and also within school by ‘protagonists’ to their peers as they gain new knowledge and skills. Schools are encouraged to reward [25] successful achievement of their goals and action plan deliverables by creating celebrations. Because the intervention engages multiple groups within a school (teachers, administration, students, and also parents), changing ideas and attitudes in different groups also creates social support [25–27] for behavioral change as the intervention progresses. Social support and specific techniques, tips, and experiences are also provided by a ‘Peer Learning Network’ of more than 100 schools using the Toolkit, moderated by Raising Voices.
Control
If the intervention is shown by the trial to be effective, control schools will be offered the following package after the end of the study: the Good Schools Toolkit, an introductory session to support implementation, and access to a peer learning support network to help them support implementation of the Toolkit. During the study, the control schools will not receive any form of programming; however, they will have the same schedule of survey, class and school level assessments as the intervention schools. If the intervention is shown by the trial to be ineffective, this will be explained to control schools. Raising Voices will refine and update the content of its programming if this is the case, and updated materials will instead be offered to control schools.
Setting
The study is being conducted in Luwero District, Uganda, which has both rural and more urban areas. Luwero has an estimated 433,100 people, 60% of whom are aged below 18 years. Luwero is demographically similar to the rest of Uganda, with an equal sex ratio of males to females and a mean household size of 4.4 persons. A total of 77% of the population aged 10 years or over was literate in 2002 (versus 68% nationally), 72% had access to safe water (versus 61% nationally), and 7.2% had access to electricity (versus 8% nationally). Both in Luwero district and at a national level, subsistence farming methods were used by 68% of households, and the majority of households (97%) relied on firewood and charcoal for cooking.
Luwero District was chosen because it is within the catchment areas of the implementing partners for this study. There are currently no prevalence or epidemiological data available on children’s experiences of violence or independent data on educational performance. Our study will provide new information for the District.
Participant selection and inclusion criteria
A two-stage selection process has been employed. In the first stage, schools were randomly selected to participate in the trial. In the second stage, within each school, all individual staff members and a random sample of Primary 5, 6 and 7 students will be invited to participate in the follow-up survey.
Schools
Our implementing partners are able to provide support to a maximum of 21 schools; with the inclusion of 21 control schools the total sample size is 42 schools. These were chosen to minimise selection bias as far as possible and to represent larger schools in Luwero. Using the official 2010 list of all 276 primary schools in Luwero as our sampling frame, we excluded 105 very small schools (with fewer than 40 registered Primary 5 students) and 20 schools with existing governance interventions, and then stratified the remaining 151 schools by the gender ratio of their pupils, into >60% girls, mixed, or >60% boys. From these 151 schools, we selected a random sample of 42 schools, proportional to size of the stratum. Of these schools, 100% agreed to participate in the study. Up to date lists of all Primary 5, 6 and 7 students, and a list of all teaching and non-teaching staff will be obtained from school head teachers at endline.
Individual participants
From these class lists, up to 130 children will be randomly selected for individual interviews at follow-up, with the aim of successfully interviewing at least 60 children per school. If a school has fewer than 130 students in P5 to 7, all children will be invited for interview. We chose to focus on students in P5 to 7 students, aged approximately 10 to 14 years, because they would be better able to respond to survey questions about their experiences versus younger children. All school staff will be invited for an interview. Research teams will be in each school for 3 to 6 days; at least one repeat visit will be made to find students who have been absent for that entire period. Students deemed unable to understand the study consent form (who will be therefore unable to provide informed consent) will be excluded.
Sensitisation and recruitment
During the planning stages of the study, Raising Voices staff visited Ministry of Education and Sports officials at national and district levels. At the national level, support for the study was indicated by the Ministry Of Education And Sports. At the district level meetings have been held with the District Education Officer in Luwero, who has given permission for the study to take place.
Invitations to individual schools selected for participation were issued by letter from Raising Voices to school Head Teachers. These letters were followed by a visit from senior program officers from Raising Voices to explain further details about what would be involved in participating in the study. Consent for participation was sought from head teachers; of these head teachers, 100% agreed to participate.
Consent for classroom level assessments, which will involve observations of whole classes, will also be sought from head teachers on the day of the assessment. At follow-up, all staff in participating schools will be invited to participate in individual level data collection by providing informed consent. An opt-out consent strategy will be used for students, consistent with other school-based research on sensitive topics in Uganda [29]. Same as at baseline, parents of children in participating schools will be informed about the study in several different ways and advised that they can opt their children out from participating. Information meetings will be held at each participating school with a staff member from school administration and a representative from the study team to explain the study to community leaders, and parents and guardians of children. These meetings will emphasize that participation of children in research data collection is voluntary, that they have the opportunity to opt out of the study at any time, and not to answer any questions that they do not want to. It will also be made clear that the study does not involve collecting of any biological specimens, and involves only asking questions of children and administering standard educational tests. Parents and community leaders will be asked to circulate word about the study to others in their community.
In participating schools, each P5, P6, and P7 child will also receive a written notice to carry home to his or her parents or caregivers. At baseline, identical procedures were used, and all of these meetings and notices were held before the baseline survey and allocation took place; schools, parents, or children were not aware of whether or not their school would receive the intervention when they completed the survey.
Individual students selected to participate in the survey will be approached within their school, and informed consent will be sought. The consent form will be read aloud to each child. This form contains a description of study procedures and will remind children that they do not have to participate and have the right to stop the interview at any time. Children will also be informed that if an interviewer feels a child’s safety is at risk, the interviewer is obligated to discuss the case with the District Probation Officer or with another party responsible for child protection locally.
Outcomes
We aim to determine the impact of the Good School Toolkit on violence, well-being and educational outcomes in children attending primary school. Specifically, we will:
-
1.
Examine whether there is a difference in children’s self-reported experience of past week physical violence by school staff, in schools that receive the intervention versus those that do not (primary outcome);
-
2.
Examine impacts of the intervention on children’s educational achievement (word recognition and reading comprehension in Luganda and English, spelling, and written numeracy), and children’s mental health (symptoms of common mental disorders and self-reported feelings of safety and well-being in school) (secondary outcomes).
Statistical power
Preliminary analysis of baseline data has shown a prevalence of past week physical violence (primary outcome) of approximately 50%, and an estimated intracluster correlation coefficient of 0.06. If we see similar levels at follow-up, allowing for a possible loss of two schools per arm and a conservative estimate of 60 pupils per school, we will be able to detect a reduction of approximately 13% in the prevalence of reported violence in the schools receiving the intervention, with 80% power and a 5% level of significance. One other pilot trial we are aware of that evaluated an educational and behavioural change program observed a more then 50% reduction in ‘negative teacher behaviours’ as rated by classroom observers over the trial period [22], so we should be well placed to detect effects of the Good School Toolkit.
Survey procedures
All questions related to violence and mental health will be asked during individual interviews with Luganda-speaking staff who receive 3 weeks of in-depth training. Individual interviews will be done using a questionnaire programmed into mobile phones (completed by an interviewer). Some educational assessments will be done on paper in groups.
Survey instruments
Students’ experiences of violence will be assessed using behaviourally specific questions about acts of violence adapted from the IPSCAN Child Abuse Screening Tool (ICAST) [30], and the WHO Multi Country Study on Women’s Health and Domestic Violence against Women (WHO MCS) [31]. We will assess acts of different severity levels and timeframes (past week, frequency in past year, and before past year). The main focus will be on violence by school staff, but we will also ask about violence from other perpetrators. Initial items were reviewed by a panel of teachers and Raising Voices staff to ensure that they would reflect the experiences of primary school children in the Ugandan context. The instrument was then pilot tested with children in Kampala and refined based on this feedback, and will be further refined based on the results of the baseline survey.
Children will be administered literacy, numeracy and sustained attention assessments adapted from an intervention study in schools in Kenya, involving one of the study team (Dr Allen) [32]. The Strengths and Difficulties Questionnaire [33] brief screening instrument will be used to measure symptoms of common childhood mental disorders, including depression, anxiety, and conduct disorder. Items measuring children’s attitudes to corporal punishment developed by the research team will also be included.
School staff will be asked about perceptions of student discipline and learning needs, attitudes about corporal punishment and violence, as well as their teaching methods, participation in school policy development and culture. Experiences of violence will be queried, using items from the WHO MCS [31]. Mental health and well-being will be assessed using the Self Report Questionnaire-20 [31], and items adapted from the ICAST (as above) will be used to ask staff about their use of violence toward students in the past week, and for other time frames.
Confidentiality and data management
Interviews will be conducted in open spaces, where interviewer-participant pairs are out of earshot but within sight of others, to protect confidentiality and ensure child safety. All data collected on mobile phones (all violence and mental health data) will be identified only by ID number, for both staff and students. All data collected by mobile phone will be stored on a password protected databases that will be online-accessible only to senior study personnel, and backed-up daily on a password-protected laptop. Students will write their names on standard educational tests, which will be double marked and double entered into a customised database, separate from other sensitive data.
Randomisation
Schools were allocated to receive either the intervention or a wait-list control at a public meeting with all school head teachers before the start of the 2012 September school term. Stratified randomisation was used to ensure balance in regard to key factors (baseline violence, whether the school was urban or rural, and a qualitative assessment of the likelihood of attrition over the course of the trial).
Analysis
Primary analysis will be carried out based upon the groups as randomised (‘intention to treat’). Results will be presented as appropriate effects sizes with a measure of precision (95% confidence intervals). Clustering by school will be allowed for in all analyses. Our main analysis of the primary outcome, the prevalence of past week experience of violence, and secondary mental health and educational outcomes will be cross-sectional analyses comparing the prevalence at follow-up between the two arms of the trial. In cross-sectional analyses, baseline school level summaries will be adjusted for, where appropriate [34]. Additional analyses will use longitudinal data and compare the change in individual scores over time between intervention and control schools. In longitudinal analyses, baseline measures of the outcomes will be adjusted for, along with other covariates such as gender and baseline age.
Trial organisation, governance, and adverse events
The trial is overseen by the principal investigator (KD) and run by a dedicated manager (JC). A data manager is responsible for monitoring data quality at baseline and follow-up data collection points, and for interim data collection on the intervention implementation. The main data monitoring concerns for a trial of this nature relate to ensuring that cases where children have been exposed to severe forms of violence are detected and that adequate steps are taken to protect them. Comprehensive referral plans have been developed for baseline and follow-up data collection (described in Ethical considerations). We will also collect ongoing monitoring data from schools during the implementation process; any child protection concerns that arise during the implementation period will proceed through the same referral pathway.
The intervention under study is behavioural and we do not anticipate adverse events occurring as a result of the intervention itself. However we will collect interim qualitative data from representatives of school staff and also from representatives of student committees about unexpected consequences and any adverse consequences of the intervention that have become apparent during implementation. This data will be collected by telephone and in-person interviews by the trial monitoring data officer, and will be entered into a computer on a continuous basis. Any adverse events will be reported immediately to the study manager (JC), the principal investigator (KD), and the implementing partners (DN). This committee will decide upon and enact appropriate responses.
Ethical considerations
In any research on violence against children, ethical considerations and child protection are paramount. We developed comprehensive protocols to ensure that children are protected and referred as necessary over the course of the study, and have received full ethical approvals from both the LSHTM (#6183) and the Uganda National Council for Science and Technology (SS 2520).
Our referral strategy for children in need of support involves working within existing child protection structures for the Ministry of Education; the health, legal, and community welfare sectors, with additional support from local NGOs; and implementing study partners to ensure proper procedures are followed. In brief, all children who participate in the study, regardless of what they disclose, will be offered the opportunity to visit with a trained counsellor who is fluent in Luganda. For children who disclose more severe experiences of violence in the past week or past year, the District Probation Officer and the representative from our local Luwero partner NGO, will be informed, and will refer cases onward in accordance with local policy. Children who disclose recent sexual violence, severe physical violence, or injury, or who have otherwise urgent conditions, will be taken immediately to a health centre, and the District Probation officer and local Luwero partner NGO representative will be informed so that further follow-up may take place.
The nature of case management and follow-up provided by the existing child protection system is the subject of a nested study within the larger Good Schools Study.