How to collect non-medical data in a pediatric trial: diaries or interviews
Trials volume 21, Article number: 36 (2020)
Non-medical data, such as the amount of time that patients and caregivers spend managing their condition, may be relevant when assessing therapeutic strategies. For chronic pediatric conditions, the time that patients and caregivers spend in seeking and providing care (which are the indirect costs in an economic evaluation) can be significantly different depending on the treatment arm. To explore methods for collecting information on the care burden for caregivers and patients, we investigated whether a patient diary provided additional information compared to retrospective investigator-led interviews and whether a diary that was completed intermittently produced more or less information than a diary completed continually. The main objective of this study was to identify which type of data collection was most effective for measuring the time spent by caregivers and for estimating indirect treatment costs over 9 months.
Start-In! is a randomized controlled trial comparing the efficacy of three strategies of real-time continuous glucose monitoring for 12 months in children and adolescents with type 1 diabetes. We designed an ancillary study to assess methods of collecting information on the time spent by patients and caregivers in managing their condition (indirect costs). Data were entered retrospectively in case report forms (CRFs) by investigators during quarterly follow-up visits, which were supplemented with diaries completed prospectively by children or caregivers either continuously or intermittently. Data about absences from school and work as well as the time that caregivers spent on diabetes care were collected and the three collection methods were compared.
At the end of the 9-month study, 42% of the study participants failed to return their diary. For the diaries that were received, less than 10% of expected data were collected versus 82% during investigators'interviews. Based on all the information collected, we calculated that over 9 months, caregivers lost on average 3.9 days of working time (€786) and 4 days of personal time, i.e. the equivalent of €526, and spent around 15 min of time on care per day, i.e. the equivalent of €1700.
The CRFs completed by investigators during quarterly visits cannot be replaced by a diary. Completing the diaries appeared to represent an important additional burden to children and their caregivers, and the diaries provided little additional information compared to investigators’ entries in the CRF.
ClinicalTrials.gov, NCT00949221. Registered on 30 July 2009. Registry name: Study of Insulin Therapy Augmented by Real Time Sensor in Type 1 Children and Adolescents (START-IN!).
Non-medical data, such as the time spent by patients and caregivers in managing their condition, may be relevant in assessing the full burden and cost of a therapeutic strategy. The pertinence of including time costs for caregivers and patients as indirect costs in economic evaluations of therapeutic strategies may be considered when writing the protocol and mainly depends upon the perspective of the evaluation and the type of illness being considered . Productivity losses can represent a significant amount of resources, especially for chronic pediatric conditions because caregivers may need to seek and also provide care for children. Information on absences from work or school is not recorded in health insurance databases, so must be estimated, either through retrospective interviews with an investigator  or from diaries completed by the parents or patients [3, 4]. Investigator-led interviews require additional work from the physicians, which could be reduced if the children or their caregivers completed diaries prospectively . However, the use of paper or electronic diaries in clinical trials requires logistical management and fastidiousness from participants, especially for studies with long follow-up periods . This may be more problematic for pediatric trials, since parental constraints may influence the acceptance of additional methods of data collection .
To determine the best way to collect information on care burden and time costs, we investigated whether or not a patient-completed diary would provide additional information compared to retrospective investigator-led interviews and whether a diary completed intermittently would produce more or less information than a continuous diary. We hypothesized first that a diary would enable investigators to shorten their interviews, which would be helped by or even replace the diaries. We assumed that patients would tend to underreport their time costs. Our second hypothesis was that keeping a continuous diary would be an onerous task and could lead to fatigue and poor data collection over time. However, the intermittent diary, while reducing the data entry burden for patients , has the risk that patients may fail to complete it if it was not a regular task.
The main objective of this study was to identify which type of data collection method was most effective for obtaining information on care burden and time costs over the 9 months of a pediatric trial. The secondary objectives were to determine the patient acceptance of an intermittent versus a continuous diary and to check whether the amount of data collected decreased over time.
Start-In! was a French multi-center trial with a randomized controlled prospective open parallel-group design, comparing three therapeutic modalities in pediatric management . We designed an ancillary study to the Start-In! trial (NCT00949221) that compared the efficacy and cost-effectiveness of three strategies of real-time continuous glucose monitoring (RT-CGM) for children with type 1 diabetes . This trial was well adapted to our study objective since insulin-dependent diabetes in children is a chronic condition with major time costs to children and their parents.
The patients in Start-In! were aged from 2 to 17 years. They had been diagnosed with type 1 diabetes for more than 1 year and were being treated by intensive insulin therapy, achieving inadequate metabolic control. The study was conducted in 11 pediatric diabetology units with expertise in pump therapy and CGM. The primary endpoint was long-term glycemic control at 12 months. The clinical study had two phases: all subjects wore glucose monitors during the first 3 months, and thereafter were randomized into three groups with different strategies for glucose monitoring during the next 9 months. The ancillary study reported here concerns the 9-month follow-up period of the patients (Fig. 1).
The Start-In! economic evaluation required information on consultations, absences from work or school due to diabetes, and the time spent by the caregiver on average each day on providing diabetes care. This information was entered for all patients by the investigators in paper case report forms (CRF) at the end of each trimester and represented the reference strategy.
To test whether diaries would provide additional information, patients were randomized into two groups. In the continuous group, children and caregivers were asked to collect data in a diary without interruption during one of the three trimesters of the study (3 months continuously). In the intermittent group, data were collected only during the last month of each of the three trimesters. Randomization was in a 3:1: ratio. In the continuous group, the diary was completed daily for one of the three trimesters of the study follow-up period, allocated randomly in a 1:1:1 ratio. To simplify the work for the investigators, the randomization for the ancillary study was by center and not by patient (Figs. 2 and 3). Data from the patient interviews and diaries were entered into the CRFs by investigators following the same schedule: one trimester for the continuous group and the last month of each trimester for the intermittent group.
The primary endpoint was the amount of information recorded in the diaries, compared to the amount of information given to the investigator during the quarterly follow-up visit. The Start-In! participants (patients and caregivers) were asked to record in their diary: (1) for the child, the dates of each period of absence from school or work, (2) for an adult caregiver, the dates of each period of absence from work, (3) the date of each consultation, (4) the number of caregivers present at each consultation, and (5) an estimate of the time spent on average each day by the caregivers to provide diabetes care such, as blood glucose monitoring and insulin administration (Additional file 1). There was no minimum or maximum for the number of absences or visits that could be recorded. The same information was also collected quarterly by the investigators and entered in the CRF with the help of the diary, which was collected at the same time.
The number of absences from work or school and the number of consultations were calculated from the data entered in the diaries for each patient. If the amount of time spent by the caregiver on average each day was not completed, it was considered to be missing data rather than zero. If the other data items were not completed at all (dates of absences from school or work, and dates of consultations), we considered that the data were missing if the average time spent by the caregiver was also missing. The overall amount of information recorded was compared between the intermittent and continuous groups, and with the amount of information entered by the investigators into the CRFs.
Care burden and indirect costs
For the indirect costs in the economic evaluation, the opportunity cost was based on the number of absences from work or school for the patients and their families. For those who were in employment, it was based on the average salary in France (€34 per hour) and for those who were unemployed, on the average salary of a house cleaner (€18.7 per hour). The average daily time spent by a caregiver administering diabetes care to a patient was valued based on the average salary of a home nurse (€28.8 per hour). All costs are in euros (€) for 2017 (1€ = 1.2US$) and were not discounted due to the short time horizon.
A total of 151 patients were randomized in the Start-In! trial, 52 into Group 1, 48 into Group 2, and 51 into Group 3. Their characteristics were similar at inclusion . They had an average age of 12 years and there was nearly a 50:50 ratio of boys to girls, although there was large differences in age and sex between the centers, as seen in Table 1. A total of 23 patients who left the study were excluded from our ancillary analysis of the diary data, as their diaries would, obviously, be empty. Thus, diaries were expected from 128 patients for analysis, 92 in the continuous group and 36 in the intermittent group. Information on care burden was additionally collected on all 151 patients by the investigators during the follow-up visits. However, almost half of the 128 patients in this analysis did not return a single diary, so the information they reported to the investigators and entered in the CRF was based solely on their memory.
Of the 92 patients, only 53.3% (n = 49) returned their diary. Of the 49 diaries returned, 44 were empty so data were considered to be missing. Four diaries reported that no absences had occurred and only one diary gave the date of an absence from work or school. That is, 89.8% of the diaries received had missing data. Overall, of the 92 diaries expected, 99.5% were either not received or had missing data.
Of the 36 patients, just under half (n = 16) returned at least one diary. Each patient should have submitted three diaries, one for each trimester, but only 41 of the 108 diaries expected were received. Of those 41 diaries, 97.6% had missing data, as only one diary was complete. Overall, 99.5% of diaries from patients were either not received or had missing data.
The same information was collected retrospectively from the patients and caregivers by investigators, who entered it into the CRFs at the trimestral follow-up visits. The information was, potentially, supplemented by the data entered prospectively in the diaries.
Table 2 shows the numbers of CRFs completed for each data item by the investigators for patients whose diaries were missing or empty, by randomization group (continuous or intermittent). For example, for the continuous group, regarding absences from work, among the 87 patients who either returned an empty diary or no diary, this information was recorded for 73 in their CRF. For the consultations section, among the 35 patients from the intermittent group with empty or no diaries, this information was recorded for 34 in their CRF. We calculated the percentage of data missing from the CRFs in the same way we did for diaries. The continuous group had more data missing in their CRFs than the intermittent group (14% vs. 5%), which is far less than the around 99% of data missing from the diaries. In conclusion, it appeared that for the diaries that were received, less than 10% of expected data were collected versus 82% during investigators'interviews.
Care burden and indirect costs
Since the CRFs, which were based upon interviews and diaries, held about 80% of the indirect cost data, we calculated productivity losses and indirect costs. Over 9 months, parents lost on average 3.9 days of working time, i.e. the equivalent of €786, and 4 days of unemployed time, i.e. the equivalent of €526, and spent around 15 min of time on care per day, i.e. the equivalent of €1700. Children were absent from school for an average of 3 days over the 9 months.
Some trials require non-medical data that are not readily available from patients’ charts or an insurance database. Investigators must either rely on patients’ recollection at the time of a follow-up visit or provide diaries. Our objective was to estimate the indirect medical costs in each treatment arm and identify possible differences that cannot be captured by calculating direct medical costs based upon information systematically retrieved from hospital insurance claims data. All investigators were asked to record in the CRFs data on caregiver and patient burden (time), and they relied on information obtained from both interviews and diaries. Our study compared two prospective methods of documenting the treatment burden for caregivers and children with type 1 diabetes during a clinical trial, which included an economic evaluation. Our major finding was the magnitude of the data missing in the diaries provided to caregivers (around 99%). While the average cost of caregivers’ time could be estimated from investigator-led interviews to be around €3000 per patient over 9 months, the information from diaries alone did not allow a reliable estimate to be calculated. It would appear that asking families to record non-medical information in a diary during a pediatric trial of a severe chronic condition is infeasible, and we do not recommend their use.
The amount of data entered by investigators for patients in the intermittent group was slightly higher than for the continuous group (95% vs. 86%). This may be because the participants were asked to remember information for only one month at the time of an interview instead of for three . In this trial, the CRFs always contained more data than the diaries since the CRFs were based upon interviews supplemented by the diaries (if the diaries had data). In fact, diaries can be a reliable source of data to complement usual reporting methods, as shown by Dunn et al. , but we were not able to confirm this conclusion because of the amount of missing diary data.
Many studies from different countries show a high concordance between self-reported and medical records of healthcare resource utilization in the general patient population [11, 12]. We were interested in pediatric trials, in which the diaries would be completed by caregivers, who may be tired from caring for a chronically ill child in addition to their jobs and may have other children to take care of. In fact, family members of someone with diabetes seems to be at higher risk of depression  and parents of a child with type 1 diabetes not only struggle with depression [14, 15] but also worry about hypoglycemia , which can lead to parental emotional distress . Difficulties are encountered with adolescent patients, who are becoming more independent but have troubled relationships with their parents  and may have transition issues, like treatment adherence [19, 20]. In those circumstances, reporting non-medical data, such as school absences, may not be a priority for the family.
Unfortunately, the patients and their caregivers were not asked to give their opinions about the different methods of data collection, so we can only speculate about why so much data were missing. This study has other limitations. The amount of missing data prevented us from performing statistical tests or evaluating our secondary objectives of patient acceptance and fatigue. Moreover, with such little data, the quality of the information provided in the diaries could not be assessed and thus, only the quantity was explored in our trial.
Trials that require non-medical data may need to rely on patients and families for information. In our study of a pediatric chronic condition, investigator-led interviews provided much more information than patient diaries. The completion of diaries appears to represent an important additional burden to children and their families, who are already struggling with many competing issues. Notwithstanding, the large indirect costs associated with the loss of parental productivity underlines the importance of collecting these data, in one way or another.
Availability of data and materials
The datasets analyzed during the current study are available from the corresponding author on reasonable request.
Haute Autorité de Santé - Choix méthodologiques pour l’évaluation économique à la HAS [Internet]. [cité 30 juill 2018]. Disponible sur: https://www.has-sante.fr/portail/jcms/r_1499251/fr/choix-methodologiques-pour-l-evaluation-economique-a-la-has
Laidi C, Prigent A, Plas A, Leboyer M, Fond G, Chevreul K, et al. Factors associated with direct health care costs in schizophrenia: Results from the FACE-SZ French dataset. European Neuropsychopharmacology. 2018;28(1):24–36.
Lin C-WC, March L, Crosbie J, Crawford R, Graves S, Naylor J, et al. Maximum recovery after knee replacement--the MARKER study rationale and protocol. BMC Musculoskelet Disord. 2009;10:69.
Rösch M, Leidl R, Tirpitz C, Reinshagen M, Adler G, König HH. Cost measurement based on a cost diary in patients with inflammatory bowel disease. Z Gastroenterol. 2002;40(4):217–28.
Dunn J, Harris B, Iqbal G, Wood J, Mistry P, O’Beirne-Elliman J, et al. Use of patient diaries in conjunction with standard reporting methods: duplication of data or a valuable resource? Trials. 2015;16(2):P183.
Hendriks MRC, Al MJ, Bleijlevens MHC, van Haastregt JCM, Crebolder HFJM, van Eijk JTM, et al. Continuous versus intermittent data collection of health care utilization. Med Decis Mak. 2013;33(8):998–1008.
von Niederhäusern B, Saccilotto R, Schädelin S, Ziesenitz V, Benkert P, Decker M-L, et al. Validity of mobile electronic data capture in clinical studies: a pilot study in a pediatric population. BMC Med Res Methodol. 2017;17(1):163.
Seidl H, Meisinger C, Wende R, Holle R. Empirical analysis shows reduced cost data collection may be an efficient method in economic clinical trials. BMC Health Serv Res. 2012;12:318.
Guilmin‐Crépon S, Carel J-C, Schroedt J, Sulmont V, Salmon A-S, Tallec CL, et al. Is there an optimal strategy for real-time continuous glucose monitoring in pediatrics? A 12-month French multi-center, prospective, controlled randomized trial (Start-In!). Pediatric Diabetes [Internet]. [cité 21 févr 2019];0(0). Disponible sur: https://onlinelibrary.wiley.com/doi/abs/10.1111/pedi.12820
Heyer GL, Perkins SQ, Rose SC, Aylward SC, Lee JM. Comparing patient and parent recall of 90-day and 30-day migraine disability using elements of the PedMIDAS and an Internet headache diary. Cephalalgia. 2014;34(4):298–306.
Lubeck DP, Hubert HB. Self-report was a viable method for obtaining health care utilization data in community-dwelling seniors. J Clin Epidemiol. 2005;58(3):286–90.
van den Brink M, van den Hout WB, Stiggelbout AM, Putter H, van de Velde CJH, Kievit J. Self-reports of health-care utilization: diary or questionnaire? Int J Technol Assess Health Care. 2005;21(3):298–304.
Jia Z, Li X, Yuan X, Zhang B, Liu Y, Zhao J, et al. Depression is associated with diabetes status of family members: NHANES (1999–2016). J Affect Disord. 2019;249:121–6.
Jaser SS, Whittemore R, Ambrosino JM, Lindemann E, Grey M. Mediators of Depressive Symptoms in Children with Type 1 Diabetes and their Mothers. J Pediatr Psychol. 2008;33(5):509–19.
Driscoll KA, Johnson SB, Barker D, Quittner AL, Deeb LC, Geller DE, et al. Risk Factors Associated with Depressive Symptoms in Caregivers of Children with Type 1 Diabetes or Cystic Fibrosis. J Pediatr Psychol. 2010;35(8):814–22.
Aalders J, Hartman E, Nefs G, Nieuwesteeg A, Hendrieckx C, Aanstoot H-J, et al. Mindfulness and fear of hypoglycaemia in parents of children with Type 1 diabetes: results from Diabetes MILES Youth – The Netherlands. Diabet Med. 2018;35(5):650–7.
Haugstvedt A, Wentzel-Larsen T, Graue M, Søvik O, Rokne B. Fear of hypoglycaemia in mothers and fathers of children with Type 1 diabetes is associated with poor glycaemic control and parental emotional distress: a population-based study. Diabet Med. 2010;27(1):72–8.
Jaser SS, Grey M. A Pilot Study of Observed Parenting and Adjustment in Adolescents with Type 1 Diabetes and their Mothers. J Pediatr Psychol. 2010;35(7):738–47.
Changes in Treatment Adherence and Glycemic Control During the Transition to Adolescence in Type 1 Diabetes | Diabetes Care [Internet]. [cité 22 août 2019]. Disponible sur: https://care.diabetesjournals.org/content/35/6/1219
Lotstein DS, Seid M, Klingensmith G, Case D, Lawrence JM, Pihoker C, et al. Transition From Pediatric to Adult Care for Youth Diagnosed With Type 1 Diabetes in Adolescence. Pediatrics. 2013;131(4):e1062–70.
The authors would like to thank Meryl Darlington for reviewing this work, as well as Neil Jouanjean and Céline Quelen for their help in managing the diary data.
Start-in! Study Group
Nadia Tubiana-Rufi, Sophie Guilmin-Crépon, Jean-Bruno Lobut, Patricia Sierra, Christine Delcroix (all CHU Robert Debré, AP-HP); Véronique Sulmont (CHU Reims, CH Annemasse); Anne-Sophie Salmon, Pierre-François Souchon (both CHU Reims); Claire Le Tallec (CHU Purpan, Toulouse); Régis Coutant, Natacha Bouhours, Sylvie Dufresne, Frederique Gatelais (all CHU Angers); Fabienne Dalla-Vale, Denis Morin (both CHU Arnaud de Villeneuve, Montpellier); Claire Stuckens, Louis Montagne, Marie Dupre, Stéphanie Coopman (all CHU Lille); Hélène Bony-Trifunovic, Karine Braun (both CHU Amiens); Hélène Crosnier, Claire Personnier (both CH Intercommunal de Poissy); François Kurtz (CH Saint-Avold); Florentia Kaguelidou, Laurence Corvez, Véronique Beruer, Evelyne Jacqz-Aigrain (all Centre d’Investigation Clinique CIC 1426, Inserm, Robert Debré Hospital); Rolande Ducrocq (Laboratoire de Biochimie Métabolique, CHU Robert Debré, AP-HP); Adyla Yacoubi, Tania Rilcy, Cécile Hoffart-Jourdain, Akim Souag, Béatrice Andriss, Damir Mohamed, Isabelle Durand-Zaleski (all Unité de Recherche Clinique Robert Debré, Unité de Recherche Clinique Economie de la Santé and Direction de la Recherche Clinique et de l’Innovation, AP-HP).
This study was supported by Direction générale de l’offre de soin and sponsored by Assistance Publique, Hôpitaux de Paris (AP-HP).
Ethics approval and consent to participate
The AFSSAPS (National Agency for the Safety of Medicines and Health Products) approved the protocol of the START-IN! trial (2007-AO1330-53). Informed consent was obtained from all study participants.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
About this article
Cite this article
Le Jeannic, A., Maoulida, H., Guilmin-Crépon, S. et al. How to collect non-medical data in a pediatric trial: diaries or interviews. Trials 21, 36 (2020). https://doi.org/10.1186/s13063-019-3997-9
- patient diary
- data collection
- time costs
- investigator-led interview