The NYCKidSeq project: study protocol for a randomized controlled trial incorporating genomics into the clinical care of diverse New York City children
Trials volume 22, Article number: 56 (2021)
Increasingly, genomics is informing clinical practice, but challenges remain for medical professionals lacking genetics expertise, and in access to and clinical utility of genomic testing for minority and underrepresented populations. The latter is a particularly pernicious problem due to the historical lack of inclusion of racially and ethnically diverse populations in genomic research and genomic medicine. A further challenge is the rapidly changing landscape of genetic tests and considerations of cost, interpretation, and diagnostic yield for emerging modalities like whole-genome sequencing.
The NYCKidSeq project is a randomized controlled trial recruiting 1130 children and young adults predominantly from Harlem and the Bronx with suspected genetic disorders in three disease categories: neurologic, cardiovascular, and immunologic. Two clinical genetic tests will be performed for each participant, either proband, duo, or trio whole-genome sequencing (depending on sample availability) and proband targeted gene panels. Clinical utility, cost, and diagnostic yield of both testing modalities will be assessed. This study will evaluate the use of a novel, digital platform (GUÍA) to digitize the return of genomic results experience and improve participant understanding for English- and Spanish-speaking families. Surveys will collect data at three study visits: baseline (0 months), result disclosure visit (ROR1, + 3 months), and follow-up visit (ROR2, + 9 months). Outcomes will assess parental understanding of and attitudes toward receiving genomic results for their child and behavioral, psychological, and social impact of results. We will also conduct a pilot study to assess a digital tool called GenomeDiver designed to enhance communication between clinicians and genetic testing labs. We will evaluate GenomeDiver’s ability to increase the diagnostic yield compared to standard practices, improve clinician’s ability to perform targeted reverse phenotyping, and increase the efficiency of genetic testing lab personnel.
The NYCKidSeq project will contribute to the innovations and best practices in communicating genomic test results to diverse populations. This work will inform strategies for implementing genomic medicine in health systems serving diverse populations using methods that are clinically useful, technologically savvy, culturally sensitive, and ethically sound.
Trial Sponsor: Icahn School of Medicine at Mount Sinai
Contact Name: Eimear Kenny, PhD (Principal Investigator)
Address: Icahn School of Medicine at Mount Sinai, One Gustave L. Levy Pl., Box 1003, New York, NY 10029
With the precipitous drop in the cost of genomic sequencing technology over the past two decades, genomic information is increasingly informing clinical decision-making across health systems . There are currently over 5500 single gene disorders and traits with a known molecular etiology (https://www.omim.org/statistics/geneMap). Since 2009, targeted gene panels (TGPs) and exome sequencing (sequencing some or all of the protein-coding regions of the genome, respectively) have been increasingly used for the diagnosis of these individually rare, but collectively common disorders. While the majority of clinical sequencing currently uses panels or exomes, there is an increasing number of pilot programs using more advanced genetic modalities, such as whole-genome sequencing (WGS), which has the potential to capture all classes of genetic variation in one analysis [2,3,4]. These advancements in genomic sequencing and testing have sparked innovation in methods for scaling genomic medicine services across health systems.
Although genomic testing is offered to patients in clinical care more routinely, a number of barriers remain to successful implementation of genomic medicine, particularly for racially and ethnically diverse populations. Genomics research in the past has been conducted predominantly in individuals of European ancestry [5, 6], which has led to significant disparities in the accuracy and clinical utility of genomic sequencing in non-European populations [7,8,9]. Non-European individuals are more likely to receive variants of uncertain significance or misclassified results, which has been demonstrated in the context of genetic testing for hypertrophic cardiomyopathy and hereditary cancer risks in under-studied populations [10,11,12,13]. Evidence suggests that this bias is likely to persist in the ongoing and upcoming efforts to sequence entire genomes. Diversifying the data pool in genomics research is a major initiative of federal funding agencies such as the National Human Genome Research Institute (NHGRI) with the goal of enhancing the accuracy, utility, and acceptability of genomic testing in clinical care of diverse populations .
Communication of genomic information in health systems poses another significant challenge to genomic medicine implementation . For the clinical benefits of genomic testing to be realized, results must be communicated effectively to patients and families undergoing this testing . Genomic test results can be complicated, and vital information can be lost if results are not conveyed in a way that patients and families understand. More studies are emerging to develop and evaluate strategies for communicating genomic results in diverse populations. In New York City (NYC), investigators have evaluated the use of a brief educational program to improve knowledge of complex genomic concepts in a predominantly Hispanic community [16, 17]. To extend the cultural competence of genetic counseling methods, studies have also designed and assessed the use of culturally tailored genetic counseling methods to convey unique aspects of breast cancer in African-American women . Other studies have explored the use of narrative educational tools for at-risk Latinas to improve psychosocial outcomes [19, 20]. The vast majority of literature on genetic counseling interventions in diverse populations has focused mainly on cancer genetics , and there is a dearth of literature on barriers to communication in racially and ethnically diverse pediatric patient populations and their families. Digital applications that display personalized genetic testing results in a way that addresses low-health literacy, includes images, and provides information in both English and Spanish can be leveraged to improve understanding.
A further barrier to the broad adoption of genomic medicine is that until recently, this discipline has been the province of a small minority of specialized physicians trained in genomics. The majority of physicians do not receive extensive training in genomics. For example, a survey of US physicians, including generalists and specialists, found that 79% and 69% of primary-care and non-primary-care physicians, respectively, report that “lack of knowledge about genomic medicine” is a barrier to its incorporation in practice [22, 23]. In a recent study of medical students, 79% felt that it was important to apply genomics to clinical care, but only 6% thought that their medical education had adequately prepared them to practice . Genetic testing technologies are moving at a fast pace, and even physicians in a single specialty area can have difficulty keeping up to date on the most effective way to order testing and interpret test reports. For clinical genetic testing laboratories, diagnostic pipelines involve a manual curation step that jointly considers the patients’ genomic variation in the context of the clinical indication for testing and the patient’s phenotypic manifestations, as provided by the physician. However, the input of the clinician’s assessment is usually limited to a few short descriptions on a test requisition that the laboratory personnel must interpret. Interactions between clinical testing labs and ordering providers must be improved to facilitate the interpretation of genomic test results.
The NYCKidSeq study is a genomic implementation research program that will assess strategies for enhanced communication of genomic information in health systems and evaluate the utility of advanced genomic sequencing technology for increased diagnosis. The study will recruit 1130 children with suspected genetic disorders in historically underserved racially and ethnically diverse communities of NYC. This is a multi-institutional research program with three participating sites: the Icahn School of Medicine at Mount Sinai (MS), Albert Einstein College of Medicine/Montefiore Medical Center (EM), and the New York Genome Center (NYGC). NYCKidSeq has three major aims: (1) to develop and evaluate the effectiveness of a novel digital application, GUÍA, to improve communication of genomic test results to patients in a randomized controlled trial (RCT); (2) to compare the diagnostic yield of WGS versus TGPs in a racially and ethnically diverse cohort; and (3) to evaluate the utility in a pilot study of the novel digital application, GenomeDiver, to enhance the interpretation of sequencing data by laboratory personnel, to direct reverse phenotyping by clinicians, and to enhance the communication between clinical and laboratory personnel in the diagnostic process. The study will also examine the utility of genetic ancestry in clinical diagnostic pipelines, evaluate costs associated with genetic testing, and assess provider attitudes toward genomic medicine. This study is one of six clinical sites funded as part of the Clinical Sequencing Evidence-Generating Research (CSER2) consortium, jointly funded by NHGRI and the National Institute of Minority Health and Health Disparities (NIMHD) .
Study design overview
Figure 1 shows the NYCKidSeq project study schema illustrating the flow of enrollment from participant referral to administration of the last parental survey. NYCKidSeq is an RCT evaluating the use of GUÍA to facilitate the return of genomic results compared to standard of care (SOC) genetic counseling. Outcomes to be assessed are parental understanding, satisfaction, feelings about the results, and participants’ subsequent behavior. Surveys will collect data at three study visits: baseline (0 months), result disclosure (ROR1, approximately + 3 months), and follow-up (ROR2, approximately + 9 months). WGS and TGPs will be performed for diagnostic purposes in 1130 children and young adults with specific, suspected genetic disorders in an effort to assess clinical utility and compare diagnostic yield of both testing methods. Prior to the launch of the RCT, a lead-in pilot phase of 30 participants was conducted to solicit input from families regarding the survey instruments and GUÍA. In designing this study, stakeholders were engaged at key stages of development to facilitate successful implementation of this genomic medicine program and contribute to its cultural appropriateness and sensitivity. As a member-site of the CSER consortium, the funding source has a role in the design of this study with regard to its recruitment goals and outcome measures.
Recruitment, enrollment, and sample size
Potential participants are receiving medical care under a physician at a participating institution (MS or EM). Participants and their families are introduced to the study by their physician during a routine clinic visit, phone call, or during an in-patient admission. Potential participants who express interest in the study and verbally consent to being contacted by study staff are referred to the study team. Study staff confirms the participant’s eligibility and obtains informed consent to complete the baseline parental survey during an in-person encounter. Surveys are conducted in either English or Spanish, depending on participant preference. Informed consent for WGS and TGP testing is obtained by a genetic counselor during an in-person baseline encounter in the participant’s preferred language (English or Spanish), and assent is obtained from capable children. Enrollment is achieved after a blood draw for the proband’s sample for genetic testing has been collected. At the same time, blood is drawn from biological parents. If one or more biological parents are not available during the visit, a saliva kit is mailed to the home to collect a saliva sample. The enrollment target is 1130 participants, including the lead-in phase. Participants enrolled in the RCT receive $80 in gift cards for completing all three study visits and those enrolled in the lead-in phase receive $120 in gift cards. Referring physicians do not receive compensation for their participation.
Inclusion and exclusion criteria
Table 1 presents the inclusion and exclusion criteria for NYCKidSeq participants. All participants are followed by a physician in the participating institutions. Patient participants are 0–21 years of age; young adults (18-21 years of age) who are cognitively intact are included in this study provided that their parent(s) or legal guardian(s) also agree to participate. All participants have a currently undiagnosed, suspected genetic cause of their specific neurologic, immunologic, or cardiac disorders. Specifically, participants have at least one of the following: seizure disorder, intellectual disability, global developmental delay, congenital heart disease, cardiomyopathy, cardiac arrhythmia, or features of a primary immunodeficiency. Patients will be excluded if they have a known molecular genetic diagnosis, an obvious genetic diagnosis based on clinical features, or if they have undergone a bone marrow transplant. Inclusion of children of European ancestry is capped at < 40% of total participants to ensure that at least 60% of participants are from underserved populations, consistent with the requirements of this funding opportunity.
Engagement with diverse populations
The NYCKidSeq project is recruiting children, young adults, and their families from low-income and minority communities which are underrepresented in genomics research and are frequently the last to benefit from advances in research and technology. Participants of all racial and ethnic backgrounds who speak English or Spanish are included with the following distribution of race/ethnicity expected: approximately 1/3 Black/African ancestry, 1/3 Latino/Hispanic ancestry, and 1/3 White/European ancestry.
Engagement of non-English-speaking patients
Recruitment and retention materials (NYCKidSeq website, brochures, and participant letters), study documents (informed consent documents and surveys), and GUÍA are offered in Spanish and English. Study materials were translated by study staff of Latin American and European descent into Spanish representing five Spanish dialects: Mexican, Cuban, Dominican, Puerto Rican, and Castilian. All grew up in exclusively or mostly Spanish-speaking homes, completed Spanish coursework in high school or college, and have worked on research projects that recruited Spanish-speaking participants of a variety of ages, countries of origin, and literacy levels. All had assisted with translation and administration of study materials for prior projects. Staff consulted several online Spanish translation resources such as Word Reference  or Linguee , as needed for the development of multi-dialect compatible content. GUÍA text that is not patient specific was translated by study staff. Participant-specific GUÍA text is translated by a study genetic counselor using Google Translate and then reviewed by Spanish-speaking staff to ensure accuracy. Translated survey measures and GUÍA were piloted during the lead-in phase of the study to obtain parents’ feedback on the understandability of the translated text.
Engagement with genomic stakeholder board
NYCKidSeq engaged the Mount Sinai Genetics and Genomics Stakeholder Board including community leaders of color, clinicians, and researchers working together for several years on bridging the gap between academia and communities likely to benefit most from genomic scientific discovery. They participated in designing the study and conceptual framework, consent procedures, patient educational materials, surveys, and recruitment strategies and materials. They meet on a bi-monthly basis to discuss study status, provide feedback to recruitment and retention challenges, and assist with the analysis of study data, using principles of community-based participatory research to guide their work and ensure meaningful participation [28, 29].
Clinical genomic testing
Participants enrolled in NYCKidSeq receive clinical WGS as well as appropriate TGP testing. Proband and biological parental samples, when available, are collected and sent to the NYGC for WGS analysis and Sema4 genetic testing laboratory for TGP analysis. WGS with mean coverage of at least 30x is performed on the NovaSeq platform and is performed as single proband, duo, or trio sequencing depending on the availability of parental samples. TGP tests offered through the study include neurodevelopmental (448 genes), immunodeficiency (250 genes), and cardiovascular (241 genes) gene panels and are performed as proband sequencing. For probands with symptoms in more than one specified disease area of interest to this study (neurological, cardiac, or immunologic), multiple TGP tests may be ordered. Participants and biological parents may opt-in to receiving secondary findings from WGS testing. For secondary findings, this study is reporting expected pathogenic variants in the 59 genes that the American College of Medical Genetics and Genomics (ACMG) recommends laboratories report . Sequencing analysis and variant classification occur at the laboratories using their individual variant interpretation pipelines, and Sanger validation of suspected pathogenic variants is performed. Separate clinical reports are released for each test ordered (i.e., participants will receive at least two test reports). Sema4 and NYGC are Clinical Laboratory Improvement Amendments (CLIA)-certified and approved by the New York State Department of Health to perform TGP and WGS for clinical purposes.
RCT intervention (GUÍA)
The Genomic Understanding, Information and Awareness (GUÍA) digital application was developed from formative research as part of the NYCKidSeq study . GUÍA facilitates delivery of individualized genomic results and clinical information to participants and families by allowing genetic counselors to walk participants through their genomic test results in a personalized, highly visual, and narrative manner. GUÍA consists of distinct pages with sub-tabs representative of the essential components of a genetic counseling result disclosure session. This includes summaries of the proband’s primary and secondary genomic results, recommendations for next steps for clinical care, inheritance information, educational modules to learn more about the basics of DNA and genomic sequencing, and web links to support groups and related resources. GUÍA presents information in a modular way, allowing the participant to control the depth of the information provided during the session. It can display text in both English and Spanish to increase accessibility for a greater number of participants and their families.
Pilot intervention (GenomeDiver)
GenomeDiver is a digital application developed as part of the NYCKidSeq study . Using the GenomeDiver web-based platform, the ordering provider is presented with Human Phenotype Ontology (HPO) terms that help to discriminate the candidacy of the highest-ranked DNA sequence variants potentially causing the patient’s phenotype. Following this reverse phenotyping, the enhanced phenotypic information is then used to re-prioritize variants, in turn generating a list of diseases for assessment by the clinician. The additional phenotypic information and any diseases flagged by the clinician as potentially matching the patient’s presentation are then returned to the diagnostic laboratory to inform their interpretation of genomic test results, with the goal of improving genomic diagnostics.
Participants are randomized to one of two study arms: the control arm designed to approximate SOC genetic counseling for result disclosure, and the GUÍA arm. “Standard of care” genetic counseling in a research setting has not been well defined in the literature. It is challenging to define SOC in genetic counseling as genetic counselors practice in a variety of clinical settings, both in-person and remotely. For the purposes of this study, SOC genetic counseling for result disclosure consists of contracting, review of the purpose of the genomic testing, and disclosure of the child’s genomic test results. For positive test results, the genetic counselor describes the diagnosis, associated symptoms, management recommendations, and life expectancy, if known. The genetic counselor then discusses the inheritance pattern and recurrence risks and identifies at-risk family members who may also require/consider testing. In the case of negative results, the genetic counselor discusses the implications of such a result, such as the possibility that there is a genetic cause for the child’s symptoms that was unable to be identified at this time by this testing. For ambiguous results, the genetic counselor explains the meaning and uncertainty associated with these types of results and provides recommendations for continued disease management. The genetic counselor also discloses any secondary findings to participants who opted to receive those results. Throughout the session, explanations are accompanied by visual aids at the discretion of the genetic counselor. Psychosocial concerns are addressed throughout the encounter. The genetic counselor provides medical and support referrals, when appropriate. As the WGS and TGP are approved for clinical purposes, reports are given to the families and incorporated into their medical records and shared with referring physicians. Letters or handouts that summarize the sessions are provided after disclosure of results to explain the findings to the patient, family, physicians, and/or insurance companies for additional services. Patient or family support resources such as syndrome- or symptom-specific parent support groups, research and awareness organization websites, and/or publicly available information booklets are provided based on the needs of the family. Families are encouraged to return to their referring provider for continued post-test clinical care.
During the GUÍA genetic counseling result disclosure, the genetic counselor follows the same procedures as those outlined for the SOC arm using GUÍA during the genetic counseling session. Prior to the result disclosure session, genetic counselors personalize GUÍA by inputting genomic test result information, clinical details, inheritance, family implications, medical recommendation, and support referrals. At the close of the result disclosure appointment, the genetic counselor provides the family a hard copy of their child’s personalized GUÍA report.
Figure 1 shows the study flow and data collection points of the NYCKidSeq project. This includes a lead-in phase of the first 30 participants and the subsequent RCT. The first 30 families who met eligibility requirements and agreed to participate were entered into the lead-in phase of the study. All other participants are enrolled into the RCT and randomized to either the SOC or the GUÍA study arm.
The participants of the lead-in phase (N = 30) were not randomized to a study arm. Instead, they were asked to provide feedback on the surveys or on GUÍA. Participants in the lead-in phase completed the same series of study visits as those in the RCT phase (study visits are described in detail below).
The first 15 participants received genomic results with genetic counseling using GUÍA. After their result session, a trained study team member collected participant feedback on GUÍA using a brief, structured feedback guide to explore parents’ reactions to GUÍA. Feedback was used to address and clarify wording/phrasing, use of images, order of information, amount and detail of information, pace, and potential Spanish translation issues.
The next 15 participants provided input about the surveys. After each study survey was administered, a trained member of study staff gathered participant feedback on the survey using a “think aloud” format. Participant feedback focused on survey question clarity, flow, and order. These participants received genomic results using SOC genetic counseling.
Randomized controlled trial
Participants in the RCT (N = 1100) are randomized using a stratified randomization scheme by disease category (cardiac, neurologic, immunologic) and clinical site as seen in Fig. 2. Randomization occurs prior to the baseline visit (BL) via a randomization module in REDCap applied by a study staff member. The REDCap randomization allocation is not revealed to study staff at any point in the study.
At the BL visit, participants complete the baseline surveys with a study staff member and receive pre-test genetic counseling by a genetic counselor designated to their arm. The consenting process consists of the family being educated about the study; the risks, benefits, and limitations of genomic testing; purpose of genomic testing; possible results of genomic testing including the option to receive or decline secondary findings; and potential implications for other family members. As part of the pre-test genetic counseling, the genetic counselor obtains a medical and family history. Families who consent for testing undergo blood or saliva collection for both TGP and WGS. Parents and cognitively intact young adults will also provide permission to use of de-identified samples in future research; sharing of de-identified data in secure, public research databases; and to be contacted by trial investigators for further informational and consent-related purposes. Participants consenting to take part in this study voluntarily agree to indefinite storage of their and their child’s blood and sequencing information by the research study, including NYCKidSeq research teams at Sema4, NYGC, EM, and MS. Samples may be used for either research or clinical purposes if additional testing is needed. Participants can decide to withdraw consent for the storage of their or their child’s biological samples at any time by contacting the principal investigator. Sample(s) or portions thereof that have not already been used will be destroyed; however, the parent or child’s sample may have already been distributed to other researchers within NYCKidSeq before the request to destroy was received and may not be able to be retrieved.
Once a participant’s results are reported, generally after 3 months, the results are sent to the genetic counselor. Results are reviewed and shared with the referring physician and/or a geneticist who then shares their interpretation about the significance of the genomic findings as well as their medical recommendations with the genetic counselor. An ad hoc discrepancy committee is available to review cases at the discretion of the genetic counselor for cases that have discrepant or unsatisfying results. The committee consists of NYCKidSeq medical geneticists, genetic counselors, laboratory directors, and referring providers. The decision of the discrepancy committee is used as a final diagnostic determination.
Each participant has a one-on-one result disclosure visit with a genetic counselor (ROR1). The referring physician can also participate in the appointment at their discretion. At the result disclosure appointment, the participant is informed of the results and their referring provider’s medical recommendations. The method of genetic counseling delivery depends on the study arm to which the participant is randomized. After genetic counseling, participants immediately complete the ROR1 survey with a study staff member. Six months after the ROR1 visit, approximately 9 months after study entry, a follow-up visit (ROR2) occurs either by phone or video. During this interaction, a study staff member administers the final survey (ROR2 survey).
Each subject’s genetic results may be reviewed every 12 months for the duration of the study. This is because information about genetic variants can change over time, as can the patient’s phenotype. As both types of information contribute to making a diagnosis, a re-analysis that recognizes reclassification of DNA sequence variants in a patient and their current phenotypic presentation can combine to change their original diagnosis. Variant reclassification information is derived by the laboratory from public databases such as ClinVar, while the refined phenotypic information is prompted by and entered into GenomeDiver following a review of electronic medical records. If results are reinterpreted, a new visit is arranged to inform the family of the finding. The visit to review the results is performed by a genetic counselor.
We will also recruit 20 referring providers, 6 genetic counselors, and 7 laboratory staff members for our GenomeDiver pilot study. Two different interventions will be performed. One is retrospective and facilitates updating participants’ phenotypic information 12 months after ROR1 to help with the review of the original diagnostic laboratory report. The second intervention is part of the ongoing recruitment of patients and occurs as part of the primary analysis of the patient’s genomic sequence. We randomize these patients into two arms, one with SOC, the other with the addition of a GenomeDiver intervention. Following the generation of the annotated Variant Call Format (VCF) file, the clinicians (referring provider and genetic counselor assigned to the patient) are contacted and requested to perform a GenomeDiver session. HPO terms that help to discriminate the variants with the highest Exomiser combined scores  are presented to the clinicians and categorized as present, absent, or uncertain for that patient. Potential phenotypes present in the patient are then displayed for clinician evaluation and possible flagging, and the enhanced, updated information is then returned to the diagnostic laboratory.
RCT survey measures
The NYCKidSeq project is assessing participant outcomes through surveys administered at BL, prior to pre-test counseling and consent for the genomic testing, following the result disclosure genetic counseling during ROR1, and 6 months after disclosure of results at ROR2. The primary study outcome is the participant’s perceived understanding of genomic testing results, with comparison of results in SOC arm to GUÍA arm. The secondary study outcomes are objective understanding of genomic testing results and understanding of medical follow-up, the actionability of genomic results, and adherence to medical follow-up recommendations, with comparison of results in SOC arm to GUÍA arm. Additional participant outcomes focus on six domains: (a) attitudes toward genomic testing, (b) perceived utility, (c) psychological, (d) behavioral, (e) social, and (f) economic impact of genomic testing. With the exception of economic impact, all outcomes will be compared between the two study arms. The CSER consortium harmonized survey measures so that CSER projects, when possible, administer standardized survey measures  to facilitate combining data into a single data set for cross-consortium analysis. Table 2 summarizes participant outcomes being assessed across the three time points. When possible, previously validated measures were used. The BL and ROR1 surveys are administered in-person by a study staff member while the ROR2 survey is administered by a study staff member over the phone or video. Surveys are administered in either English or Spanish. All survey response data is entered and maintained in the REDCap database.
Pilot study measures
The quantitative outcome sought from the GenomeDiver intervention is whether it led to an increase in the rate at which the genetic test yields a clinical diagnosis compared with SOC. We are also testing other outcomes. Laboratory personnel will be asked to assess how long analyses took for individual patients; how many variants were considered per patient; whether they gained insights into the ability of clinical colleagues to identify specific phenotypes; and whether the prospective GenomeDiver intervention overall changed the time to issue a report, as a concern is that introducing a delay in analyzing the VCF file while waiting for GenomeDiver input could lead to the report being overdue. Clinicians will be asked whether the HPO terms appeared to be appropriate for the presentation using yes, no, or maybe designations; the time spent performing sessions; the diagnostic value and ease of interpretation of HPO terms; whether any further testing was prompted by the suggested HPO terms; and any difficulty categorizing specific HPO terms because of the race/ethnicity of the patient. The analysis will also include testing whether the referring provider’s specialty or patient properties, such as age, number of notes in the electronic medical record, or length of time in the health care system are associated with the HPO term categorization patterns.
Diagnostic yield and comparison of WGS to TGP
The overall diagnostic yield of the genomic testing will be calculated as the percentage of NYCKidSeq participants with definitive or likely positive diagnoses. Individual diagnostic yield will be calculated for WGS and TGP tests as well as by disease category (neurology, cardiology, immunology). We will also investigate the concordance between the two testing modalities. Lastly, we will assess the diagnostic yield of both tests among race/ethnic groups. Genomic testing result categorization for both testing modalities is maintained in the REDCap database.
Genetic counselors and other study clinicians access the participants’ medical record to obtain relevant clinical information, such as medical diagnoses and previous genetic test results. This information is reviewed and collected in accordance with the Health Insurance Portability and Accountability Act (HIPAA). The following procedures are used at MS and EM to safeguard data: (1) train staff on data sensitivity and safeguards; (2) store and process sensitive hard copy in a centralized location; (3) secure sensitive hard copy in locked files when not in use; (4) remove names, addresses, and other direct identifiers from hard copy and computer-readable data if they are not necessary for participant tracking; (5) destroy all identifiable links to data after accuracy has been verified and final analyses have been completed; and (6) protect the patient information file, secured in our file server, by Microsoft NT-encrypted password and a separate password to access the database file on the server.
Limited identifying information of consented participants is stored in a web-based REDCap database. The REDCap server is managed by Mount Sinai IT and is firewall protected. User access to the database for study personnel is managed by the study project manager. Data access for study personnel is limited to their site's participants and what is required for their roles on the project. The NYCKidSeq program gives participants the option to consent to share de-identified genetic and related clinical information with other CSER investigators and access-restricted scientific databases.
As this study involves genetic testing for diagnostic purposes, WGS and TGP results are entered into the medical record along with the accompanying genetic counseling chart notes. These documents are maintained in the participant’s permanent medical record. The remaining clinical research records including IRB documentation are retained for at least 3 years at MS and at least 25 years at EM after the clinical research study is completed, consistent with NIH and FDA policies, or longer if required by MS. Upon completion of this period, documents will be shredded and disposed of in accordance with hospital requirements.
Analysis of outcomes
Data from the feedback sessions of parents performed during the lead-in phase of the study (N = 30) to learn about GUÍA (N = 15) and to identify any issues with the surveys (N = 15) will be reviewed. Any useful feedback from these sessions will be incorporated into GUÍA and the surveys.
Descriptive statistics will be calculated for quantitative survey instruments in the baseline, ROR1, and ROR2 surveys. In the case of missing data, when survey measures contain summary scores, a mean score will be calculated based on the responses provided. We will adjust for covariates, including age, sex, and race/ethnicity where appropriate. Repeated measures of analysis of variance (ANOVAs), chi-squared test, or regression models will be fit to the data in a simple paired design (N = 550 on each arm) to assess and identify significant improvements in parental understanding, satisfaction, and feelings about the results, and their subsequent behavior in the SOC group compared to the GUÍA group. A statistical significance criterion of p < 0.05 (after adjustment for multiple testing) will be used for all analyses.
We will also perform analysis to compare the clinical utility and diagnostic yield of WGS compared to TGP by comparing the results status (positive, negative, and uncertain) via each modality. We will focus our analysis on concordance, accuracy, and reproducibility as being most important for clinical utility. We will also examine differences in diagnostic yield of pathogenic, likely pathogenic, or uncertain variants across race/ethnicity groups.
The NYCKidSeq study aims to recruit 1130 children and their families, predominantly from Harlem and the Bronx areas of NYC to a RCT. Recruitment began on January 31, 2019, and is expected to be completed by May 31, 2021. Eligible children are suspected of having an undiagnosed genetic disorder in three disease categories: neurologic, cardiovascular, and immunologic. Two clinical genetic tests will be performed for each participant, either proband, duo or trio WGS (depending on sample availability) and proband TGPs. Clinical utility, cost, and diagnostic yield of both testing modalities will be assessed and compared. This study will evaluate the use of a novel, digital platform called GUÍA to digitize and standardize the return of genomic results and improve participant understanding, designed for both English- and Spanish-speaking families. The outcomes are parental understanding of and attitudes toward receiving genomic test results for their child, and behavioral, psychological, and social impact of genomic results. We will also conduct a pilot study to assess a digital tool called GenomeDiver designed to enhance communication between medical professionals and genetic testing labs, and evaluate its ability to increase diagnostic yield when used as a means to improve communication to share phenotypic and genotypic information, compared to standard practices.
There are several limitations to the study design, which includes the lack of blinding in the RCT. Participants and study staff are aware of participants’ randomization status as participants in the intervention arm are asked about their experiences using GUÍA specifically. In addition, GUÍA is unavailable in languages other than English and Spanish. After evaluating the use of GUÍA in these languages, we hope to expand the usability of GUÍA in other languages to be more reflective of the immense linguistic diversity of NYC.
In summary, the NYCKidSeq Study is investigating the effectiveness of integrating WGS into the clinical care of diverse and medically underserved children and their families in a variety of healthcare settings and disease specialties. This work is contributing to the broader NHGRI-funded CSER consortium, now in its second funding cycle. Goals of the consortium include assessing the clinical utility of WGS, exploring medical follow-up and patient outcomes, and providing new technology to enhance communication of genomic information within health systems and communities, and evaluating patient-provider-laboratory-level interactions that influence the use of this technology. The findings from this study, and the broader CSER consortium, will inform a clearer understanding of the opportunities and barriers of providing genomic medicine in diverse populations and clinical settings and contribute evidence toward developing best practices for the delivery of clinically useful and cost-effective genomic sequencing in diverse healthcare settings.
NYCKidSeq protocol version 10, October 15, 2019. Recruitment began on January 31, 2019, and is expected to be completed by May 31, 2021.
Availability of data and materials
De-identified data for this study will be shared in secure, access-restricted scientific research databases called The Database of Genes and Phenotypes (dbGAP) at the National Institutes of Health. Interpretations of the clinical significance of variants from genetic testing will be submitted to the ClinVar database at the National Institutes of Health.
Targeted gene panel
Genomic Understanding, Information and Awareness application
Standard of care
Return of results
Icahn School of Medicine at Mount Sinai
Albert Einstein College of Medicine/Montefiore Medical Center
New York Genome Center
Abul-Husn NS, Kenny EE. Personalized medicine and the power of electronic health records. Cell. 2019;177(1):58–69.
Sanderson SC, Linderman MD, Suckiel SA, Zinberg R, Wasserstein M, Kasarskis A, et al. Psychological and behavioural impact of returning personal results from whole-genome sequencing: the HealthSeq project. Eur J Hum Genet. 2017;25(3):280–92.
Turnbull C. Introducing whole-genome sequencing into routine cancer care: the Genomics England 100 000 Genomes Project. Ann Oncol. 2018;29(4):784–7.
Farnaes L, Hildreth A, Sweeney NM, Clark MM, Chowdhury S, Nahas S, et al. Rapid whole-genome sequencing decreases infant morbidity and cost of hospitalization. NPJ Genom Med. 2018;3:10.
Popejoy AB, Fullerton SM. Genomics is failing on diversity. Nature. 2016;538(7624):161–4.
Popejoy AB, Ritter DI, Crooks K, Currey E, Fullerton SM, Hindorff LA, et al. The clinical imperative for inclusivity: race, ethnicity, and ancestry (REA) in genomics. Hum Mutat. 2018;39(11):1713–20.
West KM, Blacksher E, Burke W. Genomics, health disparities, and missed opportunities for the nation’s research agenda. JAMA. 2017;317(18):1831–2.
Petrovski S, Goldstein DB. Unequal representation of genetic variation across ancestry groups creates healthcare inequality in the application of precision medicine. Genome Biol. 2016;17(1):157.
Need AC, Goldstein DB. Next generation disparities in human genomics: concerns and remedies. Trends Genet. 2009;25(11):489–94.
Manrai AK, Funke BH, Rehm HL, Olesen MS, Maron BA, Szolovits P, et al. Genetic misdiagnoses and the potential for health disparities. N Engl J Med. 2016;375(7):655–65.
Landry LG, Rehm HL. Association of racial/ethnic categories with the ability of genetic tests to detect a cause of cardiomyopathy. JAMA Cardiol. 2018;3(4):341–5.
Caswell-Jin JL, Gupta T, Hall E, Petrovchich IM, Mills MA, Kingham KE, et al. Racial/ethnic differences in multiple-gene sequencing results for hereditary cancer risk. Genet Med. 2018;20(2):234–9.
Roberts ME, Susswein LR, Janice Cheng W, Carter NJ, Carter AC, Klein RT, et al. Ancestry-specific hereditary cancer panel yields: moving toward more personalized risk assessment. J Genet Couns. 2020; Available from: https://doi.org/10.1002/jgc4.1257.
Hindorff LA, Bonham VL, Brody LC, Ginoza MEC, Hutter CM, Manolio TA, et al. Prioritizing diversity in human genomics research. Nat Rev Genet. 2018;19(3):175–85.
Syurina EV, Brankovic I, Probst-Hensch N, Brand A. Genome-based health literacy: a new challenge for public health genomics. Public Health Genomics. 2011;14(4–5):201–10.
Sussner KM, Jandorf L, Thompson HS, Valdimarsdottir HB. Barriers and facilitators to BRCA genetic counseling among at-risk Latinas in New York City. Psychooncology. 2013;22(7):1594–604.
Hillyer GC, Schmitt KM, Reyes A, Cruz A, Lizardo M, Schwartz GK, et al. Community education to enhance the more equitable use of precision medicine in Northern Manhattan. J Genet Couns. 2020;29(2):247–58.
Halbert CH, Kessler L, Troxel AB, Stopfer JE, Domchek S. Effect of genetic counseling and testing for BRCA1 and BRCA2 mutations in African American women: a randomized trial. Public Health Genomics. 2010;13(7–8):440–8.
Sussner KM, Jandorf L, Thompson HS, Valdimarsdottir HB. Interest and beliefs about BRCA genetic counseling among at-risk Latinas in New York City. J Genet Couns. 2010;19(3):255–68.
Hurtado-de-Mendoza A, Graves KD, Gómez-Trillos S, Carrera P, Campos C, Anderson L, et al. Culturally targeted video improves psychosocial outcomes in Latina women at risk of hereditary breast and ovarian cancer. Int J Environ Res Public Health. 2019;16(23) Available from: https://doi.org/10.3390/ijerph16234793.
Southwick SV, Esch R, Gasser R, Cragun D, Redlinger-Grosse K, Marsalis S, et al. Racial and ethnic differences in genetic counseling experiences and outcomes in the United States: a systematic review. J Genet Couns. 2020;29(2):147–65.
Selkirk CG, Weissman SM, Anderson A, Hulick PJ. Physicians’ preparedness for integration of genomic and pharmacogenetic testing into practice within a major healthcare system. Genet Test Mol Biomarkers. 2013;17(3):219–25.
Rinke ML, Mikat-Stevens N, Saul R, Driscoll A, Healy J, Tarini BA. Genetic services and attitudes in primary care pediatrics. Am J Med Genet A. 2014;164A(2):449–55.
Eden C, Johnson KW, Gottesman O, Bottinger EP, Abul-Husn NS. Medical student preparedness for an era of personalized medicine: findings from one US medical school. Perinat Med. 2016;13(2):129–41.
Amendola LM, Berg JS, Horowitz CR, Angelo F, Bensen JT, Biesecker BB, et al. The clinical sequencing evidence-generating research consortium: integrating genomic sequencing in diverse and medically underserved populations. Am J Hum Genet. 2018;103(3):319–27.
Kellogg M. WordReference.com. WordReference.com; 1999.
Linguee | Dictionary for German, French, Spanish, and more [Internet]. Linguee.com. [cited 2020 Jul 28]. Available from: https://www.linguee.com.
Horowitz CR, Robinson M, Seifer S. Community-based participatory research from the margin to the mainstream: are researchers prepared? Circulation. 2009;119(19):2633–42.
Israel BA, Schulz AJ, Parker EA, Becker AB. Review of community-based research: assessing partnership approaches to improve public health. Annu Rev Public Health. 1998;19:173–202.
Kalia SS, Adelman K, Bale SJ, Chung WK, Eng C, Evans JP, et al. Recommendations for reporting of secondary findings in clinical exome and genome sequencing, 2016 update (ACMG SF v2.0): a policy statement of the American College of Medical Genetics and Genomics. Genet Med. 2017;19(2):249–55.
Suckiel SA, Odgis JA, Gallagher KM, Rodriguez JE, Watnick D, Bertier G, et al. GUÍA: a digital platform to facilitate result disclosure in genetic counseling. medRxiv. [Preprint]. Available from: http://medrxiv.org/content/early/2020/09/18/2020.09.14.20194191.abstract.
GenomeDiver [Internet]. [cited 2020 Jul 28]. Available from: https://genomediver.org.
Smedley D, Jacobsen JOB, Jäger M, Köhler S, Holtgrewe M, Schubach M, et al. Next-generation diagnostics and disease-gene discovery with the Exomiser. Nat Protoc. 2015;10(12):2004–15.
Read CY, Perry DJ, Duffy ME. Design and psychometric evaluation of the Psychological Adaptation to Genetic Information Scale. J Nurs Scholarsh. 2005;37(3):203–8.
Kohler JN, Turbitt E, Lewis KL, Wilfond BS, Jamal L, Peay HL, et al. Defining personal utility in genomics: A Delphi study. Clin Genet. 2017;92(3):290–7.
Mazor KM, Street RL Jr, Sue VM, Williams AE, Rabin BA, Arora NK. Assessing patients’ experiences with communication across the cancer care continuum. Patient Educ Couns. 2016;99(8):1343–8.
Street RL Jr, Mazor KM, Arora NK. Assessing Patient-Centered Communication in Cancer Care: Measures for Surveillance of Communication Outcomes. J Oncol Pract. 2016;12(12):1198–202.
Lobb EA, Butow PN, Moore A, Barratt A, Tucker K, Gaff C, et al. Development of a communication aid to facilitate risk communication in consultations with unaffected women from high risk breast cancer families: a pilot study. J Genet Couns. 2006;15:393–405.
Sanderson SC, Suckiel SA, Zweig M, Bottinger EP, Jabs EW, Richardson LD. Development and preliminary evaluation of an online educational video about whole-genome sequencing for research participants, patients, and the general public. Genet Med. 2016;18:501–12.
Li M, Bennette CS, Amendola LM, Ragan Hart M, Heagerty P, Comstock B, et al. The Feelings About genomiC Testing Results (FACToR) Questionnaire: Development and Preliminary Validation. J Genet Couns. 2019;28(2):477–90.
Biesecker BB, Woolford SW, Klein WMP, Brothers KB, Umstead KL, Lewis KL, et al. PUGS: A novel scale to assess perceptions of uncertainties in genome sequencing. Clin Genet. 2017;92(2):172–9.
Brehaut JC, O’Connor AM, Wood TJ, Hack TF, Siminoff L, Gordon E, et al. Validation of a decision regret scale. Med Decis Mak. 2003;23(4):281–92.
Agency for Healthcare Research and Quality (AHRQ). Medicare Expenditure Panel Survey (MEPS)-Household Component (HC), Access to Care Section (P18R5/P19R3/P20R1), Variable: Recommended Family Testing & Monitoring. US Department of Health and Human Services.
Varni JW, Seid M, Kurtin PS. PedsQLTM 4.0: Reliability and Validity of the Pediatric Quality of Life InventoryTM Version 4.0 Generic Core Scales in Healthy and Patient Populations. Med Care. 2001;39(8):800.
Wille N, Badia X, Bonsel G, Burström K, Cavrini G, Devlin N, et al. Development of the EQ-5D-Y: a child-friendly version of the EQ-5D. Qual Life Res. 2010:875–86. https://doi.org/10.1007/s11136-010-9648-y.
Hebert PL, Sisk JE, Wang JJ, Tuzzio L, Casabianca JM, Chassin MR, et al. Cost-effectiveness of nurse-led disease management for heart failure in an ethnically diverse urban community. Ann Intern Med. 2008;149:540–8.
Hoefman RJ, Van Exel NJA, Brouwer WBF. iMTA Valuation of Informal Care Questionnaire (iVICQ). Version 1.0 (December 2011). Rotterdam: iBMG / iMTA; 2011. retrieved from www.bmg.eur.nl/english/imta/publications/manuals_questionnaires/ on 01/12/2021
Haun J, Noland Dodd VJ, Graham-Pole J, Rienzo B, Donaldson P. Testing a Health Literacy Screening Tool: Implications for Utilization of a BRIEF Health Literacy Indicator. Fed Pract. 2009;26(12):24–31.
McNaughton CD, Cavanaugh KL, Kripalani S, Rothman RL, Wallston KA. Validation of a Short, 3-Item Version of the Subjective Numeracy Scale. Med Decis Mak. 2015:35, 932–936. https://doi.org/10.1177/0272989x15581800.
Horowitz CR, Abul-Husn NS, Ellis S, Ramos MA, Negron R, Suprun M, et al. Determining the effects and challenges of incorporating genetic testing into primary care management of hypertensive patients with African ancestry. Contemp Clin Trials. 2016 Mar;47:101–8.
Shea JA, Micco E, Dean LT, McMurphy S, Sanford Schwartz J, Armstrong K. Development of a Revised Health Care System Distrust Scale. J Gen Intern Med. 2008;23:727–32. https://doi.org/10.1007/s11606-008-0575-3.
United States Department of Health and Human Services. Centers for Disease Control and Prevention. National Center for Health Statistics. National Health and Nutrition Examination Survey (NHANES), 1999-2000. ICPSR Data Holdings. 2009. https://doi.org/10.3886/icpsr25501.v3.
The GenIUSS Group. Best Practices for Asking Questions to Identify Transgender and Other Gender Minority Respondents on Population- Based Surveys. Los Angeles, CA: The Williams Institute; 2014.
Jones NA. Update on the U.S. Census Bureau’s Race and Ethic Research for the 2020 Census. Survey News. 2015;3(5) https://www.census.gov/content/dam/Census/newsroom/press-kits/2014/article_race_ethnic_research_2020census_jones.pdf.
Matthews K, Phelan J, Jones NA, Konya S, Marks R, Pratt BM, et al. 2015 national content test race and ethnicity analysis report: A new design for the 21st century. DC: US Census Bureau Washington; 2017.
Goddard KAB, Angelo FAN, Ackerman SL, Berg JS, Biesecker BB, Danila MI, et al. Lessons learned about harmonizing survey measures for the CSER consortium. Journal of Clinical and Translational Science. Cambridge University Press. 2020;4:537–46.
The authors would like to thank the following individuals: Hernis de la Cruz, Mimsie Robinson, Kojo Davis, Trinisha Williams, Irma Laguerre, Melvin Gertner, Lynne Richardson, Reymundo Lozano, Wilson Heredia Nunez, David Kaufman, Jessica Oh, Julie Flom, Jao Pedro Matias Lopes, Nicole Ramsey, Maite La Vega-Talbott, Soledad Puente-Guzman, Jules Beal, Paul Levy, Sheri Escalante, Puja Patel, Robert Marion, Zachary Hena, Bradley Clark, Daphne Hsu, Neha Bhatia, Christine Walsh, Karen Ballaban-Gil, Kimberly Reidy, Daniel Lax, Manoj Gupta, Mohamad Saifeddine, Rana Jehle, Tatyana Gavrilova, Gregory Gutin, Nagma Dalvi, Elisa Muniz, Koshi Cherian, Mana Mann, Sarah Van Dine, Maris Rosenberg, Maria Valicenti-Mcdermott, Rosa Seijo, Lisa Shulman, Elisa Muniz, Meghan Keenan Breheney, Michael Satzer, Jennifer Yoffe, Dona Rani Kathirithamby, Sindhu Gupta, Gina Cassel, Parmpreet Kaur, Rachel Eisenberg, Margo Breilyn, Emily Jackness, Arjola Cosper, Joanna Grater, Gina Cassel, Emily Jackness, Isaac Molinero, Jenny Shliozberg Richard Sidlow, Larry Bernstein, Zoya Treyster, Leslie Delfiner, Susan Duberstein, and Farah Dosani. We would also like to thank the parents and children participating in this study.
Principal investigators will review topics suggested for presentation or publication. Lead author(s) will be determined by principal investigators. Any disputes regarding authorship will be resolved by the principal investigators.
Any significant changes to the protocol outlined above must be written in a formal amendment. The amendment must be approved by both IRBs at MS and EM.
Research reported in this publication was supported by the National Human Genome Research Institute and National Institute for Minority Heath and Health Disparities of the National Institutes of Health under Award Number 1U01HG0096108. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Ethics approval and consent to participate
The Institutional Review Boards of Icahn School of Medicine at Mount Sinai and Albert Einstein College of Medicine approved this study protocol. Written, informed consent to participate will be obtained from all participants.
Consent for publication
A model consent form will be made available upon request.
EEK has received speaker honorariums from Regeneron and Illumina. NSAH was previously employed by Regeneron Pharmaceuticals and has received a speaker honorarium from Genentech. AB has received consulting fees from the American Board of Family Medicine outside the submitted work. JCB has received an honorarium from LivaNova. AR has received funding by TAKEDA for two clinical trials: HYQVIA in Pediatric CVID Patients (2018-2021) and SCIG Treatment for Chronic Inflammatory Demyelinating Polyneuropathy. RZ has received consulting fees from Sema4 outside the submitted work. All other authors declare they have no competing interest.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The original online version of this article was revised: Following publication of the original article, we were notified that the originally published Table 2 was incorrect. References 1-33 were not impacted by the table update and remain unchanged, but are updated from reference 34 onward.
About this article
Cite this article
Odgis, J.A., Gallagher, K.M., Suckiel, S.A. et al. The NYCKidSeq project: study protocol for a randomized controlled trial incorporating genomics into the clinical care of diverse New York City children. Trials 22, 56 (2021). https://doi.org/10.1186/s13063-020-04953-4