Skip to main content

Advancing collaborations in health research and clinical trials in Sub-Saharan Africa: development and implementation of a biostatistical collaboration module in the Masters in Biostatistics Program at Stellenbosch University

Abstract

Background

Sub-Saharan Africa continues to carry a high burden of communicable diseases such as TB and HIV and non-communicable diseases such as hypertension and other cardiovascular conditions. Although investment in research has led to advances in improvements in outcomes, a lot still remains to be done to build research capacity in health. Like many other regions in the world, Sub-Saharan Africa suffers from a critical shortage of biostatisticians and clinical trial methodologists.

Methods

Funded through a Fogarty Global Health Training Program grant, the Faculty of Medicine and Health Sciences at Stellenbosch University in South Africa established a new Masters Program in Biostatistics which was launched in January 2017. In this paper, we describe the development of a biostatistical and clinical trials collaboration Module, adapted from a similar course offered in the Health Research Methodology program at McMaster University.

Discussion

Guided by three core principles (experiential learning; multi-/inter-disciplinary approach; and formal mentorship), the Module aims to advance biostatistical collaboration skills of the trainees by facilitating learning in how to systematically apply fundamental statistical and trial methodological knowledge in practice while strengthening some soft skills which are necessary for effective collaborations with other healthcare researchers to solve health problems. We also share some preliminary findings from the first four cohorts that took the Module in January–November 2018 to 2021. We expect that this Module can provide an example of how to improve biostatistical and clinical trial collaborations and accelerate research capacity building in low-resource settings.

Funding source

Fogarty International Center of the National Institutes of Health.

Peer Review reports

Introduction

It has been recognized as far back as 1957 that, while recent graduates who specialize in biostatistics are usually adequately equipped to do biostatistical research, they are in fact inadequately prepared to practice as biostatisticians or clinical trial methodologists [1]. Biostatisticians or methodologists working within an academic environment are generally involved in applied research by collaborating with multidisciplinary teams including epidemiologists, healthcare professionals, students from a variety of health sciences programs, and health economists, among others. The challenges in working with such diverse teams consist of dealing with various human elements, addressing a wide range of research problems or disciplines, and working with researchers with minimal statistical background [2]. Additionally, biostatisticians or methodologists themselves might come from a diversity of educational and cultural backgrounds [3]. Moreover, while it has been recognized that both short-term (i.e., consultation) and long-term (i.e., collaboration) statistical and methodological activities require sophisticated non-statistical skills [4], unfortunately, the skills needed are not usually taught in biostatistics or clinical trials postgraduate programs [5, 6]. All these aspects can impact the efficiency and effectiveness of communication between biostatisticians or clinical trial methodologists and other clinical researchers.

Sub-Saharan Africa continues to carry a high burden of communicable diseases such as tuberculosis and HIV, and non-communicable diseases such as hypertension and other cardiovascular conditions, as well as injuries [7, 8]. Although investment in conducting relevant research, and in increasing the uptake of research into healthcare policy and practice [9], has led to advances and improvements in health outcomes, a lot still remains to be done to build research and knowledge translation capacity in health [10]. Like many other regions in the world, in Sub-Saharan Africa, biostatisticians and clinical trial methodologists remain a scarce resource, which leads to an over-reliance on input from biostatisticians sourced from high-income countries or the pharmaceutical industry for writing research grants, performing advanced data analyses, publishing in high-profile journals, teaching biostatistics courses, and training postgraduate students [11]. To mitigate this issue, the Masters in Biostatistics Program (subsequently called “the Program”) at Stellenbosch University was started in 2017 after a meeting of Sub Saharan African Biostatisticians in 2014 that identified biostatistical training as a specific need in the region [12]. At the meeting, the demand to improve the trainees’ non-statistical (often called soft) skills was also highlighted, where these skills include effective collaboration and consulting skills, life-long learning, efficient communications, and leadership training, among others [12]. To enhance the capacity building of both statistical and soft skills, Stellenbosch University introduced a 12-credit Biostatistical Consulting and Collaboration Module (subsequently called “the Module”) as part of the Program running in the second year from January to November. It was offered in 2018 in collaboration with a senior biostatistician who had vast experience in developing and teaching similar courses at McMaster University [5] and an in-house module facilitator with 15-year experience in biostatistical consulting and collaborating in Sub-Saharan African university environments. In this article, we briefly introduce the outline of the Module and show some preliminary findings from the Module starting in 2018. It is expected that the Module can help improve the biostatistical collaboration and accelerate research capacity building for students in Sub-Saharan Africa.

Outline of the Module—principles, skills, lectures and practice, linkage with internship, and assessment

Figure 1 shows the overview of the Module. The Module is guided by three principles and covers both the fundamental technical and soft skills for the biostatistical and methodological collaborations. All the three parts are interactively implemented during the Module activities.

Fig. 1
figure 1

Core principles and fundamental soft and technical skills for the Module

Guiding core principles

We adopted the existing successful module at McMaster University in the Program [5]. The Module includes three guiding core principles: (1) embracing experiential learning, (2) using a multidisciplinary approach in collaborations, and (3) identifying a formal mentor (including “mentorship-beyond-borders”) (Fig. 1). Students trained with the Module at McMaster University generally proactively get involved in various research groups that engage local, national, and global communities in identifying problems and finding solutions. Such research environments provide our students experiential learning and shared opportunities that help them locate research needs and improve their collaborations. A second principle using a multidisciplinary approach is recommended because both efforts to generate primary clinical research evidence and developments of clinical guidelines leading to improvements in health require multidisciplinary backgrounds and clinical research network [13]. Embracing a multidisciplinary environment can advance biostatistical collaborations to yield high-quality outputs in an effective fashion. Another principle is related to mentorship. Evidence has shown that a strong mentorship can help achieve great career satisfaction, accelerate research productivity, expand research networks, offer more opportunities, advance academic promotion, and enhance the acquisition of important career skills, to mention a few [14,15,16,17,18]. Therefore, our students trained in the Module are always encouraged to identify an experienced mentor as an important step in their career.

Technical and soft skills for collaboration

Figure 1 also displays the technical and soft skills needed for effective biostatistical collaborations in health research. Being able to apply statistical knowledge in the design, conduct, analysis, reporting, and translation of studies is the fundamental requirement for a qualified biostatistician in collaboration with non-statisticians [3, 19]. In the Module, we seek to cover some most common and relevant aspects in real-life collaborations, including pragmatic trials (design), data safety monitoring (conduct), statistical analysis plan (analysis), transparent reporting (reporting), and meta-analyses (translation), among others. The soft skills, on the other hand, play a key role in successful collaborations with researchers from different culture and backgrounds to solve complex problems [19, 20]. We have to emphasize that such soft skills are usually not taught in regular statistics or biostatistics graduate programs, and therefore many graduates have to learn these skills through trial-and-error when they start working [21, 22]. Facilitating learning of our students in soft skills in biostatistical and methodological collaborations is an important element that makes the Module unique and intriguing.

Lecture topics and practice

Guided by the three principles, the Module, with a total of 120 h learning time, was designed to comprise of 1/3 face-to-face lectures (40 h) and 2/3 practical work (80 h) implemented throughout the module. Table 1 shows the schedules of the learning time assigned for the Module.

Table 1 Schedules of the learning time for the Module

The objectives of lectures and practice include the following: (1) integrating statistical knowledge gained in the other modules and applying it during biostatistical and methodological consulting sessions with clinical researchers and students, (2) communicating effectively with researchers to gather information required to make a link between research questions and statistical methods by asking relevant questions, (3) providing guidance in study design, sample size calculations and statistical analysis plans for researchers, (4) effectively analyzing data from consultations on research projects within a variety of biomedical fields, (5) discussing results in a clear manner orally and in the form of a written report to researchers, (6) doing research on the relevant statistical methodologies and presenting findings to the audience, and (7) displaying teamwork and leadership in research collaborations. Details on the topics and practice including their contents and key references for the Module are shown in Table 2. For example, the second topic is about the communication skills between a biostatistician and researchers. The topic aims to train students how to raise questions thoughtfully, understand the do’s and don’ts during communications, appreciate the culture and backgrounds of collaborators, make others aware of your statistical and social culture, and present statistical methodology and plans to the non-statistical audiences. Readers can refer to reports from Ehrenberg [23], Zahn and Isenberg [4], and others for further details regarding communication between biostatisticians and researchers.

Table 2 Lecture topics and practice, contents, and references for the Module

Linkage with internship

As part of the Program, students participate in an internship program during the last three months (from July to September) of their second year of study. Although the internship is not included in the Module’s 120-h learning time, we utilize and link the internship as a supplemental platform of real-life practice for students in this Module. Moreover, we use the internship to help with summative assessment of students’ performances (details below).

The internship placements may occur within research institutions or centers where biostatisticians are employed, and the students will usually organize their own placements according to their specific area of interest. The placement organization provides the student with opportunity to engage with researchers, conduct statistical consultation, and gain experience in communication and teamwork under the supervision of an experienced biostatistician. The learning outcomes are to apply theoretical concepts to day-to-day problems in a workplace environment, provide guidance in study design and statistical analysis plans, work in a collaborative environment, and successfully propose solutions to statistical problems arising from workplace environment. Thus, linkage with the internship is a significant way for students to put into practice what they have learned in the Module.

Assessment of students’ performances

The assessment of students’ performances for the Module is 50% formative and 50% summative. The key points for assessments consist of students’ ability to translate research questions into statistical questions, assistance with study design, analyses, interpretation and reporting of results, effective communications with non-statisticians, self-learning, teamwork, and leadership. The formative part is composed of a presentation (30% of the overall mark) and two written assignments (20% of the overall mark). For the presentation, students are expected to identify a biostatistical topic of interest and relevance to biomedical researchers and then present for 30 minutes to the audience from the Division of Epidemiology and Biostatistics as well as other interested researchers. Marks are awarded for choice of topic, clarity of presentation, knowledge of the subject, and how questions are handled. The written assignments are designed to cover issues that may or may not have been dealt with in the lectures but that are commonly encountered in real-life biostatistical collaboration. The rationale for the assignment settings is to expose students to the actual scenarios that biostatisticians encounter, encourage them to improve self-learning and critical thinking, and enhance their problem-solving ability. For example, one of the assignments concerns randomization in a large multi-center trial, where the treatment groups were not balanced due to chance after the trial ended. Students are asked to provide their suggested solutions, rationale for the solutions, expected analyses and reporting, and concluding remarks appropriate for future similar issues. In general, there are no explicitly correct or incorrect answers to these open questions; therefore, the assignments are marked as an essay with a rubric awarding marks to how well the student has given evidence to support their answer.

The summative part, assessed after the internship, is entirely composed of a portfolio of evidence of learning (50% of the overall mark). Since this Module is primarily applied, it is difficult to summatively evaluate the skills they have learnt using traditional examination format. A portfolio of evidence is a more valid way to help students to better incorporate the knowledge gained through lectures, readings, assignments, and during the real-life practice of their internship. Students document their learning process in biostatistical collaborations through their workplace placements using reflections, case studies reports, written reports to clients, and videos of consultations. The portfolio is assessed through a rubric that examines the depth of their reflection into the learning process as well as the overall quality of the portfolio.

Findings from the Module from 2018 to 2021

Evaluating the Module

For the first intake in 2017, we received eight applications and five met the requirements and enrolled. These five students, three from Zimbabwe and two from South Africa, had backgrounds ranging from statistics to health science. They went on to take the Module in their second year of study from January to November 2018. All students were assigned a mentor during the Module who then became their research assignment supervisor. Subsequently, the interest in the program has grown and we receive more than forty applications annually. A further five students were enrolled in 2018 who went on to take the Module in 2019, of which four were Zimbabwean and one was Namibian. The 2019 intake, which took the Module in 2020, consisted of eight students, two of whom were South Africans, and the remainder from Zimbabwe, Namibia, Kenya, and Ethiopia. Thus far, the program has produced sixteen graduates. The 2020 cohort of students who took the Module in 2021 consisted of seven students of whom three were Ethiopian, two South African, one Kenyan, and one Zimbabwean.

Immediately after each teaching block, the students were asked to rate each of the seven following domains of the Module on a scale of 1 (poor) to 7 (excellent): the Module overall so far; the clarity of the Module objectives; the organization of the Module; the relevance to career, educational goals, and interests; the usefulness of the class discussions; the usefulness of the course materials; and how students’ needs and expectations were met.

Block 1 evaluations for 2018 to 2021 cohorts are shown in Table 3. In 2018, the means of the items ranged from 5.6 to 6.2, indicating students’ satisfaction with the Module. The means of the items increased in 2019, 2020, and 2021, as reflected in Table 3.

Table 3 Student evaluation scores for the Module from 2018 to 2021

Additionally, students were given three open questions on the evaluation form: (1) What are the best features of the Module; (2) Did you feel that any aspects of the Module were not relevant; and (3) Which aspects of the Module would you change? The 2020 cohort of students were given an extra question on the online format of the course due to the COVID-19 pandemic and subsequent lockdown. In each case they were asked to elaborate on their answers. The responses regarding the best features of the Module were mainly around the lecturer’s passion, vast experience, and in-depth understanding of the subject. The real-life practical aspects were also appreciated. From 2019 onward, the responses to this question were mainly around the usefulness of the collaboration framework and the enjoyment of the role-play sessions. They also indicated a renewed appreciation for collaboration as a career and a taste of what to expect in the real world. In terms of the second question, all but one student felt that all aspects were relevant. The one student who felt differently highlighted that the Module placed too much emphasis on “soft skills” such as time management and stress management, at the expense of more important statistical topics. Suggestions for what to change included the addition of more information on sample size calculations and statistical analysis plans and research protocol development. We have considered all their feedback and are now trying to reach a consensus about how to incorporate appropriate changes into the future Module. Suggestions of what to change in future included having conversations with biostatisticians who work collaboratively in different fields or being able to observe real or simulated collaboration sessions. Responses from the 2020 cohort on the new online format of the offering were generally in favor of the format, with students feeling that not much was lost without face-to-face contact; however, one student acknowledged the challenges of internet connectivity and electricity interruptions which happened intermittently in South Africa.

One of the improvements which were done to the Module since 2018 was to add a structured framework for managing and running statistical collaborations [24]. Further improvements under consideration for future years are methodological research within clinical trials (SWAT) [25], methodological issues around systematic reviews and meta-analysis, and economic analyses in trials [26].

Evaluating students’ performances

While all students progressed well in the formative assessments, the summative portfolios displayed only surface reflections which were almost “report like.” We realize that proper reflection is a skill that has not been taught to the students and this will be incorporated into the future Module. On the positive side, the students tended to be brutally honest about the course and provided very useful feedback about what worked this year and which aspects needed improvement. Moreover, their documentation of the challenges they faced in the Module (reading, lectures, assignments, presentation, and linkage with internship) was also helpful in knowing where the facilitators can be of more assistance in the future. For example one student wrote “The department should have field visits to assess the progress of the students and know challenges faced by students. Also, students should meet on a monthly basis to present their progress on projects and other reports or problems which they face while on field attachment.” Another wrote “More so, I was not spared from problem consultations where some postgraduate students wanted the statistician to do everything for them. Some supervisor who had little understanding of statistics and epidemiological study design would cause confusion between the consulting activities. However, through communication of the problems encountered to the head and seeking advice from other professionals around, I was able to solve and move on without further friction on how to conduct biostatistical collaborations.”

Concluding remarks

Realizing the urgent need of statistical or methodological expertise and soft skills in biostatistical and clinical trial collaborations, we developed and implemented a module with the purposes of enhancing students’ capacity building at Stellenbosch University in the Program. The Module tried to advance our students’ both technical and non-technical statistical and methodological competencies, using an interactive approach combining in-class instruction, critical thinking and self-learning, and real-life practice. It not only equipped students for engagement to conduct research but also for engagement and communication to advance the use of research in healthcare decision making. Further improvement of the Module is needed in the future setting. It is our expectation that the Module can provide an example of how to improve the biostatistical and clinical trial collaborations and accelerate research capacity building in low-resource settings.

Availability of data and materials

No data available.

References

  1. Kimball A. Errors of the third kind in statistical consulting. J Am Statist Assoc. 1957;52(278):133–42. https://doi.org/10.1080/01621459.1957.10501374.

    Article  Google Scholar 

  2. Kirk RE. Statistical consulting in a university: dealing with people and other challenges. Am Stat. 1991;45(1):28–34.

    Google Scholar 

  3. Tobi H, Joop Kuik D, Dick Bezemer P, Ket P. Towards a curriculum for the consultant biostatistician: identification of central disciplines. Stat Med. 2001;20(24):3921–9. https://doi.org/10.1002/sim.1082.

    Article  CAS  PubMed  Google Scholar 

  4. Zahn DA, Isenberg DJ. Nonstatistical aspects of statistical consulting. Am Stat. 1983;37(4):297–302.

    Article  Google Scholar 

  5. Thabane L, Walter SD, Hanna S, Goldsmith CH, Pullenayegum E. Developing a biostatistical collaboration course in a health science research methodology program. J Stat Educ. 2008;16(2). https://www.tandfonline.com/doi/full/10.1080/10691898.2008.11889564.

  6. Kenett R, Thyregod P. Aspects of statistical consulting not taught by academia. Stat Neerl. 2006;60(3):396–411. https://doi.org/10.1111/j.1467-9574.2006.00327.x.

    Article  Google Scholar 

  7. Peer N. The converging burdens of infectious and non-communicable diseases in rural-to-urban migrant Sub-Saharan African populations: a focus on HIV/AIDS, tuberculosis and cardio-metabolic diseases. Trop Dis Travel Med Vaccines. 2015;1:6.

    Article  Google Scholar 

  8. Nachega JB, Uthman OA, Ho YS, Lo M, Anude C, Kayembe P, et al. Current status and future prospects of epidemiology and public health training and research in the WHO African region. Int J Epidemiol. 2012;41(6):1829–46. https://doi.org/10.1093/ije/dys189.

    Article  PubMed  PubMed Central  Google Scholar 

  9. Young T, Garner P, Clarke M, Volmink J. Evidence-based health care and policy in Africa: past, present and future. J Clin Epidemiol. 2016;S0895-4356(16):30182. https://doi.org/10.1016/j.jclinepi.2016.06.006.

    Article  Google Scholar 

  10. Laabes EP, Desai R, Zawedde SM, Glew RH. How much longer will Africa have to depend on western nations for support of its capacity-building efforts for biomedical research? Trop Med Int Health. 2011;16(3):258–62. https://doi.org/10.1111/j.1365-3156.2010.02709.x.

    Article  PubMed  Google Scholar 

  11. Gezmu M, DeGruttola V, Dixon D, Essex M, Halloran E, Hogan J, et al. Strengthening biostatistics resources in sub-Saharan Africa: research collaborations through US partnerships. Stat Med. 2011;30(7):695–708. https://doi.org/10.1002/sim.4144.

    Article  PubMed  PubMed Central  Google Scholar 

  12. Machekano R, Young T, Rusakaniko S, Musonda P, Sartorius B, Todd J, et al. The Africa Center for Biostatistical Excellence: a proposal for enhancing biostatistics capacity for sub-Saharan Africa. Stat Med. 2015;34(27):3481–9. https://doi.org/10.1002/sim.6572.

    Article  PubMed  PubMed Central  Google Scholar 

  13. Li G, Thabane L, Papaioannou A, Ioannidis G, Levine MA, Adachi JD. An overview of osteoporosis and frailty in the elderly. BMC Musculoskelet Disord. 2017;18(1):46. https://doi.org/10.1186/s12891-017-1403-x.

    Article  PubMed  PubMed Central  Google Scholar 

  14. Sambunjak D, Straus SE, Marušić A. Mentoring in academic medicine: a systematic review. Jama. 2006;296(9):1103–15. https://doi.org/10.1001/jama.296.9.1103.

    Article  CAS  PubMed  Google Scholar 

  15. Kumagai AK, Jackson B, Razack S. Cutting close to the bone: student trauma, free speech, and institutional responsibility in medical education. Acad Med. 2017;92(3):318–23. https://doi.org/10.1097/ACM.0000000000001425.

    Article  PubMed  Google Scholar 

  16. Wise MR, Shapiro H, Bodley J, Pittini R, McKay D, Willan A, et al. Factors affecting academic promotion in obstetrics and gynaecology in Canada. J Obstet Gynaecol Can. 2004;26(2):127–36. https://doi.org/10.1016/S1701-2163(16)30488-1.

    Article  PubMed  Google Scholar 

  17. Straus SE, Sackett DL. Clinician-trialist rounds: 8. Mentoring - part 2: the structure and function of effective mentoring linkage, resources, and academic opportunities. (1740-7753 (Electronic)).

  18. Thabane L, Thabane M, Goldsmith CH. Mentoring young statisticians: facilitating the acquisition of important career skills. Afr Stat J. 2007;4:123–36.

    Google Scholar 

  19. Earnest A. Essentials of a successful biostatistical collaboration: Chapman and Hall/CRC; 2016. https://doi.org/10.1201/9781315373447.

    Book  Google Scholar 

  20. Begg MD, Vaughan RD. Are biostatistics students prepared to succeed in the era of interdisciplinary science? (And how will we know?). Am Stat. 2011;65(2):71–9. https://doi.org/10.1198/tast.2011.10222.

    Article  Google Scholar 

  21. DeMets DL, Stormo G, Boehnke M, Louis TA, Taylor J, Dixon D. Training of the next generation of biostatisticians: a call to action in the US. Stat Med. 2006;25(20):3415–29. https://doi.org/10.1002/sim.2668.

    Article  PubMed  Google Scholar 

  22. Zelen M. The training of biostatistical scientists. Statistics in medicine. 2003;22(21):3427–30. https://doi.org/10.1002/sim.1642.

    Article  CAS  PubMed  Google Scholar 

  23. Ehrenberg A. Writing technical papers or reports. Am Stat. 1982;36(4):326–9.

    Google Scholar 

  24. Vance EA, Smith HS. The ASCCR frame for learning essential collaboration skills. J Stat Educ. 2019;27:265–74.

    Article  Google Scholar 

  25. Treweek S, Bevan S, Bower P, Campbell M, Christie J, Clarke M, et al. Trial Forge Guidance 1: what is a Study Within A Trial (SWAT)? Trials. 2018;19(1):139. https://doi.org/10.1186/s13063-018-2535-5.

    Article  PubMed  PubMed Central  Google Scholar 

  26. Clarke L. An introduction to economic studies, health emergencies, and COVID-19. J Evid Based Med. 2020;13(2):161–70. https://doi.org/10.1111/jebm.12395.

    Article  PubMed  PubMed Central  Google Scholar 

Download references

Acknowledgements

We thank all the staff and students who participated in this study.

Funding

Work reported in this publication was supported by the Fogarty International Center of the National Institutes of Health under Award Number D43 TW010547. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.

Author information

Authors and Affiliations

Authors

Contributions

TE, GL, and LT: conceived the idea and designed the study; TE, GL, TY, JZ, RM, and LT: collected, analyzed and interpreted data; TE and GL: generated tables and figures; TE and GL: searched the literature and wrote the manuscript; TY, JZ, RM, and LT: provided comments to the manuscript; TE and GL: incorporated comments into the revision. TE and GL contributed equally. All authors read and approved the final version of the manuscript.

Corresponding author

Correspondence to Lehana Thabane.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

We declare no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Esterhuizen, T.M., Li, G., Young, T. et al. Advancing collaborations in health research and clinical trials in Sub-Saharan Africa: development and implementation of a biostatistical collaboration module in the Masters in Biostatistics Program at Stellenbosch University. Trials 22, 478 (2021). https://doi.org/10.1186/s13063-021-05427-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13063-021-05427-x

Keywords