- Study protocol
- Open Access
- Open Peer Review
A core outcome set for clinical trials of Chinese medicine for hyperlipidemia: a study protocol for a systematic review and a Delphi survey
Trialsvolume 20, Article number: 14 (2019)
Hyperlipidemia, defined as elevated lipid levels, is the primary and major risk factor for atherosclerotic cardiovascular disease. Several studies have evaluated the effects of Chinese medicine (CM) on hyperlipidemia. However, due to the varied designs and methods of these studies, data synthesis has been difficult, restricting the practical value of the findings. Developing a core outcome set (COS) could solve these methodological concerns. In this paper, we report a protocol to develop a COS for CM clinical trials for hyperlipidemia (COS-CM-Hyperlipidemia).
The development of COS-CM-Hyperlipidemia will include four stages: (1) a systematic review to identify potential important outcomes—a study advisory group, composed of core stakeholders of hyperlipidemia, will be set up afterwards to evaluate the identified outcomes and a candidate outcome set will be developed accordingly; (2) a panel of experts will be invited to conduct a three-round Delphi survey, so that the experts’ opinions on the importance of outcomes for treating hyperlipidemia with CM will be collected; (3) a consensus meeting with clinicians, patients, and other key stakeholders will be conducted to finalize the items and definitions; and (4) COS-CM-Hyperlipidemia will be promoted and updated.
The development of this COS will improve the design and operation of CM trials on hyperlipidemia, keeping them in compliance with international standards, as well as the comparability and utility of their results.
The Core Outcome Measures in Effectiveness Trials Initiative (COMET): http://www.comet-initiative.org/studies/details/983. Registered on 25 April 2017.
Hyperlipidemia, or elevated lipid levels, presents the symptoms of elevated total cholesterol (TC), low-density lipoprotein cholesterol (LDL-C), triglyceride (TG) levels, and mixed TC and TG . It is a primary and major risk factor for atherosclerotic cardiovascular diseases, as well as a prerequisite for them [2,3,4,5]. In the United States, > 100 million adults have reported elevated TC and hyperlipidemia is the 11th leading cause for direct health expenditures ($37 billion annually) . In China, the prevalence of hyperlipidemia is 34.0% and there is a significantly higher prevalence in men than women (41.9% and 32.5%, respectively) .
LDL-C has been recommended as the primary target of lipid-lowering interventions to reduce the incidence of atherosclerotic cardiovascular disease by the mainstream clinical guidelines [3, 4, 8]. In addition to conventional pharmaceutical therapies such as atorvastatin, Chinese medicine (CM) has abundant resources for lipid-lowering interventions. In CM theory, pattern (also called syndrome) is a diagnostic conclusion based on pathological changes in a disease at a certain stage. It includes features such as the nature of disease, cause, location, and development trends . CM physicians treat patients according to syndrome differentiation, so it is important in clinical trials to measure syndrome change so as to embody the characteristics of CM .
Many studies have reported CM treatments that are effective against hyperlipidemia. For example, a meta-analysis indicated that the effect of Xuefuzhuyu decoction on hyperlipidemia was better than that in the control group . Another systematic review showed that Yinchenwuling powder was more effective at decreasing the levels of TC and TG, while increasing the level of high-density lipoprotein cholesterol without serious adverse effects . A Cochrane systematic review also found that Xuezhikang was the most commonly used herbal formula investigated for hypercholesterolemia; a significant effect on decreasing TC was shown in favor of Xuezhikang when compared with inositol nicotinate .
However, these promising findings need to be interpreted with caution due to methodological problems in CM trials [11,12,13,14]. Outcome measures are one of the key factors . For instance, a systematic review of 35 trials studying the efficacy of a Chinese patented drug on hyperlipidemia reported as many as 30 different outcomes, but none focused on atherosclerotic cardiovascular disease events . The main issues concerning hyperlipidemia outcome measures are: (1) the outcome measures vary from one trial to the next and this heterogeneity jeopardizes data consistency, resulting in reporting bias and patient irrelevance ; (2) there is no agreed upon or standardized outcome; (3) surrogate endpoints, such as biochemical indicators, have been widely adopted [11,12,13,14,15,16]; and (4) no standardized outcome sets to measure syndrome change for hyperlipidemia. Due to these problems, the practical value of clinical trials has been limited.
A core outcome set (COS) refers to a minimal set of important outcome measures identified by key stakeholders that should be collected and reported in all clinical trials of a specific health area, as a standard criterion [17, 18]. The Core Outcome Measures in Effectiveness Trials (COMET) Initiative brings together people interested in the development and application of COS. It collects and reports core outcomes that will promote the comparison and synthesis of separate trial results where appropriate . Several studies on the development of COS for CM have already begun [20,21,22], but none focus on a COS for hyperlipidemia. This study presents a protocol for the development and promotion of a COS for hyperlipidemia in clinical trials. It evaluates the efficacy of CM in the treatment of hyperlipidemia compared with conventional medications. Therefore, a COS for CM clinical trials for hyperlipidemia is urgently need.
This study aims to present a protocol of the development of a COS for CM clinical trials for hyperlipidemia (COS-CM-Hyperlipidemia).
This study has been registered on the COMET website (number 983) . The development of this COS-CM-Hyperlipidemia will include four stages: (1) a systematic review will be adopted to identify potential important outcomes—meanwhile, a study advisory group (SAG), composed of core hyperlipidemia stakeholders, will be set up to evaluate the identified outcomes and a candidate outcome set will be developed later; (2) a panel of experts will be selected and a three-round Delphi survey will be conducted to assess their opinions on the importance of hyperlipidemia outcomes in CM; (3) a consensus meeting, composed of clinicians, patients, and other key stakeholders, will be conducted to finalize the items and definitions; and (4) promotion and updates. Figure 1 illustrates the design of our study.
Stage 1: Systematic review
A systematic review will be conducted to screen the range of potentially important outcomes according to the recommendations of the COMET Initiative . Previous COS studies on CM have also used systematic reviews to initiate research [20,21,22]. All clinical trials (regardless of study type) that report hyperlipidemia outcome(s) will be included in this study.
Types of studies, interventions, and participants
Both interventional and observational studies (e.g. case reports, case-control studies, cohort studies, randomized controlled trials, and systematic reviews) published in either English or Chinese, will be included in our systematic review. Patients with hyperlipidemia, including hypercholesterolemia, hypertriglyceridemia, mixed hyperlipidemias, and lowered high-density lipoprotein cholesterol, treated by CM alone or combined with conventional medicine, will be considered for addition into our systematic review. Only those in which the hyperlipidemia diagnosis and the effectiveness assessment standard were clear will be eligible for inclusion. All patients, regardless of sex, age, or race, will be included. Patients who suffer from coronary heart disease, hypertension, diabetes, hypothyroidism, or liver disease will be excluded. If a study is mainly on secondary hyperlipidemia, as opposed to other serious diseases, the full text will be read to decide whether to include it.
Data will be searched in CENTRAL, PubMed, Embase, Wangfang database, China National Knowledge Infrastructure, and Chinese BioMedical Database using a comprehensive, electronic search strategy. Two independent reviewers will assess the titles and abstracts to identify whether studies should be included. Any disagreements will be resolved through discussion after a thorough reading of the paper or by consulting a third researcher. Any duplicate studies will be excluded. Studies will also be excluded if they do not describe hyperlipidemia outcomes.
Data extraction and analysis
Two reviewers will independently extract the data by reading the full texts. For each included study, the following data will be extracted if available: author(s); year of publication; country; patient source; study type; treatment duration; length of follow-up; number of patients included; number of patients who withdrew; patients’ age; type of hyperlipidemia; CM syndrome types; interventions details; outcomes and their definitions; time-point; and method of outcome measurement. If any data are incomplete, the reviewers will contact the studies’ authors by email or telephone to get the missing data. Before the data are extracted, consistency evaluation will be conducted between the two reviewers to ensure all analytical details are reliable. Discussion and consulting with a third researcher will be conducted to resolve any disagreements. The data will then be entered into a form and the outcomes will be classified into different domains (patient-reported, clinician-reported, and proxy-reported outcomes) by one of our researchers. Classification will be confirmed by another researcher. The number of definitions of outcomes, the number of outcomes of each domain, and the number of measurement methods will be calculated and presented.
Study advisory group
A SAG composed of at least two hyperlipidemia patients and two clinicians, methodologists, researchers, ethicists, and statisticians will be created. The patients will be selected out of outpatients at the Guangdong Provincial Hospital of Chinese Medicine (GPHCM); clinicians will be selected from endocrinologists and cardiovascular experts of CM/integrated Chinese and Western medicine from both China and abroad; methodologists, researchers, and statisticians will be chosen from GPHCM and ethicists from GPHCM’s Ethics Committee.
Development of COS candidate items
The identified outcomes will be presented on a checklist which will be distributed to the SAG. The SAG will evaluate the checklist and domains of each outcome and will list additional outcomes if they think important ones have been left off the checklist. The final outcomes will be the candidate items for the COS in Questionnaire 1 and will be scored in Delphi process 1. The SAG will not participate in the Delphi process, but will take part in the consensus meeting.
Stage 2: Delphi survey
A wide variety of stakeholders, including hyperlipidemia patients, clinicians, methodologists, regulators, researchers, ethicists, statisticians, journal editors, and other relevant experts, will be recruited. There are no guidelines for the sample size of the Delphi study ; however, in general, the more panelists who participate, the more reliable the group judgment will be . We therefore expect to select 50 panelists (including at least five patients and 10 experts) using “snowball sampling” (Table 1). The sample size will be based on the Construction of Core Outcome Set of Traditional Chinese Medicine Clinical Trials (COS-TCM) and the Implementation Specifications of the Delphi Method . According to the principle of representativeness and authority, the experts will be selected from the following five professional fields: CM/integrated Chinese and Western medicine, clinical pharmacy, clinical epidemiology, statistics, and editors of important relevant journals. The identification of stakeholders will begin with a preliminary list of experts and patients. The preliminary patients will be selected from outpatients at GPHCM and the preliminary experts will be recognized authors of high impact papers and relevant senior physicians of the Chinese Association of Integrative Medicine. The preliminary list will then be augmented with authors of studies included in the systematic review. Finally, the experts and stakeholders will recommend whomever else they think should be included as a relevant stakeholder . An email will be sent to the identified panelists who registered with informed consent to explain this study and the importance of the Delphi survey. Out of consideration for the response rate, we will email more related stakeholders than we plan to include. Each panelist will have a unique ID number corresponding to their contact information and their responses for each round of the Delphi survey.
Candidate items will be grouped by topic. In each group, candidate items will be sorted alphabetically. In this way, it will be convenient to both score items and avoid ranking bias in each group. For patient panelists, lay equivalents of each outcome will be presented instead of scientific terms . Candidate items will be measured using a 9-point Likert scale, where 7, 8, and 9 mean “critical importance;” 4, 5, and 6 mean “not critically important;” and 1, 2, and 3 mean “low importance” . This scale has been recommended by the COMET Initiative for measuring outcomes  and has been widely adopted by other COS development studies [17, 30,31,32].
Delphi round 1
In round 1 of the Delphi survey, all panelists will be asked to score all candidate items, along with their name, contact information, and other relevant personal information registered. This will confirm their stakeholder groups. After scoring all of the candidate outcomes, panelists will have the opportunity to add one or two additional outcomes they believe are important to hyperlipidemia but which had not been included in the questionnaire. A three-week period is planned for data collection and a reminder will be sent via email or SMS to panelists who have not completed their questionnaires by the end of the second week. A second reminder will be sent via email or SMS when only two days are left.
The number of panelists who complete Questionnaire 1 will be recorded. Each outcome will be analyzed using descriptive statistics, both by stakeholder group and as a whole. The SAG will assess the additional outcomes to decide whether they are representative of new outcomes. Those identified as new outcomes will be included in Delphi round 2. All scored outcomes will be included into Delphi round 2 (Questionnaire 2).
Delphi round 2
Panelists who complete the Delphi round 1 will be invited to take part in the Delphi round 2. They will re-score the outcomes listed on Questionnaire 2, where the scores they assigned in Delphi round 1 will be presented. The descriptive statistical analysis results of Delphi round 1, including the responses and distribution of each outcome from each different stakeholder group, will be attached. Panelists can make their scores in light of insight from others in his/her stakeholder group.
Descriptive statistics will be performed again. For each stakeholder group, we will calculate the number of participants and describe the distribution of each outcome score. A comparison will be conducted between each stakeholder group and the response group as a whole. Outcomes whose median is ≥ 4 (by any stakeholder group) will continue to Delphi round 3 (Questionnaire 3).
Delphi round 3
Panelists who complete Delphi rounds 1 and 2 will participate in round 3 to re-score Questionnaire 3. Questionnaire 3 will include those scores they gave in round 2. Descriptive statistical analysis of round 2, including the responses and distribution of each outcome, will be attached. Panelists will be asked to make their scores in light of the insight of all stakeholder groups.
After conducting the Delphi round 3 analysis, the outcomes will be categorized into three domains: “consensus out,” “consensus in,” or “without consensus,” using the definitions in Table 2. These prespecified definitions have also been suggested by the COMET initiative  and have been used by other COS development studies [33,34,35,36]. They can minimize the chance of consensus being defined post hoc and further decrease bias resulting from the pre-existing beliefs of the research team . The consistency of these results across stakeholders, as well as across groups, will be analyzed. This analysis will ensure the relevance of outcomes across all participating stakeholders, making sure all voices are heard and that no minority stakeholders are suppressed . The results of this process will be forwarded to the consensus meeting.
Stage 3: Consensus meeting
To ensure a range of opinions of all hyperlipidemia stakeholder groups, the purposeful sampling method will be adopted to select participants from those who have completed all three rounds of the Delphi survey. The consensus group will be composed of at least two representatives from patients, endocrinologists, cardiologists, hyperlipidemia specialist nursing, and hyperlipidemia researchers, as well as all members of the SAG.
A roundtable consensus meeting will be held at GPHCM to identify the final COS-CM-Hyperlipidemia. The results of each Delphi round and the classification of candidate outcomes from round 3 will be represented. Outcomes of “consensus in” will be voted either “yes” or “no” electronically and anonymously. Those voted for by at least 70% of the participants will be included in the final COS-CM-Hyperlipidemia. Outcomes scored “consensus out” will be excluded from the final COS-CM-Hyperlipidemia. Outcomes of “without consensus” in round 3 will be discussed. Participants will be asked to re-score all of the “without consensus” outcomes anonymously using the same 9-point Likert scale at the meeting. Outcomes for which at least 70% were scored 1–3 and at most 15% received a score of 7–9 will be removed from the final COS-CM-Hyperlipidemia. Those for which at least 70% were scored 7–9 and at most 15% were scored 1–3 will be included in the final COS-CM-Hyperlipidemia. The remaining outcomes will be re-scored until a final consensus is reached [28, 31].
Stage 4: Promotion and update
A priority of this study is to improve the quality and value of CM trials for hyperlipidemia by applying the COS-CM-Hyperlipidemia. We will collaborate with systematic review groups, clinical guideline makers, clinicians, journal editors, and ethics committees to promote the broad application of COS-CM-Hyperlipidemia. We hope this COS-CM-Hyperlipidemia will be published in the COMET Initiative and be recommended by relevant industry associations.
As medical research continues, understandings of disease, diagnosis, treatment, and evaluation will be updated. Therefore, the COS-CM-Hyperlipidemia needs constant evaluation and upgrading in accordance with the latest achievements of basic and clinical research. In the process of promotion, weaknesses in the current COS-CM-Hyperlipidemia should be noted. They should then be revised in line with the standard procedures based on the number and importance of these shortcomings. With constant evaluation and updates, the COS-CM-Hyperlipidemia can maintain its value and advantages in clinical and basic research of CM for hyperlipidemia. This is accomplished by adding new outcomes and new evaluation tools to ensure practicality and advancement.
To date, there has been no COS for CM clinical trials on hyperlipidemia. The development of COS-CM-Hyperlipidemia will improve the design and operation of CM trials, keeping them in compliance with international standards and guaranteeing the credibility of their results . Outcomes important to key stakeholders, particularly to physicians and patients with hyperlipidemia, will be identified through the development of the COS-CM-Hyperlipidemia. We hope that the use of the developed COS-CM-Hyperlipidemia will ensure the consistency of important outcomes, lower reporting bias, improve the comparability of future studies, and improve the methodological quality of CM clinical research and the utility of study results.
The development of the COS-CM-Hyperlipidemia is currently in the systematic review stage.
Core Outcome Measures in Effectiveness Trials
Core outcome set
Core outcome set of CM clinical trials for hyperlipidemia
Guangdong Provincial Hospital of Chinese Medicine
Low-density lipoprotein cholesterol
Nelson RH. Hyperlipidemia as a risk factor for cardiovascular disease. Prim Care. 2013;40(1):195–211.
Owens AR, Byrnes JR, Mackman N. Hyperlipidemia, tissue factor, coagulation, and simvastatin. Trends Cardiovasc Med. 2014;24(3):95–8.
Jellinger PS, Handelsman Y, Rosenblit PD, et al. American Association of Clinical Endocrinologists and American College of Endocrinology Guidelines for management of Dyslipidemia and Prevention of Atherosclerosis. Endocr Pract. 2017;23(Suppl 2):1–87.
Chinese Joint Committee of Revising the Guidelines for Prevention and Treatment of Dyslipidemia in Chinese Adults. Guidelines for Prevention and Treatment of Dyslipidemia in Chinese Adults (2016). Chin J Cardiol. 2016;44(10):833–53.
Tietge UJ. Hyperlipidemia and cardiovascular disease: inflammation, dyslipidemia, and atherosclerosis. Curr Opin Lipidol. 2014;25(1):94–5.
Mozaffarian D, Benjamin EJ, Go AS, et al. Heart Disease and Stroke Statistics-2016 Update: A Report from the American Heart Association. Circulation. 2016;133(4):e38–360.
Pan L, Yang Z, Wu Y, et al. The prevalence, awareness, treatment and control of dyslipidemia among adults in China. Atherosclerosis. 2016;248:2–9.
Anderson TJ, Grégoire J, Pearson GJ, et al. 2016 Canadian Cardiovascular Society Guidelines for the Management of Dyslipidemia for the Prevention of Cardiovascular Disease in the Adult. Can J Cardiol. 2016;32(11):1263–82.
World Health Organization. WHO International Standard Terminologies on Traditional Medicine in the Western Pacific Region. Geneva: World Health Organization; 2007.
Lai SL. Key points about clinical effectiveness assessment of traditional Chinese medicine. J Guangzhou Univ Tradit Chin Med. 2002;19(4):245–50.
Liao J, Tian J, Li T, et al. Xuefuzhuyu decoction for hyperlipidemia: a systematic review and meta-analysis of randomized clinical trials. J Tradit Chin Med. 2014;34(4):411–8.
Yao H, Zhang Z, Wang J, et al. Efficacy and safety of Yinchenwuling powder for hyperlipidemia: a systematic review and Meta-analysis. J Tradit Chin Med. 2016;36(2):135–43.
Liu ZL, Liu JP, Zhang AL, et al. Chinese herbal medicines for hypercholesterolemia. Cochrane Database Syst Rev. 2011;7:CD008305.
Liu J, Zhang J, Shi Y, et al. Chinese red yeast rice (Monascuspurpureus) for primary hyperlipidemia: a meta-analysis of randomized controlled trials. Chin Med. 2006;1:4.
Zhang L, Zhang J, Chen J, et al. Clinical research of traditional Chinese medicine needs to develop its own system of core outcome sets. Evid Based Complement Alternat Med. 2013;2013:202703.
Tai HX. Systematic review of Zhibituo in treatment of hyperlipidemia. Tianjin : Tianjin Medical University; 2015.
Williamson P, Altman D, Blazeby J, et al. Driving up the quality and relevance of research through the use of agreed core outcomes. J Health Serv Res Policy. 2012;17(1):1–2.
Williamson P, Altman D, Blazeby J, et al. The COMET (Core Outcome Measures in Effectiveness Trials) Initiative. Trials. 2011;12(Suppl 1):A70.
COMET Initiative. Overview. http://www.comet-initiative.org/about/overview. Accessed 9 Aug 2017.
Hongcai Shang, Ruijin Qiu, Xiaoyu Zhang, et al. A core outcome set for clinical trials of Traditional Chinese Medicine in non-valvular atrial fibrillation. http://www.comet-initiative.org/studies/details/941. Accessed 9 Aug 2017.
Yanan Sun, Changhe Yu, Liyun He, et al. Core Outcome Sets of integrity of modern and Traditional Chinese Medicine on treatment of chronic Hepatitis B. http://www.comet-initiative.org/studies/details/669. Accessed 9 Aug 2017.
Junhua Zhang, Dongmei Xing, Hongcai Shang. Developing a Core Outcome Set for Traditional Chinese Medicine for Stable Angina Pectoris. http://www.comet-initiative.org/studies/details/391. Accessed 9 Aug 2017.
Zehuai Wen, Xiankun Chen, Geng Li, et al. A core outcome set for clinical trials of Chinese Medicine in hyperlipidemia. http://www.comet-initiative.org/studies/details/983. Accessed 9 Aug 2017.
Williamson PR, Altman DG, Blazeby JM, et al. Developing core outcome sets for clinical trials: issues to consider. Trials. 2012;13:132.
Prinsen CA, Vohra S, Rose MR, et al. Core Outcome Measures in Effectiveness Trials (COMET) initiative: protocol for an international Delphi study to achieve consensus on how to select outcome measurement instruments for outcomes included in a ‘core outcome set’. Trials. 2014;15:247.
Murphy MK, Black NA, Lamping DL, et al. Consensus development methods, and their use in clinical guideline development. Health Technol Assess. 1998;2:i–iv 1–88.
Zhang D, Zhang M, Zheng W, et al. The Construction of Core Outcome Set of Traditional Chinese Medicine Clinical Trials (COS-TCM) and Implementation Specification of the Delphi Method. J Tradit Chin Med. 2017;58(1):20–2.
Allin B, Bradnock T, Kenny S, et al. NETS1HD: study protocol for development of a core outcome set for use in determining the overall success of Hirschsprung’s disease treatment. Trials. 2016;17(1):577.
Guyatt GH, Oxman AD, Kunz R, et al. GRADE guidelines: 2. Framing the question and deciding on important outcomes. J Clin Epidemiol. 2011;64(4):395–400.
Kelly LE, Jansson LM, Moulsdale W, et al. A core outcome set for neonatal abstinence syndrome: study protocol for a systematic review, parent interviews and a Delphi survey. Trials. 2016;17(1):536.
Iyengar S, Williamson PR, Schmitt J, et al. Development of a core outcome set for clinical trials in rosacea: study protocol for a systematic review of the literature and identification of a core outcome set using a Delphi survey. Trials. 2016;17(1):429.
Blackwood B, Ringrow S, Clarke M, et al. Core Outcomes in Ventilation Trials (COVenT): protocol for a core outcome set using a Delphi survey with a nested randomised trial and observational cohort study. Trials. 2015;16:368.
Gargon E, Gurung B, Medley N, et al. Choosing important health outcomes for comparative effectiveness research: a systematic review. PLoS One. 2014;9(6):e99111.
Taylor J, Böhnke JR, Wright J, et al. A core outcome set for evaluating self-management interventions in people with comorbid diabetes and severe mental illness: study protocol for a modified Delphi study and systematic review. Trials. 2017;18(1):70.
Hirsch M, Duffy JM, Barker C, et al. Protocol for developing, disseminating and implementing a core outcome set for endometriosis. BMJ Open. 2016;6(12):e013998.
Gillies K, Entwistle V, Treweek SP, et al. Evaluation of interventions for informed consent for randomised controlled trials (ELICIT): protocol for a systematic review of the literature and identification of a core outcome set using a Delphi survey. Trials. 2015;16:484.
Tong A, Manns B, Hemmelgarn B, et al. Standardised outcomes in nephrology - Haemodialysis (SONG-HD): study protocol for establishing a core outcome set in haemodialysis. Trials. 2015;16:364.
Harman NL, Bruce IA, Callery P, et al. MOMENT--Management of Otitis Media with Effusion in Cleft Palate: protocol for a systematic review of the literature and identification of a core outcome set using a Delphi survey. Trials. 2013;14:70.
This work was supported by the Specific Research Fund for TCM Science and Technology of Guangdong Provincial Hospital of Chinese Medicine (No. YN2015QN11).
Availability of data and materials
Ethics approval and consent to participate
Ethical approval to conduct this study has been granted by the Ethics Committee of Guangdong Provincial Hospital of Chinese Medicine. Consent will be requested from all study participants.
Consent for publication
The authors declare that they have no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.