Skip to main content

How pragmatic are the randomised trials used in recommendations for control of glycosylated haemoglobin levels in type 2 diabetic patients in general practice: an application of the PRECIS II tool

Abstract

Background

Recommendations for good clinical practice have been reported to be difficult to apply in real life by primary care clinicians. This could be because the clinical trials at the origin of the guidelines are based on explanatory trials, conducted under ideal conditions not reflecting the reality of primary care, rather than pragmatic trials conducted under real-life conditions. The objective of this study was to evaluate how pragmatic are the clinical trials used to build the French High Authority of Health’s recommendations on the management of type II diabetes.

Methods

Trials from the 2013 Cochrane meta-analysis that led to the 2013 French High Authority of Health’s recommendations on the management of type II diabetes were selected. Each trial was analysed by applying the PRECIS-2 tool to evaluate whether the trial was pragmatic or explanatory, according to the nine domains of PRECIS-2. Each domain was scored between 1 (very explanatory) and 5 (very pragmatic) by two blinded researchers, and consensus was reached with a third researcher in case of discrepancy. Median scores were calculated for each of the nine domains.

Results

Twenty-three articles were analysed. Eight out of nine domains – namely eligibility, recruitment, setting, organisation, flexibility of delivery, flexibility of adherence, follow-up, and primary outcome – had a median score of less than 3, indicating a more explanatory design. Only the primary analysis domain had a score indicating a more pragmatic approach (median score of 4). In more than 25% of the articles, data to score the domains of recruitment, flexibility of delivery, flexibility of adherence, and primary analysis were missing.

Conclusions

Trials used to build French recommendations for good clinical practice for the management of type 2 diabetes in primary care were more explanatory than pragmatic. Policy-makers should encourage the funding of pragmatic trials to evaluate the different strategies proposed for managing the patient’s treatment according to HbA1C levels and give clinicians feasible recommendations.

Peer Review reports

Introduction

In France, recommendations to improve the management of patients with type II diabetes are based on the glycosylated haemoglobin (HbA1C) levels [1]. Practitioners have to adapt the patient’s treatment for them to achieve the recommended HbA1C levels according to their condition [1]. For most patients with type 2 diabetes, an HbA1c target less than or equal to 7% is recommended. For elderly or frail patients, the HbA1C target may be 8% or even 9% [1]. For newly diagnosed young and healthy patients, the HbA1C target is 6.5% [1].

These recommendations for managing the patient’s treatment according to HbA1C levels are based mainly on experts’ views of existing evidence. This evidence is often based on randomised trials. The latest guidelines are based on a 2013 Cochrane review that considered the effects of targeted intensive glycaemic control compared with conventional glycaemic control in patients with type 2 diabetes [2]. The review included randomised trials comparing mortality, macrovascular and microvascular complications and adverse events depending on predefined HbA1c targets [2]. However, there is a lack of studies comparing the different strategies, based on morbidity and mortality outcomes. Of the 45 recommendations, none is grade A (i.e., high quality of evidence, usually from well-performed randomised controlled trials), four are grade B (i.e., moderate quality of evidence, usually from randomised controlled trials with important limitations or very strong evidence from other designs) and 41 are grade C (i.e., low quality of evidence, usually from observational studies, clinical experience or controlled trials with serious flaws) [1].

However, an intensive strategy of management of HbA1C levels is not without risk: over a treatment period of five years, 117 to 150 patients would need to be treated to avoid one myocardial infarction and 32 to 142 patients to avoid one episode of microalbuminuria, whereas one severe episode of hypoglycaemia would occur for every 15 to 52 patients [3]. Severe hypoglycaemia was defined as a blood glucose level of less than 2.8 mmol/L (50 mg/dL) in patients with transient dysfunction of the central nervous system who were unable to treat themselves (requiring help from another person) [4].

For different reasons, practitioners do not follow recommendations [5]: lack of trust in the scientific basis of the recommendations, difficulties in applying the recommendations in real life, or for other reasons particular to the practitioner and their professional environment. The lack of trust in the scientific basis of the recommendations may come from the fact that the trials the recommendations are based on are not pragmatic, leading to results that cannot be applied in usual care.

A pragmatic trial is designed for testing the effectiveness of an intervention under real-world conditions, whereas an explanatory trial is designed for testing an intervention under ideal experimental conditions [6]. As mentioned by Godwin et al., “The explanatory trial seeks to maximise the internal validity by assuring rigorous control of all variables other than the intervention. The pragmatic trial seeks to maximise external validity to ensure that the results can be generalized” [7]. Trials are rarely wholly pragmatic or explanatory, and different elements of a trial may be more pragmatic or more explanatory. An assessment of how pragmatic or explanatory a trial is can be made by using PRECIS-2, a tool designed to evaluate whether a trial is more explanatory or pragmatic across nine different domains: eligibility criteria, recruitment, setting, organisation, flexibility (delivery), flexibility (adherence), follow-up, primary outcome, and primary analysis [8].

Therefore, we decided to determine whether the trials at the basis of the recommendations for the management of type II diabetes were more pragmatic or explanatory by using the PRECIS-2 tool. We decided to apply it to trials that were selected to build the recommendations on the management of type II diabetes on the basis of HbA1C levels [1]. Our aim was to determine whether those trials were more explanatory or pragmatic according to the PRECIS-2 tool.

Methods

Selection of randomised trials

Studies were eligible if they were randomised trials and included in the most recent Cochrane meta-analysis conducted on the topic that was cited in the recommendations and published in 2013 [2]. We restricted eligibility to the most recent Cochrane meta-analysis as recommendations were based on the results of the trials included in this review. Articles that reported studies that were not randomised trials or meta-analyses of randomised trials, articles reporting ancillary studies (i.e., studies derived from an original study), and articles written in a language other than English or French were excluded.

Data collection

Each selected trial was analysed by using the PRECIS-2 tool [8]. Scores for each of the nine domains were from 1 (very explanatory) to 5 (very pragmatic) using a 5-point Likert scale [8]. A score of 3 was defined as equally pragmatic and explanatory [8]. In order to harmonise the scoring of the PRECIS-2 tool, the tool was tested on three randomly selected trials by the two researchers who would analyse all of the trials (ET and CD-D) and discussions on the scoring of the domains were carried out with all of the researchers (ET, CD-D, IE-A, SE, and GF) before the analysis of the remaining trials. The nine domains of the PRECIS-2 tool are described in Additional file 1: Appendix 1, and the result of the discussions on the scoring is detailed in Additional file 1: Appendix 2. Data were collected by two researchers (ET and CD-D) blinded from each other and using a standardised form developed by the researchers (CD-D, IE-A, SE, and GF). Collected data included the characteristics of the trials (details of the publication, design and main result) and scoring for the nine domains of the PRECIS-2 tool. In case of insufficient information in the selected article, the protocol of the article was consulted if it was referenced and published. After having collected the data, the two researchers (ET and CD-D) shared the results and discussed disagreeing scores to reach a consensus. If consensus between ET and CD-D could not be reached, the opinion of a third researcher (IE-A, SE or GF) was used to establish a consensus.

Data analysis

The characteristics of the trials were analysed descriptively. The median score and interquartile range (Q1; Q3) over all trials were calculated for each domain in the PRECIS-2 tool. We also performed a descriptive analysis of the articles for which scoring needed a consensus between researchers and where there were missing data in the articles to score the domains of the PRECIS-2 tool.

Results

Selection of articles

From the Cochrane meta-analysis published in 2013 on the HbA1C target-based therapeutic strategy of type 2 diabetes, 28 randomised clinical trials were identified [2]. Of these 28 randomised trials, 23 trials were included in the analysis (see Fig. 1).

Fig. 1
figure 1

Flowchart of the selected trials

Characteristics of the included trials

The characteristics of the included trials are shown in Table 1. 2009 was chosen as a threshold date because it corresponds to the date of the first publication of the PRECIS tool [9]. All of the trials evaluated an intervention involving delivering a drug to achieve strict targets of HbA1C. The control group was a group with a usual care strategy or with less stringent blood glucose targets than the intervention group. The details of the intervention and primary criteria for each of the 23 trials are given in Additional file 1: Appendix 3. Of the 23 studies analysed, 17 were conducted in hospital settings.

Table 1 Characteristics of the 23 included trials

Scoring of trials according to the PRECIS-2 tool

The median scores for each domain in the PRECIS-2 tool are shown in Table 2.

Table 2 Median scores of the 23 trials for each domain of the PRECIS-2 tool

Eight out of nine PRECIS-2 domains had a median score of less than 3. “Recruitment”, “Flexibility of adherence” and “Primary outcome” were the hardest domains for reaching the consensus between explanatory and pragmatic.

The results were graphically represented with the “wheel” of PRECIS-2 in Fig. 2.

Fig. 2
figure 2

Graphical representation of the PRECIS-2 wheel

The details of the median scores before and after consensus are available in Additional file 1: Appendix 4.

For every PRECIS-2 domain, between 60% and 85% of the articles required a consensus, mainly on the degree to which the trial was pragmatic (between the scores 4 or 5) or explanatory (between scores 1 or 2). Disagreements between pragmatic score (>3) and explanatory scores (<3) occurred in between 0% and 22% of articles, depending on which domain was considered.

Discussion

Summary of results

We found that clinical trials contributing to recommendations on the therapeutic strategy of type 2 diabetes were more explanatory than pragmatic. In fact, only the domain on primary analysis was more pragmatic with a median score greater than 3 (4). The lack of information reported in the article to score its explanatory or pragmatic aspect concerned mainly recruitment, flexibility of delivery, flexibility of adherence and primary analysis, and more than 25% of articles had insufficient information. The majority of the scoring has required a consensus to define the degree to which the trial was pragmatic or explanatory.

Interpretation of results

For eligibility, the selection of the population was more explanatory than pragmatic. Most trials evaluated the intervention under optimal conditions and not with populations that would present in primary care. For example, patients with comorbidities that could interfere with study outcomes, patients living too far from the study site, or patients who were unable to manage insulin and cope with hypoglycaemia were excluded [10,11,12,13]. These highly selective samples excluded patients usually encountered in general practice, which may be one of the barriers to implementing recommendations in usual care. Furthermore, some studies included very specific populations, such as veterans, that were not comparable with the typical French diabetic population [14].

Most of the trials were carried out in hospital settings, leading to the scores for domains of setting, recruitment, organisation, primary outcome and flexibility of delivery being more explanatory than pragmatic. If recommendations of good clinical practice were addressed to the management of hospitalised patients with diabetes, those results would have been more pragmatic in that context. However, some characteristics of the organisation or intervention that are provided in the hospital setting would not be available in primary care, such as involvement of specialised staff to administer the treatment and educate the patient or more intensive monitoring of the patient [15,16,17,18,19,20,21]. In the hospital, the treatments were administered by nurses, which ensured regular intake and reduced the risk of non-compliance. On the other hand, the outpatient was autonomous, and there was no guarantee of compliance with the treatment prescribed by the general practitioner. For the primary outcome, the median score of 2 in favour of a rather explanatory methodological choice was because either the primary outcome was not observable in general practice or it required the intervention of specialists [20, 22, 23]. For instance, the study by Natarajan et al. used the change of volume of intimal hyperplasia within the stent as primary outcome [22].

The domain on the primary analysis was the only one out of the nine with a median score greater than 3, suggesting that the majority of trials were more pragmatic in relation to this domain. This was because most of the studies analysed the results following the intention-to-treat principle [24]. This sort of analysis is recommended for intervention’s regulatory approval and in CONSORT (Consolidated Standards of Reporting Trials) guidelines.

Strengths and limitations

Our study was the first from the perspective of the French primary care setting to use a graphic tool to illustrate that the trials on which recommendations for good clinical practice were based were not pragmatic. We chose to use the PRECIS-2 tool because it had already been used successfully for a systematic review in order to assess whether the pragmatism of the trials was a source of heterogeneity in the trial results [25]. As a prevention for a desirability bias according to our initial hypothesis of an excess of explanatory trials, the researchers had different backgrounds: an experienced general practitioner (CD-D) and a medical student (ET) independently scored the trials, whereas consensus was reached thanks to a third general practitioner (IE-A) and two biostatisticians (SE and GF) who studied the applicability of the PRECIS-2 tool [26]. However, in our study, many scores were finalised only after discussion between researchers. Some areas of the PRECIS-2 tool were subject to interpretation despite the examples given in the article by Loudon et al. [8]. This can be explained by the purpose for which the PRECIS-2 tool was created, namely the evaluation of protocols intended for researchers and not the evaluation of the published trials. To be applicable for the latter objective would require a greater precision in the way of scoring each domain, to ensure a better homogeneity of the evaluations of the trials.

There was also a lot of missing data in the articles to score certain domains of PRECIS-2. However, missing data were less frequent for domains that were detailed in the CONSORT guidelines for reporting randomised trials [24].

Implications

According to our results, the clinical trials on which recommendations for good clinical practice were based had primarily explanatory features, which could constitute an obstacle to the application of recommendations to the therapeutic strategy of type 2 diabetes in general practice. However, we cannot be sure that the lack of pragmatism of the evidence of the guidelines is the only reason why general practitioners do not follow the guidelines. This can contribute to this lack of trust, but other controversies around the evidence might also play a role: in particular, the Cochrane review concluded that there was no benefit of the intensive glycaemic control on mortality compared with the conventional glycaemic control, and French guidelines still recommended an HbA1C targets less than 7% in most of the cases.

Ideally, both types of trials would be required: explanatory trials to demonstrate the efficacy of interventions under ideal conditions and pragmatic trials performed under the usual conditions of practice to improve the applicability of the results in general practice. This requires the involvement of general practitioners and their patients in clinical research and also methodological and organisational adaptations to integrate research into daily care practice.

Conclusions

Our study has highlighted the fact that the clinical trials leading to recommendations on the therapeutic strategy of type 2 diabetes were more explanatory than pragmatic. Diabetes researchers could concentrate on more pragmatic trials. Policy-makers should encourage the funding of pragmatic trials to evaluate the different strategies proposed in the recommendations for managing the patient’s treatment according to HbA1C levels.

Availability of data and materials

All of the dataset is available on request.

Abbreviations

CONSORT:

Consolidated Standards of Reporting Trials

HbA1C:

Glycosylated haemoglobin

PRECIS-2:

Pragmatic-explanatory continuum indicator summary 2nd version

References

  1. French Health Authority 2013. Treatment strategy for glycaemic control in type 2 diabetes. Available at: https://wwwhas-santefr/portail/jcms/c_1022476/fr/strategie-medicamenteuse-du-controle-glycemique-du-diabete-de-type-2 January 2013.

    Google Scholar 

  2. Hemmingsen B, Lund SS, Gluud C, Vaag A, Almdal TP, Hemmingsen C, et al. Targeting intensive glycaemic control versus targeting conventional glycaemic control for type 2 diabetes mellitus. Cochrane Database Syst Rev. 2013;11:CD008143.

    Google Scholar 

  3. Boussageon R, Bejan-Angoulvant T, Saadatian-Elahi M, Lafont S, Bergeonneau C, Kassaï B, et al. Effect of intensive glucose lowering treatment on all cause mortality, cardiovascular death, and microvascular events in type 2 diabetes: meta-analysis of randomised controlled trials. BMJ. 2011;343:d4169.

    Article  Google Scholar 

  4. Zoungas S, Patel A, Chalmers J, de Galan BE, Li Q, Billot L, et al. Severe hypoglycemia and risks of vascular events and death. N Engl J Med. 2010;363(15):1410–8.

    Article  CAS  Google Scholar 

  5. Cogneau J, Lehr-Drylewicz AM, Bachimont J, Letoumy A. Guidelines and practices for type 2 diabetes. Patients’ and doctors’ preconceptions prevent effective patient education. Presse Med. 2007;36:764–70.

    Article  Google Scholar 

  6. Schwartz D, Lellouch J. Explanatory and pragmatic attitudes in therapeutical trials. J Clin Epidemiol. 2009;62(5):499–505.

    Article  Google Scholar 

  7. Godwin M, Ruhland L, Casson I. Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity. BMC Med Res Methodol. 2003;3:28.

    Article  Google Scholar 

  8. Loudon K, Treweek S, Sullivan F, Donnan P, Thorpe KE, Zwarenstein M. The PRECIS-2 tool: designing trials that are fit for purpose. BMJ. 2015;350(2147):1–11.

    Google Scholar 

  9. Thorpe KE, Zwarenstein M, Oxman AD, Treweek S, Furberg CD, Altman DG, et al. A pragmatic-explanatory continuum indicator summary (PRECIS): a tool to help trial designers. J Clin Epidemiol. 2009;62(5):464–75.

    Article  Google Scholar 

  10. Becker A, van der Does FEE, van Hinsbergh VWM, Heine RJ, Bouter LM, Stehouwer CDA. Improvement of glycaemic control in type 2 diabetes: favourable changes in blood pressure, total cholesterol and triglycerides, but not in HDL cholesterol, fibrinogen, von Willebrand factor and (pro)insulin. Neth J Med. 2003;61(4):129–36.

    CAS  PubMed  Google Scholar 

  11. Action to Control Cardiovascular Risk in Diabetes Study Group, Gerstein HC, Miller ME, Byington RP, Goff DC Jr, Bigger JT, et al. Effects of intensive glucose lowering in type 2 diabetes. N Engl J Med. 2008;358(24):2545–59.

    Article  Google Scholar 

  12. Blonde L, Merilainen M, Karwe V, Raskin P. Patient- directed titration for achieving glycaemic goals using a once-daily basal insulin analogue: an assessment of two different fasting plasma glucose targets - the TITRATE study. Diabetes Obes Metab. 2009;11(6):623–31.

    Article  CAS  Google Scholar 

  13. Malmberg K, Ryden L, Wedel H, Birkeland K, Bootsma A, Dickstein K, et al. Intense metabolic control by means of insulin in patients with diabetes mellitus and acute myocardial infarction (DIGAMI 2): effects on mortality and morbidity. Eur Heart J. 2005;26(7):650–61.

    Article  CAS  Google Scholar 

  14. Duckworth W, Abraira C, Moritz T, Reda D, Emanuele N, Reaven PD, et al. Glucose control and vascular complications in veterans with type 2 diabetes. N Engl J Med. 2009;360(2):129–39.

    Article  CAS  Google Scholar 

  15. Melidonis A, Stefanidis A, Tournis S, Manoussakis S, Handanis S, Zairis M, et al. The role of strict metabolic control by insulin infusion on fibrinolytic profile during an acute coronary event in diabetic patients. Clin Cardiol. 2000;23(3):160–4.

    Article  CAS  Google Scholar 

  16. UK Prospective Diabetes Study (UKPDS) Group. Intensive blood-glucose control with sulphonylureas or insulin compared with conventional treatment and risk of complications in patients with type 2 diabetes (UKPDS 33). Lancet. 1998;352:837–53.

    Article  Google Scholar 

  17. Jaber LA, Halapy H, Fernet M, Tummalapalli S, Diwakaran H. Evaluation of a pharmaceutical care model on diabetes management. Ann Pharmacother. 1996;30(3):238–43.

    Article  CAS  Google Scholar 

  18. Gaede P, Vedel P, Parving HH, Pedersen O, Gaede P, Vedel P, et al. Intensified multifactorial intervention in patients with type 2 diabetes mellitus and microalbuminuria: the Steno type 2 randomised study. Lancet. 1999;353(9153):617–22.

    Article  CAS  Google Scholar 

  19. Knatterud GL, Klimt CR, Levin ME, Jacobson ME, Goldner MG. Effects of hypoglycemic agents on vascular complications in patients with adult-onset diabetes. Mortality and selected nonfatal events with insulin treatment. JAMA. 1978;240(1):37–42.

    Article  CAS  Google Scholar 

  20. Ohkubo Y, Kishikawa H, Araki E, Miyata T, Isami S, Motoyoshi S, et al. Intensive insulin therapy prevents the progression of diabetic microvascular complications in Japanese patients with non-Insulin-dependent diabetes- mellitus - A randomized prospective 6-year study. Diabetes Res Clin Pract. 1995;28(2):103–17.

    Article  CAS  Google Scholar 

  21. de Souza Fantin S, Wainstein MV, Polanczyk CA, Ledur P, Lazzari CM, Klein C, et al. Inflammatory and oxidative stress markers after intravenous insulin in percutaneous coronary intervention with stent in type 2 diabetes mellitus: a randomized controlled trial. J Clin Endocrinol Metab. 2011;96(2):478–85.

    Article  Google Scholar 

  22. Natarajan MK, Strauss BH, Rokoss M, Buller CE, Mancini GB, Xie C, et al. Randomized trial of insulin versus usual care in reducing restenosis after coronary intervention in patients with diabetes. the STent Restenosis And Metabolism (STREAM) study. Cardiovasc Revasc Med. 2012;13(2):95–100.

    Article  Google Scholar 

  23. Stefanidis A, Melidonis A, Tournis S, Zairis M, Handanis S, Beldekos D, et al. Effect of intravenous insulin administration on left ventricular performance during non- ST-elevation acute coronary events in patients with diabetes mellitus. Am J Cardiol. 2003;91(10):1237–40.

    Article  CAS  Google Scholar 

  24. Moher D, Schulz KF, Altman D, CONSORT Group. The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA. 2001;285(15):1987–91.

    Article  CAS  Google Scholar 

  25. Aves T, Allan KS, Lawson D, Nieuwlaat R, Beyene J, Mbuagbaw L. The role of pragmatism in explaining heterogeneity in meta-analyses of randomised trials: a protocol for a cross-sectional methodological review. BMJ Open. 2017;7(9):e017887.

    Article  Google Scholar 

  26. Forbes G, Loudon K, Treweek S, Taylor SJC, Eldridge S. Understanding the applicability of results from primary care trials: lessons learned from applying PRECIS-2. J Clin Epidemiol. 2017;90:119–26.

    Article  Google Scholar 

Download references

Acknowledgements

Not applicable.

Funding

No funding.

Author information

Authors and Affiliations

Authors

Contributions

IE-A conceived the study, created the standardised form, analysed the data, interpreted the results and drafted the manuscript. ET collected and analysed the data and drafted the manuscript. GF and SE created the standardised form, interpreted the results and drafted the manuscript. CD-D conceived the study, collected and analysed the data, interpreted the results and drafted the manuscript. All authors, external and internal, had full access to all of the data (including statistical reports and tables) in the study and take responsibility for the integrity of the data and the accuracy of the data analysis. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Isabelle Ettori-Ajasse.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary information

Additional file 1: Appendix 1.

The nine domains of the PRECIS-2 tool. Appendix 2. Consensus on the scoring of the domains of the PRECIS-2 tool. Appendix 3. Intervention, control and primary outcome of the 23 included trials. Appendix 4. Median scores before and after consensus.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ettori-Ajasse, I., Tatin, E., Forbes, G. et al. How pragmatic are the randomised trials used in recommendations for control of glycosylated haemoglobin levels in type 2 diabetic patients in general practice: an application of the PRECIS II tool. Trials 21, 281 (2020). https://doi.org/10.1186/s13063-020-4215-5

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13063-020-4215-5

Keywords