Efficacy of a complex intervention comprising the guideline evidence-based health information and a training programme on the quality of health information: study protocol of a randomised controlled trial

Background Evidence-based health information (EBHI) is a prerequisite for informed and shared decision-making. The criteria for EBHI have been described comprehensively but the implementation in practice is still insufficient. The guideline evidence-based health information addresses providers of health information. Its goal is to improve the quality of health information. The evidence-based guideline emerged from the German Network for Evidence-based Medicine (DNEbM) and was published in February 2017. In addition, the competences of providers of health information were explored and a training programme was developed. Aim of this study is to evaluate the efficacy of an intervention comprising the guideline evidence-based health information and a training programme addressing providers of health information in a randomised controlled trial. We expect the intervention to improve the quality of health information in comparison to provision of the guideline only. Methods The trial uses a superiority randomised control group design with ten months follow-up. 26 providers of health information (groups with up to ten members) will be enrolled to compare the intervention (guideline & training programme) with usual care (guideline publicly available). The 5-day training programme comprises an evidence-based medicine training module and a module to prepare the application of the guideline. The primary outcome measure is the quality of the health information. Quality is operationalised as the extent of adherence to the guideline’s recommendations. Each provider will prepare a single health information informing a health-related decision. The quality of this information will be rated using the MAPPIN Health Information Quality instrument. An accompanying process evaluation will then be conducted. Discussion The study results will show whether the efficacy of the intervention justifies implementation of the training programme to enhance health information developers’ competences in evidence-based medicine and to ensure high quality EBHI in the long-term. Trial

approaches to reform the guideline development processes in order to reduce bias (15)(16)(17)(18)(19)(20)(21), and to GRADE (Grading of Recommendations, Assessment, Development and Evaluation) (22). The guideline development group consisted of providers of health information, health scientists and patient representatives. Based on GRADE (23) In general, guidelines may help to establish quality criteria and to improve practice (25).
With regard to implementation, various strategies have been discussed and it would seem that implementation in combination with a training programme might be successful (26).
Especially for the implementation of the guideline evidence-based health information, the training of the providers seems to be an appropriate method. Interviews with providers revealed shortcomings regarding their competences in evidence-based medicine (ebm) (27). However, these competences are prerequisites for fulfilling the requirements for the development and presentation of EBHI. Therefore, we have developed a training programme for the providers of health information. The programme comprises two modules: an ebm training module and a module to prepare the application of the guideline. We have developed a blended learning programme considering existing face-toface training programmes and tested it for acceptance and feasibility with providers of health information in a qualitative pilot study (28).

Objectives
The aim of this study is to evaluate the efficacy of an intervention comprising the guideline evidence-based health information and a training programme addressing providers of health information in a randomised controlled trial. The observation unit is the provider. We expect the intervention to improve the quality of health information in comparison to provision of the guideline only.

Methods
The reporting of this protocol follows the criteria of the standard protocol items: recommendations for interventional trials (SPIRIT) (29)

and the UK Medical Research
Council (MRC) Framework for the development and evaluation of complex interventions (30). For the completed SPIRIT checklist, see additional file 1.

Design
The trial uses a superiority randomised control group design, comparing two groups of equal sizes. In addition, the trial includes a formative evaluation seeking to fully understand the involved mediating mechanisms as well as the barriers towards the implementation process.

Setting and participants
The intervention is made available on-site at providers of public available health 6 information (e.g. health insurances, self-help associations or foundations, health portals, hospitals, rehabilitation clinics, nursing facilities and physicians) in the German-speaking area, mainly in Germany and Austria. Providers are defined as institutions or working groups rather than individuals and may therefore comprise several individuals (e.g. up to ten).

Eligibility and recruitment
Eligibility criteria Providers will be eligible if they are responsible for the production and publishing of health information. The use of external services (e.g. counselling by experts, graphic design, external performing of pilot tests or external literature searches) is not a reason for exclusion, as long as the provider is designated as the responsible publisher/editor. Providers should have regularly and currently published information. Therefore, they are eligible if they have published any information in the last 18 months. In addition, at least a single information produced in the last three years has to fulfil the following criteria: The information… has to inform a health related decision, has to address patients or medical laypersons, has to discuss different options regarding one specified health problem, does not inform about a single option, procedure or healthcare system and it does not give general advice on health and wellbeing.
It is essential that the participating providers assent to comply with the training programme and the production of one item of health information. Above all, the providers will be notified about the purpose and schedule of the intervention and their obligation to produce the new health information within ten months.
Members of the working groups on the guideline evidence-based health information or the GPGI are not eligible. The same applies to providers producing exclusively (drug) fact(s) boxes or offering counselling services such as medical online consulting websites, health related blogs, forums or communities or (online) encyclopaedias.
The providers themselves will choose the participants for the training programme. No other criteria are defined for these participants. If possible, all the individuals in the working group, who are closely involved in the development of the health information, should participate.

Recruitment
Providers will be identified by means of internet searches. We assume that most providers have a web appearance, even if they issue printed material. We will search via Google and MetaGer with different terms for online health information and webpages that refer to information (print or pdf). Data of the providers will be collected via these information and webpages (e.g. contact details). In addition, data of well-known providers will be listed.
The identified providers will be assessed for their eligibility (membership in the working groups guideline evidence-based health information and/or GPGI, the amount and kind of information published by them in the last three years). Contact details and the persons in charge will be identified online or by telephone. We will continue the search until 100 eligible providers are identified. These will receive a cover letter informing them briefly about the study and the guideline evidence-based health information as well as an invitation to take part in this project (see study flow, figure 1). Providers who do not reply within two weeks will be contacted personally by telephone with an offer of further information. If providers are interested, a telephone or online meeting will be scheduled to assess further eligibility criteria and to give an overview of the course of the study.
Afterwards, a personal meeting will be set up with eligible providers who intend to participate. The meeting is thought to give further information, to obtain informed consent, to assess baseline data and to perform the allocation. Further information about the study will be presented with special reference to the relevance of EBHI, the intervention and the primary outcome. The providers have to know about the requirements and advantages associated with their participation. Requirements are for example: releasing employees from work for the training programme, providing time and financial resources to develop a single health information in the study period, possibly with more effort in the intervention group in order to realise the guideline recommendations. In return, a final symposium with the presentation of the best information will be scheduled after completion of the project. With their consent, the providers' logos will be published on the project webpage after completing the study. In addition, the health information produced in the study may be published on the webpage along with the results of the quality rating. After receiving notification about requirements and incentives, the providers must assent to comply with the training programme and to produce one item of health information informing a health-related decision within ten months. In addition, written informed consent will be obtained from the individual participants and baseline data will be assessed by the researchers. Recruitment and randomisation will be performed consecutively.

Randomisation and blinding
Concealed allocation of participating provider groups to either the intervention or the control group will be determined by randomisation using a computer-generated list with randomly permuted blocks of length 2, 4, 6 or 8. An independent external person will prepare sealed opaque envelopes. After baseline assessment of the respective provider group, researchers will open the sealed opaque envelope and reveal the centre's allocation on-site.
Due to the nature of the intervention, blinding of the participating providers and researchers conducting the training programme is not possible. However, assessment of the primary endpoint and all analyses will be blinded towards group allocation.

Intervention group (IG)
Providers in the intervention group receive the guideline evidence-based health information and an adjunctive training, which is intended to facilitate the appropriate use of the guideline.
Guideline evidence-based health information The guideline defines the quality criteria of EBHI. It comprises general and ethical requirements regarding the development process, target group orientation and content of EBHI as well as 21 evidence-based recommendations assigned to the topics: presentation of frequencies, application of graphics, pictures and drawings, narratives, value clarification tools, formats and involvement of the target group. Table 1 gives an overview of the guideline and its recommendations.

Training programme
Workshop for persons in charge for provided health information (module 0) Even though the actual target group of the training programme are people directly involved in developing and producing health information, our intervention also needs to address the providers on the level of responsible decision-makers and leaders in the organisation. Attitudes of these persons can be both facilitators and/or barriers to the implementation of the EBHI criteria. Therefore, they will be invited to a workshop with the purpose to discuss and reflect on the benefits of using the EBHI criteria. In particular, the systemic structures required for developing EBHI will be addressed. If possible, the workshop (up to two hours) should take place on-site directly after randomisation into the intervention group. If this is not possible, a second personal or online meeting will be scheduled.

Training programme (module 1 and 2)
The free training programme is based on a problem-based instructional design that is learner-centred and enables learners to link theory with practice and to apply knowledge and skills to develop EBHI to a defined health problem for special target groups (31).
Case-based learning links theory to practice through the application of knowledge about the cases and by using inquiry-based learning methods (32). We set up a case about smoking cessation. This was chosen since it has relevance for several health professions and smoking is still a widely discussed topic. Moreover, (passive) smoking concerns everyone in daily life. Good health information is needed because most people in Germany try to quit smoking on their own, which is the less promising option (33). People need to be informed about evidence-based methods for smoking cessation, such as counselling, medications (e.g. antidepressants) and nicotine replacement therapy. Therefore, it is also a relevant topic for providers of health information.
The training programme comprises module 1 "ebm training" and module 2 "application of the guideline". It is designed in a blended learning format (34) where face-to-face and web-based learning activities alternate (see figure 2).

Figure 2: Modular training structure
The ebm training module is divided into five sub-modules: introduction, treatment studies, evidence syntheses and guidelines, diagnostic studies and systematic literature search.
The teaching goals are broadly defined (table 2).
The first module (two days of face-to-face training followed by one day of web-based training) aims to impart competences in searching for, selecting, critically appraising and extracting relevant literature according to the principles of ebm.
The second module is designed as a converted classroom scenario (one day of webbased training followed by one day of face-to-face training) (35) The CHC test comprises four different scenarios which allows testing at different time points using different scenarios. The test will be carried out at baseline and as part of the process evaluation after the training programme in IG and CG. Critical health literacy will be aggregated on group level using the maximum of individual values of the CHC test.
Outcomes and data collection

Primary Outcome
The primary outcome measure is the quality of health information produced by the participating providers. Quality is operationalised as the extent of adherence to the guideline's recommendations. The quality of this information will be rated using the MAPP'INFO instrument.
By that time, MAPP'INFO will have passed all the validation steps and the study protocol for the validation study will be in the process of publication. Information on reliability, internal consistency, construct validity, criterion validity and divergent validity will be available. The instrument has been developed to score the extent of compliance with the guideline criteria, observable based on the pure information or the directly linked sources; this means the transparency may be established by a report or documentation of the methods. MAPP'INFO is supposed to be the best possible estimate for the overall quality of health information. The instrument comprises 21 to 23 test items (according to the nature of the addressed decision) in four categories. Adherence to the criteria checked by MAPP'INFO is essential but not sufficient to provide material enhancing informed decisionmaking. Non-adherence with one of them may already challenge the entire information process. Therefore, all the criteria should be met. Nevertheless, the instrument provides percentage scores for each category and for each criterion. In this study, scores based on MAPP'INFO will be expressed as a percentage of the fully met criteria. Scores for information quality will be generated. MAPP'INFO coding works without previous training.
The information will be coded by two independent and blinded raters. DEVELOPMENT: "The health information target group has been involved in the development process using appropriate methods." All these secondary outcomes are binary.

Sample size
The sample size was calculated for a t-test comparing IG and CG regarding guideline compliance to EBHI as measured by MAPP'INFO. It is assumed that MAPP'INFO is normally distributed with the same standard deviation σ of 10% in both groups. Based on a sample size of 26 provider groups (13 in each study group) a quality improvement of 15% from 20% to 35% adherence can be detected with 90% power using a two-sided significance level of 5% after excluding at most 10% dropouts.
Sample size calculation is based on the assumption of σ = 10% from pretest data for MAPP'INFO, and the clinically relevant difference of 15% was chosen taking into account some pragmatic considerations starting from Cohen's d corresponding to a strong effect.
The power was calculated using the statistical software package SAS version 9.4 (PROC POWER).

Procedure
During recruitment, both groups will be informed about the requirement to produce a health information within ten months, according to the defined criteria.
In one personal meeting, information on the requirements for study participation will be given and the providers must agree to comply with these requirements. Written informed consent will be obtained from individual participants. Afterwards, baseline data will be assessed (t -1 ) and randomisation will be performed (t 0 ) (see SPIRIT- Figure, figure 3). In the IG, the workshop for the persons responsible (module 0) will be conducted and a timely convenient appointment for the training programme (module 1 and 2) will be made (t 1 ). The training should take place on-site at the provider's institution. If possible in terms of time and place, two or more groups may attend the training together. When the training is completed, they can begin with their production of the health information. The providers in the CG can start directly after the randomisation.
The information, produced within the study period, has to inform a decision in healthcare (e.g. therapy, diagnostic procedure, screening, prevention or rehabilitation). It addresses patients or medical laypersons and discusses at least two options regarding a specified health problem. Material regarding procedures (e.g. how to perform peritoneal dialysis at home) or the healthcare system, resources giving general advice for health and wellbeing as well as (drug) fact(s) boxes without additional text are not appropriate. Despite these requirements, the providers are free to choose the topic and target group of the information. Therefore, they can further develop already planned information. The providers may use external services (e.g. counselling by experts, graphic design, external performing of pilot tests or external literature searches), but they are responsible for the methods applied and for the content and design of the final information.
During the time of production, we will contact the providers every two months to keep ourselves informed about the progress and to ask about any unexpected difficulties. The study team will be accessible and will answer upcoming questions regarding requirements and the course of the study. The IG will have access to the training material on the online learning platform, including resources such as updated search strategies. We realise that the lack of access to relevant references is an important barrier in the development process and therefore the providers in the IG and CG will equally receive support regarding the acquisition of research articles. No further support will be provided.
Completed health information can be provided to the study team at any time. After ten months, the development process should be concluded and the providers will be asked for the material. A provider who cannot deliver after ten months may get a 2-month extension of the deadline. He should give a justification for the delay and a reasonable timeline for the following two months.
After twelve months (t 2 ), even unfinished material must be submitted. The information will be rated with regard to the primary outcome. Single criteria (or else the entire information) that do not meet the MAPP'INFO requirements, are coded as zero. Providers of both groups (IG and CG) who completed their information will be asked to give further information on the development process (t 3 ). Interviews will be conducted in the IG to explore barriers and facilitators of implementation (cf. process evaluation). The providers in the CG will be invited to make appointments for the training programme (t 3 ). After the provided information has been rated, providers may receive an individual feedback on their material. This is optional, but this way especially providers in the CG may benefit from such feedback.

Data management, protection and analyses
Data survey and data processing are used for the scientific purpose of the study only and are appropriate and indispensable for that purpose. Figure 4 shows data flow. Risk assessment according to §35 General Data Protection Regulation (DS-GVO) revealed a very low risk for the planned study. Special categories containing personal data will not be collected. In addition, the assessment using the Black List / White List of the competent supervisory authority of Saxony-Anhalt did not reveal the necessity for a privacy impact assessment. The remaining very low risk can be further minimised by the planned procedures (e.g. data storage using password-protected servers at the computer centre, no disclosure of data to third parties).
Withdrawal from the study is possible at any time without giving reasons and without disadvantages for the participants. In the case of withdrawal, data can be deleted only before anonymization (after completion of the analysis). Privacy and data protection will be taken into account according to the German Data Protection Act (37). A code number will be assigned to every participant. All data will be marked with this code number and thus pseudonymized. Identification of single participants is only possible with the coding key, which is stored on password-protected servers at the Martin Luther University Halle-Wittenberg. Only the project team has access to the coding key which is protected securely from third parties. Once data analysis has finished, the coding key will be deleted. From then on, data can no longer be deleted in the case of study withdrawal.
All data provided by the study participants are treated as absolutely confidential.
Researchers adhere to the data protection rules. All data will be stored on passwordprotected servers and there will be no disclosure to third parties. Publication and storage of data will only be performed in an anonymised form. Audio recordings will be deleted after transcription. Data will be stored at the Martin Luther University Halle-Wittenberg for 10 years.
The primary analyses will be performed from the intention-to-treat perspective. Provider's characteristics (institutional and individual level) will be analysed descriptively. Baseline variables of individuals will be described on individual level by means, standard deviations, percentiles, and frequency tables depending on their distributions. The IG and CG will be described separately.
The primary outcome will be analysed on provider level assuming approximately normal distribution by comparing means between the IG and CG using the t-test. The distribution of MAPP'INFO will be described graphically. In case of deviations from the normal distributions, transformations or alternative tests will be discussed as secondary sensitivity analyses. Frequencies of missing values will be described in detail. In the primary analysis, multiple imputations will be performed if enough data are available.
Unfinished information will not be regarded as missing values. For the binary secondary outcomes, relative frequencies will be described in both provider groups and compared using Fisher's exact test. A secondary analysis per protocol will be performed as a complete case analysis (without multiple imputations for the primary outcome).
A data monitoring committee will not be necessary, as the trial does not involve a highrisk intervention and participants do not belong to a vulnerable population. We do not expect adverse events or other unintended effects of the intervention. During the entire study period, participants will have the possibility to contact the study centre.

Process evaluation
To support future implementation of the guideline evidence-based health information and the training programme, a comprehensive analysis of the underlying processes of this complex intervention is indispensable (38). Barriers and facilitators of implementation should be assessed. Additionally, high quality of the training programme should be ensured. We will focus on parameters such as recruitment, reasons for participation or non-participation, intervention fidelity, structure and process-related factors, attitudes toward the intervention, response of individuals and organisations, and unintended consequences (38,39). Mixed methods will be applied (40) according to the MRC framework for process evaluation of complex interventions (38).
Structured documentation will be used to assess data of recruitment and intervention fidelity (e.g. recruitment process; numbers of institutions invited, responses and participants in each training session; location, time and duration of the training; completeness of modules and online tasks and reasons for deviations; and unexpected difficulties). Feasibility and acceptance of the training programme will be assessed at the end of the training sessions using structured feedback and all statements will be documented. Critical health literacy will be assessed using the CHC test (36) after the IG and CG have completed the training programme.
To map the development process and the methodological quality of the produced health information, all the providers (IG and CG) will be asked to provide the MAPP'INFO selfdeclaration after completing the health information. The self-declaration comprises freetext questions on the selection of the reported options and outcome measurements, search strategies, methods of data extraction and critical appraisal, reasons for the use of pictures, graphics, fact boxes, value clarification tools, animations and interactive tools.
Experts will review the results and if there are outstanding issues, interviews will be conducted. In addition, to identify influencing factors, resources and support used in the development process as well as any training attended during the study period will be assessed.
In the IG (theoretical sampling), semi-structured interviews will be conducted with single participants as well as with persons in charge in order to explore the implementation barriers and facilitators. Relevant factors for acceptance and usability of the educational contents and materials may be further assessed. In particular, the use of educational contents in daily routines and barriers against putting the guideline recommendations into practice will be explored. If the participant consents, interviews will be audio recorded and then transcribed.
Data will be collected at various points of time. An iterative process of collecting and analysing qualitative data will allow exploring unexpected aspects in further interviews (38). Data will be analysed in accordance with the method of collection (40). Descriptive statistics will be used for quantitative data. For qualitative data, a qualitative content analysis according to Mayring will be performed (40).

Discussion
The aim of the proposed trial is to evaluate the efficacy of the guideline evidence-based health information in combination with a training programme for providers of health information in comparison to having access to the guideline only. We expect the combined intervention to improve the quality of health information and thus promote informed patient decisions.
However, the training programme is extensive and time consuming; it does not only provide guidance on the application of the guideline but also extensive ebm knowledge.
That is because a previous qualitative study revealed the providers' shortcomings regarding their ebm competences. Participants must be released from their work duties in order to attend the training and in addition, the development of information in compliance with the guideline recommendations is time and cost-intensive, e.g. comprehensive literature searches must be conducted by the providers themselves or contracted out. The lack of access to databases or references can constitute a further barrier. These potential barriers may limit the study results because of problems in recruitment and study compliance (attending the training programme in person and online; producing information in accordance with the requirements). To address the barriers, the providers will receive comprehensive information on the study course and its requirements, incentives will be provided (publicity for providers of high quality information) and the providers will receive support for purchasing literature. The attitudes of the providers regarding EBHI will play a crucial role and therefore these attitudes will be addressed in The Ethical Principles for Medical Research Involving Human Subjects ( World medical association Declaration of Helsinki) will be respected in this project (43). Interested providers of health information (institutions and individuals) will receive detailed written information before participation. Written informed consent will be obtained from individual participants. The corresponding documents will have previously been revised by the ethic committee. The authors provide these documents on request. Withdrawal from the study is possible at any time without giving reasons and without disadvantages for the participants.

Not applicable
Availability of data and materials The study centre will coordinate the intra-study data sharing process. The data sets will be available for all the principal investigators.
After study completion, adjusted data will be stored and made publicly accessible via a special data repository to fulfil national (44) and international (45,46) requirements for sharing clinical trial data. For this purpose, the European Open Science Cloud (EOSC), a collaborative tool for data sharing and reuse, was recently implemented and is to serve all researchers from all research domains (47).

Competing interests
The authors declare that they have no competing interests. 1, 53227 Bonn, Germany. The funding institution will not interfere in any part of the study.

Authors' contributions
The principal investigator of the study is ASt. ASt, JL, BBH, JH and JK conceived the study.  Quality criteria regarding understandable language, accessibility of information cultural differences and issues related to age and gender of the users, due to et requirements.

Content requirements
The criteria on content and transparency were derived from the ethical guidelin the UK General Medical Councils (GMC), defining which information patients sho receive before they consent to medical interventions (5). In Germany, the requirements are legally anchored in the German Act on Patients' Rights (6).

Presentation of frequencies
Recommendations (n=5) for verbal and numerical presentation of frequencies.

Application of graphics
Recommendations (n=6) for the application and design of different types of graphics.

Application of pictures
Recommendations (n=5) for the application of anatomical pictures, cartoons, ph pictographs and drawings.

Application of narratives
Recommendation (n=1) for the application of narratives.

Application of value clarification tools
Recommendation (n=1) for the application of value clarification tools.

Formats
Recommendations (n=2) for the application of interactive tools and fact boxes. 2.2.9 Involvement of the target group Recommendation (n=1) for the involvement of the target group in the developm process of EBHI. • Participants start to consider EBHI as the prerequisite informed decision-making.
1.2 Treatment studies • Participants understand why association is not the sa causation and that RCTs are designed to establish a c relationship.
• Participants know the characteristics of RCTs.
• Participants are able to interpret the results of RCTs a critically appraise them.

Evidence syntheses and guidelines
• Participants are able to interpret the results and critic appraise systematic reviews and meta-analyses.
• Participants describe the development process of guid and are aware of their limitations.  Additional File 1_SPIRIT-Checklist.doc