Skip to main content


We're creating a new version of this page. See preview

  • Study protocol
  • Open Access
  • Open Peer Review

Practical health co-operation - the impact of a referral template on quality of care and health care co-operation: study protocol for a cluster randomized controlled trial

  • 1, 2Email author,
  • 2,
  • 2 and
  • 2, 3

  • Received: 29 May 2012
  • Accepted: 13 December 2012
  • Published:
Open Peer Review reports



The referral letter plays a key role both in the communication between primary and secondary care, and in the quality of the health care process. Many studies have attempted to evaluate and improve the quality of these referral letters, but few have assessed the impact of their quality on the health care delivered to each patient.


A cluster randomized trial, with the general practitioner office as the unit of randomization, has been designed to evaluate the effect of a referral intervention on the quality of health care delivered. Referral templates have been developed covering four diagnostic groups: dyspepsia, suspected colonic malignancy, chest pain, and chronic obstructive pulmonary disease. Of the 14 general practitioner offices primarily served by University Hospital of North Norway Harstad, seven were randomized to the intervention group. The primary outcome is a collated quality indicator score developed for each diagnostic group. Secondary outcomes include: quality of the referral, health process outcome such as waiting times, and adequacy of prioritization. In addition, information on patient satisfaction will be collected using self-report questionnaires. Outcome data will be collected on the individual level and analyzed by random effects linear regression.


Poor communication between primary and secondary care can lead to inappropriate investigations and erroneous prioritization. This study’s primary hypothesis is that the use of a referral template in this communication will lead to a measurable increase in the quality of health care delivered.

Trial registration

This trial has been registered at The trial registration number is NCT01470963


  • Cluster randomized trial
  • General practice
  • Quality of care
  • Referral


Quality of healthcare is defined by the American Institute of Medicine as ‘the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge’ [1]. The focus on prevention of medical errors and improved quality of medical care continues to increase [2, 3]. This focus is evident in Norway by the publication of a national strategy for quality improvement in health and social services [4], and the multitude of clinical guidelines available from governmental and professional sources.

The referral of a patient from a general practitioner (GP) to a hospital environment represents a transition of care, in which the major information exchange is through the written referral letter [5]. This transition of care represents an important step in the quality of the care process, and it has been shown that key clinical information may not be communicated adequately at the transition of care interface [6]. There has been considerable research into the quality of a referral and its impact on the process of care. A Norwegian study from 2007 amongst elderly patients demonstrated that both referral and discharge letters were missing vital information [7]. The consequences might represent a health hazard to older patients [7]. A Finnish assessment of the quality of referral letters for patients with asthma concludes that 45% of the referrals were of poor or unacceptable quality, based upon quality criteria developed by GPs and hospital respiratory consultants [8]. Others have also found varying quality and content of referrals [915].

Many studies have been designed for improving the referral process from GPs to the hospital [16, 17]. A recent Cochrane review on interventions to improve outpatient referrals from primary to secondary care concludes that surprisingly few interventions on the referral system have been rigorously evaluated. Many of the studies evaluated focused only on referral rates or referral quality. The review highlights the complexities of research in this area, especially as no single study managed to present findings on all aspects of the referral process (referral behavior, management of non-referred patients, secondary care management of patients, the flow of patients through the referral system, patient outcomes and satisfaction, and resource use). However, structured referral sheets and local education interventions have an impact on referral rates [18].

The primary objective of the present study is to evaluate whether the implementation of a referral template in the referral from general practice to hospital care can lead to a measurable increase in the quality of care delivered at the hospital. As secondary objectives, we will assess patient satisfaction and effective prioritization at secondary care.


Study design

This study is a cluster randomized trial, with the GP clinics as the cluster. The local GP clinics are randomly assigned either to use a referral template or to continue standard referral practice.


The 14 community GP practices in the area primarily served by University Hospital of North Norway Harstad (UNN Harstad) were included in the randomization process, with a total list size of 35,490 patients. In Norway, each individual has a regular GP. These GPs act as gatekeepers to secondary care. The health care system is relatively uniform throughout the country. In the study area, access to specialist care is practically impossible without a GP referral, whereas some access is possible in other areas of the country.

The study population will consist of patients referred to the medical department at UNN Harstad. The referrals received are, almost exclusively, electronic. Children (<18 years of age) and patients with reduced capacity to consent will be excluded from participation in the study.


The GP clinics were randomized stratified by location, to ensure adequate selection of cases and equal sociodemographic background data. Five of the centers are larger town-based centers and nine are smaller, more rural centers. The location of the center was not expected to influence the outcome variables. Initially, two centers approached declined the invitation to participate in the study, and therefore two additional GP clinics were randomly selected, as illustrated by Figure 1.
Figure 1
Figure 1

Flow chart of randomization process. ‡ From the four rural clinics initially randomized to the intervention group, two clinics refused. Therefore two additional rural clinics, from the five rural clinics initially randomized to the control group were randomized, and consented to, the intervention.


New patients referred to the medical department within one of the four diagnostic groups described below will receive written information and a consent form together with their appointment letter. These will be sent out by a clinic nurse unaware of the status of the GP center sending the referral (intervention or control). Patients will be orally reminded at the appointment and may be given a new consent form. This process is illustrated in Figure 2.
Figure 2
Figure 2

Recruitment process. Recruitment of patients in both the intervention and control groups follow the same procedure.


The referral templates have been developed based upon international literature [1927] and in collaboration with local specialists in the appropriate medical field. The templates have also been through a process of clinical assessment from subspecialists in other northern Norwegian hospitals. In acknowledging the problems in earlier studies with the uptake of referral interventions [17, 18] we have deliberately reduced the number of items in the referral templates, to ensure ease of uptake. Only information that the medical consultants thought imperative in the referral have been included as items in the templates. The study will implement referral guidelines for the following four diagnostic groups:

  • patients referred with dyspepsia;

  • patients referred with suspected colonic malignancy;

  • patients referred with chest pain;

  • patients referred with chronic obstructive pulmonary disease or suspected chronic obstructive pulmonary disease.

These diagnostic groups were chosen as they represent a substantial number of the referrals to a medical department (own data, 2008). They also represent a clear diagnostic challenge in primary care and are adept for simple referral guidelines.

The GPs at the intervention offices will use the referral template when initiating a new referral process for a patient. To ensure adequate uptake of intervention, the templates have been distributed as electronic templates as well as hard copies. The templates function as guidelines, but are not implemented as compulsory electronic checklists. The intervention referrals are sent to a separate electronic inbox at the hospital. The further evaluation and process of care has not been altered in the intervention group compared with the standard referral practice in the control group (Figure 3).
Figure 3
Figure 3

Referral pathway. Flow of referral and process of care in the intervention (blue arrows) and control group (red arrows).

In addition, questionnaires have been developed that assess patient experiences within the care framework, with one questionnaire designed for inpatients and one for outpatients. These questionnaires have been produced by combining questions from previously validated questionnaires regarding patient experiences in general and with transitional care. They include questions from a Norwegian patient experiences questionnaire [28]); two questions about health interaction from the Commonwealth Fund Survey 2010 [29]; the Care Transitions Measure 3 [30]; and demographic questions.

The questionnaires have been reviewed through an interview process with patients. This was done to ensure that the forms are acceptable to patients and to highlight possible issues that patients felt were missing from the questionnaires.


We aim to examine the impact of a referral template on the process of care, at the individual level. The primary hypothesis is that the use of a referral template in the communication between the GP and secondary care will lead to a measurable increase in the quality of health care delivered. Secondary hypotheses include that the use of a referral template in the communication between a GP and secondary care will lead to:
  • a measurable improvement in referral quality;

  • a change (up or down) in the number of patients defined as being in need of prioritization (as defined in national guidelines for prioritization in health care [20, 23, 24]);

  • more appropriate prioritization, as measured by final diagnosis;

  • an increase in the ‘appropriateness’ of the referrals (positive predictive value (PPV) of referral [31];

  • and better patient satisfaction, as measured by self-report questionnaires.


The primary outcome will be a quality indicator score compared between the intervention and the control group. The quality indicators have been generated from previous international quality assessment tools and national and international treatment guidelines. Some adaptation to locally accepted practice has been made, as demonstrated by others when quality indicators are used in a new context [32]. Each patient care process will be scored according to the criteria. Scoring will be done by a panel of specialists blinded to the intervention status of the patient. We will calculate the quality score as adherence scores (number of quality criteria met divided by number of applicable criteria expressed as a percentage) as illustrated by Ashton et al. [33]. If a criterion is applicable, but no information can be found (applicable, but not answerable), it will be noted ‘not met’ for statistical purposes [34]. Weighting of the criteria based upon clinical importance will not be used, as this adds complexity without adding much to the clinical findings, a finding discussed by Lyons and Payne in 1975 [35]. The scores will be compared between the intervention and control groups.

As secondary outcomes, the quality of the referrals will be evaluated against the developed referral template, to determine if the intervention has led to improved referral quality. It is important to measure referral quality, as the premise in the study is that more information will lead to improved care. In addition, health process outcomes such as waiting time from referral to appointment, number of appointments before a diagnosis is reached, time before treatment is initiated, the application or not of appointment prioritization, and the outcome of any given referral (appointment, return information or referral rejected) will be collected and compared between the groups. Bennett et al. used the PPV of a referral as a measure of the appropriateness of the referral [31]. In adapting this concept from glue ear in otolaryngology to a medical department we have defined it as the proportion of the GP referrals that result in a histological diagnosis, diagnostic clarification or change in medical management. We will assess and compare the PPV of referrals in the study groups. Patient experiences will be compared between the intervention and control group. Finally, the possible outcome diagnoses have been grouped according to severity. We will correlate the continuous outcome variable ‘waiting time’ with the grouped severity, to see whether the prioritization in the intervention group is more aligned with disease severity.

Sample size

Sample size was calculated based upon the primary outcome, and the initial calculation was done without regard for clustering. A change in the quality score of 10% was determined to be clinically interesting. Setting the significance level at 0.05 and using PASS 2008 (NCSS, LLC, Kaysville, UT, USA) for the calculation provides that the study would require 855 patients in the control and 855 in the intervention group, for an 80% power to detect a 10% difference between the groups.

To correct for clustering, the design effect (DE) may be calculate as:

DE = 1 + ρ(m-1)

where ρ denoted the intracluster correlation coefficient (ICC) and m is the size of each cluster [36]. No ICC for equivalent designs was identified from literature searches. Reviewing primary care-based interventions from the literature [3739], an expected ICC ranging from 0.001 to 0.08 does not seem improbable, giving a DE of 1.15 and 12.9, respectively, for a cluster size of 150 patients. Because only 14 GP clinics were available for randomization, further inflation of the number of clusters to achieve higher power was not possible, although this could have been advisable [40].

Based upon a review of patient data at UNN Harstad from 2008, the study is expected to achieve this relatively high inclusion number by recruiting over a two-year period (personal data).


The referring GP cannot be blinded to the trial, as the intervention is actively used by the GP. The patient will not be aware of the intervention, but no active effort has been made to keep the patients blinded. The patients will be mixed with the general caseload to avoid bias in the treatment process at the hospital. For the GPs that use the electronic referral template, this usage will be visible to the hospital doctor in the presentation of the referral letter on the computer, but for the majority of the cases the hospital doctor will be blinded to the intervention status of the patient. The outcome assessors will be blinded.

Data gathering

Data will be extracted by both automated computer reports (for example, waiting times, number of appointments) and manual chart review (for example, PPV, group of final diagnosis). Data will be collected after the process of care that the referral initiated is completed.

Statistical methods

We will collect the following baseline characteristics:

  • patient age (mean and confidence interval) and sex (number and percentage)

  • practice list size (median and interquartile range, or mean and confidence interval if normally distributed)

  • referral type - electronic or paper (number and percentage)

  • referred by - GP or other doctor (number and percentage).

For the primary outcome (quality score), we will calculate adherence scores as described above and compare between treatment arms. We expect substantial variation in cluster size. Because of the small number of clusters, analysis based upon the cluster level was considered [41]. However, as there is no prior accurate estimate of variation between clusters, weighting for cluster size could not be achieved [41]. To offer increased precision and take into account between-cluster variation, random effects linear regression will be used [42] to generate estimates of intervention effect. It has been suggested that this can be used for studies with as few as 10 clusters [36]. The estimated effect and confidence intervals will be reported. A P-value <0.05 will be regarded as statistically significant. Intention to treat analysis will be employed.

The referrals will be scored using a simple scoring system related to the referral templates. Each unit of information specified in the referral template (for example, presence of weight loss specified) will provide one point in the scoring system, with no weighting applied. Scores will be compared between the groups as noted above.

For outcome severity, random effects linear regression will be used, with the severity group score as a categorical variable, and the relationship compared between the intervention and control group.

In the questionnaire, answers noted as ‘not applicable’ or no answer will be counted as missing data. The questionnaire will be scored according to a pre-set scoring system. Scores will be analyzed using the regression technique outlined above. The data will also be analyzed to determine if factors such as self-perceived health, age, gender, and education level have an impact on patient experience.

The trial will be reported according to the CONSORT standards for reporting cluster randomized trials [43].

Pilot study

No pilot study has been carried out. To ensure acceptability of the intervention, GPs were invited to, and participated in, the development of the referral template. To ensure feasibility, the authors have collected all data specified in the protocol from the 20 patients included first. To ensure an adequate uptake of the intervention, regular reviews of all referrals received at UNN Harstad will be undertaken.


The study will follow the directions in the Helsinki Declaration, and was presented to the Regional Ethical Committee for Medical Research in northern Norway, who determined it not to be within the scope of the Health Research Act (REK NORD 2010/2259). The project has been approved by the Data Protection Official for Research. The study is registered at The trial registration number is NCT01470963. All patients must provide written informed consent.


Transitions of care represent a point of frequent adverse events [44, 45]. The referral is the main form of communication in the transition from primary to secondary care [13, 46]. Although many referral interventions have been evaluated, there appears to be limited knowledge on how the referral letters affect specialist care. A recent study protocol describes a similar project within mental health care [47], although this study is still ongoing. The primary objective of our study is to assess whether an improved referral letter will lead to a measurable change in the quality of care delivered in a medical department. The aim is to go beyond an assessment of referral quality and waiting times per se, and evaluate quality of care and the appropriateness of waiting times and treatment, and, as such, help fill parts of the knowledge gap identified in a Cochrane review on the subject [18].

However, research at the interface between primary and secondary care can be challenging [18, 48]. The choice of using an intervention with intuitive content was made to make it acceptable in normal general practice. The assumption underlying this research project is that a referral guideline will increase the amount of information available to the hospital specialist, and that this increase in information will translate into better care.

The cluster design was chosen because randomization with this approach is well suited for interventions implemented at the level of the health care organizational unit [42]. In addition, randomizing at the individual patient level would undermine findings, as the GPs could use the information learned from the referral template in their non-intervention referrals and as such contaminate the data. For similar reasons, the GP clinic, as opposed to the individual GP, was randomized, as contamination between GPs in the same clinic was to be expected.

In choosing a cluster randomized design, we have a design that is less statistically efficient than a standard randomized design. A recent study involving a more complex intervention [49] used randomization at the patient level to avoid this problem. We feel that the dangers of contamination with individual randomization in our design would be so large that results would be difficult to interpret.

In cluster randomized trials, post randomization bias has been identified as a concern [40]. This entails the recruitment of different cohorts in the intervention and control groups, as the patients are recruited after randomization of the clusters. We hope to reduce this by actively recruiting the patients (obtaining signed consent) in conjunction with the hospital appointment, both for the intervention and control groups.

The intervention is intentionally simple to ensure that an effect seen can be attributed to the intervention. However, an intervention at an interface in a complicated health care system can quickly affect the entire process, in ways we have not yet envisaged.

There is also a risk of performance bias as systematic differences in the care may not be due to the intervention, but rather because the doctors will be aware of the study protocol. By ensuring the mixing of cases in normal workloads and, as much as possible, blinding the doctors involved, we hope to minimize this bias. The fact that the care process is being studied may change the behavior of the doctors in general, akin to a Hawthorn effect. This will potentially attenuate the effects of the intervention, as the quality of care may improve for both intervention and control patients.

The authors also recognize that many referrals from primary to secondary care are not made only to identify major pathology. Referrals are also made to reassure the patient, reduce medico-legal risk, obtain a second opinion or for handing over of care [50]. The authors fully appreciate these as valid reasons for referral. We, therefore, do not aim to reduce the number of referrals, but rather assess the effect on hospital care of improved referrals.

This study aims to add to the knowledge regarding the effect of the referral on the patient pathway and quality of care. Simple diagnostic groups have been chosen. If the study can identify benefits from improving referrals in these areas, this may lead the way to further implementations of referral proforma, preferably electronically integrated into the standard software packages. This could improve the overall referral process to enable better care and effective prioritization based upon the need of the individual patients.

Trial status

The study began including patients in the fall of 2011 and inclusion is planned for approximately two years.



general practitioner


positive predictive value.



The study is funded by the Northern Norway Regional Health Authority. The funding source did not have any role in designing the study or writing the manuscript.

Authors’ Affiliations

Department of Community Medicine, Faculty of Health Sciences, University of Tromsø, 9037 Tromsø, Norway
University Hospital of North Norway Harstad, St. Olavsgate 70, 9480 Harstad, Norway
Centre for Sami Health Research, University of Tromsø, 9037 Tromsø, Norway


  1. Lohr KN: Medicare: A Strategy for Quality Assurance, Volume I. 1990, Washington, DC: The National Academy PressGoogle Scholar
  2. Institue of Medicine: To Err is Human: Building a Safer Health System. 2000, Washington, DC: National Academy PressGoogle Scholar
  3. Varkey P, Reller MK, Resar RK: Basics of quality improvement in health care. Mayo Clinic Proceedings. 2007, 82: 735-739.View ArticlePubMedGoogle Scholar
  4. The Norwegian Directorat of Health: and it’s going to get better! National Strategy for Quality Improvement in Health and Social Services (2005–2015). 2005, Oslo: The Norwegian Directorat of HealthGoogle Scholar
  5. Piterman L, Koritsas S: Part II. General practitioner-specialist referral process. Intern Med J. 2005, 35: 491-496. 10.1111/j.1445-5994.2005.00860.x.View ArticlePubMedGoogle Scholar
  6. Coleman EA: Falling through the cracks: challenges and opportunities for improving transitional care for persons with continuous complex care needs. J Am Geriatr Soc. 2003, 51: 549-555. 10.1046/j.1532-5415.2003.51185.x.View ArticlePubMedGoogle Scholar
  7. Garasen H, Johnsen R: The quality of communication about older patients between hospital physicians and general practitioners: a panel study assessment. BMC Health Serv Res. 2007, 7: 133-10.1186/1472-6963-7-133.View ArticlePubMedPubMed CentralGoogle Scholar
  8. Tuomisto LE, Erhola M, Kaila M, Brander PE, Kauppinen R, Puolijoki H, Kekki P: The Finnish national asthma programme: communication in asthma care–quality assessment of asthma referral letters. J Eval Clin Pract. 2007, 13: 50-54. 10.1111/j.1365-2753.2006.00645.x.View ArticlePubMedGoogle Scholar
  9. Ong SP, Lim LT, Barnsley L, Read R: General practitioners’ referral letters - do they meet the expectations of gastroenterologists and rheumatologists?. Aust Fam Physician. 2006, 35: 920-922.PubMedGoogle Scholar
  10. Dentith GE, Wilson KE, Dorman M, Girdler NM: An audit of patient referrals to the sedation department of Newcastle Dental Hospital. Prim Dent Care. 2010, 17: 85-91. 10.1308/135576110791013857.View ArticlePubMedGoogle Scholar
  11. Syed AA, Large DM: Quality of GPs’ referral letters to diabetes secondary care. Practical Diabetes International. 2003, 20: 165-169. 10.1002/pdi.488.View ArticleGoogle Scholar
  12. Newton J, Hutchinson A, Hayes V, McColl E, Mackee I, Holland C: Do clinicians tell each other enough? An analysis of referral communications in two specialties. Fam Pract. 1994, 11: 15-20. 10.1093/fampra/11.1.15.View ArticlePubMedGoogle Scholar
  13. Shaw I, Smith KM, Middleton H, Woodward L: A letter of consequence: referral letters from general practitioners to secondary mental health services. Qual Health Res. 2005, 15: 116-128. 10.1177/1049732304270725.View ArticlePubMedGoogle Scholar
  14. Jacobs LG, Pringle MA: Referral letters and replies from orthopaedic departments: opportunities missed. BMJ. 1990, 301: 470-473. 10.1136/bmj.301.6750.470.View ArticlePubMedPubMed CentralGoogle Scholar
  15. Bodek S, Ghori K, Edelstein M, Reed A, MacFadyen RJ: Contemporary referral of patients from community care to cardiology lack diagnostic and clinical detail. Int J Clin Pract. 2006, 60: 595-601. 10.1111/j.1368-5031.2006.00902.x.View ArticlePubMedGoogle Scholar
  16. Thomas RE, Grimshaw JM, Mollison J, McClinton S, McIntosh E, Deans H, Repper J: Cluster randomized trial of a guideline-based open access urological investigation service. Fam Pract. 2003, 20: 646-654. 10.1093/fampra/cmg605.View ArticlePubMedGoogle Scholar
  17. Jiwa M, Skinner P, Coker AO, Shaw L, Campbell MJ, Thompson J: Implementing referral guidelines: lessons from a negative outcome cluster randomised factorial trial in general practice. BMC Fam Pract. 2006, 7: 65-10.1186/1471-2296-7-65.View ArticlePubMedPubMed CentralGoogle Scholar
  18. Akbari A, Mayhew A, Al-Alawi MA, Grimshaw J, Winkens R, Glidewell E, Pritchard C, Thomas R, Fraser C: Interventions to improve outpatient referrals from primary care to secondary care. Cochrane Database Syst Rev. 2008, 4: cd005471-PubMedGoogle Scholar
  19. Jiwa M, Mathers N, Walters S: Quality of information on referrals to colorectal surgeons: towards consensus. Curr Med Res Opin. 2002, 18: 72-77. 10.1185/030079902125000309.View ArticlePubMedGoogle Scholar
  20. Nygaard E, Kårikstad V: Prioriteringsveileder – Fordøyelsessykdommer (Prioritization guidelines – digestive diseases). 2009, Oslo: HelsedirektoratetGoogle Scholar
  21. Jiwa M, Coleman M, McKinley RK: Measuring the quality of referral letters about patients with upper gastrointestinal symptoms. Postgrad Med J. 2005, 81: 467-469. 10.1136/pgmj.2004.027516.View ArticlePubMedPubMed CentralGoogle Scholar
  22. British Thoracic Society Standards of Care Committee: BTS statement on criteria for specialist referral, admission, discharge and follow-up for adults with respiratory disease. Thorax. 2008, 63: 16-View ArticleGoogle Scholar
  23. Nygaard E, Kårikstad V: Prioriteringsveileder – Lungesykdommer (Prioritization guidelines – respiratory diseases). 2008, Oslo: HelsedirektoratetGoogle Scholar
  24. Nygaard E, Kårikstad V: Prioriteringsveileder - Hjertemedisinske sykdommer (Prioritization guidelines – cardiac diseases). 2009, Oslo: HelsedirektoratetGoogle Scholar
  25. Snow V, Barry P, Fihn SD, Gibbons RJ, Owens DK, Williams SV, Weiss KB, Mottur-Pilson C: Evaluation of primary care patients with chronic stable angina: guidelines from the American College of Physicians. Ann Intern Med. 2004, 141: 57-64.View ArticlePubMedGoogle Scholar
  26. Malm S, Schirmer H: Hjertesak: riktigere prioritering av pasienter (Heart issue: more correct prioritization of patients). Fastlegenytt (2). 2010, 0012: 3-6.Google Scholar
  27. Erhardt L, Herlitz J, Bossaert L, Halinen M, Keltai M, Koster R, Marcassa C, Quinn T, van Weert H: Task force on the management of chest pain. Eur Heart J. 2002, 23: 1153-1176. 10.1053/euhj.2002.3194.View ArticlePubMedGoogle Scholar
  28. Sjetne IS, Bjertnæs ØA, Iversen HH, Olsen RV: Pasienterfaringer i spesialisthelsetjenesten. Et generisk, kort spørreskjema (Patient experiences in specialist health care: A short generic questionnaire). 2009, Oslo: Norwegian Knowledge Centre for the Health ServicesGoogle Scholar
  29. Skudal KE, Bjertnæs ØA, Holmboe O, Bukholm G, Rottingen JA: The 2010 Commonwealth Fund survey: Results from a comparative population survey in 11 countries. 2010, Oslo: Norwegian Knowledge Centre for the Health ServicesGoogle Scholar
  30. Parry C, Mahoney E, Chalmers SA, Coleman EA: Assessing the quality of transitional care: further applications of the care transitions measure. Med Care. 2008, 46: 317-322. 10.1097/MLR.0b013e3181589bdc.View ArticlePubMedGoogle Scholar
  31. Bennett K, Haggard M, Churchill R, Wood S: Improving referrals for glue ear from primary care: are multiple interventions better than one alone?. J Health Serv Res Policy. 2001, 6: 139-144. 10.1258/1355819011927387.View ArticlePubMedGoogle Scholar
  32. Marshall MN, Shekelle PG, McGlynn EA, Campbell S, Brook RH, Roland MO: Can health care quality indicators be transferred between countries?. Qual Saf Health Care. 2003, 12: 8-12. 10.1136/qhc.12.1.8.View ArticlePubMedPubMed CentralGoogle Scholar
  33. Ashton CM, Kuykendall DH, Johnson ML, Wray NP: An empirical assessment of the validity of explicit and implicit process-of-care criteria for quality assessment. Med Care. 1999, 37: 798-808. 10.1097/00005650-199908000-00009.View ArticlePubMedGoogle Scholar
  34. Ashton CM, Kuykendall DH, Johnson ML, Wun CC, Wray NP, Carr MJ, Slater CH, Wu L, Bush GR: A method of developing and weighting explicit process of care criteria for quality assessment. Med Care. 1994, 32: 755-770. 10.1097/00005650-199408000-00001.View ArticlePubMedGoogle Scholar
  35. Lyons TF, Payne BC: The use of item importance weights in assessing physician performance with predetermined criteria indices. Med Care. 1975, 13: 432-439. 10.1097/00005650-197505000-00005.View ArticlePubMedGoogle Scholar
  36. Eldridge S, Kerry S: A Practical Guide to Cluster Randomised Trials in Health Services Research. 2012, Chichester, UK: John Wiley & Sons, LtdView ArticleGoogle Scholar
  37. Smeeth L, Ng ES-W: Intraclass correlation coefficients for cluster randomized trials in primary care: data from the MRC Trial of the Assessment and Management of Older People in the Community. Control Clin Trials. 2002, 23: 409-421. 10.1016/S0197-2456(02)00208-8.View ArticlePubMedGoogle Scholar
  38. Cosby RH, Howard M, Kaczorowski J, Willan AR, Sellors JW: Randomizing patients by family practice: sample size estimation, intracluster correlation and data analysis. Fam Pract. 2003, 20: 77-82. 10.1093/fampra/20.1.77.View ArticlePubMedGoogle Scholar
  39. Adams G, Gulliford MC, Ukoumunne OC, Eldridge S, Chinn S, Campbell MJ: Patterns of intra-cluster correlation from primary care research to inform study design and analysis. J Clin Epidemiol. 2004, 57: 785-794. 10.1016/j.jclinepi.2003.12.013.View ArticlePubMedGoogle Scholar
  40. Medical Research Council: Cluster Randomised Trials: Methodological and Ethical Considerations. 2002, London: Medical Research CouncilGoogle Scholar
  41. Hayes JH, Moulton LH: Cluster Randomised Trials. 2009, Boca Raton: Chapham & Hall/CRCView ArticleGoogle Scholar
  42. Ukoumunne OC, Gulliford MC, Chinn S, Sterne JA, Burney PG: Methods for evaluating area-wide and organisation-based interventions in health and health care: a systematic review. Health Technol Assess. 1999, 3: 92-Google Scholar
  43. Campbell MK, Elbourne DR, Altman DG: CONSORT statement: extension to cluster randomised trials. BMJ. 2004, 328: 702-708. 10.1136/bmj.328.7441.702.View ArticlePubMedPubMed CentralGoogle Scholar
  44. Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW: The incidence and severity of adverse events affecting patients after discharge from the hospital. Ann Intern Med. 2003, 138: 161-167.View ArticlePubMedGoogle Scholar
  45. Richard IC, Marta R, David DW: Gaps in the continuity of care and progress on patient safety. BMJ. 2000, 320: 791-794. 10.1136/bmj.320.7237.791.View ArticleGoogle Scholar
  46. Newton J, Eccles M, Hutchinson A: Communication between general practitioners and consultants: what should their letters contain?. BMJ. 1992, 304: 821-824. 10.1136/bmj.304.6830.821.View ArticlePubMedPubMed CentralGoogle Scholar
  47. Hartveit M, Biringer E, Vanhaecht K, Haug K, Aslaksen A: The Western Norway mental health interface study: a controlled intervention trial on referral letters between Primary Care and Specialist Mental Health Care. BMC Psychiatry. 2011, 11: 177-10.1186/1471-244X-11-177.View ArticlePubMedPubMed CentralGoogle Scholar
  48. Faulkner A, Mills N, Bainton D, Baxter K, Kinnersley P, Peters TJ, Sharp D: A systematic review of the effect of primary care-based service innovations on quality and patterns of referral to specialist secondary care. Br J Gen Pract. 2003, 53: 878-884.PubMedPubMed CentralGoogle Scholar
  49. Augestad K, Revhaug A, Vonen B, Johnsen R, Lindsetmo RO: The one-stop trial: does electronic referral and booking by the general practitioner (GPs) to outpatient day case surgery reduce waiting time and costs? A randomized controlled trial protocol. BMC Surgery. 2008, 8: 14-10.1186/1471-2482-8-14.View ArticlePubMedPubMed CentralGoogle Scholar
  50. Coulter A, Noone A, Goldacre M: General practitioners’ referrals to specialist outpatient clinics. I. Why general practitioners refer patients to specialist outpatient clinics. BMJ. 1989, 299: 304-306. 10.1136/bmj.299.6694.304.View ArticlePubMedPubMed CentralGoogle Scholar


© Wåhlberg et al.; licensee BioMed Central Ltd. 2013

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.


By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate. Please note that comments may be removed without notice if they are flagged by another user or do not comply with our community guidelines.