Skip to main content

Effectiveness of a computerized drug-monitoring program to detect and prevent adverse drug events and medication non-adherence in outpatient ambulatory care: study protocol of a randomized controlled trial



Medications are an effective intervention for managing and preventing health problems but their benefit can be undermined by non-adherence or adverse drug events (ADEs). Since these issues may be interconnected, efforts to improve non-adherence should also include reduction of ADEs. We have developed the ISTOP-ADE system (Information Systems-enabled Outreach for Preventing Adverse Drug Events), which enables timely monitoring and managing of ADEs. The objectives of this study are to determine whether the ISTOP-ADE system, compared to routine care, will reduce: a) the probability of discontinuing the use of prognosis-altering medications; b) the probability of a patient experiencing a severe ADE; c) the proportion of patients experiencing ADEs, preventable ADEs and ameliorable ADEs; and d) health services utilization.


We will randomly assign 2,200 adult ambulatory patients in the province of Québec who have been prescribed an incident medication for the management or prevention of a chronic health condition, to routine care or the ISTOP-ADE system. The ISTOP-ADE system consists of an interactive voice response system (IVRS) paired with pharmacist support. The IVRS will call patients at 3 and 17 days post-prescription to determine if they are experiencing any problems and connect them with a pharmacist when required or desired by the patient. We will evaluate medication persistence at 180 days and health-care utilization using provincial administrative data. Two blinded physicians will ascertain ADE status through a case review.


We expect the ISTOP-ADE intervention to be feasible and to improve the quality of patient care through improved medication adherence, reduced ADE duration and reduced number of ADEs resulting in an emergency department or inpatient encounter. This in turn could lower health-care utilization, saving costs and lowering the burden on emergency departments and family practices. The success of ISTOP-ADE would present opportunities to implement this intervention through health systems, health insurance agents and commercial pharmacies.

Trial registration Identifier: NCT02059044. Date registered: 10 January 2014.

Peer Review reports


Medications are an effective intervention for managing and preventing many acute and chronic health-care problems. Their benefit is commonly reduced by non-adherence, in which patients either do not take or stop taking the medication as prescribed, or by adverse drug events (ADEs), in which patients experience side effects of the medications. These issues are interrelated. Some patients do not take their medication due to their fear of an impending ADE, while others stop taking the medication because of an ADE. Therefore, efforts to increase adherence should include efforts to target ADE reduction specifically.

To address these concerns, we have developed the Information Systems-enabled Outreach for Preventing ADEs (ISTOP-ADE) system (protocol version 1, 1 March 2012). The ISTOP-ADE system consists of a set of business processes supported by a technology infrastructure. The information technology is designed to connect providers to patients and includes an electronic health record system, a clinical data repository and an interactive voice response system (IVRS). This system enables timely monitoring of patients for ADEs and couples it with an effective arm to manage ADEs if, and when, they occur.

We recently established the feasibility and acceptability of the ISTOP-ADE system in a prospective cohort study. The pilot study included 628 patients who were prescribed an incident prescription. The ISTOP-ADE system successfully contacted 90% of patients. Important measures of the system’s effectiveness included the identification of 45% of ADEs, 50% of all potential ADEs and 30% of primary non-adherence episodes. In terms of potential benefits, it positively influenced the management of 42% of all medication-related problems. In terms of acceptability, almost all patients found it easy to use and understand and reported they would use the system again if it was provided to them. Physicians and pharmacists also supported the system [1, 2]. While these results are favorable, a controlled clinical trial in which patients are randomly assigned to the ISTOP-ADE system or routine care is required to determine the system’s effectiveness. We therefore designed this study.


This study has been approved by the McGill Institutional Review Board (A01-B02-02A) and the Ottawa Health Science Network Research Ethics Board (20130285-01H).


This research will occur during ambulatory care in the province of Québec, Canada. We will study prescriptions in patients receiving primary care from a selected group of physicians using the Medical Office of the 21st Century (MOXXI) electronic health record system. MOXXI was developed by the McGill University’s Clinical and Health Informatics Research Group. It is currently used by over 100 primary care physicians in Montréal and Québec City. The MOXXI system is linked to the provincial insurance company, Régie de l’assurance maladie du Québec (RAMQ), resulting in an electronic medical record that displays a drug profile of prescribed as well as dispensed drugs, emergency department visits and hospitalizations in the past 12 months, a list of active and past health problems, and an electronic prescribing tool. Data stored by MOXXI have been used by the McGill research team for medical research in the past [3, 4].

Primary objective

The primary objective is to determine whether the ISTOP-ADE system, compared to routine care, will reduce the probability of discontinuing the use of prognosis-altering medications in patients prescribed an incident medication to manage or prevent a chronic health condition.

Secondary objectives

  1. a.

    Determine whether the ISTOP-ADE system, compared to routine care, will reduce the probability of a patient experiencing a severe ADE (defined as an ADE lasting at least 7 days or causing an emergency department visit).

  2. b.

    Determine whether the ISTOP-ADE system, compared to routine care, will reduce the proportion of patients experiencing ADEs, preventable ADEs and ameliorable ADEs.

  3. c.

    Determine whether the ISTOP-ADE system, compared to routine care, will reduce health services utilization (comprising physician visits, emergency department encounters and hospitalizations) at 12 months following an incident prescription.

  4. d.

    Measure the costs associated with the ISTOP-ADE system.


This study will include French- and English-speaking adult patients (age >18) who receive a high-risk incident prescription, use RAMQ insurance to pay for medications and are followed by a physician who has consented to be in the MOXXI research network. We will exclude patients if their physicians do not deem them appropriate for study inclusion.

We define high-risk prescriptions as medications commonly used to prevent progression or complications related to the following conditions: congestive heart failure, hypertension, dyslipidaemia, type 2 diabetes mellitus, coronary artery disease, cerebrovascular disease, atrial fibrillation, chronic renal failure, chronic obstructive lung disease, depression, dementia, autoimmune conditions, seizures, and chronic pain. The selected medications (Table  1) have an important impact on outcomes or they have been shown to have a high frequency of ADEs [5, 6]. We will consider prescriptions as ‘incident’ if the patient had not received a prior prescription for that medication in the 12 months preceding the index event.

Table 1 Included diseases and drug classes for the ISTOP-ADE study

Eligibility assessment and consent

When a physician uses MOXXI to prescribe a new medication for a patient, a pop-up window will appear on their computer screen to indicate the patient’s eligibility. This pop-up window will prompt the doctor to inform the patient of the study, confirm their eligibility and seek consent. We will obtain informed consent from each participant. If the patient consents, the physician will confirm the patient’s phone number, preferred language and preferred time of day to be called. If the physician deems the patient ineligible or the patient does not consent, then the reason will be captured in MOXXI using a standard classification.


The unit of analysis for the study is the individual patient, each of whom we will randomly assign to the intervention or control arms using a computer algorithm. We will perform stratified randomization at the physician practice level with block sizes of four. We will only randomize patients once a physician confirms the patient’s eligibility for enrolment. We will update the ISTOP-ADE system with the patient’s treatment group but this information will not be available to the prescribing physician or the patient.


The intervention consists of an IVRS call coupled with pharmacist support. We will program the IVRS to call patients automatically on day 3 and day 17 following their prescription. If a call is not answered, the system will make as many as eight separate attempts (two per day over a maximum of 4 days) before marking the call as unsuccessful. If a call is answered, the IVRS will proceed through a calling algorithm. The following questions will be asked at each call:

  1. 1.

    Since receiving your medication, have you started taking it?

  2. 2.

    Since receiving your medication, have you had any problems?

  3. 3.

    Since receiving your medication, have you developed any new symptoms?

  4. 4.

    Would you like to speak to a pharmacist?

If the patient answers ‘No’ to the first question or ‘Yes’ to any of the three remaining questions, the IVRS will inform the patient that a pharmacist will personally contact them within 2 business days. The IVRS will send an email to a pharmacist informing them of the patient’s request. In addition, the IVRS will be capable of receiving inbound calls from patients regarding their medications at any time. This will also connect the patient to the pharmacist according to the same business rules as for the outbound IVRS calls.

When required, the pharmacist will contact the patient. Prior to contact, the pharmacist will access the electronic health record to obtain background information on the patient’s health status and medication use. During the call, the pharmacist will document the patient’s problems as well as their recommendation for resolving the problem in the electronic health record. If the pharmacist deems the patient’s problems are significant, then the pharmacist can escalate the issue to the patient’s primary care physician using the electronic health record.


The control arm consists of routine care.


Interview at 21 days

We will perform a standard interview of patients at 21 days to determine the development of new symptoms, medication use and health services use since their prescription. The study personnel performing the interview will not have access to knowledge regarding the study group to which the patient is allocated.

Administrative database interrogation

We will link the study database to the drug benefit and hospital record files contained at RAMQ using the patient’s provincial health insurance number.

Outcome assessment

Primary outcome: medication persistence at 180 days

We define persistence as ‘having a continuous supply of at least one medication used to treat the index condition, with gaps no greater than 90 days, in the first 6 months of treatment’. We will use validated methods to measure medication persistence using administrative claims data on prescription refills [711]. Prescription refills provide a good proxy measure of medication use because refill rates are strongly correlated with targeted clinical outcomes such as blood pressure [7]. We will construct medication dispensing histories for all patients and medications during the study follow-up period using the RAMQ medication file.

Secondary outcome: ADEs within 21 days

We will use previously developed methods to ascertain ADE status [1217]. Specifically, a clinician will create a case summary describing the post-prescription course for each patient experiencing a problem. Problems will be identified during the 21-day interview. The clinician will compile information derived from the post-intervention interview and a review of the patient’s MOXXI record using a standardized format. The summary will elaborate on the patient’s underlying health conditions, the medications currently being used (including the index prescription), the timing of the problem, the management of the problem, the timing of the resolution and what the patient was told about the cause of the problem.

To define ADE status, two physicians (blinded to allocation group) will review each case summary and determine whether the case represents an ADE. This determination is made using a six-point classification system with a cut-point of 3. A case is considered to be an ADE if both reviewers agree that the case was an undesirable outcome and that outcome was more likely caused by medication use than the patient’s underlying health condition. This method has been widely adopted as the standard approach for ADE classification in patient safety research [1217]. For an ADE, the reviewers will further classify its duration, preventability, amelioration and severity using standard definitions [1219].

Health services utilization

Standard methods to measure health services use using provincial claims data will be used. All physician visits, emergency department visits, hospitalizations and medication claims during the 12 months following index prescription will be identified using the RAMQ data through MOXXI.

Intervention costs

We will record the entire amount of time the study pharmacist spends supporting patients in the intervention group. This will allow us to calculate costs at the patient level as well as aggregated at the population level. We will also track implementation and support costs for the IVRS, including calling charges. These costs will be prorated across the study population.

Explanatory variables

To determine factors that influence outcomes, we will use data in the MOXXI system to derive baseline information on patients, including demographic information (age, gender, language of choice and location of residence), active and past medical diagnoses and medication use in the preceding year (number of new drug starts and adherence).


Due to the nature of the study, blinding of patients and health-care providers will not be possible. However, we will blind personnel performing the interviews and physicians performing the assessment of ADE status of study group allocation.

Sample size

We have based our sample size on the following assumptions:

  1. a.

    A baseline probability of non-persistence of 25% (based on a prior research study evaluating persistence with anti-hypertensive therapy using the same methods we will for this study) [11]

  2. b.

    A minimally clinical important difference of 20%

  3. c.

    A 10% loss to follow-up (based on our pilot study)

  4. d.

    An enrollment rate of 50% (based on our pilot study)

  5. e.

    Other specifications: α = 0.05, β = 0.20 and two-sided tests for significance will be used

Given these assumptions, we will need to approach 4,708 patients in total (2,354 patients in each treatment arm being eligible and approached for enrollment). This will result in 2,354 patients being enrolled (1,177 patients in each arm) and 2,118 patients evaluated for the composite outcome (1,059 patients in each arm). To be more conservative, we will enroll 1,100 patients in each arm.

Data management and analysis

Data will be exported from the MOXXI system and analyzed using SAS software (Cary, NC, USA). All data analysis will be performed in secure physical locations using hardware that is password protected. Only study personnel will have access to the data. Simple descriptive statistics to describe the patient population, call statistics and outcome measures will be used. Continuous variables will be described using median and inter-quartile ranges, while categorical variables will be described using frequency distributions. The statistical significance of any associations between group assignment and measures will be assessed using the Wilcoxon rank sum test and or the chi-square test, respectively, for continuous and categorical variables. Since the primary outcome is binary, tests to compare outcomes between the intervention and control groups will be simple chi-square statistics. An alpha of 0.05 as a threshold to determine statistical significance using a two-tailed distribution will be applied.

As a sensitivity analysis, logistic regression modeling will be used to assess possible baseline factors that are associated with the outcome and differ between the intervention and control groups. Analogous analyses to assess significance for the individual components of the composite primary outcome and the binary secondary outcome variables will also be performed (ADE occurrence, preventable ADE occurrence and ameliorable ADE occurrence).

To compare health services utilization, since the outcomes are event data, which follow a Poisson distribution, incidence rates for the outcome will be calculated. Incidence rates will be calculated using PROC GENMOD to create a t-test. Multivariable Poisson regression will be used to compare the rates, while adjusting for baseline factors. The model will use the treatment group as an independent covariate. If over-dispersion is noted, a negative binomial model will be used.

Ethical considerations

The physicians and their patients enrolled in the MOXXI program are covered under an umbrella consent program, which is approved and monitored by McGill’s Institutional Review Board. In addition, patients will be given opportunities to become informed about the project and to withdraw, as appropriately defined under the Tri-Council’s guidelines.


The entire ISTOP-ADE study will take 5 years from start to finish. Patient enrolment will run for 3 years to ensure sample size requirements are met. Patient follow-up will continue for 1 year from enrolment. Case summaries and physician reviews, as well as data validation and quality checks will occur throughout the lifespan of the study. We have budgeted 12 months to accomplish the analysis and knowledge translation activities.


We have planned this randomized trial to assess the effectiveness and cost-effectiveness of an automated outreach program to monitor patients receiving a new ambulatory care prescription. It is an important study as the intervention has shown promising results in a pilot study. Some health payers and pharmacies may feel that this intervention is already warranted on face value and based on our preliminary findings. We have advised against this approach as the system has not been compared against routine care. In a situation where there are limited resources, it is not appropriate to invest in interventions of unknown effectiveness.

We chose adherence with prognosis-altering medications as our primary outcome. This outcome is important as many studies have demonstrated poor adherence – approximately 50% of patients with chronic illness do not take their medication as prescribed – and this in turn has been estimated to have major impacts on disease progression and health-care costs [20, 21]. We will assess this outcome using administrative data. This ascertainment method has been demonstrated to be accurate and will not be biased by an inability to blind patients to the intervention. Our secondary outcomes are similarly important. In particular, ADEs are common following incident prescriptions [15, 16]. While many of the symptoms related to ADEs are short in duration, a substantial proportion of patients have symptoms lasting longer than 7 days and can lead patients to discontinue their medications on their own. Collectively, these outcomes will allow us to assess adequately the value of our intervention to patients and the health-care system.

Our study will also give us an opportunity to assess further implementation issues. We will track the costs of the intervention closely, including the workload required by our pharmacist to follow up the automated calls and by the physicians, who may need to do extra work to respond to patient complaints. We will also assess the failed automated calls to assess which patients do not accept this type of intervention. Finally, we will conduct an assessment of patient factors associated with adherence. All of this information will guide decisions pertaining to whether and how this system should be implemented, and to whom it should be directed.

This is an ambitious study, which will run in over 100 primary care practices in two large urban settings. It will take 3 years to complete the recruitment and follow-up. We are randomizing patients at the patient rather than provider level because there is little risk of contamination or co-intervention. The patients are unlikely to interact with each other, and, based on our pilot study, there was very little interaction between patients and doctors within the 3 weeks of their prescription. There are some sources of bias in our design to be considered. The primary risk is the lack of blinding in patients and primary care doctors. This is a concern as knowledge of allocation could change behavior. However, based on findings from our pilot study, we predict this is unlikely. In addition, outcome assessment is done objectively or by blinded reviewers. In short, this study will have internal validity and will be generalizable.

In summary, we have previously developed an automated outreach program for patients prescribed new medications. We have tested this intervention in a pilot study. This work found the intervention to be feasible and suggested it might improve the quality of patient care through reduced ADE duration and improved medication adherence. We have designed this randomized trial to test whether the intervention is effective compared to routine care. If the ISTOP-ADE system were demonstrated to be effective, then there would be significant opportunities to implement this intervention through health systems, health insurance agents and commercial pharmacies.

Trial status

The ISTOP-ADE study is anticipated to start early 2015.



adverse drug event


Information Systems-enabled Outreach for Preventing ADEs


interactive voice response system


Medical Office of the 21st Century


Régie de l’assurance maladie du Québec.


  1. Auger C, Forster AJ, Oake N, Tamblyn R: Usability of a computerised drug monitoring programme to detect adverse drug events and non-compliance in outpatient ambulatory care.BMJ Qual Saf 2013, 22:306–16. 10.1136/bmjqs-2012-001492

    Article  PubMed  Google Scholar 

  2. Forster AJ, Auger C: Using information technology to improve the monitoring of outpatient prescribing.JAMA Intern Med 2013, 173:382–4. 10.1001/jamainternmed.2013.2002

    Article  PubMed  Google Scholar 

  3. Tamblyn R, Huang A, Taylor L, Kawasumi Y, Bartlett G, Grad R, et al.: A randomized trial of the effectiveness of on-demand versus computer-triggered drug decision support in primary care.J Am Med Inform Assoc 2008, 15:430–8. 10.1197/jamia.M2606

    Article  PubMed  PubMed Central  Google Scholar 

  4. Eguale T, Tamblyn R, Winslade N, Buckeridge D: Detection of adverse drug events and other treatment outcomes using an electronic prescribing system.Drug Saf 2008, 31:1005–16. 10.2165/00002018-200831110-00005

    Article  PubMed  Google Scholar 

  5. Fischer MA, Choudhry NK, Brill G, Avorn J, Schneeweiss S, Hutchins D, et al.: Trouble getting started: predictors of primary medication nonadherence.Am J Med 2011, 124:1081.

    Article  PubMed  Google Scholar 

  6. Fischer MA, Stedman MR, Lii J, Vogeli C, Shrank WH, Brookhart MA, et al.: Primary medication non-adherence: analysis of 195,930 electronic prescriptions.J Gen Intern Med 2010, 25:284–90. 10.1007/s11606-010-1253-9

    Article  PubMed  PubMed Central  Google Scholar 

  7. Choo PW, Rand CS, Inui TS, Lee ML, Cain E, Cordeiro-Breault M, et al.: Validation of patient reports, automated pharmacy records, and pill counts with electronic monitoring of adherence to antihypertensive therapy.Med Care 1999, 37:846–57. 10.1097/00005650-199909000-00002

    Article  CAS  PubMed  Google Scholar 

  8. Sikka R, Xia F, Aubert RE: Estimating medication persistency using administrative claims data.Am J Manag Care 2005, 11:449–57.

    PubMed  Google Scholar 

  9. Steiner JF, Koepsell TD, Fihn SD, Inui TS: A general method of compliance assessment using centralized pharmacy records. Description and validation.Med Care 1988, 26:814–23. 10.1097/00005650-198808000-00007

    Article  CAS  PubMed  Google Scholar 

  10. Steiner JF, Prochazka AV: The assessment of refill compliance using pharmacy records: methods, validity, and applications.J Clin Epidemiol 1997, 50:105–16. 10.1016/S0895-4356(96)00268-5

    Article  CAS  PubMed  Google Scholar 

  11. Tamblyn R, Abrahamowicz M, Dauphinee D, Wenghofer E, Jacques A, Klass D, et al.: Influence of physicians’ management and communication ability on patients’ persistence with antihypertensive medication.Arch Intern Med 2010, 170:1064–72. 10.1001/archinternmed.2010.167

    Article  PubMed  Google Scholar 

  12. Bates DW, Leape LL, Petrycki S: Incidence and preventability of adverse drug events in hospitalized adults.J Gen Intern Med 1993, 8:289–94. 10.1007/BF02600138

    Article  CAS  PubMed  Google Scholar 

  13. Forster AJ, Asmis TR, Clark HD, Al Saied G, Code CC, Caughey SC, et al.: Ottawa Hospital Patient Safety Study: incidence and timing of adverse events in patients admitted to a Canadian teaching hospital.CMAJ 2004, 170:1235–40. 10.1503/cmaj.1030683

    Article  PubMed  PubMed Central  Google Scholar 

  14. Forster AJ, Murff HJ, Peterson JF, Gandhi TK, Bates DW: Adverse drug events occurring following hospital discharge.J Gen Intern Med 2005, 20:317–23. 10.1111/j.1525-1497.2005.30390.x

    Article  PubMed  PubMed Central  Google Scholar 

  15. Gandhi TK, Weingart SN, Borus J, Seger AC, Peterson J, Burdick E, et al.: Adverse drug events in ambulatory care.N Engl J Med 2003, 348:1556. 10.1056/NEJMsa020703

    Article  PubMed  Google Scholar 

  16. Gurwitz JH, Field TS, Harrold LR, Rothschild J, Debellis K, Seger AC, et al.: Incidence and preventability of adverse drug events among older persons in the ambulatory setting.JAMA 2003, 289:1107–16. 10.1001/jama.289.9.1107

    Article  PubMed  Google Scholar 

  17. Rothschild JM, Landrigan CP, Cronin JW, Kaushal R, Lockley SW, Burdick E, et al.: The critical care safety study: the incidence and nature of adverse events and serious medical errors in intensive care.Crit Care Med 2005, 33:1694–700. 10.1097/01.CCM.0000171609.91035.BD

    Article  PubMed  Google Scholar 

  18. Aronson J, Ferner R: Clarification of terminology in drug safety.Drug Saf 2005, 28:851–70. 10.2165/00002018-200528100-00003

    Article  PubMed  Google Scholar 

  19. Edwards I, Aronson J: Adverse drug reactions: definitions, diagnosis, and management.Lancet 2000, 356:1255–9. 10.1016/S0140-6736(00)02799-9

    Article  CAS  PubMed  Google Scholar 

  20. Nieuwlaat R, Wilczynski N, Navarro T, Hobson N, Jeffery R, Keepanasseril A, et al.: Interventions for enhancing medication adherence.Cochrane Database Syst Rev 2008. Issue 2. Art. No.: CD000011. doi:10.1002/14651858.CD000011.pub3

    Google Scholar 

  21. Peterson AM, Takiya L, Finley R: Meta-analysis of trials of interventions to improve medication adherence.Am J Health Syst Pharm 2003, 60:657–665.

    PubMed  Google Scholar 

Download references


The ISTOP-ADE study is receiving funding from the Canadian Institutes of Health Research (ResearchNet ID: 159794).

Author information

Authors and Affiliations


Corresponding author

Correspondence to Alan J Forster.

Additional information

Competing interests

The authors declare that they have no competing interests.

Authors’ contributions

AF, RT, CA, DB and CVW conceived the study and participated in its design and oversight. AF, AJ and TEE participated in writing the manuscript. All authors read and approved the final version of the manuscript.

Rights and permissions

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly credited. The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Forster, A.J., Erlanger, T.E., Jennings, A. et al. Effectiveness of a computerized drug-monitoring program to detect and prevent adverse drug events and medication non-adherence in outpatient ambulatory care: study protocol of a randomized controlled trial. Trials 16, 2 (2015).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: