Open Access
Open Peer Review

This article has Open Peer Review reports available.

How does Open Peer Review work?

The effects of an Internet based self-help course for reducing panic symptoms - Don't Panic Online: study protocol for a randomised controlled trial

  • Wouter van Ballegooijen1, 2Email author,
  • Heleen Riper1, 3,
  • Annemieke van Straten1,
  • Jeannet Kramer2,
  • Barbara Conijn2 and
  • Pim Cuijpers1
Trials201112:75

https://doi.org/10.1186/1745-6215-12-75

Received: 15 November 2010

Accepted: 11 March 2011

Published: 11 March 2011

Abstract

Background

Internet based self-help for panic disorder (PD) has proven to be effective. However, studies so far have focussed on treating a full-blown disorder. Panic symptoms that do not meet DSM-IV criteria are more prevalent than the full-blown disorder and patients with sub-clinical panic symptoms are at risk of developing PD. This study is a randomised controlled trial aimed to evaluate an Internet based self-help intervention for sub-clinical and mild PD compared to a waiting list control group.

Methods

Participants with mild or sub-clinical PD (N = 128) will be recruited in the general population. Severity of panic and anxiety symptoms are the primary outcome measures. Secondary outcomes include depressive symptoms, quality of life, loss of production and health care consumption. Assessments will take place on the Internet at baseline and three months after baseline.

Discussion

Results will indicate the effectiveness of Internet based self-help for sub-clinical and mild PD. Strengths of this design are the external validity and the fact that it is almost completely conducted online.

Trial registration

Netherlands Trial Register (NTR): NTR1639 The Netherlands Trial Register is part of the Dutch Cochrane Centre.

Background

Anxiety disorders are highly prevalent, affecting one in six people during their lifetime [1], and cause a substantial loss of quality of life [2]. Direct and indirect costs caused by anxiety disorders are estimated at half a billion dollars per 1 million adults per year [3]. One of those anxiety disorders is panic disorder (PD). A study among the population in the Netherlands shows PD affects 1.5% of all adults each year, while 2% of the population is affected by sub-clinical PD [4]. Sub-clinical PD indicates infrequent panic attacks or frequent panic attacks that are relatively mild. It can be defined as panic symptoms that do not meet DSM-IV criteria for PD. Sub-clinical PD is a substantial burden for both the patient and society [4]. Panic symptoms are often comorbid with other mental health problems, like depression or other anxiety disorders [5]. Comorbid anxiety and depressive symptoms can lead to suicidal ideation [6] and PD elevates the risk at suicide attempts when comorbid with a mood disorder, substance abuse or another anxiety disorder [7].

While PD can be effectively treated by cognitive behavioural therapy and pharmacotherapy [8], it is known that only 25% of people with anxiety symptoms seeks help [9] and only 15% of people with panic symptoms receives effective treatment [10]. Help-seeking might be too difficult or fearful for people with panic symptoms, especially when they suffer from agoraphobia as well. In addition, help-seeking could be further compromised by fear of stigmatisation, misinterpretation of panic symptoms as a physical problem, or little awareness of effective treatment opportunities or available services.

A more accessible and perhaps more acceptable alternative to face-to-face interventions for PD is Internet based self-help. Several studies show promising effects of Internet based self-help for PD [1113]. Moreover, Internet based self-help courses save clinicians' time and cost little compared to face-to-face treatment or pharmacotherapy [14].

Internet based self-help interventions for panic symptoms focus mainly on full-blown PD [1113]. An accessible self-help course, tailored for the large group of sub-clinical and mild cases, would be beneficial to a group of patients that is not yet targeted by Internet interventions. Moreover, an Internet based self-help course for sub-clinical and mild panic symptoms could close the gap between prevention and early intervention of panic disorder.

This study is designed to evaluate Don't Panic Online (Geen Paniek Online) [15], an Internet based self-help course for sub-clinical and mild cases of PD with minimal guidance. Don't Panic Online (DPO) is based on Don't Panic (Geen Paniek) [16], a face-to-face group course for sub-clinical and mild panic symptoms. Don't Panic has proven to be cost-effective [17, 18].

We will conduct a randomised controlled trial, comparing an experimental group that takes the course Don't Panic Online to a waiting list control group that will get access to an information website. The research question is the following: what is the effect of the Internet based self-help course Don't Panic Online on sub-clinical to mild panic symptoms?

Methods

Design

We will conduct a randomised controlled trial with two arms: (a) Internet based self-help with minimal guidance, (b) waiting list control group with information website. The Medical and Ethical Committee of VU Medical Centre approved the study protocol.

Study population

The study population consists of adults with sub-clinical or mild PD. The inclusion criteria are the following: 18 years of age or older, Internet access, sub-clinical or mild PD (PDSS-SR score 5-15). Consequently, individuals with too mild (PDSS-SR score 1-4) or too severe (PDSS-SR score 15 or higher) panic symptoms will be excluded. Participants who report moderate to high suicide risk, as measured by a self-report version of the MINI, will be excluded as well. Applicants who report severe panic symptoms or suicide risk will be contacted by e-mail and advised to contact their general practitioner.

Sample size

Power calculations are based on a moderate effect size of d=.50, comparing the PDSS-SR scores of the intervention group to the control group with a two-sided t-test (alpha .05, power 80%). To show this effect each group will consist of 64 participants [19], so the total sample size adds up to N = 128. Missing values will be imputed using regression imputation.

Recruitment

Participants will be recruited in the general population by means of a Facebook advertisement campaign, messages on panic or anxiety related online forums, banners on health related websites and advertisements in newspapers. Interested individuals will be directed to a study website, where they can find further information as well as an informed consent form. They can apply for participation by printing and signing the informed consent form, which can be scanned and sent by e-mail or sent by postal mail. After application, the researchers will send the participants a link to an online questionnaire.

Randomisation and procedure

After screening and completion of the baseline questionnaires (t0), the participant will be contacted for a diagnostic interview by telephone. The interview will be conducted within two weeks after baseline by a trained and experienced interviewer. All participants will be randomised to one of the two groups, regardless of the presence or absence of a diagnosis. Randomisation will be stratified for the presence or absence of agoraphobia symptoms (PDSS-SR item 4 score > 1) and the use of medication. Randomisation lists are generated with a computer program. Blinding of the participants and researchers is not possible due to the design of this research. The post-treatment assessment (t1) is scheduled at 3 months after baseline. Both the baseline and the post-treatment assessments are self-report and will be conducted through the Internet.

Intervention

Don't Panic Online (DPO) is an individual guided web-based self-help course, based on cognitive behavioural therapy. It was developed by the Trimbos Institute, which is the Netherlands Institute for Mental health and Addiction, in collaboration with GGNet, a Dutch mental health care institute. The course consists of six sessions, in which the participants will learn to control their panic symptoms by applying various cognitive behavioural techniques and skills. The course's format is based on Colour Your Life, an evidence based Internet intervention for depressive symptoms [20, 21].

DPO is a protocolised treatment and consists of the following components: keeping a log of panic attacks; analysis of fearful situations; challenging thoughts that enable feelings of panic; replacing these thoughts by more realistic, constructive thoughts that reduce anxiety; behavioural exercises; and ranking exercises from manageable to difficult and carrying them out in order of difficulty. Each session consists of text, voice over, animated diagrams and video. A typical session will take about thirty minutes and consists of an introduction, a discussion of the previous lesson's homework, new theory and homework for the next week. The course is designed to be followed on a weekly basis until session five, while the sixth lesson can be followed four weeks after the fifth. The course can be completed in eight weeks.

Besides the lessons, the participant has several online resources at his or her disposal: a homework station, a panic attack log, a library for extra information, reading tips and a discussion board.

The DPO website contains an e-mail system for contact between a participant and a coach. For the current study, the coach will reply to questions about the course, its exercises and the participant's mental health. He will also contact the user weekly to ask after his progress. Participants will be supported for a maximum of three months. The coaches will consist of students who are in the final phase of their study of clinical psychology. All will receive a brief training.

Participants in the control group will receive access to DPO after completing the t1 measurement. In the mean time, they will have access to an information website about PD.

Instruments

All variables will be measured at t0 and t1, except for demographic data, the diagnosis and satisfaction with the intervention. Demographic data and diagnosis are only measured at t0, while satisfaction with the intervention will only be measured at t1.

Demographic data

This section contains questions about age, gender and education.

Diagnosis

The Composite International Diagnostic Interview (CIDI) [22] will be used to ascertain the presence or absence of PD, other anxiety disorders and depression. The CIDI has been developed by the WHO and is an extensive, fully structured diagnostic interview to assess DSM-IV Axis-I diagnoses [22]. Only the subscales depression, PD, agoraphobia, GAD, social phobia and post-traumatic stress disorder will be used in this study. The reliability of these subscales is sufficient (inter-rater κ = .94 - .99; test-retest κ = .62 - .84) [23]. In this study, a trained interviewer will administer the CIDI by telephone.

Anxiety and panic symptoms

For severity of panic symptoms the Panic Disorder Severity Scale - Self Report (PDSS-SR) will be used. The PDSS is originally a face-to-face interview [24] and was adapted to self-report by Houck et al. [25]. The instrument contains seven items that assess the severity of seven dimensions of panic disorder and associated symptoms. The PDSS-SR generates a total score ranging from 0 to 28, with a higher score indicating more severe panic symptoms. The questionnaire has good psychometric properties with Cronbach's alpha = 0.92 and intraclass correlation coefficient = 0.81 [25]. In the current study, a score of 0-4 will count as no clinically relevant symptoms and 16 or higher as severe PD.

Anxiety symptoms in general will be measured with the Beck Anxiety Inventory (BAI) [26]. The BAI contains 21 short questions. Internal consistency is high, with Cronbach's alpha ranging from .90 to .94 [26]. The score varies from 0 to 63. A score of 30 or higher is interpreted as severe anxiety symptoms.

Depressive symptoms

Depressive symptoms we will be measured with the CES-D [27]. The CES-D is a 20-item self-report questionnaire. Every item ranges from 0 to 3 and the total score ranges from 0 (no feelings of depression) to 60 (severe feelings of depression). Radloff [27] reports high internal consistencies among different populations (.79 to .92). The optimal cut-off score varies between 16 and 27 [28]. In the current study, we will regard 27 and higher as a high score.

Suicidal risk

Suicidal risk and ideation will be measured with the specific section of the Mini-International Neuropsychiatric Interview (MINI) [29, 30]. The MINI suicide section consists of 6 items and classifies subjects into four groups: no suicidal risk, low suicidal risk, moderate suicidal risk, and high suicidal risk. In this study, the items are self-report. Participants with moderate to high suicidal risk will be excluded from this study.

Quality of life

Quality of life will be measured with the Dutch version of the EuroQol Questionnaire (EQ-5D) [31, 32]. This short list contains five dimensions: mobility, self-care, usual activities, pain/discomfort and anxiety/depression. The EQ-5 D generates a total of 243 unique health states, each of which is associated with a utility score ranging from 0 (poor health) to 1 (perfect health). The EQ-5 D is a validated instrument for measuring general health-related quality of life [3133].

Loss of production and health care consumption

An indication of loss of production and use of care will be made with the Trimbos and Institute of Medical Technology Assessment Questionnaire on Costs Associated with Psychiatric Illness (TiC-P) [34]. This list consists of three parts. Part I measures health care consumption of individuals who suffer from mental disorders. Part II examines loss of production (indirect costs) and part III general, demographic variables. In the current study, part III will be omitted, because demographic variables are already measured elsewhere.

Satisfaction and track-and-trace

A track-and-trace system will keep a record of the dates participants of the intervention group log on or finish a lesson. This system will also pose a question after each lesson: "Was this lesson useful to you?", which can be answered on a 5-point Likert scale. At t1, these participants will receive questions about their experience with the intervention. This includes the number of lessons followed, the amount of time spent on homework, satisfaction with the intervention and satisfaction with the coaching.

Analyses

Differences in demographic and baseline clinical characteristics will be computed with Chi-square tests, t-tests, and analysis of variance (ANOVA). The data of post-treatment measurements will be analysed in agreement with the intention to treat principle. Assuming missing data will be missing at random, the Linear Mixed Modeling (LMM) procedure will be used for all analyses to estimate missing values. LMM includes incomplete cases in the analysis and employs restricted maximum likelihood estimation to calculate parameter estimates.

Effects between the experimental and control group on continuous measures will be calculated with Cohen's d. Cohen's d is computed by subtracting the mean post-test score of the control group from the average score of the experimental group and dividing the difference by the pooled standard deviation. Effect sizes of 0.8 can be assumed to be large, while effect sizes of 0.5 are moderate, and effect sizes of 0.2 are small [35]. Estimated data from the LMM procedure will be used to analyse effect sizes.

Discussion

Internet based self-help for PD has proven to be effective. However, existing interventions focus on treating a full-blown disorder, while sub-clinical PD is highly prevalent and can develop into full-blown PD. This study is a randomised controlled trial aimed to evaluate an Internet based self-help intervention for sub-clinical and mild PD compared to a waiting list control group. The primary outcome measure is the severity of panic and anxiety symptoms. Secondary outcome measures include depression, quality of life, loss of production and health care consumption.

Methodological considerations

The primary research question is whether DPO is an effective intervention for mild and sub-clinical PD. We hypothesise the experimental group will show more improvement on the outcome measures than the control group, because the experimental group receives an accessible intervention tailored for its specific mental health problems.

A limitation of this study is the low number of exclusion criteria. The presence of a comorbid disorder or substance abuse is not an exclusion criterion. Comorbidity may influence this study's outcome measures, because panic symptoms may not be the participant's primary mental health problem. However, in reality, many people who suffer from panic symptoms have comorbid mental health problems, especially other anxiety disorders and depression [6]. Therefore, it is preferable not to exclude participants based on DSM diagnoses for these disorders. An indication of the effects of comorbidity on the effectiveness of DPO could be obtained by mediator analyses.

Both a strength and a limitation of this study is that, apart from the diagnostic interview, the measurements are conducted online. Some evidence suggests psychometric properties may change when a test is conducted via the web [36, 37]. Consequently, results of this study may differ from studies into panic interventions that applied paper-pencil questionnaires. On the other hand, the strength of online self-report measurements is the accessibility of participation [38]. Therefore, this study is feasible and should be able to answer the research questions.

Conclusion

Existing online interventions for panic symptoms focus on treating full-blown PD, while sub-clinical PD is highly prevalent and may develop into a full-blown disorder. This study evaluates an Internet based self-help intervention for sub-clinical and mild PD. Results will contribute to the ongoing research into Internet based interventions and treatment of panic symptoms.

Author's contributions

All authors have contributed to, read and approved this manuscript.

Declarations

Acknowledgements and funding

Godelief Willemse, Jessica Herzmanatus and Peter Meulenbeek have contributed to the content of Don't Panic Online.

This study is funded by the Trimbos Institute and VU-University Amsterdam.

Authors’ Affiliations

(1)
Department of clinical psychology and EMGO Institute, VU-University
(2)
Trimbos Institute (Netherlands Institute of Mental Health and Addiction)
(3)
GGZ InGeest

References

  1. Somers JM, Goldner EM, Waraich P, Hsu L: Prevalence and incidence studies of anxiety disorders: a systematic review of the literature. Can J Psychiatry. 2006, 51 (2): 100-13.PubMedGoogle Scholar
  2. Angermeyer MC, Kilian R, Wilms HU, Wittmund B: Quality of life of spouses of mentally ill people. Int J Soc Psychiatry. 2006, 52 (3): 278-85. 10.1177/0020764006067186.View ArticlePubMedGoogle Scholar
  3. Smit F, Cuijpers P, Oostenbrink J, Batelaan N, de Graaf R, Beekman A: Costs of nine common mental disorders: implications for curative and preventive psychiatry. J Ment Health Policy Econ. 2006, 9 (4): 193-200.PubMedGoogle Scholar
  4. Batelaan N, Smit F, de Graaf R, van Balkom A, Vollebergh W, Beekman A: Economic costs of full-blown and subthreshold panic disorder. J Affect Disord. 2007, 104 (1-3): 127-36. 10.1016/j.jad.2007.03.013.View ArticlePubMedGoogle Scholar
  5. Kessler RC, Chiu WT, Demler O, Merikangas KR, Walters EE: Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Arch Gen Psychiatry. 2005, 62 (6): 617-27. 10.1001/archpsyc.62.6.617.View ArticlePubMedPubMed CentralGoogle Scholar
  6. Norton PJ, Temple SR, Pettit JW: Suicidal ideation and anxiety disorders: Elevated risk or artifact of comorbid depression?. J Behav Ther Exp Psychiatry. 2008, 39 (4): 515-25. 10.1016/j.jbtep.2007.10.010.View ArticlePubMedGoogle Scholar
  7. Nock MK, Hwang I, Sampson NA, Kessler RC: Mental disorders, comorbidity and suicidal behavior: Results from the National Comorbidity Survey Replication. Mol Psychiatry. 2010, 15 (8): 868-76. 10.1038/mp.2009.29.View ArticlePubMedGoogle Scholar
  8. Furukawa TA, Watanabe N, Churchill R: Psychotherapy plus antidepressant for panic disorder with or without agoraphobia. Br J Psychiatry. 2006, 188: 305-12. 10.1192/bjp.188.4.305.View ArticlePubMedGoogle Scholar
  9. Roness A, Mykletun A, Dahl AA: Help-seeking behaviour in patients with anxiety disorder and depression. Acta Psychiatr Scand. 2005, 111 (1): 51-8. 10.1111/j.1600-0447.2004.00433.x.View ArticlePubMedGoogle Scholar
  10. Craske MG, Zucker BG: Consideration of the apa practice guideline for the treatment of patients with panic disorder: strengths and limitations for behavior therapy. Behavior Therapy. 2001, 32: 259-281. 10.1016/S0005-7894(01)80005-8.View ArticleGoogle Scholar
  11. Carlbring P, Bohman S, Brunt S, Buhrman M, Westling BE, Ekselius L, Andersson G: Remote treatment of panic disorder: a randomized trial of Internet-based cognitive behavior therapy supplemented with telephone calls. Am J Psychiatry. 2006, 163 (12): 2119-25. 10.1176/appi.ajp.163.12.2119.View ArticlePubMedGoogle Scholar
  12. Marks IM, Kenwright M, McDonough M, Whittaker M, Mataix-Cols D: Saving clinicians' time by delegating routine aspects of therapy to a computer: a randomized controlled trial in phobia/panic disorder. Psychol Med. 2004, 34 (1): 9-17. 10.1017/S003329170300878X.View ArticlePubMedGoogle Scholar
  13. Klein B, Richards JC, Austin DW: Efficacy of Internet therapy for panic disorder. J Behav Ther Exp Psychiatry. 2006, 37 (3): 213-38. 10.1016/j.jbtep.2005.07.001.View ArticlePubMedGoogle Scholar
  14. Cuijpers P, Marks IM, van Straten A, Cavanagh K, Gega L, Andersson G: Computer-Aided Psychotherapy for Anxiety Disorders: A Meta-Analytic Review. Cogn Behav Ther. 2009, 66-82. 10.1080/16506070802694776. 2Google Scholar
  15. Kramer J, Conijn B, Herzmanatus J, Riper H: Productbeschrijving Geen Paniek Online. 2008, Utrecht: Trimbos-instituutGoogle Scholar
  16. Meulenbeek P, Herzmanatus J, Smit F, Willemse G, Van der Zanden R: Cursusmap: Geen Paniek, leren omgaan met paniekklachten. 2005, Utrecht: Trimbos-instituut/GGNetGoogle Scholar
  17. Meulenbeek P, Willemse G, Smit F, van Balkom A, Spinhoven P, Cuijpers P: Early intervention in panic: randomized controlled trial and cost-effectiveness analysis. Trials. 2008, 27 (9): 67-10.1186/1745-6215-9-67.View ArticleGoogle Scholar
  18. Smit F, Willemse G, Meulenbeek P, Koopmanschap M, van Balkom A, Spinhoven P, Cuijpers P: Preventing panic disorder: cost-effectiveness analysis alongside a pragmatic randomised trial. Cost Eff Resour Alloc. 2009, 24 (7): 8-10.1186/1478-7547-7-8.View ArticleGoogle Scholar
  19. Lipsey MW: Design sensitivity; statistical power for experimental research. 1993, Newbury Park: Sage publicationsGoogle Scholar
  20. Warmerdam L, van Straten A, Twisk J, Riper H, Cuijpers P: Internet-based treatment for adults with depressive symptoms: randomized controlled trial. J Med Internet Res. 2008, 10 (4): e44-10.2196/jmir.1094.View ArticlePubMedPubMed CentralGoogle Scholar
  21. Spek V, Cuijpers P, Nyklícek I, Smits N, Riper H, Keyzer J, Pop V: One-year follow-up results of a randomized controlled clinical trial on internet-based cognitive behavioural therapy for subthreshold depression in people over 50 years. Psychol Med. 2008, 38 (5): 635-9. 10.1017/S0033291707002590.View ArticlePubMedGoogle Scholar
  22. Robins LN, Wing J, Wittchen HU, Helzer JE, Babor TF, Burke J, Farmer A, Jablenski A, Pickens R, Regier DA: The Composite International Diagnostic Interview. An epidemiologic Instrument suitable for use in conjunction with different diagnostic systems and in different cultures. Arch Gen Psychiatry. 1988, 45 (12): 1069-77.View ArticlePubMedGoogle Scholar
  23. Wittchen HU: Reliability and validity studies of the WHO--Composite International Diagnostic Interview (CIDI): a critical review. J Psychiatr Res. 1994, 28 (1): 57-84. 10.1016/0022-3956(94)90036-1.View ArticlePubMedGoogle Scholar
  24. Shear MK, Rucci P, Williams J, Frank E, Grochocinski V, Vander Bilt J, Houck P, Wang T: Reliability and validity of the panic disorder severity scale: replication and extension. J Psychiatr Res. 2001, 35 (5): 293-6. 10.1016/S0022-3956(01)00028-0.View ArticlePubMedGoogle Scholar
  25. Houck PR, Spiegel DA, Shear MK, Rucci P: Reliability of the self-report version of the Panic Disorder Severity Scale. Depress Anxiety. 2002, 15: 183-185. 10.1002/da.10049.View ArticlePubMedGoogle Scholar
  26. Beck AT, Epstein N, Brown G, Steer RA: An inventory for measuring clinical anxiety: psychometric properties. J Consult Clin Psychol. 1988, 56 (6): 893-7. 10.1037/0022-006X.56.6.893.View ArticlePubMedGoogle Scholar
  27. Radloff LS: The CES-D Scale: A self-report depression scale for research in the general population. Appl Psychol Meas. 1977, 1: 385-10.1177/014662167700100306.View ArticleGoogle Scholar
  28. Haringsma R, Engels GI, Beekman AT, Spinhoven P: The criterion validity of the Center for Epidemiological Sutdies Depression Scale (CES-D) in a sample of self-referred elders with depressive symptomatology. Int J Geriatr Psychiatry. 2004, 19 (6): 558-63. 10.1002/gps.1130.View ArticlePubMedGoogle Scholar
  29. Sheehan DV, Lecrubier Y, Sheehan KH, Amorim P, Janavs J, Weiller E, Hergueta T, Baker R, Dunbar GC: The Mini-International Neuropsychiatric Interview (M.I.N.I.): the development and validation of a structured diagnostic psychiatric interview for DSM-IV and ICD-10. J Clin Psychiatry. 1998, 59 (Suppl 20): 22-33. quiz 34-57PubMedGoogle Scholar
  30. Lecrubier Y, Sheehan D, Weiller E, Amorim P, Bonora I, Sheehan KH, Janavs J, Dunbar G: The Mini International Neuropsychiatric Interview (MINI). A short diagnostic structured interview: reliability and validity according to the CIDI. European Psychiatry. 1997, 12: 224-231. 10.1016/S0924-9338(97)83296-8.View ArticleGoogle Scholar
  31. Brooks R: EuroQol: the current state of play. Health Policy. 1996, 37 (1): 53-72. 10.1016/0168-8510(96)00822-6.View ArticlePubMedGoogle Scholar
  32. De Beurs E, den Hollanders-Gijsman ME, Buwalda V, Trijsburg W, Zitman FG: De Outcome Questionnaire OQ-45. Psychodiagnostisch gereedschap. De Psycholoog. 2005, 40: 393-400.Google Scholar
  33. Fransen M, Edmonds J: Reliability and validity of the EuroQol in patients with osteoarthritis of the knee. Rheumatology. 1999, 38 (9): 807-13. 10.1093/rheumatology/38.9.807.View ArticlePubMedGoogle Scholar
  34. Hakkaart-van Roijen L, van Straten A, Donker M, Tiemens B: Handleiding Trimbos/iMTA questionnaire for Costs associated with Psychiatric illness (TiC-P). 2002, Rotterdam: Insituut voor Medische Technology Assessment, Erasmus MCGoogle Scholar
  35. Cohen J: Statistical Power Analysis for the Behavioural Sciences. 1988, Hillsdale, NJ: Lawrence Erlbaum Associates, 2Google Scholar
  36. Buchanan T, Ali T, Heffernan TM, Ling J, Parrott AC, Rodgers J, Scholey AB: Nonequivalence of On-Line and Paper-and-Pencil Psychological Tests: The Case of the Prospective Memory Questionnaire. Behav Res Methods. 2005, 37 (1): 148-54. 10.3758/BF03206409.View ArticlePubMedGoogle Scholar
  37. Cuijpers P, Boluijt P, Van Straten A: Screening of depression in adolescents through the Internet: sensitivity and specificity of two screening questionnaires. Eur Child Adolesc Psychiatry. 2008, 17: 32-38. 10.1007/s00787-007-0631-2.View ArticlePubMedGoogle Scholar
  38. Whitehead LC: Methodological issues in Internet-mediated research in the field of health: An integrated review of the literature. Soc Sci Med. 2007, 65: 782-791. 10.1016/j.socscimed.2007.03.005.View ArticlePubMedGoogle Scholar

Copyright

© van Ballegooijen et al; licensee BioMed Central Ltd. 2011

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.