This article has Open Peer Review reports available.
Over 50 years of trial in Acta Psychiatrica Scandinavica: a survey
© Matar et al; licensee BioMed Central Ltd. 2009
Received: 10 September 2008
Accepted: 27 May 2009
Published: 27 May 2009
Randomised controlled trials are the gold standard for evaluating mental health care interventions. We assessed the content and quality of trials published in Acta Psychiatrica Scandinavica and Supplementum since 1948.
All trials were identified manually, quality assessed, data extracted, and sought on Medline.
About 8.6% of all reports in the journal were clinical trials (n = 582) with the peak frequency in the 1980s. Most originate from Europe (80%) and focus on depression (~38%) or schizophrenia (27%). The median sample size is 44. We found only two trials that fully met the criteria of quality reporting RCTs set by CONSORT statements (0.34%) since 1996. Less than 50% of records were possible to identify by a Medline search using broad methodological terms.
Acta is a major source of health trials. The standard of reporting is similar to other journals but better adherence to CONSORT would ensure higher quality of reports and better dissemination.
Randomised trials are now widely accepted as the "gold standard" for providing evidence on the effectiveness of health care interventions in general  as well as mental health care interventions in particular . Random assignment greatly reduces the potential for bias in the allocation of interventions. It ensures that comparison groups are as similar as possible to each other in terms of both known and unknown possible predictors of treatment response . Empirical evidence suggests that non-randomised studies tend to overestimate the effects of healthcare . In addition, the extent and the direction of bias in these studies is often impossible to predict .
The use of randomised trials was largely restricted to evaluations of new drug therapies, either against an existing standard agent or against a placebo control. More recently, however, randomised controlled trials are being used to evaluate a much broader range of interventions .
Poorly conducted randomised trials, however, can provide biased results . For example, allocation concealment, blinding, intention-to-treat analysis, and loss to follow-up are the critical items in the design and conduct of randomised trials, and an inadequately conceived, conducted and reported randomised trial may overestimate or underestimate true effects of interventions [8, 9]. In order to improve design, conduct and reporting of trials CONSORT guidance was drawn up first in 1996  and updated in 2001 .
Acta Psychiatrica Scandinavica is one of the leading psychiatric journals with impact factor 3.857 (2006) , since the first report of a randomised trial being used in clinical medicine was published in 1948 , we aimed to assess the content and quality of trials published Acta Psychiatrica Scandinavica and Acta Psychiatrica Scandinavica Supplementum from 1948 to 2006.
All randomised controlled trials or controlled clinical trials were identified by manually searching every page of all relevant volumes of Acta Psychiatrica Scandinavica and Acta Psychiatrica Scandinavica Supplementum from 1948 to Issue 4 of 2006, including Acta Psychiatrica et Neurologica (1948–1950) and Acta Psychiatrica et Neurologica Scandinavica (1951–1961) as Acta predecessors. Working independently, HEM, MQA, and MOA used a standardised data collection sheet to extract information from the papers. Any disagreements were resolved by discussion. We recorded year of publication, type of report, country of origin (address of first author); characteristics of participants including diagnosis, sex, age (adults, elderly or children), setting (hospital or community); variables reporting trial design such as description of randomisation, size, presence of power calculation, consent, duration, parallel or crossover, type of intervention (pharmacological or non-pharmacological) and outcomes.
As allocation concealment is pivotal when describing the likelihood of inclusion of over estimates of effect in trials  this was assessed using the Cochrane Collaboration criteria. These criteria consider allocation concealment as 'adequate' when papers report sequentially labelled, sealed, opaque envelopes or central or pharmacy randomisation. By the same criteria, allocation concealment is 'unclear' when papers report randomisation methods used in terms such as "patients were randomised to..." or "treatment was assigned in random order..." Finally, when methods of allocation were explicitly not at random, or by alternation, number of medical record, or date of birth, we categorised these as 'inadequate' randomisation. In addition, each included trial was rated using Jadad scale . This measures a wider range of factors that impact on trial quality. Not only is randomisation rated (adequate = 2, unclear = 1, inadequate = 0), but also double-blindness (adequate = 2, unclear = 1, inadequate = 0) and description of withdrawals (Yes = 1, No = 0). These particular data relevant to quality were chosen because the Jadad rating, in particular, has been validated for mental health trials, and because poor quality scores are associated with over-estimates of effects of treatment [4, 8].
We also calculated the proportion of follow up by dividing the number of participants for whom the primary outcome measure was obtained and by the total number randomised. Finally we recorded source of funding of trials (pharmaceutical companies or not) and whether authors were employed by these sources. Electronic records of relevant records on the National Library of Medicine's Medline database, on the PubMed platform, were downloaded into reference manger software (ProCite). We searched for the methodological indexing of all records.
We calculated simple proportions. For trends over time, we grouped data by decade and for this used SPSS (version 13, SPSS Inc., Chicago, IL, USA).
Acta Psychiatrica Scandinavica and Acta Psychiatrica Scandinavica Supplementum contain 582 reports of clinical trials published between 1953 and 2006. According to PubMed records, this is about 8.6% of all reports published in the Journals since that time. Of these trials, most (423, 73%) were randomised studies but 159 (27%) were 'controlled clinical trials'. In this type of study, randomisation is implied rather than made explicit. Of the 582 trials, most were published as original papers (560, 96%), with only ten (1.7%) being abstracts alone. The remaining 12 (2.3%) were either conference proceedings or letters.
Reports by Region
Reports by region
Number of reports of trials
South East Asia
Sample size and participants
Diagnosis of trial participants
Number of reports of trials
Jadad score by time and impact factor
3–5 (good quality)
Interventions and outcomes
Interventions used in trials
Types of Interventions
Source of funding
Trials reported source of funding 183 trials (31.5%) with pharmaceutical companies supporting 82 (45%) and the remaining 101 trials funded by national medical research counsels and academic institutes. Principal investigators in 21/183 trials were directly employed by those providing funding.
By searching Medline and Old Medline, we found 566 records for the 582 trials (97.3%). The National Library (NLM), who index for Medline, do not, by policy, index conference abstracts and indexing of letters is somewhat haphazard  and it is for this reason that 2.7% of studies in the Actas are not in Medline or Old Medline. Searching the indexing terms by the simple phrase "RANDOM*" identified only 54 records (9.3%). Expanding this to "RANDOM* OR (DOUBLE-BLIND* or (DOUBL* AND BLIND*))" still only identified 273 records (47%). Searching both NLM index terms combined with free text (including abstract and title) by "RANDOM*" identified 237 records (41%) and the "RANDOM* OR (DOUBLE-BLIND* or (DOUBL* AND BLIND*))" phrase 396 records (68%).
This study shows how Acta Psychiatrica Scandinavica and Acta Psychiatrica Scandinavica Supplementum have published a wide range of trials over the last five decades. The proportion of all studies that were RCTs or CCTs (8.6%) being very similar to that in other specialties (8.4%) . Most trials in these European journals originated from Europe. This 'home advantage' has also been demonstrated for other journals  but it is encouraging that the proportion of trials from outside Europe in Acta is increasing. We are unclear why the number of trials in the journals seems to be decreasing across time. This may be that studies are being published elsewhere. There is an ever-increasing number of journals vying for authors, Acta's editorial policy or the attractiveness of the journals could have changed over time – for example, by decreasing impact factor. Certainly, other information suggests that there are not fewer mental health trials being published over the same period [20–22].
The content and quality of trials in Acta Psychiatrica Scandinavica and Acta Psychiatrica Scandinavica Supplementum seems similar to those in other major sources [21, 22]. Most studies are small (only 1.9% even reported undertaking a power calculation), short (>50% 6 weeks or less) and problematic to apply to everyday care. Moreover, they are so poorly reported that readers have to be concerned as to the validity of their findings. From Table 3 it can be seen that a reasonably high proportion of trials do fall into the 'good quality' category and the impression of improvement (too underpowered to clearly show a statistically significant incline) is paralleled by increasing impact of Acta. This proportion of 'good quality' studies is more than has been seen in preceding similar work [20, 21]. However, over 60% of trials continue to have minimal reporting of methods and this is associated with exaggerated estimates of effect [4, 8].
We implemented the Jadad scale as it is the only scale that has been developed using established standards for scales and where low scores have been associated with increased effect estimates . However, despite its thorough development and validation, the scale has more focus on the quality of reporting than on methodological quality . As for randomisation, the scale addresses the sequence generation but not concealment of allocation as even adequate concealment of allocation may not prevent against selection bias if the sequence generation is deciphered by the persons enrolling patients .
In undertaking a survey of this kind there is a danger of judging work from decades ago by standards of today. However, when this has been applied to other mental health journals findings have been surprising with, for example, the quality of reporting in one major journal declining across time . The CONSORT statement, published over a decade ago, does not seem to have been incorporated into Acta's reporting style.
The indexing of randomised studies could also be improved. Just less than 70% are identified by a broad methodology search in Medline. Even though 123 records did use the terms 'randomised' or 'randomly assigned/allocated' or 'double blind' in the title or abstract of the work, these were not indexed as such by The National Library of Medicine. Medline and Old Medline indexing contains much [understandable] human error. There is, however, a suggestion that 32% of randomised or controlled clinical trials did not mention a methodological term in an obvious way so as to allow easy indexing. It is encouraging that this proportion as steadily decreased across time (95% 1960's, 48% 1970's, 24% 1980's, 16% 1990's), but that 10% of randomised trials since the year 2000 still do not mention key methodological terms in the title or abstract is not good and could be improved by editorial policy. In this way both trialists and editors could be more assured that valuable studies in Acta Psychiatrica Scandinavica and Acta Psychiatrica Scandinavica Supplementum will be identified, used, cited and contribute to impact.
Acta Psychiatrica Scandinavica and Acta Psychiatrica Scandinavica Supplementum are major sources of mental health trials undertaken in the last half century. The small but increasing proportion of trials from outside of Europe is encouraging, although why the number of trials in the journals is declining is not clear. The standard of reporting is as high, if not higher in these journals compared with their competitors – but could improve with little additional effort. Better adherence to CONSORT would ensure higher quality of reports, better indexing and, as a consequence, wider dissemination.
We would like to thank the efforts of the Warneford Search Party of many years ago undertaking the first manual searches late in the library in Oxford.
- Sackett DL, Straus SE, Richardson WS, Rosenberg W, Haynes RB: Evidence-Based Medicine: How to Practice and Teach EBM. 2000, Edinburgh, Scotland: Churchill LivingstoneGoogle Scholar
- WHO Scientific Group on Treatment of Psychiatric Disorders. Evaluation of methods for the treatment of mental disorders. 1991, Geneva: WHOGoogle Scholar
- Cochrane AL: Effectiveness and efficiency: Random Reflections on Health Services. 1972, London, England: Nuffield Provincial Hospitals TrustGoogle Scholar
- Schulz KF, Chalmers I, Hayes RJ, Altman DG: Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995, 273: 408-412. 10.1001/jama.273.5.408.View ArticlePubMedGoogle Scholar
- Kunz R, Oxman AD: The unpredictability paradox: review of empirical comparisons of randomised and non-randomised trials. BMJ. 1998, 317: 1185-1190.View ArticlePubMedPubMed CentralGoogle Scholar
- Silagy CA, Jewell D: Review of 39 years of randomized controlled trials in the British Journal of General Practice. British Journal of General Practice. 1994, 44: 359-363.PubMedPubMed CentralGoogle Scholar
- Kjaergard LL, Villumsen J, Gluud C: Reported methodological quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med. 2001, 135: 982-989.View ArticlePubMedGoogle Scholar
- Huwiler-Muntener K, Juni P, Junker C, Egger M: Quality of reporting of randomized trials as a measure of methodologic quality. JAMA. 2002, 287: 2801-2804. 10.1001/jama.287.21.2801.View ArticlePubMedGoogle Scholar
- Schulz KF, Chalmers I, Grimes D, Altman DG: Assessing the quality of randomization from reports of controlled trials published in obstetrics and gynecology journals. JAMA. 1994, 272: 125-128. 10.1001/jama.272.2.125.View ArticlePubMedGoogle Scholar
- Begg C, Cho M, Eastwood S, Horton R, Moher D, Olkin I, Pitkin R, Rennie D, Schulz KF, Simel D, Stroup DF: Improving the quality of reporting of randomized controlled trials. The CONSORT statement. JAMA. 1996, 276: 637-639. 10.1001/jama.276.8.637.View ArticlePubMedGoogle Scholar
- Moher D, Schulz KF, Altman D: The CONSORT statement: revised recommendations for improving the quality of reports of parallel-group randomized trials. JAMA. 2001, 285: 1987-1991. 10.1001/jama.285.15.1987.View ArticlePubMedGoogle Scholar
- Munk-Jørgensen P, Christiansen B, Licht RW: Happy New Year 2008. Acta Psychiatr Scand. 2008, 117: 1-3.View ArticleGoogle Scholar
- Medical Research Council: Streptomycin treatment of pulmonary tuberculosis. BMJ. 1948, 2: 769-782. 10.1136/bmj.2.4582.769.View ArticleGoogle Scholar
- Jüni P, Altman D, Egger M: Systematic reviews in health care: Assessing the quality of controlled clinical trials. BMJ. 2001, 323: 42-46. 10.1136/bmj.323.7303.42.View ArticlePubMedPubMed CentralGoogle Scholar
- Alderson P, Green S, Higgins JPT: Cochrane Reviewers' Handbook 4.2.2. 2004, The Cochrane LibraryGoogle Scholar
- Jadad AR, Moore RA, Carroll D, Jenkinson C, Reynolds DJ, Gavaghan DJ, McQuay HJ: Assessing the quality of reports of randomized clinical trials: is blinding necessary?. Control Clin Trials. 1996, 17: 1-12. 10.1016/0197-2456(95)00134-4.View ArticlePubMedGoogle Scholar
- Munk-Jørgensen P: Editorial: Happy New Year 2005. Acta Psychiatr Scand. 2005, 111: 1-3. 10.1111/j.1600-0447.2004.00477.x.View ArticleGoogle Scholar
- Hay P, Adams CE, Lefebvre C: The efficiency of searches for randomized controlled trials in the International Journal of Eating Disorders: a comparison of handsearching, EMBASE and PsycLIT. Health Libraries Review. 1996, 13: 91-96. 10.1046/j.1365-2532.1996.1320091.x.View ArticleGoogle Scholar
- Gnanalingham MG, Robinson SG, Hawley DP, Gnanalingham KK: A 30 year perspective of the quality of evidence published in 25 clinical journals: Signs of change?. Postgrad Med J. 2006, 82: 397-399. 10.1136/pgmj.2005.041251.View ArticlePubMedPubMed CentralGoogle Scholar
- Gill DB, Adams CE: Randomized controlled trials in the Journal of Psychosomatic Research; 1956–1993: a prevalence study. J Psychosom Res. 1995, 39: 949-956. 10.1016/0022-3999(95)00013-5.View ArticlePubMedGoogle Scholar
- Ahmed I, Soares K, Seifas R, Adams CE: Randomised controlled trials in Archives of General Psychiatry (1959–1995): a prevalence study. Archives of General Psychiatry. 1998, 55: 754-755. 10.1001/archpsyc.55.8.754.View ArticlePubMedGoogle Scholar
- Thornley B, Adams CE: Content and quality of 2000 controlled trials in schizophrenia over 50 years. BMJ. 1998, 317: 1181-1184.View ArticlePubMedPubMed CentralGoogle Scholar
- Moher D, Pham B, Jones A, Cook DJ, Jadad AR, Moher M, Tugwell P, Klassen TP: Does quality of reports of randomised trials affect estimates of intervention efficacy reported in meta-analyses?. Lancet. 1998, 352: 609-613. 10.1016/S0140-6736(98)01085-X.View ArticlePubMedGoogle Scholar
- Jüni P, Witschi A, Bloch R, Egger M: The hazards of scoring the quality of clinical trials for meta-analysis. JAMA. 1999, 282: 1054-1060. 10.1001/jama.282.11.1054.View ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.