Overview

In the section "discussion" this methodological approach is compared to a more formal consensus development method (e.g. Delphi method, nominal group technique).

Considered regulations and guidelines

In developing the ECRIN standard, compliance with ICH GCP [3] was seen as the central requirement. The European directives and guidelines dealing with clinical research all refer to this document (Figure 1), indicating the central importance of GCP for data management in clinical trials. ICH GCP section 5 [3] describes some requirements for the use of electronic trial data and computer systems, e.g. the sponsors operating such computer systems must validate their systems, maintain SOPs for their use, ensure an audit trail for each data change and provide for data security. In addition, further relevant documents were consulted. In summary, the following documents were taken into consideration.

Process of expert consensus

Based on requirements extracted from the documents listed above, an already existing ECRIN document on GCP-compliant data management in multinational trials [20], and especially with help from the groundwork of the UK DIMS Project Team [12], a requirements catalogue for GCP-compliant computer system based data management was developed based on expert consensus and formulated as a standard.

To structure the consensus process, the ECRIN Working Group was divided in two subgroups: an Information Technology (IT) subgroup and an Data Management (DM) subgroup, each group working on different requirements. This divison was useful in distinguishing general IT aspects from trial related aspects of data management, though interactions between these two areas were also taken into consideration.

A computer system used for clinical trials cannot be regarded as an isolated application. It must be seen as part of the IT infrastructure of the trial centre using it. The IT subgroup therefore looked at various aspects of that infrastructure as well as systems specifically designed to support clinical trials. Similarly, the DM group examined not just trial specific components of data management systems (in particular the clinical data management application) but also the general aspects of data management systems that are used by all trials.

This cycle was repeated with several iterations (7 telephone conferences). In addition, the group conducted three face-to-face-meetings, one with participation of international accreditation experts (14 September 2009) and a final full day meeting (March 2010) examining all the remaining contentious issues in detail, and in particular the relationships between the IT and DM standards, as well as the final format of the document.

Standards and requirements for GCP-compliant data management

In establishing a consensus for GCP-compliant data management, the underlying IT infrastructure and the relevant indicators of good GCP practice have been the driving force for the ECRIN Working Group on Data Centres. The development of a standard was initiated by a perceived lack of clarity in this area and the existing heterogenity within academic clinical trial data centres [2]. The aim of this approach and the standard requirements developed is to bring ECRIN and other data centres to the same level of quality and standardisation and to make them evolve towards a common quality level. But there is no claim that the ECRIN standard represents the definitive standard for all clinical trial centres. The validity and practicality of the ECRIN standard will need evaluation by clinical trial units data centres themselves. The scope of the standard is limited to the structure, function and use of computer systems for clinical trials and the value, accuracy and integrity of the data generated and does not cover other aspects of clinical trials. Therefore, the ECRIN standard should not be perceived or managed in isolation from other quality management systems in clinical trial units.

There is no attempt to justify each requirement in any detail, though some are commented more specifically. This paper does not consider the possible consequences of non-compliance in any detail and it does not specify any risk of non-compliance. That is seen as an issue for those who might audit a data management centre.

Clinical data management centres vary considerably in their size, in the available resources and in the extent of quality management [2]; and so some centres are much more likely to have reached certain levels of GCP compliance than others. One important discussion point in the Working Group was that the standard has to reflect what the majority of ECRIN partners or other trial units providing data management services can realistically achieve with their available resources. This is a point of concern for many academic research centres.

For instance, system validation plays an important part in ensuring GCP-compliance of a computer system but can be problematic. Academic units do not, in general, have the resources available in the pharma industry to conduct or outsource a 'full' validation for every system component, including the vendor audit, and to maintain complete change management. In addition, there is no simple way to know how much system validation is necessary or sufficient [21] and the extent and depth of validation required may depend on the interpretation of a particular auditor and whether a commercial software or an inhouse developed software is used. We do acknowledge, however, that system validation probably needs to be improved in many ECRIN centres [2].

Accordingly, the ECRIN standard does not stipulate the best possible IT infrastructure, but instead tries to define minimum requirements and best practice. In this way it allows flexibility wherein centres can develop their resources and improve their operations. Best practices are often worthy of implementation, and may over time become more attainable for all centres. In this way, standards, if they are sufficiently supported and explained, may not only become generally accepted but can also be used as a tool for improvement. At this time it would be impractical, for instance, to insist on the use of specific data standards (e.g. the use of CDISC ODM for metadata) despite their many advantages, especially for international trials [22]. Nonetheless, an updated standard may in future include addtional requirements for metadata export using ODM [23] and centres would be expected to comply with it.

Academic clinical trial units are often part of their corresponding university IT infrastructure. In some cases a part or all of the functionality covered by the standard will not be the direct responsibility of the centre or the trials unit itself, falling instead within the remit of the computer centre of the university or even an outside service provider. For example, IT infrastructure services may be provided by the parent organisation, or a commercial host, or another collaborating trials unit. In such cases, the data centre using the ECRIN requirements should have formal written agreements, e.g. in the form of contracts or service level agreements (SLAs), that ensure that the relevant requirements of the standard will be met by the organisation(s) that provides them.

This attempt to introduce a new standard into the field of computer systems in clinical trials must deal with two related questions: a) What is the justification for a new requirements standard for trials units? and b) how does this new standard fit in with existing requirements for IT systems, validation, data management etc?

The prime justification for the standard stems from the recognition that a deficit in clinical research should result in the establishment of a tool to overcome and improve the situation. Clinical research is dependent upon high quality, reliable and GCP-compliant tools, including appropriate IT and data management systems. However and as decribed in "background", no freely available and sufficiently detailed standard currently exists to evaluate such systems.

Similar difficulties have arisen with several other recent initiatives, e.g. the evaluation of breast cancer centres (EUSOMA) [24] and phase I trial centres (MHRA) [25], or the accreditation of centres for first-in-man studies (JACIE) [26]. In these cases too, requirements were first developed to have a standard available to judge the quality of a structure, process or service. The IT/DM requirements developed by ECRIN follow the same path and offer a quality standard for all interested academic clinical centres that conduct electronic data management in clinical trials to support quality assessment within these centres. This standard is, however, not restricted to academic centres; it may also be useful for commercial software and solutions in the pharmaceutical industry.

Methodological approach

The way in which this standard fits with other existing requirement schemes has been partly covered in 'Methods' - i.e. it was certainly informed by those other standards, particularly ICH GCP, but has a different focus and level of detail. Existing validation schemes and security audits, for instance, often have an industry focus (and assume industry-level resources). Detailed examination of data management may be based on FDA demands, that again tend to originate in industry and have corresponding expectations. With the ECRIN requirements list, however, we publish a set of concrete requirements for academic clinical research centres, within the specific perspective of European clinical trials. In this way the new standard complements the different existing approaches and systems. It provides an easy to use and systematic tool for clinical trial units developing or improving their data management system. Our standard is designed to support the validation efforts of centres by defining agreed, minimal requirements that are GCP compliant and that are realistic for academic units.

In order to develop standards different methodological approaches are used. Formal consensus methods, such as Delphi method or nominal group technique offer a structured, transparent and replicable way of synthesizing individual judgements and have been used, for example, in the development of many guidelines. We did not follow strictly a formal consensus development technique but used certain features of these consensus methods in our structured and standardised consensus process. These features covered iterations and controlled feedback with several rounds based on strict versioning of documents with track changes.

Anonymous input was not used in building the consensus. Instead a Working Group of experts was established, which created strong motivation and willingness to participate, provided a useful variety of professional backgrounds in different disciplines, and allowed us to make use of effective communication and collaboration skills developed over many years. Work was based on efficient and regular telephone conferences and face-to-face meetings dedicated solely to the development of the standard requirements. During the process different opinions were expressed by the experts and minority responses dealt with, but there were no severe disagreeements or diversities of opinion on priorities within the group. Nevertheless, there may be limitations in our approach, which could have been overcome by a more formal consensus technique.

Two further aspects have to be discussed. The first aspect is related to the lack of empirical evidence available to the group in coming to a consensus about the most suitable standard to be stipulated. This particular standard is not alone in this - the other guidelines consulted were also all essentially statements of principle, based on ethical principles and domain expertise. The reason for this is simply that there is no (non-anecdotal) evidence - at least that the working party members were aware of - that relates 'good' or 'bad' practice within trial unit data management and IT to particular policies or systems. This underlines the need for a concrete operational set of standards, such as the ECRIN requirements, because before empirical research can be carried out - measuring some index of overall utility, efficiency, safety (etc.) against the practices employed by units - we have to agree on the important aspects of practice to assess. Most of the existing guidelines and policies are too vague to support such a process. The ECRIN standard, however, is specific enough to potentially be useful for this purpose.

The ECRIN standard should therefore be seen as part of a continuously evolving set of tools, to be employed not just in assessing or accrediting trials units, but able to contribute to future data collection, and thus better identification and understanding of the key aspects of organising systems within trials units.

The fact that the requirements developed, whatever their ultimate source, were filtered through the opinions and expertise of group members, is related to the second aspect - the size of the working group. Though the 25 members of the group could claim considerable experience and expertise between them, they obviously did not cover all types of trials units and all the possible variations in practice.

The working group was very aware of this and was therefore keen to disseminate the standard widely - the principal motivation for submitting this paper. It is envisaged that once available publicly the standard will be the subject of further discussion and will evolve further within a wider framework, In particular the group is anxious to engage national regulatory authorities, with a view to exploring how the standard could be used as widely as possible and embedded in normal practice, rather than being seen as an additional set of requirements only linked to ECRIN activities.

Certification

The assembled standard requirements list will be the basis for the planned certification of ECRIN data centres, with a certified centre expected to meet all of the minimal requirements. Best practice standards are not mandatory but have been included to provide guidance to data centres who want to improve their data management practices. Certification as an ECRIN approved data centre will demonstrate, first, compliance of the certified centre with regulations and standards, including GCP, second compliance with recommendations of ECRIN in terms of data management, third, that the centre is staffed by expert personnel, and fourth, that the centre is competent in the management of data for international, multicentre clinical trials. Thus, by implementing the certification procedure based on requirements that are GCP-compliant, ECRIN will guaranty a standard quality level for data management performed by academic trial units in pan European trials.