Table 2 Differing themes and challenges from diagnostic test accuracy trials

Ethics/governance

Patients approached for recruitment require information that is sensitive to their knowledge of their disease.

Timing is sensitive as information may need prior discussion with the participant about what condition tests are designed to detect or rule out. This requires careful consideration of the trial title and the patient information sheet.

Explaining the purpose of the trial, where trial design collects patient samples to evaluate and develop new tests.

Explaining the purpose of the trial to patients can be more complicated where the trial does not list all tests based on patient samples.

Explaining the purpose of trial developing a new test may lead to questions about deficiencies in the current test pathway.

Explaining the purpose of trial evaluating current tests or developing a new test can be complicated, as it may raise questions about the current diagnostic pathway, its accuracy and uncertainties.

No direct benefit to participants.

Trial may be about developing a new test, so trial participants have on direct benefit.

Trial test results availability required for patient care where appropriate.

For ethical reasons, test results may need to be revealed for patient care, particularly in trials comparing tests in current clinical use.

Accessing patient populations

Screening burden.

Eligible patients may need to be screened manually from the hospital clinic list where the clinic conducts a test used to diagnose several diseases, but the trial only relates on one type of referral.

Accessing patients at a clinically relevant time point.

Hospital clinics receive patients through different referral pathways, only some of which may be relevant to the trial.

Hospitals arrange clinical care pathways differently.

Recruitment

Patients eligible for the trial should represent a clinical referral population without clinical selection (spectrum bias)

Requirement for an additional clinician referral required by the trial can affect who gets recruited.

Clinicians can misunderstand eligibility criteria and select participants by referral plus their clinical suspicion.

Recruitment requires collaboration between specialist disease and general clinical teams

Patient recruitment may occur in the radiology department for trial diagnosing a specific disease requiring recruitment in clinics without research staff directly involved in the trial.

Collaboration across clinical settings

Patient pathway in the hospital may depend on the interpretation of tests made by primary care clinicians.

Patient preferences

Research burden.

Participants may find it difficult to attend additional hospital appointments or travelling to a different hospital site when they are feeling ill, or it takes extra time from family care or work.

Reference test may be too onerous to patients.

For some participants, follow-up may require an additional invasive or unpleasant test, which may be too burdensome.

Test processes, clinical pathways and sample storage

Trial may require a change in clinic patient pathway or referral to specialist test clinic.

Trial pathway may require referral to staff trained in a new test method.

Test may not be available in a recruiting centre but may take place at a specialist hospital clinic.

An extra test or patient sample is required for trial.

Different clinics may need to order or interpret the test, resulting in delays or reduced recruitment.

Test may require extra staff training to conduct or interpret test results.

Can restrict recruitment to time periods when staff with specialist test training are on duty.

Extra staff training may be required.

Trial includes standardisation of test processes to a higher standard than normal clinical practice.

Tests may require additional burden on clinics of specialist equipment or calibration of equipment or additional software or extra clinical interpretation.

Patient sample may need immediate testing by a specialist laboratory.

Recruitment sites had to be within a certain distance of laboratory to enable sample testing within a required time.

Requires additional requirements for collection and storage of trial samples, in addition to normal clinical practice.

Additional burden to clinics including anonymisation of images, cloud/server storage of images, sending samples, collation of patient information for interpretation at a second site, etc.

Trial may store samples of blood or tissue for future test development.

Requires curation and future proofing tissue storage.

Uncertainty of test results

In tests requiring interpretation by clinicians, clinicians can feel their ability is being tested, rather than the test.

Clinicians may be cautious or report some data intermittently if they feel the trial is measuring their performance rather than the performance of the test in a typical clinical setting.

Calibration of the index and reference tests across time can be problematic.

Calibration and standardisation of tests and equipment over time can affect diagnostic performance.

Verifying diagnosis (reference standard)

Reference standard may require formation of an expert consensus group to review and interpret patient diagnosis based on several tests.

Requires extra time from staff and clinicians to prepare and attend meetings.

Processes to resolve disagreements are needed.

Reference test may depend on the results from the index tests.

Additional tests may be required when index tests disagree.

Reference test requires a specialist test.

If a specialist test is required as part of the reference test, it may not be available at all hospital sites and may require specialist training.

Follow-up

Patient follow-up is often needed to confirm disease status.

Participants may be unwilling to return for “unnecessary” appointments.

Clinicians may be unwilling to refer participants for additional follow-up tests not part of usual clinical practice.

Clinical guidelines can change and affect tests forming part of trial follow-up.

Follow-up tests and procedures can be different for participants, depending on their test results during the trial.

Follow-up for a patient can vary from further imaging, surgery, histology of sample from surgery to follow-up of primary care records.

Follow-up may require collaboration across clinics or healthcare settings.

Follow-up in participants with no disease may require follow-up through primary care, patient contact or national databases.

Adverse effects or harms

Can be difficult for clinical staff to understand which AE in the clinical pathway should be recorded as relevant to the trial.

Discussion of AE relevant to the trial needs careful definition differentiating AE from the test itself or AE in patient pathway alternation by use of the test.

AE can be caused by consenting a patient for testing within a trial.

Informed consent may make participants more aware of the range of diagnoses from tests than discussed in clinical practice. Trial participation may also prompt more discussion about diagnostic uncertainty from current tests.

Diagnostic impact

Measuring the impact of tests on patient management.

Patient management decisions can be captured within diagnostic accuracy trials. The highest level of evidence is from an RCT of using tests as an intervention, but alternative designs reporting patient management decisions instead of patient outcomes can, for some tests, provide important evidence on test impact faster and at a lower cost.

Understanding the incremental benefit of a particular test within a complex patient pathway is difficult.

Separating a test to measure its individual impact can become artificial.

Understanding what difference in diagnostic accuracy or diagnostic confidence is required to change clinical practice.

Often adding a test into a test pathway has an incremental value on certainty or timing of diagnosis.