Skip to main content

Advertisement

Table 3 Summary of item scores by survey round and outcome of consensus meeting

From: Development of a standardised set of metrics for monitoring site performance in multicentre randomised trials: a Delphi study

Metric and domain Round 1 scores Round 2 scores Round 3 scores % of consensus meeting participants voting to retain item
1–3 4–6 7–9 10 1–3 4–6 7–9 10 1–3 4–6 7–9 10
n (%) respondents per rating category (1-3 ‘not important’, 4-6 ‘important but not critical’, 7-9 ‘critical’, 10 unable to score)
Recruitment and retention 1. Total actual recruitment versus total target recruitment a,c 1 (0.5) 36 (17.1) 172 (81.5) 2 (0.9) 0 16 (7.6) 194 (91.9) 1 (0.5) 0 14 (6.6) 197 (93.4) 0 100
2. Time from the site opening to first participant randomised 8 (3.8) 108 (51.2) 94 (44.5) 1 (0.5) 3 (1.4) 118 (55.9) 90 (42.7) 0 5 (2.4) 126 (59.7) 80 (37.9) 0  
3. Number of days/weeks since the most recent participant was randomised 21 (10.0) 112 (53.1) 77 (36.5) 1 (0.5) 12 (5.7) 142 (67.3) 57 (27.0) 0 11 (5.2) 155 (73.5) 45 (21.3) 0  
4. Percentage of potential participants screened who have been randomised 4 (1.9) 88 (41.7) 117 (55.5) 2 (0.9) 2 (0.9) 83 (39.3) 125 (59.2) 1 (0.5) 0 76 (36) 134 (63.5) 1 (0.5)  
5. Percentage of potential participants who could have been screened, who were screened 13 (6.2) 97 (46.0) 92 (43.6) 9 (4.3) 10 (4.7) 90 (42.7) 103 (48.8) 8 (3.8)  
6. Percentage of potential participants screened who were eligible 9 (4.3) 106 (50.2) 93 (44.1) 3 (1.4) 6 (2.8) 110 (52.1) 92 (43.6) 3 (1.4)  
7. Percentage of potential participants eligible who have consented b,c 8 (3.8) 81 (38.4) 119 (56.4) 3 (1.4) 3 (1.4) 77 (36.5) 128 (60.7) 3 (1.4) 95
8. Percentage of potential participants who have consented and have been randomiseda 5 (2.4) 71 (33.6) 131 (62.1) 4 (1.9) 2 (0.9) 57 (27) 150 (71.1) 2 (0.9) 35
9. Percentage of randomised participants who have withdrawn consent to continue in the studya,c 8 (3.8) 76 (36.0) 125 (59.2) 2 (0.9) 4 (1.9) 60 (28.4) 147 (69.7) 0 4 (1.9) 46 (21.8) 161 (76.3) 0 83
10. Percentage of randomised participants lost to follow-upa 10 (4.7) 59 (28.0) 140 (66.4) 2 (0.9) 3 (1.4) 38 (18) 169 (80.1) 1 (0.5) 3 (1.4) 24 (11.4) 183 (86.7) 1 (0.5) 22
11. Percentage of screening logs returned on time out of all those that should have been returned 40 (19.0) 135 (64.0) 33 (15.6) 3 (1.4) 29 (13.7) 159 (75.4) 22 (10.4) 1 (0.5) 23 (10.9) 167 (79.1) 20 (9.5) 1 (0.5)  
12. Percentage of screening items completed of those required 32 (15.2) 105 (49.8) 67 (31.8) 7 (3.3) 20 (9.5) 114 (54) 72 (34.1) 5 (2.4) 18 (8.5) 117 (55.5) 72 (34.1) 4 (1.9)  
13. Percentage of randomised participants with a consent form that is incomplete or inaccuratea 11 (5.2) 51 (24.2) 148 (70.1) 1 (0.5) 8 (3.8) 31 (14.7) 172 (81.5) 0 9 (4.3) 14 (6.6) 187 (88.6) 1 (0.5) 13
14. Percentage of all expected forms that have been receiveda      8 (3.8) 69 (32.7) 128 (60.7) 6 (2.8) 4 (1.9) 50 (23.7) 154 (73) 3 (1.4) 39
15. Percentage of randomised participants with any issues or problems with consenta 10 (4.7) 68 (32.2) 129 (61.1) 4 (1.9) 6 (2.8) 53 (25.1) 150 (71.1) 2 (0.9) 4 (1.9) 34 (16.1) 169 (80.1) 4 (1.9) 26
16. Percentage of randomised participants for whom documentation of consent is missing from their medical recordsa 15 (7.1) 69 (32.7) 123 (58.3) 4 (1.9) 9 (4.3) 47 (22.3) 154 (73) 1 (0.5) 7 (3.3) 31 (14.7) 172 (81.5) 1 (0.5) 0
Data quality 17. Percentage of randomised participants with the time between data collection and either data entry (electronic case report form) or central receipt of paper case report form within the target timeframe 12 (5.7) 129 (61.1) 66 (31.3) 4 (1.9) 8 (3.8) 156 (73.9) 45 (21.3) 2 (0.9) 7 (3.3) 170 (80.6) 32 (15.2) 2 (0.9)  
18. Percentage of randomised participants with a query/queries for primary outcome data a,c 4 (1.9) 59 (28.0) 145 (68.7) 3 (1.4) 3 (1.4) 36 (17.1) 170 (80.6) 2 (0.9) 4 (1.9) 23 (10.9) 182 (86.3) 2 (0.9) 65
19. Percentage of randomised participants with query/queries for secondary outcome data 16 (7.6) 128 (60.7) 65 (30.8) 2 (0.9) 8 (3.8) 156 (73.9) 46 (21.8) 1 (0.5) 8 (3.8) 162 (76.8) 40 (19) 1 (0.5)  
20. Time taken between sending a data query and resolution of the query 17 (8.1) 140 (66.4) 52 (24.6) 2 (0.9) 10 (4.7) 164 (77.7) 36 (17.1) 1 (0.5) 9 (4.3) 167 (79.1) 34 (16.1) 1 (0.5)  
21. Percentage of randomised participants with complete data for primary and important secondary outcomesa,c 2(0.9) 44 (20.9) 163 (77.3) 2 (0.9) 1 (0.5) 20 (9.5%) 189 (89.6) 1 (0.5) 1 (0.5) 11 (5.2) 198 (93.8) 1 (0.5) 96
22. Percentage of randomised participants with complete data 3 (1.4) 85 (40.3) 120 (56.9) 3 (1.4) 0 88 (41.7) 122 (57.8) 1 (0.5) 0 91 (43.1) 119 (56.4) 1 (0.5)  
23. Percentage of unresolved serious adverse event queries > 30 calendar days from the date the query was generated a 3 (1.4) 44 (20.9) 163 (77.3) 1 (0.5) 1 (0.5) 24 (11.4) 186 (88.2) 0 1 (0.5) 12 (5.7) 198 (93.8) 0 9
24. Total number of adverse events and serious adverse events reported per number of randomised participants 32 (15.2) 101 (47.9) 74 (35.1) 4 (1.9) 19 (9) 130 (61.6) 59 (28.0) 3 (1.4) 17 (8.1) 138 (65.4) 53 (25.1) 3 (1.4)  
25. Number of serious adverse events reported per number of randomised participants 25 (11.8) 84 (39.8) 98 (46.4) 4 (1.9) 16 (7.6) 90 (42.7) 102 (48.3) 3 (1.4) 15 (7.1) 102 (48.3) 91 (43.1) 3 (1.4)  
26. Number of adverse events reported per number of randomised participants b,c 40 (19) 106 (50.2) 60 (28.4) 5 (2.4) 27 (12.8) 136 (64.5) 45 (21.3) 3 (1.4) 24 (11.4) 148 (70.1) 36 (17.1) 3 (1.4) 81
Protocol compliance 27. Percentage of randomised participants with at least one protocol violation a,c 6 (2.8) 78 (37) 124 (58.8) 3 (1.4) 1 (0.5) 64 (30.3) 145 (68.7) 1 (0.5) 0 47 (22.3) 163 (77.3) 1 (0.5) 76
28. Percentage of randomised participants receiving allocated intervention as intended per protocol a,c 2 (0.9) 48 (22.7) 158 (74.9) 3 (1.4) 0 19 (9.0) 191 (90.5) 1 (0.5) 0 11 (5.2) 199 (94.3) 1 (0.5) 100
29. Number of missed visits per number of randomised participants a 7 (3.3) 93 (44.1) 107 (50.7) 4 (1.9) 5 (2.4) 75 (35.5) 128 (60.7) 3 (1.4) 4 (1.9) 52 (24.6) 152 (72) 3 (1.4) 10
30. Number of late visits per number of randomised participants 18 (8.5) 128 (60.7) 61 (28.9) 4 (1.9) 10 (4.7) 157 (74.4) 41 (19.4) 3 (1.4) 9 (4.3) 162 (76.8) 37 (17.5) 3 (1.4)  
31. Number of critical or major audit findings per number of randomised participants a 6 (2.8) 43 (20.4) 152 (72) 10 (4.7) 4 (1.9) 23 (10.9) 179 (84.8) 5 (2.4) 3 (1.4) 14 (6.6) 190 (90) 4 (1.9) 0
Staff 32. Number of contacts from site staff to the central trial team within a given time period 79 (37.4) 112 (53.1) 14 (6.6) 6 (2.8) 76 (36) 124 (58.8) 6 (2.8) 5 (2.4%) 81 (38.4) 120 (56.9) 6 (2.8) 4 (1.9)  
33. Time between protocol amendment being sent and principal investigator sign-off 22 (10.4) 110 (52.1) 73 (34.6) 6 (2.8) 16 (7.6) 127 (60.2) 65 (30.8) 3 (1.4) 15 (7.1) 140 (66.4) 53 (25.1) 3 (1.4)  
34. Cumulative number of staff included on the delegation of duties log 105 (49.8) 87 (41.2) 12 (5.7) 7 (3.3) 116 (55.0) 83 (39.3) 7 (3.3) 5 (2.4)  
  1. 1–3 not important, 4–6 important but not critical, 7–9 critical and 10 unable to score
  2. Scores shown for the 211 participants who completed all three rounds of the Delphi survey. Cells containing a dash indicate metrics that were added at round 2 and therefore not scored in round 1
  3. aMetrics (n = 15) reaching consensus in status after survey round 3 and taken forward to the consensus meeting
  4. b Metrics 7 and 26 were also discussed and voted on at the meeting
  5. cMetrics receiving > 50% of the vote at the meeting and retained in the final set