Participants’ understanding of a randomized controlled trial (RCT) through informed consent procedures in the RCT for breast cancer screening, J-START

Abstract

Background

It is often difficult to enrol healthy volunteers into a randomized controlled trial (RCT) as there are barriers to participants’ proper understanding of a trial. This study aimed to evaluate degrees of understanding of the informed consent (IC) process among healthy volunteers who participated in an RCT. Additionally, factors associated with degree of understanding were investigated.

Methods

The J-START (the Japan STrategic Anti-cancer Randomized controlled Trial) is an RCT investigating the effectiveness of ultrasonography screening for breast cancer in women aged 40 to 49 years. To evaluate participants’ understanding of the J-START, we administered questionnaires to 376 Japanese women on the day of enrolment at five study sites across Japan. The respondents were asked to complete the anonymous questionnaire within 2 weeks. We assessed objective understanding and perceived subjective understanding of IC using a Japanese version of the Quality of Informed Consent scale (QuIC). Then we analyzed the characteristics of women whose understanding was poor, and clarified the association between providing information and their understanding of the study protocol.

Results

The average QuIC scores were 78.2 and 82.2 (out of 100 each) for objective and subjective understanding, respectively. These are generally acceptable scores for participants’ understanding of an RCT. However, there were four domains with low scores, indicating poor understanding: (1) experimental nature of the study, (2) potential risks or discomfort, (3) benefit to self, and (4) compensation.

Conclusions

Healthy volunteers generally well understood the J-START. Nevertheless, there were some domains in need of improvement. In order to facilitate participants’ understanding, it is necessary to provide training to reduce differences in information-providing procedures between medical centres and to endeavour to provide consistent information and conditions.

Trial registration

The J-START was registered with the University Hospital Medical Information Network Clinical Trial Registration (UMIN-CTR), Japan (registration number: UMIN000000757), on July 1, 2007.

Background

It has been suggested that the complex design of a clinical trial, such as a randomized controlled trial (RCT), is difficult to understand for participants. Although participants are enrolled in a trial only after undergoing the informed consent (IC) process, their understanding of the trial in which they are participating is often insufficient [1–9]. The following two factors might account for this insufficient understanding. First, participants cannot distinguish already established, evidenced examination and treatment procedures from study protocols necessitated by treatment that has not yet been established [1–5]. Second, it is difficult to understand the concepts that are characteristic of RCTs, such as randomization and placebos [5–9].

To our knowledge, no large-scale RCT with more than 70,000 healthy volunteers from the general population had been conducted in Japan before the Japan STrategic Anti-cancer Randomized controlled Trial (J-START) was initiated [10]. The J-START is an RCT on the effectiveness of ultrasonography in breast cancer screening that was developed to assess breast cancer screening in Japan in 2007.

According to a survey conducted in Japan on the attitudes of the healthy general population toward clinical trials [11], participation in them is widely recognized as a volunteer activity that provides an opportunity to contribute to society. However, it has been reported that participants do not sufficiently understand the definitions and contents of clinical trials [11]. There are no reports on the understanding of an RCT targeting a general population of healthy women in Japan. Moreover, there are guidelines but no laws about clinical trials except for pharmaceutical products in Japan. Therefore, as a matter of ethics, clinical trials conducted in Japan should make an effort to provide understandable ICs for eligible participants. However, there is no validity of the ICs provided in Japan with respect to ethics compared with trials in other countries. Thus, we surveyed the degrees of objective and subjective understanding of the IC process among J-START participants. Additionally, factors associated with degrees of understanding were investigated. This is a survey study of the quality of IC given to participants of an RCT to clarify what would be helpful for future trials.

Methods

Overview of J-START

J-START is a multi-centre RCT with 42 participating medical centres in 23 prefectures across Japan. It is supported by science research funds from the Japan Ministry of Health Labour, and Welfare. The aim of the J-START is to assess the effectiveness of screenings for breast cancer among women in their 40s. As of 31 March 2011, 76,196 healthy female volunteers had enrolled [12]. The primary endpoints were sensitivity, specificity, and advanced cancer ratio in both screening groups: the intervention group (women who received ultrasonography in addition to mammography) and the control group (women who received mammography only). The research coordinators verbally explained the purpose and method of the RCT to all eligible participants while collecting ICs.

J-START participants were recruited by trained research coordinators. Potential participants were given a supplementary leaflet by mail before visiting the medical centres. At the medical centres, an educational video recording of the study was presented and a verbal description and a booklet were provided for IC. The total time of the educational video was almost 7 minutes, and it played continuously in the visitors’ waiting rooms of the medical centres.

Participants and study methods

Of the 42 medical centres participating in J-START, five medical centres in Iwate, Yamagata, Tokyo, Okayama, and Okinawa participated in our study. From 10 November 2010 to 31 January 2011, we distributed anonymous questionnaires to newly enrolled participants regarding their understanding of the RCT at the five medical centres in different Japanese prefectures. At the time we received approval for this survey in September 2010, only the above-cited medical centres were recruiting new participants; furthermore, only those centres had the proper personnel for conducting the accompanying research.

We distributed the anonymous questionnaires regarding participants’ understanding of the RCT at the medical centres. Informed consent for participating in our investigation of participants’ understanding of the RCT was assumed by response to the questionnaire. The set of documents distributed to each participant included a letter describing the purpose and method of the study, the questionnaire, and a return envelope. Ethical approval was obtained from the Institutional Review Board of Tohoku University Graduate School of Medicine on 27 September 2010 (No. 2010–279).

Questionnaire

Objective understanding (actual knowledge) and subjective understanding (self-assessed understanding) were measured using a Japanese version of the Quality of Informed Consent (QuIC), which is a scale established in the United States to assess participants’ knowledge and self-assessment of clinical trials [6]. The items cover the basic elements regarding the protection of human participants as stated in US federal regulations (45CFR46) [13]. In addition to English [6, 14–16], the QuIC has been translated into French [17] and Swedish [18]. It has been used not only in clinical trials for developing anticancer agents but also in studies on social support [19] and in a genome cohort study [14].

Part A of the Japanese version of the QuIC contains 16 items measuring actual knowledge of the basic elements of IC. The total QuIC Part A score represents an average of the 16 items on objective understanding, ranging from 0 (lowest) to 100 (highest) [6]. Each item is measured with a triple-bounded binary-choice format: “Totally agree”, “Unsure”, and “Strongly disagree”. The points were given as follows: 100 points for the correct answer, 50 points for “Unsure”, which is neither correct nor incorrect, and 0 points for a wrong answer. Answers without a response were marked “No reply”. There were four negatively-phrased items, for which the correct answer was “Disagree”, so as to avoid agreement bias that might result from all correct answers being “Agree” [6].

Part B of the Japanese version of the QuIC consists of 14 items assessing subjective understanding of important elements of the trial. The total score of the QuIC Part B is an average of the scores for the 14 items [6]. Each item was measured with a quintuple-bounded binary-choice format: 100 points for “Understood very well”, 75 points for “Generally understood”, 50 points for “Neither understood nor did not understand”, 25 points for “Almost did not understand”, and 0 points for “Did not understand”.

The J-START is an RCT that enrolled healthy volunteers as the participants for breast cancer screening. For that reason, the QuIC questions that pertained only to Phase I clinical studies were deleted (in the original QuIC full version: questions A6, A7, A8, and A10). We then had one Japanese and one Chinese healthcare worker who use English and Japanese on a daily basis translate the remaining questions into Japanese sentences that would be readily understandable in the context of current Japanese culture. Next, we had back-translated that Japanese into English by a different person who used English and Japanese on a daily basis. Finally, that back-translated version was checked by a person whose native language was English. The comprehensibility of the technical terms and expressions in not only the Japanese QuIC, but also the full questionnaire including original items, was checked by having a medical ethics specialist and medical staffers complete the entire questionnaire. We then made revisions to the materials based on their replies. To ensure the validity of the full questionnaire, we evaluated the materials’ reliability by test-retest as following. We asked the new participants in J-START, which has the same conditions as the present study, to answer the full questionnaire, including the Japanese version of QuIC and other items, twice, at a 2-week interval. Twenty-one participants answered all the questions in the questionnaire twice. Weighted kappa statistics for QuIC ranged from 0.32 (objective risks) to 0.67 (subjective experimental nature of study), respectively. Cronbach’s alpha values for QuIC for internal consistency were 0.57 and 0.89 for Parts A and B, respectively.

Other questionnaires

We administered additional questionnaires to measure age, educational level, marital status, work status, and study allocation of the RCT. These also assessed impressions of materials provided and prior knowledge about J-START and RCTs. These questionnaires and the Japanese QuIC were validated by the pilot study at the same time.

Multimedia information materials had been prepared for J-START (for example, leaflets, educational DVDs). Their effectiveness was assessed with quintuple-bounded binary-choice items (“Very helpful”, “Almost helpful”, “Neither helpful nor unhelpful”, “Almost unhelpful”, and “Not at all helpful”). In addition, alternative responses were prepared as follows: “Have not read”, and “Have not watched”.

The effectiveness of the verbal description during IC was assessed with the following five questions: (1) “The research coordinator facilitated your understanding”. (2) “You had an opportunity to ask questions”. (3) “You had sufficient time to understand the process”. (4) “It was easy to say ‘no’, if you did not want to continue your participation”. (5) “You sought further explanation of the study”. The answer to these five questions was coded as follows: 1 for “Yes” and 0 for “No” or “Unsure”.

Statistical analysis

Continuous variables are reported with means and standard deviations, whereas categorical variables are reported as percentages. Bivariate correlations were conducted between the QuIC scores and each independent variable. Correlations were assessed using chi-squared tests. All analyses were conducted using SAS statistical software (version 9.3; SAS Institute Inc., Cary, NC, USA).

Results

Participant characteristics

Participant characteristics are shown in Table 1. Of the 745 questionnaires distributed, 376 (50.4%) were returned. Among 376 survey respondents, almost 70% graduated from vocational school, junior college, or above. There were 280 married participants (74.5%), and 308 participants (81.9%) were employed. In relation to group allocation of the RCT, 198 participants (52.7%) were assigned to the intervention group, and 162 participants (43.1%) were assigned to the control group.

Table 1 Characteristics of the study participants

Objective understanding

The results of the objective understanding (QuIC Part A) are shown in Table 2. The average QuIC Part A score was 78.2 (95% CI, 68.6 to 87.8). Participants generally understood most question items well. However, the accuracy rates for some items were low. Specifically, there were accuracy rates of 14.6% for Question A4 on the “Experimental nature of study”, 14.1% for Question A8 on “Potential risks or discomforts”, 34.6% for Question A9 on “Benefit to self”, and 33.0% for Question A13 on “Compensation”.

Table 2 Percentage of participants by QuIC Part A score: an objective understanding of J-START

Subjective understanding

The results of the subjective understanding (QuIC Part B) are shown in Table 3. The average QuIC Part B score was 82.2 (95% CI, 69.3 to 95.1). However, for Question B11 on “Compensation”, “Who will pay for treatment if you are injured or become ill because of participation in J-START?”, the combined percentage of participants who indicated that “I understood very well” and “I generally understood” was only 44.7%. For Question B6 on “Potential risks or discomforts”, the total percentage of participants who indicated that “I understood very well” and “I generally understood” was 78.2%. This question item corresponded to Question A8 on “Potential risks or discomforts”. It is worth noting that, although objective understanding of this item was low, subjective understanding was high.

Table 3 Percentage of participants by QuIC Part B score: a subjective understanding of J-START

Assessment of explanatory materials and information-providing procedures

In order to examine how various information-providing procedures used during the IC process related to participants’ understanding, we asked participants about their prior knowledge and asked them to give their impression of the materials used for explanation and verbal description, and to evaluate each procedure (Table 4). Although informational leaflets explaining the trial had been mailed to the participants in advance, 279 (74.2%) indicated that they did not have any prior knowledge of J-START. The participants highly valued the informational leaflets and the educational videos explaining the trial. These were regarded as “Helpful for understanding” by 266 (70.7%) and 277 participants (73.7%), respectively. Three hundred and forty-six participants (92.0%) reported that their understanding had been confirmed by the research coordinator at the end of the oral description during the IC process. It would be expected that, at the time of that interaction, the research coordinator would become cognizant of the participant’s questions and points of confusion, but we do not know what sort of communication took place later in this study. Only 10 participants (2.7%) indicated that they sought further explanation of the trial at that time.

Table 4 Questionnaire on participants’ impression of the informed consent process

Factors associated with low objective understanding

Table 5 shows the association between factors at the time of the IC process and the four objective understanding items with low accuracy rates; that is, A4, A8, A9, and A13 of QuIC Part A. The medical centre showed a correlation with two questions: Question A8 on “Potential risks or discomforts” (P = 0.009) and Question A13 on “Compensation” (P < 0.0001). The existence of prior knowledge of the RCT or J-START itself was statistically significant only for Question A4 on “Experimental nature of study”, (RCT, P = 0.002; J-START, P = 0.003). The educational video showed a correlation with the two questions A8 on “Potential risks or discomforts” (P = 0.003) and A13 on “Compensation” (P = 0.001). Sufficient opportunity to ask a question showed correlations with Question A8 on “Potential risks or discomforts” (P = 0.025), Question A9 on “Benefit to self” (P = 0.023), and Question A13 on “Compensation” (P = 0.049), while enough time to achieve understanding showed correlations with Question A8 on “Potential risks or discomforts” (P = 0.007) and Question A13 on “Compensation” (P = 0.013). The atmosphere at the venue when the decision to participate was made showed correlations with Question A9 on “Benefit to self” (P = 0.001) and Question A13 on “Compensation” (P = 0.043), but for each case there was a low percentage of participants who felt the atmosphere made it hard to refuse to participate. No statistically significant correlation was found between the degree of understanding and the following factors: education level, marital status, work status, informational leaflet, whether the research staff confirmed whether the participant understood, or whether the participant wanted to ask further questions at the time of IC.

Table 5 Factors associated with low comprehension scores of QuIC Part A items

Discussion

The present study aimed to evaluate the degree of participants’ understanding of an RCT and investigate the associated factors. We administered questionnaires to 376 healthy Japanese women on the day of enrolment at five study sites using a Japanese version of the QuIC. Although healthy volunteers generally well understood J-START, there were some domains in need of improvement. Until the present study, there have been no reports on either the understanding of an RCT targeting the general population of healthy women or reports concerning evaluations using international and validated scales in Japan. The present study has demonstrated hints of improvement in the IC process in an RCT that targeted the general population of healthy people. The participants generally provided correct responses to the majority of the questions. Furthermore, the QuIC scores in the present survey were comparable to the scores in the preceding studies [14–16, 18]. The results of the present study revealed a tendency for the degree of subjective understanding to be higher than the degree of objective understanding, which is similar to the findings of earlier studies. The higher degree of subjective understanding can be thought to be related to the ease of understanding the IC and/or a feeling of satisfaction with the amount of information, but the reason for discrepancy with the degree of objective understanding warrants further study. Thus, the purpose of IC seems to have largely been achieved. However, among the items specified by the US Federal Regulation (Chapter 45, Part 46) that should be explained to individuals eligible for participation in a clinical trial, “Experimental nature of study”, “Potential risks or discomforts”, “Benefit to self”, and “Compensation” were not correctly understood by some participants. In previous studies, “Experimental nature of study” [1, 8, 14, 18], “Potential risks or discomfort” [8, 14, 18, 20],“Benefit to self” [5], and “Compensation” [14, 18] were reported to be difficult for participants to understand. Therefore, strategies need to be developed to facilitate better understanding of these items among participants.

It is necessary to consider the current status of breast cancer screening in Japan, giving consideration to these three factors: “Experimental nature of the study”, “Potential risks or discomforts”, and “Compensation”. Regarding “Experimental nature of the study” and “Potential risks or discomforts”, because of the comfort level and familiarity derived from the fact that ultrasonography is a well-known method, participants are unlikely to understand that a trial is conducted due to the lack of any established intervention, and that there are potential risks or discomforts involved. Moreover, regarding “Compensation”, participants might not have paid attention to the explanation of compensation for any problems about intervention. Mammography, which is currently used for breast cancer screening, is known to sometimes cause pain from the compression of a breast, as well as minor exposure to x-rays during imaging. Ultrasonography is generally perceived to be more comfortable than mammography. Therefore, it might be difficult to imagine that health problems could be caused by ultrasonography. Moreover, because breast ultrasonography has already been employed as a breast cancer screening method for opportunistic screening (for example, complete physical examination), a perception of this procedure as a well-established screening method might not reflect the true situation. In fact, scientific evidence for breast ultrasonography as a method for mass screening has remained under investigation in J-START since 2007. Because the procedure has been employed as an effective screening method in routine practice in breast surgery departments, the present survey might have included participants who had previously undergone ultrasonography as a component of their medical care.

There were three possible reasons for participants’ misunderstanding of IC, which have been referenced in previous studies. First, the explanatory materials might have been difficult to understand [21, 22]. Second, the comprehension ability of participants might have been insufficient [4, 23]. Third, the information-providing procedures used during the IC process might have varied between medical centres [9].

First, in relation to difficult explanatory materials, well-considered wording [22] and explanations using written, paper-based information (for example, leaflets and booklets) appear to improve understanding of RCTs [21]. Further, it is preferable to provide information with a combination of several materials [24]. When we prepared the informational leaflets and the educational video, we sought to use simple words and considered a combination of materials. The resultant material was also approved by the institutional ethical review board. As shown in Table 4, the majority of the participants indicated that the explanatory materials were “Helpful for understanding”. This demonstrates that the explanatory materials were perceived to be sufficiently easy to understand. Table 5 indicates that prior knowledge influenced misunderstanding of “Experimental nature of the study”, but prior knowledge had not always been given by our leaflet, and the educational video did not help correct misconceptions. In a study simulating a situation in which children and their parents undergo the IC process, it was reported that multimedia materials with visual and auditory information are preferable to paper-based information and could be expected to improve understanding, particularly among parents [25]. Other studies have indicated that a simplified IC form with emphasis added by verbal description [26], and user testing, which examines not only the wording of leaflets, but also the layout and paper thickness, in order to design interesting materials for participants to read, leads to improved explanatory materials and better understanding [27]. However, in systematic reviews of understanding of RCTs [28] and of trials and ICs [29], it was reported that multimedia informational materials were not as effective for improving participants’ understanding as test-feedback quizzes or discussion of the IC process [28, 29]. The result of our survey corroborated the systematic reviews. Improving face-to-face communication would foster better understanding than improving written materials.

Second, regarding the comprehension ability of participants, approximately 70% of the participants in the present study graduated from vocational school, junior college, or above (Table 1). Because the illiteracy rate in Japan is low due to the 9 years of mandatory education, insufficient comprehension ability is an unlikely explanation.

Regarding the third possible reason for participants’ misunderstanding of IC, there were, unfortunately, significant gaps or inconsistencies between the IC information-providing procedures of the different medical centres (Table 5). Regrettably, we did not have the data on how IC was obtained from each of the participants. For the standardized information dissemination process in J-START, the manual included specific procedures concerning IC and the use of materials, and held several training sessions for the research coordinators, who had previous experience working at a medical centre (for example, nurses and public health nurses). To avoid misconceptions arising from verbal communication, explanatory materials were also integrated into the IC process. Despite this, a certain number of participants reported as being unsure about these materials. However, we did not have the IC process data, and it was not clear that the process was lost or the participants did not read or watch. Given that explanatory materials are reported to improve participants’ understanding [8], the differences in understanding observed in the present study might be attributable to insufficient use of the materials. A previous systematic review reported that having a long discussion time with a team member and a neutral educator was effective for participants’ understanding, and it worked even over the telephone [29]. In J-START, this type of discussion was not programmed officially. If we could have had the opportunity to include such a discussion, IC and accompanying materials might have been delivered more effectively. In order to more effectively perform the IC process, we must create opportunities for the team members to have frequent discussions with the educator, with the topic of discussion to include better information-providing procedures.

A limitation of the present study was that not all of the J-START study sites were included. However, the validity of the QuIC was evaluated and the reliability of the questionnaire was tested. Significantly more participants in the intervention group returned the questionnaires than in the control group. Therefore, the present findings were not derived from random sampling with respect to which participants received an explanation of J-START. Unfortunately, we had to use the most convenient sampling method, and could not collect data on non-responders without authorization.

Conclusions

Healthy volunteers generally well understood the RCT in which they participated. The results of the present study suggest that when an RCT on a minimally invasive intervention is conducted, both the researchers who provide explanations and the participants might neglect the following four items: the fact that interventions under investigation have not been standardized, the possible benefits and disadvantages of participating in the trial, and compensation in case of injury resulting from participation. In order to facilitate participants’ understanding, it is necessary to provide training for the research team members to reduce differences in information-providing procedures between medical centres and to endeavour to provide consistent information and conditions.