Exploring Understanding of “Understanding”: The Paradigm Case of Biobank Consent Comprehension

Abstract

Data documenting poor understanding among research participants and real-time efforts to assess comprehension in large-scale studies are focusing new attention on informed consent comprehension. Within the context of biobanking consent, we previously convened a multidisciplinary panel to reach consensus about what information must be understood for a prospective participant’s consent to be considered valid. Subsequently, we presented them with data from another study showing that many U.S. adults would fail to comprehend the information the panel had deemed essential. When asked to evaluate the importance of the information again, panelists’ opinions shifted dramatically in the direction of requiring that less information be understood. Follow-up interviews indicated significant uncertainty about defining a threshold of understanding and what should happen when prospective participants are unable to grasp key information. These findings have important implications for urgently needed discussion of whether consent comprehension is an ethical requirement or an ethical aspiration.

Keywords:

• Informed consent
• comprehension
• autonomy
• voluntariness
• research ethics
• biobanking

This article is referred to by:

Informed Consent Requires Understanding: Complete Disclosure Is Not Enough

Disruption, Diversity, and Global Biobanking

Reciprocity and the Quest for Meaningful Disclosure

Rejecting “Understanding”: An Ethical Proposal Whose Time Has Come

The Value of Consent for Clinical Research Does Not Always Hinge on Understanding

Ignorance Isn’t Bliss: Retaining a Meaningful Comprehension Requirement for Consent to Research

Moving From Understanding of Consent Conditions to Heuristics of Trust

Understanding as an Ethical Aspiration in an Era of Digital Technology-Based Communication: An Analysis of Informed Consent Functions

Measuring Understanding and Respecting Trust in Biobank Consent

The Unbearable Requirement of Informed Consent

Understanding (in) Consent for Governance

Being Polite: Why Biobank Consent Comprehension Is Neither a Requirement nor an Aspiration

What Is Adequate Understanding?

INTRODUCTION

A confluence of events has shone a spotlight on the concept of comprehension, long considered a fundamental pillar of informed consent (National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research 1979). Responding to considerable empirical evidence that many participants fail to understand basic aspects of studies in which they have enrolled, new federal regulations require that consent forms be organized in a way that facilitates comprehension of the information a reasonable person would want to know (Menikoff, Kaneshiro, and Pritchard 2017; Hodge and Gostin 2017). In addition, changing research methods and advances in information technology have given rise to new approaches to informed consent (Grady et al. 2017). For example, the advent of precision medicine research (such as the All of Us Research Program [National Institutes of Health 2018; Sankar and Parker 2017]), collaborative pragmatic trials (such as the ADAPTABLE study [Johnston, Jones, and Hernandez 2016; Jones et al. 2016]), and other large-scale translational research is hastening the use of self-directed electronic consent procedures. Given little or no interaction between prospective participants and study staff, these approaches present challenges to assessing and ensuring consent comprehension. Joint guidance from the Office for Human Research Protections and the Food and Drug Administration on the use of electronic consent suggests that comprehension questions could be used during the process to help gauge subject understanding of important study elements (U.S. Department of Health & Human Services 2016).

Developments like these trigger important questions concerning the meaning and implications of comprehension in informed consent. Regulations and best practice guidelines set forth topics that should be covered in consent forms, but as many commentators have observed, complete knowledge of all the information deemed important for disclosure is not necessary to give valid consent (Appelbaum 2010; Wendler 2004; Wendler and Grady 2008). Furthermore, a common definition of the phenomenon of “understanding” is lacking (Agre et al. 2003; Sand, Kaasa, and Loge 2010). Without a shared definition of what it means to understand in the context of informed consent, it is difficult to operationalize researchers’ ethical obligations to promote comprehension and ascertain whether it is adequate for prospective participants to make informed, voluntary decisions (National Commission for the Protection of Human Subjects of Biomedical and Behavioral Research 1979).

These issues of what it means to understand were brought to the forefront when a national survey suggested that approximately one-third of the U.S. adult population may be unable to answer basic questions about information in a biobank consent form (Beskow et al. 2017). We consulted a multidisciplinary panel of prominent experts to further explore whether and how to assess consent comprehension, and what should happen when prospective participants are unable to demonstrate grasp of key information.

BACKGROUND

The findings reported here are the culmination of a long-term program of research to improve informed consent for biobanking. We began this program by exploring the general public’s understanding of and opinions about informed consent for biobanking (Beskow and Dean 2008), and then asked key stakeholder groups to identify the information most important for decisions about biobank participation (Beskow et al. 2010b). We used the results to develop a simplified consent form for biobanking (Beskow et al. 2010a) and then sought to test whether the simplified form led to better comprehension compared to a traditional form. To perform this test, we first devised a comprehension measure by conducting a formal Delphi process with a multidisciplinary expert panel (Beskow et al. 2015). The objective was to determine which information in a biobank consent form prospective participants must grasp for their consent to be considered valid. After three rounds, the panel achieved consensus (≥70% agreement) on a concise set of points, which we translated into a series of brief quiz items that we assessed and refined via approximately 60 cognitive interviews to ensure they were understood as intended.

We implemented this measure in a national online survey in which participants were randomized to receive either the simplified or traditional consent form (Beskow et al. 2017). After reading the form, participants completed the comprehension quiz. For each item answered incorrectly, they reviewed the corresponding consent form section and then answered another quiz item on that topic. One of several striking results was that, even after review and retesting, approximately one-third of the sample (weighted to the U.S. adult English-speaking population) did not achieve the quiz score of 100% needed to demonstrate adequate comprehension (i.e., grasp of the information the Delphi panel agreed was essential). This finding raised pressing questions about what should happen with regard to enrollment of these individuals in real-life biobank consent settings.

To address these questions, we returned to our Delphi panel for another round of online input. In this fourth round, we explained how we had developed a comprehension quiz that directly reflected the set of essential information on which they had reached consensus. For each quiz item, we displayed our national survey data on the proportion of people who were unable to answer correctly even upon second try and asked panelists what should happen: Should these individuals still be allowed to enroll, not be allowed to enroll, undergo additional review, or some other consequence?

Round 4 responses confirmed the importance of many of the elements of information the Delphi panel had originally deemed essential for prospective biobank participants to understand. For other elements, however, Round 4 results suggested that consensus had been lost: Although ≥70% of panelists previously agreed that these elements were essential for adequate understanding, >30% of Round 4 respondents said that individuals should still be allowed to enroll even if unable to answer corresponding quiz items correctly. See Supplementary Appendix A for complete details on Round 4 methods and results.

To better understand the issues raised by these results, we conducted semistructured interviews with approximately half of the panel to ask their reactions to the outcomes of the Delphi process and to explore in depth their thoughts on informed consent comprehension, including whether there should be a threshold for “adequate” understanding in biobanking consent and a consequence for not meeting it, and whether the level of risk involved in a study would affect any such threshold. See Supplementary Appendix B for further methodologic details; findings from these interviews are described in the following.

REACTIONS TO DELPHI ROUND 4 SURVEY RESULTS

When asked for their general reactions to Round 4 results in follow-up interviews, some panelists chose to explain their own opinions about the importance of specific biobank consent topics (Table 1). The justifications they offered for these opinions included reference to one or more underlying themes. Some evaluated a topic’s importance in terms of its bearing on the risks of participation. Some took misunderstanding of one topic as an indicator of a larger problem with comprehension. Some saw certain topics as less important because risks could be mitigated by protections other than informed consent, as well as the ongoing availability of information beyond initial consent. Finally, some cited the difference between the need to disclose information and the need to understand it, i.e., that certain information should be disclosed during the consent process but that understanding it should not be a condition for participation.

Table 1. Interviewee opinions regarding the importance of specific biobank consent topics: Selected quotes.

	Less important to understand	More important to understand
GINA	“I was pleased to see GINA sitting at the top of the list of things that probably shouldn’t be make-or-break for keeping people out. I have had qualms about being forced to put that information in the consent form ever since it was issued … I don’t think we’d be putting it in there except the regulations say we have to. So I’m entirely comfortable thinking somebody could get into a project like this without being able to regurgitate what GINA’s all about.” (14, Ethics)
Confidentiality protections	“When you’re talking about confidentiality, that’s where you start to get into things like the privacy statements that you get from Google and Facebook and all of those kinds of things. And who actually reads those? … In today’s world that is not something that would be a gatekeeper.” (04, Participant)	“Risk of breach of confidentiality and privacy is the one consistent risk across most biobank studies. Many people have argued, in fact, that there are near to no, or no other risks involved in biobanking. So I’m surprised to see that so many people said that it’d be okay to enroll people into the biobank if they didn’t understand the confidentiality protections.” (18, Ethics)
Study contact information	“My tendency as an educator is: people need to know the entire consent form. But going through this process, I’m like, ‘Well, no, they don’t necessarily have to have a complete understanding of that. They’re given this. It’s in print. They can always refer to it,’ like the contact information and all of that. So it gave me a different perspective on how to look at the consent process.” (09, Participant)	“I felt that people really needed to have an understanding that there’s somebody who owns this process or who owns the research project … I was really concerned at this point, where I’m like, ‘If you don’t even understand that there’s somebody that you can contact to learn something about what you just got involved in, that’s a problem.’” (10, Biobank)
Privacy risks	“After the initial blood draw, the biggest risks you’re dealing with are privacy risks, but there are very robust measures in place in every [biobank] I’m familiar with to protect privacy … So I’m not very worried about it.” (17, Ethics)	“If somebody’s … consenting because they have some understanding that there’s 100 percent absolute privacy, then I think that’s something that should be a screen-out.” (05, Participant)
Benefits	“There were a couple that I thought, ‘What’s most important to me?’ It’s not really the benefits, because they’re not really getting any benefits. It’s more about the risks. So if they don’t know what the benefits are, is it really all that important?” (06, Biobank)	“People need to understand what they would get out of it or what they would not get out of it … I think that’s the most important. It’s where people have the most feeling of betrayal: if they’re expecting a benefit that doesn’t come.” (05, Participant) “Once [people] understand that this has sort of a social benefit, a medical/social benefit for society and not necessarily for them, then the rest of the understanding becomes a little less important in my mind.” (20, Biobank)
Withdrawal	“‘I have the right to leave the project’—that’s pretty clear. Not being able to withdraw or get back the samples from studies is something that’s a little more of a difficult concept. So, having that statement that ‘I have the right to leave the project, period’—I felt like even if they didn’t understand the second part, that was okay. They will have heard that they have the right to withdraw. If they choose to do that, they’ll have a discussion that explains the parameters with them.” (04, Participant)	“If someone doesn’t understand that they have the right to withdraw from the biobank, really, it would be a flagrant transgression of the fundamental ethical purpose of conducting an informed consent process to go ahead and enroll that person. Regardless of whether we agree or disagree about the level of risk involved in the biobank.” (18, Ethics)
Return of results	“There will be policies and procedures that will be set in place with respect to return of results that provide additional support to the individual … The institution that’s performing the study or providing the results—they are trying to look out for the best interests of this individual. I don’t think that they’re doing so in a way that puts an individual in harm’s way. You’re doing so in a way that you believe it’s the morally right thing to do … So that’s why I look at that as: there are other factors at play than ‘Does this person understand return of results?’” (10, Biobank)	“If you’re going to return results then you absolutely have to have teach-back for that, because that’s a really big deal … The business of returning results is different because it turns your life on its head—that’s a game changer.” (03, Ethics)
Blood draw	“When there is a significant danger at hand, we have a greater responsibility to explain things to people. So at the initial blood draw, when there is a small amount of risk, I would try explaining it to people again: it’s not a blood draw for their health, but a blood draw for research, so they understand it. Because there is some degree of risk. [But] if they still don’t get it, it’s not a big deal.” (17, Ethics)	“If I don’t get that you’re going to stick me with a needle and take my blood, then you probably shouldn’t stick me with a needle and take my blood.” (12, Ethics)
Purpose		“What is actually more important to people considering enrolling in a research study or a biobank than anything else: ‘Why are you doing this?’ … They want to know that there’s actually thought behind this, and that they’re going to contribute to something meaningful.” (04, Participant) “The one where I got the most concerned, where I’m like, ‘It’s not even worth having a discussion about this one,’ was [biobank purpose]. If you had no idea that this thing is going to be used in the future or what it’s going to be used for, then you don’t know what you’re getting involved in. You just don’t understand this. And I just don’t know if there’s anything else that I can do to explain this to you.” (10, Biobank)

Other interviewees, when asked about Round 4 results, perceived resistance among their fellow panelists to making the ultimate decision that a willing person should not be allowed to enroll:

I’m seeing two dominant responses, one of them being, “They can still enroll.” I’m guessing people chose that option when they themselves personally didn’t think the issue was all that important for understanding, which is interesting. Then the other big response was: “Try again…” So what I’m finding interesting is that people really do not want to choose that you’re not allowed to enroll even though you haven’t understood these critical items. People just don’t want to choose that. (02, Ethics)

Reflecting on the apparent loss of consensus concerning the essential nature of some of the information, many interviewees said that it was influential to see our national survey data on the proportion of prospective participants who might answer quiz items incorrectly: “When you see what a group of people actually got right and wrong, I think that had a big impact on the group” (09, Participant). In particular, many characterized the data as providing a reality check (Table 2) and, as one stated, “We all kinda backed down” (05, Participant). A few interviewees said the data were irrelevant to determining what information must be understood, but were helpful in identifying topics in need of additional education or attention during the consent process.

Table 2. The influence of empirical data on comprehension expectations.

Data as a reality check

Data modify experts’ judgments about what people can understand: “The most … eye-opening part for me is: I honestly believed when I first went through it and said, ‘These things are essential for consent,’ I really thought that I was putting myself in the role of every person. And thought, ‘Well, certainly people would understand this.’ What surprised me is how I had taken for granted how many people would be able to understand these elements who in fact didn’t.” (05, Participant) “It’s a gut check … It helps me to modulate the focus of my response or the severity of the lines that I’m drawing, based on the reality of the field.” (13, Participant) “In the absence of those data, people are making the obviously wrong assumption that everybody’s going to get all of these right … We’ve already whittled the list down from all of the garbage that we’re forced to put in consent forms these days or that we put in of the goodness of our hearts … So people are coming to the table thinking, ‘Okay, that means that everybody will surely get these right because we think they’re important and it’s a shorter list.’ But now we’ve got evidence that, well, in fact: no, they’re not going to.” (14, Ethics) “It was quite interesting. And kind of you get a little test of reality in terms of what it means to expect people to understand what it means to participate in biobank research. I started out with a much higher bar in terms of what people needed to know in order to give valid consent.” (19, Ethics)

Data modify experts’ judgments about what should be considered important: “Seeing how many people would actually get it wrong, it made you reconsider the importance of the information.” (06, Biobank) “If you see a question where a large percentage of people are getting it wrong, then you think, ‘Huh. Is it really that important? Would I bounce them out of the study for not knowing this?’” (11, Biobank)

Data as an indication of education need

Indicator of need for more participant education: “If the majority got it wrong, that tells me … that more information was needed and more discussion was needed in order for them to understand it … So I need to pay more attention to those areas, look at the subject matter … Maybe it’s something we need to tweak, or the way it was written or said was not getting across to the majority of our participants.” (09, Participant)

Indicator of need for more public education: “It’s a time for when you can have a critical view about what’s out of whack in the general public’s perception of what’s going on. So I feel like it’s a great opportunity for education. Because if the goal is to get informed responses … then you want to know which of those questions are the ones that seem trickiest, and maybe put more resources there, or more public policy efforts around those.” (07, Participant) “There are a lot of questions, for instance, where it seemed people understood the concept a lot faster, right? At which point it may be that there are just things that’re either commonsense or just the general status quo in society where people understand this because that’s the generally accepted model. So they get it. So there’s not a reason to spend a lot of time on that in the consenting process because this is just something that people now have at their disposal, or they come to expect it.” (10, Biobank)

COMPREHENSION AND INFORMED CONSENT FOR BIOBANKING

We further explored interviewees’ thoughts and opinions concerning whether there should be a defined threshold for what constitutes adequate understanding for biobanking consent and, if so, whether there should be a consequence for not meeting it. Responses varied to an exceptional degree, with interviewees addressing different aspects of these nuanced topics, as well as expressing diverse opinions. Some answered the questions in a straightforward manner, conveying a clear and consistent set of beliefs. But most of these experts found the questions and ensuing discussion to be challenging—thinking aloud, voicing inner turmoil, and occasionally making statements that were internally inconsistent. As one reflected at the conclusion of the interview:

I’m struggling here because I think valid consent is a concept people believe in, but it’s really difficult to define what valid consent means in terms of the aspects of participation that people need to understand … And what you do if they don’t understand. I don’t know—I guess in clinical trials, you sit with someone and make sure they understand what it means to participate. But even then I’m not sure that it catches everyone who doesn’t really quite get what they’re doing. So the whole issue is challenging … I started out thinking there’s consent and people need to understand, and you need to explain the concepts in a way that people can understand. Now I’m not sure if it’s always possible—and whether you can say, “Okay, you don’t understand. You can’t participate.” (19, Ethics)

Collectively, interviewees described the role and importance of comprehension throughout the initial informed consent process—including in the development of consent materials, consideration of thresholds for adequate understanding, assessment of actual comprehension, and decisions regarding consequences for lack of understanding.

Development of Consent Materials

Our Delphi process did not focus on what information should be disclosed in consent forms and processes, but rather which information participants must grasp to demonstrate adequate understanding of a given model consent form. Even so, for some interviewees, deep commitment to the development of tailored, high-quality consent materials was the prime location of researchers’ obligations with regard to comprehension—rather than setting and testing knowledge thresholds:

I think researchers have a responsibility to do their best to provide all the relevant information that a reasonable person would wish, and do their best to provide it in language that is likely to be understandable by their participants. And if they have done that, then I’m not sure they need to test participants … I’m not trying to say that I’ve just said something simple. Because how you know that you have provided all the information that’s relevant, and how you know that you’ve provided it in reasonable language is a question. (02, Ethics)

Some emphasized that researchers must be keenly mindful of the specifics of their studies and needs of their study populations:

We start with this template … of an informed consent document that’s used in a given institution. Then we just adapt that to any given study too easily, without thinking about what’s important for this study? What’s important for the participants in this particular study? I don’t think we sit back and reflect enough about that. (11, Biobank)

Some underscored researchers’ ethical responsibilities in determining which information is even available for prospective participants to consider:

The researchers … start from a position of power in that they already have a perspective and a comprehensive understanding of what is at stake. Subjects start from a position of disadvantage because they don’t have the perspective of that understanding, and we are asking them to participate in something that is potentially harmful or risky to them. So it is our responsibility to ensure their protection. We have to acknowledge that we are paternally making decisions … about what is important for them to know. (13, Participant)

Several interviewees explicitly called for researchers to seek input from study populations as an important way to craft consent materials that focus on the information most important to prospective participants:

There’s a lot of things that happen in IRB [institutional review board] land … that have very much to do with just people’s gut sense and intuition, and very little to do with being guided by data. I’ve always thought that was problematic. Because it’s very subjective; it’s not necessarily informed and it’s biased … We should listen to the intended audience. (14, Ethics)

Thresholds for Adequate Understanding

More than three-fourths of interviewees supported the concept of a defined set of information that is essential for prospective biobank participants to understand:

I buy into this idea of a minimum adequate understanding threshold. I think that is the sensible way to approach informed consent, be it in a context of biobanks or clinical trials or any type of research. It’s certainly the most realistic way of approaching it, based on what we know empirically about informed consent. (18, Ethics)

However, where to set the threshold was acknowledged to be the crucial question:

Obviously it’s a huge challenge to get valid consent versus the justice and access issues—to get the balance right. It’s vital to take this on directly because the stakes are so high. If you get too literal about what constitutes understanding, you’re going to have too few people in the studies. (07, Participant)

Most interviewees argued that, for biobanking, the threshold should be low—oftentimes summarizing “in a nutshell” the information they found most essential. Many said a low threshold was appropriate because the risk involved in the research is low:

I think the more people understand, the better. But I also think that, in research with a databank where there’s not a huge health risk, then I’d have more tolerance for what people may not understand. (05, Participant)

I tend to view biobanking frankly as a pretty low-risk enterprise. So I would set the threshold fairly low. But that’s not to say there shouldn’t be one at all. (14, Ethics)

Several believed that comprehension requirements should be low because of other protections in the system beyond informed consent:

I don’t think people have to understand much. It’s important to explain it to people—but if they don’t get it, it’s not a big deal. When you’re talking about biobank research that has very low risk and that’s gone through IRB review and a biobank oversight committee review, I don’t think people need to understand much. (17, Ethics)

To further inform where to set the threshold, several interviewees highlighted the need to incorporate perspectives from the target study population: “It’s important to hear from research participants and find out what they see as important—which may be very different from what I see as important” (06, Biobank). In seeking out truly lay perspectives (“Being on a community advisory board for about seven years now, we have many discussions about the fact that … we no longer perhaps fill that role as being the general public or the Joe Blow”; 15, Participant), some emphasized the importance of gathering informed input:

The positions [people] come to after a process of reflection and conversation where they can challenge each other on issues gives you much better conclusions, much more reliable conclusions, than asking for people’s opinions: “Off the top of your head, what do you think?” (17, Ethics)

Some focused specifically on the question of which information would make a material difference to biobank participation decisions:

I would not say: what do they want to know? You can ask that, but that’s more about disclosure than understanding. They probably want to know everything, and I think we probably should tell them, in some form or fashion—at least give them access to all the information that we have. So not: what would they want know? But: what do they think they need to know to make a decision? (12, Ethics)

For the minority of interviewees who did not support the concept of setting a threshold for adequate understanding, a prominent theme was risk/benefit trade-off:

One of the things that this area really lacks is an understanding that this emphasis on the informed consent process—even though it’s not going to lead to meaningful informed consent—it’s going to come at enormous cost… Realistically, it’s hard for people to understand everything, even when they work really hard. And that understanding comes at a not-trivial cost. (03, Ethics)

There is some utility in trying to define [a threshold] if nothing else but it helps people see how difficult it is to have one. The trade-offs between the importance of the research that gets done, and the scientific benefit and to society, and the individual risk is something that we all think about in this work. (19, Ethics)

Another argument against setting a threshold was based on direct appeals to respect and autonomy—allowing individuals to decide for themselves which information is important to their participation decision:

People have pretty terrible recall. Whether that’s understanding or recall or both, it’s hard to tell. But they really don’t know what they’re signing up for when they sign these consent forms. What we found is that, to a large extent, they really don’t care that they don’t know. As long as they feel like they’re being adequately respected by being asked, the fact that they missed what I would consider really important stuff, that didn’t seem to bother them too much. Because they had been asked and they had the opportunity to know and look at it more carefully if they wanted to. (12, Ethics)

Respecting autonomy requires that you inform people and you let them do things their own way. If they like to think about McDonald’s for lunch while you’re explaining it to them and they don’t really get it, that’s not your problem. From the perspective of your duty to respect autonomy, you’re supposed to leave them alone and let them make decisions whatever way they do it. (17, Ethics)

Assessment of Actual Comprehension

In general, many interviewees found the use of a comprehension quiz acceptable (“A person really needs to have a very basic understanding of what the consent says … it’s worth a participant having to take this quiz in order to validate their consent”; 15, Participant), particularly given large-scale, low risk research (“For biobanking, I thought the quiz format was appropriate”; 13, Participant). However, they often said a quiz was acceptable while expressing a preference for other approaches to assessing comprehension when possible—teach-back, in particular:

The best way of measuring understanding is for people to be able to restate the content in their own words. You can do simpler things such as a true/false or a multiple choice; true/false really is probably easier. But really the best assessment of understanding is that people can restate it in their own words. (11, Biobank)

Some were intrigued by the possibility of implementing a comprehension quiz not as an objective test, but rather to help prospective participants gauge their own levels of understanding and comfort with the information:

What I’d be really interested in is whether it made a difference to people, whether it increased their satisfaction with the study process. I’m not sure it would, but it might—I need empiric data on that. (02, Ethics)

Several interviewees, however, were disinclined toward testing comprehension and suggested looking instead to subjective understanding: “If the patient can say ‘Yes, I understood it,’ then I think that’s sufficient” (16, Biobank). In terms of rationale, one observed, “We don’t do [comprehension testing] anywhere else in medicine, so I don’t know why we’d do it here” (03, Ethics). Another described the downside of repeated cycles of testing and review:

There’s a tension there [regarding] how far we can go testing and re-testing participants. It can get pretty onerous in terms of both the time and resources that consenters need to invest, as well as pretty uncomfortable for participants to really feel like they’re being put on the spot. So the tension lies between those burdens and the possibility—and, of course we only know that it’s a possibility, and not probability—that they will come out after being re-tested multiple times and understand something better. (18, Ethics)

A particular concern was that individual testing treads on autonomy:

People may think that they have all the information they need and are comfortable making a voluntary decision … They heard a bunch of stuff we said and they’re satisfied that they understand the situation enough to say yes or no. That’s the point at which I’m loath to interfere and test them. Because if they truly have received all the information they needed to make the decision and they have decided, then it isn’t my place to step in and say, “Wait a minute. I’m not sure you’ve really really really understood.” (02, Ethics)

For this interviewee, relying on subjective understanding was grounded in two vital conditions: ensuring a situation where prospective participants can ask questions and where researchers are confident they have provided key information in an understandable way:

Caveat number one is that people who are deciding whether or not they have enough information need to be very clear that they’re within a context where they can ask any questions. In other words, part of the situation is: they need to not just be told they can ask questions, but there’s someone sitting there who they feel comfortable asking the questions, who’s made it their business to be open to questions … The other caveat is that the researcher, in discharging the duty, has to feel confident that they have provided all the material information and they have provided it under circumstances where it’s a reasonable expectation that the people receiving the information understand it. (02, Ethics)

Consequence for Lack of Understanding

Among interviewees who endorsed the concept of a threshold for adequate understanding, approximately two-thirds believed there should be a consequence for not meeting it. This estimation is complicated, however, by interviewees’ varied expectations concerning the likely success of additional review. For example, some who said there should be a consequence seemed to have faith that it would seldom be imposed because individuals could be brought to adequate understanding through additional review: “The most important thing is patience, patience in explaining … I think you can explain everything” (16, Biobank).

Others mentioned the issue of capacity: that individuals who are competent to give consent can be brought to adequate understanding through additional review or, conversely, those who cannot be brought to adequate understanding must, by definition, lack capacity:

For somebody who could possibly understand, I think you try to explain it to them again. Ultimately, if they really are not getting it—it’s actually pretty hard for me to imagine that if you explain it and you have a good explainer … that they’re not going to get it? If they forget it five minutes later, fine. But if they have the capacity to consent, you just keep trying until they get it … I think if you make a sincere effort to somebody who’s not cognitively impaired, you can explain these things. (17, Ethics)

As a prerequisite for expecting that additional review would be successful, one interviewee spoke at length about the obligation to ensure a quality consent process:

It depends on the quality of that remedial attempt. Is it simply restating the explanation as it appears in the consent document? Or is the consenter able to re-word the concept … in a way that anybody with basic cognitive competency would likely get? … I would argue that we really have to have excellent consenters. We have to have people who can really communicate, who understand what a biobank is and understand all the different factors and dimensions of it, so they can unpack these elements for people … I know there are a lot of elements that make it difficult to administer a quality informed consent process, but it ought to be possible. (18, Ethics)

Among interviewees who focused less on additional review as the solution per se, some firmly stated that individuals who fail to demonstrate adequate understanding should not be allowed to enroll. A few referenced the basic meaning of informed consent:

Let’s just say for the sake of argument that the bottom four questions [in Figure 1 of the online supplementary material Appendix A] are the ones we really think they shouldn’t be sneaking by with. If on additional review they still can’t get it, well then yeah, I guess they don’t get in. Because consent has really failed somehow, or this person’s ability to comprehend is impaired. If we’re going to get consent, it ought to mean something. (14, Ethics)

I don’t think that it’s true consent if they don’t understand deal-breakers … Consent is: “I understand what I have been told and read,” and if they don’t get that … they should not be allowed to enroll. (09, Participant)

For one interviewee, part of the concern about allowing those who have not demonstrated adequate understanding to enroll was the potential to spread misunderstanding in their communities: “I just feel very strongly, particularly strongly, that misrepresenting [the study] because you misunderstood and didn’t have clarification—that you become a mouthpiece for a really potentially messed up view of what’s going on” (07, Participant).

About half of interviewees, however—including approximately one-third of those who said there should be a threshold—said that individuals who do not meet the threshold for adequate understanding should still be allowed to enroll: “If one option is don’t allow them to enroll versus re-educate them, I’d always say ‘re-educate them’—but I would still allow them to enroll even if they continued to get it wrong” (12, Ethics). Some were surprised to find themselves reaching this conclusion:

I was surprised that I care so much about what’s in a consent form and whether it covers everything and whether the language is understandable or too simple or too complex, and yet I’m not willing to say that if people … don’t understand it then they shouldn’t be allowed to participate … That’s what I find so troubling—that I could think that it’s really important for people to know and ask them as many times as possible, but then in the end, let them participate. (19, Ethics)

Several interviewees offered ethical reasons for still allowing enrollment, including arguments based on autonomy, voluntariness, and the countervailing value of research (Table 3). For others, their opinion that people should still be allowed to enroll was premised on the research involving little risk, especially in light of other available protections:

Table 3. Ethical arguments for allowing enrollment.

General: “So the [Delphi panelists who said], ‘That’s it’ after two attempts, they don’t get to enroll in the biobank—you know, I’d really like to talk to some of those folks. Because there’s a tension there too. Now you’re saying you’re going to deny people access if they don’t understand certain elements of the biobank. And that’s problematic as well, right? From an ethics standpoint … And how does that work in practice? How do you actually say that to someone after having just told them that there’s this thing called a biobank, ‘We’d like you to consider it,’ et cetera, and then you say, ‘Well, no, sorry. You just don’t get it. So we can’t let you go in.’ It just doesn’t seem in the least bit patient-centric or compassionate.” (18, Ethics)

Autonomy: “It almost feels like if we, mea culpa, even state that ‘It’s beyond our ability to bring you to the level of understanding we’d like you to have’ and they still wish to enroll, then there you are: they wish to enroll … Then it’s the autonomy of those individuals that should take greater weight.” (01, Biobank) “There’s a tension between autonomy and paternalism. If the autonomous responder is saying, ‘You told me enough; don’t worry about it; don’t worry about what I understand or don’t understand; I believe you have told me enough; we’re done; I’m saying yes or I’m saying no,’ I can respect that the ultimate decision needs to rest with the participant.” (02, Ethics)

Voluntariness: “In the ideal world, you’ve given them all the information you think they would want. But it’s up to them to decide what information they need in order to say yea or nay. For us to say, ‘We’re going to give you this information and it’s voluntary—you can say yes or no,’ and we’re going to say whether you’re even able to say yes or no, based on whether you understand the information—you’re putting an extra level of requirement that maybe isn’t reasonable.” (02, Ethics)

Ethical value of research: “I’m not so happy about saying that people have to understand the ins and outs of biobanking before they can be a participant. Because I think the result is that, in the name of a thick understanding of autonomy, we’re going make it so that research can’t go forward. There’s value in learning things that improve human health.” (03, Ethics)

There has to be a level of trust with the research enterprise and the protections that we have in place within the enterprise … When you take that in conjunction with comprehension, comprehension doesn’t have to be perfect and our protections don’t have to be perfect. (06, Biobank)

Just the whole notion of consent—I mean, I still believe in it. I just think there needs to be lots of other pieces, other parts of the research infrastructure that also protect participants … So if there are other kinds of advisory boards that are looking at the infrastructure and how the studies are being chosen to use the data … there’s a scientific advisory board, a community advisory board. I don’t think there’s any one kind of protection that I would say, “Okay, this would be fine as long as this were happening.” It’s a whole constellation of factors that’re put in place for a well-designed biobank. (19, Ethics)

Some, in fact, urged greater emphasis on these other protections and lowered expectations for what informed consent can achieve:

We have given a lot of weight to informed consent over the years. IRBs spent a lot of time tweaking documents, et cetera. And the discouraging news is that, if the leg on which research must stand is whether participants completely understand what they’re involved in, the answer is they don’t … Now, I don’t think it means that we should give up on informed consent. But … the fact that you all spent so much time really focusing on how to do this in language that’s comprehensible, meets the reading requirements, etcetera—much more than any IRB ever would—shows that there’s just a limit on how far we’re going to get. (03, Ethics)

Finally, several interviewees mentioned burden on both participants and institutions, and the trade-offs in creating arguably excessive obstacles to enrollment:

Ideally, we want people to make a perfect score and we want them to understand everything. But what will we accept? Because we know we don’t have eight hours—at least, up to this point, we haven’t said that we’re going to require eight hours of consent studying in order to let people join a study. (06, Biobank)

You could tell somebody, “Okay, the amount of resources that you can now spend to get people in the study is such that you only get half the number of people in your study if you spend all your time doing additional review.” … This question of utility needs to be brought to the table. (10, Biobank)

INFORMED CONSENT COMPREHENSION FOR HIGHER RISK RESEARCH

After discussing these questions in the context of biobanking consent, we asked interviewees to consider whether their views might differ given research involving higher risk, such as a phase III trial of a new oncology agent. In general, all interviewees believed that comprehension was even more important for such research because of the interventional nature and risk of physical harm. Even so, nearly all expressed the same opinions as they had for biobanking regarding whether there should be a threshold of knowledge defining “adequate” understanding and whether there should be a consequence for not meeting it. For example, those who had supported the concept of a threshold and a consequence for biobanking typically described even higher expectations for a phase III trial:

I would think, logically, we would be more concerned about somebody getting into a study with high personal risk to them. I would think we’d want more right answers and have more concern if they couldn’t grasp some of the essential elements than we would in a lower-risk study. (14, Ethics)

Interviewees who had supported the concept of a threshold but not a consequence for biobanking were similarly concerned that prospective clinical trial participants needed to understand more, but remained reluctant to impose a consequence:

What’s different is that people in a phase three oncology trial need to know a whole lot more before deciding to enroll in that study than they need to know before giving their blood for biobanking … What I’m struggling with is: knowing what we know—which is that people really don’t understand a lot when they’re signing up for research, they don’t really understand what a randomized controlled trial is—is it absolutely critical for them to understand to be able to say “yes”? Or should we say, “Well, you didn’t get that, so you can’t participate in the study”? I would be reluctant to do that. (12, Ethics)

Interviewees who supported neither threshold nor consequence for biobanking recognized the higher stakes and greater responsibility for effective disclosure in a phase III trial, yet maintained their basic stance:

In that situation, you really have to tell them ”This is research. There’s some chance that it may help you … but there’s also a chance that this won’t help you and actually might hurt you.” That’s the fundamental issue in a clinical trial for phase three oncology and I do think people have to be told that. But the fact that [the person might not] get it doesn’t mean that he shouldn’t be in the trial. But you have to make a real good faith effort to get that point across. The stakes are so much higher than what we’re talking about in biobanking. (03, Ethics)

Among the few interviewees who said their opinions about a phase III trial would be substantively different than for biobanking, one shifted to endorsing a consequence for failure to grasp information deemed essential:

I’m going to contrast. In the biobank setting, the risks are a lot to do with confidentiality, privacy, that kind of thing. The risks in a phase three study are much more significant. I would argue that there probably is a pretty strong threshold of understanding required to enroll somebody, with the understanding that there are toxicities that may happen that could make you very sick or potentially even kill you. I think there’s an obligation that the patient needs to understand those risks. I wouldn’t walk away saying, “Well, they didn’t understand it, but enroll them anyhow.” (20, Biobank)

Examples of other substantive changes included being less swayed by empirical data showing how many people would answer comprehension questions about a phase III trial incorrectly (“If we’re looking at a medical risk, a health risk—I don’t know how much value [empirical data] would have, because I think we need to come up with what’s absolutely necessary for consent, even if that sets the bar pretty high”; 05, Participant), and requiring in-person consent (“If you’re going to take a drug that has some amount of risk to your health … you should have the opportunity that consent is done in person; for biobank research, it’s often not done in person”; 19, Ethics).

DISCUSSION

Major developments in clinical and translational research have brought renewed focus on the foundational concept of comprehension in informed consent. Our study was initially motivated by the goal of testing the effectiveness of a simplified biobanking consent form and thus the need to develop a measure of “adequate” comprehension. Over the course of this research, however, we realized that implementing such a measure brings into stark relief the question of what should happen when prospective participants fail to grasp the requisite information—a phenomenon that has been amply documented in many research contexts (Beardsley, Jefford, and Mileshkin 2007; Bergenmar et al. 2008; Bergenmar, Johansson, and Wilking 2011; Jefford et al. 2011; Joffe et al. 2001; Koh et al. 2012; Lipton et al. 2011; Montalvo and Larson 2014), including biobanking (McCarty et al. 2007; Ormond et al. 2009; Rahm et al. 2013). In further exploring this question, we discovered fundamental discord both within and between nationally recognized experts regarding the role and implications of consent comprehension.

Specifically, despite achieving a high degree of consensus on what constitutes minimally adequate comprehension for biobanking, opinions among our diverse panel of Delphi experts shifted dramatically when confronted with empirical data regarding the proportion of individuals who may be unable to meet this threshold of understanding (Supplementary Appendix A). Although many continued to support the idea of a threshold, they indicated that the set of essential information was even smaller than the panel had originally agreed upon. In addition, many in Round 4 opted for additional review as the preferred action for many of the quiz items—perhaps in some cases taking this path as a temporary reprieve from the quandary of having to choose between prohibiting or allowing enrollment.

In follow-up interviews, some panelists persisted in believing that additional review could bring competent individuals to adequate understanding. Ultimately, however, although some decided that individuals who do not meet a defined threshold of knowledge should not be allowed to enroll, about half of interviewees—sometimes with great self-reported cognitive dissonance—said that such individuals should still be allowed to enroll. For the most part, these opinions remained the same when we asked about higher risk research.

The questions highlighted by these results are worthy of urgent debate, both as an ethical matter and as a practical policy matter that researchers will increasingly confront—particularly those using electronic consent and other self-directed approaches that involve less human interaction and thus require ways to gauge comprehension other than study staff judgment. At the heart of the debate is whether adequate comprehension should be viewed as an ethical requirement or an ethical aspiration (Bromwich 2015; Sreenivasan 2003), and the relationship between informed consent and other requirements for ethical research (Emanuel, Wendler, and Grady 2000; Joffe and Miller 2008).

Our research brings rich, multidimensional data to this discussion. Across the various domains about which interviewees commented—from the development of consent materials through enrollment decisions—the themes that emerged could be arrayed along a continuum describing the locus of responsibility or activity. One end of the continuum is more researcher-centered, with the researcher not only determining the information disclosed, but also objectively assessing comprehension and deciding whether prospective participants have adequate understanding to make an enrollment decision. The other end of the continuum is more participant-centered, with individuals—given the information disclosed—deciding which is important to their own interests, weighing their level of comfort with that information, and making enrollment decisions based on subjective understanding. Some themes reflected a middle ground whereby the information highlighted as part of disclosure and the details deemed important for adequate understanding are directly informed by input from the study population. Each of these points on the continuum leads to a very different outcome regarding the meaning of comprehension and how enrollment decisions are eventually made. Although a few of our interviewees made comments that placed them consistently at one point or another, most shifted back and forth along the continuum: for example, evincing a researcher-centered approach to determining what should be disclosed and understood, and a participant-centered approach to assessing actual understanding and readiness to make an enrollment decision.

There are several notable design features and limitations of our study. As descriptive qualitative research, the approximate proportions of interviewees who supported (or not) the concepts of threshold and consequence should be considered preliminary and relative. Further, given the already-challenging nature of these interviews and the density of the topics addressed, we did not follow up on all relevant repercussions of the opinions expressed. For example, among interviewees who favored additional review, we did not probe how they thought this would work in the context of online consent processes. Our goal at this juncture was to capitalize on our existing panel of experts—who had already dedicated substantial time to systematically thinking about consent comprehension as part of our project—by gathering their culminating, in-depth input in support of further debate and investigation.

Although we assigned these experts a particular perspective for the purpose of recruiting a multidisciplinary panel (and for the purpose of labeling direct quotes here), very few represented only one stakeholder group. For this reason, as well as the qualitative nature of our study, we did not attempt to assess similarities and differences in opinion between groups. Further investigation of the extent to which perspectives differ between groups, as well as the origins and prevalence of relevant differences, is a vital area for future research.

Our findings have at least two important implications for the research community. First, large-scale translational research endeavors are already struggling with decisions about whether or not to implement comprehension questions and how responses to such questions should best be used to promote the ethical conduct of research. Our findings suggest that study teams that wish to set a threshold for understanding ought to first collect data on how many potential participants will meet various thresholds, because judgments about where that threshold is placed (as well as the development of materials to address comprehension gaps) may be strongly influenced by such data.

Second, our findings contribute to ongoing debates concerning “understanding” in the context of informed consent. Much of bioethical discourse regarding comprehension is based implicitly or explicitly on it being an ethical requirement, with the only remaining questions concerning what should be understood. But the reflections and realizations that occurred for panelists over the multiple rounds of our Delphi process suggest that the combined notion of thresholds with consequences is difficult to put into practice. If thoughtful colleagues struggle with how to act upon the notion of understanding as an ethical requirement, perhaps viewing understanding as an ethical aspiration is more consistent with the range of values and considerations involved in making real-life decisions about research participation.

ACKNOWLEDGEMENTS

The authors thank Catherine Hammack, Zachary Lampron, Martina Bresciani, Emily Gao, and Kathleen Brelsford for their assistance.

FUNDING

This work was supported by a grant from the National Human Genome Research Institute (R01-HG-006621). The content is solely the responsibility of the authors and does not necessarily represent the official views of NHGRI or NIH.

AUTHORS’ CONTRIBUTIONS

Study conception and design: LMB, KPW; Acquisition of data: LMB, KPW; Analysis and interpretation of data: LMB, KPW; Drafting of manuscript: LMB; Critical revision: LMB, KPW.▪