Taking into Account Patient Preferences: A Consensus Study on the Assessment of Psychological Dimensions Within Patient Preference Studies

Abstract

Patient preferences are gaining recognition among key stakeholders involved in benefit-risk decision-making along the medical product lifecycle. However, one of the main challenges of integrating patient preferences in benefit-risk decision-making is understanding differences in patient preference, which may be attributable to clinical characteristics (eg age, medical history) or psychosocial factors. Measuring the latter may provide valuable information to decision-makers but there is limited guidance regarding which psychological dimensions may influence patient preferences and which psychological instruments should be considered for inclusion in patient preference studies. This paper aims to provide such guidance by advancing evidence and consensus-based recommendations and considerations. Findings of a recent systematic review on psychological constructs having an impact on patients’ preferences and health-related decisions were expanded with input from an expert group (n = 11). These data were then used as the basis for final recommendations developed through two rounds of formal evaluation via an online Delphi consensus process involving international experts in the field of psychology, medical decision-making, and risk communication (n = 27). Three classes of recommendations emerged. Eleven psychological constructs reached consensus to be recommended for inclusion with the strongest consensus existing for health literacy, numeracy, illness perception and treatment-related beliefs. We also proposed a set of descriptive and checklist criteria to appraise available psychological measures to assist researchers and other stakeholders in including psychological assessment when planning patient preference studies. These recommendations can guide researchers and other stakeholders when designing and interpreting patient preference studies with a potential high impact in clinical practice and medical product benefit-risk decision-making processes.

Keywords:

• patient preferences
• patients reported outcomes
• psychological assessment
• decision-making

Key Points

• In the medical products benefit-risk decision-making process and in the research field of preference studies, patient preferences are gaining recognition among key stakeholders. There is a strong need for evidence-based guidance on what psychological dimensions may influence patient preferences and which should be considered for inclusion in patient preferences studies.

• This study fills an important gap in patient preference methodology. It helps stakeholders selecting psychological constructs and measurements to use for evaluating how patients form, maintain, change and negotiate preferences in medical and healthcare context. In addition, psychological assessments can assist in explaining preference heterogeneity in cases where research participants’ preferences diverge.

• Evidence- and consensus-based considerations on what psychological constructs to include in patient preference studies are advanced. A criteria checklist to select appropriate psychological measurements to include in patient preference studies are also advanced. Recommendations and criteria proposed will inform stakeholders involved in the medical products benefit-risk decision-making of the patient’s formative abilities and describe preference heterogeneity.

Introduction

Patient preferences are gaining recognition among key stakeholders involved in benefit-risk decision-making along the medical product lifecycle.^1,2 A patient preference (PP) may be defined as

A statement of the relative desirability or acceptability to patients of specified alternatives or choices among outcomes or other attributes that differ among alternative health interventions.¹

PPs are specific type of patient perspective. Patient perspective can be defined as any information related to patients’ experiences with a medical condition and its management. Overall, this information allows a better comprehension of the disease and its impact, the effective identification of outcomes that are more relevant for patients, and a more fine-grained understanding of the benefit-risk trade-off for treatment.³ PPs are especially relevant for the evaluation of the benefit-risk profile of medical products in case of preference-sensitive decisions, namely situations where medical equipoise among alternative treatments exists.⁴ In the clinical setting, the attention given to PP is relevant in different theoretical and practical approaches, such as the shared decision-making model that underlies the relevance of recognising PP to empower patients and personalise treatment and care.^5–7 The present paper is a direct result of the tasks carried out within the framework of The Patient Preferences in Benefit-Risk Assessments during the Drug Life Cycle (PREFER) project, a European undertaking supported by the Innovative Medicines Initiative (IMI). The main objective of PREFER is to strengthen patient-centric decision-making throughout the medical product life-cycle by developing evidence-based recommendations to guide industry, regulatory authorities, health technology assessment (HTA)/payer bodies, reimbursement agencies, academia, and health care professionals on how and when patient-preference studies should be performed and the results used to support and inform decision-making throughout the medical products lifecycle and enable the implementation of personalised medicine.⁸ However, as reported by a recent systematic review by Huls and colleagues,⁹ one of the main challenges of integrating PP in benefit-risk decision-making is taking into account patient preference heterogeneity and formation. PP heterogeneity can be defined as “differences in preferences among a sample”¹⁰ with the existence of subgroups of patients with relevant differences in preferences. Within this framework, the measurement of psychological constructs might be relevant to obtain a psychological profile of subpopulations of patients with relevant differences in preferences. The identification of subgroups of patients with heterogeneous preferences could

Be particularly valuable when there is a suggestion that these differences are important enough to alter the decision whether to approve a product for at least one subgroup of patients.¹⁰

The need to better understand sources of PP heterogeneity is also stressed by a qualitative study carried out within the PREFER project investigating stakeholders’ attitudes, expectations and concerns towards measuring and including PP in decision-making along the medical product lifecycle. Its results highlighted that stakeholders were often concerned about whether and how psychological and emotional factors may influence PP. Stakeholders reasoned that these factors and their impact on preferences are often unknown for researchers and, hence, difficult to control for in PP studies.^11,12

All this empirical evidence and the practical demands called for a need of identifying, describing and assess the feasibility of using different ways to profile psychological variables that, alongside demographic (eg, age, gender, culture) and clinical (eg, disease stage, comorbidity) characteristics of patients, can affect the construction, elicitation, and interpretation of PP. With this purpose, a systematic review was carried out within the PREFER project to explore the existing instruments which are currently used in PP studies or health-related decision studies to measure the psychological variables that can affect the formation of PP, its elicitation and its heterogeneity.¹³ This initial step provided an overview of the literature on the psychological constructs measured in PP or health-related decisions studies and the instruments used to measure these constructs. This review resulted in the identification of 18 constructs and 33 psychological instruments indicating that health literacy, numeracy, and locus of control have an impact on health-related preferences and decisions The present work builds upon these results and aims at providing researchers and stakeholders with useful criteria and tools to select psychological constructs to include in PP studies to take into account patient heterogeneity and preferences formation. Specifically, three main tasks were undertaken to achieve this goal: (i) identifying a list of candidate psychological constructs by supplementing the results of the PREFER systematic review¹³ with input from experts in the field of PP studies, health psychology, and psychological assessments; (ii) proposing recommendations on which psychosocial constructs to include in PP studies reached through a consensus-based process, that is a two-round Delphi method; (iii) advancing a set of criteria and a check list that can assist in describing and selecting psychological instruments to be included in PP studies.

Psychological Constructs Linked to Patient Preferences

To provide researchers and other stakeholders with practical advice on psychological constructs that can influence PP formation and its heterogeneity, we propose a list of psychological constructs empirically and theoretically linked to PP. Specifically, a pool of 19 international experts in the fields of PP studies, health psychology, and psychological assessment were invited to review an existing list of psychological constructs identified during an earlier systematic review conducted within the PREFER project.¹³ Reviewers were internationally recognised experts both within and external to the PREFER project who were not involved in the original systematic review. The reviewers were invited to contribute via e-mail. Out of 19 experts contacted, 11 provided their feedback. Participants were provided with the list of psychological constructs from the systematic review, and they were asked to supplement the list with: Psychological dimensions that can empirically or theoretically have a bearing on patients’ preferences formation and/or explain PP heterogeneity. Three authors (SR, DM, GP) independently analysed the experts’ contributions to evaluate whether the suggested constructs met the aforementioned inclusion criteria. Disagreements in construct selection were resolved through discussion between the researchers until consensus was reached. The panel of experts indicated 13 constructs which supplemented the 18 constructs identified in the systematic review leading to the identification of a total of 31 psychological constructs which could be relevant for future PP studies (Table 1).

Table 1 List and Definition of Psychological Constructs Identified by the Systematic Literature Review from Russo et al¹³ and the Panel of Experts. They are Reported in Alphabetic Order

Construct	Description of Construct
Anxiety	A distinction between state and trait anxiety has become commonplace.^Citation14,Citation15 State anxiety is defined as an unpleasant emotional arousal in face of threatening demands or dangers. A cognitive appraisal of threat is a prerequisite for the experience of this emotion.^Citation16 Trait anxiety, on the other hand, reflects the existence of stable individual differences in the tendency to respond with state anxiety in the anticipation of threatening situations.
Assertiveness	Assertiveness is a proactive response in difficult situations to contrast with passive or aggressive reactions.^Citation17
Autonomy preference	Autonomy is self-governance over decisions; a decision or choice of action is considered autonomous if it comes from within and is free from external control or influence.^Citation18,Citation19 Thus, autonomy preference reflects patient’s preference for self-governance in medical care.
Behavioural inhibition and activation	Behavioural inhibition system and the behavioural approach system are two general motivational systems that underlie behaviour. The behavioural approach system is believed to regulate appetitive motives, in which the goal is to move toward objectives and something desired. A behavioural avoidance (or inhibition) system is said to regulate aversive motives, in which the goal is to move away from something unpleasant.^Citation20
Conservatism	Conservatism is defined as the disposition to preserve tradition and established institutions and resist and oppose to change.^Citation21
Control preference	Control preference reflects patient’s preferred level of their own versus their physician’s control or a collaborative role over a treatment decision.^Citation22
Coping style	Coping style is defined as the habitual pattern of individuals when reacting to stress either across different situations or over time.^Citation23
Decision-making styles	Decision-making style is the habitual pattern individuals use in decision-making, or characteristic mode of perceiving and responding to decision-making tasks.^Citation24,Citation25
Depression	Depression is a state of low mood and aversion to activity that can affect a person’s thoughts, behaviour, feelings, and sense of well-being.^Citation26
Dispositional optimism	Dispositional optimism is defined as generalized expectancy for positive future events.^Citation27
Health anxiety	Health anxiety is defined as a worry about physical health and can range from mild concern to severe or persistent anxiety such as that found in hypochondriasis.^Citation26
Health literacy	Health literacy is the patient’s ability to read, understand and use healthcare information appropriately.^Citation28
Health locus of control	Health locus of control is defined as a generalized expectation about whether one’s health is controlled by one’s own behaviour or forces external to oneself.^Citation29 An individual with an internal locus of control believes that outcomes are a direct result of his or her own behaviour. An individual with an external locus of control believes that outcomes are a result of either chance or powerful other people, such as physicians.
Health numeracy	Health numeracy refers to the patient’s ability to apply and manipulate numerical concepts in the healthcare context.^Citation30,Citation31
Health orientation	Health orientation is an individual-differences concept defined as an individual’s motivation to engage in healthy attitudes, beliefs, and behaviours.^Citation32
Illness perception	Illness perception is defined as patients’ own implicit and common-sense beliefs about their illness.^Citation33
Mastery	Mastery motivation has been defined as a psychological force that stimulates an individual to attempt independently, in a focused and persistent manner, to solve a problem or master a skill or task which is at least moderately challenging for him or her.^Citation34
Mood states	In contrast to emotion mood is defined as a transient, low-intensity, nonspecific, and subtle affective state that often has no definite cause.^Citation35
Need for closure	The need for cognitive closure is conceptualized as a cognitive-motivational factor that underlies how laypersons approach and form their knowledge about the social world.^Citation36 Need for closure varies along a continuum with one end representing a need to attain cognitive closure and the other end representing a need to avoid cognitive closure. People with high need for closure can be described as having a preference for quick decision-making, predictability, structure, and low tolerance for ambiguity. Individuals with a low need for closure prefer variety, flexibility, uncertainty, and slow decision-making.
Need for cognition	Need for cognition refers to peoples “tendency to engage in and enjoy effortful cognitive endeavors.”^Citation37 Individuals with a high need for cognition enjoy the thinking process, are motivated to apply their thinking skills and are more likely to be able to process and systematize information.^Citation38
Patient Activation	Patient activation refers to the degree to which an individual has knowledge, motivation, skills, and confidence to make effective health-related decisions.^Citation39
Personality	Personality is the dynamic organisation within the individual of those psychophysical systems that determine his characteristic behaviour and thought.^Citation40
Psychological well-being	Psychological well-being is defined as a combination of several aspects of positive psychological functioning, which includes self-acceptance, positive relations with others, autonomy, environmental mastery, purpose in life, and personal growth.^Citation41
Rational and experiential thinking styles	Rational and experiential thinking styles are habitual pattern of information processing.^Citation42 While the rational style emphasizes a conscious and analytical approach, the experiential style emphasizes a pre-conscious and a more affective approach.
Resilience	Resilience is defined as the process of adapting well in facing traumas, adversities, threats, tragedies, and sources of stress.^Citation43
Risk propensity	Risk propensity is described as a function of the person’s perception of risk and the person’s willingness to take on this risk.^Citation44
Self-efficacy	Self-efficacy is an individual’s belief in his or her capacity to master the cognitive, motivational, and behavioural resources required to perform a specific action in a given situation.^Citation45
Sensation Seeking	Sensation seeking is defined as the seeking of various, novel, complex, and intense sensations and experiences, and the willingness to take physical, social, legal, and financial risks for the sake of such experience.^Citation46
Sense of coherence	The sense of coherence is a construct that refers to the extent to which one sees one’s world as comprehensible, manageable, and meaningful.^Citation47
Social support	Social support is defined as a “social network’s provision of psychological and material resources intended to benefit an individual’s ability to cope with stress”.^Citation48
Treatment-related beliefs	Treatment-related beliefs are defined as the specific patient’s perception of the need to take medication and concerns about it as well as the general beliefs about pharmacotherapy.^Citation49

Note: Psychological constructs proposed by experts which supplement the list of the systematic review from Russo et al¹³ are indicated in bold.

We grouped the psychological constructs considering the available scientific evidence supporting the relevance of the psychological constructs for PP studies. Four researchers (UK, DM, CAP, SR) organised the constructs in three classes basing their grouping on the quality and soundness of the available evidence of the association between psychological constructs and PP within the context of decision-making processes. In particular, to classify these psychological constructs the researchers considered: (i) the number of studies assessing the association between each psychological construct and PP; (ii) the quality of these studies; (iii) the overall agreement and consistency of their finding. For each identified construct emerging from the systematic review an overall rating of the quality of the empirical evidence was considered. In Russo et al¹³ each study received a score based on its quality ranging from 1 to 3 (1=weak; 2=moderate; 3=strong), then summed to the score of the other studies investigating the same construct and the mathematical average of the resulting value was categorised as follow: from 1 to 1.4 weak; from 1.41 to 1.8 weak to moderate; from 1.81 to 2.2 moderate; from 2.21 to 2.6 moderate to strong; from 2.61 to 3 strong. The constructs suggested by the experts were included in the Group C as no empirical data are yet available on the association between these constructs and PP (see Table 2). Disagreements in grouping between researchers were resolved through discussion until consensus was reached.

• Group A: psychological constructs for which there are strong and consistent results regarding their role in influencing patients’ preferences and decisions.

• Group B: psychological constructs that could be theoretically promising in the field of PP studies, but the number of available studies with consistent results is not yet satisfactory.

• Group C: psychological constructs for which data are not available, or there are inconclusive or inconsistent results on their role in influencing PPs and decisions.

Table 2 List of Identified Psychological Constructs Organised by Strength of Available Empirical Evidence (Alphabetically Ordered Within Each Class)

	Constructs
Group A	Health literacy
	Health locus of control
	Health numeracy
Group B	Assertiveness
	Autonomy preference
	Conservatism
	Control preference
	Coping style
	Decision-making style
	Dispositional optimism
	Health orientation
	Patient activation
	Resilience
	Risk propensity
	Self-efficacy
	Treatment-related beliefs
Group C	Anxiety
	Behavioural inhibition and activation
	Depression
	Health anxiety
	Illness perception
	Mastery
	Mood states
	Need for closure
	Need for cognition
	Personality
	Psychological well-being
	Rational and experiential thinking styles
	Sensation seeking
	Sense of coherence
	Social-support

Development of Recommendations: A Delphi Consensus Process

To further assist researchers and stakeholders with the selection of psychological constructs for preference studies, we developed recommendations based on the empirical evidence and experts’ contributions previously gathered using a consensus-based process. Formal expert review via an online two-round Delphi panel was used to reach consensus.⁵⁰

Materials and Procedures

Although there is no standardized sample size for the Delphi approach, 8 to 15 panel members may be adequate for a focused Delphi study⁵¹ such as the one reported here. Kezar and Maxey⁵² suggest that a smaller expert panel is appropriate when the panel is heterogeneous and up to 30 or more panel members may be required with a homogenous panel of experts. Accordingly, a minimum sample of 15 experts was targeted for the panel. Researchers and professionals with a diverse range of expertise were invited via email to participate in the online Delphi consensus process. Fields of expertise included health preference research, clinical and health psychology, health economics, public health, risk communication, and decision-making. Panel members were recruited through searching relevant literature and recommendations from the project team. Additionally, five international organisations (European Health Psychology Society, Society for Health Psychology, European Association for Communication in Healthcare, Society for Medical Decision Making, and International Society for Pharmacoeconomics and Outcomes Research) were contacted and asked to forward the Delphi survey invitation to their associates. No remuneration was offered to participants. Participants were sent a web link to the consent form and once informed consent was provided, participants were presented with the study survey.

Participants were asked to provide demographic and professional information and rate their level of agreement on considering and measuring each of the 31 constructs in PP studies (Table 1) on a 5-point Likert scale (“strongly disagree” to “strongly agree”) and also provided open text space for participants to give reasons for their evaluations. For each psychological construct outlined, the proportion of panel members that either agreed or disagreed with the inclusion/usefulness of each corresponding construct in PP studies was calculated. Criteria for the level of consensus were defined a priori based on agreement responses on the Likert scale [“agree” or “strongly agree”], and included the following categories of consensus: “unanimity” of inclusion, “consensus”, “majority” and “discrepancy” when 100%, 80%, ≥70%, and <70% of participants agreed with inclusion. Two drag-and-drop items asked participants to indicate which of the outlined psychological constructs could account for preference heterogeneity and misunderstanding differences in PP, respectively. “Preference heterogeneity” was defined as differences in preferences across sub-populations or classes of people, while “misunderstanding differences” were defined as differences in patient preferences due to patients not understanding or not interpreting the patient preference study questions and information as meant by the researcher. After presenting the two operational definitions, participants were also asked to select one of five figures visually presenting the possible relations between the two concepts (see Figure 1).

Figure 1 Item asking participants’ understanding of the relations between heterogeneity differences and misunderstanding differences in patient preference studies. Note: (A) Hetereogeneity differences as subset of misunderstanding differences; (B) misunderstanding differences as subset of heterogeneity differences; (C) misunderstading and heterogeneity differences completley overlap; (D) misunderstanding and heterogeneity differences are completely distinct; (E) misunderstading and heterogeneity differences partially overlap.

Participants who completed the questionnaire in round one of the Delphi survey were invited via email to complete round two of the survey. For each construct, participants were presented with the rating they provided during round one and the overall mean and standard deviation from the panel and were asked to rate again their level of agreement for inclusion of each construct in PP studies using the same 5-point Likert scale (“strongly disagree” to “strongly agree”).

Results

Delphi Panel Demographic and Professional Features

Twenty-seven out of the 34 experts who participated in round one participated in round two of the Delphi survey. As reported in Table 3, panel members had a mean age of 41.19 years (SD= 9.77) and 59.3% were female. They mainly worked in the European Union (41%), followed by North America (33%), the United Kingdom (15%), with the remainder split among Asia, Australia, and Switzerland. The panel was heterogeneous in terms of both work sector and field of expertise. Most panel members (74%) worked in the academic sector, followed by industry, government agencies, and other sectors (ie, consulting services and hospitals). When considering field of expertise, participants were allowed to provide more than one area of expertise if applicable: 41% of panel members declared to work in the health psychology, 15% in clinical psychology, 22% in risk communication, 30% in medical decision-making, 30% in other types of decision-making, 26% in public health, and 22% in other fields (ie, health preference research, medical education, and sociology). The average years of experience in their field of expertise was 13.48 (SD= 8.40).

Table 3 Delphi Panel Sociodemographic and Professional Characteristics

Characteristics	n= 27
Gender, n (%) female	16 (59.3%)
Age years, mean (SD)	41.19 (±9.77)
Field of expertise, n (%)^a
Health psychology	11 (40.7%)
Clinical psychology	4 (14.8%)
Risk communication	6 (22.2%)
Medical decision-making	8 (29.6%)
Decision-making (non-medical)	8 (29.6%)
Public health	7 (25.9%)
Other fields	6 (22.2%)
Work sector, n (%)
Academic sector	20 (74.1%)
Government	1 (3.7%)
Industry	4 (14.8%)
Other	2 (7.4%)
Years of expertise, mean (SD)	13.48 (±9.77)
Expertise level in patient preference study (from 1 “not at all expert” to 5 “to a large extent expert”)	3.04 (±1.43)
Country of work, n (%)
European Union	11 (40.8%)
North America	9 (33.3%)
United Kingdom	4 (14.8%)
Asia	1 (3.70%)
Australia	1 (3.70%)
Switzerland	1 (3.70%)

Note: ^aPercentages sum up to more than 100, as more than one option could be selected.

Preference Heterogeneity and Misunderstanding Differences

In round one of the Delphi survey, experts were asked to report their opinion regarding the relationship between PP heterogeneity and misunderstanding differences in PPs by choosing one among five figures graphically representing different relationship between them (Figure 1). Most experts (44%) reported that PP heterogeneity and misunderstanding differences in PPs are different but overlapping aspects (Figure 1E); 26% that they are different and not overlapping phenomena (Figure 1D); 18% that misunderstanding differences is a specific subtype of PP heterogeneity (Figure 1B); 7% that PP heterogeneity and misunderstanding differences refer to the same concept (Figure 1C), and % reported that PP heterogeneity is a specific subtype of misunderstanding differences (Figure 1A).

Delphi Consensus

Based on the results of round two of the Delphi survey, three classes of consensus-based recommendation on inclusion of the listed psychological constructs in PP were created (Table 4):

• Class I: psychological constructs for which there is unanimity or consensus regarding their inclusion in PP studies. The evaluation of these constructs is recommended for inclusion in PP studies when deemed relevant to address the research question and describing preference heterogeneity.

• Class II: psychological constructs for which the majority of experts agreed for their inclusion in PP studies. The evaluation of these constructs could be recommended to understand PP in a healthcare setting. The decision about including or not including a psychological construct should be made by considering: its theoretical link with PP, the previous empirical evidence about its role in influencing PP, the possible cognitive load for patients in including a further instrument to complete and other feasibility criteria (eg timelines, costs).

• Class III: psychological constructs for which there is discrepancy among experts regarding their inclusion in PP studies. Therefore, their evaluation could not be recommended based on current expert evaluation to understand PP in a healthcare setting.

Table 4 Consensus-Based Recommendations at the Two Rounds of the Delphi Method

Psychological Construct	1st Round (% Agree and Agree Strongly)	2nd Round (% Agree and Agree Strongly)	Group of the Empirical Evidence Available	Class of Recommendation
Anxiety	44.4%	37.0%	C	III
Assertiveness	33.3%	22.2%	B	III
Autonomy preference	59.3%	74.1%^a	B	II
Behavioural inhibition and activation	25.9%	18.5%	C	III
Conservatism	33.3%	14.8%	B	III
Control preference	70.4%^a	85.2%^b	B	I
Coping style	48.1%	48.1%	B	III
Decision-making style	59.3%	70.4%^a	B	II
Depression	44.4%	51.9%	C	III
Dispositional optimism	40.7%	33.3%	B	III
Health anxiety	55.6%	59.3%	C	III
Health literacy	85.2%^b	96.3%^b	A	I
Health locus of control	70.4%^a	85.2%^b	A	I
Health numeracy	81.5%^b	92.6%^b	A	I
Health orientation	70.4%	74.1%^a	B	II
Illness perception	70.4%^b	92.6%^b	C	I
Mastery	33.3%	25.9%	C	III
Mood states	22.2%	11.1%	C	III
Need for closure	37.0%	25.9%	C	III
Need for cognition	37.0%	33.3%	C	III
Patient activation	70.4%^a	85.2%^b	B	I
Personality	37.0%	29.6%	C	III
Psychological well-being	59.3%	55.6%	C	III
Rational and experiential thinking styles	48.1%	33.3%	C	III
Resilience	44.4%	33.3%	B	III
Risk propensity	77.8%^b	88.9%^b	B	I
Self-efficacy	55.9%	63.0%	B	III
Sensation seeking	25.9%	14.8%	C	III
Sense of coherence	29.6%	11.1%	C	III
Social support	59.3%	55.6%	C	III
Treatment-related beliefs	74.1%^a	92.6%^b	B	I

Notes: ^aHighlighted majority; ^bhighlighted consensus.

As reported in Table 4, the percentages of agreement at the second round of the Delphi ranged between 11% and 96%, with the lowest percentage for the inclusion of mood states and sense of coherence, and the highest percentages for the inclusion of health literacy. Specifically, consensus was reached for the inclusion of eight psychological constructs, namely health literacy, health numeracy, illness perception, treatment-related beliefs, risk propensity, health locus of control, control preference, and patient activation (Class I). Three constructs, namely autonomy preference, decision-making style, and health orientation, reached the majority consensus (Class II). Twenty out of the 31 psychological constructs considered by the Delphi panel did not reach consensus about their inclusion or exclusion when considering PP studies falling into the Class III of recommendation.

As reported in Table 5 amongst the 11 constructs included in Class I and II, health literacy and health numeracy were mainly conceptualized as psychological constructs that can account for understanding differences in patient preferences due to patients not understanding or not interpreting the patient preference study questions and information as meant by the researcher (ie, misunderstanding differences in PPs). On the contrary, the remaining nine constructs were mainly considered as relevant psychological characteristics that could account for PP heterogeneity.

Table 5 Percentages of Selection of the Class I and Class II Psychological Constructs as Relevant Information to Account for PP Heterogeneity or Misunderstanding Differences in PPs

Constructs	Account for PP Heterogeneity (%)	Account for Misunderstanding Differences in PPs (%)
Autonomy preference	59.3%	14.8%
Control preference	51.9%	22.2%
Decision-making style	37.0%	29.6%
Health literacy	44.4%	85.2%
Health locus of control	59.3%	25.9%
Health numeracy	37.0%	81.5%
Health orientation	33.3%	11.1%
Illness perception	55.6%	29.6%
Patient activation	29.2%	22.2%
Risk propensity	51.9%%	14.8%
Treatment-related beliefs	55.6%	37.0%

Descriptive Criteria and Checklist for Selecting Measurement Instruments

After deciding which construct(s) to measure depending on evidence- and consensus-based considerations, stakeholders and researchers should appraise available instruments in terms of quality, usability, restrictions, and shortcomings and select the best tool to evaluate these candidate psychological differences. This step is especially relevant when more than one instrument is available for the same construct.

We therefore propose a set of descriptive criteria (see Supplementary Material A) and a criteria checklist (see Supplementary Material B) to assist researchers and other stakeholders respectively in i) assessing the overall quality and usability of each tool, comparing different instruments when more than one instrument is available for the same construct, and selecting the best measure and ii) evaluating the feasibility of measuring candidate psychological differences in the context of a specific PP study. In detail, two researchers (SR, DM) developed a set of descriptive criteria and a checklist to evaluate psychological instruments and to guide researchers to decide which tool best fit the research needs of a specific PP study. The initial lists of descriptive criteria and the initial draft of the checklist were extended and revised based on conversations and feedback from the authors and other members of PREFER consortium with expertise and experience in the fields of PP studies, health psychology, and psychological assessment.

Starting from the final list of descriptive criteria (see Supplementary Material A), here we are proposing a briefer set of questions addressing the main features of psychological instruments that could support stakeholders and researchers in evaluating and selecting available measures. This checklist – reported in Table 6 - may be especially helpful when more than one instrument is available for the same construct. By given answers to each question, stakeholders are assisted in assessing the overall quality and usability of each tool.

Table 6 Brief Checklist for the Evaluation and Selection of Available Measures to Assess Psychological Constructs

Criteria	Recommendations
1. Is the instrument valid and reliable?	It could be recommended to use in a PP study only psychological measures with consistently proven reliability and validity in the specific population under investigation.
2. Is the instrument available in the language the patient preference study will be conducted?	The use of psychological instruments specifically developed or properly adapted to the language and culture of the target population for the PP study is recommended.
3. Is this instrument designed for a particular population of patients or a specific disease?	Psychological instruments developed specifically for the population being targeted by the PP study should be preferred to other tools.
4. What is the outcome measure of the instrument?	Psychological measures with standardized and norm-referenced scores should be preferred over instruments with only raw scores.
5. Does the instrument provide cut-offs classifying patients?	If relevant, psychological measures providing cut-off scores are preferred over instruments with only raw or standardized scores.
6. Is the instrument protected by copyright/license?	When budget constraints do not allow the use of psychological instruments with fees, researchers might consider alternative measures with equivalent characteristics but not requiring payment.
7. What is the average cognitive burden and time commitment for respondents completing the instrument?	Required time for completion and associated patient-reported respondent burden should be investigated through pretesting and piloting studies. When available psychological instruments are deemed to be equivalent in their informative values, the less cognitively burdensome and time-consuming tool should be preferred over more cognitively and time-demanding ones.

Strong knowledge and skills concerning psychometrics, psychological assessment and testing are essential to properly evaluate available psychological measures and select the best measurement tool based on its features and psychometric characteristics. Specifically, the first criterion advanced deals with two essential features of any psychological tool: validity and reliability. They refer respectively to the extent to which any tool actually measures what it is intended to measure⁵³ and the precision of psychological measures in terms of internal consistency or the consistency of observed scores over different administration of the same instrument.⁵⁴ Since validity and reliability of an instrument should be investigated and demonstrated in a specific context of use, empirical evidence coming from previous validation studies in similar contexts of the PP study should be considered to assess the appropriateness of candidate psychological measures. Stakeholders are recommended to use only those psychological tools with consistently proven reliability and validity in the specific population under investigation in the PP study. Second, psychological instruments are generally developed, constructed, and validated for a specific language and culture. They can be adapted for use in a new country, culture, and/or language through “cross-cultural adaptation”, namely a unique and complex method to reach equivalence between the original and the translated versions of the instrument.⁵⁵ Caution should be taken when using in a PP study a translated version of an instrument that has not been properly adapted to the new language and culture. Thus, stakeholders are always recommended to choose only those psychological instruments specifically developed or properly adapted to the language and culture of the target population of their PP study. Third, since psychological instruments are often constructed and validated for a specific population of patients, caution should be taken when using the tool to a different population without previous empirical evidence regarding its validity and reliability in the new population. Thus, stakeholders are recommended to prefer those psychological instruments that have been developed specifically for the population being targeted by the PP study over other available tools. Fourth, while some psychological instruments return only raw scores that may be difficult to interpret without knowledge of how one raw score compares to a norm group, other psychological tools adopt a reference group and can provide standardized scores. The latter might help stakeholders to get a clearer picture of the position of each patient in a predefined population for the psychological differences being measured. Thus, if relevant, psychological measures with standardized and norm-referenced scores should be preferred over instruments with only raw scores. Fifth, some psychological tools can also provide meaningful cut-off scores to classify patients into groups based on their scores. These cut-off scores are generally used with screening purposes to differentiate clinical populations from non-clinical ones or differentiate among people with adequate or inadequate abilities. If relevant, stakeholders are recommended to prefer psychological measures providing cut-off scores over tools providing only raw or standardized scores. Sixth, while the noncommercial use of psychological instruments is often allowed without requiring written permission, a license or a fee, sometimes developers require explicit approval to use their instrument.⁵⁶ Other authors use the copyright status and put limits on use, adaptations, and translations. Finally, other instrument developers can charge fees to use their tools, namely licensing fee, administration fee or fee for obtaining scoring instructions. When budget constraints do not allow the use of psychological instruments with fees, stakeholders might consider alternative measures with equivalent characteristics but not requiring payment. Finally, the protection of people participating in research demands investigators and regulatory bodies to minimize the respondent burden.⁵⁷ Respondent burden is a multifaceted phenomenon encompassing mainly cognitive burden and time commitment to take part in a study,⁵⁸ as well as participants’ perceived psychological, physical, and economic discomfort caused by the participation in research.⁵⁹ Psychological instruments can vary in both their completion time and required cognitive effort for completion. Thus, stakeholders should balance the benefits of including psychological profiling within PP studies with the possible additional respondent burden. Moreover, when two psychological instruments are deemed to be equivalent in their informative values, the less cognitively burdensome and time-consuming tool should be preferred over the more cognitively and time demanding.

We would like to stress that the criteria and checklist advanced here is not intended to be rigidly applied to PP studies but rather to function as an initial guide to support PP researchers and other relevant stakeholders in selecting psychological instruments to be included in PP studies.

Discussion and Limitations

The present paper provides scholars and other stakeholders involved in the field of PP studies with consensus-based recommendations for the inclusion of psychological dimensions in PP studies and a checklist for the appraisal and selection of psychological instruments to assess the candidate psychological constructs. Building upon literature on the relationship between psychological dimensions and PP and on contributions from experts, we identified a list of 31 psychological constructs which could be relevant when planning PP studies. A panel of experts in PP studies and related disciplines enlarged a list of psychological constructs that emerged from a systematic literature on psychological dimensions known to have an impact on patients’ preferences and health-related decisions. To provide the reader with a snapshot of the current scientific data on the relationship of each construct with PP, the authors grouped the final list of psychological constructs in three clusters according to the strength and soundness of the existing empirical evidence.

Furthermore, the psychological dimensions underwent a consensus-based process to identify those psychological constructs to be recommended for inclusion in PP studies. Consensus for inclusion was reached for 11 psychological constructs. A consensus-based classification of the psychological constructs also emerged. Three classes of recommendation have been detected. Class I-constructs, namely control preference, health literacy, health locus of control, health numeracy, illness perception, patient activation, risk propensity, and treatment-related beliefs have been recognised as those with a theoretical basis for forming preferences and predicting preference heterogeneity. These constructs should be a basic consideration for inclusion in preference studies conducted for medical product decision-making. Constructs in Class II (autonomy preference, decision-making style, and health orientation, reached the majority consensus) have been considered as promising constructs to better understand PPs. In Class III are listed psychological constructs for which there was not consensus among experts regarding their inclusion in PP studies. Our expert panel did not recognise the potential contribution of these psychological constructs in shedding light on PP formation or heterogeneity. Because of lack of consensus, their evaluation could not be recommended. Nevertheless it has to be noted that expert consensus on inclusion might fail to be reached due to lack of scientific evidence on the relationship between PP and the psychological dimensions considered. Out of the 11 psychological dimensions reaching expert agreement for inclusion, 10 belong to Group A or B showing association between expert-consensus and the empirical data. Psychological constructs that have an important role in PPs may not yet have been studied systematically. Moreover, it should be underlined that available empirical evidence concerning the impact of psychological constructs on PP is still limited to specific contexts, patient populations, or diseases. Thus, the generalizability of these results to other medical conditions might not be guaranteed. More investigations are needed on the link between PP and psychological dimensions. It is particularly relevant to explore how particular psychological dimensions connect to PP heterogeneity. As the Medical Device Innovation Consortium highlighted¹⁰ differences in PP amongst subgroups of patients may be highly relevant to guide decision in medical products and devices approval process. From this perspective, detecting patient preference heterogeneity can be considered as a complement of understanding heterogeneity of treatment effects based on other observable characteristics because it might assure that sponsors and regulators have appropriately identified the specific population for which the medical product should be indicated. Understanding patient preference heterogeneity might be especially relevant if the benefits of a medical product outweigh its risks only for a subgroup of patients but not for others. In this case, it would be meaningful to infer the size of the overall patient population presumably outweighing benefits over risks and identify specific patients’ characteristics that are associated with the likelihood of being in this group.¹⁰ The identification of heterogeneity in patient preferences might inform patient subgroup considerations as part of the benefit-risk assessments.¹ Specifically, it is relevant to identify any potential population-, condition-, and treatment-variability in patient preferences to obtain a clear picture of preferences of patients from the full spectrum of disease for which the medical product is intended to be used. Working in this direction our study explored how our expert panel members understand the role that psychological constructs can play in explaining differences in PP. In particular, in the context of PP studies, we asked them which psychological constructs could explain both preference heterogeneity and possible differences springing from misunderstanding between experimental requests and patients’ understanding of those.

Amongst the 11 constructs that reached agreement on inclusion, health literacy and health numeracy were the ones considered the more informative when it comes to differences due to misunderstanding. For example, in a PP study aimed to determine maximal acceptable risk levels for a certain treatment in elderly people with a chronic disease, the assessment of health literacy and numeracy may be especially relevant to identify patients that may not able to properly understand complex numerical and/or medical information or the scenario proposed by the researcher.

We furthermore investigated what relationship our experts assumed exist between heterogeneity in PP defined as “differences in preferences across sub-populations or classes of people” and differences in misunderstanding differences defined as “differences in patient preferences due to patients not understanding or not interpreting the patient preference study questions and information as meant by the researcher.” The majority of the expert panel members reported that the two phenomena are similar, sometimes overlapping, but distinct.

Further research is necessary to expand the recommendations advanced here and base them on scientific data to further them towards more comprehensive and evidence-based guidance. Specifically, the classes of recommendation identified here are a first attempt to develop a common framework to further facilitate sharing of information and the accumulation of evidence to demonstrate how a specific psychological construct relates to PPs and which measures should be considered within a given context. It is particularly relevant to explore how a particular psychological dimension connects to PP heterogeneity.

Once the psychological dimensions to be investigated and included in a PP study have been identified, researchers need to select appropriate measurement instruments. We have suggested a checklist and set of descriptive criteria to assist and guide scholars in the selection of relevant psychological instruments to be include in PP studies. It should be stressed that the applicability and usefulness of the checklist and descriptive criteria advanced here are not confined to the assessment of PP. Our checklist and criteria are all grounded in the common need for good study design principles and are intended to support researchers to select or develop an optimal instrument to address specific research questions. Overall, the decisions concerning the inclusion of psychological constructs and related psychological tools for their measurement should be made by a multidisciplinary team with clinical and psychological expertise, and study management experience. Specifically, psychological knowledge is needed to identify relevant psychological differences that can be related to the formation and heterogeneity of patient preferences. Moreover, strong knowledge and skills concerning psychological assessment and measurement are essential to properly evaluate available psychological measures and select the best measurement tool based on its psychometric characteristics, such as overall validity, reliability, and measurement properties.

Conclusions

The list of psychological constructs identified in this study, along with the classes of recommendations, the checklist and descriptive criteria to evaluate psychological measures are valuable tools to assist researchers and stakeholders when designing PP studies in order to obtain results that can inform decision-making along the medical product lifecycle and enable the implementation of personalised medicine and patient-centred care. The classification and criteria advanced here are meant to assist and to be considered in light of existing guidelines on qualities and characteristics of PP information from relevant agencies and authorities such as the FDA’s Center for Devices and Radiological Health guidance on Patient Preference Information.⁶⁰ The present study serves as a common framework to stimulate further research, and foster reflection and discussion on the formalisation of guidelines to assist stakeholders in whether and how to include psychological dimensions and measurements when designing PP studies.

Acknowledgments

We would like to thank the PREFER members of the work package 2.5 who contributed providing their precious insight and expertise: Jorien Veldwijk, Rosanne Janssens, Chiara Jongerius, Isabelle Huys, Richard Hermann, Flavia Faccio, Silvia Pizzoli. We furthermore are deeply grateful to the experts who contributed to the study as either advisory experts or Delphi panel members.

Disclosure

The authors report no conflicts of interest in this work.

Additional information

Funding

This research was supported by The Patient Preferences in Benefit-Risk Assessments during the Drug Life Cycle (PREFER) project. PREFER has received funding from the Innovative Medicines Initiative 2 Joint Undertaking under grant agreement No 115966. This Joint Undertaking receives support from the European Union’s Horizon 2020 research and innovation programme and EFPIA. This text and its contents reflects the author’s and PREFER project’s view and not the view of IMI, the European Union or EFPIA.