A systematic review of questionnaires about patient’s values and preferences in clinical practice guidelines

Figures & data

Table 1 Items of the psychometric methods

1. Item and dimension generation
The item generation phase should include all important elements of patient’s preferences by reviewing the existing questionnaires, literature, and opinions from experts and patient focus groups. The authors should describe the generation of initial items, including literature review,^Citation26 the Delphi survey,^Citation27 and consensus meetings,^Citation28 etc.	⎕ Yes
	⎕ No
	⎕ Uncertain
	⎕ Not applicable
2. Pretest and pilot testing
The authors should state that they pretested the items of questionnaires. The process of pretest and pilot testing is for revision of the questionnaire into the final validated version by using the response from the protest group. Items with ambiguous meanings can be eliminated to maximize the reliability and validity of the questionnaire.	⎕ Yes
	⎕ No
	⎕ Uncertain
	⎕ Not applicable
3. Validity testing
Content validity. It is usually reported in questionnaires judged by the panel after literature reviews and focus group interviews. During the testing phase, patients are asked to review the items in the questionnaire for content validity on patient’s values and preferences before it is administered to study participants.	⎕ Yes
	⎕ No
	⎕ Uncertain
Construct validity. Because there is no standard for measuring patient’s satisfaction, researchers usually compare their questionnaire with other validated instruments or other related questions for this correlation. Discriminant validity requires that the construct should not show correlation with dissimilar variables.	⎕ Not applicable
4. Reliability testing
The authors should state that they tested the reliability.	⎕ Yes
Internal consistency. Cronbach’s α should be reported, the value should be 0.7–0.9, as a value above 0.9 may indicate that the questionnaire is too narrow in scope.	⎕ No
	⎕ Uncertain
Test-retest reliability. The minimum value of the correlation coefficient should be 0.7.	⎕ Not applicable
Intrarater and interrater agreements. Intrarater agreement is the agreement between observations made by the same rater on two different occasions.
5. Feasibility/acceptability
Time to complete. The authors should provide the time to complete the questionnaires.	⎕ Yes
	⎕ No
Response rates. The authors should provide the response rates of the questionnaires. A response rate of 50% is considered adequate for analysis.^Citation29	⎕ Uncertain
	⎕ Not applicable

Figure 1 Trial selection flow chart.

Table 2 Characteristics of included studies

Author (year)	Disease domain	Study design	Study duration (months)	Setting	No of patients	Provided training to patients	Calculated sample size	Involved family members	Examined knowledge about diseases
Bo et al (2014)^Citation30	Lung cancer	NR	19	Inpatient	135	NR	NR	No	Yes
Welsh and Tiffin (2014)^Citation31	Psychosis	Brief self-report survey	3	NR	75	NR	NR	Yes	No
Vu et al (2015)^Citation32	Colonoscopy	Single-center, prospective survey	NR	Outpatient	500	Yes	NR	No	Yes
Tong et al (2016)^Citation33	Lung cancer	Pilot study	NR	NR	225	Yes	NR	No	Yes
Sekimoto et al (2004)^Citation34	T2DM	NR	3	Outpatient	133	Yes	NR	Yes	Yes
Rid et al (2015)^Citation35	Incapacity	Self-administered quantitative survey	20	Inpatient Outpatient	1,116	Yes	Yes	No	Yes
Noble et al (2015)^Citation36	Thrombosis	Conjoint analysis surveys	NR	Inpatient	100	Yes	Yes	No	No
Mazur et al (1999)^Citation37	Prostate cancer	Cross-sectional study	NR	Outpatient	228	Yes	NR	No	Yes
Matti et al (2010)^Citation38	Optic neuritis	NR	5	Outpatient	27	Yes	NR	No	Yes
Maciver et al (2016)^Citation39	Heart failure	Qualitative study	12	Outpatient	25	NR	Yes	No	Yes
Sanford et al (2014)^Citation40	Myeloid leukemia	NR	3	Outpatient	56	Yes	NR	Yes	Yes
Koh et al (2010)^Citation41	ACL reconstruction	NR	22	Outpatient	129	Yes	NR	No	Yes
Ha and Mcdonald (2017)^Citation42	Preterm birth	Cross-sectional survey	2	Outpatient	311	No	Yes	Yes	Yes
Gareen et al (2015)^Citation43	Colorectal cancer	NR	12	Outpatient	2,310	No	NR	No	Yes
Fiks et al (2013)^Citation44	ADHD	NR	6	Outpatient	148	No	NR	Yes	Yes
Eckman et al (2015)^Citation45	Thrombus	Cross-sectional study	NR	Outpatient	123	Yes	NR	No	Yes
Choudhry et al (2015)^Citation46	Melanoma	NR	12	Outpatient	301	NR	NR	No	No
Calderwood et al (2011)^Citation47	Colorectal cancer	Cross-sectional survey	14	Outpatient	100	Yes	Yes	No	Yes
Bolge et al (2016)^Citation48	RA	Self-administered online survey	NR	Outpatient	243	No	NR	No	Yes
Hofman et al (2011)^Citation49	T1DM	Open-label, randomized, crossover usability test	NR	Inpatient	15	Yes	NR	Yes	Yes

Abbreviations: ACL, anterior cruciate ligament reconstruction; ADHD, attention deficit/hyperactivity disorder; NR, not reported; RA, rheumatoid arthritis; T1DM, type 1 diabetes mellitus; T2DM, type 2 diabetes mellitus.

Table 3 Characteristics of included questionnaires

Questionnaire from studies	Mode of administration	Format						Incentives	Ethics statement
		No of items	No of pages	The setting of the question options for the questionnaires	No of sections	Response options (No of items)	Range of scores		Anonymous	Informed consent
Bo et al (2014)^Citation30	NR	9	1	SCQ	2	–	–	NR	Yes	Yes
Welsh and Tiffin (2014)^Citation31	Interview	10	3	CT	4	4-point scale (10)	Very likely, likely, unlikely, very unlikely	NR	NR	Yes
Vu et al (2015)^Citation32	Interview	12	2	SCQ+CT+OeQ	2	4-point scale (3)	Very important to not important at all	NR	Yes	NR
Tong et al (2016)^Citation33	Interview	50	17	SCQ+CT	4	5-point scale (3)	Strongly prefer left, somewhat prefer left, indifferent, somewhat prefer right, strongly prefer right	Yes	Yes	Yes
Sekimoto et al (2004)^Citation34	Interview	7	2	SCQ+CT	2	–	–	NR	Yes	Yes
Rid et al (2015)^Citation35	Interview	46	6	SCQ+CT+OeQ	4	4-point scale (23)	Extremely important, moderately important, just a little important, not important at all	NR	Yes	NR
Noble et al (2015)^Citation36	Interview	10	7	SCQ+CT+OeQ	2	5-point scale (10)	Do not agree at all to fully agree	NR	Yes	Yes
Mazur et al (1999)^Citation37	Interview	2	1	SCQ+OeQ	1	–	–	NR	NR	Yes
Matti et al (2010)^Citation38	Interview	19	3	MCQ+SCQ	3	–	–	NR	Yes	Yes
Maciver et al (2016)^Citation39	Interview	6	3	SCQ+OeQ	2	–	–	NR	Yes	Yes
Sanford et al (2014)^Citation40	Interview	18	4	SCQ+CT+OeQ	5	5-point scale (11)	Nil, minor, tolerable, tolerable but I adjust my activities or take medications for it, intolerable	NR	Yes	Yes
Koh et al (2010)^Citation41	NR	7	1	MCQ+SCQ	1	–	–	NR	Yes	Yes
Ha and Mcdonald (2017)^Citation42	NR	17	6	SCQ+CT	6	5-point scale (3)	Extremely important, slightly important, neutral, slightly not important, not at all important	NR	Yes	NR
Gareen et al (2015)^Citation43	Interview	3	9	SCQ+OeQ	6	–	–	NR	NR	NR
Fiks et al (2013)^Citation44	Interview	9	1	CT	3	6-point scale (9)	Not at all, a little, somewhat, very much, completely	NR	Yes	Yes
Eckman et al (2015)^Citation45	Interview	2	1	SCQ	2	–	–	NR	Yes	Yes
Choudhry et al (2015)^Citation46	Interview	11	4	SCQ+CT+OeQ	3	–	–	NR	No	Yes
Calderwood et al (2011)^Citation47	Interview	20	12	SCQ+CT	3	5-point scale (13)	Strongly agree, agree, neither agree nor disagree, disagree, strongly disagree	Yes	Yes	Yes
Bolge et al (2016)^Citation48	online survey	11	3	SCQ+CT	2	5-point scale (2) 4-point scale (1)	Not at all open, not very open, somewhat open, very open, extremely open; strongly prefer self-injection to strongly prefer IV infusion	NR	Yes	Yes
Hofman et al (2011)^Citation49	Interview	13	2	CT+OeQ	2	6-point scale (13)	Very difficult–very easy; not at all likely–very likely	NR	Yes	Yes

Abbreviations: CT, choice task; IV, intravenous; MCQ, multiple-choice question; NR, not reported; OeQ, open-ended question; SCQ, single-choice question.

Table 4 The evaluation of components of questionnaires

	Item generation	Pilot testing and redesign	Validity testing	Reliability testing (Cronbach’s α)	Acceptability
					Time to complete	Response rates
Bo et al (2014)^Citation30	No	No	No	No	No	No
Welsh and Tiffin (2014)^Citation31	No	No	No	No	No	Yes
Vu et al (2015)^Citation32	No	No	No	No	No	No
Tong et al (2016)^Citation33	Yes	Yes	No	No	Yes	Yes
Sekimoto et al (2004)^Citation34	No	No	Yes	No	No	Yes
Rid et al (2015)^Citation35	Yes	Yes	Yes	No	Yes	Yes
Noble et al (2015)^Citation36	Yes	No	Yes	No	Yes	No
Mazur et al (1999)^Citation37	No	No	No	No	No	No
Matti et al (2010)^Citation38	Yes	No	No	No	Yes	Yes
Maciver et al (2016)^Citation39	Yes	Yes	No	No	No	No
Sanford et al (2014)^Citation40	No	No	No	No	Yes	Yes
Koh et al (2010)^Citation41	Yes	No	Yes	No	No	No
Ha and Mcdonald (2017)^Citation42	Yes	No	Yes	No	Yes	Yes
Gareen et al (2015)^Citation43	No	No	Yes	No	Yes	Yes
Fiks et al (2013)^Citation44	Yes	Yes	Yes	Yes	No	Yes
Eckman et al (2015)^Citation45	Yes	No	No	No	No	No
Choudhry et al (2015)^Citation46	Yes	No	No	No	No	Yes
Calderwood et al (2011)^Citation47	No	No	No	No	Yes	No
Bolge et al (2016)^Citation48	No	No	No	No	Yes	No
Hofman et al (2011)^Citation49	No	No	No	No	No	No

Table 5 Summary of domains and items for reviewing the questionnaires addressing values and preferences

Domain	Study ID, author (year)	Examples	Scale	Type of disease
Effectiveness
The convenience of treatment	Bolge et al (2016)^Citation48	Would you prefer to take your biologic medication by self-injection at home or have the medication given to you by a health care professional through intravenous infusion? 1=strongly prefer self-injection, 2=somewhat prefer self-injection, 3=no preference between self-injection and intravenous infusion, 4=somewhat prefer intravenous infusion, 5=strongly prefer the intravenous infusion	5-point scale	RA
	Hofman et al (2011)^Citation49	How easy or difficult it was to detach/remove the needle cap? 1=very difficult, 6=very easy	6-point scale	Adolescents with diabetes
The frequency required for testing	Calderwood et al (2011)^Citation47	Which of the following attributes most influences your choice? The frequency required for testing Liability concerns Cost of test/coverage Complication rates	SCQ	Colorectal cancer
The types of drug preparations and therapy	Welsh and Tiffin (2014)^Citation31	How likely would you be to accept this kind of support? Usually given as tablets Tablet or liquid format Talking therapy	4-point scale	Psychosis
The benefit of the interventions	Ha and Mcdonald (2017)^Citation42	Did the medication work very well? (extremely important, slightly important, neutral, slightly not important, not at all important)	5-point scale	Premature or early birth
	Ha and Mcdonald (2017)^Citation42	When considering cerclage treatment, how important or unimportant is each of the following Whether the medication works well (extremely important, slightly important, neutral, slightly not important, not at all important)	5-point scale	Premature or early birth
	Sekimoto et al (2004)^Citation34	Regarding the treatment options, what kind of information do you require? Risks and benefits Outcome probabilities Name of the famous specialist Prognosis Physician’s recommendations	SCQ	T2DM
Treatment time	Tong et al (2016)^Citation33	If these procedures were identical in all other ways, which would you prefer? Open surgery: 3 hours Minimally invasive surgery: 3 hours Focused radiation: 1-hour outpatient visit	SCQ	Lung cancer
	Tong et al (2016)^Citation33	What bothers you the most about focus radiation? Need to be still for 1 hour for each treatment The tumor is not removed, only killed Difficult traveling for outpatient treatments	SCQ	Lung cancer
The frequency of drug administration	Noble et al (2015)^Citation36	Which frequency of administration do you prefer? Twice daily (duration of action over 24 hours) Once daily (duration of action over 12 hours)	SCQ	Cancer-associated thrombosis
	Sanford et al (2014)^Citation40	How likely are you to take your medications every day? Very unlikely, unlikely, neither unlikely nor likely, likely, very likely	5-point scale	CML
Hospital stay time	Tong et al (2016)^Citation33	What bothers you the most about open surgery? 5-night hospital stay 6- to 8-week recovery time at home	SCQ	Lung cancer
Safety
The risk of adverse effects	Bo et al (2014)^Citation30	Reasons for treatment decision making (choose the three answers with the most consideration): Fewest side effects Physician’s recommendation Least painful	SCQ	Lung cancer
The risk of complications	Ha and Mcdonald (2017)^Citation42	When considering cerclage treatment, how important or unimportant is each of the following: Concerns about potential harm to the baby (extremely important, slightly important, neutral, slightly not important, not at all important)	5-point scale	Premature or early birth
	Calderwood et al (2011)^Citation47	Which of the following attributes mostly influences your choice? The frequency required for testing Liability concerns Cost of test/coverage Complication rates	5-point scale	Colorectal cancer
Communicating the risk of the interventions	Mazur et al (1999)^Citation37	Which do you prefer when your physician communicates to you the risk of the intervention? Words Numbers Other	SCQ	Lung cancer
	Sekimoto et al (2004)^Citation34	Regarding the treatment options, what kind of information do you require? Risks and benefits Outcome probabilities Name of the famous specialist Prognosis Physician’s recommendations	SCQ	T2DM
The risk of disease transmission	Koh et al (2010)^Citation41	If you choose the allograft, what are the reasons for it? Better healing after grafting Less expensive No risk of tissue rejection or disease transmission Surgeon’s explanation	MCQ	ACL reconstruction
Prognosis
The risk of relapse	Sanford et al (2014)^Citation40	In the following questions, if the risk of relapse is different, how likely are you to stop medications for your CML? If the risk of relapse were 20%, 40%, 60%?, I would absolutely stop, I would likely stop, I would be neutral, I would likely not stop, I would absolutely not stop	SCQ	CML
	Sekimoto et al (2004)^Citation34	Regarding the treatment options, what kind of information do you require? Risks and benefits Outcome probabilities Name of the famous specialist Prognosis Physician’s recommendations	SCQ	T2DM
	Matti et al (2010)^Citation38	What is your view about having an action plan in the event of a relapse? Not interested, may be useful but unsure, will consider, will definitely have an action plan	SCQ	Optic neuritis
Better healing after treatment/recover time	Koh et al (2010)^Citation41	If you choose the allograft, what are the reasons for it? Better healing after grafting Less expensive No risk of tissue rejection or disease transmission Surgeon’s explanation	MCQ	ACL
Others
The cost patients have to pay	Ha and Mcdonald (2017)^Citation42	When considering cerclage treatment, how important or unimportant is each of the following Whether there would be a cost for you (extremely important, slightly important, neutral, slightly not important, not at all important)	5-point scale	Premature or early birth
	Calderwood et al (2011)^Citation47	Which of the following attributes mostly influences your choice? The frequency required for testing Liability concerns Cost of test/coverage Complication rates	SCQ	Colorectal cancer
	Koh et al (2010)^Citation41	If you choose the allograft, what are the reasons for it? Better healing after grafting Less expensive No risk of tissue rejection or disease transmission Surgeon’s explanation	MCQ	ACL reconstruction
Physician’s experience	Tong et al (2016)^Citation33	How important is your physician’s experience to you? Unimportant compared with other factors, somewhat important, very important, extremely important	4-point scale	Lung cancer
Physician’s recommendation	Bolge et al (2016)^Citation48	When deciding on a treatment for rheumatoid arthritis, which of the following best applies? 1=I make the final treatment decision, 2=I make the final treatment decision after considering my rheumatologist’s recommendations, 3=I share responsibility with my rheumatologist when deciding on a treatment, 4=my rheumatologist makes the final treatment decision, but seriously considers my opinion, 5=I leave all treatment decisions to my rheumatologist	5-point scale	RA
	Bolge et al (2016)^Citation48	How open are you to having your RA medication administered at home by self-injection if your rheumatologist suggested it? 1=not at all open, 2=not very open, 3=somewhat open, 4=very open, 5=extremely open	5-point scale	RA
	Sekimoto et al (2004)^Citation34	Regarding the treatment options, what kind of information do you require? Risks and benefits Outcome probabilities Name of the famous specialist Prognosis Physician’s recommendations	SCQ	T2DM
	Bolge et al (2016)^Citation48	When deciding on a treatment for rheumatoid arthritis, which of the following best applies? 1=I make the final treatment decision, 2=I make the final treatment decision after considering my rheumatologist’s recommendations,3=I share responsibility with my rheumatologist when deciding on a treatment, 4=my rheumatologist makes the final treatment decision, but seriously considers my opinion, 5=I leave all treatment decisions to my rheumatologist	5-point scale	RA
	Koh et al (2010)^Citation41	If you choose the allograft, what are the reasons for it? Better healing after grafting Less expensive No risk of tissue rejection or disease transmission Surgeon’s explanation	MCQ	ACL reconstruction
	Bo et al (2014)^Citation30	Reasons for treatment decision-making: (choose the three answers with the most consideration) Fewest side effects Physician’s recommendation Least painful	SCQ	Lung cancer
Initiation of the decision process	Maciver et al (2016)^Citation39	Who do you think should start these discussions? Would you want the physician to wait until you asked about it or would you want the physician to start the discussion?	OeQ	ICD treatment

Abbreviations: ACL, anterior cruciate ligament; CML, chronic myeloid leukemia; ICD, implantable cardioverter-defibrillators; MCQ, multiple-choice question; OeQ, subjective questions; RA, rheumatoid arthritis; SCQ, single-choice questions; T2DM, type 2 diabetes mellitus.

NieroMMartinMFingerTA new approach to multicultural item generation in the development of two obesity-specific measures: the Obesity and Weight Loss Quality of Life (OWLQOL) questionnaire and the Weight-Related Symptom Measure (WRSM)Clin Ther200224469070012017412

 PubMed Web of Science ®Google Scholar

HewittMRDELPHI SurveyLeading Edge2002261831

 Google Scholar

CellaDFLloydSRWrightBDCultural instrument equating: Current research and future directionsSpilkerBQuality of Life and Pharmacoeconomics in Clinical Trials2nd edPhiladelphia, PALippincott-Raven Publishers1996707715

 Google Scholar

BabbieERSurvey Research Methods2nd edBelmont, CAWadsworth Publishing Co Inc1990257318

 Google Scholar

BoYFangZZongZPreferences for treatment of lobectomy in Chinese lung cancer patients: video-assisted thoracoscopic surgery or open thoracotomy?Patient Preference Adher20148813931397

 Google Scholar

WelshPTiffinPAAssessing adolescent preference in the treatment of first-episode psychosis and psychosis riskEarly Interv Psychiatry20148328128523870289

 PubMed Web of Science ®Google Scholar

VuHTSayukGSGuptaNHollanderTKimAEarlyDSPatient preferences of a resect and discard paradigmGastrointest Endosc201582238138425910663

 PubMed Web of Science ®Google Scholar

TongBCWallaceSHartwigMGD’AmicoTAHuberJCPatient Preferences in Treatment Choices for Early-Stage Lung CancerAnn Thorac Surg201610261837184427623277

 PubMed Web of Science ®Google Scholar

SekimotoMAsaiAOhnishiMPatients’ preferences for involvement in treatment decision making in JapanBMC Fam Pract20045111015053839

 PubMedGoogle Scholar

RidAWesleyRPavlickMMaynardSRothKWendlerDPatients’ priorities for treatment decision making during periods of incapacity: quantitative surveyPalliat Support Care20151351165118325273677

 PubMed Web of Science ®Google Scholar

NobleSMatzdorffAMaraveyasAHolmMVPisaGAssessing patients’ anticoagulation preferences for the treatment of cancer-associated thrombosis using conjoint methodologyHaematologica2015100111486149226294737

 PubMed Web of Science ®Google Scholar

MazurDJHickamDHMazurMDHow patients’ preferences for risk information influence treatment choice in a case of high risk and high therapeutic uncertainty: asymptomatic localized prostate cancerMed Decis Making199919439439810520677

 PubMed Web of Science ®Google Scholar

MattiAIKeaneMCMcCarlHKlaerPChenCSPatients’ knowledge and perception on optic neuritis management before and after an information sessionBMC Ophthalmol2010101720302669

 PubMed Web of Science ®Google Scholar

MaciverJTibblesABilliaFRossHPatient perceptions of implantable cardioverter-defibrillator deactivation discussions: A qualitative studySAGE Open Med20164205031211664269

 Google Scholar

SanfordDKyleRLazo-LangnerAPatient preferences for stopping tyrosine kinase inhibitors in chronic myeloid leukemiaCurr Oncol2014212241249

 PubMed Web of Science ®Google Scholar

KohHSInYKongCGWonHYKimKHLeeJHFactors affecting patients’ graft choice in anterior cruciate ligament reconstructionClin Orthop Surg201022697520514263

 PubMedGoogle Scholar

HaVMcDonaldSDPregnant women’s preferences for and concerns about preterm birth prevention: a cross-sectional surveyBMC Pregnancy Childbirth20171714928143534

 PubMed Web of Science ®Google Scholar

GareenIFSiewertBVannessDJHermanBJohnsonCDGatsonisCPatient willingness for repeat screening and preference for CT colonography and optical colonoscopy in ACRIN 6664: the National CT Colonography trialPatient Prefer Adherence201592104326229451

 PubMed Web of Science ®Google Scholar

FiksAGMayneSDebartoloEPowerTJGuevaraJPParental preferences and goals regarding ADHD treatmentPediatrics2013132469270223999959

 PubMed Web of Science ®Google Scholar

EckmanMHAlonso-CoelloPGuyattGHWomen’s values and preferences for thromboprophylaxis during pregnancy: a comparison of direct-choice and decision analysis using patient specific utilitiesThromb Res2015136234134726033397

 PubMed Web of Science ®Google Scholar

ChoudhryAHongJChongKPatients’ preferences for biopsy result notification in an era of electronic messaging methodsJAMA Dermatol2015151551325831475

 PubMed Web of Science ®Google Scholar

CalderwoodAHWasanSKHeerenTCSchroyPCPatient and Provider Preferences for Colorectal Cancer Screening: How Does CT Colonography Compare to Other Modalities?Int J Canc Prev20114430733825237287

 PubMedGoogle Scholar

BolgeSCGorenABrownDGinsbergSAllenIOpenness to and preference for attributes of biologic therapy prior to initiation among patients with rheumatoid arthritis: patient and rheumatologist perspectives and implications for decision makingPatient Prefer Adherence2016101079109027390518

 PubMed Web of Science ®Google Scholar

HofmanPLilleøreSKTer-BorchGNeedle with a novel attachment versus conventional screw-thread needles: a preference and ease-of-use test among children and adolescents with diabetesJ Diabetes Sci Technol2011561480148722226269

 PubMedGoogle Scholar