The Patient-Reported Experience Measure for Improving qUality of care in Mental health (PREMIUM) project in France: study protocol for the development and implementation strategy

Figures & data

Figure 1 Study flow chart.

Abbreviation: CAT, computerized adaptive testing.

Table 1 Selection criteria

Inclusion criteria
Phase 1 (item bank and CAT development)	Men or women, aged over 18 and under 65 years
	Subjects cared for in one of the investigator centers of this study
	Cared for in psychiatry; diagnosis of schizophrenia, bipolar disorder or depression according to DSM V, regardless of current or previous therapies, duration, and severity of illness
	Subjects who have no comprehension disorders and are able to read and write, agree to participate in the study, and have signed informed consent
Phase 2 (qualitative study)	Any patient or health system stakeholder from an investigator health care establishment or an associated supervisory organization: professionals involved in the care of patients (doctors, nurses, and psychologists), department heads, health executives, members of the hospital administration, and public institutions
Exclusion criteria
	Aged less than 18 or greater than 65 years
	Subject not cared for in psychiatry or untreated in one of the investigative centers, without a diagnosis of schizophrenia, bipolar disorder, or depression according to DSM V
	Subject unlikely to participate in an interview or complete a self-administered questionnaire
	Subject refusing to participate in the study or to sign informed consent
	Vulnerable persons (pregnant women, parturient or nursing mothers, persons deprived of liberty, persons admitted to a health or social institution for other purposes than research, and adults who are subject to a legal protection measure)
	Subjects with decompensated organic disease or mental retardation
	Subject withdrawing his/her consent before the end of the study

Abbreviations: CAT, computerized adaptive testing; DSM V, Diagnostic and Statistical Manual of Mental Disorders, fifth edition.

Figure 2 CAT algorithm.

Abbreviation: CAT, computerized adaptive testing.

Table 2 Collected data to assess the quality of mental health care

Measure	Instruments	Number of items and format	Number of dimensions	Description	Steps of Phase 1*
Sociodemographic characteristics	NA	NA	NA	Gender, age, educational level, marital status, work situation	1, 2, 3, and 4
Clinical data	NA	NA	NA	Disease duration, psychiatric diagnosis	1, 2, 3, and 4
Severity	Clinical Global Impression- Severity (CGI-S)^Citation123,Citation124	1 item, 7-point Likert	1	Clinician’s assessment of current severity of the patient’s symptoms	4
General functioning	General Assessment Functioning (GAF)^Citation125	Numeric scale (ranged from 1 to 100)	1	Clinician’s assessment of the individual’s overall level of functioning (social, occupational, and psychological)	4
Satisfaction	Client Satisfaction Questionnaire^Citation126	8 items, 4-point Likert	1	Client’s assessment of satisfaction with mental health services	4
Therapeutic alliance	4-point ordinal Alliance Self-report^Citation127	11 items, 4-point Likert	2	Assessment of the quality of the therapeutic relationship	4
Medication adherence	Medication Adherence Rating Scale^Citation128	10 items, Yes/No	3	Assessment of the patient’s medication adherence	4
Quality of life	The Short-Form (36) Health Survey^Citation129	36 items	8	Assessment of a person’s self-report quality of life	4

Notes:

* Phase 1 includes 4 steps as described in the “Study procedure” section. All 4 steps were considered for some measures, whilst step 4 only was considered for other measures. Abbreviation: NA, not applicable.

Table 3 Sample size

	Sample size	Justification
Phase 1
Step 1	45 face-to-face semi- structured interviews	In the absence of basic guidelines, the concept of saturation^Citation130 is considered the guiding principle of qualitative research to determine sample size.^Citation131 We chose to include 45 subjects in accordance with the recommendations of Ritchie et al^Citation132 that an optimal sample should contain less than 50 subjects and that beyond 60 inclusions data collection does not provide additional information necessary for understanding the purpose of the study.
Step 2	A minimum of 45 face-to- face directive interviews
Step 3	600 patients	There is no consensus on sample size requirements for estimating the parameters of an IRT model.^Citation133,Citation134 However, some guidelines have been formulated. In accordance with those formulated by Tsutakawa and Johnson,^Citation135 a sample higher than 500 individuals is suitable to obtain sufficiently accurate estimates when a multi-parameter model is used.^Citation136
Step 4	600 patients	There is a lack of formal guidelines on how to calculate a priori sample size for psychometric validation studies.^Citation137 The minimum size depends on the methods considered. To obtain a heterogeneous sample covering the spectrum of possible responses, we chose to include 600 topics as in the previous step (1.3).
Phase 2	A minimum of 45 patients and 45 health stakeholders	The number of subjects to be included in this second phase was determined according to the recommendations based on the saturation concept used in steps 1.1 and 1.2.

GuyWClinical Global Impressions ECDEU Assessment Manual for PsychopharmacologyRockville, MDUS Department of Health, Education, and Welfare Public Health Service Alcohol, Drug Abuse, and Mental Health Administration1976

 Google Scholar

BerkMNgFDoddSThe validity of the CGI severity and improvement scales as measures of clinical effectiveness suitable for routine clinical useJ Eval Clin Pract200814697998318462279

 PubMed Web of Science ®Google Scholar

EndicottJSpitzerRFleissJCohenJThe global assessment scale. A procedure for measuring overall severity of psychiatric disturbanceArch Gen Psychiatry1976336766771938196

 PubMed Web of Science ®Google Scholar

AttkissonCCZwickRThe client satisfaction questionnaire. Psychometric properties and correlations with service utilization and psychotherapy outcomeEval Program Plann19825323323710259963

 PubMedGoogle Scholar

MisdrahiDVerdouxHLançonCBayleFThe 4-Point ordinal Alliance Self-report: a self-report questionnaire for assessing therapeutic relationships in routine mental healthCompr Psychiatry200950218118519216896

 PubMed Web of Science ®Google Scholar

FialkoLGaretyPAKuipersEA large-scale validation study of the Medication Adherence Rating Scale (MARS)Schizophr Res20081001–3535918083007

 PubMed Web of Science ®Google Scholar

LeplegeAEcosseEVerdierAPernegerTVThe French SF-36 Health Survey: translation, cultural adaptation and preliminary psychometric evaluationJ Clin Epidemiol19985111101310239817119

 PubMed Web of Science ®Google Scholar

MorseJMThe significance of saturationQual Health Res199552147149

 Web of Science ®Google Scholar

MasonMSample size and saturation in PhD studies using qualitative interviewsForum Qual Soc Res2010113Art 8

 Google Scholar

RitchieJLewisJElamGDesigning and selecting samplesRitchieJLewisJQualitative Research Practice: A Guide for Social Science Students and ResearchersThousand Oaks, CASage Publications200377108

 Google Scholar

EdelenMOReeveBBApplying item response theory (IRT) modeling to questionnaire development, evaluation, and refinementQual Life Res200716S151817375372

 PubMed Web of Science ®Google Scholar

NguyenTHHanHRKimMTChanKSAn introduction to item response theory for patient-reported outcome measurementPatient201471233524403095

 PubMedGoogle Scholar

TsutakawaRKJohnsonJCThe effect of uncertainty of item parameter estimation on ability estimatesPsychometrika1990552371390

 Web of Science ®Google Scholar

CappelleriJCJason LundyJHaysRDOverview of classical test theory and item response theory for the quantitative assessment of items in developing patient-reported outcomes measuresClin Ther201436564866224811753

 PubMed Web of Science ®Google Scholar

AnthoineEMoretLRegnaultASébilleVHardouinJBSample size used to validate a scale: a review of publications on newly-developed patient reported outcomes measuresHealth Qual Life Outcomes201412117625492701

 PubMedGoogle Scholar

Data availability

The datasets used and/or analyzed during the current study are available from the corresponding author on reasonable request.