Developing a Preliminary Conceptual Framework for Guidelines on Inclusion of Patient Reported-Outcome Measures (PROMs) in Clinical Quality Registries

Figures & data

Figure 1 Flow chart in PRISMA consort diagram of the screening and selection of studies.

Table 1 Characteristics of Included Studies (N=10) on Capturing Patient-Reported Outcomes Measures in Clinical Registries

Study Details	Rationale	Setting	Ethics	Instrument(s)	Administration		DATA MANAGEMENT			STATISTICAL METHOD	Reporting
Author (Year) Country Aims	Rationale for collecting PROMs	Population Sample Size (N)	Patient Consent Confidentiality	New/Existing Generic/Specific	Frequency	Mode & Methods	Entry & Quality Check	Storage	Data Completeness	Data Analysis	Dissemination
Individual registry studies
Ashley et al (2011)^Citation30 UK, cancer registry To describe technical development of the electronic PROMs from cancer survivors system, for linking PROMs with clinical data in cancer registries	To improve understanding of cancer survivors’ experiences and outcomes to inform development and targeted provision of support services and interventions.	Non-metastatic breast, colorectal and prostate cancer patients NS	Written informed consent	Existing Both	≤6, 9, and 15 months post-diagnosis	Online	Via unique ID and password to log in and complete PROMs	Within the registry	Extent of missing responses recorded in the electronic system	Summary statistics	NS
Azad et al (2016)^Citation27 USA, spine surgery registry To detail the methodology of constructing an integrated electronic health record system to collect QoL metrics and demonstrate the effect of data collection on routine clinical workflow.	To determine value of surgical care	All new patients after spine surgery 791	NS	Existing Both	3, 6, and 12 months post-operative	Online, integrated electronic health record (EHR) system	Electronic entry	Within the registry	98%	NS	Via clinician-facing dashboard
Helsten et al (2016)^Citation28 USA, unselected surgical patient registry To describe methods of Services Following Yearlong Surgical Outcomes Surveys (SATISFY-SOS) registry, which combines perioperative PROMs with granular electronic data to track patients’ postoperative trajectories.	To track patient recovery after surgery and to answer how the presence or absence of reported postoperative complications might impact QoL and functional recovery after surgery	All patients attending assessment center 44,081	Consent	Existing Both	Baseline (enrolment), 30 days, and 1-year postoperatively	Online, mail, or telephone. If a patient did not respond to initial e-mail, reminders sent on day 2 and 4 after the first contact	Via unique ID	Aggregate data stored on a dedicated research database server	30-day survey (62% response rate), 1-year survey (71%)	NS	NS
Moloczij et al (2018)^Citation32 Australia, lung cancer registry To describe the design and structure of a study to develop and pilot test a PRO framework to systematically and longitudinally collect PRO data from a cohort of lung cancer patients at a comprehensive cancer centre.	To provide important information that have potential to translate to meaningful service improvement	Patients with lung cancer selected for the pilot 30–60	Opt-in	Existing Disease-specific	Baseline, 2, 6, and 12 months	Paper, online	Manually entered and checked	REDCap/Microsoft Access database	To be analyzed	Summary statistics	NS
Morris et al (2016)^Citation31 UK, spinal osteopathy service registry To report the feasibility of implementing routine registry data collection and sustaining a high rate of data capture within a secondary care spinal osteopathy service, using the Spine Tango Conservative registry data collection tool with multiple PROMs.	To provide important indicators of treatment effect	All patients 349	Consent	Existing Both	Baseline, end of 1st treatment cycle, beginning of 2nd treatment cycle, discharge	Clinic. Paper	By data entry/analysis staff	Kept in spreadsheets and patients notes	55−90%	Summary statistics	NS
Rana (2016)^Citation29 USA, joint arthroplasty registry To describe one hospital’s experience in building and maintaining an electronic PROM database for a practice of 6 board-certified orthopedic surgeons.	To improve delivery of health care, including quality of care.	Primary and revision hip, knee, and shoulder arthroplasties >4500	Informed consent to allow use of data in future research	Existing Both	Preoperative (baseline), postoperative 6 weeks, 3 months, 1, 2 and 5 years	Electronic tablet + email	Database	Electronic PROM database	Baseline (91% response rate), post 6 weeks (60%), 12 months (70%)	Summary statistics	Patients; and surgeons: graphically present patients’ PROM data at their follow-up office visit.
Reviews, Guidelines and Recommendations
Breckenridge et al (2015)^Citation24 Europe, renal registries To describe the main points learned from a consensus meeting on how to routinely collect PROMs in renal registries in Europe.	PROMs: to highlight relevant symptoms/changes in symptoms, promote patient engagement in treatment, improve patient outcomes, inform patients’ choice of treatment, and assess quality of care across different hospitals. PREMs: to improve quality in clinical services.	All patients on renal replacement therapy N/A	Consider ethics/consent/data protection	Existing Both	At least annually; preferably not during dialysis	Unassisted self-report; no clear preference for paper/web-based	NS	NS	NS	NS	NS
Rolfson et al (2016)^Citation25 International, joint arthroplasty registries To describe the Working Group recommendations for selection, administration, and analysis of PROMs by arthroplasty registries.	To provide important information that matters to patients.	All – for quality improvement a sample – for research purposes	Informed consent	Existing Both	Pre- and post- joint replacement as determined	Paper-and electronic based. In clinic	Entered in an electronic form	Registry	Missing values should be reported per item	Propensity scores, regression, risk adjustment	Clinicians
Franklin et al (2013)^Citation33 [Review] International, total joint arthroplasty registries To review current PRO use by existing national registries, challenges to integrating PROs in national registries, lessons from national registries that have integrated PROs, and suggestions to guide future adoption of PROs.	To quantify optimal outcomes from total joint arthroplasty procedures as defined by the patient, as a measure of positive outcomes (eg, pain relief and improved function) and as a marker of risk for negative outcomes (eg, persistent pain or higher risk for implant failure).	All total joint arthroplasty patients or a representative subset of patients if 100% registry coverage	Consent	Existing Both	Pre-arranged times Advantages of direct-to-patient model independent of clinical visits	Electronic, computer-adapted measures are preferable.	Data infrastructure and data collection procedures must support a complete and accurate database.	Data infrastructure and data collection procedures must support a complete and accurate database.	Data infrastructure and data collection procedures must support a complete and accurate database.	NS	NS
Franklin et al (2017)^Citation20 International, orthopedic registries To describe framework and steps that were identified by stakeholders as best practices to guide PROM capture and use in the arthroplasty registries	1. Patients and Clinicians/Individual patient care decisions: Individual patient-centered decisions to prioritize, treat, and monitor disease symptoms and health status. 2. Hospital Leaders and Clinicians/Quality improvement: Monitor and improve aggregate patient outcomes as compared to national best practice and benchmarks 3. Insurers and Hospital Leaders/Value-based payment: Measure outcomes as compared to costs and utilization to optimize health care value 4. Researchers, Policy makers, and Funders/Population health and research: Generate new evidence for best clinical practices across patients to achieve optimal health status over time.	Clear definition of patient population is critical as the disease, symptoms, or procedure will inform the frequency and duration of PROM collection.	Consent	Existing Both Recommended checklist for PROM selection: content, patient acceptance, cost/licensing, ease of clinic integration, and external considerations	Consistent time intervals when patients are likely to achieve peak outcomes Advantages of direct-to-patient model independent of post-TJR clinical visits	Paper, web portals (incl. HER portals), tablets, personal computers, mobile phones, automated voice surveys, and scannable paper forms	PROM collection infrastructure must be flexible to meet demands of varied patients and diverse settings; depending on use of data, available resources and technology	Stored PROMs should include both the item level responses and the summary scores	Completeness of PROM collection is most likely if data collection is embedded in standard clinic workflow, supported by electronic means, and serves patient and clinician decisions	Scoring, risk-adjustment and clear visual displays	Appropriate clinician alerts; data aggregated at intervals to compare risk factors and outcomes; integration of clinical data and PROMs

Abbreviations: QoL, quality of life; PROMs, patient-reported outcome measures; PROs, patient-reported outcomes, NS, not stated.

Table 2 Survey Responses for the Registries Who Collect PROMs in Australia (N=19)

Survey Question	N (%)
Registry Information
Type of the registry
Condition/disease	11 (57.9)
Procedure	6 (31.6)
Drug/device	2 (10.5)
Population coverage
<20%	8 (42.1)
20–39%	2 (10.5)
40–59%	1 (5.3)
60–79%	4 (21.1)
>80%	4 (20.1)
Mean (SD) years registries operating	8.9 (6.2)
Mean (SD) years registries collecting PROMs	7.4 (6.3)
PROMs COLLECTION
Rationale^a
Quality of care and safety improvement	13 (68.4)
Monitoring outcomes of care	13 (68.4)
Population health studies	6 (31.6)
Measuring burden of disease	6 (31.6)
Shared decision making and patient-centered care	4 (21.1)
Other (clinical trials, measuring quality indicators)	6 (31.6)
Consent model
Opt-out model – part of the registry	14 (73.7)
Opt-in (written-informed) – separate from the registry	3 (15.8)
Opt-in (written-informed) – part of the registry	2 (10.5)
Instrument type^a
Generic	13 (68.4)
Disease/condition specific	13 (68.4)
Performance/process measures (patient reported experience measures)	3 (5.8)
How the instrument were chosen?^a
Practical/clinical utility	14 (73.7)
Reliability and validity of the instrument	12 (63.2)
Based on global standards	9 (47.4)
Availability and cost	8 (42.1)
Length of time required to complete	8 (42.1)
On recommendations of others	5 (26.4)
Style of the instrument	1 (5.3)
Developed a new instrument	1 (5.3)
Patient population
All	15 (78.9)
Specific	4 (21.1)
How routinely PROMs are captured?^a
At multiple occasions	15 (78.9)
At baseline	12 (63.2)
At single time point	4 (21.1)
Mode of administration^a
Postal	11 (57.9)
Telephone call	10 (52.6)
On paper, given to the patient (e.g., in clinic)	6 (31.6)
Via email	4 (21.1)
SMS with/without a link to an email	1 (5.3)
How is the PROMs data collected?
Directly from electronic patient responses	9 (47.4)
By dedicated data collectors	7 (36.8)
By clinicians	3 (15.8)
Where is the PROMs data stored?
As a part of the main registry	15 (78.9)
On a stand-alone platform	2 (10.5)
Other	2 (10.5)
Completeness at baseline
<50%	5 (26.3)
>70%	5 (26.3)
Not described	9 (47.3)
Completeness at other time points
<50%	2 (10.5)
50–70%	2 (10.5)
70–90%	6 (31.6)
>90%	3 (15.8)
No known	6 (31.6)
Missing data handling
Proportion of missing data is unknown	9 (42.1)
Exclude missing data from the analysis	7 (36.9)
Use imputation models to replace missing data	3 (15.8)
Major challenges related to PROMs collection^a
Cost of data collection	10 (52.6)
Data completeness issues	8 (42.1)
Timing of administration	7 (36.8)
Integration with other outcome measures	4 (21.1)
Consent issues	2 (10.5)
Other	3 (15.8)
REPORTING AND IMPACT OF PROMs
How PROMs are reported?^a
In aggregated form	15 (78.9)
Benchmarking reports (between sites)	8 (42.1)
Others –a for publications, or yet to report	6 (31.8)
Used for comparisons	2 (10.5)
At individual level	1 (5.3)
To whom PROMs are reported?^a
Conferences and forums	10 (52.6)
Health services	9 (47.4)
Peer-reviewed publication and journals	8 (42.1)
Individual clinicians	6 (31.6)
Funders and industry	6 (31.6)
Government departments	5 (26.3)
Publicly available annual reports	1 (5.3)
Have not been decided	5 (26.3)
Available to clinicians	6 (31.6)
Has made an impact	2 (10.6)

Note: ^aMultiple response options were possible.

Figure 2 Conceptual framework for patient-reported outcome inclusion in clinical registries.

Ashley L, Jones H, Thomas J, et al. Integrating patient reported outcomes with clinical cancer registry data: a feasibility study of the electronic Patient-Reported Outcomes From Cancer Survivors (ePOCS) system. J Med Internet Res. 2013;15(10):e230. doi:10.2196/jmir.276424161667

 PubMed Web of Science ®Google Scholar

Azad TD, Kalani M, Wolf T, et al. Building an electronic health record integrated quality of life outcomes registry for spine surgery. J Neurosurg Spine. 2016;24(1):176–185. doi:10.3171/2015.3.SPINE14112726431073

 PubMed Web of Science ®Google Scholar

Helsten DL, Ben Abdallah A, Avidan MS, et al. Methodologic considerations for collecting patient-reported outcomes from unselected surgical patients. Anesthesiology. 2016;125(3):495–504. doi:10.1097/ALN.000000000000121727355128

 PubMed Web of Science ®Google Scholar

Moloczij N, Gough K, Solomon B, et al. Development of a hospital-based patient-reported outcome framework for lung cancer patients: a study protocol. Health Qual Life Outcomes. 2018;16(1):10. doi:10.1186/s12955-017-0837-z29325571

 PubMed Web of Science ®Google Scholar

Morris S, Booth J, Hegarty J. Spine Tango registry data collection in a conservative spinal service: a feasibility study. Eur Spine j. 2016;25(9):2984–2992. doi:10.1007/s00586-016-4697-827439415

 PubMed Web of Science ®Google Scholar

Rana AJ. Building a patient-reported outcome metric database: one hospital’s experience. J Arthroplasty. 2016;31(6):1151–1154. doi:10.1016/j.arth.2016.03.01727215139

 PubMed Web of Science ®Google Scholar

Breckenridge K, Bekker HL, Gibbons E, et al. How to routinely collect data on patient-reported outcome and experience measures in renal registries in Europe: an expert consensus meeting. Nephrol Dialysis Transplant. 2015;30(10):1605–1614. doi:10.1093/ndt/gfv209

 Web of Science ®Google Scholar

Rolfson O, Bohm E, Franklin P, et al. Patient-reported outcome measures in arthroplasty registries Report of the Patient-Reported Outcome Measures Working Group of the International Society of Arthroplasty Registries Part II. Recommendations for selection, administration, and analysis. Acta Orthop. 2016;87(Suppl 1):9–23. doi:10.1080/17453674.2016.118181627228230

 PubMed Web of Science ®Google Scholar

Franklin PD, Harrold L, Ayers DC. Incorporating patient-reported outcomes in total joint arthroplasty registries: challenges and opportunities. Clin Orthop Relat Res. 2013;471(11):3482–3488. doi:10.1007/s11999-013-3193-223897504

 PubMed Web of Science ®Google Scholar

Franklin P, Chenok K, Lavalee D, et al. Framework to guide the collection and use of patient-reported outcome measures in the learning healthcare system. EGEMS. 2017;5(1):17.29881737

 PubMedGoogle Scholar