Time difference in retrieving clinical information in Patient-overview Prostate Cancer compared to electronic health records

Abstract

Background

Patients with advanced prostate cancer (PCa) typically undergo numerous lines of treatment leading to large amounts of information in Electronic Health Records (EHRs). The Patient-overview Prostate Cancer (PPC) presents clinical information in a graphical overview. The aim of this study was to measure time spent on retrieving clinical information in PPC compared to EHRs, to assess if retrieved data was correct and to explore usability of PPC.

Material and methods

Oncologists, urologists and nurses in three hospitals in Sweden were timed when filling out questionnaires about patients using PPC and two different EHRs; Melior and COSMIC. Time and number of errors were analysed using linear mixed models (LMMs). Usability of PPC was measured with the System Usability Scale.

Results

The LMM showed a significantly shorter time to retrieve information in PPC compared to EHRs. The estimated time to complete one questionnaire was 8 minutes (95% CI = 6–10, p < 0.001) in PPC compared to 25 minutes in Melior and 21 minutes in COSMIC. Compared to PPC, the estimated time difference was 17 minutes longer in Melior (95% CI = 14–20, p < 0.001) and 13 minutes longer in COSMIC (95% CI = 10–17, p < 0.001). The LMM showed significantly fewer errors in PPC compared to Melior. No significant difference in the number of errors was found between PPC and COSMIC. The usability of PPC was rated as excellent by oncologists, urologists and nurses.

Conclusion

A graphical overview of a patient’s medical history, as in PPC, gives health staff rapid access to relevant information with a high degree of usability.

Keywords:

• Patient overview
• electronic health record
• EHR
• prostate cancer
• metastatic
• castration resistant
• decision support
• graphic visualization
• usability

Introduction

Patients with advanced prostate cancer (PCa) typically undergo numerous lines of treatment during several years, leading to large amounts of information collected in Electronic Health Records (EHRs) [1]. Healthcare staff involved in prostate cancer care work in multidisciplinary teams and therefore there is a strong need for easy access to information that facilitates communication and coordination of care [2].

In Electronic Health Records (EHRs), information on a patient’s medical history and vital clinical data is registered and displayed [3]. However, EHRs have been criticized by healthcare staff as cumbersome and that too much time, prior to and during consultation, is spent on searching and finding information about the patient [4,5]. EHRs often consist of multiple user interfaces that do not match the clinical workflow, resulting in non-intuitive entry and presentation of data with a risk of missing critical information that may compromise patient safety [6].

Thus, there is a need for accessible and user-friendly clinical information systems that can collect, collate and present data to facilitate patient–doctor interaction, team communication, support decision-making in cancer care and secure quality assessment and research.

The Patient-overview Prostate Cancer (PPC) a part of the National Prostate Cancer Register (NPCR) [7], was created by an interdisciplinary team involving both physicians and nurses from different hospitals in Sweden together with patient representatives. The aim was to create a user-friendly decision support by collecting longitudinal clinically important information for each patient with advanced PCa presented in an interactive graphical display, trying to overcome gaps of EHRs [8]. The aim of this study was to compare time spent on retrieving clinical information on patients with advanced PCa in PPC and EHRs, to assess if retrieved data was correct and to assess the usability of PPC.

Materials and methods

Electronic healthcare records and Patient-overview Prostate Cancer

Two of the most frequently used electronic healthcare records (EHRs) in Sweden; Melior and COSMIC, were compared to the Patient-overview Prostate Cancer (PPC) in three different hospitals; one academic hospital, Sahlgrenska University Hospital (SU) in Gothenburg, using Melior; and two regional hospitals, Växjö Hospital in Växjö and St Göran Hospital in Stockholm, both using COSMIC. Data in EHRs depends on manual entry or transcription of the treating healthcare worker’s dictation and part of the clinical data, i.e., imaging, laboratory and pathology data, is stored in external programs linked to the EHRs.

The Patient-overview Prostate Cancer (PPC) a part of the National Prostate Cancer Register (NPCR) [7], integrates clinical and laboratory data, imaging, medication and Patient Reported Outcome Measures (PROM), registered from initiation of hormonal treatment until death, and presents them on a timeline in a graphical display (Figure 1). Data in PPC depends on manual data entry both by the treating healthcare worker as well as patients at each outpatient visit and is to be used together with the EHR. PPC is held on the platform ‘Information Network on CAncer care’ (INCA) shared by all 21 healthcare regions in Sweden, which holds data for all clinical cancer registers in Sweden [9]. Data in PPC can therefore be accessed regardless of type of EHR, region or hospital. Caregiver access to both EHR and INCA requires personal authentication and the patient-overview for each individual patient is reached through a link between the EHR and INCA in the same way as other external platforms are reached, i.e., imaging and laboratory data. Data in PPC can be automatically transferred to the national quality register.

Figure 1. Graph in Patient-overview Prostate Cancer (PPC) with longitudinal overview of treatment effects including clinical assessment and Patient Reported Outcome Measures (PROM). Reprinted by permission of Taylor & Francis Ltd, http://www.tandfonline.com, on behalf of Acta Chirurgica Scandinavica Society. ‘Set-up and preliminary results from the Patient-overview Prostate Cancer. Longitudinal registration of treatment of advanced prostate cancer in the National Prostate Cancer Register of Sweden’, Franck Lissbrant et al. [8], Scandinavian Journal of Urology, copyright © Acta Chirurgica Scandinavica Society.

Study participants

Sixteen clinicians (10 oncologists and six urologists) and eight registered nurses were included in the study (Table 1). Both residents and consultants participated. Participants differed in work experience and in familiarity with PPC.

Table 1. Characteristics of study participants; health staff timed while retrieving clinical information about patients and filling out questionnaires.

Health staff	Electronic Health Record
	Melior	COSMIC	Total
	(n = 16)	(n = 8)	n (%)
Hospital
Sahlgrenska	16	0	16 (67)
St Göran	0	2	2 (8)
Växjö	0	6	6 (25)
Age, years
20–30	2	0	2 (8)
31–40	4	2	6 (25)
41–50	5	4	9 (38)
51–60	4	2	6 (25)
61–70	1	0	1 (4)
Experience of PPC, months
≤6	6	7	13 (54)
7–24	3	0	3 (13)
≥25	7	1	8 (33)
Profession
Oncologist	8	2	10 (42)
Urologist	0	6	6 (25)
Urology nurse	8	0	8 (33)
Gender
Woman	10	3	13 (54)
Male	6	5	11 (46)
Working experience, years
≤5	5	0	5 (21)
6–10	4	2	6 (25)
≥11	7	6	13 (54)

PPC, Patient-overview Prostate Cancer.

Patients

Twenty-four patients with advanced PCa were included (Table 2). We chose only to include patients no longer alive to make sure that there was information on all different stages of the disease spectrum, including palliative care. Patients treated with a minimum of two lines of therapy in the metastatic castration resistant phase of PCa and registered in PPC were included.

Table 2. Characteristics of patients. The study included clinical information on patients registered in two of the most commonly used Electronic Health Records (EHRs) in Sweden; Melior and COSMIC.

Patients	Electronic Health Record
	Melior	COSMIC	Total
	(n = 8)	(n = 16)	n (%)
Hospital
Sahlgrenska	8	0	8 (33)
St Göran	0	8	8 (33)
Växjö	0	8	8 (33)
Risk category at diagnosis
Low risk	0	3	3 (13)
Intermediate risk	2	1	3 (13)
High risk/locally advanced	4	4	8 (33)
Metastatic	2	8	10 (42)
Primary treatment
WW/AS	0	2	2 (8)
RT/RP	3	4	7 (29)
Hormonal treatment	5	10	15 (63)
Time, years from diagnosis to death
<3	0	8	8 (33)
4–9	6	3	9 (38)
>10	2	5	7 (29)

WW, watchful waiting; AS, Active surveillance; RT, Radiation therapy; RP, Radical prostatectomy.

Questionnaire

An electronic questionnaire with 15 questions on diagnostics, treatment, and follow-up relevant for patients with advanced PCa was created (Supplementary Table). The questions aimed to address information of clinical importance from as many parts of the disease spectrum as possible. All answers were retrievable in both PPC and the EHRs. The electronic questionnaire was created in Dynareg [10], a data system that provided double authentication verification on log in and enabled participants to be automatically timed from start to completion of each questionnaire.

Measuring time for retrieving data from EHR and PPC

All participants underwent a 15-minute introduction to PPC and Dynareg and were familiar with the EHR at their clinic, Melior and COSMIC, respectively. Participants filled out one questionnaire for each of the eight patients at their clinic, retrieving information either in PPC or the EHR, four patients per system. By randomization, participants were instructed in which order and which system (PPC or EHR) they should use when retrieving information to fill out the questionnaire (see statistics). Time was automatically measured in seconds from the opening of one questionnaire until completion.

Correct answers

Prior to the study, correct answers were extracted from the EHRs at each clinic by the principal investigator (PI) of the study together with local PIs at the different hospitals. Correct answers were compared to the participant’s answers and the number of errors were counted for each questionnaire. Small discrepancies in the answers were accepted focusing on patient safety. For example, when a question asked for month and year of an event, an answer with 1 month less or more was approved as correct.

Time for data entry into PPC

Four randomly selected doctors at SU participating in the study were asked to measure time spent on data entry in PPC after completing the study. They were asked to clock the time they spent on filling in clinical information after four encounters in daily clinical practice for four randomly chosen patients in their out-patient clinic, both first visits and revisits.

Usability

System Usability Scale (SUS) was used to evaluate usability of PPC [11]. SUS consists of 10 statements regarding the usability of a system, with responses ranging from 1 (strongly agree) to 5 (strongly disagree). All participants were asked to rate the usability of PPC by answering the 10 statements after completion of the study.

Statistical methods

To estimate the number of participants and patients included in the study a pilot test was performed with a power analysis to obtain 80% power to detect a time difference of 50% between PPC and EHR. A cyclic design was used to balance the patients between method (i.e. PPC or EHR) and health staff (i.e. oncologist, urologist or nurse). To account for repeated measures by health staff within patients a linear mixed model was used, with system (PPC, Melior and COSMIC) as a fixed effect and health staff and patients as random effects. The linear mixed model was fitted using restricted maximum likelihood. Kenward-Roger’s approximation method was used to calculate p-values and perform F-tests for each fixed factor. The outcomes were the time to fulfil the questionnaire and number of errors. We treated time and number of errors as continuous variables. The mean difference between PPC, Melior and COSMIC with the corresponding 95% confidence intervals were estimated with the linear mixed model.

Usability was analysed according to the SUS model, where the score for each question is converted to a new number, added together and then multiplied by 2.5. The final SUS score is on a scale of 0–100, where <51 is rated as ‘poor or worse’, 51–70 is rated as ‘OK’, 71–84 as ‘good’, 85–92 as ‘excellent’ and >92 as ‘best imaginable [12]. Usability score was calculated and presented for oncologists, urologists and nurses, respectively.

All data management and statistical analyses were conducted using R, version 3.6.1 (Vienna, Austria).

Results

Time for retrieving data

The linear mixed model found that the estimated mean time to complete one questionnaire was 8 minutes in PPC (95% Confidence Interval, CI = 6–10, p < 0.001) compared to 25 minutes in Melior and 21 minutes in COSMIC. The estimated time difference was 17 minutes longer in Melior (95% CI = 14–20, p < 0.001) and 13 minutes longer in COSMIC (95% CI = 10–17, p < 0.001), compared to PPC, respectively. There was less variation between health staff regarding time used to complete one questionnaire when using PPC compared to Melior and COSMIC, respectively, see Figure 2.

Figure 2. Distribution of time to complete questionnaire, including mean time in minutes using Patient-overview Prostate Cancer, PPC (orange) = 8.4 (SD = 3.7); the electronic health care records; COSMIC (blue) = 21.7 (SD = 9.9), and Melior (yellow) = 25.6 (SD = 11.4).

Number of errors

In the linear mixed model, the estimated mean number of errors in one questionnaire in PPC was 1.8 (95% CI = 1.3–2.4, p < 0.001) compared to 3.2 errors in Melior and 2.5 errors in COSMIC. The mean number of errors was significantly lower in PPC compared to Melior. The estimated difference in number of errors was on average 1.4 more (95% CI = 0.6–2.1, p = 0.002) in Melior and on average 0.7 more (95% CI = −0.1 to 1.5, p = 0.088) in COSMIC compared to PPC.

Data entry

Mean time spent on filling in clinical information into PPC after clinical encounters, both first visits and revisits, was on average 2 minutes (standard deviation 1 minute) per patient.

Usability of PPC

The response rate of the SUS was 100%. The SUS score was calculated to a mean value of 89 for oncologists, 85 for urologists and 90 for nurses, corresponding to a usability rated as ‘excellent’ by all participants according to SUS (Supplementary Figure). Most of the participants strongly agreed that the system was easy to learn and use and declared that they would like to use this system frequently (Figure 3).

Figure 3. System Usability Scale questions with results from 24 participants.

Discussion

This randomized trial comparing PPC to two EHRs showed that health staff find information of clinical importance faster and with less variation in time in PPC than in two frequently used EHRs in Sweden; Melior and COSMIC. The information found in PPC was more or equally accurate compared to that in the EHRs and health staff rated the usability of PPC as excellent.

The strengths of this study are that study participants were oncologists, urologists and nurses with varying clinical experience, typical for a multidisciplinary team responsible for advanced prostate cancer care in Sweden. The study was conducted in secondary and tertiary care centres and PPC was compared to two of the most common EHRs in Sweden. By using a cyclic design and a linear mixed model, variation within and between individuals was accounted for. Limitations of this study are that PPC was not compared to all EHRs in Sweden and that it was conducted only in men with prostate cancer.

The large time difference in information retrieval between PPC and EHRs observed in this study may have several explanations. Men with advanced PCa typically undergo several lines of treatments and visit various members in a multidisciplinary team, resulting in large amounts of information. Since information in EHRs is not systematically organized and not seldom found in externally linked programs, it takes time to obtain full understanding of the clinical information. We do not know of any previously published studies comparing time spent on retrieving information between EHRs and graphical decision supports such as PPC. However, when an EHR was implemented at a Danish hospital in 2002, physicians expressed that they had lost the overview in the medical record and that it took appreciably longer time to use the EHR than paper-based records [3]. An early systematic review also showed that the EHRs were more time-consuming than paper-based records [13]. A graphical display of a uniform set of variables presented on a time-line, as in PPC, has in other studies been shown to be an effective and rapid way of communicating information, since all data is presented in one image [14].

Fewer errors were made in information retrieval in PPC compared to Melior, whereas no significant difference was found in comparison to COSMIC. In PPC information is gathered according to a predefined variable list and a template, and therefore data from the same visit is never repeated. In contrast, in EHRs data is entered as free text with the possibility and risk of repetition, thereby risking redundancy, which may be prone to error [15,16].

All participants ranked usability of PPC as excellent according to SUS, despite little user-experience of the system and the need for manual recording. An analysis studying factors associated with high user satisfaction of EHRs suggested that the combination of ease of use and the degree to which the system supports communication, clinical workflow and cognitive clinical reasoning are of great importance [4,17,18]. Health staff and patients were engaged early in the design of PPC, both in selecting relevant data, in the user interface design and in the iterative refinements of PPC according to the needs of users in order to ensure that PPC is effectively integrated into care pathways. All new versions of PPC have gone through acceptance testing and a user manual has been produced based on a generic format. A coordinator, specialized in implementation, supports and facilitates the implementation of PPC in the clinics both by ensuring adequate training for users and by making plans for the introduction and management. The results of the SUS observed in this study confirm that ease of use and well-integrated functions were highly rated, indicating that workflow may be facilitated [19].

Several other clinical ‘disease-oriented’ decision supports and registries have been developed for patients with chronic diseases [20]. Both the Swedish Rheumatology Quality Register (SRQ) and the National Quality Registry for HIV (InfCare HIV) and Hepatitis (InfCare Hepatit) have successfully been implemented into clinical care both in Sweden and other countries [21]. InfCareHIV has for example been instrumental in making Sweden become the first country in the world to reach the WHO target for HIV care, showing that these systems may play an important role in developing care as an aid in patient-centred clinical work, as metrics for quality assurance and quality improvement and for research [22]. Currently, similar patient overviews to PPC have been constructed on a generic platform at INCA for several other cancer types, including renal cell carcinoma, lung cancer, breast cancer, CNS tumours, ovarian cancer and myeloma in a project managed by the Federation of Regional Cancer Centres. During this work much effort has been put into finding and agreeing on generic variables that are important for management of cancer patients and visualization regardless of diagnosis, a prerequisite for enabling automatic transfer of data into the patient overviews. In the future, patient overviews will hopefully be seamlessly integrated with the EHRs, allowing users to rapidly access information of clinical importance.

Conclusion

This randomized study shows that PPC is less time-consuming and visualizes more or equally accurate information than two of the most used EHRs in Sweden with a high degree of usability.

Acknowledgements

This study was made possible by the participation of health staff at Växjö Hospital, St. Göran Hospital and Sahlgrenska University Hospital. We thank Olga Zajc-Hansson, Therese Hallberg, Joakim Örtengren and Christofer Lagerros. We thank the PPC working group: Nina Hageman, Maria Nyberg, Anna Cedvall Gustavsson, Erik Andersson, Hans Joelsson, Magnus Törnblom, Marie Hjälm-Eriksson, Pär Stattin and Ingela Franck Lissbrant. We also thank the National Prostate Cancer Register of Sweden (NPCR) steering group: Pär Stattin (chair), Ingela Franck Lissbrant (deputy chair), Johan Styrke, Camilla Thellenberg Karlsson, Lennart Åström, Eva Johansson, Stefan Carlsson, Marie Hjälm-Eriksson, Magnus Törnblom, Olof Akre, David Robinson, Mats Andén, Ola Bratt, Johan Stranne, Jonas Hugosson, Maria Nyberg, Per Fransson, Fredrik Sandin, Karin Hellström, Hans Joelsson and Gert Malmberg.

Disclosure statement

Charlotte Alverbratt, Hanna Vikman, Marie Hjälm Eriksson, Pär Stattin, Ingela Franck Lissbrant: none declared.

Additional information

Funding

This study was supported by funding from ProLiv Väst. The development of PPC was funded by the Prostate Cancer Patient Federation, The Federation of Regional Cancer Centers, Swelife and Sjöbergsstiftelsen.