Validation study of an end-of-life questionnaire from the Swedish Register of Palliative Care

Abstract

Quality in health care is important to measure and palliative care is no exception. The Swedish Register of Palliative Care (SRPC) is a national quality register that focuses on the last week of life. It collects data with an end-of-life questionnaire (ELQ), which is validated in this study. Material and methods. This study included 169 consecutive patients who had died at a palliative unit. That unit had developed a computerized end-of-life medical record module that enabled a comparison between reported data and medical records, illustrating the validity of the registry questionnaire. The paper versions of the ELQs filled in at the unit were also inspected to determine the extent of registration mistakes when completing the web questionnaire. Results. Data from the medical records and data from the ELQs reported to the SRPC showed a congruity of 22 to 100%. A working limit of acceptable congruity was set to 80%. Eight questions fell below that line. Some of these differences were caused by systematic errors. The paper versions filled in at the unit and the data from the ELQs reported to the SRPC had a congruity of between 96% and 100%, with the exception of one question about pain evaluation, which had 91% congruity. Discussion. The results in this study will be used to improve and further develop the register. Some questions need to be more specific to promote more valid registrations. Consensus on quality issues in end-of-life care would simplify the work of writing and answering the questionnaire. It is desirable to perform a similar study at hospital wards that do not specialize in palliative care; however, the anticipated lack of palliative documentation could make such a study difficult to perform.

In Sweden, all health care is financed by taxes and organized as a result of democratic decisions. The Swedish Health Care Law requires that the person with the greatest need for health care be given priority [1]. The political ambition is to offer all citizens a dignified and well-operating health care at the end of life regardless of level of care – own home, hospital, or nursing home [2]. The society is now facing the challenge to maintain and improve quality in health care in spite of a growing population of older people with increasing needs and fewer people available in the work force. This situation tends to increase the gap between needs and available resources.

The last three to six months of a person's life often require intensive health care. As approximately 1% of the population dies annually and a clear majority of these persons have palliative care needs [2], end-of-life care is one of the most common tasks in health care and occupies substantial resources. These patients often are unable to speak for themselves and their care needs can be complex. In the case of lack of recourses, these patients can easily be even more marginalized. Thus, monitoring and improving the quality of end-of-life care is essential both for the individual patient and for society.

The best way to measure quality in health care is not completely clear. Today, government agencies in Sweden are investing in 71 national quality registers to compare and improve health care in their specific domain. Data can also be used for local management and control purposes. The Swedish Register of Palliative Care (SRPC) funded in 2005 is a national quality register financed by the Swedish Association of Local Authorities and Regions to evaluate and improve care at the end of life. The register collects data by two Swedish questionnaires; the end-of-life questionnaire (ELQ) is validated in this study. This questionnaire is answered by ward staff as soon as a patient dies at their unit and gives an indication whether or not the recent deceased patient had received different care interventions connected with the main areas in palliative care, symptom alleviation, support to the patient's next of kin, and adequate nursing interventions. (See Appendix 1 for a translated version of the 27 questions included in the ELQ, http://informahealthcare.com/doi/abs/10.3109/0284186X.2011.554434).

The register data of the ELQ covers the provided care content during the last week of life and thus naturally consists of retrospective data. Because prospective data are generally regarded as more accurate, the aim of this study was to compare the registered data with prospective data from medical records of the deceased patients. A secondary aim was to evaluate the extent of registration mistakes. The staff involved in the end-of-life care jointly completed a paper version of the ELQ. Then a ward secretary transferred these answers to a web-based form on the SRPC's home page. Accordingly, this study evaluates two essential steps of the registration process: the congruity of data when comparing medical records with registered data and mistakes when transferring the paper form to the web.

This study is a part of the register's overall validation performed in 2009. This study was approved by the ethics committee at Linköping University, Sweden. Published data from a similar study have not been found.

Material and methods

Initially, 169 consecutive patients who had died and been reported to the register were included in this study. All of these patients belonged either to the specialized palliative home care or to the palliative ward 96 (same staff and management), both at the Linköping University Hospital, Sweden. All these patients died between October 2008 and May 2009. This unit was chosen because of its special way of documenting important aspects of palliative care in a computerized medical end-of-life module included in the patients’ regular medical records. All included patients had a complete end-of-life module, which was filled in by their physician when the patients had been diagnosed to pass beyond the transition point to end-of-life care, i.e. to be dying from their illnesses. Data from this module as well as daily record notes with palliative information and other medical record notes were used to complete the ELQ of these patients. All medical record data was collected by the same person (first author) who had no knowledge about the true ELQ registration submitted to the SRPC. Thirty-five patients were excluded because they had not been recognized as end-of-life patients and accordingly lacked the end-of-life module. For 33 patients, an ELQ in paper form at the ward could not be found; these patients were excluded. One patient had not been reported to the register, although there existed a completed ELQ in paper form at the ward. This patient was also excluded. One hundred patients remained in the study. Data was analyzed in the statistics program Statview. The six questions in the ELQ with purely administrative natures were not analyzed. The rate of identical answers between the ELQ registrations and the ELQs filled in from medical records or paper versions was analyzed with χ²-analysis.

Results

Cancer was reported as a main disease in 93 patients (93%); 53 patients (53%) died in their own home with support from specialized home care and 46 patients (46%) patients died at the palliative ward. The remaining patient (1%) died in a nursing home with support from specialized home care. Many of those patients who died in the palliative ward had also been cared for in their own home with support from specialized home care before admission to the palliative ward and 99 of 100 death cases were reported as expected. One patient had gone through a forensic autopsy according to the ELQ registration, but this was incorrect according to the medical record. If the answer to question number 10 about forensic autopsy was “Yes”, the subsequent questions cannot be answered in the web-based form. Nevertheless, the whole paper version for this patient was answered and the ELQ filled in from medical records was fully answered. This caused answer diversity for this patient from question 11 to 25 in both comparisons below, but to avoid this systematic error this patient was excluded from the calculations for these questions.

Comparison between ELQ registrations and paper versions

Per question, ELQ registrations and paper versions had a congruity of 96–100%, with the exception of question number 16 (about pain evaluation), which had a congruity of 91%. The non-identical answers could be classified into three different categories depending on cause (Figure 1). One category consisted of cases where questions had not been answered in the paper version. Because questions cannot be left empty in the web-based form, the person who transferred the answers to the web had to find out or guess the right answer. In 15 cases, a question was left empty in the paper version. In eight cases, it was possible to choose the answer “Don't know” in the web-based form, which was also done. Another category consisted of the cases where the paper version was answered in a way that could not be answered in the web-based form. For example, in question number 16 although only one alternative could be answered in the web-based form, both one of the two alternatives “Yes” or “No” and the alternative “The patient cannot participate” was chosen in six cases in the paper version. The third category consisted of cases where the answer was changed when transferred to the web-based form even though the paper version was completed in a technically correct way. Therefore, the differences in this category are most probably mistakes, but can also consist of corrections.

Figure 1. Answers that differed between ELQ registrations and corresponding paper versions per question, divided into three categories.

Comparison between ELQ registrations and ELQs filled in based on the medical records

A comparison between ELQ registrations and ELQs filled in based on the medical records showed a very diverse congruity, which varied from 22 to 100% (Figure 2). In this study, a working limit of acceptable congruity was set to 80%. Eight questions fell below that line. The cause of this difference could be divided into three categories (Figure 3). One reason for differences was that the questions sometimes could not be answered at all as medical records did not contain enough information to answer the questions. This could be a false difference. That is, the staff may have known the right answer to the question without it being documented in the medical record. Another reason for differences was found to be different interpretations of the question between the unit and the register; e.g. the unit had a less specific time limit than the register. The third category of different answers was unexplained differences, because no trend in how the answers differed could be found. This could be due to a mix of different interpretations of the questions by different staff members or due to the same person answering the questions in various ways from time to time. The questions with congruities below 80% and more than two given answering alternatives are reported with further details in tables. See Table I and II for questions number 13, 14 and 15.

Figure 2. Congruity between ELQ registrations and ELQs filled in from medical records.

Figure 3. Answers that differed between ELQ registrations and ELQs filled in from medical records per question, divided into three categories.

Table I. Contingency table comparing reported data with data from medical record concerning question number 13 and 14.

	Medical record/Data reported	Yes, by a doctor	Yes, by a nurse	Yes, by both a doctor and a nurse	No	Don't know
Information to patient	Yes, by a doctor	10	0	4	5	0
	Yes, by a nurse	0	0	0	0	0
	Yes, by both a doctor and a nurse	30	0	24	7	1
	No	3	0	2	9	0
	Don't know	2	0	1	1	0
Information to next of kin	Yes, by a doctor	6	0	3	0	0
	Yes, by a nurse	0	2	0	0	0
	Yes, by both a doctor and a nurse	46	1	40	0	0
	No	0	0	0	0	0
	Don't know	0	1	0	0	0

Table II. Contingency table comparing reported data with data from medical record concerning question number 15.

	Medical record/Data reported	Hours	Days	Weeks	Months	Don't know
Time for loss of ability of self-determination	Hours	30	13	0	0	9
	Days	10	22	1	0	5
	Weeks	1	0	2	0	0
	Months	0	0	0	0	0
	Don't know	1	3	0	0	2

Question number 16 (whether VAS/NRS scale had been used for pain evaluation in the last week) showed the second lowest congruity of all questions (29%). The discrepant answers in a majority of cases (n = 48) were due to systematic misinterpretations. These misinterpretations consisted mainly of an under-use of the alternative “The patient cannot participate” (n = 37), but also of cases where the staff had done the evaluation instead of the patient and still answered that pain evaluation had been done (n = 11). Only a low amount of differences could be explained by lack of documentation in the medical records (n = 5). See Table III for further details.

Table III. Contingency table comparing reported data with data from medical record concerning question number 16.

	Medical record/Data reported	Yes	No	Don't know	The patient cannot participate
Pain evaluation with VAS/NRS	Yes	12	11	2	25
	No	8	6	2	12
	Don't know	1	3	0	1
	The patient cannot participate	3	1	1	11

The question with the lowest congruity in this study (22%) is question number 17. This question is about which symptoms were not fully alleviated during the patient's last week. The question has nine different answers that can be combined in different ways. Studied individually, these alternatives had both higher and scattered congruity. The alternative “Shortness of breath” had the highest congruity (82%), while “Pain” had a lower congruity (67%). The alternative “Confusion” had the lowest congruity (54%) of these answering alternatives. Confusion was underreported for the patient group in this study; in fact, only 1% confusion was reported to the SRPC although 28% of the patients had confusion according to the patients’ journals. See Table IV for further details. When collecting data from the medical records, a symptom was only negated if the medical record specified that the patient did not show this symptom.

Table IV. Contingency table comparing reported data with data from medical record concerning question number 17.

	Medical record/ Data reported	Yes	No	Don't know
Shortness of breath	Yes	1	0	0
	No	2	79	4
	Don't know	0	12	1
Confusion	Yes	1	0	0
	No	27	51	7
	Don't know	0	12	1
Nausea	Yes	0	1	0
	No	2	77	6
	Don't know	0	12	1
Death rattle	Yes	3	11	1
	No	4	65	2
	Don't know	0	12	1
Pain	Yes	3	7	0
	No	8	62	6
	Don't know	2	10	1
Anxiety	Yes	1	15	1
	No	3	63	3
	Don't know	2	10	1
Other	Yes	1	2	0
	No	14	67	2
	Don't know	0	12	1

Question number 19 asks whether the patient had pressure ulcers during the last week. The ulcers are then graded from one to four according to the European Pressure Ulcer Advisory Panel (EUPAP). There was often a higher grade of pressure ulcer reported in the medical records than there was in the ELQs. There was also a higher amount of the overall incidence of pressure ulcers in the medical records than was reported to the SRPC. This caused an overall under-reporting to the SRPC of 18%; i.e. a relatively high amount of pressure ulcers were detected but not reported. Some of this could be due to different interpretations of the question, since the question does not specify what to answer in the case of different ulcer grades during the last week.

One other question that had a high proportion of misinterpretations was question number 20 (whether or not drugs were prescribed for use as needed in the form of injections for four different symptoms at least one day before death). Although these drugs were prescribed later than one day (24 h) before death, the unit still had answered positive to the question in 45 cases. Furthermore, in one case the question was answered positive although the drug was prescribed for other indications only.

Discussion

With eight questions below the line of 80% congruity, this study showed that there was a diverse result in the validity of the ELQ. These results will be used to develop the ELQ further. Some questions will be rewritten so that they are more specific. It is also clear that consensus in different palliative areas would simplify the work of writing and answering the questionnaire. Question number 16 about pain evaluation is a good example of the lack of consensus in the palliative area. Does pain evaluation with VAS/NRS mean that the patient actually has to produce the number or is it sufficient for the staff to evaluate the patient's pain? While the SRPC thinks that the patient has to do the evaluation, the unit had often reported that pain evaluation had been done when only the staff had done the pain evaluation; however, according to the literature the pain evaluation done by hospital staff is not equal to the patient's pain evaluation [3]. Although question number 16 already has the disadvantage of lack of consensus in the area, this question is also a good example of a need for a more specific question. Many people answering this question seem to have misunderstood the answering alternative “The patient cannot participate”.

There was shown to be an underreporting of confusion, with only 1% confusion reported to the ELQ while 28 of 100 patients had confusion according to their medical records. The overall register confusion rate for cancer patients is 5%. Is the rate of 1–5% or the rate of 28% more likely to be accurate? One study found a delirium prevalence of 32% in patients with advanced cancer [4], another 47% [5].

Yet another study found that terminal delirium occurred in 88% of deaths in patients with advanced cancer [6]. This indicates that a rate of 1% confusion in terminally ill cancer patients (this study included 93% cancer patients) is highly unlikely. Also, the low number of overall rate of confusion for cancer patients (5%) in the SRPC indicates that this is due to a systematic error caused by the questionnaire.

This systematic error in question number 17 (not fully alleviated symptoms) could be due to a number of plausible reasons. First, the default answer to specific symptoms is “No” due to the unfortunate design of the question where it is possible to leave most of the alternatives without making active selections. It is possible that the alternative of confusion is overshadowed by symptoms that tend to be considered more important, such as pain. Second, the question may be understood as meaning only symptoms that had been possible to alleviate. Since confusion is common in terminally ill patients and often hard to alleviate, it could be considered more of a natural part of the dying process and therefore nothing to report. Third, the question is very complex and takes effort to answer, which could lead to carelessness and trouble interpreting the question. Fourth, there is no way to grade the symptoms. If, for example, a patient had very severe pain and was moderately confused, answering both pain and confusion would create the wrong impression of the patient's situation.

When comparing the congruity of different questions, one should remember that the questions differ much in complexity. Subjects that are difficult to measure are those dealing with quality of life, ability of self-determination, and not fully alleviated symptoms. For those questions, perhaps a lower limit of congruity has to be accepted. As seen in Table I concerning question number 13 and 14 about information to patient and next of kin, the low congruity of these questions were largely caused by lack of documentation of information given by nurses. Figure 4 illustrates the information flow before it reaches SRPC. The grey arrows represent information flows that were controlled in this study. To gain a more complete picture of the provided patient care and the true validity of the used ELQ, the non-studied black arrows also need to be investigated.

Figure 4. Flow of information from the patient to the SRPC.

This study was performed on cancer patients at a unit specializing in palliative care and with a special way of documenting palliative aspects in its medical records. The medical records were selected so that only those with a completed end-of-life module were included. Despite this, question number 19 (pressure ulcers) indicates that data in the medical records are not used when answering the ELQ. Results from this question showed that higher amounts of pressure ulcers and higher grades of pressure ulcers were detected and reported in the medical records than were reported in the SRPC. Although proved significant in this study, the information flow directly between staff and paper version (Figure 4) was probably smaller than usual because of the special palliative data documentation at the unit that participated in this study. Since this study examined how valid the questionnaire is in quite a narrow patient group, it is desirable to perform a similar study at another type of unit. However, at a hospital ward without specialized knowledge about palliative medicine and care and without special ways of documenting palliative aspects, the presumed substantial flow of information that is not documented would possibly make such a study more difficult or even impossible to perform.

Declaration of interest: The authors report no conflicts of interest. The authors alone are responsible for the content and writing of the paper.