1,061
Views
0
CrossRef citations to date
0
Altmetric
Research Article

A comprehensive view to reflection on the palliative care approach for family medicine residents: A modified Delphi method

, , , , , , & | (Reviewing editor) show all
Article: 1704137 | Received 21 Jul 2019, Accepted 09 Dec 2019, Published online: 22 Dec 2019

Abstract

Effective reflection is essential to allow family medicine residents to cope with difficult and negative experiences during palliative care. We developed a multidisciplinary, consensus-based comprehensive view to reflection on the palliative care approach for family medicine residents. We used a modified Delphi method and a multidisciplinary panel to build a systematic consensus. We developed a prototype comprehensive view based on a literature review, cognitive flexibility theory, and discussion by the research team. The 12 panelists consisted of 8 physicians, 3 nurses, and 1 medical social worker. The final comprehensive view consisted of 11 situations and 6 viewpoints regarding patterns of rapid and intermittent functional decline, respectively, with a total of 508 statements. Six of the 11 situations were common to both illness trajectories, and there were 5 unique situations for each disease trajectory. The 6 viewpoints were physical evaluation, mental/psychological evaluation, social evaluation, spiritual evaluation, cooperation among multidisciplinary professionals, and professional judgment. A comprehensive view to reflection on the palliative care approach was developed for family medicine residents using a modified Delphi method with multidisciplinary input. This comprehensive view may help family medicine residents and faculty become more reflective practitioners through interactive communication.

PUBLIC INTEREST STATEMENT

We developed a multidisciplinary, consensus-based comprehensive guide to reflection on the palliative care approach for family medicine residents. This comprehensive view helps family medicine residents refer to their reflective feedback in a self-directed manner, which assists faculty in communicating interactively with them to help them become reflective practitioners. The comprehensive view consisted of 11 situations and 6 viewpoints regarding patterns of rapid and intermittent functional decline, respectively. The six viewpoints were physical evaluation, mental/psychological evaluation, social evaluation, spiritual evaluation, cooperation among multidisciplinary professionals, and professional judgment. The 11 situations were involved time elements, for example at diagnosis, during treatment or follow-up, and during the period shortly before death.

1. Introduction

It is essential that family physicians receive education on a palliative care approach regarding communication with patients, family members, and other healthcare professionals, decision-making, and goal-setting in accordance with the principles of palliative care. (Radbruch, Citation2009)

Reflection is a metacognitive process that creates a greater understanding of both the self and the situation so that future actions can be informed by this understanding. (K V, Holmes, Hayes, Burge, & Viscount, Citation2001) Family physicians engage both in reflection on action and reflection in action, which explains how professionals meet the challenges of their work by improvisation that improves through practice (Arai et al., Citation2017). Therefore, learning how to apply deep reflection is essential for family medicine residents, with a focus on the illness trajectory and the complexity of the patient from the perspectives of continuity and comprehensiveness. In addition, a recent study demonstrated that it is important to pay attention to the emotional state of residents and to offer them words of affirmation when they are caring for the dying. (American Academy of Family Physicians, Citation2015) Although several countries, including Japan, have recommended training goals for family medicine residents in terms of palliative and end-of-life care (Paes & Wee, Citation2008; Royal College of General Practitioners, Citation2016; The College of Family Physicians of Canada, Citation2016; The Royal Australian College of General Practitioners, Citation2011), there is no material to facilitate the greater understanding of both the self and the situation in terms of palliative care in primary care.

Thus, to improve the quality of reflection during learning in the daily practice of palliative care, a comprehensive view is required to help family medicine residents refer to their reflective feedback in a self-directed manner and to help faculty members assist residents in becoming reflective practitioners through interactive communication.

We aimed to develop a multidisciplinary, consensus-based comprehensive view to facilitate the reflection on the palliative care approach for family medicine residents in Japan, with the goal that this comprehensive view will serve as a metacognitive framework.

2. Materials and methods

A consensus method based on the subjective opinions of several experts is considered to be appropriate for developing one of the metacognitive frameworks on an educational perspective with a clear methodology. (Grauel et al., Citation1996; MacDonald, Mount, Boston, & Scott, Citation1993; Schonwetter & Robinson, Citation1994; Spiro et al., Citation1992; Ury, Arnold, & Tulsky, Citation2002) Using this concept, we developed a comprehensive view of a palliative care approach for family medicine residents by employing a modified Delphi method to build systematic consensus.

We applied cognitive flexibility theory as a theoretical framework to develop the comprehensive view. The strength of this theory is that it focuses on revisiting the same material at different times, in rearranged contexts, and for different purposes; this process leads to different perspectives that are essential for attaining the goals of advanced knowledge acquisition (Sasahara et al., Citation2009). Therefore, cognitive flexibility theory should be useful to support reflection by residents, since it focuses on the nature of learning in complex and ill-structured domains, such as the palliative care approach. We initially shared this strength of the cognitive flexibility theory with research team members and Delphi members by e-mail, video conferences, and face-to-face meetings.

The study protocol was approved by the Institutional Review Board of the University of Tsukuba (No.991, approved on 16 September 2015). Written informed consent was obtained from all participants. Confidentiality of individual responses was ensured by coding the data.

2.1. Research team members

All eight research team members belonged to the Japan Primary Care Association and had expertise in family medicine and palliative medicine, either in research, education, or clinical experience, and were also knowledgeable about the status of family medicine residency programs in Japan.

2.2. Selection of the delphi panelists

Although there are no universally accepted criteria for the selection of Delphi panelists, using panelists from a broad range of disciplines will best represent the variety of viewpoints available (Fukuma et al., Citation2017). To gather a wide range of opinions, research team members discussed the number and types of expertise areas of potential Delphi panelists. Based on a recent study, we concluded that at least 10 panelists would be needed (Kunz et al., Citation2017; Murray et al., Citation2017; Ueda, Ohtera, Kaso, & Nakayama, Citation2017), with expertise in family medicine, palliative medicine, home care, geriatrics, nursing, social work, and medical education. We asked several academic societies in the fields of family medicine, palliative medicine, home care, and geriatrics to recommend representatives as Delphi panelists, and all did so. We chose additional Delphi panelists from the following specialty areas: palliative care nursing, geriatric nursing, social work, and medical education.

2.3. Preparatory phase: research team members develop a prototype comprehensive view for palliative care

The eight research team members performed a literature review using PubMed and subsequently communicated with each other by e-mail and video conferences. We concluded that there was no appropriate prior study or guide to deepen multifaceted reflective learning in palliative care, though we found several key articles that described the nature of the illness trajectory at the end of life. (Beernaert et al., Citation2016; Cavers et al., Citation2012; Kendall et al., Citation2015; Kimbell, Murray, Macpherson, & Boyd, Citation2016; Lloyd, Kendall, Starr, & Murray, Citation2016; Lynn, Citation1986; Mason et al., Citation2016) Research team members shared their opinions with each other by e-mail, video conferences, and face-to-face meetings (during which audio was recorded) from April to September 2017.

2.4. Delphi round 1: evaluation and modification of the prototype comprehensive view by research team members

In the first round, each research team member evaluated the prototype comprehensive view, which consisted of trajectory patterns, situations, viewpoints, and statements, by completing an e-mail survey based on a 4-point scale in October 2017.

We adopted a 4-point scale for the assessment based on Lynn’s determination and quantification of content validity: 1 = very valid; 2 = almost valid; 3 = not quite valid; and 4 = not valid (Polit & Beck, Citation2008). Full consensus was defined as 78% or more of the responders giving a rating of 1 or 2, according to a previous study. (Morita et al., Citation2015) We also asked each research team member to suggest different wording or to propose additional trajectory patterns, situations, viewpoints, and statements.

2.5. Delphi round 2: evaluation of the draft comprehensive view by Delphi panelists via an e-mail survey

We performed a similar e-mail survey among the Delphi panelists in November 2017 to evaluate statements in the draft comprehensive view before conducting a face-to-face meeting.

We asked the Delphi panelists to rate the statements by using the same assessment criteria and definition of full consensus as employed in Delphi round 1. We also asked the panelists to suggest different wording, propose additional trajectory patterns, situations, viewpoints, and statements. Non-responders were sent e-mail reminders. No financial incentives were provided.

2.6. Delphi round 3: development of the comprehensive view by Delphi panelists during a panel meeting

On the basis of the results of Delphi round 2, in December 2017 we conducted a face-to-face panel meeting to develop the comprehensive view from the perspective of panel members’ expertise and the cognitive flexibility theory. All of the e-mail survey response scores and all feedback comments were compiled and distributed anonymously to the Delphi panelists.

2.7. Delphi round 4: evaluation of the comprehensive view by Delphi panelists via an e-mail survey

In December 2017, we conducted an e-mail survey based on a 4-point scale and the same definition of full consensus as in Delphi rounds 1 and 2.

2.8. Delphi round 5: review of the modified comprehensive view by Delphi panelists via e-mail

We modified the comprehensive view on the basis of the results obtained by Delphi round 4. We then asked the Delphi panelists to review the full set of trajectory patterns, situations, viewpoints, and statements in January 2018.

3. Results

The backgrounds of research team members and Delphi panelists are shown in Table . The details of the Delphi process are shown in Figure . A total of 12 multidisciplinary Delphi panelists were selected: 8 physicians (3 females), 3 nurses (3 females), and 1 medical social worker (1 female).

Table 1. Background of research team members and Delphi panelists

Figure 1. The process of Delphi rounds begins with Delphi panelist selection and a preparatory phase by research team members. We had a total of 5 Delphi rounds.

Figure 1. The process of Delphi rounds begins with Delphi panelist selection and a preparatory phase by research team members. We had a total of 5 Delphi rounds.

3.1. Preparatory phase: prototype comprehensive view developed by research team members

Based on a literature review (Beernaert et al., Citation2016; Cavers et al., Citation2012; Kendall et al., Citation2015; Kimbell et al., Citation2016; Lloyd et al., Citation2016; Lynn, Citation1986; Mason et al., Citation2016) and discussion among research team members, we decided that the comprehensive view regarding the characteristics of end-of-life care should be based on different patterns of the illness trajectory, specifically those marked by rapid, intermittent, and gradual functional decline. Since the intermittent and gradual functional decline patterns overlap in many ways, the research team members selected the rapid and intermittent functional decline trajectories to use as patterns in the prototype.

Given the nature of the palliative care approach and the comprehensive perspective of family physicians, it seemed ideal to take into account total pain, self-management, and continuity of context. Thus, we created two axes for each functional decline pattern: the multifaceted viewpoint and the consecutive situation. For each combination of situation and viewpoint, several concrete statements were devised to facilitate reflection.

Research team members created 11 situations and 6 viewpoints for each functional decline pattern in the prototype, including 135 statements about rapid functional decline and 219 statements about intermittent functional decline. Six situations were common to both trajectories, whereas the other five situations differed between the two trajectory patterns. The six viewpoints were common to both patterns. These viewpoints were based on the concept of total pain, which is a principle of palliative care, as well as interprofessional collaborative practice and professionalism, which are essential components of family practice. (Paes & Wee, Citation2008; Royal College of General Practitioners, Citation2016; The College of Family Physicians of Canada, Citation2016; The Royal Australian College of General Practitioners, Citation2011)

3.2. Delphi round 1: evaluation and modification of the prototype comprehensive view by research team members

All eight research team members evaluated the illness trajectory patterns, situations, viewpoints, and statements in the prototype comprehensive view via e-mail. The research team members failed to reach full consensus on one situation and one viewpoint. They did not reach full consensus on 8 of the 135 statements about rapid functional decline and 5 of the 219 statements about intermittent functional decline. Based on the suggestions in the e-mail survey regarding different wording and additional statements, we modified “at the time of death” to “at the diagnosis of death” in the situations, and changed “existential evaluation” to “spiritual evaluation” in the viewpoints. As an example of changes to the statements, we modified “The patient has a place to express the pain of physical care” to “The patient has a place to express the pain of physical care, such as assistance with excretion” in the rapid functional decline trajectory. Finally, the research team members formulated a draft guide that consisted of 135 statements about rapid functional decline and 211 statements about intermittent functional decline, with each statement categorized into 11 situations and 6 viewpoints.

3.3. Delphi round 2: evaluation of the draft comprehensive view by Delphi panelists via an e-mail survey

All 12 Delphi panelists responded to the e-mail survey. We achieved full consensus on 2 trajectory patterns and on 11 situations and 6 viewpoints for each trajectory. However, there was not full consensus on 5 of the 135 statements about rapid functional decline and 28 of the 211 statements about intermittent functional decline. Panelists also suggested several wording changes and proposed additional statements.

3.4. Delphi round 3: modifications of draft comprehensive view by Delphi panelists in a face-to-face panel meeting

Among the 12 panelists, 10 (83.3%) participated in the face-to-face panel meeting. Three research team members were present during the face-to-face panel meeting, though they only facilitated the discussion among the Delphi panelists. As a result of the meeting, statements were added and deleted, or the wording was changed for statements without full consensus. Finally, the modified draft guide consisted of 245 statements about rapid functional decline and 263 statements about intermittent functional decline.

3.5. Delphi round 4: evaluation of the modified comprehensive view by Delphi panelists via an e-mail survey

All 12 Delphi panelists responded to the e-mail survey. There was full consensus on 245 statements about rapid functional decline and 263 statements about intermittent functional decline. Several changes were proposed regarding the wording and combination of statements (Table , A).

3.6. Delphi round 5: review of the re-modified comprehensive view by Delphi panelists

On the basis of the proposals regarding wording changes and combined statements in round 4, the research team members developed the final version. Subsequently, we asked all 12 Delphi members to review the final version, which they all approved. The final version consisted of 2 trajectory patterns, with 11 situations and 6 viewpoints for each trajectory, and 508 statements in total. The summary of the number of statements and the agreement rate are shown in Table , and the 11 situations and 6 viewpoints for each trajectory are shown in Table . The details of the statements about each trajectory are shown in Table ,.

Table 2. Summary of number of statements and agreement rate

Table 3. The 11 situations and 6 viewpoints for each trajectory

4. Discussion

We used the modified Delphi method with multidisciplinary input to develop the first comprehensive view, which was kind of metacognitive framework in primary palliative care, that helps family medicine residents refer to their reflective feedback in a self-directed manner, and that assists faculty in communicating interactively with residents so as to help them become reflective practitioners. This metacognitive framework could improve the quality of reflection in both family medicine residents and faculty while they learn in the daily practice of palliative care.

This comprehensive view is based on cognitive flexibility theory. Since this theory focuses on the nature of learning in complex and ill-structured domains, such as the palliative care approach, it encourages the representation of knowledge acquired from different experiences and case perspectives for later use, and promotes the ability to use different case representations to construct a knowledge ensemble tailored to the needs of the situation (Sasahara et al., Citation2009). Since the thoughts, interpretations, and emotions of residents are largely influenced by their interactions with patients, family members, and members of multidisciplinary teams, the concepts in cognitive flexibility theory are applicable to fostering skills for handling uncertainty, contradictions, and limitations in medicine, which are inherent components of family medicine and palliative care.

In relation to utilizing this comprehensive view in the clinical setting, we considered several important points. First, it is necessary to prepare a safe environment for learners. For example, faculty should confirm whether residents find it difficult to talk about a topic or would rather not do so, and should clarify that residents can stop the conversation.   

In addition, faculty should do their best to avoid being judgmental and overly investigative, and should instead adopt empathic and supportive attitudes with residents. Second, instead of using this comprehensive view as a checklist, faculty should deepen the reflections of residents while appropriately asking questions based on the contents of the metacognitive framework so that residents can reflect about the cases they have experienced. Third, faculty should share the feelings and concerns of residents, and ask questions about emotions and consciousness to fill the context, in order to deepen the reflection of residents based on the concepts of cognitive flexibility theory. If a resident has difficulty with verbalization, the faculty member should make deductions about the state and thinking process of the resident and help the resident summarize their opinions in a consistent manner to support learning. Thus, this comprehensive view could prompt residents to deepen their experience and emotions, as suggested by a recent qualitative study (American Academy of Family Physicians, Citation2015), by establishing an explicit learning opportunity in palliative care. Fourth, this comprehensive view focused on relationships among patients, family members, and multidisciplinary professionals. This is based on the concept of relationalism, which is a unique cultural practice of Asian countries that may motivate an individual to take the most suitable and ideal action when people encounter interpersonal difficulties (Bito et al., Citation2007). In addition, because this comprehensive view includes many statements regarding relationships among patients, family members, and multidisciplinary professionals, it could be suitable for facilitating a group-oriented approach to end-of-life decision-making with the family, which is preferred by Japanese people. (Jünger, Payne, Brine, Radbruch, & Brearley, Citation2017) Fifth, the defining feature of this comprehensive view is that it might clarify patients’ and their families’ stories and practices in two types of palliative trajectories, functioning as a type of consultation theory. The previous consultation theories, such as the Calgary-Cambridge guide, are learning aids to help physicians develop their own consultation skills, focus on patients’ perspectives such as the bio-psycho-social aspects of their condition, and facilitate a patient-centered approach. Therefore, the value of this comprehensive view is that it incorporates a time element into the patient’s palliative phase, enabling family medicine residents to reflect on their own behavior and multi-professional collaboration.

The chief strength of this comprehensive view is that it was developed using the modified Delphi method with multidisciplinary input. A recent qualitative study suggested that attending physicians should establish explicit learning opportunities that allow residents to communicate with multidisciplinary team members, and also respect the autonomy of residents and share responsibility in developing their professional identities in relation to end-of-life care (American Academy of Family Physicians, Citation2015). Thus, we developed this multifactorial, consensus-based and authentic comprehensive view using the multidisciplinary Delphi method.

It is noteworthy that the Delphi panelists emphasized that family medicine residents should collaborate with multidisciplinary health-care providers to assess and manage various situations. This is one reason that the largest number of statements was related to interprofessional collaborative practice. Therefore, this comprehensive view encourages residents to communicate with multidisciplinary team members. In addition, this metacognitive framework could help alleviate the conflict that residents might feel because of their inexperience in coping with a great deal of pressure both as individuals and as physicians, and their confusion due to the highly individual and uncertain nature of the palliative care approach.

This comprehensive view has several limitations. First, the number of Delphi panelists was small compared with other Delphi studies. The reason is that both the reflection and palliative care approaches are not well understood among health-care professionals, except in family medicine. Therefore, we asked only a limited number of academic societies to recommend Delphi panelists.

Second, the comprehensive view was developed for family medicine residents in Japan; thus, it might be difficult to use in other countries with different cultural and religious backgrounds, such as countries with a high proportion of Christians. However, we believe that this comprehensive view is useful in that the value of palliative care was clarified by the Delphi panelists, who consisted of multidisciplinary professionals and educational theoreticians.

Third, this comprehensive view was not reviewed or approved by an external authority, and both processes are recommended in guidance on conducting and reporting Delphi studies (CREDES) in palliative care (Arai et al., Citation2017),0 although our multidisciplinary Delphi panelists were nominated by several academic societies. Thus, we need to obtain review and approval by an external authority in the future.

Fourth, as this was a development preliminary study, we did not verify the feasibility of residents and attending physicians using this comprehensive view, although we suggested several important points regarding its use in the clinical setting. Thus, we have planned a feasibility study to assess whether residents and faculty use the comprehensive view as a metacognitive framework.

Fifth, the discussion time was limited and 2 of the 12 panelists were unable to participate in the face-to-face meetings.

5. Conclusion

A comprehensive view to reflection on palliative care approach was developed for family medicine residents in Japan using the modified Delphi method with multidisciplinary input. This comprehensive view as a metacognitive framework may help family medicine residents and faculty become reflective practitioners through interactive communication.

Conflict Disclosure

The authors have declared no competing financial interests.

Acknowledgements

The authors would like to acknowledge the following Delphi panel members: Dr. Mariko Shuto, Dr. Maiko Kodama, Dr. Ai Oishi, Prof. Katsuya Iijima, Dr. Hideyuki Kashiwagi, Dr. Koki Katou, Dr. Muneyoshi Aomatsu, Dr. Kazuhisa Motomura, Ms. Satsuki Uno, Dr. Keiko Abe, and Prof. Machiko Ohara. And the authors also would like to thank for Dr. Tesshu Kusaba for supporting our study.

Additional information

Funding

This project received funding from Sasakawa Memorial Health Foundation (2017A-7). The funder had no role in the design and conduct of the study; the collection, management, analysis, and interpretation of data; the preparation, review, or approval of the manuscript; or the decision to submit the manuscript for publication.

Notes on contributors

Jun Hamano

Our research team aims to improve the quality of palliative care delivery and palliative care practice in the primary care setting. Research themes of our team include palliative care education in primary care and the construction of a training system for primary care physicians who are interested in palliative care.

The first author’s personal research interests include primary palliative care, prognostication in cancer and non-cancer patients, advance care planning, and home care.

References

  • American Academy of Family Physicians. (2015). Recommended curriculum guidelines for family medicine residents: Palliative and end-of-life care [internet]. Retrieved from https://www.aafp.org/dam/AAFP/documents/medical_education_residency/program_directors/Reprint269_Palliative.pdf
  • Arai, K., Saiki, T., Imafuku, R., Kawakami, C., Fujisaki, K., & Suzuki, Y. (2017). What do Japanese residents learn from treating dying patients? The implications for training in end-of-life care. BMC Medical Education, 17(1), 205. doi:10.1186/s12909-017-1029-6
  • Beernaert, K., Pardon, K., Van den Block, L., Devroey, D., De Laat, M., Geboes, K., … Cohen, J. (2016). Palliative care needs at different phases in the illness trajectory: A survey study in patients with cancer. European Journal of Cancer Care, 25(4), 534–51. doi:10.1111/ecc.2016.25.issue-4
  • Bito, S., Matsumura, S., Singer, M. K., Meredith, L. S., Fukuhara, S., & Wenger, N. S. (2007). Acculturation and end-of-life decision making: Comparison of Japanese and Japanese-American focus groups. Bioethics, 21(5), 251–262. doi:10.1111/biot.2007.21.issue-5
  • Cavers, D., Hacking, B., Erridge, S. E., Kendall, M., Morris, P. G., & Murray, S. A. (2012). Social, psychological and existential well-being in patients with glioma and their caregivers: A qualitative study. Canadian Medical Association Journal, 184(7), E373–82. doi:10.1503/cmaj.111622
  • The College of Family Physicians of Canada. (2016). Specific standards for family medicine residency programs accredited by the college of family physicians of Canada [internet]. Retrieved from https://www.cfpc.ca/uploadedfiles/red%20book%20english.pdf
  • Fukuma, S., Shimizu, S., Niihata, K., Sada, K.-E., Yanagita, M., Hatta, T., … Shibagaki, Y. (2017). Development of quality indicators for care of chronic kidney disease in the primary care setting using electronic health data: A RAND-modified Delphi method. Clinical and Experimental Nephrology, 21(2), 247–256. doi:10.1007/s10157-016-1274-8
  • Grauel, R. R., Eger, R., Finley, R. C., Hawtin, C., Keay, T., O’Brien, W., & Ross, D. D. (1996). Educational program in palliative and hospice care at the University of Maryland School of Medicine. Journal of Cancer Education, 11(3), 144–147. doi:10.1080/08858199609528417
  • Jünger, S., Payne, S. A., Brine, J., Radbruch, L., & Brearley, S. G. (2017). Guidance on conducting and REporting DElphi Studies (CREDES) in palliative care: Recommendations based on a methodological systematic review. Palliative Medicine, 31(8), 684–706. doi:10.1177/0269216317690685
  • K V, M., Holmes, D. B., Hayes, V. M., Burge, F. I., & Viscount, P. W. (2001). Community family medicine teachers’ perceptions of their teaching role. Medical Education, 35(3), 278–285. doi:10.1046/j.1365-2923.2001.00769.x
  • Kendall, M., Carduff, E., Lloyd, A., Kimbell, B., Cavers, D., Buckingham, S., & Murray, S. A. (2015). Different experiences and goals in different advanced diseases: Comparing serial interviews with patients with cancer, organ failure, or frailty and their family and professional carers. Journal of Pain and Symptom Management, 50(2), 216–224. doi:10.1016/j.jpainsymman.2015.02.017
  • Kimbell, B., Murray, S. A., Macpherson, S., & Boyd, K. (2016). Embracing inherent uncertainty in advanced illness. BMJ, 354, i3802. doi:10.1136/bmj.i3802
  • Kunz, D., Pariyadath, M., Wittler, M., Askew, K., Manthey, D., & Hartman, N. (2017). Derivation of a performance checklist for ultrasound-guided arthrocentesis using the modified Delphi method. Journal of Ultrasound in Medicine, 36(6), 1147–1152. doi:10.7863/ultra.16.08004
  • Lloyd, A., Kendall, M., Starr, J. M., & Murray, S. A. (2016). Physical, social, psychological and existential trajectories of loss and adaptation towards the end of life for older people living with frailty: A serial interview study. BMC Geriatrics, 16(1), 176. doi:10.1186/s12877-016-0350-y
  • Lynn, M. R. (1986). Determination and quantification of content validity. Nursing Research, 35(6), 382–386. doi:10.1097/00006199-198611000-00017
  • MacDonald, N., Mount, B., Boston, W., & Scott, J. (1993). The Canadian palliative care undergraduate curriculum. Journal of Cancer Education, 8(3), 197–201. doi:10.1080/08858199309528229
  • Mason, B., Nanton, V., Epiphaniou, E., Murray, S. A., Donaldson, A., Shipman, C., … Boyd, K. (2016). “My body’s falling apart.” Understanding the experiences of patients with advanced multimorbidity to improve care: Serial interviews with patients and carers. BMJ Supportive & Palliative Care, 6(1), 60–65. doi:10.1136/bmjspcare-2013-000639
  • Morita, T., Oyama, Y., Cheng, S.-Y., Suh, S.-Y., Koh, S. J., Kim, H. S., … Tsuneto, S. (2015). Palliative care physicians’ attitudes toward patient autonomy and a good death in East Asian countries. Journal of Pain and Symptom Management, 50(2), 190–199.e1. doi:10.1016/j.jpainsymman.2015.02.020
  • Murray, S. A., Kendall, M., Mitchell, G., Moine, S., Amblàs-Novellas, J., & Boyd, K. (2017). Palliative care from diagnosis to death. BMJ, 356, j878. doi:10.1136/bmj.j878
  • Paes, P., & Wee, B. (2008). A Delphi study to develop the ASSOCIATION FOR PALLIATIVE MEDICINE CONsensus syllabus for undergraduate palliative medicine in Great Britain and Ireland. Palliative Medicine, 22(4), 360–364. doi:10.1177/0269216308090769
  • Polit, D. F.,Beck, C. T. (2008). Developing and testing self-report scale. In Nursing research. eighth (pp. 474–486). Philadelphia: Lippincott Williams & Wilkins.
  • Radbruch, L. P. S. (2009). White Paper on standards and norms for hospice and palliative care in Europe: Part 1. European Journal of Palliative Care, 16(6), 278–289.
  • The Royal Australian College of General Practitioners [internet]. The RACGP Curriculum for Australian General Practice. (2011). Palliative care. Retreived from https://www.racgp.org.au/FSDEDEV/media/documents/Education/Curriculum/2011-Curriculum.pdf
  • Royal College of General Practitioners. (2016). Position statement on palliative and end of life care [internet]. Retrieved from: https://www.rcgp.org.uk/clinical-and-research/resources/a-to-z-clinical-resources/end-of-life-care-resources-for-gps.aspx
  • Sasahara, T., Kizawa, Y., Morita, T., Iwamitsu, Y., Otaki, J., Okamura, H., … Bito, S. (2009). Development of a standard for hospital-based palliative care consultation teams using a modified Delphi method. Journal of Pain and Symptom Management, 38(4), 496–504. doi:10.1016/j.jpainsymman.2009.01.007
  • Schonwetter, R. S., & Robinson, B. E. (1994). Educational objectives for medical training in the care of the terminally ill. Academic Medicine, 69(8), 688–690. doi:10.1097/00001888-199408000-00021
  • Spiro, R. J., Feltovich, P. J., Jacobson, M. J., & Coulson, R. L. (1992). Cognitive flexibility, constructivism, and hypertext: Random access instruction for advanced knowledge acquisition in ill-structured domains. In T. M. Duffy & D. H. Jonassen (Eds.), Constructivism and the technology of instruction: A conversation (pp. 57–75). Lawrence Erlbaum Associates, Inc.
  • Ueda, K., Ohtera, S., Kaso, M., & Nakayama, T. (2017). Development of quality indicators for low-risk labor care provided by midwives using a RAND-modified Delphi method. BMC Pregnancy and Childbirth, 17(1), 315. doi:10.1186/s12884-017-1468-4
  • Ury, W. A., Arnold, R. M., & Tulsky, J. A. (2002). Palliative care curriculum development: A model for a content and process-based approach. Journal of Palliative Medicine, 5(4), 539–548. doi:10.1089/109662102760269779

Table A1. The details of number of statements and agreement rate in rapid functional decline

Table A2. The details of number of statements and agreement rate in intermittent functional decline

Table A3. Comprehensive view for rapid functional decline

Table A4. Comprehensive view for intermittent functional decline