548
Views
0
CrossRef citations to date
0
Altmetric
Research Article

Adapting Systems Training for Emotional Predictability and Problem Solving (STEPPS) for older adults with personality disorder: a Dutch Delphi study

ORCID Icon, , , & ORCID Icon
Pages 62-69 | Received 09 May 2023, Accepted 12 Sep 2023, Published online: 02 Oct 2023

ABSTRACT

Objective

Systems Training for Emotional Predictability and Problem Solving (STEPPS) is a group therapy in which patients with personality disorder are taught emotion and behaviour regulation skills. Since older adults regulate their emotions differently than younger adults, age-specific aspects are suggested to be taken into account in the treatment. Objectives of this study were to determine expert consensus on the applicability of STEPPS for older adults and determine which adjustments should possibly be implemented for a better fit for older adults with personality disorder.

Method

A group meeting involving seven experts, with expertise in psychological therapies with older adults, was held. The Delphi method was applied in order to reach consensus on three main statements regarding the applicability of STEPPS in older adults with personality disorder.

Results

Most experts (86%) voted against the statement implying the current STEPPS program is applicable for older adults. All experts voted in favour of the statement that adjustments to the STEPPS program at the content level are required. Consensus (≥66.7%) was reached among the experts that adjustments to the STEPPS program are also required at the process level.

Conclusions

STEPPS is considered to be applicable for older adults with emotion regulation problems and personality disorder but needs adaptations to enhance its feasibility and efficacy in later life.

Key Points

What is already known about this topic:

(1)   Systems Training for Emotional Predictability and Problem Solving (STEPPS) is an evidence-based psychotherapeutic treatment for patients with borderline personality disorder.

(2)   Promising results have been found for STEPPS in older adults.

(3)   Several studies recommend adapting psychotherapeutic treatments for older adults.

What this topic adds:

(1)   Consensus was reached among experts, stating that STEPPS needs adaptations to enhance its feasibility and efficacy in later life.

(2)  Adapting STEPPS for the young old (60–75 years) is recommended as a useful starting point.

(3)   A main recommendation concerned including themes into the STEPPS program that are more relevant for ageing.

Introduction

In ancient Greece, the oracle of Delphi was one of the most visited and most revered sacred places. People came from far and wide to hear the divine prophecies of the Pythia, the high priestess. The oracle was consulted for important political and personal decisions by prominent people. The Pythia sat in a cauldron on a tripod and went into a religious trance. She passed on revelations from the gods, giving the consultants guidance on a subject where they needed advice. Nowadays, in topics lacking empirical data, the scientific method named after Delphi can offer solace. The Delphi technique is a systematic method, in which independent participants with relevant expertise are subjected to questionnaires, in order to reach reliable consensus (Dalkey & Helmer, Citation1963). Since little is known about evidence-based treatment of emotion regulation problems in later life, the Delphi technique might be a useful method in providing consensus-based recommendations on the matter.

There is increasing evidence that psychological therapies are effective in late-life mental disorders, such as depression and generalised anxiety disorder (Cuijpers et al., Citation2014, Citation2020; Gonçalves & Byrne, Citation2012). Several studies also investigated the feasibility and associated effects of psychotherapy for personality disorders (PD) in older adults, such as Systems Training for Emotional Predictability and Problem Solving (STEPPS; Ekiz et al., Citation2022), dialectical behaviour therapy (Lynch et al., Citation2007), and schema therapy (Videler et al., Citation2018). STEPPS has been found feasible in the treatment of PD in later life (Ekiz et al., Citation2022). STEPPS is a 20-week group therapy based on cognitive behavioural therapy principles. During STEPPS, patients are given psychoeducation about borderline personality disorder (BPD) and are taught emotion and behaviour regulation techniques to cope with emotional vulnerability in day-to-day events (Blum et al., Citation2002). The emotion regulation techniques help patients cope with intense emotions and maladaptive cognitions. The behaviour regulation skills help patients manage their daily life in such a way that patients can build healthy lifestyle patterns and prevent emotion regulation problems to arise.

STEPPS focuses on PD-related emotional dysregulation, as it is an important underlying factor of BPD (Putnam & Silk, Citation2005). Most studies investigating the efficacy of STEPPS have been conducted in younger adults with a mean age of 33 years (Ekiz et al., Citation2023). Black et al. (Citation2018) reported that the response to the STEPPS program did not differ by age. However, in this study, age was dichotomised, comparing patients younger than 40 years old (N = 60) with patients of 40 years and older (N = 15). Importantly, the oldest participating patient in this study was 50 years of age (Black et al., Citation2013). Research by Hofer et al. (Citation2015) suggests that older adults (i.e., ≥60 years) regulate their emotions differently compared to younger adults. For instance, older adults are more likely to use suppression as an emotion regulation strategy. A study concerning 169 patients diagnosed with BPD found that younger patients (16–25 years) reported higher levels of aggression and suicide attempts, whereas older patients (>45 years) reported higher levels of somatisation, symptoms of depression and anxiety, and more functional deficits (Frías et al., Citation2017). Beatson et al. (Citation2016) confirm that the phenomenology of BPD in older adults differs from younger adults, complicating the (timely) diagnosis in older adults. The authors state that PD-related emotional dysregulation persists into old age, with older adults with BPD demonstrating other forms of self-harm (e.g., drug abuse, non-compliance with therapeutic advice). These results suggest that emotion regulation problems in patients with BPD might have different manifestations in different age groups and therefore might need a different therapeutic approach or emphasis in its treatment.

A common recommendation in studies in older adults seems to be that psychological treatments for older adults need adaptations (e.g., Knight, Citation2004; Lynch, Citation2000; Videler et al., Citation2017). In a Delphi study involving older adults with PD, consensus was reached among 35 experts about the importance of integrating gerontological aspects as themes in psychotherapy with older persons (Van Alphen et al., Citation2012). These gerontological aspects include beliefs about and consequences of somatic disorders, as well as cohort-related and sociocultural beliefs. Similar consensus on adaptations in psychotherapy for older adults with PD emerged in a cross-validation study in the United States (Rosowsky et al., Citation2018). Nonetheless, empirical data on adapted psychological treatments for older adults with PD are scarce. As pointed out by Konrat et al. (Citation2012), older adults are often not included in scientific research, as age limits of 18–65 are generally used. In the psychiatric research field, not only age but also exclusion criteria such as substance abuse, suicide risk, comorbidity, and use of medication are frequently seen (Kennedy-Martin et al., Citation2015). This largely prevents older adults’ participation in research, a situation that is disproportionate to the rapidly growing group of older adults in society. Although research on older adults is challenging because of the interactions among biological, psychological, and social factors, it is also necessary if society is to provide for the specific needs of this population.

Since there are indications that emotion regulation strategies and their manifestation differ during the life span, it has been suggested that the STEPPS program needs adaptations to better serve older adults with PD (Ekiz et al., Citation2022). The aim of the present study was (1) to determine the level of expert consensus on the applicability of the present STEPPS program for older adults with PD (≥60 years), and (2) consensus on which adjustments should possibly be implemented in the program for a better fit for older adults.

Method

In this study, the Delphi technique was applied. This method is relevant to explore topics that lack empirical data. Some key principles of the Delphi technique have been described in the literature, including anonymity of the expert panel, multiple rounds of responses by the panel, controlled feedback between the rounds, and a statistical summary of the group consensus (Rowe & Wright, Citation1999). However, the application of the Delphi technique is extremely varied and heterogeneous; the technique comes in different guises (Powell, Citation2003). Following the method as used by Legra et al. (Citation2017), we performed a Delphi study involving a group meeting instead of distributing questionnaires (Yousuf, Citation2007). In this group meeting, experts in the field were asked to express their opinions and offer suggestions for possible adjustments to the current STEPPS program. Consensus was defined as a minimum of 66.7% voters in favour or against the statement. We chose a group meeting, and thus a non-anonymous Delphi method, since there is, to the best of our knowledge, a limited number of experts on the subject. An advantage of this approach is that rounds of responses by the panel and feedback by the chair are more fluent and iterative. Another advantage of the Delphi method is that a group of experts is guided to a final decision (McKenna, Citation1994). Iteration seems a crucial principle for this process. The iteration process of the Delphi method allows generating a high quantity and quality of creative insights and ideas (Donohoe & Needham, Citation2009) and ensures reliability and generalisability of outcomes (Day & Bobeva, Citation2005). Since one of the aims of the current study was to make STEPPS more suitable for older adults, it was important to create an optimal setting for new and creative ideas to arise.

Selection of the chairman and experts

The selection of the chairman was carefully considered by the research team. The established criteria were an experienced clinical worker with (1) expertise in STEPPS and (2) a certain distance from the field of psychotherapy with older adults, to ensure an independent position. We invited Horusta Frije, clinical psychologist, who was largely responsible for the introduction and dissemination of STEPPS in the Netherlands since 1998, and is experienced in psychotherapy in younger adults with PD.

Experts were defined as follows: (1) possessing a minimum of 5 years of experience in psychological therapies with older adults with PD and (2) having affinity and/or experience with the STEPPS program in the Netherlands. This led to 10 identified potential experts who were invited by email to participate. Seven (70%) potential experts agreed to participate, two (20%) decided not to participate due to a lack of time, and one (10%) agreed to participate but later withdrew due to health issues. Most experts (85.7%) were experienced mental health professionals at a Clinical Centre of Excellence for Personality Disorders in Older Adults. The seven participants included three psychologists (43%), one nurse practitioner (14%), and three nurses (43%) working in geriatric psychiatry. All experts (100%) were Dutch.

All subjects were provided oral and written information concerning the study. Subjects’ informed consents were documented. The study was approved by research ethics committee of GGz Breburg, Tilburg, the Netherlands (reference number CWO 2020-03).

Procedure

The structure and content of the Delphi meeting were determined by the research team and the chairman in several pre-meetings. Three statements, one for each round, were established in these meetings. The statements served as a starting point for each round of discussion in the face-to-face Delphi meeting. The statements were as follows:

  1. The current STEPPS program is sufficiently applicable to older adults in mental health settings.

  2. Adjustments to the STEPPS program at the content level are required.

  3. Adjustments to the STEPPS program at the process level are required.

For the Delphi meeting, no preparation was required for the experts. Each statement was presented on a PowerPoint slide and briefly introduced by the chairman, after which the discussion started. During the discussion, the chairman encouraged the experts to engage in open dialogue and to think creatively and unconventionally. At the end of each round, the chairman provided the panel a short summary of the discussed topics. Furthermore, voting for the statement (yes/no) took place in an open, non-anonymised manner. The meeting lasted 3 hours in total, with a break of 20 minutes after the completion of the voting regarding the second statement. Forty minutes were reserved for the introduction, voting round, and discussion of each starting-point statement. The meeting was video-recorded for qualitative analysis afterwards.

Results

The participation rate was high, with all seven experts (100%) joining in discussion in all three rounds. Discussion of the statements proceeded in a natural manner, which had the effect that the content of the discussion was sometimes about another proposition that had yet to come or had been discussed before. At the end, full consensus was reached on the second statement, and sufficient consensus was reached on the first and third statement (see also ).

Table 1. Overview of agreement among the Delphi experts per statement.

Statement 1: the current STEPPS program is sufficiently applicable for older adults in mental health settings

Six of the seven experts (86%) considered the statement to be false, implying that the current STEPPS program is not sufficiently applicable to older adults with PD. The discussion then revealed several concerns about particular parts of the program being ill-suited for the target group, i.e., older adults with PD. First, the examples, pictures, and movie clips used in the current program do not always apply to the lived experience of older adults. Second, the themes included in the current STEPPS program (e.g., work and intimate relationships), are not fully suitable for older adults, and more relevant themes (i.e., experiences of loss and mourning, physical limitations, polypharmacy defined as using five or more medications, and finding meaning in life) are being missed. Moreover, some existing themes are relevant, but experts stated these themes focus mainly on younger adults with PD. For example, the manifestation of destructive behaviour as described in the current program is more overt (e.g., “attacking boyfriend with a vase and smashing it on his head”), whereas the manifestation of destructive behaviour among older adults with PD is being described as more hidden and ego-syntonic (e.g., neglect of self-care). Additionally, older adults may find their social networks have become quite small, and they may encounter difficulty in marshalling a reinforcement group, which is part of the STEPPS program. Third, in group therapy with older adults, discussion material often includes reminiscing memories of “the old days”. This is not explicitly included in the current STEPPS program. Five experts (72%) stated this should be added for older adults, to provide them the opportunity for life review and to help them achieve closure. Lastly, it was noted that there are differences between the younger cohort of older adults with PD (60–75 years) and the older cohort of older adults with PD (75 years and above). Each group might need other adaptations to the program.

Statement 2: adjustments to the STEPPS program at the content level are required

Regarding this statement, consensus was reached with all seven experts (100%) stating that they agreed with this statement. First, the experts suggested emphasising positive psychology and positive interventions (e.g., mentioning adaptive cognitive schemas next to maladaptive ones) throughout the program (100%). Second, some used terminology, like BPD, was considered to be stigmatising. Experts suggested referring to “emotion regulation problems” better reflects the common core problem participants with different PDs deal with (86%). Third, all experts agreed with the suggestion that including an abstract at the end of every session in the manual would be of significant value. Fourth, experts stated that the transition from the second component of the treatment program (i.e., emotion regulation techniques) to the third component (i.e., behaviour regulation techniques) is too abrupt for older adults (100%). Also, the third component entails different life domains. Experts stated that certain changes in life, typical for later life (e.g., acquiring new social roles, experiences of loss and mourning), remain underexposed (86%).

Consensus was not tested with regard to the following suggestions, which were offered for consideration only. The experts suggested adding a discussion around values in life. Furthermore, experts recommended rewriting existing topics to be more appropriate for the target population. An example is a session related to healthy eating habits, which for older adults should also focus on eating sufficiently and choosing a diet that keeps blood-sugar levels within healthy ranges for diabetic older adults. Finally, an additional session regarding self-destructive behaviours typical for older adults with PD was recommended.

Statement 3: adjustments to the STEPPS program at the process level are required

Although partial consensus was reached regarded this statement, points made by the experts largely concerned content level rather than process level. First, the experts’ opinions varied on the topic whether to add one or more follow-up or booster sessions within a year after the last session. Consensus was not reached (43%) regarding this topic. Differences in personal experiences with follow-up sessions formed the basis for the variation in opinions. Second, full consensus was reached regarding expanding the focus on early maladaptive schemas, part of the emotion regulation skill ‘challenging thoughts’, from one to two group sessions. Experts stated that the pace of the sessions regarding this emotion regulation skill was too high for older adults. Third, all experts agreed that the original protocol should be rewritten into shorter sentences to increase readability. Fourth, all experts agreed the sessions should follow a recurring structure, with several adjustments (e.g., two relaxation exercises, one at the start and one after the break, instead of one relaxation exercise). Lastly, there was no agreement (43%) on increasing the font size of the STEPSS workbook.

Discussion

The aim of this study was to examine possible adaptations to the STEPPS program, to increase its applicability to older adults with poor emotion regulation strategies and PD. Given the current lack of knowledge about STEPPS in older adults, we chose an exploratory technique through a face-to-face meeting of a group of experts, following the Delphi method. The main outcome was that experts suggested adapting the STEPSS protocol for older adults with PD. Main recommendations concerned including themes that are more relevant with ageing into the program (such as physical limitations, life review, polypharmacy, experiences of loss and mourning), including an abstract at the end of every session in the manual, adding extra group sessions to the program to generate more time for several high-paced themes, using short sentences to increase readability, and using a recurring session structure. Furthermore, experts recommended that examples, pictures, and movie clips used in the sessions should better fit the lived experience of older adults. Experts also suggested that referring to “emotion regulation problems” better reflects the common core problem older aged participants with different PDs participating in STEPPS deal with. As Chanen (Citation2021) has stated, it is important to understand and recognise the heterogeneity and uniqueness of BPD in different people. This is even more relevant for BPD in later life, as biopsychosocial ageing processes can lead to a different manifestation of BPD symptoms (Van Alphen et al., Citation2012), and as stigma and therapeutic nihilism are even more prevalent towards older people with BPD (Khasho et al., Citation2023). Finally, in accordance with recent attention on integrating positive schemas, such as success and emotional fulfilment, in schema therapy with older adults (Van Donzel et al., Citation2021; Videler et al., Citation2020), the expert panel suggested focusing more on positive aspects of life by adding those to the protocol. These findings are in line with earlier studies suggesting that psychological treatments for older adults need adaptations (e.g., Knight, Citation2004; Lynch, Citation2000; Videler et al., Citation2017). As for STEPPS, this may be due to the findings that older adults regulate their emotions differently than younger adults (Hofer et al., Citation2015) and that personality pathology manifests differently in older adults compared to younger adults (Van Alphen et al., Citation2012).

Not all themes included in the original protocol were considered applicable to both the third and fourth phases of life. The third phase refers to the ages 60 to 75, the years following one’s working life, in which the older person is still actively involved in many pursuits (Wahl & Ehni, Citation2020). In this phase, the loss of positions and social roles predominates, necessitating the acquisition of new roles. In the fourth phase of life, at approximately 80 years and older, frailty and deterioration in various areas become more central (Wahl & Ehni, Citation2020). In this phase, loss is a central concept, and it can refer to the loss of health and loved ones; as a result, these older adults find themselves facing impending death, which often induces existential anxiety. The distinction between the “young old” and the “older old” also emerged as an important consideration among the experts. This raises the question: for which age group of older adults with PD should the STEPPS protocol be adjusted – the young old or the older old? Based on our clinical experience in which STEPPS has been provided to adults between the ages of 56 to 72 (Ekiz et al., Citation2022), we suggest adapting the protocol for older adults in the third phase of life as a useful starting point.

There are several strengths and limitations to this study. This study is a first step into integrating an adjusted STEPPS into geriatric psychiatry. In order to achieve this, experts were brought together to determine the level of consensus on what these adjustments should be for a better fit of STEPPS for older adults with PD. The Delphi technique, which is a widely used systematic method, was used to close a scientific gap. A strength of this study is that several authors are experienced in conducting Delphi studies in scientifically unexplored areas of geriatric psychiatry (e.g., Heijnen-Kohl et al., Citation2022; Legra et al., Citation2017; Rosowsky et al., Citation2018; Van Alphen et al., Citation2012, Citation2003). Another strength is that the iterative character of the Delphi technique, as used in the current study, allowed for mutual discussion and the emergence of creative ideas. However, a consensus method such as the Delphi technique can be characterised as gathering limited scientific evidence and having a low level of reliability (Yousuf, Citation2007). When considering the levels of evidence for therapeutic studies, expert opinion can be seen as Level V (i.e., lowest level) evidence (Burns et al., Citation2011). Furthermore, although the face-to-face Delphi method has some advantages, such as enhancing creativity and encouraging an open discussion, there are also constraints. Possible risks are mutual influence, dominant individuals, and social pressure (Legra et al., Citation2017). In the evaluation and the qualitative analysis of the video-recording afterwards, we concluded that these risks were minimal regarding the experts, but it may have been difficult for the chairman to take a fully independent position, prompted by her broader interest in improving the existing protocol for more than only older adults. Another limitation is that the expert panel consisted of a small number of experts and that all experts were Dutch. The latter may have cultural implications, reducing the generalisability of the findings to an international context. Although the small number of experts is representative of the current (Dutch) research and clinical practice on STEPPS for older adults, results should be interpreted with caution. Potential blind spots of the expert panel could be overcome by having a broader representation of (international) experts. Lastly, the experts were all clinicians, meaning that the patient perspective was not included in this study, and possible useful input from this important group was missed.

The overall conclusions are that STEPPS is considered to be applicable to older adults with PD-related emotion regulation problems, but it needs adaptations to enhance its feasibility and, possibly, its efficacy in later life. A first step in determining which adaptations are useful has been taken by means of this study. To make the expert opinion results more robust, a cross-validation Delphi study is recommended. The STEPPS protocol has been translated into various languages, including German, Italian, and Spanish, implying the use of STEPPS in multiple countries (Black & Blum, Citation2017). It is recommended to include as many countries as possible in which STEPPS is applied in the cross-validation study, in which professionals who meet the same criteria as used in the present Delphi study are invited to participate. Furthermore, involving the patient’s perspective through qualitative research is advocated. An adapted protocol that meets the needs of older adults with PD and that better fits their lived experience could be tested first in a mixed-method design, examining its effects on, for example, emotion regulation, personality functioning, and psychological distress. A final step would be a randomised controlled trial of a further adapted STEPPS protocol, fine-tuned for older adults with PD, to assess its efficacy.

Ethical statement

The authors have abided by the ethical principles of the Declaration of Helsinki. All subjects were provided oral and written information concerning the study. Subjects’ informed consents were documented. The study was approved by research ethics committee of GGz Breburg, Tilburg, the Netherlands (reference number CWO 2020–03).

Acknowledgments

We would like to thank Horusta Freije and the participating experts for offering their valuable time and input in the study.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Data availability statement

The data that support the findings of this study are available from the corresponding author upon reasonable request.

Additional information

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

References

  • Beatson, J., Broadbear, J. H., Sivakumaran, H., George, K., Kotler, E., Moss, F., & Rao, S. (2016). Missed diagnosis: The emerging crisis of borderline personality disorder in older people. Australian and New Zealand Journal of Psychiatry, 50(12), 1139–1145. https://doi.org/10.1177/0004867416640100
  • Black, D. W., Blum, N., & Allen, J. (2018). Does response to the STEPPS program differ by sex, age, or race in offenders with borderline personality disorder? Comprehensive Psychiatry, 87, 134–137. https://doi.org/10.1016/j.comppsych.2018.10.005
  • Black, D. W., Blum, N., McCormick, B., & Allen, J. (2013). Systems training for emotional predictability and problem solving (STEPPS) group treatment for offenders with borderline personality disorder. Journal of Nervous and Mental Disease, 201(2), 124–129. https://doi.org/10.1097/nmd.0b013e31827f6435
  • Black, D. W. & Blum, N. S. (2017). Systems training for emotional predictability and problem solving for borderline personality disorder: Implementing STEPPS around the globe. Oxford University Press. https://doi.org/10.1093/med:psych/9780199384426.001.0001
  • Blum, N., Pfohl, B., St. John, D., Monahan, P., & Black, D. W. (2002). STEPPS: A cognitive-behavioral systems based group treatment for outpatients with borderline personality disorder – A preliminary report. Comprehensive Psychiatry, 43(4), 301–310. https://doi.org/10.1053/comp.2002.33497
  • Burns, P., Rohrich, R. J., & Chung, K. C. (2011). The levels of evidence and their role in evidence-based medicine. Plastic and Reconstructive Surgery, 128(1), 305–310. https://doi.org/10.1097/prs.0b013e318219c171
  • Chanen, A. M. (2021). Bigotry and borderline personality disorder. Australasian Psychiatry, 29(6), 579–580. https://doi.org/10.1177/10398562211045151
  • Cuijpers, P., Karyotaki, E., Eckshtain, D., Ng, M. Y., Corteselli, K. A., Noma, H., Quero, S., & Weisz, J. R. (2020). Psychotherapy for depression across different age groups: A systematic review and meta-analysis. JAMA Psychiatry, 77(7), 694–702. https://doi.org/10.1001/jamapsychiatry.2020.0164
  • Cuijpers, P., Karyotaki, E., Pot, A. M., Park, M., & Reynolds, C. F. (2014). Managing depression in older age: Psychological interventions. Maturitas, 79(2), 160–169. https://doi.org/10.1016/j.maturitas.2014.05.027
  • Dalkey, N. & Helmer, O. (1963). An experimental application of the Delphi method to the use of experts. Management Science, 9(3), 458–467. https://doi.org/10.1287/mnsc.9.3.458
  • Day, J. E. & Bobeva, M. (2005). A generic toolkit for the successful management of Delphi studies. Electronic Journal of Business Research Methods, 3(2), 103–116.
  • Donohoe, H. M. & Needham, R. D. (2009). Moving best practice forward: Delphi characteristics, advantages, potential problems, and solutions. International Journal of Tourism Research, 11(5), 415–437. https://doi.org/10.1002/jtr.709
  • Ekiz, E., Van Alphen, S. P. J., Ouwens, M. A., Van De Paar, J., & Videler, A. C. (2023). Systems training for emotional predictability and problem solving for borderline personality disorder: A systematic review. Personality and Mental Health, 17(1), 20–39. https://doi.org/10.1002/pmh.1558
  • Ekiz, E., Videler, A. C., Ouwens, M. A., & Van Alphen, S. P. J. (2022). Systems training for emotional predictability and problem solving in older adults with personality disorders: A pilot study. Behavioural and Cognitive Psychotherapy, 51(1), 105–109. https://doi.org/10.1017/s1352465822000443
  • Frías, Á., Palma, C., Solves, L., Martínez, B. O. H., & Salvador, A. (2017). Differential symptomatology and functioning in borderline personality disorder across age groups. Psychiatry Research-Neuroimaging, 258, 44–50. https://doi.org/10.1016/j.psychres.2017.09.081
  • Gonçalves, D. D. & Byrne, G. J. (2012). Interventions for generalized anxiety disorder in older adults: Systematic review and meta-analysis. Journal of Anxiety Disorders, 26(1), 1–11. https://doi.org/10.1016/j.janxdis.2011.08.010
  • Heijnen-Kohl, S., Hitzert, B., Schmidt, R., Geurts, H. M., & Van Alphen, S. (2022). Features and needs of autistic older adults: A Delphi study of clinical experiences. Clinical Gerontologist, 1–11. https://doi.org/10.1080/07317115.2022.2060157
  • Hofer, M. D., Burkhard, L., & Allemand, M. (2015). Age differences in emotion regulation during a distressing film scene. Journal of Media Psychology, 27(2), 47–52. https://doi.org/10.1027/1864-1105/a000134
  • Kennedy-Martin, T., Curtis, S., Faries, D. E., Robinson, S., & Johnston, J. A. (2015). A literature review on the representativeness of randomized controlled trial samples and implications for the external validity of trial results. Trials, 16(1). https://doi.org/10.1186/s13063-015-1023-4
  • Khasho, D. A., van Alphen, S. P. J., Ouwens, M. A., Arntz, A., Heijnen‐Kohl, S. M., & Videler, A. C. (2023). The effectiveness of individual schema therapy in older adults with borderline personality disorder: A multiple‐baseline case series design. Clinical Psychology & Psychotherapy. Advance online publication. https://doi.org/10.1002/cpp.2900
  • Knight, B. G. (2004). Psychotherapy with older adults. Sage Publications. https://doi.org/10.4135/9781452204574
  • Konrat, C., Boutron, I., Trinquart, L., Auleley, G., Ricordeau, P., & Ravaud, P. (2012). Underrepresentation of elderly people in randomised controlled trials. The example of trials of 4 widely prescribed drugs. PLOS ONE, 7(3), e33559. https://doi.org/10.1371/journal.pone.0033559
  • Legra, M. J. H., Verhey, F. R., & Van Alphen, S. (2017). A first step toward integrating schema theory in geriatric psychiatry: A Delphi study. International Psychogeriatrics, 29(7), 1069–1076. https://doi.org/10.1017/s1041610217000412
  • Lynch, T. J. (2000). Treatment of elderly depression with personality disorder comorbidity using dialectical behavior therapy. Cognitive and Behavioral Practice, 7(4), 468–477. https://doi.org/10.1016/s1077-7229(00)80058-9
  • Lynch, T. J., Cheavens, J. S., Cukrowicz, K. C., Thorp, S. R., Bronner, L., & Beyer, J. L. (2007). Treatment of older adults with co-morbid personality disorder and depression: A dialectical behavior therapy approach. International Journal of Geriatric Psychiatry, 22(2), 131–143. https://doi.org/10.1002/gps.1703
  • McKenna, H. (1994). The Delphi technique: A worthwhile research approach for nursing? Journal of Advanced Nursing, 19(6), 1221–1225. https://doi.org/10.1111/j.1365-2648.1994.tb01207.x
  • Powell, C. E. (2003). The Delphi technique: Myths and realities. Journal of Advanced Nursing, 41(4), 376–382. https://doi.org/10.1046/j.1365-2648.2003.02537.x
  • Putnam, K. M. & Silk, K. R. (2005). Emotion dysregulation and the development of borderline personality disorder. Development and Psychopathology, 17(4), 899–925. https://doi.org/10.1017/s0954579405050431
  • Rosowsky, E., Young, A. S., Malloy, M. J., Van Alphen, S., & Ellison, J. M. (2018). A cross-validation Delphi method approach to the diagnosis and treatment of personality disorders in older adults. Aging & Mental Health, 22(3), 371–378. https://doi.org/10.1080/13607863.2016.1261796
  • Rowe, G. & Wright, G. E. (1999). The Delphi technique as a forecasting tool: Issues and analysis. International Journal of Forecasting, 15(4), 353–375. https://doi.org/10.1016/s0169-2070(99)00018-7
  • Van Alphen, S., Bolwerk, N., Videler, A. C., Tummers, J., Van Royen, R., Barendse, H., Verheul, R., & Rosowsky, E. (2012). Age-related aspects and clinical implications of diagnosis and treatment of personality disorders in older adults. Clinical Gerontologist, 35(1), 27–41. https://doi.org/10.1080/07317115.2011.628368
  • Van Alphen, S. P. J., Engelen, G. J. J. A., Kuin, Y., van der Staak, C. P. F., & Derksen, J. J. L. (2003). Een Delphi-onderzoek naar persoonlijkheidsdiagnostiek in de (A)GGZ bij ouderen [Personality diagnostics in the elderly psychiatric patients by a Delphi technique]. Tijdschrift voor Gerontologie en Geriatrie, 34(5), 208–214.
  • Van Donzel, L., Ouwens, M. A., Van Alphen, S., Bouwmeester, S., & Videler, A. C. (2021). The effectiveness of adapted schema therapy for cluster C personality disorders in older adults – Integrating positive schemas. Contemporary Clinical Trials Communications, 21, 100715. https://doi.org/10.1016/j.conctc.2021.100715
  • Videler, A. C., Van Alphen, S. P. J., Van Royen, R. J. J., Van Der Feltz-Cornelis, C. M., Rossi, G., & Arntz, A. (2018). Schema therapy for personality disorders in older adults: A multiple-baseline study. Aging & Mental Health, 22(6), 738–747. https://doi.org/10.1080/13607863.2017.1318260
  • Videler, A. C., Van Royen, R., Heijnen-Kohl, S., Rossi, G., Van Alphen, S., & Van Der Feltz-Cornelis, C. (2017). Adapting schema therapy for personality disorders in older adults. International Journal of Cognitive Therapy, 10(1), 62–78. https://doi.org/10.1521/ijct.2017.10.1.62
  • Videler, A. C., Van Royen, R. J. J., Legra, M. J. H., & Ouwens, M. A. (2020). Positive schemas in schema therapy with older adults: Clinical implications and research suggestions. Behavioural and Cognitive Psychotherapy, 48(4), 481–491. https://doi.org/10.1017/s1352465820000077
  • Wahl, H. & Ehni, H. (2020). Advanced old age as a developmental dilemma: An in-depth comparison of established fourth age conceptualizations. Journal of Aging Studies, 55, 100896. https://doi.org/10.1016/j.jaging.2020.100896
  • Yousuf, M. I. (2007). Using experts’ opinions through Delphi technique. Practical Assessment, Research & Evaluation, 12(4), 1–8. https://doi.org/10.7275/rrph-t210