Assessing the relative importance of correlates of loneliness in later life. Gaining insight using recursive partitioning

ABSTRACT

Objectives: Improving the design and targeting of interventions is important for alleviating loneliness among older adults. This requires identifying which correlates are the most important predictors of loneliness. This study demonstrates the use of recursive partitioning in exploring the characteristics and assessing the relative importance of correlates of loneliness in older adults.

Method: Using exploratory regression trees and random forests, we examined combinations and the relative importance of 42 correlates in relation to loneliness at age 68 among 2453 participants from the birth cohort study the MRC National Survey of Health and Development.

Results: Positive mental well-being, personal mastery, identifying the spouse as the closest confidant, being extrovert and informal social contact were the most important correlates of lower loneliness levels. Participation in organised groups and demographic correlates were poor identifiers of loneliness. The regression tree suggested that loneliness was not raised among those with poor mental wellbeing if they identified their partner as closest confidante and had frequent social contact.

Conclusion: Recursive partitioning can identify which combinations of experiences and circumstances characterise high-risk groups. Poor mental wellbeing and sparse social contact emerged as especially important and classical demographic factors as insufficient in identifying high loneliness levels among older adults.

KEYWORDS:

• Loneliness
• psycho-social interventions
• recursive partitioning
• random forest
• regression trees

Introduction

Late-life loneliness is a harmful state with serious physical and mental consequences (Cacioppo et al., 2010; Christiansen, Larsen, & Lasgaard, 2016; Patterson & Veenstra 2010; Shiovitz-Ezra & Ayalon 2010; Wilson et al., 2007). Due to the many adverse outcomes of loneliness, an important part of ensuring a healthy, happy and worthwhile old age is knowing how to alleviate and prevent loneliness (De Jong Gierveld, 1998; Hawkley, 2015; Wenger, Davies, Shahtahmasebi, & Scott, 2008). However, identifying appropriate groups to whom interventions should be targeted and the design of the intervention is an area of much discussion (Dickens, Richards, Greaves, & Campbell, 2011). Many correlates of loneliness have been identified but there is little consensus as to which are the most important (Cacioppo et al., 2002; De Jong Gierveld, 1998; Victor, Scambler, Bowling, & Bond, 2005). A more comprehensive understanding of the combination of characteristics of lonely older adults may enable better opportunities to identify high risk groups and create effective interventions (Cacioppo & Hawkley, 2009).

The aim of this paper is to clarify the characteristics and experiences that are related to older adults being lonely. We aim to answer the following questions: Which combinations of characteristics and experiences identify high loneliness levels and what is their relative importance? We further assess the possibility of identifying high risk groups using demographic predictors that are readily available for commissioners and service providers.

Assessing the relative importance of loneliness correlates

Most studies focus on correlates of loneliness separately using traditional regression techniques (Cicchetti & Rogosch, 1996; Dahlberg & McKee, 2014). However, many correlates are of a reciprocal nature and have complicated interrelations with loneliness (De Jong Gierveld, 1998). This makes the use of traditional regression techniques difficult due to the risk of multicollinearity, overfitting and the need to explicitly include interaction terms (James et al., 2014; Victor & Scharf, 2005). Considering the number of identified correlates in the literature, such a model would be practically unfeasible both in terms of specification and interpretation.

Recursive partitioning techniques such as regression trees and random forests can handle the challenges mentioned above. They can uncover and investigate complex dependencies among a large set of predictor variables (Strobl, Boulesteix, Kneib, Augustin, & Zeileis, 2008). Regression trees divide the study population into smaller sub-populations in terms of loneliness levels based on the included correlates. This process makes it possible to create rules such as if an individual x is z, t and s they are most likely to have loneliness level y. In the absence of actual information on loneliness such an approach might be able to inform the design of interventions by identifying subgroups of the population with disproportionate high loneliness levels (Kuchibhatla & Fillenbaum, 2003). The random forest technique ranks the included correlates according to their ability to identify loneliness levels across many separate regression trees (Scott, Jackson, & Bergeman, 2011; Strobl, Malley, & Tutz, 2009). That way, recursive partitioning makes it possible to pinpoint those correlates that best identify differing loneliness levels reflecting both their main effects and their interactions (Strobl et al., 2009). Recursive partitioning has previously been used to identify markers of dementia (Kuchibhatla & Fillenbaum, 2003) and stress (Scott et al., 2011) and pathways to positive and negative affect (Gruenewald et al., 2008) in later life. A more detailed explanation of the two techniques is provided in the methods-section.

Loneliness and its correlates

Most commonly conceptualized as the discrepancy between the degree of objective social isolation and the perceived social needs, loneliness is a subjective feeling referring to the perceived inadequacy of one's social relationships (Cacioppo & Patrick, 2008; De Jong Gierveld, 1987, 1998; Hawkley, 2015; Hawkley & Cacioppo, 2010). Weiss (1973) further distinguished social and emotional loneliness. The former refers to a lack of social network and the latter refers to an absence of fulfilling intimate relationships. Several theoretical models of loneliness had been proposed to explain why loneliness occurs. De Jong-Gierveld's (1987) theoretical model of loneliness describes how personality characteristics, objective and subjective measures of social relations, affective states and demographic characteristics affect the possibility of becoming lonely with Victor, Scambler, Bond, and Bowling (2000) further adding ill health as a possible precursor to loneliness. In the empirical literature, personality characteristics associated with higher degrees of loneliness are social anxiety, introversion (Cheng & Furnham, 2002), pessimism, low self-esteem (Hawkley, 2015) and low levels of mastery (Nicolaisen & Thorsen 2012). Golden et al. (2009) found that loneliness was independently associated with affective states such as poor wellbeing, depressed mood, and hopelessness among community-dwelling elderly. Perlman and Peplau (1981) further suggest that loneliness may interfere with the effective channelling of one's energies to complete social tasks through an oversensitivity to minimal cues and a tendency to misinterpret or to exaggerate the hostile or affectionate intent of others. Demographic characteristics associated with greater loneliness include being unmarried (Hawkley, 2015), and having lower education (Victor et al., 2005) or income (Savikko, Routasalo, Tilvis, Strandbergc, & Pitkälä, 2005). The evidence on the association between gender and loneliness is mixed both in regard to whether gender matters and for which gender loneliness levels is highest (Ayalon, Shiovitz-Ezra, & Palgi, 2013; Dykstra & de Jong Gierveld 2004; Pikhartova, Bowling, & Victor, 2015). Aspects of social relations, e.g. fewer friends, lower frequency of contact, less frequent social participation as well as poorer quality of relations are associated with loneliness (Hawkley, 2015; Heylen, 2010; Newall et al., 2009). Some studies report an inverse association between religious belief and loneliness for men, but not for women (Baumeister & Storch, 2004) while others have found the opposite (Kirkpatrick, Shillito, & Kellas, 1999). Some studies have found that attending religious activities (church etc.) – and not belief – is related to lower levels of loneliness (Johnson & Mullins, 1989). Finally, several authors have shown poor health to be associated with loneliness measured through physical health symptoms (Hawkley et al., 2008), disablement (Warner & Adams, 2016) and disability (Korporaal, Broese van Groenou, & van Tilburg, 2008).

In this study, we assess 42 variables within the five domains – personality characteristics, affective states, demographic characteristics, social relations and health – that have we have identified conceptually and empirically as correlates of loneliness in terms of their relative importance of associations to loneliness levels and the combined effect of these same correlates.

Methods

Study population

The Medical Research Council National Survey of Health and Development (NSHD) is the longest running birth cohort study in the world. The study is based on a representative sample of 5,362 males and females born within marriage during a week in March 1946 in England, Scotland or Wales. As part of the 24th follow-up conducted between 2014 and 2015 the remaining eligible study members were asked to complete a postal questionnaire at age 68 (Kuh et al., 2016). In total, 2453 study members answered the postal questionnaires (Figure 1).

Figure 1. Flow chart of the study sample at the 24th data collection in 2014–2015.

Variables and measures

All the variables are collected at age 68 unless otherwise specified. A list of the variables and how they are scored can be found in the online supplementary materials.

Outcome

Loneliness was measured using the 3-item short scale UCLA measure (Hughes, Waite, Hawkley, & Cacioppo, 2004). Participants are asked how often they (1) feel that they lack companionship, (2) feel left out and (3) feel isolated from others. The three response options; hardly ever, some of the time and often are summed into a composite scale ranging from 3-9 with 3 indicating being the least lonely and 9 being the most. The scale has convergent and discriminant validity in a US setting (Hughes et al., 2004). The short scale validity has not been assessed in a British setting so using a similar approach to Uysal-Bozkir et al. (2015) we assessed its reliability and structural validity (discriminant and convergent) comparing the results to the US-based validation study (Hughes et al., 2004). We found evidence of both convergent, discriminant and structural validity. The results can be found in the online supplementary materials.

Correlates

Four personality measures were included; extroversion, agreeableness, neuroticism (at age 26) and conscientiousness (Goldberg, 1990). Personal mastery was measured through the Pearlin Mastery Scale (Pearlin & Schooler 1978). Included measures of affective states were the Warwick-Edinburgh Mental Well-being Scale (WEMWBS) ranging from low to high wellbeing (Tennant et al., 2007) and four measures of the degree of either physical and mental fatigue from participating in or hosting a social activity, respectively (Glynn et al., 2015). Socio-demographic characteristics were measured by midlife occupational social class at age 53 and coded per the UK Registrar General's Classification. We included home ownership and number of cars in the household. Educational level at age 26 was coded as no qualification/sub GCE, ordinary secondary qualifications (typically attained by age 16), advanced secondary qualifications (typically attained by age 18) or higher qualifications (a degree or equivalent). We further included whether the participants had retired from their main occupation, their age at retirement, gender, marital status and change in marital status. Health was measured by self-rated health, any longstanding illness or health problem, and the degree to which this had limited their daily activities. Social relationship measures were frequency of visits to/by friends or relatives, how many friends or relatives the participants saw at least once a month, number of hours of voluntary work per week, number of children (until age 53), how close they live to their nearest child, death of children (until age 37), if they have grandchildren and are regularly visited by them, frequency of participation in social activities (recreational groups, civic-political groups, community fund-raising clubs and informal social activities such as going to pubs, cinema, theatre etc. with others, or online social networking). Study members were also asked the identity of the person they felt closest to and the level of emotional support and negative aspects of this relationship (Stansfeld & Marmot, 1992). Religious characteristics were measured through the frequency of participation in church and religious meetings, the frequency of prayer and meditation, whether religion provides purpose and the importance of faith.

Statistical analysis

Assessing the importance of loneliness correlates using recursive partitioning

Recursive partitioning is a non-parametric approach originating from the field of machine learning. Random forests are based on the results of an ensemble of single regression trees where each individual tree recursively splits the data into smaller subsets that contain participants with similar response values. For a single regression tree the algorithm compares the included correlates across levels to determine which of these is the optimal predictor and its cut-off point such that the within-group variance on the loneliness is minimized (Scott et al., 2011; Strobl et al., 2009). The recursive binary splitting approach employed by the algorithm starts with the entire sample and then uses the least squares criterion to determine how best to split the sample in terms of loneliness levels across the included correlates. The identified correlate and its respective cut-off point that minimizes the within-group variance of loneliness produces two groups. Next, this binary splitting is repeated for each of the two groups at each step identifying the correlate and its respective cut-off that optimally divides the data into two further groups. Thus, for each splitting the smaller and smaller groups become more similar both in terms of loneliness levels and values of the identified correlates. The binary splitting is repeated until the algorithm reaches a stopping criterion and the predicted loneliness levels for each final subsample is calculated (Hastie, Tibshirani, & Friedman, 2009; James et al., 2014; Strobl et al., 2008, 2009). This algorithm results in individual trees with low bias but with a tendency to overfit the data. The desirability of random forest is that each tree in the ensemble uses a slightly different subset of the study sample as well as a subset of the included correlates. When a strong correlate is left out of a tree, interactions between other correlates that might have been missed in a single tree may now be identified. With a high number of trees in the ensemble this approach ensures that most possible interrelations between included correlates are being considered and results in an overall tree-average that is much less variable and thus more reliable than any single tree prediction. The importance of a given variable is then measured through the amount that the residual sum of squares is decreased due to splits over a given predictor, averaged over all the trees (James et al., 2014). Lastly, based on the variable importance measure the variables can now be ranked in their importance of how well they identify loneliness levels through variable importance plots. In total the relative ranking of correlates was assessed across 5000 estimated regression trees using the r-package randomForest (Liaw & Wiener, 2002) in the statistical software environment R (R Core Team, 2015). In the results, we show one of these single regression trees to demonstrate how combinations of correlates indicate different groupings of participants with similar loneliness levels.

We imputed missing data using the random forest technique through the MICE package in R (Stekhoven, 2013; Stekhoven & Bühlmann, 2012). Single item missingness for all variables were below 9% except mental fatigue when hosting social activities (10.9%), physical fatigue when participating in social activities (15%), and mental fatigue when participating in social activities (17.9%).

Results

Socio-demographic characteristics

The demographic characteristics of the 2453 participants are summarized in Table 1. Most had retired from their main occupation by age 68 and had a low level of education by age 26. The majority owned the home they were living in, had a longstanding illness or health problem, were married, and had a mean loneliness score of 3.8.

Table 1. Characteristics of the NSHD (n = 2453) at age 68.

Characteristic	n (%)
Gender
Male	1177 (48.0)
Female	1276 (52.0)
Age at retirement from main occupation
Retired before age 50	139 (5.7)
Retired before age 60	599 (24.4)
Retired between 60 and 68	1601 (65.3)
Still working	114 (4.6)
Educational attainment at age 26
None/sub-GCE	976 (39.8)
O-level	520 (21.2)
A level or equiv.	675 (27.5)
Degree or higher	282 (11.5)
Home ownership
Own outright	2018 (82.3)
With a mortgage or loan	179 (7.3)
Rent—private landlord	80 (3.3)
Rent–council/housing association	144 (5.9)
Other	32 (1.3)
Have a longstanding illness
No	1012 (41.3)
Yes	1441 (58.7)
Marital status
Married	1863 (75.9)
Divorce	292 (11.9)
Single	89 (3.6)
Widowed	209 (8.5)
	Mean (sd)
Loneliness	3.85 (1.4)

The relative importance of correlates

Figure 2 shows the 10 most and 10 least important identifiers of loneliness ranked by how much the residual sum of squares is decreased due to splits over the correlate in question, averaged over all 5000 trees. The relative ranking of correlates should be assessed and not their absolute values (Scott et al., 2011). In total 32% of the variation in loneliness could be explained by the 42 correlates included in this analysis. Wellbeing explained the largest amount of variation in loneliness, with higher wellbeing being associated with lower levels of loneliness (Table 2). Higher levels of mastery and extroversion, being married, identifying the spouse as the closest confidante, and higher frequency of both the number of friends and visits in a month were all associated with lower loneliness levels and were in the top 10 most important correlates. Among the 10 least important correlates were number of cars owned and educational level, civic-political participation, having a longstanding health condition and marital dissolution in the last six years.

Figure 2. Ranking of the 10 most (top) and 10 least (bottom) important correlates across 5000 trees*.

Table 2. Spearman's rank correlation coefficient (ρ) with loneliness for continuous and ordinal scaled correlates and mean loneliness levels for nominal scaled correlates.

Ten most important correlates	ρ
Wellbeing	−0.42
Mastery	−0.34
Extroversion	−0.25
Number of visits to/by friends	−0.21
Number of friends and/or relatives seen in a month	−0.26
Low degree of limiting health problem	−0.18
Good self-rated health	0.19
Number of visits to/by relatives	−0.17
Identity of closest confidante	Mean loneliness level
Partner	3.63
Relative	4.48
Child	4.41
Other	5.23
Friend	4.62
No-one	5.95
>1 given	3.92
Marital status	Mean loneliness level
Married	3.68
Divorced	4.25
Single	4.64
Widowed	4.33
Ten least important correlates	ρ
Neuroticism	0.16
Divorced in last 6 years	0.07
Religious faith provides meaning	−0.04
Having a longstanding health issue	0.08
Religious faith is important	−0.02
Civic participation (recreational groups)	0.07
Civic participation (community service)	0.06
Widowed in last 6 years	0.06
Civic participation (political)	0.03
Number of cars owned	−0.11

Loneliness levels based on social demographic characteristics

We conducted a sub-analysis with just the social demographic characteristics listed in Table 1. Marital status emerged as the most important correlate of loneliness followed by midlife social class with the lower social classes, lower educational attainment and having a health issue for the last six months having higher loneliness levels (Table 2 and Figure 3). The random forest with only socio-demographic characteristics explained −1.7% of variance in loneliness, indicating that the grand mean provides a better prediction than the inclusion of these covariates. Removing the correlates with a negative explained variance only marginally improves the overall model fit to 1.3%.

Figure 3. Ranking of socio-demographic correlates in variable importance.

Combinations of correlates best indicating subgroups with similar loneliness levels

Figure 4 illustrates how combinations of correlates identify subgroups of the population with similar loneliness levels. The regression tree demonstrates a compounding of risk. For example, participants who had a combination of low levels of wellbeing, not indicating a partner as their closest confidante, and who had reported a low perceived relationship quality with their closest confidante (3 points below the mean) had the highest mean loneliness (m = 7.5). The group (n = 1030) who experienced high wellbeing (above 45.5) and high mastery reported the lowest mean loneliness (m = 3.4). In contrast, participants who scored high on wellbeing but low on mastery had an average loneliness (m = 3.9 or 5.0 depending on whether they had nominated their partner or child as close confidante or not).

Figure 4. Illustration of a regression tree with size of subsample (n) and mean (M) at each stage*.

Discussion

In this study we aimed to examine the relative importance of different correlates of loneliness. Among the 42 included correlates considered here, mental wellbeing, personal mastery, identifying one's partner as the closest confidant and being extrovert were identified as the most important. We also demonstrated how recursive partitioning can help identify combinations of experiences and circumstances that characterise high-risk groups among older adults.

With regard to the first aim, affective state played the largest part in differing loneliness levels compared to other correlates included in this study. This is in support of the theoretical underpinnings of loneliness that is itself defined as an affective state (Perlman & Peplau, 1981). Whilst marital status was also found to be among the 10 most important correlates (in support of the notion that a partner is important for facilitating a wider social contact and providing emotional stability which in turn helps protect against loneliness (Dykstra & de Jong Gierveld, 2004; Hawkley, 2015; Victor & Bowling, 2012), the identity of their closest confidante was more predictive of loneliness levels than marital status. While simply having a partner is associated with lower loneliness levels it is more important that your partner is also your closest confidante. As anticipated, personality measures (notably extraversion and personal mastery) were strongly associated with lower loneliness, though in contrast to previous studies agreeableness was not identified here (Peerenboom et al., 2015).

Structural social relationship measures such as frequency of seeing friends or relatives and number of friends or relatives are also among the most important when identifying loneliness levels, as others have found Dahlberg and McKee (2014) and supporting the view that social isolation plays a part in who becomes lonely and who does not (Hawkley, 2015). This is in support of the findings from Dahlberg and McKee (2014) who found that low contact with friends and family were significant predictors of loneliness. Contact with friends and regularly seeing a larger number of people explained more variation in loneliness than did contact with relatives.

As part of the analysis, we investigated whether socio-demographic characteristics readily available from public registers can be used to identify high-risk groups. Our results clearly indicate that identifying lonely older adults is highly imprecise using information on marital status, house and car ownership, educational level, midlife social class, having a longstanding health issue, gender and age at retirement only. If we want to identify older adults who are lonely, classical demographic variables are simply insufficient predictors. This supports the notion that identifying groups at high-risk of loneliness is a complicated process (De Jong Gierveld, 1998; Victor & Yang, 2012) where we must look to several aspects instead of more classical demographic factors alone.

With regard to the second aim, our analyses identified examples of combinations of experiences and circumstances associated with high loneliness, in line with different pathways to loneliness suggested by de Jong Gierveld (1998). The regression tree provide examples of equifinality – how different pathways can lead to similar outcomes in loneliness as well as examples of compounding of risk or protection through the interaction of different correlates (Scott et al., 2011). For a given level of loneliness, different factors contributed to this. Efforts to intervene in terms of increasing frequency of social contact – a common intervention strategy in terms of diminishing the risk of loneliness (Dickens et al., 2011; Hawkley, 2015) – would most likely benefit participants in the left of the tree more than those to the right (Figure 3), though we note that informal contact with friends and relatives rather than social participation in organised groups were more strongly related to loneliness. The regression tree also shows examples of combined effects and demonstrates cumulative pathways where there is a compounding risk of loneliness. For example, having a higher sense of mastery seems to negate the negative effect of seeing less than three different friends or relatives a month. Similarly, indicating a partner as closest confidante and seeing more than three friends or relatives a month seems to offset the higher loneliness levels for the group with low levels of wellbeing. These are examples of compensatory pathways (Seroczynski & Cole, 1997) in which being low risk in one domain offsets being high risk in another.

Strengths and limitations

Among the strengths of this study, we count the population-based data source, the comprehensive number of included variables in the analysis and the choice of recursive partitioning as the analytic method. Additionally, by using the NSHD we can rule out potential bias resulting from age differences and the large sample allows for more precise estimates. However, limitations are also present. The data originated mostly from the same sweep and loneliness was not measured before age 68, and thus we have refrained from implying causal directions in the interpretation of results. Neuroticism and death of children were collected at previous ages and it is possible that the importance of these variables would change if up-to-date information had been included in the model. Srivastatva et al. (2003) found that neuroticism changed only slightly for women but not for men throughout adulthood, giving some credence to the validity of using this measure from an earlier age. For death of children it is likely that adding information on death of children when the participants where in their 40–60 s would show a greater importance of this correlate of loneliness. We cannot rule out a possible information bias with those feeling lonely tending to answer in certain ways. Also, we cannot rule out measurement overlap between the WEMWBS and the UCLA scale but we note that associations with loneliness were seen for all wellbeing items and not only those capturing social wellbeing (data available from the authors). While our analyses show that the levels of loneliness in the NSHD are comparable to those from the English Longitudinal Study of Ageing, we cannot rule out a potential attrition bias. Socioeconomic advantage was associated with a greater likelihood of responding, though affective symptoms were not (Stafford et al., 2013). Thus, participants with a lower socioeconomic position may be different in terms of loneliness to those who did not respond. Previous analyses at age 60-64 show that the NSHD was broadly comparable to the UK population of the same age on social class and unemployment rate but over-represented owner-occupiers (Stafford et al., 2013). Generalizing to other age groups should be done with caution. The analysis was able to account for about a third of the variance in loneliness. This suggests a need to broaden our scope of analysis if we want to have a greater understanding of both the different ways people become lonely as well as the ways inventions should be designed as to best alleviate loneliness. Thus, a possible next step could be to consider possible determinants across the life course as suggested by Victor and Yang (2012). Future research may also benefit by distinguishing between correlates of social and emotional loneliness (De Jong Gierveld & Van Tilburg, 2010) and how correlates of the two may differ in importance across the life course.

Conclusion

This study adds to the knowledge of the relative importance of correlates of loneliness among older adults demonstrating how recursive partitioning can be used to gain insight in this regard. The results have several implications for the field. Identifying older adults who are lonely using information on only socio-demographic factors does not discriminate those who are lonely. When analysing a wide range of determinants together, affective states explain the most variation in loneliness, emphasizing the highly qualitative nature of loneliness and helping to explain why objective measures of social isolation only moderately correlate with the negative subjective evaluation of social isolation that constitutes loneliness. Nevertheless, informal social contact may modify the association between poor mental wellbeing and loneliness, suggesting that encouraging this kind of contact in older age may be beneficial even in the context of low affect. Future research may benefit from inclusion of a broader range of age groups at different life stages to get a deeper understanding of the complex phenomenon of loneliness.

Acknowledgments

The authors thank the study participants for their continuing participation in the MRC NSHD. They also thank members of the NSHD scientific and data collection teams who have been involved in the NSHD data collections.

Data used in this publication are available bona fide researchers upon request to the NSHD Data Sharing Committee via a standard application procedure. Furhter details can be found at http://www.nshd.mrc.ac.uk/data (dui: 10.5522/NSHD/Q101; doi: 10.5522/NSHD/Q102).

This study and MS and DK were funded by the UK MRC (program codes MC_UU_12019/1; MC_UU_12019/5).

Disclosure statement

No potential conflict of interest was reported by the authors.