Predictive validity of the Chelsea Critical Care Physical Assessment tool (CPAx) in critically ill, mechanically ventilated adults: a prospective clinimetric study

Abstract

Purpose

To investigate the predictive validity of the Chelsea Critical Care Physical Assessment tool (CPAx) at intensive care unit (ICU) discharge in critically ill adults for their 90-day outcomes.

Materials and methods

This prospective clinimetric study investigated four theory-driven, a-priori hypotheses in critically ill adults recruited within 72–144 h of mechanical ventilation. The primary hypothesis was a moderate accuracy (AUROC = 0.750) in predicting residence at home within 90 days. Secondary hypotheses included discrimination between hospital discharge destinations, correlation with subsequent health-related quality of life and length of ICU stay.

Results

We observed a good accuracy (AUROC = 0.778) of the CPAx at ICU discharge in predicting a return to home within 90 days. The CPAx score significantly increased between the discharge groups “undesirable” ≤ “rehabilitation” ≤ “home” (p < 0.001), but was not associated with 90-day health-related quality of life (physical: r = 0.261, mental: r = 0.193). Measured at baseline, CPAx scores correlated as expected with length of ICU stay (r = −0.443).

Conclusions

The CPAx at ICU discharge had a good predictive validity in projecting residence at home within 90 days and general discharge destinations. The CPAx might therefore have clinical value in prediction, though it does not seem useful to predict subsequent health-related quality of life. TRIAL REGISTRATION: German Clinical Trials Register (DRKS) identification number: DRKS00012983, registered on September 20, 2017

IMPLICATIONS FOR REHABILITATION

• The CPAx is a valid and reliable measurement instrument to evaluate critically ill adults’ physical function and activity, in addition the CPAx might be useful to predict rehabilitation needs.

• The CPAx had a moderate to good predictive validity with three out of four a-priori hypotheses accepted.

• A CPAx score of ≥18 at critical care discharge has a sensitivity of 80% and a specificity of 70% in predicting a return to home within 90 days.

• The CPAx might consequently be valuable to identify critically ill adults’ rehabilitation needs, to advise on their potential trajectory of recovery or to screen patients for follow-up after hospital discharge.

Keywords:

• Physiotherapy
• rehabilitation
• critical illness
• predictive value of tests
• patient outcome assessment
• critical care outcomes
• measurement instrument
• physical function

Introduction

Survivors of critical illness often experience functional disability and reduced health-related quality of life after hospitalisation [1–3]. Only about 33% will return to work within three months of discharge, which raises concerns about financial security or social isolation potentially worsening health-related quality of life [4]. Early rehabilitation starting in the intensive care unit (ICU) improves functional outcomes and reduces the duration of mechanical ventilation and ICU length of stay [5,6]. However, it may not be solely sufficient to counter long-term functional disability [6] or to increase health-related quality of life [7]. Returning home is an important outcome for patients and their families [8]. Early identification of patients at risk for subsequent functional impairment may help to provide targeted multidisciplinary interventions.

The Chelsea Critical Care Physical Assessment tool (CPAx) is a performance-based measurement instrument to assess respiratory function, functional mobility and grip strength in critically ill adults [9]. The evaluation of the CPAx is based on observation and its ten items are rated on a 6-point Guttman scale from 0 (=dependent/unable) to 5 (=independent). The CPAx has established clinimetric properties such as an excellent interrater-reliability [9], construct and cross-sectional validity across the ICU and hospital stay [10], responsiveness [11] along with low floor and ceiling effects in a general ICU population [12]. The clinical value of the CPAx therefore lies in the evaluation of critically ill adults’ physical function and activity across the ICU and hospital. However, the CPAx may also have a relevant role in predicting patients in need of further multidisciplinary rehabilitation as indicated by one previous study exploring hospital discharge destinations [12]. More research is therefore needed to study the usefulness of the CPAx for prediction in survivors of critical illness.

In order to expand the clinimetric properties of the CPAx and to test whether the CPAx has clinical value in population-prediction, this study aimed to investigate the predictive validity of the CPAx in critically ill adults assessed at ICU discharge. We hypothesised that the CPAx at ICU discharge would have a moderate accuracy in predicting a good outcome, which has been defined as residing at home 90 days after ICU discharge.

Material and methods

Design and setting

This prospective, single-centre, clinimetric study was the second part of a larger clinimetric project conducted in the mixed ICU of an academic hospital in Switzerland (Department of Intensive Care Medicine, Inselspital, Bern University Hospital, Switzerland). The first part aimed to investigate the measurement properties (validity and reliability) of the German CPAx and has been published previously [10]. The local Ethics Committee approved the project on 14 August 2017 (ID 2017-01396), which was subsequently performed in accordance with the Declaration of Helsinki, Swiss federal law and Good Clinical Practice guidelines. We based our methodology on the consensus-based standards for the selection of health measurement instruments (COSMIN) [13] and report trial-results according to the STROBE statement [14].

Participants

Eligible participants were critically ill adults (aged >18 years) who were mechanically ventilated for ≥72 h and in sufficient command of German. We excluded patients who were wheelchair users, lived in a care facility, had already participated in ≥2 studies (confounding) or who had a planned discharge within the next 24 h, a neurological ICU admission diagnosis (CPAx not yet validated), ongoing palliative treatment, a pre-existing mental disability, transfer from an external ICU (with >3 days length of stay at external ICU), or an ICU re-admission (within the same hospital stay). Potential participants were screened by a research nurse and a senior physiotherapist. Written informed consent was obtained from eligible patients’ relatives prior to commencement of data collection, and then from participants once they regained the capacity to provide informed consent.

Procedures and measurements

The complete study procedures have been published earlier [10]. In brief, the CPAx score was obtained after a standard physiotherapy session by certified physiotherapists at baseline (between 72 and 144 h of ventilation), at ICU discharge and at hospital discharge. All therapists completed the short, official online training (<2 h) before using the CPAx during routine care. This study was conducted in Switzerland, accordingly, the cross-culturally validated, highly reliable (ICC >0.9) German CPAx version was used [10]. A research nurse collected demographic and hospital data such as duration of mechanical ventilation and hospital discharge destination. For the purpose of establishing predictive validity of the CPAx, all participants were followed-up for 90 days after ICU discharge by the research team. We prospectively defined a good outcome as “residence at home 90 days after ICU discharge.” Non-survivors were identified from hospital databases. Survivors were contacted by phone and asked for their current residence, working status and whether they would agree to complete a questionnaire about their health-related quality of life. Short Form 36 (SF-36) questionnaires were then sent by post with a stamped, addressed return-envelope. The SF-36 is a multidimensional, patient-reported measurement instrument for health-related quality of life that provides two component summaries for physical and mental health that incorporate normative data (based on the US population 1990: population mean = 50 ± 10) and have established validity and reliability in critically ill patients [15].

Hypotheses

Predictive validity determines the ability of a measurement instrument to predict a selected criterion in the future [16]. We constructed the following a-priori hypotheses, based on our theoretical conceptual model [10] which takes into account previous research as well as influencing factors of constructs that are not measured by the CPAx (for example social support or pain):

Primary outcome

1. The CPAx score at ICU discharge would have a moderate accuracy with an Area Under the Receiver Operating Characteristic curve (AUROC) of 0.75 in predicting a good outcome defined as residence at home 90 days after ICU discharge.

Secondary outcomes (order of clinical importance)

2.  The CPAx score at ICU discharge can differentiate between three hospital discharge destinations (home, rehabilitation or an undesirable discharge such as a transfer to an external hospital or death). Accordingly, we hypothesised that the CPAx at ICU discharge would increase in the following ranked order: “undesirable” ≤ “rehabilitation” ≤ “home.”

3.  The CPAx score at ICU discharge would have a positive, low correlation with the SF-36 mental component summary 90 days after ICU discharge (r = 0.2–0.4) and a positive, moderate correlation with the SF-36 physical component summary 90 days after ICU discharge (r = 0.4–0.7).

4.  The CPAx score at ICU baseline (measured between 72 and 144 hours of mechanical ventilation) would have a negative, low to moderate correlation with days of mechanical ventilation and the length of ICU stay (r = −0.3 to −0.6).

Statistical analysis

Descriptive statistics were used to investigate patient demographics and score distribution. The CPAx is an ordinal scale, accordingly, data is presented as medians with interquartile range (IQR) and categorical data as numbers with percentages. Analyses were performed with SPSS (IBM SPSS Statistics Version 25).

We performed a prospective sample size calculation for the primary outcome of returning home within 90 days of leaving the ICU with MedCalc (version 19.3.1). According to the study by Corner et al. [12] we anticipated a ratio of 2:1 for a good outcome “home within 90 days” whereby Corner’s study indicated “home after hospital discharge.” With a power of 0.8, an α of 0.05 and an expected AUROC of 0.75 as per our a-priori hypothesis (null hypothesis: AUROC = 0.5) a total sample size of 42 (n = 28 for a good and n = 14 for a negative outcome) was necessary [17,18]. To compensate for drop-outs, we aimed to recruit 60 participants. The primary hypothesis was analysed using a receiver operating characteristic (ROC) and the respective area under the ROC curve (AUROC). The optimal cut-off point with the corresponding sensitivity and specificity was chosen based on ROC-curve coordinates and clinical relevance. To investigate hypothesis number two, the three prespecified discharge groups “undesirable,” “rehabilitation,” and “home” were tested for an increased ranked order with the nonparametric Jonckheere–Trend–Test [19,20]. The assumption of homogeneous variances between the three groups was investigated with the Levene Test. Statistical significance was defined as p < 0.05 using Bonferroni correction for pairwise comparisons. Hypotheses three and four were investigated with Spearman’s rank correlation coefficient.

Results

Sixty patients were recruited for this study from 21 November 2017 to 25 May 2019 of whom 50 had CPAx scores assigned at ICU discharge. The study flow including the 90-day follow-up is reported in Figure 1. Descriptive data about participants’ demographics, baseline characteristics and outcomes are presented in Table 1.

Figure 1. Study flow.

From 60 included patients, 2 have withdrawn their consents and 8 died in the ICU leading to 50 CPAx measurements at ICU discharge. At 90-day follow up, there were 58 outcomes available and 28 SF-36 questionnaires returned, whereby 2 were unable, 19 died and 9 did not return the questionnaire.

Table 1. Descriptive participant data.

Variables	Number (n)	Median [25%, 75%] or n (%)
Age (years)	58	68 [56, 73]
Gender (male)	58	41 (70.7%)
BMI (kg/m^2)	58	26 [22.8, 29.1]
Weight (kg)	58	75.5 [70, 94]
APACHE II score (0–71)^a	58	32 [28, 36]
SOFA score (0–24)^a	58	12 [10, 14]
ICU diagnosis on ICU admission
Cardiac surgery	58	15 (25.9%)
Other surgery		7 (12.1%)
Respiratory insufficiency		15 (25.9%)
Hemodynamic insufficiency		13 (22.4%)
Trauma		5 (8.6%)
Neurology/neurosurgery		–
Gastroenterology		1 (1.7%)
Other		2 (3.4%)
ICU days until study inclusion (days)	58	3.84 [3.41, 4.70]
Length of stay in ICU (days)^b	58	7.97 [6.69, 12.85]
Duration of mechanical ventilation (days)^b	58	6.81 [5.30, 11.51]
Length of stay in hospital (days)^b	58	21.08 [11.72, 27.18]
ICU mortality^b	58	8 (13.8%)
ICU and hospital mortality^b	58	9 (15.5%)
90-day mortality	58	19 (32.8%)
CPAx at baseline (0–50)	58	5 [3, 7]
CPAx at ICU discharge (0–50)	50	18.5 [14, 26]
CPAx at hospital discharge (0–50)	32	43.5 [37.5, 46.5]
ICU discharge destination
Hospital wards	58	34 (58.6%)
External ICU		13 (22.4%)
External hospital		3 (5.2%)
Died		8 (13.8%)
Hospital discharge destination
Rehabilitation	58	24 (41.4%)
External hospital		20 (34.5%)
Home		5 (8.5%)
Died		9 (15.5%)
Outcomes 90 days after ICU discharge
Home	58	30 (51.7%)
ICU		1 (1.7%)
External hospital		2 (3.4%)
Rehabilitation facility		6 (10.3%)
Died		19 (32.8%)
Return to work 90 days after ICU discharge	36^d	14 (38.9%)
SF-36 physical health sum-score^c	28	38.7 [33.0, 44.9]
SF-36 mental health sum-score^c	28	52.2 [46.9, 58.4]

^aICU admission.

^bAt original hospital (excluding external hospitals).

^cT-values where the population mean is 50 and the SD is 10; based on US-population 1990.

^dUnknown status: n = 3 of 39 survivors.

BMI: Body mass index; APACHE: acute physiology and chronic health evaluation; SOFA: sequential organ failure assessment; ICU: intensive care unit; SF-36: Short Form 36 questionnaire.

Primary outcome

Hypothesis 1: The AUROC of the CPAx at ICU discharge predicting a good outcome was 0.778 (95%-CI 0.644–0.912, p-value = 0.001, standard error 0.068, n = 50: positive n = 30, negative n = 20) (Figure 2). Accordingly, a CPAx score of ≥18 predicts a good outcome (residence at home within 90 days) with a sensitivity of 80% and a specificity of 70% (Supplemental file 1, Table S1). For this cut-off score, the positive predictive value is 80% and the negative predictive value 70% (Supplemental file 1, Table S2). The hypothesis is accepted.

Figure 2. ROC-Curve for a good outcome “residence at home within 90 days of ICU discharge.” Sensitivity: true positive rate, 1-specificity: false positive rate

The relationship between sensitivity on the y-axis and 1-specificity on the x-axis is illustrated with a ROC-curve. The curve proceeds at 70 degrees to the top-left corner which lies at a sensitivity of 0.8 and a 1-specificity of 0.3.

Secondary outcomes

Hypothesis 2: The three discharge groups met the assumption of homogenous variance (Levene Test p = 0.198 based on medians, p = 0.115 based on means). The CPAx score at ICU discharge increased in a ranked order between the three hospital discharge destinations (p < 0.001, n = 50: undesirable n = 21, rehabilitation n = 24, home n = 5) (Figure 3). The hypothesis of an increasing ranked order of CPAx scores (“undesirable” ≤ “rehabilitation” ≤ “home”) could therefore be accepted. Additionally, the CPAx scores at ICU discharge significantly differed after Bonferroni-adjustments for repeated testing for the following discharge destinations: “undesirable” versus “rehabilitation” (p = 0.001), “undesirable” versus “home” (p = 0.001) and “rehabilitation” versus “home” (p = 0.023).

Figure 3. CPAx score at ICU discharge across the three discharge groups.

The three boxplots per discharge group show an increasing CPAx score from undesirable outcome to rehabilitation to discharge to home.

Hypothesis 3: The correlations of the CPAx score at ICU discharge with the SF-36 mental component summary and the physical component summary 90 days after ICU discharge were very low and outside the expected ranges (Table 2). Consequently, the hypothesis was rejected.

Table 2. Hypothesis-testing with correlation.

Prior hypothesis	n	Spearman’s rank correlation coefficient (r)	Confirmed
CPAx at ICU discharge with
SF-36 mental sum-score (r = 0.2–0.4)	28	0.193	No
SF-36 physical sum-score (r = 0.4–0.7)	28	0.261	No
CPAx at ICU baseline with
days of mechanical ventilation (r = −0.3 to −0.6)	58	−0.499**	Yes
ICU length of stay (r = −0.3 to −0.6)	58	−0.443**	Yes

**p < 0.001.

Bold text indicates acceptance of the a-priori hypothesis.

Hypothesis 4: The CPAx score at ICU baseline correlated within the expected values with the subsequent days of mechanical ventilation and ICU length of stay (Table 2). The hypothesis was accepted.

Discussion

This prospective, clinimetric study found that the CPAx at ICU discharge has a good predictive validity in predicting residence at home within 90 days of discharge as well as hospital discharge destination (home, rehabilitation, undesirable discharge, e.g., transfer to an external hospital or death). However, the CPAx did not seem suitable to predict health-related quality of life within 90 days of ICU discharge. A CPAx assessment between 72 and 144 h of mechanical ventilation may further be used to predict the length of ICU stay and the duration of mechanical ventilation. The CPAx can therefore be partially recommended for prediction, for example, to identify candidates for a targeted, in-hospital rehabilitation, to advise patients on their probable trajectory of recovery or to recognise candidates for multidisciplinary follow-up after ICU discharge. Similarly, the CPAx at ICU baseline could be used to identify risk for prolonged mechanical ventilation, which is a risk factor for subsequent functional disability [21].

The evaluation of the predictive validity of the CPAx has not been the primary aim of other studies. However, our results are somewhat supported by Milton et al. [22] who investigated predictors for poor physical function within 3 months of ICU discharge. Although they did not aim to investigate the predictive validity of the CPAx with pre-specified hypotheses as recommended by the COSMIN guidelines, they found that the CPAx remained the only significant predictor among the risk factors explored including age, previous comorbidities, severity of illness or ICU length of stay. Their calculated AUROC (0.68 [95%-CI 0.61–0.76]) was somewhat lower, but comparable to ours (0.78 [0.64–0.91]). However, caution is warranted because only half of the ten CPAx items were assessed in the study by Milton et al. [22] and there may have been bias due to measurement error (interrater reliability for four items: kappa ≤0.60). The CPAx at ICU discharge has been previously identified as being able to discriminate between seven hospital discharge destinations (p < 0.001) [12]. Given that this was a single-centre study in the UK, it is important to generalise this finding to other settings. We included less destinations because our hospital is a tertiary centre with many referrals from the region and thus only few people are directly discharged home; yielding too little data. Instead we investigated a previously defined ranked order with the Jonckheere–Trend–Test, which has more power than a traditional Kruskal–Wallis–Test [19]. Together these results confirm that the higher the CPAx score at ICU discharge, the higher the likelihood of a hospital discharge to home. More importantly, this might help clinicians to identify candidates in need of targeted rehabilitation after ICU discharge, for example, on acute wards. The clinical value of using the CPAx for prediction may further include patient-information on a probable trajectory of recovery or to recognise candidates for multidisciplinary follow-up after discharge. The CPAx could therefore be helpful to adapt expectations.

The CPAx at ICU discharge did not seem useful to predict health-related quality of life after 3 months. Estrup et al. [23] similarly did not find an association between activity levels in the first week of ICU discharge and physical functioning after 3 months. Reasons may include the small sample size and a high drop-out rate due to patient death in both our study and theirs. However, it seems most likely that other factors such as pre-ICU comorbidities [24] or social support [25] are more relevant to 3-month health-related quality of life than physical functioning at ICU discharge. Further prospective cohort studies are needed to investigate individuals’ determinants of quality of life.

The predictive validity of the CPAx is comparable to the Functional Status Score for the ICU (FSS-ICU), which is another excellent measure of physical function in the ICU. Similar to the CPAx, the FSS-ICU is useful to predict hospital discharge destination and/or length of subsequent hospital stay [26]. Thereby a cut-off score of FSS-ICU ≥19/35 at ICU discharge predicted a hospital discharge to home with a sensitivity of 83% and a specificity of 74% [27]. However, to our knowledge, there is no study that has investigated the predictive validity of the FSS-ICU at up to 3-months after ICU discharge. Still, the preference of using one measurement over the other may rather be due to different physiotherapy practices in respiratory care between the American and European continents [12]. Accordingly, we would recommend the CPAx to physiotherapists with a role in weaning from mechanical ventilation because in contrast to the FSS-ICU the CPAx takes respiratory function into account.

This study has limitations. The study aimed to investigate the predictive validity of the CPAx with a-priori hypotheses in the target population. While we established that the validity of the CPAx for population-based prediction, a larger sample size with narrower confidence intervals is needed for individual prediction. The chosen cut-off may therefore be overestimated for individual prediction and should be confirmed in future studies [28]. However, the patient-centred outcomes in our study have clinical value. They do not aim to influence decision making processes, but rather to help patients shape their expectations about their potential trajectory of recovery. Our definition of a “good outcome” did not consider independency or the amount of support at home. While physical health-related quality of life was substantially below population norms, half of survivors were able to go back to work within 90 days. Results from this single-centre study may not be generalisable to other health-care systems or non-tertiary regional hospitals with more diverse discharge destinations (e.g., care-home). The predictive validity of the CPAx only applies to the investigated factors (residence at home within 90 days, hospital discharge destination, subsequent ICU length of stay) and was only sufficiently powered for the primary outcome (residence at home within 90 days). More research would be necessary to investigate whether the CPAx would be useful to predict other outcomes, for example, functional capacity or independency at three months. The major strength of this study is the assessment of the CPAx in the target population during routine care which was independent of the patient-reported outcomes such as the SF-36 and the 90-day outcome.

Conclusions

The CPAx at ICU discharge has clinical value to predict a return to home within 90 days of discharge in critically ill, mechanically ventilated adults, thereby a cut-off score of ≥18/50 indicates a return to home with a sensitivity of 80% and a specificity of 70%. The CPAx may also be used to anticipate hospital discharge destination and thus the need for further rehabilitation. However, contrary to our hypothesis the CPAx did not prove useful to predict 3-month health-related quality of life.

Acknowledgements

We would like to thank all participating physiotherapists and research nurses as well as the patients and their families. We also thank Roger Hilfiker for his statistical advice.

Disclosure statement

SE, MLV, VS, AK, DS, CHGB declare that they have no competing interests. JS, BZ report the following potential conflicts of interest: The Department of Intensive Care Medicine has or has had in the past research and development/consulting contracts with Edwards Lifesciences Services GmbH, Phagenesis Limited and Nestlé. The money was paid into a departmental fund, and none of the authors received any financial gain. The Department of Intensive Care Medicine has received in the past unrestricted educational grants from the following organizations for organizing bi-annual postgraduate courses in the fields of critical care ultrasound, management of ECMO and mechanical ventilation: Pierre Fabre Pharma AG (formerly known as RobaPharm), Pfizer AG, Bard Medica S.A., Abbott AG, Anandic Medical Systems, PanGas AG Healthcare, Orion Pharma, Bracco, Edwards Lifesciences AG, Hamilton Medical AG, Fresenius Kabi (Schweiz) AG, Getinge Group Maquet AG, Dräger Schweiz AG, Teleflex Medical GmbH. No personal financial gain resulted from respective development/consulting contracts and/or grants. None of these companies had a role in this study.

Data availability statement

The dataset supporting the conclusions of this article is included within the article as a supplemental file (Supplemental file 2).

Additional information

Funding

This research was funded by the PhD Grant 2018 from the Swiss Foundation for Physiotherapy Science and by a research grant from physioswiss, both received by SE. The funders had no role in the study design, the collection, analysis and interpretation of data, or in writing the manuscript.