1. Neurobehavioural disability (NBD) and its assessment
A serious consequence of acquired brain injury (ABI) is NBD which underpins severe chronic social handicap. NBD comprises elements of executive and attentional dysfunction, poor insight, problems of awareness, and social judgement, labile mood, altered emotional expression, poor impulse control, and a range of personality changes. This complex pattern of disability is the product of interactions between damaged neural systems and neurocognitive impairment, further modified by premorbid personality traits and post-injury learning. NBD imposes serious constraints on psychosocial recovery; behavior disorders are enduring, create severe difficulties for families and are associated with poor engagement in rehabilitation. Furthermore, challenging behavior symptomatic of NBD is a greater impediment to community reintegration than chronic physical disability [Citation1,Citation2].
Given the potentially catastrophic impact of NBD, it is essential appropriate tools are available to enable assessment of people with ABI to ensure timely delivery of services. A challenge to assessment is the diverse range of symptoms people present with: consequently, a combination of different appraisal methods to capture the complete range is recommended. These include interviewing, direct observation, questionnaires, and rating scales. While an assortment of focal assessments enabling detailed examination of specific symptoms is available, such as for aggression [Citation3], use of ‘global’ assessment tools, which usually consist of rating scales that provide the necessary overview of the rich diversity of potential symptoms has proved problematic. These instruments can fulfill a number of important functions. In order for specialized rehabilitation to successfully reduce NBD in individual patients, reliable and valid methods of identifying symptoms, using reliable and valid means of assessment are two essential requirements. Additionally, instruments need to be repeatable to track recovery. Results of assessment should be meaningful and capable of informing rehabilitation. Finally, assessment data should be able to be collated to provide collective information to inform audit, create benchmarks of service efficacy, and for research.
2. Threats to assessment of NBD
The complexity of NBD creates a number of potential threats capable of undermining validity and reliability of assessment instruments. For example, neurocognitive impairment, especially memory, and attention, has been noted to impact adversely on the reliability of self-report measures, resulting in unrealistic self-appraisal, and a tendency for ABI survivors to under-report difficulties [Citation4]. Relying on proxy measures is also not unproblematic. Assessment based on observations of other people is potentially subject to a number of biases, including how well they know the person being assessed and their relationship with them. For example, relatives can underestimate loved ones difficulties, while a single carer or clinician may have only had limited contact with the person during the assessment period. The stage of recovery the person is at and the context in which assessment takes place is also noteworthy. For example, in the early stages of recovery, the person may be well-supported if they are in a residential rehabilitation service, which will mask symptoms of NBD; these symptoms are more likely to be evident after the person has been discharged and exposed to situations that make demands on their ability to function autonomously.
Comprehensive reviews have described in detail attributes of instruments concerned with global measurement of NBD using questionnaires and rating scales [Citation5,Citation6]. A range of issues limiting their usefulness was found, especially regarding: (1) methodological problems of definition, conceptualization, and measurement of NBD; (2) missing/weak psychometric properties undermining validity, reliability, and responsiveness; (3) lack of appropriate normative data limiting interpretation of assessments; and (4) scores lacked clinical and social meaningfulness and were difficult to apply to rehabilitation.
3. St Andrews–Swansea Neurobehavioural Outcome Scale (SASNOS): reliable, valid assessment of NbR
The SASNOS was subsequently developed to address these shortcomings, providing a ‘fit-for-purpose’ assessment tool capable of measuring the unique characteristics of NBD [Citation7]. SASNOS comprises 49 items that capture five major domains of NBD (interpersonal behavior, cognition, aggression, inhibition, and communication), each of which has two to three sub-domains. Each item consists of a statement regarding a symptom of NBD whose prevalence is rated using a 7-point scale (‘never’ to ‘always’). Both proxy and self-completion versions are available. Using the proxy version, ratings are made through observation of the person during the preceding 2-week period by a family member, carer, or clinician. When the person has been admitted into a residential neurorehabilitation programme, it is recommended that representatives of the clinical team complete assessment using SASNOS in the context of a case review. This can help reduce possible bias and increase reliability as multiple raters are more likely than a single assessor to capture the complete range of possible NBD symptoms evident during the assessment period. Where possible, it is instructive to get the person being assessed to rate themselves. Comparison of the proxy and self-completion ratings enables identification of jointly agreed treatment goals, and also provides a measure of self-awareness of the person with ABI.
The sum of ratings for the total score, 5 domains, and 13 subdomains are converted into standardized scores, using the T-distribution, with a mean of 50 and standard deviation of 10, derived from ratings of neurologically healthy controls. This group comprised 100 individuals rated by Swansea University staff and students; proxy ratings regarding this group were not influenced by age or gender with the exception of the ‘Aggression’ domain, where younger males were rated as presenting with more symptoms than younger females [Citation7]. Having a normative reference group has many advantages. Transformations are constructive, so higher scores reflect greater autonomy. As standardized scores are anchored using ratings of neurological healthy people, results are readily interpretable, being both clinically and socially meaningful. Scores provide a profile of strengths and weaknesses, identification of potential rehabilitation goals is easily achieved through consideration of which domain and sub-domain scores fall more than one standard deviation below the mean (less than 40).
SASNOS has robust psychometric properties, having utilized methods from classical test theory and item response theory (including Rasch analysis) in its construction. The authors of SASNOS described multiple indices of validity including content, face, convergent/divergent, discriminant/diagnostic validity, and internal consistency. Construct validity was good with Cronbach’s alpha exceeding .90 for 3 of the 5 SASNOS domains. Regarding diagnostic validity, discriminant function analysis demonstrated the total SASNOS score correctly classified nearly 90% of cases correctly (ABI vs. neurologically healthy controls). Both inter-rater reliability and test-retest reliability were excellent with intraclass coefficient correlations achieved for the total score of .81 and .94, respectively.
A website supports SASNOS and encourages the development of a user community -https://projects.swan.ac.uk/sasnos/- from where the measure is freely obtained. Website analytics confirm international interest in SASNOS and its employment in clinical work and research. An annual conference has been held in Swansea since 2016 to further raise awareness about NBD, its impact, assessment, and management.
4. Tracking response to rehabilitation through repeated assessment
Since publication in 2011, further work has been undertaken to improve the measure. Repeating assessment using global measures is routinely undertaken to track the effectiveness of rehabilitation. In order to interpret any differences in scores, it is essential that psychometric properties concerning responsiveness, the ability of a measure to detect change, is known. To reflect improvement due to rehabilitation, a difference in scores collected at two different assessments needs to exceed two thresholds: first, the smallest difference between scores that falls outside the measurement error of an instrument (minimally detectable change); and second, the smallest difference in scores that is clinically significant or indicative of meaningful change (minimally important change). However, responsiveness indices are elusive for the majority of NBD measures [Citation5] which greatly limits their use. Alderman, Williams, Knight, and Wood [Citation8] confirmed responsiveness as a psychometric property of SASNOS and reported five reliable change indices to assist interpretation of differences in scores from repeated assessments for both the group and individual contexts. Two were especially recommended. For measuring change between assessments in groups of rehabilitation participants, the authors endorsed use of the standardized response mean, a version of Cohen’s effect size. ‘Medium’ or better effect size is associated with clinically meaningful change. Regarding change in scores at the individual level, the authors sanctioned the standard error of measurement (SEM) is utilized. Differences in scores that exceeded two SEM are indicative of change beyond error associated with the measure and consistent with the achievement of meaningful change. Thresholds are given for the overall SASNOS score and each of the five domains, and the authors provide a worked example to illustrate their application in clinical practice.
5. Importance of context
Another issue is the results of assessment inevitably reflect the context in which they are made, which can ironically adversely impact on recovery. SASNOS ratings falling in the expected range for neurologically healthy controls (scores of 40 and higher) can be the product of the amount of support people receive, especially when assessed in a rehabilitation service. This can sometimes hasten discharge when apparently ‘normal’ ratings are thought to reflect true autonomy, which can prove problematic if scores are instead the result of ongoing rehabilitation. In these cases, discharge without support can prove disastrous. Alderman, Williams, and Wood [Citation9] published a solution to this dilemma, proposing a supplementary scoring system where a rating of 1–3 is assigned to each item reflecting the degree of support given. These dependency ratings are used to recalibrate standardized SASNOS scores to reflect what the profile would look like in the absence of support. Using this scheme, patients who require continued support to maintain lack of NBD symptoms can readily be identified.
Free availability of SASNOS gives clinicians, service managers, commissioners, and researchers an instrument that for the first time enables valid, reliable assessment of NBD, that can be employed across a range of contexts, for both clinical and research purposes. Unlike most other measures SASNOS was specifically developed to measure NBD, has comprehensive psychometric properties, normative data, and generates meaningful scores that inform rehabilitation.
6. Embedding outcome measures into services
However, having suitable instruments does not provide the total solution for solving the puzzle of measuring NBD. Measures need to be fully integrated within the clinical fabric of rehabilitation services if assessments are to truly reflect needs and play a central role in enabling patients on the road to greater independence. Managers, in particular, need to require measures are used, and services need to ensure policies and procedures are in place establishing when and how measures are employed, from pre-admission to discharge and beyond. Goal planning, assessment, formulation, and determining outcomes all benefit from use of standardized measures. If services use an electronic patient record then dashboards can greatly facilitate completion of assessments and produce outputs to inform individual care, audits, benchmarking, and marketing activities.
7. Providing a level playing field when benchmarking service
The importance of services concerned with reduction of NBD in using measures designed to capture symptoms and behavior arising from this is further highlighted when external agencies impose requirements to utilize assessment tools that lack relevance.
An accepted position is that ABI populations are diverse and complex, a basket of outcome measures is required to capture the full range of difficulties and needs. In the UK there is an expectation that services contribute data centrally using a defined basket of measures to the UK Rehabilitation Outcomes Collaborative (UKROC) in order to meet a range of ambitions [Citation10]. A variety of measures populate the UKROC basket, including tools that measure patient outcomes, patient complexity, and clinical inputs required to meet these. One goal is for UKROC to utilize these tools to categorize the complexity of cases a service can manage and establish a financial tariff for daily bed rates based on patient dependency (high, medium, and low). A criticism of this scheme is the choice of measures is heavily biased towards acute medical rehabilitation services. However, diversity of neurorehabilitation services means the choice of measures in the UKROC basket is not a ‘one-size-fits-all’ solution. This was especially felt to be the case for neurobehavioural rehabilitation (NbR) services which comprise the principal means by which NBD is managed, that are organized and implemented quite differently to medically focused programmes [Citation11]. A particular concern is that classification of services and performance benchmarking using measures that lack sensitivity in reflecting the success of NbR programmes will have a direct impact on funding. This issue was addressed by the Independent Neurorehabilitation Providers Alliance (INPA) who facilitated research which confirmed there were shortcomings in applying many of the UKROC tools to NbR services. Furthermore, this research clearly demonstrated the benefits of using SASNOS and other measures relevant to this population [Citation12].
8. Concluding remarks
In conclusion, SASNOS continues to make a valid contribution to the assessment of NBD and in determining rehabilitation outcomes. All rehabilitation services need to employ global measures of outcome to fulfill a range of needs, including clinical work, service evaluation, benchmarking, marketing, and research. It is only relatively recently that SASNOS, a measure of NBD that is valid, reliable and capable of reflecting meaningful change, has become available. By doing so, clinicians and service managers have an opportunity to finally utilize an instrument in their basket of outcome measures capable of capturing a range of symptoms and behaviors that potentially have a catastrophic impact on long-term outcome, and to properly demonstrate to external agencies and regulators the important work achieved in reducing the debilitating impact of NBD.
Declaration of interest
N Alderman is an employee of Elysium Healthcare, and an Honorary Professor at Swansea University and Visiting Professor at the University of the West of England. The author has no other relevant affiliations or financial involvement with any organization or entity with a financial interest in or financial conflict with the subject matter or materials discussed in the manuscript apart from those disclosed.
Reviewer disclosures
Peer reviewers on this manuscript have no relevant financial or other relationships to disclose.
Additional information
Funding
References
- Alderman N, Wood Rl. Neurobehavioural approaches to the rehabilitation of challenging behaviour. NeuroRehabilitation. 2013;32(4):761–770.
- Tam S, McKay A, Sloan S, et al. The experience of challenging behaviours following severe TBI: a family perspective. Brain Inj. 2015;29(7–8):813–821.
- Alderman N. Acquired brain injury, trauma and aggression. In: Dickens G, Picchioni M, Sugarman P, Eds. Handbook of specialist secure inpatient mental healthcare. London: The Royal College of Psychiatrists 2015;163-178.
- Cooper-Evans S, Alderman N, Knight C, et al. Self-esteem as a predictor of psychological distress after severe acquired brain injury: an exploratory study. Neuropsychol Rehabil. 2008 Oct-Dec;18(5–6):607–626.
- Tate R. A compendium of tests, scales and questionnaires: the practitioner’s guide to measuring outcomes after acquired brain impairment. East Sussex: Psychology Press; 2010.
- Wood Rl, Alderman N, Williams C. Assessment of neurobehavioural disability: a review of existing measures and recommendations for a comprehensive assessment tool. Brain Inj. 2008 Nov;22(12):905–918.
- Alderman N, Wood Rl, Williams C. The development of the St Andrew’s-Swansea Neurobehavioural Outcome Scale: validity and reliability of a new measure of neurobehavioural disability and social handicap. Brain Inj. 2011;25(1):83–100.
- Alderman N, Williams C, Knight C, et al. Measuring change in symptoms of neurobehavioural disability: responsiveness of the St Andrew’s – swansea Neurobehavioural Outcomes Scale (SASNOS). Arch Clin Neuropsychol. 2017 Dec 1; 32(8):951–962.
- Alderman N, Williams C, Wood Rl. When normal scores don’t equate to independence: recalibrating ratings of neurobehavioural disability from the ‘St Andrew’s – swansea Neurobehavioural Outcome Scale’ to reflect context-dependent support. Brain Inj. 2018;32(2):218–229.
- Turner-Stokes L, Williams H, Sephton K, et al. Engaging the hearts and minds of clinicians in outcome measurement – the UK rehabilitation outcomes collaborative approach. Disabil Rehabil. 2012 Nov;34(22):1871–1879.
- Worthington A, Wood Rl, McMillan T. Neurobehavioural disability over the past four decades. In: McMillan T, Wood Rl, Eds. Neurobehavioural disability and social handicap following traumatic brain injury. 2nd ed. Abingdon, Oxon: Psychology Press, Routledge. 2017;3-14.
- Independent Neurorehabilitation Providers Alliance. INPA outcomes study: evaluation and recommendations from INPA of a selection of measures used in the independent sector. London: Independent Neurorehabilitation Providers Alliance; 2018.