The Spinal Cord Injury – Quality of Life (SCI-QOL) measurement system: Development, psychometrics, and item bank calibration

Though spinal cord injury (SCI) was historically regarded as an ailment not to be treated,¹ medical, pharmaceutical, and technological advances in the 20th and 21st centuries have improved acute and long-term SCI rehabilitation outcomes. Consequently, SCI has become an increasingly common cause of long-term disability, with over 250 000 Americans^2,3 and over 85 000 Canadians⁴ living with SCI. Traumatic SCI is a catastrophic injury that changes the lives of individuals in a split second. SCI is characterized by a broad and unique set of functional limitations and secondary complications that affect physical (e.g. altered urinary and bowel function,^5–7 pressure ulcers,^8,9 chronic and neuropathic pain) cognitive,^10,11 emotional (e.g. depression,^12,13 anxiety disorders),¹⁴ and social (e.g. unemployment)¹⁵ areas of health and functioning. Individuals who sustain SCI must adjust immediately to a new way of life that is often characterized by significant physical limitations, alterations to basic physiological functions, intense emotions, disruption of social relationships, and barriers to participating in their usual activities – essentially, every possible area of health-related quality of life (HRQOL). Individuals with SCI have described the secondary complications of SCI to be equally or even more troublesome than the primary functional limitations of SCI, such as the inability to walk.¹⁶ Furthermore, SCI is heterogeneous because the associated functional impairments and secondary medical issues are directly related to the location and neurological completeness of injury. An individual who sustains an American Spinal Injury Association (ASIA) Impairment Scale (AIS) grade D injury may be able to walk unassisted, while an individual with high-level and complete (AIS grade A) tetraplegia will be unable to move below the neck and will require constant mechanical ventilation. Due to the suddenness and severity of SCI, the wide range of potential secondary complications, and the diversity of functioning and complications within the population of individuals with SCI, healthcare professionals must assess a wide variety of areas of functioning, examine changes over time, and identify and mitigate potential risk factors. To do so, the healthcare provider must be able to measure and monitor a wide variety of issues that a person with SCI might experience. Until now, there have not been the proper tools to do so.

The lack of available SCI-relevant measurement instruments to conduct standardized, effective assessment of a wide variety of HRQOL domains has been rather disheartening. In 2001, Tulsky¹⁷ chaired a state-of-the-science conference for rehabilitation professionals that focused on the current state of quality of life measurement for individuals with disabilities. Several keynote addresses,^18,19 as well as Tulsky and Rosenthal's synthesis¹⁷ of the conference, pointed out that, by and large, when HRQOL variables were utilized in clinical trials, rehabilitation researchers, including specialists in SCI medicine, were forced to use existing general scales that were developed and intended for the general population. These measurement tools did not capture areas of functioning that were important to individuals with physical disabilities, and often contained items that were irrelevant, inappropriate, or even offensive.^20–22 Unfortunately, despite the flaws in these measurement tools for use with persons with disabilities, there was simply no alternative at that time. Outcomes measurement, in general, had not received the same level of attention and methodological rigor in rehabilitation as it had received in other fields of medicine.

In their summary of the conference talks, Tulsky and Rosenthal²³ outlined a vision to improve rehabilitation outcomes measurement. This vision included John Ware's discussion of state-of-the-art assessment models using item response theory (IRT) that could provide flexible, dynamic and practical assessments,²⁴ along with a paper providing detailed illustrations on how test items could be developed as ‘item banks’ for use across different groups of persons with disabilities.²⁵ Though the vision was there, it was unclear if there would be support to implement such a strategy for individuals with SCI and other disabilities.

At the same time, rapid advances were taking place in the healthcare field in general. The first decade and a half of the new millennium was quickly establishing itself as a golden age for measurement, as new measurement methods and initiatives for healthcare and clinical trials research were being introduced. State-of-the-art measurement strategies from other fields such as education were being implemented in health research settings. In 2004, the National Institutes of Health (NIH) Common Fund (then called the NIH Roadmap) established the Patient Reported Outcomes Measurement Information System® (PROMIS®)²⁶ and the National Institute of Neurological Disorders and Stroke (NINDS) embarked on a critical path to develop the Quality of Life in Neurological Disorders Measurement System (Neuro-QOL).²⁷ The overarching goal was to develop state-of-the-art measurement scales to be used (for PROMIS®) across medical populations and (for Neuro-QOL) across individuals with Neurological Disorders. Unfortunately, these new initiatives did not target individuals with SCI.

Given the unique constellation of SCI-related symptoms, secondary complications, and potential alterations to social and emotional functioning, Dr. Tulsky and colleagues secured 2 grants to embark on a research initiative to fill this measurement gap. Dr. Tulsky received funding (in the form of an R-01) from the Eunice Kennedy Shriver National Institute on Child Health and Human Development/National Center on Medical Rehabilitation Research and the NINDS, as well a separate line of funding (in the form of a Model Systems Collaborative ‘Modular’ project) from the National Institute on Disability and Rehabilitation Research's Spinal Cord Injury Model System (SCIMS) program. An extensive network of collaboration between SCIMS, PROMIS, and Neuro-QOL investigators ensued and the research group set out to develop a psychometrically advanced measurement system that would be tailored for individuals with SCI and appropriate for use in both research and clinical settings. Ten years after the initial conference that Dr. Tulsky chaired, Tulsky, Carlozzi, and Cella²⁸ painted a very different picture of the state of rehabilitation outcomes measurement. In contrast to the ‘doom and gloom’ picture from a decade earlier, Tulsky and colleagues reported that emerging, state-of-the-art measurement strategies were having a significant impact on the field.^26,27,29,30 Tulsky and colleagues reported that new measurement initiatives, designed specifically for individuals with SCI (as well as for individuals with other chronic disabilities, such as traumatic brain injury) would provide SCI researchers and clinicians with valid and reliable outcome measures that addressed subjectively important issues to individuals with SCI.^16,31–34 No longer was outcomes measurement an afterthought in SCI medicine. Instead, SCI outcomes researchers were leading the initiatives, including the Spinal Cord Injury – Quality of Life (SCI-QOL), which would transform outcomes measurement for rehabilitation research and practice.

The series of manuscripts that make up this special issue represent the culmination of the initial development of these SCI-specific item banks. These articles describe the detailed development work and psychometric calibration of the majority of SCI-QOL item banks. The purpose of this special issue is both to formally introduce the SCI-QOL to the field, and also to serve as a technical manual for use of the SCI-QOL item banks. The first manuscript in the issue provides a conceptual overview of the goals and outputs of the SCI-QOL project. Following the introductory overview,³⁵ Tulsky, Kisala, Victorson, Choi, Gershon, Heinemann, and Cella³⁶ provide a thorough description of the development and calibration methodology used across all SCI-QOL banks. They provide a description of five sequential phases of the SCI-QOL development work, each with unique goals, samples, and methods. Their paper provides detail on the research methodology that was used for the majority of manuscripts that follow in the special issue.

Following these introductory manuscripts, two manuscripts introduce SCI-QOL measures of secondary medical complications of SCI. First, Tulsky, Kisala, Tate, Spungen, and Kirshblum³⁷ present the development of item banks to measure bowel management difficulties and bladder management difficulties, and a short scale to measure bladder complications. Next, Kisala, Tulsky, Choi, and Kirshblum present an IRT-calibrated scale to measure the impact of pressure ulcers on quality of life.

The next set of 8 manuscripts present the development of the SCI-QOL item banks related to emotional health following SCI. Kisala, Tulsky, Kalpakjian, Heinemann, Pohlig, Carle, and Choi³⁸ present the SCI-QOL Anxiety item bank, which is a version of the PROMIS scale that has been tailored and optimized for the SCI population. The authors linked the SCI-QOL Anxiety scale with a frequently used brief measurement scale to assess anxiety in the general population (the Generalized Anxiety Disorder scale – 7 item version³⁹; GAD-7). Post-traumatic stress, or psychological panic reactions due to traumatic events, have traditionally been classified as anxiety disorders. However, they have recently been reclassified as Trauma and Stress-or-Related Disorders under the new psychiatric/psychological classification system⁴⁰ to reflect psychological reactions to a trigger that result from external traumatic events, such as exposure to actual or threatened death or serious injuries. An article by Kisala, Victorson, Pace, Heinemann, Choi, and Tulsky⁴¹ presents the SCI-QOL Trauma scale and will mark the first time that a scale has been developed to track this emotional reaction in individuals with SCI. The next two manuscripts are related to depressed feelings. Tulsky, Kisala, Kalpakjian, Bombardier, Pohlig, Heinemann, Carle, and Choi⁴² present the SCI-QOL Depression bank, which is an optimized version of the PROMIS scale for the SCI population. Given the common use of the Patient Health Questionnaire – 9 (PHQ-9)^43,44 in individuals with SCI, the authors also report on the use of item response theory methods to convert scores on the PHQ-9 to SCI-QOL Depression scores. Qualitative input from individuals with SCI early in SCI-QOL development process led Tulsky and colleagues to develop a bank of items to assess the emotional components of a grief reaction, stemming from a sudden loss of functioning and difficulty adjusting to life, which differs from the traditional construct of depression. Kalpakjian, Tulsky, Kisala, and Bombardier⁴⁵ present a new item bank to measure grief and loss after SCI. Next, two articles focus on positive psychological variables and emotional states. Bertisch, Kalpakjian, Kisala, and Tulsky⁴⁶ present the SCI-QOL Positive Affect and Well-being item bank. This is a version of the Neuro-QOL Positive Affect and Well-being bank that has been optimized specifically for individuals with SCI and will provide researchers and clinicians with an efficient way to integrate constructs of positive affect and emotional well-being into SCI research and clinical practice. During the qualitative stage of the SCI-QOL's development, individuals with SCI described the necessity of accepting their injury and moving on with their ‘new’ life—of not just turning over a new page in life, but starting an entirely new book.¹⁶ The feedback underscored the importance of resilience following a traumatic injury such as SCI, and Victorson, Tulsky, Kisala, Kalpakjian, Weiland, and Choi⁴⁷ present the development of the SCI-QOL Resilience item bank. The final two item banks focus on the feelings of appraisal, judgement, or stigmatization that individuals with SCI experience. Self-esteem refers to the cognitive, emotional, and evaluative perceptions of the self. Following SCI, individuals might experience self-generated negative emotions about themselves secondary to their injury. Kalpakjian, Tate, Kisala, and Tulsky⁴⁸ present the SCI-QOL Self-esteem item bank. Parallel to such internal appraisals and due to the visible nature of a physical disability like SCI, there can be a stigmatizing effect of injury, especially in social settings. To measure these effects in individuals with SCI, Kisala, Tulsky, Pace, Victorson, Choi, and Heinemann⁴⁹ describe the development and calibration of the SCI-QOL Stigma item bank. Several items were derived from the Neuro-QOL Stigma item bank and the SCI-QOL bank was placed on the Neuro-QOL metric.

The final manuscripts report on the SCI-QOL measures of social and physical functioning. Heinemann, Kisala, Hahn, and Tulsky⁵⁰ report on the SCI-QOL Ability to Participate in Social Roles and Activities and the SCI-QOL Satisfaction with Social Roles and Activities item banks. Both of these item banks have utilized the items from the Neuro-QOL but have optimized the item banks for individuals with SCI. The final article, by Jette, Slavin, Ni, Kisala, Tulsky, Heinemann, Charlifue, Fyffe, Tate, Morse, Marino, Smith, and Williams.⁵¹ presents new enhancements to the SCI-QOL physical functioning scales (i.e. Spinal Cord Injury – Functional Index; SCI-FI).^31,52 These item banks were developed using a separate (though parallel) sample and methodology and focus on how individuals perform activities with the use of assistive technology (AT). In contrast to the original SCI-FI items which ask participants about performing activities ‘without any devices or assistance,’ this paper by Jette et al. describes the development of the SCI-FI/AT – an enhancement of the original SCI-FI item banks to reflect the use of assistive technology when performing physical functions.

Each manuscript in this special issue presents one or more SCI-QOL item banks and reviews the included construct(s), item development/selection and reduction, item response theory analyses and further item reduction, calibration data, and brief, fixed-length ‘short form’ versions that have been developed for each bank. The SCI-QOL item banks provide a much richer assessment of functioning than traditional assessments and have included direct input from individuals with SCI throughout the development process. In each manuscript, we have presented all of the technical information related to the presented bank including the IRT calibration parameters (slope and threshold values), which can be useful if others want to develop a customized short forms or program a stand-alone computer adaptive test for a given item bank.

This series of manuscripts describes much of the SCI-QOL development work. However, as indicated by Tulsky, Kisala, Victorson, Choi, Gershon, Heinemann, and Cella,³⁶ there are a few item pools/banks that have not been included in this special issue. Most notably, a detailed description of the SCI-QOL Pain Interference item bank and Pain Behavior scale, and the related calibration data, are not included here. Similarly, the development and calibration of the original 5 SCI-FI physical functioning item banks (i.e. Basic Mobility, Fine Motor function, Self-Care, Wheelchair Mobility, and Ambulation) have already been reported^31,52 and as such are not included here. A SCI-QOL Independence item bank has been developed and is available on Assessment Center, though there is additional scaling work that has yet to be performed and it has not been included in this special issue. Finally, there are other pools of items that have been tested but not calibrated (e.g. Respiratory and Sexual Functioning subdomains). Due to the distribution of responses to these items, there was insufficient data to analyze these items with graded response model IRT analyses. Therefore, the Respiratory and Sexual Functioning item banks have not been calibrated and their description is beyond the scope of this special issue.

This special issue represents the contributions of many individuals. We extend our profound appreciation to the following individuals for their participation on this ambitious study.

First and foremost, we thank Dr. David Cella and Dr. David Victorson from the Northwestern University Department of Medical Social Sciences (MSS) for close partnership on this project from its initial stages through the entire item development and calibration process. Drs. Cella and Victorson transferred knowledge from the PROMIS and Neuro-QOL initiatives and ensured that we were conducting state-of-the-art measurement work in every aspect of the project. Dr. Victorson moderated all focus groups and could be counted on to assist us at every step along the way. Dr. Seung Choi and his team of psychometricians, Natalie McKinney and Tracy Podrabsky, conducted all of the item response theory analyses using the most meticulous procedures (and creating new procedures when there was no guide). Dr. Richard Gershon led the efforts to program the SCI-QOL item banks into Assessment Center assisted by several collaborators, including Dr. Nan Rothrock, Michael Bass, Maria Varela Diaz, Manpreet Lakhan, Monica Prudencio, and the entire MSS information technology team. Excellent technical support for our team's use of Assessment Center was provided by Warren Francis and Odessa Castro. Vitali Utsinovich provided materials and helped us understand the Neuro-QOL item banks and procedures. Jin-Shei Lai provided additional psychometric help and expertise. Dr. Cella, himself, provided inspiration for this project and provided the resources for the MSS scientists and staff and we remain eternally grateful.

Equally important were the contributions from all of our co-investigators from the SCI Model Systems for their hard work and significant contributions throughout the research project. We heartily thank Dr. Steven Kirshblum from the Kessler Institute of Rehabilitation and Dr. Trevor Dyson-Hudson, Dr. Denise Fyffe, and Dr. Amanda O'Brien Milleisen from the Kessler Foundation Research Center; Dr. Denise Tate, Dr. Claire Kalpakjian, and Martin Forchheimer from the University of Michigan; Dr. Susie Charlifue from Craig Hospital; Dr. Allen Heinemann from the Rehabilitation Institute of Chicago; Dr. Charles Bombardier and Dr. Dagmar Amtmann from the University of Washington; Dr. Alan Jette, Dr. Mary Slavin, and Dr. Pensheng Ni from Boston University; and Dr. Marcel Dijkers and Dr. Jeanne Zanca from Mount Sinai School of Medicine. We could not have dreamed of a better group of co-investigators with whom to have partnered over the last 9 years.

For a project of this magnitude, each institution brought their own army of data collectors and staff working effortlessly day after day on this project. At Kessler Foundation, Donna Servidio managed administrative aspects of the study and LeighAnn Martinez identified potential participants and assisted with all recruitment efforts. Data collectors included Rachel Byrne, Julia Stoumbos, Caitlin Seifert Miranda, Joseph McCabe, Erinn Stivala Nakahara, Michael Pino, Arielle Teitcher, Adrianna Maldonado, Brianne Smith, Heather McGowan, Katherine Czado Aquino, and Lila Inglima-Pereira; at RIC, the data collectors included Sara Jerousek, Ana Miskovic, Dustin Williams, Kyle Seanor, Brian Weiland, and Martha Bailey; at University of Michigan, Rachel Tocco helped with Assessment Center and data collectors included Jane Walters, Kate Donnelly, Siera Goodnight, Rachel Hartwig, Angela Miciura, Laury Elwell, Sam Leaf, and Sonya Sutherland; at Craig Hospital, data collectors included Amy Dannels-McClure, Susan Solnick, and Caroline Rose; and at University of Washington, Rana Salem helped with project coordination and data collectors included Kara Bogusz, Thayer Wild, Missy Takahashi, Matt Smith, and Meighan Rasley. At Boston University, data collectors included Diana Pernigotti and Vanessa Oliveira, and at Mt. Sinai, the data collection team included Marilyn Gomez, Jeannie Chan, Mila Babaev, Michelle Dziedzic, and Rana Searfoss. To develop the customized computer software that was utilized in the SCI-QOL calibration study, we thank Kunal Jain from VisionStream LLC.

We are very appreciative of the backing of the Kessler Foundation especially when this project was being developed. We thank Rodger DeRose, Dr. Mitchell Rosenthal, and Dr. Joel DeLisa, all of whom ensured that we have the appropriate financial and personnel support in the early phases of the project that established a strong foundation from which we could successfully complete a project of this magnitude. Matt Weiner and John Giraud provided incredible support from the business office to manage the project through its early years. Similarly, we thank our colleagues from the James J. Peters Department of Veterans Affairs Medical Center in the Bronx, NY: Dr. Ann Spungen, Kel Morin, and Shevana Swaby, for ensuring that the input of veterans was included so that their needs would be accounted for in the measurement system. Dr. Spungen is leading a VA sponsored study to validate SCI-QOL in a VA population.

Several investigators and staff worked on the early measurement initiatives that prepared us for future SCI-QOL funding and are deserving of our gratitude. Dr. Carol Miklos conducted our initial pilot interviews with individuals with SCI and Dr. Tamara Mills and Dr. Sandra Mercedes prepared initial item banks as we began to learn what issues were most important to individuals with SCI. Rachel Gold Tadduni, Kate Francis Hardy, and Amy Bullman Giles helped test these items in new pilot studies. This work set the foundation for SCI-QOL and helped us ‘hit the ground running’ once funding was secured. We would also like to thank the members of the Northern New Jersey SCI System Community Advisory Board who provided invaluable input throughout the first several years of the project.

Dr. Rita Bode, Dr. Adam Carle, and Dr. Ryan Pohlig provided psychometric and statistical help in the later stages of the project. Others who helped at various stages include Jerry Wright who helped with participant tracking, Dr. Jerry Slotkin who helped coordinate writing of the articles when the special issue was just being proposed, Brad Trumpower and Shale Maulana who helped prepare many of the tables included in this issue, Emily Buchanan who helped finalize the first set of manuscripts for submission, and Dr. Barton Palmer for conducting a preliminary peer review of some of the manuscripts. Similarly, we are grateful to Dr. Matthew Cohen for diligently reading all manuscripts and providing us with invaluable editorial comments. At the Journal of Spinal Cord Medicine, we would like to thank Dr. J. Scott Richards for serving as a guest editor for this issue and Stephen Cavanaugh for the hands-on management and editing of the issue. Finally, and most importantly, we would like to thank the approximately 1500 individuals with SCI who participated in the development, calibration, and reliability assessment phases of the study. The primary goal of the SCI-QOL was to ensure conceptual relevance by including input from individuals with SCI at every stage of the project. We will be forever indebted to the study participants who gave of their time and themselves by sharing their personal accounts of living with SCI, giving the research team a truly honest opinion of all of the preliminary items, and responding to seemingly endless sets of interview questions about some of their most personal feelings and experiences.