Abstract
Objective
To develop an item response theory (IRT) calibrated Grief and Loss item bank as part of the Spinal Cord Injury – Quality of Life (SCI-QOL) measurement system.
Design
A literature review guided framework development of grief/loss. New items were created from focus groups. Items were revised based on expert review and patient feedback and were then field tested. Analyses included confirmatory factor analysis (CFA), graded response IRT modeling and evaluation of differential item functioning (DIF).
Setting
We tested a 20-item pool at several rehabilitation centers across the United States, including the University of Michigan, Kessler Foundation, Rehabilitation Institute of Chicago, the University of Washington, Craig Hospital and the James J. Peters/Bronx Department of Veterans Affairs hospital.
Participants
A total of 717 individuals with SCI answered the grief and loss questions.
Results
The final calibrated item bank resulted in 17 retained items. A unidimensional model was observed (CFI = 0.976; RMSEA = 0.078) and measurement precision was good (theta range between −1.48 to 2.48). Ten items were flagged for DIF, however, after examination of effect sizes found this to be negligible with little practical impact on score estimates.
Conclusions
This study indicates that the SCI-QOL Grief and Loss item bank represents a psychometrically robust measurement tool. Short form items are also suggested and computer adaptive tests are available.
Introduction
Loss is a universal experience. Bereavement, the loss of a loved one, and grief, the distress resulting from bereavement,Citation1 are concepts that patients and their loved ones naturally use to understand reactions to catastrophic losses like spinal cord injury (SCI). However, questions remain about the applicability of grief and bereavement to understanding adjustment to SCI.Citation2 Most salient is that grief and bereavement classically focus on the loss of a loved one. Grief also has been used to characterize reactions to other losses, but the extent to which models of grief can be applied to SCI-related loss of physical capacity, social or occupational role function, and life goals remains an empirical question. Moreover, measuring grief and loss, the focus of this paper, must be distinguished from theories about the process of grief, especially those characterized by Kubler-Ross' stages of grief.Citation3
Despite its heuristic appeal, grief is an abandoned concept in SCI rehabilitation. Without empirical support, grief was thought to be universal after injury, required to achieve positive adjustment, and occurring in a particular way, commonly described as ‘stage theory.’Citation4,Citation5 Grief also lacked a strong foundation in measurement. Rehabilitation psychologists eventually moved towards the measurement of core psychopathological constructs, such as depressionCitation6 and post-traumatic stress disorder,Citation7 as well as positive psychological constructs, such as coping and appraisal.Citation8,Citation9
In the meantime, significant theoretical, empirical, measurement, and treatment advancements have occurred in the field of grief and bereavement. In terms of theory, Stroebe and ShutCitation10 have described the influential dual-process model of coping with bereavement. From this perspective, the bereaved are thought to oscillate between a focus on loss and a focus on restoration in daily activities. In a groundbreaking prospective study of spousal bereavement, Bonanno et al.Citation11 described five trajectories of responses to loss, including resilience as the modal response. Since then, similar trajectories have been demonstrated in response to other forms of trauma, including SCI.Citation12,Citation13 Newer conceptualizations of grief also suggest that the context of the loss, characterized by factors such as suddenness, unexpectedness, or violence, may be critical predictors of outcome.Citation14
Controversy remains about how to define concepts such as normal, complicated and prolonged grief. Nevertheless, there is growing evidence that pathological grief occurs in a significant minority of people who sustain loss and is distinct from ‘normal’ grief, anxiety, and depression.Citation15 Furthermore, pathological grief is associated with functional impairment, physiological changes, reduced quality of life, poor self-care, and elevated suicidal ideation and suicide attempts, after controlling for depression and anxiety.Citation15 Prigerson et al.Citation15 posit that key symptoms of pathological grief include yearning for what was, accompanied by other cognitive, emotional and behavioral symptoms, such as difficulty accepting the loss, emotional numbness, avoidance of reminders and confusion about one's role in life.
Distinguishing abnormal grief from depression also may have significant treatment implications. An influential clinical trial based on the dual process model of grief demonstrated that interpersonal therapy (IPT) alone was not as effective a treatment for complicated grief as was a combination of IPT and exposure therapy.Citation16 Making meaning of losses is another emerging dimension of grief and bereavement, particularly when the loss is sudden or unexpected.Citation17 The search for meaning after injury has been associated with positive adaptation to injury.Citation18 Qualitative work suggests a process of adaption and meaning-making, characterized by seeking a balance between holding onto pre-injury goals and adaptation to life after injury.Citation19
Given theoretical advances in the field of grief and bereavement, we speculated that these perspectives may also benefit our understanding of adjustment to losses associated with SCI. The concept of examining trajectories of adjustment has already been applied to people with SCI; however, with few exceptions,Citation20 assessment of adjustment to SCI has been limited to measures of depression and anxiety.Citation13,Citation21 Popular scales such as Core Bereavement Items,Citation22 Texas Revised Inventory of Grief,Citation23 or the Inventory of Complicated GriefCitation24 focus on the loss of a loved one. Construct validity of such scales is examined almost exclusively in respondents who have experienced the death of a loved one. As such, the usefulness and validity of such items is limited for persons with SCI, where losses are focused not on the loss of another, but on losses related to the self. To address the gap in the availability of relevant and valid measurement tools to measure grief processes after SCI, we sought to create a new item bank to capture features of grief/loss that were highly relevant to the losses experienced by persons with SCI.
Methods
This study was approved by the Institutional Review Board at all sites. The first study activity was to develop and refine a Grief and Loss item pool. Next, grief and loss items were administered to a large sample of people with SCI using a computerized data collection platform and interview format, so that each question was read to the respondent by a trained interviewer and responses were directly entered into the database. Each of these steps is described in detail in Tulsky et al.Citation25 and is also outlined briefly in the section below.
Development of a grief and loss item pool
The Grief and Loss item bank assesses emotional reactions or grief such as anger, guilt, anxiety, sadness, and despair. Items comprising the Grief and Loss item bank were drawn from various sources. We began by identifying candidate items from our initial pilot work, which included individual, semi-structured interviews and focus groups with individuals with SCI and clinicians with SCI (see Tulsky et al.Citation25,Citation26 for a full description). From these data, we developed a set of two preliminary items related to grief and loss. To develop these new items, specific phrases or concepts were drawn from the interviews and focus group transcripts (12 items from pilot interviews and 20 items from focus groups). For example, from the patient focus group quote, ‘[T]he first thing that came to my mind is, I guess, the things that you miss, you know, that you could do before. Yeah, even people say you can do them, you can do them in a different way, but I can't walk my dog effortlessly like I used to. If he gets loose, I can't just go over there and grab him and bring him back…’ we drafted the item, It made me sad to think about the things I used to enjoy. Additionally, one item from the Neuro-QOLCitation27 Stigma bank, I lost friends by telling them that I have this [injury], was originally categorized as a SCI-QOL Grief/Loss item raising the total pool to 33 items.
Expert item review (EIR), the first phase of the qualitative item review process used by the PROMISCitation28 and Neuro-QOLCitation27 project teams, was used to optimize this preliminary pool of 33 items. Items were considered candidates for deletion if they were redundant, overly generic, too specific, or if they failed to represent the construct or domain definition. Items that were verbose, double-barreled (i.e. one item with content related to two different issues), poorly or inconsistently worded, or contained high-level vocabulary were revised. For example, the item ‘It made me sad to think about the things I used to enjoy’ (mentioned above) was to read ‘I missed the activities I used to do.’ Based on EIR feedback, 30 items were retained in the preliminary Grief and Loss item pool. Preliminary items then underwent an additional phase of item review and modification by members of the investigative team. Items were arranged on a hierarchy of ‘difficulty’, from items indicating the lowest degree of grief and loss to the highest degree of grief and loss. Team members removed redundant items where there was oversaturation in the middle range of the hierarchy, and suggested new items to fill gaps in content coverage. Specifically, 3 items were deleted during this phase of review, 6 were moved to other item banks (e.g. Trauma, Stigma) and 5 new items were added.
We then conducted cognitive interviewsCitation29 with at least five individuals with SCI (see Introduction paper in this issue for details on the methodology) to assess item comprehension, decision making, and response retrieval processes. After cognitive interviewing, 7 items were removed and 4 items were modified. After this phase, the final 20 items were reviewed for translatability (for method, please see Eremenco et al.Citation30) and reading level (using the Lexile frameworkCitation31). Slight modifications were made to 3 items after the translatability and cultural review. For example, the item ‘I was upset about everything that has happened to me’ was changed to ‘I was overwhelmed by everything that has happened to me,’ since there is no generic word for ‘upset’ in some other languages such as Spanish. All items were written at or below the 5th grade reading level. A final item pool of 20 items was then field-tested.
Calibration study participants and data collection procedures
As a part of a large-scale, multi-site item calibration study (sites included the Kessler Foundation, University of Michigan, Rehabilitation Institute of Chicago, University of Washington, Craig Hospital and the James J. Peters/Bronx Veterans Administration hospital), we administered the 20 grief and loss items along with other item pools reflecting different emotional health subdomains to a sample of people with SCI.
The calibration sample included 717 participants with SCI. Inclusion criteria were 18 years of age and older, ability to read and understand English, and had a medically documented traumatic SCI. The sample was stratified by level (paraplegia versus tetraplegia), completeness of injury (complete vs. incomplete), and time since injury (<1 year, 1–3 years, and >3 years) to ensure that the final sample was a heterogeneous sample of individuals with SCI. Each participant's diagnosis was confirmed by medical record review; neurologic level was documented by their most recent American Spinal Injury Association Impairment Scale (AIS) rating.Citation32 To ensure a consistent mode of administration across participants, all items were presented in a structured interview format, either in person or over the phone. The context (time frame) for all items was, ‘In the past 7 days…,’ and the response options were Never/Rarely/Sometimes/Often/Always. A more detailed description of the study methodology and procedures is provided elsewhere in this issue.Citation25
Data analyses
Analysis involved confirmation of construct unidimensionality, use of a graded-response IRT model to calibrate item parameters, and examination of DIF. We used CFA to determine if our items conformed to a unidimensional model. Acceptable model fit indices were: CFI > 0.90, RMSEA < 0.08, good; CFI > 0.95, RMSEA < 0.06, excellent). Calibration was performed using iterative methods to reduce the item pool and obtain the best-fitting item parameters that would best allow estimation of a participant's standing on a trait of grief and loss. With each successive analytic iteration, we identified poorly fitting items by examining item fit to the 2-PL IRT model, DIF, local dependence between items (residual correlations >|0.15|), and significant loadings on the single factor (values >0.30). We then removed these items from the item pool and repeated the analytic steps. Once an acceptable solution was reached with CFA statistics that supported a unidimensional model, and all items showing misfit to the model or DIF were removed, the final IRT parameters were utilized to develop a computer adaptive test (CAT) algorithm for the Grief and Loss item bank. The CAT was programmed on the Assessment Center website (http://www.assessmentcenter.net) and can be administered directly from there. The item parameters were also used to select items for a static short form which can also be downloaded as a PDF from the Assessment Center website. Tulsky et al.Citation25 within this special issue described the detailed methodology and data analysis plan.
Development of short forms
To select items for short, fixed-length forms (as an alternative to CATs), project investigators reviewed item difficulty (item location) and slope (discrimination). As a starting point, items were divided into quintiles based on location; at each quintile, the first and/or second items with the highest slope were selected. Other considerations for item selection were clinical relevance, item wording and similarity to other items with the goal of having short form items as diverse as possible. Therefore, selection of items for short forms used both item statistics and qualitative characteristics.
Results
Participant characteristics
Demographic and injury characteristics are summarized in . Please see Tulsky et al.Citation33 introductory article within this special issue for additional details on the calibration sample, including education, income and mechanism of injury.
Table 1. Demographic and injury characteristics of the calibration sample
Preliminary analysis and item removal
Of the original 20 items that were tested, 3 were removed for the following reasons: local item dependence, low item-total correlation, and DIF for gender (Grief_31, “I cried when I was reminded of the abilities I used to have”), respectively. For the final 17 retained items, internal consistency was excellent (Cronbach's α = 0.947) and item-total correlations ranged from 0.59 to 0.78. All of the items but one had more than 20% of the sample selecting the first category of category of 1 (‘Never’). One case was deleted due to excessive missing data. No items had sparse data (fewer than five responses) in any category and no items had a category inversion. No further items were removed at this time.
Dimensionality
Using CFA, a unidimensional model was observed (CFI = 0.976; RMSEA = 0.078). The R2 values for all 17 of the items were greater than 0.40 In terms of local dependence, no item pairs were identified (i.e. residual correlations >|0.15|). Eigenvalue ratio (first to second) was 11.8.
IRT parameter estimation and model fit
Slopes ranged from 1.65 to 3.15; thresholds ranged from −1.48 to 2.48 (see ).
Table 2. Grief and Loss items and item bank parameters
The measurement precision in the theta range between −0.8 and 1.8 is roughly equivalent to a classical reliability of 0.95 or better ().
Figure 1. SCI-QOL Grief and Loss Item Bank Information and Precision.
The S-X2 model fit statistics were examined using the IRTFIT macro program. All items had adequate or better model fit statistics (P < 0.05), with marginal reliability equal to 0.947 and no item pairs were flagged (|r|> = 0.4) for local dependence.
Differential item functioning
DIF was examined using lordifCitation34 for six categories: age (≤49 vs. ≥50), sex (male n = 559 vs. female n = 158), education (some college and lower n = 523 vs. college degree and above n = 194), diagnosis (tetraplegia n = 388 vs. paraplegia n = 325), injury severity (incomplete n = 374 vs. complete n = 339), and time post injury (<1 year n = 196 vs. >1 year n = 521). Items were flagged for possible DIF when the probability associated with the χ2 test was <0.01 and the effect size measures (McFadden's pseudo R2) >0.02, which is a small but non-negligible effect. Overall, 10 of the final items were flagged for DIF in at least one category based on the chi-square test; however, when the effect size measures were examined, the DIF was negligible and all 17 items were retained in the final, calibrated item bank.
Short form selection and mode of administration
Once the SCI-QOL Grief and Loss item bank was finalized, all items and parameters were programmed into the Assessment CenterSM Citation35 platform and the bank is now freely available as a CAT. Since the purpose of calibrating items using IRT is that only a subset of items needs to be administered from a given bank in order to estimate an individual's score, there is flexibility as to how the items are selected and administered. On the Assessment Center platform, the CAT administration parameters can be modified to reduce standard error variance (e.g. maximize reliability), or to reduce test burden. There is also a predetermined static short form that can be downloaded. Finally, the individual items are present and could be selected if the end user wanted to administer a specific item. These administration options are reviewed below.
The SCI-QOL utilizes the same default CAT discontinue criteria as PROMIS; namely, the CAT minimum number of items to administer is four and the maximum is 12 with a maximum standard error of 0.3. In other words, in the default settings, the CAT will always administer at least 4 items, then will discontinue when the standard error of the individual's score estimate drops below 0.3 or a maximum of 12 items is reached (and the standard error variance criterion cannot be met).
Alternatively, the user could change the ‘discontinue criteria’ of the CAT so that it will administer additional items and obtain a more precise assessment of functioning. For instance, if the user selected an option that the CAT administers a minimum of 8 items before discontinuing, a lengthier test would be administered, but a more reliable score will be obtained. In some cases, greater precision over test burden is desirable based on factors such as resource allocation where specificity is critical.
However, in some cases it is neither possible (e.g. internet unavailable) nor practical (e.g. laptop/tablet computer equipment beyond budget of project) to administer items via CAT. To address this need, the Grief and Loss and other SCI-QOL item banks are also available as short forms. The project investigators utilized psychometric and clinical input to develop a fixed, 9-item ‘short form’ version of the Grief and Loss item bank. The goal of the short form selection process was to include the most informative items across a wide range of ‘difficulty’, or amount of the underlying trait. Since all items are calibrated on the same metric, scores on the short form are directly comparable to those on the CAT or full item bank. The correlation of the short form and various CATs with the full bank are given in . Short forms may be administered directly within Assessment Center, or may be downloaded for administration by paper and pencil or an alternate data capture platform or system. Individual investigators or clinicians could also develop additional, custom short forms, which could then be scored on the same IRT-based metric with the help of a psychometrician.
Table 3. Accuracy of variable- and fixed-length CAT and 9-item short form: correlations with full-bank score
To determine the degree of measurement precision and error for these assessments, we compared the reliability of the full bank, 9-item short form, and variable-length CAT with the default minimum of 4 items. When we compared the reliability of a CAT that was either fixed to 8 items, or a variable-length CAT with a minimum of 8 items, CAT values for both reliability () and precision () demonstrated improvement over the short form values. presents the mean, standard deviation, range, and standard error ranges for the various administration modes. Additionally, reliability curves for the full bank, short form, variable length CAT (minimum of 4 items) and fixed-length CAT (8 items) are given in .
Figure 2. Measurement reliability by T-Score and assessment method.
Table 4. Breadth of coverage for variable length CAT, fixed length CAT, 9-item short form, and full item bank
Scoring
SCI-QOL Grief and Loss scores are standardized on a T-metric, with a mean of 50 and a standard deviation of 10; this is based on the SCI-QOL calibration data; that is, a mean of 50 reflects the mean of an SCI population rather than the general population. All CAT administrations of the SCI-QOL Grief and Loss item bank are automatically scored by Assessment Center. When administering the short form, whether via Assessment Center, paper and pencil, or another data capture platform, an individual must complete all 8 component items in order to receive a score. The raw score for the short form is computed by simply summing the response scores for the individual component items. The T-score and associated standard error for each raw score value is given in .
Table 5. T-score lookup table for SCI-QOL Grief/Loss SF9a
Reliability
As a part of the reliability study described in the Tulsky et al.Citation25 methods paper in this issue, we compared Grief/Loss scores at Baseline with those from the 1-2 week retest assessment. In a sample of 245 individuals with SCI, Pearson's r = 0.84 and ICC (2,1) = 0.83 (95% CI = 0.78 to 0.87).
Discussion
We developed the SCI-QOL Grief and Loss item bank to assess an individual's subjective experience of losses that is relevant to people with SCI. This new item bank is a departure from the status quo of currently available grief measures that focus primarily on death of a loved one. In contrast, items in the new item bank reflect losses of a life before injury, vision of a future life, time and opportunities, and the self they used to be. All of the items in the Grief and Loss item bank were developed directly from verbatim interview or focus group quotes; no items were drawn from PROMIS or Neuro-QOL. As a result, the new SCI-QOL Grief and Loss item bank fills an important gap in existing grief and bereavement instruments as well as these important patient oriented outcome measurement initiatives.
The use of IRT to calibrate the SCI-QOL Grief and Loss items has yielded a variety of administration options including a short form and CAT. The CAT outperformed the short form in reliability and precisions; as such, if a user's goal is to optimize reliability, especially at the ceiling and floor of the distribution, we would recommend administering the Grief and Loss item bank as a CAT with a minimum of 8 items. In cases where it may not be feasible or practical to administer items via CAT/Assessment Center, or if having participants answer the same subset of 8 items is necessary to answer a given research question, we would recommend short form administration. An additional administration option is to administer both the CAT and any short form items not included in the CAT by using the ‘no duplicates’ option in Assessment Center. In this way, the user could optimize reliability and have the option of directly comparing individuals' responses on specific items to each other or to themselves over time.
Clinical applications
The flexibility of methods to administer the SCIQOL Grief and Loss item bank provides scientists and clinicians with an efficient and accessible way to integrate the measurement of grief and loss specific to SCI into research and ultimately clinical practice. Next, research is needed to examine the validity of this Grief and Loss measure. For example, studies are needed to determine how grief symptoms change over time and whether grief and loss can be discriminated from depression and anxiety in people with SCI. It will also be important to determine whether the magnitude or persistence of grief and loss predict other important outcomes such as quality of life, functional impairment, and self-care after controlling for depression, anxiety and other salient variables. If, as is the case with other type of loss,Citation14 there is a significant minority of persons who have prolonged, disabling grief symptoms, tailored grief treatment approaches should be studied.Citation16
The integration of this measure in clinical practice can be particularly useful for guiding therapy to target specific areas of difficulty. For example, losses of a future, loss of a sense of self, or acceptance of injury all have important implications for tailoring therapy to individual needs. Use of the measure in clinical settings can also aid the clinician for tracking change over time to assess the effectiveness of therapy and guide changes in their approach. By design, interpretation of SCI-QOL scores is simplified by a standardized metric that allows the clinician to readily identify how the individual compares to the broader SCI population. Finally, the use of this new measure can aid clinicians in the distinguishing grief and loss from depression to guide treatment.
Conclusion
The final SCI-QOL Grief and Loss item bank contains 17, IRT-calibrated items. This item bank is the first, to our knowledge, to specifically capture grief and loss after SCI. Because of the flexibility of IRT-based measures, the use of CATs is also possible with this item bank, enabling researchers and clinicians to administer only the most precise and informative items based on an individual's responses. This has important implications for the use of such innovative applications in symptom monitoring and self-management in post-acute care settings and community living.
To the best of our knowledge, this is the first time that a patient-centered, modern measurement theory derived approach has been used to develop and test a grief and loss self-reported measurement tool specifically for individuals with SCI. Our work using focus groups and interviews strengthened our understanding of grief and loss as it is experienced in the context of SCI. While this work has made great strides forward, there remains work to be done to more comprehensively define grief and loss in this population. This new item bank makes an important first step towards reliably measuring the construct of grief and loss in persons with SCI. Such measurement will significantly contribute to the development of conceptual models. Future work will also include establishing discriminant validity from depression and other grief measures to further ensure that the SCI-QOL Grief and Loss item bank captures the unique losses experienced after injury. The knowledge gained from the use of this new measure will be essential for the development of grief and loss conceptual models to inform treatment approaches to maximize HRQOL after SCI.
Disclaimer statements
Contributors All authors have contributed significantly to the design, analysis and writing of this manuscript. The contents represent original work and have not been published elsewhere.
Funding This study was supported by grant #5R01HD054659 from the National Institutes of Health – Eunice Kennedy Shriver National Institute of Child's Health and Human Development/National Center on Medical Rehabilitation Research and the National Institute on Neurological Disorders and Stroke.
Conflicts of interest No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit upon the authors or upon any organization with which the authors are associated. All SCI-QOL items and parameters are © 2015 David Tulsky and Kessler Foundation. All rights reserved. All items are freely available to the public via the Assessment Center platform (http://www.assessmentcenter.net). There are currently no plans for Dr. Tulsky or Kessler Foundation to benefit financially from the copyright.
Ethics approval The Institutional Review Board at each site reviewed and approved this project.
References
- Ayers T, Balk D, Bolle J, Bonanno GA, Connor SR, Cook AS, et al. Report on bereavement and grief research. Death Stud 2004;28(6):491–575.
- Niemeier JP, Burnett DM. No such thing as ‘uncomplicated bereavement’ for patients in rehabilitation. Disabil Rehabil 2001;23(15):645–53.
- Kubler-Ross E. On death and dying, London, UK: Routelage; 1969.
- Frank R, Elliott T, Corcoran J, Wonderlich S. Depression after spinal cord injury: is it necessary? Clin Psychol Rev 1987;7(6):611–30.
- Wortman CB, Silver RC. The myths of coping with loss. J Consult Clin Psychol 1989;57(3):349–57.
- Elliott TR, Frank RG. Depression following spinal cord injury. Arch Phys Med Rehabil 1996;77(8):816–23.
- Radnitz CL, Hsu L, Tirch DD, Willard J, Lillian LB, Walczak S, et al. A comparison of posttraumatic stress disorder in veterans with and without spinal cord injury. J Abnorm Psychol 1998;107(4):676–80.
- Buckelew SP, Baumstark KE, Frank RG, Hewett JE. Adjustment following spinal cord injury. Rehabil Psychol 1990;35:101–9.
- Kennedy P, Lowe R, Grey N, Short E. Traumatic spinal cord injury and psychological impact: a cross sectional analysis of coping strategies. Br J Clin Psychol 1995;34:627–39.
- Stroebe M, Schut H. The dual process model of coping with bereavement: rationale and description. Death Stud 1999;23(3):197–224.
- Bonanno GA, Wortman CB, Lehman DR, Tweed RG, Haring M, Sonnega J, et al. Resilience to loss and chronic grief: a prospective study from preloss to 18-months postloss. J Pers Soc Psychol 2002;83(5):1150–64.
- Bonanno GA, Westphal M, Mancini AD. Resilience to loss and potential trauma. Annu Rev Clin Psychol 2011;7:511–35.
- Bonanno GA, Kennedy P, Galatzer-Levy IR, Lude P, Elfstrom ML. Trajectories of resilience, depression, and anxiety following spinal cord injury. Rehabil Psychol 2012;57(3):236–47.
- Bonanno GA, Kaltman S. Toward an integrative perspective on bereavement. Psychol Bull 1999;125(6):760–76.
- Prigerson HG, Horowitz MJ, Jacobs SC, Parkes CM, Aslan M, Goodkin K, et al. Prolonged grief disorder: psychometric validation of criteria proposed for DSM-V and ICD-11. Plos Medicine. 2009;6(8):e1000121.
- Shear K, Frank E, Houck PR, Reynolds CF. Treatment of complicated grief – a randomized controlled trial. JAMA 2005;293(21):2601–8.
- Davis CG, Nolen-Hoeksema S. Loss and meaning: how do people make sense of loss? Am Behav Sci 2001;44(5):726–41.
- de Roon-Cassini TA, de St. Aubin E, Valvano A, Hastings J, Horn P. Psychological well-being after spinal cord injury: perception of loss and meaning making. Rehabil Psychol. 2009;54(3):306–14.
- Angel S, Kirkevold M, Pedersen BD. Getting on with life following a spinal cord injury: regaining meaning through six phases. Int J Qual Stud Health Well-being 2009;4(1):39–50.
- Niemeier J, Kennedy R, McKinley W, Cifu D. The loss inventory: preliminary reliability and validity data for a new measure of emotional and cognitive responses to disability. Disabil Rehabil. 2004;26(10):614–23.
- Quale AJ, Schanke AK. Resilience in the face of coping with a severe physical injury: a study of trajectories of adjustment in a rehabilitation setting. Rehabil Psychol. 2010;55(1):12–22.
- Burnett P, Middleton W, Raphael B, Martinek N. Measuring core bereavement phenomena. Psychol Med. 1997;27(1):49–57.
- Faschingbauer TR, Devaul RA, Zisook S. Development of texas inventory of grief. Am J Psychiatry. 1977;134(6):696–7.
- Prigerson HG, Maciejewski PK, Reynolds CF, Bierhals AJ, Newsom JT, Fasiczka A, et al. Inventory of complicated grief: a scale to measure maladaptive symptoms of loss. Psychiatry Res. 1995;59(1–2):65–79.
- Tulsky DS, Kisala PA, Victorson D, Choi SW, Gershon R, Heinemann AW, et al. Methodology for the development and calibration of the SCI-QOL item banks. J Spinal Cord Med. 2015;38(3):270–87.
- Tulsky DS, Kisala PA, Victorson D, Tate D, Heinemann AW, Amtmann D, et al. Developing a contemporary patient-reported outcomes measure for spinal cord injury. Arch Phys Med Rehabil. 2011;92(10):S44–51.
- Cella D, Nowinski C, Peterman A, Victorson D, Miller D, Lai J-S, et al. The neurology quality-of-life measurement initiative. Arch Phys Med Rehabil. 2011;92 (10, Supplement):S28–36.
- DeWalt DA, Rothrock N, Yount S, Stone AA. Evaluation of item candidates: the promis qualitative item review. Med Care. 2007;45 (5 Suppl 1):S12–21.
- Willis GB. Cognitive interviewing: a tool for improving questionnaire design. Thousand Oaks, CA: Sage Publications; 2005.
- Eremenco SL, Cella D, Arnold BJ. A comprehensive method for the translation and cross-cultural validation of health status questionnaires. Eval Health Prof. 2005;28(2):212–32.
- MetaMetrics. The lexile framework for reading. Durham, NC: MetaMetrics Inc.; 1995.
- American Spinal Injury Association. Reference manual for the international standards for neurological classification of spinal cord injury. Chicago, IL: American Spinal Injury Association; 2003.
- Tulsky DS, Kisala PA, Victorson D, Tate DG, Heinemann AW, Cella DF. Overview of the Spinal Cord Injury – Quality of Life (SCI-QOL) measurement system. J Spinal Cord Med. 2015;38(3):257–69.
- Choi SW, Gibbons LE, Crane PK. Lordif: an r package for detecting differential item functioning using iterative hybrid ordinal logistic regression/item response theory and monte carlo simulations. J Stat Softw. 2011;39(8):1–30.
- Gershon R, Rothrock NE, Hanrahan RT, Jansky LJ, Harniss M, Riley W. The development of a clinical outcomes survey research application: assessment center. Qual Life Res. 2010;19(5):677–85.