Abstract
Objective
To develop a spinal cord injury (SCI)-focused version of PROMIS and Neuro-QOL social domain item banks; evaluate the psychometric properties of items developed for adults with SCI; and report information to facilitate clinical and research use.
Design
We used a mixed-methods design to develop and evaluate Ability to Participate in Social Roles and Activities and Satisfaction with Social Roles and Activities items. Focus groups helped define the constructs; cognitive interviews helped revise items; and confirmatory factor analysis and item response theory methods helped calibrate item banks and evaluate differential item functioning related to demographic and injury characteristics.
Setting
Five SCI Model System sites and one Veterans Administration medical center.
Participants
The calibration sample consisted of 641 individuals; a reliability sample consisted of 245 individuals residing in the community.
Results
A subset of 27 Ability to Participate and 35 Satisfaction items demonstrated good measurement properties and negligible differential item functioning related to demographic and injury characteristics. The SCI-specific measures correlate strongly with the PROMIS and Neuro-QOL versions. Ten item short forms correlate >0.96 with the full banks. Variable-length CATs with a minimum of 4 items, variable-length CATs with a minimum of 8 items, fixed-length CATs of 10 items, and the 10-item short forms demonstrate construct coverage and measurement error that is comparable to the full item bank.
Conclusion
The Ability to Participate and Satisfaction with Social Roles and Activities CATs and short forms demonstrate excellent psychometric properties and are suitable for clinical and research applications.
Introduction
Participation in social roles and activities, a long term outcome following rehabilitation, is valued highly by persons living with the consequences of spinal cord injury (SCI). Health insurance reform legislation emphasizes the need to measure the long term outcomes of rehabilitation services using standardized measures; such measures are critical to new delivery and payment reform models, including Accountable Care Organizations, bundling demonstrations, the Continuing Care Hospital concept, and the Independence at Home Demonstration program. Needed are long term outcome measures to improve the quality of healthcare and reduce cost without sacrificing care. Thus, this report describes the development of standardized measures of long term rehabilitation outcomes focused on participation in social roles and activities and satisfaction with participation in social roles and activities.
Participation measurement
The World Health Organization defines participation limitations within the context of the International Classification of Disability and Health (ICF). Participation is defined by many rehabilitation researchers as chapters 6 to 9 of ICF's Activity and Participation domain, those pertaining to Domestic Life, Interpersonal Interactions and Relationships, Major Life Areas, and Community, Social and Civic Life,Citation1 and the definition we adopt in this report. Participation measurement is complicated by the World Health Organization's decision to combine activities and participation in a single classification making it impossible to distinguish performance at the individual level (activity) and performance at the societal level (participation). Needed are reliable and valid measures of participation that allow the separation of objective participation performance and subjective satisfaction with participation.
Research on participation measurement
Magasi and PostCitation2 reviewed the content of contemporary participation measures. They identified 8 participation measures developed using classical test theory methods that demonstrated moderate to good validity and reliability, though reporting of measurement information was often incomplete. The measures most often assessed the ICF domains of mobility; domestic life; social interactions; major life domains; and community, social, and civic life.
DijkersCitation3 noted that participation is defined frequently as the primary valued outcome of rehabilitation, but that vague and incomplete definitions and inadequate measurement of participation severely limit clinicians’ and investigators’ ability to measure participation outcomes. He described several major issues related to participation measurement and offers conceptual and methodological guidance.
WhiteneckCitation1 observed that the large and growing number of participation measures limits rehabilitation research and practice. He described desirable types and characteristics of participation measures and made recommendations for future participation research. He noted that participation measures vary in regards to specificity, conceptual models, development approach, psychometric properties, dimensionality, objective vs. subjective perspectives, and use of norms He encouraged development of better participation measures built on consensus regarding conceptual models.
Over the last ten years, the National Institutes of Health (NIH) has supported the development of instruments to provide common data elements for clinical research.Citation4,Citation5 The Patient Reported Outcomes Measurement Information System (PROMIS)Citation6 is a family of instruments that measure patient-reported health status.Citation4,Citation5 The National Institutes of Neurological Disorders and Stroke has taken the lead in developing core data sets for neurologic populations, including epilepsy, stroke, and Parkinson's disease among other conditions through the Quality of Life in Neurological Disorders (Neuro-QOL) project.Citation7 Neuro-QOL provides a conceptual framework and a common language for clinicians and investigators.
The PROMIS Social Health Workgroup conducted a systematic review of social health definitions, content, and item wording, and implemented a qualitative item review process that included identification of items, development of new items, focus group exploration of domain coverage and cognitive interviews.Citation8–Citation10 Validation testing in diverse samples (n = 2,208 English; n = 644 Spanish) resulted in psychometrically-sound and culturally-appropriate measures of social function, including Ability to Participate in Social Roles and Activities (SRA), and Satisfaction with SRA.Citation10 Work also began on social relationships, focusing on social support and social isolation.Citation8–Citation11 The extent to which these concepts and measures apply to persons with SCI has not been evaluated.
Study objectives
The objectives of this study were to (1) develop item banks of social functioning that are relevant to persons with SCI; (2) link the item banks to either the PROMIS or Neuro-QOL (or both) through common items and transform the score to the PROMIS or Neuro-QOL metric; (3) evaluate the psychometric properties of item pools developed for adults with SCI; and (4) report information that facilitates clinical and research use.
Methods
Item set development
Based on focus group feedback, we identified initially 3 realms that are relevant to social participation: family and friends, leisure, and work. We began with 129 items generated during individual interviewsCitation12,Citation13 and developed 214 new items based on the focus group transcriptsCitation14 to form the basis of item banks. Given the work completed by the PROMISCitation10 and Neuro-QOL collaborators, we decided to adopt PROMIS’ domains of Ability to Participate and Satisfaction with SRA. We found these subdomains to be more useful than our initial 3 realms given the assumption of unidimensionality required for item response theory (IRT) analyses and computer adaptive testing (CAT) programming. We binned the items generated from individual interviews and focus groups with 82 Neuro-QOL social domain items into subdomains of Ability to Participate in SRA and Satisfaction with SRA. Two investigators independently assigned items to subdomains; when they disagreed, they discussed and reconciled differences.
Because a goal of the SCI-QOL project was to evaluate the Neuro-QOL's social domain items in an SCI population and supplement them with SCI-relevant items, we prioritized the 38 Neuro-QOL Ability to Participate items and the 44 Satisfaction items. Due to the Neuro-QOL's linkage with PROMIS, this set included 23 PROMIS Satisfaction items. The project team reviewed the remaining items and determined that all but 4 Ability and 4 Satisfaction items were redundant with the Neuro-QOL content. The team winnowed the item pool to 50 Ability and 49 Satisfaction items
Following PROMIS instrument development standards,Citation15 we scheduled cognitive debriefing interviewsCitation16 with individuals with SCI (n ≥ 5 per item). Because the Neuro-QOL items had already undergone this extensive review, only the 8 new SCI-QOL social items underwent this high level of review. No items required modification based on cognitive interviews.
Next, we completed a translatability review to ensure that the SCI-QOL items would be amenable to translation into other languages. An experienced team conducted a thorough review of the items to evaluate issues with vocabulary or grammar that could affect the meaning of items translated to Spanish. We modified one Ability to Participate and one Satisfaction item based on translatability feedback. We rephrased ‘I am able to navigate a crowded social situation’ as two new items: ‘I am able to interact with people in social situations’ and ‘I am physically able to move through a crowded room.’ We removed ‘amount of’ from ‘I am satisfied with the amount of control I have over my daily activities.’ Finally, we evaluated the reading level of new items using the Lexile FrameworkCitation17 to assure that they did not exceed a fifth grade level.
Field testing
We administered the 50 Ability to Participate and 49 Satisfaction items to a sample of individuals with SCI to provide data for IRT calibration. As outlined by Tulsky and colleagues (overview paper, this issue)Citation13, 5 SCI Model System sites (Craig Hospital, Kessler Rehabilitation Hospital, Rehabilitation Institute of Chicago, University of Michigan, and University of Washington) and the James J. Peters Veterans Hospital participated in data collection. Interviewers read each question aloud from a computer screen and entered responses into a secure data platform. Institutional Review Boards at each site approved this study. We developed a procedure manual and used it to provide interviewers with training to certify their competence.
Calibration sample
A total of 641 individuals with traumatic SCI completed the two item sets. Inclusion criteria were SCI due to traumatic event, age 18 or older at the time of study participation, and ability to read and understand English. There were no further exclusion criteria. Level and severity of injury were confirmed through medical record review.
Reliability sample
An independent sample of 245 individuals from 4 SCI Model Systems centers (University of Michigan, Kessler Institute for Rehabilitation, Rehabilitation Institute of Chicago, Craig Hospital) completed the item banks twice as part of a larger project in which we tested SCI-QOL CATs and short forms at multiple intervals.Citation12 Each site's Institutional Review Board reviewed and approved the study protocol. Eligibility criteria were similar to the calibration study: traumatic SCI, 18 years or older, and ability to read, speak, and understand English fluently. We stratified the sample by level and completeness of injury as well as time since injury (≤2 years, 2>years). Participants were community-dwelling and sustained SCI more than 4 months before the assessment. Our goal was to have participants complete the second assessment 7–14 days after the first assessment.
Data analysis
We evaluated dimensionality of the Ability to Participate and Satisfaction item sets using confirmatory factor analysis with MPLUS version 6. Criteria for unidimensionality were a comparative fit index (CFI) and a Tucker-Lewis Index (TLI) >0.9, a root mean square error of approximation (RMSEA) <0.08 for good fit and <0.05 for excellent fit.Citation18
2-parameter item response theory analyses
We used a graded response IRT modelCitation19 using MULTILOG software and examined the S-X2 model fit statistics using the IRTFIT macro program.Citation20 We removed items that demonstrated local item dependence (residual correlation >|0.2|), significant (P < 0.05), misfit (S-X2 test),Citation21 or differential item functioning (DIF)Citation22 due to sex, age (<50 vs. ≥50), education (some college or less vs. college degree or higher), level of injury (paraplegia vs. tetraplegia), injury severity (complete vs. incomplete), and time post injury (<1 year vs.≥1 year).
We transformed SCI-QOL item parameters to Neuro-QOL's general population metric following the procedure reported by Tulsky and colleagues.Citation12
Results
Calibration sample characteristics
Tulsky and colleagues describe the demographic characteristics of the focus group participants.Citation13 shows that the sample of 641 respondents was middle-aged (mean 42.9 years old) with a range from 18 to 91 years. Reflecting the epidemiology of SCI, men outnumbered women 3 to 1; the largest group was single and never married. The sample reflects the US population in terms of ethnicity; racial minority groups were somewhat over-represented. Educational attainment included high school or less education (37.6%), some college (33.9%), and a baccalaureate degree or higher (28.6%). Household income ranged from less than $20,000 (26.4%), 20,000 to $74,999 (36.4%), to $75,000 or more (22.5%); 14.8% did not know their household income or declined to report it. Most participants (89.6%) resided in a private residence. The average age at SCI was 35.6 years (SD = 15.6). Primary means of mobility included walking (33.2%), propelling a manual wheelchair (53.4%), and using a power wheelchair (40.4%).
Table 1. Demographic Characteristics of the Calibration Sample
Item characteristics
shows descriptive statistics for the retained Ability to Participate items shows the same information for the Satisfaction items
Table 2. Item Statistics for Retained Ability to Participate in Social Roles and Activities Items
Table 3. Item Statistics for Retained Satisfaction with Social Roles and Activities Items
Confirmatory factory analysis
Ability to participate
We deleted 23 items due to a significant χ2 value indicating misfit (9), bimodal distributions (3), residual correlations greater than |0.2| indicating local item dependence (11), or r2 values less than 0.3 indicating low item-total correlation (1). Some of the Neuro-QOL items pertaining to work demonstrated a bimodal distribution of responses, suggesting that people who are employed complete the items in one manner and people who are unemployed complete the items differently. The items displayed misfit, local dependence, and differential item functioning (DIF), all of which could bias the results and performance of the scale in this population. Therefore, we removed several employment related items
Satisfaction: We deleted 14 items due to bimodal distributions (6), local item dependence (11), and misfit (2).
IRT calibrations
shows the calibration statistics, including slopes and response category thresholds, for the 27 retained Ability to Participate items; all of them are Neuro-QOL items. These calibration parameters have been optimized for an SCI population but, as described below, have been transformed to the Neuro-QOL metric so they are interpretable as Neuro-QOL scores.
Table 4. Item Calibration Statistics for Ability to Participate in Social Roles and Activities
shows the calibration statistics, including slopes and response category thresholds, for the 35 retained Satisfaction items; 34 of them are Neuro-QOL items and 15 of these are PROMIS items. Similar to the Ability to Participate domain, these calibration parameters are optimized for an SCI population and transformed to the Neuro-QOL metric.
Table 5. Item Calibration Statistics for Satisfaction with Social Roles and Activities items
Differential item functioning
Seven Ability to Participate items demonstrated statistically significant DIF as did 8 Satisfaction items. When we examined the DIF effect sizes, the practical effects were negligible; thus, we decided to retain them.
Transformation to Neuro-QOL metric
We computed SCI-specific calibrations using the SCI-QOL calibration sample, then we linked these scores to the Neuro-QOL calibrations that had been developed using a general population reference group using the items that are common to SCI-QOL and Neuro-QOL. We utilized Stocking-Lord equating methodsCitation23 to calculate transformation slope and intercept parameters. We applied these parameters to create linear transformations so that the SCI-QOL measure maps to the Neuro-QOL metric and scores are reported in terms of general population norms and are equivalent to Neuro-QOL scores. shows how the mean scores for each were transformed. SCI-QOL scores are about 5 T-score points lower than Neuro-QOL scores and have less variance
Table 6. Scoring Before and After Transformation
Test information function and reliability
illustrates the test information function across the range of the Ability to Participate measure; illustrates the same function for the Satisfaction items. Reliability exceeds 0.95 within a range of –2.3 to +1.2 theta for Ability to Participate and –2.3 and +1.6 theta for Satisfaction.
Figure 1. Ability to Participate Item Bank Information and Precision (i = 27).
Figure 2. Satisfaction with SRA–Item bank information and precision (i = 35).
Short form item selection
We selected items to comprise a fixed-length short form version of each item bank for situations in which CAT administration is not practical. First, we reviewed item locations and discrimination. Then, we assigned items into quintiles based on location and selected 1 to 2 items within each quintile with the highest slope. We considered clinical relevance, wording, and similarity to other candidate items. Our goal was to maximize the diversity of short form item content. Following PROMIS naming conventions, the short forms are titled Ability to Participate in SRA SF10a and Satisfaction with SRA SF10a, respectively.
illustrates the reliability of the Ability to Participate SF10a vs. a variable length CAT (set to the default minimum of 4 items, maximum of 12 items, and maximum standard error 0.3), a 10-item fixed-length CAT, and the full item bank. Reliability exceeds 0.80 between T-scores of 20 and 60. illustrates reliability values for the various modes of administration of the Satisfaction bank. Reliability exceeds 0.80 between T-scores of 26 and 60.
Figure 3. Reliability of Ability to Participate full bank, 10-item short form, variable-length CAT, and 10-item fixed-length CAT.
Figure 4. Reliability of Satisfaction with SRA full bank, 10-item short form, variable-length CAT, and 10-item fixed-length CAT.
shows the breadth of coverage for the Ability to Participate SF10a compared with 4- and 8-item minimum variable-length CATs, a 10-item fixed length CAT, and the full item bank. The correlation between the 10-item Ability to Participate SF10a with the full item bank exceeds 0.96. shows the same information for the Satisfaction SF10a. T-score means were essentially identical as was the range of scores and standard errors. The SF10a has somewhat greater ceiling effects than the other versions.
Table 7. Ability to Participate: Accuracy of Variable- and Fixed-Length CAT and 10-Item Short Form
Table 8. Satisfaction with SRA: Accuracy of Variable- and Fixed-Length CAT and 10-item Short Form
provides means, ranges, and standard errors for the Ability to Participate variable length CATs, 10-item CAT, 10-item short form, and full bank. Means and standard deviations, ranges, percent of the sample at the ceiling and floor, and standard errors are essentially identical.
Table 9. Ability to Participate: Breadth of Content Coverage for Variable Length CAT, Fixed Length CAT, 10-item Short Form, and Full Item Bank
provides the same information for Satisfaction with SRA. Again, means and standard deviations, ranges, percent of the sample at the ceiling and floor, and standard errors are essentially identical across different versions.
Table 10. Satisfaction with SRA: Breadth of Content Coverage for Variable Length CATs, Fixed Length CAT, 10-Item Short Form, and Full Item Bank
Tables and provide the raw to scaled score conversions for the 10-item short forms for Ability to Participate and Satisfaction with SRA, respectively.
Table 11. Lookup Table for SCI-QOL v1.0 Ability to Participate in Social Roles and Activities SF10a
Table 12. Lookup Table for SCI-QOL v1.0 Satisfaction with Social Roles and Activities SF10a
Test-retest reliability
The 245 retest participants completed the second testing 7–14 days following the first assessment. The Pearson's test-retest correlation for Ability to Participate was 0.75 and 0.78 for Satisfaction (P < 0.001), accounting for more than 50% shared variance. The ICC (2,1) for Ability to Participate was 0.74; 95%CI = 0.67, 0.79 and for Satisfaction the ICC (2,1) was 0.77; 95% CI = 0.72, 0.82.
Discussion
The objectives of this study were to (1) develop item banks measuring social domains of functioning in an SCI population; (2) link the measure(s) to PROMIS and Neuro-QOL; (3) evaluate the psychometric properties of item pool developed for adults with SCI; and (4) report information that facilitates clinical and research use. The 27-item Ability to Participate and a 35-item Satisfaction with Participation item bank fulfill these objectives.
The SCI-QOL Ability to Participate and Satisfaction with SRA banks are optimized versions of the Neuro-QOL/PROMIS v1.0 social item banks for use by individuals with SCI. Items calibrations were developed using a large, heterogeneous, and representative sample. When administered as a CAT, items will be selected based on their functioning in an SCI sample. Similarly, item content that is tailored to an SCI population could be added and items that do not function well in an SCI population could be identified and removed. For example, several Neuro-QOL/PROMIS items related to employment had bimodal distributions or demonstrated poor model fit, local dependence, or differential item functioning, likely due to high rates of un- and under-employment in this sample. Given the importance of employment issues, the research team was presented with a dilemma of retaining content that could bias the overall measurement scale or remove poorly functioning items We decided to remove most problematic items at the expense of reducing content coverage. We recommend that future research develop item banks focused exclusively on employment issues. We excluded all but one (RSATIS_56, ‘I am satisfied with the amount of physical contact I have with others’) of the newly developed items from the final item banks. With the exception of the employment items as described above, the PROMIS and Neuro-QOL Ability to Participate and Satisfaction with SRA items describe most of the participation-related issues that are relevant for individuals with SCI.
Another significant advancement of this project is the development of calibrations optimized for individuals with SCI and items that are relevant and appropriate for this population. No other participation measure is customized to persons with SCI and used IRT methods in their development. Transformation to the Neuro-QOL metric enables direct comparison of SCI-QOL with Neuro-QOL social bank scores from other disability samples. Notably, the Ability to Participate and Satisfaction with SRA banks demonstrate lower mean scores (∼45 vs. ∼50) and restricted range (SD ∼6 vs. SD ∼10) when transformed to reflect the general population norms. This result suggests that people with SCI are limited in both their ability to participate and satisfaction with participation, and that the range of engagement in social roles and activities is more limited in individuals in SCI than in the general population. Social participation is an important issue for clinicians to target for clinical services, and for investigators to develop effective interventions.
The psychometric properties of the SCI-QOL Ability to Participate and Satisfaction CATs and short forms are excellent. Full item banks and short forms are available as PDF files from the authors; CATs may be administered through the NIH Assessment Center, which provides users with options for customizing stopping rules such as the minimum and maximum number of items to administer, and maximum standard error.
Study limitations
This convenience sample was drawn from only 6 hospitals; it may not reflect the diversity of people with SCI living in the United States. Future studies should evaluate sensitivity to change, demonstrate known groups validity, and provide information on interpretability and meaningfulness.
Clinical applications
Screening during outpatient visits allows clinicians to identify individuals with low levels of Ability to Participate or Satisfaction with Social Roles and Activities. Discussions with patients about the meaning and consequences of the scores could help guide referrals and treatment plans.
Research opportunities
These 2 SCI-QOL social domain variables can be used in epidemiological studies to monitor population health and the effects of interventions designed to enhance participation, among other applications.
Conclusions
The Ability to Participate and Satisfaction with Social Roles and Activities items provide state-of-the-art measures that can be administered using CAT or short forms. The measures are compatible with the PROMIS and Neuro-QOL family of measures. They complement the physical, emotional, and other social domain item banks.
Suppliers
Mplus Statistical Analysis with Latent Variables User's Guide [computer program]. Version 6. Los Angeles: Muthen & Muthen; 2007.
MULTILOG: Multiple, categorical item analysis and test scoring using item response theory [computer program]. Chicago, IL: Scientific Software; 1991.
Acknowledgements
We are indebted to David Victorson, Ph.D. for his assistance with conceptual and methodological input and Seung Choi, PhD for data analyses.
Disclaimer statements
Contributors All authors have contributed significantly to the design, analysis and writing of this manuscript. The contents represent original work and have not been published elsewhere. No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit upon the authors or upon any organization with which the authors are associated.
Funding Funding was provided by the National Institutes of Health – Eunice Kennedy Shriver National Institute of Child Health & Human Development/National Center for Medical Rehabilitation Research and the National Institute of Neurological Disorders and Stroke (grant no. 5R01HD054659), and by the National Institute on Disability and Rehabilitation Research (grant nos. H133N110014, H133N110002, H133N110006, and H133N110020).
Conflicts of interest All SCI-QOL items and parameters are copyright © 2015 David Tulsky and the Kessler Foundation. All rights reserved. All Neuro-QOL items are copyright © 2008–2013 David Cella on behalf of the National Institute for Neurological Disorders and Stroke (NINDS). All items are freely available to the public via the Assessment Center platform (www.assessmentcenter.net). There are currently no plans for Dr. Tulsky or Kessler Foundation to profit from the use of the copyrighted material.
Ethics approval The Institutional Review Board at each site reviewed and approved this project.
References
- Whiteneck GG. Issues affecting the selection of participation measurement in outcomes research and clinical trials. Arch Phys Med Rehabil 2010;91(9):S54–S9.
- Magasi S, Post MW. A comparative review of contemporary participation measures’ psychometric properties and content coverage. Arch Phys Med Rehabil 2010;91(9):S17–28.
- Dijkers MP. Issues in the conceptualization and measurement of participation: an overview. Arch Phys Med Rehabil 2010;91(9Suppl1):S5–S16.
- NIH Common Fund 2013 [updated May 20, 2013]. National Institutes of Health Web Site]. Available from: http://www.nihroadmap.nih.gov/.
- Quatrano L, Cruz T. Future of outcomes measurement: impact on research in medical rehabilitation and neurologic populations. Arch Phys Med Rehabil 2011;92(10 Suppl):S7–11.
- Cella D, Riley W, Stone A, Rothrock N, Reeve B, Yount S, et al. The Patient-Reported Outcomes Measurement Information System (PROMIS) developed and tested its first wave of adult self-reported health outcome item banks: 2005–2008. J Clin Epidemiol 2010;63:1179–94.
- Stone K. NINDS Common data element project: a long-awaited breakthrough in streamlining trials. Ann Neurol 2010;68:A11–A3.
- DeWalt DA, Rothrock N, Yount S, Stone AA, Group PC. Evaluation of item candidates: the PROMIS qualitative item review. Med Care 2007;45(5 Suppl 1):S12–21.
- Castel LD, Williams KA, Bosworth HB, Eisen SV, Hahn EA, Irwin DE, et al. Content validity in the PROMIS social-health domain: a qualitative analysis of focus-group data. Qual Life Res 2008;17(5):737–49.
- Hahn EA, DeWalt DA, Bode RK, Garcia SF, DeVellis RF, Correia H, et al. New English and Spanish social health measures will facilitate evaluating health determinants. Health Psychol 2014;33(5):490–9.
- Hahn EA, Devellis RF, Bode RK, Garcia SF, Castel LD, Eisen SV, et al. Measuring social health in the patient-reported outcomes measurement information system (PROMIS): item bank development and testing. Qual Life Res 2010;19(7):1035–44.
- Tulsky DS, Kisala PA, Victorson D, Choi SW, Gershon R, Heinemann AW, et al. Methodology for the development and calibration of the SCI-QOL item banks. J Spinal Cord Med 2015;38(3):270–87.
- Tulsky DS, Kisala PA, Victorson D, Tate D, Heinemann AW, Charlifue S, et al. Overview of the Spinal Cord Injury – Quality of Life (SCI-QOL) measurement system. J Spinal Cord Med 2015;38(3):257–69.
- Tulsky DS, Kisala PA, Victorson D, Tate D, Heinemann AW, Amtmann D, et al. Developing a contemporary patient-reported outcomes measure for spinal cord injury. Arch Phys Med Rehabil 2011;92(10):S44–51.
- PROMIS. PROMIS Instrument Development and Psychometric Evaluation Scientific Standards 2012 [12/27/2012]. Available from: http://www.nihpromis.org/Documents/PROMIS_Standards_050212.pdf.
- Willis GB. Cognitive Interviewing: a ‘how to’ guide. Research Triangle Park, NC: Research Triangle Institute, 1999.
- MetaMetrics. The LEXILE framework for reading. Durham, NC: MetaMetrics Inc; 1995.
- Hu LT, Bentler PM. Cutoff criteria for fit indexes in covariance structure analysis: Conventional criteria versus new alternatives. Struct Equ Modeling 1999;6(1):1–55.
- Samejima F. Graded response model. Handbook of modern item response theory. In: van der Liden WJ, Hambleton R, eds. New York: Springer; 1997. p. 85–100.
- Bjorner JB, Smith K, Stone C, Sun X. IRTFIT: A Macro for Item Fit and Local Dependence Tests under IRT Models. 2007.
- Orlando M, Thissen D. Further investigation of the performance of S-X2: An item fit index for use with dichotomous item response theory models. Appl Psych Meas 2003;27(4):289–98.
- Choi SW, Gibbons LE, Crane PK. Lordif: An R package for detecting differential item functioning using iterative hybrid ordinal logistic regression/item response theory and Monte Carlo simulations. J Stat Softw. 2011;39(8):1.
- Stocking ML, Lord FM. Developing a common metric in item response theory. Appl Psych Meas 1983;7(2):201–10.