Methodology for the development and calibration of the SCI-QOL item banks

Figures & data

Figure 1. SCI-QOL Development and Calibration: Phases and Goals.

Table 1. Glossary of Terms

Term	Description
Calibration	Using item response theory – in this case, graded response model analyses – to place all items on a single underlying metric so that any subset of items will yield a score that is directly comparable to the score on any other subset of items
Computer Adaptive Test (CAT)	‘Smart’ test that customizes administration for each individual. Each item that is administered is selected based on the participant's response to the previous item.
Construct	Conceptual area of interest/relevance; a construct may represent a domain, subdomain, or subtopic within a subdomain
Domain	Overarching, multidimensional conceptual area, e.g. emotional health, social participation, physical-medical health, physical function.
Instrument	Instruments are scales, assessment devices, or psychological tests. In this context, they are patient-reported outcomes measures.
Item Bank	Final set of items that have been calibrated with item response theory and are available for CAT administration
Item Pool	Set of preliminary items that have not yet been calibrated with item response theory
Measure	An assessment device or psychological test or scale. The terms ‘measure’ and ‘instrument’ refer to the same thing. In this context, we are referring to a patient reported outcomes scale and the ‘measure’ may be an item bank, short form, or CAT.
Metric	The underlying value that is used to understand the score and how it is scaled so that meaning can be derived from a score. For the SCI-QOL, the reported metric is a ‘T Metric’ with an average of 50 and standard deviation unit of 10. For PROMIS, the metric reflects the general population average. For SCI-QOL, the metric reflects either general population (when anchored to a PROMIS or Neuro-QOL scale) or to the SCI-population (when it is a new bank that does not have a comparable PROMIS or Neuro-QOL bank). The metric is relative the population that was used to calibrate the items
Phase	Major section of the original SCI-QOL development work with unique methods and goals. There are 5 distinct phases reported here.
Scale	A group of items measuring a similar construct. The scales indicate that the items are not administered via CAT technology but rather are a fixed-form set of items. These may or may not be calibrated using IRT.
Short Form	Brief (e.g. 6–10 items) fixed-length subset of items from a larger test or a calibrated item bank. When the short form is derived from a calibrated item bank, the short form score is directly comparable to the full bank or CAT score.
Subdomain	Subcomponent of a domain; unidimensional conceptual area that is amenable to item banking

Table 2. Preliminary Items Generated from Study 1: Item Generation Interviews

Domain	Subdomain	n	Overall Percent	Percent within Domain
Emotional	Attitude	118	9.2	33.3
	Body image	10	0.8	2.8
	Coping	47	3.6	13.3
	Emotional well-being	92	7.1	26.0
	Life satisfaction	51	4.0	14.4
	Spirituality	36	2.8	10.2
Environmental Factors	Assistive devices	51	4.0	16.2
	Accessibility/environmental	82	6.4	26.1
	Finances	65	5.0	20.7
	Housing	18	1.4	5.7
	Home health attendant	34	2.6	10.8
	Insurance	14	1.1	4.5
	Transportation	50	3.9	15.9
Other	Other	31	2.4	100.0
Physical-Functional	Physical well-being	120	9.3	100.0
Physical-Medical	Aging with SCI	4	0.3	5.8
	Bowel/Bladder management	37	2.9	53.6
	Pain	28	2.2	40.6
Sexual Functioning	Sexuality	43	3.3	100.0
Social/Participation	Able-bodied persons	59	4.6	16.5
	Communication	22	1.7	6.2
	Education	18	1.4	5.0
	Employment (or school)	38	2.9	10.6
	Future directions	4	0.3	1.1
	Independence	71	5.5	19.8
	Leisure or Hobby	20	1.6	5.6
	Relationships – family	28	2.2	7.8
	Relationships – Parenting	8	0.6	2.2
	Relationships – Professional	9	0.7	2.5
	Relationships – Romantic	5	0.4	1.4
	Relationships – Social/Friends	51	4.0	14.3
	Support	25	1.9	7.0
		1289	100.0

Table 3. Subdomains for development, field testing, and analysis

Domain	Developed	Tested	Analyzed
Emotional Health
Positive Affect & Well-Being	X	X	X
Depression	X	X	X
Anxiety	X	X	X
Stigma	X	X	X
Resilience	X	X	X
Grief/Loss	X	X	X
Self-Evaluation	X	X	X
Psychological Trauma	X	X	X
Physical-Medical Health
Pressure Ulcers	X	X	X
Bladder Management Difficulties	X	X	X
Bladder Complications	X	X	X
Bowel Management Difficulties	X	X	X
Pain Interference	X	X	X
Pain Behavior	X	X	X
Respiratory Functioning	X	X
Social Participation
Ability to Participate	X	X	X
Satisfaction with Social Roles & Activities	X	X	X
Independence	X	X	X
Control over Participation	X	X	X
Involvement in Life Situations	X	X	X
Environmental Barriers & Supports	X
Physical Functioning
Ambulation	X	X	X
Basic Mobility	X	X	X
Fine Motor Function	X	X	X
Self Care	X	X	X
Wheelchair Mobility	X	X	X
Sexual Functioning
Sexual Function–Performance	X	X
Sexual Function–Satisfaction	X	X
Sexual Self Esteem	X	X
SCI Interfering Factors	X	X

Table 4. Calibration Sample Demographics

Variable	Total Sample	Session 1 (Physical)	Session 2 (Emotional)	Session 3 (Social)
	(n = 877)	(n = 757)	(n = 717)	(n = 641)
Age	42.9 years (SD 15.4)	42.9 years (SD 15.5)	43.0 years (SD 15.3)	42.9 years (SD 15.3)
Age at Injury	36.3 years (SD 16.6)	36.3 years (SD 15.8)	36.1 years (SD 16.8)	35.9 years (SD 16.9)
Sex
Male	693 (79%)	599 (79%)	559 (78%)	496 (77%)
Female	184 (21%)	158 (21%)	158 (22%)	145 (23%)
Ethnicity
Hispanic	93 (11%)	80 (11%)	82 (11%)	66 (10%)
Non-Hispanic	771 (88%)	665 (88%)	631 (88%)	571 (89%)
Not provided	13 (2%)	12 (2%)	4 (1%)	4 (1%)
Race
Caucasian	613 (70%)	536 (71%)	505 (70%)	460 (72%)
Black or African-American	155 (18%)	130 (17%)	125 (17%)	110 (17%)
Asian	13 (2%)	11 (2%)	8 (1%)	6 (1%)
American Indian/Alaska Native or Native Hawaiian/Pacific Islander	8 (1%)	7 (1%)	7 (1%)	5 (1%)
More than one race	13 (2%)	11 (2%)	9 (1%)	7 (1%)
Other	60 (7%)	51 (7%)	50 (7%)	42 (7%)
Not provided	15 (2%)	11 (2%)	13 (2%)	10 (2%)
Education
High school or less	338 (39%)	290 (38%)	275 (38%)	241 (38%)
Some college	297 (34%)	253 (34%)	248 (35%)	217 (34%)
Bachelor's degree or more	240 (27%)	212 (28%)	194 (27%)	183 (29%)
Total Household Income
<$20,000	223 (25%)	184 (24%)	193 (27%)	169 (26%)
$20,000–74,999	306 (35%)	269 (36%)	268 (37%)	233 (36%)
>$75,000	195 (22%)	167 (22%)	148 (21%)	144 (22%)
Unknown/Not provided	153 (17%)	137 (18%)	108 (15%)	95 (15%)
Time Since Injury
<1 year post injury	246 (28%)	218 (29%)	196 (27%)	139 (22%)
1–3 years post injury	239 (27%)	210 (28%)	186 (26%)	192 (30%)
>3 years post injury	392 (45%)	329 (44%)	335 (47%)	310 (48%)
Diagnosis
Paraplegia Complete	200 (23%)	173 (23%)	171 (24%)	151 (24%)
Paraplegia Incomplete	152 (17%)	134 (18%)	136 (19%)	116 (18%)
Tetraplegia Complete	193 (22%)	167 (22%)	152 (21%)	129 (20%)
Tetraplegia Incomplete	284 (32%)	249 (33%)	225 (31%)	215 (34%)
Not confirmed*	48 (6%)	34 (5%)	31 (4%)	27 (4%)
Wheelchair Use (not mutually exclusive)
Manual Wheelchair	462 (53%)	386 (61%)	386 (54%)	342 (53%)
Power Wheelchair	367 (42%)	313 (41%)	305 (43%)	259 (40%)
Cause of Injury
Motor vehicle accident	294 (34%)	245 (32%)	241 (34%)	219 (34%)
Gunshot wound or other violence	105 (12%)	89 (12%)	83 (12%)	77 (12%)
Fall	193 (22%)	169 (22%)	164 (23%)	140 (22%)
Dirt bike accident	7 (1%)	6 (1%)	6 (1%)	7 (1%)
Motorcycle accident	23 (3%)	20 (3%)	20 (3%)	16 (3%)
Bicycle accident	10 (1%)	9 (1%)	9 (1%)	8 (1%)
Other Sports	58 (7%)	56 (7%)	45 (6%)	47 (7%)
Diving	62 (7%)	50 (7%)	57 (8%)	49 (8%)
ATV accident	8 (1%)	4 (1%)	4 (1%)	6 (1%)
Medical or surgical accident	29 (3%)	28 (4%)	27 (4%)	18 (3%)
Other	51 (6%)	47 (6%)	49 (7%)	41 (6%)
Not reported	37 (4%)	81 (11%)	12 (2%)	13 (2%)

Note: Participant demographic information on the physical functioning sample may be found in Tulsky et al.¹² and Jette et al.¹⁶

^*For the small percentage of individuals without confirmed medical record information, self-reported diagnosis and completeness were used throughout the analyses.

Table 5. Linkages with PROMIS and Neuro-QOL

Subdomain/Bank	# SCI-QOL Items	Linked to	# PROMIS Items	# Neuro-QOL Items	Reference Population
Bladder Management Difficulties	15	–	0	0	SCI
Bladder Complications	5	–	0	0	SCI
Bowel Management Difficulties	26	–	0	0	SCI
Pressure Ulcers	14	–	0	0	SCI
Pain Interference	7	PROMIS	18	0	General
Pain Behavior	4	PROMIS	3	0	General
Positive Affect & Well-being	5	Neuro-QOL	0	14	General
Depression	6	PROMIS	18	22	General
Anxiety	2	PROMIS	18	5	General
Stigma	5	Neuro-QOL	0	12	Neuro
Resilience	21	–	0	0	SCI
Grief/Loss	17	–	0	0	SCI
Self-Evaluation	19	–	0	3	SCI
Psychological Trauma	19	–	0	0	SCI
Ability to Participate in Social Roles & Activities	0	Neuro-QOL	0	19	General
Satisfaction with Social Roles & Activities	2	Neuro-QOL	3	14	General + Neuro
Independence/Autonomy	8	–	0	0	SCI
Ambulation	39	–	4	9	SCI
Basic Mobility	54	–	6	9	SCI
Fine Motor Function	36	–	9	6	SCI
Self Care	90	–	11	10	SCI
Wheelchair Mobility	56	–	0	0	SCI

Note: Bold text indicates number of items that are statistically linked.

Abbreviations: Neuro, neurological population consisting of individuals with stroke, epilepsy, Parkinson's, multiple sclerosis, and amyotrophic lateral sclerosis.

Table 6. SCI-QOL Item Bank Statistics

Domain	Bank	Scoring Direction	No. Items Tested	No. Final Items	No. Items in SF	N	CFI	RMSEA	α	Item Total Correlations	Slope Range	Threshold Range
Physical Health	Pain Interference	Severe Symptom	28	25	10	757	0.983	0.063	0.968	0.47 to 0.86	1.22 to 4.27	0.00 to 2.66
	Pain Behavior	Severe Symptom	18	7	–	757	0.996	0.076	0.899	0.59 to 0.81	2.26 to 5.39	−0.62 to 2.31
	Bladder Mgmt. Difficulties	Severe Symptom	38	15	8	757	0.965	0.093	0.905	0.38 to 0.78	1.05 to 4.21	0.10 to 3.23
	Bowel Mgmt. Difficulties	Severe Symptom	53	26	9	757	0.955	0.078	0.951	0.32 to 0.79	0.92 to 4.90	−0.01 to 3.68
	Pressure Ulcers	Severe Symptom	30	14^†	7	189	0.961	0.124	0.924	0.56 to 0.80	2.17	−0.85 to 2.02
	Bladder Complications	Severe Symptom	8	5^†	–	297	0.982	0.080	0.794	0.28 to 0.68	1.51	−0.33 to 3.34
Emotional Health	Positive Affect & Well-being	Better Function	32	28	10	717	0.947	0.094	0.970	0.61 to 0.82	2.25 to 4.55	−2.19 to 1.79
	Depression	Severe Symptom	35	28	10	716	0.968	0.066	0.964	0.51 to 0.81	1.39 to 4.23	−0.68 to 3.14
	Anxiety	Severe Symptom	38	25	9	716	0.953	0.069	0.946	0.50 to 0.74	1.49 to 2.95	−0.91 to 3.38
	Stigma	Severe Symptom	30	23	10	611	0.941	0.088	0.936	0.39 to 0.72	1.81 to 4.72	−0.17 to 2.77
	Resilience	Better Function	32	21	8	717	0.968	0.074	0.951	0.54 to 0.78	1.48 to 3.26	−3.26 to 1.11
	Grief/Loss	Severe Symptom	20	17	9	716	0.976	0.078	0.947	0.59 to 0.78	1.65 to 3.15	−1.48 to 2.48
	Self-Esteem	Better Function	30	23	8	716	0.946	0.087	0.950	0.50 to 0.81	1.28 to 3.74	−3.94 to 1.38
	Psychological Trauma	Severe Symptom	31	19	8	716	0.954	0.065	0.903	0.45 to 0.69	1.17 to 2.52	−0.78 to 3.50
Social Participation	Ability to Participate in SRA	Better Function	50	27	10	641	0.946	0.096	0.963	0.51 to 0.81	2.11 to 5.66	−2.08 to 0.25
	Satisfaction with SRA	Better Function	50	35	10	641	0.914	0.093	0.971	0.50 to 0.77	2.63 to 5.84	−2.06 to 0.07
	Independence	Better Function	13	8	8	641	0.980	0.111	0.894	0.55 to 0.79	1.54 to 3.74	−2.59 to 1.05
Physical Functioning	Ambulation	Better Function	40	39	11	855	0.999	0.039	0.991	0.45 to 0.94	3.23 to 7.29	0.48 to 3.35
	Basic Mobility	Better Function	65	54	11	855	0.969	0.081	0.985	0.54 to 0.90	1.19 to 7.84	−2.15 to 1.73
	Fine Motor Function	Better Function	39	36	9	850	0.998	0.049	0.990	0.58 to 0.94	1.95 to 6.96	−1.82 to 1.05
	Self Care*	Better Function	99	90	11	850	0.992	0.052	0.995	0.21 to 0.92	0.90 to 5.73	−5.05 to 0.75
	Wheelchair Mobility	Better Function	63	56	10^a/9^b	709	0.929	0.063	0.993	0.29 to 0.82	0.90 to 4.31	−3.79 to 1.64

^aNumber of items in Manual Wheelchair Short Form.

^bNumber of items in Power Wheelchair Short Form.

*For Males, 85 items are included in Self-Care, CFI is .992, and RMSEA is 0.049. For Females, 84 items are included, CFI is 0.993, and RMSEA is 0.049.

^†The Pressure Ulcers and Bladder Complications scales each include one additional non-scored screener item.

Table 7. SCI-QOL Domain Definitions

Physical Health

Bladder Management Difficulties

A range of difficulties associated with bladder management, ranging from ability to carry out their bladder program to worry about bladder accidents, performing their bladder program, and impact on everyday living

Bladder Complications

A range of difficulties associated with bladder complications, such as urinary tract infection (UTI), UTI impact on everyday living, and bladder issues affecting sexual function

Bowel Management Difficulties

A range of difficulties associated with bowel management, ranging from ability to carry out their bowel program to worry about bowel accidents, performing their bowel program, and impact on everyday living

Pressure Ulcers

A range of difficulties associated with Pressure Ulcers, such as the extent to which pressure ulcers hinder engagement in social, cognitive, emotional, physical, and recreational activities

Pain Interference^†

Consequences of pain on relevant aspects of one's life, including the extent to which pain hinders engagement with social, cognitive, emotional, physical, and recreational activities

Pain Behavior^†

Self-reported external manifestations of pain; behavior, verbal or non-verbal and voluntary or deliberate, that typically indicate to others that an individual is experiencing pain, including observable displays, pain severity behaviors, and verbal reports of pain

Emotional Health

Positive Affect & Well-being*

Aspects of one's life that relate to a sense of well-being, life satisfaction or an overall sense of purpose and meaning

Depression*

Self-reported negative mood (sadness, guilt), views of self (self-criticism, worthlessness), and social cognition (loneliness, interpersonal alienation), as well as decreased positive affect and engagement (loss of interest, meaning, and purpose

Anxiety*

Symptoms that reflect autonomic arousal and experience of threat, including self-reported fear (fearfulness, panic), anxious misery (worry, dread), hyperarousal (tension, nervousness, restlessness), and somatic symptoms related to arousal (racing heart, dizziness)

Stigma^‡

Others’ perceptions of oneself and publically enacted negativity, prejudice, and discrimination as a result of injury-related manifestations

Resilience

Subjective experience of the process and outcome of successfully adapting to difficult or challenging life experiences, especially highly stressful or traumatic events

Grief/Loss

Emotional reactions of grief (the natural process of reacting to a loss) such as anger, guilt, anxiety, sadness, and despair

Self-Evaluation

Assessment of one's emotional, evaluative, and cognitive perceptions of competence and worth

Psychological Trauma

An overwhelming experience of fear, helplessness or horror as a result of exposure to actual or perceived threat(s) to life, bodily integrity or the mind, usually rendering an individual unable to adequately cope

Social Participation

Ability to Participate in Social Roles and Activities*

Degree of involvement in one's usual social roles, activities, and responsibilities, including work, family, friends, and leisure

Satisfaction with Social Roles and Activities*

Satisfaction with involvement in one's usual social roles, activities, and responsibilities, including work, family, friends, and leisure

Independence/Autonomy

Perceived independence, ability to tell others about one's needs, and sense of control over one's life

Physical Functioning

Basic Mobility

Ability to carry out activities involving changing and maintaining body positions, transfers, moving and carrying objects, moving around in different locations

Self-Care

Ability to carry out activities involving eating, dressing, grooming, bathing and toileting, including managing bowel and bladder programs

Fine Motor Function

Ability to manually hold, manipulate and move objects that require varying degrees of dexterity and/or strength

Wheelchair Mobility

Ability to transfer in and out of a wheelchair, maneuver a wheelchair under different conditions, engage in activities from a wheelchair and manage wheelchair parts

Ambulation

Ability to engage in walking activities in different locations that vary based on speed, time and condition and the ability to manage stairs under different conditions

*Definition from Neuro-QOL.³⁵

^†Definition from PROMIS.³⁶

^‡Adapted from Neuro-QOL.³⁵

Table 8. Correlations between Short Form and simulated CAT scores

		Calibration Data (CAT + SF simulations with full-bank data)	Reliability Study Data (CAT + SF administration with no duplicates)
	n	Pearson's r	n	Pearson's r
Ambulation	825	0.98**	n/a	n/a
Ability to Participate	641	0.93**	164	0.92**
Anxiety	716	0.94**	175	0.93**
Basic Mobility	773	0.94**	85	0.95**
Bladder Management Difficulties	757	0.94**	168	0.94**
Bowel Management Difficulties	757	0.95**	169	0.95**
Depression	716	0.94**	175	0.95**
Fine Motor	849	0.95**	84	0.96**
Grief/Loss	716	0.97**	174	0.96**
Independence	629	0.99**	168	0.99**
Pain Interference	757	0.96**	171	0.96**
Positive Affect& Well-Being	716	0.95**	175	0.94**
Psychological Trauma	716	0.96**	89	0.95**
Resilience	716	0.96**	174	0.95**
Satisfaction with Social Roles & Activities	641	0.92**	166	0.91**
Self Care	788	0.94**	85	0.97**
Self-Esteem	716	0.92**	89	0.91**
Stigma	611	0.92**	175	0.91**
Wheelchair Mobility: Power WC SF	355	0.91**	45	0.91**
Wheelchair Mobility: Manual WC SF	435	0.89**	41	0.93**

**P<0.01.

Note: Ambulation items were not administered in the Reliability Study.

Table 9. SCI-QOL Test-Retest Reliability

SCI-QOL Item Banks	Pearson's r	Intraclass Correlation Coefficient
		ICC (2,1)	95% CI
Ability to Participate in SRA	0.75**	0.74	(0.67, 0.79)
Anxiety	0.80**	0.80	(0.75, 0.84)
Basic Mobility	0.93**	0.93	(0.90, 0.94)
Bladder Management Difficulties	0.77**	0.76	(0.70, 0.81)
Bowel Management Difficulties	0.74**	0.74	(0.68, 0.79)
Depression	0.80**	0.80	(0.75, 0.84)
Fine Motor	0.95**	0.95	(0.93, 0.96)
Grief	0.84**	0.83	(0.78, 0.87)
Independence (n = 159)	0.84**	0.84	(0.78, 0.88)
Pain Interference (n = 244)	0.84**	0.83	(0.78, 0.87)
Positive Affect & Well Being	0.78**	0.78	(0.72, 0.82)
Resilience	0.79**	0.79	(0.74, 0.83)
Satisfaction with SRA	0.78**	0.77	(0.72, 0.82)
Self Care	0.96**	0.96	(0.94, 0.97)
Self Esteem	0.84**	0.84	(0.80, 0.88)
Stigma	0.80**	0.79	(0.74, 0.84)
Trauma	0.84**	0.84	(0.80, 0.88)
Wheelchair Mobility (n = 195)	0.92**	0.92	(0.90, 0.94)
SCI-QOL Scales
Bladder Complications	0.70**	0.69	(0.61, 0.76)
Skin-Pressure Ulcers	0.79**	0.79	(0.74, 0.84)

**P < 0.01.

Note: Test-retest reliability was not assessed for Ambulation or Pain Behavior.

Tulsky DS, Kisala PA, Victorson D, Tate D, Heinemann AW, Amtmann D, et al. Developing a Contemporary Patient-Reported Outcomes Measure for Spinal Cord Injury. Arch Phys Med Rehabil 2011;92(10):S44–S51.

 PubMed Web of Science ®Google Scholar

Glaser BGS, A.L. The discovery of grounded theory: Strategies for qualitative research. New Brunswick, NJ: Aldine Transaction; 1967.

 Google Scholar

Cella D, Gershon R, Lai JS, Choi S. The future of outcomes measurement: item banking, tailored short-forms, and computerized adaptive assessment. Qual Life Res 2007;16(Suppl 1):133–41.

 PubMedGoogle Scholar

Gershon R, Rothrock NE, Hanrahan RT, Jansky LJ, Harniss M, Riley W. The development of a clinical outcomes survey research application: Assessment Center. Qual Life Res 2010;19(5):677–85.

 PubMed Web of Science ®Google Scholar