Survey response burden in caregivers of civilians and service members/veterans with traumatic brain injury

Abstract

Purpose

Caregivers have expressed interest in survey research, yet there is limited information available about survey response burden, ie, the time, effort, and other demands needed to complete the survey. This may be particularly important for caregivers due to excessive time demands and/or stress associated with caregiving.

Method

Survey response burden indicators were collected as part of a study to develop and validate a patient-reported outcome (PRO) measurement system for caregivers of civilians or service members/veterans (SMVs) with traumatic brain injury (TBI).

Results

Compared to the group caring for civilians (n=335), the group caring for SMVs (n=123) was comprised of all women, was younger, had fewer racial/ethnic minorities, had more education, and nearly all were the spouse of a person with TBI. All PRO outcomes were poorer for the group caring for SMVs. Although the caregivers of SMVs had poorer PRO outcomes compared to caregivers of civilians, they were more likely to report that they would recommend the study to others. Caregivers with less education and those from racial/ethnic minority groups had more favorable ratings of their study participation experience, even though they needed more help using the computer or answering the questions.

Conclusion

The results of this study provide useful information about the acceptability of computer-based survey administration for caregiver PROs. PROs are widely gathered in clinical and health services research and could be particularly useful in TBI care programs. More data are needed to determine the best assessment strategies for individuals with lower education who are likely to require some assistance completing PRO surveys. Studies evaluating PROs administered by multimedia platforms could help researchers and clinicians plan the best strategies for assessing health-related quality of life in TBI caregivers.

Keywords:

• patient-reported outcomes
• computer-based survey administration
• study participation ratings
• military personnel

Introduction

Caregivers of people with moderate to severe traumatic brain injury (TBI) are at risk for poor health-related quality life (HRQOL).^1–7 Caregivers of service members or veterans (SMVs) with TBI face unique challenges navigating the military health care system, and may experience even greater burden and worse HRQOL than caregivers of civilians with TBI.^5,8–13 Assessing HRQOL in caregivers of individuals with TBI is complicated by the lack of a gold standard assessment tool specific to this population. To address this need, the TBI-CareQOL study developed a patient-reported outcome (PRO) measurement system that captures both the generic and unique aspects of HRQOL significant to caregivers of civilians or SMVs with TBI.^4,5,14 Briefly, TBI-CareQOL measurement development involved qualitative methods to identify relevant aspects of HRQOL for both caregivers of civilians and SMVs. Areas requiring new development included several iterations of item development and review. Quantitative field testing of items was conducted, and detailed analyses included both classical test theory approaches and item response theory analytical approaches for final item selection. The study utilized modern psychometric and health information technology methods from PROMIS (Patient-Reported Outcomes Measurement Information System).¹⁵

Health information technology advances have increased the number of available PRO survey administration options. However, different methods of administration require different skills and resources of people being asked to complete the survey; this means that the choice of method of administration may result in differing levels of respondent burden.¹⁶ One important methodological standard for a PRO instrument is to minimize respondent burden, ie, the time, effort, and other demands needed to complete the instrument.^17–21 This may be particularly important for caregivers due to excessive time demands and/or stress associated with caregiving.^22–24 Caregivers have expressed interest in survey research,^4,5,25 yet there is limited information available about respondent burden.²⁶ In addition to addressing item wording, literacy level, questionnaire length, and questionnaire formatting during instrument development, it is useful to obtain respondent feedback after completion of the questionnaire.^19,27–29

As part of the TBI-CareQOL study to develop and validate a PRO measurement system for TBI caregivers, respondents were asked to complete a short evaluation survey at the end of the online assessment. The purpose of this paper is to present the results of this evaluation survey, which included indicators of response burden. It was hypothesized that caregivers with less education would report more burden completing online computer surveys compared to those with more education. There were no specific hypotheses about response burden for caregivers of civilians vs SMVs or for racial/ethnic minorities, although there was interest in evaluating burden across these subgroups.

Methods

Participants

Caregivers of civilians with TBI were recruited from the Kessler Foundation, the Rehabilitation Institute of Michigan (RIM), The Institute for Rehabilitation and Research (TIRR) Memorial Hermann Rehabilitation Hospital, and the University of Michigan Medical School. Caregivers of SMVs with TBI were recruited primarily through military caregiver support organizations, eg, Hearts of Valor. Multiple recruitment strategies for convenience sampling were implemented, including existing TBI caregiver databases, medical record data capture systems,³⁰ and hospital-based and community outreach efforts. For both groups, caregivers were at least 18 years old, English-speaking, and were caring for an individual who sustained a medically documented TBI after age of 16 years. The individual with the TBI had to be at least 1 year post-injury. The TIRR Institutional Review Board (IRB) approved a waiver of consent because this research was considered to be of low risk. The Kessler Foundation IRB, RIM: Wayne State University IRB, and the University of Michigan Medical School IRB approved both a waiver of written informed consent (to allow for telephone consent) as well as in-person written consent. Each participant received US$50 for completing the study.

Measures and method of administration

Sociodemographic data were obtained by self-report on the survey, and information about the person with TBI was obtained by medical record review or by caregiver report. The Mayo-Portland Adaptability Inventory-Fourth Edition (MPAI-4)³¹ was completed by caregivers as a measure of the functional ability of the person with TBI. MPAI-4 items represent the range of physical, cognitive, emotional, behavioral, and social problems that people may encounter after an acquired brain injury. MPAI-4 scores were dichotomized to indicate low functioning/more impairment (score ≥60) vs high functioning/less impairment (score <60).³² Caregivers completed four TBI-CareQOL item pools; all items were written at approximately a fifth-grade reading level. The 66-item Caregiver Strain item pool captured feelings of being overwhelmed, stressed, self-defeated, downtrodden, and “beat-down” related to the caregiver role. The 81-item Caregiver-Specific Anxiety item pool captured feelings of worry and anxiety specific to general safety, health, and future well-being of the person with the injury. The 28-item Feeling Trapped item pool captured feelings of being unable to go places or do things due to caregiving responsibilities. The 98-item Feelings of Loss item pool captured feelings of loss for the caregivers themselves (including loss of self, relationships, activities, and future plans) and feelings of loss with regard to the person with TBI (including loss of abilities, loss of potential/failure, and changes in behavior/personality).

Participants next completed 10 measures from PROMIS¹⁵ which used approximately a sixth-grade reading level for most items.³³ The PROMIS measures were Depression (feelings of sadness and worthlessness), Anxiety (fear, anxiety, and hyperarousal), Anger (irritability and frustration), Ability to Participate in Social Roles and Activities (involvement in one’s usual social roles and activities), Satisfaction with Social Roles and Activities (satisfaction in one’s usual social roles and activities), Emotional Support (feelings of being cared for and having supportive relationships), Informational Support (ability to access helpful information and advice), Social Isolation (feelings of being emotionally cut off from other people), Fatigue (feelings of tiredness and exhaustion related to being overwhelmed), and Sleep Disturbance (sleep quality, depth, and restorative sleep). All PROMIS measures were administered as computer-adaptive tests. Additional instruments were administered in this study, but were not included in this paper.

Participants were asked to complete PRO surveys through the PROMIS Assessment Center online platform (http://www.assessmentcenter.net), either at a designated computer at the site or on their own computer. After all PROs were completed, a short five-item evaluation survey was presented: 1) “Compared to what you expected, how would you rate your experience participating in this research study?” (a lot worse than expected, a little worse than expected, about the same as expected, a little better than expected, a lot better than expected); 2) “Would you recommend this research study to other people?” (no, maybe, yes); 3) “Do you have any comments or suggestions?” (open-ended); 4) “What is your overall rating of the design of the screens, including the colors and the layout?” (poor, fair, good, very good, excellent); and 5) “Did you receive any help using the computer or answering any of the questions?” (not at all, a little bit, some, a lot). Since these five evaluation questions were presented at the end of the online assessment, they were not presented to participants who stopped before completing all of the PRO surveys. Two or three of the authors independently coded responses to the open-ended evaluation question; all three coders met to resolve all discrepancies. Participant responses were coded as positive, negative, psychological, or other. Responses were coded as “positive” if most of the content of the response indicated that the participant had a positive opinion of the study and/or questionnaires, eg, “I think I learned a lot about myself.” Brief comments like “thank you” or “good luck” were also coded as “positive.” Responses were coded as “negative” if most of the content of the response indicated that the participant had a negative opinion of the study and/or questionnaires, eg, “I think the survey was a little too long.” Responses were coded as “psychological” if the participant “told a story” without content relevant to study burden, eg, “My daughter is in a nursing home and doesn’t go out much.” Responses were coded as “other” if they did not fit into the other categories, eg, “Make a mobile version.” If a response included multiple themes, classification as positive or negative took precedence. The amount of time needed to complete the surveys was automatically recorded by the assessment platform.

Statistical analysis

Item response theory-based scores for all TBI-CareQOL and PROMIS measures were standardized using T-scores (M=50, SD=10)¹⁵; higher scores indicate more of the trait being measured. Thus, for negative traits (Caregiver Strain, Caregiver-Specific Anxiety, Feeling Trapped, Feelings of Loss, Depression, Anxiety, Anger, Social Isolation, Fatigue, Sleep Disturbance), higher scores indicate worse HRQOL, whereas for positive traits (Ability to Participate in Social Roles and Activities, Satisfaction with Social Roles and Activities, Emotional Support, Informational Support) higher scores indicate better HRQOL.

The total time to complete the assessment was greater than 3 hours for 31% of the participants (n=140). Since many of these participants had data that spanned across multiple days, it was assumed that these individuals took breaks; thus, they were excluded from analyses that examined number of minutes for completion. Chi-squared tests, Fisher’s exact tests, and analysis of variance methods were used to compare the two caregiver groups on sociodemographic and clinical characteristics, PROs, and response burden indicators. Separate multivariable logistic regression models were estimated to evaluate the effects of caregiver group, education (more than high school vs less education), race/ethnicity (non-His-panic white vs others), MPAI-4 (low vs high functioning), and PROs on study participation experience (a little/a lot better vs same/worse), study recommendation (yes vs no/maybe), and help completing the assessment (some/a little/a lot vs none). ORs and 95% CIs were estimated. A chi-squared test was used to evaluate the association between help provided in completing the assessment and evaluation comments (positive vs negative vs psychological/other). A nominal significance level of 0.05 was used to interpret the results.

Results

Of the 473 individuals who were enrolled in the TBI-CareQOL study, 458 participants (97%) completed the evaluation questions at the end of the assessment and were included in the analyses; 335 were caregivers for civilians with TBI and 123 were caregivers for SMVs with TBI. The two groups were very different (Table 1). Compared to the group caring for civilians, the group caring for SMVs was comprised of all women, was younger, had fewer racial/ethnic minorities, had more education, and nearly all were the spouse of a person with TBI (all P<0.001). There was a higher proportion of low functioning SMVs compared to civilians (51% vs 12%; P<0.001). All HRQOL outcomes were poorer for the group caring for SMVs (all P<0.001).

Table 1 Sociodemographic and clinical characteristics, and health-related quality of life outcomes of study participants, by caregiver type

Sociodemographic and clinical characteristics	Caregivers of civilian TBI (n=335)	Caregivers of SMV TBI (n=123)	Total (n=458)	P-valuea
Caregiver
Female gender	259 (77%)	123 (100%)	382 (83%)	<0.001
Age (years)	51.8 (13.9)	35.6 (8.6)	47.5 (14.6)	<0.001
Ethnicity, race
Hispanic, any race	36 (11%)	10 (8%)	48 (10%)	<0.001
Non-Hispanic, white	212 (63%)	88 (72%)	300 (66%)
Non-Hispanic, black	68 (20%)	6 (5%)	74 (16%)
Other	12 (4%)	13 (11%)	25 (5%)
Missing	5 (1%)	6 (5%)	11 (2%)
Highest education
Less than HS	45 (13%)	3 (2%)	48 (10%)	<0.001
HS/GED	45 (13%)	5 (4%)	50 (11%)
More than HS	245 (73%)	115 (94%)	360 (79%)
Relationship to person with TBI
Spouse	123 (37%)	118 (96%)	241 (53%)	<0.001
Parent	121 (36%)	5 (4%)	126 (28%)
Child/other family member	64 (19%)	0	64 (14%)
Other	27 (8%)	0	27 (6%)
Years in caregiver role	6.0 (6.5)	5.6 (2.9)	5.9 (5.7)	0.410
Person with TBI
Female gender	85 (25%)	0	85 (19%)	<0.001
Age (years)	42.4 (14.7)	36.1 (7.3)	40.7 (13.4)	<0.001
Time since injury (years)	6.9 (5.5)	8.6 (4.7)	7.4 (5.4)	0.003
Level of functioning (MPAI-4)
High functioning/low impairment	293 (87%)	57 46%)	350 (76%)	<0.001
Low functioning/high impairment	42 (13%)	63 (51%)	105 (23%)
Missing	0	3 (2%)	3 (1%)
Caregiver health-related quality of life
TBI-CareQOL
Caregiver strain	47.5 (9.8)	55.2 (7.5)	49.6 (9.8)	<0.001
Caregiver-specific anxiety	47.1 (9.3)	56.8 (8.0)	49.8 (9.9)	<0.001
Feeling trapped	46.5 (9.2)	57.2 (6.6)	49.4 (9.8)	<0.001
Feelings of loss-self	48.2 (9.8)	55.4 (7.5)	50.2 (9.8)	<0.001
Feelings of loss-person with TBI	49.6 (10.2)	52.9 (7.3)	50.5 (9.6)	<0.001
PROMIS
Anger	51.2 (9.8)	59.2 (9.4)	53.4 (10.3)	<0.001
Anxiety	52.4 (9.1)	61.3 (8.6)	54.8 (9.8)	<0.001
Depression	50.2 (9.6)	57.5 (9.7)	52.2 (10.1)	<0.001
Ability to participate in social roles and activities	51.9 (9.5)	43.3 (8.0)	49.6 (9.9)	<0.001
Satisfaction with social roles and activities	48.7 (9.0)	41.2 (6.9)	46.7 (9.1)	<0.001
Emotional support	49.7 (9.2)	44.4 (8.5)	48.3 (9.3)	<0.001
Informational support	50.0 (10.7)	46.3 (9.9)	49.0 (10.6)	<0.001
Social isolation	48.0 (9.4)	56.8 (9.3)	50.3 (10.1)	<0.001
Fatigue	51.8 (9.6)	59.5 (9.2)	53.9 (10.1)	<0.001
Sleep disturbance	51.3 (9.4)	59.7 (9.8)	53.6 (10.2)	<0.001

Notes: Entries in the table represent the number of participants (percentage), or the mean (SD).

a P-value: chi-squared test, Fisher’s exact test, or independent t-test for comparison of the caregiver groups, excluding missing data.

Abbreviations: GED, General Educational Development Test (High School Equivalency Diploma); HS, high school; MPAI-4, Mayo-Portland Adaptability Inventory-Fourth Edition; TBI, traumatic brain injury; SMV, service member/veteran; PROMIS, Patient-Reported Outcomes Measurement Information System.

Table 2 summarizes information about the survey evaluation and response burden indicators. About half in each caregiver group reported that their study participation experience was about the same as expected, and about half reported that their experience was better than expected; only 6% in each group reported that their experience was worse than expected. The group caring for SMVs was more likely to report that they would recommend this study to others (P=0.002), and they were more likely to have comments about their study participation (P=0.006). Among those who had comments, about half in each group had negative comments. About half in each group rated the screen design as very good or excellent. The group caring for civilians needed more help to complete the questionnaires (P=0.03) but took about the same length of time to complete them as the group caring for SMVs.

Table 2 Survey evaluation and response burden, by caregiver type

	Caregivers of civilian TBI (n=335)	Caregivers of SMV TBI (n=123)	Total (n=458)	P-valuea
Study participation experience
A lot worse/a little worse than expected	21 (6%)	8 (6%)	29 (6%)	0.440
About the same as expected	149 (44%)	61 (50%)	210 (44%)
A little better than expected	87 (26%)	23 (19%)	110 (23%
A lot better than expected	78 (23%)	31 (25%)	109 (23%)
Would recommend this study to others	271 (81%)	115 (94%)	386 (84%)	0.002
Comments about study participation
None	164 (49%)	77 (63%)	241 (53%)	0.006
Type of comment
Positive	59 (35%)	19 (41%)	78 (36%)	0.321
Negative	82 (48%)	24 (52%)	106 (49%)
Psychological	20 (12%)	2 (4%)	22 (10%)
Other	10 (6%)	1 (2%)	11 (5%)
Overall rating of screen design
Poor/fair	32 (10%)	14 (11%)	46 (10%)	0.890
Good	122 (36%)	44 (36%)	166 (36%)
Very good	109 (32%)	43 (35%)	152 (33%)
Excellent	49 (15%)	22 (18%)	71 (16%)
Missing	23 (7%)	0	23 (5%)
Help using the computer or answering the questions
Not at all	266 (79%)	110 (89%)	376 (82%)	0.030
A little bit	27 (8%)	4 (3%)	31 (7%)
Some	15 (4%)	6 (5%)	21 (5%
A lot	27 (8%)	3 (2%)	30 (7%)
Time needed to complete the questionnaires
≤3 hours	229 (68%)	89 (72%)	318 (69%)	0.466
Minutes to complete the questionnairesb	78 (30)	71 (28)	76 (29)	0.070

Notes: Entries in the table represent the number of participants (percentage), or the mean (SD).

a P-value: chi-squared test, Fisher’s exact test, or independent t-test for comparison of the caregiver groups, excluding missing data;

b including only those who took 3 hours or less (n=318).

Abbreviations: TBI, traumatic brain injury; SMV, service member/veteran.

Caregivers were more likely to rate their study participation experience as a little/a lot better than expected if they had lower education or were racial/ethnic minorities (Table 3). Caregiver group, MPAI-4 function, and HRQOL outcomes were not associated with study participation experience (all P>0.10).The group caring for SMVs was more likely to report that they would recommend this study to others. Education, race/ethnicity, MPAI-4 function, and HRQOL outcomes were not associated with study recommendation (all P>0.10).

Table 3 Logistic regression models for evaluation/burden indicators

Dependent variable	Independent variable
	SMV vs civilian	More than HS education vs less	Non-Hispanic white vs other
Study participation experience: a little/a lot better than expected vs same/worse	–	0.51 (0.32, 0.82)	0.54 (0.36, 0.81)
Would recommend this study to others: yes/maybe vs no	3.40 (1.58, 7.31)	–	–
Help using the computer or answering the questions: some/a little/a lot vs not at all	–	0.44 (0.26, 0.75)	0.36 (0.22, 0.59)

Participants with a more favorable overall rating of their study participation experience had a greater proportion of positive comments compared to those with less favorable ratings (Figure 1; P<0.001). Specifically, 56% and 48%, respectively, of those who rated their experience as a lot better or a little better than expected had positive comments compared to 4% and 27%, respectively, for those who rated their experience as worse or about the same.

Figure 1 Association between rating of study participation experience and open-ended comments (P<0.001).

Those with less education and those from racial/ethnic minority groups needed more help using the computer or answering questions (Table 3). Caregiver group, MPAI-4 function, and HRQOL outcomes were not associated with the need for help (all P>0.10).

Discussion

Participant ratings of survey response burden provide useful information for researchers and clinicians on the acceptability of a PRO instrument for a particular population and use. This is the first study to measure multiple indicators of PRO survey response burden in caregivers of civilians and SMVs with TBI. Overall, caregivers had generally favorable ratings of study participation experience, most would recommend the study to others, and most needed little help using the computer or answering the questions. Multivariable analyses of these burden indicators showed that caregivers with less education and those from racial/ethnic minority groups had more favorable ratings of their study participation experience, even though they needed more help using the computer or answering the questions. Receiving help with survey completion allowed them to participate successfully in the study. It is possible that some individuals may consider survey participation as a form of social support.

Although caregivers of SMVs in this study had poorer HRQOL outcomes compared to caregivers of civilians, they were more likely to report that they would recommend the study to others. HRQOL outcomes were not associated with study recommendations. It may be that these caregivers were simply more appreciative to be the focus of research efforts designed to better understand and ultimately improve their HRQOL. Anecdotally, many caregivers mentioned that they have few opportunities to participate in research. In addition, qualitative data suggest that caregivers of SMVs often do not feel like they have the opportunity to express their emotions, and in fact they feel the need to put on a brave face for others.⁵ Thus, it may also be possible that providing them with the opportunity and a relatively anonymous space to express their feelings might have been cathartic and may have led them to recommend this study to others.

There are some limitations to this study. Because convenience sampling was used, generalizability is limited with regard to racial/ethnic minorities, male caregivers, and caregivers who are parents or relations other than spouses. Caregivers of SMVs were primarily recruited through support organizations, and so results may not be representative of those who care for a person with a military-related TBI. It will be important to recruit caregivers through military medical centers in future studies. In addition, SMVs with TBI have high rates of other comorbid clinical conditions³⁴ which could increase the caregiver burden. Comorbidities were not collected for this study, but would be important to incorporate in the design of future studies. The majority of the caregivers completed the survey at home; thus, it is possible that their survey evaluation was influenced by others.

PROs are widely gathered in clinical and health services research; they are increasingly collected and used in clinical practice settings as well.^35–40 PROs are relevant for many activities: helping patients and their clinicians make informed decisions about health care, monitoring the progress of care, setting policies for coverage and reimbursement of health services, improving the quality of health care services, and tracking or reporting on the performance of health care delivery organizations.²¹ It might be useful to incorporate PROs into the TBI care programs developed by the Department of Defense and the Veterans Affairs Health Care Systems.⁴¹

The TBI-CareQOL measurement system offers unique measures of caregiver-specific HRQOL. The results of this study provide useful information about the acceptability of computer-based survey administration in caregivers of civilians and SMVs. Future work could consider adapting the TBI-CareQOL measurement system for caregivers of people with other conditions, eg, Alzheimer’s disease. More data are needed to determine the best assessment strategies for individuals with lower education who are likely to require some assistance in completing PRO surveys. For example, studies evaluating PROs administered by multimedia platforms^42–47 could help researchers and clinicians plan the best strategies for assessing HRQOL in TBI caregivers.

Acknowledgments

This work was supported by the National Institute of Nursing Research under Grant #R01-NR013658.

Disclosure

The authors report no conflicts of interest in this work.