Examining the Construct Validity of Enacted Support: A Multitrait–Multimethod Analysis of Three Perspectives for Judging Immediacy and Listening Behaviors

Abstract

Scholars of supportive communication are primarily concerned with how variations in the quality of enacted support affect individual and relational health and well-being. But who gets to determine what counts as enacted support? There is a large degree of operational heterogeneity for what gets called enacted support, but little attention has been afforded to the issue of whether these assessments are substitutable. In two studies we use self-reports, conversational partner-reports, and third-party ratings of two quintessential behavioral support indicators, namely, listening and immediacy. Using a multitrait–multimethod (MTMM) design, Study 1 found (1) little association between the enacted support assessments and (2) a high degree of common method variance. A second study found moderate-to-high degrees of effective reliability (i.e., consistency of judgments within a set of judgments, or mean judgments) for enacted support evaluations from the perspective of unacquainted and untrained third-party judges. In general, our data provide cautionary evidence that when scholars examine evaluations of enacted support, perspective matters and might ultimately contribute differently to well-being and health.

Keywords:

• Active Listening
• Comfort
• Stress
• Judgment Studies
• Perceived Support
• Social Support

Acknowledgments

The authors would like to thank Tim Levine for serving as a consultant on Study 1. The authors also would like to thank Jonathan Denham, Michelle Pence, Trey Gibson, Logan Sacco, McCade McDaniel, Elizabeth McKee, Daniel Chapman, Lori Castano, Amanda Legrand, Nickole Hojnowski, Allison O'Neill, Dan Barberio, Billy Boland, and Kristin Carlson for their assistance with various aspects of data collection and coding. This study was supported by a Pilot Funding for New Research (Pfund) grant and a Research Competitiveness Subprogram (RCS) grant, both awarded to Graham D. Bodie from the Louisiana Board of Regents.

Notes

[1] Of these 171 dyads, 41 consisted of a trained confederate, whereas 130 paired two students unfamiliar with each other. This study reports on a subset of the 130 stranger dyads, namely, the 103 with full information to construct the MTMM matrix. Information about other aspects of this project is available upon request.

[2] We considered as adequate models that exhibited a CFI value at or above .90 and a SRMR value at or below .08. Chi-square values were not considered in judging model fit. Although the approximation error (RMSEA) is reported for all scales, this value, when high, is not interpreted as an indication of poor model fit. Model fit is influenced by sample size and degrees of freedom (Kenny, Kaniskan, & McCoach, 2011), and our data featured small sample sizes for all measures and low degrees of freedom for many of the scales, which consequently attenuated model fit.

[3] We also computed several other measures of internal consistency, each of which was comparable to the alpha values. Because most readers will be familiar with Cronbach's alpha, we report those values for convenience.

[4] In the case of Priem et al., the observers were only minimally trained. They were provided a definition of supportiveness and were permitted to view the video multiple times before submitting evaluations. Other researchers, including the second author, have used more extensive training techniques (e.g., Jones & Guerrero, 2001).

[5] The data presented for Study 2 have not been previously published, although the data-set under question has been used for other purposes.

[6] We then randomly split the 72 conversations into two sets of 36. Although the initial plan was to collect data for 36 conversations at each research site, many students at the University of Minnesota, Twin Cities, who had signed up for time slots did not show up to the lab. Consequently, the majority of the data were collected at Louisiana State University A&M (n = 309). All 72 conversations were utilized, and between two and seven participants viewed each video.

[7] We asked listeners three questions regarding the normalcy of their behavior, and each of these three questions generated a mean above the midpoint (M = 5.69, SD = 1.34). Likewise, disclosers responded that they did not hide feelings (M = 2.36, SD = 1.48) and that they talked about their “true feelings” (M = 5.72, SD = 1.37). Nevertheless, we realize that the situation in which we placed participants is far from a “normal” conversational setting and, thus, has ramifications for the generalizability of our data.