Reliability in coding social interaction: A study of confirmation

This study sought (a) to determine, under relatively ideal conditions, whether confirmation and disconfirmation can be coded reliably by trained observers, and (b) to provide a tutorial on coding reliability using data from the study of reliability in confirmation. After discussing five problems in determining and reporting the reliability of systems for coding social interaction, including confirmation and disconfirmation, the findings from the analysis of ten videotaped interviews are reported. The tapes were scored by two experienced and trained coders, using the Confirmation Disconfirmation Rating Instrument. Unitizing reliability was good, and global reliability was excellent. Category‐by‐category reliability, both inter‐rater and test‐retest, was acceptable for the two major categories of confirmation and disconfirmation, but unacceptable for the six subcategories. The implications section draws attention particularly to the necessity in all studies involving the coding of social interaction (a) of using appropriate statistics to calculate reliability and (b) of determining category‐by‐category reliability coefficients. Implications to future studies of confirmation are also discussed.

Notes

Kenneth N. Cissna is an Associate Professor of Communication, University of South Florida. Bonnie J. Garvin is an Associate Professor of Family and Community, and Carol W. Kennedy is an Associate Professor of Life Span Process, College of Nursing, The Ohio State University.