Surveying Pornography Use: A Shaky Science Resting on Poor Measurement Foundations

ABSTRACT

A great deal of pornography research relies on dubious measurements. Measurement of pornography use has been highly variable across studies and existing measurement approaches have not been developed using standard psychometric practices nor have they addressed construct validation or reliability. This state of affairs is problematic for the accumulation of knowledge about the nature of pornography use, its antecedents, correlates, and consequences, as it can contribute to inconsistent results across studies and undermine the generalizability of research findings. This article provides a summary of contemporary measurement practices in pornography research accompanied by an explication of the problems therein. It also offers suggestions on how best to move forward by adopting a more limited set of standardized and validated instruments. We recommend that the creation of such instruments be guided by the careful and thorough conceptualization of pornography use and systematic adherence to measurement development principles.

Acknowledgments

We would like to thank our research assistant, Emma M. Guimaraes, for all of the work she provided with respect to searching for, collecting and organizing the 100 pornography measurement articles we reviewed in this manuscript.

Notes

¹ Indeed, some have suggested that “feminine hygiene” products with implied contraceptive functions (e.g., antiseptic douches, creams, jellies, etc.) were introduced by marketers in the 1920s to circumvent the legal regulation of “obscenity” (Sarch, 1997) which at the time was widely applied to both explicit representations of sexuality as well as information about birth control and birth control devices (e.g., condoms).

² Unbeknownst to us, while we were conducting this review, parallel efforts were also underway by others (see Marshall & Miller, 2019), who published findings that were generally consistent with Short et al.’s (2012) while our manuscript was undergoing peer review.

³ The question of how pornography use should be operationalized in the experimental literature is a related but somewhat separate issue as other aspects of validity must also be considered. Regardless, all operational definitions, including experimental manipulations, should stem from a well-considered conceptual definition of the construct of interest, and while our focus in the current paper involves the measurement of pornography use, the preceding discussion may provide some useful guidance for experimental work as well. We would note that the application of a common conceptual definition of pornography use across surveys and experiments would make both branches of research more mutually informative of one another. For this reason, we encourage the field to further consider the extent that past, current, and future experimental manipulations conform to conceptual definitions of pornography use that are used in survey research and vice-versa.

⁴ However, as Fisher and Barak (2001) have cautioned, it is critical to avoid over-crediting the potential impact of intermittent contact with pornography and to avoid “monkey see, monkey do” assumptions about effects of pornography. As these authors noted, the monkey has a brain – an extensive learning history signaling what actions are likely to be acceptable and what is likely to be punished and guilt inducing. Such learning histories accrue over a lifetime of frequent social interactions with others and should heavily influence the effects of comparatively intermittent exposure to pornography.

⁵ We note that there is an emerging literature that uses eye-tracking equipment to examine what people attend to when they look at sexual imagery (Wenzlaff, Briken, & Dekker, 2016). Our point here is not to overlook such research, but to suggest that there is simply not enough studies of this sort to fully inform our theoretical understanding of specific conceptual definitions of sexually violent pornography.

Additional information

Funding

This research was funded by an Insight Grant (PI: Dr. W. Fisher) provided by the Social Sciences and Humanities Research Council.