Reporting Mobile Social Media Use: How Survey and Experience Sampling Measures Differ

ABSTRACT

Quantifying the ubiquitous, ephemeral, and highly diverse patterns of mobile social media (MSM) use is a challenge for communication research. Most researchers employ retrospective survey measurement, thus depending on the accuracy of users’ memories and generalizations. Alternatively, some researchers rely on in-situ measurement, being less dependent on users’ memories and generalizations, but requiring random situation samples. To assess differences and similarities between these two measurement approaches we analyzed whether characteristics (duration and frequency of a usage episode, habit, elaboration, and gratifications) of MSM use (regarding Facebook, WhatsApp, and YouTube) vary between retrospective survey and mobile experience sampling measurement. We observe a consistent pattern of higher estimates in retrospect as compared to individual averages of in-situ reports. The absolute magnitude of these differences varies considerably between platforms and characteristics studied. Nonetheless, for most constructs and platforms we find low significant positive correlations between retrospective and aggregated in-situ values.

Acknowledgments

We would like to thank the editor and the anonymous reviewers for their thoughtful comments on an ealier version of the article.

Notes

1. We chose this time frame as the by far biggest part of media use of young people (ages 14–29; Feierabend et al., 2016) in Germany happens during these hours. In addition, we did not want to burden our participants any further by alerting them during night time, when they might be sleeping.

2. The arithmetic mean is the standard aggregation procedure to compute individual level data from repeated in-situ observations of individual participants. To test whether the aggregation procedure influences the results when comparing aggregated in-situ values (2) and retrospective values (1), we applied different aggregation procedures regarding duration, habit, and elaboration for the three platforms. For instance, we derived at aggregated in-situ values (2a) by computing the median of each respondent’s in-situ measures and (2b) by computing the arithmetic mean of each respondent’s in-situ measures excluding probable outliers (z ≥ 1.96) of the respective individual respondent. However, the aggregated in-situ values computed as arithmetic mean (2) correlated strongly (r above .90) and significantly (p < .001) with the aggregated in-situ values computed as median (2a) and computed as arithmetic mean excluding outliers from the aggregation (2b). This holds for all tested constructs (duration, habit, elaboration, and gratifications) and for all three platforms. Thus, in the paper we will only report results of the standard aggregation using the arithmetic mean (2).

3. More detailed analyses show that correlations and partial correlations between retrospective and aggregated in-situ values have very similar levels when only controlling for report latency. Additionally, introducing the number of ESFs per participant and platform as control variable leads to three significant changes (see Tables 1 and 4). Thus, we assume that while report latency does not introduce relevant measurement distortion, a greater number of ESF reports on which the aggregation of in-situ data is based stabilizes the aggregated MESM values.