Considering the ‘With Whom’: Differences Between Event- and Signal-Contingent ESM Data of Person-Specific Social Interactions

Abstract

Experience sampling studies often aim to capture social interactions. A central methodological question in such studies is whether to use event- or signal-contingent sampling. The little existing research on this issue has not taken into account that social interactions occur with unique interaction partners (e.g., Anna or Tom). We analyze one week of social interaction data of 286 students from the University of Pittsburgh (60.8% male, mean age 19.2 years), taking into account the unique interaction partners of each student. Specifically, we investigate the differences between event- and signal contingent sampling in (1) the total number of unique interaction partners captured, as well as (2) the kinds of relationships, and (3) the quality of social interactions with these captured interaction partners. Apart from a larger quantity of interactions and unique interaction partners in the event-contingent sampling design, our analyses indicate subtle differences between the two designs when aiming to assess social interactions with more distant interaction partners, such as coworkers or strangers. Most importantly, in our analyses, specific interaction partners and social roles explained a considerable amount of variance in the quality of social interactions (up to 20.5%), suggesting that future research would benefit greatly from considering “with whom” individuals interact.

Keywords:

• Ecological momentary assessment
• ambulatory assessment
• experience sampling methodology
• social context
• social interactions

In recent years, it has become easier to study mood in daily life due to advances in methodology (Silva & Cotter, 2021). Researchers more often focus on the bidirectional relationship between social interactions and mood, as this is relevant to many fields, such as mental and physical health (e.g., Brown et al., 2011; Elmer et al., 2020; Kopala-Sibley et al., 2012), attachment (Kaurin et al., 2022), well-being (Achterhof et al., 2022), or job satisfaction (Dimotakis et al., 2011). One method for capturing these dynamics is experience sampling methodology (ESM), also known as Ecological Momentary Assessment (EMA), which involves participants completing short smartphone surveys throughout their daily lives.

ESM is an increasingly popular tool to study social processes within daily life as it allows researchers to capture real-time, person-specific social interactions and their associations with, for example, momentary well-being by offering a dynamic and ecologically valid perspective (Myin-Germeys & Kuppens, 2021). By minimizing recall bias and providing rich, momentary data, ESM enables researchers to examine social processes as they naturally unfold, ultimately contributing to a better understanding of social experiences, their perceptions, and their implications for psychological processes (Stadel et al., 2023).

Designing an ESM study can, however, be complex, and to date, there are few established methodological standards. A central question when setting-up an ESM study is the sampling design (Myin-Germeys & Kuppens, 2021; Silva & Cotter, 2021). Two common designs are event- and signal-contingent sampling. When applying an event-contingent design, participants are asked to initiate assessments themselves, any time an event of interest (e.g., a social interaction, smoking a cigarette, or a binge-eating episode) occurs. In contrast, a signal-contingent design entails using a predefined number of (semi-)random or fixed assessment prompts distributed over the course of a day. These prompts ask about occurrences of the event of interest (e.g., social interactions) since the last assessment or in the moment itself.

To date, little is known to which extent the decision to use an event- or signal-contingent sampling design to capture social interactions affects gathered data and thus the substantive findings of a study. To our knowledge, there are only two studies explicitly comparing the two designs (Dawood et al., 2020; Himmelstein et al., 2019). These studies come to different conclusions but also differ considerably in their aims and methodology.

Himmelstein et al. (2019) researched whether collecting data on social interactions with event- or signal-contingent experience sampling leads to systematic differences in data quality or quantity. They recruited students from the University of Pittsburgh who were randomly assigned to either event- or signal-contingent sampling of social interactions. Based on comparisons of mean and variance of interaction quality (i.e., affect, warmth, and dominance) between the two conditions they concluded that the two sampling designs may be used interchangeably. They further concluded that participants in the event-contingent condition indicated approximately one-third more interactions than the ones in the signal-contingent condition. Additionally, participants in the event-contingent condition were more likely to report interactions during the day (between 7 am and 7 pm) and later evening (between 10 pm and 1 am). Between 7 and 10 pm, the likelihood of interaction reporting was similar in both conditions. Overall, Himmelstein et al. note that while there is a difference in data quantity between recording interactions with event-contingent as opposed to signal-contingent designs, there appears to be none in quality.

Dawood et al. (2020) researched whether affect ratings differ when collected with a signal- vs. event-contingent sampling design. They conducted a within-subject study (N = 40) with psychotherapy-outpatients who reported on their affect for 21 days via six semi-random ESM prompts per day. During the same 21-day period, participants also reported on their social interactions in an event-contingent manner. These event-contingent reports included momentary affect ratings as well. The results of Dawood et al. show that semi-random affect reports and event-contingent affect reports tied to a social interaction differ in mean and variability. Specifically, patients reported on average higher arousal and more positive valence in event-contingent interaction assessments. The signal-contingent assessments, on the other hand, showed more within and between-person mood variability. Thus, Dawood et al. showed that affect tied to social interactions is on average more positive and with a higher arousal than affect measured semi-randomly throughout the day. As this investigation did not contrast signal-contingent reports of social interactions with event-contingent reports of social interaction specifically, a comparison to the investigation of Himmelstein et al. is difficult.

Given the limited prior research, many questions regarding the effects of sampling design on the assessment of daily-life social interactions remain open. Importantly, none of the studies considered with whom a social interaction took place—neither unique interaction partners (e.g., Anna or Tom), nor a broader social role classification (e.g., friend or coworker). Differentiating between interaction partners allows us to examine their unique effects on interaction quality and study with whom social interactions are for example particularly positive or negative. Even if it is not the main interest of a study to make such a differentiation, the fact that social interactions take place with a specific interaction partner, and that likely multiple interactions happen with the same partner, creates dependencies between assessments. This dependence can and should be accounted for in analyses (Perry et al., 2018) and not doing so may have masked differences between the two sampling designs in previous research.

Additionally, considering with whom in ESM research enables a more comprehensive exploration of social support networks. By identifying key individuals in an person’s daily life, researchers can assess how certain interactions contribute to momentary well-being or serve as sources of stress. This knowledge can inform interventions aimed at enhancing social support. Furthermore, studying specific interaction partners or roles can shed light on the development of relationships over time. It allows researchers to assess changes in interaction quality or frequency with particular individuals, providing valuable data for understanding the dynamics of evolving social relationships. Lastly, accounting for the “with whom” aspect in ESM research aligns with the broader goal of personalization and context-specificity in psychological studies. It recognizes that individuals may have unique patterns of interaction and emotional responses with different people, emphasizing the need for tailored interventions and treatment strategies based on an individual’s specific social context (Stadel et al., 2023).

Yet, to date, it is not known how omitting such with whom information could bias analyses aiming to explain social interaction outcomes (e.g., quality, affect). In other words, we do not know how much variance in social interaction outcomes can be attributed to information on with whom individuals interact.

The current study

The goal of the current study is to reanalyze the data collected by Himmelstein et al. (2019), differentiating between different interaction partners. We illustrate a more elaborate modeling approach for social interaction data than common ESM analysis strategies (i.e., two-level models with social interactions nested in participants) by including interaction partners as a further level of analysis. This results in a three-level model, with social interactions nested within interaction partners nested within participants (see Perry et al., 2018, p. 262). Such more detailed modeling can help to shed light on complex social processes in daily life by taking information about specific interaction partners/social network members into account. Figure 1 shows an illustration of the three-level nested structure.

Figure 1. Data structure of ESM social interaction assessments.

Specifically, we test whether the data obtained using the two ESM sampling designs differ with regard to the number of captured unique interaction partners and the quality of relationships with these partners. We also reexamine differences in the quality of reported social interactions—now taking into account the unique interaction partner with whom an interaction takes place. Considering the with whom allows us to further compare social interaction data obtained via an event- or signal-contingent sampling design.

Differences in the number of unique interaction partners

Given that fewer interactions are captured with signal-contingent ESM (Himmelstein et al., 2019), it is likely that also fewer unique interaction partners are captured. Thus, our first research question (RQ1) is whether systematically fewer unique interaction partners (Level 2 in Figure 1) are captured with signal- vs. event-contingent experience sampling reports.

Differences in the quality of relationships with captured interaction partners

Besides a different number of captured interaction partners, event- and signal-contingent sampling may result in assessing different kinds of interaction partners in terms of social roles or relationship quality (e.g., closeness). As an event-contingent design captures a substantial number of interactions outside of the common signal-contingent period (i.e., 10 am to 10 pm), it may be that this design captures more interactions with leisure-related social roles that take place, for example, with friends and family as opposed to teachers, superiors, and colleagues.

Naturally, participants have a different relationship with friends than with colleagues. Hence, the closeness of interaction partners may differ as well between the two sampling designs. Additionally, signal-contingent sampling may bias participants toward reporting more memorable interactions and thus underrepresent shorter encounters (e.g., chit-chat with a colleague) as interactions are not recorded immediately (Leertouwer et al., 2021). Thus, our second research question is: Do event- and signal-contingent sampling capture different kinds of relationships with interaction partners in terms of (a) social roles (RQ2a) and (b) relationship quality ratings, such as closeness (RQ2b)?

Differences in the quality of the captured social interactions

Further, we are interested to see if the two sampling designs differ in terms of the quality of captured interactions when taking interaction-partner-specific effects into account. For example, interactions with Alice are of higher quality than the average interaction of the participant. Such interaction-partner-specific effects would be masked in the original analyses of Himmelstein et al. due to averaging characteristics of social interactions across all interaction partners. Therefore, we expand on their analyses by using the three-level framework presented in Figure 1 to answer the following research question: Do the event- and signal-contingent sampling designs differ in the quality of captured interactions (i.e., warmth and dominance of self and other, positive and negative affect) when taking into account unique interaction partners (RQ3)?

In the following, we will describe our methods and results regarding our three research questions. We pre-registered our analyses using the registration template for secondary data analysis (Akker et al., 2019; Weston et al., 2019). The registration, a document outlining deviations from the registration, an R-Markdown file with all analysis code and results as well as Supplementary Materials can be found on the Open Science Framework.¹

Methods

Participants

We used data collected by Himmelstein et al. (2019). This sample is composed of undergraduate students of Pittsburgh University in the US, who received course credits as compensation for their research participation. Additionally, participants were entered into a draw for Apple Watches, with their winning chances increasing with higher experience sampling compliance rates. The dataset includes a total of 10,260 observations (6028 event-contingent and 4232 signal-contingent) spread across 286 participants (38.8% female) with a mean age of 19.22 years (SD = 1.74). As Himmelstein et al. also asked participants to indicate the initials of their interaction partner, it is possible to link ESM social interaction assessments to unique interaction partners.

The presented analyses only include observations that captured a social interaction and contained the initials of the unique interaction partner. All initials were checked for eligibility and, when necessary, recoded into a consistent format. A detailed overview of recoded and excluded initials as well as the applicable reasons can be found in the Supplementary Materials on OSF. Further, we removed observations according to our pre-registered inclusion criteria for ensuring data quality on the initials variable: (a) We identified five participants (51 observations) who interacted with fewer than two different interaction partners during the study period and (b) 13 participants (278 observations) for whom <⅔ of all interaction data contained usable initials. Four participants presented with both issues. The final dataset included 6252 social interactions of 269 participants, of which 3810 social interactions of 139 participants were in the event-contingent condition and 2442 social interactions of 130 participants in the signal-contingent condition.

The social roles of interaction partners were unfortunately only available for 4932 observations of 212 participants, of which 3105 social interactions of 112 participants were in the event-contingent condition and 1827 social interactions of 100 participants in the signal-contingent condition. Missingness of these social role data is due to erroneous coding of this variable for a substantive part of the original sample (Number of observations = 1320, Number of participants = 57). Because the correct raw answers to this question are unrecoverable, for these participants all answers about the social role of their interaction partner are missing. A full overview of our participant and observation flow can be found in Figure 2.

Figure 2. Participant flow.

Procedure

First, participants were invited to the lab for explanations and instructions about the study procedure. During this session, participants also filled in a baseline questionnaire. Then, participants were randomly assigned to one of the two sampling design conditions. In the following seven days, participants were asked to report on social interactions with either event- or signal-contingent experience sampling. Social interactions were defined as an exchange between the participant and at least one other person lasting for at least 5 min. All experience sampling assessments were done with the MetricWire application (Metricwire Inc. | Real-World Data | Real-Life Impact, 2022) on students’ personal smartphones.

Participants in the signal-contingent condition received six semi-random prompts per day between 10 am and 10 pm for seven days. Prompts were sent at least 90 min apart and participants were reminded to respond to a given prompt after 15 min. The questionnaire would become unavailable 30 min after the original prompt was sent. During each of the assessments, participants were asked to report on the most significant interpersonal interaction with a single individual since the last assessment. Participants only received their full credit compensation for a compliance rate of 67% (on average four out of the six daily prompts).

Participants in the event-contingent condition were asked to self-initiate assessments between 10 am to 10 pm immediately after the occurrence of each social interaction (as defined above). For compensation, participants were required to complete at least four assessments per day. Participants in this condition could also report outside of the daily 12-h assessment period if they wanted to. The content of the ESM questions was the same as in the signal-contingent condition.

Measures

In the following, we describe the measures that are used for the analyses of the present article. For more information on all measures and further details about the data collection, we refer to Himmelstein et al. (2019).

Interaction partner

For every social interaction, participants were asked to indicate the initials or nickname of the interaction partner and which social role the partner had (Boss/Teacher = 0, Coworker = 1, Family member = 2, Friend/Acquaintance = 3, Romantic Partner/Spouse = 4, Roommate = 5, Someone I've not met before = 6, Other = 7).

Relationship quality

Three items from the Social Provisions Scale (SPS; Cutrona & Russell, 1987) were used to measure different facets of relationship quality with the current interaction partner: “Feeling Respected” (“The person I interacted with respects my skills and abilities”), “Closeness & Emotional Security” (“I have a close relationship with the person I interact with that provides me with a sense of emotional security and well-being”) and “Trustworthiness & Advice Seeking” (“The person I interacted with is a trustworthy person I could turn to for advice if I were having problems”). Participants were asked to respond on a visual analog scale from Strongly Disagree (= 0) to Strongly Agree (= 100). These items were assessed for every social interaction observation and were considered as separate variables.

Dominance and warmth

Participants were also asked for a rating of their own behavior in the interaction, as well as a rating of the interaction partner’s behavior using the Visual Interpersonal Analogue Scale (Woods et al., 2022): “Please rate how the other person behaved toward you during this interaction” and “Please rate your behavior toward the other person during this interaction” from Accommodating/Submissive/Timid (= −50) to Assertive/Dominant/Controlling (= 50) for “Dominance” and from Cold/Distant/Hostile (= −50) to Warm/Friendly/Caring (= 50) for “Warmth.”

Momentary affect

To measure affect, the 10-item version of the Positive and Negative Affect Schedule (PANAS; Watson & Clark, 1994) was used. This included the following five positive affect dimensions: “content,” “relaxed,” “happy,” “proud,” and excited.” The five negative affect dimensions were: “sad,” “nervous,” “angry,” “hostile,” and “ashamed.” Participants were asked to rate to what extent the adjective described how they felt on a 100-point visual analog scale from not at all (= 1) to extremely (= 100). Measures were always tied to a social interaction (e.g., “During the interaction, how happy did you feel?”).

Analysis

RQ1: Differences in the number of unique interaction partners

We determined the number of unique interaction partners for each participant, by using the captured initials and social roles (i.e., each unique initial/social role combination constituted a unique interaction partner). Then, we compared this variable between the two conditions with a two-tailed two-sample t-test. Because participants in the event-contingent condition had more opportunities to report social interactions (and in fact also did report more interactions), we additionally ran an analysis with a relative score of interaction partners. Specifically, we divided the raw number of unique interaction partners of a participant by the total number of interactions reported by that participant. We used a two-tailed two-sample t-test on this rate of reporting unique interaction partners given the total number of opportunities to report an interaction partner.

RQ2a: Differences in the reporting rates of specific social roles

To compare the event and signal-contingent sampling designs with regards to the captured social roles we examined the counts of unique interaction partners with a given social role (e.g., the number of captured friends) using a series of multilevel Poisson regression models. Similar to our strategy in answering the first research question, we correct for the larger number of opportunities to capture new interaction partners (i.e., the larger number of captured social interactions) in the event-contingent condition. This means that we are estimating the rate of interaction partners of a given role. Such data (i.e., counts and rates) cannot be modeled appropriately with a linear multilevel model. Hence, we turn to the Poisson regression framework (see e.g., Gelman & Hill, 2006). With this model, we can determine whether an independent variable, for example, sampling-design, is associated with differences in reporting rates of a given role. In our model, the rates on Level 1 are nested in participants on Level 2. Note that this is a different nesting structure than depicted in Figure 1. As the main variable of interest is social role, these analyses were performed on the social roles subsample (see Figure 2). When examining the distributions of the interaction partner counts for each social role separately, it became apparent that all these distributions show overdispersion except for friends/acquaintances. This can be explained by the fact that whereas all respondents report one or more interactions with friends, many respondents report zero interactions for the other social roles (e.g., no interactions with a partner, boss, or stranger). Thus, we explored also models accommodating overdispersion (negative binomial) and an excess of zeros (zero-inflated Poisson and negative binomial (see, e.g., Hilbe, 2011). As a negative binomial model fits the data the best, we report the results of this analysis.²

We employed a stepwise modeling approach, starting with an empty model. In the second modeling step, to model the reporting rates of each social role, we added one dummy variable per social role as Level 1 predictors. To directly compare the rates of each social role between the two conditions, we used no intercept (Othman, 2014). Therefore, the main effects of the dummy variables for the social roles indicate the expected reporting rates per social role. In the following step, we added the main effect of the condition on Level 2. Lastly, we included interactions between the social role dummies and condition which indicate the differences in reporting rates per role between the two sampling designs.

RQ2b: Differences in the quality of relationships with captured interaction partners

We further wanted to examine differences between event- and signal-contingent sampling regarding the quality of the captured relationships. Thus, we estimated three-level random intercept models with the nesting structure displayed in Figure 1 (Level 1: social interactions, Level 2: unique interaction partners, Level 3: participants). By considering the unique interaction partners as a separate level, we can control for the nesting of social interactions within specific interaction partners.

In a stepwise approach, we build a three-level linear regression model for each dependent variable (i.e., “Feeling Respected,” “Closeness & Emotional Security,” and “Trustworthiness & Advice Seeking”) to see if there were differences in relationship quality between the two sampling designs. The main predictor of interest to answer RQ2b was the effect of condition (Level 3). We further considered relevant variables (Level 2: social role of the interaction partner, Level 3: sex and age of the participant) and interactions between these variables and conditions.³

For one specific part of the sample (i.e., one semester) all social roles of interaction partners are missing. We created a social role category “missing,” which can be considered an average across social roles, assuming that the students from the semester with missing social roles are comparable to students from the other semesters.⁴ With this strategy, it was possible to use all available data and investigate whether there is a bias in our outcome variables for interaction partners with missing social role information.

RQ3: Differences in the quality of captured social interactions

Lastly, we were interested to see whether and how the quality of reported social interactions differs between the event- and signal-contingent sampling design. For this we also estimated three-level linear regression models (Level 1: social interactions, Level 2: unique interaction partners, Level 3: participants) for each interaction quality rating (i.e., self and other’s warmth, self and other’s dominance, and affect) as a separate dependent variable. Starting from empty models, we added predictors in a stepwise manner. The main predictor of interest was condition (Level 3). We further considered relevant variables on all three levels (Level 1: timing of the social interaction represented by dummy variables covering 3-h time blocks, Level 2: social role of the interaction partner, Level 3: sex and age of the participant) and interactions between these variables and condition.

Exploratory analyses: How important is modeling “with whom”?

Novel in our analyses is that we examine ESM social interaction data while taking into account with whom an interaction took place (i.e., the unique interaction partner or the social role of the interaction partner). Thus, we were specifically interested in finding out what the added value of explicitly modeling unique interaction partners as nesting level or including social roles as predictors is. For this purpose, we examined the intraclass correlations (ICCs) on Level 2 and Level 3 in the models for RQ2b and RQ3. The ICCS indicates how much dependence there is between interactions of the same participant with the same interaction partner (Level 2) relative to the dependence between interactions by the same participant (Level 3). Moreover, we calculated the proportion of explained variance by the social role dummies and the dummy variables modeling the time of the day (see Snijders & Bosker, 2011). This will give us a direct comparison between the importance of including time and including social roles when modeling social interaction outcomes (i.e., relationship and interaction quality).

Results

Due to space restrictions, we only report descriptives and analyses directly relevant to our main research questions. Additional descriptives, visualization, and explorations of the data, as well as all modeling steps, can be found on our OSF repository. We further refrain from in-depth interpretation of the effects of control variables.

Descriptive statistics

Participants reported on average 23.24 (SD = 12.95) social interactions over the seven days of the study, with an average of 12.49 (SD = 6.39) different interaction partners. Figure 3 displays the timing of interactions over the course of the day split by condition and split by social role. As anticipated given the design, participants in the signal-contingent condition mainly reported interactions between 10 am and 10 pm Participants in the event-contingent condition also reported interactions during the late evening and night, despite being instructed to also constrain reporting to 10 am to 10 pm These nighttime interactions—only captured by the event-contingent design—take place mainly with friends, partners, and roommates.

Figure 3. Timing of interactions split by condition and social role of the interaction partner.

Do the conditions differ in the number of unique interaction partners? (RQ1)

Based on visual exploration, more social interactions, and more unique interaction partners were captured in the event-contingent condition (see Figure 4). A t-test on the raw number of unique interaction partners shows that indeed significantly more interaction partners were captured in the event-contingent condition [Signal Contingent M = 10.32, Event-Contingent M = 14.53; t(df) = −5.73 (266.62), p < 0.001]. However, when adjusting the number of interaction partners for the number of reported interactions a participant had, the two conditions are not significantly different [Signal Contingent M = 0.60, Event-Contingent M = 0.58; t(df) = 1.11 (257.86), p = 0.27]. Thus, the reporting rate of unique interaction partners does not seem not to differ between conditions.

Figure 4. Number of interactions and number of unique interaction partners split by condition.

Do the conditions differ in the reporting rates of social roles of interaction partners? (RQ2a)

Table 1 displays the counts of unique interaction partners of a certain social role split by condition. Students reported most interactions with friends/acquaintances, followed by family members. A large portion of social roles is missing due to the earlier described data loss. Given that participants in the event-contingent condition reported more social interactions, the counts for almost all social roles are higher in the event-contingent condition. However, the differences are not equally large for each role (e.g., the difference in counts of partners is very small while the difference in count of friends is larger). This indicates that there may be a difference in which kinds of interaction partners are captured with the two sampling designs. The table further shows the mean rate of interactions with a certain social role. These mean rates were computed across participants from the subsample with complete social role data split by condition. Thereby, the rate is the count of unique interactions partners of a given role divided by the total number of interactions a participant had. It thus presents a measure that corrects for the larger number of opportunities to encounter interaction partners (i.e., the larger number of captured social interactions) in the event-contingent condition and allows for a fairer comparison of the two conditions. Based on these “corrected counts” the differences between the two designs become smaller for some social roles (e.g., bosses/teachers and friends), meaning that the two designs capture these roles at a similar rate.

Table 1. Descriptive statistics of count and rate of unique interaction partners of a certain social role split by condition.

	Signal			Event
	Total count	M (SD) count across participants	M (SD) rate across participants	Total count	M (SD) count across participants	M (SD) rate across participants
Boss/Teacher	68	0.52 (1.00)	0.0306 (0.0741)	119	0.86 (1.15)	0.0307 (0.0395)
Coworker	37	0.28 (0.73)	0.0173 (0.0471)	34	0.24 (0.73)	0.0119 (0.0490)
Family	105	0.81 (1.31)	0.0417 (0.0643)	189	1.36 (1.42)	0.0536 (0.0618)
Friend/Acquaintance	590	4.54 (4.06)	0.2680 (0.1935)	1018	7.32 (5.73)	0.2779 (0.1923)
Partner	56	0.43 (0.54)	0.0300 (0.0583)	63	0.45 (0.55)	0.0194 (0.0312)
Roommate	80	0.62 (0.90)	0.0376 (0.0638)	106	0.76 (1.11)	0.0300 (0.0512)
Stranger	44	0.34 (0.82)	0.0184 (0.0447)	78	0.56 (1.09)	0.0193 (0.0382)
Other	23	0.18 (0.52)	0.0098 (0.0300)	51	0.37 (0.81)	0.0122 (0.0263)
Missing	338	–	–	361	–	–

Note. k = 3360 interaction partners nested in N = 269 participants used to determine descriptive statistics for the count; k = 2661 interaction partners, N = 212 participants used to determine descriptive statistics for the rate.

The results of the two-level negative binomial models of the reporting rates are shown in Table 2. This model tests the differences in social role reporting rates between the two sampling designs. Given that the model contains no intercept, the main effects of the social roles indicate the expected rate for the respective social role on the log scale. The estimated average rate of interaction partners per reported social interaction of the category friend/acquaintance for participants in the signal-contingent condition for instance is $e^{-1.11}$ = 0.33. The rate is highest for friends and lowest for partners, meaning that friends are the most prevalent social role reported and partners the least. To answer our research question, we need to consider the interaction effects of social roles and conditions, as these indicate whether the two conditions differ in the reporting rate of a certain interaction partner role. The two largest interaction effects we find are between condition and coworker ($e^{-0.62}$ = 0.54) as well as condition and roommate ($e^{-0.41}$ = 0.66). Both indicate that the event-contingent condition has a significantly lower reporting rate of coworkers and roommates than the signal-contingent condition. For the other social roles, however, there were only small and non-significant differences in the reporting rates between the event and signal condition.

Table 2. Negative binomial random-intercept models with social roles nested in participants.

	Rate of interaction partners on the log scale
Fixed effects (SE)
Boss/Teacher	−3.30 (0.13)***
Coworker	3.90 (0.17)***
Family	−2.87 (0.10)***
Friend/Acquaintance	−1.11 (0.05)***
Partner	−4.38 (0.21)***
Roommate	−3.48 (0.14)***
Stranger	−3.13 (0.12)***
Other	−3.74 (0.16)***
Boss/Teacher * Condition	0.04 (0.16)
Coworker * Condition	−0.62 (0.24)*
Family * Condition	0.07 (0.13)
Friend/Acquaintance * Condition	0.00 (0.07)
Partner * Condition	0.26 (0.26)
Roommate * Condition	−0.41 (0.19)*
Stranger * Condition	−0.24 (0.16)
Other * Condition	0.05 (0.19)
Dispersion parameter	10.8
Random effects
Level 2 Intercept SD	0.10
Model fit measures
Deviance	4249.4
Chi^2 (df) vs. empty model	1991.7 (16)***

Note. *z < 0.05, ***z < 0.001; k = 2661 interaction partners, N = 212 participants. Condition is coded 0 = signal-contingent, social role is dummy coded with no reference category (i.e. one dummy variable per role).

Do the conditions differ in the quality of relationships with captured interaction partners? (RQ2b)

Table 3 shows descriptive statistics of the relationship quality ratings split by condition and social role of the interaction partner. Social roles differ in their relationship quality ratings as one would expect (e.g., participants rated family, partner, and friends as the highest quality relationships). Particularly the Closeness & Emotional Security ratings are different between the social roles. Differences between the two sampling designs are less clear: Some social roles (bosses/teachers, coworkers, family members, partners) are rated higher in the event-contingent condition than the signal-contingent condition, while for other role categories (friends/acquaintances, roommates, strangers, others, missing) the opposite is the case.

Table 3. Means (standard deviations) of outcome variables split by condition and social role.

Social role	Condition	N	Feeling respected	Closeness and emotional security	Trustworthiness and advice seeking	Dominance/ submission-self	Dominance/ submission-other	Warm/cold-self	Warm/cold-other	Positive affect	Negative affect
Boss/Teacher	Signal	70	64.79 (21.44)	38.39 (25.45)	52.41 (25.60)	1.16 (9.59)	4.05 (8.92)	21.41 (10.80)	21.91 (11.40)	48.74 (12.66)	17.52 (13.92)
	Event	129	74.13 (18.00)	41.32 (27.49)	54.52 (26.75)	−2.12 (10.32)	1.01 (10.30)	26.95 (11.16)	25.02 (11.53)	51.56 (10.64)	16.23 (11.96)
Coworker	Signal	38	65.42 (21.04)	37.08 (25.96)	49.00 (24.18)	1.4 (8.42)	3.71 (8.77)	24.14 (11.28)	23.47 (10.58)	46.61 (14.61)	14.9 (14.89)
	Event	39	71.15 (19.93)	43.85 (28.04)	44.77 (26.32)	−1.3 (12.34)	−1.22 (9.98)	25.96 (13.72)	24.73 (14.05)	47.78 (10.68)	12.68 (11.77)
Family	Signal	224	75.97 (21.40)	75.10 (22.86)	74.72 (23.68)	2.67 (11.07)	4.15 (10.41)	21.71 (13.71)	20.41 (13.50)	50.21 (14.27)	15.02 (13.31)
	Event	390	85.14 (16.40)	86.22 (17.47)	86.46 (18.69)	−2.59 (12.55)	0.29 (11.03)	28.75 (11.40)	27.42 (11.36)	53.73 (10.95)	15.5 (10.52)
Friend/ Acquaintance	Signal	972	76.44 (16.81)	67.51 (23.91)	69.11 (23.78)	1.63 (10.43)	0.97 (9.54)	24.41 (10.68)	23.45 (11.04)	52.40 (12.74)	13.79 (11.32)
	Event	1749	75.73 (18.38)	66.56 (24.08)	67.08 (25.36)	−1.37 (11.81)	0.60 (11.20)	25.1 (12.32)	23.7 (12.20)	51.66 (11.95)	16.47 (10.98)
Partner	Signal	232	87.24 (17.48)	87.66 (15.80)	88.00 (17.15)	0.48 (9.35)	0.98 (9.59)	28.05 (10.17)	27.78 (10.92)	56.70 (12.98)	14.2 (9.71)
	Event	348	89.90 (16.11)	91.02 (15.56)	91.26 (15.14)	−3.74 (10.97)	−2.31 (11.20)	26.51 (13.19)	25.42 (13.14)	48.04 (14.99)	14.5 (11.90)
Roommate	Signal	222	72.61 (17.19)	68.52 (18.78)	68.23 (23.54)	0.6 (9.64)	0.83 (8.75)	22.49 (10.59)	21.87 (10.09)	49.36 (9.91)	13.83 (11.32)
	Event	307	70.37 (23.74)	66.75 (26.21)	67.97 (26.10)	−3.01 (12.96)	−1.38 (12.25)	23.74 (12.30)	22.56 (12.48)	52.08 (9.42)	16.99 (9.94)
Stranger	Signal	45	58.49 (20.47)	34.87 (26.39)	33.67(22.59)	3.41 (12.34)	1.75 (11.34)	19.34 (11.91)	17.92 (12.18)	52.43 (10.19)	21.64 (17.57)
	Event	79	53.28 (25.63)	19.77 (24.27)	25.44(25.71)	−4.59 (10.63)	−2.11 (11.07)	25.26 (11.88)	23.37 (12.65)	48.68 (10.86)	16.69 (11.34)
Other	Signal	24	68.46 (18.36)	43.33 (30.38)	44.88(31.96)	3.28 (9.41)	2.18 (8.62)	22.32 (14.42)	21.81 (13.96)	47.54 (10.77)	18.66 (16.84)
	Event	64	64.38 (25.93)	42.22 (32.39)	53.39 (35.00)	−2.58 (10.26)	−1.58 (10.37)	25.94 (12.22)	24.73 (12.79)	48.30 (12.57)	15.63 (11.47)
Missing	Signal	615	76.63 (18.21)	71.20 (23.85)	70.83 (24.28)	0.97 (7.45)	−2.65 (7.65)	24.03 (10.35)	22.76 (9.50)	52.64 (11.66)	17.41 (11.21)
	Event	705	74.52 (21.01)	67.11 (27.34)	67.97 (26.79)	1.33 (8.30)	0 (6.61)	22.48 (13.70)	21.94 (13.34)	49.02 (11.07)	16.47 (12.56)

Note. N refers to the number of interactions with a given social role for a given condition.

The results of the final three-level models with the three relationship quality ratings as outcomes are presented in Table 4.⁵ The main effects of the condition show that there are no significant differences between the event- and signal-contingent conditions in relationship quality for the reference category (i.e., friends). The significant main effects of social roles indicate differences between the role in question and the reference category (i.e., friends) in the signal-contingent condition: Bosses/teachers, coworkers, strangers, and others were rated significantly lower than friends/acquaintances, confirming the general definitions of these social roles as being less close than friends. Family members and partners were rated significantly higher than friends.

Table 4. Final three-level random-intercept models for relationship quality ratings.

	Feeling respected	Closeness and emotional security	Trustworthiness and advice seeking
Fixed effects (SE)
Intercept	77.95 (1.11)***	66.21 (1.26)***	67.83 (1.28)***
Condition (ref. signal)	4.19 (2.22)	−1.58 (1.81)	1.07 (2.55)
Social role (ref. friend)
Boss/Teacher	−5.30 (1.37)***	−23.46 (1.89)***	−11.79 (1.93)***
Coworker	−6.69 (2.10)**	−22.10 (2.89)***	−17.84 (2.95)***
Family	7.06 (1.06)***	18.16 (1.50)***	16.02 (1.54)***
Partner	10.20 (1.39)***	22.28 (2.03)***	20.41 (2.09)***
Roommate	−2.90 (1.21)*	2.51 (1.74)	2.46 (1.79)
Stranger	−17.30 (1.68)***	−36.46 (2.31)***	−36.03 (2.35)***
Other	−9.50 (2.18)***	−21.56 (3.01)***	−17.49 (3.07)***
Missing	−0.53 (1.62)	2.16 (1.83)	1.64 (1.84)
Sex (ref. male)	−5.42 (1.37)***	−4.73 (1.52)**	−5.30 (1.54)***
Boss/Teacher * Condition	7.75 (2.73)**	1.34 (3.77)	2.24 (3.85)
Coworker * Condition	6.54 (4.20)	9.58 (5.78)	1.09 (5.90)
Family * Condition	4.95 (2.13)*	10.70 (3.01)***	11.91 (3.08)***
Partner * Condition	1.42 (2.77)	3.68 (4.07)	3.24 (4.19)
Roommate * Condition	−1.62 (2.42)	−0.50 (3.48)	2.12 (3.58)
Stranger * Condition	−9.36 (3.35)**	−18.88 (4.62)***	−8.82 (4.71)
Other * Condition	−8.26 (4.35)	−4.47 (6.02)	5.11 (6.14)
Missing * Condition	−1.19 (3.24)	−3.22 (3.60)	−0.07 (3.69)
Sex * Condition	−8.15 (2.74)**	–	−6.55 (3.08)*
Random effects (SE)
Level 3—participants	89.27 (9.45)	94.28 (9.71)	93.36 (9.66)
Level 2—interaction partners	149.59 (12.23)	371.04 (19.26)	400.27 (20.01)
Level 1—social interactions	119.63 (10.94)	142.95 (11.96)	135.44 (11.64)
Deviance	51664.81	54386.40	54372.2
L. ratio vs. empty model	374.96***	918.40***	667.58***

Note. *p < 0.05, **p < 0.01, ***p < 0.001.

Condition is coded 0 = signal-contingent, 1 = event-contingent; Sex is coded 0 = male, 1 = female; Social role is dummy coded with the category “friends” as reference category.

When looking more closely at the interactions between condition and social roles, some differences between the sampling designs can be observed: The significant interaction between the category boss/teacher and condition for “Feeling Respected” indicates, for example, that the “Feeling Respected” ratings of bosses/teachers are lower compared to friends in the event-contingent condition, while in the signal-contingent condition bosses/teachers are rated higher than friends. The same pattern holds for coworkers. The positive interaction parameter estimates between family and condition—that are significant for all relationship quality ratings—imply that the higher ratings of family members (compared to friends) in the signal-contingent condition are amplified in the event-contingent condition. The interaction between stranger and condition highlights the significantly lower “Closeness & Emotional Security” ratings of strangers vs. friends in the event-contingent condition, compared to the already lower ratings in the signal contingent condition. A similar, but less extreme pattern holds for the interaction between condition and stranger with regards to “Feeling Respected.” Thus, the two sampling schemes differ in the quality of captured relationships for interaction partners of specific social roles (i.e., bosses, family members, and strangers).

Do the conditions differ in the quality of momentary social interactions? (RQ3)

Table 3 shows descriptive statistics of the interaction quality ratings split by condition and social role of the interaction partner. No clear patterns differentiating the two conditions can be observed.

The three-level random-intercept models with social interactions on Level 1, interaction partners on Level 2, and participants on Level 3 can be found in Table 5 (for dominance/submission and warmth/cold) and Table 6 (for affect).⁶ To answer our third research question, we examine the main and interaction effects of the condition. There is no significant main effect of the condition for any of the outcomes. The results show several large, significant main effects of the social role of the interaction partner, all in a direction as one would expect (e.g., social interactions with romantic partners were rated significantly warmer from both sides than interactions with friends).

Table 5. Three-level random-intercept models for interaction quality.

	Dominance/submission-self	Dominance/submission-other	Warm/cold-self	Warm/cold-other
Fixed effects (SE)
Intercept	0.03 (0.77)	−0.54 (0.73)	29.81 (1.14)***	28.54 (1.13)***
Condition (ref. signal)	−2.05 (1.55)	−1.17 (1.45)	1.40 (1.61)	0.35 (1.61)
Social role (ref. Friend)
Boss/Teacher	−14.94 (1.35)***	14.07 (1.45)***	−5.90 (1.32)***	−4.31 (1.43)**
Coworker	−3.63 (2.06)	8.12 (2.21)***	−5.62 (2.02)**	−6.26 (2.20)**
Family	−0.78 (0.94)	3.01 (1.04)**	2.38 (0.96)*	4.35 (1.06)***
Partner	0.30 (1.05)	−0.29 (1.21)	5.84 (1.15)***	6.43 (1.28)***
Roommate	0.22 (1.00)	−1.47 (1.12)	−3.54 (1.05)***	−5.00 (1.16)***
Stranger	−4.32 (1.66)**	4.79 (1.78)**	−8.36 (1.62)***	−7.40 (1.76)***
Other	0.36 (2.13)	3.43 (2.30)	−5.44 (2.09)**	−8.15 (2.28)***
Missing	1.40 (1.59)	−0.06 (1.47)	−1.51 (1.68)	−0.60 (1.67)
Sex (ref. male)	–	–	−8.94 (1.42)***	−9.34 (1.41)***
Age (centered)	–	−0.80 (0.37)*	–	–
Boss/Teacher * Condition	−1.26 (2.71)	1.46 (2.89)	6.46 (2.63)*	3.76 (2.87)
Coworker * Condition	−5.50 (4.11)	3.19 (4.43)	3.42 (4.04)	6.49 (4.39)
Family * Condition	2.85 (1.88)	−2.26 (2.08)	2.56 (1.93)	5.30 (2.12)*
Partner * Condition	−3.69 (2.10)	−4.11 (2.42)	0.50 (2.31)	1.42 (2.57)
Roommate * Condition	−2.47 (2.00)	−0.64 (2.24)	−3.60 (2.11)	−0.99 (2.33)
Stranger * Condition	−12.59 (3.32)***	0.99 (3.57)	−2.12 (3.24)	−8.97 (3.52)*
Other * Condition	−4.59 (4.26)	6.00 (4.59)	−8.52 (4.18)*	−8.15 (4.55)
Missing * Condition	1.43 (3.18)	2.15 (2.95)	−3.54 (3.30)	−1.19 (3.29)
Age (centered) * Condition	–	−1.77 (0.74)*	–	–
Random effects variance (SE)
Level 3—participants	94.27 (9.71)	72.03 (8.49)	103.07 (10.15)	96.36 (9.82)
Level 2—interaction partners	34.92 (5.91)	64.33 (8.02)	68.18 (8.26)	91.19 (9.55)
Level 1—social interactions	238.12 (15.43)	250.22 (15.82)	187.71 (13.70)	212.12 (14.56)
Deviance	53140.85	53823.53	52523.47	53430.0
L. ratio vs. empty model	182.20***	149.89***	182.96***	211.40***

Note. *p < 0.05, **p < 0.01, ***p < 0.001.

Condition is coded 0 = signal-contingent, 1 = event-contingent; Sex is coded 0 = male, 1 = female; Social role is dummy coded with the category “friends/acquaintances” as reference category.

Table 6. Final three-level random-intercept models for affect.

	Positive affect	Negative affect
Fixed effects (SE)
Intercept	52.20 (0.85)***	12.90 (1.22)***
Condition (ref. signal)	1.14 (1.70)	0.49 (1.70)
Social role (ref. friend)
Boss/Teacher	−8.47 (1.43)***	2.22 (1.08)*
Coworker	−9.58 (2.19)***	4.54 (1.65)**
Family	1.46 (1.03)	0.76 (0.75)
Partner	8.80 (1.19)***	0.40 (0.83)
Roommate	−6.28 (1.11)***	0.73 (0.79)
Stranger	−8.52 (1.76)***	6.35 (1.32)***
Other	−8.59 (2.27)***	5.69 (1.70)***
Missing	−2.34 (1.75)	3.11 (1.78)
Sex (ref. male)	–	3.60 (1.54)*
Age (centered)	−1.34 (0.40)**	–
Boss/Teacher * Condition	−0.08 (2.87)	2.27 (2.15)
Coworker * Condition	6.47 (4.39)	1.81 (3.29)
Family * Condition	1.70 (2.05)	−1.44 (1.50)
Partner * Condition	−3.50 (2.39)	−1.25 (1.67)
Roommate * Condition	0.90 (2.22)	−0.81 (1.59)
Stranger * Condition	−6.88 (3.53)	1.43 (2.65)
Other * Condition	−9.78 (4.54)*	1.25 (3.40)
Missing * Condition	−7.46 (3.50)*	0.75 (3.50)
Random effects variance (SE)
Level 3—intercept	114.43 (10.70)	131.92 (11.49)
Level 2—intercept	60.62 (7.79)	20.46 (4.52)
Level 1—residual	245.28 (15.66)	152.43 (12.35)
Deviance	53802.54	50554.89
L. ratio vs. empty model	237.41***	62.06***

Note. *p < 0.05, **p < 0.01, ***p < 0.001.

Condition is coded 0 = signal-contingent, 1 = event-contingent; Sex is coded 0 = male, 1 = female; Social role is dummy coded with the category “friends/acquaintances” as reference category.

Whereas the main effects for self- and other-ratings are in the same direction (i.e., either positive or negative) for the warm/cold interaction quality dimension, for the dominance/submission dimension they are in the opposite direction for almost all social roles (compared to friends). For both dimensions, the accompanying interaction effects are mostly in the same direction, pointing to larger differences between the social roles (compared to friends) in the event-contingent condition (compared to the signal-contingent condition). The largest—negative—difference between the signal-contingent and the event-contingent condition is found for the dominance/submission self-rating with strangers. There are a few other significant interactions between condition and social role. For example, in the event-contingent condition, the difference between the warmth ratings of family members and friends is approximately twice as large as in the signal-contingent condition. Thus, the two sampling schemes differ in the quality of captured momentary social interactions for interaction partners of specific social roles, particularly strangers.

Exploratory analyses: How important is modeling “with whom”?

Intraclass correlation coefficients relationship quality (RQ2b) and interaction quality (RQ3)

Table 7 shows the intraclass correlation coefficients (ICCs) for the empty models with a three-level structure (Level 1: social interactions, Level 2: unique interaction partners, Level 3: participants) and each relationship and social interaction quality rating as an outcome (Snijders & Bosker, 2011). The ICCs provide evidence for the dependence of relationship quality ratings with the same interaction partner, reported by the same participant which underscores the importance of a three-level modeling approach, including the level of unique interaction partners (Level 2 ICC between 0.7 and 0.82). Moreover, the ICCs indicate the relative importance of the unique interaction partner level when compared with the participant level (Level 3 ICC between 0.16 and 0.37). Note that “Feeling Respected” shows a less extreme pattern in the ICC compared to the other two relationship quality ratings, with a much higher correlation on the participant level than the others. This indicates that “Feeling Respected” differs more between respondents than the other two relationship quality ratings.

Table 7. Intra class correlation coefficients for the empty 3-level models of relationship quality and interaction quality.

		Feeling respected	Closeness and emotional security	Trustworthiness and advice seeking	Dominance/ submission- self	Dominance/ submission- other	Warm/ cold-self	Warm/ cold-other	Positive affect	Negative affect
Level 2 and 3	$\frac{{\phi }^2+{\tau }^2}{{\phi }^2+{\tau }^2+{\sigma }^2}$	0.70	0.82	0.82	0.37	0.38	0.52	0.52	0.45	0.51
Level 3 (Relative)	$\frac{{\phi }^2}{{\phi }^2+{\tau }^2}$	0.37	0.16	0.17	0.69	0.51	0.62	0.53	0.62	0.87
Level 3 (Absolute)	$\frac{{\phi }^2}{{\phi }^2+{\tau }^2+{\sigma }^2}$	0.26	0.13	0.14	0.25	0.19	0.32	0.28	0.28	0.44

Note. Social interactions (Level 1) nested in interaction partners (Level 2) nested in participants (Level 3); φ² Level 3 variance, τ² Level 2 variance, σ² Level 1 variance; for the relevant formulas and interpretation guidance see also p. 68 in Snijders and Bosker (2011).

The ICCs of the empty models for the five interaction quality measures indicate that for the interaction quality ratings in contrast to the relationship quality ratings there is more dependence attributed to the participant level (Level 3 ICC between 0.51 and 0.87). This is especially the case for negative affect ratings. There is, however, still considerable dependence on the quality of interactions within a unique interaction partner (Level 2 ICC between 0.37 and 0.52).

Variance explained by time and social role

Table 8 shows the proportion of explained variance in the relationship and interaction quality variables (outcomes RQ2b and RQ3) that can be attributed to either variables modeling time or modeling social role. Social role explains a considerable portion of the variance in the three relationship quality ratings (between 7.6 and 20.5%), which suggests that social role is an important predictor when studying relationship quality. With regards to the interaction quality ratings, less variance is explained by social role (between 2.1 and 4.0%). In general, it appears that social role variables were more important predictors than the time variables (explaining between 0.0 and 0.2% of the outcome variance) when modeling the social interaction data.

Table 8. Proportion of explained variance by time blocks and social roles.

	Feeling respected	Closeness and emotional security	Trustworthiness and advice seeking	Dominance/ submission-self	Dominance/ submission-other	Warm/ cold-self	Warm/ cold-other	Positive affect	Negative affect
R^2 time blocks	–	–	–	0.1%	0.0%	0.1%	0.1%	0.2%	0.1%
R^2 social roles	7.6%	20.5%	15%	2.2%	2.5%	3.1%	3.7%	4.0%	2.1%

Note. R² was calculated based for the three-level models with social interactions (Level 1) nested in interaction partners (Level 2) nested in participants (Level 3).

Discussion

We used previously collected data on daily social interactions in a student sample (Himmelstein et al., 2019) to investigate differences between signal- and event-contingent sampling designs of experience sampling data in (1) the number of interaction partners that are captured, (2) the kinds of captured interaction partners (i.e., social role and relationship quality), and (3) the quality of interactions (i.e., warmth, dominance, negative affect, positive affect) taking into account that they take place with a specific interaction partner. Furthermore, we quantified the importance of modeling with whom a social interaction took place by examining the intraclass correlations (ICCs) attributed to the interaction partner level as well as the variance explained by the social role of an interaction partner.

As expected, event-contingent sampling captured more social interactions and thereby also more unique interaction partners (i.e., a larger absolute data quantity). The numbers of captured social interactions and unique interaction partners are, however, in proportion to each other, so that the reporting rate (i.e., the relative data quantity) of unique interaction partners is similar between the two conditions (RQ1). When examining what kinds of interaction partners are captured with the two sampling designs, we found that also here the relative data quantity is similar. For most social roles the reporting rate, so the number of captured interactions with a specific role relative to the total number of captured interactions, is similar. However, in event-contingent condition, the rate at which we captured coworkers and roommates, is lower than in the signal-contingent condition (RQ2a). Regarding relationship quality (RQ2b) and the quality of social interactions (RQ3), a pattern emerged when considering the interaction effects of condition and social role: The large and significant interactions indicate differences between the conditions in how specific social roles, such as family members or strangers, are rated in terms of relationship and interaction quality. While in many regards the two sampling schemes appear to be comparable, there seem to be differences in how they capture emotionally close or self-selected interaction partners (friends) in contrast to more distant or routine interaction partners (coworkers, roommates, strangers). These effects are impacting broader relationship quality ratings more so than quality ratings of momentary social interactions.

Leertouwer et al. (2021) suggest that the assessments with the two sampling designs differ in the extent to which they represent the remembering self (i.e., a biased summary of past experiences) vs. the experiencing self (i.e., the current moment) of participants. A signal-contingent design that asks to report on a limited number of interactions since the last beep is by definition closer to the remembering self and may lead participants toward providing a biased default quality rating especially for more routinely encountered (i.e., less memorable) social roles, such as coworkers or strangers. In the present dataset, the by far most prevalent social role was friends/acquaintances. All other social roles could be considered “rare” in this case. An event-contingent sampling design requires participants to self-monitor their behavior more closely as they need to initiate assessments themselves. Thereby it leads to capturing more social interactions in absolute numbers (i.e., counts), including more instances of social interactions with distant social roles. Thus, despite fewer reports of rare social roles in relative terms (i.e., lower rates or proportions), event-contingent sampling may provide a more fine-grained assessment of interactions with these social roles. The observed subtle differences between the two sampling schemes for specific social roles may be rooted in the fact that fewer instances are captured when sampling in a signal-contingent manner and that participants’ recollections of these interactions may be somewhat biased.

Most importantly, our analyses highlight that it is insightful to use three-level models including the interaction partner level to account for the dependence between interactions within unique interaction partners. Our results show that when modeling social interactions, the social interactions of a participant that take place with the same interaction partner are more similar to each other than social interactions with different interaction partners (i.e., there is considerable dependence between interactions with the same partner).

Moreover, our analyses showed that including the social role of interaction partners when analyzing social interactions is important. The social role of the interaction partner was a robust predictor in all our analyses explaining between 7.6 − 20.5% of the total variance. These findings are in line with previous research highlighting the association between interaction partner characteristics, such as social role and participants’ interpersonal behaviors (Moskowitz et al., 1994; Suh et al., 2004).

Interestingly, modeling the timing of social interaction observations (with dummies that represent 3-h time blocks of the day) did not improve our models significantly in terms of explained variance. This suggests that the effect of social roles might dominate the differences observed across various timings. Thus, when wanting to examine the quality of social interactions it matters more whether one interacted with a family member or a coworker as opposed to whether it was morning or afternoon when this interaction took place.

Besides highlighting the importance of information about an interaction partner, our results add nuance to the conclusion made by Himmelstein et al. (2019) that event- and signal-contingent experience sampling may be used interchangeably to capture the quality of social interactions. In terms of relative numbers (i.e., the reporting rates of social interactions with different social roles) the two sampling schemes deliver mostly similar data. The data, however, differs not only in the absolute quantity of captured interactions (and unique interaction partners) but in the granularity of relationship and interaction quality ratings of less frequently encountered social roles. The event-contingent design captures more interactions and thereby also more interactions with distant social roles providing a more fine-grained picture of these interaction instances. Since signal-contingent sampling captures fewer social interactions, differences between the frequency and ratings of the interactions with more distant interaction partners may not be captured to the same extent as with event-contingent sampling.

Commonly used signal-contingent sampling designs capture not only fewer interactions but also specifically do not capture social interactions late in the evening or at night, which may shift the representation of social interaction with interaction partners of a given social role. This may not have severe effects on data quality in a student sample where a less clear day structure is present (i.e., often no regular 9 to 5 working hours) and social life is generally structured differently than in later life phases (Litwin & Stoeckel, 2013). There may, however, be different patterns in other populations, such as a general population with more regular working hours.⁷

Thus, when considering the choice for either signal- or event-contingent experience sampling, the research question and the population of interest is important. If a researcher aims to capture all social interactions and the full social life of a participant or is specifically interested in interactions with less frequently encountered social roles, then event-contingent sampling may be most appropriate. Signal-contingent sampling schemes tailored to the day structure of each participant, or a combination of the two sampling designs, may be feasible alternatives as well but need further investigation. When a researcher is only interested in overall interaction quality or in interactions with frequently encountered social roles, such as friends, then a common signal-contingent design (e.g., six prompts between 10 am and 10 pm) is sufficient.

Limitations and future directions

The current analyses come with several limitations: As mentioned above, our data was collected in a student sample, which has a specific day structure and organization of social life that influences the results. While the effects we found seem to not be strong in student samples, they may be more impactful when being interested in the general population which, as mentioned, needs further investigation.

Furthermore, collecting the initials of the interaction partner did not always allow for straightforward matching of social interactions with the same interaction partner. The variable needed some manual correction and some observations needed to be excluded due to ambiguity. Additionally, there are unique interaction partners with the same initials that could only be distinguished based on social role in this sample. Still, it is likely that there are also interaction partners with the same social role and the same initials (e.g., two friends with the initials MS).

Despite some ambiguities in defining unique interaction partners, our analyses highlight the importance of considering the interaction partner level when analyzing social interaction data. To include the interaction partner level or at least the social role of the interaction partners in future analyses, information on this level should be collected. Assessing the initials of interaction partners has shown some important disadvantages: Two interaction partners may have the same initial and participants often switch answering styles. Future studies should consider assessing full names, nicknames, or the first name and the initial of the last name, as this would allow for clearer matching. However, asking participants to type a name or initials and assessing conceptually rather stable relationship quality ratings at every social interaction creates an unnecessary burden for the participant. An even better option may be to assess a personal network (Perry et al., 2018; Stadel et al., 2022) before an experience sampling study and then let participants pick their interaction partner from the list of network members or add new ones. This topic, however, warrants further research regarding its practical implementation and feasibility, which is currently in progress (see Stadel & Stulp, 2022).

Lastly, to date, there is only one dataset specifically contrasting social interactions captured with event-contingent vs. signal-contingent sampling (Himmelstein et al., 2019). Replication of the present results in different samples and with different signal-contingent schemes (e.g., a larger sampling window or more prompts per day) is warranted. Similarly, within-subject studies exploring both sampling designs are needed.

Conclusion

Our analyses highlight that distinguishing between unique interaction partners, or at least between interaction partners of different social roles, is important when collecting and analyzing social interaction data. We showed and discussed different ways in which this can be done. Further, we observed subtle differences between event- and signal-contingent sampling of social interactions in daily life: (1) Event-contingent sampling captures more social interactions and interaction partners in absolute numbers. However, in terms of relative numbers, the two sampling schemes deliver very similar data for most social roles. (2) Quality ratings of relationships and momentary social interactions with infrequently encountered social roles are impacted by the sampling design. This may be due to the fact that event-contingent sampling simply provides a larger absolute data quantity and thereby a more fine-grained picture of rare events.

Article information

Conflict of interest disclosures: Each author signed a form for disclosure of potential conflicts of interest. No authors reported any financial or other conflicts of interest in relation to the work described.

Ethical principles: The authors affirm having followed professional ethical guidelines in preparing this work. These guidelines include obtaining informed consent from human participants, maintaining ethical treatment and respect for the rights of human or animal participants, and ensuring the privacy of participants and their data, such as ensuring that individual participants cannot be identified in reported results or from publicly available original or archival data.

Funding: This work was supported by Grant PZ00P1_208742 from the Swiss National Science Foundation (SNSF).

Role of the funders/sponsors: None of the funders or sponsors of this research had any role in the design and conduct of the study; collection, management, analysis, and interpretation of data; preparation, review, or approval of the manuscript; or decision to submit the manuscript for publication.

Acknowledgments: The authors would like to thank Gert Stulp for his comments on prior versions of this manuscript. The ideas and opinions expressed herein are those of the authors alone, and endorsement by the authors’ institutions or the funding agency is not intended and should not be inferred.

Open Scholarship

This article has earned the Center for Open Science badges for Preregistered. The materials are openly accessible at https://osf.io/yr563.

Data availability statement

Data can be requested from Aidan G. C. Wright.

Notes

1 Data access can be requested by contacting A. G. C. Wright.

2 All four modelling approaches (Poisson, zero-inflated Poisson, negative binomial and zero-inflated negative binomial) lead to the same main conclusions. All analyses and results can be found in our analysis script on OSF.

3 Control variables (i.e., sex and age) as well as interactions with control variables were only retained in the final model if they significantly improved deviance.

4 Modeling the missing category versus all the other social roles as well as the interaction with this category and condition for each dependent variable confirmed this assumption as there were no significant effects (see supplementary materials on our OSF repository).

5 Note that no parameters for time (block) are included in the final model. This is because the relationship quality ratings are relatively stable over time, as is also indicated by the relatively low residual variance at level 1 in the three analyses.

6 Interestingly, adding variables capturing the timing of interactions (modeled as dummies for three-hour time blocks) did not improve deviance significantly and none of the time blocks was a significant predictor in any of the fully specified models, thus our final models do not explicitly model time.

7 For further sample evaluation and constraints on generality please refer to Himmelstein et al. (2019).