Does Question Format Matter in Assessing the Prevalence of Sexual Aggression? A Methodological Study

ABSTRACT

As research on sexual aggression has been growing, methodological issues in assessing prevalence rates have received increased attention. Building on work by Abbey and colleagues about effects of question format, participants in this study (1,253; 621 female; 632 male) were randomly assigned to one of two versions of the Sexual Aggression and Victimization Scale (SAV-S). In Version 1, the coercive tactic (use/threat of physical force, exploitation of the inability to resist, verbal pressure) was presented first, and sexual acts (sexual touch, attempted and completed sexual intercourse, other sexual acts) were presented as subsequent questions. In Version 2, sexual acts were presented first, and coercive tactics as subsequent questions. No version effects emerged for overall perpetration rates reported by men and women. The overall victimization rate across all items was significantly higher in the tactic-first than in the sexual-act-first conditions for women, but not for men. Classifying participants by their most severe experience of sexual victimization showed that fewer women were in the nonvictim category and more men were in the nonconsensual sexual contact category when the coercive tactic was presented first. Sexual experience background did not moderate the findings. The implications for the measurement of self-reported sexual aggression victimization and perpetration are discussed.

The last decades have seen a continuous increase in research seeking to establish the prevalence of sexual aggression (Depraetere et al., 2018; Fedina et al., 2018; Muehlenhard et al., 2017). This increase has been accompanied by growing attention to the methodological issues involved in estimating the prevalence of sexual aggression. Sexual aggression may be defined as behavior carried out with the intent or result of making another person engage in sexual activity despite his or her unwillingness to do so (Krahé et al., 2015, p. 683). It can only be assessed via self-reports from victims and/or perpetrators, and the way questions are phrased may have a substantial impact on the number of participants who report victimization experiences and perpetration behavior (Fisher, 2009; Krebs et al., 2017). Acknowledging this challenge, a growing body of research has examined the impact of methodological decisions on the prevalence rates derived from different measures of sexual aggression. Studies have examined the convergence of prevalence rates assessed by different instruments (Anderson & Delahanty, 2020; Testa et al., 2015) and shown that variations in item format and number of items produce different prevalence figures (Cook et al., 2011; Kolivas & Gross, 2007; Koss, 1993; Krahé et al., 2014). Other studies have found that using behaviorally specific questions produce significantly higher prevalence rates compared with narrow direct questions regarding attempted or completed rape (e.g., “Have you been raped?”) (e.g., Fisher et al., 2000), potentially because they provide better memory cues. Based on this research, the use of multiple and behaviorally specific items is considered today to be the gold standard for establishing the prevalence of sexual aggression victimization and perpetration (e.g., Koss et al., 2007; Krahé & Vanwesenbeeck, 2016).

Ensuring that prevalence rates of sexual aggression are based on reliable measurement tools is critical for several reasons. It is required for generating a knowledge base about the scale of the problem, for identifying vulnerable groups and predicting the consequences of sexual victimization. Moreover, it is the precondition for evidence-based interventions as well as policy decisions, including legal measures and funding allocations, designed to tackle sexual aggression as a societal problem.

An important aspect of the reliability of tools for measuring sexual aggression is the demonstration that the prevalence rates they yield are robust against variations of question format, mode and context of question administration, and other variables that might affect responses to questions tapping into the same space of experience. Due to the prominence of the Sexual Experiences Survey (SES; Koss et al., 1987, 2007, 2008), this instrument has been examined in several methodological studies (Anderson et al., 2017; Krahé et al., 1999; McCallum & Peterson, 2017). In the revised version of the SES (Koss et al., 2007, 2008), the questions are phrased in gender-neutral language to assess nonconsensual sexual contacts in different gender constellations of victims and perpetrators. The SES includes several nonconsensual sexual acts (i.e., fondling, kissing, oral sex, and penetration with penis, fingers, or other objects) in combination with several coercive tactics (e.g., threat or use of physical harm, telling lies, or threatening to end the relationship). The available evidence, although limited, shows that the measure is both reliable and valid, with validity established in terms of the convergence of reports on the SES with reports on other measures of sexual aggression (Anderson et al., 2017; Buday & Peterson, 2015) and consistency across different scoring approaches (Davis et al., 2014). At the same time, some open questions remain. For example, McCallum and Peterson (2017) found that victimization reports by Black and White women differed when the items were administered in a paper-and-pencil format, but not when they were presented in a computerized format. Another study examined whether the order in which research participants were prompted to think about (a) nonconsensual sexual acts and (b) the coercive tactics used by the perpetrator to make them engage in these nonconsensual acts affected self-reported prevalence rates (Abbey et al., 2005). Although the item content was identical, the version that presented the coercive tactic first produced higher self-reported prevalence rates of victimization among women and higher perpetration rates among men compared with the version in which the type of sexual act was presented first (see next section for a detailed description of this study).

Building upon the study by Abbey et al. (2005), the present study was designed to examine potential effects of question framing on a different instrument for eliciting reports of sexual aggression, the Sexual Aggression and Victimization Scale (SAV-S; Krahé & Berger, 2013). The SAV-S, originally developed in Germany, has been used in a range of different countries to assess the prevalence of sexual aggression victimization and perpetration among young adults and to study risk and vulnerability factors as well as outcomes of victimization and perpetration (e.g., Krahé & Berger, 2017; Krahé et al., 2015; Schuster, Krahé, Ilabaca Baeza, et al., 2016; Schuster, Krahé, & Toplu-Demirtaş, 2016; Tomaszewska & Krahé, 2018). Going beyond the SES, the SAV-S additionally addresses the relationship between victim and perpetrator. The cross-classification of three coercive tactics (use or threat of physical force, exploitation of the victim’s inability to resist, verbal pressure), four sexual acts (sexual touch, attempted sexual intercourse/penetration, completed sexual intercourse/penetration, and other sexual acts, such as oral sex), and three victim-perpetrator constellations (current/former partner, friend/acquaintance, stranger) results in 36 items eliciting reports of sexual victimization and 36 parallel items eliciting reports of perpetration. As an online tool, the SAV-S offers the possibility of assigning participants to tailored versions depending on their gender and sexual experience background (i.e., whether they had exclusively opposite-sex contacts, exclusively same-sex contacts, or both opposite- and same-sex contacts). Because there is evidence that individuals who engage in sex with both men and women are more likely to report sexual victimization (e.g., Canan et al., 2019; Krahé & Berger, 2013; Wegner & Davis, 2020), assessing the victimization experiences of these individuals constitutes an important task in research on the prevalence of sexual aggression, as facilitated by the SAV-S.

The SAV-S has been validated in a qualitative interview study in nine European countries, which revealed that participants interpreted the items as intended by the researchers (Krahé et al., 2016). The current study aimed to test the reliability of the SAV-S by examining whether self-reported prevalence rates would vary depending on the way in which the two components of the items (coercive tactic and sexual act) are presented. Before presenting the design and objective of our study in detail, we discuss existing evidence on the effect of differences in item format on reports of sexual aggression victimization and perpetration.

Effects of Item Format on Self-Reports of Sexual Aggression

Behaviorally specific questions about sexual aggression victimization or perpetration contain two elements: a description of the coercive tactic used by the perpetrator, such as the use of force or verbal pressure, and a description of the sexual act in which the victim was made to engage, such as attempted or completed sexual intercourse. To decide whether to answer “yes” or “no” to a question, participants must therefore search for experiences that meet both of these aspects. For example, to answer a victimization question, they need to decide whether or not they have ever been exposed to the specific coercive tactic to make them engage in sex against their will, and what specific sexual act was involved. The question addressed in the present study is whether the order in which these search processes are activated by the way the question is presented impacts the likelihood of arriving at a “yes” or “no” response.

Order effects in response to survey questions have been demonstrated in several studies. For example, it has been shown that participants’ responses to an item are impacted by their responses to the previous item. The previous item activates certain cognitions that may provide a kind of interpretative framework that guides responses to subsequent items (e.g., Terentev & Maloshonok, 2019; Tourangeau et al., 2003). Although several empirical studies have demonstrated order effects in questionnaires in different fields (e.g., Shorey et al., 2016; Tourangeau et al., 2003), the study by Abbey et al. (2005) is the only one that has examined order effects in questionnaires measuring sexual aggression. They found that the less invasive sexual acts, such as fondling and kissing, were more affected by the manipulation of question format than the more invasive, penetrative acts, with higher rates in the tactic-first version. The differences were small for attempted and completed sexual intercourse and oral sex. Order effects were also larger for the tactics referring to verbal pressure than for the tactics involving the administration of alcohol or drugs or the use of physical force. To explain their findings, the authors reasoned that items presenting the coercive tactic rather than the sexual activity first might activate memories of victimization and perpetration more effectively, which would have a greater impact on the less severe forms of sexual aggression than on the more severe forms, as the latter may be assumed to have a higher chronic salience.

The Current Study

The purpose of the present study was to examine whether the effects of question format found by Abbey et al. (2005) would also hold for the SAV-S. Moreover, our study extended their scope in several ways, thereby contributing to the methodological discourse in sexual aggression research beyond its implications for the SAV-S. First, our assessment of victimization experiences also included men, and our assessment of perpetration behavior also included women, with a sample size that was twice as high for women and almost four times as high for men. Second, we collected victimization and perpetration reports for different relationship constellations between victims and perpetrator, facilitating a more fine-grained examination of reports of sexual aggression that might be more or less susceptible to order effects. Third, we sought to test the statistical significance of differences between the two versions not only for the dichotomous overall victim and perpetrator status, but also for the individual items. Fourth, we examined whether the nonredundant severity score proposed by Koss et al. (2008) to classify victimization and perpetration reports by the most severe form of sexual aggression would be affected by the order manipulation. This analysis is especially important because the ordinal scoring procedure proposed by Koss et al. (2007) is widely used in the prediction of sexual aggression victimization and perpetration (e.g., Johnson et al., 2017; Schuster & Krahé, 2019a, 2019b; Zinzow & Thompson, 2015) and was found to be valid in the analysis by Davis et al. (2014). Like Abbey et al. (2005), we focused on young adults because they are the target group that is most often studied with self-report instruments, such as the SES and the SAV-S.

To test order effects, we created two versions of the SAV-S: The original version starts with the description of the coercive tactic and presents different sexual activities in different relationship constellations under each coercive tactic. The reversed version starts with the description of the sexual activity, and groups different coercive tactics in different relationship constellations under the respective sexual act (see examples of each version in the Supplementary Material). Based on the findings and theoretical reasoning by Abbey et al. (2005), we expected that the tactic-first version would result in higher prevalence rates of both victimization and perpetration than the version presenting the sexual acts first (Hypothesis 1). We further expected that this effect would be mainly due to the less severe forms of sexual aggression, as reflected at the level of individual items about nonconsensual touch through the use of verbal pressure, and at the level of nonconsensual sexual contact (the least severe form of sexual aggression) in the ordinal score proposed by Koss et al. (2007) (Hypothesis 2). Finally, we expected that the response patterns would be the same for victimization and perpetration reports, for men and women, and for participants with exclusively opposite-sex and both opposite- and same-sex experiences, as there is no theoretical reason to assume that differences in the memory search triggered by the two versions would vary by gender or sexual experience background. We tested potential differences in relation to these variables in an exploratory analysis.

Method

Sample

A total of 1,341 participants completed the survey. Because we wanted to focus on young adults, we excluded participants below the age of 18 (n = 2) and above the age of 35 (n = 73). In addition, we excluded 13 participants who self-identified as non-binary on the question about gender, because their number was too small for separate analyses. The final sample consisted of 1,253 participants (621 female, 632 male). The mean age of the sample was 25.1 years (SD = 4.12; range: 18–35 years), which is similar to the Abbey et al. (2005) study, where the mean age was 24.2 years (SD = 3.4). Women were slightly younger (M = 24.8, SD = 4.12) than men (M = 25.5, SD = 4.09), t(1245) = − 2.79, p =.005. No age differences were found between the two order conditions. Most participants were German nationals (90.5%). More than half the sample were college students (65.6%). The remaining participants were working (29.6%), unemployed (2.0%), self-employed (2.0%), unable to work (0.3%), or housewives/househusbands (0.5%). Regarding relationship experiences, 90.5% of the participants reported that they had been in a steady relationship, either at the time of the survey or before. With respect to sexual experiences, more than half of the participants (65.8%) had opposite-sex contacts only, 25.6% had both same‐sex and opposite-sex contacts, 3.8% had same-sex contacts only, and 4.7% reported that they had no consensual sexual experiences with either a woman or a man. More women than men reported sexual interactions with both opposite-sex and same-sex partners (34.0% vs. 19.5%), and fewer women than men reported exclusively same-sex contacts (1.6% vs. 6.5%) and exclusively opposite-sex partners (64.3% vs. 74.1%), ps < .001. Mean age at first sex was 17.1 years (SD = 2.70), with women (M = 16.6, SD = 2.73) reporting a lower age than men (M = 17.6, SD = 2.57), t(1176) = − 6.14, p < .001. No differences in sexual experiences and age at first sex were found between the two experimental conditions.

Instruments

Sexual Aggression Victimization and Perpetration

The Sexual Aggression and Victimization Scale (SAV-S; Krahé & Berger, 2013) was used to assess the prevalence of sexual aggression victimization and perpetration since the age of 14, the age of consent in Germany. Building on Koss et al.’s (2007) revised version of the SES, the SAV-S differentiates between three coercive tactics (threat or use of physical force, exploitation of the inability of the victim to resist, and use of verbal pressure) and four sexual activities (sexual touch, attempted sexual intercourse/penetration, completed sexual intercourse/penetration, and other sexual acts, e.g., oral sex). In addition, the SAV-S breaks down the reports by three different relationship constellations between victim and perpetrator (current/former partner, friend/acquaintance, and stranger). Altogether, the SAV-S comprises 36 items each for victimization and perpetration (three coercive tactics × three victim-perpetrator constellations × four sexual acts). The instrument was developed in German and has since been used in a range of other languages (e.g., Krahé et al., 2015).

In the original version of the SAV-S, the coercive tactic is presented at the superordinate question level (e.g., “Has a man ever made (or tried to make) you have sexual contact with him against your will by threatening to use force or by harming you (e.g., by hurting you, holding you down, or threatening to do so)?”). Then, participants respond to the four sexual-act questions for each of the three victim-perpetrator constellations. This version was used in the tactic-first condition of the present study. In the sexual-act-first version, the questions started with the description of the sexual act (e.g., “Has a man ever sexually touched (or tried to sexually touch) you against your will?”). Below this superordinate question, participants responded to the three coercive tactics within each of the three victim-perpetrator constellations.

In both conditions, participants received a questionnaire version tailored to their gender and sexual experience background. Three screening questions were used: (1) gender (female/male/other), (2) whether they ever had sex with a member of the opposite sex, and (3) whether they ever had sex with a person of the same sex (male/female). For example, women who reported only opposite-sex contacts were asked questions referring to a male perpetrator (victimization part) and a male victim (perpetration part). Women who reported only same-sex contacts were asked the questions referring to a female perpetrator (victimization part) and a female victim (perpetration part). The response format was no (0) or yes (1) for the opposite-sex contact only and same-sex contact only questionnaire versions, and no (0), yes, a man (1), and yes, a woman (2) for both opposite-sex and same-sex contact questionnaire versions. “Yes” responses in that version were collapsed into a dichotomous score (no/yes). All items were presented in German.

Procedure

The study protocol and all instruments were approved by the Institutional Review Board of the authors’ university. Data were collected via an online survey entitled “Study on nonconsensual sexual experiences”. The study was advertised on social media platforms (e.g., groups of university students in general and psychology students), online forums (e.g., counseling agencies, LGBTI groups), and mailing lists of student councils. Additionally, flyers with the study link were distributed in supermarkets. Information about the study was provided on the first page of the survey. After providing active consent, participants were randomly assigned to the tactic-first (original) version of the SAV-S or to the sexual-act-first version created for the purpose of the current study. A list with contact details of counseling agencies on sexual aggression could be accessed via a “help button” on all pages that contained items on sexual aggression victimization and perpetration. Participants could choose to take part in a raffle of twenty gift cards worth 25 Euros in return for participation.

Plan of Analysis

The data analysis proceeded in the following steps: First, we calculated the overall prevalence rates of victimization and perpetration, assigning ‘0’ to those who responded “no” to all items and ‘1’ to those who endorsed at least one victimization/perpetration item. Second, we calculated the percentage of “yes” responses for each of the victimization and perpetration items. Finally, we created nonredundant scores of victimization and perpetration for all participants based on their most severe victimization experience and perpetration behavior reported, using the classification proposed by Koss et al. (2007): (0) non-victim/non-perpetrator: “no” responses to all victimization/perpetration items; (1) sexual contact: endorsement of at least one item of nonconsensual sexual contact without penetration (i.e., sexual touch) or other sexual acts, but none of the items referring to attempted coercion, coercion, attempted rape, and rape; (2) attempted coercion: at least one experience of attempted sexual intercourse through verbal pressure, but not coercion, attempted rape, and rape; (3) coercion: endorsement of at least one item of completed sexual intercourse through verbal pressure, but no attempted rape or rape; (4) attempted rape: endorsement of at least one experience of attempted sexual intercourse through the exploitation of the victim’s inability to resist or the use or threat of physical force, but no completed rape; (5) rape: endorsement of at least one item of completed sexual intercourse through the exploitation of the victim’s inability to resist or the use or threat of physical force. Because cell frequencies for attempted and completed sexual coercion were low, the two categories were combined into a single category of sexual coercion, including both attempted and completed acts. This approach follows other studies that used the same classification (Johnson et al., 2017). The findings reported below for the ordinal scores are based on this final five-level categorization.

For the tests of differences between the two order versions, we adopted the criterion of more than 20 cases for all categories to allow reliable estimates, guided by the approach adopted by Black et al. (2011). Significant differences between the two versions in the frequencies for each item were tested by Chi² tests. To control for multiple testing, we applied the Holm adjusted p-value approach rather than the more conservative Bonferroni correction (Chen et al., 2017). Order effects on the severity scores for the distribution of victimization as a whole for men and women and for participants with exclusively opposite-sex and those with both opposite- and same-sex contacts were examined by Chi² tests, followed up by post-hoc Chi² tests for each category. Holm adjustment of the alpha level of p < .05 was again applied for the post-hoc tests.

Results

Frequency Counts in the Total Sample

Across both order versions, the percentage of participants who endorsed at least one of the SAV-S victimization items was 80.5% for women and 46.6% for men, χ² (1, N = 1221) = 151.80, p < .001. The prevalence rates for perpetration were 19.3% for men and 12.0% for women, χ² (1, N = 1206) = 12.01, p = .001.

Effects of Question Format

For the victimization reports, the majority of individual items met the criterion of a frequency count of n > 20 for women, but not for men. Therefore, only women’s responses were used to test order effects at the item level. The nonredundant severity scores met the n > 20 criterion for most categories for women and men and were therefore included in the hypothesis testing for both gender groups. None of the perpetration items met the criterion of n > 20 participants with “yes” responses in either the male or the female sample. Therefore, our hypotheses regarding order effects based on the perpetration reports could only be tested for the dichotomous overall score. To document the levels of perpetration in the present sample, a summary table of prevalence rates for perpetration based on the nonredundant severity scores is presented in the Supplementary Material (Table SM1).

To test our hypothesis that the tactic-first version would result in a higher prevalence of sexual victimization compared with the version that presented the sexual act first, we first tested order effects for the dichotomous overall score. In the tactic-first version, 85.7% of women reported at least one experience of sexual victimization since the age of 14, compared with 75.6% of women in the sexual-act-first version. The difference was significant, χ² (1, N = 616) = 10.13, p = .001. Among men, the victimization rate was 49.8% in the tactic-first condition and 43.9% in the condition with the sexual act presented first. This difference was not significant, χ² (1, N = 605) = 2.08, p = .149. For perpetration, the prevalence rates for women were 14.2% in the tactic-first and 9.9% in the sexual-act-first condition, χ² (1, N = 599) = 2.70, p = .100. For men, the perpetration rates were 22.3% in the tactic-first and 16.7% in the sexual-act-first condition, χ² (1, N = 607) = 3.02, p = .082. Thus, our first hypothesis that there would be a higher rate in the tactic-first condition was confirmed on the basis of the overall dichotomous scores only for women’s victimization rates, but not for men’s victimization and for perpetration rates of men and women.

In our second hypothesis, we predicted that the order manipulation would especially influence prevalence rates of sexual touch, the least invasive form of sexual victimization. Women’s victimization rates at the item level in the two conditions are presented in Table 1.

Table 1. Women’s reports of sexual victimization since age 14 by coercive tactic, victim–perpetrator relationship, type of sexual act, and condition; % (n)

					Coercive Tactic
		Use/Threat of Physical Force			Exploitation of Inability to Resist			Verbal Pressure
Victim-Perpetrator Relationship	Sexual act	Tactic first	Sexual act first	Diff	Tactic first	Sexual act first	Diff	Tactic first	Sexual act first	Diff
(Ex-)Partner	Sexual touch	21.0 (63)	15.3 (48)	5.7	12.6 (38)	14.4 (45)	-1.8	23.6 (71)	22.8 (71)	0.8
	Attempted sex. inter.	17.1 (51)	9.9 (31)	7.2	10.3 (31)	9.2 (29)	1.1	21.3 (64)	13.5 (42)	7.8
	Completed sex. inter.	10.7 (32)	7.7 (24)	3.0	6.0 (18)	8.3 (26)	-2.3	18.0 (54)	11.5 (36)	6.5
	Other (e.g., oral sex)	13.7 (41)	8.3 (26)	5.4	5.3 (16)	6.1 (19)	-0.8	17.9 (54)	14.4 (45)	3.5
Friend/ Acquaintance	Sexual touch	32.5 (98)	15.5 (48)	17.0*	31.5 (94)	28.9 (90)	2.6	21.9 (66)	17.4 (54)	4.5
	Attempted sex. inter.	16.3 (49)	12.5 (39)	3.8	18.1 (54)	18.3 (57)	-0.2	14.6 (44)	14.1 (44)	0.5
	Completed sex. inter.	8.3 (25)	6.1 (19)	2.2	14.0 (42)	13.4 (42)	0.6	8.9 (27)	9.9 (31)	-1.0
	Other (e.g., oral sex)	12.6 (38)	7.7 (24)	4.9	10.0 (30)	12.6 (39)	-2.6	12.0 (36)	11.6 (36)	0.4
Stranger	Sexual touch	42.3(127)	23.1 (72)	19.2*	34.1(102)	28.1 (87)	6.0	19.9 (60)	9.4 (29)	10.5*
	Attempted sex. inter.	9.8 (29)	13.1 (41)	-3.3	10.7 (32)	16.7 (52)	-6.0	8.3 (25)	6.4 (20)	1.9
	Completed sex. inter.	4.4 (13)	4.8 (15)	-0.4	6.7 (20)	8.7 (27)	-2.0	4.7 (14)	2.9 (9)	1.8
	Other (e.g., oral sex)	9.8 (29)	5.8 (18)	4.0	11.0 (33)	8.3 (26)	2.7	5.6 (17)	3.8 (12)	1.8

Note. Diff = difference between the two versions. Multiple responses were possible. Figures in italics denote items below the minimum frequency of n > 20, for which no comparisons were made.

*p < .05 (based on Holm adjustment)

Out of the 27 comparisons, three significant differences were found, all for the sexual touch category. In the tactic-first version, more women reported nonconsensual sexual touch through the use of threat of physical force by a friend or acquaintance and by a stranger as well as nonconsensual sexual touch through verbal pressure by a stranger. Aggregating across the three victim–perpetrator relationships and coercive strategies, significantly higher rates of nonconsensual sexual touch were reported in the tactic-first condition. The results of this analysis are presented in the Supplementary Material (Table SM2). A parallel analysis for men’s victimization reports showed that higher rates of nonconsensual sexual touch through the use or threat of force was found in the tactic-first condition (see Table SM3 in the Supplementary Material). Hence, our second hypothesis was partly supported inasmuch as all significant differences related to sexual touch as a less severe form of sexual victimization.

Table 2 shows the prevalence rates for the ordinal severity scores of sexual victimization for women and men in the two conditions.

Table 2. Sexual victimization of women and men since age 14 based on scoring proposed by Koss et al. (2007, 2008), % (n)

	Women (n = 616)			Men (n = 605)
	Tactic first (n = 301)	Sexual act first (n = 315)	Diff	Tactic first (n = 275)	Sexual act first (n = 330)	Diff
(0) No victimization	14.3 (43)	24.4 (77)	-10.1*	50.2 (138)	56.1 (185)	-5.9
(1) Nonconsensual sexual contact	23.3 (70)	19.7 (62)	3.6	25.1 (69)	13.6 (45)	11.5*
(2) Sexual coercion	9.0 (27)	9.5 (30)	-0.5	4.7 (13)	7.6 (25)	-2.9
(3) Attempted rape	21.6 (65)	16.2 (51)	5.4	8.4 (23)	8.5 (28)	-0.1
(4) Rape	31.9 (96)	30.2 (95)	1.7	11.6 (32)	14.2 (47)	-2.6

Note. Diff = difference between the two versions. No comparison was made for the sexual coercion category for men because one cell size did not meet the minimum number of > 20 cases.

*p < .05 (based on Holm adjustment).

The overall frequency distributions differed significantly between the two order conditions for both women, χ² (4, N = 616) = 11.66, p = .020, and men, χ² (4, N = 605) = 14.14, p = .007. These overall effects were followed up by post-hoc Chi² tests for each category. Among women, lower rates were found for the no victimization category in the tactic-first version compared with the version presenting the sexual act first. This finding means that victimization rates were higher when the coercive tactic was presented first. Among men, higher rates were found for the nonconsensual sexual contact category in the tactic-first version compared with the sexual act-first version. No further differences between the two versions emerged.

Comparing Participants with Different Sexual Experience Backgrounds

The final set of analyses examined potential differences in the version effects on participants differing in sexual experience background, comparing participants with exclusively opposite-sex contacts (65.8%) and participants who reported sexual contacts with both members of the opposite and the same sex (25.6%). The groups of participants with exclusively same-sex participants (3.8%) and with no consensual sexual experience (4.7%) were too small to warrant separate analyses. On the overall score of victimization, the two order versions differed for participants with only opposite-sex contacts, χ² (1, N = 803) = 11.68, p = .001. No significant version effect emerged for participants with both opposite- and same-sex contacts, χ² (1, N = 316) = 0.77, p = .381. Next, we replicated the analysis reported in Table 2 by conducting post-hoc Chi² tests on the ordinal measure of victimization, as presented in Table 3.

Table 3. Sexual victimization by sexual experience background since age 14 based on scoring proposed by Koss et al. (2007, 2008), % (n)

	Opposite-sex contacts only (n = 803)			Same‐sex and opposite-sex contacts (n = 316)
	Tactic first (n = 372)	Sexual act first (n = 431)	Diff	Tactic first (n = 162)	Sexual act first (n = 154)	Diff
(0) No victimization	32.5 (121)	44.3 (191)	-11.8*	22.8 (37)	18.8 (29)	4.0
(1) Nonconsensual sexual contact	24.5 (91)	15.3 (66)	9.2*	22.8 (37)	20.1 (31)	2.7
(2) Sexual coercion	7.8 (29)	9.0 (39)	-1.2	6.2 (10)	9.7 (15)	-3.5
(3) Attempted rape	16.7 (62)	12.8 (55)	3.9	14.8 (24)	13.0 (20)	1.8
(4) Rape	18.5 (69)	18.6 (80)	-0.1	33.3 (54)	38.3 (59)	-5.0

Note. Diff = difference between the two versions. No comparisons were made for the sexual coercion and attempted rape categories in the group with both same- and opposite-sex contacts because the overall Chi² test was not significant.

*p < .05 (based on Holm adjustment)

The analyses for the opposite-sex-only group replicated the findings for the sample as a whole. Fewer participants were in the group that reported no victimization when the tactics were presented first, which was due to a higher percentage of participants in the nonconsensual contact category. The other three levels of victimization did not differ significantly between the two versions, nor did any of the perpetration categories. For the subgroup of participants with both same-sex and opposite-sex experiences, no post-hoc tests were conducted because the overall Chi² test was nonsignificant.

Discussion

The present study was designed to contribute to the methodological advancement of sexual aggression research by testing the reliability of a self-report instrument for studying prevalence rates of sexual aggression victimization and perpetration. Specifically, we sought to examine the potential impact of variations in question format on self-reports of sexual aggression victimization and perpetration by a sample of male and female young adults. Past research has shown that prevalence rates of victimization and perpetration vary depending on the way the questions are worded, with behaviorally specific questions yielding higher rates than broad questions using summary labels, such as “rape” (Cook et al., 2011). In our study, we compared two versions of a validated tool for assessing sexual aggression that used identical item contents, but presented them in two different formats, leading either with the coercive tactic or with the sexual act. The study was guided by previous research by Abbey et al. (2005), who experimentally varied the format of introducing questions about coercive tactics and sexual acts in a modified version of the widely used Sexual Experiences Survey (SES) (Koss et al., 1987, 2007). They found that presenting the tactics first resulted in higher rates of reporting victimization by women and perpetration by men compared with presenting the sexual acts first, especially for verbally coercive tactics.

In our study, we used a different instrument for eliciting victimization and perpetration reports, the Sexual Aggression and Victimization Scale (SAV-S). As a guideline, we followed the approach adopted by Black et al. (2011) that there should be more than 20 participants per category to yield reliable estimates. Despite a substantial sample size of over 1,200 participants, the rates of perpetration were not high enough to warrant a comparative analysis of question format beyond the overall rates of perpetration based on a dichotomous yes/no classification of participants’ perpetrator status. For victimization, the majority of scores could be included in the comparative analysis based on that standard.

Regarding the overall reported rate of victimization, we found that the percentage of women who endorsed at least one of the victimization items was higher in the tactic-first than in the sexual-act-first version. This finding is in line with the results of Abbey et al. (2005). However, no parallel effect was found for men’s victimization reports or for the overall prevalence rates of perpetration in both gender groups. The effect of question format on the overall rates of self-reported victimization by women raises the issue of which of the two formats is more likely to reflect the “true” scale of sexual victimization. In the absence of an external standard, two possible interpretations need to be considered: one is that presenting the tactics first may result in an over-reporting of experiences of victimization, the other is that presenting the sexual acts first may lead to an under-reporting of pertinent experiences. Arguments for the latter possibility may be derived from empirical findings in other areas of recall of significant personal events and from theoretical concepts of cognitive information processing. Empirical evidence on the recall of significant experiences from areas where external evidence is available suggests that survey participants do under-report experiences that would appear to be memorable, such as involvement in car accidents or hospitalization (Abbey et al., 2005). When such memories are made salient, for instance, by more specific questions, the discrepancies between the number of experiences reported and the number of experiences established by external evidence are reduced. In the domain of sexual aggression, many studies have found that a substantial number of women are “unacknowledged rape victims”. They answer “no” to the question “Have you ever been raped?”, but report experiences amounting to the definition of rape in response to behaviorally specific questions (see Wilson & Miller, 2016, for a meta-analysis).

In terms of theoretical models of information processing, the first part of an item may be conceptualized as a prime that activates the search for relevant experiences. Abbey et al. (2005) hypothesized that the type of coercive tactic would be most salient for (female) victims. Hence, the version in which the coercive tactic is presented first should prompt respondents to search for coerced sex rather than specific sexual acts (regardless of their consensual or nonconsensual nature), which should produce higher rates of sexual victimization. Their findings, as well as our results, are in line with this reasoning.

The three significant effects found in women’s victimization reports at the item level as well as aggregated across victim-perpetrator relationships and coercive strategies referred to sexual touch. This finding is consistent with our prediction and with the results by Abbey et al. (2005), who also observed the largest differences for the category of fondling/kissing. A tentative interpretation of these results could be that for more severe forms of sexual aggression, which are likely to be salient in participants’ memory and leave less room for interpretation, the relevant experiences are retrieved in response to the survey question regardless of the order in which tactics and sexual acts are presented. By contrast, less severe forms of sexual victimization may be more ambiguous, particularly in interactions with friends or strangers, so prompting coercive tactics first may facilitate the retrieval of nonconsensual sexual touch experiences to a greater extent than prompting touch as a sexual act before thinking about whether or not it was nonconsensual.

For men’s victimization reports, only the self-reported rate of nonconsensual sexual touch through the use or threat of force was affected by the order manipulation, with higher rates in the tactic-first condition, paralleling the effects for women. Why men’s reports were largely unaffected by the manipulation of question format cannot be explained conclusively on the basis of the present data, as this is the first study to examine effects of question format on male victimization reports. Studies have shown that the rate of unacknowledged victims may be even higher among men than among women (Anderson et al., 2017), so based on the priming explanation offered above, one would expect a similar finding of higher rates in the tactic-first condition. Alternatively, one might argue that the threshold for labeling an experience as sexual victimization may be even higher for men than for women because of the widely held stereotype that men cannot be sexually assaulted (Reed et al., 2020). By that reasoning, the prime included in the leading part of the question (tactic vs. sex act) may not have been strong enough to reduce the lack of acknowledgment in either condition. This post-hoc explanation is tentative and requires further research, for example, by using stronger manipulations to increase the salience of the nonconsensual nature of experiences covered by the survey items.

In a second step, we examined differences between the two versions on a five-level ordinal score of sexual victimization that classified participants in terms of the most severe form of sexual aggression experienced, based on a classification approach developed by Koss et al. (2007, 2008) for the SES. Both women’s and men’s reports could be included in this analysis. Ordinal severity scores are frequently used in analyses linking sexual aggression perpetration and/or victimization to risk or vulnerability factors (e.g., Gilmore et al., 2018; Santos-Iglesias & Sierra, 2012). Therefore, it is critical to examine whether or not these scores are susceptible to variations in question format. When severity scores were computed across all three victim-perpetrator relationships, one significant difference between the two versions emerged in each gender group: The percentage of women who reported no victimization was significantly lower in the tactic-first than in the sexual-act-first version. The additional cases of victimization identified in the tactic-first condition were mainly due to the nonconsensual contact and attempted rape categories, but post-hoc analyses showed that the differences between the two versions within these categories were not significant. In men’s victimization reports, the prevalence in the category of nonconsensual sexual contact was higher in the tactic-first condition, consistent with the proposition that order effects would be more likely for the less invasive/more ambiguous forms of sexual victimization. When the distributions were analyzed based on participants’ sexual experience background, it was found that in the group with exclusively opposite-sex contacts, fewer nonvictims were counted in the tactic-first condition, which was due to a higher count in the nonconsensual contact category. No significant order effects emerged for participants who had both opposite-sex and same-sex partners.

Over all analyses, the present findings demonstrate that the SAV-S produces higher overall prevalence rates of self-reported victimization by women when the questions lead with the coercive tactic than when they lead with the coerced sexual act. The analysis of the ordinal severity scores confirmed this difference, as the percentage of participants in the “no victimization” category was lower among women and participants with exclusively opposite-sex contacts. As argued above, there is reason to assume that leading with the coercive tactics facilitates the retrieval of relevant experiences to a greater extent than leading with the sexual acts, which supports the original question format used in the SAV-S.

Strengths, Limitations, and Directions for Future Research

We believe our study had several strengths. First, it included a much larger sample than the previous study by Abbey et al. (2005) which inspired the current research. This enabled us to adopt a more stringent criterion for the minimum number of observations per analytic category to obtain more reliable estimates, even though the numbers were still too small to allow analyses of order effects beyond the overall scores for male victimization and for both male and female perpetration. Second, whereas the earlier study obtained victimization reports from women and perpetration reports from men, our study asked women and men to report on both victimization and perpetration. Third, we used an instrument for assessing sexual aggression victimization and perpetration that was similar in several respects to the SES but differed in other respects, for example, in the consideration of different victim-perpetrator relationships. Therefore, the findings can contribute insights about question format that complement earlier research.

At the same time, several limitations must be noted. First, the small number of cases reporting perpetration precluded the testing of order effects except for the broadest measure of a dichotomous score. Much larger sample sizes would be needed to overcome this problem. For example, future surveys of sexual assault victimization and perpetration based on large, nationally representative samples, such as the National Intimate Partner and Sexual Violence Survey (NISVS; Black et al., 2011), might consider implementing a manipulation of question format to test the robustness of prevalence reports.

A second limitation of our study is the reliance on a convenience sample of young adults. Victimization rates – independent of the experimental manipulation – were substantially higher in the present sample than in previous studies conducted in Germany (e.g., Krahé & Berger, 2013), or in other studies conducted in the United States (see reviews by Fedina et al., 2018; Muehlenhard et al., 2017). This may be due, at least partly, to the recruitment strategy, advertising the study as a survey about nonconsensual sexual experiences. Given the sensitive nature of questions about sexual aggression victimization and perpetration, it is ethically necessary to inform participants about the study content, which may have led to a self-selection of participants with victimization experiences into the study. At the same time, this may explain the lower perpetration rate compared with other studies, including Abbey et al. (2005), who advertised their study as a study on health topics. However, our focus was not on establishing prevalence rates per se but on testing the manipulation of question format, and we randomly assigned participants to the two SAV-S versions. Therefore, even if self-selection might have increased prevalence rates in the sample as a whole, it is unlikely to have differentially affected responses in the two order conditions.

Third, varying the order of coercive tactics and sexual acts is only one test of possible effects of question format. Further effects could be due to the order in which the three coercive tactics (physical force, exploiting the inability to resist, and verbal pressure) or the three victim-perpetrator relationships (current or former partner, friend or acquaintance, stranger) are presented. Studying whether prevalence rates differ for these categories depending on the order of presentation would further strengthen the methodological rigor of instruments assessing the prevalence of sexual aggression.

A final limitation of our study, which is shared with the Abbey et al. (2005) design, is that we did not include measures that would allow us to clarify the exact mechanisms underlying responses to the different orders. For example, future studies could use thought-listing techniques to get insight into the cognitive search space opened up by the different question formats (e.g., Brown & Gold, 2014). Moreover, qualitative studies eliciting participants’ interpretations of the items used in quantitative surveys could identify points at which different question formats result in different decisions to endorse or reject a specific item (e.g., Canan et al., 2020; Krahé et al., 2016; Strang & Peterson, 2017).

In the expanding field of sexual aggression research, significant methodological progress has been made in recent years. This includes comparing results from instruments designed to measure similar constructs (Strang et al., 2013; Testa et al., 2015), comparing one-stage approaches that present specific items to all participants with two-stage approaches that ask specific questions only to those who endorsed a broad screening question (e.g., Cook et al., 2011), and exploring the interpretation of survey items in qualitative studies (Buday & Peterson, 2015; Krahé et al., 2016). The present findings were designed to contribute to these developments, which will lead to a more coherent and more rigorously tested set of tools for measuring sexual aggression.

Supplementary Material

Supplemental data for this article can be accessed on the publisher’s website.