Indirect scaling methods for testing quantitative emotion theories

Abstract

Two studies investigated the utility of indirect scaling methods, based on graded pair comparisons, for the testing of quantitative emotion theories. In Study 1, we measured the intensity of relief and disappointment caused by lottery outcomes, and in Study 2, the intensity of disgust evoked by pictures, using both direct intensity ratings and graded pair comparisons. The stimuli were systematically constructed to reflect variables expected to influence the intensity of the emotions according to theoretical models of relief/disappointment and disgust, respectively. Two probabilistic scaling methods were used to estimate scale values from the pair comparison judgements: Additive functional measurement (AFM) and maximum likelihood difference scaling (MLDS). The emotion models were fitted to the direct and indirect intensity measurements using nonlinear regression (Study 1) and analysis of variance (Study 2). Both studies found substantially improved fits of the emotion models for the indirectly determined emotion intensities, with their advantage being evident particularly at the level of individual participants. The results suggest that indirect scaling methods yield more precise measurements of emotion intensity than rating scales and thereby provide stronger tests of emotion theories in general and quantitative emotion theories in particular.

Keywords:

• Emotion intensity
• Emotion measurement
• Direct scaling methods
• Indirect scaling methods
• Rating scales
• Graded pair comparisons
• Maximum likelihood difference scaling
• Quantitative emotion theories

We thank Siegfried Macho for his helpful comments on a previous version of the manuscript.

We thank Siegfried Macho for his helpful comments on a previous version of the manuscript.

Notes

¹ We conceptualise emotions as mental states that are subjectively experienced as feelings and that manifest themselves in self-reports, expressive behaviours, and actions. Readers who prefer a multi-component view of emotion (according to which expressive behaviours and actions are components rather than indicators of emotions) should read our term “emotion” as referring to the feeling component of the multi-component state.

² Another probabilistic scaling model suitable for graded pair comparisons is the cumulative probit model (e.g., Boschman, 2001; Greene & Hensher, 2010). The cumulative probit model (also called ordinal regression) is similar to the AFM model in that it uses the original graded comparison judgements as input, and similar to the MLDS model in that it requires only ordinal data. Its main disadvantage is that it has many more parameters than MLDS because, in addition to the m scale values and the error variance, m−1 threshold parameters that mark the category boundaries need to be estimated. This can lead to estimation problems, particularly at the individual level. In addition, we prefer MLDS because it is based on a representational measurement model (Krantz et al., 1971; see also the general discussion). However, it should be noted that Boschman (2001) found that the scale values obtained with the cumulative probit model were highly similar to those obtained with the AFM model.

³ In a recent study (Junge & Reisenzein, 2013a), we compared the MLDS solutions obtained from quadruple comparisons of emotional stimuli with those derived from graded pair comparisons of the same stimuli. For most participants, high agreements of the solutions were found.

⁴ In keeping with the common representation of subjective probability and utility, we assume that b(p), the belief strength, is represented by real numbers from [0,1], with 1 denoting certainty that p, 0.5 maximal uncertainty, and 0 certainty that not-p; and that d(p) represents the direction and strength of the desire for p, with values >0 denoting positive desire, 0 indifference, and values <0 denoting negative desire, or an aversion to p.

⁵ In the case of belief, it is actually not quite correct to speak of a “psychophysical” function because b(p) also includes (and in our view, even mainly reflects) the weight attached to perceived probabilities in the emotion-generating process. The case here is parallel to that of the decision-weighting function proposed in prospect theory (Fox & Poldrack, 2009; Kahneman & Tversky, 1979). Analogous to the decision weights of prospect theory, we propose speaking of “emotion weights” (Junge & Reisenzein, 2010). However, important as this clarification (or amendment) of the belief–desire theory of emotion is on a theoretical level, it makes no difference to the empirical tests of the emotion models reported here.

⁶ Losses can also cause disappointment, and gains can cause relief, under certain circumstances: namely, when participants focus on avoiding the possible loss or on missing the possible gain. However, we have found that without prompting (e.g., by explicitly asking participants how disappointed they feel about failing to avoid a possible loss), only a subset of the participants spontaneously construe the lottery outcomes in this way.

⁷ We also tried nonmetric multidimensional scaling (restricted to one dimension) for this purpose, but obtained inferior results.

⁸ In addition, the reliabilities of the MLDS scalings were higher (.98/.99) than those of the AFM scalings (.90/.91). We attribute this to the fact that the split-half scale values whose correlation was used as the index of reliability were estimated from many more, as well as more diverse data points, in the case of MLDS than in the case of AFM. As a result, the MLDS estimates were more stable.

⁹ The model is called semi-quantitative because it does not specify the form of the proposed function beyond declaring it to be increasing; however, a more precise specification seems within reach (e.g., Tversky, 1977).

¹⁰ Interaction effects were larger for MLDS than for AFM in Study 2 (see Figure 5) and partly also in Study 1 (for relief). However, without additional investigation (e.g., simulation studies), we find it difficult to say whether this finding reflects greater sensitivity of MLDS for interaction effects or a tendency of MLDS to overestimate the size of these effects.