Categorical and dimensional perceptions in decoding emotional facial expressions

Abstract

We investigated whether categorical perception and dimensional perception can co-occur while decoding emotional facial expressions. In Experiment 1, facial continua with endpoints consisting of four basic emotions (i.e., happiness–fear and anger–disgust) were created by a morphing technique. Participants rated each facial stimulus using a categorical strategy and a dimensional strategy. The results show that the happiness–fear continuum was divided into two clusters based on valence, even when using the dimensional strategy. Moreover, the faces were arrayed in order of the physical changes within each cluster. In Experiment 2, we found a category boundary within other continua (i.e., surprise–sadness and excitement–disgust) with regard to the arousal and valence dimensions. These findings indicate that categorical perception and dimensional perception co-occurred when emotional facial expressions were rated using a dimensional strategy, suggesting a hybrid theory of categorical and dimensional accounts.

Keywords:

• Hybrid theory of emotion
• Dimension
• Category
• Facial expressions

In recent years, psychology has been dominated by two theories of emotion: a categorical theory and a dimensional theory. The categorical theory proposes the presence of six basic, distinct, and universal emotions (Ekman, 1992; Ekman & Friesen, 1971, 1976; Ekman, Sorenson, & Friesen, 1969): happiness, anger, sadness, surprise, disgust, and fear (Ekman & Friesen, 1971; Johnson-Laird & Oatley, 1992; Tomkins & McCarter, 1964), whereas the dimensional theory proposes fundamental dimensions that constitute emotional spaces (Russell, 1980; Russell & Bullock, 1985). Dimensional theorists posit two fundamental dimensions: valence, which represents the hedonic tone or pleasantness–unpleasantness continuum, and level of arousal (Russell, 1980; Russell & Bullock, 1985) or tension (Schlosberg, 1954), which refers to the level of energy.

Research on facial expressions, a powerful medium for emotional communication, has demonstrated categorical and dimensional perception in decoding facial expressions. For example, based on the categorical-perception effect (Harnad, 1987), Etcoff and Magee (1992) explored categorical perception of facial expressions by morphing images from two face drawings into prototypes representing the six basic emotions. Two tasks, identification and discrimination, were used to assess categorical perception. During the identification task, participants identified each stimulus from two alternative options derived from prototypical facial expressions. During the discrimination task, they discriminated pairs of stimuli along a continuum based on the physical features of faces. Etcoff and Magee (1992) found that participants were able to discriminate particular pairs of stimuli more accurately than other pairs of stimuli and they identified a categorical boundary that separated the two. That is, they found evidence for a human ability to categorise emotional facial expressions, which was particularly robust in the case of facial expressions depicted in photographs (Calder, Young, Perrett, Etcoff, & Rowland, 1996; Young et al., 1997). In recent years, the nature of categorical perception of facial expressions has been further explored. Roberson and Davidoff (2000) found that verbal interference during a discrimination task diminished the categorical-perception effect, but visual interference did not, indicating that categorical perception involves a verbal code representing target expressions. As this finding has been replicated (Roberson, Damjanovic, & Pilling, 2007), it appears that the categorical-perception effect, in which cross-category pairs are discriminated better than are within-category pairs, emerges from the generation of verbal coding (see Roberson, Damjanovic, & Kikutani, 2010, for a review).

Contrary to accounts focusing on categorical perception, several studies have reported that emotional facial expressions were recognised based on dimensions of emotional space. Takehara and Suzuki (1997) collected psychological ratings in response to morphed facial images of a real person. They analysed the data using multidimensional scaling, which enables visualisation of similarities or dissimilarities in the data by locating the more similar stimuli closer to one another on a plot. Takehara and Suzuki (1997) indicated that each original facial expression created a circumplex structure that included both valence and arousal dimensions. Notably, they also found that the morphed images did not cluster into categories, but were located intermediately between the two original facial expressions. This result suggests that facial expressions are recognised dimensionally. The dimensional account of facial expression recognition has been replicated (Katsikitis, 1997; Takehara & Suzuki, 2001).

Thus, evidence supporting both categorical and dimensional approaches to facial expression recognition has been produced. In response to the conflict between these approaches, a hybrid of the categorical and dimensional theories has been proposed (Christie & Friedman, 2004; Panayiotou, 2008; Russell, 2003). According to the hybrid theory, people can use both categorical perception and dimensional perception to decode facial expressions. However, the relative dominance of categorical or dimensional perception and the ways in which these modes interact with each other remain unclear.

Interpretations of previous research findings showing that either categorical or dimensional perception is dominant should consider the response formats used for the psychological ratings and the methods used for the data analysis. For example, identification tasks require participants to classify a facial expression into one emotional category. Discrimination tasks can assess the ability to detect a difference between two facial expressions. These explicit strategies to evaluate facial expressions may lead to evidence of categorical perception. On the other hand, multidimensional scaling is appropriate to an alternative approach in which emotional space is seen as derived from similarities of facial stimuli, and valence and arousal dimensions are viewed as arbitrarily determined (Katsikitis, 1997; Takehara & Suzuki, 1997). Acknowledgement that the response format and the method of data analysis may influence evidence for categorical or dimensional perception is crucial in evaluating research on the recognition of facial expressions.

The present study investigated how a hybrid of categorical and dimensional theories can be applied to the recognition of facial expressions. The facial stimuli were created by a morphing technique so that they could be interpreted from both categorical and dimensional perspectives. In Experiment 1, two facial continua represented changes in valence and arousal. The endpoints of these continua were members of the six basic emotions (i.e., happiness–fear for valence and anger–disgust for arousal). In Experiment 2, two additional facial continua, surprise–sadness and excitement–disgust, were constructed from high- and low-arousal expressions to represent a fuller range of the arousal dimension. In terms of response formats, we employed identification and discrimination (e.g., Young et al., 1997) tasks for the categorical strategy and the Affect Grid (Russell, Weiss, & Mendelsohn, 1989) for the dimensional strategy. The Affect Grid is a means of assessing affect along the dimensions of valence and arousal.

To test the hybrid theory, we examined whether categorical or dimensional perception was totally dependent on response format. If so, category boundaries within each continuum would be found only by use of the identification and discrimination tasks, and each stimulus would be located along the continuum defined by the valence or arousal dimension of the Affect Grid. On the other hand, if categorical perception and dimensional perception can occur irrespective of response format, it is possible that the two co-occur within one strategy. That is, if people use both categorical perception and dimensional perception simultaneously, a continuum of facial stimuli should be divided into two clusters, and facial stimuli should be simultaneously arranged within each cluster according to their emotional intensity on the Affect Grid. With respect to the identification and discrimination tasks, we should find an explicit categorical boundary on the facial-stimuli continuum. In addition, task performance in response to each stimulus within each cluster should show linear changes, indicating that dimensional perception partly contributed to a categorical strategy. Resolution of these issues can provide useful evidence supporting the existence of a hybrid of categorical and dimensional theories.

EXPERIMENT 1

Experiment 1 investigated whether categorical or dimensional perception occurs irrespective of the response format (i.e., the categorical or dimensional strategy). To allow the two types of perception to be demonstrated, facial continua representing valence and arousal were created from facial depictions of four basic emotions.

Method

Participants

Twenty-two adults (12 men and 10 women; M _age 33.3±SD 4.17 years) participated in the study. They were recruited by advertisements placed by an intermediary company, and their occupational backgrounds varied widely. They received a reward for participating in the experiment. All of the participants were native Japanese speakers and had normal or corrected-to-normal vision.

Facial stimuli

The emotional faces were depicted by two models: a man selected from Pictures of Facial Affect (Ekman & Friesen, 1976) and a woman drawn from examples of facial stimuli (Russell, 1997). To create continua that changed according to valence and arousal, we chose expressions of happiness and fear and of anger and disgust as endpoints. These are also prototypical facial expressions that are included in the six basic emotions (Ekman, 1992; Ekman & Friesen, 1971). The disgusted expression portrayed by the woman was originally defined as “sadness” or “fatigue” by Russell (1997). However, this face was identified by Japanese participants as demonstrating “disgust” (Ogawa, Fujimura, & Suzuki, 2005). Therefore, we decided to define the face as “disgust” in this study because all of our participants were native Japanese speakers.

To represent the emotional continua according to valence and arousal, seven morphed images were created for each pair of faces. Each continuum consisted of nine photographs of faces (i.e., the original faces and morphed images; see Figure 1). For example, the happiness–fear continuum included faces blending the two emotions in the following proportions: 100:0 (happiness 100%, fear 0%), 87.5:12.5 (happiness 87.5%, fear 12.5%), 75:25 (happiness 75%, fear 25%), 62.5:37.5 (happiness 62.5%, fear 37.5%), 50:50 (happiness 50%, fear 50%), 37.5:62.5 (happiness 37.5%, fear 62.5%), 25:75 (happiness 25%, fear 75%), 12.5:87.5 (happiness 12.5%, fear 87.5%), and 0:100 (happiness 0%, fear 100%). We defined the facial stimuli using 12.5% increments in the degree of morphing with respect to happiness or anger. The happiness–fear continuum represents change in the pleasantness–unpleasantness continuum, and the anger–disgust continuum represents change in the arousal–sleepiness continuum.

Figure 1.  Facial stimuli used in this experiment. The original faces were drawn from Ekman and Friesen (1976).

Apparatus

Experimental events were controlled by a program written in Inquisit 3.0 (Millisecond) and were implemented on a computer (Vostro 420, Dell) using the Microsoft Windows® operating system, Windows XP^®. Stimuli were presented on a 19-inch LCD monitor (E1902S, Iiyama; 1024×768 pixels, 75 Hz refresh rate) and subtended a visual angle of about 10.0°×7.3°.

Procedure

Participants rated facial stimuli using three types of tasks: the identification task, the ABX discrimination task, and the Affect Grid. No time restrictions were applied. Four training trials were conducted before each task. The order of the three tasks was counterbalanced across participants.

Identification task

Participants were asked to identify a facial expression by choosing between the two emotions on the endpoints of the continuum to which the depiction belonged. For example, participants identified the happiness 87.5% stimulus as either “happiness” or “fear”. Each trial began with a 250 ms presentation of a fixation point, followed by a 250 ms blank screen, and then a 300 ms facial stimulus. After a 250 ms mask consisting of a cluster of asterisks, two emotional words were presented, and participants chose one word by pressing the assigned button. Each face was presented eight times in random order, yielding a total of 288 trials. These trials were divided into four blocks on the basis of two models and the two continua (i.e., happiness–fear and anger–disgust). The order of the four blocks was randomly determined across participants.

ABX discrimination task

The ABX discrimination task required participants to discriminate between faces on a continuum. Each trial began with a fixation point presented for 250 ms, followed by a blank screen lasting 250 ms, and then three successive images of faces. The first (A) and second (B) faces were presented for 300 ms each, and the third (X) face was presented for 1000 ms. The blank interval between A and B was 250 ms, that between B and X was 1000 ms, and that after X was 250 ms. Participants were asked to press a response button to indicate whether X matched A or B.

In each trial, facial stimuli A and B differed by two steps on a continuum, yielding a 25% gap between paired faces (e.g., happiness 100% and happiness 75%), resulting in seven possible pairs on each continuum. The third face, X, was always identical to either A or B. Four presentation orders were possible: (ABX)=(ABA) (ABB) (BAA) (BAB). The same order was presented twice for each pair, yielding a total of 56 trials for each continuum. One block consisted of pairs from one continuum, resulting in a total of four blocks. The order of trials within a block and the blocks themselves were randomised across participants.

Affect Grid

The 9×9 Affect Grid assesses affect along the dimensions of valence and arousal (Russell et al., 1989). Participants were asked to rate the emotion expressed by a face by using a computer mouse to select the appropriate location on a two-dimensional square representing emotional space. Each trial began with a 250 ms fixation point, followed by a 250 ms blank screen, and then a facial stimulus presented for 300 ms. Following a mask of asterisks lasting 250 ms, the Affect Grid was displayed until the participant responded. Each facial stimulus was presented twice in random order, yielding a total of 72 trials that were divided into two blocks based on our models. The order of blocks was counterbalanced across participants.

Results

The upper portion of Figure 2 shows the mean percentages for two identified emotions on each continuum: happiness or fear and anger or disgust. Visual inspection shows that identification rates were nonlinearly distributed, indicating an abrupt category shift. In terms of the happiness–fear continuum, happiness 62.5% or 50% seemed to constitute the category boundary between happiness and fear. Identification rates for the anger–disgust continuum also showed patterns similar to those manifested in response to the happiness–fear continuum.

Figure 2.  (a) The mean identification rates for the happiness–disgust and anger–disgust continua. These rates show the frequency of the identification of happiness or fear and anger or disgust. Labels along the x-axis indicate the percentage of a particular emotion in facial stimuli. For example, “Ha 87.5” means that happiness represents 87.5% of the face and that fear represents 12.5% of the face. (b) The mean of the observed discrimination rates and the predicted data for each continuum. These rates were based on the frequency of correct responses in the ABX discrimination task. The labels show which facial stimuli were paired. For example, “Ha 50–25” indicates a trial in which happiness 50% and happiness 25% were presented as A and B, respectively.

To assess the occurrence of categorical perception, we applied a method used in previous studies (Calder et al., 1996; Young et al., 1997). First, we predicted subjects’ performance in the ABX discrimination task on the basis of the identification and ABX discrimination data. This approach assumes that two factors determine the ability to discriminate between two facial expressions: the physical differences between pairs of facial stimuli irrespective of their expressions, and the contribution of the categorical perception of facial expressions. To estimate the first factor, we employed the mean of the discrimination rates for the pairs at the endpoints of each continuum. Categorical perception did not contribute significantly to these results because these stimuli were near prototypical facial expressions. For the second factor, we calculated the differences between the identification rates for the two relevant stimuli in each pair and multiplied the difference by 0.25 (a constant). By summing the two estimates, we obtained the predicted performance on the discrimination task. If these predicted values correlated with the observed ABX discrimination data, we could conclude that categorical perception occurred within that continuum.

The bottom portion of Figure 2 shows the predictions and the mean actual correct rates for the discrimination task. As the observed and predicted curves seem to fit, correlations between the observed and predicted results for each continuum were significant, happiness–fear: r=.90, t(5)=4.62, p<.01; anger–disgust: r=.85, t(5) = 3.67, p<.01. That is, categorical perception made at least some contribution to responses to the facial stimuli within each continuum. If categorical perception occurs for each continuum, participants should discriminate better between a pair of facial stimuli that cross a categorical boundary than between pairs of facial stimuli within the same category. To confirm this hypothesis, the peak correct rate was contrasted with the mean of the correct rates on all the other pairs. A t-test revealed that the correct rate for the happiness 75% and 50% pair was significantly higher than for other pairs on the happiness–fear continuum, t(21)=3.12, p<.01. On the anger–disgust continuum, performance for anger 75% and 50% was also significantly better than that for all other pairs, t(21) = 4.83, p<.01. These results indicate that happiness 62.5% and anger 62.5% may constitute a category boundary for each continuum.

The mean scores on the Affect Grid are shown in Figure 3a. Facial stimuli within the happiness–fear or the anger–disgust continuum changed in accordance with valence or arousal. Notably, the happiness–fear continuum seemed to show a gap between happiness 62.5% and 50% and happiness 50% and 37.5%, indicating the likelihood of a category boundary. To visualise the distributions of the data on the Affect Grid, frequency histograms for the happiness–fear and the anger–disgust continua, in which each facial stimulus was rated on a grid of a dimension, are shown in Figure 3b. The rating data for each morphing of the stimuli were averaged for each participant. We applied a Gaussian-mixture model (GMM; McLachlan & Basford, 1988), which uses models of probabilities to account for clustering in the distribution of data. The Bayesian information criterion (BIC) was used to evaluate the fitness of a model. Smaller BIC values indicate more appropriate models. We found two normal distributions in the data for the happiness–fear continuum (single Gaussian distribution: BIC=852.23; two Gaussian distributions: BIC=835.37). On the other hand, one normal distribution appeared for the anger–disgust continuum (single Gaussian distribution: BIC=596.73; two Gaussian distributions: BIC=604.06). Valence was divisible into two clusters, whereas arousal was represented on a continuum.

Figure 3.  The results for Experiment 1. (a) The mean ratings on the Affect Grid. (b) Histograms showing how frequently each facial stimulus was assigned on a grid for the valence or the arousal scale. The data were obtained from all participants and were averaged for each type of stimulus across all participants. Solid lines show normal distributions. (c) The mean distances between faces using a two-step method of the valence ratings for the happiness–fear continuum and the arousal ratings for the anger–disgust continuum. The label “100–75” refers to the distance between happiness (or anger) 100% and happiness (or anger) 75%, respectively, on valence or arousal ratings.

To verify the occurrence of categorical perception even in the presence of ratings based on continua in emotional space, we calculated the distance between facial stimuli differing by two steps on each continuum with respect to valence or arousal ratings. Figure 3c shows the mean distance of the pairs of facial stimuli with respect to valence ratings on the happiness–fear continuum and with respect to arousal ratings on the anger–disgust continuum. We conducted an analysis identical to that used in the ABX discrimination task to confirm whether the distance between happiness 67.5% and 37.5% on the valence rating was the largest on the happiness–fear continuum. The peak distance was compared to the mean distances of all the other pairs combined. A t-test showed that the distance between happiness 62.5% and 37.5% was significantly larger than was the average distance across other pairs on the happiness–fear continuum, t(21)=4.69, p<.01. Moreover, we found a significant difference in the distance on the arousal dimension between surprise (excitement) 62.5% and 37.5% and the average distances between all other pairs, surprise–sadness: t(21)=4.17, p<.01; excitement–disgust: t(21)=2.98, p<.01. These findings indicate that the fifty-fifty faces within the surprise–sadness and the excitement–disgust continua were category boundaries on the arousal as well as on the valence dimension.

Discussion

Consistent with previous studies (Calder et al., 1996; Young et al., 1997), we found a category boundary on the happiness–fear and anger–disgust continua in the identification and discrimination tasks. The identification performance on both continua of stimuli showed a rapid shift in the middle of each continuum to the prototypical facial expressions, indicating non-linear change. If psychological ratings reflected the physical features of facial stimuli, identification rates for each facial stimulus would correspond to the intensity of emotion depicted in the facial stimulus. However, we found a steep shift in the identification rates on the happiness–fear and the anger–disgust continua, which likely constituted category boundaries. The discrimination data show that participants discriminated better between a pair of facial stimuli that crossed a category boundary than between those within a category boundary. Consequently, categorical perception contributed to the discrimination of facial expressions over and above the contribution of physical differences. Moreover, significant correlations between the observed and the predicted discrimination rates for both continua emerged. These results provide evidence of the categorical-perception effect.

With respect to the Affect Grid ratings, we found that the happiness–fear continuum was divided into two clusters in terms of valence ratings. The gap between happiness 67.5% and 37.5% was larger than that between other pairs, indicating that happiness 50% was a boundary between pleasantness and unpleasantness. In addition, the distribution of valence ratings on the continuum was divided into two clusters based on the fit of two normal distributions. These results provide evidence that categorical perception occurred in response to the happiness–fear continuum even when using a dimensional strategy. This is the first finding to indicate that categorical perception contributes to ratings of facial expressions even when an explicit dimensional strategy such as the Affect Grid is used. This suggests that the relative dominance of categorical or dimensional perception does not totally depend on the response format (i.e., on a categorical or a dimensional strategy).

Given that categorical perception was totally dominant on the happiness–fear continuum, even with a dimensional strategy, the morphed facial stimuli around the prototypical faces likely merged (i.e., happiness and fear). However, our results also indicate that the facial stimuli were arranged in the correct order, which corresponded to the physical changes in each cluster. Constant physical differences in the facial continuum reflected the intensity of happiness and fear expressed in the facial stimuli. That is, participants may have rated facial stimuli by clustering them into pleasant and unpleasant and then evaluated their emotional intensity with regard to valence. This suggests that categorical perception and dimensional perception co-occur in valence ratings of facial expressions. Consequently, our findings support a hybrid theory that combines dimensional and categorical accounts (Christie & Friedman, 2004; Panayiotou, 2008; Russell, 2003).

However, the anger–disgust continuum showed no boundary across clusters, suggesting that arousal level, in contrast to valence, is continuously represented. There are two possible explanations for the discrepancy between the results for the happiness–fear continuum and those for the anger–disgust continuum. First, happy faces have superiority in identification and discrimination tasks. Of the six basic emotions, happy faces are the most likely to be recognised with considerable accuracy (see Ekman, 1994, for a review). Furthermore, Calvo and Marrero (2009) reported that a happy face had a greater advantage in a visual search task compared with the five other basic emotions. Taken together, these findings suggest that it is possible that the happiness–fear pairs were discriminated better than the anger–disgust pairs due to the superiority of a happy face. Second, the anger–disgust continuum might be insufficient to fully represent the arousal dimension. The results of the Affect Grid indicate that the facial stimuli constituting the anger–disgust continuum were distributed within a relatively narrow range compared with those making up the happiness–fear continuum. Therefore, it is possible that the arousal dimension represented by the anger–disgust continuum was ineffective in eliciting categorical perception in a dimensional strategy. To test this possibility, we used two additional continua that encompassed the arousal dimension better than the anger–disgust continuum.

EXPERIMENT 2

Experiment 2 examined whether categorical perception occurs in the context of a dimensional strategy when rating facial expressions in terms of arousal as well as valence. To encompass a full range of arousal, we created two facial continua. One blended surprised and sad faces that represented extreme high- and low-arousal facial expressions. The other was constructed using excitement and disgust faces that represented high arousal and positive expressions or moderate arousal and negative expressions, which were therefore located in opposite quadrants in the emotional space.

Method

Participants

Twenty-two adults (8 men and 14 women; M _age 29.5±SD 4.81 years) who were staff members at the RIKEN institute participated in the study. All of the participants were native Japanese speakers and had normal or corrected-to-normal vision.

Facial stimuli

We created two facial continua using the same morphing technique as in Experiment 1. One continuum was from the original surprised and sad faces posed by the male model in Experiment 1. The other continuum was prepared by blending the excitement and disgust faces of the female model in Experiment 1. Each continuum consisted of nine photographs of faces, including the two original faces. We defined the facial stimuli using 12.5% increments in the degree of morphing with respect to surprise or excitement. That is, “surprise 62.5%” means that the face consisted of 62.5% surprise elements and 37.5% sadness elements.

Procedure

The apparatus and presentation setting were identical to those used in Experiment 1. Participants were asked to rate the emotions expressed by the faces using the Affect Grid. The timing order of the presentation was the same as in Experiment 1. Each facial stimulus was presented four times in random order, yielding a total of 72 trials that were divided into two blocks.

Results

The mean scores on the Affect Grid for the surprise–sadness and the excitement–disgust continua are shown in Figure 4a. Both facial continua formed clusters at the end of the continuum, providing evidence of categorical perception.

Figure 4a.  The results of Experiment 2. (a) The mean ratings on the Affect Grid.

As for the distributions of the data on the Affect Grid, frequency histograms of valence and arousal ratings for the surprise–sadness and the excitement–disgust continua are shown in Figure 4b. We applied a GMM to estimate clustering in the distribution of data in both the valence and arousal dimensions. For the surprise–sadness continuum, we found two normal distributions for the arousal data (single Gaussian distribution: BIC=853.15; two Gaussian distributions: BIC=813.83), but only one normal distribution for the valence data (single Gaussian distribution: BIC=630.21; two Gaussian distributions: BIC=632.14). For the excitement–disgust continuum, two normal distributions yielded a better fit than a single normal distribution for the valence data (single Gaussian distribution: BIC=942.92; two Gaussian distributions: BIC=815.10), but a single normal distribution was a better fit for the arousal data (single Gaussian distribution: BIC=771.20; two Gaussian distributions: BIC=772.27). The valence rating for the excitement–disgust continuum and the arousal rating for the surprise–sadness continuum were divisible into two clusters.

Figure 4b.  (b) Histograms showing how frequently each facial stimulus was assigned on a grid for the valence and the arousal scale. The upper panels show the distribution of data for the surprise–sadness continuum. The lower panels show the data for the excitement–disgust continuum. Solid lines show normal distributions. (c) The mean distances between two-step faces in terms of the valence and arousal ratings. The label “100–75” refers to the distance between surprise (or excitement) 100% and surprise (or excitement) 75% in valence or arousal ratings.

To assess the occurrence of categorical perception with regard to valence and arousal ratings, we calculated the distance between facial stimuli differing by two steps on each continuum, as in Experiment 1. Figure 4c shows the mean distance of the pairs of facial stimuli within the surprise–sadness continuum and the excitement–disgust continuum. For both continua, the distance between surprise (excitement) 62.5% and 37.5% was larger than that between all other pairs for the valence and arousal ratings. That is, a fifty-fifty face on each continuum was the likely category boundary for the valence and arousal ratings. To confirm whether a category boundary existed for valence and for arousal, the maximum-distance pairs for the valence and the arousal ratings were compared with the mean distances of all the other pairs combined. For the valence ratings, t-tests showed that the distance between surprise 62.5% and 37.5% was significantly larger than was the average distance across other pairs on the surprise–sadness continuum, t(21)=2.30, p<.05. The excitement–disgust continuum yielded identical results to those of the surprise–sadness continuum, t(21)=6.69, p<.01. Moreover, we found a significant difference in the distance on the arousal dimension between surprise (excitement) 62.5% and 37.5% and the average distances between all other pairs, surprise–sadness: t(21)=4.17, p<.01; excitement–disgust: t(21)=2.98, p<.01. These findings indicate that the fifty-fifty faces within the surprise–sadness and the excitement–disgust continua were category boundaries on the arousal as well as on the valence dimension.

Discussion

We found distinct boundaries in both the surprise–sadness and the excitement–disgust continua on the Affect Grid ratings. The results of the analysis of distance on the Affect Grid indicate that the facial stimuli on the continuum could be divided into two clusters on the valence dimension, pleasantness and unpleasantness. For both continua, the facial expression straddling the divide was a fifty-fifty face (i.e., surprise 50% or excitement 50%). Moreover, a category boundary was also demonstrated in the arousal ratings for both continua. The results indicate that categorical perception occurred on the arousal dimension with regard to the surprise–sadness and the excitement–disgust continua. Thus, categorical perception probably contributes to the rating of facial expressions when using a dimensional strategy, not only for valence but also for arousal.

However, although the differences were subtle, the BIC showed that for the valence ratings for the surprise–sadness continuum and for the arousal ratings for the excitement–disgust continuum a single normal distribution yielded a better fit than two normal distributions. That is, these continua did not have two clear clusters as defined by a normal distribution in the valence or arousal ratings. There are some possible reasons for this finding. First, this pattern of results may have been obtained as a result of the range of the data. For the surprise–sadness continuum, the arousal ratings were distributed over a wider range of data than were the valence ratings, whereas for the excitement–disgust continuum, the reverse was true. Given these findings, it is possible that the range of the data with regard to valence for the surprise–sadness continuum and arousal for the excitement–disgust continuum was too narrow to form two normal distributions. Second, the hallmarks of the categorical-perception effect that resulted from analysis using distance on the Affect Grid and GMM were fundamentally different. Analysis using the Affect Grid ratings tested whether a category boundary existed on the continuum by exploring the specific pairs of faces that were farthest apart. On the other hand, GMM assessed whether a single distribution or two normal distributions fitted the distribution of the data for the ratings on the Affect Grid. That is, GMM assumed that all of the data were clusters. Therefore, if the ratings converge at the centre of the distribution of data, corresponding to a category-boundary, a single normal distribution would fit better than two normal distributions. This may explain why the arousal ratings of the excitement–disgust continuum were not divided into two clusters based on GMM. Nevertheless, the fact that the facial continuum could be divided by a category boundary is evidence for the categorical-perception effect (Calder et al., 1996; Etcoff & Magee, 1992; Young et al., 1997). Hence, our results showing that a boundary existed on the Affect Grid ratings are sufficient to indicate that a categorical-perception effect did occur regarding arousal for the excitement–disgust continuum and valence for the surprise–sadness continuum. Furthermore, the continua showing the widest range on each dimension (i.e., the surprise–sadness continuum for arousal and the excitement–disgust continuum for valence in Experiments 2) were separated by category boundaries and exhibited two clusters defined by normal distributions. This is robust evidence that categorical perception emerged for the emotional dimension.

Additionally, facial stimuli were arranged according to their emotional intensity in terms of the valence and arousal dimensions, even within categories. Consequently, categorical perception and dimensional perception co-occurred when facial expressions of emotion were rated on the Affect Grid. This suggests that a hybrid theory that combines categorical and dimensional accounts is applicable to arousal ratings as well as to valence ratings.

GENERAL DISCUSSION

The current study revealed that category perception occurs even when using a dimensional strategy. A category boundary was found within the happiness–fear continuum with respect to the valence dimension (Experiment 1) and within the surprise–sadness and excitement–disgust continuum with regard to both the valence and arousal dimensions (Experiment 2). This suggests that the mode of perception for emotional facial expressions is not limited by a given response format. Categorical perception is not totally subject to a categorical strategy (i.e., identification or discrimination task). Furthermore, the facial stimuli were arranged in sequence according to their physical changes even within a category. This indicates that people can simultaneously use both categorical perception and dimensional perception, suggesting a hybrid theory of emotion (Christie & Friedman, 2004; Panayiotou, 2008; Russell, 2003).

One possible explanation for the categorical-perception effect within a dimensional strategy is the role that language plays when rating facial expressions. Previous research has suggested that verbalisation significantly contributes to the category-perception effect (Roberson et al., 2007, Roberson et al., 2010; Roberson & Davidoff, 2000). In particular, generation of verbal coding in response to a facial expression arose even when participants were not required to label the stimulus (Roberson & Davidoff, 2000). Consistent with this notion, it is possible that verbal labelling (e.g., “happiness” or “fear”) of facial expressions automatically arose and led to categorical perception, although the Affect Grid rating did not explicitly require participants to label the facial expressions.

Additionally, there is evidence for the robustness of the categorical-perception effect within a dimensional strategy. In the current study, all of the participants were native Japanese speakers, who are considered to have more varied interpretations of facial expressions than do Western individuals (Russell, Suzuki, & Ishida, 1993). That is, compared with Western individuals, it is more difficult for Japanese individuals to attribute one label to a specific facial expression. Therefore, it is remarkable that a categorical-perception effect arose for Japanese participants, given that categorical perception partly relies on one-to-one labelling between a face and an emotional word. This finding provides robust evidence for categorical perception when using a dimensional strategy to rate facial expressions of emotion.

In summary, the present study produced two major findings. First, we found a categorical-perception effect even in the context of a dimensional strategy, the Affect Grid. Therefore, the relative dominance of categorical perception and dimensional perception appears not to be totally dependent on the response format. To obtain a category boundary that straddles two emotions in a dimensional strategy, a facial continuum stimulus would need to encompass emotional dimensions such as the excitement–disgust continuum for valence and the surprise–sadness continuum for arousal. Second, we found that categorical perception and dimensional perception co-occurred in ratings of facial expressions. The continuum was divided into two categories with respect to valence and arousal, but each of the morphed images was arranged in the correct order, corresponding to their physical changes. This finding supports a hybrid theory of categorical perception and dimensional perception. In sum, categorical and dimensional accounts of emotion are not fundamentally contradictory but might be complementary. Future research should investigate the interaction between categorical perception and dimensional perception using various methodologies to assess emotional responses and experiences.