Affective feature knowledge interaction for empathetic conversation generation

ABSTRACT

A popular chatbot can generate natural and human-like responses, and the crucial technology is the ability to understand and appreciate the emotions and demands expressed from the perspective of the user. However, some empathetic dialogue generation models only specialise in commonsense and neglect emotion, which can only get a one-sided understanding of the user's situation and makes the model unable to express emotion better. In this paper, we propose a novel affective feature knowledge interactive model named AFKI, to enhance response generation performance, which enriches conversation history to obtain emotional interactive context by leveraging fine-grained emotional features and commonsense knowledge. Furthermore, we utilise an emotional interactive context encoder to learn higher-level affective interaction information and distill the emotional state feature to guide the empathetic response generation. The emotional features are to well capture the subtle differences of the user's emotional expression, and the commonsense knowledge improves the representation of affective information on generated responses. Extensive experiments on the empathetic conversation task demonstrate that our model generates multiple responses with higher emotion accuracy and stronger empathetic ability compared with baseline model approaches for empathetic response generation.

KEYWORDS:

• Emotional features
• emotional lexicon
• commonsense knowledge
• EmpatheticDialogues dataset
• empathetic conversation generation

1. Introduction

Humans have the special ability to convey subtle emotions in a more natural communication way. Most neural conversation models (J. Li, Galley, Brockett, Gao, et al., 2016; Shang et al., 2015; Vinyals & Le, 2015; Wolf et al., 2019) successfully generate fluent and relevant responses grounded in the given user inputs. However, these models only focus on semantic understanding and the quality of generated responses, while ignoring the empathetic ability in conversations. This ability is called emotional transference in psychology. Empathetic responding refers to the natural and human-like utterances generated by the system through understanding and appreciating both the emotions and demands expressed by them from the perspective of users. In other words, the most critical communication skill of the empathetic conversation system is to recognise the user's emotions and understand the user's feelings, and then generate appropriate responses related to the user's emotions through emotional associations. We show a conversation from the EmpatheticDialogues dataset in Table 1.

Table 1. A conversation from the EmpatheticDialogues dataset, where the speaker stands for user and the listener stands for the system mentioned in this paper.

Emotion: Jealous

Conversation:

Speaker: My sister is always the sibling that my parents talk about. Sometimes i wonder if they even notice me.

Listener: I am sorry to hear that. Have you ever talked to your parents about that?

Speaker: I haven't. I am scared that it would just make the situation worse. Do you think i should tell them how i feel?

Target: Yes, you do. Otherwise, you are going to carry that pain the rest of your life.

A grouping body of studies has shown that empathetic conversations can improve users' satisfaction and availability in lots of fields (Brave et al., 2005; Wright & McCarthy, 2008; L. Zhou et al., 2020). For example, K. K. Liu and Picard (2005) denotes that embedding empathy into human-computer interactive systems, namely conscious empathy systems, plays a crucial role in building a good user experience. Recent research (Ma et al., 2020) has been attracted to how to give the ability of empathetic responses to the conversation system. However, numerous studies (Hasegawa et al., 2013; Ruan & Ling, 2021; Song et al., 2019; H. Zhou et al., 2018; X. Zhou & Wang, 2018) are based on an externally given emotion label to generate the response with a specific emotion. These methods generate emotional responses to some extent, but they are not conducive to improving the emotional content quality of generated responses and are not necessarily empathetic to the user either. Meanwhile, some recent works (Lin et al., 2019; Lin, Xu, et al., 2020; Majumder et al., 2020; Rashkin et al., 2019) have focussed on improving the ability of conversation models to understand the emotions expressed by the user, which makes models somewhat more empathetic. Therefore, one of the challenges faced by open-domain conversation systems is how to better understand the user's emotions and make the corresponding empathetic responses without specifying emotion labels.

We mainly focus on the basic research of the empathetic conversation generation model in open-domain dialogue systems. A conscious empathetic conversation system built by embedding empathy into human-computer interaction systems can not only improve user satisfaction but also play a crucial role in natural language processing. Therefore, how to give the ability of empathetic responses to the conversation system is the focus of our research. However, there are still some issues in most existing research: despite conversation generation models have made great progress in semantic understanding (Lin et al., 2019; Rashkin et al., 2019) and response generation (Majumder et al., 2020; Welivita & Pu, 2020), there are still great challenges in improving the quality of empathetic response generation by leveraging both emotional features and commonsense knowledge. In this paper, the purpose of training our model is to generate empathetic responses similar to the user's emotions by extracting the emotional state feature from the context that combines emotional features and commonsense knowledge.

Toward this end, we propose a new framework for conversation generation of affective feature knowledge interactive model, named AFKI, which is a Transformer-based empathetic response generation model and leverages multi-head attention to learn higher-level affective interaction information. AFKI model is composed of three components. (1) The emotional interactive context is obtained from the interaction of emotional features, conversation history, and commonsense knowledge, which is the expression of conversation history enriched by affective information and enhances the system's understanding of emotional knowledge in semantic information. (2) The emotional interactive context encoder is used to encode the interactive context to obtain the vector representation of the emotional interactive context information, and we extract the emotional state feature to guide the empathetic response generation. (3) The emotional response decoder can effectively inject the emotional state feature into the model training to generate appropriate empathetic responses. In this paper, the emotional state feature is injected into each decoding layer so that the multi-attention mechanism can effectively capture the affective information and enhance the system's empathy expression in response generation.

The main contributions in this work are summarised as:

1. We extract the contextual emotional features and retrieve the external concepts, which can help the system well capture the subtle differences in the user's emotional expression and improve the expression of affective information in the conversation history.

2. We obtain the vector representation of emotional interactive context information utilising the proposed emotional interactive context encoder and extract the emotional state feature to guide the target response generation.

3. Extensive experimental evaluations containing automatic and human evaluations indicate that AFKI is superior to several competing baseline models based on the EmpatheticDialogues dataset.

The remainder of this paper is arranged as follows: the related work is described in Section 2, Section 3 describes our proposed model in detail, Section 4 presents the experiments, and our paper is summarised in Section 5.

2. Related work

In this section, we introduce the related work of diverse response generation, emotional response generation, and empathetic response generation in open-domain conversation systems.

2.1. Diverse response generation

Open-domain conversation systems (Serban et al., 2015, 2016; Shang et al., 2015; Vinyals & Le, 2015; Weizenbaum, 1966; Wolf et al., 2019) have been extensively studied. The researchers noted that responses generated by the encoder-decoder models tended to be generic and repetitive, like “I don't know”. Some recent work (J. Li, Galley, Brockett, Gao, et al., 2016; J. Li, Galley, Brockett, Spithourakis, et al., 2016; Lin, Winata, et al., 2020; Serban et al., 2017; Zhang et al., 2018; T. Zhao et al., 2017) utilised different means including modifying the learning objective, introducing additional information, or integrating random potential variables to improve the diversity of generated responses. The enhancement of diversity can enrich the generated responses. Due to the lack of emotion, the responses of the system can only confirm to user's idea, but cannot give an affirmation to the user and fail to express the desire to accompany the user.

2.2. Emotional response generation

Human beings are rich in emotions, and all kinds of emotions show up in conversations. To address the issue that open-domain conversation systems cannot control the emotional expression of the generated responses, the emotional conversation models (Hasegawa et al., 2013; Ruan & Ling, 2021; Shen & Feng, 2020; Song et al., 2019; Zhong et al., 2021; H. Zhou et al., 2018; X. Zhou & Wang, 2018) which can improve the users' satisfaction and experimental results came into being X. Zhou & Wang (2018) constructed a large-scale annotated emotion dataset using Twitter data labelled with emojis, then adopted several neural models and generated emotional responses grounded in arbitrarily specific emotions. To express a designated emotion naturally in the target response, H. Zhou et al. proposed a sequence-to-sequence language model with an internal memory and an external memory to generate emotional sentences according to a given sentence and affective categories. Song et al. utilised lexicon-based attention to extend the sequence-to-sequence model and guided the target response generation process by applying an emotion classifier. Zhong et al. leveraged commonsense and emotion to generate more accurate and commonsense-aware responses with desired emotions. However, most parts of emotional conversation models produce specific emotional responses conditioned on designated emotion labels. Although these generated responses are emotional, they still cannot produce natural and meaningful responses through real emotions. In other words, these responses are not necessarily empathetic.

2.3. Empathetic response generation

Early studies on emotional response generation were limited to sentiment labels annotated by a small number of humans. Hence, X. Zhou & Wang (2018) introduced a large-scale labelled Twitter dataset naturally annotated with emojis. Rashkin et al.(2019) presented a novel benchmark for generating empathetic conversations. EmpatheticDialogues is a new publicly available dialogue dataset based on specific emotional situations with 25k conversations, where the speaker feels a given emotion label and the listener responds to the speaker. The development of empathetic conversations benefits not only from conversation corpora with emotion labels but also from empathetic conversation models. Previous research on emotional conversation generation is devoted to generating specific emotional responses according to designated affective categories. In recent years, more and more researchers have paid increasing attention to the field of empathetic conversation generation (Casas et al., 2021; Gao et al., 2021; Q. Li, Chen, et al., 2020; Lin et al., 2019; Lin, Xu, et al., 2020; Majumder et al., 2020; Rashkin et al., 2019; Zandie & Mahoor, 2020; Zeng et al., 2021). Rashkin et al. promoted empathetic response generation by constructing new models from existing models and generated more empathetic responses by identifying the users' emotions. The model designed by Lin et al. first identified the user's emotions and then fused the emotion distribution information from multiple decoders to generate an empathetic response. Majumder et al. (2020) introduced the method of emotion grouping and emotion mimicry to produce more empathetic responses, in which emotion mimicry was used to mimic the user's emotion. Q. Li, Li et al. (2020) facilitated empathetic response generation by enriching conversation history with multi-type knowledge. Recently, Zeng et al. (2021) designed an effective method named affective decoding and added the auxiliary dual emotion encoder for more empathetic responses.

3. The proposed model

In this section, we present the proposed AFKI model, which is built on the standard Transformer (Vaswani et al., 2017), as shown in Figure 1. According to most of the previous work on neural response generation, we leverage an encoder-decoder architecture to construct the AFKI model. The input of the encoder is the emotional interactive context, which is connected by the emotional features and knowledge context graph. Among them, emotional features are retrieved from conversation history, and the knowledge context graph G is composed of conversation history and external concepts obtained from the commonsense knowledge base ConceptNet. The attentional mechanism selectively weights different parts of the sentence during decoding, so that the decoder is used to decode the emotional features and the knowledge context graph differently.

Figure 1. In our proposed framework, emotional feature extraction is first carried out for the given conversation history to obtain the emotional feature matrix. Then, we extract the commonsense knowledge using the commonsense knowledge base ConceptNet and obtain the emotional interactive context by interacting with the emotional features, commonsense knowledge, and conversation history. Finally, the emotional state feature extracted from the emotional interactive context is used to guide the model to produce more empathetic responses.

3.1. Task definition

Given a conversation history $\mathcal{U}=\left[u_1,\dots ,u_a\right]$ consisting of a set of utterances, the ith utterance is made up of $W_i$ tokens, i.e. $u_i=\left[x_1^i,\dots ,x_{W_i}^i\right]$, where a and $W_i$ are the number of utterances in the conversation history and the number of tokens in the ith utterance, respectively. The ath and bth utterances belong to the user and the system, respectively, where a represents the even number and b represents the odd number. Following the MoEL (Lin et al., 2019), the utterances in $\mathcal{U}$ are flattened into a single token sequence $\mathcal{U}=\left[x_1,\dots ,x_b\right]$. Besides, we add a token before the sequence to get $\mathcal{U}=\left[CLS,x_1,\dots ,x_b\right]$. The emotional words $\mathcal{E}=\left[e_1,\dots ,e_o\right]$ are extracted from conversation history with the emotion lexicon NRC_VAD (Mohammad, 2018), where o is the number of emotional words selected by sorting emotional words in descending order. We also add a emotion token before the sequence to get $\mathcal{E}=\left[LAB,e_1,\dots ,e_o\right]$. Our task can be expressed as follows: given a conversation history $\mathcal{U}$, the general knowledge base ConceptNet (Speer et al., 2017), the emotion lexicon NRC_VAD and the emotional words $\mathcal{E}$, the system generates a natural and emotionally appropriate empathetic response.

3.2. Knowledge context graph

Inspired by Q. Li, Li et al. (2020), we utilise the emotion lexicon NRC_VAD and the general knowledge base ConceptNet as knowledge data sources in the proposed AFKI model to enrich conversation history.

NRC_VAD is an emotion lexicon that is a list of over 20k English words and their corresponding VAD scores. It gives human ratings of English words for valence (a.k.a V), arousal (a.k.a A), and dominance (a.k.a D). The affective intensity $\gamma \left(x_i\right)$ of words $x_i$ is calculated by using VAD values as shown below: (1) $\gamma \left(x_i\right)=min-max\left({\Vert V\left(x_i\right)-1/2,A\left(x_i\right)/2\Vert }_2\right),$(1) where $V\left(x_i\right)\in \left[0,1\right]$, $A\left(x_i\right)\in \left[0,1\right]$, $\Vert \cdot {\Vert }_m$ represents $l_m$ norm, $V\left(x_i\right)$ and $A\left(x_i\right)$ are the valence value and arousal value of word $x_i$ in the VAD vector, respectively. If $x_i$ is not in NRC_VAD, we set both $V\left(x_i\right)$ and $A\left(x_i\right)$ to 0.5.

ConceptNet is an open large-scale knowledge graph representing relations between words. Edges in a knowledge graph (D. Zhao et al., 2022) are knowledge units that connect one vertex to another. In our knowledge context graph G, we set edges to 1. We succinctly denote an assertion as a triple of its start vertex, a relation label, and an end vertex, i.e. “a monkey has a mouth” can be expressed as (monkey, HasA, mouth). Each (start concept, relation, end concept) triple denotes an assertion, and each assertion has a confidence score. Therefore, we represent each triplet and its corresponding confidence as a tuple $\psi =\left(r,c,s\right)$ of non-stop word $x_i$. ConceptNet contains 5.9 million assertions, 3.1 million concepts, and 38 relations in English. Usually, the confidence score s of each assertion belongs to $\left[1,10\right]$. We scale s between 0 and 1 by using min-max normalisation: (2) $min-max\left(s\right)=\frac{s-s_{min}}{s_{max}-s_{min}},$(2) where $s_{min}=1$, $s_{max}=10$.

For the token sequence $\mathcal{U}=\left[x_1,\dots ,x_b\right]$, we retrieve a set of tuples of non-stopword token $x_i$ from ConceptNet: $\left(r_1,c_1,s_1\right)$, $\left(r_2,c_2,s_2\right)$,…, $\left(r_{p_i},c_{p_i},s_{p_i}\right)$, where $r_{p_i}$ is the relation between token $x_i$ and concept $c_{p_i}$, $c_{p_i}$ is the $p_i$th concept connected with token $x_i\in \mathcal{U}$, $s_{p_i}$ is the confidence of the $p_i$th concept, respectively. To better enrich the conversation history with external knowledge, we perform the following operations on knowledge: First, the concepts and candidate tuples whose confidence values $s_{p_i}$ are below the minimum threshold α are filtered. Second, we perform relational filtering to filter out the tuples that belong to the list $l_{ur}$ of unrelated relations. Third, the affective intensity $\gamma \left(c_{p_i}\right)$ of the concept $c_{p_i}$ and cosine similarity $\cos \left(x_i,c_{p_i}\right)$ between token $x_i$ and concept $c_{p_i}$ are calculated, and we finally sum them with $s_{p_i}$ to obtain the combined score $g\left(x_i\right)$ of each candidate tuple: (3) $g\left(x_i\right)=\gamma \left(c_{p_i}\right)+\cos \left(x_i,c_{p_i}\right)+s_{p_i}.$(3) The tuples are sorted in descending order of combined scores $g\left(x_i\right)$, and the top $p^{\mathrm{\prime }}$ candidate tuples $\{\left(r_{p_i},c_{p_i},s_{p_i}\right){\}}_{p_i=1,\dots ,p^{\mathrm{\prime }}}$ are taken as the emotional knowledge of per token $x_i$. Then the knowledge retrieved from the emotion lexicon NRC_VAD and the knowledge base ConceptNet is applied to enrich conversation history. Finally, vertices $\mathcal{V}=\left[v_1,\dots ,v_m\right]$ in the enriched conversation history consist of a CLS token, the words $x_i\in \mathcal{U}$, and concepts $c_{p_i}$ of the words $x_i\in \mathcal{U}$.

3.3. Encoder

Our encoder is built on L identical and independently parameterised Transformer layer blocks, where L is the number of blocks. Each Transformer encoder block is composed of a graph attention mechanism, a multi-head self-attention mechanism, and a fully connected feed-forward network model.

In a multi-turn conversation, it is essential to ensure that the model can distinguish vertices in different utterances. Therefore, we incorporate a conversation state embedding $E^v$ for each vertex. And we employ three different ways of embedding in the vector representation of vertex $v_i$. The vertex $v_i$ is finally represented as: (4) $v_i=E^w\left(v_i\right)+E^p\left(v_i\right)+E^v\left(v_i\right),$(4) where $E^w\in {\mathbb{R}}^d$, $E^p\in {\mathbb{R}}^d$, $E^v\in {\mathbb{R}}^d$, d is the embedding dimension. $E^w$, $E^p$, and $E^v$ denote the word embedding, the position embedding, and the conversation state embedding, respectively.

Next, each vertex $v_i$ contextualises itself by focussing on other neighbouring vertices connected to it in the knowledge context graph. We employ the N-headed graph attention mechanism and obtain the new representation vector of each vertex $v_i$ with the local context to model the vertex interactions. $a^n$ denotes a self-attention mechanism module of the nth head, as shown below: (5) $\begin{array}{rl}{a}^n\left(q_i,k_j\right)& =\frac{\exp \left({\left(W_k^n{k}_j\right)}^{\mathrm{T}}{W}_q^n{q}_i\right)}{\sum _{j^{\mathrm{\prime }}\in N_i}\left({\left(W_k^n{k}_{j^{\mathrm{\prime }}}\right)}^{\mathrm{T}}{W}_q^n{q}_i\right)},\end{array}$(5) (6) $\begin{array}{rl}{\alpha }_{ij}^n& =a^n\left(v_i,v_j\right),\end{array}$(6) (7) $\begin{array}{rl}{\hat{v}}_i& =v_i+\underset{n=1}{\overset{N}{\parallel }}\sum _{j\in N_i}{\alpha }_{ij}^n{W}_v^n{v}_j,\end{array}$(7) where $\mathcal{V}=\left[{\hat{v}}_1,\dots ,{\hat{v}}_m\right]$. N denotes the number of self-attention heads, ∥ denotes the concatenation of N heads, $N_i$ represents the neighbourhood of $v_i$ in the adjacency matrix $N$, $v_j$ represents the neighbouring vertex of $v_i$, $W_q^n\in {\mathbb{R}}^{d^{\mathrm{\prime }}\times d^{\mathrm{\prime }}}$, $W_k^n\in {\mathbb{R}}^{d^{\mathrm{\prime }}\times d^{\mathrm{\prime }}}$, $d^{\mathrm{\prime }}=d/N$.

To obtain the emotional interactive context and make the generated responses more appropriate and empathetic, we introduce emotional words into the enriched conversation history. The emotion lexicon NRC_VAD is utilised to extract emotional words from conversation history, and the emotional features are ranked in descending order according to affective intensity values. Then, we select top-o words as the emotional words for each conversation history and add an emotion token to infer the emotional state of the conversation context, i.e. $\mathcal{E}=\left[e_0,e_1,\dots ,e_o\right]$, where $e_0=LAB$. Finally, we employ three different ways of embedding in the vector representation of the word $e_i$. The word $e_i$ is finally represented as: (8) $e_i=E^w\left(e_i\right)+E^p\left(e_i\right)+E^e\left(e_i\right),$(8) where $E^e\in {\mathbb{R}}^d$ denotes the conversation state embedding. We concatenate the emotional word vector $\mathcal{E}$ and the vertex representation vector $\mathcal{V}$ obtained previously as the emotional interactive context vector: (9) $C=\left[\mathcal{V};\mathcal{E}\right],$(9) where $C=\left[{\hat{r}}_1,\dots ,{\hat{r}}_t\right]$, $[;]$ denotes the concatenation operator, t is the total number of vertices and emotional words. The remaining operators are to utilise the transformer layer to optimise all the tokens $C=\left[{\hat{r}}_1,\dots ,{\hat{r}}_t\right]$, which will serve as input to the transformer model shown in Figure 2. We apply the residual connection (He et al., 2016) around each multi-head attention mechanism and feed-forward network, followed by the layer normalisation (Ba et al., 2016): (10) $\begin{array}{rl}{\tilde{r}}_i& =LayerNorm\left(r_i^{\mathrm{\prime }}+FFN\left(r_i^{\mathrm{\prime }}\right)\right),\end{array}$(10) (11) $\begin{array}{rl}{r}_i^{\mathrm{\prime }}& =LayerNorm\left({\hat{r}}_i+MHAtt\left({\hat{r}}_i\right)\right),\end{array}$(11) where $C=\left[{\tilde{r}}_1,\dots ,{\tilde{r}}_t\right]$. MHAtt is the multi-head attention with N attention heads, LayerNorm is the layer normalisation, and FFN is a position-wise and simple fully connected feed-forward network model. Each layer of the encoder contains a fully connected feed-forward network model consisting of two linear transformation layers with an activation function ReLu in the middle.

Figure 2. In our Transformer encoder, the obtained emotional interactive context vector is input into the encoder for encoding.

To facilitate the generation of empathetic responses, the emotional state feature is extracted from the vector representations of the fusion of knowledge context graph G and emotional words. We first compute the affective intensity value ${\gamma }_i$ of ${\tilde{r}}_i$ in C according to Equation (1(1) $\gamma \left(x_i\right)=min-max\left({\Vert V\left(x_i\right)-1/2,A\left(x_i\right)/2\Vert }_2\right),$(1) ) and take the weighted sum of all the vector representations that are outputs of the encoder to obtain the emotionally augmented context representation $c_e$. We then apply a linear transformation to $c_e$ to obtain the emotional state feature $e_p$ that will be input into the decoder to facilitate the generation of empathetic responses, and the representation $e_p$ is fed into the emotional category distribution $P_e$ to promote the generation of empathetic responses: (12) $\begin{array}{rl}{c}_e& =\sum _{i=1}^t\frac{\exp \left({\gamma }_i\right)}{\sum _{j=1}^t\exp \left({\gamma }_j\right)}{\tilde{r}}_i,\end{array}$(12) (13) $\begin{array}{rl}{e}_p& =W_e{c}_e+b_e,\end{array}$(13) (14) $\begin{array}{rl}{P}_e\left(e|C\right)& =softmax\left(e_p\right),\end{array}$(14) where $W_e\in {\mathbb{R}}^{q\times d}$ and $b_e\in {\mathbb{R}}^q$ denote the learnable weights and the learnable bias for the linear transformation layer, respectively. q is the number of affective categories, and d is the embedding dimension.

3.4. Decoder

Our decoder is built on L independently parameterised Transformer layer blocks, where L is the number of blocks. Each Transformer decoder block consists of a multi-head self-attention mechanism, a multi-head attention mechanism, and a fully connected feed-forward network model. The middle layer of the decoder performs multi-head attention by attending to the output of the encoder stack.

The vector representation $Y=\left[y_0,\dots ,y_{j-1}\right]$ is the connection between emotion vector $e_p^{\mathrm{\prime }}\in {\mathbb{R}}^{1\times d}$ and the embeddings of the previously generated $j-1$ words $\left[y_1,\dots ,y_{j-1}\right]$, where $y_0=e_p^{\mathrm{\prime }}$, emotion vector $e_p^{\mathrm{\prime }}\in {\mathbb{R}}^{1\times d}$ is transformed from $e_p\in {\mathbb{R}}^{1\times q}$ by using a linear transformation. The vector representation Y is fed into the decoder.

The decoder of the proposed model is grounded in the Transformer layers. The emotional words, conversation history, and external concepts are concatenated with special tokens as inputs to the decoder. We adopt the multi-headed attention mechanism with N attention heads to calculate the cross sentiment context vector $c_a$ from the fusion of emotional features and knowledge context graph G. $c_a$ is calculated using the multi-headed attention contextualised by $y_{j-1}$: (15) $\begin{array}{rl}{c}_a& =\underset{n=1}{\overset{N}{\parallel }}\sum _{i=1}^t{\alpha }_i^n{W}_G^n{\tilde{r}}_i,\end{array}$(15) (16) $\begin{array}{rl}{\alpha }_i^n& =a^n\left(y_{j-1},{\tilde{r}}_i\right),\end{array}$(16) where N is the number of self-attention heads, ∥ denotes the concatenation of N heads, $y_{j-1}$ is the projection of the outputs of the previous decoder layer, and ${\tilde{r}}_i$ is the projection of the outputs of the interactive Transformer encoder. $a^n$ denotes the self-attention module of the nth head as shown below: (17) $a^n\left(y_{j-1},{\tilde{r}}_i\right)=\frac{\exp \left({\left(W_{\tilde{r}}^n{\tilde{r}}_i\right)}^T{W}_h^n{y}_{j-1}\right)}{\sum _{\omega =1}^t\left({\left(W_{\tilde{r}}^n{\tilde{r}}_{\omega }\right)}^T{W}_h^n{y}_{j-1}\right)}.$(17) We construct the final emotional context vector by concatenating the cross sentiment context vector $c_a$ with the emotion vector $c_e$ to enhance the empathetic ability of the target response. Each layer of the decoder contains a fully connected feed-forward network model FFN consisting of two linear transformation layers with a ReLu activation function in the middle: (18) $\begin{array}{rl}B& =Y+W_m\left[c_a;c_e\right]\end{array}$(18) (19) $\begin{array}{rl}\hat{B}& =LayerNorm\left(B\right),\end{array}$(19) (20) $\begin{array}{rl}\hat{Y}& =LayerNorm\left(\hat{B}+FFN\left(\hat{B}\right)\right),\end{array}$(20) where $[;]$ denotes the concatenation operator and $\hat{Y}=\left[{\hat{y}}_1,\dots ,{\hat{y}}_j\right]$.

3.5. Response generator

The decoder generates tokens one by one that makes up the target response $\hat{Y}=\left[{\hat{y}}_1,\dots ,{\hat{y}}_j\right]$. We utilise the obtained sequence of tokens as the context to predict the next token. The softmax function is used as a readout to calculate the vocabulary probability distribution ${\alpha }^G$ of the next token $y_j$ generated by the response generator: (21) ${\alpha }^G\left(y_j|y_{<j},C\right)=softmax\left(W_o{\hat{y}}_j\right).$(21) Gu et al. introduced the copying mechanism and proposed CopyNet, which can not only generate words regularly but also copy appropriate segments of the input sequence. Later, See et al. (2017) proposed a hybrid pointer-generator network, which is similar to CopyNet. By using external concepts and emotional words to generate empathetic responses, we calculate the probability $P_G$ of copying a word from vertices $\left[{\tilde{r}}_1,\dots ,{\tilde{r}}_t\right]$ using the sigmoid function in a way similar to See et al. (2017) and calculate the final probability distribution $P\left(y_j\right)$ of eventually predicting the next token. Next, the standard maximum likelihood estimator (MLE) is used as the loss function ${\mathcal{L}}_G$ to optimise the response prediction, and ${\mathcal{L}}_G$ is computed as follows: (22) $\begin{array}{rl}{P}_G& =sigmoid\left(W_G{\hat{y}}_j+b_G\right),\end{array}$(22) (23) $\begin{array}{rl}P\left(y_j\right)& =P_G\ast {\alpha }^G+\left(1-P_G\right)\ast {\alpha }^c,\end{array}$(23) (24) $\begin{array}{rl}{\mathcal{L}}_G& =-\log P\left(y_j|y_{<j},C\right),\end{array}$(24) where vector $W_G$ and scalar $b_G$ are learnable parameters. $P_G$ is used to generate a word from the vocabulary which is sampled from ${\alpha }^G$, and $1-P_G$ is used to copy a word from the knowledge-enriched conversation context by sampling from the attention probability distribution ${\alpha }^c$. ${\alpha }^c$ is the concatenation of attention scores, i.e. ${\alpha }_i^c=\alpha \left({\hat{y}}_j,{\tilde{r}}_i\right),{\tilde{r}}_i\in C$. During training, we adopt the negative log-likelihood (NLL) loss between the golden label $e^{\ast }$ of the conversation history and the emotional category distribution $P_e$ to optimise the weights: (25) ${\mathcal{L}}_M=-\log \left(P_e\left(e^{\ast }|C\right)\right).$(25) Finally, we jointly optimise the two objectives of A prediction and B prediction. The total loss function ${\mathcal{L}}_{all}$ is: (26) ${\mathcal{L}}_{all}=\alpha {\mathcal{L}}_M+\text{β}{\mathcal{L}}_G,$(26) where α and β are hyper-parameters that we utilise to balance the two losses. During our training, we set both α and β to 1.

4. Experiments

In this part, we mainly introduce the public dataset, evaluation metrics, baseline models, results and analysis, and settings, which are related to the following experiments.

4.1. Dataset

We evaluate our model on the first large-scale multi-turn empathetic dialogue dataset, namely EmpatheticDialogues (Rashkin et al., 2019). Table 2 presents the statistics of the EmpatheticDialogues dataset used in the experiments. This dataset is obtained from the Amazon Mechanical Turk platform, containing 25k one-to-one open-domain conversations. The dataset provides 32 emotion labels, which are evenly distributed. As shown in Table 3, we divide 32 emotions into two categories: positive emotions and negative emotions.

Table 2. The statistics of the EmpatheticDialogues dataset.

Dataset	Train	Validation	Test	Labels
EmpatheticDialogues	40,250	5,734	5,255	32

Table 3. Two categories of 32 emotion labels.

Positive Labels	Negative Labels
surprised, hopeful, excited,	annoyed, terrified, lonely, angry,
impressed, confident, proud,	nostalgic, furious, apprehensive,
prepared, faithful, content,	afraid, embarrassed, devastated,
grateful, trusting, caring,	anxious, ashamed, disappointed,
joyful, anticipating	sentimental, disgusted, jealous,
	guilty, sad

4.2. Evaluation metrics

In terms of the evaluation metrics, we utilise automatic and human evaluations to evaluate the models on the EmpatheticDialogues dataset.

4.2.1. Automatic metrics

We evaluate all models using two automatic metrics. Perplexity (Serban et al., 2015): Perplexity indicates the inverse likelihood of predicting the responses on the test set. Accuracy: The emotion classification accuracy of the responses measured by our emotion classifier is used to evaluate the model at the emotional level.

4.2.2. Human evaluations

We also conduct three human evaluation metrics rated on a Likert scale (1: not at all, 3: somewhat, 5: very much) (Alexandrov, 2010; Ji et al. 2022; Rashkin et al. 2019) to evaluate the models. Specifically, we randomly sample 112 instances and ask five participants to compare responses generated by our model and baseline models with those ground truth responses to judge their Empathy, Relevance, and Fluency. The scores of 112 samples and five participants are averaged to get the final human ratings. Empathy: The replies express whether the system understands or cares about the feelings expressed by the user. Relevance: The responses are whether consistent with the conversation and on-topic. Fluency: You are whether understand the generated responses, and the responses are whether accurate.

4.3. Baselines

To conduct a comprehensive performance evaluation, we compare our model over the following typical baselines.

• Transformer (Vaswani et al., 2017): is trained by using a Maximum Likelihood Estimation (MLE) loss.

• EmoPrepend-1 (Rashkin et al., 2019): is a Transformer-based model with an external supervised classifier. It leverages the top-1 predicted emotion label trained from the emotion classifier as a prepended token which is added to the beginning of the token sequence as the input of the encoder.

• MoEL (Lin et al., 2019): consists of an emotion tracker, emotion-aware listeners, and a meta listener. The model uses a decoder for each emotion, referred to as the listener, optimised to respond to one type of emotion. The meta listener gently combines the output representations of these decoders and produces a response.

• EmpDG (Q. Li, Chen, et al., 2020): is an extension of the Transformer model, which utilises an empathetic generator and interactive discriminators to enhance the performance of empathetic response generation. Interaction discriminators consist of a semantic discriminator and an emotional discriminator. The semantic discriminator ensures that the generated responses are contextual, while the emotional discriminator ensures that the generated responses are empathetic.

• MK-EDG (Q. Li, Li, et al., 2020): is another extension of the Transformer model, which incorporates an emotional context graph, a multi-type knowledge-aware context encoder, and a multi-type knowledge-aware response generator. The framework exploits multiple types of knowledge to understand and express appropriate emotions in the generation of empathetic conversation.

4.4. Results and analysis

In this section, we discuss the automatic evaluation results, human evaluation results, and case study respectively.

4.4.1. Automatic evaluation results

We present the evaluation results of the automatic evaluation indicators for the baseline models and our AFKI model in Table 4. The EmpDG model has the lowest accuracy among the baseline models, while it obtains slightly lower perplexity than Transformer. The Emoprepend-1 model and MoEL model achieve slightly better accuracy, however their perplexity increases. Compared with other baseline models, the accuracy of the MK-EDG model increases slightly and perplexity decreases slightly, suggesting that commonsense knowledge is helpful in improving emotion accuracy and reducing perplexity. Compared with all the baseline models, our AFKI model scores the highest in the evaluation results of emotional accuracy, and the perplexity is equivalent to that of the best baseline model, which illustrates that our model is effective in empathy expression.

Table 4. Results of automatic evaluations and human evaluations between multiple models on EmpatheticDialogues dataset.

Methods	Accuracy	Perplexity	Empathy	Relevance	Fluency
Transformer (Vaswani et al., 2017)	–	37.73	3.16	3.48	3.65
EmoPrepend-1 (Rashkin et al., 2019)	34.52	38.30	3.21	3.50	3.73
MoEL (Lin et al., 2019)	35.24	38.04	3.36	3.84	3.67
EmpDG (Q. Li, Chen, et al., 2020)	30.53	37.08	3.39	3.83	3.64
MK-EDG (Q. Li, Li, et al., 2020)	37.52	36.77	3.44	3.87	3.66
AFKI	39.13	36.82	3.48	3.93	3.68

4.4.2. Human evaluation results

We list the evaluation results of human evaluations, including Empathy, Relevance, and Fluency, in Table 4. According to the human evaluation results, we observe that the proposed AFKI model significantly outperforms baseline models in terms of empathy and relevance, indicating that both emotional features and commonsense knowledge contribute to the system's understanding of implicit emotions and the generation of more empathetic responses. Furthermore, it is easily observed from Table 4 that the fluency results of the response generated by all models remain basically stable despite the change of models, which can be explained by the excellent fluency of the Transformer model. Both the baseline models and our model are based on the Transformer, thus they have a steady fluency.

4.4.3. Case study

To intuitively display the expression of emotion in the generated responses, we provide three samples generated by the baseline models and our model in Table 5. In the first sample, our model can express empathy similar to that of the user based on the user's utterance. The generated utterance not only expresses the same joy when the user gets the job he wants but also expresses the system's blessing that he hopes the user will get a raise soon. Our model produces richer, more empathetic, and more consistent content than other baseline models. For the second example, the responses generated by our model are coherent and more contextually consistent, whereas the responses generated by the baselines are less relevant, that is, the baselines fail to generate responses that are more context-appropriate based on semantic content. For the third sample, our model can follow the emotional context and advise the owner to fix the cat, whereas the baseline models can express similar empathy but produce meaningless responses. These samples indicate that the AFKI model is superior to the baselines in the emotional and content balance of context.

Table 5. Examples generated by the baselines and the proposed AFKI model.

Emotion: Furious

Conversation:

U: I was given the opportunity to work at a new job that i was hoping to get! I never thought it would be possible!

Transformer: That is awesome! What kind of job is it?

EmoPrepend-1: That is awesome! I hope you find a new job!

MoEL: That is awesome! I am sure you will have a new job!

EmpDG: That is great! I am so sorry! I am so sorry! I am sure you will get it!

MK-EDG: What kind of job did you get?

AFKI: That is great. I hope you get a raise too.

Target: Congratulations! I am sure you earned it!

Emotion: Embarrassed

Conversation:

U: I hate when i fart in public it is so embarrassing.

Transformer: Oh no, i am so sorry to hear that.

EmoPrepend-1: I am sorry to hear that. I am sure it was a good time.

MoEL: Oh no! I hate when that happens. I do not know what they are doing in the end.

EmpDG: Oh no! I am so sorry! I would have been a pain!

MK-EDG: I am sure it will be fine.

AFKI: I am sure it was not a good thing to do.

Target: You are unable to hold them in?

Emotion: Furious

Conversation:

U: A while back my cat knocked over and broke my mothers urn.

S: Oh i am so sorry. Cats can be very mischievous sometimes.

U: I was so mad for the longest time but i have gotten around to not being so made about it.

Transformer: I am sorry to hear that. I hope you find a cat, and you can get a cat…

EmoPrepend-1: I am sorry to hear that. I hope you find a cat back to the doctor.

MoEL: I am sorry to hear that. I hope you have a cat who did not get to get a cat.

MK-EDG: I am sorry to hear that. I hope you can find a pet to be a little bit.

AFKI: I am sorry to hear that. I hope you have a cat and you can get it fixed.

Target: Yea i feel you. At the end of the day you can’t blame the cat. It didn't do this on purpose.

Note: U: denotes the speaker. S: denotes the listener.

4.5. Settings

We train all models in PyTorch (Paszke et al., 2017) with a batch size of 16 and initialise the word embedding using 300-dimensional pre-trained Glove vectors (Pennington et al., 2014). The hidden size for all respective components is set as 300. The maximum values of external concepts introduced into each conversation and each token are set to 10 and 5, respectively. All models are implemented on a single GPU of NVIDIA-SMI 430.40 and CUDA Version 10.1. Early stopping is a technique applied when training.

5. Conclusion

In this paper, through studying the problem of empathetic response generation, we propose a novel affective feature knowledge interactive model named AFKI, which can integrate emotional features and commonsense knowledge into the conversation context. At the same time, the emotional interactive context encoder is utilised to learn an emotional interactive context and distill the emotional state feature to guide the empathetic response generation. Experimental results on automatic evaluation and human evaluation tasks demonstrate that our work outperforms the baselines in Accuracy, Empathy, and Relevance. Our proposed approach can improve the empathy ability of chatbots. Under the background of the application of deep learning technology (Huan et al., 2022; X. Liu & Qi, 2021; Wei et al., 2022; Zhang et al., 2021), the proposed model can be applied in the field of customer service to improve user satisfaction and relieve the service pressure of human customer service. However, the current method still has limitations, such as the introduction of emotional features is not conducive to improving the diversity of empathetic response generation. In future work, we will leverage maximum mutual information as the objective loss function and integrate random potential variables to promote diversity in generated responses.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

This work was supported by the grants from National Natural Science Foundation of China (Project No. 62076092) and Special Project of Foshan Science and Technology Innovation Team (Project No. FS0AA-KJ919-4402-0069).