A proposed role for glucocorticoids in mediating dopamine-dependent cue-reward learning

Abstract

Learning to respond appropriately to one’s surrounding environment is fundamental to survival. Importantly, however, individuals vary in how they respond to cues in the environment and this variation may be a key determinant of psychopathology. The ability of seemingly neutral cues to promote maladaptive behavior is a hallmark of several psychiatric disorders including, substance use disorder, post-traumatic stress disorder, eating disorders and obsessive-compulsive disorder. Thus, it is important to uncover the neural mechanisms by which such cues are able to attain inordinate control and promote psychopathological behavior. Here, we suggest that glucocorticoids play a critical role in this process. Glucocorticoids are primarily recognized as the main hormone secreted in response to stress but are known to exert their effects across the body and the brain, and to affect learning and memory, cognition and reward-related behaviors, among other things. Here we speculate that glucocorticoids act to facilitate a dopamine-dependent form of cue-reward learning that appears to be relevant to a number of psychiatric conditions. Specifically, we propose to utilize the sign-tracker/goal-tracker animal model as a means to capture individual variation in stimulus-reward learning and to isolate the role of glucocorticoid-dopamine interactions in mediating these individual differences. It is hoped that this framework will lead to the discovery of novel mechanisms that contribute to complex neuropsychiatric disorders and their comorbidity.

Keywords:

• Glucocorticoids
• reward
• Pavlovian conditioning
• incentive salience
• dopamine
• individual differences

Introduction

The ability to adapt and respond appropriately to the surrounding environment increases one’s chance of survival. Indeed, one must learn to identify cues that are predictive of positive (e.g. food) or negative (e.g. predator) events, as such cues will thereby signal the appropriate response of approach or avoidance, respectively. For some individuals, however, encountering particular cues in the environment can trigger complex emotional responses that, in turn, promote maladaptive behavior and unwanted consequences. For example, when individuals with substance use disorder encounter people, places, or paraphernalia previously associated with their drug-taking experience, the consequence is often relapse. Aberrant responses to environmental stimuli are, in fact, characteristic of a number of psychiatric disorders, including post-traumatic stress disorder (PTSD; VanElzakker et al., 2014), pathological gambling (Hellberg et al., 2019), eating disorders (Belfort-DeAguiar & Seo, 2018; Berridge et al., 2010), schizophrenia (Howes & Nour, 2016; Kapur, 2003), and obsessive-compulsive disorder (Aouizerate et al., 2004; Apergis-Schoute et al., 2017). Yet, it remains to be determined whether there is a common psychological and neurobiological process subserving each of these disorders. Here we postulate that glucocorticoid function plays a critical role in mediating individual differences in response to environmental stimuli and thereby may represent a common denominator of cue-driven psychopathologies.

Glucocorticoids (GCs), although classically recognized as the “stress” hormone (Henry, 1992), play a wide-ranging role in adaptive physiology and behavior (Myers et al., 2014). Beyond the stress response, glucocorticoid function has been implicated in metabolism (Rose et al., 2010; Vegiopoulos & Herzig, 2007), inflammation (De Bosscher & Haegeman, 2009; Scheschowitsch et al., 2017), immune response (Cain & Cidlowski, 2017), development (Busada & Cidlowski, 2017; De Kloet et al., 1988), memory (Roozendaal, 2000; Sandi & Pinelo-Nava, 2007), cognition (De Kloet et al., 1999; Sandi, 2013), and reward-processing (Leal & Moreira, 1997; Piazza et al., 1989). It is not surprising, therefore, that GC function has been recognized in the pathophysiology and neurobiology of disease and affective disorders for more than 70 years (see McEwen & Akil, 2020; Selye, 1955). Hyper- and hypo-concentrations of GCs, as well as altered GC receptor expression levels, have been reported in individuals with depression, schizophrenia, and PTSD (Akil, 2005; Muck-Seler et al., 1999; Szeszko et al., 2018). Further, individuals with excessive glucocorticoid secretion (e.g. Cushing’s syndrome) (Sonino et al., 2010), or those prescribed GCs (e.g. for treating inflammation), often report depressed mood (Marques et al., 2009). While there is an abundance of data in support of a relationship between glucocorticoid function, emotional state and maladaptive behavior, the point of intersection remains to be determined. Below, we will briefly summarize the role of glucocorticoids in the stress response, learning and memory, and reward, before presenting a framework that may enable us to elucidate the role of glucocorticoids in cue-driven psychopathologies and a potential mechanism underlying this relationship.

Glucocorticoid function and the stress response

Historically, the hypothalamic-pituitary-adrenal (HPA) axis has been viewed as the primary biological system activated by stress (e.g. Dallman & Jones, 1973). Upon perceiving a stressor (see Figure 1), neural signals are elicited throughout the brain, including the prefrontal cortex, hippocampus, amygdala and brainstem (for review see Herman & Cullinan, 1997). This information converges at the paraventricular nucleus (PVN) of the hypothalamus via direct projections or indirectly through the bed nucleus of the stria terminalis (BNST) and neighboring hypothalamic nuclei (for review see Herman et al., 2003). The HPA axis is thereby activated by secretion of corticotropin-releasing hormone (CRH) and arginine vasopressin (AVP) from the PVN. Adrenocorticotropic hormone (ACTH) is subsequently released from the pituitary, and ultimately, GCs are synthesized and released from the cortex of the adrenal glands (for review see Herman et al., 2016; Spencer & Deak, 2017). Even in the absence of stress, GCs—cortisol in humans and corticosterone in rodents—circulate both peripherally and centrally in fluctuating concentrations that follow an ultra-radian and circadian rhythm (Kalsbeek et al., 2012; Lightman & Conway-Campbell, 2010; Qian et al., 2012; Sarabdjitsingh et al., 2010; Spiga et al., 2014). Approximately 85% of circulating GCs are inactive and bound to a glycoprotein, corticosteroid-binding globulin (CBG) (for review see Moisan et al., 2014). At baseline, ∼5% of GCs are free to enter the brain, however, under circadian peak or stress levels this percentage is increased (McEwen et al., 1968; Qian et al., 2012).

Figure 1. Overlapping neural circuits mediating HPA-axis “stress responsivity” and individual differences in cue-motivated behaviors. The key elements mediating the hypothalamic-pituitary-adrenal (HPA)-axis “stress response” are depicted (in black), and start with the conversion of neural information from the prefrontal cortex (PFC), hippocampus (Hipp), amygdala (Amy), and brainstem at the paraventricular nucleus (PVN) of the hypothalamus. Information is relayed directly to the PVN or indirectly via the bed nucleus of the stria terminalis (BNST). The HPA axis is triggered with the release of corticotropin releasing hormone (CRH) from the PVN. Consequently, ACTH secretion from the pituitary is elicited; and, subsequently, the synthesis and secretion of glucocorticoids from the adrenal cortex occurs. Glucocorticoids are diffused across the body and brain. At baseline levels, they exert their effects upon Type I-MRs (in green) located primarily within the hippocampus (Hipp) and less so within the PFC, Amy, and paraventricular nucleus of the thalamus (PVT). Under stress or circadian peak, they activate Type II-GRs (in yellow) located ubiquitously across the pituitary, PVN, PVT, Amy, PFC, Hipp, lateral septum (LS), nucleus accumbens (NAc), ventral tegmental area (VTA), nucleus tractus solitarus (NTS), locus coeruleus (LC), and cerebellum (Cb). (Ahima & Harlan, 1990; Chao et al., 1989; Fuxe et al., 1985; 1985; Jaferi & Bhatnagar, 2006; Reul & de Kloet, 1985). Structures that comprise the “motive circuit” (Kalivas & Volkow, 2005; Kelley et al., 2005) and have been implicated in individual differences in cue-learning (Flagel & Robinson, 2017) are shown in light beige. This includes: medial PFC(mPFC) (Campus et al., 2019; Haight et al., 2017), orbitofrontal cortex (OFC) (Stringfield et al., 2017), Hipp (Fitzpatrick et al., 2016), NAc (Saunders & Robinson, 2012), ventral pallidum (VP) (Ahrens et al., 2016), Amy (Flagel et al., 2011), lateral hypothalamus (LH) (Haight et al., 2017), PVT (Campus et al., 2019), and VTA (Flagel et al., 2011).

Glucocorticoids are lipophilic and bind to two types of receptors that act as ligand-gated transcription factors. Upon activation, type I-mineralocorticoid receptors (MRs) and type II-glucocorticoid receptors (GRs) (for review see Evans & Arriza, 1989) are translocated from the cytoplasm to the nucleus (Madan & DeFranco, 1993). The effects of GR and MR activation can be fast and non-genomic, or slow, genomic, and long lasting (Groeneweg et al., 2011; Scheschowitsch et al., 2017). MRs are considered the low capacity, high-affinity receptors and are activated and occupied by baseline levels of GCs; whereas GRs are considered the high capacity, low-affinity receptors and are activated by circadian peak and stress levels of GCs (Herman et al., 1989; Reul et al., 1987b; Reul & de Kloet, 1985). MRs are mainly concentrated in the hippocampus, amygdala, prefrontal cortex (PFC) and paraventricular nucleus of the thalamus (PVT) (see Figure 1; Chao et al., 1989; Jaferi & Bhatnagar, 2006; Reul & de Kloet, 1985). In contrast, GRs are expressed in most cells throughout the body (for review see De Kloet et al., 2000) and ubiquitously expressed throughout the brain including, the lateral septum, hippocampus, nucleus tractus solitarius, amygdala, hypothalamus, locus coeruleus, cerebellum, PFC, ventral tegmental area (VTA), PVT, and striatum (see Figure 1; Ahima & Harlan, 1990; Chao et al., 1989; Fuxe et al., 1985; Fuxe et al., 1985; Jaferi & Bhatnagar, 2006; Reul & de Kloet, 1985).

One of the major roles of GC-GR interaction is to protect us from further stress via a negative feedback loop (Akil, 2005; De Kloet & Reul, 1987; Herman & Mueller, 2006; Reul et al., 1987a; Reul & de Kloet, 1985). GR activation within the hippocampus (Jacobson & Sapolsky, 1991) or HPA-axis prevents GCs from rising to levels that are threatening to homeostasis by inhibiting further synthesis and secretion of CRH and ACTH (for review see Gjerstad et al., 2018; Herman et al., 2012). Glucocorticoid regulation of the HPA axis, therefore, is critical for survival, as it promotes adaptive physiological responses and governs our ability to cope with stress. As outlined below, however, GC-GR interaction extends beyond maintenance of the HPA axis and has been implicated in a number of complex behaviors of relevance to neuropsychiatric disorders.

The role of glucocorticoids in learning and memory

Acquiring salient information (learning) and recalling it (memory) are fundamental processes underlying an individual’s response to the surrounding environment. The relationship between glucocorticoid signaling and learning and memory has been a primary focus of stress neurobiology research for decades (Lupien & McEwen, 1997). This work has demonstrated that glucocorticoids can act to either improve or impair learning and memory, with the effects dependent, at least in part, on the level of circulating glucocorticoids (McEwen & Sapolsky, 1995). For example, corticosterone alters long-term potentiation (LTP) in hippocampal neurons (Bennett et al., 1991; Dubrovsky et al., 1987), but does so in an inverted-U shaped manner (Diamond et al., 1992; Pavlides et al., 1993). That is, low concentrations of GCs increase LTP, while high concentrations decrease LTP. It is not surprising, therefore, that an increase in the induction of LTP is mediated by MR activation, whereas a decrease is mediated by GR activation (Conrad et al., 1999; Pavlides et al., 1995). As expanded upon below, dose, timing, and receptor specificity are all critical factors mediating glucocorticoid effects on learning and memory, and the impact of each is dependent on the behavior being assessed.

Spatial learning and memory

Much of the work examining the effects of glucocorticoids on learning and memory has centered around spatial tasks such as the Morris Water Maze, which is known to rely on the hippocampus (e.g. Morris et al., 1990). For example, administration of corticosterone immediately following each training session of acquisition improves long-term retrieval on the Morris Water Maze task, but in an experience-dependent manner (Sandi et al., 1997). That is, experimentally-induced endogenous corticosterone levels were identified as a critical factor determining the cognitive consequences (Sandi et al., 1997). On the other hand, chronic exposure (3 months) to corticosterone (via pellet implants) prior to training impairs learning on a Morris Water Maze task, and similar impairments were evident following pre-exposure to chronic social stress (Bodnoff et al., 1995). Given the density of glucocorticoid receptors in the hippocampus, it is not surprising that spatial learning is also affected by blocking central MRs and GRs via intracerebroventricular administration of selective antagonists (Oitzl et al., 1998; Oitzl & de Kloet, 1992). The behavioral effects reported in these studies suggest a differential role for MRs vs. GRs in spatial learning and memory, with MRs mediating spatial learning strategies and GRs playing a critical role in the consolidation of spatial information (Oitzl et al., 1998; Oitzl & de Kloet, 1992). Importantly, as with corticosterone administration, GR blockade has been shown to both improve and impair cognitive function, with effects largely dependent on the treatment regimen and timing of administration (e.g. Oitzl et al., 1998). Together, these studies highlight an important role for GC function in spatial learning and memory, and reveal the complexities of studying such a ubiquitous and wide-ranging physiological system.

Stimulus-response learning

In addition to spatial learning, GCs have also been implicated in stimulus-response learning (e.g. Atsak et al., 2016; Schwabe et al., 2010). On the circular hole board (CHB) task rodents are trained to locate an open hole or exit using either distant cues, which are reflective of spatial learning, or proximal cues, which signify stimulus-response learning. When rodents are stressed or administered corticosterone prior to a test of the “preferred” learning strategy (i.e. spatial vs stimulus-response), they rely mainly on stimulus-response learning (Schwabe et al., 2010). Further, blocking MRs prevents stimulus-response learning and rescues spatial learning strategies, but not performance (Schwabe et al., 2010). Importantly, unlike hippocampal-dependent spatial learning, stimulus-response learning relies on the dorsal striatum (Packard & McGaugh, 1996; Packard & Wingard, 2004). Thus, these findings highlight a role for GCs in learning and memory in brain regions outside of the hippocampus.

Consistent with the rodent work described above, humans also exhibit a strategy-shift from spatial to stimulus-response learning following stress (Schwabe et al., 2008) or the administration of glucocorticoids (Guenzel et al., 2014; Schwabe et al., 2009). Specifically, individuals with a self-reported history of “high” chronic stress are more likely to exhibit a stimulus-response learning strategy relative to those with a history of “low” chronic stress (Schwabe et al., 2008). The same patterns follow exposure to an acute psychosocial stressor, and individuals who exhibit a stress-induced stimulus-response strategy tend to show greater cortisol levels during the spatial learning task (Schwabe et al., 2007). Further, administration of hydrocorticosterone prior to a stimulus-response task has been shown to enhance performance (Guenzel et al., 2014). Together, both animal and human data suggest that GCs influence learning strategies, and do so in a brain region-specific manner (e.g. Iaria et al., 2003).

Pavlovian conditioning

Glucocorticoids also play a role in Pavlovian associative learning. Most research in this regard has centered around fear conditioning (for review see Kim & Jung, 2006), which often consists of a single session in which a tone (conditioned stimulus, CS) predicts the delivery of a footshock (unconditioned stimulus, US). Following associative learning, the tone-CS comes to elicit the conditioned response (CR) of freezing behavior (Fanselow, 1980). This conditioning experience elicits a rise in corticosterone (Marchand et al., 2007). Under these experimental conditions, a rise in corticosterone is expected, as acute stressors including, footshock, restraint stress, and forced swim test are known to increase corticosterone in rodents (e.g. Hueston et al., 2011). Importantly, however, the rise in corticosterone is greater in rodents that received paired presentation of the cue (CS) and footshock (US), relative to those that received random presentation of the CS and US (e.g. Marchand et al., 2007). Consistent with these findings, exposure to stress will facilitate the CS-US association (Shors et al., 1992). For example, exposure to acute stress (i.e. tail shock) prior to a conditioning session results in an increase in the magnitude of a CS (white noise)-elicited eye-blinking CR, and this effect appears to be corticosterone-dependent (Beylin & Shors, 2003; Shors, 2001). That is, the effect of stress is no longer apparent following adrenalectomy (Beylin & Shors, 2003). Interestingly, administration of corticosterone after a conditioning session will enhance the CS-elicited CR observed on subsequent sessions (Lesuis et al., 2018; Zorawski & Killcross, 2002). GC function, therefore, seems to be involved in both establishing a relationship between discrete environmental cues (CS) and aversive stimuli (US), and in expressing the corresponding behavioral response (CR).

Although less research has investigated a potential role for GCs in appetitive Pavlovian conditioning, there is some supporting evidence (Tomie et al., 2002, 2004). Following a Pavlovian conditioning session consisting of a lever-CS paired with food-US, Tomie and colleagues (Tomie et al., 2002, 2004) demonstrated an increase in corticosterone levels, which was apparent following either the 1st or 20th conditioning session. Similar to the data described above for fear conditioning, the rise in corticosterone was greater in rats that received paired presentations of the lever-CS and food-US, relative to those that received random presentations. GC function, therefore appears to play a role in both aversive and appetitive Pavlovian conditioning. The exact mechanism remains to be determined, but it is presumed that both overlapping and distinct processes are involved in the GC-mediated effects on aversive vs. appetitive conditioning. Below we will discuss one possible mechanism in the context of appetitive Pavlovian conditioning.

The role of glucocorticoids in reward

Although we lack a specific understanding of the role of GCs in appetitive Pavlovian conditioning, there is a large body of literature in support of a role for GCs in reward-related behaviors. Much like stress, exposure to rewards such as food, sex, and drugs elicit a rise in corticosterone (for review see Piazza & Le Moal, 1997). In fact, it has been suggested that corticosterone itself is reinforcing, as some rats willingly self-administer stress levels of corticosterone in a dose-dependent manner (Deroche et al., 1993; Piazza et al., 1993). Further, both stress (e.g. restraint, footshock, tailshock, isolation) (Abercrombie et al., 1989; Hall et al., 1998; Kalivas & Duffy, 1995; Puglisi-Allegra et al., 1991) and administration of corticosterone (Imperato et al., 1989; Piazza et al., 1996) have been reported to increase dopamine within the nucleus accumbens (NAc), a primary locus for reward-processing; and corticosterone dose-dependently affects the reinforcing properties of drugs of abuse (Deroche et al., 1997) as well as the locomotor response to drugs (i.e. sensitization) (Cador et al., 1993; Marinelli, Rouge-Pont, Deroche et al., 1997; Marinelli et al., 1997). Together, these findings formed the foundation for the “pathophysiological chain” framework which was put forth by Piazza and LeMoal more than 20 years ago to explain individual differences in drug-taking behaviors of relevance to substance use disorder (Piazza & Le Moal, 1996; 1998). At baseline, corticosterone and dopamine concentrations are typically low, as is the propensity to self-administer drugs. Under stress, however, the pathophysiological chain is triggered, and increased levels of corticosterone and dopamine are thought to interact in the NAc and, in turn, affect the reinforcing effects of drugs of abuse. Importantly, within this framework, individual differences are accounted for, as the initial rise in corticosterone or heightened sensitivity to the effects of this hormone could be inherently present in some individuals or induced by stress in others (Piazza & Le Moal, 1996). Indeed, this framework was based in large part on an animal model that captured such individual differences, as described briefly below.

Individual differences

Like humans, only some rats readily self-administer drugs of abuse, and Piazza and colleagues showed that this tendency to take drugs could be predicted by individual differences in response to a novel environment (Piazza et al., 1989). That is, high-responder (HR) rats that exhibit a greater locomotor response to a novel environment have an increased propensity to acquire drug self-administration relative to low-responder (LR) rats that exhibit low levels of novelty-induced locomotion (Piazza et al., 1990; 2000). These distinct patterns of drug self-administration between HRs and LRs are apparent across drug classes, including psychostimulants (Marinelli & White, 2000; Piazza et al., 1989), nicotine (Suto et al., 2001), morphine (Ambrosio et al., 1995) and ethanol (Nadal et al., 2002). Thus, these individual differences do not appear to be a function of drug-induced effects directly on the dopamine system. Rather, the behavioral characteristics of high- and low-responder rats appear to be driven largely by GC function. Relative to LRs, HRs exhibit a greater and more prolonged corticosterone response to a novel environment (Piazza et al., 1990; 1991). This corticosterone profile has been attributed to lower GR mRNA expression in the hippocampus, or a faulty negative feedback system in HR rats (Kabbaj et al., 2000). Further, experimenter-administered corticosterone increases drug self-administration of LRs, the less vulnerable or “resistant” phenotype; but decreases drug self-administration of HRs, the more vulnerable phenotype (Piazza et al., 1991). These phenotype-dependent effects of corticosterone on drug-taking behavior have been attributed to inherent differences in circulating levels of corticosterone which, in turn, were postulated to interact with the dopamine system (Piazza et al., 1991).

Differences in DA transmission do indeed accompany the behavioral phenotypes of HRs and LRs. Relative to LRs, HRs have higher stress- and drug-induced DA within the nucleus accumbens, and, importantly, these differences are dependent on corticosterone, as they are not apparent following adrenalectomy (Rouge-Pont et al., 1998). In fact, adrenalectomy decreases nucleus accumbens DA levels in HRs such that they are indistinguishable from LRs. Beyond the nucleus accumbens, HRs and LRs also differ in stress-induced dopamine-metabolite/dopamine ratios in the dorsal striatum and prefrontal cortex (Piazza et al., 1991). Together, these data support an integrative role for glucocorticoid function and dopamine in mediating individual differences in reward-related behaviors.

Potential mechanisms for glucocorticoid-dopamine interactions

The mechanism by which corticosterone and dopamine interact to affect behavior remains to be determined. There are, however, a few possibilities that can be considered in this regard (see Figure 2; Mukhara et al., 2018; Piazza & Le Moal, 1996). For one, the positive correlative relationship between glucocorticoids and dopamine may be mediated via tyrosine hydroxylase (TH), the rate limiting enzyme of dopamine production. In support, exposure to stress or exogenous GCs have been reported to increase TH throughout the mesocorticolimbic dopamine system, including the VTA (Ortiz et al., 1995) and NAc (Hall et al., 1998). Alternatively, glucocorticoids may act via the DA-degrading enzyme, monoamine oxidase (MAO), such that the lower the levels of MAO, the greater amount of available dopamine. In agreement, following administration of exogenous glucocorticoids, MAO is inhibited (Veals et al., 1977); and following adrenalectomy, MAO levels are increased (Caesar et al., 1970). Dopamine availability might also be increased via GC effects on pre-synaptic uptake. Indeed, systemic administration of GCs has been shown to decrease DA uptake by DA synaptosomes (Gilad et al., 1987) and cells within the NAc of rats (Wheeler et al., 2017). In relation, stress causes a decrease in the expression of the dopamine transporter (DAT) in the NAc (Meaney et al., 2002). More recent studies, however, suggest that the GC-mediated decrease in dopamine clearance is dependent on a different uptake protein, the organic cation transporter 3 (Graf et al., 2013). As stress and GCs are known to alter excitatory synapses via GR activation, this has been suggested as another potential mechanism within the VTA by which dopamine activity could be potentiated (Polter & Kauer, 2014; Saal et al., 2003; Stelly et al., 2016). Further, the absence of GRs within DA-receptive medium spiny neurons results in a decrease in neural activity within DA-inervated brain regions, such as the nucleus accumbens, and consequentially decreases the reinforcing properties of rewards (Ambroggi et al., 2009; Barik et al., 2010; Parnaudeau et al., 2014). Importantly, such ablation of GR in dopaminoceptive neurons does not affect anxiety-related behaviors (Ambroggi et al., 2009).

Figure 2. A proposed role for glucocorticoid-dopamine interactions in appetitive conditioning. The integrative response of dopamine (DA) and corticosterone (CORT) under appetitive Pavlovian conditioning is depicted. With repeated pairings of the 8-second presention of a lever-conditioned stimulus (CS) followed by a food-unconditioned stimulus (US), CORT (pink circles) increases both peripherally and centrally, while DA (blue circles) increases within the nucleus accumbens (NAc, orange structure). Corticosterone may potentiate dopamine release by acting within the ventral tegmental area (VTA, maroon structure) to: (1) increase tyrosine hydroxylase (TH), the rate limiting enzyme of DA biosynthesis, or (2) decrease monoamine oxidase (MAO), one of the degradation enzymes of DA. Alternatively, CORT may act presynaptically to mediate DA clearance and/or synaptic uptake by: (3) inhibiting the organic cation transporter (OCT3, red structure) or (4) the dopamine transporter. Finally, CORT may increase DA transmission (5) via glutamatergic synapses or (6) by acting directly upon glucocorticoid receptors (GR) within DA-receptive neurons.

Ultimately, the relationship between glucocorticoids and dopamine remains complex. Understanding the psychological and neurobiological circumstances in which these two systems interact to regulate motivated behavior will be critical. The availability of techniques capable of unmasking points of intersection with higher specificity, such as region-specific GR receptor manipulations (e.g. Scheimann et al., 2019), will advance our understanding. Further, the proper use of animal models capable of capturing functional relevance (see Bale et al., 2019), is crucial. Here we propose the use of the sign-tracker/goal-tracker animal model to elucidate the relationship between glucocorticoids and dopamine in cue-motivated behaviors of relevance to psychopathology.

The role of glucocorticoids in mediating individual differences in the propensity to attribute incentive salience to reward cues

Daily, individuals are exposed to an overwhelming amount of environmental cues (e.g. smell, sound, sight) that are associated with prior experiences; for example, a captivating restaurant sign (cue) associated with a delicious meal (experience). Such cues can have great influence over behavior. In a state of hunger, the restaurant sign may quickly grab your attention and prompt you to stop and eat your favorite meal. It is through Pavlovian conditioning that one learns to associate such cues with biologically relevant stimuli (e.g. food, sex, danger) (Rescorla, 1988). Once learned, these cues become predictors of positive or negative stimuli and elicit an adaptive response. Such cues, however, can also be attributed with incentive motivational value (i.e. incentive salience) (Robinson & Berridge, 1993) and thereby transformed into incentive stimuli that promote maladaptive patterns of behavior. For example, for some individuals, the same restaurant sign may become a “motivational magnet” that urges them to stop and eat even in a state of satiety. Indeed, when attributed with excessive incentive motivational value, environmental stimuli attain inordinate control over behavior and can trigger psychopathological behavior. As indicated above, when individuals with substance use disorder encounter cues previously associated with their drug-taking experience, they often relapse despite knowing the adverse consequences. This transformation of drug-associated cues into powerful incentive stimuli is thought to be one cause of relapse (Berridge & Robinson, 2016; Saunders & Robinson, 2013).

Until recently, predictive and incentive learning appeared to be intertwined. However, the sign-tracker (ST)/goal-tracker (GT) animal model has shown that this is not the case, as this model allows us to parse the psychological and neurobiological processes involved in each form of learning (Robinson & Flagel, 2009). The ST/GT model consists of exposing rats to a Pavlovian conditioning paradigm, in which a lever-CS precedes impending food-US delivery (see Figure 2). With training, different conditioned responses (CR) emerge – goal-tracking and sign-tracking. Goal-tracking behavior is directed toward the location of reward delivery upon cue presentation. In contrast, sign-tracking behavior is directed toward the cue itself. Some rats, intermediate responders (IRs), vacillate between the two CRs. Both STs and GTs attribute predictive value to the cue, but only for STs does the cue also become attributed with incentive value. Relative to GTs, STs also show greater impulsivity (Lovic et al., 2011), enhanced fear response to aversive cues (Morrow et al., 2011; 2015), attentional deficits (Paolone et al., 2013), increased sensitized response to cocaine (Flagel et al., 2008), and greater cue- and drug-induced reinstatement of drug-seeking behavior (Saunders & Robinson, 2010, 2011). Thus, although characterized based on Pavlovian conditioned approach behavior, the ST/GT animal model seems to capture an endophenotype of relevance to a number of psychopathologies, including PTSD (Morrow et al., 2011; 2015), impulse control disorders (Lovic et al., 2011) and substance use disorder (Saunders & Robinson, 2010, 2011).

Neural circuitry underlying individual differences in the attribution of incentive salience to reward cues

The ST/GT model has revealed different neural circuits underlying predictive vs. incentive stimulus-reward learning (for review see Flagel & Robinson, 2017; Kuhn et al., 2018 ). We know that sign-tracking behavior is DA-dependent (Flagel et al., 2011; Saunders & Robinson, 2012) and relies heavily on subcortical “bottom-up” circuits (Flagel et al., 2011; Haight et al., 2017; Yager et al., 2015). In contrast, goal-tracking behavior is thought to rely on cortical “top-down” mechanisms (Campus et al., 2019; Sarter & Phillips, 2018). The brain regions involved in mediating individual differences in stimulus-reward learning (see Figure 1) include the prelimbic cortex (PrL) (Campus et al., 2019; Haight et al., 2017), orbitofrontal cortex (OFC) (Stringfield et al., 2017), ventral hippocampus (vHPC) (Fitzpatrick et al., 2016), lateral habenula (LHb) (Flagel et al., 2011), dorsal striatum (DS) (DiFeliceantonio & Berridge, 2016; Yager et al., 2015), NAc (Saunders & Robinson, 2012), paraventricular nucleus of the thalamus (PVT) (Campus et al., 2019; Haight et al., 2017), basal lateral amygdala (BLA) (Yager et al., 2015), medial nucleus of amydgala (MeA) (Flagel et al., 2011), ventral pallidum (VP) (Ahrens et al., 2016), lateral hypothalamus (LH) (Haight et al., 2017), and VTA (Flagel et al., 2011). These brain regions largely comprise the classic “motive circuit” (Kalivas & Volkow, 2005). Importantly, however, the ST/GT animal model has shown that this circuitry is, for the most part, activated only when a reward cue is attributed with incentive value (Flagel et al., 2011). That is, this system is engaged to a greater extent in STs relative to GTs (Flagel et al., 2011). Of particular importance here is the extensive overlap between the structures that comprise the “motive circuit”, or those that encode the incentive value of reward cues, and GC function, as highlighted in Figure 1.

The effects of stress on the propensity to attribute incentive salience to reward cues

Given the overlapping brain structures and circuits (Figure 1), it is perhaps not surprising that stress has been shown to impact the propensity to attribute incentive salience to reward cues (i.e. sign-tracking). Specifically, it appears that early life stress sets the stage for enhanced sign-tracking behavior in adulthood. For example, inadequate early-life social experience, consisting of artificial-rearing (i.e. deprived of mother and litter) from postnatal day (PND) 5, increases sign-tracking behavior in adulthood (PND 90–160) (Lomanowska et al., 2011). Further, early life adversity, consisting of isolation, forced swim, restraint, and exposure to predator odor from PND 21–35, potentiates sign-tracking behavior in adulthood (PND 60+) (Hynes et al., 2018). The mechanisms by which early life stress affects the propensity to attribute incentive salience to reward cues remains to be determined. It is likely, however, that the impact of early life stress on brain development and connectivity in adulthood (Chen & Baram, 2016) renders one more likely to sign-track. Indeed, such experiences are known to alter stress responsivity and response to drugs of abuse in adulthood via alterations in brain function (Ladd et al., 2004; Sinha, 2001). Specifically, exposure to stress early in life has been reported to alter hippocampal glucocorticoid receptor (Type I and II) expression in adulthood and increase stress-induced corticosterone response (Brunton & Russell, 2010; C. Henry et al., 1994; Ladd et al., 2004). Early life stress also results in a decrease in dopamine transporter expression and an increase in stress-induced dopamine activity (Meaney et al., 2002). We speculate, therefore, that the impact of early life stress on GC function and dopamine activity may alter the way in which these two systems interact and, in turn, promote DA-dependent stimulus-response learning, or sign-tracking behavior.

Importantly, exposure to stress later in life, seems to attenuate the attribution of incentive salience to reward cues. A single prolonged stressor, consisting of restraint, forced swim, and ether exposure in early adulthood, attenuates the acquisition of sign-tracking behavior (Fitzpatrick et al., 2019). These same rats show an attenuation of stimulated dopamine release within the nucleus accumbens. Thus, while the dopamine system appears to be a primary point of intersection for stress-induced effects on sign-tracking behavior, such effects are dependent on the type of stress and timing of exposure, with differential impacts early vs. later in life.

The role of glucocorticoids on the propensity to attribute incentive salience to reward cues

Although limited, there are also data to support a specific role for GC function in incentive learning processes. We have shown that a single session of Pavlovian conditioning—consisting of lever-CS presentation paired with food-US delivery—elicits a rise in corticosterone in all rats, but to a greater extent in those that become sign-trackers (Flagel et al., 2009). Thus, individual differences in corticosterone response are apparent even before a sign-tracking or goal-tracking conditioned response is established. Further, systemic administration of a GR antagonist decreases sign-tracking behavior in Japanese quail (Rice et al., 2018) and does so in a dose-dependent manner (Rice et al., 2019). Together, these data highlight a potential role for glucocorticoids in determining individual differences in the propensity to attribute incentive salience to reward cues.

A proposed integrative role for glucocorticoid-dopamine interactions in determining individual differences in the propensity to attribute incentive salience to reward cues

We currently lack in our understanding of the relationship between corticosterone and dopamine in STs vs GTs. Based on the data outlined above, we postulate that glucocorticoids enhance dopamine transmission and, in turn, the propensity to sign-track (Figure 2). Specifically, we hypothesize that, following Pavlovian conditioning, corticosterone cumulatively increases with the incentive motivational value of the reward cue (Figure 2). Thus, while this increase may be apparent following a single Pavlovian conditioning session (see Flagel et al., 2009), it becomes even more apparent once the conditioned responses become established, and is greater for sign-trackers relative to goal-trackers (Flagel et al., 2009). Further, we expect that this increase in corticosterone impacts the role of dopamine in encoding the incentive value of the reward cue. As we have previously demonstrated using this animal model, dopamine acts specifically to encode the incentive value of reward cues, not the predictive value (Flagel et al., 2011). Thus, as the CS-US relationship is learned, there is a shift in the dopamine response from the US to the CS, but this shift only occurs in sign-trackers (Flagel et al., 2011). We suspect, therefore, that glucocorticoids are acting in a time-sensitive manner to prime the dopamine system to respond selectively to the reward-cue (CS). The mechanism by which glucocorticoids and dopamine interact in this regard remains to be determined. However, given that sign-tracking behavior is dependent on dopamine in the NAc (Flagel et al., 2011; Saunders & Robinson, 2012), we expect the primary point of intersection to be within the VTA or NAc (Figure 2).

The temporal dynamics of corticosterone and dopamine over the course of stimulus-reward learning should be considered within this framework. An increase in dopamine overflow within the NAc can be captured across a Pavlovian conditioning session (Campus et al., 2019); as can sub-second changes in DA in response to the reward cue (Flagel et al., 2011). However, only a few studies have captured corticosterone levels within the NAc (e.g. Palamarchouk et al., 2009), and the dynamics of corticosterone in this regard are largely unknown. Thus, it remains to be determined whether the effects of GCs on stimulus-reward learning are rapid (taking seconds to minutes) and/or slow and genomic (taking hours to days) (De Kloet et al., 2008). We speculate that it is likely a combination of both. As reported above, we know that GCs rise immediately following a single Pavlovian conditioning session (Flagel et al., 2009; Tomie et al., 2002), and do so to a greater extent in STs (Flagel et al., 2009). However, pharmacological blockade of GR does not appear to have an immediate effect on sign-tracking behavior (Rice et al., 2018; 2019). Further, the ability of GCs to strengthen excitatory synapses of VTA-DA neurons via GR activation can take up to 2 hrs (Daftary et al., 2009; Polter & Kauer, 2014; Saal et al., 2003; Stelly et al., 2016); and stress-induced translocation of GRs to the nucleus of DA neurons occurs on a similar time scale, indiciative of genomic effects (Hensleigh & Pritchard, 2013). It is possible, therefore, that activity of GCs at GR can impact behavior during a Pavlovian conditioning session, but that the interaction between GCs and dopamine that promote incentive learning rely more on genomic mechanisms.

Another potential point of intersection for glucocorticoids and dopamine within the context of incentive motivational processes, is the DAT. Interestingly, relative to GTs, STs show greater DAT surface expression in ventral striatal synaptosomes and faster dopamine uptake in the NAc (Singer et al., 2016). These findings suggest that greater DAT surface expression promotes the attribution of incentive salience to discrete reward cues (Singer et al., 2016). It is possible that glucocorticoids are playing a facilitatory role in this regard. Currently, our understanding of the mechanism by which GCs alter DAT within the NAc are limited, but are presumed to be GR-dependent and to take days (Sarnyai et al., 1998); thus, indicative of genomic effects. Importantly, while we can learn a great deal about glucocorticoid function from the field of stress neurobiology, we recognize that the manner (genomic vs nongenomic) and mechanism by which glucocorticoids are impacting reward-processing and reward-related behaviors is likely to be quite different. We look forward to utilizing the ST/GT animal model to elucidate these relationships and to increase our understanding of the role of glucocorticoids in psychopathological behaviors.

Conclusion

The overwhelming amount of cues that surround us on a daily basis can help guide our behavior in a beneficial manner. However, for some, these environmental stimuli become triggers of complex emotions and, consequentially, maladaptive behavior. Thus, the way individuals respond to environmental stimuli may be indicative of vulnerability to psychopathology. While glucocorticoids have been implicated in a number of psychiatric conditions, their proposed role in this regard has centered primarily around stress responsivity. Importantly, however, glucocorticoids are constantly engaged throughout the body, both peripherally and centrally, and are vital in both physiological and behavioral adaptation. Given their wide-ranging effects beyond stress responsivity - including those on learning and memory, cognition and reward-related behaviors—it is reasonable to speculate that glucocorticoids may play a critical role in mediating individual differences in response to environmental stimuli. Similar to the “pathophysiological chain” framework put forth by Piazza and Le Moal more than 20 years ago, we believe that glucocorticoids and dopamine interact to mediate individual differences in reward-related behaviors. Here, however, we speculate that this interaction is perhaps most relevant to a distinct form of reward learning associated with psychopathology. Specifically, we propose that glucocorticoids are involved in dopamine-dependent incentive learning, a process that transforms neutral environmental stimuli into “motivational magnets” that attain inordinate control over behavior and elicit maladaptive behavior. Utilizing an animal model that allows us to isolate and probe the neural mechanisms underlying this specific form of reward learning will advance our understanding of glucocorticoid function and, hopefully, uncover novel mechanisms that contribute to complex neuropsychiatric disorders and their comorbidity.

Acknowledgements

We would like to thank the University of Michigan Department of Psychiatry for internal funding sources. We would also like to acknowledge Dr. Paolo Campus for assistance with the illustrations and for comments on earlier versions of this manuscript.

Disclosure statement

No potential conflict of interest was reported by the author(s).

Additional information

Funding

We thank current and previous funding sources for SAL, including the National Institute of Neurological Disorders and Stroke of the National Institutes of Health [1F99-NS115337-01], the National Science Foundation Graduate Research Fellowship Program, the National Institute on Drug Abuse Training Program in Neuroscience [T32-DA7281-21], and the Early Stage Training in the Neurosciences Training Grant [T32-NS076401]