Differential effects of dopamine-directed treatments on cognition

Abstract

Dopamine, a prominent neuromodulator, is implicated in many neuropsychiatric disorders. It has wide-ranging effects on both cortical and subcortical brain regions and on many types of cognitive tasks that rely on a variety of different learning and memory systems. As neuroscience and behavioral evidence for the existence of multiple memory systems and their corresponding neural networks accumulated, so did the notion that dopamine’s role is markedly different depending on which memory system is engaged. As a result, dopamine-directed treatments will have different effects on different types of cognitive behaviors. To predict what these effects will be, it is critical to understand: which memory system is mediating the behavior; the neural basis of the mediating memory system; the nature of the dopamine projections into that system; and the time course of dopamine after its release into the relevant brain regions. Consideration of these questions leads to different predictions for how changes in brain dopamine levels will affect automatic behaviors and behaviors mediated by declarative, procedural, and perceptual representation memory systems.

Video abstract

Point your SmartPhone at the code above. If you have a QR code reader the video abstract will appear. Or use:

http://youtu.be/WSx3_3n_5lg

Keywords:

• dopamine
• cognition
• memory systems
• learning

Introduction

In the half a century since its discovery, dopamine (DA) has been among the most widely studied neurotransmitters in the brain.¹ Dopaminergic systems have been studied in the context of neuropsychiatric and other disorders (eg, schizophrenia, attention deficit hyperactivity disorder (ADHD), Parkinson’s disease [PD], and drug addiction), as well as from the perspectives of cognitive development and aging. This broad interest in DA primarily reflects its important neuromodulatory function in many different subcortical and cortical neural networks, and the wide range of motor control, motivational, and cognitive functions that recruit these networks.

Not surprisingly, brain DA levels have been shown to affect performance of many different behaviors.² Even so, DA can have qualitatively different effects on some behaviors and on other behaviors it can have little if any effect. There are several reasons for this. First, DA neurons project to much, but not all, of the brain, so behaviors mediated within brain regions that are not DA targets will be relatively unaffected by DA-related medications compared with behaviors mediated within brain regions that receive rich DA projections. Second, the effects of phasic bursts of DA neurons have a vastly different time course in the cortex^3,4 and basal ganglia,^5,6 and thus, the effects of DA-related medications on cortical-mediated behaviors will be much different than on behaviors mediated by the basal ganglia. Third, there is now overwhelming evidence that humans have multiple learning and memory systems that for the most part are neuroanatomically and functionally distinct.^7,8 For example, declarative memory systems are mediated largely within frontal cortical areas that receive a dense DA projection and where DA release has a slow time course.^3,4 Procedural memory systems are mediated largely within the basal ganglia, which receive a dense DA projection that has a fast time course,^5,6 and the perceptual representation memory system is mediated largely within posterior cortical regions that receive almost no DA projection, at least compared with the heavily innervated frontal cortex.^9,10 Thus, to make accurate predictions of how DA-related medications will affect a behavior, it is critical to understand: which memory system is mediating that behavior, the neural basis of that system, the DA projections into the relevant brain regions, and the time course of DA release in those regions.

This article reviews the current literature on each of these four questions. We begin, in the next section, with a brief overview of some of the most common memory systems that have been identified by memory researchers. The following section reviews the neuroanatomy and neurophysiology of the DA system, and subsequent sections highlight the differing roles of DA in different memory systems and examines some of the reasons why brain DA levels can vary so much across individuals. The final section closes with some general comments and conclusions.

Memory systems

The existence of multiple memory systems is supported by a rich armamentarium of behavioral neuroscience studies in non-human animals (eg, single-unit recordings, and lesion and pharmacological studies) and cognitive neuroscience and neuropsychology studies in humans (neuropsychiatric patient studies and neuroimaging studies). A common taxonomy is to distinguish between declarative and nondeclarative systems.⁸ Briefly, declarative memories are those accessible to conscious awareness, whereas nondeclarative memories are stored and retrieved without conscious awareness.

Declarative memory

Declarative memory is a phylogenetically advanced system that allows conscious manipulation of information. It is typically thought to include three interrelated subsystems, ie, working memory, episodic memory, and semantic memory. Working memory is a temporary, limited capacity memory system that can store information for brief time periods (up to ~1 minute). Episodic memory is the long-term memory of specific events or episodes and semantic memory is the long-term memory of facts.

Working memory

Working memory is a limited capacity system that temporarily stores and maintains roughly 3–5 chunks of information¹¹ for online manipulation and organization, thereby supporting the human thought process by providing an interface between perception, long-term memory, and action.¹² The term “working memory” has largely superseded the term “short-term memory”, since the latter traditionally only describes retention, whereas the former also includes the ability to manipulate information.

Based on experiments that demonstrate low levels of interference during the maintenance of information acquired via different sensory modalities, it appears that working memory reflects a multidimensional process supported by different brain structures that are controlled by a central executive located in the frontal lobes.¹² Executive-attention and working memory circuits dependent on the prefrontal cortex (PFC) and the anterior cingulate cortex (ACC) are thought to select and maintain newly acquired stimulus representations from different sensory modalities, integrate these sensory inputs with long-term memory traces, action plans, and goal representations to form strategies that guide behavior.^12–14 To accommodate this higher-order manipulation of information, working memory processes are subject to prefrontal executive control, such as inhibition of non-relevant information, attentional guidance, and associative monitoring.^12,15,16 Thus, working memory processes may be grossly divided into two parts, ie, maintenance of information and the dynamic integration and manipulation of this information, which is heavily dependent on the PFC.^16–18

Working memory maintenance appears to be a volatile phenomenon that is reflected in short-term electrical/synaptic activity, in contrast with long-term memory, which is encoded in stable structural changes that depend on genetic transcription and are preserved over long periods of time until information is recalled and structural traces become unstable.^19,20 By this, we mean that working memories seem to be encoded via the sustained firing of neurons during the maintenance period. When the firing stops, the memories are lost, unless they have been transferred to some long-term structural store. In support of this hypothesis, many single-unit recording studies have found neurons that show sustained activity during delay periods during which an animal is motivated to maintain the memory of the location of some reward. Neurons of this type have been reported in a wide variety of different brain areas, including the PFC,²¹ the posterior parietal cortex,²² the medial dorsal nucleus of the thalamus,²³ the caudate nucleus,^24,25 and the globus pallidus.²⁶ Ashby et al²⁷ proposed a computational model in which sustained activation in all these structures contributes to working memory maintenance.

As part of the declarative system, working memory is an important conscious gate in the creation of declarative long-term memories. The transition from short-term to long-term memory is a dynamic process that includes acquisition, working memory maintenance, transition to intermediate-term memory, consolidation to long-term memory, and destabilization and restabilization of existing long-term memory traces during retrieval and integration of new information.¹⁹

Episodic and semantic memory

Declarative long-term memory is typically subdivided into the episodic memory of events and the semantic memory of facts.²⁸ Episodic and semantic memories are characterized by high specificity, capacity for partial retrieval, conscious awareness, capacity limitations, and slow access. The term “remember” is commonly used to refer to retrieval from these memory systems.²⁹ Declarative memory formation is subject to motivational and emotional control³⁰ and is potentiated by attention.³¹

The encoding of declarative information depends on conceptually driven, top-down processing of stimulus associations and experiences. Declarative memories are flexible by nature; flexibility arises because memories are stored in a relational network in which semantic information is shared among multiple episodes.³² The processing sequences that were implemented during encoding also regulate recall. When retrieving information from episodic and semantic memory, a major distinction is made between recollection, which is a slow controlled search process, and familiarity, which is a fast perceptual process.³³ The recall of an episodic or semantic memory requires recollection, but recognition can occur via either process.

Episodic and semantic memory systems depend on a widespread neural network. The hippocampus is critical for the consolidation of episodic memories,³⁴ whereas the consolidation of semantic memories also depends on surrounding medial temporal lobe structures, including the entorhinal, perirhinal, and parahippocampal cortices.³⁵ Long-term storage of the memories is distributed across much of the neocortex,³² whereas recall depends on the PFC, medial temporal lobe structures, and posterior midline regions.¹⁷

At the cellular level, long-term consolidation depends on long-term potentiation (LTP) at synapses within the hippocampus.³⁶ Hippocampal LTP begins with activation of post-synaptic N-methyl-D-aspartate (NMDA) receptors. A complex chemical cascade is then initiated, which eventually leads to gene transcription, causing structural changes that strengthen the synapse.²⁰

Nondeclarative memory

Nondeclarative memories, which are often called implicit memories, are expressed through performance rather than conscious recollection. In contrast with declarative memories that describe the “what” dimension of experience, implicit memories are concerned with the mechanism of “how” the organism interacts with its environment. Memories of this type mature early, appear to be highly specific, perceptually based, habitual, and inflexible. Non-declarative memory is an umbrella term with several subcategories tied to specific brain regions. Here we will focus on two subcategories, ie, the procedural and perceptual representation systems.

Procedural memory

Procedural memory systems store memories of habits and skills that are learned via repeated practice.³⁷ Compared with declarative memories, the major distinguishing characteristics of procedural memories are: little conscious recollection or even awareness of what was learned; slow and incremental learning; learning that requires immediate and consistent feedback;^37,38 and learning that includes a motor component.^39,40 Traditionally, this type of memory refers to motor skills, such as playing a musical instrument, riding a motorcycle, or tying a shoe (known colloquially as muscle memories).

More recently, however, it has been discovered that many more purely cognitive behaviors also recruit procedural learning and memory systems. The best evidence for this comes from information-integration category learning tasks. In category learning tasks, two or more categories of unfamiliar visual images are presented to the subject one at a time and the subject’s task is to use trial-by-trial feedback about response accuracy to learn to assign each image to its correct category. Information-integration category learning tasks are those in which accuracy is maximized only if information from two or more non-commensurable stimulus dimensions is integrated at some predecisional stage.⁴¹ Typically, the optimal strategy in information-integration tasks is difficult or impossible to describe verbally, so it is unlikely to be discovered via logical reasoning or by a process of explicit hypothesis generation and testing. Tasks in which these latter explicit strategies succeed are called rule-based category learning tasks. Figure 1 shows examples of rule-based and information-integration tasks that are frequently used in laboratory studies. A real life example of an information-integration task might be learning to discriminate between German shepherds and wolves. Much evidence suggests that success in information-integration tasks depends on procedural learning. For example, switching the locations of the response keys interferes with information-integration categorization, but not with rule-based categorization.^39,42

Figure 1 Examples of rule-based and information-integration category structures.

Notes: Each stimulus is a sine wave disk that varies across trials in the number of bars per disk (or bar narrowness) and bar orientation. For each task, three illustrative category A and category B stimuli are shown. The plus signs and open circles denote the specific values of all stimuli used in each task. In the rule-based task, only the number of bars per disk carries diagnostic category information, so the optimal strategy is to respond with a one-dimensional bar narrowness rule (thin vs thick). In the information-integration task, both the number of bars per disk and bar orientation carry useful but insufficient category information. The optimal strategy requires integrating information from both dimensions in a way that is impossible to describe verbally.

Another difference between the declarative and procedural systems is that learning in tasks that depend on declarative memory is flexible with regards to feedback timing, in the sense that long timing delays often have no detrimental effect on learning. In contrast, for procedural learning, the timing of feedback is critical. Learning is best when feedback immediately follows the behavior. Several studies have shown that rule-based category learning is unaffected by feedback delays as long as 10 seconds,^38,43 supporting the notion that declarative and especially working memory systems are recruited. In contrast, information-integration category learning is best with a feedback delay of 500 milliseconds, slightly worse with delays of 0 or 1,000 milliseconds,⁴⁴ and highly impaired with feedback delays of 2.5 seconds or longer.^38,43 This complex pattern of results suggests that there is an optimal time for feedback to arrive after a response.

There is now overwhelming evidence that procedural learning depends on the basal ganglia, and especially on the striatum. This evidence comes from single-unit recording studies in non-human animals,^45,46 animal lesion experiments,^47–50 neuropsychological patient studies,^51–53 and human neuroimaging studies.^54–56

Perceptual representation memory

The perceptual representation memory system mediates “improvement in identifying or processing a stimulus as the result of its having been observed previously”.⁵⁷ This type of learning is often referred to as repetition priming and it occurs even in the absence of feedback.⁵⁸ Phenomenologically, perceptual representation memory can mediate the feeling of visual familiarity that occurs when one sees a visual image that has been seen before, but perceptual representation memory is an implicit system that can operate without conscious awareness.⁵⁹ Behavioral effects of this type of memory can be observed after only a single stimulus repetition⁶⁰ and can persist for months.⁶¹

Neuroimaging data and neuropsychological patient studies suggest a locus within the visual cortex. For example, perceptual priming is associated with a repetition-dependent suppression of activity in extrastriate regions of the visual cortex, such as the extrastriate occipital cortex.^17,62 Similarly, animal studies show reduced cellular activity in the inferior temporal cortex that is proportional to stimulus repetition.⁶³ However, these data do not provide a clear cellular mechanism for perceptual learning within the perceptual representation system.

Memory of automatic behaviors

Many actions performed by adults are automatic. When we brush our teeth, make our morning coffee, or ride a bicycle, our actions are typically automatic. The evidence is now good that automatic behaviors are not stored via any of the declarative or nondeclarative memory systems discussed so far.⁶⁴ For example, PD patients have impaired declarative and nondeclarative memories. Even so, some PD patients are nevertheless able to emit an automatic motor response when presented with a familiar visual cue (eg, kicking a ball), despite difficulties in initiating novel voluntary movements.⁶⁵

Ashby et al⁶⁶ hypothesized that after practicing a behavior that was initially acquired via procedural learning long enough for it to become automatic, its representation is entirely cortical. According to this view, the development of automaticity is associated with a gradual transfer of control from the striatum to cortical–cortical projections from the relevant sensory areas directly to the premotor areas that initiate the behavior.

Dopamine systems

Neuroanatomy

Although DA cells are found in a number of central nervous system sites, for the purposes of understanding cognitive behaviors, the most important DA neurons originate in the midbrain regions of the substantia nigra pars compacta (SNpc) and the ventral tegmental area (VTA). DA neurons in these areas widely innervate various subcortical and cortical regions via the nigrostriatal, mesolimbic, mesocortical, and thalamic ascending DA projection pathways. The cell bodies of the nigrostriatal DA system, which are located in the SNpc, send dense projections to the striatum. The mesolimbic DA system originates from a more diffuse collection of neurons in the VTA. One portion of these neurons projects to limbic regions such as the nucleus accumbens, the amygdala, the hippocampus, and the ACC. A third pathway, referred to as the mesocortical DA system, also originates from the VTA and projects to virtually all of the frontal cortex. A fourth pathway, recently identified in the primate brain, projects to the thalamus and may be independent from the other systems.⁶⁷

It is important to note that DA neurons in the VTA and SNpc projecting to striatal, limbic, and cortical areas are at least partly intermixed, suggesting that ascending DA systems interact.⁶⁸ For example, it is true that most cortical DA comes from the VTA, but there are notable exceptions, such as a group of neurons in the dorsolateral SNpc that project to the dorsolateral PFC,⁶⁹ and another group in the ventromedial SNpc that projects to the ventromedial PFC, ACC, and orbitofrontal cortex.^70,71

Within the basal ganglia, an additional organizational scheme is that striatal neurons (within patch compartments) send direct inhibitory projections to those DA neurons from which they receive DA input and also to other DA neurons that in turn project to a different striatal subregion.⁷² Thus, ascending DA pathways interact and enable functional interactions via a series of parallel cortical-striatal-cortical loops.⁷³ This spiral organization allows for information flow from ventral parts of the basal ganglia important in emotional learning to dorsal parts related more to cognitive and motor function.⁷⁴

Through these pathways, the DA systems contribute to a wide variety of cognitive behaviors. Not surprisingly, DA dysfunction in neurological and neuropsychiatric disorders such as PD, schizophrenia, depression, ADHD, anorexia, autism spectrum disorders, and drug addiction is associated with multiple types of impairments, including of motor control, cognition, and motivation.^75–79

Neurophysiology

DA neuron firing includes both phasic and tonic phases.⁸⁰ The phasic bursts are brief, on the time scale of milliseconds, while the tonic mode includes spontaneous single spike activity that provides a baseline firing rate, which can briefly pause under certain conditions. Spike bursts trigger a high amplitude phasic DA release into targeted synapses,^81,82 whereas the tonic firing of DA neurons is responsible for the baseline DA concentration.

After its release, free DA is either reabsorbed into the presynaptic terminal for reuse or broken down by enzymes. DA finely tunes its own synthesis and release with an inhibitory feedback loop mediated by presynaptic (D2) autoreceptors. Dopamine active transporter (DAT) is a membrane-bound, presynaptic protein that serves as a regulator of the synaptic concentration of DA at nerve terminals.⁸³ DAT provides a rapid and efficient mechanism for reuptake of synaptic DA and is essential for the regulation of DA neurotransmission.⁸⁴

The concentration of DAT serves as a marker of the homeostatic tone of the DA system.⁸⁵ The highest concentrations of DAT are found in the striatum, with much lower concentrations in the brain stem, thalamus, basolateral amygdala, ACC, and neocortex, including the PFC.^86,87 The high striatal concentrations of DAT cause exceptionally fast DA reuptake and therefore rapid clearing of DA from striatal synapses.⁵ This is an extremely important feature that makes the function of striatal DA unique, where a single phasic DA burst has the ability for temporal coding of the event that generated it. In contrast, low concentrations of cortical DAT cause DA to persist for much longer in cortical synapses.^3,88 For example, the delivery of a single food pellet to a hungry rat elevates DA levels in the PFC for approximately 30 minutes.⁸⁹ In the cortex, DA must either diffuse out of the synapse or be degraded by enzymes such as catechol-o-methyltransferase (COMT).⁹⁰

Most DA receptors are located on post-synaptic neurons. Five receptor subtypes (D1–D5) are currently identified. The DA receptor subtypes have distinct anatomical distributions in the brain⁹¹ and can be viewed as markers for different clusters of DA-related functions. The five subtypes are grouped into two families on the basis of structural homology and biochemical characteristics. The family of D1-like receptors includes the D1 and D5 subtypes, and the family of D2-like receptors includes the D2, D3, and D4 subtypes. Of all DA receptors, the D1 and D2 receptors are, by far, the most common. Among these two, D1 receptors are more abundant than D2 receptors, reflecting high concentrations not only in the striatum but also throughout the neocortex.⁹² For example, in the frontal cortex, there are approximately ten times as many D1 as D2 receptors.⁹³ D2 receptors are highly concentrated in the striatum, lower concentrations are expressed in the brainstem and thalamus, and concentrations are minute in the neocortex.^86,94

Receptors of the D1 family are expressed throughout the brain. In contrast, receptors in the D2 family exhibit more regional specificity: D2 receptors are expressed primarily in the dorsal striatum, D3 receptors in the ventral striatum, including the nucleus accumbens but less so in the dorsal striatum, and D4 receptors in the frontal cortex and limbic regions.⁹⁵ These differences have important consequences for cognition and behavior.^96,97

The D1 and D2 receptors are slow-acting metabotropic receptors that are linked to G proteins. Activation of D1 receptors leads to an increase in intracellular levels of cyclic AMP, whereas activation of D2 receptors leads to decreases in the levels of intracellular cyclic AMP. In addition, phasic vs tonic DA release may have differential effects on D1 and D2 receptors.⁸⁰

Post-synaptic DA receptors are located on neurons that are the targets of the most common excitatory neurotransmitter in the brain, namely glutamate, and there is now substantial evidence that DA modulates the postsynaptic effects of glutamate.^98,99 There are two primary mechanisms via which DA exerts these neuromodulatory effects, ie, fast and slow. The fast effect of increases in synaptic DA levels is to increase the signal-to-noise ratio of the postsynaptic glutamate response,^100–102 resulting in immediate effects on behavior that persist until DA levels decline. The slower effect is to potentiate LTP and long-term depression (LTD), leading to increased levels of synaptic plasticity^103,104 which causes long-term changes in learned behaviors.

Neurons select meaningful signals out of noise by suppressing relatively weaker signals, while amplifying stronger ones. To achieve this dual effect, glutamate exerts different effects on two classes of receptors, ie, NMDA and non-NMDA. A common member of the non-NMDA class is the α-amino-3-hydroxy-5-methyl-4-isoxazolepropionic acid (AMPA) receptor. AMPA and NMDA are both ionotropic receptors, but NMDA requires higher levels of presynaptic glutamate release for activation than AMPA. Therefore, at low levels of stimulation, the postsynaptic glutamate response is dominated by activity at AMPA (and other non-NMDA) receptors, whereas at high levels of stimulation, the glutamate response is dominated by activity at NMDA receptors. Evidence suggests that DA potentiates the glutamate response through the NMDA receptor^102,105,106 and depresses the glutamate response through the AMPA receptor.^100,101 The net effect of these modulations is to weaken the postsynaptic response when presynaptic stimulation is weak and to potentiate the postsynaptic response when presynaptic stimulation is strong. This increases the overall signal-to-noise ratio because noise should cause weak presynaptic stimulation, whereas signal should cause the presynaptic stimulation to be strong. Several mathematical models of this modulatory effect have been proposed.^107,108

DA is known to have pronounced effects on synaptic plasticity. This has been observed in hippocampal,¹⁰⁹ striatal,^110,111 and PFC neurons.¹¹² Evidence suggests that presynaptic glutamate release initiates several postsynaptic intracellular cascades that alter the strength of these synapses, and that DA plays an important role in these processes. Much evidence suggests that, when presynaptic and postsynaptic activation are both strong, DA levels above baseline promote LTP.¹⁰⁴ Furthermore, within the striatum at least, if strong presynaptic and postsynaptic activation are present, but DA release is below baseline, then LTD is observed.¹⁰³

A large literature shows that DA neurons in the VTA and SNpc increase their firing above baseline following unexpected rewards and decrease their firing below baseline following failure to receive an expected reward.^113–115 Thus, DA-enhanced LTP should be in effect following an unexpected reward in any brain region that is a target of VTA or SNpc DA neurons.

The mechanisms via which DA potentiates LTP are reasonably well understood. One such mechanism is mediated by chemical cascades that are initiated when NMDA receptors are activated by glutamate, which begins the phosphorylation of calcium/calmodulin-dependent protein kinase II (CaMKII).¹¹⁶ During a brief period of time (0.3–2 seconds)¹¹⁷ when CaMKII is partially phosphorylated, a different chemical cascade that is initiated when DA binds to D1 receptors can potentiate the LTP-inducing effects of CaMKII. The critical step in this DA-induced cascade may be the phosphorylation of DARPP-32 (dopamine and cAMP-regulated phosphoprotein) because the phosphorylated version of DARPP-32 deactivates proteins (eg, PP-1) that reduce the LTP effects of CaMKII. Thus, when DA binds to D1 receptors, an important LTP-inhibiting action is reduced. The effect of phosphorylating Ca²⁺-dependent enzymes, such as CaMKII is to signal the activation of transcription factors and genes that encode for the synthesis of membrane and cytoskeletal proteins to mediate actual structural changes in dendritic spines.¹¹⁸

In the machine-learning literature, a distinction is made between Hebbian learning and reinforcement learning.^119,120 In Hebbian learning, any synapses that are (simultaneously) highly active are strengthened and any synapses that are weakly active are weakened. Reinforcement learning is similar to Hebbian learning with the additional requirement of a learning (or reinforcement) signal. If the signal is present, then the learning rules are the same as in Hebbian learning, but if the signal is absent, then all synapses are weakened, regardless of activity. Many researchers have argued that DA plays the role of the reinforcement signal in the basal ganglia.^6,121

A necessary feature of any reinforcement signal is high temporal resolution. Consider a task in which the goal is to learn a number of stimulus–response associations. Following a correct response, DA must be released into the relevant synapses quickly, before the critical traces disappear. But after the correct synapses have been strengthened, it is also essential that excess DA be quickly cleared from the synapse. If it is not, and the response on the next trial is an error, then the residual DA will strengthen inappropriate synapses, namely, those responsible for producing the incorrect response. This would undo the beneficial learning that occurred following correct responses, and prevent discrimination learning. Within the striatum, DA is quickly cleared from synapses by DAT and, as a result, the temporal resolution of DA in the striatum is high enough for DA to serve as a trial-by-trial reinforcement signal. However, the low concentrations of DAT in the frontal cortex cause cortical DA levels to change slowly. Thus, the first rewarded behavior in a training session is likely to cause frontal cortical DA levels to rise, and the absence of DAT will cause DA levels in the frontal cortex to remain high throughout the training session. As a result, all synapses that are activated during the session are likely to be strengthened, regardless of whether the associated behavior is appropriate or not. Thus, although DA may facilitate LTP in the frontal cortex, it appears to operate too slowly to serve as a trial-by-trial reinforcement signal.⁴ Instead, it is thought that cortical LTP/LTD follows Hebbian learning rules,^121,122 which means that cortical LTP can occur regardless of whether the resulting behavior was rewarded. Recent work suggests, however, that within the PFC at least, this Hebbian learning depends on baseline DA levels. If the background DA is too low, then activity that might normally facilitate LTP can actually induce LTD.^123,124 Thus, disorders that reduce tonic cortical DA levels could have profound effects on the learning of PFC-mediated behaviors.

In summary, DA has two prominent neuromodulatory effects, ie, a fast effect in which the signal-to-noise ratio of the postsynaptic glutamate response is increased, and a slower effect in which synaptic plasticity is potentiated. The consequences of these two different effects are vastly different in the basal ganglia, where DAT concentrations are high, and in other targets of VTA and SNpc DA neurons where DAT concentrations are low (such as in the frontal cortex). For example, in the frontal cortex, a single phasic burst from DA neurons could increase signal-to-noise ratio for many minutes, whereas this increase would likely only last a few seconds in the basal ganglia. Furthermore, these regional differences in DAT concentration mean that synaptic plasticity follows reinforcement learning rules within the striatum and Hebbian rules in the frontal cortex and in other DAT-poor targets of VTA and SNpc DA neurons.

Effect of DA on behaviors mediated by declarative learning and memory systems

Declarative learning and memory systems receive a dense DA projection, mainly from the VTA. Furthermore, DAT concentrations are low in almost all brain regions that contribute to declarative systems. Thus, we should expect that increases in brain DA levels (at least up to some optimal level) should cause persistent increases in signal-to-noise ratio and enhanced Hebbian learning for any behavior mediated by declarative systems.

A wide range of research supports these general predictions, although as we will see, the evidence suggests that there is an optimal DA level for cognitive function, that is, performance is compromised if DA levels are either higher or lower than this optimal level.

Deficits due to DA reductions

PD is caused by the accelerated death of DA-producing neurons. Although the earliest damage is typically to the nigrostriatal system, there is also concomitant damage to the mesocorticolimbic system. In particular, production of DA in the VTA is substantially diminished as the disease progresses.^125,126 Since DA neurons die in both the SNpc and the VTA, DA levels decrease in both the PFC and the striatum.² Within the SNpc, cell loss is predominantly found in the ventral tier with less (but still extensive) damage in the dorsal tier.^127,128 This is opposite to the dorsal-to-ventral pattern of the less severe cell loss occurring in normal aging. Parkinsonian motor symptoms appear after a 60%–70% loss of SNpc neurons and a 70%–80% loss of DA levels in the striatal nuclei.^128,129 These motor symptoms include tremor, rigidity, bradykinesia, and akinesia.

In addition to motor deficits, non-demented PD patients present cognitive symptoms that resemble those observed in patients with PFC damage (especially those with dorsolateral PFC lesions).¹³⁰ Numerous studies documenting the cognitive deficits of PD patients have revealed impairment in a wide variety of tasks mediated by declarative memory systems, including working memory, logical reasoning, recall from episodic memory, rule learning, and tasks that require executive attention.^131–133

For example, PD patients are impaired in rule-based category learning, which requires hypothesis testing about what simple explicit rules can separate perceptual stimuli into categories.¹³³ The most widely known example of a rule-based task is the Wisconsin Card Sorting Test (WCST),¹³⁴ and as expected, PD patients are impaired on the WCST.¹³⁵ Specifically, they show a tendency to perseverate on rules that are no longer successful.

Owen et al argued that perseverative errors can occur on the WCST for two reasons.¹³⁰ One is a failure to select an appropriate alternative stimulus feature when feedback suggests that the feature that is the current focus of attention is incorrect, and the second is a failure to switch attention from the current unsuccessful feature to the newly selected feature. Using a clever design that isolated these two cognitive operations, Owen et al found that patients with frontal damage were impaired on attentional switching but not on feature selection, whereas (unmedicated) PD patients were impaired on both operations. Based on these and other results, Ashby et al proposed that feature selection is mediated by a circuit that includes the ACC and the PFC, whereas attentional switching is mediated by a circuit that includes the PFC and the head of the caudate nucleus.¹³⁶

One line of evidence supporting this hypothesis is that lesioning the DA fibers that project from the VTA into the PFC actually improves the performance of monkeys on a simplified version of the WCST.¹³⁷ If switching occurs entirely within the PFC, then such lesions should impair switching performance (as seen, for example, in patients with PD). If the basal ganglia play a key role, however, then lesioning DA fibers into the PFC should have no direct effect on switching. How then can one explain that lesioning DA fibers into the PFC actually improves performance on the WCST? An important clue to this apparent paradox comes from reports that such lesions tend to increase DA levels in the basal ganglia.¹³⁷ If the basal ganglia are responsible for switching, if switching is enhanced by DA, and if lesioning the DA fibers that enter the PFC increases DA levels in the basal ganglia, then lesioning the DA fibers in the PFC should improve switching. Data such as these support the hypothesis that switching is mediated primarily by the basal ganglia whereas selection is mediated primarily by the ACC.

PD patients also suffer from proactive interference in rule application, which also has switching and selection elements. In testing PD patients in the Odd-Man-Out choice discrimination task (a task where subjects need to pick the odd-man-out from a grouping of three stimuli), Flowers and Robertson¹³⁸ found that PD patients were relatively unimpaired on the first block of trials using one rule (ie, performance was quite close to controls), but were subsequently impaired in later blocks using either a different rule or the same original rule. In fact, their performance decrement was only slightly improved when told explicitly what rule to use when selecting the Odd-Man-Out stimulus: subjects never reacquired the same performance level as at the beginning of the test. In a similar task where subjects were required to alternate their response strategy on a trial-by-trial basis, PD patients produced more false alarms than controls, but only for long time intervals between targets.¹³⁹ Taken together, these results show that PD patients are susceptible to proactive interference, which reflects deficits both in switching away from an early response strategy or rule, and selecting a new one.

Episodic memory can be tested by recall (eg, “What did you have for breakfast?”) or by recognition (“Did you have cereal?”). Compared with recognition, recall depends much more heavily on the ability to select the appropriate retrieval strategy, maintain it, and switch to a new strategy if the current strategy is ineffective.¹⁴⁰ These processes are similar to those required in rule-based category learning. Not surprisingly, therefore, PD patients are more impaired in recall than in recognition, recalling fewer words than controls while still showing the usual serial position effects of recency and primacy.¹⁴¹ Evidence that this deficit is due to PFC dysfunction comes from studies showing that patients with frontal lobe damage use fewer strategies than healthy controls during episodic memory encoding and recall,¹⁴² and have difficulty remembering the sequence (temporal order) and the source (who said what) of items.¹⁴³

Facilitation due to DA increases

Whereas many studies have shown that DA depletion impairs the performance of behaviors that depend on declarative memory systems, many fewer have directly investigated the effects of increasing DA levels in healthy subjects. Even so, some promising results have been reported. For example, several studies have reported improvements in the working memory of healthy humans who were given small doses of a DA agonist.^144,145 Although high doses of DA-targeting drugs, such as psychostimulants, impair PFC function, low doses preferentially improve PFC cognitive function of both healthy subjects and patients with neuropsychiatric disorders such as schizophrenia and ADHD.¹⁴⁶ Even so, most of the data on how DA increases might affect declarative memory-based tasks are indirect.

Ashby et al proposed that events that induce mild positive affect (ie, improved mood) in healthy adults are also often associated with increased cortical DA levels.¹⁴⁷ For example, receiving an unexpected gift improves mood and it should also cause cortical DA levels to rise since DA neurons are well known to increase their firing following unexpected rewards.¹¹⁴ There is other evidence linking positive mood to DA. First, drugs that mimic the effects of DA (ie, DA agonists) or that enhance dopaminergic activity elevate feelings. These drugs include morphine and apomorphine (agonists), cocaine (which blocks DA reuptake), amphetamines (which increase DA release), and naturally produced endorphins.¹⁴⁸ Finally, DA antagonists (ie, neuroleptics) are thought to flatten affect and render reward ineffective.¹⁴⁹

If positive affect increases cortical DA levels, and if increased cortical DA improves executive function, then mild positive affect should be associated with improved executive function. Considerable evidence supports this prediction. In particular, many studies have shown that mild positive affect improves creative problem-solving, facilitates recall, and generally improves cognitive flexibility.¹⁴⁷ Perhaps the strongest such evidence comes from studies of creative problem-solving using tasks such as the Remote Associates Test¹⁵⁰ and the Duncker candle task.¹⁵¹ Success in both of these requires the selection of a nondominant response (eg, finding unusual compound words that relate usually unrelated words and using the match box as a platform for the candle instead of just storing matches). Thus, both tasks make heavy demands on the selection operation identified by Owen et al¹³⁰ and hypothesized to depend on an ACC/PFC circuit.¹³⁶ A number of studies have reported that positive affect improves performance in both tasks.^152–154

More recently, Nadler et al showed that relative to neutral affect controls, positive affect subjects performed better in rule-based categorization, but not in information-integration categorization.¹⁵⁵ These results are expected because the abundant DAT in the striatum should cause the DA released to the positive affect-inducing event to be cleared quickly out of the striatum, and therefore not influence the subsequent information-integration category learning. In contrast, the relative lack of cortical DAT means that the DA released to the positive affect-inducing event should elevate cortical DA levels above baseline for 20–30 minutes, ie, long enough to facilitate the selection and attentional switching processes needed for success in the rule-based task.

Positive affect has also been shown to facilitate the encoding of neutral and positive episodic memories, which is demonstrated by better later recall.^156,157 One possibility is that the positive affect facilitates hippocampal-mediated memory consolidation. VTA DA projects heavily to the hippocampus, an important part of the medial temporal lobes thought to be necessary for the consolidation of episodic memories.^158,159 Normal hippocampal function depends critically on the neurotransmitter acetylcholine (eg, reductions in hippocampal cholinergic activity produce spatial memory deficits in rats).¹⁶⁰ DA has been shown to increase acetylcholine release in the hippocampus¹⁶¹ and to improve memory consolidation in a brightness discrimination task.¹⁶² Thus, it is plausible that positive affect-induced DA release improves episodic memory consolidation.

Optimal DA levels

Although declarative memory-mediated behaviors seem to improve when cortical DA levels rise modestly, evidence also strongly suggests that large increases impair performance. In other words, DA appears to have an “inverted U-shaped” influence on PFC function, where either decreases or increases from an optimal level of DA impair declarative memory-mediated behaviors.¹⁶³ For example, blockade and excessive stimulation of D1 receptors both impair spatial working memory.^164,165

Other supporting evidence comes from studies that induced negative affect. One might speculate that if positive affect is associated with increased cortical DA levels, then negative affect may be associated with decreased levels. However, stressful or anxiety-provoking events, which presumably would produce a negative affective state in humans, actually appear to increase DA levels in certain brain regions. In particular, animal studies indicate that stressful events (eg, foot shock, tail pinch) cause increased DA release from the VTA but have little or no effect on DA release from the SNpc.^166,167 Working memory deficits during stress exposure are likely mediated by excessive D1 receptor stimulation, because they are prevented by D1 antagonist administration. At high levels of D1 receptor stimulation, the responsiveness and signal-to-noise ratio of PFC neurons are both reduced. Thus, these adverse effects of stress could be due to overstimulation of the DA system.

Changing environmental demands and internal states may necessitate shifts of control from one memory system to another, or a fine-tuned coordination between them. It appears that DA participates in influencing this interplay between memory systems. Under optimal levels of DA, PFC functions such as working memory and selective attention are supported, thereby enabling deliberate, flexible regulation of behavior within a changing environment. Sliding to either side of the “inverted U” function of DA compromises this state. Under chronic stress exposure (eg, over 2 weeks), a decrease in baseline PFC DA levels is observed.¹⁶⁸ Such DA reductions are associated with anhedonia, social withdrawal (as in schizophrenia),¹⁶⁹ and depression.¹⁷⁰

On the other hand, acute uncontrollable stress can produce high levels of DA and norepinephrine, with the effect that control of behavior is switched to the amygdala and other more primitive brain circuits.¹⁷¹ Several lines of evidence suggest that DA may initiate this shift: DA levels in the amygdala are increased under stress;^172,173 DA produces disinhibition in the basolateral amygdala;¹⁷⁴ and release of DA enhances amygdala-related behavior,¹⁷⁵ whereas inhibition of the DA system depresses learned fear responses.¹⁷⁶ Shifting control of behavior from the PFC to the amygdala and other subcortical systems creates a bias toward sensory-driven emotional responses, fear conditioning, and stressor-associated procedural and spatial memory formation. The adaptive value of such switches may be that responding becomes more rapid and less effortful. This might save one’s life when there is danger and a need to react rapidly, but it could be detrimental in situations that require thoughtful analysis and inhibitory control.

Effect of DA on behaviors mediated by procedural learning and memory systems

Procedural learning and memory systems receive a dense DA projection, from both the SNpc and the VTA. In addition, DAT concentrations are high in the most critical procedural system brain regions, namely the basal ganglia. Thus, we should expect that increases in DA levels will cause transient increases in signal-to-noise ratio and enhanced reinforcement learning for any behavior mediated by procedural systems. Again, the available data support these general predictions.

First, the induction of positive affect improves learning on declarative memory-mediated rule-based categorization, but not on procedural memory-mediated information-integration categorization.¹⁵⁵ This result follows from the assumption that the event that induced the positive affect increases cortical signal-to-noise ratios throughout the subsequent period of category learning, whereas the increased striatal signal-to-noise ratios caused by the same event are too brief to affect the category learning.

Second, many studies have shown that PD patients, who have reduced striatal DA levels, are impaired in procedural learning. The two most heavily studied procedural-learning tasks are information-integration category learning and the serial reaction time task,¹⁷⁷ which requires subjects to press keys as quickly as possible in response to stimuli that appear in various locations on the screen. A large improvement in response time is observed when the stimulus sequence is repeated, even when subjects are unaware that a sequence exists. Several studies have reported that patients with PD show reduced implicit learning in the serial reaction time task.^178,179 Other studies have reported that PD patients are impaired in information-integration learning, especially when the task is difficult.^52,53,180 These results are consistent with the hypothesis that reductions in striatal DA levels decrease cortical-striatal synaptic plasticity.

Effect of DA on behaviors mediated by other learning and memory systems

Perceptual representation memory system

The perceptual representation memory system is mediated primarily within sensory association areas of the cortex. The DA systems do not project in any significant way to the visual or auditory cortex,^9,10 so the effects of changes in brain DA levels should be minimal on behaviors mediated by perceptual representation memory.

This prediction is surprisingly difficult to test. On the one hand, some seemingly counter results have been reported. For example, transient stimulation of VTA DA neurons during presentation of an auditory tone increases the extent of the representation of this tone in the auditory cortex.¹⁸¹ Similarly, several functional magnetic resonance imaging studies have reported reward-based learning effects in the visual cortex that are consistent with DA-mediated synaptic plasticity.^182,183 One complication, however, in interpreting these results is that the effects of reward in sensory association areas should be similar to the effects of attention, because attention is biased toward stimuli that predict reward.¹⁸⁴ In fact, it has been proposed that within the visual cortex, plasticity is mediated by attention-gated reinforcement learning that uses acetylcholine as the reinforcement signal, rather than DA.¹⁸⁵ In support of this prediction, stimulation of cholinergic neurons in the basal forebrain during presentation of an auditory tone also increases the extent of the representation of this tone in the auditory cortex.¹⁸⁶ Other studies have shown that cortical learning is impaired following acetylcholine depletion.¹⁸⁷ To complicate matters even more, there is abundant evidence that the DA and cholinergic systems interact, so altering DA levels is likely to have concomitant effects on acetylcholine levels.^188,189 Clearly, more work is needed on this issue.

Memory of automatic behaviors

If automatic behaviors are stored in the cortex, as hypothesized by Ashby et al,⁶⁶ then their production should be modestly affected by changes in brain DA levels. The motor and premotor cortex receive a relatively dense DA projection from the VTA, but the visual and auditory cortex do not. These predictions are largely untested, although, as noted earlier, PD patients, who have DA reductions and striatal dysfunction, are impaired in procedural learning but are relatively normal in producing automatic skills.⁶⁵

Individual differences in DA function

Many factors are known to affect brain DA levels, including age, genetic predisposition, drug-taking history, and neuropsychological patient status.¹⁴⁷ For example, brain DA levels are known to decrease by approximately 7% per decade of life during normal aging. However, adolescence is associated with sustained overexpression of DA receptors in the PFC that persists until adulthood.¹⁹⁰ Interestingly, this inverted U-shaped peak and decline of DA receptor levels occur earlier in the striatum than in the PFC.¹⁹¹ The firing rate of midbrain DA neurons also peaks during this distinctive developmental period.¹⁹² Not surprisingly, adolescents differ from adults on cognitive tasks that are PFC-dependent and DA-dependent, such as decision-making, planning, working memory, and inhibitory control.^193,194 These and other developmental shifts in DA function¹⁹¹ may contribute to the heightened vulnerability to drug abuse and stress during adolescence.¹⁹⁵

Neurological diseases that impact the DA system may result from aberrant development at different stages. For example, schizophrenia typically develops during late adolescence and early adulthood,¹⁹¹ while ADHD develops in early childhood, but in 60%–70% of cases symptoms persist into adulthood.^196,197 Similarly, single gene mutation diseases such as Fragile X syndrome and Rett syndrome are neurodevelopmental autism spectrum disorders marked by derailment of brain development (including synaptic plasticity) from infancy, and especially during the critical period from ages 1 to 3 years.^198–200

Not surprisingly, different aspects of the DA system malfunction in these three DA-related disorders. A recent meta-analysis²⁰¹ found that dopaminergic abnormality in schizophrenia is mainly presynaptic, which affects DA synthesis, baseline synaptic DA levels, and DA release. Current treatment mechanisms rely on D2 receptor blockade, which fail to target these abnormalities. ADHD stimulant medications target DAT levels, which are lower in ADHD patients.²⁰² In Fragile X syndrome and Rett syndrome, the DA system is disrupted and hypofunctional,^79,108 and dopaminergic treatment shows promise in ameliorating at least some of the devastating symptoms.^203–205

It is important to note that DA medications do not have uniform effects throughout the brain. When PD patients are given DA-related medications to improve their motor function (eg, L-dopa), they show a modest improvement on many of the memory and executive tasks on which they are normally impaired.¹³¹ However, such drugs may improve some processes but have a minimal, or detrimental, effect on others. For example, Cools et al reported that DA replacement therapy improved the task-switching performance of PD patients, but made the same patients impulsive.²⁰⁶ They argued that by increasing levels of DA, normal function was restored to the system supporting task-switching but the system responsible for decision-making became “overdosed”.

Could DA agonists ameliorate memory problems in people with other kinds of frontal lobe dysfunction, such as due to aging or focal lesion? In cases of focal lesion, this may be only successful if enough tissue is left to respond to the DA drug.²⁰⁷ In one such successful attempt, McDowell et al²⁰⁸ gave either a DA agonist (bromocriptine) or placebo to traumatic brain injury patients (who showed impaired executive function, and probably had some frontal damage). The drug selectively improved performance on many of the executive measures, such as reducing perseverative errors on the WCST.

Another source of variability is genetic factors that affect DA neurotransmission.²⁰⁹ Thanks to recent progress in genomic research, individual differences in a variety of DA-relevant genotypes have been associated with a range of cognitive functions, such as working memory, attention, episodic memory, and reward processing, as well as with psychiatric disorders^210,211 and addiction.²¹² At this point it is too soon to draw any strong conclusions because of possible complex interactions among genes and between genes and the environment. Even so, guided by the framework laid out in this article, some predictions are possible and even supported by recent genomic studies.

For example, several studies have shown that subjects with a COMT genotype (met/met) associated with higher PFC DA levels have better working memory and perform better on the WCST test than subjects with a COMT genotype (val/val) associated with lower levels of PFC DA.^213,214 Another example is a recent study showing that carriers of a specific DRD2 genotype that is associated with higher striatal D2 receptor availability were better at information-integration category learning than participants with a genotype associated with lower D2 availability.²¹⁵ Similarly, other studies have examined the physiological and behavioral effects of DARPP-32.^97,216

Conclusion

The multiple roles via which DA affects behavior have been topics of scientific research for half a century. In contrast, the widespread acceptance of multiple learning and memory systems has occurred only recently. To date, research on DA function has proceeded mostly independently of research on memory systems. But to predict the effects of DA-related medications on a cognitive behavior, it is vital to know what memory systems are mediating that behavior. This knowledge will indicate which brain regions are relevant and whether those regions receive dopaminergic input. The next step in the prediction process is to determine the density of DAT in these relevant regions. Since DAT concentrations are high only in the basal ganglia, essentially this step reduces to determining whether the basal ganglia play a significant role in mediating the behavior. For behaviors that are mediated within brain regions that receive a rich DA projection but lack high concentrations of DAT, the general prediction should be that even a brief rise in brain DA levels could enhance performance for a period of 20–30 minutes. The major exception is if the person already has high DA levels, perhaps because he or she is a healthy adolescent or under the influence of some DA-related drug, in which case the opposite effect should be expected. For behaviors primarily mediated within the basal ganglia, the same brief rise in DA levels should have little if any effect because DAT will quickly clear the excess DA from basal ganglia synapses. However, a treatment that can raise DA levels for a more prolonged period of time (eg, L-dopa) should improve procedural learning, facilitate the initiation of novel movements, and improve attentional switching. Finally, behaviors mediated in brain regions that do not receive a prominent DA projection (eg, the visual cortex) should be relatively unaffected by DA-related treatments, although this prediction is complicated by the known interactions between DA and acetylcholine.

Acknowledgments

Preparation of this article was supported in part by a grant from the National Institute of Mental Health (2R01MH063760).

Disclosure

The authors report no conflicts of interest in this work.