Striatal dopamine, reward, and decision making in schizophrenia

Abstract

Elevated striatal dopamine function is one of the best-established findings in schizophrenia. In this review, we discuss causes and consequences of this striata! dopamine alteration. We first summarize earlier findings regarding striatal reward processing and anticipation using functional neuroimaging. Secondly, we present a series of recent studies that are exemplary for a particular research approach: a combination of theory-driven reinforcement learning and decision-making tasks in combination with computational modeling and functional neuroimaging. We discuss why this approach represents a promising tool to understand underlying mechanisms of symptom dimensions by dissecting the contribution of multiple behavioral control systems working in parallel. We also discuss how it can advance our understanding of the neurobiological implementation of such functions. Thirdly, we review evidence regarding the topography of dopamine dysfunction within the striatum. Finally, we present conclusions and outline important aspects to be considered in future studies.

El aumento de la funcion dopaminergica estriatal es uno de los ha I la zg os mejor establecidos en la esquizofrenia. En esta revision se discuten las causas y consecuencias de esta alteration de la dopamina estriatal. inkialmente se resumen los primeros hallazgos en relation con los procesos de recompensa y anticipation en el estriado empleando neuroimage nes funcionales. En segundo lugar se presenta una serie de estudios recientes que ejemplifican una forma especial de investigation: una combination de aprendizaje por refuerzo basado en la teoria y tareas de toma de detisiones en combination con modelos computacionales y neuroimagenes funcionales. Se discute por que esta forma representa una herramienta promisor'! a para comprender los mecanismos subyacentes de dimensiones sintomatkas mediante la diseccion de la contribution de sistemas de control conductual multiple trabajando en paralelo. Tambien se discute como se puede avanzar en la comprension de la implementation neurobiologica de tales funciones. En tercer lugar, se revisa la evidencia relacionada con la to po gratia de la dis funcion dopaminergica dentro del estriado. Por ultimo se presentan las conclusiones y se destacan aspectos importantes para ser considerados en futuros estudios.

La schizophrénie se caractérise par un taux élevé de dopamine striatale. Dans cet article, nous analysons les causes et les conséquences de cette modification de la dopamine striatale. Nous résumons tout d'abord les premiers résultats obtenus par neuro-imagerie fonctionnelle, concernant l'anticipation et le processus de récompense striatale. Deuxièmement, nous présentons une série d'études récentes emblématiques d'une recherche particulière: une association d'apprentissages par renforcement basés sur la théorie et de tests de prise de décision combinés à une modélisation informatique et à la neuro-imagerie fonctionnelle. Nous analysons pourquoi cette approche représente un outil prometteur pour la compréhension des mécanismes symptomatiques sous-jacents en étudiant l'apport de multiples systèmes de contrôle comportementaux travaillant en parallèle. Nous analysons aussi la façon dont cela peut améliorer notre compréhension de la mise en place neurobiologique de telles fonctions. Troisièmement, nous passons en revue les arguments sur la topographie de la dysfonction de la dopamine au sein du striatum. Enfin nous présentons nos conclusions et soulignons les aspects importants à considérer dans des études ultérieures.

Keywords:

• computational modeling
• computational psychiatry
• decision making
• reward anticipation
• reward prediction error
• schizophrenia
• striatal dopamine

Introduction

The dopamine hypothesis of schizophrenia has been one of the most influential concepts in schizophrenia research.^1,2 Initially based upon the observation that dopamine D₂ receptor antagonists reduce auditory hallucinations and delusions, early neurochemical imaging studies using positron emission tomography (PET) and single-photon emission tomography (SPECT) with radiotracers of the dopamine system provided evidence for elevated striatal dopamine levels in patients suffering from schizophrenia.^3,4 To date, and in line with the early formulation of this hypothesis, elevation of striatal presynaptic dopamine function is one of the best-established findings in schizophrenia (eg, see ref 5; for meta-analyses and for detailed review of different neurochemical PET studies in schizophrenia, see refs 6-10). The dopamine hypothesis was reformulated several times (for a historical summary, see ref 1) to include the notion of prefrontal dopaminergic “hypofrontality” and also as a combined dopamine-glutamate hypothesis.¹¹ A comprehensive neurobiological theory of schizophrenia has described the disease as an illness of disordered synaptic plasticity,^12-14 and disrupted mechanisms of learning have been proposed to play a crucial role both in the development of positive symptoms,^2,15,16 in particular delusions, as well as negative symptoms.¹⁷,¹⁹ In this review, we do not cover the development of dopamine hypothesis itself. Instead, we aim to describe likely consequences and potential causes of the well-documented excess in striatal presynaptic dopamine function by mainly focusing on task-based functional neuroimaging studies to examine motivational and cognitive aspects of learning. In addition, we aim to explain the putative relevance for symptomatology.

Around thesame time as early PET and SPECT studies in schizophrenia, animal research demonstrated that phasic firing patterns of midbrain dopamine neurons signal temporal-difference reward prediction errors.^20,21 This neuronal teaching signal represents the difference between experienced events—eg, rewards—and expectations about the environment and is fundamentally involved in learning.“ ²² Cognitive neuroscience research regarding such dopaminergic learning signals was influenced by computational models of learning and decision making,^23,24 and this has profoundly enhanced our understanding of the underlying neurobiological mechanisms.

This review first summarizes findings from functional neuroimaging studies about reward processing and anticipation and their potential link to elevated striatal dopamine in schizophrenia. As neuroimaging has become ubiquitous in psychiatric research in the last decade, we go on to describe a series of very recent and selected studies that are exemplary for the utilization of a particular research approach: a combination of theory-driven reinforcement learning and decision-making tasks in combination with computational modeling and multimodal functional neuroimaging. We outline why this represents a promising tool to understand the neurobiological implementation of these functions, with a particular focus on the role of striatal dopamine and associated learning signals in schizophrenia.

Striatal activation during reward anticipation and feedback processing

Up to now, many clinical neuroimaging studies have used the monetary incentive delay (MID) task or variations of this task, originally presented in a study by Knutson and colleagues.²⁵ In this task, individuals are presented with abstract geometric cues that are indicative of a subsequent monetary outcome. Before functional magnetic resonance imaging (fMRI), participants are trained on cue-outcome contingencies to minimize the effect of learning. During fMRI, individuals are instructed to press a button as fast as possible to obtain a reward. As task difficulty is adapted according to individual differences in reaction times, all participants experience the expected reward in 70% of their responses. Thus, brain activation can be studied during two phases of the task, the anticipation of a reward and its subsequent delivery, the outcome. This can be applied equivalently for monetary wins and losses as well as for different values of wins and losses. In the following, we mainly focus on anticipation, as this can be measured robustly with this task. The delivery of outcomes in this task remains confounded with aspects of learning via prediction errors and can, in our view, be more appropriately examined by using dynamic learning tasks probing the coding of prediction errors directly (see section on striatal learning signals). However, we also describe findings from the outcome phase, although more briefly.

Reward anticipation

Juckel et al²⁶ applied the MID task for the first time in a group of unmedicated patients suffering from schizophrenia and found reduced ventral striatal activation during reward anticipation (Figure 1A). Consistently, this initial finding was replicated in two larger cohorts of medication-naive, first-episode patients.^27,28 The reduction in ventral striatal activation during reward anticipation appears to be modulated by antipsychotic medication: patients treated with first-generation (or “typical”) antipsychotics (FGAs) showed this reduction, whereas it was not observed in patients treated with second-generation (or “atypical”) antipsychotics (SGAs).²⁹ In line with this finding, switching patients from FGAs to SGAs³⁰ or subsequent treatment of medication-naive, first-episode patients with SGAs³¹ revealed that striatal activation during reward anticipation was not distinguishable from controls when patients were medicated with SGAs. Supporting evidence comes from further cross-sectional studies comparing controls and patients treated with SGAs, which did not find group differences in striatal activation during anticipation of monetary reward.³²“³⁷ Interestingly, a recent study demonstrated ventral striatal activation during reward anticipation to be reduced in first -degree relatives of patients suffering from schizophrenia, thus supporting the idea of reduced ventral striatal activation as an intermediate phenotype of the disorder.³⁸

Figure 1 Ventral striatal dysfunction in unmedicated schizophrenia patients. (A) Reduced ventral striatal activation during reward anticipation using the monetary incentive delay (MID) task as reported by Juckel et al.²⁶ (B) Coding of reward prediction errors in ventral striatal activation during reversal learning in healthy controls. (C) Reduced coding of reward prediction errors in ventral striatal activation in unmedicated schizophrenia patients compared with healthy controls. A is reproduced from ref 26: Juckel G, Schlagenhauf F, Koslowski M, et al. Dysfunction of ventral striatal reward prediction in schizophrenia . Neuroimage. 2006;29(2):409-416. Copyright ©Elsevier Inc. 2005. B and C are reproduced from ref 52: Schlagenhauf F, Huys QJ, Deserno L, et al. Striatal dysfunction during reversal learning in unmedicated schizophrenia patients. Neuroimage. 2014;89:171-180. (No permission required as this article is available under the terms of the Creative Commons Attribution License.)

Potentially consistent with the observation of reduced ventral striatal activation to rewards in unmedicated and medication-naive patients as well as nontreated first-degree relatives (as described above) is the observation of elevated presynaptic striatal dopamine function in these groups.^4,5,39 Thus, one might ask how elevated levels of dopamine could specifically drive the results obtained from functional neuroimaging during reward anticipation. It was found that amphetamine administration (a paradigm to induce massive dopamine release) was accompanied by reduced ventral striatal activation during reward anticipation.⁴⁰ Further evidence was provided by a study demonstrating that reward-induced dopamine release, as measured via radioligand displacement, was positively correlated with ventral striatal activation during reward anticipation.⁴¹ Recently, it was shown that administration of ketamine (a well-established model of psychosis) also reduces reward anticipatory activation in the ventral striatum both in humans and in animals.⁴² This translational “disruption model” ⁴² with acute subanesthetic ketamine has been used for a long time and also revealed elevation in striatal dopamine levels measured via radioligand displacement⁴³ as well as an extra boost of amphetamineinduced striatal dopamine release under ketamine.⁴⁴ More specifically, animal research revealed that N-methyl-D-aspartate (NMDA) receptor blockade in the prefrontal cortex induces elevated release of dopamine in the striatum.^45,46 Using an indirect measurement of glutamate levels via magnetic resonance spectroscopy, prefrontal glutamate was found to be negatively correlated with ventral striatal presynaptic dopamine as measured via F-fluorodihydroxyphenylalanine (FDOPA) PET in healthy human individuals.⁴⁷ In patients and individuals at risk for psychosis, prefrontal activation during working memory was negatively correlated with striatal presynaptic dopamine levels.^48,49 Taken together, these studies provide cumulative evidence for a proposed mechanism¹³: disrupted glutamatergic synaptic plasticity, most likely NMDA-receptor mediated, could lead to elevated striatal dopamine, which interferes with ventral striatal activation during reward anticipation.

Outcome delivery

One explicitly examining the delivery of outcome (reward or punishment) during the MID or similar tasks, it was mainly found that unexpected outcomes elicit abnormal responses in patients. In unmedicated patients, Schlagenhauf et al⁵⁰ observed elevated medial prefrontal activation when an expected reward was not delivered, and found ventral striatal activation to be reduced when comparing trials of successful against unsuccessful avoidance of punishment. In a similarly designed study, investigators reported reduced medial and lateral prefrontal cortex activation when comparing win-versus-loss trials in patients medicated with SGAs.³⁴ In line with this, another study found lower saliency processing in patients medicated with SGAs in right ventrolateral prefrontal cortex.³² Using juice as a reward, one study found abolished striatal and midbrain responses to expected and unexpected rewards in medicated patients.⁵¹ Also supporting the notion of altered coding of expected versus unexpected events in schizophrenia patients, ventral striatal responses to expected rewards were found to be exaggerated, whereas they were blunted in response to unexpected outcomes.⁵¹ Results of these studies have been interpreted as dysfunctional coding of reward prediction errors in schizophrenia; more direct support for this notion comes from studies implementing a computational modeling approach to this question,⁵² which we will discuss in the section on striatal learning signals.

Relation to symptoms

It has long been thought that subcortical dopamine excess mainly contributes to the positive symptoms of the disorder; this notion was built upon the potential of dopamine-enhancing drugs to induce psychosis, and also on the attenuating effects of D₂ antagonists used to treat these symptoms.^1,2 Supporting evidence mainly comes from pharmacological challenge studies in combination with SPECT or PET: while baseline D₂ receptor availability does not differ between groups, amphetamine-induced dopamine release, measured as a reduction in binding potential compared with baseline, is elevated in medication-free patients and this correlates with the change in the degree of induced positive symptoms.^3,4 When using a dopamine depletion paradigm, studies revealed larger increases in D₂ availability in patients than in controls, suggesting elevated endogenous dopamine levels,⁵³ a finding that was predictive for treatment response of positive symptoms to antipsychotic medication. On the basis of these findings, one could have expected a correlation of reduced ventral striatal activation during reward anticipation with positive symptoms. However, the initial study by Juckel et al²⁵ found a significant correlation between ventral striatal activation during reward anticipation and negative symptoms, which is in line with the idea that ventral striatal dopamine dysfunction could impair motivation and increase apathy.²

However, Juckel et al²⁶ also observed a statistical trend for correlation with positive symptoms.²⁶ Subsequently, in medication-naive patients, a correlation between ventral striatal activation during anticipation with positive symptoms was indeed found,^27,28 and the normalization of ventral striatal activation after treatment with amisulpride was correlated with the improvement in positive symptoms.²⁹ A correlation between ventral striatal activation during reward anticipation and positive symptoms was also reported in individuals at risk for psychosis.⁵⁴ However, other studies using the MID task contribute to a mixed picture: while overall group differences in striatal reward anticipation seem to be restricted to medication-free patients,^30,31 some studies found correlations of ventral striatal reward anticipation with negative symptoms,'³⁴ apathy,^32,35and between anticipatory signals in dorsal striatum and avolition.³⁷ Another study found depressive symptoms across five major psychiatric disorders to be correlated with ventral striatal anticipation of reward.⁵⁵ Thus, although reward anticipation as measured with the MID task has reliably provided insight in neurobiological disease correlates, these symptom correlations raise new questions: they call for future studies probing their predictive value regarding the clinical development of patients in longitudinal studies with overall larger sample sizes,⁵⁶ refinement and consensus of clinical and psychometric ratings,⁵⁷ and the elucidation of the exact biological mechanisms underlying such brain activation patterns.^13,58,59 Regarding the last point, it is conceivable that measures such as ventral striatal reward anticipation are most likely conflating affective and cognitive processes at work in parallel. We will return to this issue in the section on striatal learning signals of this article.

The association of dopamine and positive symptoms as described in neurochemical PET and SPECT studies has also influenced the hypothesis of aberrant salience attribution in schizophrenia^2,15,16: whereas elevated presynaptic dopamine may lead to “drowning” of phasic signals assigning salience to motivationally relevant stimuli in the noise of chaotic dopamine release,⁶⁰ this may go along with aberrant salience attribution to otherwise motivationally irrelevant stimuli. Some of the studies discussed above indeed support the idea of reduced neural processing of motivationally relevant stimuli.^26,27 Regarding the elevated processing of motivationally irrelevant stimuli, it was found that schizophrenic patients rate neutral pictures as more salient than controls.⁵⁶ Esslinger et al²⁸ combined the MID task with a second task possibly reflecting salience, and found in an exploratory correlation analysis that pronounced ventral striatal hypoactivation during reward anticipation was associated with salience attribution to neutral stimuli. Another study reported higher dorsolateral prefrontal cortex activation elicited by neutral outcomes during the MID task in unmedicated patients.²⁷ In a study using classical conditioning, ventral striatal activation to stimuli preceding neutral events relative to aversive events was elevated in patients compared with controls,''' and midbrain activation to stimuli preceding neutral events predicted a higher degree of delusion severity.⁶³ The latter study also applied a temporal-difference model to the data and found reduced correlates of prediction errors elicited by aversive events in the midbrain of patients when compared with controls. Intriguingly, when modeling prediction errors for neutral events, this signal was coded in midbrain of patients but not in controls.⁶³

One task that was specifically developed to measure aberrant salience is the “salience attribution test” as described by Roiser et al,⁶⁴ which assesses speeding up of reaction times to relevant and irrelevant dimensions in an instrumental reward paradigm. Initially, this task revealed no overall group differences in reaction time measures of aberrant salience; however, among medicated patients, the severity of delusions was positively correlated with explicit measures of aberrant salience (ratings after the task).⁶⁴ Unmedicated people at risk for psychosis exhibited greater measures of explicit aberrant salience, and this was correlated with severity of delusion-like symptoms.⁶⁵ A recent study for the first time reported elevated implicit aberrant salience in medicated patients compared with healthy controls, based on reaction-time measurements during the task.⁶⁶

Healthy individuals with subclinical delusional experiences showed intermediate aberrant salience levels, though this was not significantly different from healthy controls.⁶⁶ In this study by Pankow et al, implicit aberrant salience was also negatively correlated with neural processing of self-reference in medial prefrontal cortex of medicated patients. Another study in healthy individuals reported a negative correlation between aberrant salience and coding of reward prediction errors in ventral striatum and orbitofrontal cortex.⁶⁷ Both studies support the idea that high levels of aberrant salience can interfere with coding motivationally relevant stimuli and are in line with a dimensional perspective on positive symptoms — in particular, delusions — in the population.^68,69 Nevertheless, these findings require further validation, most importantly among unmedicated patients. Probing the driving variables in these seemingly an ti correlated processes would be of great relevance but can be challenging to achieve experimentally.⁵⁹ Adding to the existing evidence for the potential of dopamine agonists to induce psychosis, Boehme et al⁶⁷ found a positive correlation of ventral striatal dopamine synthesis capacity with implicit aberrant salience in healthy individuals, and a study in Parkinson disease supports this link more directly via a pharmacological design with dopamine agonists.⁷⁰ In studies with patients, a correlation of striatal dopamine synthesis capacity with positive symptoms was found to be less consistent than in the PET studies that challenge dopamine release,⁶ but the great clinical relevance of this measure of presynaptic dopamine function is underlined by the observation that it is elevated before the onset of the illness,^71,72 predicts the transition to schizophrenia,^73,74 and distinguishes treatment responders from nonresponders.⁷⁵ These studies have also pointed toward some important regional specificity with respect to the latter findings, which we will describe in the section of this review addressing topography within the striatum.

Striatal learning signals

So far, we have discussed fMRI studies of reward anticipation and processing mainly using the MID task in conjunction with neurochemical PET studies of striatal dopamine function. As described in the previous section, these neurochemical PET studies have great clinical validity in terms of striatal dopamine being involved in positive symptoms and their treatment response, as well as elevation of striatal dopamine synthesis capacity predicting the course of the disorder and treatment resistance. Regarding task-based fMRI studies, the picture is less clear, in particular due to a lack of longitudinal studies (but see ref 30 for a notable exception) and the inconsistent results regarding symptom correlations. To target the latter question, it seems important to examine the temporal dynamics of reward-based learning and decision making more closely. This can help to identify specific behavioral readouts to address heterogeneity in symptoms. Encouraging the hope that this promise will hold is evidence from cognitive (neuro-) science that many behavioral tasks are not process pure. For example, even simple reinforcement learning tasks put some load on working memory: thus, even many supposedly very basic processes receive influences from higher-order systems⁷⁶ One comprehensive principle stems from so-called dual (or multiple) system theories proposing reflexive versus reflective, automatic versus deliberative, slow versus rapid, or habitual versus goal-directed systems.⁷⁷ It is important to note that these functions evolve dynamically over time, which is also a fundamental principle of brain signals. In recent years, cognitive neuroscience has provided convincing evidence that such systems work in parallel, with considerable interindividual differences, but that their integration is much tighter than previously assumed, behaviorally, but most surprisingly also regarding their neural instantiation.⁷⁸ One illustrative example is that the ventral striatum codes components of both habitual and goal-directed decision-making systems with interindividual differences reflecting the influence of both systems.⁷⁸ This behavioral has been replicated twice^79,80 and also found to be similar with different tasks in other studies.^81-83 The isolation of such components in behavioral and neural data could improve the specificity of relationships between neural data and clinical ratings tremendously. However, this requires a constant mutual refinement of experiments and their analytic tools—in particular, computational models—as they allow for a priori simulation when planning new experimental designs and a posteriori analysis on a trial-by-trial level after the data are collected. Therefore, addressing the temporal dynamics underlying learning and decisionmaking can provide a deeper understanding into symptom-specific and disease-specific processes associated with schizophrenia.⁵⁸ In the following, we will discuss three illustrative examples for this approach,^52,84,85 which has recently been framed within the larger agenda of computational psychiatry.^59,86-88

First, based on a series of studies by Gold, Waltz, and their colleagues in conjunction with neurocomputational models of the dopamine system by Frank and colleagues, there is evidence for a deficit in patients to learn from rewards, whereas learning from punishment remains spared (for a summary, see ref 89). After the initial findings, Gold et al⁸⁵ examined learning from rewards contrasted with learning from punishments by using an instrumental decision-making task in medicated, chronically ill patients. The deficit in learning from rewards was particularly pronounced in patients with high levels of negative symptoms, whereas learning from punishment was relatively intact. A similar finding was also reported in two independent studies.^90,91 Gold et al⁸⁵ also collected data from a post-acquisition test where participants could select previously learned stimuli from newly combined pairs across reward and punishment conditions. In controls, two patterns of behavior were observed in this test phase: (i) “choosing frequent winners over frequent loss avoiders,” demonstrating sensitivity to actual outcome values, which are always higher in the reward condition; (ii) “choosing the frequent loss avoiders over infrequent winners,” showing no sensitivity to actual outcome values. Intriguingly, patients with high levels of negative symptoms showed the latter pattern of behavior. To gain an understanding of the behavioral mechanisms, the authors tested two variants of reinforcement learning to explain the observed behavior. One algorithm was Q-learning, where in reward and punishment conditions, each of the two stimuli available for action is associated with a single value that signals the expected outcome; these expected values are used to compute a prediction error to update values. Therefore, this model represents specific expected outcome value and thus results in “choosing frequent winners over frequent loss avoiders.”⁸⁵ In contrast, a more rigid actor-critic algorithm cannot give this pattern of behavior, but it can explain choices in line with “choosing the frequent loss avoiders over infrequent winners.” This is because it learns only one state value for reward and punishment conditions via prediction errors. Consistent with this, frequent loss avoiders are learned similarly from positive prediction errors as frequent winners. Thus, at the end of learning, frequent loss avoiders are preferred over infrequent winners. As both aspects of behavior were observed across patients and controls, a combination of both algorithms explained the behavior best potentially reflecting influences of a rapid prefrontal-orbitofrontal system (Q-learning) versus a slow striatal system (actor-critic). Strikingly, a weighting parameter, reflecting the relative influence of each mechanism in an individual, was biased toward actor-critic learning in patients with high levels of negative symptoms. This study represents one illustrative example of how specific cognitive decisionmaking tasks can be parsed into different mechanisms at work that map on a specific symptom dimension in patients—in this case, negative symptoms.

The second example stems from work into flexible behavioral adaptation using reversal-learning tasks. Worth mentioning from a clinical perspective, medication-naive, first-episode patients display impaired flexible adjustment of behavior to changes in the environment,⁹² indicating that these deficits are not due to neuroleptic medication or chronic disease course and remain relatively stable over time (6 years), independent of general effects of IQ?³ Targeting the putative mechanism underlying this important behavioral deficit, Schlagenhauf et al'² studied reversal learning during fMRI complemented by computational modeling of the observed choice behavior, notably in unmedicated patients. The computational modeling analysis showed that a flexible, belief-based hidden Markov model was the best fit with the behavioral data when compared with variants of reinforcement-learning models. The hidden Markov model dynamically updates the probability of being in one of the two task states, hence the probability that one of the two stimuli is the one with the higher reward probability. This represents the anti-correlated or counterfactual structure inherent to this reversal task⁵² and to most serial reversal learning tasks.⁹⁴ In line with the observed choices, this model revealed an increased tendency to switch to be a key mechanism of impaired reversal learning in patients. One subgroup of patients who performed the task rather poorly and whose choices could not be explained better than chance by any model also showed reduced sensitivity to rewards and significantly higher levels of positive symptoms. On the neural level, ventral striatal learning signals were reduced in unmedicated patients, a finding based not only on prediction errors from a reinforcement-learning model (Figure IB and 1C), but also on punished trials that were informative about shifts in action-outcome contingencies as inferred by the hidden Markov model.

Interestingly, another subgroup of patients with insight into the task structure showed relatively intact prefrontal activation to these informative punishments, whereas ventral striatal learning signals remained reduced.⁵² This striatal finding is consistent with the assumption of impaired dopamine-dependent striatal error signaling, potentially due to excessive striatal presynaptic dopamine function. This interpretation can also account for the increased tendency to switch in unmedicated patients. This study nicely shows that computational models of choice data acquired during reversal learning can enable the identification of behavioral key mechanisms and helps to isolate neural differences—here, reduced ventral striatal learning signals—as a likely consequence of interindividual variation in striatal dopamine excess. Supporting this view, one study found reduced coding of reward prediction errors in ventral striatum (and medial prefrontal cortex) under methamphetamine challenge in healthy individuals⁹⁵ Consistent with this, ventral striatal reward prediction errors from a reinforcement learning model were reported in two independent studies to be negatively correlated with ventral striatal presynaptic dopamine in healthy individuals.^79,96 In addition to this negative correlation, Deserno et al⁷⁹ could also demonstrate that ventral striatal presynaptic dopamine positively relates to neural signatures of goal-directed behavioral control in the prefrontal cortex, underlining that the computational approach to decision making can also crucially inform multimodal imaging studies — in this case, fMRI and neurochemical PET. This result is consistent with the notion that striatal dopamine may gate environmental information to the prefrontal cortex.⁹⁷ Importantly, the study by Schlagenhauf et al⁵² investigated unmedicated patients; thus, medication effects are unlikely to play a role. Future studies should clarify whether these striatal and behavioral effects generalize to medicated patients. Interestingly, in healthy individuals, pretreatment with amisulpride did not protect against the disruption of ventral striatal reward prediction by methamphetamine.⁹⁵ Behaviorally, increased switching was, however, observed in two independent studies using probabilistic reversal learning in medicated patients, but these studies did not apply computational modeling.^98,99 Systematic investigations of the behavioral and neural processes as a function of medication status remain to be carried out.

The above-described examples have contrasted more flexible versus more rigid learning algorithms to analyze behavioral choice data. There is overwhelming evidence for a cognitive deficit in schizophrenia, eg, with respect to working memory. In line with dual-process theories, it is conceivable that even a very simple motivational task, such as deterministic reinforcement learning, could be influenced by higher cognitive functions, such as working memory.⁷⁵ Thus, it would be of great interest to explore to what extent reinforcement learning deficits can actually be attributed to workingmemory impairments. This requires appropriate experimental designs and computational analysis to dissociate the underlying mechanisms. One such example is the study by Collins et al in patients receiving long-term medication; by using a deterministic reinforcement learning task with systematic variation of set size, the authors found that the overall observed deficit in learning was entirely attributed to parameters derived from the working-memory component of the model. Validity of the working-memory parameters obtained from computational modeling was demonstrated via principle component analysis on model parameters and a correlation of the identified working memory component with independent measures of working memory.⁸⁴ This exemplary study is of importance, as cognitive deficits and negative symptoms share some variance and are both predictive for clinical outcome.^100,101 A further dissection of these mechanisms could greatly improve our understanding of symptom dimensions.

Topography within the striatum

So far, we have discussed likely consequences, and some potential causes, of striatal presynaptic dopamine excess as it is observed in schizophrenia. Currently, two meta-analyses have confirmed this finding when looking at the entire striatum.^6,8 However, there is considerable variability within the striatum. The so-called capacity for dopamine release,⁹ a summary label for FDOIA PET and the challenge studies, shows higher signals along a dorsal to ventral, or sensorimotor to limbic, gradient in healthy individuals.¹⁰² Most of the discussed task-based functional imaging studies revealed the ventral striatum (or limbic striatum, Figure 2 ) to code reward anticipation as well as prediction errors. This appears plausible based on the dense innervation that this striatal subregion receives from the ventral tegmental area (Figure 2), where phasic dopamine bursts signal a reward prediction error.²¹ Interestingly, although presynaptic striatal dopamine is most likely elevated in all subregions, there is some evidence that this effect is most pronounced in more dorsorostral parts,¹⁰³ also named the associative striatum (Figure 2). Interestingly, in the latter study, this finding was related to negative symptoms in patients. This clearly points toward the idea that this observation results from interactions within frontostriatal loops (Figure 2). In this regard, the associative striatum has relatively specific connections with the frontal cortex.¹⁰⁴ A dysfunction of this region has been proposed to be a key player in schizophrenia,¹⁰⁵ in particular for cognitive deficits,^106,107 and was shown to be related to striatal hyperdopaminergia in patients⁴⁸ and at-risk individuals.⁴⁹

Figure 2. Schematic illustration of frontostriatal connections with a focus on the ventral striatum. Blue arrows, inputs; gray arrows, outputs; Amy, amygdala; BNST, bed nucleus stria terminalis; dACC, dorsal anterior cingul ate cortex; DPFC, dorsal prefrontal cortex; Hipp, hippocampus; Hypo, hypothalamus; MD, mediodorsal nucleus of the thalamus; NB, nucleus basal is; OFC, orbitofrontal cortex; PPT, pedunculopontine nucleus; S, shell; SN, substantia nigra; THAL, thalamus; VP, ventral pallidum; VTA, ventral tegmental area; vmPFC, ventral medial prefrontal cortex. Reprinted from ref 104: Haber SN, Knutson B. The reward circuit: linking primate anatomy and human imaging. Neuropsychopharmacology. 201 0:35(1 ):4-26. Copyright © Nature Publishing Group, 2009

Animal research demonstrated that experimentally induced overexpression of D₂ receptors can result in prefrontal dysfunction associated with low prefrontal dopamine function.¹⁰⁸ However, findings regarding changes in D₂ receptors in schizophrenia patients remain debatable and are likely secondary to antipsychotic treatment.⁶ It was also shown that developmentally early temporolimbic lesions result in an impaired prefrontal regulation of striatal dopamine function.^109,110 Studies focusing on glutamate (NMDA receptor)-mediated plasticity also support this notion of a secondary striatal dopamine excess.¹² Recent advances in neurochemical PET imaging provided important results by showing that cortical and extrastriatal subcortical dopamine release is widely blunted.¹¹¹ This supports the idea of an overall cortical dopamine deficit in addition to the well-known presynaptic elevation in the striatum; however, these findings are inconclusive regarding a causal chain of these alterations. One well-known developmental animal model of schizophrenia has established a key, most likely causal, role for increased glutamate release from the ventral hippocampus to the ventral striatum underlying excessive dopamine dysregulation.¹¹² This emphasizes the role of hippocampal-striatal connectivity, again supportive of striatal dopamine dysregulation being secondary to dysfunctions in other brain areas. Predictions of this model are being translated to human studies.¹¹³ In line with this, a recent combined magnetic resonance spectroscopy and FDOPA PET study found local (ventral) striatal glutamate levels, probably conveyed by hippocampal inputs, to be positively correlated with ventral striatal presynaptic dopamine in healthy individuals.⁴⁷ Overall, the presented view fits well with the neurodevelopmental hypothesis^114-116 of schizophrenia and with recent twin studies,¹¹⁷ which point toward important shared genetic and environmental effects on neuromodulatory function in frontohippocampal and frontostriatal networks.

Conclusion and outlook

In this review, we have discussed functional and behavioral consequences of elevated striatal dopamine function as observed in schizophrenia. Consistently, ventral striatal reward anticipation and ventral striatal coding of reward prediction errors were reduced in unmedicated patients.^26,52 For studies of reward anticipation and processing, there is a mixed picture with respect to clinical correlations. In the section on striatal learning signals, we have suggested specifically designed decision-making tasks in conjunction with computational models of choice behavior as a promising way to overcome these inconsistencies and have discussed three exemplary studies using this approach in patients.^52,84,85 Delineating more clearly how such computationally specified cognitive and motivational subprocesses relate to the functional topography of dopamine within the striatum would be illuminating. This requires a translational bridge between animal and human research as well as computational and clinically guided neuroscience. Computational psychiatry is a promising framework to achieve this.^59,86-88

In this brief outlook section, we would like to point toward three aspects that appear of particular importance for future studies. Firstly, using decision-making tasks with computational modeling has proven fruitful, as described in the section on striatal learning signals. This can help to dissociate neural signatures in fMRI data, and multimodal imaging with neurochemical measurements have also proven to further reveal specific associations with neurotransmitter systems⁷⁴ and are applicable in patients.^48,49 Secondly, antipsychotic medication remains a crucial issue. Although it is worth emphasizing the importance of still relatively rare studies with unmedicated or medication-naive patients,^{27,52,111 111} systematic comparisons of medication status are warranted. In combination with the computational approach to decision making pursued here, this could provide important insight regarding treatment response by subgrouping patients with respect to specific behavioral mechanisms or the neuronal implementation of these mechanisms.^88,118 In fact, knowing “who should be treated how” based on a biological stratification may greatly improve clinical effects. This seems promising because such decision-making models mirror important aspects of the function of the main neuromodulatory systems targeted by the well-known antipsychotics: most prominently dopamine,^20,79 but also serotonin¹¹⁹ and acetylcholine.¹²⁰ Furthermore, such tasks and tools are feasible and patient friendly. In contrast, although neurochemical imaging has substantially improved our insight into the dopamine system in schizophrenia, these measurements are expensive and are associated with a high logistic burden, which renders any future clinical application unlikely. Thirdly, despite the promising advances based on computational approaches, there is unfortunately a lack of longitudinal studies in psychiatric neuroscience in general. These studies are urgently required to assess the relevance for clinical outcome prediction of any of the measures discussed here. Thus, a combination of computational approaches to reward- based decision making with functional neuroimaging in longitudinal multicenter studies, also in adolescents or at-risk individuals,¹²¹ can pave the way to promising progress.

Selected abbreviations and acronyms

FDOPA	=	18F-fluorodihydroxyphenylalanine
FGA	=	first-generation antipsychotic
fMRI	=	functional magnetic resonance imaging
MID	=	monetary incentive delay
NMDA	=	N-methyl-D-aspartate
PET	=	positron emission tomography
SGA	=	second-generation antipsychotic
SPECT	=	single-photon emission tomography

This study was supported by the Max Planck Society and grants from the German Research Foundation awarded to FS (DFG SCH LI 969/1-1, DFG SCHL 1969/2-1).

Finantial disdosures: All authors report no biomedical financial interests or potential conflicts of interest.