RESUMEN

The tail of the striatum (TS) is increasingly recognized as a unique subdivision of the striatum, characterized by its dense sensory inputs and projections received from a distinct group of dopamine neurons. Separate lines of research have characterized the functional role of TS, and TS-projecting dopamine neurons, in three realms: saccadic eye movement towards valuable visual stimuli; tone-guided choice between two options; and defensive responses to threatening stimuli. We propose a framework for reconciling these diverse roles as varied implementations of a conserved response to salient stimuli, with dopamine in TS providing a teaching signal to promote quick attentional shifts that facilitate stimulus-driven orientation and/or avoidance.

RESUMEN

Midbrain dopamine neurons are thought to signal reward prediction errors (RPEs), but the mechanisms underlying RPE computation, particularly the contributions of different neurotransmitters, remain poorly understood. Here, we used a genetically encoded glutamate sensor to examine the pattern of glutamate inputs to dopamine neurons in mice. We found that glutamate inputs exhibit virtually all of the characteristics of RPE rather than conveying a specific component of RPE computation, such as reward or expectation. Notably, whereas glutamate inputs were transiently inhibited by reward omission, they were excited by aversive stimuli. Opioid analgesics altered dopamine negative responses to aversive stimuli into more positive responses, whereas excitatory responses of glutamate inputs remained unchanged. Our findings uncover previously unknown synaptic mechanisms underlying RPE computations; dopamine responses are shaped by both synergistic and competitive interactions between glutamatergic and GABAergic inputs to dopamine neurons depending on valences, with competitive interactions playing a role in responses to aversive stimuli.

Asunto(s)

Neuronas Dopaminérgicas , Ácido Glutámico , Ratones , Animales , Neuronas Dopaminérgicas/fisiología , Dopamina/fisiología , Recompensa , Mesencéfalo , Área Tegmental Ventral/fisiología

RESUMEN

Dopamine (DA) plays multiple roles in a wide range of physiological and pathological processes via a large network of dopaminergic projections. To dissect the spatiotemporal dynamics of DA release in both dense and sparsely innervated brain regions, we developed a series of green and red fluorescent G-protein-coupled receptor activation-based DA (GRABDA) sensors using a variety of DA receptor subtypes. These sensors have high sensitivity, selectivity and signal-to-noise ratio with subsecond response kinetics and the ability to detect a wide range of DA concentrations. We then used these sensors in mice to measure both optogenetically evoked and behaviorally relevant DA release while measuring neurochemical signaling in the nucleus accumbens, amygdala and cortex. Using these sensors, we also detected spatially resolved heterogeneous cortical DA release in mice performing various behaviors. These next-generation GRABDA sensors provide a robust set of tools for imaging dopaminergic activity under a variety of physiological and pathological conditions.

Asunto(s)

Dopamina , Núcleo Accumbens , Ratones , Animales , Núcleo Accumbens/fisiología , Receptores Dopaminérgicos , Encéfalo , Receptores Acoplados a Proteínas G

RESUMEN

Midbrain dopamine neurons are thought to signal reward prediction errors (RPEs) but the mechanisms underlying RPE computation, particularly contributions of different neurotransmitters, remain poorly understood. Here we used a genetically-encoded glutamate sensor to examine the pattern of glutamate inputs to dopamine neurons. We found that glutamate inputs exhibit virtually all of the characteristics of RPE, rather than conveying a specific component of RPE computation such as reward or expectation. Notably, while glutamate inputs were transiently inhibited by reward omission, they were excited by aversive stimuli. Opioid analgesics altered dopamine negative responses to aversive stimuli toward more positive responses, while excitatory responses of glutamate inputs remained unchanged. Our findings uncover previously unknown synaptic mechanisms underlying RPE computations; dopamine responses are shaped by both synergistic and competitive interactions between glutamatergic and GABAergic inputs to dopamine neurons depending on valences, with competitive interactions playing a role in responses to aversive stimuli.

RESUMEN

Dopamine (DA) plays multiple roles in a wide range of physiological and pathological processes via a vast network of dopaminergic projections. To fully dissect the spatiotemporal dynamics of DA release in both dense and sparsely innervated brain regions, we developed a series of green and red fluorescent GPCR activation-based DA (GRABDA) sensors using a variety of DA receptor subtypes. These sensors have high sensitivity, selectivity, and signal-to-noise properties with subsecond response kinetics and the ability to detect a wide range of DA concentrations. We then used these sensors in freely moving mice to measure both optogenetically evoked and behaviorally relevant DA release while measuring neurochemical signaling in the nucleus accumbens, amygdala, and cortex. Using these sensors, we also detected spatially resolved heterogeneous cortical DA release in mice performing various behaviors. These next-generation GRABDA sensors provide a robust set of tools for imaging dopaminergic activity under a variety of physiological and pathological conditions.

RESUMEN

Social grouping increases survival in many species, including humans1,2. By contrast, social isolation generates an aversive state (loneliness) that motivates social seeking and heightens social interaction upon reunion3-5. The observed rebound in social interaction triggered by isolation suggests a homeostatic process underlying the control of social drive, similar to that observed for physiological needs such as hunger, thirst or sleep3,6. In this study, we assessed social responses in multiple mouse strains and identified the FVB/NJ line as exquisitely sensitive to social isolation. Using FVB/NJ mice, we uncovered two previously uncharacterized neuronal populations in the hypothalamic preoptic nucleus that are activated during social isolation and social rebound and that orchestrate the behavior display of social need and social satiety, respectively. We identified direct connectivity between these two populations of opposite function and with brain areas associated with social behavior, emotional state, reward, and physiological needs, and showed that animals require touch to assess the presence of others and fulfill their social need, thus revealing a brain-wide neural system underlying social homeostasis. These findings offer mechanistic insight into the nature and function of circuits controlling instinctive social need and for the understanding of healthy and diseased brain states associated with social context.

RESUMEN

Animals both explore and avoid novel objects in the environment, but the neural mechanisms that underlie these behaviors and their dynamics remain uncharacterized. Here, we used multi-point tracking (DeepLabCut) and behavioral segmentation (MoSeq) to characterize the behavior of mice freely interacting with a novel object. Novelty elicits a characteristic sequence of behavior, starting with investigatory approach and culminating in object engagement or avoidance. Dopamine in the tail of the striatum (TS) suppresses engagement, and dopamine responses were predictive of individual variability in behavior. Behavioral dynamics and individual variability are explained by a reinforcement-learning (RL) model of threat prediction in which behavior arises from a novelty-induced initial threat prediction (akin to "shaping bonus") and a threat prediction that is learned through dopamine-mediated threat prediction errors. These results uncover an algorithmic similarity between reward- and threat-related dopamine sub-systems.

Asunto(s)

Cuerpo Estriado , Dopamina , Animales , Ratones , Dopamina/fisiología , Cuerpo Estriado/fisiología , Refuerzo en Psicología , Recompensa , Aprendizaje/fisiología

RESUMEN

A large body of evidence has indicated that the phasic responses of midbrain dopamine neurons show a remarkable similarity to a type of teaching signal (temporal difference (TD) error) used in machine learning. However, previous studies failed to observe a key prediction of this algorithm: that when an agent associates a cue and a reward that are separated in time, the timing of dopamine signals should gradually move backward in time from the time of the reward to the time of the cue over multiple trials. Here we demonstrate that such a gradual shift occurs both at the level of dopaminergic cellular activity and dopamine release in the ventral striatum in mice. Our results establish a long-sought link between dopaminergic activity and the TD learning algorithm, providing fundamental insights into how the brain associates cues and rewards that are separated in time.

Asunto(s)

Dopamina , Recompensa , Animales , Señales (Psicología) , Dopamina/fisiología , Neuronas Dopaminérgicas/fisiología , Aprendizaje Automático , Mesencéfalo , Ratones

RESUMEN

Different regions of the striatum regulate different types of behavior. However, how dopamine signals differ across striatal regions and how dopamine regulates different behaviors remain unclear. Here, we compared dopamine axon activity in the ventral, dorsomedial, and dorsolateral striatum, while mice performed a perceptual and value-based decision task. Surprisingly, dopamine axon activity was similar across all three areas. At a glance, the activity multiplexed different variables such as stimulus-associated values, confidence, and reward feedback at different phases of the task. Our modeling demonstrates, however, that these modulations can be inclusively explained by moment-by-moment changes in the expected reward, that is the temporal difference error. A major difference between areas was the overall activity level of reward responses: reward responses in dorsolateral striatum were positively shifted, lacking inhibitory responses to negative prediction errors. The differences in dopamine signals put specific constraints on the properties of behaviors controlled by dopamine in these regions.

Asunto(s)

Axones/fisiología , Cuerpo Estriado/fisiología , Toma de Decisiones/fisiología , Neuronas Dopaminérgicas/fisiología , Animales , Femenino , Masculino , Ratones , Ratones Endogámicos C57BL , Odorantes , Refuerzo en Psicología , Recompensa , Olfato

RESUMEN

Rapid phasic activity of midbrain dopamine neurons is thought to signal reward prediction errors (RPEs), resembling temporal difference errors used in machine learning. However, recent studies describing slowly increasing dopamine signals have instead proposed that they represent state values and arise independent from somatic spiking activity. Here we developed experimental paradigms using virtual reality that disambiguate RPEs from values. We examined dopamine circuit activity at various stages, including somatic spiking, calcium signals at somata and axons, and striatal dopamine concentrations. Our results demonstrate that ramping dopamine signals are consistent with RPEs rather than value, and this ramping is observed at all stages examined. Ramping dopamine signals can be driven by a dynamic stimulus that indicates a gradual approach to a reward. We provide a unified computational understanding of rapid phasic and slowly ramping dopamine signals: dopamine neurons perform a derivative-like computation over values on a moment-by-moment basis.

Asunto(s)

Dopamina/metabolismo , Transducción de Señal , Potenciales de Acción/fisiología , Animales , Axones/metabolismo , Calcio/metabolismo , Señalización del Calcio , Cuerpo Celular/metabolismo , Señales (Psicología) , Neuronas Dopaminérgicas/fisiología , Fluorometría , Masculino , Ratones Endogámicos C57BL , Modelos Neurológicos , Estimulación Luminosa , Recompensa , Sensación , Factores de Tiempo , Área Tegmental Ventral/metabolismo , Realidad Virtual

RESUMEN

The prefrontal cortex modifies the sensory system to focus attention. In this issue of Neuron, Nakajima et al. (2019) fill the gap between the prefrontal cortex and the sensory system with an overlooked basal ganglia pathway.

Asunto(s)

Ganglios Basales , Corteza Prefrontal , Atención , Filtrado Sensorial , Tálamo

RESUMEN

Midbrain dopamine neurons are well known for their role in reward-based reinforcement learning. We found that the activity of dopamine axons in the posterior tail of the striatum (TS) scaled with the novelty and intensity of external stimuli, but did not encode reward value. We demonstrated that the ablation of TS-projecting dopamine neurons specifically inhibited avoidance of novel or high-intensity stimuli without affecting animals' initial avoidance responses, suggesting a role in reinforcement rather than simply in avoidance itself. Furthermore, we found that animals avoided optogenetic activation of dopamine axons in TS during a choice task and that this stimulation could partially reinstate avoidance of a familiar object. These results suggest that TS-projecting dopamine neurons reinforce avoidance of threatening stimuli. More generally, our results indicate that there are at least two axes of reinforcement learning using dopamine in the striatum: one based on value and one based on external threat.

Asunto(s)

Reacción de Prevención/fisiología , Cuerpo Estriado/citología , Neuronas Dopaminérgicas/fisiología , Vías Nerviosas/fisiología , Refuerzo en Psicología , Animales , Benzazepinas/farmacología , Encéfalo/citología , Encéfalo/efectos de los fármacos , Encéfalo/fisiología , Channelrhodopsins/genética , Channelrhodopsins/metabolismo , Fosfodiesterasas de Nucleótidos Cíclicos Tipo 1/genética , Fosfodiesterasas de Nucleótidos Cíclicos Tipo 1/metabolismo , Dopaminérgicos/farmacología , Proteínas de Transporte de Dopamina a través de la Membrana Plasmática/genética , Proteínas de Transporte de Dopamina a través de la Membrana Plasmática/metabolismo , Neuronas Dopaminérgicas/efectos de los fármacos , Proteínas Luminiscentes/genética , Proteínas Luminiscentes/metabolismo , Ratones , Ratones Endogámicos C57BL , Ratones Transgénicos , Oxidopamina/farmacología , Unión Proteica/efectos de los fármacos , Reconocimiento en Psicología/efectos de los fármacos , Proteína 2 de Transporte Vesicular de Glutamato/genética , Proteína 2 de Transporte Vesicular de Glutamato/metabolismo

RESUMEN

Dopamine and serotonin play critical roles in flexible behaviors and are related to various psychiatric and motor disorders. This paper reviews the global organization of dopamine and serotonin systems through recent findings using a modified rabies virus. We first introduce methods for comprehensive mapping of monosynaptic inputs. We then describe quantitative comparisons across the data regarding monosynaptic inputs to dopamine neurons versus serotonin neurons. There is surprising similarity between the input to dopamine neurons in the ventral tegmental area (VTA) and the input to serotonin neurons in the dorsal raphe (DR), suggesting functional interactions between these systems. We next introduce studies of mapping monosynaptic inputs to subpopulations of dopamine neurons specified by their projection targets. It was found that the population of dopamine neurons that project to the tail of the striatum (TS) forms an anatomically distinct outlier, suggesting a unique function. From these series of anatomical studies, we propose that there are three information flows that regulate these neuromodulatory systems: the midline stream to serotonin neurons in median raphe (MR) and B6, the central stream to value-coding dopamine neurons and serotonin neurons in rostral DR, and the lateral stream to TS-projecting dopamine neurons. Finally we introduce a new approach to investigate firing patterns of monosynaptic inputs to dopamine neurons in behaving animals. Combining anatomical and physiological findings, we propose that within the central stream, dopamine neurons broadcast a central teaching signal rather than personal teaching signals to multiple brain areas, which are computed in a redundant way in multi-layered neural circuits. Examination of global organization of the dopamine and serotonin circuits not only revealed the complexity of the systems but also revealed some principles of their organization. We will also discuss limitations, practical issues and the possibility of future improvements of the rabies virus-mediated tracing system.

Asunto(s)

Neuronas Dopaminérgicas/fisiología , Vectores Genéticos , Virus de la Rabia/genética , Neuronas Serotoninérgicas/fisiología , Sinapsis/fisiología , Animales , Encéfalo/fisiología , Vías Nerviosas/fisiología

RESUMEN

The ability to predict future outcomes increases the fitness of the animal. Decades of research have shown that dopamine neurons broadcast reward prediction error (RPE) signals-the discrepancy between actual and predicted reward-to drive learning to predict future outcomes. Recent studies have begun to show, however, that dopamine neurons are more diverse than previously thought. In this review, we will summarize a series of our studies that have shown unique properties of dopamine neurons projecting to the posterior "tail" of the striatum (TS) in terms of anatomy, activity, and function. Specifically, TS-projecting dopamine neurons are activated by a subset of negative events including threats from a novel object, send prediction errors for external threats, and reinforce avoidance behaviors. These results indicate that there are at least two axes of dopamine-mediated reinforcement learning in the brain-one learning from canonical RPEs and another learning from threat prediction errors. We argue that the existence of multiple learning systems is an adaptive strategy that makes possible each system optimized for its own needs. The compartmental organization in the mammalian striatum resembles that of a dopamine-recipient area in insects (mushroom body), pointing to a principle of dopamine function conserved across phyla.

RESUMEN

Although modified rabies viruses have emerged as a powerful tool for tracing the inputs to genetically defined populations of neurons, the toxicity of the virus has limited its utility. A recent study employed a self-inactivating rabies (SiR) virus that enables recording or manipulation of targeted neurons for months.

Asunto(s)

ADN , Virus de la Rabia/genética , Neuronas

RESUMEN

Dopamine neurons facilitate learning by calculating reward prediction error, or the difference between expected and actual reward. Despite two decades of research, it remains unclear how dopamine neurons make this calculation. Here we review studies that tackle this problem from a diverse set of approaches, from anatomy to electrophysiology to computational modeling and behavior. Several patterns emerge from this synthesis: that dopamine neurons themselves calculate reward prediction error, rather than inherit it passively from upstream regions; that they combine multiple separate and redundant inputs, which are themselves interconnected in a dense recurrent network; and that despite the complexity of inputs, the output from dopamine neurons is remarkably homogeneous and robust. The more we study this simple arithmetic computation, the knottier it appears to be, suggesting a daunting (but stimulating) path ahead for neuroscience more generally.

Asunto(s)

Encéfalo/fisiología , Dopamina/fisiología , Aprendizaje/fisiología , Red Nerviosa/fisiología , Recompensa , Animales , Humanos , Vías Nerviosas/fisiología

RESUMEN

Dopamine neurons are thought to encode novelty in addition to reward prediction error (the discrepancy between actual and predicted values). In this study, we compared dopamine activity across the striatum using fiber fluorometry in mice. During classical conditioning, we observed opposite dynamics in dopamine axon signals in the ventral striatum ('VS dopamine') and the posterior tail of the striatum ('TS dopamine'). TS dopamine showed strong excitation to novel cues, whereas VS dopamine showed no responses to novel cues until they had been paired with a reward. TS dopamine cue responses decreased over time, depending on what the cue predicted. Additionally, TS dopamine showed excitation to several types of stimuli including rewarding, aversive, and neutral stimuli whereas VS dopamine showed excitation only to reward or reward-predicting cues. Together, these results demonstrate that dopamine novelty signals are localized in TS along with general salience signals, while VS dopamine reliably encodes reward prediction error.

Asunto(s)

Señales (Psicología) , Neuronas Dopaminérgicas/fisiología , Estriado Ventral/fisiología , Animales , Dopamina/metabolismo , Fluorometría , Ratones , Simpatomiméticos/metabolismo

RESUMEN

Dopamine is thought to regulate learning from appetitive and aversive events. Here we examined how optogenetically-identified dopamine neurons in the lateral ventral tegmental area of mice respond to aversive events in different conditions. In low reward contexts, most dopamine neurons were exclusively inhibited by aversive events, and expectation reduced dopamine neurons' responses to reward and punishment. When a single odor predicted both reward and punishment, dopamine neurons' responses to that odor reflected the integrated value of both outcomes. Thus, in low reward contexts, dopamine neurons signal value prediction errors (VPEs) integrating information about both reward and aversion in a common currency. In contrast, in high reward contexts, dopamine neurons acquired a short-latency excitation to aversive events that masked their VPE signaling. Our results demonstrate the importance of considering the contexts to examine the representation in dopamine neurons and uncover different modes of dopamine signaling, each of which may be adaptive for different environments.

Asunto(s)

Neuronas Dopaminérgicas/fisiología , Aprendizaje , Mesencéfalo/fisiología , Recompensa , Animales , Modelos Neurológicos , Ratas

RESUMEN

Dopamine neurons encode the difference between actual and predicted reward, or reward prediction error (RPE). Although many models have been proposed to account for this computation, it has been difficult to test these models experimentally. Here we established an awake electrophysiological recording system, combined with rabies virus and optogenetic cell-type identification, to characterize the firing patterns of monosynaptic inputs to dopamine neurons while mice performed classical conditioning tasks. We found that each variable required to compute RPE, including actual and predicted reward, was distributed in input neurons in multiple brain areas. Further, many input neurons across brain areas signaled combinations of these variables. These results demonstrate that even simple arithmetic computations such as RPE are not localized in specific brain areas but, rather, distributed across multiple nodes in a brain-wide network. Our systematic method to examine both activity and connectivity revealed unexpected redundancy for a simple computation in the brain.

Asunto(s)

Condicionamiento Clásico/fisiología , Neuronas Dopaminérgicas/fisiología , Vías Nerviosas/fisiología , Recompensa , Sinapsis/fisiología , Animales , Mapeo Encefálico/psicología , Cuerpo Estriado/citología , Cuerpo Estriado/fisiología , Masculino , Ratones

RESUMEN

Combining rabies-virus tracing, optical clearing (CLARITY), and whole-brain light-sheet imaging, we mapped the monosynaptic inputs to midbrain dopamine neurons projecting to different targets (different parts of the striatum, cortex, amygdala, etc) in mice. We found that most populations of dopamine neurons receive a similar set of inputs rather than forming strong reciprocal connections with their target areas. A common feature among most populations of dopamine neurons was the existence of dense 'clusters' of inputs within the ventral striatum. However, we found that dopamine neurons projecting to the posterior striatum were outliers, receiving relatively few inputs from the ventral striatum and instead receiving more inputs from the globus pallidus, subthalamic nucleus, and zona incerta. These results lay a foundation for understanding the input/output structure of the midbrain dopamine circuit and demonstrate that dopamine neurons projecting to the posterior striatum constitute a unique class of dopamine neurons regulated by different inputs.

Asunto(s)

Encéfalo/anatomía & histología , Cuerpo Estriado/anatomía & histología , Neuronas Dopaminérgicas/fisiología , Vías Nerviosas/anatomía & histología , Vías Nerviosas/fisiología , Animales , Ratones