RESUMEN

Biological accounts of reinforcement learning posit that dopamine encodes reward prediction errors (RPEs), which are multiplied by a learning rate to update state or action values. These values are thought to be represented in synaptic weights in the striatum, and updated by dopamine-dependent plasticity, suggesting that dopamine release might reflect the product of the learning rate and RPE. Here, we leveraged the fact that animals learn faster in volatile environments to characterize dopamine encoding of learning rates. We trained rats on a task with semi-observable states offering different rewards, and rats adjusted how quickly they initiated trials across states using RPEs. Computational modeling and behavioral analyses showed that learning rates were higher following state transitions, and scaled with trial-by-trial changes in beliefs about hidden states, approximating normative Bayesian strategies. Notably, dopamine release in the nucleus accumbens encoded RPEs independent of learning rates, suggesting that dopamine-independent mechanisms instantiate dynamic learning rates.

RESUMEN

Gonadal hormones act throughout the brain 1 , and nearly all neuropsychiatric disorders vary in symptom severity with hormonal fluctuations over the reproductive cycle, gestation, and perimenopause 2-4 . Yet the mechanisms by which hormones influence mental and cognitive processes are unclear. Exogenous estrogenic hormones modulate dopamine signaling in the nucleus accumbens core (NAcc) 5,6 , which instantiates reward prediction errors (RPEs) for reinforcement learning 7-16 . Here we show that endogenous estrogenic hormones enhance RPEs and sensitivity to previous rewards by regulating expression of dopamine reuptake proteins in the NAcc. We trained rats to perform a temporal wagering task with different reward states; rats adjusted how quickly they initiated trials across states, balancing effort against expected rewards. Dopamine release in the NAcc reflected RPEs that predicted and causally in-fluenced subsequent initiation times. When fertile, females more quickly adjusted their initiation times to match reward states due to enhanced dopaminergic RPEs in the NAcc. Proteomics revealed reduced expression of dopamine transporters in fertile stages of the reproductive cycle. Finally, genetic suppression of midbrain estrogen receptors eliminated hormonal modulation of behavior. Estrogenic hormones therefore control the rate of reinforcement learning by regulating RPEs via dopamine reuptake, providing a mechanism by which hormones influence neural dynamics for motivation and learning.

RESUMEN

The value of the environment determines animals' motivational states and sets expectations for error-based learning1-3. How are values computed? Reinforcement learning systems can store or cache values of states or actions that are learned from experience, or they can compute values using a model of the environment to simulate possible futures3. These value computations have distinct trade-offs, and a central question is how neural systems decide which computations to use or whether/how to combine them4-8. Here we show that rats use distinct value computations for sequential decisions within single trials. We used high-throughput training to collect statistically powerful datasets from 291 rats performing a temporal wagering task with hidden reward states. Rats adjusted how quickly they initiated trials and how long they waited for rewards across states, balancing effort and time costs against expected rewards. Statistical modeling revealed that animals computed the value of the environment differently when initiating trials versus when deciding how long to wait for rewards, even though these decisions were only seconds apart. Moreover, value estimates interacted via a dynamic learning rate. Our results reveal how distinct value computations interact on rapid timescales, and demonstrate the power of using high-throughput training to understand rich, cognitive behaviors.

Asunto(s)

RESUMEN

The value of the environment determines animals' motivational states and sets expectations for error-based learning1-3. How are values computed? Reinforcement learning systems can store or "cache" values of states or actions that are learned from experience, or they can compute values using a model of the environment to simulate possible futures3. These value computations have distinct trade-offs, and a central question is how neural systems decide which computations to use or whether/how to combine them4-8. Here we show that rats use distinct value computations for sequential decisions within single trials. We used high-throughput training to collect statistically powerful datasets from 291 rats performing a temporal wagering task with hidden reward states. Rats adjusted how quickly they initiated trials and how long they waited for rewards across states, balancing effort and time costs against expected rewards. Statistical modeling revealed that animals computed the value of the environment differently when initiating trials versus when deciding how long to wait for rewards, even though these decisions were only seconds apart. Moreover, value estimates interacted via a dynamic learning rate. Our results reveal how distinct value computations interact on rapid timescales, and demonstrate the power of using high-throughput training to understand rich, cognitive behaviors.

RESUMEN

We examined the circadian rhythms of locomotor activity in three spider species in the Family Theridiidae under light-dark cycles and constant darkness. Contrary to previous findings in other organisms, we found exceptionally high variability in endogenous circadian period both within and among species. Many individuals exhibited circadian periods much lower (19-22 h) or much higher (26-30 h) than the archetypal circadian period. These results suggest relaxed selection on circadian period as well as an ability to succeed in nature despite a lack of circadian resonance with the 24-h daily cycle. Although displaying similar entrainment waveforms under light-dark cycles, there were remarkable differences among the three species with respect to levels of apparent masking and dispersion of activity under constant dark conditions. These behavioral differences suggest an aspect of chronotype adapted to the particular ecologies of the different species.

Asunto(s)

RESUMEN

PURPOSE: A substantial clinical need exists for an alternate to vitamin K antagonists for treating deep vein thrombosis in many patients. Long-term low-molecular-weight heparin (LMWH), body-weight adjusted, avoids anticoagulant monitoring and may be associated with less bleeding. We evaluated the effectiveness and safety of long-term LMWH compared with vitamin K antagonist therapy in a broad spectrum of patients with proximal vein thrombosis. METHODS: We performed a multicenter, randomized, open-label clinical trial using objective outcome measures comparing therapy for 3 months. Outcomes were assessed at 3 and 12 months. RESULTS: Of 737 patients, 18 of 369 receiving tinzaparin (4.9%) had recurrent venous thromboembolism at 3 months compared with 21 of 368 (5.7%) receiving usual care (absolute difference, -0.8%, 95% confidence interval -4.1-2.4). Hemorrhagic complications occurred less frequently in the LMWH group largely because of less minor bleeding: 48 of 369 patients (13.0%) versus 73 of 368 patients (19.8%) receiving usual-care anticoagulation (absolute difference -6.8%; P = .011; risk ratio = 0.66). New major bleeding events ceased early (by day 23, P = .034) for patients receiving LMWH but persisted throughout the study treatment interval for patients receiving vitamin K antagonist therapy. No mortality advantage was shown for LMWH. CONCLUSION: Our study shows that LMWH is similar in effectiveness to the usual-care vitamin K antagonist treatment for preventing recurrent venous thromboembolism in a broad spectrum of patients. It causes less harm and enhances the clinicians' therapeutic options for patients with proximal deep vein thrombosis. Our findings reported here suggest the possibility of a broader role for long-term LMWH in selected patients.

Asunto(s)

RESUMEN

PURPOSE: A substantial clinical need exists for an alternative to vitamin K antagonists for treating deep-vein thrombosis in cancer patients who are at high risk of both recurrent venous thromboembolism and bleeding. Low-molecular-weight heparin, body-weight adjusted, avoids anticoagulant monitoring and has been shown to be more effective than vitamin-K-antagonist therapy. SUBJECTS AND METHODS: Subjects were patients with cancer and acute symptomatic proximal-vein thrombosis. We performed a multi-centre randomized, open-label clinical trial using objective outcome measures comparing long-term therapeutic tinzaparin subcutaneously once daily with usual-care long-term vitamin-K-antagonist therapy for 3 months. Outcomes were assessed at 3 and 12 months. RESULTS: Of 200 patients, 100 received tinzaparin and 100 received usual care. At 12 months, the usual-care group had an excess of recurrent venous thromboembolism; 16 of 100 (16%) versus 7 of 100 (7%) receiving low-molecular-weight heparin (P=.044; risk ratio=.44; absolute difference -9.0; 95% confidence interval [CI], -21.7 to -0.7). Bleeding, largely minor, occurred in 27 patients (27%) receiving tinzaparin and 24 patients (24%) receiving usual care (absolute difference -3.0; 95% CI, -9.1 to 15.1). In patients without additional risk factors for bleeding at the time of randomization, major bleeding occurred in 0 of 51 patients (0%) receiving tinzaparin and 1 of 48 patients (2.1%) receiving usual care. Mortality at 1 year was high, reflecting the severity of the cancers; 47% in each group died. CONCLUSION: Our findings confirm the limited but benchmark data in the literature that long-term low-molecular-weight heparin is more effective than vitamin-K-antagonist therapy for preventing recurrent venous thromboembolism in patients with cancer and proximal venous thrombosis.

Asunto(s)

RESUMEN

PURPOSE: Clot-burden change in patients receiving anticoagulant therapy, by predicting subsequent recurrent venous thromboembolism, may provide a clinically relevant surrogate endpoint of prognostic importance. The validity of this objective measure is yet to be established. METHODS: A PubMed search was performed to retrieve articles published up to December 2003. We identified 11 randomized trials reported from 1990 to 2003 that met our study identification and selection criteria. Anticoagulant therapy subsequently approved by regulatory affairs was assessed by clot-burden change and the validated outcome measure, long-term venous thromboembolism. Two additional randomized trials, partly meeting the inclusion criteria, were included in the sensitivity analysis. RESULTS: Individual studies suggested a predictive relationship between clot-burden change and likelihood of recurrent venous thromboembolism irrespective of the particular anticoagulant. The summary treatment effects strongly favored the therapy under evaluation and were in harmony for improved clot-burden (relative risk 0.82; 95% CI, 0.76-0.88; P <0.001) and for recurrent venous thromboembolism (relative risk 0.56; 95% CI, 0.42-0.76; P <0.001). The aggregate data show a striking predictive correlation for clot-burden change and subsequent recurrent venous thromboembolism using meta-regression analysis; (correlation = 0.81, P = 0.005) validating quantitative clot-burden assessment. CONCLUSION: Clot-burden change predicts long-term outcome, providing clinically relevant, patient-specific prognostic findings that may guide duration of anticoagulant therapy as well as provide a valid surrogate endpoint for clinical trials of innovative antithrombotic therapy, allowing more efficient trials exposing far fewer patients to the hazards of ineffective therapy than is required for outcome studies. Noninvasive assessment (duplex ultrasonography) of clot-burden change is currently being deployed for use in clinical trials.

Asunto(s)

Asunto(s)