TY - JOUR A1 - Schad, Daniel A1 - Juenger, Elisabeth A1 - Sebold, Miriam A1 - Garbusow, Maria A1 - Bernhardt, Nadine A1 - Javadi, Amir-Homayoun A1 - Zimmermann, Ulrich S. A1 - Smolka, Michael N. A1 - Heinz, Andreas A1 - Rapp, Michael A. A1 - Huys, Quentin J. M. T1 - Processing speed enhances model-based over model-free reinforcement learning in the presence of high working memory functioning JF - Frontiers in psychology N2 - Theories of decision-making and its neural substrates have long assumed the existence of two distinct and competing valuation systems, variously described as goal-directed vs. habitual, or, more recently and based on statistical arguments, as model-free vs. model-based reinforcement-learning. Though both have been shown to control choices, the cognitive abilities associated with these systems are under ongoing investigation. Here we examine the link to cognitive abilities, and find that individual differences in processing speed covary with a shift from model-free to model-based choice control in the presence of above-average working memory function. This suggests shared cognitive and neural processes; provides a bridge between literatures on intelligence and valuation; and may guide the development of process models of different valuation components. Furthermore, it provides a rationale for individual differences in the tendency to deploy valuation systems, which may be important for understanding the manifold neuropsychiatric diseases associated with malfunctions of valuation. KW - decision-making KW - reward KW - cognitive abilities KW - model-based and model-free learning KW - fluid intelligence KW - habitual and goal-directed system Y1 - 2014 U6 - https://doi.org/10.3389/fpsyg.2014.01450 SN - 1664-1078 VL - 5 PB - Frontiers Research Foundation CY - Lausanne ER - TY - JOUR A1 - Sebold, Miriam A1 - Garbusow, Maria A1 - Jetzschmann, P. A1 - Schad, Daniel A1 - Nebe, S. A1 - Schlagenhauf, Florian A1 - Heinz, A. A1 - Rapp, Michael A. A1 - Romanczuk-Seiferth, Nina T1 - Reward and avoidance learning in the context of aversive environments and possible implications for depressive symptoms JF - Psychopharmacology N2 - Background Aversive stimuli in the environment influence human actions. This includes valence-dependent influences on action selection, e.g., increased avoidance but decreased approach behavior. However, it is yet unclear how aversive stimuli interact with complex learning and decision-making in the reward and avoidance domain. Moreover, the underlying computational mechanisms of these decision-making biases are unknown. Methods To elucidate these mechanisms, 54 healthy young male subjects performed a two-step sequential decision-making task, which allows to computationally model different aspects of learning, e.g., model-free, habitual, and model-based, goal-directed learning. We used a within-subject design, crossing task valence (reward vs. punishment learning) with emotional context (aversive vs. neutral background stimuli). We analyzed choice data, applied a computational model, and performed simulations. Results Whereas model-based learning was not affected, aversive stimuli interacted with model-free learning in a way that depended on task valence. Thus, aversive stimuli increased model-free avoidance learning but decreased model-free reward learning. The computational model confirmed this effect: the parameter lambda that indicates the influence of reward prediction errors on decision values was increased in the punishment condition but decreased in the reward condition when aversive stimuli were present. Further, by using the inferred computational parameters to simulate choice data, our effects were captured. Exploratory analyses revealed that the observed biases were associated with subclinical depressive symptoms. Conclusion Our data show that aversive environmental stimuli affect complex learning and decision-making, which depends on task valence. Further, we provide a model of the underlying computations of this affective modulation. Finally, our finding of increased decision-making biases in subjects reporting subclinical depressive symptoms matches recent reports of amplified Pavlovian influences on action selection in depression and suggests a potential vulnerability factor for mood disorders. We discuss our findings in the light of the involvement of the neuromodulators serotonin and dopamine. KW - Reward learning KW - Avoidance learning KW - Reinforcement learning KW - Computational psychiatry KW - Decision-making KW - Affective modulation KW - Depression symptoms Y1 - 2019 U6 - https://doi.org/10.1007/s00213-019-05299-9 SN - 0033-3158 SN - 1432-2072 VL - 236 IS - 8 SP - 2437 EP - 2449 PB - Springer CY - New York ER - TY - JOUR A1 - Friedel, Eva A1 - Sebold, Miriam A1 - Kuitunen-Paul, Sören A1 - Nebe, Stephan A1 - Veer, Ilya M. A1 - Zimmermann, Ulrich S. A1 - Schlagenhauf, Florian A1 - Smolka, Michael N. A1 - Rapp, Michael A. A1 - Walter, Henrik A1 - Heinz, Andreas T1 - How Accumulated Real Life Stress Experience and Cognitive Speed Interact on Decision-Making Processes JF - Frontiers in human neuroscienc N2 - Rationale: Advances in neurocomputational modeling suggest that valuation systems for goal-directed (deliberative) on one side, and habitual (automatic) decision-making on the other side may rely on distinct computational strategies for reinforcement learning, namely model-free vs. model-based learning. As a key theoretical difference, the model-based system strongly demands cognitive functions to plan actions prospectively based on an internal cognitive model of the environment, whereas valuation in the model-free system relies on rather simple learning rules from operant conditioning to retrospectively associate actions with their outcomes and is thus cognitively less demanding. Acute stress reactivity is known to impair model-based but not model-free choice behavior, with higher working memory capacity protecting the model-based system from acute stress. However, it is not clear which impact accumulated real life stress has on model-free and model-based decision systems and how this influence interacts with cognitive abilities. Methods: We used a sequential decision-making task distinguishing relative contributions of both learning strategies to choice behavior, the Social Readjustment Rating Scale questionnaire to assess accumulated real life stress, and the Digit Symbol Substitution Test to test cognitive speed in 95 healthy subjects. Results: Individuals reporting high stress exposure who had low cognitive speed showed reduced model-based but increased model-free behavioral control. In contrast, subjects exposed to accumulated real life stress with high cognitive speed displayed increased model-based performance but reduced model-free control. Conclusion: These findings suggest that accumulated real life stress exposure can enhance reliance on cognitive speed for model-based computations, which may ultimately protect the model-based system from the detrimental influences of accumulated real life stress. The combination of accumulated real life stress exposure and slower information processing capacities, however, might favor model-free strategies. Thus, the valence and preference of either system strongly depends on stressful experiences and individual cognitive capacities. KW - chronic stress KW - model-based learning KW - model-free learning KW - decision making KW - cognitive speed KW - real-life events Y1 - 2017 U6 - https://doi.org/10.3389/fnhum.2017.00302 SN - 1662-5161 VL - 11 SP - 1 EP - 9 PB - Frontiers Research Foundation CY - Lausanne ER - TY - GEN A1 - Sebold, Miriam A1 - Garbusow, Maria A1 - Nebe, S. A1 - Sundmacher, L. A1 - Kuitunen-Paul, Sören A1 - Wittchen, H. U. A1 - Smolka, M. A1 - Zimmermann, U. A1 - Rapp, Michael A. A1 - Huys, Q. A1 - Schlagenhauf, Florian A1 - Heinz, A. T1 - From goals to habits in alcohol dependence BT - association with treatment outcome and cognitive bias modification training T2 - European psychiatry : the journal of the Association of European Psychiatrists Y1 - 2018 SN - 0924-9338 SN - 1778-3585 VL - 48 SP - S274 EP - S274 PB - Elsevier CY - Paris ER - TY - JOUR A1 - Garbusow, Maria A1 - Nebe, Stephan A1 - Sommer, Christian A1 - Kuitunen-Paul, Sören A1 - Sebold, Miriam A1 - Schad, Daniel A1 - Friedel, Eva A1 - Veer, Ilya M. A1 - Wittchen, Hans-Ulrich A1 - Rapp, Michael A. A1 - Ripke, Stephan A1 - Walter, Henrik A1 - Huys, Quentin J. M. A1 - Schlagenhauf, Florian A1 - Smolka, Michael N. A1 - Heinz, Andreas T1 - Pavlovian-To-Instrumental Transfer and Alcohol Consumption in Young Male Social Drinkers BT - Behavioral, Neural and Polygenic Correlates JF - Journal of Clinical Medicine N2 - In animals and humans, behavior can be influenced by irrelevant stimuli, a phenomenon called Pavlovian-to-instrumental transfer (PIT). In subjects with substance use disorder, PIT is even enhanced with functional activation in the nucleus accumbens (NAcc) and amygdala. While we observed enhanced behavioral and neural PIT effects in alcohol-dependent subjects, we here aimed to determine whether behavioral PIT is enhanced in young men with high-risk compared to low-risk drinking and subsequently related functional activation in an a-priori region of interest encompassing the NAcc and amygdala and related to polygenic risk for alcohol consumption. A representative sample of 18-year old men (n = 1937) was contacted: 445 were screened, 209 assessed: resulting in 191 valid behavioral, 139 imaging and 157 genetic datasets. None of the subjects fulfilled criteria for alcohol dependence according to the Diagnostic and Statistical Manual of Mental Disorders-IV-TextRevision (DSM-IV-TR). We measured how instrumental responding for rewards was influenced by background Pavlovian conditioned stimuli predicting action-independent rewards and losses. Behavioral PIT was enhanced in high-compared to low-risk drinkers (b = 0.09, SE = 0.03, z = 2.7, p < 0.009). Across all subjects, we observed PIT-related neural blood oxygen level-dependent (BOLD) signal in the right amygdala (t = 3.25, p(SVC) = 0.04, x = 26, y = -6, z = -12), but not in NAcc. The strength of the behavioral PIT effect was positively correlated with polygenic risk for alcohol consumption (r(s) = 0.17, p = 0.032). We conclude that behavioral PIT and polygenic risk for alcohol consumption might be a biomarker for a subclinical phenotype of risky alcohol consumption, even if no drug-related stimulus is present. The association between behavioral PIT effects and the amygdala might point to habitual processes related to out PIT task. In non-dependent young social drinkers, the amygdala rather than the NAcc is activated during PIT; possible different involvement in association with disease trajectory should be investigated in future studies. KW - Pavlovian-to-instrumental transfer KW - amygdala KW - alcohol KW - polygenic risk KW - high risk drinkers Y1 - 2019 U6 - https://doi.org/10.3390/jcm8081188 SN - 2077-0383 VL - 8 IS - 8 PB - MDPI CY - Basel ER - TY - GEN A1 - Garbusow, Maria A1 - Sommer, Christian A1 - Nebe, Stephan A1 - Sebold, Miriam A1 - Kuitunen-Paul, Sören A1 - Wittchen, Hans-Ulrich A1 - Smolka, Michael N. A1 - Zimmermann, Ulrich S. A1 - Rapp, Michael A. A1 - Huys, Quentin J. M. A1 - Schlagenhauf, Florian A1 - Heinz, Andreas T1 - Multi-level evidence of general pavlovian-to-instrumental transfer in alcohol use disorder T2 - Alcoholism : clinical and experimental research ; the official journal of the American Medical Society on Alcoholism and the Research Society on Alcoholism Y1 - 2018 SN - 0145-6008 SN - 1530-0277 VL - 42 SP - 128A EP - 128A PB - Wiley CY - Hoboken ER - TY - GEN A1 - Sebold, Miriam A1 - Nebe, Stephan A1 - Garbusow, Maria A1 - Schad, Daniel A1 - Sommer, Christian A1 - Rapp, Michael A. A1 - Smolka, Michael N. A1 - Huys, Quentin J. M. A1 - Schlagenhauf, Florian A1 - Heinz, Andreas T1 - Neurobiological correlates of learning and decision-making in alcohol dependence T2 - European psychiatry : the journal of the Association of European Psychiatrists Y1 - 2017 U6 - https://doi.org/10.1016/j.eurpsy.2017.01.084 SN - 0924-9338 SN - 1778-3585 VL - 41 SP - S11 EP - S11 PB - Elsevier CY - Paris ER - TY - JOUR A1 - Nebe, Stephan A1 - Kroemer, Nils B. A1 - Schad, Daniel A1 - Bernhardt, Nadine A1 - Sebold, Miriam A1 - Mueller, Dirk K. A1 - Scholl, Lucie A1 - Kuitunen-Paul, Sören A1 - Heinz, Andreas A1 - Rapp, Michael A. A1 - Huys, Quentin J. M. A1 - Smolka, Michael N. T1 - No association of goal-directed and habitual control with alcohol consumption in young adults JF - Addiction biology N2 - Alcohol dependence is a mental disorder that has been associated with an imbalance in behavioral control favoring model-free habitual over model-based goal-directed strategies. It is as yet unknown, however, whether such an imbalance reflects a predisposing vulnerability or results as a consequence of repeated and/or excessive alcohol exposure. We, therefore, examined the association of alcohol consumption with model-based goal-directed and model-free habitual control in 188 18-year-old social drinkers in a two-step sequential decision-making task while undergoing functional magnetic resonance imaging before prolonged alcohol misuse could have led to severe neurobiological adaptations. Behaviorally, participants showed a mixture of model-free and model-based decision-making as observed previously. Measures of impulsivity were positively related to alcohol consumption. In contrast, neither model-free nor model-based decision weights nor the trade-off between them were associated with alcohol consumption. There were also no significant associations between alcohol consumption and neural correlates of model-free or model-based decision quantities in either ventral striatum or ventromedial prefrontal cortex. Exploratory whole-brain functional magnetic resonance imaging analyses with a lenient threshold revealed early onset of drinking to be associated with an enhanced representation of model-free reward prediction errors in the posterior putamen. These results suggest that an imbalance between model-based goal-directed and model-free habitual control might rather not be a trait marker of alcohol intake per se. KW - alcohol KW - goal-directed KW - reinforcement learning Y1 - 2017 U6 - https://doi.org/10.1111/adb.12490 SN - 1355-6215 SN - 1369-1600 VL - 23 IS - 1 SP - 379 EP - 393 PB - Wiley CY - Hoboken ER - TY - JOUR A1 - Sebold, Miriam A1 - Spitta, G. A1 - Gleich, T. A1 - Dembler-Stamm, T. A1 - Butler, Oisin A1 - Zacharias, Kristin A1 - Aydin, S. A1 - Garbusow, Maria A1 - Rapp, Michael A. A1 - Schubert, Florian A1 - Buchert, Ralph A1 - Gallinat, Jürgen A1 - Heinz, A. T1 - Stressful life events are associated with striatal dopamine receptor availability in alcohol dependence JF - Journal of neural transmission N2 - Stress plays a key role in modulating addictive behavior and can cause relapse following periods of abstinence. Common effects of stress and alcohol on the dopaminergic system have been suggested, although the precise mechanisms are unclear. Here, we investigated 20 detoxified alcohol-dependent patients and 19 matched healthy controls and assessed striatal D2/D3 availability using [F-18]-fallypride positron emission tomography and stressful life events. We found a strong association between striatal D2/D3 availability and stress in patients, but not in healthy controls. Interestingly, we found increased D2/D3 receptor availability in patients with higher stress levels. This mirrors complex interactions between stress and alcohol intake in animal studies and emphasizes the importance to investigate stress exposure in neurobiological studies of addiction. KW - Stressful life events KW - Dopamine D2 KW - D3 receptor KW - Positron emission tomography KW - Striatum KW - Alcohol dependence Y1 - 2019 U6 - https://doi.org/10.1007/s00702-019-01985-2 SN - 0300-9564 SN - 1435-1463 VL - 126 IS - 9 SP - 1127 EP - 1134 PB - Springer CY - Wien ER - TY - JOUR A1 - Sebold, Miriam A1 - Chen, Hao A1 - Önal, Aleyna A1 - Kuitunen-Paul, Sören A1 - Mojtahedzadeh, Negin A1 - Garbusow, Maria A1 - Nebe, Stephan A1 - Wittchen, Hans-Ulrich A1 - Huys, Quentin J. M. A1 - Schlagenhauf, Florian A1 - Rapp, Michael A. A1 - Smolka, Michael N. A1 - Heinz, Andreas T1 - Stronger prejudices are associated with decreased model-based control JF - Frontiers in psychology N2 - Background: Prejudices against minorities can be understood as habitually negative evaluations that are kept in spite of evidence to the contrary. Therefore, individuals with strong prejudices might be dominated by habitual or "automatic" reactions at the expense of more controlled reactions. Computational theories suggest individual differences in the balance between habitual/model-free and deliberative/model-based decision-making. Methods: 127 subjects performed the two Step task and completed the blatant and subtle prejudice scale. Results: By using analyses of choices and reaction times in combination with computational modeling, subjects with stronger blatant prejudices showed a shift away from model-based control. There was no association between these decision-making processes and subtle prejudices. Conclusion: These results support the idea that blatant prejudices toward minorities are related to a relative dominance of habitual decision-making. This finding has important implications for developing interventions that target to change prejudices across societies. KW - subtle and blatant prejudice KW - immigrant KW - social behavior; KW - decision-making KW - computational modeling KW - reinforcement learning Y1 - 2022 U6 - https://doi.org/10.3389/fpsyg.2021.767022 SN - 1664-1078 VL - 12 PB - Frontiers Research Foundation CY - Lausanne ER -