TY - JOUR A1 - Wyckmans, Florent A1 - Otto, A. Ross A1 - Sebold, Miriam A1 - Daw, Nathaniel A1 - Bechara, Antoine A1 - Saeremans, Mélanie A1 - Kornreich, Charles A1 - Chatard, Armand A1 - Jaafari, Nemat A1 - Noël, Xavier T1 - Reduced model-based decision-making in gambling disorder JF - Scientific reports N2 - Compulsive behaviors (e.g., addiction) can be viewed as an aberrant decision process where inflexible reactions automatically evoked by stimuli (habit) take control over decision making to the detriment of a more flexible (goal-oriented) behavioral learning system. These behaviors are thought to arise from learning algorithms known as "model-based" and "model-free" reinforcement learning. Gambling disorder, a form of addiction without the confound of neurotoxic effects of drugs, showed impaired goal-directed control but the way in which problem gamblers (PG) orchestrate model-based and model-free strategies has not been evaluated. Forty-nine PG and 33 healthy participants (CP) completed a two-step sequential choice task for which model-based and model-free learning have distinct and identifiable trial-by-trial learning signatures. The influence of common psychopathological comorbidities on those two forms of learning were investigated. PG showed impaired model-based learning, particularly after unrewarded outcomes. In addition, PG exhibited faster reaction times than CP following unrewarded decisions. Troubled mood, higher impulsivity (i.e., positive and negative urgency) and current and chronic stress reported via questionnaires did not account for those results. These findings demonstrate specific reinforcement learning and decision-making deficits in behavioral addiction that advances our understanding and may be important dimensions for designing effective interventions. Y1 - 2019 U6 - https://doi.org/10.1038/s41598-019-56161-z SN - 2045-2322 VL - 9 PB - Nature Publ. Group CY - London ER - TY - JOUR A1 - Sekutowicz, Maria A1 - Guggenmos, Matthias A1 - Kuitunen-Paul, Sören A1 - Garbusow, Maria A1 - Sebold, Miriam A1 - Pelz, Patricia A1 - Priller, Josef A1 - Wittchen, Hans-Ulrich A1 - Smolka, Michael N. A1 - Zimmermann, Ulrich S. A1 - Heinz, Andreas A1 - Sterzer, Philipp A1 - Schmack, Katharina T1 - Neural Response Patterns During Pavlovian-to-Instrumental Transfer Predict Alcohol Relapse and Young Adult Drinking JF - Biological psychiatry : a journal of psychiatric neuroscience and therapeutics ; a publication of the Society of Biological Psychiatry N2 - BACKGROUND: Pavlovian-to-instrumental transfer (PIT) describes the influence of conditioned stimuli on instrumental behaviors and is discussed as a key process underlying substance abuse. Here, we tested whether neural responses during alcohol-related PIT predict future relapse in alcohol-dependent patients and future drinking behavior in adolescents. METHODS: Recently detoxified alcohol-dependent patients (n = 52) and young adults without dependence (n = 136) underwent functional magnetic resonance imaging during an alcohol-related PIT paradigm, and their drinking behavior was assessed in a 12-month follow-up. To predict future drinking behavior from PIT activation patterns, we used a multivoxel classification scheme based on linear support vector machines. RESULTS: When training and testing the classification scheme in patients, PIT activation patterns predicted future relapse with 71.2% accuracy. Feature selection revealed that classification was exclusively based on activation patterns in medial prefrontal cortex. To probe the generalizability of this functional magnetic resonance imaging-based prediction of future drinking behavior, we applied the support vector machine classifier that had been trained on patients to PIT functional magnetic resonance imaging data from adolescents. An analysis of cross-classification predictions revealed that those young social drinkers who were classified as abstainers showed a greater reduction in alcohol consumption at 12-month follow-up than those classified as relapsers (Delta = -24.4 +/- 6.0 g vs. -5.7 +/- 3.6 g; p = .019). CONCLUSIONS: These results suggest that neural responses during PIT could constitute a generalized prognostic marker for future drinking behavior in established alcohol use disorder and in at-risk states. KW - Alcohol dependence KW - Future drinking behavior KW - Multivoxel classification KW - Pavlovian-to-instrumental transfer KW - Relapse Y1 - 2019 U6 - https://doi.org/10.1016/j.biopsych.2019.06.028 SN - 0006-3223 SN - 1873-2402 VL - 86 IS - 11 SP - 857 EP - 863 PB - Elsevier CY - New York ER - TY - JOUR A1 - Sebold, Miriam A1 - Spitta, G. A1 - Gleich, T. A1 - Dembler-Stamm, T. A1 - Butler, Oisin A1 - Zacharias, Kristin A1 - Aydin, S. A1 - Garbusow, Maria A1 - Rapp, Michael A. A1 - Schubert, Florian A1 - Buchert, Ralph A1 - Gallinat, Jürgen A1 - Heinz, Andreas T1 - Stressful life events are associated with striatal dopamine receptor availability in alcohol dependence JF - Journal of neural transmission N2 - Stress plays a key role in modulating addictive behavior and can cause relapse following periods of abstinence. Common effects of stress and alcohol on the dopaminergic system have been suggested, although the precise mechanisms are unclear. Here, we investigated 20 detoxified alcohol-dependent patients and 19 matched healthy controls and assessed striatal D2/D3 availability using [F-18]-fallypride positron emission tomography and stressful life events. We found a strong association between striatal D2/D3 availability and stress in patients, but not in healthy controls. Interestingly, we found increased D2/D3 receptor availability in patients with higher stress levels. This mirrors complex interactions between stress and alcohol intake in animal studies and emphasizes the importance to investigate stress exposure in neurobiological studies of addiction. KW - Stressful life events KW - Dopamine D2 KW - D3 receptor KW - Positron emission tomography KW - Striatum KW - Alcohol dependence Y1 - 2019 U6 - https://doi.org/10.1007/s00702-019-01985-2 SN - 0300-9564 SN - 1435-1463 VL - 126 IS - 9 SP - 1127 EP - 1134 PB - Springer CY - Wien ER - TY - JOUR A1 - Sebold, Miriam A1 - Schad, Daniel A1 - Nebe, Stephan A1 - Garbusow, Maria A1 - Juenger, Elisabeth A1 - Kroemer, Nils B. A1 - Kathmann, Norbert A1 - Zimmermann, Ulrich S. A1 - Smolka, Michael N. A1 - Rapp, Michael A. A1 - Heinz, Andreas A1 - Huys, Quentin J. M. T1 - Pavlovian-to-Instrumental Transfer Effects Rely Less on Model-based Reinforcement Learning JF - Journal of cognitive neuroscience N2 - Behavioral choice can be characterized along two axes. One axis distinguishes reflexive, model-free systems that slowly accumulate values through experience and a model-based system that uses knowledge to reason prospectively. The second axis distinguishes Pavlovian valuation of stimuli from instrumental valuation of actions or stimulus–action pairs. This results in four values and many possible interactions between them, with important consequences for accounts of individual variation. We here explored whether individual variation along one axis was related to individual variation along the other. Specifically, we asked whether individuals' balance between model-based and model-free learning was related to their tendency to show Pavlovian interferences with instrumental decisions. In two independent samples with a total of 243 participants, Pavlovian–instrumental transfer effects were negatively correlated with the strength of model-based reasoning in a two-step task. This suggests a potential common underlying substrate predisposing individuals to both have strong Pavlovian interference and be less model-based and provides a framework within which to interpret the observation of both effects in addiction. Y1 - 2016 U6 - https://doi.org/10.1162/jocn_a_00945 SN - 0898-929X SN - 1530-8898 VL - 28 SP - 985 EP - 995 PB - MIT Press CY - Cambridge ER - TY - JOUR A1 - Sebold, Miriam A1 - Nebe, Stephan A1 - Garbusow, Maria A1 - Guggenmos, Matthias A1 - Schad, Daniel A1 - Beck, Anne A1 - Kuitunen-Paul, Sören A1 - Sommer, Christian A1 - Frank, Robin A1 - Neu, Peter A1 - Zimmermann, Ulrich S. A1 - Rapp, Michael A. A1 - Smolka, Michael N. A1 - Huys, Quentin J. M. A1 - Schlagenhauf, Florian A1 - Heinz, Andreas T1 - When Habits Are Dangerous: Alcohol Expectancies and Habitual Decision Making Predict Relapse in Alcohol Dependence JF - Biological psychiatry : a journal of psychiatric neuroscience and therapeutics ; a publication of the Society of Biological Psychiatry N2 - BACKGROUND: Addiction is supposedly characterized by a shift from goal-directed to habitual decision making, thus facilitating automatic drug intake. The two-step task allows distinguishing between these mechanisms by computationally modeling goal-directed and habitual behavior as model-based and model-free control. In addicted patients, decision making may also strongly depend upon drug-associated expectations. Therefore, we investigated model-based versus model-free decision making and its neural correlates as well as alcohol expectancies in alcohol-dependent patients and healthy controls and assessed treatment outcome in patients. METHODS: Ninety detoxified, medication-free, alcohol-dependent patients and 96 age-and gender-matched control subjects underwent functional magnetic resonance imaging during the two-step task. Alcohol expectancies were measured with the Alcohol Expectancy Questionnaire. Over a follow-up period of 48 weeks, 37 patients remained abstinent and 53 patients relapsed as indicated by the Alcohol Timeline Followback method. RESULTS: Patients who relapsed displayed reduced medial prefrontal cortex activation during model-based decision making. Furthermore, high alcohol expectancies were associated with low model-based control in relapsers, while the opposite was observed in abstainers and healthy control subjects. However, reduced model-based control per se was not associated with subsequent relapse. CONCLUSIONS: These findings suggest that poor treatment outcome in alcohol dependence does not simply result from a shift from model-based to model-free control but is instead dependent on the interaction between high drug expectancies and low model-based decision making. Reduced model-based medial prefrontal cortex signatures in those who relapse point to a neural correlate of relapse risk. These observations suggest that therapeutic interventions should target subjective alcohol expectancies. KW - Alcohol dependence KW - Alcohol expectancy KW - Goal-directed control KW - Medial prefrontal cortex KW - Reinforcement learning KW - Treatment outcome Y1 - 2017 U6 - https://doi.org/10.1016/j.biopsych.2017.04.019 SN - 0006-3223 SN - 1873-2402 VL - 82 SP - 847 EP - 856 PB - Elsevier CY - New York ER - TY - JOUR A1 - Sebold, Miriam A1 - Garbusow, Maria A1 - Jetzschmann, P. A1 - Schad, Daniel A1 - Nebe, S. A1 - Schlagenhauf, Florian A1 - Heinz, Andreas A1 - Rapp, Michael A. A1 - Romanczuk-Seiferth, Nina T1 - Reward and avoidance learning in the context of aversive environments and possible implications for depressive symptoms JF - Psychopharmacology N2 - Background Aversive stimuli in the environment influence human actions. This includes valence-dependent influences on action selection, e.g., increased avoidance but decreased approach behavior. However, it is yet unclear how aversive stimuli interact with complex learning and decision-making in the reward and avoidance domain. Moreover, the underlying computational mechanisms of these decision-making biases are unknown. Methods To elucidate these mechanisms, 54 healthy young male subjects performed a two-step sequential decision-making task, which allows to computationally model different aspects of learning, e.g., model-free, habitual, and model-based, goal-directed learning. We used a within-subject design, crossing task valence (reward vs. punishment learning) with emotional context (aversive vs. neutral background stimuli). We analyzed choice data, applied a computational model, and performed simulations. Results Whereas model-based learning was not affected, aversive stimuli interacted with model-free learning in a way that depended on task valence. Thus, aversive stimuli increased model-free avoidance learning but decreased model-free reward learning. The computational model confirmed this effect: the parameter lambda that indicates the influence of reward prediction errors on decision values was increased in the punishment condition but decreased in the reward condition when aversive stimuli were present. Further, by using the inferred computational parameters to simulate choice data, our effects were captured. Exploratory analyses revealed that the observed biases were associated with subclinical depressive symptoms. Conclusion Our data show that aversive environmental stimuli affect complex learning and decision-making, which depends on task valence. Further, we provide a model of the underlying computations of this affective modulation. Finally, our finding of increased decision-making biases in subjects reporting subclinical depressive symptoms matches recent reports of amplified Pavlovian influences on action selection in depression and suggests a potential vulnerability factor for mood disorders. We discuss our findings in the light of the involvement of the neuromodulators serotonin and dopamine. KW - Reward learning KW - Avoidance learning KW - Reinforcement learning KW - Computational psychiatry KW - Decision-making KW - Affective modulation KW - Depression symptoms Y1 - 2019 U6 - https://doi.org/10.1007/s00213-019-05299-9 SN - 0033-3158 SN - 1432-2072 VL - 236 IS - 8 SP - 2437 EP - 2449 PB - Springer CY - New York ER - TY - JOUR A1 - Sebold, Miriam A1 - Deserno, Lorenz A1 - Nebe, Stefan A1 - Schad, Daniel A1 - Garbusow, Maria A1 - Haegele, Claudia A1 - Keller, Juergen A1 - Juenger, Elisabeth A1 - Kathmann, Norbert A1 - Smolka, Michael N. A1 - Rapp, Michael A. A1 - Schlagenhauf, Florian A1 - Heinz, Andreas A1 - Huys, Quentin J. M. T1 - Model-based and model-free decisions in alcohol dependence JF - Neuropsychobiology : international journal of experimental and clinical research in biological psychiatry, pharmacopsychiatry, Biological Psychology/Pharmacopsychology and Pharmacoelectroencephalography N2 - Background: Human and animal work suggests a shift from goal-directed to habitual decision-making in addiction. However, the evidence for this in human alcohol dependence is as yet inconclusive. Methods: Twenty-six healthy controls and 26 recently detoxified alcohol-dependent patients underwent behavioral testing with a 2-step task designed to disentangle goal-directed and habitual response patterns. Results: Alcohol-dependent patients showed less evidence of goal-directed choices than healthy controls, particularly after losses. There was no difference in the strength of the habitual component. The group differences did not survive controlling for performance on the Digit Symbol Substitution Task. Conclusion: Chronic alcohol use appears to selectively impair goal-directed function, rather than promoting habitual responding. It appears to do so particularly after nonrewards, and this may be mediated by the effects of alcohol on more general cognitive functions subserved by the prefrontal cortex. KW - Alcohol dependence KW - Decision-making KW - Reinforcement learning KW - Dopamine KW - Computational psychiatry Y1 - 2014 U6 - https://doi.org/10.1159/000362840 SN - 0302-282X SN - 1423-0224 VL - 70 IS - 2 SP - 122 EP - 131 PB - Karger CY - Basel ER - TY - JOUR A1 - Sebold, Miriam A1 - Chen, Hao A1 - Önal, Aleyna A1 - Kuitunen-Paul, Sören A1 - Mojtahedzadeh, Negin A1 - Garbusow, Maria A1 - Nebe, Stephan A1 - Wittchen, Hans-Ulrich A1 - Huys, Quentin J. M. A1 - Schlagenhauf, Florian A1 - Rapp, Michael A. A1 - Smolka, Michael N. A1 - Heinz, Andreas T1 - Stronger prejudices are associated with decreased model-based control JF - Frontiers in psychology N2 - Background: Prejudices against minorities can be understood as habitually negative evaluations that are kept in spite of evidence to the contrary. Therefore, individuals with strong prejudices might be dominated by habitual or "automatic" reactions at the expense of more controlled reactions. Computational theories suggest individual differences in the balance between habitual/model-free and deliberative/model-based decision-making. Methods: 127 subjects performed the two Step task and completed the blatant and subtle prejudice scale. Results: By using analyses of choices and reaction times in combination with computational modeling, subjects with stronger blatant prejudices showed a shift away from model-based control. There was no association between these decision-making processes and subtle prejudices. Conclusion: These results support the idea that blatant prejudices toward minorities are related to a relative dominance of habitual decision-making. This finding has important implications for developing interventions that target to change prejudices across societies. KW - subtle and blatant prejudice KW - immigrant KW - social behavior; KW - decision-making KW - computational modeling KW - reinforcement learning Y1 - 2022 U6 - https://doi.org/10.3389/fpsyg.2021.767022 SN - 1664-1078 VL - 12 PB - Frontiers Research Foundation CY - Lausanne ER - TY - JOUR A1 - Schad, Daniel A1 - Juenger, Elisabeth A1 - Sebold, Miriam A1 - Garbusow, Maria A1 - Bernhardt, Nadine A1 - Javadi, Amir-Homayoun A1 - Zimmermann, Ulrich S. A1 - Smolka, Michael N. A1 - Heinz, Andreas A1 - Rapp, Michael A. A1 - Huys, Quentin J. M. T1 - Processing speed enhances model-based over model-free reinforcement learning in the presence of high working memory functioning JF - Frontiers in psychology N2 - Theories of decision-making and its neural substrates have long assumed the existence of two distinct and competing valuation systems, variously described as goal-directed vs. habitual, or, more recently and based on statistical arguments, as model-free vs. model-based reinforcement-learning. Though both have been shown to control choices, the cognitive abilities associated with these systems are under ongoing investigation. Here we examine the link to cognitive abilities, and find that individual differences in processing speed covary with a shift from model-free to model-based choice control in the presence of above-average working memory function. This suggests shared cognitive and neural processes; provides a bridge between literatures on intelligence and valuation; and may guide the development of process models of different valuation components. Furthermore, it provides a rationale for individual differences in the tendency to deploy valuation systems, which may be important for understanding the manifold neuropsychiatric diseases associated with malfunctions of valuation. KW - decision-making KW - reward KW - cognitive abilities KW - model-based and model-free learning KW - fluid intelligence KW - habitual and goal-directed system Y1 - 2014 U6 - https://doi.org/10.3389/fpsyg.2014.01450 SN - 1664-1078 VL - 5 PB - Frontiers Research Foundation CY - Lausanne ER - TY - JOUR A1 - Schad, Daniel A1 - Garbusow, Maria A1 - Friedel, Eva A1 - Sommer, Christian A1 - Sebold, Miriam A1 - Hägele, Claudia A1 - Bernhardt, Nadine A1 - Nebe, Stephan A1 - Kuitunen-Paul, Sören A1 - Liu, Shuyan A1 - Eichmann, Uta A1 - Beck, Anne A1 - Wittchen, Hans-Ulrich A1 - Walter, Henrik A1 - Sterzer, Philipp A1 - Zimmermann, Ulrich S. A1 - Smolka, Michael N. A1 - Schlagenhauf, Florian A1 - Huys, Quentin J. M. A1 - Heinz, Andreas A1 - Rapp, Michael A. T1 - Neural correlates of instrumental responding in the context of alcohol-related cues index disorder severity and relapse risk JF - European archives of psychiatry and clinical neuroscience : official organ of the German Society for Biological Psychiatry N2 - The influence of Pavlovian conditioned stimuli on ongoing behavior may contribute to explaining how alcohol cues stimulate drug seeking and intake. Using a Pavlovian-instrumental transfer task, we investigated the effects of alcohol-related cues on approach behavior (i.e., instrumental response behavior) and its neural correlates, and related both to the relapse after detoxification in alcohol-dependent patients. Thirty-one recently detoxified alcohol-dependent patients and 24 healthy controls underwent instrumental training, where approach or non-approach towards initially neutral stimuli was reinforced by monetary incentives. Approach behavior was tested during extinction with either alcohol-related or neutral stimuli (as Pavlovian cues) presented in the background during functional magnetic resonance imaging (fMRI). Patients were subsequently followed up for 6 months. We observed that alcohol-related background stimuli inhibited the approach behavior in detoxified alcohol-dependent patients (t = -3.86, p < .001), but not in healthy controls (t = -0.92, p = .36). This behavioral inhibition was associated with neural activation in the nucleus accumbens (NAcc) (t((30)) = 2.06, p < .05). Interestingly, both the effects were only present in subsequent abstainers, but not relapsers and in those with mild but not severe dependence. Our data show that alcohol-related cues can acquire inhibitory behavioral features typical of aversive stimuli despite being accompanied by a stronger NAcc activation, suggesting salience attribution. The fact that these findings are restricted to abstinence and milder illness suggests that they may be potential resilience factors. KW - Alcohol dependence KW - Human neuroimaging KW - Nucleus accumbens KW - Pavlovian-instrumental transfer KW - Relapse Y1 - 2018 U6 - https://doi.org/10.1007/s00406-017-0860-4 SN - 0940-1334 SN - 1433-8491 VL - 269 IS - 3 SP - 295 EP - 308 PB - Springer CY - Heidelberg ER -