TY - JOUR
A1 - Nicenboim, Bruno
A1 - Vasishth, Shravan
T1 - Statistical methods for linguistic research: Foundational Ideas-Part II
JF - Language and linguistics compass
N2 - We provide an introductory review of Bayesian data analytical methods, with a focus on applications for linguistics, psychology, psycholinguistics, and cognitive science. The empirically oriented researcher will benefit from making Bayesian methods part of their statistical toolkit due to the many advantages of this framework, among them easier interpretation of results relative to research hypotheses and flexible model specification. We present an informal introduction to the foundational ideas behind Bayesian data analysis, using, as an example, a linear mixed models analysis of data from a typical psycholinguistics experiment. We discuss hypothesis testing using the Bayes factor and model selection using cross-validation. We close with some examples illustrating the flexibility of model specification in the Bayesian framework. Suggestions for further reading are also provided.
Y1 - 2016
U6 - https://doi.org/10.1111/lnc3.12207
SN - 1749-818X
VL - 10
SP - 591
EP - 613
PB - Wiley-Blackwell
CY - Hoboken
ER -
TY - GEN
A1 - Stone, Kate
A1 - Nicenboim, Bruno
A1 - Vasishth, Shravan
A1 - Rösler, Frank
T1 - Understanding the effects of constraint and predictability in ERP
T2 - Zweitveröffentlichungen der Universität Potsdam : Humanwissenschaftliche Reihe
N2 - Intuitively, strongly constraining contexts should lead to stronger probabilistic representations of sentences in memory. Encountering unexpected words could therefore be expected to trigger costlier shifts in these representations than expected words. However, psycholinguistic measures commonly used to study probabilistic processing, such as the N400 event-related potential (ERP) component, are sensitive to word predictability but not to contextual constraint. Some research suggests that constraint-related processing cost may be measurable via an ERP positivity following the N400, known as the anterior post-N400 positivity (PNP). The PNP is argued to reflect update of a sentence representation and to be distinct from the posterior P600, which reflects conflict detection and reanalysis. However, constraint-related PNP findings are inconsistent. We sought to conceptually replicate Federmeier et al. (2007) and Kuperberg et al. (2020), who observed that the PNP, but not the N400 or the P600, was affected by constraint at unexpected but plausible words. Using a pre-registered design and statistical approach maximising power, we demonstrated a dissociated effect of predictability and constraint: strong evidence for predictability but not constraint in the N400 window, and strong evidence for constraint but not predictability in the later window. However, the constraint effect was consistent with a P600 and not a PNP, suggesting increased conflict between a strong representation and unexpected input rather than greater update of the representation. We conclude that either a simple strong/weak constraint design is not always sufficient to elicit the PNP, or that previous PNP constraint findings could be an artifact of smaller sample size.
T3 - Zweitveröffentlichungen der Universität Potsdam : Humanwissenschaftliche Reihe - 829
KW - N400
KW - anterior PNP
KW - posterior P600
KW - probabilistic processing
KW - constraint
KW - predictability
KW - entropy
Y1 - 2023
U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:517-opus4-587594
SN - 1866-8364
IS - 829
ER -
TY - JOUR
A1 - Stone, Kate
A1 - Nicenboim, Bruno
A1 - Vasishth, Shravan
A1 - Rösler, Frank
T1 - Understanding the effects of constraint and predictability in ERP
JF - Neurobiology of language
N2 - Intuitively, strongly constraining contexts should lead to stronger probabilistic representations of sentences in memory. Encountering unexpected words could therefore be expected to trigger costlier shifts in these representations than expected words. However, psycholinguistic measures commonly used to study probabilistic processing, such as the N400 event-related potential (ERP) component, are sensitive to word predictability but not to contextual constraint. Some research suggests that constraint-related processing cost may be measurable via an ERP positivity following the N400, known as the anterior post-N400 positivity (PNP). The PNP is argued to reflect update of a sentence representation and to be distinct from the posterior P600, which reflects conflict detection and reanalysis. However, constraint-related PNP findings are inconsistent. We sought to conceptually replicate Federmeier et al. (2007) and Kuperberg et al. (2020), who observed that the PNP, but not the N400 or the P600, was affected by constraint at unexpected but plausible words. Using a pre-registered design and statistical approach maximising power, we demonstrated a dissociated effect of predictability and constraint: strong evidence for predictability but not constraint in the N400 window, and strong evidence for constraint but not predictability in the later window. However, the constraint effect was consistent with a P600 and not a PNP, suggesting increased conflict between a strong representation and unexpected input rather than greater update of the representation. We conclude that either a simple strong/weak constraint design is not always sufficient to elicit the PNP, or that previous PNP constraint findings could be an artifact of smaller sample size.
KW - N400
KW - anterior PNP
KW - posterior P600
KW - probabilistic processing
KW - constraint
KW - predictability
KW - entropy
Y1 - 2022
U6 - https://doi.org/10.1162/nol_a_00094
SN - 2641-4368
VL - 4
IS - 2
SP - 221
EP - 256
PB - MIT Press
CY - Cambridge, MA, USA
ER -
TY - JOUR
A1 - Nicenboim, Bruno
A1 - Roettger, Timo B.
A1 - Vasishth, Shravan
T1 - Using meta-analysis for evidence synthesis
BT - the case of incomplete neutralization in German
JF - Journal of phonetics
N2 - Within quantitative phonetics, it is common practice to draw conclusions based on statistical significance alone Using incomplete neutralization of final devoicing in German as a case study, we illustrate the problems with this approach. If researchers find a significant acoustic difference between voiceless and devoiced obstruents, they conclude that neutralization is incomplete, and if they find no significant difference, they conclude that neutralization is complete. However, such strong claims regarding the existence or absence of an effect based on significant results alone can be misleading. Instead, the totality of available evidence should be brought to bear on the question. Towards this end, we synthesize the evidence from 14 studies on incomplete neutralization in German using a Bayesian random-effects meta-analysis. Our meta-analysis provides evidence in favor of incomplete neutralization. We conclude with some suggestions for improving the quality of future research on phonetic phenomena: ensure that sample sizes allow for high-precision estimates of the effect; avoid the temptation to deploy researcher degrees of freedom when analyzing data; focus on estimates of the parameter of interest and the uncertainty about that parameter; attempt to replicate effects found; and, whenever possible, make both the data and analysis available publicly. (c) 2018 Elsevier Ltd. All rights reserved.
KW - Meta-analysis
KW - Incomplete neutralization
KW - Final devoicing
KW - German
KW - Bayesian data analysis
Y1 - 2018
U6 - https://doi.org/10.1016/j.wocn.2018.06.001
SN - 0095-4470
VL - 70
SP - 39
EP - 55
PB - Elsevier
CY - London
ER -
TY - GEN
A1 - Nicenboim, Bruno
A1 - Logacev, Pavel
A1 - Gattei, Carolina
A1 - Vasishth, Shravan
T1 - When High-Capacity Readers Slow Down and Low-Capacity Readers Speed Up
BT - Working Memory and Locality Effects
N2 - We examined the effects of argument-head distance in SVO and SOV languages (Spanish and German), while taking into account readers' working memory capacity and controlling for expectation (Levy, 2008) and other factors. We predicted only locality effects, that is, a slowdown produced by increased dependency distance (Gibson, 2000; Lewis and Vasishth, 2005). Furthermore, we expected stronger locality effects for readers with low working memory capacity. Contrary to our predictions, low-capacity readers showed faster reading with increased distance, while high-capacity readers showed locality effects. We suggest that while the locality effects are compatible with memory-based explanations, the speedup of low-capacity readers can be explained by an increased probability of retrieval failure. We present a computational model based on ACT-R built under the previous assumptions, which is able to give a qualitative account for the present data and can be tested in future research. Our results suggest that in some cases, interpreting longer RTs as indexing increased processing difficulty and shorter RTs as facilitation may be too simplistic: The same increase in processing difficulty may lead to slowdowns in high-capacity readers and speedups in low-capacity ones. Ignoring individual level capacity differences when investigating locality effects may lead to misleading conclusions.
T3 - Zweitveröffentlichungen der Universität Potsdam : Humanwissenschaftliche Reihe - 288
KW - locality
KW - working memory capacity
KW - individual differences
KW - Spanish
KW - German
KW - ACT-R
Y1 - 2016
U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:517-opus4-90663
SP - 1
EP - 24
ER -
TY - JOUR
A1 - Nicenboim, Bruno
A1 - Logacev, Pavel
A1 - Gattei, Carolina
A1 - Vasishth, Shravan
T1 - When High-Capacity Readers Slow Down and Low-Capacity Readers Speed Up
BT - Working Memory and Locality Effects
JF - Frontiers in psychology
N2 - We examined the effects of argument-head distance in SVO and SOV languages (Spanish and German), while taking into account readers' working memory capacity and controlling for expectation (Levy, 2008) and other factors. We predicted only locality effects, that is, a slowdown produced by increased dependency distance (Gibson, 2000; Lewis and Vasishth, 2005). Furthermore, we expected stronger locality effects for readers with low working memory capacity. Contrary to our predictions, low-capacity readers showed faster reading with increased distance, while high-capacity readers showed locality effects. We suggest that while the locality effects are compatible with memory-based explanations, the speedup of low-capacity readers can be explained by an increased probability of retrieval failure. We present a computational model based on ACT-R built under the previous assumptions, which is able to give a qualitative account for the present data and can be tested in future research. Our results suggest that in some cases, interpreting longer RTs as indexing increased processing difficulty and shorter RTs as facilitation may be too simplistic: The same increase in processing difficulty may lead to slowdowns in high-capacity readers and speedups in low-capacity ones. Ignoring individual level capacity differences when investigating locality effects may lead to misleading conclusions.
KW - locality
KW - working memory capacity
KW - individual differences
KW - Spanish
KW - German
KW - ACT-R
Y1 - 2016
U6 - https://doi.org/10.3389/fpsyg.2016.00280
SN - 1664-1078
VL - 7
SP - 1
EP - 24
PB - Frontiers Research Foundation
CY - Lausanne
ER -
TY - JOUR
A1 - Schad, Daniel
A1 - Nicenboim, Bruno
A1 - Bürkner, Paul-Christian
A1 - Betancourt, Michael
A1 - Vasishth, Shravan
T1 - Workflow techniques for the robust use of bayes factors
JF - Psychological methods
N2 - Inferences about hypotheses are ubiquitous in the cognitive sciences. Bayes factors provide one general way to compare different hypotheses by their compatibility with the observed data. Those quantifications can then also be used to choose between hypotheses. While Bayes factors provide an immediate approach to hypothesis testing, they are highly sensitive to details of the data/model assumptions and it's unclear whether the details of the computational implementation (such as bridge sampling) are unbiased for complex analyses. Hem, we study how Bayes factors misbehave under different conditions. This includes a study of errors in the estimation of Bayes factors; the first-ever use of simulation-based calibration to test the accuracy and bias of Bayes factor estimates using bridge sampling; a study of the stability of Bayes factors against different MCMC draws and sampling variation in the data; and a look at the variability of decisions based on Bayes factors using a utility function. We outline a Bayes factor workflow that researchers can use to study whether Bayes factors are robust for their individual analysis. Reproducible code is available from haps://osf.io/y354c/.
Translational Abstract
In psychology and related areas, scientific hypotheses are commonly tested by asking questions like "is [some] effect present or absent." Such hypothesis testing is most often carried out using frequentist null hypothesis significance testing (NIIST). The NHST procedure is very simple: It usually returns a p-value, which is then used to make binary decisions like "the effect is present/abscnt." For example, it is common to see studies in the media that draw simplistic conclusions like "coffee causes cancer," or "coffee reduces the chances of geuing cancer." However, a powerful and more nuanced alternative approach exists: Bayes factors. Bayes factors have many advantages over NHST. However, for the complex statistical models that arc commonly used for data analysis today, computing Bayes factors is not at all a simple matter. In this article, we discuss the main complexities associated with computing Bayes factors. This is the first article to provide a detailed workflow for understanding and computing Bayes factors in complex statistical models. The article provides a statistically more nuanced way to think about hypothesis testing than the overly simplistic tendency to declare effects as being "present" or "absent".
KW - Bayes factors
KW - Bayesian model comparison
KW - prior
KW - posterior
KW - simulation-based calibration
Y1 - 2022
U6 - https://doi.org/10.1037/met0000472
SN - 1082-989X
SN - 1939-1463
VL - 28
IS - 6
SP - 1404
EP - 1426
PB - American Psychological Association
CY - Washington
ER -
TY - JOUR
A1 - Nicenboim, Bruno
A1 - Vasishth, Shravan
A1 - Gattei, Carolina
A1 - Sigman, Mariano
A1 - Kliegl, Reinhold
T1 - Working memory differences in long-distance dependency resolution
JF - Frontiers in psychology
N2 - There is a wealth of evidence showing that increasing the distance between an argument and its head leads to more processing effort, namely, locality effects: these are usually associated with constraints in working memory (DLT: Gibson, 2000: activation-based model: Lewis and Vasishth, 2005). In SOV languages, however, the opposite effect has been found: antilocality (see discussion in Levy et al., 2013). Antilocality effects can be explained by the expectation based approach as proposed by Levy (2008) or by the activation-based model of sentence processing as proposed by Lewis and Vasishth (2005). We report an eye-tracking and a self-paced reading study with sentences in Spanish together with measures of individual differences to examine the distinction between expectation- and memory based accounts, and within memory-based accounts the further distinction between DLT and the activation-based model. The experiments show that (i) antilocality effects as predicted by the expectation account appear only for high-capacity readers; (ii) increasing dependency length by interposing material that modifies the head of the dependency (the verb) produces stronger facilitation than increasing dependency length with material that does not modify the head; this is in agreement with the activation-based model but not with the expectation account; and (iii) a possible outcome of memory load on low-capacity readers is the increase in regressive saccades (locality effects as predicted by memory-based accounts) or, surprisingly, a speedup in the self-paced reading task; the latter consistent with good-enough parsing (Ferreira et al., 2002). In sum, the study suggests that individual differences in working memory capacity play a role in dependency resolution, and that some of the aspects of dependency resolution can be best explained with the activation-based model together with a prediction component.
KW - locality
KW - antilocality
KW - working memory capacity
KW - individual differences
KW - Spanish
KW - activation
KW - DLT
KW - expectation
Y1 - 2015
U6 - https://doi.org/10.3389/fpsyg.2015.00312
SN - 1664-1078
VL - 6
PB - Frontiers Research Foundation
CY - Lausanne
ER -
TY - JOUR
A1 - Nicenboim, Bruno
A1 - Vasishth, Shravan
A1 - Gattei, Carolina
A1 - Sigman, Mariano
A1 - Kliegl, Reinhold
T1 - Working memory differences in long-distance dependency resolution
JF - Frontiers in psychology
N2 - There is a wealth of evidence showing that increasing the distance between an argument and its head leads to more processing effort, namely, locality effects; these are usually associated with constraints in working memory (DLT: Gibson, 2000; activation-based model: Lewis and Vasishth, 2005). In SOV languages, however, the opposite effect has been found: antilocality (see discussion in Levy et al., 2013). Antilocality effects can be explained by the expectation-based approach as proposed by Levy (2008) or by the activation-based model of sentence processing as proposed by Lewis and Vasishth (2005). We report an eye-tracking and a self-paced reading study with sentences in Spanish together with measures of individual differences to examine the distinction between expectation- and memory-based accounts, and within memory-based accounts the further distinction between DLT and the activation-based model. The experiments show that (i) antilocality effects as predicted by the expectation account appear only for high-capacity readers; (ii) increasing dependency length by interposing material that modifies the head of the dependency (the verb) produces stronger facilitation than increasing dependency length with material that does not modify the head; this is in agreement with the activation-based model but not with the expectation account; and (iii) a possible outcome of memory load on low-capacity readers is the increase in regressive saccades (locality effects as predicted by memory-based accounts) or, surprisingly, a speedup in the self-paced reading task; the latter consistent with good-enough parsing (Ferreira et al., 2002). In sum, the study suggests that individual differences in working memory capacity play a role in dependency resolution, and that some of the aspects of dependency resolution can be best explained with the activation-based model together with a prediction component.
KW - locality
KW - antilocality
KW - working memory capacity
KW - individual differences
KW - Spanish
KW - activation
KW - DLT
KW - expectation
Y1 - 2015
U6 - https://doi.org/10.3389/fpsyg.2015.00312
SN - 1664-1078
VL - 6
IS - 312
PB - Frontiers Research Foundation
CY - Lausanne
ER -
TY - GEN
A1 - Nicenboim, Bruno
A1 - Vasishth, Shravan
A1 - Gattei, Carolina
A1 - Sigman, Mariano
A1 - Kliegl, Reinhold
T1 - Working memory differences in long-distance dependency resolution
N2 - There is a wealth of evidence showing that increasing the distance between an argument and its head leads to more processing effort, namely, locality effects; these are usually associated with constraints in working memory (DLT: Gibson, 2000; activation-based model: Lewis and Vasishth, 2005). In SOV languages, however, the opposite effect has been found: antilocality (see discussion in Levy et al., 2013). Antilocality effects can be explained by the expectation-based approach as proposed by Levy (2008) or by the activation-based model of sentence processing as proposed by Lewis and Vasishth (2005). We report an eye-tracking and a self-paced reading study with sentences in Spanish together with measures of individual differences to examine the distinction between expectation- and memory-based accounts, and within memory-based accounts the further distinction between DLT and the activation-based model. The experiments show that (i) antilocality effects as predicted by the expectation account appear only for high-capacity readers; (ii) increasing dependency length by interposing material that modifies the head of the dependency (the verb) produces stronger facilitation than increasing dependency length with material that does not modify the head; this is in agreement with the activation-based model but not with the expectation account; and (iii) a possible outcome of memory load on low-capacity readers is the increase in regressive saccades (locality effects as predicted by memory-based accounts) or, surprisingly, a speedup in the self-paced reading task; the latter consistent with good-enough parsing (Ferreira et al., 2002). In sum, the study suggests that individual differences in working memory capacity play a role in dependency resolution, and that some of the aspects of dependency resolution can be best explained with the activation-based model together with a prediction component.
T3 - Zweitveröffentlichungen der Universität Potsdam : Humanwissenschaftliche Reihe - paper 273
KW - locality
KW - antilocality
KW - working memory capacity
KW - individual differences
KW - Spanish
KW - activation
KW - DLT
KW - expectation
Y1 - 2015
U6 - http://nbn-resolving.de/urn/resolver.pl?urn:nbn:de:kobv:517-opus4-75694
ER -