TY  - JOUR
A1  - Mertzen, Daniela
A1  - Lago, Sol
A1  - Vasishth, Shravan
T1  - The benefits of preregistration for hypothesis-driven bilingualism research
JF  - Bilingualism : language and cognition
N2  - Preregistration is an open science practice that requires the specification of research hypotheses and analysis plans before the data are inspected. Here, we discuss the benefits of preregistration for hypothesis-driven, confirmatory bilingualism research. Using examples from psycholinguistics and bilingualism, we illustrate how non-peer reviewed preregistrations can serve to implement a clean distinction between hypothesis testing and data exploration. This distinction helps researchers avoid casting post-hoc hypotheses and analyses as confirmatory ones. We argue that, in keeping with current best practices in the experimental sciences, preregistration, along with sharing data and code, should be an integral part of hypothesis-driven bilingualism research.
KW  - preregistration
KW  - open science
KW  - bilingualism
KW  - psycholinguistics
KW  - confirmatory analysis
KW  - exploratory analysis
Y1  - 2021
U6  - https://doi.org/10.1017/S1366728921000031
SN  - 1366-7289
SN  - 1469-1841
VL  - 24
IS  - 5
SP  - 807
EP  - 812
PB  - Cambridge Univ. Press
CY  - Cambridge
ER  - 
TY  - JOUR
A1  - Jäger, Lena Ann
A1  - Mertzen, Daniela
A1  - Van Dyke, Julie A.
A1  - Vasishth, Shravan
T1  - Interference patterns in subject-verb agreement and reflexives revisited
BT  - a large-sample study
JF  - Journal of memory and language
N2  - Cue-based retrieval theories in sentence processing predict two classes of interference effect: (i) Inhibitory interference is predicted when multiple items match a retrieval cue: cue-overloading leads to an overall slowdown in reading time; and (ii) Facilitatory interference arises when a retrieval target as well as a distractor only partially match the retrieval cues; this partial matching leads to an overall speedup in retrieval time. Inhibitory interference effects are widely observed, but facilitatory interference apparently has an exception: reflexives have been claimed to show no facilitatory interference effects. Because the claim is based on underpowered studies, we conducted a large-sample experiment that investigated both facilitatory and inhibitory interference. In contrast to previous studies, we find facilitatory interference effects in reflexives. We also present a quantitative evaluation of the cue-based retrieval model of Engelmann, Jager, and Vasishth (2019).
KW  - Sentence processing
KW  - Cue-based retrieval
KW  - Similarity-based interference
KW  - Reflexives
KW  - Agreement
KW  - Bayesian data analysis
KW  - Replication
Y1  - 2020
U6  - https://doi.org/10.1016/j.jml.2019.104063
SN  - 0749-596X
SN  - 1096-0821
VL  - 111
PB  - Elsevier
CY  - San Diego
ER  - 
TY  - JOUR
A1  - Vasishth, Shravan
A1  - Mertzen, Daniela
A1  - Jaeger, Lena A.
A1  - Gelman, Andrew
T1  - The statistical significance filter leads to overoptimistic expectations of replicability
JF  - Journal of memory and language
N2  - It is well-known in statistics (e.g., Gelman & Carlin, 2014) that treating a result as publishable just because the p-value is less than 0.05 leads to overoptimistic expectations of replicability. These effects get published, leading to an overconfident belief in replicability. We demonstrate the adverse consequences of this statistical significance filter by conducting seven direct replication attempts (268 participants in total) of a recent paper (Levy & Keller, 2013). We show that the published claims are so noisy that even non-significant results are fully compatible with them. We also demonstrate the contrast between such small-sample studies and a larger-sample study; the latter generally yields a less noisy estimate but also a smaller effect magnitude, which looks less compelling but is more realistic. We reiterate several suggestions from the methodology literature for improving current practices.
KW  - Type M error
KW  - Replicability
KW  - Surprisal
KW  - Expectation
KW  - Locality
KW  - Bayesian data analysis
KW  - Parameter estimation
Y1  - 2018
U6  - https://doi.org/10.1016/j.jml.2018.07.004
SN  - 0749-596X
SN  - 1096-0821
VL  - 103
SP  - 151
EP  - 175
PB  - Elsevier
CY  - San Diego
ER  - 
TY  - GEN
A1  - Vasishth, Shravan
A1  - Mertzen, Daniela
A1  - Jäger, Lena A.
A1  - Gelman, Andrew
T1  - Corrigendum to: Shravan Vasishth, Daniela Mertzen, Lena A. Jäger, Andrew Gelman; The statistical significance filter leads to overoptimistic expectations of replicability. - Journal of Memory and Language. - 103 (2018), pg. 151 - 175
T2  - Journal of memory and language
Y1  - 2018
U6  - https://doi.org/10.1016/j.jml.2018.09.004
SN  - 0749-596X
SN  - 1096-0821
VL  - 104
SP  - 128
EP  - 128
PB  - Elsevier
CY  - San Diego
ER  -